IJLTER.ORG
p-ISSN: 1694-2493 e-ISSN: 1694-2116
International Journal of Learning, Teaching And Educational Research
Vol.20 No.8
International Journal of Learning, Teaching and Educational Research (IJLTER) Vol. 20, No. 8 (August 2021) Print version: 1694-2493 Online version: 1694-2116
IJLTER International Journal of Learning, Teaching and Educational Research (IJLTER) Vol. 20, No. 8
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically those of translation, reprinting, re-use of illustrations, broadcasting, reproduction by photocopying machines or similar means, and storage in data banks.
Society for Research and Knowledge Management
International Journal of Learning, Teaching and Educational Research The International Journal of Learning, Teaching and Educational Research is a peer-reviewed open-access journal which has been established for the dissemination of state-of-the-art knowledge in the fields of learning, teaching and educational research.
Aims and Objectives The main objective of this journal is to provide a platform for educators, teachers, trainers, academicians, scientists and researchers from over the world to present the results of their research activities in the following fields: innovative methodologies in learning, teaching and assessment; multimedia in digital learning; e-learning; m-learning; e-education; knowledge management; infrastructure support for online learning; virtual learning environments; open education; ICT and education; digital classrooms; blended learning; social networks and education; etutoring: learning management systems; educational portals, classroom management issues, educational case studies, etc.
Indexing and Abstracting The International Journal of Learning, Teaching and Educational Research is indexed in Scopus since 2018. The Journal is also indexed in Google Scholar and CNKI. All articles published in IJLTER are assigned a unique DOI number.
Foreword We are very happy to publish this issue of the International Journal of Learning, Teaching and Educational Research. The International Journal of Learning, Teaching and Educational Research is a peer-reviewed open-access journal committed to publishing high-quality articles in the field of education. Submissions may include full-length articles, case studies and innovative solutions to problems faced by students, educators and directors of educational organisations. To learn more about this journal, please visit the website http://www.ijlter.org. We are grateful to the editor-in-chief, members of the Editorial Board and the reviewers for accepting only high quality articles in this issue. We seize this opportunity to thank them for their great collaboration. The Editorial Board is composed of renowned people from across the world. Each paper is reviewed by at least two blind reviewers. We will endeavour to ensure the reputation and quality of this journal with this issue.
Editors of the August 2021 Issue
VOLUME 20
NUMBER 8
August 2021
Table of Contents Washback Impact of the MUET: The Before and After Effect of a High-stake University English Test in Malaysia 1 Najihah Mahmud, Nor Hazwani Munirah Lateh, Nazirah Mahmud, Ariezal Afzan Hassan, Amaal Fadhlini Mohamed, Siti Amirah Ahmad Tarmizi The Efficacy of Promoting Financial Literacy with MOOC among Economics Pre-Service Teachers ...................... 18 Khoo Yin Yin, Derek Watson, Rohaila Yusof The Relationships between Experience, Qualification and Subject Specialization and Content Knowledge Mastery of Economic and Management Sciences Teachers: A Case of Accounting Teaching ................................................... 36 Habasisa Molise Effect of the Use of WebQuest in a Chinese for Tourism Course at a Thai University: An Exploratory Study ....... 50 Pairin Srisinthon Multiple Means of Engagement Strategies for Maximising the Learning of Mathematics in Pandemic-regulated Classrooms............................................................................................................................................................................. 68 Matshidiso M. Moleko Online Design Thinking Problems for Enhancing Motivation of Gifted Students....................................................... 91 Ayed H. Ziadat, Mohammad Abed Sakarneh Teaching-Learning Strategies to Production Planning and Control Concepts: Application of Scenarios to Sequencing Production with Virtual Reality Support ................................................................................................... 108 Fernando Elemar Vicente dos Anjos, Luiz Alberto Olibveira Rocha, Rodrigo Pacheco, Debora Oliveira da Silva Medical Students’ Evaluation of Online Assessment: A Mixed-Method Account of Attitudes and Obstacles ..... 126 Abed Alkarim Ayyoub, Oqab Jabali The Systematic Implementation of an Innovative Postgraduate Online Learning Model in the Middle East ....... 139 Nessrin Shaya, Laila Mohebi Autonomy and Creative Thinking Skills of Prospective Elementary School Teacher Students in Learning Mathematics with Science Phenomena assisted by the Learning Management System ........................................... 160 Suprih Widodo, Turmudi ., Rizki Rosjanuardi E-Learning Implementation Barriers during COVID-19: A Cross-Sectional Survey Design .................................... 176 Ishaq Al-Naabi, Abdullah Al-Abri Implications of Social Media Addiction on Academic Performance among Generation Z Student-athletes during COVID-19 Lockdown ......................................................................................................................................................... 194 Wan Ahmad Munsif Wan Pa, Muhammad Sofwan Mahmud, Mohd Syazwan Zainal The Role of Non-Academic Service Quality in an Academic-Oriented Context: Structural Equation Modeling amongst Parents of Secondary Education Students ....................................................................................................... 210
Thao-Thanh Thi Phan, Ha-Giang Thi Tran, Le-Huong Thi Nguyen, Tam-Phuong Pham, Thang-The Nguyen, Hiep-Hung Pham, Thai-Quoc Cao Developing A Framework Peace Education for Primary School Teachers in Indonesia ........................................... 227 Sahril Buchori, Sunaryo Kartadinata, Syamsu Yusuf, Ilfiandra Ilfiandra, Nurfitriany Fakhri, Sofwan Adiputra The Measurement of Knowledge Construction in A Course of Diagnostic Evaluation of Learning Disorders in Psychology Students........................................................................................................................................................... 240 Guadalupe Elizabeth Morales-Martinez, Michelle Garcia-Torres, Maria del Carmen Castro-Gonzalez, Yanko Norberto Mezquita-Hoyos The Effect of Resource-Based Instructions on Pre-service Biology Teachers’ Attitudes towards Learning Biology ............................................................................................................................................................................................... 262 Josiane Mukagihana, Catherine M. Aurah, Florien Nsanganwimana The Effectiveness of a Self-Questioning Strategy at Developing Academic Achievement and Critical-Thinking Skills among Secondary-School Students in Saudi Arabia ............................................................................................ 278 Ali Tared Aldossari, Mohammed Msnhat Aldajani The Implementation of Formal Assessments in Intermediate Phase Mathematics at Primary Schools in South Africa .................................................................................................................................................................................... 300 Senzeni Sibanda, Awelani M. Rambuda
1
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 1-17, August 2021 https://doi.org/10.26803/ijlter.20.8.1 Received May 31, 2021; Revised Aug 16, 2021; Accepted Aug 30, 2021
Washback Impact of the MUET: The Before and After Effect of a High-stake University English Test in Malaysia Najihah Mahmud and Nor Hazwani Munirah Lateh Universiti Malaysia Kelantan, Malaysia https://orcid.org/0000-0001-7938-1499 https://orcid.org/0000-0002-6968-0513 Nazirah Mahmud Universiti Sultan Azlan Shah, Malaysia https://orcid.org/0000-0001-6343-6555 Ariezal Afzan Hassan, Amaal Fadhlini Mohamed and Siti Amirah Ahmad Tarmizi Universiti Malaysia Kelantan, Malaysia https://orcid.org/0000-0002-9806-0874 https://orcid.org/0000-0001-8559-4618 https://orcid.org/0000-0002-4932-6007
Abstract. This study seeks to explore the washback effect of the Malaysian University English Test (MUET), a high-stakes compulsory university entry test in the context of Malaysia. As simple and linear as it commonly appears, washback has been found to be far more complex than simply looking at the impact that a test might or might not have on the stakeholders. Therefore, this study aims to fill in this knowledge gap by systematically re-examining the beliefs on washback by investigating the relationship between the students’ perceptions of the MUET in terms of its importance and difficulty, with their language learning strategies whilst preparing for the test and after sitting the test. Using a mixed methods approach, a student questionnaire and student interview were utilised to elicit data from 30 male and 46 female students. The students were further divided into two groups, specifically those who were preparing for the MUET and those who had already sat the MUET. The findings suggest that preparing for the MUET encouraged the students to utilise a certain language learning strategy more compared to the others, specifically focusing on language skills that had not been formally tested before. Although the students’ perception did not significantly shape the students’ course of action when preparing for the test, it still had an impact on their overall view of the whole test-taking matter. This study is
©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
2
intended to add more insights to the less explored areas of washback, specifically the students’ perceptions and washback over time. Keywords: washback; language assessment; high-stake test; language testing; backwash
1. Introduction In the field of education, it is well known that tests, especially high stakes tests, have a huge influence on the teaching and learning process. Specifically, in language education, the influence that the tests exert over teaching and learning is known as ‘washback’ or ‘backwash’. Washback has always been associated with the negative consequences of tests. However, this phenomenon was not empirically investigated until the 1980s. Since then, language education researchers have been looking at washback differently as the findings from the previous research on washback continually reveal how complex it is (Dong, 2020; Khan et al., 2019; Hughes, 2021). Alderson and Wall (1993), in their pioneering study on washback in Sri Lanka, hypothesised that “Tests that have important consequences will have washback” and conversely “Tests that do not have important consequences will have no washback” (p. 120). In other words, the higher the stakes of a test, the more of an impact it will have on the teaching and learning process. According to Qi (2007), the authorities are always tempted to resort to manipulating high stakes testing in the name of ‘curriculum innovation’ for an immediate outcome as it is claimed to be a “quick and most cost-effective way to improve education” (p.52). Studies on the washback of high-stakes testing reported that the test could change the students’ learning behaviour by motivating them to put more effort into their learning (Cho, 2004; Pan & Newfields, 2012; Thomas, 2005) while also promoting learner autonomy (Pan, 2014; Stecher, 2002) when preparing for the test if the stakes are sufficiently high.
2. Literature Review The connection between (1) testing, and (2) the teaching and learning practices has been commonly explored based on the research in the field (Barrows et al., 2013; Cheng et al., 2011; Gebril & Eid, 2017; Luong-Phan & Effeney, 2015) sparked by the ground-breaking study on washback by Alderson and Wall in the late 1980s. Alderson and Wall (1993), in their significant publication on washback, raised the notion of the complexity of washback and the needs for more in-depth research attempting to not only describe what washback looks like but also to account for what occurs. They also pointed out that although it is widely known that a relationship between testing, teaching and learning practices does exist, the complexity of the washback concept itself makes it difficult to prove how directly tests affect the teaching and learning practices without considering other mediators or variables that may or may not contribute to the change. Similarly, Stoneman (2006) observed that the washback phenomenon has yet to be clearly explained and deeply explored despite the abundant literature on language assessments focusing on the impact of testing on teaching and learning.
http://ijlter.org/index.php/ijlter
3
In a doctoral study by Mahmud (2018), it was hypothesised that the students’ perception of the test has an impact when it comes to determining the students’ course of action when preparing for the test as opposed to their actual proficiency level. For instance, even if a student has a weak command of the English language, if the students think that they can perform well in the test, they will be very motivated when preparing for the test and vice versa. In a more recent study undertaken in China involving 3,105 high school students, Dong (2020) found that the students’ positive perception of the test increased the positive washback of the test. Test preparation activities may help to increase the students’ probability of success but it only works in the short-term. If it is too excessive, it can be counterproductive regarding the students’ achievements in terms of their test score. Aside from being a waste of the students’ time and energy, Dong (2020) emphasised that it defeats the purpose of learning and in turn, creates negative washback. The MUET was first introduced in 1999 with the aim (1) of “bridge the gap in English language needs between secondary and tertiary education (Malaysian Examination Council); and (2) to consolidate and enhance the English proficiency of students preparing to enter Malaysian public universities” (Lee, 2004, p. 1). The four language skills of reading, writing, listening and speaking are tested in the MUET. It is designed and administered by the Malaysian Examination Council and it is recognised in Malaysia and Singapore (Othman & Nordin, 2013). This test is significant for pre-degree students as it serves as an indicator of their English language proficiency which enables them to enrol on their desired course (Kaur & Nordin, 2006). This study aimed to investigate the washback effect of a high-stake language test, the Malaysian University English Test (MUET), while preparing for the test and after sitting the test. This study investigated the Malaysian students’ perception of the MUET by exploring its relationship with the language learning strategies that they employed when preparing for the test. In addition, it also explored how long the washback effect of MUET lasted after the students sat the test. The research questions guiding this study were as follows: 1. What are the students’ perceptions of the MUET? 2. To what extent do the students’ perceptions seem to have a washback impact on the students’ language learning strategies? 3. Is there a difference in the washback impact before and after sitting for the MUET?
3. Methodology The use of both quantitative and qualitative data gathering techniques assisted the researcher in illuminating different aspects of the same issue and providing a more complete picture of the study (Denscombe, 2014). As this study aimed to generate data on the students’ perception of the test and their language learning strategies in relation to the washback effect of the MUET over time, an analytical comparison was carried out between the students who were preparing for the MUET and the students who had already sat the test. To provide multiple perspectives and methods, the data was gathered through the collection of the
http://ijlter.org/index.php/ijlter
4
students’ perceptions using a questionnaire and through interviews. Two groups of students were recruited for this study, specifically students who were preparing for the MUET (Group A, n=30) and students who had already sat the MUET and who were currently undertaking a general English language course at a public university in Malaysia (Group B, n=46). Table 1: Participants Gender Male Female
Group A 11 19
Group B 13 33
Total 24 52
Data from Group A was used to describe the washback effect of the MUET whilst data from Group B was used to explore the washback impact over time, also known as the washback length. For this study, the questionnaire was adapted from established questionnaires from relevant previous studies as detailed in Table 2. Table 2: Questionnaire Sections Background information Section 1 : Perception of the Test Section 2 : Perception of the Test Importance Section 3 : Perception of the Test Consequences Section 4 : Language Learning Strategies
Sources N/A Xie & Andrews (2013) Bodas (2006)
Cronbach’s alpha (α) N/A N/A .833
Bodas (2006)
.805
Oxford (1990) Green (2007)
.890
In general, the questionnaire consisted of two main sections. The first section covered the demographic questions such as gender, field of study, English proficiency level etc. Section two consisted of four sub-sections, dealing with the students’ (1) perception of the test (MUET) in general, (2) their perception of the test importance, (3) their perception of the test consequences and (4) Language Learning strategies (Oxford, 1990). All of the variables in section two were assessed using a Likert scale. This study also utilised the students’ individual interviews to gather the qualitative data. As pointed out by Atkins and Wallace (2012), interviews not only allow the researchers to engage with the participants individually, but it also allows them to collect various types of in-depth data, for example, factual data, views and opinions, personal narratives and histories. According to Ary et al. (2013), there is no general rule for determining the number of participants for the purpose of collecting qualitative data. For this study, 12 open-ended questions with several probing questions on the students’ experiences of learning English were prepared for the student interviews. The questions were adapted based on
http://ijlter.org/index.php/ijlter
5
the previous studies on washback (Hsu, 2010; Mahmud 2018; Shih, 2013) focusing on the learners’ general perception of their English language learning and the test under investigation, the MUET.
4. Findings Item analysis of the mean scores of the students’ perceptions for all constructs was carried out to better understand how the students responded to each item individually before analysing the items according to their grouping. The student questionnaire consisted of four constructs: (i) the perception of the MUET in general, (ii) the perception of the test importance, (iii) the perception of the test consequences and (iv) the language learning strategies. In the item analysis, the constructs were treated as the dependent variables, with gender as fixed factors, and English language proficiency and experience within the test as covariates. Taking into consideration that the data was not normally distributed, nonparametric analyses, the Mann-Whitney U-test and the Kruskall-Wallis test were employed to analyse the aforementioned constructs both as individual items and as an overall scale. To ensure a greater level of reliability, only significant differences in the mean scores (p < .05) were taken into consideration. The findings were arranged according to the sub-sections as follows. 4.1 Students’ Perception of the MUET 4.1.1 Perception of the MUET in general The mean scores for all items were calculated and arranged in descending order as illustrated in Table 3. A Likert scale (comprised 5 for strongly agree, 4 for agree, 3 for undecided, 2 for disagree and 1 for strongly disagree) was used to score the items. Table 3: The MUET in general Items
.611
4.29
.629
4.24
.671
1.11 More listening activities should be conducted in MUET preparation class.
4.16
.590
1.6 The MUET made me practise my reading skills more than before.
4.11
.723
4.11
.759
1.10 More reading activities should be conducted in MUET preparation class.
4.03
.765
1.3 The MUET made me practise my writing skills more than before.
3.99
.721
3.89
.665
2.46
1.113
1.13 More writing activities should be conducted in MUET preparation class.
1.2 My English language learning was improved by practising MUET past year questions. 1.9 I think that the MUET preparation class that I took was not very helpful.
http://ijlter.org/index.php/ijlter
Agree
1.5 The MUET made me practise my speaking skills more than before.
Agree
1.12 More speaking activities should be conducted in MUET preparation class.
Strongly Agree
4.30
Disag ree
1.4 The MUET made me practise my listening skills more than before.
Std. Mean Deviation
6
The trend seems to be that the highest-ranking items with mean scores that are within the range of ‘Strongly Agree’ are those related to the listening and speaking skills. These two skills are seldom tested in other standardised English language test in Malaysia as opposed to reading and writing. This clearly indicates the washback that the MUET had on these students, especially towards the two skills which were not tested before. The mean scores for the set of items regarding the students’ increase in effort connected to their reading and writing skills ranges from 3.99 to 4.11, which are ranked very high. Item 1.9 has the lowest mean score (2.46) which falls under the ‘Disagree’ range. This indicates that most of the students perceived the MUET preparation class that they took as being beneficial to them. Next, the analysis of the overall scale was carried out. The Mann-Whitney U-test was run and the analysis revealed there to be no significant differences in terms of the students’ general perception of the MUET in relation to their (i) gender, p = .906, (ii) experience with the test, p = .074, and (iii) proficiency level, p = .442. The result suggests that the students' general perception of the MUET did not change even after they had already taken the test. The students’ proficiency level in English did not affect their general perception of the MUET as well. 4.1.2 Perception of Test Importance The perception of the test importance construct consisted of five items as shown in Table 4. Table 4: Test Importance Std. Mean Deviation
2.3 It is very important for me that I do well in the MUET 2.4 It is very important for my future undertakings that I do well in the MUET
Strongly Agree
Items
2.2 It is very important for my teacher that I do well in the MUET 2.1 It is very important for my parents that I do well in the MUET
Agree
2.5 Every student who wants to get into the university should pass the MUET
4.42
.753
4.38
.816
4.13
.900
3.97
1.045
3.70
.980
The top two items are items relating to the students’ priority of their own self, as opposed to external factors like their teachers (item 2.2) and parents (item 2.1), which were ranked as the bottom two. At the item level, both the Mann-Whitney U-test and the Kruskal-Wallis test showed there to be no significant differences across gender (p = .506), English proficiency level (p = .619), and experience with the test (p = .619) for each item in the perceived test importance construct. This shows that the students of all English proficiency levels regarded the MUET as an important test. This finding confirms the status of the MUET as a high-stakes test. The students were also
http://ijlter.org/index.php/ijlter
7
asked about their perception of the consequences that they had to face if they failed the MUET, which has been presented in the next section. 4.1.3 Perception of the Consequences of Exam Failure The perception of the test consequences construct consisted of six items as shown in Table 5. A Likert scale (comprising 5 for extremely to 1 for not at all) was used to score the items. Table 5: Consequences of Exam Failure Std. Mean Deviation
Items
4.14
.948
3.1 How upset would you be for letting your parents down?
3.99
1.052
3.93
1.037
3.4 Will your chance to enrol on your desired course be affected?
3.91
1.048
3.6 Will your ability to communicate in the English language affected?
3.03
1.376
3.03
1.366
3.5 Will your motivation to learn the English language be affected?
Undecided
3.3 Will your chance to get into top universities be affected?
Very
3.2 How upset would you be for letting your teachers down?
Interestingly, the top two items with the highest mean score for the test consequences construct are the items that are extrinsic in nature, pertaining to living up to others’ expectations, namely their teachers (item 3.2) and parents (item 3.1). Following very closely are the items related to the main objective of the MUET, which is where it acts as an entry test into university. The students agreed and were aware that their application to their desired university (item 3.3) and courses (item 3.4) would be affected if they performed poorly on the MUET. However, they appeared to be unsure whether their ability to communicate and their motivation to learn English would be affected if they did poorly in the MUET. Based on the overall ranking, it is safe to assume that the severity of the consequences of exam failure for the MUET is perceived to be quite threatening to the students. Next, the Mann-Whitney U-test was run to determine whether there were any differences in the perception of the test consequences between the three independent variables of gender (p = .699), experience with the test (p = .321) and English language proficiency (p = .496). The findings revealed that there were differences between the variables, but they were not statistically significant. Similar to the findings for perceived test importance, the students’ proficiency level in English did not seem to affect their perception of the test consequences. The findings thus far show that there were no significant differences reported between the students’ perception of the test and the three independent variables. However, at the item level, the Kruskal-Wallis test revealed there to be significant differences across the English language proficiency levels for item 3.6 (Will your ability to communicate in the English language affected?, p=.042) with the mean
http://ijlter.org/index.php/ijlter
8
ranking across the grouping in an ascending manner being as follows: Low (22.83), Average (39.60) and High (47.06). Interestingly, the findings reveal that the students might regard the MUET as one of the main determinants of their English language ability, especially for high proficiency students. 4.2 Relationship between the students’ perceptions and washback impact on language learning strategies In order to explore the washback impact of the MUET on the students’ learning, item analysis was carried out to determine the students’ pattern of responses based on the frequency of their perceived language learning strategy usage. The mean scores for all items were calculated and arranged in descending order as illustrated in Table 6. A Likert scale (comprising 5 for all the time, 4 for most of the time, 3 for undecided, 2 for sometimes and 1 for never) was used to score the items. Table 6: Language Learning Strategies Items
4.11 I was NOT sure how to improve my English skills. 4.4 I did the MUET practice tests in my free time. 4.7 I reviewed my English class notes or textbook in my free time. 4.9 I read English without looking up every new word. 4.5 I studied extra English outside of my MUET preparation class.
http://ijlter.org/index.php/ijlter
High Medium
4.15 When writing in English, I tried to translate from my language. 4.10 When reading in English, I tried to translate it into my language to help me understand. 4.6 If I couldn’t think of an English word, I used a word or phrase that means the same thing. 4.16 I thought about the goals that I wanted to achieve in this English language course. 4.18 When I received corrected work from the teacher, I thought about how to improve next time. 4.14 To understand unfamiliar English words, I tried to guess their meaning. 4.8 I encouraged myself to use English even when I was afraid of making a mistake. 4.17 I tried to improve my writing by analysing the work of other writers. 4.2 I tried to find better ways of learning English. 4.3 I tried to improve my English by asking others to correct my mistakes. 4.12 I used new English words in sentences so then I could remember them. 4.13 When I learned a grammar rule, I tested myself to make sure that I really knew it. 4.1 I memorised English words by saying or writing them several times.
Mean
Std. Deviation
4.08
1.043
4.00
1.007
3.99
1.026
3.97
1.078
3.89
1.078
3.82
1.186
3.82
1.055
3.71
1.198
3.70
1.178
3.66
1.250
3.54
1.259
3.51
1.301
3.09
1.246
2.92 2.83
1.393 1.320
2.82
1.230
2.75
1.406
2.67
1.258
9
The findings revealed that the students reported a moderate-to-high frequency of using language learning strategies to prepare for the MUET. Specifically, the students used meta-cognitive strategies (items 4.2, 4.11, and 4.18), often associated with learning success, at a high frequency. A high frequency of usage was also reported for cognitive strategies related to rote-translation (item 4.15 and 4.10). As depicted in Table 6, in terms of the individual learning strategies for all of the strategy items belonging to the meta-cognitive (planning, organising and evaluating learning) and compensation (to overcome difficulties in communication) categories, the students reported a high frequency of use with a mean score ranging from 3.70 to 3.99. Meanwhile, none of the language learning strategies were reported at a low frequency of use. For the strategies used at moderate frequencies, the findings show that most of the strategies were related to the learning activities that the students did in their free time, for example, item 4.4 (I did MUET practice tests in my free time) and item 4.7 (I reviewed my English class notes or textbook in my free time). Finding opportunities to practice outside of the classroom (item 4.5) was the least used strategy among the students, which is not surprising. This strategy requires self-initiation and courage, which many young learners in a second language context might not possess. To determine the relationship between the students’ perception of the test importance and their language learning strategies, Kendall's tau-b correlation was carried out. There was found to be a weak positive association that was not statistically significant between perceived test importance and language learning strategies as a whole, τb = .140, p = .089, and direct learning strategies, τb = .149, p = .076, and a very weak positive association between perceived test importance and indirect language learning strategies, τb = .091, p = .283. There was no statistically significant association between the students’ perceived test importance and their use of language learning strategies. Next, there was found to be a weak positive association, which was not statistically significant, between the perceived test consequences with language learning strategies, τb = .154, p = .058 and direct learning strategies, τb = .153, p = .065, and a very weak positive association between the perceived test consequences and indirect language learning strategies, τb = .082, p = .326. There was no statistically significant association found between the students’ perceived test consequences and their use of language learning strategies. Therefore, the null hypothesis cannot be rejected, and the alternative hypothesis cannot be accepted. The data suggests that the students’ perception of test importance and the test consequences did not seem to affect the students’ usage of language learning strategies. 4.3 The before and after effect of the MUET This section presents the analysis of the individual interviews with two students from Group B, which consisted of students who had already taken the MUET. The two students were labelled R1 (Female, MUET Band 2) and R2 (Male, MUET Band 3) to ensure anonymity. Students R1 and R2 were randomly chosen from a list of the students who volunteered to be interviewed.
http://ijlter.org/index.php/ijlter
10
When it comes to English language learning, it was evident from the qualitative data that both students seemed to be particularly concerned with grammar: “For me it’s difficult, in terms of grammar. And also, the usage of the words.” (R2) “Even when we enter the university, we still have to learn grammar because it’s easy to forget. We have to really focus in the class.” (R1) This finding can be attributed to the washback effect of the 11 years of formal English language education in Malaysia. This is because grammar comes first in the syllabus before any of the four language skills. Therefore, most Malaysian students are under the impression that before they work on their language skills, they must make sure that their grammar is good. Another skill that they were concerned with when it comes to English language learning was speaking skills. Both agreed that for them to improve their speaking skills, they had to practice: “We have to practice it a lot. We have to make it a habit. For example, when communicating with the teacher, we are supposed to use the English language.” (R2) “For speaking skills, we have to always use it. If we want to improve our speaking skills, we should mix around with the Chinese students, that’ll help.” (R1) The interview participants were also asked about their current English language learning experience. When asked to describe how their current English language class is, the following are their descriptions: “Normally the teacher would emphasize on the syllabus from the textbook. For example, we were involved in drama production for an event called ‘Drama Night’, so the teacher asked us to develop our script which took months to finished. Then, after the event, we continued with the syllabus from the textbook.” (R1) “The teacher taught us based on the textbook, and if there was any question, we would discuss it together. Then, the teacher also asked us to present in front of the class.” (R2) Obviously, emphasis was given to the activities in the textbook. However, these particular students prefer something more interactive as opposed to being constrained or restricted by the syllabus. They mentioned the activities that were conducted in their previous semester’s English class, which was English I. They seemed to like it and perceived it as helping them to improve their English language skills: “I remember last semester’s English teacher, for example, when we were learning about grammar, the teacher would make up a story about it or the teacher would create songs about the grammar that we were learning.
http://ijlter.org/index.php/ijlter
11
That way, we could memorize it better. Well actually we memorized the song, but at the same time, we were learning grammar as well.” (R1) “My teacher last semester conducted some sort of game. In my opinion, after we played that game, it has somehow helped improve my English because throughout the game, the teacher made it compulsory for us to use English language, forcing us to speak in English. I think that’s quite helpful.” (R2) Moving on to the MUET, the students were asked to recall their experience preparing for the high-stakes language test. Questions pertaining to what they did inside and outside of their classes were asked to see what kind of learning activities and strategies they used, and to determine the intensity of the washback of the MUET. The participants were first asked about their perception of the MUET in general compared to the other English language tests that they had taken before. The following were their responses: “For me, MUET is important because it helps us improve especially our speaking skills, it encouraged the students to speak in English more because they definitely did not want to get low marks, so they would try to speak with other people to practice. And then for listening skills, for example my friends, if previously they were mostly listening to Malay songs, but because listening was tested in MUET, they started to listen to English songs more.” (R1) R1 described the MUET as having more challenging questions for the writing component and more questions for the reading component. She also mentioned that the MUET encourages students to speak more in English because they do not want to get low marks for the speaking component. Being tested on all four language components forced the students to work on all four skills. Improvising the four skills required different learning strategies as mentioned by R2 below: “MUET does not consist of only one test, but it has different papers for listening, speaking, writing, and reading, so for each of these skills, different preparation strategies are required. For example, when I took SPM in the past, it was more on the writing skills only, but for MUET, we must be able to grasp all of the skills, and for speaking, we have to be able to communicate well.” (R2) R2 mentioned that the previous English language test that he took mainly focused on writing skills. Since the MUET also tested his speaking and listening skills, he had to apply different learning strategies to prepare himself for it. He stated that his teacher devoted one whole period of the English lesson to teaching each language skill separately: “For example, if for this period, the teacher decided to focus only on listening skills, for other period, the teacher will focus only on speaking skills and so on.” (R2) He also mentioned the use of textbooks in the MUET preparation class. In R1’s case, her teacher emphasised the listening and speaking activities more in the classroom:
http://ijlter.org/index.php/ijlter
12
“The teacher focuses more on the listening and speaking skills. For listening skills, the teacher provided us with MUET text book. The teacher would play the CD in the classroom for us, then the teacher would ask us to answer the questions in the text book. For speaking skills, the teacher divided us into groups of four, because in MUET speaking test, the students are divided in the same way, so the teacher would imitate the real situation in MUET speaking test.” (R1) Similar to R2’s MUET preparation class, R1’s teacher also utilised textbooks to prepare the students for the test. The textbook that R1 mentioned is a commercialised MUET preparation textbook sold at ‘Popular’, one of the main bookstore chains in Malaysia. In her comment, R1 mentioned that for speaking, her teacher mimicked the real MUET situation to familiarise the students with the format. However, her teacher only did this when the date of the MUET drew closer, which was two weeks before the actual test took place: “The teacher would act as the examiner. But, during that time, it was just two weeks before the actual test took place. Before that, the teacher just asked us to practice on our own.” (R1) The researcher also asked the students to comment on the importance of the MUET to them. R1 initially was not aware of the purpose of the MUET or the use of the MUET result, not until after she had taken the test: “When I sat for MUET at the matriculation college, I did not even know that MUET result would affect my university admission result. My teacher did not say anything about it. My teacher just mentioned that it is important, that is all.” (R1) R1 stated that the reason why her teacher did not say anything on the importance of the MUET was because her teacher did not want to pressure her students. Her teacher thought that there would not be any problem for the students when it came performing well in the MUET as most of them managed to score above Band 3 in the mock-MUET practice test. “My teacher did not tell us about it because he did not want us to be stressed out. It was also because when we did a pre-test for MUET, he told us that all of us would be able to get Band 3 and above. That was why my teacher did not say anything.” (R1) For R2, the only thing that he knew about the objective of the MUET was that it was one of the requirements of applying to tertiary education in Malaysia. “All I know is MUET is compulsory to gain entry into the university, that’s all.” (R2) R1 specifically described her difficulties when applying for her desired course as most of the degree courses required at least Band 3 results in the MUET: “When I was applying for the university, there were a lot of courses that I could not apply because these courses require at least Band 3. It was quite difficult for me. So, I just chose Band 2 courses because I only got Band 2 in MUET.” (R1)
http://ijlter.org/index.php/ijlter
13
She ended up choosing courses with a set minimum requirement of MUET Band 2, as those were her only options. However, despite getting a low MUET Band, she did not let her MUET Band define her English language proficiency: “For speaking skills, the percentage for the overall score is less than reading and writing, hence, those who like to read books, or overly focused when answering the questions, they would be able to perform well. I mean, unless if the percentage for all skills are equal, then maybe MUET can be used to really measure the overall English language ability. For me, no. Imagine those who manage to get Band 4, but are not able to speak fluently, it’s still the same.” (R1) She stated that since the MUET has different weightages for the different components, like the reading component contributes the highest percentage to the overall Band, students who have good reading skills might be able to perform better. For her, getting a higher Band in the test with no ability to speak the language well would still be pointless.
5. Discussion 5.1 Students’ Perception of the MUET As can be seen in the findings of this study, although limited in scope, there is clear evidence of the washback effect of the MUET especially for the listening and speaking skills as indicated by the quantitative data. Although the MUET consists of all four language skills, reading and listening have not been formally tested before in any standardised English language test in Malaysia other than the MUET. Therefore, once these two skills were added to the test, the learners started to pay more attention to those two skills as well. A scrutiny of the qualitative data showed that the students were particularly concerned with their speaking and listening skills as well as opposed to their writing and reading skills. They also mentioned that their teachers focused more on these two skills during the English lessons. As suggested by Nambiar and Ransirini (2012), both the teachers and students tend to focus more on the tasks that they consider to be imperative to the outcome of the test. Different washback effects depend on the perceived task importance. Although the MUET tested all four skills, it is safe to assume that due to the novelty effect, the students and teachers decided to focus more on the listening and speaking skills because they have been dealing with reading and writing skills for the past 11 years of formal education in Malaysia. This was also the case in Dong’s (2020) study where the students were found to rarely engage in communicative learning, for instance speaking, as it was not tested. The quantitative findings also suggest that the students’ general perceptions about the MUET were not influenced by either their gender or their English proficiency level. Their perceptions of the MUET also did not change even after they had taken the test. Similarly, when it comes to the students’ perception of the importance of the MUET to them, their gender, English proficiency level, and their experience with the MUET did not seem to affect their perception as well. This shows that regardless of their English language proficiency, the students regard the MUET as an important test. This confirms the status of it as a high-stakes test. The findings also revealed that failing the test threaten the students’ chances of enrolling on their desired course and to their intended university. This frustration
http://ijlter.org/index.php/ijlter
14
was clearly expressed by one of the interview respondents as her choices when applying to tertiary education were limited due to her low MUET score. However, when it comes to the test consequences, the students who were preparing for the MUET appeared to be more worried about the consequences of failing the test and how it would affect their motivation to learn and their ability to use the English language. This can be attributed to a phenomenon known as test anxiety, as they had yet to sit for the MUET and did not know what to expect. Those who had already sat the MUET seemed less concerned. This might be due to the fact that they have gone already through the whole experience of the MUET and managed to pull through well in the tertiary level as they were currently studying in a university at the time that this study was conducted. 5.2 Language Learning Strategies One of the objectives of this study was to see if the MUET encourages students to employ more language learning strategies when preparing for it. The quantitative findings revealed that the students reported a moderate-to-high frequency of language learning strategy use to prepare for the MUET. None of the language learning strategies were reported at a low frequency of use. A closer analysis of the quantitative data disclosed that specifically, the students used cognitive strategies related to rote-translation at a high frequency. The qualitative data revealed similar findings in that the students utilised translation techniques to help them learn. Cognitive strategies like rote-translation are forms of direct learning strategies that, according to Pan (2014), are not deep learning strategies that can really help the students acquire the necessary language skills. In his study, he found that most of the students reported to have frequently used traditional language learning activities such as reading textbooks, memorising vocabulary and idioms, and practicing sentence patterns to name a few. A moderate obsession with grammar was also observed in the responses given by the interview respondents in this study. Similar findings were reported by Shih (2013). He found that most of the students in his study seemed to employ more surface strategies in their English language learning process rather than deep strategies. This type of strategy is mostly geared towards scoring on the test and will not benefit the students in the long run. To see whether the students’ perception of the MUET could be associated with their language learning strategy use, correlation analysis was carried out on perceived test importance and perceived test consequences in relation to the language learning strategies. The results suggest that the students’ perception of the test importance and test consequences are not statistically significantly associated with the students’ usage of language learning strategies. 5.3 The before and after effect of the MUET As reported by the qualitative data in this study (see section 4.3), it can be seen that the students were made to practice the skills that had not been tested before in their formal education in Malaysia more, which were speaking and listening skills. This is because the MUET tests all four language skills. However, they perceived that their speaking skills had particularly improved since they practiced both at school and in real life. Although the students expressed a
http://ijlter.org/index.php/ijlter
15
reservation when speaking in English, it was encouraging to see a slight selfrealisation regarding the need to speak the language more in order to help them improve their skills. As a high-stakes test, the MUET results affect the students significantly, as the MUET is one of the requirements for university entry in Malaysia. Certain courses like medicine, engineering and TESL require a slightly higher MUET band compared to others. Not being able to score the minimum MUET band requirement for their desired course would result in the student having to take another course at the university. This explains the strong washback impact that the MUET can impose on the students due to its high stakes. If utilised correctly by the stakeholders, especially the educators, the problems related to encouraging the students to practice their language skills more can be tackled in due time. 5.4 Limitation of the study It should be noted that the number of participants in this study was fairly small and that the grouping was not normally distributed. The findings from the analysis should not be taken at face value. More holistic data needs to be collected when attempting to explore a complex phenomena like washback, hence, more student interviews need to be carried out for both groups, not only two Group B students. 5.5 Implications of the study It can be seen in this study that the perception of the test could be one of the important factors involved in determining test washback. According to Dong (2020), among the stakeholders, the teacher is determined to be the most important individual affecting the teaching and learning process. Hence, the teacher can promote the proper perspective of the test among the students which in return could help them to promote positive washback and improve the students’ performance in the long run. Several studies tapping into washback have indicated that washback changes over time, hence more research should investigate this. The present study attempted to explore a part of the washback length element by comparing the before and after effect of washback. This study is hoped to add more insights to the less explored areas on washback, specifically the students’ perceptions and washback over time.
6. Conclusion It was apparent from the findings that the students’ actual proficiency in the English language did not have a washback impact on the students’ course of action when preparing for a high-stakes test like the MUET. Although the quantitative data revealed a similar result in relation to the students’ perceptions, the qualitative data appeared to reveal a glimpse into the relationship between the students’ perceptions and their influence on their selected language learning strategies when preparing for a test. This calls for further exploration regarding the washback impact of a test utilising a much bigger sample with more diverse participants and instruments.
http://ijlter.org/index.php/ijlter
16
Acknowledgments This work was supported by a research grant provided by Universiti Malaysia Kelantan (R/SGJP/A0400/01060A/001/2019/00596).
7. References Alderson, J. C., & Wall, D. (1993). Does washback exist? Applied Linguistics. https://doi.org/10.1093/applin/14.2.115 Ary, D., Jacobs, L., Sorensen, C., & Walker, D. (2013). Introduction to research in education: Cengage Learning. Atkins, L., & Wallace, S. (2012). Qualitative research in education. Sage Publications. Barrows, J., Dunn, S., & Lloyd, C. A. (2013). Anxiety, self-efficacy, and college exam grades. Universal Journal of Educational Research, 1(3), 204-208. Bodas, J. (2006). Intra-individual and Extra-individual predictors of text anxiety in Indian children: A cross-cultural perspective (Doctoral dissertation, Virginia Tech). Cheng, L., Andrews, S., & Yu, Y. (2011). Impact and consequences of school-based assessment (SBA): Students’ and parents’ views of SBA in Hong Kong. Language Testing, 28(2), 221-249. Cho, D. (2004). Use of standardized tests as university graduation requirement. English Teaching, 59(1), 251–266. Denscombe, M. (2014). The Good Research Guide: For Small-scale Social Research Projects (5th Edition). UK: McGraw-Hill Education. Dong, M. (2020). Structural relationship between learners’ perceptions of a test, learning practices, and learning outcomes: A study on the washback mechanism of a highstakes test. Studies in Educational Evaluation, 64, 100824. https://doi.org/10.1016/j.stueduc.2019.100824 Gebril, A., & Eid, M. (2017). Test Preparation Beliefs and Practices in a High-Stakes Context: A Teacher’s Perspective. Language Assessment Quarterly, 14(4), 360–379. https://doi.org/10.1080/15434303.2017.1353607 Green, A. (2007). IELTS washback in context: Preparation for academic writing in higher education (Vol. 25). Cambridge University Press. Hsu, H. F. (2010). The impact of implementing English proficiency tests as a graduation requirement at Taiwanese universities of technology (Doctoral Dissertation, University of York). Hughes, L. (2021). Washback and the assessment practices of ESL instructors at Japanese university. Language Literacy: Journal of Linguistics, Literature, and Language Teaching, 5(1), 1-9. https://doi.org/10.30743/ll.v5i1.3238 Kaur, N., & Nordin, R. (2006). A case for reconstruction of the pedagogy of the Malaysian University English test (MUET) through thematic units instruction. Journal of Institutional Research South East Asia, 4(1), 5–16. Khan, A. B. M. A., Aziz, M. S. A., & Stapa, S. H. (2019). Examining the factors mediating the intended washback of the English language school-based assessment: Preservice ESL teachers’ accounts. Pertanika Journal of Social Sciences & Humanities, 27(1), 51-68. Lee, K. S. (2004). Exploring the connection between the testing of reading and literacy: The case of the MUET. GEMA Online Journal of Language Studies, 4(1), 41–50. Luong-Phan, N. H., & Effeney, G. (2015). TOEFL iBT and language learning motivation: An investigation into teaching styles and influential factors for Vietnamese adolescents. International Journal of Research Studies in Language Learning, 4(3), 3-18. Mahmud, N. (2018). Investigating the Washback Effect of the MUET as a University Entry Test on Students in Malaysia (Doctoral dissertation, University of York).
http://ijlter.org/index.php/ijlter
17
Nambiar, M., & Ransirini, S. (2012). Teaching MUET, not English: A study of the washback effect of the Malaysian University English Test (MUET). English in Multicultural Malaysia. Pedagogy and Applied Research, 49-62. Othman, J., &Nordin, A. B. (2013). MUET as a predictor of academic achievement in ESL teacher education. GEMA Online Journal of Language Studies, 13(1), 99–111. Oxford, R. L. (1990). Language learning strategies: What every teacher should know. Boston: Heinle & Heinle Publishers. Pan, Y. C. (2014). Learner washback variability in standardized exit tests. TESL-EJ: Teaching English as a Second or Foreign Language, 18(2), 1–30. Pan, Y. C., & Newfields, T. (2012). Tertiary EFL proficiency graduation requirements in Taiwan: A study of washback on learning. Electronic Journal of Foreign Language Teaching, 9(1), 108–122. Qi, L. (2007). Is testing an efficient agent for pedagogical change? Examining the intended washback of the writing task in a high‐stakes English test in China. Assessment in Education: Principles, Policy & Practice, 14(1), 51–74. https://doi.org/10.1080/09695940701272856 Shih, P. C. (2013). The English benchmark policy for graduation: An investigation of perception, motivation, and approaches to learning at a university of technology in Central Taiwan (Doctoral dissertation, Durham University). Stecher, B. M. (2002). Consequences of large-scale, high stakes testing on school and classroom practice. Making Sense of Test-Based Accountability in Education, 79–100. Stoneman, B. W. H. (2006). The impact of an exit English test on Hong Kong undergraduates: a study investigating the effects of test status on students' test preparation behaviours (Doctoral dissertation, The Hong Kong Polytechnic University). Thomas, R. M. (2005). High-stakes testing: Coping with collateral damage. Routledge. Xie, Q., & Andrews, S. (2013). Do test design and uses influence test preparation? Testing a model of washback with Structural Equation Modeling. Language Testing, 30(1), 49-70.
http://ijlter.org/index.php/ijlter
18
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 18-35, August 2021 https://doi.org/10.26803/ijlter.20.8.2 Received Jun 29, 2021; Revised Aug 16, 2021; Accepted Aug 30, 2021
The Efficacy of Promoting Financial Literacy with MOOC among Economics Pre-Service Teachers Khoo Yin Yin Sultan Idris Education University https://orcid.org/0000-0003-4850-2184 Derek Watson University Sunderland https://orcid.org/0000-0002-1944-3544 Rohaila Yusof Sultan Idris Education University https://orcid.org/0000-0001-5304-8970
Abstract. It is important to educate financial literacy to pre-service teachers to manage their money well and to disseminate the knowledge to their students. The study aimed to examine the effectiveness of MOOC in promoting financial literacy among pre-service teachers. This paper also examines the correlation among the variables of financial literacy, collaborative skills, problem-solving, accountability, and decisionmaking. Pre-service teachers' views were discussed after attending the MOOC. The study adopted a mixed-method research design. A total of 100 pre-service teachers were selected through a random-sampling technique to participate in the survey. A purposive sampling technique was used to pick five of the participants for the follow-up interview. The quantitative data were analyzed with descriptive and inferential statistics, while the qualitative data were analyzed on the basis of content analysis. The quantitative method yielded favourable descriptive statistics; and they also showed a significant relationship between financial literacy and MOOC. The variables indicated some correlations. The qualitative results also revealed the positive responses of financial literacy, collaborative skills, problem-solving, accountability, and decision-making. The course design and participants' engagement are the key factors of the success of MOOCs. Future research should emphasize the incorporation of financial literacy into different subjects. Educators and researchers could well provide more online learning in addition to those activities that combine financial literacy in other courses. Keywords: financial literacy; Economics pre-service teachers; MOOCs; collaborative skills; accountability; problem-solving ©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
19
1. Introduction Massive open online course (MOOC) is a widespread application of informationtechnology education (Haron et al., 2019). Initially introduced in Malaysia in 2015, MOOCs are now considered a new e-learning platform that was later implemented by the Malaysian Higher Education Institutions (Kumar & AlSamarraie, 2018). By utilizing the platform of MOOCs, it is possible to create a conducive learning forum that allows learning to happen anywhere and in any place (Hassan et al., 2015 Qaffas et al., 2020). It is also recognized that MOOC provides open learning to the enrolled participants; and the students can follow the course at their own place and pace. Most of the MOOCs in Malaysia offer academic courses; nevertheless, a minority of courses have failed to embrace the essential skills, such as financial literacy. If financial literacy is not prioritized, there could well be negative consequences. For example, a total of 47% of bankruptcies in Malaysia involved young adults (Nurul Afiqah, 2016; Khoo & Fitzgerald, 2017). There is a significant tendency for millennials to spend beyond their means; since they are often easily influenced by the social media; and, in consequence, they are inclined to buy expensive and branded goods (Caixeta et al., 2016; Khan et al., 2019). For example, 38.8% of young people between the ages of 20 and 24 repeatedly upgrade their smartphones to the latest model (Malaysian Communications and Multimedia Commission, 2015). In contrast, the consequences of the financial crisis have elevated the importance of financial literacy. The Organization of Economic Co-Operation and Development (2014) raised the importance of financial literacy; and it was highly recommended that it should be included within the school curriculum. The OECD further stressed the importance of educating financial literacy to all citizens, in order to instil better financial management of their own funds and to enhance their financial well-being. The Malaysian Government supported this study; and it is evident in other countries, such as the United States and Australia, which have integrated these recommendations (Cordero & Pedraja, 2019). Financial literacy has been the only cross-curricular element in Malaysia's curriculum since 2017, resulting in the Financial Education Network being launched in 2019 (Financial Education Network, 2019). Teachers are encouraged to include financial literacy in their classes, in order to enhance their students' awareness of financial literacy (Sawatzki & Sullivan, 2017). Evidence suggests that teachers no longer regard financial literacy as not being a key priority; since it is not a standalone subject in the Malaysian secondary-schooling system (Mohd Aziz & Kassim, 2020). This view has influenced students’ perception in learning financial literacy. Furthermore, teachers rarely attend developmental courses; and consequently, they struggle to integrate this critical skill into their schemes of work effectively. However, research in this area is rather limited; and this reflects the research gap. In addition, the academic debate of utilizing MOOCs in financial literacy is limited; and in consequence, there is a research gap to be filled. This issue is the research gap that is yet to be filled in this study. Pre-service teachers, who will become qualified teachers, should be exposed to an incorporated financial literacy, which would later be transmitted to the students during class.
http://ijlter.org/index.php/ijlter
20
Unfortunately, some teachers claimed that they could not cover financial literacy in class, due to time constraints (Rakow, 2019). The same study also mentioned that many teachers are frequently pressed for time; and they are forced to omit specific topics, such as financial literacy. These barriers could be bridged if teachers were to undertake proper training on the pedagogical application and positive impacts of financial literacy. However, due to the lack of financial literacy for graduates, they must seek advice from financial planning services for personal financial management. On a more positive note, researchers have indicated that students, who studied financial education developed financial literacy, when compared to those who did not attend the class (Murugiah, 2016; Zhu, 2018; Haneger & Cude, 2019). The OECD further stated that education could help bridge the financial-literacy gap, and that financial education programmes have been found to successfully improve students' and adolescents' financial knowledge and attitudes (OECD, 2013). Many researchers only focused on pedagogical skills and on the content knowledge of teacher-training courses; in fact, some essential life skills should also be emphasized among them (Prajapati et al., 2017). Since every pre-service teacher has both generic and bespoke learning styles, the practical learning platform should be tailored, in ordered to meet the learners' needs. The learning platform must have the necessary flexibility to encompass various learning pedagogies, such as live-case studies, presentations, etc. MOOC is a popular learning mode that offers such flexibility; and furthermore, it is an affordable way to learn. In addition, prior research showed evidence that financial literacy can be effectively taught, together with the help of digital tools (Angel, 2018; Kuntze et al., 2019). Angel (2018) and Kuntze et al. (2019) shared the same views that online learning using different devices, could enhance financial literacy. Therefore, MOOCs, together with words and videos, are the most suitable platform from which to implement financial literacy. The main contribution of the current study to the practical gap, is to help preservice teachers to acquire important skills, such as collaborative skills, and to incorporate financial literacy into their lessons through practical activities. In addition, this study has also contributed to the literature on Economics education that focuses on community pre-service teachers. Despite the lack of financial literacy among young adults, there has also been a lack of research that could improve financial knowledge and promote financial awareness among preservice teachers. The population of pre-service teachers concerning the effectiveness of financial literacy through digital platforms is the significant result of this study.
2. Objective This paper aimed to answer the following research objectives: 1. To explore the effectiveness of MOOC in promoting financial literacy among pre-service teachers.
http://ijlter.org/index.php/ijlter
21
2. To investigate the correlation among financial-literacy variables, collaborative skills, accountability, problem-solving skills and decision-making. 3. To examine pre-service teachers' views on financial literacy after attending the MOOC.
3. The Literature review 3.1 The Conceptual Framework This study is based on Mayer's Multimedia Learning Theory. In Mayer's Multimedia Learning Theory, the student engages in five important cognitive processes. Mayer (2014) stated that meaningful learning from words and pictures occurs, when the learner engages in five cognitive processes, which are: 1. Choosing the relevant words for processing in verbal-working memory; 2. Selecting relevant images for processing in visual-working memory; 3. Organizing the selected words into a verbal model; 4. Organizing selected images into a pictorial model; and 5. Integrating the verbal and pictorial representations with prior knowledge. These cognitive processes determine what information is to be drawn upon in the working memory, when knowledge is retrieved from one’s long-term memory and integrated with new information, in order to construct new knowledge. The new knowledge in the working memory will be relocated and transferred to long-term memory through the process of encoding (Mayer, 2014). Refer to Figure 1. MOOCs have met the criteria of Multimedia Learning; since they are colorful, animated pictures and can transform words into pictures. Furthermore, students can cope with the knowledge of financial literacy by utilising their tacit knowledge.
Multimedia Presentation Words
Sensory Memory Ears
Long-Term Memory
Working Memory Selecting Words
Sound
Organizing Words
Verbal Mode Integrating
Pictures
Eyes
Selecting Images
Images
Organizing Images
Prior Knowledge
Pictoral Mode
Figure 1: Conceptual Framework: Mayer Multimedia Learning Theory
3.2 Learning with MOOCs Learners worldwide have benefitted from MOOCs (Sidek et al., 2019; Gabaree et al., 2020;). During the Covid-19 pandemic, learners were able to evaluate their courses at home. Consequently, MOOCs were elevated as a new teaching method that overcame the limits of traditional teaching platforms, by allowing students to learn from any place and at any time (Wang, 2021). Learners invariably benefit from observing videos or by reading lecture notes online. This is a type of remote learning that enables students to learn outside the physical classroom. As a result,
http://ijlter.org/index.php/ijlter
22
MOOCs are created under the open-concept or open-education umbrella (Quiliano Terreros et al., 2009). This platform has demonstrated that it supports lifelong learning, in acquiring key knowledge and skills and the much-needed life-skill interactions with peers and live-project exposure (Sonwalkar & Maheshkar, 2015). Researchers have identified the key approaches to incorporate MOOCs into classroom lesson plans and schemes of work, including explicitly defining the content, selecting material and subjects, specifying the MOOC and its range, and ensuring that the scope of the MOOC is easily accessible to learners. Additional strategies, such as learning aim, objectives, teaching activities, and assessment, are also central in the preplanning and in delivering effective lessons (de Jong et al., 2019). MOOCs grades, behavioral (Phan et al., 2016) outcomes and social engagement were found to have favorable connections (Gillani & Eynon, 2014; Torres & Beier, 2018). According to these studies, educators and learning designers are encouraged to include learning activities in MOOCs relevant to career demands, in order to increase learners' extrinsic motivations (Deng et al., 2020). In this report, the researchers also said that learners' engagement might be further stimulated by cognitively engaging the course and by inserting suitable formative assessment into such tasks. The length of MOOCs should likewise be kept to a minimum, in order to maintain pupils’ attention. As with all online programmes, the most challenging aspect of MOOCs is student dropout. Students would invariably leave the course if they become demotivated; and if they fail to manage their time effectively. To compensate for this, MOOCs should last between 1 and 16 weeks. However, the MOOCs’ inventor strongly suggests that programmes should last no longer than seven weeks. Various studies reported that financial literacy could be effectively enhanced through online learning (Kalmi, 2017). All previous studies suggested that using digital tools, such as videos or games, could effectively enhance students' financial literacy. Additional studies also revealed that videos, or other digital tools, could effectively improve financial knowledge, whether in college or in high school (Zhu, 2019; Popovich et al., 2020). MOOCs with multimedia, in line with the conceptual framework of this research, were selected as being the most suitable platform for promoting financial literacy in this paper.
4. Methodology 4.1 Research Design A mixed-methods research design was used in this study. A survey design was used in quantitative research; whereas, in qualitative research, an interview was used. The researchers selected the mixed-method research method; since they felt a need to comprehend this topic throughout various research phases, and as part of the validation process (Creswell & Clark, 2017).
http://ijlter.org/index.php/ijlter
23
4.2 Population and Sampling technique The target population of this study is 100 pre-service Economics teachers in Peninsular Malaysia. Consequently, a total of 100 pre-service teachers, majoring or minoring in Economics, were selected as quantitative samples. Each sample selected was between the ages of 20 and 21. A random sampling procedure was used to select the respondents. The target respondents were represented with a code number, and the selection was based on the randomly generated number by using Microsoft Excel. A method known as purposive sampling was employed for the interviewed respondents. Five teacher candidates were also chosen for a follow-up interview. The goal of the follow-up interview was to ensure the triangulation of the data. The participants were selected on the basis of their daily expenditure. Although gender was not considered in this study, two boys and three girls were selected to participate in the interview. 4.3 Instrument The principal researcher created a 50-item closed-ended questionnaire, with five Likert scales, which were then checked by professionals: Financial literacy, collaborative skills, accountability, problem-solving, and decision-making were the five main components of this set of questionnaires. There were ten questions in each section. The questionnaire's reliability was tested in a pilot study with 53 pre-service teachers; while a group of information-technology experts tested the MOOC platform's functionality. All the items in this survey were found to be reliable and acceptable, with a value of 0.70 or above (Sekaran & Bougie, 2010); because the questionnaire's Cronbach Alpha was 0.832. The pre-service teachers were given an online assessment with 20 questions regarding financial literacy, in order to test their content understanding. 4.4 The Research Procedure All of the pre-service teachers attended a six-week MOOC course. During handson activities, the candidates were divided into 25 groups of four each. This learning activity should improve their collaborative learning via peer dialogue and debate. The following was the schedule for the six-week course: All pre-service teachers were required to attend a two-week course on understanding fundamental financial literacy for the first two weeks. The preservice teachers progressed to learning how to incorporate financial literacy into their lessons. Financial planning was also taught to the pre-service teachers. On week three, the pre-service teachers were asked to choose a secondary school economics sub-topic and to construct teaching activities relevant to financial literacy. From weeks three to week six, the researchers peer-shared videos that they had produced. The pre-service teachers were responsible for designing teaching aids, based on their lesson plan in week four. Pre-service teachers could construct a game on conserving money and knowledge about trust and bonding for money and banking. They evaluated their peers' work from different groups and provided
http://ijlter.org/index.php/ijlter
24
constructive feedback and remarks in week five. During the final unit of week six, the pre-service teachers were required to prepare a short case relating to their daily life, together with a financial literacy component. During this week, preservice teachers were required to complete an online quiz and a questionnaire regarding financial literacy. Finally, five pre-service teachers, from various groups, were selected to participate in the interview in week seven. The five pre-service teachers were all third-year students at a public university. Three females and two males were selected as interviewees. They all participated in the MOOC as self-directed learners, devoting at least one hour every week to the course. Although they were from various groups, they all completed and submitted the tasks on time. 4.5 The Data-Collection Method After watching all of the video clips, the questionnaires were handed out in person, and the data were collected anonymously by using a code-number system. This study was also conducted prior to the global pandemic; and it was modelled on a real-life classroom scenario. Five pre-service teachers were chosen to attend the interview session separately, and each session lasted about 30 minutes. With the permission of the pre-service teachers, the interviews were recorded. The following were the open-ended interview questions: • How do you incorporate financial literacy into teaching? • Do you feel that your collaborative skills have improved after attending MOOC? • Could you understand the importance of accountability in financial literacy? • Could you tell me about the problem-solving skill that you learnt, after attending the course? • Do you agree that decision-making is an essential skill in financial literacy? And if so, why? 4.6 The Data Analysis The quantitative data were analyzed with descriptive and inferential statistics. The five main components of the questionnaire were used to compare the mean score; and a t-test was used to determine the significant difference after implementing MOOC among the pre-service teachers. An unstandardized coefficient model was developed to calculate the coefficient. The Pearson correlation was also employed, in order to check the relationship among the variables. SPSS version 23 was used to compute the data. The qualitative data were then analyzed via content analysis. Content analysis techniques were used to interpret the transcripts of each subject, both explicitly and implicitly throughout the analytical process (Krippendorff, 2019). It commenced with the process of making transcriptions for five interviews, after the completion of three interview sessions. According to Miles and Huberman (2019), the data that were collected from the interviews should be systematically compiled after transcriptions had been made. The interview recording was
http://ijlter.org/index.php/ijlter
25
listened to several times, in order to ensure the comprehension thereof; and then the data were stored electronically.
5. The Results 5.1 To examine the effectiveness of MOOC for promoting financial literacy among pre-service teachers The survey results were examined and summarized in Table 1. According to the questionnaire summary, the subscales of the surveys were financial literacy, collaborative skills, accountability, problem-solving, and decision-making. The highest average mean score for ten categories was problem-solving (M=3.80, SD=.776), followed by financial knowledge (M=3.75, SD=0.772). Accountability received the lowest mean score (M=3.41, SD=0.934). Table 1: Summary of the Questionnaire on MOOC Sub-scale Financial Knowledge Collaborative Skills Accountability Problem Solving Decision-Making
Item 10 10 10 10 10
Average Mean Score 3.75 3.36 3.41 3.80 3.52
S.D 0.772 1.033 0.934 0.776 0.846
The results in Table 2 revealed that both the standardized and the unstandardized beta coefficients were significant. The value of the unstandardized coefficients was 69.238 (SD=6.837), and the t value was 10.097 (p<0.05). On the other hand, the standardized coefficients showed a value of 0.073 with a t value of 0.721 (p<0.05). According to the unstandardized coefficient, increases in the financial literacy of one unit should boost students' use of MOOC. The variance in financial literacy is examined through the semi-partial correlation = 20% (0.4522). The Tolerance value is 0.751,which higher than 0.63 (1- R2), which showed that there could not be any multi-collinearity. Table 2: Coefficients Model
Unstandardized Coefficients Β Std. error
Constant
69.238
6.857
MOOC
0.067
0.093
Standardized Coefficients Beta
0.073
t
Sig.
10.097
0.00
0.72
0.00
Correlations Zeroorder
Partial
Part
0.478
0.45
0.37
Collinearity Statistic Tolerance VIF
0.751
1.315
* significant at p < .05 DV: Financial literacy Adjusted R2 = .37
5.2 To investigate the correlation among the variables of financial literacy, collaborative skills, accountability, problem-solving and decision-making The results of the Pearson correlation are shown in Table 3. There were ten different correlations, which means that ten coefficients were statistically significant. Financial literacy was significantly correlated with collaborative skills, r = 0.118 (p < 0.05). In addition, financial literacy also correlated with accountability significant, r = 0.216 (p < 0.05). In addition, financial literacy
http://ijlter.org/index.php/ijlter
26
correlated with decision-making, r= 0.227 (p < 0.001) and problem-solving r = 0.240 (p > 0.001). On the other hand, collaborative skill was significantly correlated with all the variables. For example, collaborative skills correlated with accountability, with r = 0.288 (p < 0.001), problem-solving with r = 0.202 (p < 0.001), decision-making with r = 0.346 (p < .001). Accountability was correlated with all the variables, such as problem-solving with r = 0.369 (p <0.001), accountability with r = 0.327 (p <0.001) and decision-making r = 0.494, (p <0. 001). Table 3. Pearson Correlations among the variables Correlations
Financial Collaborative Accountability Problem literacy skills Solving Pearson Correlation Sig. (2-tailed) N Pearson Collaborative Correlation skills Sig. (2-tailed) N Pearson Correlation Accountability Sig. (2-tailed) N Pearson Correlation Problem Solving Sig. (2-tailed) N Pearson Correlation Decision Making Sig. (2-tailed) N
Decision Making
1
0.118*
0.216**
-0.240**
0.227**
100 0.118*
0.018 100 1
0.000 100 0.288**
0.427 100 0.202**
0.000 100 0.346**
0.018 100 0.216*
100 0.288**
0.000 100 1
0.000 100 0.369**
0.000 100 0.494**
0.000 100 -0.040
0.000 100 0.202**
100 0.369**
0.000 100 1
0.000 100 0.327**
-0.240** 100 .227**
0.000 100 .346**
0.000 100 .494**
100 .327**
0.000 100 1
0.000 100
0.000 100
0.000 100
0.000 100
100
Financial literacy
*. Correlation is significant at the 0.05 level (2-tailed). **. Correlation is significant at the 0.01 level (2-tailed).
5.3 Pre-service teachers' views after attending MOOC A total of five pre-service teachers attended the interview. To protect their identities, they used pseudonyms. Their opinions on financial literacy, collaboration, accountability, problem-solving, and decision-making were polled. All of these enquiries were related to the questionnaires. 5.3.1 Financial literacy Eighty per cent of the participants said that they strongly believed their financialliteracy understanding had improved, and that they could incorporate the lessons' skills. They were well-versed in how to instil financial literacy in students; and they had created appropriate teaching aids. When asked how to incorporate financial literacy into teaching, these individuals said:
http://ijlter.org/index.php/ijlter
27
"… select a suitable topic, like demand and supply, by using a video clip." (Male 1) "… I would start with a story; and I would then give the class a problem-solving task." (Female 2) "… set an online platform, with an element of financial literacy, and then allow students to have self-directed learning." (Female 3) This cohort of pre-service teachers had mastered many skills through MOOC; and they expressed their enthusiasm for MOOC learning. They had also gained financial knowledge through the platform (Deng & Benckendorf, 2021). Preservice teachers created teaching aids, shared their ideas with their colleagues, and commented on their peers' work. 5.3.2 Collaborative Skills During group discussions, one can always improve collaborative skills. MOOC provides a venue for pre-service teachers to learn and share. To finish the assignment, they had to work together. Each of the participants has a certain role to play, based on the work at hand. Each week, the group leaders were rotated among the pre-service teachers. The job had to be distributed equitably among the pre-service teachers by the leaders (Duret et al., 2019). For example, during a session of creating teaching aids, one person would prepare the video's storyline and script, two members would prepare the video, and a third member would get the task of editing the video. The successful completion relied heavily on team collaboration. When asked whether they had developed collaborative skills after attending the MOOC, they expressed their delight; and they noted that they each had contributed something: "… I did my work, according to our leader's instruction, and I completed my task within the allocated timeframe. We learnt how to help each other every week, and we learnt from each other too. Everybody acquired different strengths." (Female 1) Aside from collaborative abilities, pre-service teachers were accountable for their work. As a result, responsibility is a necessary skill in the teaching profession. It also has a significant impact on financial literacy. 5.3.3 Accountability Accountability is an important element in financial literacy (Rakow, 2019). All the pre-service teachers agreed that they understood the importance of accountability in financial literacy. They also practised it when they had completed the task given during MOOC learning. They learnt the meaning of accountability in financial literacy through the weekly videos provided. Some of them tried to incorporate the value of accountability, when designing the teaching aids. "… I put the value of accountability into my teaching aid. I made a music video related to financial literacy for the chapter on money and banking; then I incorporatedl this value." (Male 2) All of the pre-service teachers agreed that accountability in financial literacy is an important element. They practised it as well; since they completed the MOOC learning activity. Through the weekly videos, they learned about the need for
http://ijlter.org/index.php/ijlter
28
accountability in financial literacy. When creating instructional tools, some of the pre-service teachers attempted to incorporate the concept of accountability. 5.3.4 Problem-solving One of the pre-service teachers provided a proactive problem-solving example. During the MOOC, she used a case study, as a teaching tool. She asked the students to assume the scenario that they were alone, without savings or investments. The scenario included the visualization of their payday, in which a snatch robber took their handbag/wallet. Meanwhile, they had to imagine that they had received a call from their hometown, informing them that their mother was gravely ill, and required financial assistance. The question that was posted among pre-service teachers was: 'What are your options for resolving this issue?' "… I try to figure out the suitable situation for high-school students who had just started to learn financial literacy. I made the question a little trickier; and I let them think that I had set a few learning activities for me to incorporate into the lessons during my teaching practicum." (Female 1) Other pre-service teachers provided a good example of problem-solving, too. The activities that the pre-service teachers devised could be used during their practicum. By doing so, the pre-service teachers gained further insights and content knowledge through practical exposure and case-study analysis (Wright et al., 2017). 5.3.5 Decision-Making The comments of the pre-service teachers suggested that one of the components they had to acquire in financial literacy and collaborative work in MOOC was decision-making. They all agreed that financial literacy requires good decisionmaking skills. Decision-making is an essential skill in financial literacy (Sonwalker & Maheshkar, 2015). They also expressed their opinions on the necessity of deciding to complete the work. For example, they chose their lesson plan or learning activities and the teaching aids they created were based on mutual agreement and decision-making. In order to meet the requirements of incorporating financial literacy into lessons, they had to decide on a topic, what materials to use as teaching aids, and what learning activities to implement. "… I kept on discussing matters with my members, when I was the leader of the group. I cannot decide on my own; I have to consider all their abilities and strengths. Some of them are good in making videos, but others are good in animation or singing." (Male 2)
6. Discussion The effectiveness of MOOC in increasing financial literacy was investigated in this study. In descriptive statistics, the quantitative results demonstrated positive results for financial literacy. they showed a medium-high average, ranging from 3.41 to 3.80, because the MOOC was implemented in a short duration. On a shortterm basis, the influence of learning, particularly value, could not be seen. However, in terms of regression, the results were significant (t value 10.097) (p<0.05) for an unstandardized model and a t value of 0.721 (p<.05) for a standardized model.
http://ijlter.org/index.php/ijlter
29
The Pearson coefficient revealed a correlation for financial literacy. The results showed that financial literacy was statistically related to collaborative skills, accountability, problem-solving, and decision-making. The results are consistent with the literature review of Bakar and Bakar (2020) and Compen et al. (2020). It was discovered that financial literacy might well be improved through MOOCs and digital learning. MOOC is a learning platform that allows learners to learn from professionals worldwide, and thereby go enhance their abilities (Clark et al., 2017). In Mayer’s Learning Theory, students could learn new knowledge better by combining words and pictures, for example, videos in MOOC. The qualitative results revealed that MOOC could improve collaboration skills, problem-solving skills, accountability, and decision-making skills. Previous research had found that online or blended learning improves collaborative skills (Compen et al., 2020). Studies also reported that online learning is able to enhance problem-solving (Yeen-Ju et al., 2015). On the other hand, online learning also promotes accountability (Pulukuri & Abrams, 2020), as well as the decisionmaking ability of learners (Galvis, 2018; Compen et al., 2020). During the course, pre-service teachers were required to create videos, in order to obtain practical experience and to learn how to include financial literacy into the curriculum. The videos were created by pre-service teachers; and they included technology, animation, and problem-solving activities. These videos serve as a wake-up call for students interested in learning about financial literacy. Preservice teachers also chose a problem that was relevant to their students' daily lives. These qualitative results supported the conclusions of Kuntze et al. (2019) that videos are an effective online method for delivering financial literacy knowledge outside the classroom, and for reinforcing it. Additionally, videos can be used as post-class reinforcement tools for increasing students' learning interest, allowing them to study more interactively (Pulukuri & Abram, 2020). The MOOC course also provided a forum for pre-service teachers to socialize, share ideas, and engage in practical exercises. When pre-service teachers were involved in the learning process, they developed a positive outlook (Gurvitch & Lund, 2014). This mindset should benefit them in both their behaviour and in their future teaching. This finding is consistent with the research of Amagir et al. (2019). In addition, previous research mentioned that ICT significantly benefits learners (Gaboy et al., 2020), especially in financial literacy (Lusardi et al., 2015), which is similar to the results found in this study. Both studies also mentioned visual tools; videos that can enhance financial literacy more interactively. In addition, with the assistance of fingertip technology, students can learn anywhere and at any time (Shah et al., 2019). Moreover, universities can engage with schools and invite students to engage in their learning platforms with the assistance of mobile phone technology (Che Kob et al., 2020). Furthermore, previous research has found that ICT benefits learners (Gaboy et al., 2020), particularly in financial literacy (Amagir et al., 2019), which is consistent
http://ijlter.org/index.php/ijlter
30
with the results of this study. Both types of research addressed visual aids, as well as videos that can help people to learn more about money. Furthermore, students can learn anywhere and at any time, by using finger-tip technology (Shah et al., 2019). With the advancement of mobile phone technology, universities may engage with schools and ask students to participate in their learning platforms (Che Kob et al., 2020). The use of mobile phone technology might be a solution for teachers who claim that they do not have enough time to incorporate financial literacy into the curriculum because it explores a learning platform that allows self-directed learning beyond the classroom.
7. Limitations The following were some noticeable limitations in this study: Firstly, the samples of this study were restricted to pre-service teachers from major or minor economics only. Pre-service teachers from other courses were not included in this study. Therefore, studies should be conducted among different courses before any generalizations of financial literacy can be made. Secondly, inservice teachers were excluded from this study. This scenario has limited the results regarding the generalization of the efficacy of financial literacy for MOOCs.
8. Future Research Pre-service teachers should produce and share projects on various topics about financial literacy across the curricula in various subjects when using MOOCs. A possible issue to investigate the incorporation of financial literacy in different subjects, such as Accounting and Business Management will probably become the future research direction. Financial planning is another future research direction; since this issue is only a small segment of the current research. In future research, the researchers can consider doing in-depth financial literacy training for preservice and in-service teachers in terms of investment and possible retirement plans.
9. Recommendations From the results, we would like to make some recommendations that might be useful to educators and researchers. These engagements were divided into two categories: course design and participation engagement. • Course design Course design is an essential aspect of attracting and retaining participants. A robust course considering the students to get the basic knowledge and make the students learn with pleasure would result in a better success percentage. The course content could consider elements related to students' prior knowledge, whereas learning activities can design familiar activities. When educators design any courses related to financial literacy, learning activities such as videos, quizzes, and games could be a good choice for initial learning. • Participants’ engagement in participation
http://ijlter.org/index.php/ijlter
31
To make the course successful, one should try to include as many activities as possible that need to be implemented in the participation. Hand-on activities are one way to let students engage in the learning activities; for example, students can work in a group to produce a video related to financial literacy. Teachers' lively teaching approach, or group activities coulc also attract students' participation. Students could gain basic financial knowledge through a platform that attracts their interest. Another option is to send a notification or a push button to participants to inform them of the task's deadline.
10. Conclusion In this study, pre-service teachers were exposed to various events that put them in perspective and inspired them to be innovative in teaching their students' financial-planning skills and principles. Alternatively, financial planning is a vital part of children's discipline, which ought to be instilled. This study researched the various approaches to teach pre-service teachers how to instil supportive ideals in their students to become responsible for themselves. The positive results of this study are encouraging, given the literature review related to financial literacy related to collaborative skills, accountability, problem-solving and decisionmaking. The mentioned skills are transferable skills that are needed to enhance financial literacy. From the results, the study suggests that MOOC coulc promote financial literacy among pre-service teachers effectively. Therefore, educators and researchers could provide more online learning opportunities that combine financial literacy in Economics, Accounting, Mathematics, or even in languages. Personal finance is rarely discussed at school or at home; educators should seize the opportunity to teach the future generation the importance thereof. We
are assisting parents in protecting the future of their children by shaping the next generation. Acknowledgement This research has been carried out under the Fundamental Research Grants Scheme (FRGS/1/2019/SS08/UPSI/02/3) provided by the Ministry of Education of Malaysia. The authors would like to extend their gratitude to Universiti Pendidikan Sultan Idris (UPSI), that helped to manage the grant. The authors also would like to thank Vasiliki Kondou Watson for editing in this paper.
11. References Angel, S. (2018). Smart tools? A randomized controlled trial on the impact of three different media tools on personal finance. Journal of Behavioral and Experimental Economics, 74, 104-111. https://doi.org/10.1016/j.socec.2018.04.002 Amagir, A., Groot, W., Brink, H. M. V. D., & Wilschut, A. (2019). SaveWise: the design of a financial education program in the Netherlands. Citizenship Social & Economics Education, 18(2), 100-120. https://doi.org/10.1177/2047173419870053 Bakar, M. Z. A., & Bakar, S. A. (2020). Prudent financial management practices among Malaysian Youth: The moderating roles of financial education. The Journal of Asian Finance, Economics and Business, 7(6), 525-535. http://doi.org/10.13106/jafeb.2020.vol7.no6.525
http://ijlter.org/index.php/ijlter
32
Caixeta, C. C., Nascimento, J. A., & Abrem, A. M. (2016). The influence of social media on consumer perception on brand. Espacious, 37(27), 17-26. http://dx.doi.org/10.5195/emaj.2019.173 Che Kob, C. G., Kannapiran, S., & Shah, A. (2020). The usage of mobile learning: comparative studies among technical and vocational education students in selected universities. International Journal of Interactive Mobile Technologies, 14(5), 203-209. https://doi.org/10.3991/ijim.v14i05.13355 Clark, K. R., Vealé, B. L, Watts, L. K. (2017). A Review of the Use of Massive Open Online Courses (MOOCs) in Medical Imaging Education. The Internet Journal of Allied Health Sciences and Practice, 23, 15(2), 1-6. https://nsuworks.nova.edu/cgi/viewcontent.cgi?article=1659&context=ijahsp Compen, B., Witte, K. D., & Schelfhout, W. (2020). The impact of teacher engagement in an interactive webinar series on the effectiveness of financial literacy education. British Journal of Educational Technology, 1-15. http://doi.org. 10.1111/bjet.13013 Cordero, J. M., & Pedraja, F. (2019). The effect of financial education training on the financial literacy of Spanish students in PISA. Applied Economics, 51(16), 1679-1693. http://doi.org.10.1080/00036846.2018.1528336 Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed-methods research (3rd ed.). California: Sage Publication. de Jong, P. G. M., Pickering, J. D., Hendriks, R. A., Swineerton, B. J., Goshtasbpour, F., & Reinders, M. E. J. (2019). Twelve tips for integrating massive open online course content into classroom teaching. Medical Teacher, 42(4), 393-397. https://doi.org/10.1080/0142159X.2019.1571569 Deng, R., Benckendorff, P., & Gannaway, D. (2020). Linking learner factors, teaching content and engagement patterns with MOOC learning outcome. Journal of Computer Assisted Learning, 1-18. https://doi.org/10.1111/jcal.12437 Deng, R., & Benckendorff, P. (2021). What are the key themes associated with the positive learning experience in MOOCs? An empirical investigation of learners' ratings and reviews. International Journal of Education Technology in Higher Education, 18 (9), 1-28. https://doi.org/10.1186/s41239-021-00244-3 Duret, D., Christley, R., Denny, P., & Senior, A. (2019). Collaborative learning with peerwise. Research in Learning Technology, 26, 1-14. http://doi:0.25304/rlt.v26.1979 Financial-Education Network. (2019). Empowering financial freedom. https://www.fenetwork.my/about/ Gabaree, L., Rodeghiero, C., Presicce, C., Rusk, N., & Jain, R. (2020). Designing creative and connected online learning experiences. Information and Learning Sciences, 121 (7/8), 655-663. https://doi.org/10.1111/jcal.12437 Gaboy, R. G., Mabalay, M. C., Mananghaya, M. E., Mercado, M. G. M., & Romblon, B. M. (2020). Coping with the new norm: ICT-Pedagogy integration awareness and competency of TEI faculty. Journal of Research, Policy & Practice of Teachers & Teacher Education, 10(2), 49-62. http://doi.org.10.37134/jrpptte.vol10.2.4.2020 Galvis, A. H. (2018). Supporting decision-making processes on blended learning in higher education: literature and good practices review. International Journal of Educational Technology in Higher Education, 15(25), 2-38. http:// 10.37134/jrpptte.vol10.2.4.2020 Gillani, N., & Eynon, R. (2014). Communication patterns in massively open online courses. The Internet and Higher Education, 23, 18-26. http://dx.doi.org/10.1016/j.iheduc.2014.05.004 Gurvitch R., & Lund J. (2014). Animated video clips: Learning in the current generation. Journal of Physical Education, Recreation and Dance, 85(5), 8–17. https://doi.org/10.1080/07303084.2014.897566
http://ijlter.org/index.php/ijlter
33
Haron, H., Yusof, A. R. M., Samad, H., Ismail, N., Juanita, A., & Yusof, H. (2019). The platform of MOOC (Massive Open Online Course) on open learning: Issues and challenges. International Journal of Modern Education, 1(3), 01-09. http://doi.org/10.35631/ijmoe.13001 Hassan, H., Mailok, R., & Johan, R. (2015). UPSI students' perceptions on the use of ICT in learning: comparison between knowledge testing and task-based testing on ICT competency. Journal of ICT in Education, 2, 5577. https://ejournal.upsi.edu.my/index.php/JICTIE/article/view/2600 Haneger, R., & Cude, B. J. (2019). Financial literacy of high school graduates: Long and short-term financial behaviour by age group. Journal of Family and Economic Issues, 40, 564-575. http://10.1891/1052-3073.27.1.3 Kalmi, P. (2017). The effects of financial education: Evidence from Finish lower secondary schools. Economic notes, 9999 (9999), 1-34. http://dx.doi.org/10.1111/ecno.12114 Krippendorff, K. (2019). Content analysis: An Introduction to its methodology. Sage publication. https://us.sagepub.com/en-us/nam/content-analysis/book 258450 Khan, H. U. R., Lim, C. K., Tan, K. L. & Khan, M. R. (2019). Location-based social media content: A conceptual paper on travellers’ purchase decisions. Management Research Journal, 8, 30-40. https://doi.org/10.37134/mrj.vol8.3.2019 Khoo, Y. Y., & Fitzgerald, R. (2017). Pocket cartoons: Learning financial literacy with mobile cartoons in Malaysia. International Journal of Mobile and Blended Learning, 9(4), 49-64. https://doi.org/10.37134/mrj.vol8.3.2019 Kumar, J. A., & Al-Samarraie, H. (2018). MOOC in the Malaysian higher education institutions: The instructors' perspectives. The Reference Librarian, 1-16. https://doi.org/10.1080/02763877.2018.1458688 Kuntze, R., Wu, C., Ross, W., & Whuang, Y. (2019). Improving financial literacy in college of business students: Modernizing delivery tools. Emerald Insight, 37(4), 976-990. https://doi.org/10.1108/IJBM-03-2018-0080 Lusardi, A., Samek, A., Kapteyn, A., Glinert, L., Hung, A., Heinberg, A. (2015). Visual tools and narratives: New ways to improve financial literacy. Journal of Pension Economics and Finance, 16(3), 297-323. https://doi.org/10.1017/S1474747215000323 Malaysian Communications and Multimedia Commission. (2015). Hand phone user survey 2014. Kuala Lumpur. https://www.mcmc.gov.my/ms/home. Mayer, R. E., Lee, H., & Peebles, A. (2014). Multimedia Learning in a Second Language: A Cognitive Load Perspective. Applied Cognitive Psychology, 28(5), 653– 660. https://doi.org/10.1002/acp.3050 Miles, M. B., & Huberman, A. M (2019). Qualitative Data Analysis 4th Edition. USA: Sage Publications. Mohd Aziz, N. I., & Kassim, S. (2020). Does financial literacy really matter for Malaysians? A review. Advanced International Journal of Banking, Accounting and Finance, 2(2),1320. http://dx.doi.org/10.35631/AIJBAF.22002 Murugiah, L. (2016). The level of understanding and strategies to enhance financial literacy among Malaysians. International Journal of Economics and Financial, 6(S3), 130-139. https://www.econjournals.com/index.php/ijefi/article/view/2621 Nurul Afiqah. (2016). The relationship between the level of financial literacy and investment by decision-making millennials in Malaysia. Taylor's Business Review, 6, 39-47. http://dx.doi.org/10.18488/journal.137.2020.42.110.126 Phan, T., McNeil, S. G., & Robin, B. R. (2016). Students' patterns of engagement and course performance in a massive open online course. Computers & Education, 95, 36-44. https://doi.org/10.1016/j.compedu.2015.11.015
http://ijlter.org/index.php/ijlter
34
Popovich, J. J., Loibl, C., Zirkle, C., & Whittington, M. S. (2020). Community-college students' response to a financial literacy intervention: An exploratory study. International Review of Economics Education, 34 (2020), 1-15. http://dx.doi.org/10.1016/j.iree.2020.100182 Prajapati, R., Sharma, F. B., & Sharma, F. D. (2017). Significance of life skills education. Contemporary Issues in Education Research, 10(1), 1-6. https://eric.ed.gov/?id=EJ1126842 Pulukuri, S., & Abrams, B. (2020). Incorporating an online interactive video platform to optimize active learning and improve student accountability through educational videos. Journal of Chemical Education, 97, 4505-4514. https://pubs.acs.org/doi/10.1021/acs.jchemed.0c00855 Rakow, K. C. (2019). Incorporating financial literacy into the accounting curriculum. Accounting Education, 28(4), 384-400. https://doi.org/10.1080/09639284.2019.1578247 Sawatzki, C., & Sullivan, P. (2017). Teachers' Perceptions of financial literacy and the implications for professional learning. Australian Journal of Teacher Education, 42(5), 51-65. http://dx.doi.org/10.14221/ajte.2017v42n5.4 Sekaran, U., & Bougie, R. (2010). Research methods for business: A skill building approach (5th ed). New York, NY: John Wiley & Sons Ltd. Sidek, S. F., Mohamad Yatim, M. H., Ariffin, S. A., & Nurzid, A. (2019). The acceptance factors and effectiveness of MOOC in the blended learning of computer architecture and organization course. Universal Journal of Education Research, 8(3), 909-915. https://doi.org/10.30534/ijatcse/2019/3281.32019 Shah, A., Suhailiezana, Che Kob, C. G., & Khairudin, M. (2019). Effectiveness of M Learning applications for design and technology subject. International Journal of Interactive Mobile Technologies, 13(10), 120-133. http://dx.doi.org/10.3991/ijim.v13i10.11324 Torres, W. J., & Beier, M. E. (2018). Adult development in the wild: The determinants of autonomous learning in a Massive Open Online Course. Learning and Individual Differences, 65, 207–217. https://doi.org/10.1016/j.lindif.2018.06.003 Qaffas, A. A., Kaabi, K., Shadiev, R., & Essalui, F. (2020). Towards an optimal personalization strategy in MOOC. Smart Learning Environment, 7(14), 1-18. https://doi.org/10.1186/s40561-020-0117-y OECD. (2013). Education at a glance 2013: OECD indicators. OECD. http://www.oecd.org/education/eag2013%20(eng)FINAL%2020%20June%2020 13.Pdf OECD. (2014). Education at a glance. http://www.oecd.org/daf/fin/financialeducation/financial-education-for-youth.htm Quiliano Terreros, A., Del Carmen, R., Jesús, P., & Durán, B. (2009). Systematic mapping study 2012-2017: Quality and effectiveness measurement in MOOC. Turkish Online Journal of Distance Education, 20 (1), 223247.http://hdl.handle.net/11285/632793 Wang, Z. (2021). Financial management teaching mode based on MOOC under the background of informatization. In: Sugumaran V., Xu Z., Zhou H. (Eds.) Application of Intelligent Systems in Multi-modal Information Analytics. MMIA 2020. Advances in Intelligent Systems and Computing, 1233. Retrieved from https://link.springer.com/book/10.1007/978-3-030-15740-1 Wright, M. C., Bergom, I., & Bartholomew, T. (2017). Decreased class size, increased active learning? Intended and enacted teaching strategies in smaller classes. Active Learning in Higher Education, 1-17. https://doi.org/10.1177%2F1469787417735607
http://ijlter.org/index.php/ijlter
35
Yeen-Ju, T., H, Neo, M., & Salvaretnam, B. (2015). Enhancing problem-solving skills in an authentic blended learning environment: a Malaysian context. International Journal of Information and Education Technology, 5 (11), 841-846. http://dx.doi.org/10.7763/IJIET.2015.V5.623 Zhu, A. Y. F. (2018). Financial literacy among Hong Kong's Chinese adolescents: testing the validity of a scale and evaluating two conceptual models. Youth & Society, 1– 26. https://doi.org/10.1177%2F0044118X17753813 Zhu, A. Y. F. (2019). School financial education and parental financial socialization: Findings from a sample of Hong Kong adolescents. Children and Youth Service Review, 107, 1-10. http://doi.org/10.1016/j.childyouth.2019.104532
http://ijlter.org/index.php/ijlter
36
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 36-49, August 2021 https://doi.org/10.26803/ijlter.20.8.3 Received Mar 29, 2021; Revised Jul 30, 2021; Accepted Aug 30, 2021
The Relationships between Experience, Qualification and Subject Specialization and Content Knowledge Mastery of Economic and Management Sciences Teachers: A Case of Accounting Teaching Habasisa Molise* University of Limpopo, Polokwane, South Africa Department of Education Studies, School of Education https://orcid.org/0000-0002-3971-0718
Abstract. This paper investigates the relationship between teaching experience, qualification, and subject specialization on the content mastery of teachers of financial literacy to Grade 9 learners. The sample consisted of 89 Grade 9 teachers of financial literacy in the Thabo Mofutsanyana education district of South Africa. Data was collected using a questionnaire that elicited biographical data and required teachers of financial literacy to answer questions on a variety of accounting topics, to diagnose the effect of teaching experience, qualification, and subject specialization on the content knowledge mastery of the teachers. The data revealed three distinct themes, namely, solving the accounting equation, understanding accounting concepts, and the connection between topics and application of accounting principles. The items of the questionnaire were clustered into three constructs: accounting equation, accounting concepts, and accounting principles and subsidiary journals. The results show that having experience of teaching the subject, having relevant qualifications, and having a minor or major subject specialization in financial literacy enhances teachers’ content knowledge and competence. This result suggests that teachers of financial literacy should have a minimum of three years of teaching experience before they teach senior classes, possess a relevant diploma or degree, and have had accounting as a subject until at least the second to the third year of their studies. In general, teachers should not be expected to teach subjects they did not specialize in, and their level of experience should be a determining factor in workload/subject allocation for teachers, especially for accounting (also known as financial literacy). Keywords: accounting teaching; content knowledge; teaching experience; subject specialization; financial literacy
*
Corresponding author: Habasisa Molise; Email: habasisa.molise@ul.ac.za
©Author This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
37
1. Introduction The paper explores the correlation between teaching experience, qualifications and subject specialization and the content mastery of teachers of financial literacy (FL) to Grade 9 learners. FL is a component of the subject economic and management sciences (EMS), commonly known as accounting in South Africa (Molise, 2020). Generally, teaching experience, qualifications and subject specialization are viewed as important variables, which contribute to the content mastery of teachers (Antony et al, 2019; Bamidele & Adekola, 2017). When teachers are selected for employment, these variables may serve to determine the suitability of the teachers for teaching certain subjects (Bamidele & Adekola, 2017). Much research has been done on the effect of teaching experience, qualifications and subject specialization on learners’ academic achievement (Antony et al., 2019; Donkoh, 2017; Ladd & Sorensen, 2017), yet we do not know whether teaching experience, qualification, and subject specialization are related to accounting content mastery by teachers (Emmanue & Ambe, 2014; Musau & Abere, 2015). Subject specialization level refers to the highest qualification obtained in the subject by a teacher (Myrberg et al., 2018). I argue that, in order to teach accounting, teachers should have studied accounting for at least two or three years of a diploma or degree qualification. The South Africa, the minimum qualification for a teacher is a Bachelor’s degree. However, most teachers at rural schools lack relevant qualifications to teach EMS, and they teach EMS because of their teaching experience in subjects such as economics and business studies (Ngwenya, 2020). Thus, EMS teachers struggle to teach FL, because schools do not consider teachers’ qualifications and subject specializations when subject allocations are done (Modise & Letlhoenyo, 2020). Studies have investigated the effect of teachers’ experience of teaching, and their qualifications, to determine if these characteristics contribute to learner achievement (Nixon et al., 2016; Podolsky et al., 2019; Papay & Kraft, 2015), and have found that the number of years spent teaching a particular subject, and qualifications relating to this subject, do not necessarily translate into academic achievement of learners (Nixon et al., 2016; Samad & Nurusus, 2015). The studies found that several factors, in addition to experience and qualifications, could contribute significantly to academic achievement of learners, and the teaching strategies and methods teachers use (Podolsky et al., 2019; Hatlevik, 2017; Bamidele & Adekola, 2017). Studies found that teachers teach subjects in which they did not major, and that they lack the requisite experience and qualifications to teach those subjects (Modise & Letlhoenyo, 2020). This situation compromizes learner achievement, also in the subject of FL. Ngwenya (2019) appreciates the importance of teachers’ content knowledge mastery of accounting, which ensures that content is correctly transferred to learners. Accordingly, Samad and Nurusus (2015) argue that learners develop negative attitudes towards learning accounting, because it is presented in an abstract manner. Donkoh (2017) acknowledges that teachers are passionate about teaching, and suggests that teachers work tirelessly to make sure that EMS learners are taught, even though teachers may have poor content mastery of the subject. The findings reported above motivated this researcher to
http://ijlter.org/index.php/ijlter
38
explore whether teachers’ experience of teaching accounting, their teaching qualifications and whether they had specialized in accounting contribute to their content knowledge mastery. This article focuses on the relationship that teaching experience, qualification, and subject specialization have with content mastery of teachers of FL to Grade 9 learners, as expressed by knowledge of the accounting equation, accounting concepts, and accounting principles.
2. Literature review In this section, literature is presented to determine whether teaching experience, qualification, and subject specialization in accounting contribute to content knowledge mastery of teachers of EMS to Grade 9 learners. Shulman (1986) distinguishes between different kinds of knowledge that an effective teacher should possess. These include knowledge about the subject matter they are teaching (content knowledge), knowledge of specific strategies for teaching a particular subject matter (pedagogical content knowledge, PCK), and knowledge of the materials and media by which instruction and assessment are carried out (curricular knowledge). Content knowledge involves knowledge of the substance of the field; specialization concepts, principles, and procedures, and the relationships between these elements. Thus, “teachers must have an in-depth knowledge of the specific accounting topics that they teach as well as the accounting that their learners will learn in the future as teachers’ subject knowledge impacts their behavior and thus indirectly affects learner achievement” (Molise, 2020, p. 460). Several studies identified factors that can be associated with problems relating to content knowledge mastery of accounting teachers (Melo et al., 2020; Modise, 2016; Peter et al., 2017). These factors are classified as misconceptions, difficulties associated with defining accounting concepts, inability to manipulate accounting equations, and difficulty in applying accounting principles and procedures (Modise & Letlhoenyo, 2020). These studies confirm that misconceptions are generally undesirable and must be identified and addressed (Melo et al., 2020; Peter et al., 2017). It is assumed that the reason for insisting on identifying and addressing misconceptions is to enhance the process of teaching and learning through effective content mastery by accounting teachers (Molise, 2020). I, thus, argue that, to improve learners’ academic achievement, teachers should be able to present content in an appropriate way, and should ensure that the content is simplified by using context-based examples that learners can relate to when they learn about the accounting equation, concepts and principles. Antony et al. (2019, p. 1) sought to assess secondary school teachers’ characteristics in relation to qualifications and experience and found that teacher qualifications and teaching experience have a significant effect on biology teachers’ TPACK. Findings of research into teacher qualifications (that is, the type of degree) are inconclusive. Some studies found a correlation between learner achievement and qualifications (Croninger et al., 2007; Brante, 2013), while others found negative effects (Musau & Abere, 2015). Thus, “some argue that the
http://ijlter.org/index.php/ijlter
39
requirement of a second degree raises the cost in terms of teacher education and the time it involves and may prevent quality candidates from choosing this profession” (Nixon et al., 2016, p. 1636). Teacher qualification refers to skills and knowledge an individual brings to improve teaching and learning conditions (Samad & Nurusus, 2015), and it is suggested that a teacher has the ability to transfer knowledge to learners based on institutional recognition (Emmanue & Ambe, 2014). In this study, teaching qualification relates to formal qualifications possessed by teachers, at any tertiary level. Several studies confirm the importance of subject specialization (Nixon et al., 2015; Emmanue & Ambe, 2014). These studies report that teachers have limited accounting content knowledge because they are generalists rather than specialists (Nixon et al., 2015), and they choose to spend their time and effort teaching other subjects. Secondary school teachers have to complete a specialization in their specialty subject before teaching it (Modise & Letlhoenyo, 2020); the area of study is considered to be the teacher’s area of expertise. In the senior phase, teachers are not required to have specialized in a subject, and it is doubtful whether teachers can show the same level of competence in all subjects (Modise, 2016). When they teach accounting, teachers need to focus on helping learners visualize accounting concepts and procedures (Qhosola, 2015) by using pictures and diagrams, rather than focusing only on doing calculations correctly (Molise, 2020). Several research studies report a positive relationship between teachers' preparation in the subject matter they later teach, and student achievement (Podolsky et al., 2016, p. 88), while others have less unequivocal results (Modise & Letlhoenyo, 2020). DarlingHammond et al. (2017) argue that there is a correlation between teacher preparation and student achievement, and it can sometimes be negative. Thus, Myrberg et al. (2018, p 12) “find a positive relationship in mathematics, but none in science”. Also, Nixon et al. (2016, p 38) “report a positive relationship between student achievement and teachers’ majoring in mathematics”. Musau and Abere (2015, p. 84), however, found that having majored in mathematics does not affect learner performance, and could have a significant negative effect on teachers with more coursework in physical science. I argue that, when teachers have a limited conceptual understanding of accounting content knowledge, it influences their teaching effectiveness and ability to refine their mathematics instruction to match students’ achievement. Therefore, teachers must teach the content in the simplest manner, to enable learners to understand it. The next section will focus on the PCK model, which was adapted to interrogate the effect that teaching experience, qualification, and subject specialization have on the content knowledge mastery of accounting of teachers teaching FL to Grade 9 learners.
3. Conceptual framework The PCK model is used in this paper to investigate the content mastery of teachers of FL, which the proponent of this model, Shulman, refers to as the “missing paradigm” in research on teaching and teacher knowledge (1986). PCK
http://ijlter.org/index.php/ijlter
40
emphasizes representations and conceptions/misconceptions (McKlin et al., 2019), as used in this paper, “broadened ideas about how knowledge might matter to teaching” (Kuhn et al., 2016, p. 9), which shows that “it is not just knowledge of content, on the one hand, and knowledge of pedagogy” (Ngwenya, 2019, p. 22). There is a need for a relationship between content and pedagogy to exist (HungHsi, 2017). Content knowledge comprises “knowledge of the subject and its organizational mechanisms” (Qhosola, 2015). Modise and Letlhoenyo (2020) believe that comprehending a teaching subject involves more than knowing its facts and concepts. According to Ngwenya (2020), educators must have full knowledge of institutional expectations of what is right and wrong to teach (Rahmi, 2018; Qhosola, 2015, p. 216). Moreover, the instructor is required to grasp why a specific topic is essential to a field, whilst another topic may be peripheral (Molise, 2020). This framing assisted me to explore how teaching experience, qualifications, and subject specialization contribute to the content mastery of EMS teachers in relation to the accounting equation, accounting concepts, and principles, as entrenched throughout the EMS curriculum and South Africa’s Curriculum and Assessment Policy Statement (CAPS). The framework assisted me to understand how teachers' orientations to content influenced the way they taught that content.
4. Method This descriptive study employed a quantitative research approach to investigate the effect teaching experience, qualifications, and subject specialization have on the content knowledge mastery of FL teachers. A descriptive research study, according to Hlalele and Mosia (2020, p. 112), depicts “the particular characteristics of a scenario, a social context, a connection, or a depiction of a phenomena as it occurs naturally”. The approach used in this study helped “to answer questions about the relationships among the measured variables such as teaching experience, qualifications and subject specialization with the purpose of explaining, predicting and controlling phenomena” (Bamidele & Adekola, 2017, p. 3). It also helped me to determine the relationship between independent and dependent variables, with the aim of confirming a hypothesis about a phenomenon. The research was carried out in the Free State province's ThaboMofutsanyane education district. All Grade 9 teachers were requested to participate by completing a content-based FL questionnaire. I informed the participants of their rights, such as that they could disengage from the study without consequences, and that data generated through this research would be used for research purposes only (Touron et al., 2018). Therefore, the purpose of this study was to investigate the relationship between teaching experience, qualifications and subject specialization and the content mastery of FL teachers. The following sections will provide more details about the participants. 4.1 Profile of participants The population comprised Grade 9 EMS teachers (n=97) at rural intermediate schools in Thabo Mofutsanyane education district in the Free State province, South Africa; the study had 89 participants who were randomly selected. Data
http://ijlter.org/index.php/ijlter
41
was generated by administering a questionnaire to EMS teachers. The questionnaire comprised two sections; the first section had 25 accounting questions, and the second elicited information on the biographical characteristics of teachers. Table 1: Participants’ demographic information Demographic variables Experience
Category Less than 3 years 3 to 5 years 10 to 15 years 15 years and above
Qualification(s)
Accounting level
Degree/Diploma
n 64 15 7 3 58
% 72% 17% 9% 3% 65%
Degree plus PGCE* Honors degree
29 2
33% 2%
No accounting First Year Second Year Third Year
25 29 21 14
28% 32% 24% 16%
*Postgraduate Certificate in Education
4.2 Instrumentation A closed-ended questionnaire was administered to collect data on the relationships teaching experience, qualifications and subject specialization have on the content mastery of teachers who teach FL to Grade 9 learners. The questionnaire teased the problem areas of accounting topics, and was easy to complete, yet effective in collecting data. After revision, the 23-item content-based questionnaire, with its four subscales/domains, covered the following subscales/domains: accounting equation; accounting concepts; accounting principles and FL (questions 9, 10, 12, 16, 18). The questionnaire elicited biographical information (gender, age, teaching experience, qualifications, and subject specialization), to contextualize the teachers’ answers and content challenges. 4.3 Data analysis I used Statistica for data analysis. The mode of data analysis (Gaertner & Brunner, 2018) underpinned the goal of the study, which was determining the relationships between teaching experience, qualification and subject specialization and the content knowledge mastery of teachers of FL to Grade 9 learners. Furthermore, the independent and dependent variables were also determined to either confirm or dispute the hypothesis relating to the phenomenon (Chang, et al., 2020; Gudmundsdottir & Hatlevik, 2018; Mosia, 2014). 4.4 Validity and reliability The tools used for data collection by this study were developed by a scholar at the University of the Free State, a senior accounting subject advisor who has 25 years of experience teaching and researching in the field of education. Officials of the Free State Education Department and the Educational, Training and Practices
http://ijlter.org/index.php/ijlter
42
Development Sector Education and Training Authority evaluated and commented on the questionnaire. These government authorities were asked to verify whether the instrument did actually relate to the material necessary for FL education in Grade 9. The Cronbach’s Alpha coefficient was used to assess the survey's reliability (Hlalele & Mosia, 2020). A reliability value of 0.870 or above is deemed acceptable, and a reliability coefficient greater than 0.90 indicates that some questions are redundant, as they are evaluating the same question, but in a different form.
5. Results This section presents data on the relationships between the variables of teaching experience, qualifications, and subject specialization level and the content mastery of teachers teaching FL to Grade 9 learners, concerning the constructs of accounting equation, accounting concepts, and accounting principles. 5.1 Solving of the accounting equation Figure 1 depicts the proportion of teachers who gave incorrect answers in response to questions on solving accounting equations – a theme in Grade 9 FL – according to participants’ teaching experience, qualifications, and subject specialization. The teaching experience of the teachers ranged from two years to more than 15 years, while their qualifications ranged from junior degrees with and without a teaching qualification, to honors degrees. The university level to which they had studied accounting ranged from zero to third-year level, with few of the teachers having studied accounting up to second-year level.
Figure 1. Incorrect answers on questions relating to accounting equation
Data in Figure 1 shows that, firstly, teachers with little teaching experience (no more than 2 years), and who did not have accounting as their subject specialization, had the highest number of incorrect answers - between 9 and 12. This can be compared to teachers who had done accounting to first-year level, who had between 3 and 6 incorrect answers. In contrast, teachers with the same
http://ijlter.org/index.php/ijlter
43
teaching experience (up to 2 years), a degree plus PGCE as qualification, but who had studied accounting up to their third year, had the lowest proportion of incorrect answers – at most 3. Teachers with up to 2 years of teaching experience, with a degree but no accounting specialization, had a higher proportion of incorrect answers than teachers with the same experience and years of accounting specialization, who had degrees plus PGCE. Secondly, teachers with teaching experience of maximum 3 years, with a degree plus PGCE, but without accounting as subject specialization, had 6 to 9 incorrect answers, while teachers with the same teaching experience and qualifications, who had specialized in accounting up to third year, gave the smallest proportion of incorrect answers – maximum 3. Teachers with the same teaching experience, with an honors degree and accounting at first-year level, had a lower proportion of incorrect answers than similarly experienced and qualified teachers who had not specialized in accounting at first-year level. Thirdly, teachers with teaching experience of up to 5 years, with an honors degree and accounting at first-year level, gave the fewest incorrect answers – no more than 3. Teachers with the same qualification and first-year accounting, and between 11 and 15 years’ experience, gave no more than 3 incorrect answers. Similarly, teachers with more than 15 years of experience, with an honors degree, but no specialization in accounting, gave few incorrect answers. 5.2 Comprehension of accounting concepts and relationships between topics Figure 2 depicts the proportion of teachers who gave incorrect answers to questions in the questionnaire on comprehension of accounting concepts and relationships between topics and themes of Grade 9 FL, in relation to teaching experience, qualifications and subject specialization.
Figure 2. Incorrect answers on questions relating to understanding of accounting concepts
Data in Figure 2 shows that, firstly, teachers who up to two years teaching experience and degrees plus PGCE, and who had specialized in accounting up to
http://ijlter.org/index.php/ijlter
44
third-year level, had between 6 and 9 wrong answers. Secondly, teachers with up to three years teaching experience and degrees, but who had not had accounting as their subject specialization, had the highest proportion of incorrect answers, namely between 9 and 12. Thirdly, teachers with up to five years teaching experience and a degree plus PGCE, but who had not done accounting at first-year-level, had the highest proportion of incorrect answers (between 9 and 12). In turn, teachers with the same teaching experience and qualification, but with accounting at third-year level, had a lower proportion of incorrect answers – maximum 3. In contrast, teachers with the same teaching experience but with accounting at third-year level, and a degree, had a higher proportion of incorrect answers than those with the same experience and specialization, but with a degree plus PGCE. Fourthly, teachers with teaching experience of between 11 and 15 years, holding a degree plus PGCE, who had not specialized in accounting, had a higher proportion of incorrect responses than teachers with the same teaching experience and accounting subject specialization level, yet with a teachers’ diploma. Lastly, teachers with teaching experience of more than 15 years, holding a teacher diploma and having done third-year-level accounting subject specialization, had the lowest proportion of incorrect answers – no more than 3 – compared to teachers with the same teaching experience and qualification, but without accounting as their subject specialization. 5.3 Application of accounting principles Figure 3 depicts the proportion of teachers who chose incorrect answers in the questionnaire on the theme application of accounting principles of Grade 9 FL teaching, according to teaching knowledge, qualifications, and subject specialization.
Figure 3. Incorrect answers on questions relating to accounting principles
http://ijlter.org/index.php/ijlter
45
Data in Figure 3 shows that, firstly, teachers who have at most two years of teaching experience, and who hold a degree plus PGCE, and who had not specialized in accounting, had the highest proportion of incorrect answers. Secondly, teachers with teaching experience of up to 3 years, who have a degree and had third-year level accounting as their subject specialization, had a lower proportion of incorrect answers, at most 3, compared to teachers with the same teaching experience and qualification, who had a high number of incorrect answers – between 6 and 9. Similarly, teachers with the same teaching experience, no more than 3 years, with a degree plus PGCE, but who did not specialize in accounting at all, had the highest number of incorrect answers, that is, between 9 and 12. Thirdly, teachers who have up to five years teaching experience, who hold a degree, and who specialized in accounting up to third-year level, had the lowest proportion of incorrect answers, that is, at most 3. Other teachers, with the same teaching experience and qualification, yet do not have accounting subject specialization, had a lower number of incorrect answers, that is, between 3 and 6. Similarly, teachers with the same teaching experience, and holding a degree plus PGCE, with no accounting specialization, had a low proportion of incorrect answers, that is, between 3 and 6. Fourthly, teachers who had teaching experience of between 11 to 15 years, who have a degree plus PGCE, but who had not specialized in accounting, had a lower proportion of incorrect answers than teachers with only a degree, but the same experience and first-year accounting. Lastly, teachers with teaching experience of more than 15 years, who hold a diploma and who had specialized in accounting at third-year level, had the lowest proportion of incorrect answers – no more than 3. Similarly, teachers with the same teaching experience and qualification, yet do not have accounting subject specialization, had a lower number of incorrect answers, that is, no more than 3. In contrast, teachers with the same years of teaching experience and who had no accounting subject specialization, and a degree plus PGCE, had a higher proportion of incorrect answers than teachers with a diploma.
6. Discussion of results This section discusses results concerning solving the accounting equation, comprehension of accounting concepts, and relations between topics and application of accounting principles. The reviewed literature confirms that teachers must have a thorough comprehension of the specific accounting topics they teach (Molise, 2020). However, most of the teachers who teach EMS are confronted with serious accounting content misconceptions, which threaten effective teaching of FL topics to Grade 9 learners (Melo et al., 2020; Modise, 2016; Peter et al., 2017). Modise and Letlhoenyo (2020) confirm that misconceptions are generally undesirable and must, therefore, be identified and addressed. The majority of the teachers in the study struggled to analyze transactions in the
http://ijlter.org/index.php/ijlter
46
accounting equation, lacked comprehension of accounting concepts and how they link with other concepts, and they were unable to apply accounting principles. Firstly, the results show that teaching experience, qualifications, and subject specialization play pivotal roles in the content mastery of EMS teachers in relation to solving the accounting equation. It was evident from the analysis that teachers with a degree plus PGCE, and who had specialized in accounting at third-year level, had a lower number of incorrect answers – at most 3 – and the number decreased further as they gained more teaching experience. Similarly, the number of wrong answers of teachers with a degree plus honors, with accounting at firstyear level, decreased from 3 to 0 as they gained more experience, first from 3 to 5 years and then from 11 to 15 years. Even teachers with no accounting specialization, with a degree plus honors, are likely to experience a decrease in the proportion of incorrect answers when they reach 15 years of teaching experience. Therefore, more teaching experience is acquired by accounting teachers who have a relevant degree and who had specialized in accounting for three years – they had a lower proportion of incorrect answers. Secondly, it is evident from the analysis that teachers with a degree plus PGCE, with a third-year level accounting subject specialization, decreased the proportion of incorrect answers (from 3 to 0), as they gained more teaching experience (from 3 to 5 years). Similarly, teachers with a diploma, with a third-year accounting specialization, showed a considerable decrease in the proportion of incorrect answers (from 3 to 0), as they reached 15 years of experience. Thirdly, it is evident from the analysis that there was a considerable decrease in the proportion of incorrect answers, from 3 to 0, for teachers holding a degree with a third-year accounting subject specialization as they gained more teaching experience (from 3 to 5 years). Similarly, teachers with a diploma, with no accounting specialization, show a considerable decrease in the proportion of incorrect answers, from 3 to 0, as they reach 15 years of experience. I recommend appointing EMS teachers who have a degree or diploma that included the study of accounting, economics, and business studies up to second to third-year level for them to ensure content mastery, especially the accounting part (referred to as FL in the CAPS document) of EMS. The author argues that content misconceptions by teachers (as illustrated by incorrect answers on the questionnaire) are not unique to accounting, and that intervention strategies through content-specific workshops and cluster teaching collaboration must be implemented to improve teachers’ content skills in all subjects.
7. Limitations The research was done in a single education district of the Free State province, South Africa, and data was generated using a single tool, a closed-ended questionnaire.
http://ijlter.org/index.php/ijlter
47
8. Conclusion This paper aimed to explore the relationships between teaching experience, qualification, and subject specialization on the content mastery of teachers of FL (the accounting part of EMS) to Grade 9 learners. The results show that the experience of the teacher in teaching the subject, relevant qualifications, and having specialized in accounting to the second or third-year level enhances teachers' content knowledge and competence. It is worth noting that teachers who had relevant teaching qualifications with accounting as their subject of specialization at either second or third-year level contributed less to the proportion of incorrect answers regarding the accounting equation, accounting concepts, and principles. However, most of the teachers who contributed to a higher proportion of incorrect answers did not have a relevant accounting teaching qualification and had not specialized in the subject during their training. Similarly, teachers who had 3–15 years of teaching experience had fewer incorrect answers because they have been teaching the subject for more than 3 years. I, therefore, recommend that teachers strive to gain at least three years of teaching experience before they teach senior classes, that they have a relevant diploma or degree that involved studying accounting to at least the second or third year. Teachers should refrain from teaching subjects in which they did not specialize; the teacher’s level of experience should be a determining factor in doing workload/subject allocation for EMS teachers, especially for accounting/FL. Teachers must receive continuous professional development support, to help address inherent content knowledge misconceptions they may have.
9. References Antony, M. K., Paidy, H., Subali, B., Pradana, S. P., Hapsari, N., & Astuti, F. E. C. (2019). Teacher’s TPACK profile: The effect of teacher qualification and teaching experience. Journal of Physics: Conference Series, 6, 1-6. https://doi.org/10.1088/1742-6596/1397/1/012054 Bamidele, A. D., & Adekola, F. F. (2017). Effects of teacher’s qualifications and teaching experience on students’ academic achievement in basic science in junior secondary school. International Journal of Education and Evaluation, 3(2), 1–9. Brante, T. (2013). The professional landscape: the historical development of professions in Sweden. Professions and Professionalism, 3(2), 1-18. https://doi.org/10.7577/pp.558 Chang, S. H., Ward, P. & Goodway, J. D. (2020) The effect of a content knowledge teacher professional workshop on enacted pedagogical content knowledge and student learning in a throwing unit. Physical Education and Sport Pedagogy, 25(5), 493-508. https://doi.org/10.1080/17408989.2020.1743252 Croninger, R. G., King Rice, J. K., Rathbun, A., & Nishio, M. (2007). Teacher qualifications and early learning: effects of certification, degree, and experience on first-grade student achievement. Economics of Education Review, 26(3), 312–324. Darling-Hammond, L., Burns, D., Campbell, C., Goodwin, A. L., Hammerness, K., Low, E. L. McIntyre, A., Sato, M., & Zechner, K. (2017). Empowered educators: How highperforming systems shape teaching quality around the world. John Wiley & Sons. Donkoh, S. (2017). Investigating the effect of teaching experience on teacher knowledge. International Journal of Scientific and Research Publications, 7(6), 319–327.
http://ijlter.org/index.php/ijlter
48
Emmanue, I. A. & Ambe, B. A. (2014). Influence of teachers, professional qualification and area of specialisation on the implementation of environmental education curriculum in Cross River State – Nigeria. International Conference on Chemical, Environment & Biological Sciences (CEBS) (155–160). http://dx.doi.org/10.15242/IICBE.C914120 Gaertner, H., & Brunner, M. (2018). Once good teaching, always good teaching? The differential stability of student perceptions of teaching quality. Educational Assessment, Evaluation and Accountability, 30, 159-182. https://doi.org/10.1007/s11092-018-9277-5 Gudmundsdottir, G. B. Hatlevik, O. E. (2018). Newly qualified teachers’ professional digital competence: Implications for teacher education. European Journal of Teacher Education, 41, 214-231. https://doi.org/10.1080/02619768.2017.1416085 Hatlevik, I. K. (2017). The impact of prospective teachers’ perceived competence on subsequent perceptions as schoolteachers. Teachers and Teaching: Theory and Practice, 23(7), 810–828. https:/doi.org/10.1080/13540602.2017.1322056. Hlalele, D., & Mosia, M. (2020). Teachers’ sense of community in rural learning ecologies. Alternation, 27(2), 101–124. https://doi.org/10.29086/2519-5476/2020/v27n2a6 Hung-Hsi, W. (2017). The content knowledge mathematics teachers need. In Y. Li, W. J. Lewis, & J. Madden (Eds.), Mathematics Matters in Education. Essays in honour of Roger E. Howe (43–91). Springer. Kuhn, C., Alonzo, A.C., & Zlatkin‑Troitschanskaia, O. (2016). Evaluating the pedagogical content knowledge of pre-and in‑service teachers of business and economics to ensure the quality of classroom practice in vocational education and training. Empirical Research in Vocational Education and Training, 8(5), 1-18. https://doi.org/10.1186/s40461‑016‑0031‑2 Ladd, H. F., & Sorensen, L. C. (2017). Returns to teacher experience: Student achievement and motivation in middle school. Education Finance and Policy, 12(2), 241–279. http://direct.mit.edu/edfp/article-pdf/12/2/241/1691525/edfp McKlin, T., Lee, T., Wanzer, D., Magerko, B., Edwards, D., Grossman, S., Bryans, E., & Freeman, J. (2019). Accounting for pedagogical content knowledge in a theory of change analysis. In ICER ’19: International Computing Education Research Conference (12-14). Association for Computing Machinery. https://doi.org/10.1145/3291279.3339412 Melo, L. Cañada-Cañada, F., González-Gómez, D., & Jeong, J. S. (2020). Exploring pedagogical content knowledge (PCK) of physics teachers in a Colombian secondary school. Education Science, 10(12), 362. https://doi.org/10.3390/educsci10120362 Molise, H. (2020). Exploring the content knowledge of accounting teachers in rural contexts: A call for a decoloniality approach. International Journal of Learning, Teaching and Educational Research, 19(8), 447–458. https://doi.org/10.26803/ijlter.19.8.24 Modise, M. A. (2016). Pedagogical content knowledge challenges of accounting teachers. International Journal of Educational Science, 13(3), 291–297. Modise, M. A. & Letlhoenyo, S. (2020). Pedagogical content knowledge of accounting teachers in South Africa: Readiness of Grade 9 for accounting in Grade 10. Proceedings of ADVED 2020- 6th International Conference on Advances in Education, (79–80). Mosia, M. S. (2014). Gender differentials and the mathematics performance of Grade 10 to 12 learners at the Dilanedi Schools. Mediterranean Journal of Social Sciences, 5(23), 1426-1431. https://doi.org/10.5901/mjss.2014.v5n23p1426
http://ijlter.org/index.php/ijlter
49
Musau, L. M., & Abere, M. J. (2015). Teacher qualification and students’ academic performance in science mathematics and technology subjects in Kenya. International Journal of Educational Administration and Policy Studies, 7(3), 83–89. Myrberg, E., Johansson, S., & Rosén, M. (2018). The relation between teacher specialisation and student reading achievement. Scandinavian Journal of Educational Research, 63(5), 1–15. https://doi.org/10.1080/00313831.2018.1434826 Ngwenya, J. (2019). Accounting teachers’ experiences of communal feedback in rural South Africa. South African Journal of Education, 32(2), S1–S10. https://doi.org/10.15700/saje.v39ns2a1502 Ngwenya, J. C. (2020). Assets teachers identify for the teaching of Accounting Education in a rural school in KwaZulu-Natal. Perspectives in Education, 38(1), 72–87. http://dx.doi.org/10.18820/2519593X/pie.v38i1.6 Nixon, R. S., Campbell, B. K., & Luft, J. A. (2016). Effects of subject-area degree and classroom experience on new chemistry teachers’ subject matter knowledge. International Journal of Science Education, 2016(38), 1636–1654. https://doi.org/10.1080/09500693.2016.1204482 Papay, J. P., & Kraft, M. A. (2015). Productivity returns to experience in the teacher labor market: Methodological challenges and new evidence on long-term career improvement. Journal of Public Economics, 130, 105–119. https://doi.org/10.1016/j.jpubeco.2015.02.008 Peter, I., Ward, P., & Li, W. (2017). Effects of improved content knowledge on pedagogical content knowledge and student performance in physical education. Physical Education and Sport Pedagogy, 22(1), 71–88. Podolsky, A., Kini, T., Bishop, J. and Darling-Hammond, L. (2016), Solving the teacher shortage: How to attract and retain excellent educators. Learning Policy Institute. https://learningpolicyinstitute.org/sites/default/files/productfiles/Solving_Teacher_Shortage_Attract_Retain_Educators_REPORT.pdf Podolsky, A., Kini, T., & Darling-Hammond, L. (2019). Does teaching experience increase teacher effectiveness? A review of US research. Journal of Professional Capital and Community, 4, 286–308. https://doi.org/ 10.1108/JPCC-12-2018-0032 Qhosola, M. R. (2015). Accounting teacher preparation: A critical accounting perspective. Journal of Higher Education Association/Research in Education Southern Africa, 13(1&2), 213–222. Rahmi, E. (2018). The analysis of pedagogical content knowledge of teacher candidates. Advances in Economics, Business and Management Research, 57, 604–609. Samad, A., & Nurusus, E. (2015). The influence of teaching experience, school location and academic background on teachers’ beliefs in teaching grammar. Asian Social Science, 11(15), 257–266. Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14. Touron, J., Martin, D. R., Navarro Asencio, E., Pradas, S., & Inigo, V. (2018). Construct validation of a questionnaire to measure teachers’ digital competence (TDC). Spanish Journal of Pedagogy, 76, 25–54. https://doi.org/10.22550/REP76-1-2018-02
http://ijlter.org/index.php/ijlter
50
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 50-67, August 2021 https://doi.org/10.26803/ijlter.20.8.4 Received May 27, 2021; Revised Aug 15, 2021; Accepted Aug 30, 2021
Effect of the Use of WebQuest in a Chinese for Tourism Course at a Thai University: An Exploratory Study Pairin Srisinthon Walailak University, Nakhon Si Thammarat, Thailand https://orcid.org/0000-0003-3693-3163
Abstract. In trying to move away from teacher-centred approaches towards student-centred approaches amidst the global Covid-19 pandemic, integrating technology into foreign-language classes has been considered. Under these conditions, this study implemented WebQuest technology in a Chinese for Tourism class, followed by adopting a mixed-methods research approach to investigate the student learning. This study aimed to 1) assess the learning achievement of 63 students as part of a Chinese for Tourism course using WebQuest to provide the instructional materials alongside the face-to-face format, and 2) to collect the learners’ opinion on the WebQuestbased activities. SPSS was used for the quantitative analysis and thematic analysis was used to analyse the interview data. The development of the WebQuest lessons began with a critical and analytical review of the existing literature related to the WebQuest course design. The WebQuest lessons were then implemented for six weeks after which the students’ learning achievements were evaluated. The findings revealed that after using WebQuest, the learners’ Chinese language ability test scores were higher than before utilizing the instrument. The students were satisfied with the WebQuest lessons with varying satisfaction levels for the questionnaire items. The benefits of teaching and learning in connection to the WebQuest lessons got the highest score. From the 15 learners’ semi-structured interviews, it was found that the majority of the learners agreed that teaching and learning through the WebQuest lessons was a modern method offering convenient access due to the lack of time limitations. Keywords: tourism WebQuest; Chinese majoring students; Chinese teaching; Chinese language; mixed methods
©Author This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
51
1. Introduction According to the statistics of the Thai Tourism Department, approximately 8.8 million Chinese travelled to Thailand in 2016. This number is higher than the number of ASEAN (Association of Southeast Asian Nations) tourists travelling to Thailand. The need to use Chinese in the tourism industry is high. Based on the reports from the Centre of Cooperative Education and Career Development of Walailak University (2016-2018), most students majoring in Chinese chose to do their work-integrated learning in companies related to the tourism industry such as hotels, airports, and tour companies. Many of Thailand’s universities offer Chinese for Tourism courses to support the growth of the tourism industry. The Chinese major curricula try to meet the requirements of the tourist industry by offering students Chinese for Tourism courses to prepare them for real-world careers. Chinese for Tourism is a Chinese for Specific Purposes course dealing with specific Chinese vocabulary and expressions in the field in the areas of business, travel, and an introduction to tourist attractions (Traveling Chinese Glossary Program Group, Foreign Chinese College, Shanghai Normal University, 2008). This course assists the students in improving their Chinese language skills when it comes to travel and tourism. The Chinese for Specific Purposes course involves the teaching and learning of Chinese as a second or foreign language of which the aim is the ability to use Chinese in their career activities. Teaching and learning Chinese for specific purposes requires a methodology that differs entirely or partially from the one used when teaching for the purpose of language skills (Traveling Chinese Glossary Program Group, Foreign Chinese College, Shanghai Normal University, 2008). The instructors of such a course face the challenges of structure, technical vocabulary and field-specific knowledge (Zheng, 2018). They also encounter the challenge of a paradigm shift from the teachercentred approach to the student-centred approach (Wang, 2021; Zheng, 2018). It is important to determine the teaching processes that suit the changing world and the newer generations of students (Chen, 2018). The designers of this course must integrate the expected Chinese language skills with the intended content knowledge to support the learners’ needs (Baker, 2018). In order to move away from the teachercentred approach to concentrate on the student-centred approach, most educators suggest that teachers integrate technology into the Chinese language class. Since technology plays an important role in society today, how to take advantage of technology as part of supporting Chinese teaching is an issue worth studying. Under these conditions, WebQuest technology is an effective tool with which to engage the learners throughout the process of learning (Kaur and Kauts, 2018). In the context of teaching and learning Chinese in Thailand, a few detailed studies were conducted on the utilization of WebQuest in the Chinese classroom. Meanwhile, due to the many years of experience in teaching the Chinese language, the researchers found that the students were rarely actively involved when it came to finding educational information on the Internet. The level of knowledge seeking from the Chinese Internet network is very low. The management of the Chinese for Tourism course was made more effective by connecting to both Thai and Chinese networks,
http://ijlter.org/index.php/ijlter
52
allowing the learners to practise their language skills and learn more effectively through interactions with their teachers and peers. This study attempted to find a way to enhance the level of student learning. Exploring how WebQuest can be best used will benefit many groups of people, be they course designers, course lecturers, or students.
2. Literature Review and Theoretical Frameworks Given the technological advancements in recent years, conducting research on the application of technology in teaching that had been done over the past ten years, and integrating online technology into the learning process were found to have a positive influence on the students’ learning engagement (Dunlap & Lowenthal, 2018; Bognar, Sablic & Skugor, 2019). In order to improve the teaching methods and classroom environment, many educational institutions use new technology (e.g., tech-based learning) as part of their language classrooms (Berezova, Mudra & Yakushko, 2018; Dousti, Amirian & Nejadansari, 2021). Technology has the potential to offer quite a number of free learning materials online for learning activities (Teng, 2017; Gao, Shen & Wang, 2020). Given the advancements in technology and the ways in which the new generation of students learn (Martin & Bolliger, 2018), education institutions need to provide access to web-based teaching and stimulate the learners to utilize Internet resources as part of the learning process. Learning through authentic materials from the Internet offers an active classroom environment as part of a learner-centred activity and a level of communication among the instructors, learners, and their peers. The differences between tech-based learning and traditional learning are compared in Table 1 (see Wasim, Sharma & Siddiqui, 2014, p. 448). Table 1: Comparison of Tech-based Learning and Traditional Learning Tech-based learning -
Student-centred instruction Multi-sensory stimulation Multi-path progression Multimedia Collaborative work Information exchange Active/exploratory/inquiry-based learning Critical thinking and informed decisionmaking Authentic, real-world context
Traditional Learning Teacher-centred instruction Single-sense stimulation Single-path progression Single media Isolated work Information delivery Passive learning Factual, knowledge-based learning Isolated, artificial context
Based on Table 1, WebQuest is more ideal for teaching and learning as it offers what the students should experience, for example, student-centred instruction, collaborative work, inquiry-based learning, active learning and critical thinking (Ebadi & Rahimi, 2018), which is a more suitable approach to learning for the current world of working.
http://ijlter.org/index.php/ijlter
53
2.1 What is WebQuest? Web-based lessons (WebQuest) are designed based on the tech-based learning model created by Professor Bernie Dodge and Tom March of San Diego State University as a learning tool to gain information from the Internet or long-distance networks. It offers access to online resources under the supervision of the instructor (Sakadineca & Jansone, 2018). The learners utilize appropriate and relevant content which the instructors provide to complete their learning tasks. Dodge (2001:?:) describes WebQuest as “an inquiry-oriented activity in which some or all of the information that learners interact with comes from resources on the Internet”. March (2003) also notes that WebQuest is an effective instrument with links to web resources and tasks that encourage the students to find the answers to the open-ended questions to develop their personal expertise and group collaboration skills in order to transfer new information, leading to deeper learning. According to Kazakova and Klyoster (2018) and Kaur and Kauts (2018) two forms of WebQuest exist, namely (i) short-term WebQuest, which provides one to three lessons, focused on the learning and application of specific skills, and (ii) long-term WebQuest which takes as long as three weeks to approximately one month to master. The learners will analyse the new knowledge in a classroom setting with their peers and will learn to apply it to real-life situations. Based on the tech-based learning model, WebQuest includes five parts, namely an introduction, a task, process, evaluation and conclusion (Millsion & Downey, 2001; Sakadineca & Jansone, 2018; Zendler & Klein, 2018; Berezova et al., 2018). These warrant a brief description: a. Introduction: Briefly explains the information and activities, and tries to stimulate the students to solve the problems. b. Task: Clearly explains the tasks and important results that the students must find. c. Process: Clarifies what activities the students must perform in order to achieve their tasks, and provides information sources available on the Internet and other resources, in order for the students to use appropriate sources to solve their assigned problems, focusing on multiple sources of knowledge and diversity. d. Evaluation: Focuses on the criteria for assessing the learning tasks. e. Conclusion: Summarizes the main objectives and experiences of the students during the learning process. Several studies have been conducted using the WebQuest approach (Kazakova & Kloyster, 2018; Ebadi & Rahimi, 2018; Sakadineca & Jansone, 2018; Chen, 2019; Adanan, Adanan & Herawan, 2020). These studies have shown that the WebQuest process supports the learners’ level of learning engagement and the level of enjoyment of their studies. WebQuest also enhances the learners’ critical thinking skills by tasking them with analysing information and completing tasks. For example, Sakadineca and Jansone (2018) designed WebQuest tasks for the purpose of teaching the Latvian language and literature. The results showed that all the students in the
http://ijlter.org/index.php/ijlter
54
class positively worked in groups and were active when it came to using WebQuest to complete their tasks. Liang and Fung (2020) implemented WebQuest in an English classroom to stimulate the students’ critical thinking skills. Their findings revealed that after implementing the WebQuest-based critical thinking programme, the students’ thinking skills improved, especially in terms of their vocabulary when expressing their opinions. Similarly, Wang (2021) used WebQuest for the purpose of supporting the students to actively participate in the information technology (IT) classroom. They found that the students engaged with their learning more and that their problem-solving abilities increased. Researchers claim that the WebQuest technology, as an interactive method, can be successfully implemented in the field of teaching foreign languages. They believe that after the implementation of WebQuest and other learning methods such as collaborative projects, problem-based classrooms or flipped classrooms in the field of language acquisition, the learners’ language competency performance will be significantly improved (Awada & Dihab, 2018; Berezova et al., 2018; Adanan et al., 2020; Synekop, 2020). Awada and Dihab (2018) investigated student’s academic writing performance after integrating Student Team Achievement Division (STAD) and WebQuest into their learning process. The results of the study showed that these web-based interventions improved the students’ academic writing performance. Furthermore, all the teacher participants developed more positive perceptions of web-based instruction. Teng (2017) introduced the concept of a flipped classroom, by dividing three EFL (English Foreign Language) classes into a structured flipped classroom, a semi-structured flipped classroom, and a traditional classroom. It was found that using WebQuest in the flipped classroom was a most effective way to increase the learners’ academic performance in the pre-post test score comparison. Zheng (2018) supported using WebQuest as a teaching and learning tool in the Chinese classroom. He conducted a comparative study of two classes. The experimental group was a trial group that used online lessons, while the control group used lecture-based teaching. The instructor designed the course content according to Hanban's “International Chinese Language Course Layout” and provided the video clips about the Chinese content and culture from the Internet. The web lessons were also designed to provide sufficient interactivity between the learners and the lecturer through an online group chat. At the end of the semester, the learners took an HSK test (Chinese Proficiency Test for assessing non-native Chinese speakers’ Chinese skills). It was found that the HSK scores of the learners who followed lectures through the web were significantly higher than those of the learners in the control group. The results gathered from the questionnaires indicated that most of the learners from the web-learning group were satisfied with the selfcontrolled learning process. They also strongly agreed to watching video clips in small groups. The study proved that WebQuest encourages learners to learn, analyse, synthesize and provide solutions that have been discovered by themselves or in small groups. In addition, the group that learned Chinese online were confident when learning with Chinese students. On the contrary, the students who studied in the control group were not confident when studying with Chinese students in their own
http://ijlter.org/index.php/ijlter
55
majors. This is because they did not understand the lectures given by the Chinese instructors. A similar study by Chen (2018) showed that patients with hearing problems who studied Chinese pronunciation through online classes got higher scores after having used web lessons. They progressed in their listening performance and were able to discuss interactions with a medical professional about their hearing problems. The web lessons made it easier for the learners to communicate effectively. The integration of WebQuest in classrooms provides learners with the ability to gather information from various websites. This can help the learners to understand both the linguistic and cultural aspects, and to improve their intercultural competence (Awada and Diab, 2018; Kazakova & Klyoster, 2018; Ebadi & Rahimi, 2018). Awada and Diab (2018) divided EFL learners from eight different countries into experimental and control groups. They provided the experimental group with the Inquiry-based Intercultural Communication Technological Model. The control group received regular instructions. The findings show that the participants in the experimental group had a strong intrinsic motivation towards conducting culturally based debates in a setting that ensured intercultural communication. Learning through WebQuest allows learners to learn without time limitations. The students have to be self–directed, active and autonomous when it comes to searching for information before referring to the instructor who acts as the facilitator. This solves the problem of over-dependence on the instructors to deliver knowledge and to identify problems (Synekop, 2020). It has been proven that the students improved their self-esteem and self-confidence by taking part in a more participative course. The reviewed studies show that most of the studies discussed, were conducted on the use of WebQuest in EFL classrooms. Further studies are recommended to investigate the use and outcomes of WebQuest technology in other subjects. More studies, therefore, will be required to determine the effects of the WebQuest approach on Chinese teaching. This will help us to understand the advantages of this technology in a holistic manner.
3. Study Objectives This study aimed to investigate the effectiveness of WebQuest technology when implemented in a Chinese for Tourism class. The research questions were as follows: 1. What were the learners’ achievements after the Chinese for Tourism course through WebQuest? 2. What were the learners’ opinions about the WebQuest lessons?
4. Research Methodology 4.1 Research design The mixed-methods design was used to explore the learners’ achievements and their opinion of learning via the WebQuest lessons. The independent variable was the implemented WebQuest lessons which included the Chinese for Tourism course
http://ijlter.org/index.php/ijlter
56
content. The dependent variables included the learners’ achievements and their opinion of the WebQuest instruction. The first dependent variable was measured by comparing the pre-test and post-test scores. The second dependent variable was measured using the post-instruction questionnaires and informal group interviews. The quantitative data from the questionnaires and the pre-post-test scores were analysed using SPSS and an analysis of variance (ANOVA) respectively. The qualitative data from the informal group interviews were analysed using thematic analysis. 4.2 Research site and participants This research was conducted at a university in southern Thailand. The university emphasizes teaching; all of the teaching staff had passed the UK-based Advanced HE Training Programme (the active teaching and learning programme used for training university instructors provided by the Higher Education Academy). On this basis, its lecturers emphasize their teaching, paying close attention to the learning of individual students, adopting formative assessments and feedback, and focusing on analytical and critical thinking. Moreover, the university encourages its instructors to teach using active-based methods and to employ empirical research. The participants in this study were 63 undergraduate Thai students majoring in Chinese who were enrolled in the Chinese for Tourism course offered in the first term of the 2019 academic year. This course was selected because its objective was the use of Chinese in their professional activities or areas of specialization. The students attended five 50-minute class periods per week in a smart classroom where each student was allowed to use his/her personal computer or mobile phone. Fifteen interviewees were selected using the volunteering system. 4.3 Research instruments The instruments used in this study included WebQuest for the Chinese for Tourism Class, the pre-test and post-test, and the post-instruction questionnaire. The learners’ Chinese proficiency and the learning objectives were considered before WebQuest was created specifically for the course. The course was offered to third-year Chinese majors who had studied approximately 1,000 words for everyday use. The WebQuest lessons consisted of six steps: 1) Introduction; 2) Task; 3) Process; 4) Resources; 5) Evaluation, and 6) Conclusion (Dodge, 2001). The WebQuest lessons were divided into two main parts, one for the administrator or instructor and one for the learners. The WebQuest lessons covered communication contexts that the learners could genuinely apply elsewhere. The contents consisted of Ticket Booking, Welcome to Thailand, Check-in Procedures, Room Reservation, and Checking in and Checking out at a hotel. WebQuest on Chinese for Tourism was provided to the participants for the purpose of learning in the classroom and outside of it. It consisted of Chinese content on tourism, online exercises, links to useful internal and external websites, and communication suggestions via e-mail and Facebook. The performance test consisted of the pre-test and post-test developed from the course content. This was used to evaluate the learners’ proficiency regarding their Chinese for Tourism knowledge and their Chinese language proficiency before and
http://ijlter.org/index.php/ijlter
57
after studying the course using the WebQuest lessons. Similarly, the pre-test and post-test consisted of ten multiple-choice questions, twenty gap-filling items, ten true-false questions and five sentences to write using the words provided. The total score for the performance test was 50 points. The post-instruction questionnaire was developed by the researcher after conducting a literature review on the effectiveness of WebQuest in supporting the learning process. It was used to measure the learners’ attitude towards the use of web resources for learning as part of the Chinese for Tourism Course and it was administered at the end of said course. The questionnaire was divided into three main aspects: the WebQuest Design, the WebQuest Lesson Content, and the Usefulness of WebQuest. It used a Likert scale for the items ranging from 1 (not at all satisfied) to 5 (extremely satisfied). Three invited Item Objective Congruence IOC experts validated the questionnaire. The score obtained was over 0.67. Subsequent to the questionnaire response session, fifteen learners from the class attended the semi-structured interviews on a voluntary basis. Each interviewee had approximately ten minutes to respond to a set of open-ended questions. Notes were taken by the interviewer and a research assistant, in order to record the key ideas as part of the data analysis. All proper interview protocols were adhered to at all times, including confidentiality. 4.4 Data collection and data analysis The quantitative data were collected using a questionnaire and the qualitative data were gathered using semi-structured interviews. The Statistical Package for the Social Sciences software ( SPSS) was used for the data analysis. The data collected were analysed using the mean, standard deviation and t- test. Analysis of variance (ANOVA) was conducted to compare the pre-test and post-test score differences with a significance level of 0. 05. The questionnaire items were interpreted using the included Likert scale ranges based on Best’s criteria (Best, 1986). 1.00 - 1.80 = Not at all satisfied 1.81 - 2.60 = Slightly satisfied 2.61 - 3.40 = Moderately satisfied 3.41 - 4.20 = Very Satisfied 4.21 - 5.00 = Extremely satisfied Thematic analysis was used to analyse the data from the interviews (see Braun & Clarke, 2019; Terry & Hayfield, 2021). The interview data were identified and coded. Five major themes emerged from the data. 4.5 Research procedure The research procedure is depicted in the following table (Table 2):
http://ijlter.org/index.php/ijlter
58
Table 2: Research procedure Research Procedure
Item
Purpose
Timeline
1
Designing the WebQuest lessons, performance tests and opinion questionnaire
-To prepare the learning materials and research instruments
Before conducting the research
2
Evaluating the students’ Chinese proficiency using the pre-test
-To measure the learners’ background knowledge of Chinese for Tourism.
Before the experiment
3
Implementing WebQuest for Chinese for Tourism
-To introduce how to use the WebQuest lessons, learning tasks, and objective goals.
6 weeks
-To provide the specific Chinese skills needed in the tourism industry, focusing on the specific vocabularies around accommodation, travel methods, sightseeing and holidays. 4
Evaluating the student learning using the post-test
-To evaluate their learning achievements
7th week
5
Collection of the data through a questionnaire
-To measure the students’ opinions of/on the WebQuest lessons
8th week
6
Collection of the data through interviews
-To gain more information about the effectiveness of the WebQuest lessons
8 - 9th week
4.6 Ethical considerations Before beginning the experiment, the details of the purpose and methodology of the research were explained to the participants for them to feel respected and appreciated after they had left the project. All the participants were requested to return a copy of their completed consent form before conducting the research. They could withdraw from the study at any time. The researcher treated the information gathered from the individuals confidentially without disclosing the participants’ identity. In addition, all the data gathered from the participants were only available to the researcher and were kept in a secure location.
5. Results 5.1 Learners’ academic achievements The first research question aimed to determine the learners’ level of achievement after studying the Chinese for Tourism course through WebQuest. Table 3 depicts a comparison of the students’ pre-test and posttest scores.
http://ijlter.org/index.php/ijlter
59
Table 3: Comparison of the learners’ pre-test and post-test scores
Pair 1 Pre-test Post-test
Mean
N
Std. Deviation
Std. Error Mean
Sig.
7.30
63
4.01
0.52
.000
21.30
63
6.22
0.81
The statistics for academic performance in Table 3 indicate that the learners’ achievements were positive with higher scores after studying the Chinese for Tourism course through the WebQuest lessons at a significance level of 0.000 (Sig. < 0.005). 5.2 Learners’ opinions The second research question was answered by the questionnaire and the interview data, the findings of which are shown in the following table. Table 4: Learners’ opinions about the WebQuest lessons Item
Aspect
Mean
S.D.
Level of Satisfaction
1
Diverse formats of information and content presentation
3.00
0.53
Moderately satisfied
2
Interesting presentation
2.95
0.66
Moderately satisfied
3
Creative design
3.47
2.57
Very satisfied
4
The font used
3.26
0.72
Moderately satisfied
5
Web colours
3.38
0.72
Moderately satisfied
6
Linked tabs
3.01
0.65
Moderately satisfied
7
3.03
0.73
Moderately satisfied
8
Easy to communicate between the learners and the instructors Easy access to other resources
3.00
0.71
Moderately satisfied
9
Appropriate tasks and evaluations
3.14
0.64
Moderately satisfied
10
Online feedback of the learners
2.84
0.65
Moderately satisfied
3.11
0.65
Moderately satisfied
The overall average value
Table 4 shows that the overall average value of the learners’ satisfaction with the WebQuest design was 3.11. When considering the individual aspects, the highest mean score was for Item 3 (mean=3.47). This means that the learners seemed to be quite satisfied with the design of WebQuest even though they had never used web-based lessons before. Because of the technical problems, the aspect of online feedback got the lowest mean score.
http://ijlter.org/index.php/ijlter
60
Table 5: Learners’ Opinions about the WebQuest Lesson Content Item
Aspect
1 2
Web structure Suitable content according to the learning objective 3 Modern application of the content 4 Genuine application of the content 5 Language and content accuracy and reliability 6 Enough resources to use The overall average value
Mean
S.D.
3.92 3.85
0.58 0.65
Level of Satisfaction Very satisfied Very satisfied
4.20
0.71
Very satisfied
4.25
0.71
Extremely satisfied
4.06
0.64
Very satisfied
3.73 4.00
0.75 0.68
Very satisfied Very satisfied
According to Table 5, it can be concluded that the learners were highly satisfied with the content of the WebQuest lessons. They were satisfied with the authentic application of the content as indicated by giving it the highest mean score (mean = 4.25). The reason behind this might simply be that the learners are much more concerned about utilizing their Chinese for Tourism knowledge in real life. The learners also focused on the modern content that was beneficial for them to learn. The learners were highly satisfied with the accuracy and reliability of the language and content of the web lessons. Table 6: Learners’ Opinions of the usefulness of WebQuest Item
Aspect
1 2
Provide self-study-based learning Enable the learners to learn anytime and anywhere 3 Motivate the learners to study 4 Support the learners to solve problems better 5 Develop the relationship among the instructor, learners, and their peers 6 Develop their learning performance 7 Learners can control their learning process 8 Support critical thinking 9 Gain knowledge that meets the learning objective 10 Appropriate tasks that meet the learning objective 11 Discipline in learning The overall average value
http://ijlter.org/index.php/ijlter
Mean
S.D.
Level of Satisfaction
4.25 4.57
0.83 0.82
Extremely satisfied Extremely satisfied
4.04 4.23 4.15
0.67 0.64 0.55
Very satisfied Extremely satisfied Very satisfied
4.45 4.04 4.34 4.50
0.54 0.71 0.53 0.61
Extremely satisfied Very satisfied Extremely satisfied Extremely satisfied
4.54
0.65
Extremely satisfied
3.73 4.25
0.88 0.68
Very satisfied Extremely satisfied
61
As shown in Table 6, the learners were satisfied with the usefulness of the Chinese for Tourism WebQuest at a high level (mean = 4.25). The aspect of learning anytime and anywhere got the highest average score of 4. 57. This indicates that the students were able to study without the limitations of time and space. The tasks provided by the Chinese for Tourism WebQuest were suitable for the learning objectives, thus the satisfaction of the learners was found to be at the highest level (mean = 4.54). 5.3 Opinions on the WebQuest lessons from the interview data Based on the thematic analysis (Braun & Clarke, 2019), the opinions of the students on the WebQuest lessons collected from fifteen interviewees were coded into the following categories: Promoting autonomous learning Supporting modern and up-to-date learning content Practising technological skills Enjoyed working in cooperative groups Internet connectivity problems Table 7: Learners’ responses in each category Item 1 2 3 4 5
Category Internet connectivity problems Supporting modern and up-to-date learning content Practising technological skills Enjoyed working in cooperative groups Promoting autonomous learning
Total (person) 15 15 13 13 12
Based on Table 7, all the interviewees found the Internet connectivity problem to be a big obstacle. However, the good points were that WebQuest supports modern and up-to-date learning content, that it gave them the opportunity to practise their technological skills, that they enjoyed working in cooperative groups and that it promoted autonomous learning. Promoting autonomous learning Twelve learners responded that integrating the WebQuest approach into the Chinese for Tourism Class was a new and modern learning method that enhanced their Chinese language skills development. They were able to learn anytime and anywhere after class. One interviewee said, “I spent a lot of time preparing for this course apart from classroom presentations. The assignments provided in the web helped me to better understand knowledge of Chinese for Tourism”. Another interviewee added, “I preferred to depend on my own ability to search knowledge from Internet rather than following the teacher’s guidance. My Chinese reading and translating skills also improved while working with Chinese posted articles”. Only three learners said that they preferred not to participate in autonomous learning, and that they liked learning from the instructor’s lectures.
http://ijlter.org/index.php/ijlter
62
Supporting modern and up-to-date learning content Studying through WebQuest encouraged the students to interact with new information from the web resources. As one commented, “The WebQuest contents are modern and useful, stimulate me to engage with learning”. All interviewees commented that the information provided on WebQuest widened their knowledge of the course and that they could apply the information to real-life situations and in the workplace. Most of the interviewees reported that they saved time by getting the learning materials from the WebQuest lessons. Practising technological skills The students had no experience with the WebQuest instruction method prior to the course. Through WebQuest, they obtained technological skills and knowledge about the use of Internet-enhanced learning. Most of the interviewees noted that submitting the tasks online was easy and comfortable. Approximately 60% of the interviewees reported that the WebQuest Chinese for Tourism course had detailed and clear steps to help them carry out the learning tasks, presenting the tasks well and providing the related materials. They were able to practise more technological skills when completing their learning tasks. Enjoy working in cooperative groups Many of the learning tasks on WebQuest required being accomplished by cooperative groups. According to thirteen learners, they enjoyed working in cooperative groups to complete the assigned tasks. For example, one responded, “I was happy to work with my peers because we will help each other to complete the assigned tasks”. Another interviewee responded, “I like to brainstorm with my classmates, this helped us to do the tasks better”. Only two interviewees in the class reported that some of their peers were not responsible enough to do the tasks, meaning that they did not engage with their assigned cooperative groups. Internet connectivity problems Fifteen learners gave feedback on the technical problems including the weak wi-fi connection and the webpage not being stable. This problem interrupted the process of learning. Some of the students were worried about submitting the tasks late and getting no scores. Providing more explanations for selected vocabularies’ meanings as part of the WebQuest lessons should be considered. Some of the students asked the instructor to explain the tasks provided on the webpage again in class.
6. Discussion and Implications A number of studies focused on using WebQuest in English classrooms to develop the students’ academic performance (Adanan et al., 2020; Baker, 2018; Kazakova & Kloyster, 2018), but of these only a few studies (e.g., Zheng, 2018) have been conducted on implementing WebQuest in Chinese classrooms. This study attempted
http://ijlter.org/index.php/ijlter
63
to find a way to improve the teaching and learning of Chinese using the principles of long-term WebQuest use (Dodge, 2001) to design web lessons for the Chinese for Tourism Course. The participants studied the web lessons for six weeks. The findings from this research indicate that learning through the Chinese for Tourism WebQuest resulted in higher scores in the learner’s post-test compared to the pre-test scores at a significant level. This result agrees with the study by Zheng (2018) who revealed that the HSK (Chinese Proficiency Test) scores of the students taught through using web lessons were significantly higher. Overall, the investigation reflected the learners’ positive satisfaction with the Chinese for Tourism WebQuest lessons at a high level. They enjoyed working with their peers to complete the learning tasks. These findings are compatible with the findings on using WebQuest to enhance English writing and reading performance, conducted by Berezova et al. (2018). They found that the students were actively engaged with the authentic and collaborative tasks provided through the Internet. The students believed that the knowledge that they gained from practising Chinese for tourism through the web lessons could be applied in their daily life and in the workplace. Learning through web lessons is a new and modern style of Chinese learning. Through this mode of learning, the learners can improve their language skills. Dudeney (2003) points out that WebQuest technology is an effective tool to provide access to the Internet in a language classroom, and that it provides the learners with an opportunity to study with no time limits. Practising various tasks enabled the learners to understand the Chinese for Tourism content and to increase their knowledge of the vocabulary and expressions suitable for use in many situations when travelling. Similar results were reported by Awada and Diab (2018) and Liang and Fung (2020), namely that after implementation of the WebQuest in the EFL classrooms, the students’ language competency improved. This learning method fosters positive motivation and critical thinking. The results of this study confirm the inferences drawn from the previous studies that employing technological devices in the Chinese learning process enhances the learners’ motivation and their enjoyment when studying (Seitkazy et al., 2016; Chen, 2019; Zheng, 2018, Wang, 2021). By learning through WebQuest lessons, the learners managed their own learning process with no time limitations (Mitsikopoulou, 2014; Liu, Huang & Wosinski, 2017; Zeng & Fu, 2019). Although integrating the selected technology into the Chinese learning context in higher education contributed various advantages, problems did occur. The learners faced difficulties when trying to get connected to the Internet as they were fully dependent on the wi-fi connection provided by the university, both in the classrooms and in the dorms. A large number of users were utilizing the same access point at the same time, which was mostly at night. The Internet streaming became poor as a result, and sometimes the online connection was lost. As a multimedia programme, WebQuest lessons require a high storage capacity to run. This was the biggest problem when it came to the learners participating actively in the Chinese for
http://ijlter.org/index.php/ijlter
64
Tourism WebQuest activities created by the instructor. Similar findings were also made in previous studies by Seitkazy et al. (2016), and Yarmakeev, Valiakhmetova, Akhmadullina and Terane (2019), who reported that the main problem with learning online was the weak Internet connection and poor condition of the computer equipment. This included that the mobile phone system was not effective enough to run the Chinese for Tourism WebQuest programme, thus preventing the learners from interacting. In this study, the instructors could not contact their students through WebQuest and they changed their method of contact to using Line or e-mail for the submission of assignments instead.
7. Conclusion This research revealed that integrating the WebQuest lessons and technology into the Chinese learning process enhanced the development of Chinese regarding their tourism knowledge and Chinese language skills. The difference between the pre-test (7.30) and post-test scores (21.30) corroborates the learner’s academic achievements. As for the questionnaire and interview results, the learners were positive regarding the use of WebQuest lessons to help them with the process of learning Chinese for tourism. According to the questionnaire responses, the learners were satisfied with the usefulness of WebQuest at a high level (mean = 4.25). The instructors should motivate and encourage the learners to utilize the Internet effectively by providing useful online resources related to the learning process. According to the interview data, the learners reported in their feedback on technical problems such as a weak wifi connection or WebQuest not being stable. The university should improve its Internet connection in order to encourage students to engage with the online learning environment. To solve the wi-fi connection problem, a WebQuest design allowing it to be used on both laptops and mobile phones should be considered.
8. Acknowledgments This paper is part of the research study titled “The Innovation of Learning Chinese for Tourism by Using WebQuest”, funded by the Innovation and Research Department of Walailak University, Thailand. This research and the informed consent were reviewed and approved by the Human Research Ethics Committee of Walailak University, based on the Declaration of Helsinki (Approval No. WUEC-19029-01).
9. Recommendations It is recommended that more in-depth investigative studies and more research design types, such as experimental and control group research designs, be used to investigate technology-supported education on a wider scale. The scope should also be broadened concerning the examination of the students’ achievements after learning through WebQuest. WebQuest should be implemented in other Chinese courses such as Business Chinese, Chinese Literature and Chinese Translation to gather more essential and useful information to fill the gaps in the literature for Chinese foreign language teaching that will benefit other Chinese language teachers and students.
http://ijlter.org/index.php/ijlter
65
10. References Adanan, H., Adanan, M., & Herawan, T. (2020). M-WebQuest development: Reading comprehension of senior high school students in Indonesia. International Journal of Emerging Technologies in Learning (IJET), 15(3), 74-92. https://doi.org/10.3991/ijet.v15i03.10628 Awada, G., & Diab, H. (2018). Effect of WebQuest media on learners’ intercultural communication debate and motivation. 19th International CALL Conference (2018). Call Paper. Baker, D. M. (2018). USA and Asia hospitality and tourism students' perceptions and satisfaction with online learning versus traditional face-to-face instruction. E-Journal of Business Education and Scholarship of Teaching, 12(2), 40-54. https://doi.org/10.33915/etd.2960 Berezova, L. М., Mudra, S. М., & Yakushko, K. H. (2018). The effect of WebQuests on the writing and reading performance of university students. Information Technologies and Learning Tools, (64,2), 110-118. https://doi.org/10.33407/itlt.v64i2.1979 Best, J. W. (1986). Research in Education 1986. Prentice-Hall. Bognar, B., Sablić, M., & Škugor, A. (2019). Flipped learning and online discussion in higher education teaching. Didactics of smart pedagogy (pp. 371-392). Springer, Cham. https://doi.org/10.1007/978- 3-030-01551-0_19 Braun, V., & Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, 11(4), 589-597. https://doi.org/10.1080/2159676x.2019.1628806 Chen, J. (2019). Designing online project-based learning instruction for EFL learners: A WebQuest approach. MEXTESOL Journal, 43(2), 1-7. Chen, X. (2018). Hearing rehabilitation training system based on Web. Beijing Biomedical Engineering. 37(1), 86-90. Dodge, B. (1995). WebQuests: A technique for Internet-based learning. Distance Educator, 1(2), 10-13. Dodge, B . (2001). Site overview:The WebQuests Page. http://webquest.sdsu.edu/overview.htm. Dousti, M., Amirian, Z., & Nejadansari, D. (2021). Application of WebQuest-based Instruction in Higher Education Context: EFL Students’ Achievement in Writing Skill. Journal of English Language Teaching and Learning, 13(27), 113-136. https://doi.org/10.22034/elt.2021.42866.2312 Dudeney, G. (2003). The quest for practical web usage. Global Issues Special Interest Group. IATEFL ISSUES, 8-9. Dunlap, J., & Lowenthal, P. (2018). Online educators’ recommendations for teaching online: Crowd sourcing in action. Open Praxis, 10(1), 79-89. https://doi.org/10.5944/openpraxis.10.1.721 Ebadi, S., & Rahimi, M. (2018). An exploration into the impact of WebQuest-based classroom on EFL learners’ critical thinking and academic writing skills: A mixed-methods study. Computer-Assisted Language Learning, 31(5), 617651. https://doi.org/10.1080/09588221.2018.1449757
http://ijlter.org/index.php/ijlter
66
Gao, H., Shen, H. D., & Wang, Y. L. (2020). Diversified teaching mode reform of mechanical principle and design based on Internet plus. Life-Long Education, 9(5), 225. http://dx.doi.org/ 10.18282 /le.v9i5.1257 Hanban. (2004). Hanban's International Chinese Language Course Layout. Hanban Explained. https//everything.explained.today/Office_of_Chinese_Language_International/ Kaur, S., & Kauts, A. (2018). Impact of WebQuest on student engagement. Indian Journal of Public Health Research & Development, 9(12), 1665-1669. https://doi.org/10.5958/09765506.2018.02230.1 Kazakova, O. P., & Kloyster, A. M. (2018). Educational tasks modelled on the basis of the WebQuest technology (from the Experience of Teaching Foreign Languages). Astra Salvensis. Liang, W., & Fung, D. (2020). Development and evaluation of a WebQuest-based teaching programme: Students’ use of exploratory talk to exercise critical thinking. International Journal of Educational Research, 104, 101652. https://doi.org/10.1016/j.ijer.2020.101652 Liu, D., Huang, R.., & Wosinski, M. (2017). Smart learning in smart cities. Singapore: Springer Nature. March, T. (2003). What WebQuests Are (Really). https://tommarch.com/writings/whatWebQuests-are/ Martin, F., & Bolliger, D. U. (2018). Engagement matters: Student perceptions on the importance of engagement strategies in the online learning environment. Online Learning, 22(1), 205-222. https://doi.org/10.24059/olj.v22i1.1092 Millson, A. J., & Downey, P. (2001). WebQuest: using Internet resources for cooperative inquiry. https://rc.library.uta.edu/utair/bitstream/handle/10106/11730/WebQuest%20using%20Internet%2020resources .pdf?sequence=1 Mitsikopoulou, B. (2014). Introduction to Webquests. www.opencourses.uoa.gr/modules/docu Sakadineca, O., & Jansone, A. (2018). Students' attitudes toward learning activities organized with the means of WebQuest method. Baltic Journal of Modern Computing, 6(3), 271278. https://doi.org /10.22364/bjmc.2018.6.3.04 Seitkazy, P. B., Toleubekova, R. K., Amanova, A. K., Tashetov, A. A., Iskakova, G., & Demissenova, S. S. (2016). A Web-Quest as a teaching and learning tool. International Electronic Journal of Mathematics Education, 11(10), 3537-3549. Teng, M. F. (2017). Flipping the classroom and tertiary level EFL students' academic performance and satisfaction. Journal of Asia TEFL, 14(4), 605. https://doi.org/10.18823/asiatefl.2017.14.4.2.605 Synekop, O. (2020). WebQuest as technology of differentiated ESP instruction at university level. Journal of Teaching English for Specific and Academic Purposes,? 043-052. https://doi.org/ 10.22190 /JTESAP2001043S Terry, G., & Hayfield, N. (2021). Conceptual foundations of thematic analysis. Essentials of thematic analysis , Number? Issue? pp. 3–14. American Psychological Association. https://doi.org/10.1037/0000238-001 Travelling Chinese Glossary Programme Group, Foreign Chinese College, Shanghai Normal University. (2008). Luyou Hanyu dagang [Outline of Traveling Chinese Function]. Shanghai Normal University.
http://ijlter.org/index.php/ijlter
67
Walailak University. (2016-2018). Reports from the Centre of Cooperative Education and Career Development of Walailak University, Nakhon Si Thammarat Province, Thailand. Wang, Y. H. (2021). Integrating modified WebQuest activities for programming learning. Journal of Computer Assisted Learning. https://doi.org/10.1111/jcal.12537 Wasim, J., Sharma, S. K., Khan, I. A., & Siddiqui, J. (2014). Web-based learning. International Journal of Computer Science and Information Technologies, 5(1), 446-449. Yarmakeev, I., Valiakhmetova N., Akhmadullina, R., & Terane, G. (2019). WebQuest technologies as learning motivation. Helix. 9(5), 53075310. https://doi.org/10.29042/2019-5307-5311 Zendler, A., & Klein, K. (2018). The effect of direct instruction and WebQuest on learning outcomes in computer science education. Education and Information Technologies, 23(6), 2765-2782. Zeng, Q., & Fu, J. (2019). Teaching mode of WebQuest in college English flip class under the mobile learning environment. Proceedings of 2019 5th International Conference on Education Technology, Management and Humanities Science (ETMHS 2019). 11771181. http://doi.org/1025236/etmhs.2019.253 Zheng, Q. (2018). The teaching reform study based on Internet plus communicative task for elementary comprehensive course of TCFL. Jinhua Polytechnic Journal, 18(3), 21-27.
http://ijlter.org/index.php/ijlter
68
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 68-90, August 2021 https://doi.org/10.26803/ijlter.20.8.5 Received May 24, 2021; Revised Aug 14, 2021; Accepted Aug 30, 2021
Multiple Means of Engagement Strategies for Maximising the Learning of Mathematics in Pandemic-regulated Classrooms Matshidiso M. Moleko University of the Free State, South Africa https://orcid.org/0000-0002-1437-7218
Abstract. Many learners find mathematics learning challenging. In response to that actuality, this paper highlights mathematics teachers’ experiences of, and insights into how they adopted and implemented the principle of “Multiple Means of Engagement” (MME) to maximise learning in pandemic-regulated classrooms (in the context of the study, characterised by alternative weeks of attendance, social distancing and wearing of masks). The MME principle is one of the three universal design for learning (UDL) key principles, which guides on how diverse groups of learners can be effectively catered for. The empirical processes, premised on a phenomenological case study, commenced with focus group discussions with 8 high school mathematics teachers from a previously disadvantaged area, who have prior-training in MME. A free attitude interview (FAI) technique was used, to afford the teachers the opportunity to share their insights into the application of MME in their pandemic-regulated classrooms. The content analysis of the teachers’ reflections revealed the following aspects: clear instructions, step-by-step guides, checklists to enhance self-regulation, varying demand and resources to meet challenges, fostering collaboration, providing corrective feedback to sustain effort and persistence, addressing mathematical vocabulary and using real-life situations to recruit interest. These strategies were found not only essential in maximising learning in mathematics under normal circumstances, but also indispensable during the prevailing conditions of the pandemic. The findings therefore suggest MME as a suitable mathematical approach during this Covid19 period. Keywords: learner engagement; Mathematics teaching; maximising learning; multiple means of engagement
1. Introduction The purpose of the study was to highlight mathematics teachers’ experiences of and insights into how they implemented the multiple means of engagement (MME) principle to maximise learning in pandemic-regulated classrooms. MME ©Author This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
69
is one of the key principles of the universal design for learning (UDL) teaching framework, which encompasses strategies to engage diverse groups of learners in meaningful activities meant to motivate them, address anxiety, and maximise learning (Dalton, 2017). Many learners find mathematics learning challenging and become demotivated, anxious, and disengaged in the classroom. With respect to this, Smith (2004) notes that most mathematics classrooms are occupied by learners with negative thoughts and feelings towards mathematics. These learners often develop anxiousness about mathematics that causes them to be less likely to continue working on mathematical problems, especially when they fail to understand these the first time (Smith, 2004). This makes the teaching of mathematics even more challenging, since teachers have to ensure that they not only teach mathematics effectively but at the same time also deal with issues pertaining to a lack of motivation, learners’ negative thoughts and feelings towards mathematics, and learners’ state of being disengaged. According to Filgona et al. (2020), learners who are unmotivated tend to be disengaged, which often leads to behaviour that makes the teaching of mathematics difficult. However, Gray and Madson (2007) and Hake (1998) maintain that mathematics teachers have a responsibility to ensure that learners are not disengaged in the classroom by using interactive approaches that have proven to be useful in terms of engaging learners and providing them with actual gain in learning. Learner engagement is regarded as a way of addressing problems and emotional dispositions such as demotivation, anxiousness and disengagement that learners often experience in various school subjects and specifically in mathematics (Padgett et al. 2019). In short, learner engagement is seen as a potential solution for addressing these emotional dispositions in an endeavour to encourage learners to accomplish learning tasks, persist in the midst of challenge, develop academic satisfaction, and remain engaged with the given tasks (Halverson and & Graham, 2019). Williams and Williams (2011) view engaging learners in learning as an important ingredient for success and a key to academic motivation and persistence. Kuh et al. (2011) hold the act of engaging learners in the learning process in high esteem and regard it as crucial in terms of promoting learner success. According to Yengin et al. (2010), engaging learners in the learning process is a strategy that puts learners at the centre of their learning and inspires them to participate and take on an active role in the learning process. Learner engagement is further regarded as an essential precursor for maximising learning (Zyngier, 2007). Engaging learners in the learning process provides many benefits, including increased “attention and focus, motivation to practise higherlevel critical thinking skills and promoting meaningful learning experiences” (Park, 2003). Three domains of engagement have been identified through the literature, namely “cognitive, emotional, and behavioural engagement” (Fredricks et al. 2004). Behavioural engagement is often observed through actions such as learners paying attention and cooperating and participating in class activities (Hughes, 2012), while cognitive engagement is exhibited through selfefficacy, motivation, focus, and psychological effort from an individual (Duchesne
http://ijlter.org/index.php/ijlter
70
& Ratelle, 2010). “Emotional engagement” is noticeable through a sense of community and belonging, support in the classroom, identifying with school, and one’s feelings towards school and learning (Fredricks et al. 2004). Research shows that learners who are highly engaged often demonstrate active involvement in learning and usually have high academic achievements (Appleton et al., 2008; Fredricks et al., 2004; Leonard, 2008; Smith et al., 2010; Sirin & Rogers-Sirin, 2005). Moreover, they demonstrate a positive attitude and behaviour towards their schoolwork (Smith et al., 2010). Although learner engagement positively influences learning, many teachers seem to have difficulty in engaging learners, and this often results in low learner participation in class activities, demotivation, and disengagement (Parsons et al., 2014). Teachers’ challenges with regard to engaging learners were recently exacerbated by the Covid-19 pandemic that forced schools to close (during the lockdown) and then apply an alternating attendance system for learners to be able to maintain social distancing inside the classroom. All of these changes had a significant impact on the teaching of mathematics and raised the levels of demotivation and learner disengagement in the classroom. The fact that teachers had not been trained to teach under the prevailing conditions and lack knowledge of inclusive teaching strategies (Bansilal & Ubah, 2019) makes it even more challenging for mathematics teachers to teach and effectively engage learners under these conditions. Furthermore, the nature of teaching mathematics that necessitates learner engagement, which is currently impeded by the restrictions of the pandemic, makes it difficult for teachers to teach and create a teaching and learning space that is conducive for diverse learner population (Klein, 2000). Balwant (2018) stresses that disengagement in the classroom is a serious challenge that should be addressed in order to maximise learning. In line with Balwant’s statement, Thoonen et al. (2011) suggest the need for improving teaching practices and, more so, ways to engage learners in order to maximise learning. The fact that learner engagement serves as a precursor for maximising learning (Zyngier, 2007) raises, even more, the need for teachers to consider strategies to meaningfully engage learners in order to maximise learning. Taking all of this into account, the current study sought to highlight mathematics teachers’ experiences of and insights into how they implemented the MME to maximise learning during the Covid-19 period (in their pandemic-regulated classrooms). MME is one of the UDL principles that provides guidelines for enhancing motivation and learner engagement in order to maximise learning (Burgstahler, 2008). It addresses issues of learner variability (e.g. different learning styles and preferences) and provides flexible choices for engagement in the learning process (Dalton, 2017). Since learners have different learning preferences and needs (Boothe et al., 2018), they also differ in the manner in which they have to be engaged. MME thus requires mathematics teachers to use varied engagement strategies applicable to all types of learners, regardless of their background to ensure that learners do not undergo the psychological and emotional distress (Pokhrel, & Chhetri, 2021).
http://ijlter.org/index.php/ijlter
71
1.1 Significance of the study The MME principle serves as a guide for assisting teachers in terms of applying strategies to engage diverse groups of learners in meaningful activities meant to motivate them, address anxiety, and maximise their learning. Learners differ distinctly in terms of how they should be engaged and motivated to learn. For instance, some learners engage spontaneously and through innovation, while others frequently disengage and thus prefer a “stringent routine”. Some learners prefer to solve mathematical problems individually, while others prefer to embark on collaborative problem solving activities. This means that there is no single “means of engagement that will suit all learners in all contexts”; consequently, there is a need for teachers to “provide multiple options for engagement”. MME thus provides a comprehensive guide for engaging learners by varying “options for self-regulation, options for sustaining effort and persistence, and options for recruiting interest”. The study thus has at least two theoretical inferences: firstly, the findings contribute to the advancement of literature on learner engagement practices to be used in mathematics classrooms; and secondly, the study serves as a guide for informing inclusive planning of mathematics teaching. 1.2 Study Context The Thabo Mofutsanyane district is in the central part of South Africa in the Free State province. The district has low socio-economic status and is densely populated, with about 647 schools. Poor performance, especially in mathematics, is often reported in this district, and many teachers there are in need of training in more effective methods of learning and teaching. Three years prior to the outbreak of the Covid-19 pandemic, the researcher was a teaching and learning coordinator at the local university and was involved in the in-service training of mathematics teachers. She introduced them to the concepts of UDL and MME, which captured their interest and led to the implementation of an action research project with the purpose to empower them with teaching strategies aimed at improving their teaching practices in order to maximise learning in mathematics. The action research project ran for two years (2017-2018) through four phases, namely planning, action, observation, and reflection. Through this, the participants were empowered with knowledge and skills on how to implement UDL principles in their mathematics classrooms by the end of the project in 2018. When the pandemic started, the researcher became interested in how these mathematics teachers would adopt the MME principle during one year of pandemic regulations that had a large impact on the learning environment, such as the lockdown period (no school attendance) and social distancing, which led to a system of alternating school weeks for learners. The interest was sparked by the fact that the school in which the research was conducted was of lower socioeconomic (deprived of the luxury of online facilities). The school in which the study was conducted had approximately 1 600 learners who attended school on a fortnightly basis (i.e., two weeks per month). The teachers engaged in focus group discussions in which they shared their experiences of and insights into how they used the MME principle to engage
http://ijlter.org/index.php/ijlter
72
learners in learning, thereby “providing options for self-regulation, for sustaining effort and persistence, and for recruiting interest”. The school in which the study was conducted, was in a developing environment wherein poverty levels were high, with poor infrastructure and lack of computers which meant that the online learning was not an option. Teachers therefore had to be creative in terms of thinking of ways in which they can support learning even when they were not physically present to teach learners. Their efforts were exemplary to others in similar environment and with a possibility of adaptation to online learning as well. 1.3 Overview of the article The article commences with a brief overview of perspectives from the literature on the learner engagement theory and MME as a UDL principle. The methodology section provides detail on the design and execution of the study. In the discussion of the findings, the experiences and insights of the participants are shared to indicate how they adopted the MME principle during one year of Covid19 regulations, especially the lockdown (no school attendance) and social distancing that led to a system of alternating school weeks, which had a significant impact on the learning environment. All these views lead to a conclusion on the use of MME in the learning of mathematics by diverse groups of learners in challenging learning environments.
2. Literature review on Multiple Means of Engagement The UDL is a teaching framework that comprises of three principles, namely “multiple means of representation, multiple means of action and expression, and MME” (Center for Applied Specialised Technology, 2011). It is an approach to teaching that requires teachers to rethink their teaching strategies in order to address the needs of all learners. It affords learners opportunities to demonstrate what they know in several ways. The UDL refers to “an approach to teaching that involves the proactive design and use of inclusive teaching strategies that benefit a broad range of learners, including those with disabilities (Scott et al., 2002, p.1)”. Although UDL comprises of the three principles as already mentioned, this article focuses only on one of these principles, namely; MME. The MME principle is expedited through the application of numerous and flexible methods of engagement of students’ learning that are incorporated into teaching to support affective learning (Boothe et al., 2018; Dalton, 2017; Evmenova, 2018). It requires teachers to use diverse strategies that allow learners to be engaged and motivated to learn (Dalton et al., 2012). According to cognitive science, the affective network is part of the brain that is linked to MME, which addresses the “why” of learning (Rose & Meyer, 2002). Affective networks help regulate emotional involvement with learning such as learners’ motivation and their ability to focus on and remain engaged with a task (Darling-Hammond et al., 2020). MME, which is linked to affective networks, requires teachers to be reflective about their teaching practices in order to devise flexible engagement strategies to enhance mathematics learning.
http://ijlter.org/index.php/ijlter
73
It should be noted though that the MME principle does not only address the issue of why learners engage but also why they engage in the manner in which they do. For this reason, Hartnett (2020) stresses the need to consider it in teaching and learning contexts. Courey et al. (2013) also recommend the use of MME, as it enables teachers to recognise that no single option of engagement works for all learners and provides a comprehensive guide in terms of how teachers should provide learners with a range of engaging learning materials to maximise their learning and engage them in meaningful mathematics learning. Capp (2020) notes that MME is vital to help stimulate learners’ interests and increase their motivation to learn and, accordingly, recommends teachers to consider it. When teachers apply the MME principle, they are often advised to look into finding ways to link material to learners’ lives, interests, and experiences by giving them choices (Dalton, 2017). This principle guides engagement by providing three types of options, namely; “options for self-regulation, for sustaining efforts and persistence, and for recruiting interest” (Capp, 2020; Center for Applied Specialised Technology, 2014). In this study, teachers reflected on how they used this principle to maximise mathematics learning. The study therefore highlights the teachers’ experiences of and insights into how they provided these three options. Table 1 reflects the MME principle and the guidelines it consists of. These are the guidelines that guided or informed the study. Table 1: Multiple means of engagement principle (Center for Applied Specialised Technology, 2014) Provide multiple means of engagement Provide options for self-regulation + Promote expectations and beliefs that optimise motivation + Facilitate personal coping skills and strategies + Develop self-assessment and reflection Provide options for sustaining effort and persistence + Heighten salience of goals and objectives + Vary demands and resources to optimise challenge + Foster collaboration and a sense of community + Increase mastery-orientated feedback Provide options for recruiting interest + Optimise individual choice and autonomy + Optimise relevance, value, and authenticity + Minimise threats and distractions
3. Learner Engagement Theory The learner engagement theory was adopted as the lens underpinning the study. Engagement refers to an individual’s active participation and involvement in a particular activity, where motivation expresses impetus behind the activity (Appleton et al., 2006). Sinatra et al. (2015) postulate that the “concept of engagement” can be explored from different levels in the learning process. The
http://ijlter.org/index.php/ijlter
74
first level is “micro-level engagement”, which they define as a learner’s state of “engagement in a moment, task, or learning activity”. This is the level that is typically found in a class with a time constraint in the order of minutes to hours. The second level is “macro-level engagement” which characterises engagement across time (e.g., days and weeks), contexts (e.g., classwork, homework, and groupwork) and activities associated with the learning tasks that take place before and after learning events. According to Csikszentmihalyi (1997), the highest degree of micro-level engagement is represented by the flow concept, which is experienced when interaction with the learning task results in the learner being “in a state of effortless attention, arising through an interaction between positive affect and high attention” (De Manzano et al., 2010, p. 301). According to Engeser (2012), flow is observable when learners become “totally absorbed in” and focus on the given tasks. The improved “state of flow” takes place under three main conditions, namely; clearly defined task goals provided to the learners, prompt constructive feedback provided to the learners regarding their performance and progress made, and the learners’ current skill levels being “matched to the challenge” they are faced with (Engeser & Rheinberg, 2008). The learner engagement theory was deemed suitable for the study because it highlights different types of engagement, explains what engagement should look like, and outlines what makes it successful.
4. Research Design and Methodology In this phenomenological case study, eight teachers (with teaching experience ranging from 8 to 18 years) of one high school in the Thabo Mofutsanyane district were purposively selected to participate in the research. Of the eight teachers who participated in the study, five were males and three were females. The age of the teachers ranged between 36 to 52 years. The school in which the research was conducted is one of the biggest high schools in the district, with an enrolment of approximately 1 600 learners. The school was selected because the mathematics teachers in this school had previously undergone UDL training and, therefore, were familiar with all of its principles, including MME. These teachers were chosen because of their lengthy experience in teaching mathematics and their knowledge of the application of UDL principles in the classroom. A series of six recorded focus group discussions, each session lasting for two hours, were conducted to generate data. As the teachers had previously undergone UDL training and had implemented the principles in their classes prior to the outbreak of the pandemic, the researcher became interested in finding out how they were using these principles (especially the MME principle) given the pandemic that had forced schools to observe the rules of the national disaster (wearing masks, social distancing, etc.). Focus group discussions were conducted with the mathematics teachers so that they could provide first-hand information on how they had implemented the MME principle to maximise learning. The themes that were used were informed by the guidelines of MME as shown in Table 1. The first theme was based on the teachers’ implementation of MME to “provide options for self-regulation”; the second theme was based on the teachers’ implementation of MME to “provide
http://ijlter.org/index.php/ijlter
75
options for sustaining effort and persistence”; and the third theme was based on the teachers’ implementation of MME to “provide options for recruiting interest”. The subthemes and categories were also informed by the MME guidelines as shown in Table 1. The themes and subthemes were used to understand how the teachers had adopted MME to maximise mathematics learning. The qualitative content analysis techniques were used to analyse the data; they were partly directed and partly inductive. The techniques involved numerous steps for analysing data, beginning with the reading of the data texts, placing different labels on the texts to differentiate it (open-coding), and clustering the emerging concepts into bigger clusters (categories). The bigger clusters were further grouped into themes that came out from the data (Hsieh & Shannon, 2005). The analysis of data in this study was directed because the general themes were determined a priori. Since the study sought to explore a phenomenon using a few participants in their school setting, the findings of this study cannot be generalised. However, in a context similar to that of this study, these results could apply. Furthermore, the findings of this study may contribute to the body of knowledge by illuminating some ways in which MME can be implemented to effectively engage learners and maximise learning. The free attitude interview (FAI) technique was used for data collection during the focus group discussions. In line with FAI, the open-ended questions that were posed during the focus group discussion (based on how the teachers had implemented the MME principle by providing “options for self-regulation, for sustaining effort and persistence, and for recruiting interest” to maximise learning) made it possible for the participants to engage in lengthy discussions leading to the generation of rich data. In order to eliminate researcher bias, member checking was conducted to ensure that the data were analysed and the findings interpreted in a manner that correctly captured and portrayed the participants’ explanations, ideas, and opinions. The University of the Free State issued ethical clearance to conduct the research. The Free State Department of Education and the principal of the school where the study was conducted were approached in order to ask for their permission to conduct the study and they gave approval. The participants (teachers) in the research signed consent forms. Their right to confidentiality was emphasised, and anonymity was guaranteed. The participants were requested to participate on a voluntary basis, and they were informed that they would not be penalised should they decide not to proceed with participation at any stage of the research project.
5. Findings and Discussion The following sections highlight the mathematics teachers’ experiences of and insights into how they adopted the MME principle and applied it to provide the “options for self-regulation, options for sustaining effort and persistence, and options for recruiting interest”.
http://ijlter.org/index.php/ijlter
76
5.1 Provide Options for Self-regulation Self-regulation is of extreme importance, especially in the pandemic context in which learners were given tasks to perform individually. This necessitated the teachers to develop self-regulation strategies to enable learners to learn independently and to maximise learning. During a focus group discussion, the teachers reflected on how they had implemented the MME principle to provide options for self-regulation. They highlighted some of the strategies they had employed to “promote expectations and beliefs that optimise motivation” (Center for Applied Specialised Technology, 2011). They reflected as follows: It is important to give mathematical tasks with clear instructions on what [is] to be done. Learners usually respond better when they know what they are expected to do. They also respond better when they are given the opportunities to tackle tasks in their own ways. (Teacher 3) It also helps to let learners know what you want from them … what is it that you want them to do … what you expect of them to do with respect to tasks that you give them. (Teacher 4) I found it useful to give learners problems which they understand. They become motivated when they work with such problems. (Teacher 2) The above extracts indicate the significance of making learners understand expectations in order to successfully engage them and maximise their learning. According to Teacher 3, teachers can do so by providing learners with clear instructions on what is expected from them so that they know how to approach the given tasks. According to both Teachers 3 and 4, learners respond better when they understand the tasks they are given and know what is expected from them. According to Teacher 2, it is vital to give learners mathematical problems that they understand, as this will motivate them to attempt to solve these tasks or problems. What also seems to be important to do as part of engaging learners and maximising their learning is to give them opportunities to solve problems using their own approaches. Such opportunities will make learners feel in charge and responsible for their own learning processes and thus raise their motivation to engage in the given tasks. The form of teaching that is espoused from the extracts in which expectations are promoted is regarded as a “high impact practice”, which Ambrose et al. (2010) deem significant in terms of motivating learners to learn so that they can make the progress necessary to contribute towards their metacognitive gains. Teaching that promotes high expectations and beliefs not only improves learning but also elevates motivation, inculcates a sense of selfefficacy in learners, and results in the higher levels of engagement necessary for meaningful mathematics learning experiences and maximising learning. As learners were given tasks to work on individually whilst at home during the lockdown, teachers had to use strategies that helped facilitate the learners’ coping skills. The teachers reflected as follows: What I tried to do during lockdown was to give my learners detailed examples and step-by-step guidelines for solving problems so that they can [sic] do the problems on their own wherever they were. (Teacher 4)
http://ijlter.org/index.php/ijlter
77
I found giving tasks together with the checklists helpful because the learners are able to check how far they are with the task and also what is still outstanding. This also eliminates distractions they might come across and ease the anxiety that comes with engaging them with the task while working alone. (Teacher 1) Giving them different ways to solve the mathematical problems helps, especially when I give similar tasks because the learners were able to apply the methods that worked for them and not rely on the ones that I introduced them to. (Teacher 5) The extracts above indicate that engagement is not only a “classroom business” but an important practice that should also be implemented when learners are working on their own outside the mathematics classroom. The pandemic had thus forced teachers to use strategies that would help learners engage with the learning tasks while working individually without their teacher’s physical presence. According to Teacher 4, learners can be engaged outside the classroom with the tasks given to them by providing them with detailed, explicit examples, serving as a guide on how to solve the problems, and step-by-step guidelines on how to solve mathematical problems before they have to solve similar problems independently. According to Lipscomb et al. (2004), a step-by-step guide serves as an important practice and a useful scaffold for supporting learning by helping to bridge the task demand in light of the learners’ current level, thus enabling learners to be more successful than they would have been otherwise. Step-by-step task-orientated scaffolding also provides learners with ample support and guidance necessary to manage the main areas of difficulty when they are working independently (Smirnova, 2016). According to Teacher 1, checklists are also useful when learners work independently, as these help learners to monitor their own work and to establish where they are with the given tasks by identifying the completed and outstanding parts of the tasks. Checklists therefore serve as an adaptive strategy for helping learners to monitor, manage, and direct their emotional responses to external events, while also helping to reduce taskirrelevant distracters. Hyppönen, Hirsto & Sointu (2019) note that learners who monitor themselves and also have the ability to self-regulate their “learning processes are more likely to perform well in their academic tasks”. Moreover, Teacher 5 alluded that exposing learners to different ways of solving mathematical problems not only optimises learning but can also serve as an opportunity for teachers to assist learners in coping with the demands of the mathematical content. Introducing different methods of solving mathematical problems provides learners with an opportunity to select and use simple methods for them, which, in turn, makes it possible for learners to cope with the demands of the given tasks. Developing the self-assessment and reflection strategy was also highlighted as a good strategy for self-regulation. The teachers reflected as follows: Although we are able to monitor the learners’ progress when giving them problems to solve in class, we still have to make sure that we empower our learners to be able to monitor their own progress when they are working on their own … Eeeerrrrr, like during the lockdown. (Teacher 7)
http://ijlter.org/index.php/ijlter
78
Bearing in mind that our learners differ in so many ways, I think we need to give clear guidelines to all of them, especially those who cannot be able [sic] to do this on their own. (Teacher 8) I assisted my learners to do self-assessments and reflections, and I realised that that helped them realise areas where they were struggling as well as realise the progress they were making, and that was really motivating for them. (Teacher 6) Monitoring is an important teaching and learning practice that teachers have to inculcate in learners so that they are able to monitor their progress. This was emphasised by Teacher 7, who pointed out that teachers had the responsibility to enable learners to monitor their own progress. This means that teachers have to develop self-assessment strategies and explicitly teach and guide learners on how to do self-assessment. By doing so, the teachers will develop a better capacity for self-regulation. According to Bercher (2012), self-assessment and reflection are important because they enable learners to monitor not only their progress but also their emotions and reactivity sensibly. In accordance with Teacher 7, Teacher 8 added that learners differed in many ways and, on the basis of this, teachers had to provide a great deal of explicit instruction and guidelines for learners for them to learn how to successfully self-assess. The need for self-assessment and reflection was further stressed by Teacher 6, who encouraged teachers to help learners self-assess and reflect because the mere recognition of one’s own progress towards greater independence could be highly motivating. 5.2 Provide Options for Sustaining Effort and Persistence The discussion on how teachers implemented the MME principle revealed the need to offer alternatives for sustaining effort and persistence as another important strategy to engage learners and maximise learning. The teachers pointed out the significance of reminding learners about the lesson outcome expected to be achieved and how to achieve it. They reflected as follows: I think we need to keep on reminding learners of the lesson outcome they are expected to achieve. When learners are given a task to do, they must be given timeframes to complete it and in-between be reminded of what they have to do to achieve the lesson outcome. (Teacher 3) I often try to show various ways in which the goal can be achieved. (Teacher 1) … the same way I do in class under normal circumstance[s] to encourage learners and motivate them to learn, I found that working for me, especially when giving them meaningful feedback to enable them to correct their mistakes and address misconceptions. (Teacher 5) According to the extracts above, one way of sustaining effort and persistence when engaging learners, as highlighted by Teacher 3, is by regularly reminding learners about the intended lesson outcome to be achieved regarding the given mathematical tasks. The timeframe for completing a task must be communicated,
http://ijlter.org/index.php/ijlter
79
and teachers should be there to remind the learners about the lesson outcomes to be achieved and guide them so that the outcome can eventually be achieved. Such support is required to help sustain the effort and concentration of the learners in the face of distracters. Teacher 1 explained that the lesson outcome (i.e., intended goal) to be achieved should be displayed or demonstrated in multiple ways; hence the statement “I often try to show various ways in which the goal can be achieved”. What Teacher 1 pointed out is important in order to cater for a diverse learner population. The practice of making a goal clear to the learners, in an endeavour to maximise learning, is advocated by Anderson and Stritch (2016), who avow that even if the goal may be difficult to achieve, when learners know it and are guided well on how to achieve it, they will pursue it. Hatip (2020) also espouses the notion of making a goal clearer so to maximise learning. Being guided on how to achieve the goal thus helps in terms of raising learners’ motivation levels and inspires them to make an effort and persist in engaging in the given mathematical tasks, even in the face of challenges. According to Teacher 5, another way to sustain effort and persistence, especially in the time of the pandemic when learners are working independently, is to continually motivate and encourage the learners by providing them with immediate corrective feedback that will enable them to correct their mistakes and address misconceptions. Febrianto et al. (2020) also stress the significance of motivating learners when they are working independently particularly during the pandemic when teachers are not present. In engaging learners, teachers should keep in mind the fact that learners differ in terms of how they learn and their levels of understanding content. This implies that learners should be engaged in different ways and be given tasks that match their learning abilities and different levels of understanding. Consequently, teachers should vary the demand and resources to meet challenges when engaging learners in mathematical activities. This point was raised by Teachers 2 and 3, who narrated as follows: We teach learners who learn differently, and therefore we need to vary the resources and also give tasks that learners will be able to tackle. (Teacher 2) Knowing my learners helped me a lot because I was able to give them tasks that match their abilities. (Teacher 3) The practice of varying demand and resources to meet challenges when engaging learners is supported by the Center for Applied Specialised Technology (2011), which asserts that no single means of engagement is optimal for all learners in all contexts. According to Teacher 3, knowing the types of learners in her class was helpful in terms of devising activities that matched the abilities of her learners. Matching abilities with task demands thus helps to sustain motivation for learners to remain engaged with activities. Fostering collaboration and a sense of community was also highlighted as a useful strategy for engaging learners and maximising learning (Mishra et al.,2020). Although during the lockdown, the teachers were not able to implement the strategy, they did manage to implement it shortly after the lockdown, during the period of alternating groups of learners coming to school every week. The teachers commented as follows:
http://ijlter.org/index.php/ijlter
80
Although during [the] lockdown I could not be able [sic] to create groups for learners to work individually, I managed, though, to create the [sic] small groups during the period of alternating groups of learners that came to school after [the] lockdown. Creating such small groups in class was helpful. It gave learners opportunities to work together and learn from each other, although I had to ensure that they observe the social distancing rule and also wear masks during their interaction in class. (Teacher 6) I formed the groups in such a way that learners were able to work well together so that they can benefit from each other. (Teacher 4) I found it helpful to change the members of the groups from time to time. (Teacher 7) Fostering collaboration among learners was not possible during the lockdown period; however, when the learners went back to school the teachers managed to get them to work in small, intimate groups while observing the rules of the pandemic. The teachers deemed the practice of fostering collaboration among the learners important as part of engaging learners and maximising their learning. According to Teacher 6, fostering collaboration can be done by creating small, intimate groups. The teacher regarded group work as an important practice that should be implemented to afford learners opportunities to work cooperatively and learn from one another. Teacher 4 pointed out that the groups of learners should be carefully formed so that they could derive maximum benefit by learning from one another. Teacher 7 added that members of the groups should be changed from time. Changing the members of groups frequently allows flexibility and exposes learners to diverse ideas from time to time, which is vital for maximising learning. The practice of peer cooperation is highly espoused in teaching and learning contexts and is a practice that is highly recommended in the 21st century. According to Poellhuber et al. (2008), peer collaboration helps to “create a sense of community” that is much needed to sustain learner motivation and persistence and support learners’ commitment to studying and remaining engaged. McKeen (2019) notes that flexible grouping enables better differentiation and manifold roles, while also providing opportunities for learners to work collaboratively. Flexible grouping also allows mathematics teachers to adapt their teaching and this enables learners to achieve the learning objectives. The teachers also reflected on increasing mastery-orientated feedback as a key strategy in implementing the MME principle of “providing options for sustaining effort and persistence” (Center for Applied Specialised Technology, 2011). The teachers reflected as follows: When I engage my learners, it really helped to give them assessments and provide feedback from time to time, especially feedback that is encouraging and helpful … I think we must give them feedback that makes them realise their mistakes so that they can be able [sic] to correct the mistakes, and this needs to be done on time. (Teacher 8) Sometimes when we give feedback, we tend to compare and create unhealthy competitions amongst the learners. Again, we give feedback
http://ijlter.org/index.php/ijlter
81
that focuses only on the marks instead of addressing the errors which learners often commit. I think this is what we need to change if we want our learners to remain engaged. (Teacher 1) According to Teacher 8, one way in which teachers can provide masteryorientated feedback is by frequently giving assessment and feedback. The expression “feedback that is encouraging and helpful” refers to feedback that not only makes it possible for learners to realise their mistakes and what causes these mistakes but also indicates areas where they are doing well. This means that assessment should not only focus on weaknesses but also point out the strengths of the learners as part of motivating them. The practice of giving constructive and corrective feedback is crucial in terms of sustaining the learners’ motivation and effort towards learning (Al-Hattami, 2019). According to Teacher 1, ensuring that learners remain engaged requires teachers to give constructive and corrective feedback that is aimed at building the learners and not unfair feedback that focuses on comparing their strengths and weaknesses. The feedback should not focus only on the marks awarded to learners but also on worked-out problems. This should be done by paying attention to how the problems were solved (i.e., steps for solving the problems) in order to identify errors made so as to share them with the learners at a later stage in an endeavour to enlighten and correct them. This type of feedback is regarded as masteryorientated feedback because it serves to guide learners towards “mastery rather than a fixed notion of performance compliance” (Darling-Hammond et al., 2020). This type of feedback is vital for sustained engagement because it accentuates the importance of “process” over “product”, thus encouraging learners to embrace the notion of focusing on the attainment of the lesson objectives and appreciating the progress they are making in terms of learning content rather than focusing on the marks. This approach to feedback is also vital in terms of nurturing “mastery orientation on the part of the learners” where they seek not only to advance their understanding of content, skills and learning strategies, but also to identify relevancy of the mathematical concepts which they are learning and “meaningfulness” in the task itself (Ames, 1992; Hattie & Gan, 2011). The suggestions made by the teachers necessitate teachers to carefully look into the feedback they provide to ensure that it inspires persistence in learners to be able to remain engage in problem solving. 5.3 Provide Options for Recruiting Interest The pandemic has made learning difficult for most learners, as they had to struggle individually working on their tasks. The issue of working individually necessitated teachers to devise strategies for recruiting learners’ interest so they could remain engaged and motivated to work on the given mathematical tasks. The teachers thus highlighted some strategies that they used to optimise individual choice and autonomy as a way of recruiting learners’ interest and maximise learning. They narrated as follows: One of the things that worked for me was to try to relate content and real life. Relating content to real life helps because it makes learners realise the importance of what they are learning. (Teacher 5)
http://ijlter.org/index.php/ijlter
82
… just because we present content in a manner which other learners find easy to relate to, that should not make us assume that all learners prefer to be taught in that way. We need to vary the activities and present content in different interesting ways. Eeeehhhh … we really need to come up with different strategies that will stimulate the learners’ interest. (Teacher 7) Teacher 5 highlighted the usefulness of real-life situations when engaging learners (“because it makes learners realise the importance of what they are learning”) as one way in which teachers could maximise learning, engage learners in meaningful learning, and also recruit their interest in learning mathematical concepts. According to the teacher, the relevance of the concepts taught in class and demonstrating this relevance through authentic, meaningful activities maximise learning. Moleko and Mosimege (2020) avow that teaching mathematics in context increases engagement. This form of teaching enables learners to apply the learnt concepts in real life, helps develop their new perceptions and skills to be able to use mathematical reasoning to problems, and enhances their capability to partake in the “economy of today and tomorrow” (Umugiraneza et al., 2017). According to Teacher 7, teachers should not “assume that all learners will find the same mathematical activities or content equally relevant or valuable”; therefore, they should accordingly vary the activities or forms of content representations “in order to cater for the diverse learning preferences and styles of all the learners”. Teachers should also invest time in devising various strategies that will assist in terms of stimulating learners’ interest so they will remain engaged. However, it should be noted that teachers can only successfully do so if they know their learners. For instance, teachers can design visual activities for learners who best assimilate information when represented in visual formats and do the same with other groups of learners with different learning styles and preferences (auditory learners, tactile learners, etc.). In order for effective learner engagement to be realised and for learning to be maximised, teachers have to ensure that they minimise threats and distractions that may impede learning (Center for Applied Specialised Technology, 2011). In line with this, the teachers commented as follows: I have realised that one of the things that kill our learners, especially when we do not address it, is the mathematical vocabulary. It is difficult for learners to solve problems when they do not understand some of these words. For example, words such as “at least”, “compound”, etc. should be taught to learners so that they can know what they mean. (Teacher 2) Our teaching should not create misunderstandings because this later makes it difficult for learners to solve problems on their own. For example, two raised to exponent three should be pronounced as such and not as two to power three. (Teacher 3) Teacher 2 highlighted the significance of addressing the insufficient knowledge of mathematical vocabulary in order to eliminate knowledge gaps that could hamper the learning process. According to the teacher, lacking an understanding
http://ijlter.org/index.php/ijlter
83
of the mathematical vocabulary could be a potential threat that can impede mathematical problem solving. The teacher gave an example of terms such as “at least” and “compound”, which are not only used within a mathematical context but also in other contexts. These are words that teachers should explain so that their meanings can be understood in context to avoid confusion (Moleko & Mosimege, 2020). Furthermore, Teacher 3 noted that teaching that causes misunderstanding should be avoided in order for learners to be engaged in meaningful mathematics learning. The example the teacher gave is that of “two raised to exponent three” and “two to power three”. According to the teacher, these two expressions are often used interchangeably; yet they do not mean the same thing. For example, two raised to exponent three can be represented numerically as 23 (where 2 is a base and 3 an exponent); and 23 (a base together with an exponent) is called a “power”, mathematically speaking. Consequently, failure on the part of teachers to mind how they teach these mathematical expressions in class may result in teaching that causes distractions, which could subsequently impede learning. The teaching of mathematics therefore requires teachers to avoid or eliminate elements of distractions at all times in order to maximise learning.
6. Summary of examples of multiple means of engagement strategies to maximise learning of mathematics in pandemic-regulated classrooms 6.1 Offer the Alternatives for Self-regulation The findings of the study indicated that during the lockdown period learners worked on their own without the physical presence of the educators to guide them. This necessitated the clear expectations regarding the given mathematical tasks to be communicated so that learners could understand what is expected of them. The significance of giving learners the mathematical problems, which they possess knowledge of, was highlighted as a good strategy to motivate and ensure that they remain engaged. This therefore means that the teachers have to design mathematical examples that are clear to follow, and also provide the step-by-step guidelines in order to make it easy for learners to solve similar given problems independently. Lipscomb et al. (2004) regard the step-by-step guide as an important practice and a useful scaffold to support learning and help bridge the task demand in light of the learners’ current level. When learners are expected to learn independently, it also helps to provide them with checklists so that they can be able to monitor their own progress. Inculcating the culture of self-monitoring and empowering learners with skills to monitor their own progress is a worthwhile practice that helps facilitate independent learning (Bellini & Peters, 2008). Self-assessment and reflection were also found to be useful in terms of enabling learners to identify areas in which they were struggling with as well as those they were making good progress on. Bercher (2012) regards self-assessment and reflection practices worthwhile in terms of assisting learners not only to monitor their progress but also to monitor their emotions and reactivity sensibly. A blend of these strategies emerging from literature and empirical data thus point to self-regulation, which is constituted within the MME framework. Since learners “differ markedly in the ways in which they could be engaged”, this necessitates teachers to vary the strategies and to offer alternatives for self-regulation.
http://ijlter.org/index.php/ijlter
84
6.2 Offer the Alternatives for Sustaining Effort and Persistence Many learners find mathematics challenging and often develop senses of helplessness and resentment towards the subject. These behavioural dispositions are often exhibited by learners in classrooms, in the presence of teachers who are there to guide them. Solving mathematical problems on their own (without the presence of the teachers) thus require teachers to use strategies that will help sustain effort and persistence. Although this may be difficult to achieve, findings of the study indicate several MME strategies that can be used to sustain effort and persistence. These strategies include; regularly reminding learners about the intended lesson outcomes which should be achieved, communicating the timeframes for completing the tasks and guiding the learners on how to achieve the learning goal. Anderson and Stritch (2016), maintain that the practice of guiding learners on how to achieve the learning goal is significant because even if the learning goal may be difficult to achieve, when learners know it and are guided well on how to achieve it, they will pursue it regardless. Providing instant corrective feedback is also vital in terms of enabling learners to realise their mistakes and misconceptions soon and to correct them. Giving tasks that match the learners’ learning abilities and different levels of understanding is also important in terms of motivating learners when they are working independently. This therefore necessitates teachers to vary the demand and resources to meet challenges when engaging learners in mathematical activities. Fostering collaboration amongst the learners affords learners with opportunities to work together to learn from one another. However, the groups should be carefully formed so that learners could derive maximum benefit by learning from one another. Changing the members of groups frequently is also important as it allows flexibility and exposes learners to diverse ideas from time to time. Poellhuber et al. (2008) espouse the notion of peer collaboration since it helps to “create a sense of community that is much needed to sustain learner motivation and persistence and support learners” commitment to studying and remaining engaged. Flexibility grouping on the other hand enables better differentiation (McKeen, 2019) and also allows teachers to tailor mathematics instruction such that they meet the needs of diverse learners. The practice of providing the constructive and empowering corrective feedback is crucial in terms of sustain motivation and effort (Al-Hattami, 2019). This should be the type of feedback that not only focus on making learners aware of their mistakes but also shows areas in which they are doing well. A mixture of these strategies emerging from literature and empirical data thus point to the notion of sustaining effort and persistence which is also constituted within the MME framework. Since learners differ in terms of motivation levels, MME thus advocates the notion of offering varied strategies and alternatives for sustaining effort and persistence. 6.3 Offer the Alternatives for Recruiting Interest Recruiting interest is important to ensure that learners remain focused on the given mathematical tasks. One way to recruit interest is through the use of reallife situations. Learners learn best when real life situations which they are familiar with are used and related to the content they are being taught. This form of teaching enables learners to apply the learnt concepts in real life, to “develop new perceptions and skills to apply mathematical reasoning to problems”, and to
http://ijlter.org/index.php/ijlter
85
enhance their “capacity to participate” in the economy of today and tomorrow (Umugiraneza et al., 2017). Since learners may not find “the same mathematical activities or content equally relevant or valuable”; it is therefore advisable for teachers to vary the activities or forms of content representations in order to cater for the diverse learning preferences and styles of all the learners. Lack of understanding the mathematical vocabulary should be addressed since it can be a potential threat that can impede mathematical problem solving. Teaching that causes misunderstanding and confusion should therefore be avoided in order for learners to be engaged in meaningful mathematics learning. The findings of the study revealed two significant contributions (made by the study): firstly, the findings of the study contribute to the expansion of the teaching and learning theories in mathematics to promote; active learning, inclusivity and sustainability. Secondly, the study serves as a guide for promoting learner engagement by proposing a “new idea” of multiple means of engagement (MME) approach in mathematics, which should be regarded as an essential strategy to promote active learning. The summary of the MME strategies highlighted above were deemed useful to provide all learners a better platform for overcoming their problems with mathematics and the challenges brought about by the pandemic regulations. It should be noted though that, different environments may require different combinations/combinations of the above strategies.
7. Conclusion The study highlighted how mathematics teachers had adopted MME to maximise learning. It was revealed that “no single means of engagement will be ideal for all learners in all contexts”; therefore, it is essential for teachers to vary their teaching strategies to provide multiple options for engagement. The teachers in the study maximised mathematics learning by using MME guidelines, namely; “providing options for self-regulation, options for sustaining effort and persistence, and options for recruiting interest”. Providing options for self-regulation in the study involved promoting high expectations for optimising learner motivation to engage with the given mathematical tasks, facilitating personal coping skills and strategies for learners to sustain engagement with the tasks even in the face of difficulties, and developing self-assessment strategies and teaching learners how to use them so they were able to monitor their progress. Providing “options for sustaining effort and persistence” required the teachers to indicate the learning goal to be achieved and guide their learners on how to achieve it, to foster peer collaboration among the learners so they could engage and learn from one another, and to increase mastery-orientated feedback by providing immediate feedback that was constructive and corrective. Providing “options for recruiting interest” required the teachers to reinforce relevance and value by using real-life examples so that learners could recognise the applicability of the concepts they were learning in reality and also by eliminating elements of threats and distractions that might hamper learning. The MME approach in mathematics was also found to be useful in terms of developing purposeful and motivated learners. One of the greatest lessons the study provided is that teachers are “a crucial part
http://ijlter.org/index.php/ijlter
86
of curriculum implementation” and therefore they cannot detach themselves from the learning experiences of learners (Mapuya, 2021). Hence they have to strive to monitor, assess and engage learners even when learners are not in class due to Covid pandemic.
8. Recommendations The findings of this study suggest the need for further research to be conducted on different topics in mathematics in order for teachers to maximise learning through MME. It should be noted that different mathematics topics necessitate different ways in which learners can be engaged in the learning processes. The MME principle will therefore serve as a guideline for engaging learners in meaningful mathematics learning as well as maximising the learning thereof. Also, MME will enable teachers to cater for and provide alternative ways in which to engage a broad range of learners in meaningful mathematics learning experiences. Funding: This research is funded by the National Research Foundation (NRF) Conflicts of Interest: The author declares no conflict of interest. The study was conducted primarily for scholarly purposes. The conceptualisation of the study and all the empirical processes were not influenced by own personal interest or anticipated gain.
10. References Al-Hattami, A. A. (2019). The perception of students and faculty staff on the role of constructive feedback. International Journal of Instruction, 12(1), 885-894. Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C., & Norman, M. K. (2010). How learning works: Seven research-based principles for smart teaching. John Wiley & Sons. Ames, C. (1992). Classrooms: Goals, structures, and student motivation. Journal of Educational Psychology, 84(3), 261. https://psycnet.apa.org/doi/10.1037/00220663.84.3.261 Anderson, D. M., & Stritch, J. M. (2016). Goal clarity, task significance, and performance: Evidence from a laboratory experiment. Journal of Public Administration Research and Theory, 26(2), 211-225. https://doi.org/10.1093/jopart/muv019 Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools, 45(5), 369-386. https://doi.org/10.1002/pits.20303 Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006). Measuring cognitive and psychological engagement: Validation of the Student Engagement Instrument. Journal of School Psychology, 44(5), 427-445. https://doi.org/10.1016/j.jsp.2006.04.002 Balwant, P. T. (2018). The meaning of student engagement and disengagement in the classroom context: Lessons from organisational behaviour. Journal of Further and Higher Education, 42(3), 389-401. https://doi.org/10.1080/0309877X.2017.1281887 Bansilal, S., & Ubah, I. (2019). The use of semiotic representations in reasoning about similar triangles in Euclidean geometry. Pythagoras, 40(1), 1-10. https://doi.org/10.4102/pythagoras.v40i1.480 Bellini, S., & Peters, J. K. (2008). Social skills training for youth with autism spectrum disorders. Child and adolescent psychiatric clinics of North America, 17(4), 857-873. https://doi.org/10.1016/j.chc.2008.06.008
http://ijlter.org/index.php/ijlter
87
Bercher, D. A. (2012). Self-monitoring tools and student academic success: When perception matches reality. Journal of College Science Teaching, 41(5), 26-32. Boothe, K. A., Lohmann, M. J., Donnell, K. A., & Hall, D. D. (2018). Applying the principles of universal design for learning (UDL) in the college classroom. The Journal of Special Education Apprenticeship, 7(3), 1–13. Burgstahler, S. (2008). Equal access: Universal design of instruction. DO-IT, University of Washington. edu/doit/Brochures/Academics/equal_access_udi.htm Capp, M. J. (2020). Teacher confidence to implement the principles, guidelines, and checkpoints of universal design for learning. International Journal of Inclusive Education, 24(7), 706-720. https://doi.org/10.1080/13603116.2018.1482014 Center for Applied Special Technology. (2014). UDL guidelines – Version 2.0: Principle III provide multiple means of engagement. http://www.udlcenter.org/aboutudl/udlguidelines/principle3 Center for Applied Specialised Technology [CAST]. (2011).Universal Design for Learning Guidelines version 2.0. Wakefield, MA: Author. Courey, S. J., Tappe, P., Siker, J., & LePage, P. (2013). Improved lesson planning with universal design for learning (UDL). Teacher Education and Special Education, 36(1), 7-27. https://doi.org/10.1177%2F0888406412446178 Csikszentmihalyi, M. (1997). Flow and education. NAMTA Journal, 22(2), 2-35. Dalton, E. M. (2017). Beyond universal design for learning: Guiding principles to reduce barriers to digital & media literacy competence. Journal of Media Literacy Education, 9(2), 17-29. http://digitalcommons.uri.edu/jmle/ Dalton, E. M., Mckenzie, J. A., & Kahonde, C. (2012). The implementation of inclusive education in South Africa: Reflections arising from a workshop for teachers and therapists to introduce universal design for learning. African Journal of Disability, 1(1), 1–7. https://doi.org/10.4102/ajod.v1i1.13. Darling-Hammond, L., Flook, L., Cook-Harvey, C., Barron, B., & Osher, D. (2020). Implications for educational practice of the science of learning and development. Applied Developmental Science, 24(2), 97-140. https://doi.org/10.1080/10888691.2018.1537791 De Manzano, Ö., Theorell, T., Harmat, L., & Ullén, F. (2010). The psychophysiology of flow during piano playing. Emotion, 10(3), 301. DOI: 10.1037/a0018432 Duchesne, S., & Ratelle, C. (2010). Parental behaviors and adolescents’ achievement goals at the beginning of middle school: Emotional problems as potential mediators. Journal of Educational Psychology, 102(2), 497. DOI: 10.1037/a0019320 Engeser, S. E. (2012). Advances in flow research. Springer Science+ Business Media. https://psycnet.apa.org/doi/10.1007/978-1-4614-2359-1 Engeser, S., & Rheinberg, F. (2008). Flow, performance and moderators of challenge-skill balance. Motivation and Emotion, 32(3), 158-172. DOI 10.1007/s11031-008-9102-4 Evmenova, A. (2018). Preparing teachers to use universal design for learning to support diverse learners. Journal of Online Learning Research, 4(2), 147-171. https://www.learntechlib.org/p/181969/ Febrianto, P. T., Mas' udah, S., & Megasari, L. A. (2020). lmplementation of Online Learning during The Covid-19 Pandemic on Madura lsland, lndonesia. lnternational Joumal of Learning, Teaching and Educational Research, 19(8), 233-254. https://doi.org/10.26803/ijlter.19.8.13 Filgona, J., Sakiyo, J., Gwany, D. M., & Okoronka, A. U. (2020). Motivation in learning. Asian Journal of Education and Social Studies, 16-37. https://doi.org/10.9734/ajess/2020/v10i430273
http://ijlter.org/index.php/ijlter
88
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59-109. https://doi.org/10.3102%2F00346543074001059 Gray, T., & Madson, L. (2007). Ten easy ways to engage your students. College Teaching, 55(2), 83-87. https://doi.org/10.3200/CTCH.55.2.83-87 Hake, R. R. (1998). Interactive engagement versus traditional methods: A six-thousandstudent survey of mechanics test data for introductory physics courses. American Journal of Physics, 66(1), 64-74. https://doi.org/10.1119/1.18809 Halverson, L. R., & Graham, C. R. (2019). Learner engagement in blended learning environments: A conceptual framework. Online Learning, 23(2), 145-178. https://doi.org/10.24059 /olj.v23i2 .1481 Hartnett, M. (2020). Relationships between online motivation, participation, and achievement: More complex than you might think. Journal of Open, Flexible and Distance Learning, 24(1), 75-88. https://search.informit.org/doi/10.3316/informit.195515786038745 Hatip, A. (2020). The Transformation Of Learning During Covid-19 Pandemic Towards The New Normal Era. PROCEEDING UMSURABAYA. Hattie, J., Gan, M., & Brooks, C. (2011). Instruction based on feedback. Handbook of Research on Learning and Instruction, 249-271. Hsieh, H. F., & Shannon, S. E. (2005). Qualitative health research. Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277-1288. https://doi.org/10.1177/1049732305276687 Hughes, K. (2012). A behavioural understanding of privacy and its implications for privacy law. The Modern Law Review, 75(5), 806-836. https://doi.org/10.1111/j.1468-2230.2012.00925.x Hyppönen, L., Hirsto, L., & Sointu, E. (2019). Perspectives on University Students’ selfregulated learning, task-avoidance, time management and achievement in a flipped classroom context. International Journal of Learning, Teaching and Educational Research. https://doi.org/10.26803/ijlter.18.13.5 Klein, M. (2000). How active involvement in learning mathematics can preclude meaningful engagement: Contributions from Foucault. Pedagogy, Culture & Society, 8(1), 69-83. https://doi.org/10.1080/14681360000200079 Kuh, G. D., Kinzie, J., Schuh, J. H., & Whitt, E. J. (2011). Student success in college: Creating conditions that matter. John Wiley & Sons. Leonard, S. H. (2008). Measuring cognitive and psychological engagement in middle school students [Doctoral dissertation. University of South Dakota]. Lipscomb, L., Swanson, J., & West, A. (2004). Scaffolding. Emerging perspectives on learning, teaching and technology. http://projects.coe.uga.edu/epltt/index.php?title=Scaffolding. Mapuya, M. (2021). First-Year Accounting Student Teachers’ Constructivist Learning Experiences, The Lecturer’s Role and Implications for Curriculum Implementation. International Journal of Learning, Teaching and Educational Research, 20(1), 103-119. https://doi.org/10.26803/ijlter.20.1.6 McKeen, H. (2019). The impact of grade level flexible grouping on math achievement scores. Georgia Educational Researcher, 16(1), 48-62. https://files.eric.ed.gov/fulltext/EJ1206047.pdf Mishra, D. L., Gupta, D. T., & Shree, D. A. (2020). Online Teaching-Learning in Higher Education during Lockdown Period of COVID-19 Pandemic. International Journal of Educational Research Open, August, 100012. https://doi.org/10.1016/j.ijedro.2020.100012
http://ijlter.org/index.php/ijlter
89
Moleko, M. M., & Mosimege, M. D. (2020). Teachers’ and learners’ experiences for guiding effective teaching and learning of mathematics word problems. Issues in Educational Research, 30(4), 1375-1394. ttps://search.informit.org/doi/10.3316/informit.606279637425764 Padgett, J., Cristancho, S., Lingard, L., Cherry, R., & Haji, F. (2019). Engagement: what is it good for? The role of learner engagement in healthcare simulation contexts. Advances in Health Sciences Education, 24(4), 811-825. https://doi.org/10.1007/s10459-018-9865-7 Park, C. (2003). Engaging students in the learning process: The learning journal. Journal of Geography in Higher Education, 27(2), 183-199. https://doi.org/10.1080/03098260305675 Parsons, S. A., Nuland, L. R., & Parsons, A. W. (2014). The ABCs of student engagement. Phi Delta Kappan, 95(8), 23-27. https://doi.org/10.1177%2F003172171409500806 Poellhuber, B., Chomienne, M., & Karsenti, T. (2008). The effect of peer collaboration and collaborative learning on self-efficacy and persistence in a learner-paced continuous intake model. International Journal of E-Learning & Distance Education / Revue Internationale du e-Learning et la Formation à Distance, 22(3), 41-62. Pokhrel, S., & Chhetri, R. (2021). A literature review on impact of COVID-19 pandemic on teaching and learning. Higher Education for the Future, 8(1), 133-141. https://doi.org/10.1177%2F2347631120983481 Rose, D. H., & Meyer, A. (2002). Teaching every student in the digital age: Universal design for learning. Association for Supervision and Curriculum Development. Scott, S., McGuire, J. M., & Embry, P. (2002). Universal design for instruction fact sheet. University of Connecticut, Center on Postsecondary Education and Disability. https://www.westga.edu/student-services/counseling/assetscounseling/docs/universal_design_fact_sheet.pdf Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015). The challenges of defining and measuring student engagement in science. Educational Psychologist, 50(1), 1-13. https://doi.org/10.1080/00461520.2014.1002924 Sirin, S. R., & Rogers-Sirin, L. (2005). Components of school engagement among African American adolescents. Applied Developmental Science, 9(1), 5-13. https://doi.org/10.1207/s1532480xads0901_2 Smirnova, L. A. (2016). Scaffolding preparation for a reading exam. Journal of Teaching English for Specific and Academic Purposes, 4(2), 391-398. Smith, D. C., Ito, A., Gruenewald, J., & Yeh, H. L. (2010). Promoting school engagement: Attitudes toward school among American and Japanese youth. Journal of School Violence, 9(4), 392-406. https://doi.org/10.1080/15388220.2010.509308 Smith, M. R. (2004). Math anxiety: Causes, effects, and preventative measures [Senior honors thesis. Liberty University]. https://digitalcommons.liberty.edu/honors/255 Thoonen, E. E., Sleegers, P. J., Oort, F. J., Peetsma, T. T., & Geijsel, F. P. (2011). How to improve teaching practices: The role of teacher motivation, organizational factors, and leadership practices. Educational Administration Quarterly, 47(3), 496-536. https://doi.org/10.1177%2F0013161X11400185 Umugiraneza, O., Bansilal, S., & North, D. (2017). Exploring teachers’ practices in teaching mathematics and statistics in KwaZulu-Natal schools. South African Journal of Education, 37(2). https://doi.org/10.15700/saje.v37n2a1306 Williams, K. C., & Williams, C. C. (2011). Five key ingredients for improving student motivation. Research in Higher Education Journal, 12, 1. Yengin, İ., Karahoca, D., Karahoca, A., & Yücel, A. (2010). Roles of teachers in e-learning: How to engage students & how to get free e-learning and the future. Procedia-
http://ijlter.org/index.php/ijlter
90
Social and Behavioral Sciences, 2(2), 5775-5787. https://doi.org/10.1016/j.sbspro.2010.03.942 Zyngier, D. (2008). (Re)conceptualising student engagement: Doing education not doing time. Teaching and Teacher Education, 24(7), 1765-1776. https://doi.org/10.1016/j.tate.2007.09.004
http://ijlter.org/index.php/ijlter
91
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 91-107, August 2021 https://doi.org/10.26803/ijlter.20.8.6 Received Jun 10, 2021; Revised Aug 09, 2021; Accepted Aug 29, 2021
Online Design Thinking Problems for Enhancing Motivation of Gifted Students Ayed H. Ziadat Al-Balqa’a Applied University, Princess Rahma University College, Department of Special education, Al-Salt, Jordan https://orcid.org/0000-0002-7920-2419 Mohammad Abed Sakarneh Al-Balqa’a Applied University, Princess Rahma University College, Department of Special education, Al-Salt, Jordan https://orcid.org/0000-0001-8683-7941
Abstract. Motivation is important for harnessing abilities and competencies, even if students are naturally or potentially gifted. This study used design thinking pedagogy as an innovative approach to motivate students holistically to learn and attend school. Design thinking is an approach to acquire and apply knowledge in a real situation that involves five stages: empathy, define, ideate, prototype, and test. The study was a quantitative quasi-experimental study with a one-group design and pre and post-tests. The study evaluated intrinsic and four extrinsic motivation categories – integrated, identified, introjected, and external regulation – to evaluate gifted students’ motivation to do coursework, and used a motivation scale to gauge their willingness to attend school, and an attitude questionnaire to determine students’ satisfaction with and engagement in the design thinking class. The experimental group consisted of 77 randomly selected gifted students at the King Abdullah II School for Excellence. After engaging in the design thinking class, the gifted students’ scores on all motivation categories were higher in the post-test than they had been in the pre-test. However, gifted students’ motivation levels were not associated with their gender, grades, and SAT scores. Thus, the design thinking approach is a promising approach for educating gifted students; students found it satisfying and they exhibited high levels of engagement behavior. The study results recommend that a design thinking approach is worth pursuing to increase gifted students’ motivation. The researchers recommend considering both gifted and non-gifted students in future studies involving the design thinking approach. Keywords: design thinking approach; extrinsic motivation; gifted students; motivation; intrinsic motivation
©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
92
1. Introduction Achieving the aim of developing 21st century skills and capabilities, namely, creative thinking, critical thinking, communication, and collaboration, requires teachers to adopt a suitable pedagogy to meet students’ unique needs in the class (Sakarneh & Al-Swelmyeen, 2020; Samat & Ismail, 2020; Stith et al., 2020). Gifted students, specifically, encounter a lack of teaching support in the regular classroom, which causes them to experience more motivational issues in relation to learning and school than their non-gifted peers. These motivational issues have commonly been suggested to be a cause of underachievement (Hornstra et al., 2020). Samat and Ismail (2020) explain, furthermore, that gifted students require specially designed programs that motivate them to participate in acquiring and seeking knowledge development. The motivation of gifted students positively impacts their giftedness development (Barabwd et al., 2017). Burns and Martin (2021) and McCoach and Flake (2018) have the same perception about the role of motivation in harnessing abilities and competencies, even for students who are naturally or potentially gifted. They stated that motivation is the catalyst of giftedness; an adequate motivation level is a necessity for developing giftedness for both naturally and potentially gifted students. A study in Jordan detected a clear difference between the intrinsic motivation levels of gifted female students, their gifted male peers, and ungifted peers (female and male). Gifted male students did not exhibit a higher level of motivation than their ungifted peers of either gender. Despite gifted female and male students being recruited from the same school for excellence, gifted female students acquired substantially greater benefits than their male counterparts (Heilat et al., 2019). This finding suggests that gifted male students gain fewer benefits from education programs that do not focus on fostering their motivation levels. This study investigated the design thinking pedagogy, as an innovative approach to motivate students about learning and to attend school, so that teachers can provide gifted students with the expected levels of support, satisfaction, and relatedness. The study intended to contribute to supporting classroom organization and meeting gifted students’ needs. Studies have found that the majority of gifted students fail to recognize their abilities (Hebert, 2020; Siegle & McCoach, 2018). Implementing design thinking in the classroom may contribute to supporting students’ recognition of their cognitive abilities and competencies. Personal recognition of their abilities is likely to serve gifted students professionally, socially, and emotionally (Hebert, 2020). This study focused on harnessing the potential benefits and features of a design thinking approach to enhance and upgrade gifted students’ motivation, since motivation plays an important role in students’ progress and excellence. The benefits of a design thinking approach can be assessed by measuring the difference between students’ motivation levels before and after the implementation of the approach. Furthermore, design thinking, as a new learning experience, can introduce students to a new learning concept, and prepare them for the post-high school stage of learning.
http://ijlter.org/index.php/ijlter
93
1.1 Study Objectives The study investigated the application of a design thinking approach as a motivational tool. Thus, the study aims were to determine - Differences between pre and post-tests of gifted students’ motivation to learn and attend school; - Differences in students’ motivation that were attributable to their demographic features (gender, grade, SAT score); and - Whether students developed positive attitudes to learning by approaching problems with design thinking.
2. Literature Review 2.1 Motivation The word motivation is derived from the Latin word meveo, meaning move. A simple dictionary definition of motivation is having a reason to engage, accomplish, or do a particular thing. Motivation carries out a force to act or move to action (McCoach & Flake, 2018). According to self-determination theory, this means stimulating attitudes, values, and tendencies towards action. According to this theory, there are two motivation categories: intrinsic and extrinsic motivation (Hornstra et al., 2020). Intrinsic motivation is the ingrained, inherent, and innate inclination to engage, seek or participate in challenges to feel satisfaction or enjoyment (Heilat et al., 2019). Extrinsic motivation refers to external stimuli or forces that induce a move to action or to taking action (Hornstra et al., 2020; Fleith, 2016). External stimulation sources have various origins. The current study identifies four types of extrinsic motivation, namely, integrated, identified, introjected, and external regulation (Ryan & Deci, 2000). Integrated motivation refers to the internalization of the stimuli to the self, through which the action becomes selfdetermined. Identified motivation is the inclination to act by recognizing the personal significance of an act and accepting the act as personal regulation. Introjected motivation is a controlling incentive to act, such as acting due to a feeling of obligation; in other words, it is about acting in order to avoid guilt or anxiety (e.g., students study because their parents expect them to). External regulation refers to the motivation or stimulus to act to satisfy external demands, or to acquire external rewards through external causality. There is a further motivation category that reflects the concept of lack of intention to act, and personal causation (Ryan & Deci, 2000). Motivation and giftedness are interrelated, and motivation is considered a fundamental catalyst for developing giftedness (Burns & Martin, 2021; Hornstra et al., 2020; Samat & Ismail, 2020; McCoach & Flake, 2018; Barabwd et al., 2017). Thus, scholarly interest has turned to investigating the motivation level of gifted students, and comparing it to that of non-gifted students, for example, the Jordanian study of Heilat et al. (2019). Another research interest is the role of motivation and its impact on gifted students’ performance and progression. For example, Barabwd et al. (2017) surveyed gifted students' perceptions of the role of motivation in developing giftedness and found that they had highly positive perceptions in this regard, and believed it played a significant role in developing giftedness. In contrast, research studies (e.g., Steenbergen-Hu et al., 2020; Rubenstein et al., 2012) argue that lack of motivation can be a predictor of
http://ijlter.org/index.php/ijlter
94
underachievement by gifted students. Burns and Martin (2021) discuss recent research into the motivation of gifted students and suggest practical strategies teachers can apply to foster giftedness development of gifted students. They also provide effective strategies to address motivational issues. This study supports these suggestions and uses a design thinking approach to foster gifted students' motivation. 2.2 Design Thinking Design thinking relates to design theory, which is a philosophy that clarifies concepts and criticizes current models and practices. However, this theory is widely known in industrial settings, and not in the education field (Simeon et al., 2020). In education, design thinking is a teaching approach or environment that can be distinguished from conventional learning approaches for acquiring knowledge, which goes beyond acquiring knowledge, to the implementation and application of knowledge in the real situation. Design thinking is not only applicable to a single discipline, but to multidisciplinary contexts that include engineering, social sciences, medicine, education, and so forth (Tsai & Wang, 2020; Wrigley et al., 2018; Henriksen et al., 2017). In education, design thinking is considered to be a cognitive process that includes creation, experimentation, feedback, and redesign of whatever the field of the subject comprises (Li et al., 2019). Design thinking is a holistic approach to acquiring and applying knowledge in real situations. It involves five stages: empathy, define, ideate, prototype, and test. Empathy involves learning more about the problem and people involved. Define is about clarifying the problem and setting goals for solving it, as expected. Ideate is about investigating, doing research, and getting ideas to achieve the goals. Prototype is about selecting the best solution and creating prototypes that can be justified. Lastly, in the test stage, the prototype is tested in the real situation and evaluated with experiments (Stith et al., 2020). Studies investigated the role of a design thinking approach for different aspects or dimensions, such as increasing creative thinking and innovation (Henriksen et al., 2017), self-efficacy (Tsai & Wang, 2020), and so forth. Simeon et al. (2020) applied an afterschool-program-based design thinking approach in one discipline (physics) and evaluated its potential to increase students’ achievement in developing concepts. Statistical results detected an increase in the level of students’ achievement, with a difference between genders in favor of male students. Educators had positive perceptions of the utility of a design thinking approach for educating gifted students. Teachers said that design thinking tasks foster the development of 21st century skills, has psychological benefits, and increases the motivation level of students (Stith et al., 2020). Teachers explained that a design thinking approach increases motivation. Firstly, students exhibit active engagement, find the approach exciting and fun, and feel that they are making a contribution to society, because they have the opportunity to develop solutions on their own. The second reason why design thinking increases motivation, according to teachers, is that the task context is authentic and transferable to other contexts. Thirdly, the problems students solve are from the real word, which students experience, feel, see, and live. In the fourth place,
http://ijlter.org/index.php/ijlter
95
students are not constrained to particular ways of finding solutions; instead, they can create their own solutions in their own ways (Stith et al., 2020). A study of design thinking in a subject relating to the human–computer interface assessed students’ motivations and found that doing design thinking tasks increased students’ motivation to do coursework and attend school. Furthermore, students showed higher levels of self-efficacy and lower test anxiety, and their ability to control, manage, and regulate their effort and time improved. The study linked this increase in motivation to the facilitation of the flow of knowledge in the course (Ahmad et al., 2017). Similar results were obtained by a study that targeted university students, in which more than 80% of students stated that they had the self-motivation to pursue success in a design thinking course. Students ascribed their perseverance to the significance of these skills for their selfdevelopment, which caused their engagement to become self-determined (Wei et al., 2020). A second study used design thinking methodology to enhance university students’ motivation and performance in their graduate projects. Results confirm the significance of a design thinking approach for fostering motivation (Bordel et al., 2019). In summary, studies found that learning with a design thinking program increased self-motivation of students, and improved teachers’ perceptions about design thinking (Wei et al., 2020; Ahmad et al., 2017). However, there is no empirical evidence for this association. Thus, the current study intended to find empirical evidence for this claim and, furthermore, determine students’ attitudes toward a design thinking program.
3. Method The study used a quasi-experimental approach with a one-group design and pre and post-tests. The study examined the effect of design thinking as a learning approach on students’ motivation in a multidisciplinary context. The study investigated afterschool activities, because the setting of the school’s conventional activities was not conducive to setting design thinking problems. The researchers played a teacher role in the study, as the usual teacher was unqualified to carry out a design thinking approach. Furthermore, for students to engage in design thinking problems, they need to have integrated and resource-rich settings. All students could access various online laboratories and online research databases. The students could also use a university chemistry library or computer facilities under the researchers’ guidance. 3.1 Study Participants and Learning Environments The study used online learning management software at the King Abdullah II School for Excellence (Grade 11 to 12 (Tawjihi1)) in Jordan. Seventy-seven students were selected randomly from the secondary grades – 45 female and 32 male students. They provided signed informed consent, and parental approval was obtained. Fifty-two students were in Grade 11, and 25 students were in Grade 12. SAT scores were obtained for all students from their student profiles at the school. The SAT is a foreign test that is taken for three main subjects: reading, writing, 1
The General Secondary Education Certificate Examination in Jordan and Palestine.
http://ijlter.org/index.php/ijlter
96
and mathematics. SAT scores were used, rather than grade point averages, to avoid biasing the results due to school administration or other factors, and to ensure homogeneity. The majority of students (N=72) obtained average SAT scores in excess of 90%; only five students obtained average scores ranged between 80.4% and 90%. Design thinking problems were posed via online learning modules that the school had adopted for learning purposes. The study opted to use the online learning modules because, at the time the study was administered, the school used online learning due to global health circumstances. 3.2 Procedure All students engaged in three weekly multidisciplinary sessions via Microsoft Teams (online platform). Each session was a one-hour class period that was scheduled after the end of the school schedule. These sessions differed from conventional school lessons and involved the authentic discipline of design thinking about a problem. Most students selected tasks from a pool of design thinking problems based on their preferences. Only the first five problems were reported by the study (see Table 1). Each task was discussed in five online sessions. The first session was an introductory session, in which the problem was either stated by a teacher or sparked by trigger questions. In the second session, students presented information that they had collected during searches relating to the problem; students presented their perceptions to their peers. In the third session, students were encouraged to reveal the solutions they suggest for solving the problem, and to explain how they planned to implement the solutions. The fourth session involved the teacher encouraging students to report on their progress and discuss any obstacles they faced, so that they could overcome the obstacles, either by suggestions by group members, or by the teacher’s suggestions and guidance. In the fifth session, each group assumed complete responsibility for presenting their solution and a report to their classmates. In conclusion, the solutions were driven and offered by participants and approved by the teacher. Each problem was solved by students in a group of three gifted students. New groups were formed between design thinking problems. This setting is consistent with a student-centered classroom and student-driven environment. Table 1. Tasks list of online design thinking problems developed during the study Online design thinking problem High rate of consumption of water for irrigation purposes
Waste management
Description
Related content
A graphic presentation of the water consumption rate by each activity of daily life, and a classification of grey and black water. Identifying the best water sample to reuse for irrigation according to Jordanian standards for irrigation water. Build a simple filter to reuse grey water sample for irrigation. Feasibility study of recent solutions for waste management mechanisms,
Chemistry, social sciences, material research, life sciences
http://ijlter.org/index.php/ijlter
Administration sciences, engineering,
97
enabling features that facilitate management, and a survey of residents’ perceptions. High rate of dependency on social media
Suggest application features or interface to reduce time spent on social media.
Lowering the unemployment rate
Study solutions and provide reports related to other countries that succeeded in lowering its unemployment rate; discuss probable and unlikely solutions. Design a presentation suggesting ways to educate communities about bullying and suggest ways to cope with bullying.
Developing awareness of and reducing bullying
life sciences, computer and information sciences, social sciences Computer and information sciences, social sciences, psychology Life sciences, social sciences, psychology, financial sciences, administrative sciences Life sciences, social sciences, psychology
3.3 Instrument A student motivation questionnaire, in English, was used for pre and postassessment. It comprises scales measuring academic motivation (four items) and motivation for school (18 items distributed over four dimensions). All items of both scales are answered on a five-point Likert scale ranging from “totally not applicable to me” to “totally applicable to me”. The academic motivation scale consists of four items developed by Vallerand et al. (1992). The motivation for school scale consists of four facets: external regulation, introjected, intrinsic, and identification motivation, and was developed by Rayan and Connell (1989). Both scales have high psychometric properties. The academic motivation scale and motivation for school scale scored above the recommended reliable score (Cronbach’s alpha >0.70) (Hornstra et al., 2020; Utvaer & Haugan, 2016). The psychometric properties of the scale used in the current study also have acceptable validity, according to the Pearson correlation coefficient recorded for the scale, in which all items of the motivation scale correlate with their dimensions, and all statements’ correlation values were in excess of 0.31, which is the lowest acceptable value (Core et al., 2021). The study used SPSS software to analyze participants’ responses, and embedded descriptive and inferential statistics to validate study objectives. Table 2. Pearson correlation test to show construct validity Amotivation # Correlation 1 .802** 2 .860** 3 .824** 4 .863**
External
Introjection
Identification
Intrinsic
#
Correlation
#
Correlation
#
Correlation
#
Correlation
1 2 3 4 5
.379** .565** .400** .494** .323**
1 2 3 4 5 6
.470** .539** .326** .373** .225* .355**
1 2 3 4 5
.353** .528** .583** .409** .509**
1 2
.854** .770**
http://ijlter.org/index.php/ijlter
98
Furthermore, the reliability of the scale is also acceptable, as it recorded a Cronbach’s alpha higher than the threshold value (>0.70). See Table 3. Table 3. Cronbach Alpha Coefficient for motivation scale Dimension Amotivation External Introjection Identification Intrinsic Motivation (Total)
Cronbach’s alpha 0.856 0.776 0.812 0.760 0.704 0.873
The second measure the study used is the attitude scale, which was implemented after the completion of the design thinking classes, to assess students’ attitudes about the design thinking problem approach. The scale has two dimensions, namely, satisfaction with program (four items, responses evaluated using fivepoint Likert scale, from “strongly disagree” to “strongly agree”) and behavioral engagement in the session (five items with a five-point Likert scale ranging from “never” to “always”) developed by Nie and Lau (2009). The scale achieved a good reliability score, since Cronbach’s alpha was higher than the accepted threshold (>0.70). See Table 4. Table 4. Cronbach’s alpha coefficient for attitude scale Dimension Satisfaction Behavior Attitude (Total)
Cronbach alpha 0.775 0.741 0.830
3.4 Normality and Homogeneity Test Before starting to test the hypotheses, the researcher made sure that the data followed the normal distribution, by applying a set of special tests. See Table 5. Table 5. Normal distribution and homogeneity tests for the study data Kolmogorov-Smirnov Variables Amotivation External Introjection Identification Intrinsic Satisfaction Behavior
Value 0.957 1.567 0.874 0.698 1.002 1.068 0.888
Sig. 0.562 0.264 0.587` 0.541 0.121 0.103 0.354
Kurtosis
Skewness
0.564 0.318 0.584 0.461 0.657 0.746 0.555
-0.541 -0.214 -0.454 -0.605 -0.241 -0.326 -0.567
Tolerance 0.685 0.432 0.384 0.491 0.426 0.654 0.441
VIF 2.001 2.684 2.854 1.540 2.366 1.451 2.214
Note: Sig. = 0.05; VIF = 10 It is clear from Table 5 that the statistical significance values on the KolmogorovSmirnov test are in excess of α ≤ 0.05, and that all the values for skewness are below -1. Kurtosis is less than 7, the tolerance values are greater than 0.05, while the VIF values are below 10. These tests confirm that all values approximate a normal distribution and, thus, allow the use of parametric methods.
http://ijlter.org/index.php/ijlter
99
4. Results 4.1 Group Heterogeneity To validate the heterogeneity of the participants of the study, inferential statistics, including a T-test, was used, and heterogeneity was validated. According to Table 6, an independent sample T-test for gender, grade, and SAT score found that motivation and related facets (amotivation, external, introjection, identification, intrinsic) were not significant at the pre-test; t-values for gender were 1.448, 0.502, -1.110, -0.002, 0.242, 0.845, for grade were 0.436, 0.049, -1.449, 1.036, 0.768, 0.534, and for SAT score were 0.376, -1.054, -0.221, -1.095, -0.861, -0.860, respectively). None of these values are significant at the 0.05 level, which indicates the equivalence of all members of the study sample at the time of the pre-test. Table 6. Independent sample T-test to show the variance between motivation and its dimensions according to gender, grade, and SAT score at the pre-test Motivation facets Gender Male Amotivation Female Male External Female Male Introjection Female Male Identification Female Male Intrinsic Female Male Motivation (total) Female Grade 11 Amotivation 12 11 External 12 11 Introjection 12 11 Identification 12 11 Intrinsic 12 11 Motivation (total) 12 SAT score > 90 Amotivation > 80.2 > 90 External > 80.2 > 90 Introjection > 80.2 > 90 Identification > 80.2
N 32 45 32 45 32 45 32 45 32 45 32 45 N 52 25 52 25 52 25 52 25 52 25 52 25
Mean 2.56 2.27 3.61 3.56 2.74 2.84 3.17 3.17 2.11 2.08 2.95 2.91 Mean 2.42 2.33 3.58 3.58 2.76 2.89 3.12 3.16 2.13 2.02 2.93 2.90
Std. deviation 0.95 0.80 0.32 0.44 0.36 0.36 0.28 0.35 0.64 0.50 0.21 0.21 Std. deviation 0.88 0.88 0.41 0.38 0.38 0.31 0.32 0.31 0.55 0.59 0.21 0.22
N
Mean
72 5 72 5 72 5 72 5
2.40 2.25 3.57 3.76 2.80 2.83 3.16 3.32
http://ijlter.org/index.php/ijlter
(t) 1.448
P value .152
.502
.617
-1.110-
.271
-.002-
.999
.242
.809
.845
.401
(t) .436
P value .664
.049
.961
-1.499-
.138
1.036
.245
.768
.445
.534
.595
Std. deviation
(t)
P value
0.86 1.15 0.40 0.30 0.36 0.39 0.32 0.33
.376
.708
-1.054-
.295
-.221-
.826
-1.095-
.277
100
Intrinsic Motivation (total)
> 90 > 80.2 > 90 > 80.2
72 5 72 5
2.08 2.30 2.92 3.00
0.55 0.67 0.21 0.16
-.861-
.392
-.860-
.393
4.2 Descriptive Statistics According to the descriptive analysis results, the students had a higher average mean (M=4.04; very high motivation level) on the motivation scale in the post-test than in the pre-test (M=2.92; medium motivation level). In the pre-test, students’ scores indicated a medium level of motivation: external regulation and identification motivation levels were high and other motivation categories were medium. The statistical data of the post-test measurement show a tangible difference from the pre-test results for almost all the motivation categories. Table 7. Mean and standard deviation for (pre and post) motivation and its dimensions Dimensions Honestly, I don’t know; I really feel that I am wasting my time in school I once had good reasons for going to school; however, now I wonder whether I should continue I can’t see why I go to school and frankly, I couldn’t care less I don’t know; I can’t understand what I am doing in school Amotivation Because I'll get in trouble if I don't Because that's what I'm supposed to do So that the teacher won't yell at me Because that's the rule So others won't get mad at me External regulation Because I want the teacher to think I'm a good student Because I will feel bad about myself if I don't Because I'll feel ashamed of myself if I don't Because I want the other students to think I'm smart Because it bothers me when I don't Because I want people to like me Introjection Because I want to understand the subject Because 1 want to learn new things To find out if I'm right or wrong Because I think it is important to work on my schoolwork Because I wouldn't want (like) to do that (negative behavior) Identification I1: Because it's fun I2: Because I enjoy it
http://ijlter.org/index.php/ijlter
Pre-test Post-test Std. Std. Mean Mean Deviation Deviation 2.34
1.07
4.03
0.73
2.49
1.13
4.03
0.69
2.51
0.93
3.83
0.64
2.23
1.04
4.10
0.66
2.39 3.66 3.69 3.40 3.45 3.69 3.58
0.87 1.02 0.92 0.63 1.08 0.89 0.40
4.00 4.25 4.05 3.75 4.13 3.79 3.99
0.38 0.65 0.76 0.67 0.66 1.06 0.42
2.70
1.06
4.29
0.48
2.70 3.23 2.74 2.01 3.40 2.80 3.55 3.53 3.45
1.08 0.71 1.04 0.92 0.71 0.36 0.64 0.66 0.70
4.27 4.38 4.13 3.68 4.53 4.22 4.62 4.62 4.57
0.62 0.49 0.78 0.66 0.50 0.25 0.49 0.49 0.50
3.31
0.80
4.52
0.50
2.00
0.74
1.42
0.64
3.17 2.14 2.04
0.32 0.76 0.62
3.95 3.90 3.95
0.26 0.77 0.67
101
Intrinsic Motivation (total)
2.09 2.92
0.56 0.21
3.92 4.04
0.60 0.16
4.3 Difference in Motivation Between Pre and Post-tests The study examined if there was a difference between the motivation, measured by pre and post-test scores, of gifted students. The descriptive data of the pre and post-tests indicate a clear difference. Foremost, statistically significant differences have to be tested in order to validate differences statistically. Therefore, a paired sample T-test was applied to determine the statistically significant differences between the pre and post-test on motivation (see Table 8). Table 8. Difference between (pre and post) motivation levels (paired sample T-test) (N=77) Mean Amotivation pre Amotivation post External pre External post Introjection pre Introjection post Identification pre Identification post Intrinsic pre Intrinsic post Motivation pre total Motivation post total
2.39 4.00 3.58 3.99 2.80 4.22 3.17 3.95 2.09 3.92 2.92 4.04
Std. deviation 0.87 0.38 0.40 0.42 0.36 0.25 0.32 0.26 0.56 0.60 0.21 0.16
Paired differences
(t)
P value
-1.60
-13.374-
0.00*
-0.42
-5.845-
0.00*
-1.42
-29.693-
0.00*
-0.78
-28.996-
0.00*
-1.83
-19.009-
0.00*
-1.12
-38.000-
0.00*
The results of the paired sample T-test in Table 8 indicate that there are statistically significant differences in the results of the pre and post-tests for motivation and its related dimensions (amotivation, external, introjection, identification, intrinsic). The paired differences for these dimensions and total degree were -1.60, -0.42, -1.42, -0.78, -1.73, -1.12 respectively, and, with the significance level less than (>0.01), the differences were in favor of the post-test, which illustrates the high level of the mean values in that measurement. 4.4 Attributing Differences in Motivation to Demographic Differences The study assessed whether gender, grade, or SAT score had an effect on the motivation level of students after they had attended a design thinking class. The study applied an independent sample T-test; the results are given in Table 9. Table 9. Independent sample T-test to determine variance between motivation and its dimensions according to gender, grade, and SAT score at the post-test
Amotivation External Introjection
Gender
N
Mean
Male Female Male Female Male Female
32 45 32 45 32 45
3.97 4.02 3.99 4.00 4.25 4.20
http://ijlter.org/index.php/ijlter
Std. deviation 0.37 0.39 0.39 0.45 0.26 0.25
(t)
P value
-.547-
.586
-.128-
.898
.841
.403
102
Identification Intrinsic Motivation (total)
Amotivation External Introjection Identification Intrinsic Motivation (total)
Amotivation External Introjection Identification Intrinsic Motivation (total)
Male Female Male Female Male Female
32 45 32 45 32 45
3.94 3.96 3.98 3.88 4.04 4.04
Grade
N
Mean
11 12 11 12 11 12 11 12 11 12 11 12
52 25 52 25 52 25 52 25 52 25 52 25
3.97 4.05 3.98 4.02 4.21 4.23 3.96 3.94 3.92 3.92 4.03 4.05
Sat score
N
Mean
> 90 > 80.2 > 90 > 80.2 > 90 > 80.2 > 90 > 80.2 > 90 > 80.2 > 90 > 80.2
72 5 72 5 72 5 72 5 72 5 72 5
3.98 4.25 4.01 3.84 4.21 4.37 3.96 3.88 3.94 3.70 4.04 4.05
0.26 0.26 0.55 0.63 0.17 0.16 Std. deviation 0.41 0.30 0.43 0.40 0.23 0.31 0.25 0.28 0.60 0.61 0.17 0.16 Std. deviation 0.38 0.25 0.43 0.26 0.25 0.32 0.26 0.18 0.60 0.57 0.17 0.04
-.378-
.706
.771
.443
.101
.920
(t)
P value
-.857-
.394
-.421-
.675
-.288-
.774
.346
.730
.021
.983
-.512-
.610
(t)
P value
-1.567-
.121
.852
.397
1.303
.243
-1.068-
.341
.860
.393
-.220-
.826
The result of the independent sample T-test for the association of motivation with gender, grade, and SAT score indicates that motivation and its dimensions (amotivation, external, introjection, identification and intrinsic with total score) were not significant at post-test (t) values: -0.547, -0.128, 0.841, -0.378, 0.771, 0.101, -0.857, -0.421, -0.288, 0.346, 0.021, -0.512, -1.567, 0.852, 1.303, -1.068, 0.860, -0.220 respectively. None of these values are significant at the level of 0.05, which indicates that there were no statistically significant differences in motivation in relation to gender, grade and SAT score. 4.5 Gifted Students’ Attitudes to the Design Thinking Problem Class To determine gifted students’ perceptions of and attitudes toward the design thinking approach, all participants completed an attitude questionnaire to rate their satisfaction and engagement behavior. The descriptive statistics of their responses is presented in Table 10.
http://ijlter.org/index.php/ijlter
103
Table 10. Attitudes of gifted students towards design thinking (N=77) Attitudes I am glad to be in this program I think it is nice to participate and study in this program If I could, I would rather back to conventional study settings If I had to move to another settings, I would still want to study through such setting Satisfaction I pay attention well I keep my attention on the work during the entire lesson I listen carefully when the teacher explains something I try my best to complete class work I try my best to answer the teacher’s questions Engagement behavior Attitude (total)
Mean 4.26 3.94
Std. deviation 0.73 0.73
2.49
0.60
4.43
0.57
3.78 4.30
0.32 0.74
4.42
0.68
4.27 4.44 4.42 4.37 4.11
0.74 0.68 0.69 0.27 0.21
The descriptive statistics indicate that students’ satisfaction was high, with a total mean of 3.78 and standard deviation of 0.57. The statement with the highest score (If I had to move to another settings, I would still want to study through such setting) has a mean of 4.43 and standard deviation of 0.57, which is high. The statement which stipulated (If I could, I would rather back to conventional study settings. The result for engagement behavior is at a high level, with a mean of 4.37 and a standard deviation of 0.21. The statement, I try my best to complete class work, ranked first, with a mean of 4.44 and a standard deviation of 0.6; it is at a high level. In contrast, the statement, I listen carefully when the teacher explains something, ranked last, with a mean of 4.27 and a standard deviation of 0.74 – which is still at a high level. Total attitude obtained a mean of 4.11, with a standard deviation of 0.21, which is at a high level.
5. Discussion Descriptive statistics indicate that gifted students attend school because they are forced to do so by external regulations: they wish to avoid getting into trouble, or to avoid punishment, and they obey school regulations. Other factors that stimulated students to attend school were self-identified stimuli, such as the desire to acquire knowledge, and recognizing that attending would help them meet their needs in the future Not surprisingly, gifted students who participated in the study exhibited a medium level of association between academic motivation and their level of external regulation motivation. This indicates that, if students are not obligated to attend school, they will not do so. Their unwillingness may be attributed to the absence of fun and enjoyable activities (thus, an absence of high intrinsic motivation). This finding confirms the study by Heilat et al. (2019), which found that gifted students in Jordan have the same level of motivation as non-gifted students, which hinders gifted students’ giftedness development. The statistical inferential tests validated the significant differences found for all motivation categories levels, in favor of the post-test measurement. It is clear that
http://ijlter.org/index.php/ijlter
104
the online design thinking problems played a significant role in fostering the motivation levels of gifted students. This confirms the research results of Stith et al. (2020), Wei et al. (2020), Bordel et al. (2019) and Ahmad et al. (2017). Gifted students found the design thinking class fun and enjoyable. They were stimulated by attending the class and the learning they underwent, and their participation was self-determined (see Table 5), probably because the design thinking class gave students opportunities to develop solutions on their own (Stith, et al., 2020). These opportunities were turned into self-determined participation and fostered a feeling of belonging in relation to the subject, since students dealt with real-life situations – some students lived with these problems (Wei et al., 2020). Furthermore, introjected motivation increased through the design thinking class – students became more interested in external appreciation and pursuing a good image in the eyes of their peers or teachers (see Table 5). This finding resonates in their high satisfaction with the design thinking approach, and their engagement behavior. Their responses indicate that they shared an interest in and were committed to solving the design thinking problems they discussed in class (see Table 10). Moreover, the design thinking problems presented to students in this study were appropriate for all gifted students (female and male, Grades 11 and 12, and any SAT score band). No differences were found between motivation levels and gender, class, or SAT score categories of gifted students. This was expected, since the participants chose their preferred problem from a pool of suggested real-life problems.
6. Conclusion Motivation plays a significant role in harnessing students’ abilities and competencies, even if students are naturally or potentially gifted. The study used design thinking pedagogy as an innovative approach to motivating students to learn and attend school. Design thinking is a holistic approach to acquiring knowledge and applying this knowledge in real situations through five stages: empathy, define, ideate, prototype, and test. The study was a quantitative quasiexperimental study with a one-group design and pre and post-tests. The study evaluated intrinsic and four extrinsic motivation categories, namely integrated, identified, introjected, and external regulation, to evaluate gifted students’ motivation to do coursework. A motivation scale was used to gauge their motivation to attend school, and an attitude questionnaire explored students’ satisfaction and engagement with the design thinking class. The experimental group consisted of 77 randomly selected gifted students at the King Abdullah II School for Excellence. After engaging in the design thinking class, the students achieved higher scores than in the pre-test for all motivation categories. Furthermore, the students did not exhibit any differences in their post-test motivation levels in relation to gender, grade, and SAT scores. Thus, the design thinking approach is a promising approach for motivating gifted students; the students found it satisfying and high engagement behavior was recorded. The study design and results justify pursuing implementing a design thinking approach to increase gifted students’ motivation for different subjects and at different grade levels. Furthermore, the study did not attempt to find differences
http://ijlter.org/index.php/ijlter
105
between gifted and non-gifted students, since gifted students who were enrolled in a school for excellence were engaged in various authentic learning environments, which hindered attempts to distinguish the effect of design thinking on motivation between two groups, as the two groups were engaged in different activities. Thus, the study was limited to one group pre and post-test, and future studies are encouraged to involve two groups, to validate the current results. The researchers recommend involving both gifted and non-gifted students in future studies that use the design thinking approach, which would enable comparisons between the two groups of students. Furthermore, longitudinal studies are recommended, to track the giftedness development of gifted students using a design thinking approach to learning.
7. References Ahmad, W. F., Sarlan, A., Rokhmah, S., & Ahmad, R. (2017). Integrating design thinking approach to increase student’s motivation. 7th World Engineering Education Forum (WEEF) (pp. 268-272). IEEE. https://doi.org/10.1109/weef.2017.8467148 Barabwd, H., Nor, M., & Ishak, N. (2017). Gifted students' motivation and its impact on the development of their giftedness. Journal of Education in Black Sea Region, 3(1). https://doi.org/10.31578/jebs.v3i1.118 Bordel, B., Alcarria, R., Manso, M., & Robles, T. (2019). Enhancing students' motivation and academic results in the context of final projects through design-thinking methodologies. 23th International Technology, Education and Development Conference. INTED2019 Proceedings (pp. 8235–8239). IATED. Burns, E., & Martin, A. (2021). Motivational issues in gifted education: understanding the role of students’ attribution and control beliefs, self-worth protection and growth orientation. In S. R. Smith (Ed.), Handbook of giftedness and talent development in the Asia-Pacific (pp. 339–357). Springer. https://doi.org/10.1007/978-981-13-30414_15 Core, R., Waring, M., Hedges, L., & Ashley, L. (2021). Research methods and methodologies in education (3rd ed.). SAGA. Fleith, D. de S. (2016). Creativity, motivation to learn, family environment, and giftedness: A comparative study. Psicologia: Teoria e Pesquisa, 32, 1–9. https://doi.org/10.1590/0102-3772e32ne211 Hebert, T. (2020). Understanding the social and emotional lives of gifted students. Sourcebooks. Heilat, M. Q., Seifert, T., & Qian, M. (2019). Mental motivation, intrinsic motivation and their relationship with emotional support sources among gifted and non-gifted Jordanian adolescents. Cogent Psychology, 6(1), 1–13. https://doi.org/10.1080/23311908.2019.1587131 Henriksen, D., Richardson, C., & Mehta, R. (2017). Design thinking: A creative approach to educational problems of practice. Thinking Skills and Creativity, 26, 140–153. https://doi.org/10.1016/j.tsc.2017.10.001 Hornstra, L., Bakx, A., Mathijssen, S., & Denissene, J. A. (2020). Motivating gifted and nongifted students in regular primary schools: A self-determination perspective. Learning and Individual Differences, 80, 1–13. https://doi.org/10.1016/j.lindif.2020.101871 Li, Y., Schoenfeld, A. H., diSessa, A. A., Graesser, A. C., Benson, L. C., English, L. D., & Duschl, R. A. (2019). Design and design thinking in STEM education. Journal for STEM Education Research, 2, 93–104. https://doi.org/10.1007/s41979-019-00020-z
http://ijlter.org/index.php/ijlter
106
McCoach, D. B., & Flake, J. K. (2018). The role of motivation. In S. I. Pfeiffer, E. ShaunessyDedrick, & M. Foley-Nicpon (Eds.), APA handbook of giftedness and talent (pp. 201213). American Psychological Association. https://doi.org/10.1037/0000038-013 Nie, Y., & Lau, S. (2009). Complementary roles of care and behavioral control in classroom management: The self-determination theory perspective. Contemporary Educational Psychology, 34(3), 185–194. https://doi.org/10.1016/j.cedpsych.2009.03.001 Rayan, R. M., & Connell, J. P. (1989). Percevied locus of causality and internalization: Examining reasons for acting in two domains. Journal of Personality and Scial Psychology, 57(5), 749-761. http://doi.org/10.1037/0022-3514.57.5.749 Rubenstein, L., Siegle, D., Reis, S., Mccoach, D. B., & Burton, M. (2012). A Complex quest: The development and research of underachievement interventions for gifted students. Psychology in the Schools, 49(7), 678-694. https://doi.org/10.1002/pits.21620 Ryan, R., & Deci, E. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67. https://doi.org/10.1006/ceps.1999.1020 Sakarneh, M. A., & Al-Swelmyeen, M. B. (2020). The extent to which the Jordanian inclusive basic school teachers use the constructivism theory in teaching. Journal of Educational and Social Research, 10(1), 182–197. https://doi.org/10.36941/jesr2020-0017 Samat, F., & Ismail, E. S. (2020). Meeting the needs of Malaysian mathematically talented students by an enrichment program: A module for primary school students. Madrosatuna: Journal of IslamicElementary School, 4(1), 23–25. https://doi.org/10.21070/madrosatuna.v4i1.55 Siegle, D., & McCoach, B. (2018). Underachievement and the gifted child. In S. Pfeiffer, E. Shaunessy-Dedrick, & M. Foley-Nicpon, APA handbook of giftedness and talent (pp. 445–473). American Psychological Association. Simeon, M. I., Samsudin, M. A., & Yakob, N. (2020). Effect of design thinking approach on students’ achievement in some selected physics concepts in the context of STEM learning. International Journal of Technology and Design Education. https://doi.org/10.1007/s10798-020-09601-1 Steenbergen-Hu, S., Olszewski-Kubilius, P., & Calvert, E. (2020). The effectiveness of current interventions to reverse the underachievement of gifted students: Findings of a meta-analysis and systematic review. Gifted Child Quarterly, 64(2), 132–165. https://doi.org/10.1177%2F0016986220908601 Stith, K. M., Potts, M. L., Rubenstein, L. D., Shively, K. L., & Spoon, R. (2020). Perceptions of K-12 teachers on the cognitive, affective, and conative functionalities of gifted students engaged in design thinking. Journal of STEM Teacher Education, 55(1), 41– 57. https://doi.org/10.30707/J Tsai, M.-J., & Wang, C.-Y. (2020). Assessing young students’ design thinking disposition and its relationship with computer programming self-efficacy. Journal of Educational Computing Research, 59(3), 410-428. https://doi.org/10.1177/0735633120967326 Utvaer, B. K., & Haugan, G. (2016). The academic motivation scale: Dimensionality, reliability, and construct validity among vocational students. Nordic Journal of Vocational Education and Training, 6(2), 17-45. https://doi.org/10.3384/njvet.2242458x.166217 Vallerand, R., Pelletier, L., Blais, M., Briere, N., Senecal, C., & Vallieres, E. (1992). The academic motivation scale: A measure of intrinsic, extrinsic, and amotivation in
http://ijlter.org/index.php/ijlter
107
education. Educational and Psychological Measurement, 52, 1003–1017. https://doi.org/10.1177/0013164492052004025 Wei, L. C., Shamsuddin, W. N., Rozee, I. M., & Vesudevan, M. (2020). Degree students' self-motivation toward design thinking project. International Journal of Education and Pedagogy, 2(4), 185-192. http://myjms.mohe.gov.my/index.php/ijeap/article/view/11606 Wrigley, C., Mosely, G., & Tomitsch, M. (2018). Design thinking education: A comparison of massive open online courses. She Ji: The Journal of Design, Economics, and Innovation, 4(3), 275–292. https://doi.org/10.1016/j.sheji.2018.06.002
http://ijlter.org/index.php/ijlter
108
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 108-125, August 2021 https://doi.org/10.26803/ijlter.20.8.7 Received Jun 09, 2021; Revised Aug 20, 2021; Accepted Aug 29, 2021
Teaching-Learning Strategies to Production Planning and Control Concepts: Application of Scenarios to Sequencing Production with Virtual Reality Support Fernando Elemar Vicente dos Anjos Instituto Federal de Educação, Ciência e Tecnologia do Rio Grande do Sul (IFRS), Campus Caxias do Sul. Caxias do Sul, Brazil; and UNISINOS –University of Vale do Rio dos Sinos https://orcid.org/0000-0002-9096-2806 Luiz Alberto Oliveira Rocha UNISINOS – University of Vale do Rio dos Sinos, São Leopoldo, Brazil https://orcid.org/0000-0003-2409-3152 Rodrigo Pacheco Instituto Federal de Educação, Ciência e Tecnologia do Rio Grande do Sul (IFRS), Campus Caxias do Sul, Caxias do Sul, Brazil https://orcid.org/0000-0002-9399-4475 Débora Oliveira da Silva UNISINOS – University of Vale do Rio dos Sinos, São Leopoldo, Brazil https://orcid.org/0000-0001-7023-4927
Abstract. This paper aims to present scenarios to be applied in higher education to the theme of production planning and control, addressing factors of the production system and indicators arising from this process and the application of virtual reality to support the process. The applied method combines the development of six scenarios for virtual reality application and the discussion about the impacts in indicators from the production planning and control, for example, inventory in the process, manufacturing lead-time, use of equipment, and punctual delivery attendance. Findings revealed that the teaching-learning process of production planning and control, when applied through scenarios, generates opportunities for students to learn the impact in the indicators. The virtual reality in this environment supports creating differentiated teaching-learning environments to generate the most significant knowledge for students which positively impacts the future in the world of work. In addition, it allows people involved in the teaching-learning processes of production engineering to apply the ©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
109
concepts presented in the sequencing process, lean about the impacts of decisions on production sequencing indicators and appreciate the support of virtual reality to generate an environment more cognitive for students. Keywords: teaching-learning in production engineering; engineering dducation; planning and production control; virtual reality in teachinglearning process; production sequencing
1. Introduction When teaching-learning strategies are approached, several paths can be traced. Mizukami (1992) presented five teaching approaches: traditional, behavioural, humanistic, cognitive, and socio-cultural. Santos (2006) highlighted that cognitivism proposes that knowledge is generated through experiments in the world, analyzing aspects through intervention in the processes, rescuing the discussion on theoretical studies of cognitive psychology presented by Piaget and Vigotsky. They explained the theory of learning called constructivism. In 2019, the ASEE Annual Conference & Exposition (promoted by the American Society for Engineering Education) took place, in which a conference proceeding was published that presented a discussion of cognitive teaching approaches to Engineering. In this discussion, Crawley et al. (2019) described that the new MIT program, called New Engineering Education Transformation (NEET), brings information that alumni are better prepared to work as innovators, creators, entrepreneurs, and future leaders, when knowledge is developed through cognitive approaches, for the formation of critical, systemic and humanistic thinking. They also supported professionals to learn and think more effectively on their initiative throughout their lives. The creation of environments that seek to prioritize cognition can be organized and explored in different ways. Prensky (2001) highlighted the importance of applying technologies in the teaching-learning process, and Wang et al. (2017) pointed out in their research that the application of technologies can positively impact students' learning. Some researches present the application of mobile technology, virtual reality, and augmented reality as part of the teachinglearning strategies in cognitive environments. Examples of such uses are the investigation of crime scenes (Mayne & Green, 2020), the teaching of chemistry (Frevert & Di Fuccia, 2019), functional spectroscopy (Lamb et al., 2018), and medical surgery (Żechowicz et al., 2018), among others. Positive cases of the application of virtual reality and augmented reality in the learning and satisfaction of students can be perceived in research in several areas of engineering, such as, , data structuring (Akbulut et al., 2018), machining manufacturing processes, and robotics (Grodotzki et al., 2018), along with construction of buildings and construction environments and equipment (Sánchez et al., 2015; Shirazi & Behzadan, 2015). Anjos et al. (2020) described in their research that there are some applications of virtual reality in the teaching-learning processes of production engineering, and
http://ijlter.org/index.php/ijlter
110
there are many other opportunities for the application of virtual reality that measure the results in the teaching-learning process or also measure the satisfaction of students when using such a method. Thus, they indicated future research in production systems management, production planning, control, material handling, production simulation, production process management, metrological quality organization, work organization, accident risk analysis and prevention, work safety, process, and product ergonomics. Through the perception of the importance of teaching-learning environments with cognitive bias in the training of students and impacts on their professional career and the application of technologies in the formation of the cognitive environment and the opportunities for new research with the application of virtual reality highlighted by Anjos et al. (2020), the following research questions arise: 1- What concepts of production planning and control could be developed in virtual reality? 2- Which theoretical bases applied to the virtual reality model should be taken into account? 3- What are the model's requirements, and why would it strengthen the studied concepts? The general objective of this study is to develop a virtual reality application model with different scenarios, to be used as a teaching-learning strategy for production planning and control concepts. This research is justified because applying virtual reality models, in some cases, increases the level of knowledge retention of the subjects studied, such as from 25% to 80% after three weeks of studies carried out with virtual reality, when compared to students who used only the traditional teaching method (Laseinde et al., 2016). Students learn more using virtual reality than students who only took classes with the traditional teaching method (Inayat et al., 2016) and were more satisfied and engaged with the subjects studied (Fonseca et al., 2016). The more increase in knowledge retention, the highest level of learning, satisfaction, and engagement positively impact organizations because professionals entering the job market arrive entirely with more acquired knowledge, supporting organizations and making them more competitive in the market than its competitors (Anjos et al., 2020).This article is organized in the following sections: the theoretical framework, methods, results, analysis of results, and conclusion, along with some limitations and future research suggestions.
2. Theoretical Background 2.1. Cognitive Environments According to Lefrançois (2016), expectations that drive behaviour are formed by cognitions, and are developed after experiences with styles and rewards. This strategy is a way for teachers to stimulate students through their expectations and learning objectives formed in a cognitive environment. It is also highlighted that an essential part of the cognitive environment is related to the perception (generated by significant experiences) of the formation of concepts, memories,
http://ijlter.org/index.php/ijlter
111
languages, thinking, problem-solving, and decision making. In the same line of thought, Kanakana-Katumba and Maladzhi (2019) described that the cognitive teaching-learning environment must take into account some characteristics, such as (I) interrogative approach, (II) experimental learning, (III) problem-based learning, (IV) case-based learning, (V) project-based learning, (VI) researchbased learning, and (VII) competency-based learning. According to Mestrinho and Cavadas (2018), the interrogative approach is innovative in higher education environments and becomes a good practice for introducing the collaborative approach and bringing energy to the teachinglearning environment. Bates (2015) claimed that the experimental approach is widely used in engineering, allowing students to practice the concepts and theories developed. The problem-based methodology has the characteristic of organizing students into groups to deal with previously defined problems as well as presenting solutions through studied concepts (Bates, 2015; Tsai et al., 2015). Case-based learning uses extensive discussion in the teaching-learning processes by discussing cases and examples of possible solutions (Bates, 2015). Projectbased learning is similar to case-based learning. However, according to Bates (2015) and Mestrinho and Cavadas (2018), the scope of the work is broader and presents real challenges and tasks that often generate manual work and not theorists only. Research-based learning is similar to project-based learning, yet the instructors control the situation in the learning environment (Hwang et al., 2015; Soudien, 2010). Finally, competency learning allows students depending on their level of knowledge to learn more quickly. This approach allows one to demonstrate competence and permits students to control their learning (Bates, 2015; Kreamer et al., 2015). Quadir et al. (2019) highlighted a positive existence of cognitive learning environments if interactive activities with feedback and multimedia components are used compared to subjective and objective learning of students. 2.2. Virtual Reality and Cognitive Environments Virtual reality can create artificial environments that can be used for different purposes. According to Lamb et al. (2018), virtual reality environments happen through the interaction between man and machine, as there is a simulation of a real environment that can create interaction and communication between them. Martins and Guimarães (2012) described that these environments use multisensory technology that uses elements of computer graphics to create virtual environments. They also highlighted that these environments have characteristics related to immersion, interaction, and involvement, which work according to the relationship between the user and the virtual environment. Such characteristics are shown in Table 1.
http://ijlter.org/index.php/ijlter
112
Table 1. Basic characteristics related to the virtual world Immersion
Interaction
Involvement
No immersionwhenthecomputationallydevelopedenvironmentisviewedthr ough a desktop screenorprojectedvisualization. Withimmersion, whentheuserfeelstotallyinside a computationalenvironment, usuallyusingvisualizationglassesorCAVEs (Cave Automatic Virtual Environment). No interactionwhentheuser does notinteractwiththe virtual environment. Withinteraction, whenconnectedtothecomputer'sabilitytodetectuser input andinstantlymodifythe virtual world and its actions. It isrelatedtotheconditionofthelevelofinvolvementof a userwith a givenaction, whichcanbe passive, such as, receiving training, viewingthe virtual environment, oractive, such as reading a book, participating in virtual surgery. Adapted from Martins and Guimarães (2012).
Schlemmer and Backes (2015) emphasized that virtual reality applications bring a very realistic environment involving the participants. When applied in the teaching-learning processes, this possibility allows students to enjoy presence, immersion, interaction, and involvement, which are all combined in enhancing the learning results. In the same vein, Duncan et al. (2012) described all the benefits of the virtual environment in the teaching-learning processes, merely in the option of formative laboratory, collaborative work, socialization, and entertainment. Gilbert (2004) explained that science subjects can be abstract, and therefore, for deep learning, the virtual world is a valuable tool. Smutny et al. (2019) highlighted that, in higher education, the application of virtual reality in curricula positively impacts student engagement and motivation to learn. 2.3 Virtual Reality in the Teaching-Learning Processes of Production Planning and Control Production engineering is an extensive area. In some regions of the world, it is even called industrial engineering or management engineering. In Brazil, the guidelines for this area are organized by the Brazilian Association of Production Engineering - ABEPRO. The ABEPRO (2008) described the curricular guidelines to be applied in production engineering, organizing them in: I) Production Management; II) Quality Management; III) Economic Management; IV) Ergonomics and Workplace Safety; V) Product Management; VI) Operational Research; VII) Strategic and Organizational Management; VIII) Organizational Knowledge Management; IX) Environmental Management; and X) Education in Production Engineering. As a sub-topic of item I, the theme of production planning and control is in the same document. This theme is fundamental for organizations because it is responsible for processing all sales information and future demands and generating information on production needs for the productive, supply, logistics, and purchasing departments. (Lage Junior, 2019). The production planning and control process has another extremely relevant function. Through the information generated for all subsystems of the organization, deliveries are generated to customers within the agreed terms at a
http://ijlter.org/index.php/ijlter
113
reasonable cost (Corrêa et al., 2019). The production scheduling technique impacts the use of equipment idleness and delays in scheduled deliveries (Habib et al., 2015). Some factors need to be considered to execute production sequencing, namely rules and guidelines of the productive system and performance indicators (Corrêa et al., 2019). Among the rules and guidelines for sequencing production, some researchers highlight relevant factors in production planning and control, such as, fixed production batches or dynamic production batches (Brahimi et al., 2017; Suzanne et al., 2020), organization of production systems for discrete, continuous or project production (Armbruster et al., 2012; La Marca et al., 2010), equipment failure rate (Göttlich & Knapp, 2019), setup times and manufacturing lead time (Allahverdi & Soroush, 2008), processing time and Take time (Ayough et al., 2020), production capacity available for the execution of the planned production (Babaei et al., 2014; Oliveira & Costa, 2018), organization of the layout, according to the processing and material flow scripts (Caicedo et al., 2019). When evaluating the performance indicators of a production system, the results of decisions of the production sequencing impact in the indicators, for example, delivery attendant, delay of order, production advance, production lead time, number of overdue orders, inventory in the process, and use of equipment (Lustosa et al., 2008).
3. Method The method applied to this research is organized in six stages as described in Figure 1.
Figure 1. Method applied in the research
As displayed, steps 1, 2, and 3 of the method are described in the introduction sections and the theoretical background on the developed theme. Stage 4 of the method was segregated into two stages: defining the model's characteristics and which software to use. The virtual reality model used has a combination of factors that influence decisions about production sequencing, which will be organized in scenario formats to collect different results and how each factor impacts the results from
http://ijlter.org/index.php/ijlter
114
the production sequencing process. Each scenario was a mix of these guidelines: the size of production lots, equipment failure rate (MTFB and MTTR), setup times, processing times, production capacity, and demand. The scripts for the processing of materials and the layout did not vary between scenarios. The transfer batches will be identical to the production batches of scenarios 1 to 4. The remaining scenarios will be informed in table 7 and 8 in the results section. The evaluated indicators are: inventory in the process, punctual delivery, manufacturing lead-time, and use of equipment. The layout is organized in line. It consisted of six material processing equipment, six assembly departments, and a shipping department. With this equipment, eight different components will be processed. The processing equipment, assembly departments, and the materials that are processed are shown in Table 2. Table 2. Equipment, assembly departments and parts processed in the scenarios elaborated MachinesandEquipment Eq1 Eq2 Eq3 Eq4 Eq5 Eq6 ---
Assembly Department MTG1 MTG2 MTG3 MTG4 MTG5 MTG6 ---
Processedparts Xa Xb Xc Xd Xe Xf Xg Xh
The organization of the equipment layout and the material processing flow are described in Figure 2.
Figure 2. Layout of the production environment and material flow
http://ijlter.org/index.php/ijlter
115
For executing the scenarios, the Siemens Tecnomatix Real NC® (2020) - Plant Simulation software will be used. The decision was made because this software can simulate dynamic and productive environments in virtual reality. The characteristic of virtual reality can be observed through the desktop, or its images can be transferred to viewing glasses with characteristics of immersion in the virtual environment. An example of the images generated in virtual reality by the chosen software is shown in Figure 3.
Figure 3. Example of virtual reality application with (Siemens Tecnomatix RealNC, 2020) - Plant Simulation Source: http://www.engusa.com/pt_br/product/siemens-tecnomatix-plant-simulation
Proposals on the scenarios used to organize production planning and control are presented in the results section.
4. Results As described in the method, it was necessary to organize a mix of guidelines that impact the production planning and control process results for the preparation of the scenarios. During the creation of Scenario 1, it was defined that the available capacity of the processing equipment would be 24 hours a day and that the manufacturing batches would correspond to seven days of demand from the assembly departments. The other information applied to this scenario is available in Table 3.
http://ijlter.org/index.php/ijlter
116
Table 3. Scenario 1 for the application of virtual reality in the teaching processes of production planning and control
The objective of this scenario is to demonstrate the impact of large manufacturing batches and high setup times on production sequencing indicators. With this scenario, lead time indicators, inventory in processes, and punctuality of deliveries (for the assemblies) will suffer impacts due to the production lot and setup times. The second scenario was elaborated from the data used in Scenario 1. The same demand, capacity, cycle times, and failure rate data described in Scenario 1 were used. The manufacturing batches were changed to four days of demand and the setup times with an average reduction of 30%. The data for Scenario 2 are revealed in Table 4. Table 4. Scenario 2 for the application of virtual reality in production control teaching processes
planning and
The objective of Scenario 2 is to demonstrate the impacts on lead time and inprocess inventory with the reduction of manufacturing batches, setup times, and punctuality of deliveries in the evaluated indicators. For the development of Scenario 3, the same database as Scenario 2 was used, changing only the different data on failure rate and the available capacity among
http://ijlter.org/index.php/ijlter
117
the evaluated equipment. The structured data for the simulation of Scenario 3 are described in Table 5. Table 5. Scenario 3 for the application of Virtual Reality in the teaching processes of production planning and control
In Scenario 3, there was a reduction in the failure rate of approximately 35%. The available capacity was modified in five of the six pieces of equipment in this production system, in which Eq1 was kept available 24 hours a day; the Eq2 / Eq3 / Eq4 / Eq6 available 16 hours a day; and Eq5 available for 8 hours a day. The reduction in the failure rate increases the availability of the equipment to process materials, and the difference in the available capacity of the equipment generates high indexes of stocks awaiting processing and shortage of materials for later processes. Scenario 4 was elaborated from the previous scenario, changing processing times and material demand parameters to assess the impacts of these parameters on the indicators of manufacturing lead time, stock in process, efficiency, and punctuality of deliveries. The parameters used are displayed in Table 6. Table 6. Scenario 4 for the application of virtual reality in the teaching processes of production planning and control
http://ijlter.org/index.php/ijlter
118
In the scenario described in Table 6, there was a reduction in cycle times by approximately 20% and demand by 10% compared to that described in Table 5. The reduction in cycle times increases in the quantities produced in the same period, but negatively impacts the manufacturing lead time and an inventory increase. The demand reduction reinforces this impact because the consumption of subsequent processes is lower, generating greater inventory in the process. For elaborating scenarios 5 and 6, we tried to change the production planning decision and applied the production logic pulled into the system, in which the decision of what to produce depends on the planned stock level between operations. For the execution of Scenario 5, it was decided to use the same data as Scenario 1, but with supermarkets between operations. For Scenario 6, the data is identical to Scenario 4, also with supermarkets between operations. The layout of the production environment and the material flow for Scenarios 5 and 6 are shown in Figure 4.
Figure 4. Layout of the production environment and material flow for Scenarios 5 and 6
When evaluating the layout and material flow as shown in Figure 4, it is possible to notice a change compared to that presented in Figure 2. That is, it appears that Eq1 has its production managed by the stock levels of supermarkets called SA and SB. The inventory levels demand the production of Eq3 of supermarkets SC and SD. The production of the Eq4, Eq5, and Eq2 equipment (which has a continuous flow with the Eq6 equipment) is managed by the inventory level of the SE supermarket. The data defined for Scenario 5 are described in Table 7. Scenario 5 uses rules to control inventory levels in supermarkets to determine which components should be manufactured. When any part reaches the minimum inventory level, a new manufacturing batch can be started. The level
http://ijlter.org/index.php/ijlter
119
of in-process inventory will be reduced because the equipment worked less, even with production batches much larger than the daily demand (production batches with seven days of daily demands). On the other hand, the equipment efficiency will be reduced due to the demand lack for production in some moments. In Scenario 6, data from Scenario 4 of virtual reality simulation were used, according to Table 8. Table 7. Scenario 5 for the application of virtual reality in the teaching processes of production planning and control
Similar to Scenario 5, Scenario 6 brings all the production planning management related to the supermarket stock levels. Unlike the previous scenario, however, this one has setup times, maximum stock levels, and smaller production batches, and the production lead time and stock in process indicators show considerable improvements. Indicators of equipment efficiency and punctuality of delivery will only be possible if they are evaluated after the simulation of the model in virtual reality, using the software chosen for this task. All theoretical evidence indicates that punctuality of delivery should have better rates in the leanest scenarios (shorter setup times, shorter cycle times, shorter production batches), but the quantification of these rates is possible only after the scenarios simulation.
http://ijlter.org/index.php/ijlter
120
Table 8. Scenario 6 for the application of virtual reality in the teaching processes of production planning and control
5. Analysis of Results It appears that the application of virtual reality in the teaching-learning processes is used in several areas of performance, demonstrating superior learning results by students who use virtual reality when compared to those who only use the traditional teaching method. This application is verified, for example, in research by the authors like Quadir et al. (2019), Li et al. (2018), and Skarka et al. (2015). They demonstrated the efficiency of the approach and emphasized that it is essential to develop a learning environment that generates the opportunity to increase learning through virtual reality to have a relevant result. Deciding on the production planning and control area generates unprecedented research in an area of great relevance in production engineering. According to Lage Junior (2019), the processes in this area impact various organizational indicators, such as on-time delivery rate, inventory levels, idleness or lack of production capacity, organization of internal work at factories, and direct activities of purchasing, receiving, and storing materials. In addition, the research opportunity presented by Anjos et al. (2020) demonstrated the importance of creating differentiated teaching environments to generate the most significant possible knowledge for students. In addition, Kanakana-
http://ijlter.org/index.php/ijlter
121
Katumba and Maladzhi (2019) emphasized that the cognitive environment must consider some characteristics to be elaborated. In the scenarios proposed by the authors, an environment with an experimental approach (through virtual reality) is perceived because it generates for the student the opportunity to practice, the concepts and theories developed in a practical way. The scenarios to be applied in virtual reality were organized through a mix of guidelines. According to the guidelines change, there was a change in some indicators resulting from production planning and control. The relevance of each scenario can be highlighted; in Scenario 1, the impacts of large production lots and transfers between the equipment and the high setup times are shown ; in Scenario 2, data similar to scenario 1 are applied, but with smaller production batches and setup time. When comparing the results, impacts of indicators are perceived, such as reducing production lead time, inventory levels in processes, and punctuality of delivery. Similarly, Scenario 3 uses the reference data from Scenario 2 but with a reduction in the available capacity for production (reduction of the available time). Through this action, it will be possible to evaluate improvements in the efficiency indexes. Scenario 4 was elaborated on data from the previous scenario, but with reduced processing times for manufactured materials and reduced demand. This change influences all the indicators monitored during the virtual reality simulation. Scenarios 5 and 6 use the data applied in Scenarios 1 and 4, respectively, however, with a fundamental change, the production planning system starts to be drawn from the inventory levels for the assembly processes (SE supermarket). The validity of this model is based on the relationship between the factors that make up the production system, for example, fixed production batches or dynamic production batches (Brahimi et al., 2017; Suzanne et al., 2020), organization of production systems for discrete, continuous or project production (Armbruster et al., 2012; La Marca et al., 2010), equipment failure rate (Göttlich & Knapp, 2019), setup times and manufacturing lead time (Allahverdi & Soroush, 2008), processing time and take time (Ayough et al., 2020), production capacity available for the execution of the planned production (Babaei et al., 2014; Oliveira & Costa, 2018), organization of the layout, according to the processing and material flow scripts (Caicedo et al., 2019) and the evaluation the performance indicators of a production system, the results of decisions of the production sequencing impact in the indicators , for example, delivery attendant, delay of order, production advance, production lead time, number of overdue orders, inventory in process, and use of equipment (Lustosa et al., 2008).It is noticed that, in addition to the realism of the virtual environment Schlemmer and Backes (2015), the benefits of the training environment in virtual reality Duncan et al. (2012) and the cognition generated by the training environment Lefrançois (2016), scenarios proposed for application in virtual reality manage to articulate the guidelines that influence the planning and control of production, impact the monitored indicators, and finally support students to develop more clearly and concisely knowledge on the topic addressed.
http://ijlter.org/index.php/ijlter
122
6. Conclusion The discussion on the teaching-learning processes in production planning and control allows us to see that the combination of scenarios with virtual reality and the focus of the cognition of the teaching environment generate a combination of factors that, if well-organized, lead to positive results in student learning through the combination of dynamic data simulation and virtual reality, in which the virtual world brings students closer to the studied subject. The different scenarios provide students with two opportunities: (i) the ability to understand which factors related to the production system influence the production sequencing indicators, and (ii) with what impact each factor influences these evaluated indicators. Scenario 1 demonstrates the impact of factors (large production batches and high setup times) on the analyzed indicators and how changing the factors applied to scenario 2 improves the indicators. Scenarios 3 and 4 discuss other factors (failure rate and available capacity) and how the quantitative change of factors influences the evaluated indicators. Finally, they change the logic applied to production sequencing, transforming it into a pull system, in which production is managed by demand and all the benefits that this logic brings to the indicators of the production sequencing process.
7. Limitations Some limitations of the research are the lack of variation in process flows and factory layout and the lack of application of variability in the data used in the simulation, such as, production times, failure rates, setup times, and demand, because it is known that these types of data are usually not static.
8. Future Research For future research, the authors suggest testing other production sequencing approaches, such as drum, lung, and rope (from the theory of constraints), and developing a system with production for product stock by adding to this decision the variability of the data of the applied factors, for example, setup times, capacity and failure rate, to the sequencing of production.
9. References ABEPRO. (2008). Áreas e Sub-áreas de Engenharia de Produção [Production engineering areas and subareas] . http://www.abepro.org.br/interna.asp?p=399&m=424&ss=1&c=362 Akbulut, A., Catal, C., & Yildiz, B. (2018). On the effectiveness of virtual reality in the education of software engineering. Computer Applications in Engineering Education, 26(4), 918–927. https://doi.org/10.1002/cae.21935 Allahverdi, A., & Soroush, H. M. (2008). The significance of reducing setup times/setup costs. European Journal of Operational Research, 187(3), 978–984. https://doi.org/10.1016/j.ejor.2006.09.010 Anjos, F. E. V. dos, Rocha, L. A. O., Silva, D. O. da, & Pacheco, R. (2020). Virtual and augmented reality application in production engineering teaching-learning processes. Production, 30, 1–16. https://doi.org/10.1590/0103-6513.20190088 Armbruster, D., Fonteijn, J., & Wienke, M. (2012). Modeling production planning and transient clearing functions. Logistics Research, 5(3), 133–139. https://doi.org/10.1007/s12159-012-0087-8
http://ijlter.org/index.php/ijlter
123
Ayough, A., Zandieh, M., & Farhadi, F. (2020). Balancing, sequencing, and job rotation scheduling of a U-shaped lean cell with dynamic operator performance. Computers and Industrial Engineering, 143. https://doi.org/10.1016/j.cie.2020.106363 Babaei, M., Mohammadi, M., & Ghomi, S. M. T. F. (2014). A genetic algorithm for the simultaneous lot sizing and scheduling problem in capacitated flow shop with complex setups and backlogging. The International Journal of Advanced Manufacturing Technology, 70, 125–134. https://doi.org/10.1007/s00170-0135252-y Bates, A. W. (2015). Teaching in a digital age: Guidelines for designing teaching and learning (1st ed.). Tony Bates Associates Ltd. Brahimi, N., Absi, N., Dauzère-Pérès, S., & Nordli, A. (2017). Single-item dynamic lotsizing problems: An updated survey. European Journal of Operational Research, 263(3), 838–863. https://doi.org/10.1016/j.ejor.2017.05.008 Caicedo, A. J., Parra, J. W., & Rivera, L. (2019). Mathematical model for production sequencing in a manufacturing company. Journal of Physics: Conference Series, 1388(1). https://doi.org/10.1088/1742-6596/1388/1/012001 Corrêa, Henrique Luiz; Gianese, Irineu Gustavo Nogueira; Caon, M. (2019). Planejamento, programação e controle da produção: MRP II / ERP :Conceitos, uso e implantação : Base para SAP, oracle applications e outros softwaresintegrados de gestão [Production planning, scheduling and control: MRP II / ERP: Concepts, use and implementation: Basis for SAP, oracle applications and other integrated management software] (6th ed.). Atlas. Crawley, E., Hosoi, A., Long, G., Kassis, T., Dickson, W., & Mitra, A. (2019, June). Moving forward with the New Engineering Education Transformation (NEET) program at MIT - building community, developing projects, and connecting with industry. 2019 Paper presented at ASEE Annual Conference & Exposition Proceedings, Tampa, Florida. https://doi.org/10.18260/1-2--33124 Duncan, I., Miller, A., & Jiang, S. (2012). A taxonomy of virtual worlds usage in education. British Journal of Educational Technology, 43(6), 949–964. https://doi.org/10.1111/j.1467-8535.2011.01263.x Fonseca, D. E., Redondo, E. D., & Valls, F. (2016). Motivación y mejora académica utilizando realidad aumentada para el estudio de modelos tridimensionales arquitectónicos [Motivation and academic improvement using augmented reality for the study of architectural three-dimensional models]. Education in the Knowledge Society (EKS), 17(1), 45–64. https://doi.org/10.14201/eks20161714564 Frevert, M., & Di Fuccia, D.-S. (2019). Virtual reality as a means of teaching contemporary chemistry. Proceedings of the 2019 The 3rd International Conference on Digital Technology in Education, 34–38. https://doi.org/10.1145/3369199.3369218 Gilbert, J. K. (2004). Models and modelling: Routes to more authentic science education. International Journal of Science and Mathematics Education, 2(2), 115–130. https://doi.org/10.1007/s10763-004-3186-4 Göttlich, S., & Knapp, S. (2019). Load-dependent machine failures in production network models. SIAM Journal on Applied Mathematics, 79(4), 1197–1217. https://doi.org/10.1137/18M1193177 Grodotzki, J., Ortelt, T. R., & Tekkaya, A. E. (2018). Remote and virtual labs for engineering education 4.0: Achievements of the ELLI project at the TU Dortmund university. Paper presented at the 26th SME North American Manufacturing Research Conference,NAMRC 46,Texas, USA. https://doi.org/10.1016/j.promfg.2018.07.126
http://ijlter.org/index.php/ijlter
124
Habib, A., Jilcha, K., & Berhan, E. (2015). Performance Improvement by Scheduling Techniques: A Case of Leather Industry Development Institute. Paper presented at Afro-European Conference for Industrial Advancement,261–269). https://doi.org/10.1007/978-3-319-13572-4_21 Hwang, G.-J., Chiu, L.-Y., & Chen, C.H. (2015). A contextual game-based learning approach to improving students’ inquiry-based learning performance in social studies courses. Computers & Education, 81, 13–25. https://doi.org/10.1016/j.compedu.2014.09.006 Inayat, I., Inayat, Z., & ul Amin, R. (2016). Teaching and learning object-oriented analysis and design with 3D game. Proceedings of 14th International Conference on Frontiers of Information Technology , 46–51. https://doi.org/10.1109/FIT.2016.16 Kanakana-Katumba, M. G., & Maladzhi, R. (2019). Online learning approaches for science, engineering and technology in distance education. Paper presented at the IEEE International Conference on Industrial Engineering and Engineering Management, 930–934. https://doi.org/10.1109/IEEM44572.2019.8978892 National Association of State Directors of Career Technical Education Consortium , & Achieve. (2015, November 5). Building a strong relationship between competency-based pathways and career technical education [Blog post]. https://careertech.org/sites/default/files/CTECompetencyBasedPathways.pdf La Marca, M., Armbruster, D., Herty, M., & Ringhofer, C. (2010). Control of continuum models of production systems. IEEE Transactions on Automatic Control, 55(11), 2511–2526. https://doi.org/10.1109/TAC.2010.2046925 Lage Junior, M. (2019). Planejamento e controle da produção : teoria e prática [Production planning and control: Theory and practice] (1st ed.). LTC. Lamb, R., Antonenko, P., Etopio, E., & Seccia, A. (2018). Comparison of virtual reality and hands on activities in science education via functional near infrared spectroscopy. Computers & Education, 124, 14–26. https://doi.org/10.1016/j.compedu.2018.05.014 Laseinde, O. T., Adejuyigbe, S. B., Mpofu, K., & Campbell, H. M. (2016). Educating tomorrows engineers: Reinforcing engineering concepts through Virtual Reality (VR) teaching aid. Paper presented at the IEEE International Conference on Industrial Engineering and Engineering Management, IEEM 2015, 2016. https://doi.org/10.1109/IEEM.2015.7385894 Lefrançois, G. R. (2016). Teorias da Aprendizagem: O que o professor disse [Theories of human learning] (6th ed.). Cengage Learning. Li, Y., Zhang, D., Guo, H., & Shen, J. (2018). A novel virtual simulation teaching system for numerically controlled machining. International Journal of Mechanical Engineering Education, 46(1), 64–82. https://doi.org/10.1177/0306419017715426 Lustosa, L. J., Mesquita, M. A., Quelhas, G., Osvaldo, L., & Oliveira, R. J. de. (2008). Planejamento e controle da produção [Planning and production control] (1st ed.). Elsevier. Martins, V. F., & Guimarães, M. de P. (2012). Desafios para o uso de Realidade Virtual e Aumentada de maneira efetiva no ensino [Challenges for using virtual and augmented reality effectively in teaching]. Anais Do Workshop de Desafios Da Computação Aplicada à Educação[Proceedings of the Challenges of Computing Applied to Education Workshop], 100–109. Mayne, R., & Green, H. (2020). Virtual reality for teaching and learning in crime scene investigation. Science and Justice, 60(5), 466–472. https://doi.org/10.1016/j.scijus.2020.07.006
http://ijlter.org/index.php/ijlter
125
Mestrinho, N., & Cavadas, B. (2018). Innovation in teacher education: An integrative approach to teaching and learning science and mathematics. Proceedings, 2(21), 1343. https://doi.org/10.3390/proceedings2211343 Mizukami, M. da G. N. (1992). Ensino: A Abordagens do Processo [Teaching: Process Approaches] (1st ed.). LTC. Oliveira, F. S., & Costa, M. L. G. (2018). Capacity expansion under uncertainty in an oligopoly using indirect reinforcement-learning. European Journal of Operational Research, 267(3), 1039–1050. https://doi.org/10.1016/j.ejor.2017.11.013 Prensky, M. (2001). Digital natives, digital immigrants part 1. On the Horizon, 9(5), 1–6. https://doi.org/10.1108/10748120110424816 Quadir, B., Yang, J. C., & Chen, N. S. (2019). The effects of interaction types on learning outcomes in a blog-based interactive learning environment. Interactive Learning Environments, 1–14. https://doi.org/10.1080/10494820.2019.1652835 Sánchez, A., Redondo, E., Fonseca, D., & Navarro, I. (2015). Academic performance assessment using augmented reality in engineering degree course. Proceedings Frontiers in Education Conference, 1–7. https://doi.org/10.1109/FIE.2014.7044238 Santos, J. (2006). Teorias da aprendizagem: Comportamentalista, cognitivista e humanista [Learning theories: Behaviorist, cognitivist and humanist]. Revista Cientifica Sigma [Sigma Scientific Journal], 2, 97–111. Schlemmer, E., & Backes, L. (2015). Learning in Metaverses. IGI Global. https://doi.org/10.4018/978-1-4666-6351-0 Shirazi, A., & Behzadan, A. H. (2015). Content delivery using augmented reality to enhance students’ performance in a building design and assembly project. Advances in Engineering Education, 4(3), 1–24. Skarka, W., Wylezol, M., Januszka, M., Rzydzik, S., & Targosz, M. (2015). Improving the ability of future engineers by using advanced interactive 3D techniques in education. In R. Curran, N. Wognum, M. Borsato,J. Stjepandic & W.J.C. Verhagen (Eds.), Transdisciplinary lifecycle analysis of systems ( pp. 647–656). IOS Press. https://doi.org/10.3233/978-1-61499-544-9-647 Smutny, P., Babiuch, M., & Foltynek, P. (2019). A review of the virtual reality applications in education and training. Proceedings of the 2019 20th International Carpathian Control Conference, ICCC. https://doi.org/10.1109/CarpathianCC.2019.8765930 Soudien, C. (2010). Transformation in higher education- A briefing paper. Policy Studies Journal, 44(1), 29. https://doi.org/10.1007/1-4020-4006-7 Suzanne, E., Absi, N., Borodin, V., & van den Heuvel, W. (2020). A single-item lot-sizing problem with a by-product and inventory capacities. European Journal of Operational Research, 287(3), 844–855. https://doi.org/10.1016/j.ejor.2020.05.017 Tsai, C. W., Shen, P. Di, & Lu, Y. J. (2015). The effects of problem-based learning with flipped classroom on elementary students’ computing skills: A case study of the production of Ebooks. International Journal of Information and Communication Technology Education, 11(2), 32–40. https://doi.org/10.4018/ijicte.2015040103 Wang, H.-Y., Liu, G.-Z., & Hwang, G.-J. (2017). Integrating socio-cultural contexts and location-based systems for ubiquitous language learning in museums: A state of the art review of 2009-2014. British Journal of Educational Technology, 48(2), 653– 671. https://doi.org/10.1111/bjet.12424 Żechowicz, M., Gromadziński, L., Dudziński, L., Grabarczyk, Ł., Żechowicz, T., Zabłocki, M., & Zaborowski, P. (2018). Development and implementation of methods for training doctors and dental surgeons using virtual devices and simulation. Polish Annals of Medicine. https://doi.org/10.29089/2017.17.00002
http://ijlter.org/index.php/ijlter
126
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 126-138, August 2021 https://doi.org/10.26803/ijlter.20.8.8 Received Jul 10, 2021; Revised Aug 11, 2021; Accepted Aug 25, 2021
Medical Students’ Evaluation of Online Assessment: A Mixed-Method Account of Attitudes and Obstacles Abed Alkarim Ayyoub* Faculty of Educational Sciences and Teacher Training, An-Najah National University, Nablus, Palestine https://orcid.org/0000-0001-9111-4465 Oqab Jabali Language Center Faculty of Humanities, An-Najah National University, Nablus, Palestine https://orcid.org/0000-0003-1156-6205
Abstract. This mixed-method account explores the medical students’ perceptions and attitudes regarding online assessment in two Palestinian universities. The researchers aimed at identifying the way medical students look at online assessment, as well as the pitfalls and the drawbacks of such an atypical evaluation method at the university level. Of the large number of medical students enrolled in the two universities, 302 completed and returned the survey, and 61 students were selected to interpret their open responses qualitatively. The study findings suggest that the evaluation of Al-Quds University in Jerusalem was better and higher than that of the An-Najah National University in Nablus. Statistically significant differences were found, when some demographic variables, i.e., gender and year of study interaction were selected. Finally, the study respondents highlighted a number of drawbacks for online assessment, which the researchers sorted into eight main categories that varied in percentages. The medical students showed their dissatisfaction with this type of assessment, for many considerations, including, but not limited to, technical and administrative matters in nature. Keywords: evaluation; medical students; online assessment; perceptions; technical problems
*
Corresponding author: Abed Alkarim Ayyoub; Email: ayyoub@najah.edu
©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
1. Introduction A lot of factors have recently emphasised the necessity to opt for the adoption of online teaching and learning. The tremendous technological advances, educators’ and students’ high expectations, and the rise of unexpectedly urgent conditions and demands, all dictate that the majority of educational institutions should seriously consider the use of modern technology in learning/teaching situations. Consequently, a large number of higher educational institutions worldwide have currently been incorporating computer technology for a considerable time. Pedagogically, education is not merely communicating information to learners in a passive way. Rather, students acquire information, or knowledge, by being actively engaged socially, psychologically and cognitively, as they construct meanings. This is deeply rooted in the constructivist model, in which such factors as socio-cultural and emotional beliefs, values and attitudes significantly impact students’ learning outcomes and their interaction in schools (Ambrose et al., 2010). As such, the main premise of the constructivist approach revolves around bringing the learner to the forefront of the teaching/learning situation, while adopting relevant pedagogical strategies that build on students’ diverse needs, interests, strengths, and expectations, and that are conducive to their academic success and social development (Richardson & Fleer, 2003). Students are very likely to develop meaningful knowledge, based on their experiences away from stringent, firm classroom regulations; and they might then well suggest their own relevant assessment techniques that are very necessary in the teaching/learning process (Kottail, 2009); they also learn better when they are provided with opportunities to be part of the assessment approach that is used in the classroom. It has been argued that the application of various evaluation and assessment techniques, within the boundaries of the constructivist model, promotes students’ critical thinking skills and increases their academic achievement. Additionally, involving students in educational assessment may have future positive outcomes in their lives. It should also reduce instructors’ anxieties or concerns. In fact, relevant and meaningful forms of assessment tend effectively to measure and “evaluate judgment, attitude and behaviour, in addition to knowledge, and skills” (Harris et al., 2017, p. 605). The quality of educational assessment processes should be based on universal standards, as well as on students’ unique and preferred learning styles and intelligence levels (Harris at al., 2017). While this can be a challenging process, it might be facilitated by integrating alternative assessment strategies, including the online assessment technique (Bennett, 2011). This new type of assessment could well result in improving students’ performance outcomes. In fact, this approach can help in overcoming the gaps in traditional assessment that might not be suitable for assessing higher-level cognitive and affective skills (Kuh et al. 2014). Using technology in the process of assessment has become a reality that ranges from developing examinations to storing results for future use. It is intended for assessing learners’ prior knowledge, skills, and abilities; it also aims to create and manage materials and resources, in addition to providing feedback (Cakiroglu et
http://ijlter.org/index.php/ijlter
al., 2017). E-assessment provides new methods and opportunities for various types of assessment related to various types of knowledge (Alruwais, Wills & Wald, 2018; Chang et al., 2013; Crews & Curtis, 2010; Kuriakose & Luwes, 2016). Furthermore, electronic assessment helps to reduce the work load of instructors and students; since it is likely to be accessed at anytime and anywhere (Cukusic et al., 2014). Accordingly, it is easy for educators and school administrators to gather data, to conduct statistical analyses and to test the results (Broughton, 2013; Douglas, 2012); and consequently, to make good decisions (Duran et al., 2013). Furthermore, e-assessment can be easily reproduced and utilized; since it needs no more than a simple computer, or a smart phone with an internet connection. This would make it possible to measure students’ outcomes, and to instantaneously, help them to get direct and immediate feedback on their performance (Alruwais, Wills & Wald, 2018; Betlej, 2013). It is intended to promote authentic assessment, as well as to facilitate testing and collaboration (Guo et al., 2014; Johnson & Davies, 2012). Electronic testing is more cost-effective, and easier to handle, to administer and to score, more reliable and easier to replicate; and sometimes, it is more authentic, objective, and bias-free, when marking students’ responses (Jordan & Mitchell, 2009; Khare & Lam, 2008). It also allows instructors to use questions that promote interactivity and the use of multimedia (Lahad et al., 2004), which would undoubtedly benefit students who are subjected to this form of evaluation (Ali et al., 2021). Online assessment may face some challenges and obstacles. For instance, some students may be inexperienced in handling computer programs and online assessment techniques (Alruwais, Wills & Wald, 2018). Another challenge may relate to computer availability and internet connection (Ridgway et al., 2004), as well as to the lack of sufficient and good infrastructure (Ridgway et al., 2004), especially in poor countries like Palestine. It is also possible for some teachers to face problems, when they are forced to use e-assessment techniques, especially for the first time; some instructors may lack the necessary experience, or they may not be familiar with the technology; and consequently, this may pose problems for students, such as delaying the loading of exams (Ogletree et al., 2014; Russell & Shepherd, 2010). Furthermore, there is the open-question issue; marking open questions, such as explaining, and listing items would then be difficult to handle (Stodberg, 2012). Student identification is another major challenge that exacerbates the reluctance to adopt online assessment; needless to say, learners cannot be verified easily, although being remote. Cheating is another pitfall of online assessment; when students take an exam that is not invigilated. Cheating more than doubles in such situations (Osuji, 2012; Stodberg, 2012; Yates & Beaudrie, 2009). Students have different attitudes and perspectives towards their instructors, the teaching/learning methodologies, as well as the assessment techniques used to evaluate their performance. They also have their own reservations about the effectiveness in certain circumstances. The current study aimed to identify medical students’ evaluation of online assessment, which had been used for a
http://ijlter.org/index.php/ijlter
whole year in two Palestinian universities. The researchers attempted to answer the following two questions: 1) Do medical students' evaluation of the electronic assessment differ, according to students’ gender, level (academic year), and total average?: and 2) What are the major obstacles that face educators, when using online/electronic assessment?
2. Methods To carry out the current quantitative/qualitative study, institutional consent and approval had to be obtained from the two university vice-presidents of academic affairs, the directors of the Electronic-Learning Centre, the deans of Scientific Research Departments, as well as the research Ethics Committees. Quantitatively, a descriptive online questionnaire was compiled to explore the way medical students evaluate and perceive online assessment during the COVID-19 pandemic, in the academic year of 2019/2020. As the researchers were interested in examining the relevant information efficiently, e-assessment as an evaluation tool, which was conceived comprehensively, in order to include any possible advantage, or to exclude any disadvantage that might be relevant to instruction and/or education at the university level. The survey included 100 constructed declarative statements, which were sent to 40 arbitrators, whose scores were collected; and the correlation between each paragraph and the total was calculated; and then the paragraphs, the correlation of which was found to be less than 0.6, were deleted. Then, the mean scores for the first and last quarters were calculated by the arbitrators, for rating the paragraphs. A t-test of two independent samples between the means of the two quarters for each paragraph was calculated, and the paragraphs for which the differences were not statistically significant, were deleted – simply because the researchers were merely interested in the paragraphs that are distinguished by high discrimination. Finally, 35 high discrimination items that were of significant relevance and correlation to the concept of e-assessment were retained. To ensure the questionnaire’s validity, factorial validity was calculated by using the Kaiser-Mayer-Olkin test; this was found to be 0.0955 which ensured that the items were suitable for exploratory-factor analysis. To exclude orthogonal items, the principal-component method and the Oblimin rotation method were used; consequently, items (9, 10, 11, 24, 29, 34) the communality degree, or factor loading of which was less than 0.3, were excluded. In total, 29 items were retained, as shown in the scree plot below.
http://ijlter.org/index.php/ijlter
Figure 1: Scree plot for the number of dimensions in the questionnaire
Clearly here, there was a dominant dimension; since there was one infliction point, which represented 41% of the explained variance, as shown in Table 1 below. Consequently, the questionnaire was considered suitable, according to the factorial-validity analysis. Table: Total Variance explained for the dominant dimension of the questionnaire
Component 1
Initial Eigenvalues % Cumulative Total Variance % 11.812 40.733 40.733
Extraction Sums of Squared Loadings % Cumulative Total Variance % 11.812 40.733 40.733
To ensure the reliability of the questionnaire, Cronbach’s Alpha was calculated. Cronbach’s Alpha coefficient was 0.946 for the total of 29 items. The alpha value was higher than 0.7; this shows that the questionnaire is reliable. Eventually, the questionnaire was posted for medical students to complete online. 302 survey instruments were completed and returned for analysis. Qualitatively, the principal study tool also required students to provide information about their attitude towards online assessment, the various obstacles they faced and the ways to overcome them, in order to complement and inform the quantitative findings by providing valuable data that would help to understand online assessment and the students’ attitudes towards it. Sixty-one students from each university were randomly selected; and their responses were analyzed by using MAXQDA. The researchers sorted through the obstacles and classified them into eight different codes, as follows: ethical, administrative, training, time, technical, negative attitudes, substantive issues, and psychological problems.
http://ijlter.org/index.php/ijlter
2.1. Sample The population of this study included those students who study medicine at AnNajah National University in Nablus and those at the Al-Quds University in Jerusalem. However, few students answered the survey (n = 302), with 151 from each university. A total of 61 students’ responses were randomly selected, in order to account for the qualitative analysis of the data. 2.2. Procedures Eventually, the questionnaire was posted to the university students online; it was written in the students’ native language (Arabic), to ensure that all the students fully understood the survey items. A total of 302 survey instruments were completed and returned for analysis. Data collection was carried out during the first semester of the academic year of 2019/2020. Quantitatively, the data were normally distributed and analyzed by using descriptive statistics; factorial ANOVA was used to calculate the mean differences between the demographic elemental scores, by using the SPSS version 26. The principal study tools also required students to provide information about their attitudes, opinions and perspectives on e-assessment, in order to complement and inform the quantitative findings by providing valuable data that examined students’ attitudes towards these matters. Qualitatively, MAXQDA was used by the researchers themselves, in order to calculate the frequencies, percentages and students’ responses, and their opinions, as well as the information provided on the open-ended questions. MAXQDA is a software program designed for computer-assisted qualitative methods, data and text analysis; it offers tools for the organization and analysis of qualitative data, especially those obtained as texts, in order to attain a valuable explanation and a comprehensive understanding or interpretation of a phenomenon, or a tendency [MAXQDA: The Art of Data Analysis, n.d.]. 2.3. Analysis Quantitatively, the data were normally distributed and analyzed by using descriptive statistics; and factorial ANOVA was used to calculate the mean differences between the demographic elemental scores by using the SPSS version 26. Qualitatively, MAXQDA was used to calculate the frequencies, percentages, and students’ responses to the open questions.
3. The Results 3.1. Demographic characteristics and the evaluation of the online assessment To answer the first question, (302) students studying medicine and health sciences at An-Najah National University (n= 151) and Al-Quds University (n= 151) responded to the questionnaire. The descriptive statistics of the students’ responses were calculated, based on the demographic variables; the results are shown in Table 2.
http://ijlter.org/index.php/ijlter
Table 2: Demographic features of respondents and the results An Najah University
Gender Average
Year
Al Quds University
Male
N 51
M 3.10
S.D. 0.85
N 46
M 3.45
S.D. 0.55
Female
100
3.07
0.63
105
3.21
0.50
Excellent
23
2.79
0.81
20
3.33
0.75
Very good
57
3.25
0.60
60
3.30
0.56
Good
57
3.03
0.77
61
3.25
0.42
Fair
14
3.08
0.53
10
3.28
0.38
Freshman
17
2.78
1.12
18
3.25
0.57
Sophomore
35
3.11
0.52
35
3.34
0.50
Junior
56
3.14
0.70
39
3.30
0.64
Senior
34
3.13
0.65
32
3.17
0.53
9
3.01
0.62
27
3.33
0.33
151
3.08
0.73
151
3.28
0.53
Super senior Total
The results in Table 2 show that medical students’ evaluation of the online assessment was medium; as the mean squares ranged between 2.6 – 3.4, based on a Likert Scale. However, the evaluation of the students at the Al-Quds University was higher than those of the An-Najah University (0.2). Furthermore, there were differences among the students’ demographic variables. To ensure whether these differences were statistically significant, the researchers used Factorial ANOVA, as shown in Table 3 below. Table 3: Tests of Between-Subjects Effects (Factorial ANOVA) for demographic variables SS 1.35
Df 1
Mean Square 1.35
F 3.78
P 0.04
η2 0.02
Gender
1.09
1
1.09
3.07
0.08
0.01
Average
0.04
3
0.01
0.04
0.99
0.00
Year
0.58
4
0.14
0.41
0.80
0.01
university * gender
0.15
1
0.15
0.43
0.51
0.00
university * average
0.63
3
0.21
0.59
0.62
0.01
university * year
0.89
4
0.22
0.63
0.64
0.01
gender * average
1.61
3
0.54
1.51
0.21
0.02
gender * year
3.73
4
0.93
2.62
0.04
0.04
average * year
5.41
12
0.45
1.27
0.24
0.06
university * gender * average
0.06
3
0.02
0.06
0.98
0.00
university * gender * year
0.72
4
0.18
0.51
0.73
0.01
university * average * year
5.33
10
1.03
1.48
0.20
0.02
gender * average * year
4.65
9
0.52
1.45
0.17
0.05
university * gender * average * year
0.54
2
0.27
0.76
0.47
0.01
84.42
237
0.36
119.59
301
Source University
Error Corrected Total
http://ijlter.org/index.php/ijlter
The results showed that there was a statistically significant main effect in the evaluation of online assessment attributed to university, F (1, 237) 3.87, p 0.04, ŋ2 0.02;
looking back at Table 1, one sees that the evaluation of Al-Quds University was better than that of An-Najah University. However, no statistically significant differences were found, based on the other variables. With respect to interactions among the demographic variables, the study results showed that there were no statistically significant differences for most of them, except the interaction between gender and the year of study, F (4, 237) 2.62, p 0.04, ŋ2 0.04. Figure 2 below illustrates the trends of these differences.
Figure 2: Trend of interaction between student’s gender and year of study
Figure 2 above shows that the evaluation of male students is better than that of the female students in all the years of study, except for the freshmen, whose evaluation was higher. However, the higher the level of the student, the lower their evaluation. This is inconsistent with the findings of a study conducted in Romania, which showed acceptance of online assessment increases, as the students move to higher levels (Marius et al., 2016). To answer the second question, which required students to provide information about their attitudes towards online assessment, the various obstacles faced, and the ways to overcome them, sixty-one students from each university were randomly selected; and their responses were analyzed by using MAXQDA. The researchers sorted the obstacles and classified them into eight different codes, as follows: ethical, administrative, training, time, technical, attitudinal, substantive, and psychological problems, as outlined in Figure 3. It can be noticed that the students in both universities believe that most of the obstacles were administrative in nature; the way the two universities manage online assessment has many pitfalls, such as question types, mark distribution, marking, and the lack
http://ijlter.org/index.php/ijlter
of any feedback. Technical problems (e.g., poor internet connection, lack of infrastructure, etc.) were also detected in both universities. Attitudinal and psychological problems were more at An-Najah University than at Al-Quds University; and this explains why the evaluation of Al-Quds University was better than that of An-Najah University. 18 16 14 12 10 8 6 4 2 0
AlQuds Un An Najah Un
Figure 3: Codes of students’ attitudes to online assessment
4. Discussion The main study findings showed that the evaluation of Al-Quds University (Mean is 3.28) was better and higher than those of the other university (Mean is 3.08), as shown in Table 2 above. This could be explained by various reasons. Firstly, the marking system used in the two universities varies. An-Najah University administers at least three exams per semester, in addition to a set of assignments, research papers, projects, discussions, or presentations; while Al-Quds University administers very few exams. This means that it is very likely for students to lose marks in each exam, or in any other evaluation activity. In addition, the high percentage of the participation mark is offered to Al-Quds students; it is 20% of the total mark of each course at this university; while it is 5%-10% in very few courses at An-Najah University. Secondly, An-Najah University adopts the National Board of Medical Exams (NBME), which forces students to sit for these exams at different levels; and it contributes 40% of its weight to the final mark of many courses. Some of these exams are very demanding (Internal Medicine and Advanced Surgery); and many students do not do well in them; since they were originally prepared for students whose native language is English, but not for students that use English as a foreign language. Furthermore, An-Najah students study a course called Medical Ethics in the first year; and they are tested in this course in the fourth year, as part of the NBME; added to this, is the fact that these exams are paid for by the students themselves. On the other hand, most of these exams are held locally at Al-Quds University; and they are free of charge.
http://ijlter.org/index.php/ijlter
The study findings also showed that both gender and year interacted; and consequently, this resulted in statistically significant differences. Responses to the questionnaire items showed that the evaluation of fresh male students was lower than that of their female counterparts; female students in the first year of study were mostly more apprehensive and more anxious, due to being in a different environment, and this, together with their inability to find the necessary textbooks, resources and materials, increased their concerns. However, the evaluation of females improved considerably in the second year; but it was still lower than that of their male counterparts. This could be attributed to the fact that these females started to become familiar with the university and its system; they made considerably more effort; and they also showed more satisfaction with the exam marking and teaching methodologies. Furthermore, the study findings showed that in the third year, female students showed more dissatisfaction with online assessment; since they started to lose marks, due to adopting the online assessment approach, which lacks precision, fairness, and an explicit grading system. Similar results were reached by Wiggins (1990). Third year students began to lose marks, due to examiners’ high level of subjectivity, as stated by (Moni et al., 2002). With respect to the main obstacles that face online assessment, the researchers sorted them into eight main categories, which varied in abundance. A lot of students in the two universities stated that the time allocated to online exams was barely sufficient; consequently, these students were forced to exert more effort to study and gain higher marks. This finding is inconsistent with the findings of previous research done by Cukusic et al. (2014). Added to this is the timer on the exam; it creates more stress (Khan and Khan, 2019). This is also contrary to Betlej’s (2013) findings that online assessment is easy to handle and enables students to get feedback very speedily. The current study showed that e-assessment was not that easy, due to the lack of adequate infrastructure for online assessment and poor internet connection (Ridgway et al., 2004). Students at both universities also stated that in most online exams, students could not go back to questions that they had already answered, due to examiners’ intervention; and this relates indirectly to the marking of these exams. While these results are consistent with those of Stodberg’s (2012) research findings, they contradict those of other researchers, such as Jordan & Mitchell (2009), Khare & Lam (2008), who undermined the role of examiners. Furthermore, students argued that sitting for many exams, doing many assignments, and being tested on topics they had never studied, was very difficult and challenging – so much so, that exams become irrelevant, and arbitrary; and they are done just to acquire marks (Hawe, 2002); while most of the question items required deeper understanding; and this further reduced their motivation. Another major technical obstacle, according to students, is related to poor internet connection, which results in slow logging into the exams, and sometimes delayed loading of these exams by instructors (Russell & Shepherd, 2010). Finally, some students believed that online assessment was not fair; since it possibly allows
http://ijlter.org/index.php/ijlter
some students to cheat, due to insufficient invigilation and examinee-identity verification; such a result is consistent with other studies, including those of Osuji (2012); Russell & Shepherd, (2010); Yates & Beaudrie (2009). Consequently, online assessment reduces motivation among smart, hard-working students, who would eventually disapprove of this type of assessment.
5. Conclusion The researchers found that applying new types of assessment without careful, prior strategic planning, is very likely to create different attitudes among students, irrespective of their gender, year of study or average marks. Students’ evaluations differ, based on where they study. Medical students showed dissatisfaction with this type of assessment, for many reasons, including, but not limited to, various technical and administrative aspects. Consequently, students’ perceptions and attitudes, regarding online assessment, should be considered, in order to ascertain a smooth educational process that effectively incorporates technology. University administrations are supposed to prepare well for such a transition to online assessment, in order to persuade students of the usefulness thereof. This study showed that to convince students to accept online assessment, universities should plan well, provide adequate infrastructure, and consider students’ circumstances. Finally, the highlighted pitfalls of online assessment should be addressed seriously and carefully, in order to integrate this type of assessment – not only at universities – but also in all the other educational institutions. This study was carried out in two Palestinian universities; it could be improved if other universities or university staff members, as well as decision-makers, were involved. Added to this limitation, of course, was the limited access to students in the other university, which is located in a place that was not easy for the researchers to access, without having a permit from the Israeli occupation. The small number of participants involved in the qualitative survey was attributed to this fact. The time allocated to respond to the online questionnaire played a negative role; and it resulted in having a small, unrepresentative sample of participants, who completed the survey. Consequently, a future longitudinal study could shed more light on the benefits, challenges, and shortcomings of online assessments, as well as their impact on students’ attitudes and their academic achievements. Contributors: Both authors made substantial contributions to the writing of this paper; they were both involved in drafting this article, or in revising it critically for important intellectual content; and they both gave their final approval of the version to be published.
6. References Ali, L., Abidal Hamid. N., & Damour, H. (2021). The Shift to Online Assessment Due to COVID-19: An Empirical Study of University Students, Behaviour and Performance, in the Region of UAE. International Journal of Information and Education Technology, 11(5), 220-228. https://doi.org/10.18178/ijiet.2021.11.5.1515
http://ijlter.org/index.php/ijlter
Ambrose, S. A., Bridges, M. W., DiPietro. M., Lovett, M. C., & Norman, M. K. (2010). How Learning Works-Seven Research-Based Principles for Smart Teaching. San Francisco, CA: Jossey Bass. Bennett, R. E. (2011). Formative assessment: a critical review. Assessment in Education: Principles, Policy & Practice, 18(1), 5-25. https://doi.org/10.1080/0969594X.2010.513678 Betlej, P. (2013). E-examinations from student's perspective – The future of knowledge evaluation. Studia Ekonomiczne, 153, 9–22. Broughton, S., Hernandez-Martinez, P., & Robinson, C. (2013, July 28th - August 2nd). A definition for effective assessment and implications on computer-aided assessment practice 37th Conference of the International Group for the Psychology of Education, (PME 37), (Vol. 2, pp. 113-120). Kiel, Germany. Cakiroglu, U., Erdogdu, F., Kokoc, M., & Atabay, M. (2017). Students’ preferences in online assessment process: Influences on academic performances. Turkish Online Journal of Distance Education-TOJDE, 18(1), 132-142. Chang, C. C., Liang, C., & Chen, Y.H. (2013). Is learner self-assessment reliable and valid in a Web-based portfolio environment for high school students? Computers & Education, 60(1), 325-334. https://doi.org/ 10.18178/ijiet.2021.11.5.1515 Crews, T., & Curtis, D. (2010). Online course evaluations: Faculty perspective and strategies for improved response rates. Assessment & Evaluation in Higher Education, 36(7), 965–878. https://doi.org/10.1080/02602938.2010.493970 Cukusic, M., Garaca, Z., & Jadric, M. (2014). Online self-assessment and students' success in higher education institutions. Computers & Education, 72, 100-109. https://doi.org/10.1016/j.compedu.2013.10.018 Douglas, M., Wilson, J., & Ennis, S. (2012). Multiple-choice question tests: a convenient, flexible and effective learning tool? A case-study. Innovations in Education and Teaching International, 49(2), 111–112. https://doi.org/101080/147032972012677596 Duran, M., Mihladiz, G., & Balliel, B. (2013). The competency level of elementary school teachers towards the alternative assessment methods. Mehmet Akif Ersoy Universitesi Egitim Bilimleri Enstitusu Dergisi, 2(2), 26-37. Guo, R., Palmer-Brown, D., Lee, S., & Cai, F. (2014). Intelligent diagnostic feedback for online multiple-choice questions. Artificial Intelligence Review, 42(3), 369–383. http://doi.org/101007/s10462-013- 9419-6. Harris, P., Bhanji, F., Topps, M., Ross, S., Lieberman, S., Frank, J. R., Snell, L., Sherbino, J., & ICBME Collaborators. (2017). Evolving concepts of assessment in a competency-based world. Medical Teacher, 39(6), 603–608. http://doi.org/10.1080/0142159X.2017.1315071 Hawe, E. M. (2002). Assessment in a pre‐service teacher education program: The rhetoric and the practice of standards‐based assessment. Asia‐Pacific Journal of Teacher Education, 30(1), 93–106. http://doi.org/10.1080/13598660120115002 Johnson, G., & Davies, S. (2012). Unsupervised Online Constructed-Response Tests: Maximizing Student Learning and Results Integrity (pp. 400–408) Wellington: Paper presented at the Ascilite Conference. Jordan, S., & Mitchell, T. (2009). E-assessment for learning? The potential of short-answer free-text questions with tailored feedback. Br J Educ Technol, 40(2), 371–385. https://doi.org/10.1111/j.1467-8535.2008.00928.x Khan, S., & Khan, R. (2019). Online assessment. Exploring perspectives of university students. Education and Information Technologies, 24, 661–677. https://doiorg/101007/s10639-018-9797-0.
http://ijlter.org/index.php/ijlter
Khare, A., & Lam, H. (2008). Assessing student achievement and progress with online examinations: Some pedagogical and technical issues. International Journal on Elearning, 7(3), 383–402. Kottail, N. K. (2009). Meaning making and self-evaluation. ERIC. http://filesericedgov/fulltext. Accessed August 10, 2020. Kuh, G. D., Jankowski, N., Ikenberry, S. O., & Kinzie, J. (2014). Knowing What students Know and Can Do: The Current State of Student Learning Outcomes Assessment in US Colleges and Universities. Urbana: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). Kuriakose, R. B., & Luwes, N. (2016). Student perceptions to the use of paperless technology in assessments–a case study using clickers. Procedia - Social and Behavioral Sciences. 228(2016), 7885. https://doi.org/10.1016/j.sbspro.2016.07.012. Lahad, N., Dafoulas, G., Kalaitzakis, E., & Macaulay, L. (2004). Evaluation of online assessment: The role of feedback in learner-centered e-learning Proceedings of the 37th Hawaii International Conference on System Sciences, 2004, 1-10. Marius, P., Marius, M., Dan, S., Emilian, C. & Dana, G. (2016). Medical students’ acceptance of online assessment systems. Acta Medica Marisiensis, 62(1), 30–32. https://doiorg/101515/amma-2015-0110 MAXQDA: The Art of Data Analysis (n.d.). What is MAXQDA. https://www.maxqda.com/what-is-maxqda Moni, K. B., van Kraayenoord, C. E., & Baker, C. D. (2002). Students’ perceptions of literacy assessment. Assessment in Education, 9(3), 319–342. Nuha, A., Wills, W., & Wald, M. (2018). Advantages and Challenges of Using eAssessment International Journal of Information and Education Technology, 8(1), 34-37. https://doiorg/10.18178/ijiet.2018.8.1.1008 Ogletree, A., Ogletree, S., & Allen, B. (2014). Transition to online assessments: A personal perspective of meeting common core state standards in an elementary school in Georgia. Georgia educational researcher, 2014, 11(1), 170–187. https://doiorg/10.20429/ger.2014.110107 Osuji, U. (2012). The use of e-assessments in the Nigerian higher education system. Turkish Online J Distance Educ, 13(4), 1–13. Richardson, C., & Fleer, M. (2003). Collective Mediated Assessment: Moving towards a Socio-cultural Approach to assessing children’s learning. Journal of Australian Research in Early Childhood Education, 10, 41-55. Ridgway, J., McCusker, S., & Pead, D. (2004). Literature review of e-assessment. University of Durham: Durham, England. Russell, C., & Shepherd, J. (2010). Online role-play environments for higher education. British Journal of Educational Technology, 41(6), 992-1002. https://doiorg/101111/j1467-8535200901048x Stodberg, U. (2012). A Research Review of E-Assessment. Assessment & Evaluation in Higher Education, 37(5), 591-604. https://doi.org/10.1080/02602938.2011.557496 Yates, R., & Beaudrie, B. (2009). The impact of online assessment on grades in community college distance education mathematics courses. American Journal of Distance Education, 23(2), 62–70. https://doi.org/10.1080/08923640902850601
http://ijlter.org/index.php/ijlter
139
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 139-159, August 2021 https://doi.org/10.26803/ijlter.20.8.9 Received Jun 09, 2021; Revised Aug 11, 2021; Accepted Aug 20, 2021
The Systematic Implementation of an Innovative Postgraduate Online Learning Model in the Middle East Nessrin Shaya American University in the Emirates, Dubai, United Arab Emirates https://orcid.org/0000-0003-4201-3945 Laila Mohebi Zayed University, Dubai, United Arab Emirates https://orcid.org/0000-0003-2640-4532 Abstract. This paper aims at examining the factors contributing to effective implementation of online learning in the Middle East higher education sector, through investigating the success and learning effectiveness of an innovative online learning model offered jointly by three prestigious universities in two different Arab countries. A mixed-method research approach was employed to triangulate data collected from key stakeholders engaged with the programme, namely senior managers and enrolled students, to derive findings that would inform managers, trainers and educators, from a systemic implementation, faculty-development and course-design perspective. Qualitative data gathered from face-to-face, semistructured interviews with a number of senior managers took place delineated on the necessary enabling conditions to create, design and offer a quality online programme. Quantitative data collected from a student satisfaction survey examined factors contributing to perceived learning and student satisfaction in an online learning context. Following thematic analysis, five major factors were identified as critical for effective online learning implementation, namely: Rationale and Motivating Factors, Technology Infrastructure, E-Learning Pedagogy and Support Infrastructure, Course Design and Delivery and Lessons Learned. Results showed that Course Structure/Organisation, Learner Interaction, Student Engagement and Instructor Presence appeared to be the major factors contributing to high satisfaction and perceived learning. This paper argues that the shift towards blended and online modes of learning is inevitable, advocating the fundamental conception that online education is instrumental in expanding access to tertiary education. If properly designed and implemented, online education has the potential to alleviate academic rigor through improved faculty productivity at reduced tuition costs. Implications for further research and practical recommendations are also discussed. Keywords: course delivery; course design; higher education; online education; perceived learning; student satisfaction; success factors ©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
140
1. Introduction While the education culture across the Middle East values campus-based learning, the shift towards blended and online modes of learning is inevitable. Despite reported benefits in holding a repertoire of learning opportunities, the absence of legal frameworks supporting recognition of online degrees is the biggest hurdle towards embracing online education (The Open University, 2021). An examination of the dominant features of education and academic institutions of the third millennium shows that chief characteristics are “flexibility, inclusiveness, collaboration, authenticity, relevance and extended institutional boundaries” (Yuksel, 2010, p. 1). Responsibilities of learners and lecturers have transformed significantly as educational goals have expanded to include selfdirected learning, digital literacy, continuous global dialogue, attainment of metacognitive skills and processes that comprise holistic curricula, critical thinkers and problem-solvers. Accordingly, higher education must undergo necessary changes to adapt the traditional educational structures to the uprising knowledge age, represented through the integration of information technology denoted by the ‘digital age’. This research advocates the fundamental conception that online education is instrumental in expanding access to tertiary education, yet, has the potential to alleviate academic rigor and standards through improving faculty productivity at reduced tuition costs (Carey & Trick, 2013; Meyer, 2012). Therefore, it focuses on an innovative online programme that is the first of its kind launched in the nation, offered in a joint collaboration between three American universities in Lebanon and Egypt, breaking down a number of cultural, social and political barriers. In a country where online education is yet to be approved and recognised, this programme is revolutionary for the Lebanese community (Malaeb, 2020), capturing the attention of significant stakeholders and official regulators, monitoring results and effectiveness, poising for further growth and supporting the uprising of the nation. Therefore, the current research understudy will focus on the Lebanese context. What is more, is the fact that only a small number of studies have been carried out in the Middle East to investigate the future prospects and barriers towards implementing online education (Baytiyeh, 2017; El Turk & Cherney, 2016). However, with the constant growth of the Web influences and changes in how online courses are designed and implemented, continuation of studies of learners’ perspectives of online learning environments are needed to build effective Webbased instruction that can optimise the learning experience within this everchanging landscape. This study seeks to add to the theoretical body of knowledge and literature, the experiences of online education in Lebanon, the prospects and challenges. Over the past years, distance education has remarkably altered the landscape of modern education, advancing significant changes in the offered learning opportunities. A myriad of literature ascertains that online education can emerge as a powerful socio-economic force in combating education inequalities and poverty and driving economic development (Burns, 2017; Li & Lalani, 2020). Despite reported benefits, institutional efforts to experiment with online modalities of teaching and learning were frequently assessed by the government
http://ijlter.org/index.php/ijlter
141
(El Ghali & Nauffal, 2020). Accordingly, this paper reports on an innovative online post-graduate engineering diploma programme that is considered to be revolutionary for Lebanon, monitored by many significant stakeholders (Baytiyeh, 2017) such as the Ministry of Education and Higher Education in Lebanon. This is done to better understand the readiness and commitment of students and the impact of the educational experience in terms of learning effectiveness, academic integrity and student satisfaction. The success of the programme will contextualise the country as a possible market for online education and poises for further growth and possible approval of online education. Accordingly, this study aims to present the first approved postgraduate online diploma as a foundational stage for potential growth of the industry in Lebanon. This mixed-method case study approach aims to answer the following research questions: - What are the main factors contributing to effective implementation of online learning in the higher education sector in Lebanon? - How satisfied were students with their fully online courses? - What factors contributed to students’ satisfaction with online learning? In an attempt to get on with global progress and support the current efforts, the findings will be framed and presented as a model on the most appropriate mode of implementation of online education in Lebanon.
2. Significance of Research At times where the West has adopted and implemented online learning since long time and is continuously experimenting with and trying out innovative models as learning solutions, till now Lebanon does not approve online education and applications for equivalency of online degrees are faced with refusal, despite how prominent and reputable the graduating universities are (Baytiyeh, 2017; El Turk & Cherney, 2016). A number of hindering factors towards implementation of online education have been reported, among which is the lack of trust in underlying teaching effectiveness and academic rigor, the absence of the Lebanese Quality Assurance Agency to audit quality and abundance of higher educational institutions, relative to a small country in geographical area like Lebanon (El Amine, 2017). Therefore, it is not surprising that investments in e-learning in the Middle East fall second to last behind Africa, equivalent to $683 million by 2016, whereas North America ranks first with $23 billion worth of investments in the same fiscal year (El Amine, 2017). The Ministry of Education and Higher Education has put forward some efforts in embracing online education, such as creating a taskforce of different expertise to draft a national strategy and create a framework of action to recognise and accredit distance learning, yet the most serious efforts remain from two leading American universities in Beirut and the Arab Open University, a branch of the online learning network (El Turk & Cherney, 2016). Shaya (2018) argues that primitive systems continue to override the work of higher education legislative academic decision-makers in Lebanon which is not on par with advancements happening in the education system worldwide, hence, denying institutions the chance to maintain a competitive advantage in a fastchanging market and provide better quality and access to education. Those
http://ijlter.org/index.php/ijlter
142
universities which do not embrace online education will be left behind in the race for globalisation and technological development. It is anticipated that the study of the effectiveness of current online practices will bring about the quality of online education and bring confidence among the regulatory authorities on the need to accredit and approve this revolutionary mode of education on par with the West. On the other hand, the perception offered by faculty members teaching at leading institutions in Lebanon has the potential to expand awareness among students and parents and lead to a change in the attitudes in perceiving the need for online learning. Finally, and most importantly, the results of the study present as an implementation model for private institutions on the best ways to deliver quality online instruction in the most cost-effective ways that can meet learners, market and business needs.
3. Theoretical Framework and Literature Review The current research understudy is part of a larger stakeholder analysis study that reports on the country’s readiness to accredit and implement online modes of teaching and learning. The stakeholders are mainly higher education students, faculty, executive leaders of private universities and the federal legislative authorities. It is governed by the theoretical conception of Aparicio et al. (2016) that perceives any successful e-learning system as a function of three main pillars: e-learning stakeholders, e-learning technologies and e-learning activities. Hence, this study presents the experience of the first postgraduate online programme as a foundational ground to build upon, making use of the lessons learned, bringing confidence in the capacity of private higher education institutions to carry out such innovative models and unravel the prospects and challenges. These elements guided the investigation seeking perceptions pertaining to faculty, students and developers as main stakeholders, the e-learning team in particular, to deeply understand the status of online learning technologies incorporated, the instructional designers and faculty to report on pedagogy and online activities and students to unravel the quality of learning and instruction versus expectations. The study is the first official postgraduate online programme to be offered in Lebanon. Eighteen credits are required towards the completion of the diploma, with quality of delivery and content sought to be equivalent to face-to-face instruction, as indicated by the Ministry of Education and Higher Education in Lebanon (Baytiyeh, 2017). Many internal policies and regulations govern the diploma such as admission and registration procedures, duration of the study, curriculum and syllabus. Hence, it is deemed necessary to document the systematic implementation of the programme and assess student satisfaction.
4. Related Work A number of international studies (Archambault & Crippen, 2014; Albrahim, 2020; Basilaia & Kvavadze, 2020; Fauzi & Khusuma, 2020) have documented academic rigor following online delivery. For instance, Shutimarrungson et al. (2014) studied the implementation of the constructivist model in an online learning course and results indicated that participants achieved high score achievement and high critical thinking skills on post-tests. Studying the nature of the learning
http://ijlter.org/index.php/ijlter
143
experience it was revealed that the activities challenged the learners to develop higher-order thinking skills and encouraged them to connect with former experience and respond to the diversity of the learning needs, which lead to authentic learning. This study, among many other studies in literature, supports the conception that pedagogy shapes the efficiency of online programmes, and if properly implemented can lead to significant achievement gains and better learning outcomes. Scrutinising the most appealing pedagogical features in online education, the literature reveals the following factors. First, teachers serve primarily as guiders and facilitators of learning, not instructors. Learning is learner-centred (Brookfield, 1995; Chen, 1997; Huang, 2002; Schell & Janicki, 2012; Spitzer, 1998; Wagner & McCombs, 1995). Second, learning takes the form of realworld scenarios emphasising authentic learning (Carwile, 2007; Doolittle, 1999; Jonassen et al., 1994; Koohang et al., 2009). Third, the social presence of students by way of incorporating negotiations, discussions and debates should also be considered (Chickering & Gamson, 1991; Chickering & Ehrmann, 1996). Fourth, content should be meaningful and students should build on their pre-existing knowledge. Fifth, students should be prompted to hold themselves accountable for their learning, hence, to become self-mediated and self-aware, given that a safe environment for questioning and learning is provided. Finally, teachers should provide and encourage multiple perspectives and representations of content (Chickering & Gamson, 1991; Chickering & Ehrmann, 1996). On the other hand, upon reviewing the literature, the trends for 2017 to 2018 (Jasmini, 2017) appear to be: - Contextualised learning, represented through modernisation of current elearning system designs and approaches. - Two-way conversation in e-learning, where learners’ needs will inform content rather than available resources or classical approaches. For instance, an ‘e-learning authoring’ tool could be used to fetch needs and elicit responses through sending out polls and questionnaires that will then in turn help to shape strategy and content. - Better use of data, where option such as ‘Elucidat’ can allow viewing analytics that would help in realising pitfalls and draw plans for personalising and improving learning content. - Extensive use of videos in terms of social learning and video learning in online courses. Options like ‘Elucidat’ and ‘Periscope’ provide a variety of features to build interactive videos and for better use in designing content. - Microlearning will be used to personalise e-learning content. - Social e-learning or ‘informal learning’ provided through e-learning experiences. - Mobile learning will continue to rise and learning content will be more accessible through mobile devices. - Performance support vs learning experience. 4.1 The Growing Market of E-Learning in the Middle East The 21st century in the Middle East has witnessed growing interest in e-learning through considerable investments in innovation leading to better e-learning
http://ijlter.org/index.php/ijlter
144
solutions, while the latter market revenue has risen to $560.7 million by 2016 (Edarabia, 2016). The e-learning and online education market was valued at $558.1 million in 2016 and is expected to reach $237.1 million by 2023. The Kingdom of Saudi Arabia holds the largest shares in the Middle East and North African (MENA) market, with expectations of earnings reaching $237 million by 2023 as well. As a result, the Middle East Online Education and E-Learning Market Size, Demand, Opportunity and Growth Outlook 2023 (Research and Markets, 2017) report has been issued from Dublin to reflect and accompany the spanned growth. The report presents historical market data for the previous year, and at the same time reflects revenue estimates and forecasts till 2023. The scope includes market trends, strategical management and development issues and so on. Internationally, recruitment for online education has achieved numerous gains and progress, with a reported enrollment of 308,000 at the University of Phoenix, leader of online learning in higher education, 78,000 at Kaplan University, and about 62 universities offering at least one course on Massive Open Online Courses (MOOCS) platforms, including Stanford (Bonvillan & Singer, 2013). 4.2 The Programme Under Study In October 2012, the main hosting university in Lebanon received a grant from the European Union to offer an online joint/dual professional diploma and a degree in green technologies (Uni-Med, 2021). The project originally aimed at widening access to Information and Communication Technology ICT through incorporating the use of new technologies in teaching and learning. Three main aims resulted, with the first the development of ‘Visio-conferencing’, then online courses and an online joint postgraduate diploma.. Three leading American universities in Lebanon and Cairo partnered in offering an international diploma. The funding process was completed by August 2016 and the diploma has currently three main specialisations, namely renewable energies, green buildings and water resources. The project aimed at developing programme structure and curriculum for a postgraduate degree in green technologies, creating a platform for the development of blended learning and a training faculty.
5. Methodology Charmaz (2006) defines grounded theory as “an inductive, comparative methodology that provides systematic guidelines for gathering, synthesising, analysing and conceptualising qualitative data for the purpose of theory construction” (p.2). Hence, an inductive grounded theory methodology has been utilised in the current understudy aiming at developing theoretical models with high levels of rigor. Therefore, the research design covered two main stages: (i) Qualitatively identifying the main themes, through thematic analysis, that contribute to learning effectiveness from programme design perspective, to delineate on the necessary enabling conditions to create, design and offer a quality online programme. Particularly, the opinions offered by the managing director of the programme, teaching faculty members, the e-learning team supporting the programme comprising the general manager, instructional designers and multimedia designers, were instrumental.
http://ijlter.org/index.php/ijlter
145
(ii) Quantitatively assessing the factors contributing to student satisfaction and perceived learning. Data was collected primarily through document analysis, interviews and surveys. Fifteen audio-recorded semi-structured interviews were carried out with professionals holding senior positions in the design, development and implementation stage of the programme. The subjects’ qualifications ranged between Masters’ and PhD holders who have worked closely on developing the programme. Four of the subjects were senior college Deans and a total of twelve were faculty with ranks ranging between assistant and full professors. Ten were females and five were males. In addition, among the interviewees were senior instructional designers and IT specialists. Interviews were first piloted, where interview question guides and procedures were verified and cross-checked by two academicians who are experts in the field, at the rank of full professor and associate professor, to ensure question rigor through avoiding any potential ambiguity, bias or leading questions. Each interview lasted between 60 and 90 minutes. There were 10 to 17 questions based on each interviewee’s position and the interview questions were categorised under main themes in an attempt to facilitate data analysis. A qualitative approach was adopted to identify emerging factors under the assigned main themes and finally data was scrutinised and interpreted iteratively leading to formation of codes and themes. Subjects’ responses were organised in a table form design comprising each subject’s code name, main themes, codes revealing subjects’ individual perceptions and corresponding quotes. Then, thematic analysis was utilised to look for patterns and themes across datasets. Findings took the form of locating emerging themes and sub-themes, where emerging codes were considered as potential themes in a manner where various codes merge towards forming dominating themes. On the other hand, for the purpose of data on student satisfaction, this research study employed the Student Learning and Satisfaction in Online Learning Environments (SLS-OLE) survey that was developed by Gray and DiLoreto (2016), which has been proven for validity and reliability. A convenience sampling approach was followed. A five point Likert scale for the questionnaire was used, prompting students to assign their level of satisfaction with the programme. The questionnaire was circulated electronically to the students enrolled in the online diploma programme. A total of sixty responses were collected, representing the majority of enrolled population with a 60% response rate , given that it is a new and unique programme. Data was analysed using SPSS to generate descriptive statistical results while being continuously cross checked to verify validity of responses. The questionnaire was first factor analysed, with minimal loading cutoff score to be 0.5 (Suliman 2001). The factors were successfully loaded scoring 0.59 and above on the varimax rotation. Knowing that the determinant of the R-matrix should be greater than 0.00001 (Field 2005), the determinant in this study was found to be .002, that is larger than the necessary value. The value of KMO is .733 meaning that factor analysis results in reliable factors (2005), and the Bartlett test appears to be significant with value .03, and accordingly factor analysis yielded reliable findings. Given the scarcity of available information as the diploma is the first of its kind in the country that is online and approved, and, while the whole nation has their eyes open, namely regulatory authorities and
http://ijlter.org/index.php/ijlter
146
different university councils, to understand the pre-requisites for successful implementation and lessons learned, every single response will be of great value and worth. Many studies acknowledge the importance of the online diploma programme understudy as a European funded project that will revolutionise the education system in Lebanon, hence, it is deemed important.
6. Demography of Respondents in Quantitative Data A total of 60 students responded with meaningful answers on the survey. Around 70% of them are of the age group 24 to 34 years old, one is below 24 and the remainder are above 35. The majority of the respondents were married and were males, around 82% and 75% of the total respectively, and the remaining were unmarried. As expected, the largest group lead full time jobs (88%), whereas the rest were equally distributed as either working a part-time job or were unemployed by the time data was collected. All respondents reported CGPA between 2.00 and 4.00, and three quarters of the group scored above 3.00 and the remainder below this. Finally, 82% were of Lebanese nationality, while 18% were non-Lebanese.
7. Qualitative Findings on Implementation of Programme Five main factors emerged as important to the implementation of the online learning programme in Lebanon, namely: -
Rationale and Motivating Factors Technology Infrastructure E-Learning Pedagogy and Support Infrastructure Course Design and Delivery Lessons Learned
A- Rationale and Motivating Factors Given the European project funding, the initial aim of the project was to develop a number of online courses that would lead to a certificate. However, the desire was high and findings from feasibility studies brought about a niche in the market requiring postgraduate diploma studies in green technologies, and accordingly the decision was taken. RR: There was no initial intention to develop a diploma, rather online courses in green energy, but aspirations were high and market research showed that there was a niche to develop this into a diploma. The leading American university hosting the programme has been offering blended learning delivery for 10 years, since 2008, and has the capacity through its e-learning team comprised of 12 experts to develop quality online modules. Three universities embraced the project, namely three American universities, two in Lebanon and another in Cairo. The international partners helped the three main universities in turning the fragmented parts into one full programme, which is offered completely online, to be the first in Lebanon and some of its neighbouring countries. RR: The main leading university has an advantage because they have a small group of specialised people instructional designers to help in assessment and testing in online education, do frequent testing rather than one midterm. They helped the faculty.
http://ijlter.org/index.php/ijlter
147
The diploma programme was launched even before the funding was complete. Presently, more than 27 online courses have been designed and delivered, 90% of students are new to the world of online education, students allocate at least 5 to 7 hours per week to study per course, and they are residents from different parts of the world. Accordingly, the regulatory authority has approved the programme and is is aware of the success of the online diploma programme RR: Our degree is almost equivalent to Masters’ courses, and the ministry is very attentive to it because we are the first. Offering a post-graduate online programme extends far beyond the online diploma programme for the main leading American university, where shifting to online modes of delivery is part of the university’s strategic planning. The university aims towards increasing the international student body and to increase student enrollment by half. Given available space and resources, going virtual is therefore one element of the Master Plan. Therefore, solid efforts are placed on intensifying e-learning resources, such as reliable cloud hosting, an e-learning platform and learning management system. WW: Our strategical plan is to increase enrollment by 50% without increasing the space, without increasing facility, and the only way is to go virtual. This one of the items in the Master Plan. Therefore, we are placing extensive efforts in our e-learning resources, on reliable cloud hosting stable, even though I think we have accomplished a success story since 2005. B- Technology Infrastructure From an operational perspective, the programme functions along with three main components: admission, registration and learning management system. Moodle and Blackboard are the main learning management systems utilised. To overcome the problem of slow and low quality of Internet service, the leading university had a British partner offering an external hosting cloud in addition to the Moodle partner. WW: Our institution has a team dedicated to implementing online courses, we contacted different companies, looked for Moodle partners, not only hosting cloud, so we took a SAS, software as a service, we contacted different companies and we selected a partner in UK. All the implementation of Moodle happens locally at the leading university, where every semester the online diploma programme has 10 to 15 courses running through Moodle, that are reliable and efficient, with no incidents registered such as hacking and server problems. The e-learning team acknowledges the importance of training for students, where upon enrollment, new students receive intensive training either through Skype or Adobe Connect. The support team is available almost all the time and issues are usually resolved within two hours. C- E-Learning Pedagogy and Support Infrastructure The concept of integrating technologies into teaching and learning is not new at the main leading American university, rather an extension of efforts that stretched for several years back. Training sometimes took the shape of a full programme that would last many weeks, on best practices, reading content, then finally
http://ijlter.org/index.php/ijlter
148
authoring their own courses that would meet certain objectives. At least twelve different faculty members are called for training every semester. Regarding the online diploma pedagogy, the fundamentals of online pedagogy are well governed. A number of specialised instructional designers, who are experts in the field of online education, are on the board. The designers’ duties, in collaboration with multimedia designers and e-learning system managers, are to: - Train faculty members in using e-learning technologies and associated pedagogies prior to any activity. - Through one-on-one and collective group meetings, to closely assist faculty members in desigining their courses, then to transform them into online learning modes.Test the courses and verify their quality prior to publishing and delivery. In that case, many courses were turned down because they were not cleared by the instructional designers. - Follow up on faculty members during the semester. Custom-designed training sessions, in addition to one-on-one support, is offered to every new faculty member joining the diploma for three to four months, whereby training and designing of the course takes place. In essence, almost three quarters of the course should be ready prior to offering it up for study. The instructional design team helps in finding the best methodologies for teaching the course, relying on the most suitable learning activities, graphics and assessment, and accordingly the syllabus might need to be modified and other types of interactive engagement assignments included. HH: Every time new faculty joining, individual work of 1-1 for about 3 – 4/months to train and design a course takes place. MM: They come with topic learning outcomes and syllabus, we assist them in methodologies of teaching, design activities and learning modules, we help to choose learning activities, assessment, etc. They write the preliminary syllabus and textbook then we work on redesigning syllabus to develop appropriate assessment, as they are used to classical quizzes and exams, whereas we train them on different kinds of assessment and interactive engagement documents. With time faculty members become proficient in online learning systems, where members reported better student evaluation reports and general internal satisfaction after teaching online courses for some time. D- Course Design and Delivery Reflections on course design and delivery were brought about through the perceptions of faculty members teaching online courses in the online diploma programme. The first faculty member is an associate professor in Engineering, with teaching experience of more than 10 years. He has been engaged in a blended learning approach at the leading university since 2013 and received a total training of 10 to 12 hours after joining the online diploma programme. The workshops were just a start, rather, it is the on-the-job training and exposure that reinforce the necessary skills to deliver high quality teaching. He teaches two courses, each two credit hours, and each requires around nine to 10 sets of lectures each lasting for about 40 minutes on average.
http://ijlter.org/index.php/ijlter
149
II: For each of these courses there is like 9 or 10 set of lectures, about 40 mins each. II: It is the OJT, on job training, when you start actually experimenting with online, there is a lot of learning that takes place. Content development happens locally, where the faculty member gets the syllabus with learning outcomes along with assigned topics. Then instructional designers will undergo some modifications on instructional strategies, replace paper books with e-books and alter assessment to be quiz-based into a set of different assessment techniques such as position paper, presentation, critique and others. With the help of instructional designers, the faculty prepares the material using a PowerPoint Presentation with very interactive slides, supplemented with videos, graphics, etc. Then with the help of multimedia specialists, voice over is recorded, then through a e-learning technologies manager, the slides would be turned into online material using specialised software. A simple link would allow students to download and access the slides. Regarding assessment, the faculty confirms that conducting online quizzes remains a problem in the diploma programme as it is difficult to verify the identity of the user taking the quiz and to check that cheating is not happening. However, what helps is the nature of the candidates who are pursuing the diploma for career advancement and self-satisfaction, hence any kind of plagiarism wouldn’t help them. In all cases, minor weight is assigned to online quizzes (20%) and the main weight is on assignments. The faculty gives about eight assignments every semester, in the form of mini-projects. They are asked to undertake some reading of a material or research/paper and then a report and exercises are then shaped so that students will have to comment on each other’s work. Eventually, they are learning and critiquing and students are interacting where questions go back and forth between them, while the faculty members monitor the interaction and interfere only when required. An assignment that initiates dynamic discussion and keeps the students engaged would help in meeting the learning outcomes, rather than the grade itself. Issues like monitoring attendance are not hard, as the system allows the faculty to keep track of the activity of the student, slides visited, how much time is spent and when it is necessary to initiate and contact the student regarding attendance. E- Challenges Faced and Lessons Learned Findings were obtained from different interviewees on the lessons learned over the three years since the diploma was offered at the university, as follows: (i) Offering online degrees (or diplomas) by any reputable university takes long extensive effort, and as one senior executive mentioned “it might look like something small, but the effort is like introducing a PhD programme. The difficulty lies in meeting local, regional and international registration and accreditation of enrolling universities, which is almost equivalent to reaccreditation”. Hence, from an administrative perspective, serious commitment is required. (ii) GG: We worked on the administration to make sure the team in our university is doing its part in admission and registration, it might look something small, but the effort is
http://ijlter.org/index.php/ijlter
150
like introducing PhD program, getting IDs, online tuition fees, everything online, to get all this done banner oracle financial system huge amount of money.Despite its operational difficulties, the online diploma is very profitable. It has achieved profit from the first day of launching, indicating a good market for the online diploma, although it is typically new. (iii) GG: This is a diploma that made money from the very first semester, it actually made profit, making good money for all the 3 partners. Tuition fees is basic, very affordable, the way we designed it, the way we shared our resources, the university charges overhead and yet we are still making money. It is a very good opportunity.The presence of strong technological infrastructure is essential to ensure smooth automated progression between admission, registration and course navigation. WW: All transitions between the three units Admission, Registration and Moodle is automated, and the units are from the leading university staff, who are experienced with more than 15 years of experience to increase chances of success, hence there should be a reliable hardware and software information technology. (iv) The presence of specialised instructional designers in abundance is a must, where their main role is faculty development and to provide course design support. Training faculty members might seem brief, however, despite how knowledgeable they are, intensive training on teaching methodologies, formulation of their teaching identity and proper follow up on the students is hard to reach. (v) Ample time of no less than three to four months should be given to prepare, test and pilot online courses before they are up and running. On average 20 to 30 slides will require 18 hours of extensive work to prepare, then three hours for audio -recording and two hours for post-production, i.e. publishing and posting. In addition, expectations should be set from the beginning for faculty members regarding due dates, deliverables and consequences if the commitment isn’t fulfilled. HH: Give the course enough ample time, minimum 3-4 months of preparation and testing, we pilot it user-testing, we do that with our media specialist. It takes long to develop interactive quality lectures. (vi) Faculty members’ social presence is very important and is underestimated, otherwise there is a risk of isolation of learners and low completion rates of courses. Online learning students need to continuously feel the presence of faculty members. (vii) A reliable auditing system helps in controlling quality, where auditors are assigned to each online course and conduct satisfaction and progress surveys twice in a semester, in the middle of the semester and towards the end. DD: Each course has assigned auditor. They conduct surveys middle of the semester and by the end. Auditors make sure learning outcomes are met, and whether students are properly progressing. (viii) Carrying out live sessions, whether for the purpose of synchronised learning or meeting the students in chat rooms and getting to know them better, appears to be effective in strengthening the faculty member-learner bond. (ix) The personal effort required from a faculty member to teach an online course is by far greater compared to face-to-face instruction. Hence, monetary
http://ijlter.org/index.php/ijlter
151
rewards as extrinsic factors will help, however, the faculty member should have intrinsic motivational factors to ‘buy-in’ to online courses and diminish resistance and reluctance. KK: Yes, a lot, I am suffering in the diploma, though it is my second year of teaching online. Each semester I am changing the course. The courses I am teaching here are not introductory, rather advanced and not available. So, I have to always search for ways to make students very interested in the subject. It is nice, but very challenging. Switching online by itself is not hard but switching and making it interactive and interesting is not easy. (x) Not all students are digital natives and ‘tech-savvy’. YY: My lessons learned that never assume that students are digital natives. Many times we take it for granted they feel at ease with technology, not all students do. We assume, if we give them any online platform they will naturally figure out what to do, that’s not the case. Whether we like it or not there is a digital divide, especially in Lebanon.
8. Results of Student Satisfaction Survey The questionnaire aims to assess student learning and satisfaction in the first and only accredited online programme in Lebanon, offered by partnering American universities in Lebanon and Cairo. Around 100 students were enrolled in total by the time the study was conducted. The students’ learning and satisfaction were measured based on six dimensions, namely: Course Structure/Organisation, Learner Interaction, Student Engagement, Instructor Presence, Student Satisfaction and Perceived Learning, and comprising 34 questions. The results are as follows: A- Course Structure/Organisation: Students’ rating of the organisation of the course and structure yielded a mean of 4.15 and a standard deviation of .65. Strength appeared in clear statement of learning outcomes, ease of navigation and organised layout of course, where means were 4.25, 4.31 and 4.35 respectively. Moderate satisfaction was in the articulation of clear instructions on student participation and alignment of learning outcomes with activities. Table 1: Course Structure/Organisation Results A- Course Structure/Organisation During your diploma courses, 1 student learning outcomes was aligned to the learning activities. 2 Course navigation was logical. The layout of the course was 3 organised. Instructions about student 4 participation were clearly presented. The purpose of the course was 5 clearly presented. Total
http://ijlter.org/index.php/ijlter
Mean
SD
3.9375
0.57
4.3125
0.79
4.4375
0.63
3.8125
0.65
4.25
0.57
4.15
0.65
152
B- Learner Interaction: The overall rating of student satisfaction on interaction was moderate to high with total mean of 3.83. Items such as on opportunities to introduce oneself and the chances of engaging in active participation rated relatively high, but the rest of the items rated moderate with the least pertaining to exchange of peer comments. In general, social interaction scored moderate to high. Table 2: Learner Interaction Results B- Learner Interaction I frequently interacted with other 6 students in the course. There were opportunities for active 7 learning in this course. The learning activities promoted 8 interaction with others. I had the opportunity to introduce 9 myself to others in the class. I communicated often with other 10 students within the course. I regularly communicated with the 11 instructor of the course. I received ongoing feedback from 12 my classmates. Total
Mean
SD
3.97
0.81
3.94
0.57
3.88
0.95
4.72
0.79
3.89
1.01
3.07
0.92
3.00
0.81
3.84
0.84
C- Student Engagement The whole factor as a total scored moderate with a mean of 3.1125 and standard deviation .90. The mean in the below table indicates that the least scoring item was the learner-instructor interaction with a low mean of 2.88, and the highest was in the learner-content interaction with a high mean of 4.19, implying dedication and commitment from the students’ side, but lack of engagement from the faculty members’ side. Table 3: Student Engagement Resul ts C- Student Engagement I frequently interacted with my 13 instructor of this course. I discussed what I learned in the 14 course outside of class. I completed my readings as 15 assigned during the course. I participated in synchronous 16 and/or asynchronous chat sessions during the course. I was actively engaged in the 17 activities required in the course. Total
http://ijlter.org/index.php/ijlter
Mean
SD
2.88
0.80
3.63
0.88
3.69
0.94
3.32
0.79
4.19
0.75
3.55
0.89
153
D- Instructor Presence Student rating on instructor presence in total was 3.2 indicating moderate satisfaction levels. The five items measuring this factor rated between low (mean 2.0) to moderate and moderate-high (3.7). Satisfaction on items pertaining to instructor feedback from assignments was the highest, however, the quality of the comment was low. The feeling of being properly followed up on by the instructor also rated moderate in satisfaction. Table 4: Instructor Presence Results D- Instructor Presence The instructor’s feedback on 18 assignments was clearly stated. The instructor's feedback on 19 assignments was constructive. The instructor provided timely 20 feedback about my progress in the course. The instructor cared about my 21 progress in this course. I learned from the feedback that 22 was provided during the course. Total
Mean
SD
3.82
0.40
2.07
0.57
3.25
0.85
3.19
0.83
3.69
0.60
3.2
0.65
E- Satisfaction The overall satisfaction of the e-learner who took the survey was high with a mean of around 4.00. They appeared to be highly satisfied with the learning content and would recommend it to their peers. Less satisfaction was in the instructor himself/herself and student-student interaction. Table 5: Student Satisfaction Results E- Student Satisfaction I am satisfied with my overall 23 experience in this course. I would recommend this course to 24 other students. I am satisfied with the level of 25 student interaction that occurred in the course. I am satisfied with my learning in 26 the course. I am satisfied with the instructor of 27 the course. I am satisfied with the content of 28 the course. Total
http://ijlter.org/index.php/ijlter
Mean
SD
3.98
0.71
4.57
0.81
3.97
1.09
4.00
0.73
3.97
0.51
4.00
0.36
4.05
0.65
154
F- Perceived Learning The total mean of items on students’ perceived learning was 4.00. High scoring appeared in satisfaction with learning, learning tasks and understanding of content, acquisition of skills for their future and career. Learning, however, was not much better than what was anticipated. Table 6: Perceived Learning Results F- Perceived Learning I am pleased with what I learned in 29 the course. The learning tasks enhanced my 30 understanding of the content. I learned more in the course than I 31 anticipated. I learned skills that will help me in 32 the future. The learning activities promoted 33 the achievement of student learning outcomes. The course contributed to my 34 professional development. Total
Mean
SD
4.17
0.44
3.94
0.44
3.99
1.19
4.07
0.57
3.75
0.57
4.125
0.88
4.01
0.68
9. Discussion of Findings Findings revealed that five main factors emerged as important to the implementation of the online learning programme in Lebanon, namely: Rationale and Motivating Factors, Technology Infrastructure, E-Learning Pedagogy and Support Infrastructure, Course Design and Delivery and Lessons Learned. The rationale behind offering the online diploma programme was significant in better understanding the current context of universities in Lebanon. It brought forth serious attempts to consider online education as a strategic objective and current experimentation to reach ultimate successful modes of implementation. In their widely spread work, “Barriers to Adoption of On-Line Learning Systems in US Higher Education,” Bacow et al. (2012) highlighted four main rationales among American universities to offer online courses and programmes. The leading American universities in Lebanon are in line with two of the aforementioned rationales, namely “the desire to generate new revenue streams by reaching students who wouldn’t otherwise enroll in traditional programs” (p. 9) such as international students, and “responding to space constraints” (p. 12) through avoiding expanding or creating expensive additional facilities. On the other hand, improving learning outcomes and increasing retention didn’t surface in the conducted interviews as Lebanon is still far from considering online education as a viable option that can generate better learning outcomes and hence become a strategic objective. The presence of online education in the leading university’s Master Plan provides clear support at the level of leadership and belief in the value of online learning. The diploma programme places a major focus on the social presence of faculty members, reinforced through high levels of student– faculty interaction, which is another attestation of the close follow up of the
http://ijlter.org/index.php/ijlter
155
managing team on the progress and satisfaction of students as it is highly correlated to learning and retention rates (Drouin et al., 2015; Govindesamay, 2012), and a fundamental pillar of online education best practices (2015). The team’s fear of isolated and unhappy learners, hence, calling for more social interaction, concurs with leading universities’ attempts, such as SUNY (Pickett et al., 1999). Analysis revealed high technology infrastructure and technical capacities among addressed institutions where the managing team was able to overcome all anticipated problems prior to launching the programme, revealing the operational feasibility of implementation. Among the major contributing factors is the resourcefulness of the leading university, equipped with all the needful resources, from qualified faculty, strong technology capacity that was able to link the processes of admission, registration and course offering among the three universities, to open educational resources with a helpful digital library and experienced instructional designers. The programme was not only able to overcome perpetuated challenges, but also documented barriers in Lebanon pertaining to inconsistent Internet access and poor network (El Turk & Cherney, 2016), doubts in the feasibility of implementation (Nasser & Abouchedid, 2005), resistance among faculty members to teach online (Tarhini et al., 2013), general doubt in perceived usefulness (Nasser and Abouchedid, 2005), and so on. Four main factors contributed to high student learning and satisfaction, namely: Course Structure and Organisation, Learner Interaction, Student Engagement and Instructor Presence. These results are in line with SUNY findings, where the main factors impacting satisfaction and learning effectiveness could be summarised as: faculty-student interaction, peer interaction, social presence and resolving individual difficulties through proper course structure and strong technical infrastructure. Findings reveal that the Lebanese higher education system is on the right track towards paving the way for more creative, innovative and advanced growth in the field of online education in Lebanon. And as the e-learning team manager said during the interview, “we don’t wait for the ministry of education to come to us, we go to them.”
10. Strategical Recommendations on Modes of Implementation This research reported on the prospects and barriers in designing and implementing an innovative online learning model in the Middle East that proved to be attractive, not because of the machine-guided online instruction element, rather due to the potential of offering improved learning outcomes while bending the cost curve, poising for the growth in the industry. As leaders are contemplating the future of education, particularly given the rapid increase in adoption of online technologies worldwide, the below presents recommendations for universities aiming at developing online learning programmes in the near future. (i)
The presence of strong technological infrastructure is essential to ensure smooth automated progression between admission, registration and course navigation.
http://ijlter.org/index.php/ijlter
156
(ii)
(iii)
(iv)
(v)
(vi)
(vii)
(viii)
The presence of specialised instructional designers in abundance is a must, to extensively train faculty members, design faculty development programmes and provide course design support. Ample time of no less than three to four months should be given to prepare, test and pilot online courses before publishing. On average 20 to 30 slides will require 18 hours of extensive work to prepare, three hours for audio recordingand two hours for post-production, i.e. publishing and posting. In addition, expectations should be set from the beginning for faculty members regarding due dates, deliverables and consequences if the commitment isn’t fulfilled. Faculty members’ social presence is very important and is underestimated, otherwise there is a risk of isolation of the learner and low completion rates of courses. Online learning students need to continuously feel the presence of faculty members. A reliable auditing system helps in controlling quality, where auditors are assigned to each online course and conduct satisfaction and progress surveys twice in a semester, in the middle of the semester and towards the end. Carrying out live sessions, whether for the purpose of synchronised learning or meeting the students in chat rooms and getting to know them better, appears to be effective in strengthening the faculty member-learner bond. The personal effort required from a faculty member to teach an online course is by far huge compared to face-to-face instruction. Hence, monetary rewards as extrinsic factors will help, however, the faculty member should have intrinsic motivational factors to ‘buy-in’ to online courses and diminish resistance and reluctance. Student readiness may seem high, but not all students are tech savvy, hence, students’ development programmes are a must.
11. Conclusion, Limitations and Future Research This study aims to present the first approved postgraduate online diploma as a foundational stage for potential growth of the industry in Lebanon. It explores the main factors leading to effective implementation of online program in the higher education sector in Lebanon, students’satisfaction with online learning, and contributing factors to high satisfaction and perceived learning. It has brought forward findings that are critical in the Lebanese context as it presents a departure from the current pervasive classical modes of delivery, overcoming many obstacles documented in literature. Despite known challenges relevant to poor Internet and power quality, lack of funds to finance such projects and fundamental reluctance to teaching online courses not owned by faculty, the mentioned online programme was able to highlight and capture official and public attention. Moreover, despite public doubt regarding the feasibility of implementation of any online programme, the mentioned online programme showed high levels of satisfaction and perceived learning among e-learners. Thus, it could be argued that implementation of online education at undergraduate or post-graduate level is feasible. The rationale and motivation factors, lessons learned from the initial developmental and implementation phases along with the student satisfaction survey will present as a critical report on what has been
http://ijlter.org/index.php/ijlter
157
achieved in online education at reputable universities in Lebanon. Findings revealed that five main factors emerged as important to the implementation of the online learning programme in Lebanon, namely: Rationale and Motivating Factors, Technology Infrastructure, E-Learning Pedagogy and Support Infrastructure, Course Design and Delivery and Lessons Learned. A number of factors appeared to contribute to the high student learning and satisfaction, namely: Course Structure and Organisation, Learner Interaction, Student Engagement and Instructor Presence. Findings allowed for the proposal of strategical recommendations on modes of implementation that would optimise potential success pertinent to pedagogy, technology infrastructure, quality control system and key stakeholders’ perceptions and readiness. Hence, this study suggests that future research would further investigate online learning in Lebanon in light of that. Moreover, this study recommends including participation of a larger sample of subjects in the student satisfaction survey, while qualitatively re-examining senior stakeholders’ opinions post Covid-19.
12. References Abouchedid, K., & Eid, G. M. (2004). E‐learning challenges in the Arab world: revelations from a case study profile. Quality Assurance in Education, 12(1), 1527. https://doi.org/10.1108/09684880410517405 Albrahim, F. A. (2020). Online Teaching Skills and Competencies. TOJET: The Turkish Online Journal of Educational Technology, 19(1), 9-20. Basilaia, G., & Kvavadze, D. (2020). Transition to Online Education in Schools during a SARS-CoV-2 Coronavirus (COVID-19) Pandemic in Georgia. Pedagogical Research, 5(4). doi:https://doi.org/10.29333/pr/7937 Fauzi, I., & Khusuma, I. H. (2020). Teachers’ Elementary School in Online Learning of COVID-19 Pandemic Conditions. Jurnal Iqra’ : Kajian Ilmu Pendidikan, 5(1), 58-70. https://doi.org/10.25217/ji.v5i1.914 Aparicio, M., Bacao, F., & Oliveira, T. (2016). An e-Learning Theoretical Framework. Journal of Educational Technology & Society, 19(1), 292-307. https://www.jstor.org/stable/jeductechsoci.19.1.292 Bacow, L. S., Bowen, W. G., Guthrie, K., M., Lack, K. A., & Long, M. P. (2012). Barriers to Adoption of Online Learning Systems in U.S. Higher Education. New York: Ithaka. Basilaia, G., & Kvavadze, D. (2020). Transition to Online Education in Schools during a SARS-CoV-2 Coronavirus (COVID-19) Pandemic in Georgia. Pedagogical Research, 5(4). https://doi.org/10.29333/pr/7937 Baytiyeh, H. (2017). The flipped classroom model: when technology enhances professional skills. The International Journal of Information and Learning Technology, 34 (1), 51-62. https://doi.org/10.1108/IJILT-07-2016-0025 Brookfield, S. (1995). Adult learning: An overview. International encyclopedia of education, 10, 375-380. Burns, M. (2017). Online Education Business as a Tool to Boost Economy. Business Community. https://www.business2community.com/tech-gadgets/onlineeducation-business-tool-boost-economy-01972787 Carey, T., & Trick, D. (2013). How online learning affects productivity, cost and quality in higher education: An environmental scan and review of the literature. Higher Education Quality Council of Ontario. Carwile, J. (2007). A constructivist a roach to online teaching and learning. Inquiry, 12(1), 68-73. https://eric.ed.gov/?id=EJ833907
http://ijlter.org/index.php/ijlter
158
Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. Sage. Chen, L. L. (1997). Distance delivery systems in terms of pedagogical considerations: A reevaluation. Educational Technology, 37(4), 34-37. https://www.jstor.org/stable/44428405 Center for Education Innovations. (2016). https://www.educationinnovations.org/p/center-for-education-innovationscei Chickering, A. W., & Ehrmann, S. C. (1996). Implementing the seven principles: Technology as lever. AAHE bulletin, 49, 3-6 Chickering, A. W., & Gamson, Z. F. (1991). Appendix A: Seven principles for good practice in undergraduate education. New directions for teaching and learning, 1991(47), 6369. Doolittle, P. E., & Camp, W. G. (1999). Constructivism: The career and technical education perspective. Journal of vocational and technical education, 16(1), 23-46. https://eric.ed.gov/?id=EJ598590 Drouin, M., Stewart, J., & Gorder, K. V. (2015). Using methodological triangulation to examine the effectiveness of a mentoring program for online instructors. Distance Education, 36(3), 400-418. https://doi.org/10.1080/01587919.2015.1081735 Edarabia. (2016). Innovating Education Technology in the Middle East. https://www.edarabia.com/115413/innovating-education-technology-in-themiddle-east/ El Amine, A. (2017). Culture of law at Arab universities. Contemporary Arab Affairs, 10(3), 392-407. https://doi.org/10.1080/17550912.2017.1350368 El Ghali, H., & Nauffal, D. 2020. COVID19: Is Lebanon Ready for Online Higher Education. https://www.aub.edu.lb/ifi/news/Pages/20200330-covid19-islebanon-ready-for-online-higher-education.aspx El Turk, S., & Cherney, I.D. (2016). Perceived online education barriers of administrators and faculty at a US university in Lebanon. Creighton Journal of Interdisciplinary Leadership, 2(1), 15-31. https://eric.ed.gov/?id=EJ1152181 Field, J. (2005). Social capital and lifelong learning. Policy Press. Gray, J. A., & DiLoreto, M. (2016). The effects of student engagement, student satisfaction, and perceived learning in online learning environments. International Journal of Educational Leadership Preparation, 11(1), n1. https://eric.ed.gov/?id=EJ1103654 Huang, H. M. (2002). Toward constructivism for adult learners in online learning environments. British journal of educational technology, 33(1), 27-37. https://doi.org/10.1111/1467-8535.00236 Jonassen, D.H., Campbell, J.P., & Davidson, M.E. (1994). Learning with media: Restructuring the debate. Educational technology research and development, 42(2), 3139. https://www.jstor.org/stable/30218685 Jasmini. (2017). Online Learning Statistics And Trends. https://elearningindustry.com/online-learning-statistics-and-trends Koohang, A., Riley, L., Smith, T., & Schreurs, J. (2009). E-learning and constructivism: from theory to application. Interdisciplinary Journal of E-learning and Learning Objects, 5, 91-109. https://www.learntechlib.org/p/44824/ Li, C., & Lalani, F. (2020). The COVID-19 pandemic has changed education forever. World Economic Forum. https://www.weforum.org/agenda/2020/04/coronaviruseducation-global-covid19-online-digital-learning/
http://ijlter.org/index.php/ijlter
159
Meyer, K. A. (2012). The influence of online teaching on faculty productivity. Innovative Higher Education, 37(1), 37-52. https://link.springer.com/article/10.1007/s10755011-9183-y Malaeb, A. (2020). Amid coronavirus, students forced online, but Lebanon won't recognize online degrees. https://english.alarabiya.net/features/2020/12/11/Lebanoncrisis-Amid-coronavirus-students-forced-online-but-Lebanon-won-t-recognizeonline-degrees Nasser, R., & Abouchedid, K. (2005). Graduates' perception of university training in light of occupational attainment and university type: The case of Lebanon. Education+ Training. Pickett, A., & Shea, P. (1999). The evolution of faculty development and course design at the SUNY Learning Network. In WebNet World Conference on the WWW and Internet (pp. 1393-1394). Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/p/7721/ Research and Markets. (2017). Middle East Online Education and E Learning Market Size, Demand, Opportunity & Growth Outlook 2023. https://www.researchandmarkets.com/reports/4326517/middle-east-onlineeducation-and-e-learning Schell, G.P., & Janicki, T.J. (2012). Online course pedagogy and the constructivist learning model. Journal of the Southern Association for Information Systems, 1(1), 26-36. http://dx.doi.org/10.3998/jsais.11880084.0001.104 Shaya, N. M. (2018). Adoption and Implementation of Online Learning Systems in Lebanon: Prospects and Barriers (Doctoral dissertation, The British University in Dubai (BUiD)). https://bspace.buid.ac.ae/handle/1234/1291 Shutimarrungson, W., Pumipuntu, S., & Noirid, S. (2014). A model of e-learning by constructivism approach using problem-based learning to develop thinking skills for students in Rajabhat University. Educational Research and Reviews, 9(21), 11651172. https://doi.org/10.5897/ERR2014.1800 Spitzer, D.R. (1998). Rediscovering the social context of distance learning. Educational Technology, 38(2), 52-56. https://www.jstor.org/stable/44428438 Suliman, A. M. (2001). Work performance: is it one thing or many things? The multidimensionality of performance in a Middle Eastern context. International Journal of Human Resource Management, 12(6), 1049-1061. https://www.tandfonline.com/doi/abs/10.1080/713769689 Tarhini, A., Hone, K., & Liu, X. (2013). User acceptance towards web-based learning systems: Investigating the role of social, organizational and individual factors in European higher education. Procedia Computer Science, 17, 189-197. https://doi.org/10.1016/j.procs.2013.05.026 The Open University. (2021). Annual Report. http://www2.open.ac.uk/about/annualreport-2018-19/ Wagner, E.D., & McCombs, B.L. (1995). Learner centered psychological principles in practice: Designs for distance education. Educational technology, 35(2), 32-35. https://www.jstor.org/stable/44428961 Yuksel, U. (2010). Integrating curriculum: Developing student autonomy in learning in higher education. Journal of College Teaching & Learning (TLC), 7(8). https://doi.org/10.19030/tlc.v7i8.138
http://ijlter.org/index.php/ijlter
160
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 160-175, August 2021 https://doi.org/10.26803/ijlter.20.8.10 Received Apr 29, 2021; Revised Jul 22, 2021; Accepted Jul 31, 2021
Autonomy and Creative Thinking Skills of Prospective Elementary School Teacher Students in Learning Mathematics with Science Phenomena assisted by the Learning Management System Suprih Widodo School of Post Graduate, Universitas Pendidikan Indonesia, Bandung, Indonesia https://orcid.org/0000-0003-2709-4804 Turmudi School of Post Graduate, Universitas Pendidikan Indonesia, Bandung, Indonesia https://orcid.org/0000-0001-7976-211X Rizki Rosjanuardi School of Post Graduate, Universitas Pendidikan Indonesia, Bandung, Indonesia https://orcid.org/0000-0001-5280-6901
Abstract. This research intends to explain the autonomy and creative thinking skills of prospective elementary school student teachers in mathematics with the learning management system (LMS)-assisted science phenomenon. The data collection was done by using test instruments in the form of essay questions to capture the creative thinking skills and a non-test in the form of a questionnaire was used to capture the autonomy skills of prospective elementary school student teachers. The data were analysed through several stages, namely data reduction, data presentation, data conclusions, and data validation using triangulation. The result of the research shows that using the LMS can improve the creative thinking skills of prospective elementary school teachers, while their autonomy skills in experimental classes and control classes indicate no difference. This happens because students were given the same opportunity to explore the material independently. Thus, a way to develop autonomy through mathematics learning needs to be explored. Keywords: autonomy; creative thinking skills; learning management system; science phenomenon
1. Introduction Principals in Indonesia have indicated that the new graduates of educators have
©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
161
low levels of competence (Widodo, 2016b). Teachers with teaching experience also have low levels of pedagogical and professional competencies (Giarti, 2016). In addition to having hard skills, the competence of elementary school teachers must also include soft skills and 21st century skills (KA21) which are often referred to as the 4Cs, namely critical thinking, collaboration, communication and creativity that are useful in their personal and professional lives (P21, 2011). In mathematics learning, the Partnership for 21st Century Skills also defines the skills of the 21st century through a 21st century skill mathematics map in the field of learning and innovation skills for K12 consisting of creativity, critical thinking, communication and collaboration (P21, 2011). Based on the literary search, not much research in mathematics learning has been done related to the development of 21st century skills. Nevertheless, Salehudin et al. (2015) explained that through mathematics learning educators can provide support for students to master the skills of the 21st century. Mathematics learning research in Indonesia and some countries is still focused on mathematical skills as proposed by the National Council of Teachers of Mathematics (NCTM) (2000a). These skills consist of five process skills that students must master through mathematics learning, namely (i) problem solving (Mushlihuddin & Nurafifah, 2018; Maslukha et al., 2018; Fuchs et al., 2004); (ii) reasoning and proof (Lestari, 2019; Kramarski & Mevarech, 2003); (iii) connections (Nabilah, 2019; Fauzi, 2015); (iv) communication (Fauzi & Priatna, 2019; Brendefur & Frykholm, 2000; Buchholz, 2005) and (v) representation (Effendi, 2012). Two research studies in mathematics learning related to 21st century skills have been conducted by Turmudi et al. (2017) and Afifah (2019). In the research by Turmudi et al. (2017), they applied mathematics learning with didactic science to develop 21st century skills consisting of critical thinking skills, creativity, communication and collaboration. Afifah (2019), on the other hand, developed the 21st century skills of grade VIII students in mathematics learning with projectbased learning. Another 21st century skills development was carried out by Muhajir (2018) who trained 21st century skills in physics learning and other experiments in Malaysia by testing several 21st century skills in the fourth grade (Salehudin et al., 2015). This research will discuss the development of creative thinking skills as one aspect of the 21st century skills. Creative thinking skills are related to the skill of discovering new things that have not existed before and are original, developing new solutions to each problem, and involving the ability to generate new, varied, and unique ideas (Chiam, 2014). Mathematics as an important knowledge mastered by students can also contribute to the development of 21st century skills. However, in reality, many consider mathematics to be an area of knowledge that is often regarded as a difficult and intimidating lesson for students (Ashcrafts, 2002; Siregar, 2017). This is caused by 10 factors (Widodo, 2016a), among which is the fact that "there are not many mathematics books published in Indonesia that present problems in the form of context, as a result, mathematics feels abstract and difficult to learn". He
http://ijlter.org/index.php/ijlter
162
also explained that mathematics is considered difficult because of the perception of the student him- or herself who considers mathematics as a difficult lesson. Many parents impart to their children the notion that mathematics is difficult so that from childhood until adulthood, they have the perception that mathematics is difficult and intimidating. This relates to self-regulation, anxiety levels and student pressures studied in relation to psychological well-being. The concept of psychological well-being was originally introduced by Bernice Neugarten in 1961. She stated that psychological well-being is a psychological condition at a time when a person reaches old age (Tobin & Neugarten, 1961). The two main approaches to understanding psychological well-being, according to Ryff (2013), are, firstly, the focus on happiness in the sense of achieving happiness and preventing pain; and the second approach is the awareness of being an overall functional or whole person, including a healthy body and state of mind (Stern, 2007). Research on the well being of children has been conducted in several parts of the world. In the United Kingdom (UK), child welfare was researched (Rees et al., 2010) while in the United States (US), the well-being of children was the topic of research (Huebner & McCullough, 2010). This was also the case in Indonesia (Muqodas, 2019); however, mathematics education was not included. Taking into consideration the background above and the Covid-19 pandemic, this research adopted a 21st century learning approach. There are nine principles related to 21stcentury learning styles, namely i) being relevant to student life; ii) applying cross-disciplinary learning; iii) developing critical thinking skills; iv) encouraging the transfer of knowledge; v) teaching students how to learn; vi) conveying misunderstandings directly; vii) collaborating; viii) exploiting technology to support learning and ix) developing student creativity (Savedra & Ofter, 2012). As an implementation of the 21st century learning style, this research integrated technology and mathematical learning approaches using scientific phenomena to develop creative thinking skills and autonomy. In 2000, the National Council of Teachers of Mathematics (NCTM) made recommendations for integrating technology in mathematics learning through technology principles (NCTM, 2000b). The technology principle states that technology is important in mathematics learning and teaching activities since it influences how mathematics is taught and improves student learning activities. Furthermore, the NCTM states that technology can facilitate the process of solving mathematical problems, communication, reasoning and evidence. This is in line with the research of Ayers et al. (1988). Many research findings conclude that technology is the most suitable instrument to develop mathematics learning and teaching activities. The activity of writing, processing, and studying computer algorithm outputs can improve the understanding of mathematical concepts and principles, numeracy skills, and problem-solving abilities of students (Hatfield & Kieren, 1972). Technology is the infrastructure used in mathematics education (Kaput et al., 2019). Furthermore, they contend that to realize the technological potential, a new pedagogical method
http://ijlter.org/index.php/ijlter
163
is needed, and teacher development needs to be conducted. Kaput et al. (2019) also shared the results of their research on classroom connectivity (CC), namely classes involving technology. However, the use of technology in learning mathematics is still a rare occurrence. Some people think that learning mathematics using technology is not effective enough to increase the passion for learning. Usually, the technology used in learning mathematics comprises Cabri Geometry (Kösa et al., 2002), Kahoot for quizzes (Göksün & Gürsoy, 2019; Prieto, 2019), Maple (Blyth & Labovic, 2009), SPSS to process data (Xiao et al., 2015), calculators (Penglase & Arnold, 1996) and PhotoMath for proof (Hamadneh & Al–Masaeed, 2015). However, it is still rare for researchers to consider the use of mathematics learning technology for distance learning. It needs to be borne in mind that all students globally who are impacted by Covid-19 must learn remotely. Therefore a remote management system called the learning management system (LMS) is necessary. LMS is a management system for distance learning, online learning or e-learning. It enables students to study any time and any where. In an emergency, LMS functions as a distance learning, online learning or e-learning infratstructure By using LMS, teachers and lecturers can monitor student activities through a student attendance management system, provide diverse learning resources (text, video, audio, animation) and connect to other sources on the Internet such as structured assignments and quizzes with interesting forms, as well as enabling discussion forums. Furthermore, LMS can provide tools for collaboration and for both asynchronous learning (teachers and students access the system at different times) and synchronous (teachers and students access the system together and real-time) so that they can converse directly (Widodo, 2017). Studies in the use of LMS in learning have been carried out in universities by Bealty and Ulasewicz (2006), Kelley et al. (2016), McGill and Klobas (2009) and Weaver et al. (2008). However, research reports on the use of LMS in the field of mathematics learning are still rare, considering some of the limitations of the features in LMS that can be used. The basis of the selection of scientific phenomena in this research is that the nature of mathematics is non-independent knowledge that can help humans in understanding and mastering social, economic and natural problems (Kline, 1973). Mathematics is an emerging science, evolving from real life and serving as a practical tool for real problems (Boyer, 2011). Studies on phenomenon-based learning to improve the cognitive and affective abilities of students through mathematics learning have been conducted related to the development of mathematical abilities of students by Aisah (2015), Faturohman (2015), Hidayat (2015) and Rosyid (2015). The results of these research studies show that phenomenon-based mathematics learning can improve students' mathematical abilities (mathematical communication skills and mathematical connections) and critical thinking. Meanwhile, research that has been trying to develop 21st century skills required by students through mathematics learning has been conducted by Turmudi (2017), who developed the didactic design of mathematics learning through science phenomena and applied it to junior high school students. In
http://ijlter.org/index.php/ijlter
164
addition, 21st century skills development has been widely applied to STEM-based integrative mathematics learning by Becker and Park (2011), Bell (2010) and Kim and Park (2012). Research Problem Departing from empirical-conceptual conditions as described above, this research examines the development of autonomy as one of the aspects in psychological well-being and creative thinking skills in mathematics learning with the phenomenon of LMS-assisted science in prospective elementary school teachers. Conceptual Framework Starting with the background, the research problem, and the pandemic of Covid19, a 21st century learning approach was adopted for this study. As an implementation of the 21st century learning style, this research integrates technology and mathematical learning approaches using scientific phenomena to develop creative thinking skill and autonomy. For technology issues, the National Council of Teachers of Mathematics (NCTM) made recommendations for integrating technology in mathematics learning through technology principles (NCTM, 2000b). In terms of scientific phenomena issues, the nature of mathematics as a non-independent knowledge can help humans in understanding and mastering social, economic and natural problems (Kline, 1973). Mathematics is an emerging science, evolving from real life and serving as a practical tool for real problems (Boyer, 2011). Moreover, mathematics explains phenomena and their occurrence via teachers to students (Turmudi, 2014).
2. Theoretical Framework 2.1 Autonomy and Creative Thinking Skill Autonomous people are independent people who can determine the best for themselves (Keyes & Ryff, 1999). These individuals have an internal locus of evaluation, i.e. not seeking the consent of others but rather evaluating themselves by personal standards. Therefore, they do not think about the expectations and judgments of others towards them. Furthermore, autonomous individuals do not rely on the judgment of others to make important decisions. They do not conform to social pressures to think and act in any particular way. The first dimension of learning and innovation skill for K12 is creativity. Creativity relates to discovery, trying new approaches to solving problems and being innovative (P21, 2011). In line with the views of Chiam et al. (2014), creativity is explained as a skill for discovering new things that have not existed before, being original, developing new solutions to each problem, and involving the ability to generate new, varied, and unique ideas (Chiam et al., 2014). Creativity is the ability to uncover new relationships, to see things from a new point of view, and to form new combinations of two or more concepts that have been mastered before (Suryadi, 2005). Based on the explanation above, creative and innovative thinking skills in this research is the ability to generate new ideas that are original and different from others, and to provide many solutions to problems. The indicators used in
http://ijlter.org/index.php/ijlter
165
measuring creative thinking skills in this study refer to indicators of creative thinking skills (Torrance, 1966, 1974) known as TTCT and presented in Table 1. Table 1: Torrance creative thinking skills indicators (1966, 1974) 1 2 3
Fluency Flexibility Originality
4
Elaboration
Mention many problem-solving solutions Provide different answers of solutions Create new solutions or answers that have never been used before to solve problems Mention the complete solutions
2.2 Mathematics Learning with Science Phenomena Science phenomenon-based mathematics learning is derived from Freudenthal’s didactical phenomenology theory (Freudenthal, 2002). Didactic phenomenonbased mathematics learning in mathematical learning introduces appropriate and suitable phenomena for students in learning mathematical concepts (Freudenthal, 2002; Turmudi, 2014). In this research, the phenomena introduced to students are phenomena related to science. The science phenomenon that is introduced to students is learning that emphasizes the importance of contextual issues to introduce mathematical topics to students. Phenomenology comes from the Greek phaninomenon meaning ‘emerging’, and logos meaning ‘science’ (Turmudi, 2017). The philosophy of phenomenology is learning about consciousness as the experience of the subject (or the view of the first person) (Turmudi, 2017). The term ‘phenomenology’ is usually followed by the word ‘didactic’, which means that learning begins with or departs from an experience and consciousness. In mathematics learning, he illustrated that students can be invited to walk on the floor (tiles) then explained to raise awareness that the floor tiles have mathematical value. Then the students pause, try to observe, look deeply, and imagine the mathematical value that can be derived from the experience. Furthermore, students can try to conjecture in the form of questions, then collect data or information to support the notion in order to make abstractions or conclusions (Turmudi, 2017). 2.3 Learning Management System The Learning management system (LMS) is a system or software as an infrastructure in electronic, online or distance learning (McGill & Klobas, 2009; Watson & Watson, 2007). The existence of this LMS cannot be separated from the opportunities for learning using Internet technology and networks. The capabilities and features of the LMS depend on the needs of the organization that uses the system to manage learning. Dye and Fagerberg (2004) explain that in general an LMS must have the following capabilities: 1. Registration for administrators, teachers and students on curriculum or learning; 2. Linking curriculum and learning administration to learning content; 3. Supporting traditional virtual learning; 4. Supporting various kinds of analysis and assessment, tracking team or individual progress; 5. Ability to administer test results, assessments and reports; and
http://ijlter.org/index.php/ijlter
166
6. Capability management so that an organization can measure learning needs and identify development needs. LMS is a powerful technology that has yet to reach its full potential and is important for the Information Age paradigm of education (Watson & Watson, 2007). Because of its importance, greater care and understanding needs to be used when applying the term in research literature. In its implementation, the LMS that used was known as blended learning. Blended learning is a formal education program in which a student learns, at least in part, through online delivery of content and instruction with some element of student control over time, place, path, and pace (Staker & Horn, 2012). In the blended learning model, students study teaching materials at home, communicating with other students and teachers through online discussions. Concepts can be reinforced by the teacher through student involvement. Therefore, owing to the Covid-19 pandemic, this research study was conducted online.
3. Methods 3.1 Design The experimental design to obtain KA21 and PWB data used in this research is the posttest-only design with a nonequivalent control group as presented in Figure 1. In this design, the experimental group and the control group were given a posttest, with the experimental class being given the treatment of mathematical learning with a filtered-s least mean square (FSLMS) algorithm. The preexperiment design used is depicted below:
Group A X1 → O Group B X → O Figure 1: KA21 quantitative pre-experimental research design
Note: X1: Treatment to the experimental class X : Treatment to the control class O : Post-test This design is used to determine how LMS variables affect creative thinking skills in KA21 and autonomy in PWB. 3.2 Participants The population in this study were all the semester 1 students of prospective elementary school teacher level 2 at a university located in two provinces, namely in West Java and Banten. The samples taken to test the model in this research were four (4) classes of diverse students, two classes of which were used as experimental classes, and the remaining two (2) classes were comparison classes. The students in all four classes numbered approximately 156 students. The sampling technique used was a purposive sampling technique. Purposive sampling is a sampling technique based on knowledge of the characteristics of the population and the purpose of the research (Fraenkel, 2012). In the experimental classes from the sample, mathematics learning was carried out using LMS-assisted science phenomena mathematics learning (FSLMS). In this research, mathematics
http://ijlter.org/index.php/ijlter
167
learning with science phenomenon was applied to rational number material to the prospective elementary school student teachers level II. 3.3 Research Instruments The instruments used in this research were questionnaires related to the autonomy or independence of learning that has been validated and tested. There were 14 statements which were based on the theory of Ryff (1999) and the development of autonomous instruments (Muqodas, 2019). Tests related to rational numbers of three essay questions were also given to students to test their creative thinking skills. These had been previously validated and empirically tested. The test question refers to indicators 1, 2, 3 of Torrance's creative thinking skills (Torrance, 1966, 1974). 3.4 Settings Learning was conducted in a blended way. At each meeting, lecturers presented learning plans, teaching materials in various forms such as audio, video and presentations as well as discussion materials on the LMS. At the beginning of the lesson, the lecturer initiated the learning synchronously by providing the details, conveyed the learning objectives, scientific phenomena to be studied and quizzes. Then the lecturer gave an explanation related to the activities that would be carried out next, accompanied by a question and answer session. After that, students studied the teaching materials. This was followed by carrying out activities asynchronously. The lecturer guided learning activities through LMS and a WhatsApp group. Once the asynchronous session had been completed, the learning continued with a synchronous discussion. Students delivered practical results in turn, then shared ideas, notions and questions and reflected on the learning that had taken place.
4. Results and Discussion 4.1 Autonomy of Prospective Elementary School Student Teachers
The descriptive analysis of the autonomy of prospective elementary school teachers students is described in Table 2. Table 2: Description of PWB-A prospective elementary school teachers Description
A(1)
A(2)
Category
F(1)
%f(1)
f(2)
%f(2)
Average Score
52.17
51,48
Very High
0
0.00
0
0,00
Highest Score
71.00
71,00
High
54
62.07
39
44,83
Lowest Score
41.00
30,00
Low
33
37.93
48
55,17
Ideal Score
Very Low
84.00 84,00 0 0.00 0 0,00 StDev Total 6.26 6,92 87 100.00 87 100,00 Information: A(1): PWB-A results in the FSLMS mathematics learning group A(2): PWB-A results in the TFSLMS mathematics learning group F(1) : Frequency of students in each category in the FSLMS mathematics learning group F(2) : Frequency of students in each category in TFSLMS mathematics learning group
http://ijlter.org/index.php/ijlter
168
Based on Table 2, it was found that PWB-A students are prospective elementary teachers in mathematics learning groups with FSLMS and TFSLMS for the following reasons: 1. There was an average score difference of 0.69 where the learning group with FSLMS was higher than the TFSLMS. However, the standard deviation score of the learning group with FSLMS was lower by 0.66 than the mathematics learning group with TFSLMS; 2. The highest score in the mathematics learning group with FSLMS was the same as that of the TFSLMS group; however, the lowest score was 11 points higher than the TFSLMS math learning group. Therefore the range of PWB-A scores in the mathematics learning group with FSLMS was 9 points less than that of the TFSLMS group; 3. After treatment, neither of the learning groups with FSLMS or TFSLMS had students with very high PWB-A scores, or very low scores; and 4. The percentage of prospective elementary school student teachers who have a low PW-A score is higher in the mathematics learning group with TFSLMS. Based on the PWB-A normality and homogeneity test, it was found that PWB-A data of prospective elementary school student teachers in the mathematics learning groups with FSLMS and TFSLMS are normal and homogeneous. On that basis, the two different tests were conducted on average by using t-test using = 0.05. The test results are shown in Table 3. Table 3: Test differences in PWB-A of prospective elementary school student teachers Result t PWB-A Equal variances POSTES assumed
t-test for Equality of Means df Sig. (2- Mean Difference Std. Error tailed) Difference -0.169 172 0.866 -0.184 1.091
Based on Table 3, t-test results indicate that the sig score. is 0.866 > 0.05. Thus H0 is accepted, which means that there is no difference in the average score of PWB-A data of prospective elementary school student teachers in the mathematics learning group with FSLMS and TFSLMS. 4.2 Creative Thinking Skills (KKf) of Prospective Elementary School Student Teachers The description of creative thinking skills of prospective elementary school teacher students is shown in Table 4. Table 4: Description of KKf achievement of prospective elementary school student teachers KKf Achievement
Class FSLMS
TFSLMS
Average
7,58
3,67
Highest Score
12
10
Lowest Score
0
0
St Dev
3,64
7,61
Ideal Score
16
http://ijlter.org/index.php/ijlter
169
Based on Table 4, the achievement score of KKf of prospective elementary school student teachers in the mathematics learning group with FSLMS and TFSLMS can be explained as follows: 1. There was an average score difference of 3.91, whereas the KKf of the learning group with FSLMS was higher than the group with TFSLMS. However, the standard deviation score of the learning group with FSLMS was lower by 3.97 than the math learning group with TFSLMS; and 2. The highest score in the math learning group with FSLMS was 2 points higher, but the lowest score was the same as that of the TFSLMS group. Based on the data, the KKf achievement of prospective elementary school student teachers in the FSLMS group is better than that of the TFSLMS group. In addition, inferential tests were conducted for differences in creative thinking skills between the FSLMS and TFSLMS groups. Based on the normality test, it was found that the results of KKf achievement data of prospective elementary school student teachers in the mathematics learning groups with FSLMS and TFSLMS are not normally distributed. On that basis, the difference test was conducted using the Mann-Whitney test using = 0,05. The test results are shown in Table 5. Table 5: Test differences in KKf achievement of prospective elementary school student teachers Mann-Whitney U Wilcoxon W Z Asymp. Sig. (2-tailed)
KKf 2853,500 6681,500 -3,188 0,001
4.3 Discussion Autonomy is one aspect of psychological well-being that can affect student learning outcomes (Winch, 2002). By having good autonomy, students will have high levels of independence in learning as well. People who are positive in terms of psychological well-being (PWB) are happy, healthy, productive, and have satisfying interpersonal relationships (Ryff, 2013). In a learning situation that is affected by the Covid-19 pandemic as it is today, learning is mostly directed at blended learning or learning that combines synchronous and asynchronous techniques. These are highly dependent on the independence of students' learning because the teacher only facilitates and provides motivation from a distance. In this study, LMS-assisted mathematics learning shows that the students in the class with mathematics learning with LMS-assisted science phenomena descriptively have a higher level of autonomy. This is indicated by a higher average score. The percentage of students who have higher levels of autonomy is greater than in the class without scientific phenomena. This indicates that computers can help students learn mathematics and problem-solving (Bell, 2010; Hatfield & Kieren, 1972; Robitaille et al., 2020). However, inferentially the results show that there is no significant difference. In this research, LMS is needed as a learning tool during a pandemic to manage the learning (Robitaille et al., 2020). Learning is strongly influenced by the subject
http://ijlter.org/index.php/ijlter
170
material, the teacher, the novelty effect, and the learning content. Although the LMS offers various teaching materials such as audio-video, audio, and presentations, independent learning will determine whether students can choose one, or study all of the teaching materials presented, or even not choose any of the teaching materials delivered. Students who study independently will be able to determine the learning resources they want to learn or even study other teaching materials as enrichment. Meanwhile, students with low autonomy will depend on the instruction and supervision of the teacher or instructor. Creative thinking skills is one of the most important skills of this century, and are the highest-level skills in Bloom's taxonomy (Krathwohl, 2010). Therefore, these skills play a very important role in individual success. In this research, learning mathematics with LMS-assisted science phenomena has shown to have an impact on students' creative thinking skills as shown in Table 4.1. Descriptively, the average creative thinking skills of students who study mathematics with LMSassisted science phenomena are almost twice those of the comparison classes’ average. This is also reinforced by the results of the inferential test which shows that there is a significant difference in the alpha of 5%. Furthermore, learning with LMS-assisted science phenomena applied in this research provides opportunities for students to explore the surrounding environment through didactic phenomena that appear in students' daily lives. In this learning, students conduct experiments that can provide opportunities for them to develop their imagination, and ways and methods of completing and reporting assigned experiments. In group experiments, students can also share ideas in solving problems. This is in line with previous preliminary research on LMS which also has a positive influence on mathematical creative thinking skills (Widodo, 2017). This is supported by research by Garrison and Anderson (2003), who found that technology is a learning tool used in formal education practices to disseminate, illustrate, communicate, or support students and teachers in activities designed specifically to sustain learning (Gunawan et al., 2019). This indicates that learning using multimedia electronics is more significant than conventional learning (Munir, 2010). The analysis was continued to determine whether there is a relationship between autonomy and creative thinking skills. Based on the correlation coefficient, a score of 0.24 was obtained. Therefore, it can be concluded that there is a low relationship between the two variables. This is not according to the views of Winch (2002) who stated that autonomy is in line with critical thinking skills. Therefore, further research needs to be done on other aspects of psychological well-being related to creative thinking skills as part of 21st century skills.
5. Conclusion Based on the analysis and discussion of these results, it can be concluded that the autonomy of prospective elementary school teachers in learning mathematics with LMS-assisted science phenomena is no different from learning mathematics without LMS-assisted science phenomena. Both of these lessons produce autonomy for students because they provide equal opportunities to study a
http://ijlter.org/index.php/ijlter
171
variety of teaching materials. The creative thinking ability of students in the experimental classes is better than that of students in the control classes because students in the experimental classes are more likely to improve their creative thinking skills and are allowed to explore the surrounding environment through didactic phenomena that appear in students' daily lives. This research has shown that what needs to be followed up in further research is how elementary school candidate teachers can explore additional teaching materials independently that are not available in LMS. Therefore, to improve their ability of autonomy and creative thinking, teachers need to adopt innovative learning such as mastering LMS as learning management in selecting materials and implementing the teaching and learning process effectively. The results reported here should be interpreted in the light of these considerations.
6. References Afifah, N. M. (2019). Keterampilan abad 21 dalam pembelajaran matematika berbasis proyek [21st century skills in project-based mathematics learning]. UIN Sunan Ampel Surabaya. Aisah, L. S. (2015). Pembelajaran Matematika Berbasis Fenomena Didaktis Melalui Pendekatan Saintifik Untuk Meningkatkan Kemampuan Koneksi Matematis Dan Self-Esteem Siswa SMP [Didactic Phenomenon-Based Mathematics Learning Through a Scientific Approach to Improve Mathematical Connection Ability and Self-Esteem of Junior High School Students]. Universitas Pendidikan Indonesia. Ashcrafts, M. (2002). Math anxiety: Personal, educational, and cognitive consequences. Current Directions in Psychological Science, 11(5). https://doi.org/10.1111/14678721.00196 Ayers, T., Davis, G., Dubinsky, E., & Lewin, P. (1988). Computer experiences in learning composition of functions. Journal for Research in Mathematics Education, 19, 246–259. Bealty, B., & Ulasewicz, C. (2006). Online teaching and learning in transition: Faculty perspectives on moving from blackboard to the moodle learning management system. Journal Techtrends, 50(4), 36–45. Becker, K., & Park, K. (2011). Effects of integrative approaches among science, technology, engineering, and mathematics ( STEM ) subjects on students’ learning : A preliminary meta-analysis. Journal of STEM Education, 12(5), 23–38. https://doi.org/10.1037/a0019454 Bell, S. (2010). Project-based learning for the 21st century skills. http://doi.org/10.1080/00098650903505415 Blyth, B., & Labovic, A. (2009). Using Maple to implement eLearning integrated with computer-aided assessment. International Journal of Mathematical Education in Science and Technology, 40(7), 975–988. https://doi.org/10.1080/00207390903226856 Brendefur, J., & Frykholm, J. (2000). Promoting mathematical communication in the classroom: Two pre-service teachers’ conceptions and practices. Journal for Research in Mathematics Education, 3(125). https://doi.org/10.1023/A:1009947032694 Chiam, L. C., Hong, H., Ning, F. H., & Tay, W.Y. (2014). Creative and Critical Thinking in Singapore Schools. Office of Education Research, National Institute of Education, Nanyang Technological University. Dye, A., & Fagerberg, T. (2004). Exploring online service in a mobile environment. Technical Working Paper. Effendi, L. A. (2012). Pembelajaran Matematika Dengan Metode Penemuan Terbimbing Untuk Meningkatkan Kemampuan Representasi Dan Pemecahan Masalah
http://ijlter.org/index.php/ijlter
172
Matematis Siswa SMP [Learning Mathematics with Guided Discovery Methods to Improve Representation and Mathematical Problem Solving Skills for Junior High School Students]. Jurnal Penelitian Pendidikan, LPPM UPI, 13(2). Faturohman, D. R. (2015). Pembelajaran Matematika Berbasis Fenomena Didaktis Melalui Pendekatan Scientific untuk Meningkatkan Kemampuan Berpikir Kritis dan Kemandirian Belajar Siswa SMP [Didactic Phenomenon-Based Mathematics Learning Through a Scientific Approach to Improve Critical Thinking Ability and Independent Learning of Junior High School Students]. Universitas Pendidikan Indonesia. Fauzi, M. R., & Priatna, N. (2019). Analysis of student’s mathematical connection and communication in algebra: the exponential equations. Journal of Physics: Conference Series, 1211 012065. https://doi.org/10.1088/1742-6596/1211/1/012065 Fauzi, A. (2015). The enhancement of student’s Mathematical connection ability and selfregulation learning with metacognitive learning approach in junior high school. In Proceedings of the International Conference on Research and Education in Mathematics (ICREM7). https://doi.org/10.1109/ICREM.2015.7357048 Fraenkel, J. R., & N. E. W. (2012). How to design and evaluate research in education (8th ed.). McGraw-Hill Higher Education. Freudenthal, H. (2002). Phenomenology, didactical mathematical, O F. Springer Netherlands. Fuchs, L. S., Fuchs, D., Prentice, K., Hamlett, C. L., Finelli, R., & Courey, S. J. (2004). Enhancing mathematical problem solving among third-grade students with schema-based instruction. Journal of Educational Psychology, 96(4), 635–647. https://doi.org/https://doi.org/10.1037/0022-0663.96.4.635 Giarti, S. dan Astuti, S. (2016). Implementasi TQM Melalui Pelatihan Model in House Training untuk Meningkatkan Kompetensi Pedagogik Guru SD [Implementation of TQM through Model in House Training to Improve Pedagogic Competence of Elementary School Teachers]. Scholaria Jurnal Pendidikan Dan Kebudayaan, 6(12). https://doi.org/10.24246/j.scholaria.2016.v6.i2.p80-91 Göksün, D. O., & Gürsoy, G. (2019). Comparing success and engagement in gamified learning experiences via Kahoot and Quizizz. Journal Computers & Education, 135, 15–29. https://doi.org/10.1016/j.compedu.2019.02.015 Gunawan, G., Sahidu, H., Susilawati, S., Harjono, A., & Herayanti, L. (2019). Learning management system with Moodle to enhance creativity of candidate physics teacher. Journal of Physics: Conference Series, 1417(1), 0–6. https://doi.org/10.1088/1742-6596/1417/1/012078 Hamadneh, I. M., & Al–Masaeed, A. (2015). Math teachers’ attitudes towards photo math application in solving mathematical problem using mobile camera. Educational Research and Reviews, 10(14), 1930–1936. https://doi.org/10.5897/ERR2015.2181 Hatfield, L. L., & Kieren, T. E. (1972). Computer-assisted problem solving in school mathematics. Journal for Research in Mathematics Education, 3(2). https://doi.org/10.2307/748668 Hidayat, A. (2015). Perbandingan Kemampuan Berpikir Kreatif Matematis Serta Self-Efficacy Antara Siswa MTs yang Memperoleh Pembelajaran Berbasis Fenomena Didaktis dan Berbasis Kurikulum 2013 Melalui Pendekatan Invesitigasi [Comparison of Mathematical Creative Thinking Ability and Self-Efficacy Between MTs Students Who Received Didactic Phenomenon-Based Learning and 2013 Curriculum-Based Through Investigation Approach]. Universitas Pendiidkan Indonesia. Huebner, S. E., & McCullough, G. (2010). Correlates of school satisfaction among adolescents. Journal of Educational Research, 93(5). https://doi.org/10.1080/00220670009598725 Kaput, J., Hegedus, S., & Lesh, R. (2019). Foundations for the future in mathematics education. Routledge.
http://ijlter.org/index.php/ijlter
173
Kelley, T. R., Knowles, J. G., Abell, S., Lederman, N., Bandura, A., Bandura, A., Williams, D. (2016). A conceptual framework for integrated STEM education. International Journal of STEM Education. https://doi.org/10.1186/s40594-016-0046-z Keyes, C. L. M., & Ryff, C. D. (1999). Psychological well-being in midlife. In S. L. Willis & J. D. Reid (Eds.), Life in the middle: Psychological and social development in middle age. Academic Press. https://doi.org/10.1016/B978-012757230-7/50028-6 Kim, Y., & Park, N. (2012). The effect of STEAM education on elementary school student’s creativity improvement. Communications in Computer and Information Science, 339 CCIS, 115–121. https://doi.org/10.1007/978-3-642-35264-5_16 Kline, M. (1973). Why JoHnny can’t add: The failure of the new math. Vintage Books. Kösa, T., & Karakuş, F. (2010). Using dynamic geometry software Cabri 3D for teaching analytic geometry. Procedia - Social and Behavioral Sciences, 2(2), 1385–1389. https://doi.org/10.1016/j.sbspro.2010.03.204 Kramarski, B., & Mevarech, Z. R. (2003). Enhancing mathematical reasoning in the Cclassroom: The effects of cooperative learning and metacognitive training. American Educational Research Journal, 40(1), 281–310. https://doi.org/10.3102/00028312040001281 Lestari, S. A. P. (2019). Mathematical reasoning ability in relations and function using the problem solving approach. Journal of Physics: Conference Series, 1188(012065). https://doi.org/10.1088/1742-6596/1188/1/012065 Maslukha, M., Lukito, A., & Ekawati, R. (2018). Refractive thinking profile in solving mathematical problem reviewed from students math capability. Journal of Physics: Conference Series, 947(012022). McGill, T. J., & Klobas, J. (2009). A task–technology fit view of learning management system impact. Journal Computers & Education, 52(2), 496–508. https://doi.org/10.1016/j.compedu.2008.10.002 Merzbach, U. C., & Boyer, C. B. (2011). A history of mathematics. John Wiley & Sons. https://doi.org/10.2307/3620228 Muhajir, S. (2018). Penerapan project oriented problem based learning dengan reading infusion. Untuk melatihkan keterampilan abad Ke 21 Mahasiswa. Universitas Pendidikan Indonesia. Munir. (2010). Penggunaan learning management system (LMS). Di Perguruan Tinggi: Studi Kasus Di Universitas Pendidikan Indonesia [Use of learning management system (LMS). In Higher Education: A Case Study at the Indonesian University of Education]. Cakrawala Pendidikan, 29(1), 109–119. https://doi.org/10.21831/cp.v1i1.222 Muqodas, I. (2019). Konseling Kesejahteraan untuk Meningkatkan Kesejahteraan Psikologis Mahasiswa [Welfare Counseling to Improve Student Psychological Well-being]. Universitas Pendidikan Indonesia. Mushlihuddin, R., & Nurafifah, I. (2018). The effectiveness of problem-based learning on students’ problem solving ability in vector analysis course. Journal of Physics: Conference Series, 948(012028). https://doi.org/10.1088/1742-6596/948/1/012028 Nabilah, N. G. S., Suhendra, S., & Yulianti, K. (2019). The efforts of improving mathematical connection ability of senior high school student with 7e learning cycle model. Journal of Physics: Conference Series, 1157(042096). NCTM. (2000a). Executive summary principles and standards for school mathematics. https://www.nctm.org/uploadedFiles/Standards_and_Positions/PSSM_Executi veSummary.pdf NCTM. (2000b). Principles and standards for school mathematics overview. Journal of Equine Veterinary Science. P21. (2011). 21st century skill map. http://www.21.org
http://ijlter.org/index.php/ijlter
174
Penglase, M., & Arnold, S. (1996). The graphics calculator in mathematics education: A critical review of recent research. Mathematics Educational Research Journal, 8, 58–90. https://doi.org/10.1007/BF03355481 Rees, G., Goswani, H., & Bradshaw, J. (2010). Developing an index of children’s subjective well being in England. The Children's Society. Robitaille, D. F., Sherrill, J. M., & Kaufman, D. M. (2020). The effect of computer utilization on the achievement and attitudes of ninth-grade mathematics students. Journal for Research in Mathematics Education, 8(1), 26–32. https://doi.org/10.5951/jresematheduc.8.1.0026 Rosyid, A. (2015). Pembelajaran Matematika Berbasis Fenomena Didaktis Melalui Pendekatan Saintifik untuk Meningkatkan Kemampuan Komunikasi Matematis dan Self-Confindence Siswa SMP [Didactic Phenomenon-Based Mathematics Learning Through a Scientific Approach to Improve Mathematical Communication Skills and SelfConfindence of Junior High School Students]. Universitas Pendidikan Indonesia. Ryff, C. D. (2013). Psychological well-being revisited: Advances in the science and practice of eudaimonia. Psychotherapy and Psychosomatics, 83(1), 10–28. https://doi.org/10.1159/000353263 Salehudin, N. N., Hassan, N. H., & Hamid, N. A. A. (2015). Matematik dan Kemahiran Abad Ke-21: Perspektif Pelajar [21st Century Mathematics and Skills: Student Perspectives]. Jurnal Pendidikan Matematik., 3(1), 24–36. Savedra, A. R & Ofter, V. D. (2012). Learning 21st century skills requires 21st century teaching. The Phi Delta Kappan, 94(2), 8-13. https://doi.org/10.1177/003172171209400203 Siregar, N.R. (2017). Persepsi siswa pada pelajaran matematika: studi pendahuluan pada siswa yang menyenangi game [Students' perceptions of mathematics: a preliminary study on game-loving students]. Prosiding Temu Ilmiah Nasional X Ikatan Psikologi Perkembangan Indonesia, Vol 1. Staker, H., & Horn, M. B. (2012). Classifying K–12 Blended Learning. Stern, S. (2007). Factors that impact the health and psychological well-being of older adults shortly following institutionalization. Case Western Reserve University. Suryadi, D. (2005). Penggunaan Pendekatan Pembelajaran Tidak Langsung serta Pendekatan Gabungan Langsung dan Tidak Langsung dalam Rangka Meningkatkan Kemampuan Berpikir Matematik Tingkat Tinggi Siswa SLTP [The Use of Indirect Learning Approaches and Combined Direct and Indirect Approaches in Order to Improve High-Level Mathematical Thinking Skills for Junior High School Students]. Universitas Pendidikan Indonesia. Tobin, S.S., & Neugarten, B. L. (1961). Life satisfaction and social interaction in the aging. Journal of Gerontology, 16, 344–346. https://doi.org/10.1093/geronj/16.4.344 Torrance, E. P. (1966). The Torrance tests of creative thinking-norms-technical manual research edition -Verbal tests, Forms A and B - Figural tests, Forms A and B. Personnel Press. Torrance, E. P. (1974). The Torrance tests of creative thinking-norms-technical manual research edition - Verbal tests, Forms A and B - Figural tests, Forms A and B. Personnel Press. Turmudi, T. (2014). Pengembangan pembelajaran matematika berbasis fenomena didaktis [Development of didactic phenomenon-based mathematics learning]. Universitas Pendidikan Indonesia Turmudi, T., Hidayat, A. S., & Sispiati, R. (2017). Mathematics instruction based on science using didactical phenomenology approach in junior secondary school in Indonesia. Intech Open. http://dx.doi.org/10.5772/intechopen.68437 Watson, W. R., & Watson, S. L. (2007). An argument for clarity: What are learning management systems. What are they not, and what should they become? TechTrends, 51(2), 28–34.
http://ijlter.org/index.php/ijlter
175
http://cardinalscholar.bsu.edu/handle/123456789/194513 Weaver, D., Spratt, C., & Nair, C. S. (2008). Academic and student use of a learning management system: Implications for quality. Australian Journal of Educational Technology, 24(1). https://doi.org/10.14742/ajet.1228 Widodo. (2016a). Profesor Ini Ungkap Mengapa Matematika Dianggap Sulit [This Professor Reveals Why Mathematics Is Considered Difficult]. Suara.com: Edisi Rabu. Suara.Com. Widodo, A. (2016b). Teacher pedagogical content knowledge (PCK) and students reasoning and wellbeing. Journal of Physics: Conference Series, 812. http://doi.org/10.1088/1742-6596/812/1/012119 Widodo, S. (2017). Implementing Google Apps for education as learning management system in math education. Journal of Physics: Conference Series, 895(1). https://doi.org/10.1088/1742-6596/895/1/012053 Winch, C. (2002). Strong autonomy and education. Educational Theory, 52(1). Xiao, X., Xu, H., & Xu, S. (2015). Using IBM SPSS modeler to improve undergraduate mathematical modelling competence. Computer Applications in Engineering Education, 23, 603–609. https://doi.org/10.1002/cae.21632
http://ijlter.org/index.php/ijlter
176
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 176-193, August 2021 https://doi.org/10.26803/ijlter.20.8.11 Received May 29, 2021; Revised Aug 14, 2021; Accepted Aug 24, 2021
E-Learning Implementation Barriers during COVID-19: A Cross-Sectional Survey Design Ishaq Al-Naabi and Abdullah Al-Abri English Language Centre, University of Technology and Applied Sciences, Nizwa, Oman https://orcid.org/0000-0001-8829-2922 https://orcid.org/0000-0001-7198-9643 Abstract. Most higher education institutions have embraced emergency remote teaching (ERT) as a response to school and university closures due to the Covid-19 pandemic. We used a cross-sectional survey design to examine teachers’ views on the implementation of e-learning for ERT due to Covid-19 in Omani higher education. We examined four barrier categories: teacher-related, institutional-related, curriculum-related and student-related. We also assessed the relationship between these barrier levels and measured the differences between e-learning and teachers’ gender, academic qualifications, teaching experience and prior experience in e-learning. Through an investigation of 856 university faculty members, we found that student-level barriers had the strongest impact on the implementation of e-learning for ERT during Covid-19. Furthermore, teachers’ academic qualifications and prior experience influenced the success of e-learning practices. Based on these results, we presented some implications with respect to student preparedness, curriculum modifications and institutional readiness for the successful implementation of ERT during Covid-19 or any similar future pandemics. Keywords: Covid-19; e-learning; emergency remote teaching; Oman; higher education
1. Introduction The Covid-19 outbreak has had a serious impact on educational systems around the world. The majority of higher education institutions cancelled face-to-face classes and embraced emergency remote teaching (ERT) in an attempt to maintain social distancing. With millions of students around the world having been affected by school and university closures during Covid-19 (UNESCO, 2020), many educational institutions implemented some type of ERT (Hodges et al., 2020). To curb the spread of coronavirus, the Omani government announced the suspension of all schools and universities on the 15th of March 2020, and
©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
177
consequently, most public and private institutions used what resources they had available to shift to ERT. Technology integration for online and distance learning is a crucial component of providing quality education (UNESCO, 2014). Teachers at the tertiary level must be ‘fluent users of technology’ if they are to support student learning (US Department of Education, 2016, p. 34). Additionally, Trust (2017) argued that future teachers should be capable of using technology to enhance students’ learning. The shift to ERT has enabled higher education to provide instruction during emergencies; however, the shift has caused serious challenges (Crawford et al., 2020; Hodges et al., 2020). As many teachers were not prepared for the sudden shift to online instruction, they faced issues in preparing online content, adapting face-to-face materials, and learning new instructional pedagogies for online teaching and learning (Crawford et al., 2020; Dhawan, 2020; Rapanta et al., 2020). Gacs et al. (2020) and Karalis (2020) suggested that higher education institutions should provide sufficient support to their faculty during emergencies. Students faced issues with online learning during COVID-19 (Crawford, Butler-henderson, et al., 2020; Gaber et al., 2020; Zuo et al., 2020). Apart from teachers and students preparedness for ERT, institutions were not fully ready to shift to online instruction (Bao, 2020; Crawford et al., 2020). Since ERT is mostly dependent on teachers’ use of and competency in e-learning (Crawford et al., 2020; Demuyakor, 2020; Gacs et al., 2020; Hodges et al., 2020), it is essential that we measure the barriers to technology integration for ERT during Covid-19. Therefore, in this paper, we have sought to investigate the barriers faced by university teachers in using e-learning for ERT during Covid19 in Omani public and private higher education institutions. In addition to advancing knowledge in e-learning and online pedagogy, the findings of this paper can inform higher education practitioners in overcoming challenges associated with online instruction during emergencies. Also, it can help institutions to better plan their instruction during similar pandemics in the future. People in charge of professional development can use the results of this study as a basis for needs analysis for teacher professional development and student training provisions.
2. Literature Review ERT is a temporary shift of instructional delivery to an alternative delivery mode due to crisis circumstances (Hodges et al., 2020, p. 6). Remote teaching is not new, and it has been argued that teachers should, in fact, be prepared to teach and administer online courses (Baran & Alzoubi, 2020; Crawford et al., 2020; Gacs et al., 2020; Hartshorne et al., 2020; Hodges et al., 2020; Karalis, 2020). While responses of higher education providers to Covid-19 differed, the majority of institutions in developed and developing economies chose to migrate to online teaching (Crawford et al., 2020; Hodges et al., 2020). Karalis (2020) proposed a model that was intended to provide a good response to crises in education. He argued that a needs analysis that includes the needs of students, instructors and available resources, as well as considering the nature of
http://ijlter.org/index.php/ijlter
178
the courses themselves, is required for the success of any emergency plan. Furthermore, Baran and Alzoubi (2020) proposed a human-centred model for transferring face-to-face courses online during Covid-19. Based on their model, they asserted that human-centred design models can enhance student empathy, engage students in pedagogical problem-solving and help them build a learning community (Baran & Alzoubi, 2020). Despite these efforts, ERT is mainly aimed at ensuring the continuity of education during emergencies, which entails some sacrifices if it is to guarantee a rapid response to a crisis (Gacs et al., 2020; Hodges et al., 2020). ERT requires technology integration, which poses challenges and difficulties for both teachers and students. Crawford et al. (2020) identified a variety of challenges that can hinder institutions in their migration to online education during ERT, including lack of infrastructure, teacher skillsets and readiness to design, offer and implement online teaching and learning. Apart from these challenges, many educational systems worldwide continue to lack online instruction strategies (Crawford et al., 2020). Moreover, Bao (2020) reported that higher academic institutions lacked the preparedness to shift to online teaching and learning during Covid-19, calling for systematic planning and investments from all sectors to ensure the ability to successfully shift to ERT. Similarly, Chang and Fang (2020) reported that teachers in higher education were not familiar with the teaching tools required for online teaching which posed serious negative impact on the implementation of ERT. Nevertheless, this unpreparedness, interruption and sudden shift, according to McMaster et al. (2020), can provide great learning opportunities for teachers. Previous research has established a set of common technology integration barriers that have each been labelled, measured and rated differently, and that overlap on certain occasions (Schoepp, 2005). Common challenges include creating content for online spaces, learning new delivery tools, understanding online pedagogy, engaging parents, addressing student mental health issues, and attempting various pedagogical strategies to address both synchronous and asynchronous teaching and learning (Hartshorne et al., 2020, p. 138). Alqudah et al. (2020) used a survey design to measure the perceptions and experiences of Jordanian academic ophthalmologists in e-learning for an undergraduate course during Covid-19. The study revealed some advantages of e-learning, including convenience, flexibility of time and place for both students and teachers, increased motivation of shy students and overcoming the circumstances of lockdown measures during Covid-19 (Alqudah et al., 2020). Though the majority of their sample viewed their e-learning experience during Covid-19 as positive and successful, Alqudah et al. (2020) did report some disadvantages of e-learning, such as lack of interaction, discomfort of teaching and learning without face-to-face interactions, and lack of practical and clinical training. The study also identified some limitations to e-learning during Covid19, including poor infrastructure, shortage of e-learning training courses in ophthalmology, students’ or teachers’ poor e-learning skills and inadequate internet speed (Alqudah et al., 2020). In another study using a survey design, Mailizar et al. (2020) examined the secondary school teachers’ views of e-learning during Covid-19. This study
http://ijlter.org/index.php/ijlter
179
found that student-related barriers (skills and knowledge, motivation and elearning infrastructure) had a strong impact on teacher implementation of elearning (Mailizar et al., 2020). Student-related barriers to e-learning implementation had a strong positive correlation with school-related barriers (i.e. availability of software and hardware, internet connectivity, policies, technical support) and curriculum-related barriers (i.e. content, assessments, elearning resources, curriculum alignment to online teaching and learning). Due to the scarcity of research on this topic during Covid-19, for this study we relied on the large and growing body of literature that has investigated the barriers to e-learning implementation prior to Covid-19. Several studies (Assareh & Hosseini, 2011; Naveed et al., 2017; Pelgrum, 2001) have identified various barriers to e-learning implementation. Despite differences in labelling and classifying these barriers, they were similar and overlapping. In general, the barriers can be classified into four main categories: student-related barriers, teacher-related barriers, curriculum-related barriers and institutional-related barriers. The issues related to students included lack of ICT skills, lack of elearning knowledge, lack of English language proficiency, lack of motivation, lack of technological infrastructure and poor assessment and online learning skills (Assareh & Hosseini, 2011; Naveed et al., 2017; Pelgrum, 2001). Assareh and Hosseini (2011), Naveed et al. (2017) and Pelgrum (2001) also identified barriers related to teachers, including lack of ICT skills, lack of e-learning knowledge, instructor resistance to change, lack of time to develop e-courses, lack of motivation, difficulty in monitoring and assessing student learning and difficulty in integrating technology into teaching. With respect to curriculum, the literature identified a lack of sound instructional design, curriculum ambiguity, poor curriculum quality, unavailability of resources for e-learning and a mismatch between the existing teaching and learning process, current evaluation procedures, and the online curriculum (Assareh & Hosseini, 2011; Naveed et al., 2017; Pelgrum, 2001). Finally, studies have also reported institutional-related barriers, such as inappropriate infrastructure, low internet bandwidth, lack of technical support, lack of financial support, lack of adequate policies and lack of training on e-learning and online pedagogy. Several research studies have identified various determinants to successful elearning practices. Although gender was not a crucial factor influencing elearning use, research has reported that female teachers are more confident in utilising e-learning in their teaching (González-gómez et al., 212; Mahdizadeh et al., 2008). In addition to gender, teacher’s academic qualification and prior experience in e-learning can affect the use of e-learning (Fryer & Bovee, 2016; Jones, 2003; Sørebø et al., 2009; Wilson, 2012). These studies highlight the barriers that teachers face when using e-learning in teaching and learning at various levels. These barriers hinder the teaching and learning process in ERT during Covid-19, as ERT is entirely based on either synchronous or asynchronous online teaching and learning (Hodges et al., 2020). Assareh and Hosseini's (2011) four-dimension classification of e-learning barriers (learners, teachers, curriculum and school) were used in this study. Since the context of the current study is higher education (i.e. colleges and universities), we changed the school classification to ‘institution’. Learner-
http://ijlter.org/index.php/ijlter
180
related barriers included insufficient e-learning knowledge and skills, limited accessibility and lack of motivation. Teacher-related barriers included insufficient knowledge and skills in e-learning, attitudes and beliefs towards elearning, lack of confidence and previous experience. The barriers related to curriculum that can hinder e-learning use included in this study are disparity between curriculum and assessment, f2f curriculum that does not have elearning components and complex tasks that cannot be delivered through an elearning system. Finally, we included the following institutional-related barriers in our study: e-learning infrastructure, policies and professional development in e-learning. Our research is guided by the following research questions: 1. What barriers do teachers face in implementing e-learning for ERT during Covid-19 in Oman? 2. What is the relationship between each level of barrier to e-learning implementation? 3. Are there any significant differences between e-learning implementation barriers and teachers’ gender, academic qualifications, teaching experience and prior experience in e-learning? 4. Are there any significant relationships between e-learning implementation barriers and type of institution?
3. Methodology 3.1 Research Design The study employed a cross-sectional survey research design. Creswell (2018) defined survey research as “a set of procedures in quantitative research in which investigators administer a survey to a sample or to the entire population of people to describe the attitudes, opinions, behaviours, or characteristics of the population” (p.376). Elsewhere, Fraenkel et al. (2012) stated that using a questionnaire in survey research methodology could provide reliable, valid and generalizable quantitative and qualitative results. 3.2 Research Instrument We developed a questionnaire consisting of three main parts to gather teachers’ views on the implementation of e-learning during Covid-19. In Part A, the participants’ demographic information, including gender, level of education, teaching experience, institution, specialisation, teacher certification and devices used for e-learning was obtained. The second part comprised 39 Likert-scale items that were grouped into the four main dimensions (teacher-related barriers, institution-related barriers, curriculum-related barriers and student-related barriers). We adapted the items included in the scale from various studies (Flack et al., 2020; Haney, 2002; Mailizar et al., 2020; Naveed et al., 2017) and added some items for the purpose of this research. Participants were asked to mark their responses on a five-level agreement scale ranging from strongly disagree to strongly agree. The third part included two open-ended questions that asked about other barriers that teachers faced in implementing e-learning during COVID-19 and about their suggestions for better online pedagogy during emergencies.
http://ijlter.org/index.php/ijlter
181
Five academics from different institutions reviewed the questionnaire for clarity and relevance to ensure its content validity. Along with the questionnaire, a content validation form was provided for the reviewers. The questionnaire achieved satisfactory level of content validity (the content validity index for each item ranged between 0.8 to 0.92 across the five reviewers. In addition, we piloted the questionnaire with a group of university teachers from a private university in Oman who had experienced ERT during Covid-19. Twenty-seven teachers (55.9% males, 44.4% females) completed the survey. The majority of the participants had a master’s degree (77.8%), while 22.2% had doctoral degrees. The sample included teachers from different specialisations, such as business, education, language and literature, information and communication technology and English language teaching. To ensure the reliability of the research instrument, we computed Cronbach’s alpha reliability coefficient in SPSS, which was found to be 0.88. The reliability coefficient of the four dimensions ranged from 0.42 to 0.85. Teacher-related barriers had a lower coefficient (0.42) compared to other dimensions of the questionnaire (teacher-related barriers = 0.75; curriculum-related barriers = 0.77; student-related barriers = 0.85). Thus, we removed three items from the teacherrelated barriers dimension to increase the instrument’s reliability. Once these items were removed, the coefficient increased to 0.89. The questionnaire was distributed to teachers using Google Forms®. Participation in the study was voluntary. Although participants were not asked to fill out an informed consent, their submission of the questionnaire was regarded as their consent. Both researchers stored the data and the data was discarded once the data was analysed. 3.3 Sample Following random sampling, we drew the actual study sample from public and private higher education institutions in Oman (teachers from public institutions, N = 726; teachers from private institutions, N = 122). Note that the numbers of teachers from each sector is not equal as there are more public institutions compared to private institutions in the country. The sample included 856 university teachers who experienced ERT during Covid-19 (63.3% males, 36.7% females). As shown in Table 1, the sample included teachers with various educational degrees. Exactly 70 of the participants held bachelor’s degrees (8.2%), 535 held master’s degrees (62.5%) and 251 held doctoral degrees (29.3%). Although the teachers’ teaching experience ranged from one to more than 15 years of experience, half of the sample had more than 15 years of teaching experience. The sample also included teachers from different specialisations, including social sciences, science, computer and information systems, education, business and management, engineering, mathematics and health. Four hundred and seventy-one of the teachers (55%) had completed teacher preparation programmes, whereas 385 teachers (45%) had not taken part in a teacher education programme. The majority of the participants (83.2%) had some experience in e-learning prior to the implementation of ERT during Covid-19. During ERT, the teachers used various devices for online teaching and learning.
http://ijlter.org/index.php/ijlter
182
Table 1: Demographic information of the research sample Variable Gender Male Female Academic Qualification Bachelor’s Degree Master's Degree Doctoral Degree Teaching Experience 0 to 5 Years 5 to 10 Years 10 to 15 Years More than 15 Years Specialisation Social Sciences Science Computer Science and Information Systems Education Business and Management Engineering Mathematics Health Institution Public Private Teacher Certification Yes No Experience in E-Learning Prior to ERT Yes No Devices Used for E-Learning in ERT Mobile/Handheld Device Computer/Laptop Both Total
N
%
542 314
63.3 36.7
70 535 251
8.2 62.5 29.3
32 123 237 464
3.7 14.4 27.7 54.2
129 40 124
15.1 4.7 14.5
110 88 283 68 14
12.9 10.3 33.1 7.9 1.6
726 122
84.8 14.3
471 385
55.0 45.0
712 144
83.2 16.8
13 249 594 856
1.5 29.1 69.4 100.0
4. Findings We used a set of descriptive statistical tests including means, standard deviations, Spearman’s correlation and one-way multivariate analysis of variance (MANOVA) to analyse the quantitative data. On the other hand, we used content analysis to analyse the qualitative data obtained through the openended questions. First, we familiarised ourselves with the data by reading the responses. Then, we defined grouping themes under which the initial data was classified. For question 1, we defined six emerging themes: policies and procedures, technological and infrastructure issues, student-related issues, curriculum-related issues, time-related issues and teachers’ experience in online
http://ijlter.org/index.php/ijlter
183
pedagogy. We then read the data, categorised it into six groups, and followed the same procedures to analyse the second open-ended question. 4.1 E-Learning Implementation Barriers As indicated earlier, the barriers were classified into four main categories: teacher-related barriers, institution-related barriers, curriculum-related barriers and student-related barriers; the results of which are displayed based on this classification. Our descriptive results revealed that teacher-related barriers did not significantly hinder the implementation of e-learning during Covid-19 (see Table 2). Overall, the participants believed that e-learning was useful (M = 1.99, SD = 1.11) and convenient (M = 2.24, SD = 1.07) for ERT during Covid-19. The participants also held that they were confident (M = 2.12, SD = 1.06) as they had sufficient knowledge (M = 2.15, SD = 1.06) and the necessary skills (M = 2.13, SD = 1.03) to use e-learning during the pandemic. Table 2: Descriptive results of teacher-related barriers Statement 1. I have sufficient knowledge to use e-learning for emergency remote teaching during the Covid-19 pandemic. 2. I have the necessary skills to use e-learning for emergency remote teaching during the Covid-19 pandemic. 3. I am confident in using e-learning for emergency remote teaching during the Covid-19 pandemic. 4. E-learning is useful for emergency remote teaching during the Covid-19 pandemic. 5. The use of e-learning for emergency remote teaching during the Covid-19 pandemic is convenient for me.
N
M
SD
856
2.15
1.06
856
2.13
1.03
856
2.12
1.06
856
1.99
1.11
856
2.24
1.07
Overall, institution-related barriers did not considerably obstruct the use of elearning (see Table 3). The participants confirmed that they had enough training (M = 2.22, SD = 1.11) and technical support (M = 2.16, SD = 1.16) from their institutions in implementing e-learning during the pandemic. Moreover, they agreed that the institutions provided them with a clear assessment policy (M = 2.42, SD = 1.13) and online conferencing tools (M = 2.21, SD = 1.11) for synchronous communication with their students. However, the teachers reported that their institutions encountered challenges in monitoring the quality of online teaching (M = 2.94, SD = 1.11). Table 3: Descriptive results of institution-related barriers Statement 1. My institution has a policy for emergency remote teaching. 2. My institution’s policies and regulations support the use of elearning for emergency remote teaching during the Covid-19 pandemic. 3. My institution has a clear assessment policy for emergency remote teaching during Covid-19. 4. My institution has online conference tools for synchronous communication with students and teachers. 5. My institution provided technical support for e-learning use for emergency remote teaching during Covid-19.
http://ijlter.org/index.php/ijlter
N 856
M 2.38
SD 1.14
856
2.21
1.12
856
2.42
1.13
856
2.21
1.11
856
2.16
1.16
184
6. My institution provided training on e-learning for emergency remote teaching during the Covid-19 pandemic. 7. My institution allowed me to design my own learning experiences. 8. My institution faced difficulty in monitoring the quality of online teaching in emergency remote teaching during the Covid-19 pandemic.
856
2.22
1.11
856
2.33
1.07
856
2.94
1.11
The curriculum-related barriers appeared to be more critical than the teacherrelated and institution-related barriers (see Table 4). The teachers reported that though learning and teaching materials and textbooks were available (M = 2.24, SD = 1.01) and suitable (M = 2.43, SD = 1.05), the nature of the courses made them difficult to teach online (M = 2.95, SD = 1.22). They also stated that the courses included materials that could not be delivered properly online (M = 2.94, SD = 1.11). Furthermore, the participants stated that some courses required faceto-face attendance for practical tasks (M = 2.52, SD = 1.32) or settings that allowed for a community of learning, neither of which were attainable during elearning (M = 2.94, SD = 1.14). Table 4: Descriptive results of curriculum-related barriers Statement 1. Learning and teaching resources that are available in the e-learning system are in accordance with the curriculum. 2. The textbooks that I use are suitable for e-learning use.
N
M
SD
856
2.24
1.01
856
2.43
1.05
3. Student's assessments are in line with e-learning use.
856
2.47
1.04
856
2.95
1.22
856
2.52
1.32
4. The nature of my course makes it difficult to be taught through elearning. 5. My course has practical tasks that require students to attend in person at a specific time. 6. My course has many materials that cannot be conveyed online.
856
2.98
1.27
7. The e-learning activities address different learning styles of students.
856
2.56
0.99
8. My course requires a community of learning, which was difficult to build in the e-learning system.
856
2.96
1.14
For the student-related barriers, the results (see Table 5) indicated that English language proficiency was a strong barrier (M = 3.00, SD = 1.11) in the use of elearning for ERT during Covid-19. Lack of training (M = 2.99, SD = 1.19), resistance to participation due to cultural norms (M = 2.96, SD = 1.06), and inadequate internet connection (M = 2.95, SD = 0.94) comprised the second, third and fourth barriers, respectively. The next most significant barriers were availability of devices (i.e. laptops and tablets) (M = 2.89, SD = 1.07), lack of interest in using e-learning (M = 2.80, SD = 1.04), lack of sufficient knowledge (M = 2.68, SD = 1.17) and lack of necessary skills (M = 2.66, SD = 1.13). On the other hand, ability to access the e-learning system (M = 2.55, SD = 1.01), ability to progress (M = 2.49, SD = 0.96), prior experience (M = 2.46, SD = 1.29), raising questions (M = 2.43, SD = 0.99), and completing course assignments (M = 2.30, SD = 1.00) were the least significant barriers, respectively.
http://ijlter.org/index.php/ijlter
185
Table 5: Descriptive results of student-related barriers Statement 1. My students used e-learning prior to Covid-19.
N 856
M 2.46
SD 1.29
2. My students have sufficient knowledge in the use of e-learning.
856
2.68
1.17
3. My students have the necessary skills for the use of e-learning.
856
2.66
1.13
856
2.89
1.07
856
2.99
1.19
856
2.80
1.04
7. My students have an internet connection.
856
2.95
0.94
8. My students are able to access the e-learning system.
856
2.55
1.01
856
3.00
1.11
856
2.30
1.00
856
2.43
0.99
856
2.49
0.96
856
2.58
1.03
856
2.66
1.00
856
2.96
1.06
4. My students have devices (i.e. laptops and tablets) for the use of elearning. 5. My students received training on the use of the e-learning system prior to the Covid-19 pandemic. 6. My students are interested in using e-learning.
9. My students' English skills are a barrier to using the e-learning system. 10. My students are able to do online assignments during Covid-19. 11. My students are able to raise questions and concerns during the course in the e-learning system. 12. My students are able to progress in the course during Covid-19. 13. My students find online assessment challenging. 14. My students find e-learning hectic because of the number of online courses during Covid-19. 15. My students are resistant to participate in e-learning activities due to cultural norms.
4.2 Relationship between E-Learning Implementation Barriers We conducted an analysis of Spearman’s correlation coefficient in order to pinpoint the relationship between each category of barriers to e-learning implementation. As Table 6 demonstrates, significant positive correlations existed among all levels, but the correlations were of different strengths. The results revealed that the strongest correlation was between teacher-related barriers and institution-related barriers (rₛ = .605, p = .000, N = 856). Meanwhile, the association between teacher-level barriers and student-related barriers was moderate, but still statistically significant (rₛ = .553, p = .000, N = 856). We found a moderate positive correlation between student-related barriers and institutionrelated barriers (rₛ = .548, p = .000, N = 856) and between curriculum-related barriers and institutional-related barriers (rₛ = .444, p = .000, N = 856). Moreover, there was only a weak positive correlation between student-related barriers and curriculum-related barriers (rₛ = .364, p = .000, N = 856), and the lowest positive correlation among all the categories was between curriculum-related barriers and teacher-related barriers (rₛ = .271, p = .000, N = 856).
http://ijlter.org/index.php/ijlter
186
Table 6: The relationship between each category of barriers to e-learning implementation Teacher-Level Barriers Teacher-Level Barriers
1.000
InstitutionLevel Barriers
InstitutionLevel Barriers
CurriculumLevel Barriers
Student-Level Barriers
.605**
.271**
.553**
1.000
.444**
.548**
1.000
.364**
CurriculumLevel Barriers
Student-Level 1.000 Barriers * Correlation interpretation scale: .00–.19 = very weak, .20–.39 = weak, .40–.59 = moderate, .60– .79 = strong, .80–1.0 = very strong. ** Correlation is significant at the 0.01 level (2-tailed).
4.3 Differences in E-Learning Implementation Barriers in the Context of Teachers’ Gender, Academic Qualifications, Teaching Experience and Prior Experience in E-Learning We administered MANOVA to investigate any possible significant differences in e-learning implementation barriers in the context of the teachers’ gender, academic qualifications, teaching experience, and prior experience in e-learning (see Table 7). There was a statistically significant difference between e-learning implementation barriers and academic qualifications: F (8, 1700) = 4.51, p < .05; Wilk's Λ = 0.959, partial η2 = .021. The MANOVA also revealed significant differences in e-learning implementation barriers due to teaching experience: F (12, 2246) = 2.07, p < .05; Wilk's Λ = 0.971, partial η2 = .010. There were also significant differences with prior experience in e-learning: F (4, 851) = 12.66, p < .05; Wilk's Λ = 0.944, partial η2 = .056. Furthermore, the results did not indicate any significant difference in e-learning implementation barriers resulting from gender: F (4, 851) = 1.99, p < .05; Wilk's Λ = 0.991, partial η2 = .009. Although the difference between these variable (gender, academic qualification, teaching experience, prior experience in e-learning) are minor, the results indicate that they influenced teacher implementation of e-learning during COVID-19. Table 7: Results of the MANOVA Effect Gender Academic Qualification Teaching Experience Prior Experience in E-Learning
Wilks' Lambda Wilks' Lambda Wilks' Lambda Wilks' Lambda
http://ijlter.org/index.php/ijlter
Value
F
Sig.
Partial Eta Squared
.991
1.992b
.094
.009
.959
4.509b
.000
.021
.971
2.073
.016
.010
.944
12.662b
.000
.056
187
4.4 The Relationship between E-Learning Implementation Barriers and Type of Institution We conducted a Spearman’s correlation coefficient to investigate any possible relationship between e-learning implementation barriers and type of institution. The results revealed a very weak, but still statistically significant, positive association between curriculum-level barriers and the type of institution (rₛ = .071, p = .05, N = 848). This can be due to the different curriculum development and evaluation frameworks and guidelines implemented at different institutions. Also. Private higher education tend to have flexible curriculum guidelines where adapting and enhancing curriculum is accessible. However, curriculum change is more rigid and centralised in public higher education. Table 8 shows the relationship between e-learning implementation barriers and type of institution. The results of this test should be interpreted cautiously as the number of teachers in the sample were more from public institutions. Table 8: The relationship between e-learning implementation barriers and type of institution
Institution
TeacherLevel Barriers .000
InstitutionLevel Barriers -.067
CurriculumLevel Barriers .071*
StudentLevel Barriers .019
* Correlation is significant at the 0.05 level (2-tailed).
4.5 Results of the Open-Ended Questions There was a good response rate for the open-ended questions in the questionnaire (the number of responses for question 1 = 590, and the number of responses for question 2 = 582). The first open-ended question encouraged participants to voice any issues they faced during ERT, while the second question allowed participants to provide suggestions to improve e-learning implementation practices for ERT during Covid-19. The results revealed that teachers faced various issues with ERT during Covid19. First, the teachers reported facing issues related to technological infrastructure, including poor internet connectivity and lack of hardware and software for online teaching and learning. Consequently, these issues contributed to the failure of many of the online classes conducted during Covid19. Our results also indicated some student-related issues. According to the teachers, many students encountered challenges due to poor network coverage, and students were not motivated to attend their online classes. Teachers attributed this issue to a lack of proper training on the use of live streaming platforms, course management systems and online learning skills and strategies. Moreover, the teachers reported issues with monitoring student attendance during the online classes. Some respondents reported that these difficulties were related to the students’ joining and leaving classes frequently during the same online class due to internet connection issues.
http://ijlter.org/index.php/ijlter
188
In addition, the results revealed that skills and knowledge of online teaching and learning were insufficient for some of the teachers. Accordingly, their inexperience in online pedagogy was reported in two contexts. First, the participants who faced these challenges were not competent in using the required applications for online teaching and learning, such as course management systems and video conferencing software. Second, they demonstrated insufficient knowledge of online pedagogy. In other words, they lacked knowledge of the teaching methods required to administer online instruction. Furthermore, the participants reported issues with the curriculum. A common issue was that the existing face-to-face courses were not suitable for online teaching; the courses contained materials that were challenging for teachers to cover through online classes. Finally, the course assessments for the face-to-face classes were not suitable for online classes. Our analysis showed that there were issues in relation to institutions’ policies and guidelines. The participants reported that their institutions lacked online teaching and learning policies and guidelines. They also stated that their institutions were unable to monitor and assess the quality of e-learning during this period due to a lack of specific quality assurance measures for ERT. Meanwhile, only a few respondents mentioned issues with time; those who reported time as an issue stated that they found online teaching to be timeconsuming. The teachers who participated in our study provided a variety of recommendations to enhance online teaching and learning through ERT during Covid-19. The majority of the respondents emphasised the need for proper student training, which would benefit the quality of both teaching and learning in the online context. They recommended student training in various areas, including video streaming software, learning management systems and screen and voice recording software. The teachers also recommended enhancements to the technological infrastructure. Their recommendations can be divided into two main parts. Some of the recommendations focused on enhancing internet connectivity at both the national and institutional levels. In addition, the participants recommended providing and upgrading software, hardware and network facilities at the institutional level. Some teachers also recommended establishing video recording studios to aid in recording professional videos for online classes. Furthermore, the teachers recommended some amendments to teaching and learning policies and institutional guidelines. Some of the teachers recommended establishing new policies that are specific to ERT, including changes to attendance policies, quality assurance and quality enhancement policies, and student support services. Finally, the participants recommended a redesign of courses and assessments. Teachers who taught practical courses asserted the need to change assessment patterns to suit online instruction, and some of the teachers recommended
http://ijlter.org/index.php/ijlter
189
including short quizzes prior to and after classes to ensure student readiness and to monitor and assess their understanding of the content.
5. Discussions 5.1 Students preparedness for Emergency Remote Teaching The current study reveals that students were not prepared for ERT, and they demonstrated insufficient online learning skills, strategies and knowledge. Similar observations have been reported in previous research. Alqudah et al. (2020) identified poor e-learning skills, unpreparedness and inadequate internet accessibility for students as the main obstacles hindering the continuity of learning in the ERT period in Jordan. Research has also confirmed that student issues such as readiness, motivation, accessibility and equity negatively impacted e-learning practices during Covid-19 (Adnan & Anwar, 2020; Bao, 2020; Hartshorne et al., 2020; Landrum, 2020; Mailizar et al., 2020). Prior Covid19 research has confirmed similar findings (Assareh & Hosseini, 2011; Naveed et al., 2017; Pelgrum, 2001). Therefore, this suggests that students themselves might pose a serious challenge to e-learning implementation in an ERT context. Another possible explanation of this issue that should be considered, however, is that institutions may not have had sufficient time to prepare their students for the sudden shift to ERT. 5.2 Curriculum and E-Learning Implementation during COVID-19 The positive correlation between curriculum-related and institutional-related barriers signifies that the existing curriculum is not suited for ERT. A possible reason for this might be that most of the courses offered during ERT were not subject to sound instructional design due to the emergency situation. This is supported by Hodges et al.'s (2020) argument that online course design during ERT is often rushed with minimum resources, and little attention is paid to typical planning and preparation. This finding also confirms the work of Crawford et al. (2020) and Bao (2020), both of whom claimed that shifting courses to online delivery during Covid-19 lacked systematic course design and evaluation. Furthermore, recent research has pointed out the need for systematic planning when shifting to online instruction. Karalis (2020), for example, emphasised that piloting – piloting the new course, assessment and teaching materials – is an essential and integral part of shaping proper responses to emergencies in education. Meanwhile, along the same line of thought, Gacs et al. (2020) stressed the importance of a quick needs analysis to understand teacher and student needs in relation to technology, workload, accessibility, equity, readiness, motivation and prior knowledge. 5.3 Institutional Readiness for Emergency Remote Teaching The positive correlation between teacher-related barriers and institution-related barriers can explain the unpreparedness for the shift to ERT during Covid-19. Our results confirm the lack of digital education policies and procedures at most higher education institutions in Oman. Additionally, the dearth of quality assurance measures and polices concerned with emergency teaching and learning negatively affected ERT practices. This finding relates not only to Omani higher education, but can be generalised to educational systems worldwide (see Crawford et al., 2020). Findings from the current research indicate the need for teacher professional development, especially since
http://ijlter.org/index.php/ijlter
190
inadequate ICT skills in teachers has been reported as a crucial hindrance of elearning implementation (Alqudah et al., 2020; Assareh & Hosseini, 2011; Bao, 2020; Chang & Fang, 2020; Crawford et al., 2020; Naveed et al., 2017; Pelgrum, 2001; Sharpe et al., 2003). 5.4 Other Influential Barriers Academic qualifications and teaching experience played a significant role in elearning implementation for ERT during Covid-19. Although teachers’ academic qualifications and teaching experience have a positive effect on their implementation of e-learning, teachers advocated for training provisions specifically related to the ERT context. Previous research confirms the finding that prior experience and academic qualifications of teachers positively correlate to the successful implementation of e-learning (Cidral et al., 2018; Hartshorne et al., 2020; Mahdizadeh et al., 2008; Mailizar et al., 2020; Naveed et al., 2017; Reeves & Pedulla, 2011). Meanwhile, the type of institution did not lead to any significant differences in e-learning implementation for ERT. In other words, the experience of public and private educational providers was the same, which indicates that neither sector was fully prepared for the sudden shift to ERT. 5.5 Requirements for Successful Emergency Remote Teaching The evidence from this study suggests that higher education institutions should pay attention to student training provisions. Higher education providers should consider providing and/or enhancing students’ internet access to maximise their ability to participate in online learning. Although our findings relate to elearning implementation for ERT during Covid-19, they are applicable to normal teaching settings and face-to-face teaching that depends, to some extent, on elearning solutions. The current study also highlights the need for a national policy for digital delivery of higher education and a national policy for ERT. Although the shift to online instruction was accomplished rapidly, it is important that institutions follow a systematic plan for the shift to online instruction that considers the needs of both teachers and students. Additionally, it is necessary that periodic and quick evaluation of courses that are shifted online during ERT be implemented. While it might be argued that this may not be possible or feasible during exceptional circumstances, institutions should ensure that they review and pilot parts of their courses concurrently to online instruction. Student, teacher and parent feedback should also be considered when evaluating online instruction during ERT. Finally, this work demonstrates the importance of enhancing institutional technological infrastructure. 5.6 Limitations and Future Research A number of limitations need to be considered. First, the study only included English-speaking teachers. The questionnaire was not translated into Arabic, which is spoken by many university teachers in Oman. Second, the study sample only included teachers. Including students in the study sample would have affected the conclusions we were able to draw, especially with regard to student-related barriers to e-learning implementation. However, the findings of this study do bring up a call to teachers, institutions and policy makers to further investigate issues related to e-learning in ERT
http://ijlter.org/index.php/ijlter
191
contexts. Future research should explore student training provisions in online learning during ERT and could also investigate the professional development of higher-education teachers in ERT contexts. Design-based research can be useful for designing and piloting teacher- and student-training programmes, and would be useful to assess institutions’ readiness for online instruction and course design during COVID-19 and other unexpected emergencies in the future.
6. Conclusion Using a cross-sectional survey design, we examined teachers’ views on the implementation of e-learning in ERT during COVID-19. Student-related barriers had a strong impact on teachers’ use of e-learning with teachers’ academic qualifications and prior experience having a less significant impact on teachers’ use of e-learning in ERT during COVID-19. The study recommends training programmes for both students and teachers to prepare them for handling learning and teaching in ERT contexts. In addition, the study recommends higher education institutions to adapt their curriculum to suite ERT. As teachers were not prepared for the sudden shift to ERT during COVID-19, higher education providers need to implement professional development provisions and amend their teaching and learning policies to suite the ERT context.
7. References Adnan, M., & Anwar, K. (2020). Online learning amid the COVID-19 pandemic: Students’ perspectives. Journal of Pedagogical Sociology and Psychology, 2(1), 45–51. https://doi.org/10.33902/jpsp.2020261309 Alqudah, N. M., Jammal, H. M., Saleh, O., Khader, Y., Obeidat, N., & Alqudah, J. (2020). Perception and experience of academic Jordanian ophthalmologists with ELearning for undergraduate course during the COVID-19 pandemic. Annals of Medicine and Surgery, 59(September), 44–47. https://doi.org/10.1016/j.amsu.2020.09.014 Assareh, A., & Hosseini, B. M. (2011). Barriers to E-teaching and E-learning. Procedia Computer Science, 3, 791–795. https://doi.org/10.1016/j.procs.2010.12.129 Bao, W. (2020). COVID ‐19 and online teaching in higher education: A case study of Peking University. Human Behavior and Emerging Technologies, 2(2), 113–115. https://doi.org/10.1002/hbe2.191 Baran, E., & Alzoubi, D. (2020). Human-centered design as a frame for transition to remote teaching during the COVID-19 pandemic. Technology and Teacher Education, 28(2), 365–372. Chang, C. L., & Fang, M. (2020). E-Learning and Online Instructions of Higher Education during the 2019 Novel Coronavirus Diseases (COVID-19) Epidemic. Journal of Physics: Conference Series, 1574(1). https://doi.org/10.1088/17426596/1574/1/012166 Cidral, W. A., Oliveira, T., Felice, M. Di, & Aparicio, M. (2018). E-learning success determinants: Brazilian empirical study. Computers and Education, 122, 273–290. https://doi.org/10.1016/j.compedu.2017.12.001 Crawford, J., Butler-henderson, K., Rudolph, J., Malkawi, B., Glowatz, M., Magni, P., & Lam, S. (2020). COVID-19: 20 countries’ higher education intra-period digital pedagogy responses. Journal of Applied Learning & Teaching, 3(1), 1–20. https://doi.org/10.37074/jalt.2020.3.1.7 Creswell, J. W. (2018). Educational Research: Planning, Conducting and Evaluating Quantitative and Qualitative Research (6th edition). Pearson.
http://ijlter.org/index.php/ijlter
192
Demuyakor, J. (2020). Coronavirus (COVID-19) and online learning in higher institutions of education: A survey of the perceptions of Ghanaian international students in China. Online Journal of Communication and Media Technologies, 10(3). https://doi.org/10.29333/ojcmt/8286 Dhawan, S. (2020). Online learning: A panacea in the time of COVID-19 crisis. Journal of Educational Technology Systems, 49(1), 5–22. https://doi.org/10.1177/0047239520934018 Fryer, L. K., & Bovee, H. N. (2016). Supporting students’ motivation for e-learning: Teachers matter on and offline. Internet and Higher Education, 30, 21–29. https://doi.org/10.1016/j.iheduc.2016.03.003 Gaber, D. A., Shehata, M. H., & Amin, H. A. (2020). Online Team- Based Learning Sessions as Interactive Methodologies During The Pandemic. Medical Education, March, 14198. https://doi.org/10.1111/medu.14198 Gacs, A., Goertler, S., & Spasova, S. (2020). Planned online language education versus crisis-prompted online language teaching: Lessons for the future. Foreign Language Annals, May, 1–13. https://doi.org/10.1111/flan.12460 González-gómez, F., Guardiola, J., Martín, Ó., Ángel, M., & Alonso, M. (212 C.E.). Gender differences in e-learning satisfaction. Computers & Education, 58(1), 283– 290. Hartshorne, R., Baumgartner, E., & Kaplan-rakowski, R. (2020). Preservice and inservice professional development during the COVID-19 pandemic. Journal of Technology and Teacher Education, 28(2), 137–147. Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond, A. (2020). The difference between emergency remote teaching and online learning. Educause Review, 7. https://er.educause.edu/articles/2020/3/the-difference-between-emergencyremote-teaching-and-online-learning Jones, A. (2003). ICT and future teachers Are we preparing for e-learning. In IFIP Advances in Information and Communication Technology (Vol. 132). https://doi.org/10.1007/978-0-387-35701-0 Karalis, T. (2020). Planning and evaluation during educational disruption: Lessons learned from Covid-19 pandemic for treatment of emergencies in education. European Journal of Education Studies, 7(4), 2020. https://doi.org/10.5281/zenodo.3789022 Landrum, B. (2020). Examining students’ confidence to learn online, self-regulation skills and perceptions of satisfaction and usefulness of online classes. Online Learning Journal, 24(3), 128–146. https://doi.org/10.24059/olj.v24i3.2066 Mahdizadeh, H., Biemans, H., & Mulder, M. (2008). Determining factors of the use of elearning environments by university teachers. Computers and Education, 51(1), 142–154. https://doi.org/10.1016/j.compedu.2007.04.004 Mailizar, Almanthari, A., Maulina, S., & Bruce, S. (2020). Secondary school mathematics teachers ’ views on e-learning implementation barriers during the COVID-19 pandemic: The case of Indonesia. Eurasia Journal of Mathematics, Science and Technology Education, 16(7). https://doi.org/10.29333/ejmste/8240 McMaster, D., Veremu, M., & Santucci, C. (2020). COVID-19: opportunities for professional development and disruptive innovation. Clinical Teacher, 17(3), 238– 240. https://doi.org/10.1111/tct.13175 Naveed, Q. N., Muhammed, A., Sanober, S., Qureshi, M. R. N., & Shah, A. (2017). Barriers effecting successful implementation of E-learning in Saudi Arabian universities. International Journal of Emerging Technologies in Learning, 12(6), 94– 107. https://doi.org/10.3991/ijet.v12i06.7003 Pelgrum, W. J. (2001). Obstacles to the integration of ICT in education: Results from a
http://ijlter.org/index.php/ijlter
193
worldwide educational assessment. Computers and Education, 37(2), 163–178. https://doi.org/10.1016/S0360-1315(01)00045-8 Rapanta, C., Botturi, L., Goodyear, P., Guàrdia, L., & Koole, M. (2020). Online university teaching during and after the Covid-19 crisis: Refocusing teacher presence and learning activity. Postdigital Science and Education, 2, 923–945. http://link.springer.com/10.1007/s42438-020-00155-y Reeves, T. D., & Pedulla, J. J. (2011). Predictors of teacher satisfaction with online professional development: Evidence from the USA’s e-learning for educators initiative. Professional Development in Education, 37(4), 591–611. https://doi.org/10.1080/19415257.2011.553824 Schoepp, K. (2005). Learning and Teaching in Higher Education: Gulf Perspectives Barriers to Technology Integration in a Technology-Rich Environment. Learning and Teaching in Higher Education: Gulf Perspectives, 2, 1–24. http://www.zu.ac.ae/lthe/vol2no1/lthe02_05.pdf Sharpe, R., Benfield, G., Roberts, G., Francis, R., Garrison, D. R., Anderson, T., Welsh, E. T., Wanberg, C. R., Brown, K. G., Simmering, M. J., Sun, P. C., Tsai, R. J., Finger, G., Chen, Y. Y., Yeh, D., Selim, H. M., Guri-Rosenblit, S., Chan, T.-W., Roschelle, J., … Walton, G. (2003). What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education, 49(4), 245–258. https://doi.org/10.1111/j.1470-3327.2005.00614.x Sørebø, Ø., Halvari, H., Gulli, V. F., & Kristiansen, R. (2009). The role of selfdetermination theory in explaining teachers’ motivation to continue to use elearning technology. Computers and Education, 53(4), 1177–1187. https://doi.org/10.1016/j.compedu.2009.06.001 UNESCO. (2020). Global Education Coalition: Covid-19 Education Response. https://en.unesco.org/covid19/educationresponse/globalcoalition Wilson, A. (2012). Effective professional development for e-learning: What do the managers think? British Journal of Educational Technology, 43(6), 892–900. https://doi.org/10.1111/j.1467-8535.2011.01248.x Zuo, L., Dillman, D., & Miller Juve, A. (2020). Learning At-Home During COVID-19: A Multi-institutional Virtual Learning Collaboration. Medical Education, March, 1–2. https://doi.org/10.1111/medu.14194
http://ijlter.org/index.php/ijlter
194
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 194-209, August 2021 https://doi.org/10.26803/ijlter.20.8.12 Received Jun 19, 2021; Revised Aug 19, 2021; Accepted Aug 30, 2021
Implications of Social Media Addiction on Academic Performance among Generation Z Student-athletes during COVID-19 Lockdown Wan Ahmad Munsif Wan Pa, Muhammad Sofwan Mahmud* and Mohd Syazwan Zainal Universiti Kebangsaan Malaysia, Malaysia https://orcid.org/0000-0001-6218-4376 https://orcid.org/0000-0002-0504-4622 https://orcid.org/0000-0002-1450-2760 Abstract. With the rapid advancement of internet technology, the increasing prevalence of smartphone ownership at a young age, the uncertainty surrounding the end of the COVID-19 pandemic, and the effect of social media addiction on academic performance among students, social media use will continue to be a major point of contention. However, student-athletes are a distinct group of students who should not be overlooked due to their concurrent commitments to studying, training, and competing. As such, this study sought to investigate the effect of social media addiction on academic performance among Generation Z student-athletes during Malaysia's COVID-19 lockdown. The research design used in this study was a quantitative cross-sectional survey. This study surveyed 91 respondents and utilized the Social Media Addiction Questionnaire (SMAQ) and Grade Point Average (GPA). The findings indicated that Facebook was the preferred social media platform over Instagram, Twitter, and YouTube, and that more than half of the participants were social media addicts. The researchers concluded that social media addiction had a significant impact on academic performance, accounting for 57.6 percent of the variance. These findings imply that excessive social media use, leading to addiction, has a significant negative effect on academic performance. Solutions are desperately needed, such as turning off notifications, silencing the phone, engaging in home workouts, reconnecting with family, or taking up new hobbies. Keywords: social media addiction; academic performance; studentathletes
*
Corresponding author: Muhammad Sofwan Mahmud; sofwanmahmud@ukm.edu.my
©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
195
1. Introduction Coronavirus 2019 (COVID-19) was discovered for the first time in December 2019 in Wuhan, Hubei Province, China, causing the World Health Organization (WHO) to declare it a global pandemic which poses a physical and psychological threat to the public. The continued spread of COVID-19 is a public health emergency and a global hazard. To prevent the disease from spreading further, governments have issued emergency orders for people to stay at home, while universities and schools have been closed. It has been estimated that, by March 26, 2020, the closures affected more than 150 million children and adolescents in 165 nations (UNESCO, 2020). Due to the COVID-19 pandemic, higher education institutions have been forced to implement emergency online learning (Yahya, Rahman & Siddiq, 2021). On top of that, this pandemic has become a source of stress since it has had an impact on all sectors of human existence, including economy, social activities, health, education, politics, culture, and communication. Based on statistics provided by the Ministry of Health (2021), Selangor was one of the states in Malaysia with the highest number of daily positive COVID-19 cases during the movement control order 2.0 (MCO). During this pandemic lockdown, communication has encompassed much more than just sending messages to people. Communication is the process of conveying or exchanging information through speech, writing, or any other medium (Buck, 2019). Additionally, information and communication technologies (ICTs) are now widely used in virtually every aspect of human life, including entertainment, education, decision-making, information searching, commerce, and business. In terms of benefits, ICTs can help people to save time, effort, and money (Filimon et al., 2017). The importance of communication through social media is obviously seen as a funnel for information between family members, education, entertainment, and other activities, as well as a vehicle for a child's socialization and education from birth to adulthood. Social media platforms are one of the most widely used forms of information on the planet. Social media platforms include Facebook, Instagram, Twitter, Snapchat, YouTube, blogging platforms, Newsfeeds, Wikis, and social gaming platforms (Kuss et al., 2014). On the other hand, excessive use of social media has developed into a problem for some individuals who exhibit signs of behavioral addiction such as salience, mood modulation, withdrawal, tolerance, and conflict (Hawi & Samaha, 2019). However, a study on social media addiction among Generation Z studentathletes was limited by the fact that previous research focused exclusively on sedentary Generation Z students (Curtis et al., 2019; Gazi & Çetin, 2017; Grau, Kleiser & Bright, 2019). The requirements of a student-athlete's academic course must be balanced with their athletic activity. Ryba et al. (2015) asserted that the economic, structural, and institutional rules in place create a one-of-a-kind structure that enables a student-athlete to pursue a successful dual profession. These Generation Z student-athletes are not immune to the internet's rapid growth, technological advancements, and social media proliferation.
http://ijlter.org/index.php/ijlter
196
2. Problem Statement When COVID-19 isolates households from one another, parents and guardians may become concerned about their own financial future, making learning at home more difficult, particularly for children who lack motivation (Daniel, 2020). Students' lives have been significantly impacted by the COVID-19 pandemic in a variety of ways, depending on their level and field of study, as well as their program of study. Additionally, several students have found themselves unable to complete their university curriculum and assessments in a timely manner, and in many cases, they have been immediately cut off from their social group. Social media is critical as a communication tool for conveying information in these circumstances. Individuals frequently turn to the media in response to everyday annoyances and adversity. According to Zhao and Zhou (2021), when confronted with COVID-19 problems, such as venting emotions for mood management or joining online communities for social support, people tended to turn to social media for problem-focused activities, such as searching for health-related information and emotion-focused coping. Despite the obvious benefits of social media in an emergency, such as COVID19, increased social media use is likely to result in social media addiction (Zhao & Zhou, 2021). Social media's positive and negative effects on society are well established (Bajpai, 2018). On the other hand, they promote a sense of belonging and an excellent marketing or communication tool for businesses, political parties, and governments. Interacting with others on social media aids in the development of communication and technical abilities. Meanwhile, concerns have been raised about possible links between frequent social media use and anxiety and depression (Pa et al., 2020), as well as issues surrounding cyberbullying, online harassment, and "trolling." According to Hootsuite and We Are Social's digital usage studies, Malaysia has the fifth-highest mobile social media penetration rate in the world and the highest in Southeast Asia (Bernama, 2019). According to the same survey, internet penetration has reached 81% in Malaysia, with users spending an average of eight hours and five minutes online. During the same time period, individuals spent two hours and 59 minutes on social media platforms like Facebook, Instagram, and Twitter. On the other hand, many individuals take advantage of these opportunities by devoting significant time to social media (Schulze, Scholer & Skiera, 2015). Numerous studies have established a link between social media use and academic achievement and performance among international undergraduate university students. According to a nationwide survey of 3000 college students, 90% of students use Facebook and 37% use Twitter (Dahlstrom et al., 2011). As a result of the long hours they spend on social media, students are more susceptible to addiction, which can result in assignment procrastination, loss of focus, and can have a negative impact on their academic performance. Moreover, student-athletes are among those affected by the lockdown pandemic. They had to cancel training and tournament for an uncertain period of time. Furthermore, they have a lot of free time and can easily become addicted
http://ijlter.org/index.php/ijlter
197
social media users. This situation is supported by statistics reported by Kashif and Aziz-Ur-Rehman (2020) in which most people are becoming addicted to social media because of the spare time they have gained due to the suspension of all outside activities; many also reported feeling agitated and nervous if they did not use social media during the coronavirus lockdown. People have been spending more time on social media, which causes them to be distracted and delayed in their work. Additionally, there is evidence of the devastating effect of social media on academic success, particularly Facebook, and smartphone use (Paul, Baker, & Cochran, 2012; Junco & Cotton, 2012; Samaha & Hawi, 2016; Lepp, Barkley, & Karpinski, 2015; Samaha & Karpinski, 2016). When these tools are combined, students spend more time interacting with them than they do learning. Recent studies indicate that students' excessive use of social media can have a detrimental effect on their academic performance (Skiera, Hinz, & Martin-Span, 2015; Al-Menayes, 2014; Skiera, Hinz, & Martin-Span, 2015). Due to their pervasiveness, students are more susceptible to social media and smartphones than the general population (Giunchiglia et al., 2018). On the other hand, student-athletes are a category of students that should not be ignored because they are committed to three main goals at the same time, which are studying, training, and competing. As a result of the difficulties, the researchers gained a better understanding of the negative impact of social media addiction on academic performance. Additionally, this type of study on Malaysian Generation Z student-athletes was uncommon, as previous studies focused on the general university student population. The most frequently used social media platform by Generation Z student-athletes in Malaysia is unknown. It is critical to understand the mediators of the relationship between social media addiction and academic performance in order to identify risk factors and develop social media addiction prevention strategies. Indeed, these issues inspired the researchers to conduct a study on the impact of social media addiction on academic performance. Therefore, the present research aims to: 1. To recognize the most frequently used social media platforms by Generation Z student-athletes during MCO 2.0. 2. To identify the extent to which Generation Z student-athletes are addicted to social media. 3. To determine the impact of social media addiction on academic performance in Generation Z student-athletes. The research questions: i. What is the most common type of social media platform used by Generation Z student-athletes during MCO 2.0? ii. What is the level of social media addiction among Generation Z studentathletes? iii. What is the effect of social media addiction on academic performance among Generation Z student-athletes?
http://ijlter.org/index.php/ijlter
198
The null hypothesis: Ho: There is no significant impact of social media addiction on academic performance among Generation Z student-athletes.
3. Literature Review During the COVID-19 pandemic, excessive levels of stress and addiction to social media are positively related (Brailovskaia et al., 2019). At the same time, people are helpless because they need to maintain their social media activity and communicate with their loved ones through social media (Perrin, 2015). Indeed, many people have been spending an increasing amount of time on social media platforms like Facebook and Twitter, in addition to remote work and study, which may satisfy their needs for disaster-related information, entertainment, and interpersonal communication (Merchant & Lurie, 2020). Zhao and Zhou (2021) reported that when individuals are exposed to COVID-19-related stress, they are more likely to actively engage with social media, such as sharing personal feelings in order to alleviate unpleasant emotions. According to statistics, the total number of users on social media has been increasing by 10 percent on a yearly basis, (Hawi & Samaha, 2017). Numerous researchers and academics are examining the effects of social media technologies and applications on various aspects of people's lives, as they have become an inseparable part of daily life in recent years (Filimon et al., 2017). With the increasing use of smartphone technology, social media usage among adults aged 18 to 29 years in the United States of America has surged from 12 percent to 90 percent since 2005 (Sutherland et al., 2018). Due to the ease and low cost of Internet access and the enormous number of registered users on social media, these platforms are one of the simplest and most effective methods of spreading and obtaining information. Whether it is a sporting event, disease, or natural disaster, the overall response to important events is usually to search for information (González-Padilla, 2020). According to a recent study, users use social media for a variety of reasons, including seeking knowledge, seeking social engagement, avoiding boredom and leisure, fleeing unpleasant emotions, and seeking happy ones (Brailovskaia, Schillack & Margraf, 2020). Thus, social media has a great impact on how people connect, work, and share information. Social media networking via smartphone has also become the fastest growing method of affiliation due to its ability to quickly transmit essential information. Although social media has had an impact on a variety of businesses, it has had the most significant impact on classroom teaching and the educational system as a whole (Sivakumar, 2020). Some institutions use social media to promote programs and engage with current and potential students, as well as alumni, due to the popularity of social media. Some academics have embraced social media to improve the educational process and cooperate on projects, as well as to improve learning outcomes by communicating with students more effectively. According to the literature, undergraduate students at bachelor level are the ones who use social media platforms to engage with other students and university professors (Duggan et al., 2015; Head, 2013; Kim, Sin & Tsai, 2014).
http://ijlter.org/index.php/ijlter
199
Students' use of social media applications has exploded in the last decade, and the impact on academic performance has been mixed (Alnjadat et al., 2019). A survey of 30 male students at the Petroleum Institute in Abu Dhabi, in the United Arab Emirates (UAE), discovered that students with a high Grade Point Average (GPA) spend a significant amount of time on social media websites and applications (Tayseer et al., 2014). According to Alahmar (2017) and Boateng and Amankwaa (2016), social media was used positively in the learning process in China and Iraq, with students using the platform to discuss assignments and other coursework; receive news and updates on class schedules and locations; exchange information with peers; and express concerns about their academic work. Similarly, research conducted in Australia in 2018 reported that 67 percent of 106 undergraduate students polled cited technology as a factor in their successful academic achievement (Sutherland et al., 2018). Thus, social media has become ingrained in the lives of Internet users, and the increasing pervasiveness of smartphones is just accelerating this trend (Giunchiglia et al., 2018). According to Al-Menayes (2014), while strolling, traveling by bus or train, or even driving a car, smartphones can be used as instructional and communication devices. A decade ago, micro time intervals during which users could engage in a variety of online activities were not possible. Micro time slots, on the other hand, can result in compulsive smartphone use, which prevents face-to-face interaction and has a detrimental effect on academic performance. Students who spent an excessive amount of time interacting on social networking sites were more likely to fail their classes (Alnjadat et al., 2019). Additionally, students who were heavily involved in social networking site activities while studying lacked focus and were easily distracted (Okyeadie Mensah & Nizam, 2016). On top of that, Generation Z is the generation that has grown up with a lot of technology, gadgets, and access to the Internet. According to Glass (2007) in Salleh, Mahbob and Baharudin (2017), a generation is defined as the length of time that individuals live in a certain year or a cohort of that year. Generation Z is the most recent generation to be born, and will rule the planet for the next decades (Wiedmer, 2015; Salleh, Mahbob & Baharudin 2017). Singh (2014), Jaleniauskiene and Juceviciene (2015) and Schroer (2016) defined Generation Z as those born between 1995 and 2012. This generation appears to have been genetically modified by the Internet since birth. This is because they were born after the Internet became widely used around the world. Table 1 shows the dominant characteristics of each generation in the 20th and 21st centuries (MacKenzie & McGuire, 2016; Glass, 2007; Wiedmer, 2015): Table 1: Generation dominant characteristic Generation Traditionalist Baby Boomers
Range of birth years 1927-1945 1946-1964
Generation X
1965-1980
http://ijlter.org/index.php/ijlter
Dominant behavioral characteristics Disciplined and loyal Work ethics that are responsible and strong Thinkers who are autonomous and productive
200
Generation Y
1981-1994
Generation Z
1995-2012
More self-assured in social situations, but less self-sufficient Inadequate communication skills, excessive reliance on technology
According to Salleh, Mahbob and Baharudin (2017), most of the characteristics described have yet to manifest because this generation were born during technological explosions. Table 2 illustrates the four specific characteristics of Generation Z: Table 2: Characteristics of Generation Z Characteristics Instant gratification
Hypertext mindset Overprotected
Ineffective communication abilities
Description Generation Z has become impatient, requiring immediate gratification, introverted, and disconnected from society because of extensive internet exposure. Generation Z prefers to watch videos over reading hardcopy materials, such as books and manuals. Spoon-feeding by parents is a common occurrence in Generation Z. They are likely to demand that institutions create a similar environment in this scenario. Generation Z is described as being more individualistic, self-absorbed, and team-oriented than previous generations on a limited basis.
4. Theoretical Framework and Conceptual Framework Active use of social media could contribute to the effects of COVID-19-related stress on the relationship between social media addiction and academic performance. Active use refers to activities that enable direct communication with others, such as commenting on friends' posts, tagging, "liking," updating one's status or sharing images or videos. By contrast, passive use refers to the act of browsing news feeds or viewing other people's posts without engaging in direct communication (Burke, Kraut & Marlow, 2011). A student-athlete, according to Pato et al. (2014), is a full-time university or high school student who participates in athletics or sports independently or as a member of a federation, club, or sports organization. Therefore, student-athletes need to divide their time between training, competition, and studies as well. In this study, the researchers adopt Social Learning Theory (SLT) as a fundamental knowledge to understand the addiction to social media among Generation Z student-athletes. According to the SLT, children and adolescents learn through observing their surroundings, especially those who are close to them (Bandura, 1977). People are devising new strategies and techniques to inform, prepare, and teach others, which is a human phenomenon known as planned socialization. Both natural and deliberate socializing, according to Ransome and Sampford (2016), are beneficial to living a purposeful life. Figure 1 illustrates the theoretical framework that consists of three components under SLT, adapted from Bandura (1977) in Ghazali and Ghani (2018), while Figure 2 illustrates the conceptual framework that consists of four sections.
http://ijlter.org/index.php/ijlter
201
Personal factors
Social Learning Theory Behavior Factors
Environment factors
Figure 1: Interconnectedness of behavioral, environmental, and personal aspects that drive Generation Z behavior.
• • •
Personal factors: huge amount of free time during pandemic lock down, stress, previous media exposure, business interest. Environment factors: social reinforcement, availability, group norms, peer pressure. Behavioral factors: Social media platforms enable students to share, create, exchange, discuss, and obtain information and ideas online in networks and virtual communities. However, if the students cannot control the amount of time using the social media, this situation can potentially lead to social media addiction behaviors.
COVID-19 lockdown
Generation Z studentathletes
Social media addiction
Academic performance
Figure 2: The conceptual framework for the research
The conceptual framework is a generative framework that encapsulates the research process in its essence (Adom, Hussain & Joe 2018). The conceptual framework depicted in Figure 2 was created for each of the study's processes and the underlying variables used to review the new search results. The study concept is derived from the researchers’ reading of the literature.
5. Research Type, Sample, and Study Context The researchers used a cross-sectional survey design in this study, combined with quantitative methods. The sampling technique in this study was a probability sampling, specifically cluster (area) random sampling. This is
http://ijlter.org/index.php/ijlter
202
because the respondents were located within a 10-kilometer radius, studentathletes born between 1995 to 2012 and the data collection was performed after the first phase of COVID-19 MCO 2.0 in Malaysia (19th February 2021 until 3rd March 2021). This technique was ideal for us as researchers as we had limited finances, time, and energy resources; while also being concerned about staying safe from the threat of COVID-19 infection, and our accessibility to respondents, specifically Generation Z student-athletes who resided on the campus of Universiti Kebangsaan Malaysia. The University is a public institution of higher education in Bangi, Selangor. The institutional research board of the University granted permission for the investigation. Before volunteers began the survey, they were given a consent form explaining the study's objective and assuring them that the data collection, storage, and reporting processes would ensure confidentiality and anonymity. The Statistical Package for the Social Sciences (SPSS) version 25 was used for data entry and analysis. A simple descriptive analysis was conducted in the form of numbers and percentages. To discover the contribution between variables, simple linear regression was implemented as a test of significance, with a threshold of less than 0.05 considered significant.
6. Data Collection Instrument The survey was divided into three sections, one for demographic data collection and the other two for independent research instruments. Gender and social media type were provided in the demographic information section. The remaining sections were composed of the Social Media Addiction Questionnaire (SMAQ), adapted from Hawi and Samaha (2017) and Grade Point Average (GPA) academic performance. The highest GPA awarded is an A, which is worth ten points, and the lowest is an E, which is worth zero points. Meanwhile, The Social Media Addiction Questionnaire (SMAQ) is an eight-item questionnaire designed to assess social media addiction. Responses were recorded on a seven-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree), with higher scores indicating greater levels of addiction. Table 3 shows the seven-point Likert-type scale; Table 4 shows the three sections of the research instrument; Table 5 shows the norms for SMAQ; and Table 6 shows the breakdown of (GPA) academic performance. Table 3: 7-point Likert-type scale Strongly disagree 1
Disagree 2
Somewhat disagree 3
Neutral 4
Somewhat agree 5
Agree 6
Strongly agree 7
Table 4: Three section of Research Instrument Section Section A Section B
Item Demographic Social media addiction
http://ijlter.org/index.php/ijlter
Gender Type of social media I frequently consider social media when I am not actively using it. I frequently use social media for no discernible reason.
Category of data Nominal Ordinal (Likert scale)
203
Section C
Academic performance
Conflicts with others have arisen as a result of my social media use. When I feel the need to access social media, I pause whatever else I'm doing. When I use social media, I feel a sense of connection with others. I'm losing track of how much time I spend on social media. I'm distressed at the prospect of being unable to access social media. I've been unable to decrease my reliance on social media. GPA
Ordinal (Rank)
Table 5: Norms for SMAQ Level of social media addiction Low Moderate High
Score <7 8 – 50 >51
Table 6: Breakdown of (GPA) academic performance Grade A AB+ B BC+ C CD+ D E
Value 4.00 3.67 3.33 3.00 2.67 2.33 2.00 1.67 1.33 1.00 0.00
Rank 10 9 8 7 6 5 4 3 2 1 0
Due to lockdown and the 10 km radius of travel restrictions imposed by the Malaysian government to break the COVID-19 chain, the instrument was distributed by using email and the respondent required to answer and return to the researchers through email as well. Previously, a pilot test had been conducted with 76 Generation Z student-athletes from several public universities in Malaysia. The reliability was tested using SPSS version 25, and yielded a Cronbach alpha value between 0.67 – 0.89. According to Chua (2014), Cronbach’s alpha coefficient should be between 0.65 to 0.95 which is high and satisfactory. Meanwhile, preliminary analyses of all simple linear regression analyses were conducted to ensure that the normality, linearity, multicollinearity, and homoscedasticity assumptions were not violated.
http://ijlter.org/index.php/ijlter
204
7. Findings The study questionnaire was completed by 91 Generation Z student-athletes out of a total of 120 invitees (a response rate of 76 %); specifically, 57 (62.6 %) were male student-athletes and 34 (37.4%) female student-athletes. The range of birth year was 1995-1998 (36.3%) and 1999-2002 (63.7%). Descriptive analysis was performed to answer the first and second research questions. Figure 3 showed type of social media frequently used by respondents.
17.6%
5.5% 52.7%
24.2%
Figure 3: Type of social media frequently used by respondents during MCO 2.0.
Based on Figure 3 above, 52.7% of Generation Z student-athletes preferred to use Facebook; 24.2 % preferred Instagram; 5.5% preferred to use Twitter; and 17.6 % preferred YouTube. Figure 4 showed the level of social media addiction among Generation Z student-athletes.
60.4%
39.6%
Figure 4: Level of social media addiction among Generation Z student-athletes.
http://ijlter.org/index.php/ijlter
205
Based on Figure 4, results showed that 39.6 % of Generation Z student-athletes were moderately addicted to social media. However, 60.4 % of Generation Z student-athletes were highly addicted to social media. Table 6: Simple linear regression test of the impact of social media addiction on academic performance Model 1
Regression Residual Total
Sum of Squares .014 39.283 39.297
Mean Square .014 .441
df 1 89 90
F
Sig.
.032
.048
As shown in Table 6, the Simple Linear Regression yielded a significant value of p = 0.048 (P <0.05). In conclusion, these findings indicate that the null hypothesis (Ho1) was rejected, social media addiction had significantly contributed to academic performance. Therefore, the researchers referred to the test of Coefficients and R² value to determine the extent to which social media use affects the academic performance of Generation Z student-athletes. Table 7: Coefficients Test Unstandardized Standardized Coefficients Coefficients Model B Std. Beta Error 1 (predictor) 6.068 1.712 Social 0.48 0.270 .019 media addiction Predictor: social media addiction Dependent variable: GPA
t
Sig. 3.544 .179
R
R²
.001 .0048
0.019
Percentage (%)
0.576
57.6%
As presented in Table 7, the Coefficients test showed that the correlation between the dependent variable and the predictor variable was R = 0.019. Linear Regression Analysis was found to be significant [F (1,89) = 0.032, p = 0.048 (P <0.05)] and R² equivalent to 0.576 showed that 57.6% of the change in the dependent variable of academic performance was due to social media addiction. Thus, the researchers concluded that social media addiction significantly impacted academic performance by 57.6%. This can result in procrastination about assignments, loss of attention, and can have a negative impact on their academic achievement. Okyeadie Mensah and Nizam (2016) stated that, students who were immersed in social networking site activities while studying lacked focus and were easily distracted. These findings suggest that social media addiction has a significant negative effect on academic performance, and that solutions, such as turning off notifications, silencing the phone, engaging in home workouts, reconnecting with family, or discovering new hobbies are urgently needed to overcome it. Interventions that can assist Generation Z student-athletes regulate and lessen their social media addiction are also critically needed.
http://ijlter.org/index.php/ijlter
206
8. Discussion The COVID-19 pandemic has posed a serious threat to educational systems. This perspective offers guidance to academics, administrators, and government officials on how to address the crisis. With the rapid advancement of internet technology, the widespread use of smartphones at a young age, and the uncertainty surrounding the end of the COVID-19 pandemic, social media addiction will continue to be a topic of discussion in relation to students' academic performance. It is undeniable that social media has a huge impact on students from a variety of perspectives, as it has become ingrained in their lives, particularly social media platforms, such as Facebook, Instagram, Twitter, Snapchat, and YouTube, as well as blogging sites, newsfeeds, wikis, and social gaming platforms. Students benefit from their ease of use and accessibility because it saves them time and energy. For instance, students can now access Facebook or Instagram from anywhere and at any time via a smartphone. This social media phenomenon has resulted in a massive information flow in developed countries and Malaysia, ranging from remote areas to the middle of cities and regardless of age or race. As a result, most university students can gain access to a greater variety of information more easily and quickly. This study investigated the effects of social media on Generation Z studentathletes. The major goal of this study was to see if there was a correlation between social media usage and academic performance. Our findings revealed that more than 50% of Generation Z student-athletes preferred to use Facebook as their main social media platform compared to Instagram, Twitter and YouTube. Next, our findings showed that 60.4% of Generation Z studentathletes were highly addicted to social media during MCO 2.0 in Malaysia. This could be attributed to a huge amount of free time and stay at home restrictions imposed by the government. These findings were also in line with previous research by Kashif and Aziz-Ur-Rehman (2020), that stated most people are becoming addicted to social media because of the extra free time they now have due to the suspension of all outside activities, and many individuals are becoming agitated and frightened if they do not use it during the coronavirus lockdown. Moreover, our findings revealed that social media addiction significantly contribute 57.6% to the respondents’ academic performance. Future researchers can also expand the number of respondents and conduct interviews using a variety of methods, such as a mix of quantitative and qualitative techniques. It is also worth mentioning that the research found no gender differences. Furthermore, results must be replicated for specific demographics, such as gender and age differences. In conclusion, no one can be left behind with their social media addiction. We do not want Generation Z student-athletes to be unproductive citizens who interact infrequently with other humans and rely heavily on technology, which will inevitably result in a variety of symptoms in the future. In Malaysia, Generation Z student-athletes must be media literate users in order to comprehend and utilize social media to a degree. However, it is necessary to revert to a more normal state of life in which one understands how to divide time between oneself, the surrounding community, and spiritual needs.
http://ijlter.org/index.php/ijlter
207
9. Acknowledgements This study was fully funded by FPEND Research Grant: GG-2019-003, Pemerkasaan Penerbitan Ahli Pusat Pendidikan dan Kesejahteraan Komuniti, Faculty of Education, Universiti Kebangsaan Malaysia.
10. References Adom, D., Hussain, E. K., & Joe, A. A. (2018). Theoretical And Conceptual Framework : Mandatory Ingredients Theoretical And Conceptual Framework : Mandatory Ingredients Engineering. International Journal of Scientific Research, 7(1), 93–98. Al-Menayes, J. (2014). The relationship between mobile social media use and academic performance in university students. New Media and Mass Communication, 25, 2329. Alahmar, A. T. (2017). The influence of social networks on the grades of medical students at University of Babylon, Iraq. July. https://doi.org/10.5455/jmas.263824 Alnjadat, R., Hmaidi, M. M., Samha, T. E., Kilani, M. M., & Hasswan, A. M. (2019). Gender variations in social media usage and academic performance among the students of University of Sharjah. Journal of Taibah University Medical Sciences, 14(4), 390–394. https://doi.org/10.1016/j.jtumed.2019.05.002 Bajpai, P. (2018). Analyzing Effect of Social Media on Academic Performance of University Graduates. In Proceedings of the 2018 The 3rd International Conference on Information and Education Innovations (pp. 40-44). Bandura, A. (1977). Self-efficacy: toward a unifying theory of behavioral change. Psychological review, 84(2), 191. Bernama. (2019). Malaysia Ranks Top 5 Globally in Mobile Social Media Penetration, Highest in Region. News Straits Times. Retrieved July 13, 2019, from https://www.nst.com.my/lifestyle/bots/2019/01/456119/malaysia-ranks-top5-globally-mobile-social-media-pen etration-highest Boateng R., & Amankwaa, A. (2016). The impact of social media on students' academic life in higher education. Glob J Hum Soc Sci: G Linguistics & Education 2016; 16(4). Brailovskaia, J., Schillack, H., & Margraf, J. (2020). Tell me why are you using social media (SM)! Relationship between reasons for use of SM, SM flow, daily stress, depression, anxiety, and addictive SM use–An exploratory investigation of young adults in Germany. Computers in Human Behavior, 113, 106511. Brailovskaia, J., Rohmann, E., Bierhoff, H. W., Schillack, H., & Margraf, J. (2019). The relationship between daily stress, social support and Facebook Addiction Disorder. Psychiatry research, 276, 167-174. Buck, R. (2019). Motivation, emotion, cognition, and communication: Definitions and notes toward a grand theory. Advances in Motivation Science (Vol. 6, pp. 27-69). Elsevier. Burke, M., Kraut, R., & Marlow, C. (2011). Social capital on Facebook: differentiating uses and users. In: Paper Presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Vancouver, BC (2011). http://doi.org/10.1145/1978942.1979023 Chua, Y. P. (2014). Research Method [Kaedah Penyelidikan]. Buku 1; Edisi Kedua [Book 1; Second Edition]. McGraw Hill Malaysia Sdn. Bhd. Curtis, B. L., Ashford, R. D., Magnuson, K. I., & Ryan-Pettes, S. R. (2019). Comparison of smartphone ownership, social media use, and willingness to use digital interventions between generation z and millennials in the treatment of substance use: Cross-sectional questionnaire study. Journal of medical Internet research, 21(4), e13050.
http://ijlter.org/index.php/ijlter
208
Dahlstrom, E., de Boor, T., Grunwald, P., & Vockley, M. (2011). ECAR: National study of undergraduate students and information technology. http://net.educause.edu/ ir/library/pdf/ers1103/ers1103w.pdf Daniel, J. (2020). Education and the COVID-19 pandemic. Prospects, 49(1), 91-96. Duggan, M., Ellison, N. B., Lampe, C., Lenhart A., & Madden M. (2015). Social media update. http://www.pewinternet.org/files/2015/01/PI_SocialMediaUpdate201 44.pdf. Filimon, N., Girona, U. De, & Kiygi-calli, M. (2017). Social Media and Higher Education – An International Perspective. 10(1), 181–191. https://doi.org/10.14254/2071789X.2017/10-1/13 Gazi, M. A., & Çetin, M. (2017). The research of the level of social media addiction of university students. International Journal of Social Sciences and Education Research, 3(2), 549-559. Ghazali, S., & Ghani, N. M. (2018). Perception of Female Students towards Social MediaRelated Crimes. Pertanika Journal of Social Sciences & Humanities, 26(2). Glass, A. (2007). Understanding generational differences for competitive success. Industrial and commercial training. González-Padilla, D. A., & Tortolero-Blanco, L. (2020). Social media influence in the COVID-19 pandemic. International braz j urol, 46, 120-124. Grau, S., Kleiser, S., & Bright, L. (2019). Exploring social media addiction among student Millennials. Qualitative Market Research: An International Journal. Giunchiglia, F., Zeni, M., Gobbi, E., Bignotti, E., & Bison, I. (2018). Mobile social media usage and academic performance. Computers in Human Behavior, 82, 177-185. Hawi, N. S., & Samaha, M. (2017). The relations among social media addiction, selfesteem, and life satisfaction in university students. Social Science Computer Review, 35(5), 576-586. Hawi, N., & Samaha, M. (2019). Identifying commonalities and differences in personality characteristics of Internet and social media addiction profiles: traits, self-esteem, and self-construal. Behaviour & Information Technology, 38(2), 110-119. Head, A. (2013). Learning the ropes: How freshmen conduct course research once they enter college. SSRN, 2364080. http://dx.doi.org/10.2139/ssrn.2364080 Jaleniauskiene, E., & Juceviciene, P. (2015). Reconsidering university educational environment for the learners of generation Z. Social Sciences, 88(2), 38-53. Junco, R., & Cotten, S. R. (2012). No A 4 U: The relationship between multitasking and academic performance. Computers & Education, 59(2), 505-514. Kashif, M., & Aziz-Ur-Rehman, M. K. J. (2020). Social media addiction due to coronavirus. International Journal of Medical Science in Clinical Research and Review, 3(04), 331-336. Kim, K. S., Sin, S. C. J., & Tsai, T. I. (2014). Individual differences in social media use for information seeking. The journal of academic librarianship, 40(2), 171-178. Kuss, D., M. Griffiths, L. Karila, & J. Billieux. (2014). “Internet Addiction: A Systematic Review of Epidemiological Research for the Last Decade.” Current Pharmaceutical Design, 20(25), 4026–4052. Lepp, A., Barkley, J. E., & Karpinski, A. C. (2015). The relationship between cell phone use and academic performance in a sample of US college students. Sage Open, 5(1), 2158244015573169. MacKenzie, J., & McGuire, R. (2016). The First Generation of the Twenty-First Century. http://magid.com/sites/default/files/pdf/MagidPluralistGenerationWhitepap er.pdf Merchant, R. M., & Lurie, N. (2020). Social media and emergency preparedness in response to novel coronavirus. Jama, 323(20), 2011-2012. Ministry of Health. (2021). Situasi Terkini Covid-19 [Current Situation of Covid-19].
http://ijlter.org/index.php/ijlter
209
Okyeadie, M. S., & Nizam, D. (2016). The impact of social media on students’ academic. Int J Educ Learn Train (IJELT), 1(1): 14e21. Pa, W. A. M. W., Salamuddin, N., Zin, N. M., & Lian, D. K. C. (2020). Sports Massage Therapy Towards Pre-Competition Anxiety Among Malaysian High Performance Tennis Players. In Proceedings of the 1st Progress in Social Science, Humanities and Education Research Symposium (PSSHERS 2019), 200824. Pato, A. S., Isıdori, E., Calderón, A., & Brunton, J. (2017). An innovative European sports tutorship model of the dual career of student-athletes. UCAM. Paul, J. A., Baker, H. M., & Cochran, J. D. (2012). Effect of online social networking on student academic performance. Computers in Human Behavior, 28(6), 2117-2127. Perrin, A. (2015). Social media usage. Pew research center, 125, 52-68. Ransome, W., & Sampford, C. (2016). Ethics and socially responsible investment: A philosophical approach. Routledge. Ryba, T. V., Stambulova, N. B., Ronkainen, N. J., Bundgaard, J., & Selänne, H. (2015). Dual career pathways of transnational athletes. Psychology of Sport and Exercise, 21, 125-134. Salleh, M. S. M., Mahbob, N. N., & Baharudin, N. S. (2017). Overview of “Generation Z” Behavioural Characteristic and its Effect Towards Hostel Facility. International Journal of Real Estate Studies, 11(2), 59-67. Samaha, M., & Hawi, N. S. (2016). Relationships among smartphone addiction, stress, academic performance, and satisfaction with life. Computers in human behavior, 57, 321-325. Schroer, W. J. (2016). Generations X, Y, Z and the Others. http://socialmarketing.org/archives/generations-xy-z-and-the-others/ Schulze, C., Schöler, L., & Skiera, B. (2015). Customizing social media marketing. MIT. Singh, A. (2014). Challenges and issues of generation Z. IOSR Journal of Business and Management, 16(7), 59-63. Sivakumar, R. (2020). Effects of social media on academic performance of the students. The Online Journal of Distance Education and e-Learning, 8(2), 90-97. Skiera, B., Hinz, O., & Spann, M. (2015). Social media and academic performance: Does the intensity of Facebook activity relate to good grades?. Schmalenbach Business Review, 67(1), 54-72. Sutherland, K. E., Davis, C., Terton, U., & Visser, I. (2018). University student social media use and its influence on offline engagement in higher educational communities. Student Success, 9(2), 13-24. UNESCO. (2020). UNESCO rallies international organizations, civil society and private sector partners in a broad coalition to ensure #Learning Never Stops. 26 March 2020. https://en.unesco.org/news/unesco-rallies-international-organizationscivil-societyand-private-sector-partners-broad. Wiedmer, T. (2015). Generations do differ: Best practices in leading traditionalists, boomers, and generations X, Y, and Z. Delta Kappa Gamma Bulletin, 82(1), 51. Tayseer, M., Zoghieb, F., Alcheikh, I., & Awadallah, M. N. (2014). Social network: academic and social impact on college students. In ASEE 2014 Zone I Conference (pp. 3-5). Yahya, M. W. B. H., Rahman, T., & Siddiq, A. A. (2021). Online Learning in the Quran Reading Class during Covid-19 Pandemic. International Journal of Learning, Teaching and Educational Research, 20(5). Zhao, N., & Zhou, G. (2021). COVID-19 stress and addictive social media use (SMU): Mediating role of active use and social media flow. Frontiers in Psychiatry, 12, 85.
http://ijlter.org/index.php/ijlter
210
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 210-226, August 2021 https://doi.org/10.26803/ijlter.20.8.13 Received Jun 03, 2021; Revised Aug 19, 2021; Accepted Aug 30, 2021
The Role of Non-Academic Service Quality in an Academic-Oriented Context: Structural Equation Modeling amongst Parents of Secondary Education Students Thao-Thanh Thi Phan Thanh Do University, Kim Chung, Hoai Duc, Hanoi, Vietnam https://orcid.org/0000-0002-9511-9527 Ha-Giang Thi Tran Hanoi Metropolitan University, Cau Giay, Hanoi, Vietnam http://orcid.org/0000-0002-6061-5424 Le-Huong Thi Nguyen Hoa Binh College of Education, Dan Chu, Hoa Binh, Vietnam https://orcid.org/0000-0002-8103-2797 Tam-Phuong Pham Can Tho University, Ninh Kieu, Can Tho, Vietnam https://orcid.org/0000-0002-1745-9067 Thang-The Nguyen Vietnam Institute of Educational Sciences, Hoan Kiem, Hanoi, Vietnam http://orcid.org/0000-0002-3998-0099 Hiep-Hung Pham Phu Xuan University, Phu Hoi, Hue, Vietnam EdLab Asia Educational Research and Development Centre, Hanoi, Vietnam https://orcid.org/0000-0003-3300-7770 Thai-Quoc Cao* EdLab Asia Educational Research and Development Centre, Hanoi, Vietnam Hanoi University of Social Sciences and Humanities, Hanoi, Vietnam http://orcid.org/0000-0001-8762-4809
Abstract. In the education sector, the academic factor is considered the primary focus when assessing the service quality of schools. However, the *
Corresponding author: Thai-Quoc Cao; Email: thaicao@edlabasia.org
©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0)
211
traditional approach to service quality proposes that the supplemental indicators of quality are just as important in meeting customer expectations. As parents of secondary school students are not directly involved in the academic service of the school, non-academic factors (e.g., administrative quality [ADQ] and physical environment quality [PEQ]) can be considered alternative service quality indicators, subsequently affecting their satisfaction and loyalty. Using data obtained from a survey questionnaire with 230 parents of secondary education students in Vietnam, this study examined non-academic service factors as antecedents of parent satisfaction and loyalty. Results of the exploratory factor analysis (EFA) yielded two factors under the non-academic service dimension: PEQ and ADQ. Results of the structural equation modeling (SEM) showed that while PEQ affects loyalty both directly and indirectly, ADQ affects loyalty only indirectly through satisfaction. Implications for stakeholders, including school principals and policymakers, have been drawn from the findings of this study. Keywords: administrative quality; non-academic service quality; parent loyalty; parent satisfaction; physical environment quality; secondary education
1. Introduction With a growing population and an ever-fiercer competition in the market, Vietnam’s educational sector is striving to adapt to a fast-changing world and the rapid development of the country. Amidst this situation, the secondary education sector in Vietnam has to deal with two major obstacles. The first obstacle is unique to the public secondary school, which emerged from the inevitable transition toward fiscal autonomy of public education in Vietnam. Traditionally, with governmental support, the tuition fee for general secondary education is substantially lower than that of their private counterpart. However, with the transition toward fiscal autonomy, the public education sector has to persuade parents to become more accepting toward an increased tuition fee while improving the quality of education to retain students (Nguyen, 2019). The second obstacle is an alias of Vietnam’s booming economy and population. The substantial profitability of the quasi-market of educational service in Vietnam has led to ever more competition (Lan Anh, 2015). This tendency has pushed the typically conservative public school into the context of the already competitive educational market amongst private secondary schools. While various stakeholders are contending the education market, emphasizing the importance of applying marketing perspectives in the secondary education sector has dramatically increased, since a slight advantage can decide the survivability of a school (Kassim et al., 2010; Sharif & Kassim, 2012). From the traditional marketing approach, customer satisfaction and loyalty are critical factors related to the success of an organization (Kaura et al., 2015). This is especially true in secondary education, where the decision regarding in which school to enroll their children is a financial one made by parents. When compared with lower education levels, the decision to choose the secondary school is valued more greatly since this is the stepping stone for the children’s tertiary education or future career (Rolfsman, 2020; Savolainen et al., 2008). The school with higher
http://ijlter.org/index.php/ijlter
212
parent satisfaction and loyalty performs more effectively in the educational market (Li & Hung, 2009). Therefore, gaining the satisfaction and loyalty of parents, who are also the purchasers of the service, is decisive to gaining the competitive edge over competitors. In traditional business, improving service quality is one way to increase customer satisfaction and loyalty (Lee, 2017), a fact that also applies to educational service (Hemsley-Brown et al., 2010; Pham & Lai, 2016). However, there are also inherent differences between the education and other business sectors. While customer perception about service quality in other businesses depends on both the tangible (physical manifestation of the company) and intangible (the service provided by the employees), the assessor of education quality mainly focuses on an ethereal quality, which is the academic side of educational assistance. Indeed, previous research on student perceptions of educational quality has pointed out that academic factors have a more substantial role in the overall student assessment compared with non-academic ones (Teeroovengadum et al., 2019). However, considering parents as the purchasers of educational services, the oversight of non-academic service quality can make the evaluation of service quality imprecise. The reasons for the potential overlook lie in the similarities and differences in the service experience of parents and students in education. Like with the students, parent satisfaction and loyalty are mainly based on the academic quality of the school. However, unlike students, who directly experience the secondary school education, parents’ perception about the academic quality is based on a more indirect assessment, that of school image (Li & Hung, 2009), or their children’s academic results and personal transformation (Harvey & Green, 1993; Teeroovengadum et al., 2016). Meanwhile, most parents have to deal directly with non-academic issues related to their children’s education, such as administrative issues and financial matters. Therefore, from a marketing perspective, examining the parents’ perceptions of non-academic service quality in secondary school might provide valuable information both in terms of theory and practice regarding parents’ assessment of service quality, and helpful advice for educational institutions to increase their market competitiveness. As there is currently no research on the effect of non-academic service quality on the level of satisfaction and loyalty with secondary education students, this research, to the best of our knowledge, may be the first to strive to address this research issue. Therefore, the main goal of this study is to examine to what extent non-academic service factors affect parent satisfaction and loyalty. Particularly, two research questions have arisen from this research goal: 1. What components constitute the non-academic service factors in the context of secondary education in Vietnam? 2. How do non-academic service factors affect parent satisfaction and loyalty in the context of secondary education in Vietnam? To address the research questions, exploratory factor analysis (EFA) was conducted to examine and adapt an established questionnaire to measure parents’
http://ijlter.org/index.php/ijlter
213
assessment of non-academic service quality. Subsequently, confirmatory factor analysis (CFA) and structural equation modeling (SEM) were performed to examine the model fit and the role of non-academic service quality on parent satisfaction and loyalty. The findings of this study have both academic and practical implications. In terms of academic implications, this study, to the best of our knowledge, is the first ever effort to investigate non-academic service factors as determinants of parent satisfaction and loyalty in a secondary education context. In terms of practical implications, school principals and policymakers are amongst the key stakeholders who may benefit from this study.
2. Literature Review Parent Satisfaction and Loyalty Satisfaction and loyalty are two prominent factors examined in marketing research, and are direct antecedents of customer retention and business success (Hallowell, 1996; Kandampully & Suhartanto, 2000; Kasiri et al., 2017). Furthermore, satisfaction and loyalty are closely related concepts and should be included conjointly to explain best success of service providers, including educational institutions. Satisfaction is identified as one of the most viable direct determinants of loyalty. This assertion has been verified in many different contexts, such as tourism (Eid, 2015), retail (Kamran-Disfani et al., 2017), banking (Bloemer et al., 1998), and e-commerce (Eid, 2011). In the educational context, we follow the definition of Fantuzzo et al. (2006), which operationalizes parent satisfaction according to the degree of expectation being met. Satisfaction can also serve as the primary pathway to customer loyalty, which is also an essential factor in the marketing field. Customer loyalty in the educational context can be defined according to the long-term commitment to the favored educational institution of parents (Li & Hung, 2009). As the concept of loyalty can be specified by different theoretical approaches (e.g., behavioral loyalty, intentional loyalty), in this study, we followed Li and Hung (2009) in using the hybrid system to measure loyalty, which has been commonly applied in educational contexts. Non-Academic Service Quality The taxonomy of the educational service quality concept has been defined in various ways in literature. As summarized by Sharif and Kassim (2012), service quality can be defined either by the excellence of service delivery or the success in meeting customer expectations. In the educational context, the latter approach is often the preferred avenue to examine service quality. As previous research has mainly relied on student assessment, the expectation of service quality is primarily defined by the added value from academic indicators, while neglecting the non-academic aspect of the service (Sharif & Kassim, 2012). Teeroovengadum et al.’s (2016) scale measuring both non-academic and academic service quality has shown that the factors related to non-academic service quality have a lesser effect on students’ assessment of service quality. However, from the parents’
http://ijlter.org/index.php/ijlter
214
perspective, the aspect of service quality that they often directly have exposure to is primarily non-academic factors. Previous research on the assessment of patients’ relatives on medical service quality has shown that interpersonal relationships with the medical staff and the physical environment are important indicators of service quality since the relatives do not have direct exposure to the medical care and enough technical knowledge about the service (Mohammad Mosadeghrad, 2013). Therefore, to parents, the non-academic factors may also play an essential role in assessing service quality. According to Sharif and Kassim (2012), non-academic service quality is crucial in choosing educational institutions when the other academic criteria are identical. To better distinguish the sub-concepts of academic and non-academic service quality, we conceptualize non-academic service quality, based on Sharif and Kassim (2012), as the factors related to the infrastructure and management activities of the educational institution. Therefore, this study aimed to explore the effects of non-academic service quality on parent satisfaction and loyalty. EFA was used to unveil the factors of non-academic service quality. Previous studies, using students as studied objects, have revealed a positive association between non-academic service quality and satisfaction and loyalty (see Kardoyo et al., 2019; Mulyono et al., 2020; Sultan & Yin Wong, 2013). Subsequently, these factors were used to answer our main research hypotheses, which expect non-academic service quality elements to significantly increase parent satisfaction and loyalty. Conceptual Model Based on the above review of literature, we built our conceptual model as illustrated in Figure 1. Specifically, it is suggested that satisfaction and non-academic service factors both have a positive effect on loyalty. Meanwhile, non-academic service factors are also proposed to have a direct effect on satisfaction. In other words, satisfaction is supposed to have a partial mediating effect on the link between non-academic service factors and loyalty.
Figure 1: Conceptual model of the research
http://ijlter.org/index.php/ijlter
215
3. Methodology
Data Collection To answer the research questions, an online survey questionnaire was distributed to Vietnamese parents of secondary school students. We used the snowball data collection method to secure respondents. Specifically, we utilized our network of educators and teachers in secondary schools across Vietnam to forward the survey to the parents. Every two weeks during the data collection process, a follow-up email was sent to remind the unresponsive parents to complete the survey. The final sample comprised 230 respondents deemed suitable for SEM (Hoogland & Boomsma, 1998). Table 1 shows an overview of the demographic profile of the sample. Table 1: The demographic background of the respondents Characteristic
Frequency (n & %)
Gender Male
36 (15.7)
Female
188 (81.7)
Prefer not to say
6 (2.6)
Age 35 years old and under
18 (7.8)
36–40 years old
70 (30.4)
41–45 years old
83 (36.1)
46–50 years old
41 (17.8)
50 years old and over
18 (7.8)
Type of work Public sector
131 (57.0)
Private sector
71 (30.9)
Other
28 (12.2)
Residency Hanoi or Ho Chi Minh City
177 (77.0)
Other
53 (23.0)
Level of education Lower than undergraduate
22 (9.6)
Undergraduate education
95 (41.3)
http://ijlter.org/index.php/ijlter
216
Characteristic
Frequency (n & %)
Postgraduate education
109 (47.4)
Other
4 (1.7)
The grade of the children Lower secondary education
147 (63.8)
Upper secondary education
83 (36.1)
Total
230
Instrument Development A positivism paradigm was employed to help answer the research questions. The research questionnaire was developed based on the Higher Education Service Quality (HESQUAL) scale (Teeroovengadum et al., 2016). The measurement has a total of five main factors and eleven sub-factors. From the conceptualization mentioned above of non-academic service quality, we chose five sub-factors of the HESQUAL scale to measure non-academic service factors, as follows: (i) administrative processes (ADP); (ii) attitude and behavior of staff (ATB); (iii) support facilities quality (SFQ); (iv) learning setting (LNS); and (v) support infrastructure (SPI). To use the questionnaire on Vietnamese parents, a translated version of the survey was required. Following the recommendations of Epstein et al. (2015), we employed back-translation to translate the English questionnaire into Vietnamese. The original English version (E1) was translated into Vietnamese (V1) by a coauthor of this paper. After this, another co-author who had never seen the E1 version was asked to back-translate V1 into English (E2). The three versions of the questionnaire (i.e., E1, V1, & E2) were compared and revised by all authors of the paper, subsequently leading to the second Vietnamese version (V2). An educational practitioner and a researcher were invited to examine the V2 and E1 versions for face validity. Some revisions on the terminology in the questionnaire were made based on the comments of the two invited experts.
4. Results Exploratory Factor Analysis As the questionnaire had been translated into Vietnamese, we decided to conduct EFA to examine the latent constructs of non-academic service quality. Following the procedure of Hair et al. (1998), the items with factor loadings larger than 0.50 were considered significant and kept. Meanwhile, items with factor loadings smaller than 0.4 were excluded. Table 2 shows the factor structure of the 13 observed variables examined by the EFA. As shown in Table 2, rather than having five initial factors, this analysis revealed two main factors, labeled physical environment quality (PEQ) and administrative quality (ADQ).
http://ijlter.org/index.php/ijlter
217
Table 2: The exploratory factor analysis results Factor (Latent variable)
PEQ
ADQ
Component
SPI1: Availability of adequate cafeteria infrastructure SPI2: Availability of adequate sports infrastructure SPI3: Availability of adequate recreational infrastructure SPI4: Availability of adequate library infrastructure SFQ2: Amount of opportunity for sports and recreational facilities SFQ3: Availability of adequate IT facilities LNS1: Having adequate lecture rooms LNS2: Having quiet places to study within the campus LNS3: Availability of adequate teaching tools and equipment (e.g., projector, whiteboard) ADP1: Well-standardized administrative processes so that there is not much bureaucracy and useless difficulties ADP2: Administrative procedures are clear and well structured so that service delivery times are minimum ATB1: Willingness of administrative staff members to help students ATB2: Ability of administrative staff members to solve students’ problems
Factor loading
Cronbach alpha
0.840 0.851 0.778 0.751 0.825
0.929
0.791 0.766 0.530 0.546
0.853
0.878
0.931
0.923 0.848
Nine items loaded onto PEQ, with factor loadings ranging from 0.53 to 0.85. The second factor, ADQ, had factor loadings ranging from 0.85 to 0.92. All items used for the analysis are presented in Table 2. The EFA results therefore provide the answer for research question 1 (What components constitute the non-academic service factors in the context of secondary education in Vietnam?). There are two specific factors pertaining to the non-academic service dimension of secondary education in Vietnam, namely PEQ and ADQ. Figure 2 represents our adjusted conceptual model based on the EFA results.
http://ijlter.org/index.php/ijlter
218
Figure 2: The adjusted conceptual model of the research after exploratory factor analysis
Particularly, we propose that satisfaction has a positive effect on loyalty. At the same time, we suggest that satisfaction partially mediates the relationships of PEQ-loyalty and ADQ-loyalty. Under these circumstances, five hypotheses are proposed: H1: PEQ has a significant positive effect on satisfaction. H2: ADQ has a substantial significant positive effect on satisfaction. H3: PEQ has a significant positive effect on loyalty. H4: ADQ has a significant positive effect on loyalty. H5: Satisfaction has a significant positive effect on loyalty. Confirmatory Factor Analysis To assess how well the measurement model fits with the observed data, CFA was conducted. Specifically, all the goodness of fit (GOF) indices were satisfied with their acceptable level, indicating that the model fit with the observed data. Table 3 presents the GOF indices of the model and their respective cut-off levels for a good fit. Table 3: Results of multiple fit indices Index Chi-square Degree of freedom Chi-square/Degree of freedom GFI AGFI NFI RMSEA CFI
http://ijlter.org/index.php/ijlter
Result 254.159 174
Acceptable level -
1.461
<5
0.908 0.878 0.949 0.045 0.983
> 0.9 > 0.8 > 0.9 < 0.08 > 0.9
219
Regarding the issues of convergent validity and discriminant validity, the factor loadings, composite reliability (CR) scores, and average variance extracted (AVE) scores of the model were examined (Tables 4 and 5). Table 4: Results of factor loading for Confirmatory Factor Analysis Factor (Latent variable)
PEQ
ADQ
SAT
LOY
Item Factor loading SFQ2: Amount of opportunity for sports and recreational facilities
0.824
SFQ3: Availability of adequate IT facilities
0.803
LNS3: Availability of adequate teaching tools and equipment (e.g., projector, whiteboard)
0.691
LNS2: Having quiet places to study within the campus
0.649
LNS1: Having adequate lecture rooms
0.760
SPI4: Availability of adequate library infrastructure
0.767
SPI3: Availability of adequate recreational infrastructure
0.797
SPI2: Availability of adequate sports infrastructure
0.801
SPI1: Availability of adequate cafeteria infrastructure
0.768
ADP2: Administrative procedures are clear and well structured so that service delivery times are minimum
0.958
ADP1: Well-standardized administrative processes so that there is not much bureaucracy and useless difficulties
0.926
ATB2: Ability of administrative staff members to solve students’ problems
0.768
ATB1: Willingness of administrative staff members to help students
0.786
SAT3. This school has met my expectations
0.941
SAT2. My choice of this school was a wise one
0.964
SAT1. I am satisfied with the decision to choose this school
0.939
LOY5. When talking about school with my relatives or friends, I will praise this school voluntarily.
0.898
http://ijlter.org/index.php/ijlter
220
Factor (Latent variable)
Item Factor loading LOY4. I will encourage my relatives or friends to let their children study in this school
0.923
LOY3. When my relatives or friends need information about secondary school, I will voluntarily recommend this school
0.916
LOY2. When my child is entering secondary school, this school will be my first choice
0.824
LOY1. If I have a child studying in secondary school, I will continue to let him/her attend the same school.
0.819
Table 5: Convergent and discriminant validity Construct
CR
AVE
Factor correlation SAT
PEQ
ADQ
PEQ
0.926
0.584
0.764
ADQ
0.921
0.746
0.622
0.864
SAT
0.964
0.899
0.519
0.646
0.948
LOY
0.943
0.769
0.587
0.535
0.770
LOY
0.877
Note: PEQ – physical environment quality; ADQ – administrative quality; SAT – satisfaction; LOY – loyalty; CR – composite reliability; AVE – average variance extracted Regarding the factor loadings of the observed variables, the PEQ factor loadings ranged from 0.649 to 0.824, ADQ from 0.768 to 0.958, satisfaction from 0.939 to 0.964, and loyalty from 0.819 to 0.923. Overall, the factor loadings of the observed variables were relatively high. Regarding the convergent validity issue, as shown in Table 4, the CR and AVE scores of the concepts were higher than their thresholds, 0.7 and 0.5, respectively. The discriminant validity of the questionnaire was examined by comparing the AVE scores and the squared correlation coefficients between factors. The AVE scores were higher than the squared correlation coefficients, which indicates that the criteria for discriminant validity were satisfied. Path Analyses and Hypotheses Testing Table 6 shows the standardized regression estimates and the p-values of the exogenous variables.
http://ijlter.org/index.php/ijlter
221
Table 6: Results of the structural equation modelling Standardized β coefficient
p-value
Hypothesis
Dependent variable: SAT PEQ
0.197
0.007
H1 is supported
ADQ
0.524
***
H2 is supported
R2
44.1%
Dependent variable: LOY SAT
0.674
***
H5 is supported
PEQ
0.283
***
H3 is supported
ADQ
-0.077
0.261
H4 is not supported
R2
64.5%
Regarding the effect of non-academic service constructs on satisfaction, the regression paths of the SEM showed that PEQ (β = 0.20, p = 0.007) and ADQ (β =0.52, p < 0.001) have a significant positive effect on satisfaction, indicating that higher non-academic service constructs come with higher parent satisfaction. Therefore, hypotheses H1 and H2 are supported. Higher PEQ is also directly associated with higher loyalty (β = 0.28, p < 0.001), indicating that increasing the PEQ heightens parents’ commitment. This relationship supports hypothesis H3. However, ADQ does not significantly affect loyalty (p = 0.261), and therefore hypothesis H4 is rejected. Lastly, satisfaction showed the strongest effect on loyalty (β = 0.67, p < 0.001), which provides support for hypothesis H5. These above findings provide an answer to the second research question of this study (How do non-academic service factors affect parent satisfaction and loyalty in the context of secondary education in Vietnam?). Specifically, both PEQ and ADQ directly affect satisfaction, while only PEQ directly affects loyalty. In the same vein, satisfaction directly affects loyalty. In other words, satisfaction partially mediates the relationship between PEQ and loyalty and fully mediates the relationship between ADQ and loyalty. Overall, the non-academic service quality explained 44.1% of the variance in satisfaction. The effect of satisfaction, PEQ, and ADQ on loyalty explained 64.5% of the variance in loyalty. Total Effect of Direct and Indirect Antecedents of Parent Loyalty Table 7 shows calculations of the total effect of the antecedents on loyalty. Amongst all, satisfaction has only a direct but also the highest effect (total effect = 0.674) on loyalty. This is followed by PEQ, which has both direct and indirect effects (total effect = 0.416), and ADQ, which has only an indirect effect (total effect = 0.353).
http://ijlter.org/index.php/ijlter
222
Table 7: Direct, indirect, and total effects of physical environment quality, administrative quality, and satisfaction on loyalty Antecedent
Direct effect magnitude
PEQ
route
and
Indirect effect route and magnitude
Total effect
PEQ - loyalty: 0.28
PEQ - satisfaction - loyalty: 0.197 * 0.674 = 0.133
0.416
ADQ
–
ADQ - satisfaction - loyalty: 0.524 * 0.674 =
0.353
Satisfaction
Satisfaction - loyalty: 0.674
–
0.674
5. Discussion and Conclusion As far as we are aware, no research has been conducted on the effects of parents’ assessment of non-academic service quality on their satisfaction and loyalty in secondary education. By surveying 230 Vietnamese parents, this study fills this research gap by pointing out that besides increasing academic quality, secondary education institutions should also focus on improving ADQ and PEQ to increase parent satisfaction and loyalty. The results of this research make several theoretical contributions to the literature on educational service quality. Academic Implications First, this research has re-established that satisfaction is still the main predictor of loyalty (Brandano et al., 2019; Kandampully & Suhartanto, 2000; Kasiri et al., 2017; Pham et al., 2019). The results also showed that the non-academic dimension is composed of two factors, PEQ and ADQ. These factors have both direct and indirect effects on parent satisfaction and loyalty. Furthermore, the total direct and indirect effects of PEQ more significantly affect parent loyalty than the indirect effect of ADQ. This result aligns with previous research on students’ assessment of ADQ and PEQ. According to Grönroos (2011), the evaluation of value can be incrementally added throughout the service experience of the customer by exposure to different indicators of quality. As the exposure to PEQ is unavoidable when parents are present at the educational institution, the value added incrementally may contribute to the positive relationship with satisfaction and loyalty. This can also be used to explain the insignificant relationship between ADQ and loyalty. According to lin and Tsai (2008), students perceived that ADQ does not affect their loyalty due to the little interaction between them and the administrative staff. Similarly, this research has shown that positive ADQ experiences affect the only the satisfaction of parents. Unlike ADQ, the exposure to PEQ is consistent and can be accounted for by parent recommendation and loyalty to the school. PEQ has been found to affect both parent satisfaction and loyalty through either direct or indirect paths, which is also consistent with previous research on students’ assessment (Hemsley-Brown et al., 2010; Wiers-Jenssen et al., 2002).
http://ijlter.org/index.php/ijlter
223
Practical Implications The findings of this research have two main direct practical implications for secondary education institutions. First, secondary schools should focus on increasing the quality of PEQ and ADQ to increase parent satisfaction and loyalty. As previous research has mainly depended on the academic quality of educational institutions, schools with equal academic quality and a stronger focus on non-academic aspects can have a stronger competitive edge than their counterparts (Kassim et al., 2010). Second, when distributing resources to increase the competitive advantage over non-academic service quality, more focus should be placed on PEQ than ADQ as PEQ has a direct effect on both satisfaction and loyalty, while ADQ has a direct effect on satisfaction only but not loyalty. Additionally, the total effect on loyalty stemming from PEQ is higher than that from ADQ. Thus, school principals and governmental agencies should focus more on PEQ constitutions such as libraries, cafeterias, recreational infrastructure, sport infrastructure, and so forth.
6. Limitations and Suggestions for Future Research The current research also had several limitations (Vuong, 2020). First, as this research was one of the first attempts to examine the effect of non-academic service quality from parents’ perspective, the survey developed to test our hypotheses requires further investigation regarding its validity and reliability in different contexts. We suggest future research to replicate these results by including other measurements of non-academic service quality to predict parent satisfaction and loyalty. Second, as Asian parents in general and Vietnamese parents in specific have certain unique traits represented by cultural factors (Loh & Teo, 2017), a more representative sample across the globe is required to further confirm how non-academic service quality affects parent satisfaction and loyalty. Third, the non-significant ADQ results may be the result of overlooking other important determinants of loyalty, such as school image. We recommend that further research use a more detailed model with relevant variables to test this effect. Lastly, although we were able to collect a diverse and large sample, our sampling method relied on educators and teachers to distribute the survey to the parents. This may have triggered the demand effect, where participants try to please the authoritative individual (teacher/researcher) by behaving in a desirable manner (Nichols & Maner, 2008). Although the consent form of this research assured the anonymity of the respondents, future research should utilize a more randomized sampling method to replicate the results of this research.
Funding and Acknowledgements: This work is part of the project “Model of K-12 school governance from the lens of educational service” – number B2019 – VGK-02 (Mô hình quản trị cơ sở giáo dục phổ thông theo tiếp cận quản trị dịch vụ giáo dục, số B2019 –VGK-02). The authors would like to thank the funder for this valuable support.
http://ijlter.org/index.php/ijlter
224
7. References Bloemer, J., De Ruyter, K., & Peeters, P. (1998). Investigating drivers of bank loyalty: The complex relationship between image, service quality and satisfaction. International Journal of Bank Marketing, 16(7), 276–286. https://doi.org/10.1108/02652329810245984 Brandano, M. G., Osti, L., & Pulina, M. (2019). How motivations and satisfaction influence wine tourists’ loyalty? An analysis of the Italian case. International Journal of Culture, Tourism and Hospitality Research, 13(1), 55–69. https://doi.org/10.1108/IJCTHR-04-2018-0054 Eid, M. (2011). Determinants of e-commerce customer satisfaction, trust, and loyalty in Saudi Arabia. Journal of Electronic Commerce Research, 12(1), 78–93. http://www.jecr.org/sites/default/files/12_1_p05.pdf Eid, R. (2015). Integrating Muslim customer perceived value, satisfaction, loyalty and retention in the tourism industry: An empirical study. International Journal of Tourism Research, 17(3), 249–260. https://doi.org/10.1002/jtr.1982 Epstein, J., Osborne, R. H., Elsworth, G. R., Beaton, D. E., & Guillemin, F. (2015). Crosscultural adaptation of the Health Education Impact Questionnaire: Experimental study showed expert committee, not back-translation, added value. Journal of Clinical Epidemiology, 68(4), 360–369. https://doi.org/10.1016/j.jclinepi.2013.07.013 Fantuzzo, J., Perry, M. A., & Childs, S. (2006). Parent Satisfaction with Educational Experiences Scale: A multivariate examination of parent satisfaction with early childhood education programs. Early Childhood Research Quarterly, 21(2), 142–152. https://doi.org/10.1016/j.ecresq.2006.04.002 Grönroos, C. (2011). Value co-creation in service logic: A critical analysis. Marketing Theory, 11(3), 279–301. https://doi.org/10.1177/1470593111408177 Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (1998). Multivariate data analysis (Vol. 5). Pearson. https://www.amazon.com/Multivariate-Data-AnalysisJoseph-Hair/dp/0138132631 Hallowell, R. (1996). The relationships of customer satisfaction, customer loyalty, and profitability: An empirical study. International Journal of Service Industry Management, 7(4), 27–42. https://doi.org/10.1108/09564239610129931 Harvey, L., & Green, D. (1993). Defining quality. Assessment & Evaluation in Higher Education, 18(1), 9–34. https://doi.org/10.1080/0260293930180102 Hemsley-Brown, J., Lowrie, A., Gruber, T., Fuß, S., Voss, R., & Gläser-Zikuda, M. (2010). Examining student satisfaction with higher education services: Using a new measurement tool. International Journal of Public Sector Management, 23(2), 105–123. https://doi.org/10.1108/09513551011022474 Hoogland, J. J., & Boomsma, A. (1998). Robustness studies in covariance structure modeling. Sociological Methods & Research, 26(3), 329–367. https://doi.org/10.1177/0049124198026003003 Kamran-Disfani, O., Mantrala, M. K., Izquierdo-Yusta, A., & Martínez-Ruiz, M. P. (2017). The impact of retail store format on the satisfaction-loyalty link: An empirical investigation. Journal of Business Research, 77, 14–22. https://doi.org/10.1016/j.jbusres.2017.04.004 Kandampully, J., & Suhartanto, D. (2000). Customer loyalty in the hotel industry: The role of customer satisfaction and image. International Journal of Contemporary Hospitality Management, 12(6), 346–351. https://doi.org/10.1108/09596110010342559 Kardoyo, K., Pitaloka, L. K., Rozman, R., & Hapsoro, B. B. (2019). Analyzing universities service quality to student satisfaction; academic and non-academic analyses. International Journal of Higher Education, 9(1), 126.
http://ijlter.org/index.php/ijlter
225
https://doi.org/10.5430/ijhe.v9n1p126 Kasiri, L. A., Guan Cheng, K. T., Sambasivan, M., & Sidin, S. M. (2017). Integration of standardization and customization: Impact on service quality, customer satisfaction, and loyalty. Journal of Retailing and Consumer Services, 35, 91–97. https://doi.org/10.1016/j.jretconser.2016.11.007 Kassim, N. M., Najdawi, M., Al Azmeh, Z., & Sadiq, H. (2010). Effects of institutional reform on corporate image and value in a developing country context. Measuring Business Excellence, 14(2), 32–45. https://doi.org/10.1108/13683041011047849 Kaura, V., Durga Prasad, C. S., & Sharma, S. (2015). Service quality, service convenience, price and fairness, customer loyalty, and the mediating role of customer satisfaction. International Journal of Bank Marketing, 33(4), 404–422. https://doi.org/10.1108/IJBM-04-2014-0048 Lan Anh. (2015). Giáo dục muốn phát triển phải cạnh tranh [Competition is required for the development of education]. Vietnamnet. https://vietnamnet.vn/vn/tuanvietnam/giao-duc-muon-phat-trien-phai-canhtranh-274106.html Lee, S. Y. (2017). Service quality of sports centers and customer loyalty. Asia Pacific Journal of Marketing and Logistics, 29(4), 870–879. https://doi.org/10.1108/APJML-102016-0191 Li, C. K., & Hung, C. H. (2009). Marketing tactics and parents’ loyalty: The mediating role of school image. Journal of Educational Administration, 47(4), 477–489. https://doi.org/10.1108/09578230910967455 lin, C., & Tsai, Y. H. (2008). Modeling educational quality and student loyalty: A quantitative approach based on the theory of information cascades. Quality & Quantity, 42(3), 397–415. https://doi.org/10.1007/s11135-006-9051-5 Loh, C. Y. R., & Teo, T. C. (2017). Understanding Asian students learning styles, cultural influence and learning strategies. Journal of Education & Social Policy, 7(1), 194–210. https://www.jespnet.com/journals/Vol_4_No_1_March_2017/23.pdf Mohammad Mosadeghrad, A. (2013). Healthcare service quality: Towards a broad definition. International Journal of Health Care Quality Assurance, 26(3), 203–219. https://doi.org/10.1108/09526861311311409 Mulyono, H., Hadian, A., Purba, N., & Pramono, R. (2020). Effect of service quality toward student satisfaction and loyalty in higher education. The Journal of Asian Finance, Economics and Business, 7(10), 929–938. https://doi.org/10.13106/jafeb.2020.vol7.no10.929 Nguyen, P. (2019). Tự chủ trong giáo dục phổ thông là xu thế tất yếu [Autonomy in secondary education is an inevitable trend]. Nhan Dan. https://www.nhandan.com.vn/hangthang/item/41735702-“tu-chu-trong-giaoduc-pho-thong-la-xu-the-tat-yeu”.html Nichols, A. L., & Maner, J. K. (2008). The good subject effect: Investigating participant demand characteristics. The Journal of General Psychology, 135(2), 151–166. https://doi.org/10.3200/GENP.135.2.151-166 Pham, H.-H., & Lai, S. L. (2016). Higher education as an extended duration service. Journal of Studies in International Education, 20(5), 454–471. https://doi.org/10.1177/1028315316662978 Pham, H.-H., Lai, S. L., & Vuong, Q.-H. (2019). The role of subjective task value in forming satisfaction and loyalty among Vietnamese international students: A structural equation model. The Asia-Pacific Education Researcher, 28(5), 399–409. https://doi.org/10.1007/s40299-019-00439-3 Rolfsman, E. (2020). Swedish students in the process of transition to upper secondary education: Factors of importance for educational choice and for their future.
http://ijlter.org/index.php/ijlter
226
Education Inquiry, 11(4), 331–359. https://doi.org/10.1080/20004508.2020.1746480 Savolainen, H., Ahonen, T., Aro, M., Tolvanen, A., & Holopainen, L. (2008). Reading comprehension, word reading and spelling as predictors of school achievement and choice of secondary education. Learning and Instruction, 18(2), 201–210. https://doi.org/10.1016/j.learninstruc.2007.09.017 Sharif, K., & Kassim, N. M. (2012). Non-academic service quality: Comparative analysis of students and faculty as users. Journal of Marketing for Higher Education, 22(1), 35– 54. https://doi.org/10.1080/08841241.2012.705793 Sultan, P., & Yin Wong, H. (2013). Antecedents and consequences of service quality in a higher education context. Quality Assurance in Education, 21(1), 70–95. https://doi.org/10.1108/09684881311293070 Teeroovengadum, V., Kamalanabhan, T. J., & Seebaluck, A. K. (2016). Measuring service quality in higher education: Development of a hierarchical model (HESQUAL). Quality Assurance in Education, 24(2), 244–258. https://doi.org/10.1108/QAE-062014-0028 Teeroovengadum, V., Nunkoo, R., Gronroos, C., Kamalanabhan, T. J., & Seebaluck, A. K. (2019). Higher education service quality, student satisfaction and loyalty: Validating the HESQUAL scale and testing an improved structural model. Quality Assurance in Education, 27(4), 427–445. https://doi.org/10.1108/QAE-01-20190003 Vuong, Q.-H. (2020). Reform retractions to make them more transparent. Nature, 582(7811), 149–149. https://doi.org/10.1038/d41586-020-01694-x Wiers-Jenssen, J., Stensaker, B., & Gr⊘gaard, J. B. (2002). Student satisfaction: Towards an empirical deconstruction of the concept. Quality in Higher Education, 8(2), 183–195. https://doi.org/10.1080/1353832022000004377
http://ijlter.org/index.php/ijlter
227
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 227-239, August 2021 https://doi.org/10.26803/ijlter.20.8.14 Received Jun 16, 2021; Revised Aug 19, 2021; Accepted Aug 30, 2021
Developing a Framework Peace Education for Primary School Teachers in Indonesia Sahril Buchori, Sunaryo Kartadinata, Syamsu Yusuf and Ilfiandra Universitas Pendidikan Indonesia, Indonesia https://orcid.org/0000-0001-7184-8576 https://orcid.org/0000-0003-3767-4706 https://orcid.org/0000-0002-3775-8857 https://orcid.org/0000-0002-0886-8301 Nurfitriany Fakhri Universitas Negeri Makasar, Indonesia https://orcid.org/0000-0001-5323-8125 Sofwan Adiputra Universitas Muhammadiyah Pringsewu, Indonesia https://orcid.org/0000-0002-5100-1236
Abstract. Teachers as educators should have a sense of peace within themselves so that a conducive classroom climate can be created. The purpose of this study is to build a framework for peace education in elementary schools. To develop this capacity for peace, the researcher conducted three stages of study: the first, an empirical study; the second, developing a framework; and the third, measuring its effectiveness. This research was conducted in three schools with 12 teachers as participants. The data collection technique used an instrument of teacher peace capacity in the form of a semantic differential scale to determine the level of knowledge, skills and attitudes of teachers' peace capacity. The results show that teachers do not yet have a qualified peace capacity. This framework can be applied to teachers in the form of training. The framework for developing teacher peace capacity has proven to be effective. Keywords: framework; teacher peace capacity; peace education
1. Introduction In an era of development that is so complex, various challenges in the form of personal and social problems become important to study. One of them is the problem of living in peace and harmony. It takes a peaceful atmosphere to live together in harmony. Educational institutions are places to foster the creation of ©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
228
peace; however, they can also be places that militate against peace in society. Therefore settings and conditions are needed in educational institutions that foster peace (Setiadi et al., 2017; UNESCO-APNIEVE, 2000). Education is the means of developing students who are peaceful and harmonious. However, the situation on the ground shows a different situation. Violent behaviour by children is currently a social problem, including in schools. This is a challenge for education in Indonesia. The results of research released in 2015 by the NGO Plan International and the International Center for Research on Women (ICRW) showed that as many as 84% of children in Indonesia experienced violence in schools (Eliasa, 2017). Violence is perpetrated by students in the form of fights both at school and outside of school, brawls, bullying, shouting at other students, skipping school, and graffiti on school walls and the school environment (Chan et al., 2013; Fung et al., 2015). Violent social behaviour results in the destruction of basic assumptions about virtue, justice and security (Janoff-Bulman, 1992). The results of research (Buchori, 2018) show that in elementary schools there is often a less peaceful atmosphere in the classroom than outside the classroom. Students and teachers still display anti-peace behaviour in the form of an inability to contain anger. Students do not care and share, are rude, mock others and generally display provocative behaviour until a fight breaks out. These forms of violent behaviour, such as fighting at school, appear among both junior and senior high-school students, while drug use and the possession of weapons are rife (Furlong & Morrison, 2000). The birth of the anti-violence movement is in line with the emergence of Strengthening Character Education (SCE), which is carried out by applying the values of pancasila in character education in Indonesia. This government programme is a response to the rampant violence that occurs in schools. In addition, other movements have emerged such as the Indonesian AntiDefamation Society (MAFINDO) organisation. The MAFINDO community is concerned with educating the public, especially children, teenagers and students to prevent the spread of slander, hate speech, hoaxes, radicalism, terrorism and other anti-social behaviour. This indicates that violence has become a concern in the school environment (Nurhadi & Muchtarom, 2020). Violent behaviour by children at school is one of the aggressive behaviours that originate from cognitive mechanisms. Children's behaviour reflects an imitation of what is seen and heard around them, thus giving rise to normative beliefs in children (Baron, 1992; Huesmann & Guerra, 1997; Tentama, 2012). Children get justification for their violent behaviour in response to similar behaviour because they think that the response has become a rule of the game based on experience or general agreement that it is acceptable by those around them. This includes both physical and verbal violence (Berkowitz, 1993; Buchori & Fakhri, 2018; Henry et al., 2000; Werner & Hill, 2010). Violent behaviour by students can also be influenced by teachers’ behaviour. The teacher becomes a model for students to behave aggressively. This behaviour can be in the form of yelling, hitting with a ruler, and giving students derogatory nicknames (Harber, 2004; Hilarski, 2004; Meyer, 2006). Good and bad
http://ijlter.org/index.php/ijlter
229
imitated behaviour tend to be exhibited by elementary school students because they do not have the ability to choose or differentiate. Building peace in educational institutions has been carried out by various countries, in both formal and non-formal institutions (Harber & Sakade, 2009). Peace education programmes are conducted in primary schools in the United Kingdom (UK) (Sakade, 2008). Such a programme, the Peacemaker Project, is carried out by the West Midlands Quaker Peace Education Project (WMQPEP). This peace education programme takes the form of conflict resolution that promotes awareness of the nature of conflict and builds nonviolent conflict management skills. The roles of researchers and teachers are those of controllers and facilitators that encourage students to be able to control themselves, to keep the rules set by themselves and be responsible for their own behaviour. One of the studies on peace education in America was conducted by Hunter (2008), a teacher at Redland's Adventist Academy, California. Hunter created a classroom environment that fosters values such as inclusive compassion, social justice, service, and active peace. These educators provide the theoretical basis for an appropriate approach and curriculum for the classroom environment. The curriculum is designed to develop students' intellectual, emotional and social aspects. Peace education is carried out by referring to the curriculum comprising four main skills that will encourage a culture of peace among students, namely empathy training, respecting diversity of training, community awareness, and conflict resolution. This concept is emphasised in extracurricular activities. Classes are designed to provide a variety of books and tools that engage students in learning about peace. Posters and student work are displayed on a bulletin board as a reminder of the values of peace they are learning. Peace education in Finland has been integrated into global education, which has become an explicit part of the national education system (Kartadinata et al., 2015). Peace education is carried out continuously and holistically, and aims to develop the knowledge, skills, attitudes and values needed to create peace in the form of avoiding and overcoming conflicts and violence in their environment. The process involves teachers, students, parents and the community. Peace education was also developed by UNESCO (UNESCO-APNIEVE, 2000) under the name Peace Education, Human Rights and Democracy. Peace education reconnects peace at all levels of education, develops various methods and materials used by teachers and develops teacher education to enable them to become peacemakers. The ultimate goal is developing everyone to have universal values and the type of behaviour that form the basis for the creation of a culture of peace. Peace education, human rights and democracy are included in the curriculum in its implementation involving teachers, students, parents and the community collaborating with each other. Teaching and learning methods are directed at making peace, ensuring human rights and promoting democracy, both in the form of behaviour in school and as something that needs to be learned using active learning methods, group work, discussions on morals and ethics, and individual learning. A requirement for developing a peaceful classroom climate is a pedagogical framework in schools. This is built through research relating to developing the
http://ijlter.org/index.php/ijlter
230
capacity of students and teachers. Building a classroom climate begins with the teacher who has an important role in educating to build student capacity. (Freire, 1998; O'Connor, 2013; Van Manen, 1991). Teachers as role models are expected to be able to become examples and peacemakers in reinforcing students' peaceful behaviour. The normative belief of students in committing violence is caused by receiving justification through imitation and reinforcement to do so (Taylor et al., 2009). Based on this, the peace capacity of teachers in the form of knowledge, skills, and attitudes of peace needs to be developed. The United Nations (UN) officially recognises schools as institutions that cannot be separated from the prevalence of violent behaviour in them, whether perpetrated by students, teachers or staff (Benbenishty & Astor, 2005; Dulmus & Harber, 2009; Pinheiro, 2006; Sowers, 2004). The rampant violence that occurs in schools has prompted the government to launch the Anti-Violence Movement in Educational Environment programme. This movement is a form of prevention and control of violence in schools. The results of interviews with teachers revealed that elementary schools were prepared to be known for a peaceful classroom climate. It is necessary to strengthen the peace capacity of teachers in the form of knowledge, skills and attitudes so that peace can be created in the classroom and at school. Teachers need to be able to become peacemakers to build a peaceful classroom climate by educating students to have good morals and mutual love for fellow students and teachers as well as by preventing violent behaviour among students.
2. Literature Review Peace Education The importance of peace education was proclaimed at the 1994 International Conference on Education and endorsed by the UNESCO General Conference in 1995 (UNESCO-APNIEVE, 2000). The phenomenon of physical and psychological violence that occurs in almost all parts of the world is most concerning. Peace education is needed for children to build understanding, solidarity, compassion and tolerance among individuals or groups. Peace education is carried out by developing knowledge, values, attitudes and skills that lead to peace. It is the process and practice of developing nonviolent skills and promoting peaceful attitudes (UNESCO, 2017). The United Nations Children's Fund (UNICEF) defines peace education as the process of promoting the knowledge, skills, attitudes and values necessary to bring about behavioural change that will enable children, youth and adults to prevent conflict and violence, both directly and indirectly, overt and structural; to resolve conflicts peacefully; and to create conditions conducive to peace, whether at intrapersonal, interpersonal, inter- group, national or international levels (Saputra, 2016). Kartadinata et al. (2015) define peace education as a conscious and planned effort that responds to various kinds of conflicts and violence, whether on a personal, local, national or international scale. This behaviour is an effort to create a future that is peaceful and secure in a sustainable manner.
http://ijlter.org/index.php/ijlter
231
Peace education fosters a base of knowledge, skills, attitudes and values that seek to change patterns of thought, attitudes and behaviour related to violence. Then it transforms and builds awareness and understanding, and develops personal and social concerns that enable individuals to live in peace. It aims to create conditions and systems without violence, promote justice and care for the environment and imbue other peaceful values (Castro & Galace, 2010). Peace education has five main postulates, namely explaining the root causes of violence; teaching alternative behaviour; and looking for different forms of violence and eliminating them. In addition, peace is a process that varies according to the context while conflict has the potential to occur everywhere. The example of peace education in the family should be carried over by parents to their children. The function of parents in the family is to teach children about peaceful behaviour as well as being a model in instilling values in order to form character in children (Saputra, 2016). Teachers’ Peaceful Capacity Educators should have the capacity for peace to be peacemakers. This capacity is in the form of seven main competencies of critical peace that educators should possess, namely critical thinking and analysis, empathy and solidarity, building cooperation, participatory and democratic engagement, education and communication strategy, conflict transformation skills and continuous reflection practice (Bajaj, 2015). The peaceful capacity of teachers is their ability in the form of knowledge, skills and attitudes that reflect peace. This knowledge includes the regulation of rights, protection, strengthening of student character education, effective communication skills, conflict resolution, reflective ability, empathy, respect, democracy, and fairness. Developing professional teacher capacity can be in the form of developing teacher knowledge, skills, attitudes and competencies in teaching (Barber, 2005; Goldman & Grimbeek, 2008; Sinkinson, 2009). Teachers are expected to be able to practise non-violent skills, promote peaceful attitudes and find ways to achieve them (UNESCO, 2017). The teacher's peace capacity is the teacher's ability to create peace in the classroom. Creating peace is based on mutual respect for human rights, respect, tolerance, love, democracy, and acceptance of one another (Castro & Galace, 2010; Galtung, 1996). Peace capacity is characterised by the presence of intrapersonal peace in the form of consistency of behaviour, emotions and attitudes towards peace; peace within oneself and in all aspects of life; and having values, competencies and cognitive dispositions that tend to promote peace. Tolerance, for example, appears to be relevant to attitudes and behaviour across all domains and will tend to influence a person to be at peace with oneself, others, and other groups (Nelson, 2014).
http://ijlter.org/index.php/ijlter
232
3. Methods Research Design The research used a research and development approach (Borg & Gall, 2007). The design comprised a pre-experimental study of the impact of teachers' peace capacity. The research design used is one-group pretest-posttest design in which the subject is given a pretest, then given treatment in the form of a framework for developing the peace capacity of the teacher. This is followed by a posttest. Data Collection The instrument of teacher peace capacity was in the form of a semantic differential scale with a range of 1 to 10. This scale was developed based on indicators of the teachers’ level of knowledge, skills and attitudes of peace. This instrument aimed to collect data on the peaceful capacity of teachers in primary schools. Population and Sample This research involved three schools in Makassar City, namely Borong State Elementary School, Borong Impres Elementary School and Farida Aryani Madrasah Ibtidaiyah Makassar. The participants involved were 12 fourth and fifth-grade teachers to determine the sample using purposive random sampling. Research Stages The research procedure was carried out through six stages, namely studying the concept of teacher quality, an empirical study of the profile of teacher capacity in three schools, the development of a hypothetical training structure, training testing, intervention, and finally, evaluation of the training quality. Analysis of Research Data For testing the effectiveness of the training, an analysis was conducted using the Wilcoxon signed-rank and Kruskal-Wallis test rank. This data analysis technique was used to be able to determine the difference in values between the pretest and posttest. The research data has a small N value so that in data analysis nonparametric statistics are used.
4. Results The research process to determine the level of the students' classroom climate began by measuring the peaceful capacity of the teacher who interacts with students as well as the learning process in the classroom. The results of the study regarding the initial description of the peace capacity of teachers before treatment in the form of teacher peace capacity development training can be seen in Table 1.
Teachers’ Peaceful Capacity
Table 1: Initial profile of teachers’ peaceful capacity Category formula Score Peace Mean + 1SD ≤ X 94 ≤ X Less Peace Mean - 1SD ≤ X < Mean + 1SD 69 ≤ X < 94 Not Peace X < Mean - 1SD X < 69
http://ijlter.org/index.php/ijlter
F 1 9 2
% 8.3 75.0 16.7
233
Most of the teachers are in the ‘less peaceful’ category. Only a small number are in the ‘good’ category. Based on these data, in general the initial description of the peaceful capacity of teachers reflects the ‘inadequate’ category. The data above shows that most teachers lack the capacity for peace in building a classroom climate. The results of this study are in line with the findings of Setiadi et al. (2017) which found that the peace capacity of teachers is still low. According to the findings of Akhmad et al. (2016), the attitudes and behaviour of teachers contribute 50% to the non-peaceful events experienced by students at school. These behaviours include giving verbal and physical punishment to students, language that is not educating or appropriate, and reacting aggressively, both verbally and physically, to the negative behaviour of students. The initial description of the peaceful capacity of teachers is seen from three aspects, namely aspects of knowledge, skills and attitudes. Table 2: Aspect profile of teachers’ peaceful capacity Aspect
Category High Medium Low High Medium Low High Medium Low
Knowledge
Skills
Attitude
Formula Mean + 1SD ≤ X Mean - 1SD ≤ X < Mean + 1SD X < Mean - 1SD Mean + 1SD ≤ X Mean - 1SD ≤ X < Mean + 1SD X < Mean - 1SD Mean + 1SD ≤ X Mean - 1SD ≤ X < Mean + 1SD X < Mean - 1SD
Score 38 ≤ X 28 ≤ X < 38 X < 28 27 ≤ X 17 ≤ X < 27 X < 17 30 ≤ X 23 ≤ X < 30 X < 23
F 2 8 2 3 8 1 2 7 3
% 16.7 66.7 16.7 25.0 66.7 8.3 16.7 58.3 25.0
The level of teacher capacity seen from all aspects of peace reflects the ‘least’ category. Moreover, judging from the indicators, there are 12 indicators out of 14 indicators in the medium category. Therefore teachers need to be trained in all aspects of knowledge, skills and attitudes contained in the indicators. The capacity for peace is an important quality, especially in a teacher as an educator (UNESCO, 1996; UNESCO, 2017). However, the results of the preliminary research indicate that most teachers do not have the capacity for peace within themselves. Table 3: Results of mean and standard deviation analysis based on pre- & posttest training Training
N
Mean
Pretest Posttest
12 12
81.33 127.92
Std. Deviation 12.565 3.476
Minimum
Maximum
61 122
96 131
Table 4: Assumptions test for Wilcoxon signed-rank test Posttest – Pretest KKG
http://ijlter.org/index.php/ijlter
p .002
α 0,05
Conclusion Signifikan
234
The results of the analysis showed p = 0.002 <0.05, which means that there is a significant difference in the peace capacity or ability of the teachers before and after the training. With the difference in the mean value of pretest = 81.33 and posttest = 127.92, it can be concluded that after being given training, there is an increase in the peace capacity of teachers. Based on the analysis of the Wilcoxon signed-rank test, it can be seen that p = 0.002 < 0.05. Therefore it can be concluded that there are differences in the peace capacity of teachers after being given training when viewed in terms of each indicator, with the biggest change occurring in the peace capacity of teachers being the indicator of ‘Strengthening Character Education’. Table 5: Results of the analysis of the mean and standard deviation based on the pre& posttest aspects of teachers’ peaceful capacity Aspect Knowledge Skills Attitude
Pre & Posttest Pretest Posttest Pretest Posttest Pretest Posttest
Valid N (listwise)
N 12 12 12 12 12 12 12
Minimu m 25 53 12 34 21 34
Maximu m 40 59 29 37 31 38
Mean 33.08 56.08 21.92 35.83 26.33 36.00
Std. Deviation 4.757 1.730 5.299 1.115 3.257 1.348
Table 6: Aspect assumption test using Kruskal-Wallis analysis test rank Aspect Knowledge Skills Attitude Total
N 12 12 12 36
Mean Rank 26.67 16.63 12.21
p
α
Conclusion
.003
0.05
Conclusion
Based on the Kruskal Wallis' analysis, p = 0.003 < 0.05. Therefore it can be concluded that there is a difference in the peace capacity of teachers after being given training when viewed from each aspect, with the biggest change occurring in the peaceful capacity of teachers in the knowledge aspect.
5. Discussion After the provision of training, there was a change in the teachers’ peace capacity. There is also an increase in the number of teachers who have the capacity to be included the peaceful category. The results of this study indicate that teachers as educators should develop the capacity for peace in order to have the knowledge, skills and attitudes of peace within themselves (Arifudin, 2007; Cavanagh et al., 2012; Invernizzi, 2005; Kirkwood-Tucker, 2004; Tilaar, 1999; Widayati, 2002). The application of peaceful values should be integrated into both educational and extracurricular activities. This can be achieved by strengthening the values of peace in supporting the learning process. In extracurricular activities, reinforcing the values of peace is carried out in order to develop the potential,
http://ijlter.org/index.php/ijlter
235
interests, talents, abilities, personality, and cooperation of students in voicing and building peace or being peacemakers. The capacity for peace that teachers should have is an understanding of the concept of peace in the form of the concept of diversity and relationships among communities, accepting, respecting, working with people of different backgrounds, having empathy, and exhibiting tolerance (Montgomery & McGlynn, 2009). Teachers who have the capacity for peace are better able to provide peaceful learning (Helu, 2010). Teachers who are able to teach by applying the values of peace make students feel comfortable in receiving learning. Students who feel comfortable in participating in learning tend to find it easy to internalise the learning content. Such students should excel in class and in the community (Akhyak et al., 2013). Students who are taught with a sense of peace are inclined to have peaceful personalities and be able to apply the values of peace in every situation and behaviour. On the other hand, students who are always at the receiving end of unreasonable behaviour from their teachers and friends will be negatively affected by this, exhibiting negative behaviour in turn (Deutsch, 1993). Students who are taught with a sense of peace will have a happy personality, attitude and positive behaviour (Deutsch, 1993). Teachers who have a peaceful attitude are happier and find life meaningful (Kyte, 2016). Peace creates comfort and security which ultimately increases psychological well-being in individuals (Cohrs et al., 2013) and economic well-being for society (Cairns & Lewis, 2003; Swee et al., 2019). Parents' perceptions also affect students' peace of mind (Adiputra et al., 2019). Teachers who have the capacity for peace will teach with great compassion. Students tend to be able to follow the lessons and practise what they have learned (Naway, 2019). Moreover, teachers who have the capacity for peace are more inclined to have students with a happy personality (Biggs et al., 2008; Twemlow et al., 2005), and who can more easily deal with the various trials, challenges and problems in life (Cacioppo et al., 2011; Cohrs et al., 2013; Hetherington, 2003).
6. Conclusion Trends in the peace capacity of teachers in schools contributed to the findings of research on teacher peace capacity. Teachers who have knowledge of peace will be better able to be peaceful and have peace-making skills. Then the teachers’ ability to implement the values of peace in learning will increase and will lead to a peaceful atmosphere or climate in the classroom. Based on theoretical studies and the results of empirical studies, this training is effective in developing the peace capacity of teachers in the form of increasing knowledge, skills and attitudes of teachers' peace. Teachers who have a good peace capacity will create a peaceful classroom climate.
http://ijlter.org/index.php/ijlter
236
7. Acknowledgment The writers would like to express their gratitude to the Indonesian government, especially the Indonesian Directorate General of Higher Education (DIKTI), and the Indonesia Endowment Fund for Education (LPDP) for funding this research.
8. References Adiputra, S., Mujiyati, & Hendrowati, T. Y. (2019). Perceptions of inclusion education by parents of elementary school-aged children in Lampung, Indonesia. International Journal of Instruction, 12(1), 199-212. https://doi.org/10.29333/iji.2019.12113a Akhmad, S. N., Kartadinata, S., & Ilfiandra. (2016). Perspektif Peserta Didik tentang Kedamaian dan Resolusi Konflik di Sekolah[Learners Perspectives on Peace and Conflict Resolution in Schools]. Pedagogia: Jurnal Ilmu Pendidikan, 14(2), 343–355. https://doi.org/10.17509/pedagogia.v14i2.3884 Akhyak, M. A., Idrus, M., & Bakar, A. Y. (2013). Implementation of teachers’ pedagogy competence to optimizing learners’ development in public primary schools in Indonesia. Journal of Education and Research, 1(9), 113-122. Arifudin, I. (2007). Urgensi Implementasi Pendidikan Multikultural di Sekolah [The Urgency of Implementation of Multicultural Education in Schools]. Jurnal Pemikiran Alternatif Pendidikan Insania, 12(2), 220-233. https://doi.org/10.24090/insania.v12i2.252 Bajaj, M. (2015). Pedagogies of resistance and critical peace education praxis. Journal of Peace Education, 12(2), 154-166. https://doi.org/10.1080/17400201.2014.991914 Barber, N. (2005). Practitioner adaptations of strengths-based practice in statutory child protection work. The Australian Institute of Family Studies Newsletter, 13, 10–15. Baron, J. (1992). The effect of normative belief on anticipated emotions. Journal of Personality and Social Psychology, 63(2), 320-330. https://psycnet.apa.org/doi/10.1037/0022-3514.63.2.320 Benbenishty, R., & Astor. S. (2005). School violence in context. Oxford University Press. Berkowitz, L. (1993). Aggression: Its causes, consequences, and control. McGraw-Hill. Biggs, B. K., Vernberg, E. M., Twemlow, S. W., Fonagy, P., & Dill, E. J. (2008) Teacher adherence and its relation to teacher attitudes and student outcomes in an elementary school-based violence prevention program. School Psychology Review, 37(4), 533-549. https://doi.org/10.1080/02796015.2008.12087866 Borg, W. R., & Gall, M. D. (2007). Educational research: An introduction (8th ed.). PearsonAllyn & Bacon. Buchori, S. (2018). Profil Budaya Damai Siswa Sekolah Dasar di Cilandak Jakarta Selatan [Profile of the Peaceful Culture of Elementary School Students in Cilandak, South Jakarta]. Supriatna, M., & dan Wahidin (Eds.). Orientasi Pribadi Optimal Perspektif Bimbingan dan Konseling. UPI Press. Buchori, S., & Fakhri, N. (2018). Terapi Film dalam Mengembangkan Budaya Damai Siswa [Movie Therapy in Building a Culture of Peace in Students]. Prosiding. Kongres dan Konvensi Nasional Bimbingan dan Konseling ke XX. ABKIN bekerja sama dengan Universitas Riau. Pekanbaru [Proceedings. XXth National Congress and Convention on Guidance and Counseling. ABKIN cooperates with the University of Riau. Pekanbaru]. Cacioppo, J. T., Reis, H. T., & Zautra, A. K. (2011). Social Resilience: The Value of Social Fitness with an Application to the Military. American Psychologist, 66, 43–51. https://psycnet.apa.org/doi /10.1037/a0021419 Cairns, E., & Lewis, C. A. (2003). Empowering peace. The Psychologist, 16, 142–143.
http://ijlter.org/index.php/ijlter
237
Castro, L. N., & Galace, J. N. (2010). Peace education: A pathway to a culture of peace. Center for Peace Education, Miriam College. Cavanagh, T., Macfarlane, A., Glynn, T., & Macfarlane, S. (2012). Creating peaceful and effective schools through a culture of care. Discourse: Studies in the Cultural Politics of Education, 33(3), 443– 455. https://doi.org/10.1080/01596306.2012.681902 Chan, J. Y., Fung, A. L., & Gerstein, L. H. (2013). Correlates of pure and co-occurring proactive and reactive aggressors in Hong Kong. Psychology in the Schools, 50 (2), 181-192. https://doi.org/10.1002/ pits.21665 Cohrs, J. C., Christie, D. J., White, M. P., & Das, C. (2013). Contributions of positive psychology to peace: Toward global well-being and resilience. American Psychologist, 68(7), 590–600. https://psycnet.apa.org/doi/10.1037/a0032089 Deutsch, M. (1993). Educating for a peaceful world. American Psychologist, 48(2), 510-517. Deutsch, M. (2015). Educating for a peaceful world. Springer briefs on pioneers in science and practice, 89–103. https://doi.org/10.1007/978-3-319-15443-5_7 Dulmus, C., & K. Sowers. (Eds). (2004). Kids and violence: The invisible school experience. Haworth Press. Eliasa, E. I. (2017). Budaya Damai Mahasiswa di Yogyakarta. Journal of Multicultural Studies in Guidance and Counseling, 1(2), 175-19. https://doi.org/10.17509/jomsign.v1i2.8286 Freire, P. (1998). Pedagogy of freedom. Ethics, democracy and civic courage. Rowman & Littlefield Publishers. Fung, A. L., Gerstein, L. H., Chan, Y., & Engebretson, J. (2015). Relationship of aggression to anxiety, depression, anger, and empathy in Hong Kong. Journal of Child and Family Studies, 24(3), 821-831. http://dx.doi.org/10.1007/s10826-0139892-1 Furlong, M., & Morrison, G. (2000). The school in school violence: Definitions and facts. Journal of Emotional and Behavioral Disorders, 8(2), 71–81. https://doi.org/10.1177%2F106342660000800203 Galtung, J. (1996). Peace by peaceful means peace and conflict, development and civilization. Sage. Goldman, J. D. G., & Grimbeek, P. (2008). Student-teachers’ understanding of policy behavioural directives concerning the reporting of child sexual abuse: Findings from one Australian state. Educational Research, 50(3), 291–305. https://doi.org/10.1080/ 00131880802309457 Harber, C. (2004). Schooling as violence. Routledge Falmer. Harber, C. (2009). Forthcoming. Long time coming: Children as only occasional decisionmakers in schools. In S. Cox, S., A. Robinson-Pant, C. Dyer, & M. Schweisfurth (Eds). Children as decision makers in education. Continuum. Harber, C., & Sakade, N. (2009). Schooling for violence and peace: How does peace education differ from ‘normal’ schooling? Journal of Peace Education, 6(2), 171– 187. https://doi.org/10.1080/ 17400200903086599 Helu Thaman, K. (2010). Teacher capacities for working towards peace and sustainable development. International Journal of Sustainability in Higher Education, 11(4), 353– 364. https://doi.org/ 10.1108/14676371011077577 Henry, D., Guerra, N., Huesmann, L. R., Tolan, P., Van Acker, R., & Eron, L. (2000). Normative influences on aggression in urban elementary school classroom. American Journal of Community Psychology, 28(1), 59-81.
http://ijlter.org/index.php/ijlter
238
Hetherington, E. M. (2003). Social support and the adjustment of children in divorced and remarried families. Childhood: A Global Journal of Child Research, 10, 217–236. https://doi.org/10.1177/ 0907568203010002007 Hilarski, C. (2004). Corporal punishment: Another form of school violence. In C. Dulmus and K. Sowers (Eds). Kids and violence: The invisible school experience. Haworth Press. Huesmann, L. R., Rowell, L., & Guerra, N. G. (1997). Children’s normative belief about aggression and aggressive behavior. Journal of Personality and Social Psychology, 72(2), 408-419. https://psycnet.apa.org/doi/ 10.1037/0022-3514.72.2.408 Hunter, T. J. (2008). Creating a culture of peace in the elementary classroom. The Education Digest, 74(1), 54-58. Invernizzi, M. A., Landrum, T. J., Howell, J. L., & Warley, H. P. (2005). Toward the peaceful coexistence of test developers, policymakers, and teachers in an era of accountability. The Reading Teacher, 58(7), 610– 618. https://doi.org/10.1598/RT.58.7.2 Janoff-Bulman, R. (1992). Shattered assumptions: Towards a new psychology of trauma. The Free Press. Kartadinata, S., Affandi, I., Wahyudin, D., & Ruyadi, Y. (2015). Pendidikan kedamaian. Remaja Rosdakarya. Kirkwood-Tucker, T. F. (2004). Empowering teachers to create a more peaceful world through global education: Simulating the United Nations. Theory & Research in Social Education, 32(1), 56–74. https://doi.org/10.1080/00933104.2004.10473243 Kyte, D. (2016). Toward a sustainable sense of self in teaching and teacher education: Sustainable happiness and well-being through mindfulness. McGill Journal of Education / Revue des Sciences de l'Education de McGill, 51(3), 1143–1162 https://doi.org/10.7202/ 1039632ar Meyer, E. (2006). Gendered harassment in North America: Recognising homophobia and heterosexism among students. In F. Leach & C. Mitchell (Eds.) Combating gender violence in and around schools. Trentham Books. Montgomery, A., & McGlynn, C. (2009). New peace, new teachers: Student teachers’ perspectives of diversity and community relations in Northern Ireland. Teaching and Teacher Education, 25(3), 391–399. https://doi.org/10.1016/j.tate.2008.11.002 Naway, F. A. (2019). Sang Guru: Panduan Guru Profesional Menuju Indonesia 4.0. PGRI Gorontalo-Press. Nelson, L. L. (2014). Peacefulness as a personality trait. In G. K. Sims, L. L. Nelson, & M.R. Puopolo (Eds.). Personal peacefulness: Psychological perspectives. Springer. Nurhadi, A. M., & Muchtarom, M. (2020). Developing civic responsibility in the community through the social community “Masyarakat Anti Fitnah Indonesia (Mafindo)”. In 3rd International Conference on Learning Innovation and Quality Education (ICLIQE 2019) (pp. 1087-1094). Atlantis Press. O’Connor, P. (2013). Pedagogy of love and care: Shaken schools respond. Disaster Prevention and Management, 22(5), 425-433. https://doi.org/10.1016/j.tate.2008.11.002 Pinheiro, P. (2006). World report on violence against children. United Nations. Sakade, N. (2008). Peace education in practice? A case study of peace education in England. (Doctoral thesis in preparation). University of Birmingham. Saputra, W. N. A. (2016). Pendidikan kedamaian: Peluang penerapan pada pendidikan tingkat dasar di Indonesia [Peace education: Opportunities for application to primary education in Indonesia]. Jurnal CARE Edisi Khusus Temu Ilmiah, 03(3), hlm. 88–94.
http://ijlter.org/index.php/ijlter
239
Setiadi, R., Kartadinata, S., Ilfiandra & Nakaya, A. (2017). A peace pedagogy model for the development of peace culture in an education setting. The Open Psychology Journal, 10(1), 182-189. http://dx.doi.org/10.2174/1874350101710010182 Sinkinson, M. (2009). “Sexuality isn’t just about sex”: Pre-service teachers’ shifting constructs of sexuality education. Sex Education: Sexuality, Society and Learning, 9(4), 421–436. https://doi.org/ 10.1080/14681810903265352 Swee, E., Zhan, H., & Powdthavee, N. (2019). Do perceptions of economic well-being predict the onset of war and peace? (No. 12650). IZA Discussion Papers. Taylor, S. E., Peplau, L. A., & Sears, D. O. (2009). Psikologi social: edisi dua belas [Social psychology: twelfth edition]. Kencana Prenada Media Grup. Tentama, F. (2013). Perilaku anak agresif: Asesmen dan intervensinya [Aggressive child behavior: Assessment and intervention]. Jurnal Kesehatan Masyarakat (Journal of Public Health), 6(2), 162-232. Tilaar, H. A. R. (1999). Beberapa Agenda Reformasi Pendidikan Nasional Dalam Perspektif Abad 21 [Several Agendas for National Education Reform in 21st Century Perspective]. Indonesia Tera. Twemlow, S. W., Fonagy, P., & Sacco, F. C. (2005). A developmental approach to mentalizing communities: II. The Peaceful Schools experiment. Bulletin of the Menninger Clinic, 69(4), 282–304. https://doi.org/10.1521/bumc.2005.69.4.282 UNESCO. (1996). Learning: Treasure within. UNESCO Publishing. UNESCO. (2017). Transformative pedagogy for peace-building: A guide for teachers. UNESCOInternational Institute for Capacity Building in Africa (IICBA). UNESCO-APNIEVE. (2000). Belajar untuk Hidup Bersama dalam Damai dan Harmoni: Pendidikan Nilai untuk Perdamaian, Hak-hak Asasi Manusia, Demokrasi dan Pembangunan Berkelanjutan untuk Kawasan Asia-Pasifik [Learning to Live Together in Peace and Harmony: Values Education for Peace, Human Rights, Democracy and Sustainable Development for the Asia-Pacific Region]. Buku Sumber UNESCO-APNIEVE untuk Pendidikan Guru dan Jenjang Pendidikan Tinggi. Penerjemah W.P. Napitupulu. Bandung: Komisi Nasional Indonesia untuk UNESCO dan UPI. Van Manen, M. (1991). The tact of teaching: The meaning of pedagogical thoughtfulness. Althouse Press. Werner, N. E., & Hill, L. G. (2010). Individual and peer group normative beliefs about relational aggression. Child Development, 81(3), 826-836. https://doi.org/10.1111/j.1467-8624.2010.01436.x Widayati, S. (2002). Reformasi pendidikan dasar [Primary school education reform]. Grasindo.
http://ijlter.org/index.php/ijlter
240
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 240-261, August 2021 https://doi.org/10.26803/ijlter.20.8.15 Received Jun 15, 2021; Revised Aug 21, 2021; Accepted Aug 30, 2021
The Measurement of Knowledge Construction in A Course of Diagnostic Evaluation of Learning Disorders in Psychology Students Guadalupe Elizabeth Morales-Martinez, Michelle Garcia-Torres and Maria del Carmen Castro-Gonzalez National Autonomous University of Mexico, Mexico City, Mexico https://orcid.org/0000-0002-4662-229X https://orcid.org/0000-0003-1997-9274 https://orcid.org/0000-0001-5883-0857 Yanko Norberto Mezquita-Hoyos Autonomous University of Yucatán, Yucatan, Mexico https://orcid.org/0000-0001-6305-7440
Abstract. This study evaluated the effects of academic learning on the organization and structure of a knowledge schema among psychology students. The authors designed three studies based on the Chronometric Constructive Cognitive Learning Evaluation Model. This article describes the first method of evaluation, which included a conceptual definition task based on the Natural Semantic Networks technique. The participants' task was to define ten target concepts using verbs, nouns, or adjectives as definers, and then rate the quality of each definer, taking into account the degree of semantic relationship between it and the target concept. The results suggest that the students' initial knowledge schema underwent modifications due to the restructuration of the cognitive structure of knowledge, the assimilation of new information nodes, and the elimination or establishment of new relationships between the conceptual nodes of the knowledge schema. The measurement of these cognitive expressions of academic learning through mental representation techniques can have relevant implications for cognitive characterization in student learning and the design of new teaching strategies that take account of the cognitive psychology principles of information processing underlying academic learning. Keywords: learning evaluation; knowledge schema; cognitive construction; Natural Semantic Networks; psychology students
©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
241
1. Introduction Twenty-first century society demands that students learn information as well as skills in order to be able to select and generate new knowledge from this information. The new educational models attempt to address these two types of learning, declarative and procedural, in their study programs. Meanwhile, new technologies have allowed innovation in teaching strategies for concepts and procedures using tools such as virtual reality and digital teaching platforms. However, there are few innovations in the field of learning assessment; furthermore, there are currently no instruments which measure all aspects of declarative and procedural learning (El-Yassin, 2015). To date, the predominant instruments in evaluation have been those that measure academic performance rather than learning process. These instruments are useful in terms of providing numerical indicators of a student’s academic performance; however, these instruments measure the learning through a product without considering the context (Sadeghi & Rahmati, 2017), nor do they measure the personal characteristics of students. In general, summative assessments do not take account of the fact that each student assimilates the knowledge they review in class in a very personal way, and therefore, there is a wide range of results from academic learning even with the same teaching quality and teaching conditions (Wiliam, 2011). Arieli-Attali (2013) points out that teaching exclusively for performance deprives teachers of valuable information which would allow them to make decisions about how to modify their instructional techniques. Therefore, the academic community has begun to recognize that the cognitive needs and characteristics of the student are central elements in the design of the teaching-learning-evaluation cycle (Morales-Martinez et al., 2021). In this regard, the United Nations Educational Scientific and Cultural Organization (UNESCO, 2015) emphasizes that the design of learning evaluation tools demands a vision beyond knowledge measurement. Creating effective evaluation tools requires the purpose, scope, nature, impact, and level of the evaluation to be considered.
2. The Chronometric Constructive Cognitive Learning Evaluation Model (C3-LEM) The incorporation of technologies within education has brought about advances in educational design and instruction. However, there is still a gap between these developments and innovation in terms of the methods used for assessing learning. Most of the contributions in this field are limited to a technological encapsulation of already existing evaluation strategies, and there are few evaluation proposals native to the digital age (Morales-Martinez, 2020). Additionally, Arieli-Attali (2013) points out that the available evaluation instruments provide information on specific moments. Thus, it is not possible to obtain a complete picture of the continuous progress of a student’s learning process; therefore, it is difficult for the results obtained from these assessments to help improve a wide spectrum of aspects of teaching. Furthermore, given the nature of these instruments, it is difficult to obtain information on the essential cognitive aspects of learning. For example, there is scarce information about the ways in which students organize information, represent problems, select, and use learning strategies, and make use
http://ijlter.org/index.php/ijlter
242
of self-control skills. All these aspects are of relevance if we take into account that fact that students live in a society with an economy based on the processing of information. One way to approach this educational challenge is to use cognitive technology to design new methods for evaluating learning. Cognitive psychology includes a wide range of techniques used to create instruments that provide information about the state of a student’s knowledge, before, during, and after a course. For example, Lopez-Ramirez and MoralesMartinez (2019), Lopez et al. (2014), Morales & Lopez (2016), Morales-Martinez, Lopez-Perez, et al. (2020), Morales-Martinez et al. (2021), Morales et al. (2017) and Morales-Martinez et al. (2015) have proposed the Chronometric Constructive Cognitive Learning Evaluation Model (C3-LEM), which suggests the use of cognitive tools to measure the cognitive properties of knowledge schemas learned in an academic environment. For example, this model measures what content is in a student’s memory, how it is organized and structured, the temporal and dynamic patterns of these knowledge structures, and other aspects of a student’s knowledge state. The C3-LEM obtains the learning indicators in two phases in the evaluation (Figure 1). The first phase is the constructive cognitive evaluation, whose main objective is to measure the changes in the organization, structure, and cognitive dynamics of a student’s knowledge schema, which are assumed to be due to the learning produced by the course. The second phase involves the chronometric cognitive evaluation of the changes that occur in the temporal patterns of schematic behaviour and that are a measure of the degree of consolidation of the schema in each student's long-term memory.
Figure 1: The two phases and components of the C3-LEM Note: From “Cognitive e-tools for diagnosing the state of medical knowledge in students enrolled for a second time in an anatomy course”, by Morales-Martinez, AngelesCastellanos, et al. (2020), International Journal of Learning, Teaching and Educational Research, 19(9), p. 346 (https://doi.org/10.26803/ijlter.19.9.18). Copyright 2020 by the authors and IJLTER.ORG.
Overall, this evaluation model provides information about conceptual advances throughout academic training. To this end, the C3-LEM takes into account the
http://ijlter.org/index.php/ijlter
243
principles and laws of human processing based on two approaches to cognitive psychology: the theory of human information processing or HIP and the theory of parallel distributed processing or PDP. Since, in this paper, the main objective is to explore the cognitive construction of the knowledge schema, the following section focuses on some of the cognitive principles of knowledge construction from these two approaches. 2.1. The Cognitive Principles of Knowledge Construction From HIP, students' minds build cognitive schemas with the motor, procedural and declarative information that they learn and store in their memories throughout a course or career. Here, declarative schemas are of particular interest since, in an academic learning context, students create knowledge networks about what they learn from the semantic and conceptual information on the course. HIP posits that semantic knowledge networks are made up of information nodes (concepts) and relational links among these concepts. The organization and structure of these nodes and relational links enable the students to give a psychological meaning to their knowledge. In this regard, Figueroa-Nazuno (2007) mentions that each person creates a meaning of the world by constructing or reconstructing the knowledge schemas in their memory beyond free association. The authors of this paper define meaning construction as a cognitive process through which students obtain a personal vision of their knowledge. In an academic environment, meaning results from interaction among the cognitive and emotional characteristics of students and their context and learning experiences. Then students can create meaning or modify the meaning of their knowledge by constructing or reconstructing their cognitive structure throughout an academic learning process. Thus, learning experiences produce changes in the students’ cognitive structure. From the cognitive psychology point of view, the degree to which students can modify their knowledge structures depends on the flexibility and stability of their schemas. According to Lopez-Ramirez and Morales-Martinez (2019) schematic flexibility refers to the malleability of cognitive structure. It means the degree to which the cognitive structure can be reorganized or reconfigured without being destroyed during the assimilation of new information. On the other hand, these authors define schematic stability as the degree to which the cognitive structure can hold its cognitive configuration and organization after assimilating new information. The academic development level of students influences the cognitive flexibility and stability of knowledge schemas. Generally, students who begin their learning process in a new field of knowledge have a pre-schema with a vague organization and a structure that is not very clear (Morales-Martinez et al., 2017). The authors of this paper have hypothesized that this kind of initial cognitive structure could be modified more easily than schemas which are completely organized and configured. Furthermore, grasping the level of organization and structure of knowledge schemas provides information about the learning needs that a student
http://ijlter.org/index.php/ijlter
244
might have. According to Messick (1984), the academic objective for beginners with incipient cognitive structures of knowledge should be the assimilation of new knowledge. Thus, the learning evaluations must measure the recognition or retrieval of information in order to be consistent with this first academic level. In contrast, students at an advanced level of academic development require teaching strategies and assessments that promote the restructuring of the schemas and the use of these schemas in problem-solving. Since the learning needs of beginners are quite different from those of advanced students, it is necessary to diagnose their academic development level. To this end, it is helpful to consider Marzano and Pickering’s learning model (1997) because it proposes three learning dimensions which are directly related to the development of cognitive structures of knowledge, namely: the acquisition and integration of knowledge; extending and refining knowledge; and the meaningful use of knowledge. The first two dimensions are especially linked to this research work, and this was the focus of this study. According to Marzano and Pickering (1997), the acquisition and integration of declarative knowledge require that students store information (facts, concepts, and principles) in their memory, deliberately and consciously. Additionally, this learning dimension involves meaning construction by linking old and new knowledge. Another component of this dimension is information organization. In order to acquire declarative knowledge, students must look for relational patterns within the information. In contrast, extending and refining knowledge refers to discovering new perspectives about the knowledge or establishing new links within information by comparing, classifying, analyzing, or reasoning. In this work, the authors interpreted these two learning dimensions from the perspective of the mental representation of knowledge. The acquisition of knowledge implies incorporating new nodes of information in students' memories regardless of the organization of these concepts. Knowledge integration requires the formation of meaning based on the organization of information stored in the memory (e.g., the priority of recall of new nodes, the general patterns of recall). Extending and refining knowledge entails incorporating more specialized and accurate concepts to theorize each knowledge domain, establishing new relationships among conceptual nodes of the knowledge schema, forming new knowledge structures, or creating innovative inferences from the information stored in the memory. All these aspects of the cognitive construction of knowledge can be measured throughout the constructive cognitive evaluation, which provides evidence about the type of information that students choose as relevant, the way they organize and structure it to build their knowledge, and their academic development. This is illustrated in the next section. 2.2. Advances in the Cognitive Assessment of Knowledge Construction When students enroll on a course, teachers and the education system expect them to learn by undergoing the experiences designed specifically for that purpose.
http://ijlter.org/index.php/ijlter
245
Constructive cognitive evaluation allows the changes in students’ knowledge schema produced by the learning experiences to be measured. This kind of assessment involves applying a mental representation technique and computational simulations (Morales-Martinez et al., 2021; Morales-Martinez, Angeles-Castellanos, et al., 2020; Morales-Martinez, Lopez-Perez, et al., 2020). In this regard, the C3-LEM uses Natural Semantic Networks (NSNs) to explore the mental representation of academic knowledge. The NSNs give information about how persons construct their own meaning of the world during a process of memory construction and reconstruction of the knowledge that is stored in their memory (Figueroa-Nazuno, 2007; Figueroa et al., 1976). From a C3-LEM perspective, the main purpose of NSNs is to discover the meaning that students give to the knowledge that they learn on academic courses. Furthermore, since this technique could be applied at different points in time during the academic course, it is possible to obtain a fluid understanding of the cognitive dynamic of construction of knowledge schemas. Students modify their declarative knowledge schemas endlessly by assimilating the information they consider relevant from the material they review throughout the course. Consequently, the construction of meaning from knowledge is a continuous process through time. The result of this learning process depends upon various factors as the prior knowledge that a student has at the beginning of a course (Morales-Martinez, Lopez-Perez, et al., 2020; Urdiales-Ibarra et al., 2018; MoralesMartinez et al., 2021), the student motivation to learn, the cognitive functioning level on which the course focuses (Morales, 2020), the distribution of topics during the course and the importance assigned to each topic (Morales-Martinez, AngelesCastellanos, et al., 2020). The NSNs are helpful in detecting the characteristics and cognitive changes that the schemas undergo due to learning at different time points within an academic year. In this regard, Morales-Martinez, Lopez-Perez, et al. (2020) observed that psychology students seem to begin the courses with a pre-schema. However, these pre-schemas do not show a clear organization among their information nodes. After the course, students have learned new information nodes, eliminated other nodes, and established new relationships between the nodes they assimilated from the course. These changes are specific indicators that the student has experienced a learning process due to the academic experiences during the course (Morales-Martinez et al., 2021). In addition, the NSNs technique has made it possible to identify difficulties in the integration of information in a unified schema in students with low academic performance and in beginners, when learning a topic (Morales-Martínez, Mezquita-Hoyos, et al., 2018; MoralesMartinez, Angeles-Castellanos, et al., 2020; Morales-Martinez, Lopez-Perez, et al., 2020; Urdiales-Ibarra et al., 2018). Furthermore, Morales-Martinez, AngelesCastellanos, et al. (2020) applied this technique to explore the formation of the anatomy schema in medicine students and they discovered that other factors could affect schema configuration in the minds of students, for example the difficulties associated with academic performance, the level of academic development of students, and the emphasis and distribution of the topics to be reviewed.
http://ijlter.org/index.php/ijlter
246
The NSNs can provide information on the effectiveness of corrective learning strategies. Using this tool, Morales-Martinez, Mezquita-Hoyos, et al. (2018) observed that engineering students who did not achieve passing grades in a computational usability course could integrate the knowledge schema in this subject after attending a remedial course. Bearing in mind that NSNs are very useful for assessing several aspects of learning, Morales-Martinez, TrejoQuintana, et al. (2021) used this technique to explore the cognitive properties of knowledge acquisition on human cognition in psychology students. These authors observed that students with previous information about the evaluated topic have a cognitive pre-schema of knowledge. However, the pre-schemas of advanced students could be different in terms of the content quality, organization, and cognitive structure of knowledge from those of beginners, namely students who are enrolled on a course for the first time (Morales-Martinez, Lopez-Perez, et al., 2020). This result suggests that the changes in knowledge structure are qualitatively different depending upon the expertise of the students. Additionally, the cognitive properties of pre-schemas seem different between beginners and advanced students. Since there are very few studies based on this cognitive approach, it is necessary to provide empirical evidence on the learning properties of knowledge schemas based on the academic development level of different students. Assuming that students with expertise on a topic demonstrate better organization and structure in the knowledge schema than beginner students, the present research work explored if the cognitive structure of the diagnostic evaluation for learning disorders schema in participants in this study was well-organized and structured since students had been enrolled on previous courses on this topic. If the students had a pre-schema based on their expertise in the diagnosis and learning disorders, then, it was expected, that their pre-schema would include well-organized information and a schema configuration which was clearly integrated as observed in the study by Morales-Martinez, Trejo-Quintana, et al. (2021). In brief, this work aims to broaden our understanding of the scope of this technique in a different domain of knowledge and provide more evidence about the cognitive characteristics of knowledge schema in students with some expertise in a topic.
3. Method In this research work, the present authors explored the changes in the knowledge schema that psychology students underwent during a course on the diagnostic evaluation of learning disorders. In order to achieve this, the organizational and structural properties of the students’ knowledge schema were measured throughout the NSNs. This technique is based on a cognitive view of the mental representation of knowledge. From this approach, students form a cognitive structure of the information that they learn during academic courses. NSNs is a tool for observing the cognitive characteristic of such knowledge. In this study, the main objective was to observe the organization, structure, and dynamic properties of the cognitive schemas of knowledge in a group of psychology students.
http://ijlter.org/index.php/ijlter
247
3.1. Sample The participants were 43 undergraduate students enrolled in the 5th semester of psychology (91% female and 9% male). Their ages ranged between 18 and 25 years old (M = 19.7, SD = 1.30). Participants were chosen based on purposive sampling. Participation was voluntary, and the teacher awarded points to the students for their participation. The participants belonged to two different groups; since these groups were each made up of a small number of students, the teaching conditions for each class (time exposure, readings, activities) were the same, and the same teacher taught the topics within the same context, and academic period; thus, the researchers integrated the participation of both groups within a single data set. 3.2. Instruments and Materials The authors designed the NSNs instrument using the Protocol for the Collection of Target Concepts and Central and Deferred Definers (Morales-Martinez, 2015). This protocol provides directions to the researcher to help the teacher identify the ten target concepts for a course. In this study, the target-identified concepts were development, psychomotor, learning, dyslalia, dyslexia, dyscalculia, attention deficit, evaluation, diagnosis, and reporting. The presentation of the target concepts of the study required the use of EVCOG software. This software allows the design, application, capture and analysis of the data for cognitive studies for the C3-LEM (Morales-Martinez, Angeles-Castellanos, et al., 2020) (Morales-Martínez and López-Ramirez, 2018 a, b, c, d). 3.3. Research Design This research employed a quasi-experimental design. The authors applied the NSNs technique before and after the course to explore the changes in the schema, for the diagnostic evaluation of learning disorders, produced by the learning process undertaken on the course. The study involved a definitional task to recover the main conceptual nodes related to the evaluated knowledge schema. 3.4. Procedure The study comprised two stages; during the first one, the authors informed the participants about the objectives, the procedure, and the benefits of participating in the study. In this informative stage, the students who decided to participate voluntarily gave their informed consent. In the second stage, the students received instructions, practiced the NSNs task and answered the final study. The task was to define ten target concepts using verbs, nouns, adjectives, and pronouns as definers. Each objective had to be defined within 60 seconds, and then the participants rated the definers using a scale from 1 to 10. Low scores meant that the quality of the word as a definer was low, and high scores indicated that the definer was significantly related to the target. Three essential restrictions delimited this task; the first was that the targets had to be defined based on the course content; that is, free association was ruled out. The second one restriction was that there was a pre-established time for defining each concept (60 seconds). Finally, the targets were presented at random. The application time oscillated between 15 and 20 minutes. The NSNs task was carried out twice, at the beginning and the end of the course.
http://ijlter.org/index.php/ijlter
248
4. Data Analysis The authors examined the data using two analysis approaches. The first one was the conventional NSNs analysis of the participant data using the EVCOG. This analysis implies the computation of organization and structure indicators proposed by Figueroa et al. (1976) and modified by Lopez (1996) and Lopez and Theios (1992). First, semantic richness (J value) was computed as the number of different definers in each target concept. Semantic weight (M value) was calculated as the quality degree estimated by participants considering the semantic relationship between the definer and target. The SAM group (Semantic analysis of M) was made up of the ten definers with the greatest semantic weight for each target concept. The semantic density (G value) was computed by the closeness score between the concepts of the network. In addition, the authors calculated the inter-response time (IRT), which is the time that student needed to recover and write a definer. Furthermore, the authors computed the percentage of concepts appearing in the initial SAM groups as well as in the final ones (the conceptual constancy or CC Value). Also, the conceptual valuation consistency (CVC Value) was computed taking into account the similarity percentage between the weights assigned to the definers that were constant at the beginning and the end of the course in each SAM group. This indicator is a modification of the Q value of Figueroa et al. (1976) that measures the percentage of similarity in the hierarchies of the common definers between two semantic networks. The second analysis was qualitative and was based on a visual inspection of the organization and structure of the concepts through a GEPHI analysis (Bastian et al., 2009). This analysis first required the SASO matrix (matrix of the semantic analyzer of schemata organization) to be extracted; it is a matrix of association weights among the concepts of the NSNs (Lopez, 1996; Lopez & Theios, 1992). Then, the researchers extracted the SASO matrix by calculating the probability of co-occurrence between the concepts with the following formula: WIJ = -1n{[p(X=0 & Y = 1) p(X=1 & Y = 0)]*[p(X=1 & Y = 1) p(X=0 & Y = 0)]-1}[1] The EVCOG system allows the automatic calculation of the association weight (WIJ) between the concepts (X and Y). This software calculates the probability of co-occurrence between the pairs of concepts. First, the program calculates the joint probability that Y appears, but X does not appear in a SAM group p(X = 0 & Y = 1). The procedure is similar for each element of the formula. However, the calculation of p(X = 1 & Y = 1) involved estimating the hierarchical modulation of M-values in SAM groups. Finally, the authors fed the GEPHI software with the SASO connectivity matrix to visualize the schema (see Figure 4).
5. Results Three aspects were analyzed through the NSNs data. First, the authors determined what kind of information fitted into the initial knowledge schema of students and how this information had changed at the end of course in the students’ memories. The second aspect was to explore the changes in the dynamic of connection among the main conceptual nodes (targets). Finally, it was carried on an inspection of the structure and organization of concepts and structure schema was carried out, through a visual representation of NSNs.
http://ijlter.org/index.php/ijlter
249
5.1. Results from NSNs Analysis The analysis considered the NSNs indicators described by Lopez and Theios (1992). Tables 1 and 2 present the NSNs indicators (frequency, IRT, M, F, J, and G value) that the participants obtained at the beginning and the end of the course. Also, there were changes in the content and number of conceptual nodes through the NSNs (Tables 1 and 2). In this respect, the J value increased for each target towards the end of the course. This result means the students showed an increase in the semantic richness of their knowledge structure which was linked to the course. The increase was not homogeneous across the targets, however. Dyscalculia was the target with the most significant increase in the number of definers, followed by report. Moreover, diagnosis and learning were the targets with the smallest increase in the number of definers. Table 1. SAM groups at the beginning of the course Development F Definidor M 1 Growth 224 4 1 1 1 1 1 3 1 1
Learning Evolution Physical Maturation Stages Go forward Skills Psychological Process Valor J: 220
97 81 77 73 72 57 50 49 40
TIR 20 28 22 39 25 28 29 28 29 19
1 1 2 3 1 1 3 1 1
Valor G: 18.40
Dyslalia Definidor M TIR 2 Language 194 13 5 Disorder 148 24 1 Speech 139 23 2 Words 116 22 3 Difficulty 99 20 5 Problems 89 17 1 Pronunciation 71 35 1 Phonemes 62 44 2 To read 51 25 2 Tongue 48 38 Valor J: 233 Valor G: 14.60 Attention-deficit F Definidor M TIR 1 Distraction 147 31 5 Problems 141 24 5 Disorder 120 21 3 Kids 101 27 1 Hyperactivity 96 21 1 Concentration 61 32 4 Learning 54 35 1 Lack of attention 52 38 2 Attention 46 37 3 Difficulty 38 21 Valor J: 198 Valor G: 10.90 Report F Definidor M TIR 1 A text 180 23 2 Information 113 22 1 Investigation 76 25 1 Reading 67 37 2 Evaluation 62 26 1 Summary 61 29 1 Data 57 22 1 Essay 45 37 1 Conclusion 43 40 3 Results 43 35 Valor J: 189 Valor G: 7.00 F
Psychomotor F Definidor M 1 Movement 276 Motricity Body Development Skills Brain Mind Kids Motor Psychology Valor J: 201
116 97 86 76 69 54 47 40 37
TIR 19
1
Learning Definidor M Knowledge 153
16 21 36 32 28 24 41 28 14
1 2 2 2 3 1 2 1 1
Memory School Attention Development Skills Study Information To learn Education
Valor G: 23.90
Dyslexia Definidor 197 TIR Disorder 136 14 Words 130 18 To read 129 26 Problems 120 16 Letters 111 18 Writing 108 33 Confusion 84 32 Language 76 27 Numbers 51 29 Kids 50 34 Valor J: 233 Valor G: 8.60 Evaluation F Definidor M TIR 2 Tests 153 25 1 To rate 136 24 1 Diagnosis 92 21 1 Exam 85 21 1 Knowledge 70 22 4 Learning 59 31 1 Revision 44 43 2 Questions 42 39 3 Results 36 24 1 To know 29 49 Valor J: 207 Valor G: 12.40
F 5 2 2 5 1 1 1 2 2 3
http://ijlter.org/index.php/ijlter
F
Valor J: 235 F 2 1 5 5 4 3 1 1 1 2
F 2 2 1 5 5 1 1 1 2 3
97 88 68 64 52 51 50 39 38
TIR 20 26 22 28 34 32 20 32 14 22 Valor G: 11.50
Dyscalculia Definidor M TIR Numbers 161 14 Math 132 26 Problems 104 18 Disorder 93 29 Learning 78 39 Difficulty 55 20 Addition 46 34 Operations 44 39 Subtraction 40 33 School 31 31 Valor J: 171 Valor G: 13.00 Diagnosis Definidor M TIR Evaluation 172 17 Tests 99 30 Result 58 20 Problems 51 38 Disorder 50 24 Interview 49 29 Patient 46 32 Treatment 46 43 Questions 34 30 Results 33 22 Valor J: 217 Valor G: 13.90
250
In contrast, the reader can observe in Table 2 that the participants eliminated information nodes that appeared in the first phase, and they also included new information nodes in each SAM group. For example, the target dyslalia presented a CC value of 90% for the NSNs; nine of the ten initial concepts remained in the final NSNs (language, disorder, speech, words, difficulty, problems, pronunciation, phonemes, reading), while one of them was eliminated (language). Thus, a new concept (articulation) was assimilated in the SAM group towards the end of the course. On the other hand, evaluation had a CC value of 30% for the NSNs. Of the ten initial concepts, only three concepts (tests, diagnosis, exam) appeared again in the final SAM group, while seven concepts (qualify, knowledge, learning, review, questions, results, know) were eliminated and in their place appeared seven new concepts (treatment, interviews, observation, diagnosis, psychological, analysis, evaluate). Table 2. SAM groups at the end of course Development F Definidor M 1 Growth 138 1 Stages 120 1 Physical 107 1 Maturation 99 1 Evolution 94 1 Cognitive 91 1 Social 88 2 Process 78 1 Psychomotor 61 4 Learning 58 Valor J: 251
TIR 15 20 24 20 12 23 28 18 22 29
Psychomotor Definidor M Movement 246 Motricity 153 Development 148 Body 108 Fine motricity 76 Gross motricity 68 Laterality 57 Difficulty 54 Psychological 54 Brain 36
Valor G: 8.00
Dyslalia Definidor M TIR Language 159 18 Phonemes 157 17 Difficulty 150 18 Disorder 128 16 Articulation 104 25 Speech 79 21 Words 65 26 Problems 53 26 Pronunciation 46 49 To read 45 17 Valor J: 262 Valor G: 11.40 Attention-deficit F Definidor M TIR 5 Disorder 193 16 1 Hyperactivity 175 21 1 Attention 136 26 5 Difficulty 134 18 4 Learning 68 33 1 Distraction 67 29 1 Uneasy 63 37 2 Kids 60 34 1 Lack of attention 49 23 1 Impulsiveness 44 18 Valor J: 242 Valor G: 14.90 Report F Definidor M TIR 2 Diagnosis 180 31 2 Results 168 19 2 Evaluation 141 26 3 Tests 126 21 2 Psychological 124 17 3 Treatment 66 27 2 Information 58 36 1 A text 56 22 3 Interviews 51 17 2 Analysis 48 23 Valor J: 259 Valor G: 12.00
F 2 1 5 5 1 1 2 2 1 3
F 1 1 1 1 1 1 1 5 2 1
Valor J: 241 F 3 1 5 5 1 2 4 2 1 1
F 2 3 3 3 1 1 1 2 2 1
TIR 22 15 23 27 33 33 28 38 42 36
1 1 1 1 3 1 1 2 1 2
Valor G: 21.00
Dyslexia Definidor 197 TIR To read 224 21 Writing 178 19 Difficulty 171 13 Disorder 157 14 Letters 107 22 Language 88 22 Learning 78 37 Words 68 14 Reading-writing 59 30 Omissions 44 40 Valor J: 269 Valor G: 18.00 Evaluation Definidor M TIR Diagnosis 155 26 Tests 151 21 Treatment 75 35 Interviews 65 32 Observation 63 31 Diagnostic 52 17 Psychological 44 33 Analysis 35 36 To evaluate 30 28 Exam 29 31 Valor J: 237 Valor G: 12.60
http://ijlter.org/index.php/ijlter
F
Learning Definidor M Knowledge 163 School 75 To learn 71 Memory 56 To read 56 Experiences 55 Skills 47 Information 42 To know 40 Process 38 Valor J: 258
F 1 1 5 5 1 1 1 4 2 2
F 2 3 3 1 2 2 5 3 1 1
TIR 18 32 20 17 38 26 30 29 48 24 Valor G: 12.50
Dyscalculia Definidor M TIR Numbers 236 17 Math 173 18 Disorder 173 16 Difficulty 142 17 Subtraction 89 34 Addition 83 33 Arithmetic 82 31 Learning 68 31 Problems 65 20 Kids 60 28 Valor J: 286 Valor G: 17.60 Diagnosis Definidor M TIR Evaluation 215 18 Tests 153 23 Treatment 89 25 Functional 82 24 Results 67 22 To evaluate 63 17 Disorder 56 30 Interviews 45 40 DSM-V 41 45 Therapy 36 45 Valor J: 224 Valor G: 17.90
251
Furthermore, the constant nodes had changed their weight of relevance towards the end of the course. This change was heterogeneous through the targets. For example, development obtained a CVC of 41%. In contrast, psychomotor obtained a CVC of 38%. On another note, concerning the structural changes, three of the ten SAM groups (development, psychomotor, dyslalia) indicated a decrease in the G value, which means the closeness of the definers in these three SAM groups had increased at the end of the course. In addition, five SAM groups showed a notable increase in the dispersion of their definers at the end of the course (dyslexia, report, dyscalculia, attention deficit, diagnosis), and two SAM groups had a tiny increase in the dispersion of the definers (learning, evaluation). In addition, there were changes in connectivity between the targets. Figure 2 shows the distribution of connections at the beginning and end of the course.
Figure 2: Changes in the number of connections among targets Note: The number of connections for each conceptual node is reported with the number inside the circle. Furthermore, the connectivity graph after the course gives the number of constant connections (black number) and the new connections (blue number). Also, in the graph for after the course, three aspects of conceptual connectivity are reported: the fading of the initial target connections (red circles), the constancy in the number of target connections (gray circles), and the emergence of new target connections (blue circles).
http://ijlter.org/index.php/ijlter
252
Dyscalculia is the target with the most connections and report is the target with least connections before the course. Meanwhile, at the end of course, the target with the highest number of connections was dyslexia, and evaluation had the lowest number of connections. Dyslexia was the target with the greatest number of new connections, and evaluation presented the lowest number of connections at the end of the course. Diagnosis, attention deficit, dyslexia, and dyscalculia were the targets that retained the number of connections presented in the initial NSNs. Evaluation was the target with the highest number of lost initial connections. 5.2. Results from GEPHI Analysis The GEPHI analysis (Bastian et al., 2009) revealed changes in the conceptual organization of the NSNs at the end of the course (Figure 3). Before the course, the students had information about the subject, and they presented an organized pattern in terms of the relationships between the information nodes they had. At the beginning of course, students organized their knowledge schema into seven conceptual modules. The first one (purple colour) involved 31.43% of the NSNs definers (interview, questions, treatment, patient, information, evaluation, tests, result, results, distraction, concentration, hyperactivity, pronunciation, lack of attention, phonemes, speech, tongue, speech, development, attention, school). The main nodes in this module were problems and disorders. The second module (light green) included 22.86% of the definers (psychological, maturation, evolution, go forward, growth, physical, process, stages, exam, diagnosis, revision, to rate, knowledge). The nodes with more relevance in this group were skills and learning. The third module (light blue) included 15.71% of the definers (movement, body, motricity, mind, brain, motor, psychology, writing, letters, confusion), and the node with more connections was kids. The fourth module (orange) consisted of 10% of the definers (a text, summary, investigation, conclusion, reading, essay, data), and it was disconnected from the NSNs. The fifth module (coffee) contained 7.14% of the definers (difficulty, language, numbers, to read, words). The sixth module (pink) incorporated 7.14% of the definers (to learn, memory, education, study, knowledge). The last module (dark green) integrated 5.71% of the definers (addition, subtraction, operation, math). At the end of the course, students grouped the NSNs definers into nine conceptual modules. The first group (purple) included 21.74% of the definers (psychomotor, physical, cognitive, maturation, social, stages, growth, to read, words, language, problems, kids, evolution, problems), and the more relevant nodes were learning and process. The second module (green) included 20.29% of the definers (lack of attention, uneasy, election, impulsiveness, distraction, hyperactivity, disorder, functional, articulation, DSM-IV, phonemes, speech, pronunciation), and the node with more connections was disorder. The third module (light blue) comprised 14. 49% of the definers (information, evaluation, results, psychological, treatment, analysis, interviews, test, diagnosis, evaluate), and the main node was kids. The fourth module (coffee) contained 13.04% of the definers (brain, motricity, laterality, gross motricity, body, development, fine motricity, movement), and the most salient node was difficulty. The fifth module (orange) included 10.14% of the definers (knowledge, skill, school, memory, experiences, to learn, to know). The sixth group (pink) included 7.25% of the definers (numbers, subtraction, math, addition, arithmetic). The seventh module (dark green) included 5.8% of the definers (writing, letters, omissions, reading, writing). The eighth group (pale pink) involved 5.8% of the definers (diagnosis, observation,
http://ijlter.org/index.php/ijlter
253
exam, psychological), and it was separated from the NSNs; finally, the last group (gray) contained just 1.45% of the definers (a text).
Figure 3: Before and after of the course: GEPHI analysis Note: Each conceptual module obtained by GEPHI from NSNs is represented by a colour.
http://ijlter.org/index.php/ijlter
254
Furthermore, the organization of concepts and the meaning of several conceptual nodes had changed at the end of the course. For example, the definer difficulty attained greater centrality during the second part of the course. This result may indicate that the students had re-signified this concept based on the knowledge they acquired. At the beginning of the course, difficulty was grouped with nodes associated with general areas (language, numbers, words, reading) in which children may present learning difficulties during their development. Meanwhile, at the end of the course, difficulty was grouped with words of a more sophisticated level related to the etiology of these difficulties (brain), the specific areas of development (fine motor, gross motor, laterality) and was also connected to broader spectrum concepts (development or body). Figure 5 illustrates the change in connectivity for the definer difficulty.
Figure 5: Change in connectivity for the definer difficulty
6. Discussion Learning assessment is one of the most significant twenty-first--century classroom challenges. UNESCO (2015) noted that the effectiveness of evaluation tools depends on taking account of the purpose, scope, nature, impact, and level of evaluation in their design. In this regard, this work has demonstrated the effectiveness of constructive cognitive assessment to obtain information about students' state of knowledge before and after a course. For this, the authors applied the NSNs technique to measure the changes in the organization and structure of the knowledge schema for the diagnostic evaluation of learning disorders. The authors explored three aspects of cognitive learning, the first referring to the conceptual content that the students had in their memory about the course, the second relating to the conceptual structure interpreted as the arrangement between the conceptual nodes of the natural semantic network, and the third concerning the general meaning that results from the combination of conceptual content and its organization and structure. The results indicated that students entered the course with prior knowledge about the topic. These results are similar to those of the study by Urdiales-Ibarra et al. (2018) in biology and Morales-Martinez, Lopez-Perez, et al. (2020), and MoralesMartinez et al. (2021) in psychology. The role of these pre-schemas in academic
http://ijlter.org/index.php/ijlter
255
learning has not been widely discussed from the cognitive perspective of knowledge mental representation. Figueroa-Nazuno (2007) mentioned that people construct and reconstruct meanings based on what is in their memory. Thus, the authors hypothesize that these previous cognitive structures are a primary mechanism of knowledge construction. Students use pre-schemas to have cognitive coherence and to create a more efficient learning path. Based on HIP, the participants used their previous schemas to assimilate and accommodate new knowledge; therefore, they probably used a top-down processing style predominantly to manage the information in their memories. There is not enough information to know if this kind of processing is apparent among beginners. A comparison of the conceptual content in Tables 1 and 2 indicates a relevant change in the degree of generality of the NSNs. The initial schema included general concepts connected to the NSNs throughout a psychology macro-schema; additionally, students used common sense to build this seminal schema. However, these concepts were not associated in a strict sense with the content of the course. For example, at the beginning of the course, students defined development alongside concepts taken from a general psychology schema (e.g., psychological) or inferred by common sense (e.g., skills). Meanwhile, at the end of the course, students had eliminated these definers and included more specific concepts such as cognitive, social, and psychomotor. According to Marzano and Pickering's learning model (1997), these results suggest that the course's learning experiences influenced the dimension of the refinement of the information in the memory of these students. Another aspect that the reader can observe in Tables 1 and 2 is the absence of examples to define the targets. The use of examples is common among students who are beginners in the study of a specific topic. For example, Morales-Martinez et al. (2021) reported that at the beginning of a course on human cognition, psychology students used many examples to define the concept of cognition. This result can be associated with the students' initial expertise level on the subject. So, in the present study, the kinds of concepts used in the initial NSNs suggest that the participants had at least some knowledge on the subject, although the knowledge schema they had at the beginning of the course was quite general. In fact, according to their career curriculum, students had received some information on the subject from previous courses. In addition, some concepts showed greater conceptual constancy (Tables 1 and 2). For example, the target dyslalia retained 90% of its definers in the final SAM group, while evaluation had retained only 30% of its definers by the end of the course. Although there is no theoretical discussion about the meaning of conceptual constancy, the authors hypothesize that the constancy of a definer in the NSNs over time is related to the condition of necessity or sufficiency of the definer in defining the target. If the definer is an indispensable or central property of the target, this will be presented constantly overtime in the NSNs. On the other hand, conceptual migration (elimination of conceptual nodes) may be related to circumstances when the initial nodes are not indispensable attributes for the concept; then the definer appeared incidentally in the initial NSNs. For example,
http://ijlter.org/index.php/ijlter
256
when the definers are part of a macro schema of knowledge and this schema is the only conceptual resource that students have, these general definers will appear in the NSNs while the schema evolves towards a more sophisticated one. In addition, a high CC may suggest that the target presents invariance of meaning; thus, the greater the invariance of meaning, the lower the level of flexibility in terms of the schema being modified by new incoming information. In this study, the objectives with the highest CC were dyslalia, attention deficit, dyscalculia, dyslexia, and development; therefore, these targets underwent less migration or conceptual change. The authors suggest that the conceptual stability of these targets could be related to the type of schemata; if a target is clearly defined (classic schemata) or there are clear guides to identify its conceptual properties (probabilistic schemata), then the constancy of definers increases. There is agreement about which characteristics are necessary and sufficient to define learning disorder in psychology. If students learn the criteria to diagnosis these conditions from standardized diagnostic manuals (e.g., Diagnostic and Statistical Manual of Mental Disorders), they demonstrated major consensus in their definitions across time. The second factor influencing the CC could be that students were familiar with these five topics since they had reviewed this knowledge domain throughout their entire academic training. The opposite happened with evaluation, which was the target with the lowest CC, and which therefore underwent greater modification in its final NSNs group. These results support the authors' idea that when a student is a beginner in a certain subject, knowledge schemas can be more flexible, and therefore, these structures can be more easily modified through learning experiences. As students gain more knowledge of a topic, their flexibility in terms of the schemas may decrease, and the stability of the schema may increase. On the other hand, the constructive cognitive evaluation explored the structural characteristics of the knowledge schema that the students constructed during the course. The G value indicates the dispersion between the concepts of the NSNs; in this regard, the reader can observe in Tables 1 and 2 that the target development showed a marked decrease in the dispersion of the defining concepts, while report and dyslexia markedly increased their dispersion. Interpreting these results is not an easy task, given that there is no detailed description of the meaning of network dispersion or density from the field of mental representation of cognitive psychology (e.g., Figueroa et al., 1976; Lopez et al., 2014; Morales-Martinez, Angeles-Castellanos, et al., 2020; Morales-Martinez et al., 2021; Urdiales-Ibarra et al., 2018). The authors suggest that the G values may indicate the degree of similarity among the definers concerning the conceptual belonging they have regarding the target. Low G values could suggest a major homogeneity in definers in defining the concept, while high values may indicate greater variability in the degree to which each definer conceptually typifies a target. The reasons for the changes in G values are unknown; the decrement in this NSNs dispersion could suggest that the learning experiences increase the quality of the selection process for definers. Therefore, the closeness of meaning among these definers increase. Another possibility is that learning experiences influence the valuation process for the
http://ijlter.org/index.php/ijlter
257
grade of relatedness among definers and their target because students acquire abilities to find relations among the definers and their targets. Concerning the pattern of connections between the targets for the NSNs, the results indicated that the targets related to disorders were the most stable in terms of the number of connections and the kind of definers involved in these connections. Since there are few reports about the cognitive nature of NSNs connectivity, it is difficult to explain these results from the cognitive perspective of mental representation. For example, Morales-Martinez et al. (2021) reported on the NSNs connectivity of anatomy in students who did not achieve a passing grade in this subject; however, they did not provide information about the cognitive nature of the connection pattern. In this study, the authors suggest that the persistence of the connection of the schema structure over time could relate to the level of expertise demonstrated by a student. Since, participants in this study had read on learning disorders, they could have formed a schema of this subject very early in their academic development. Then, their NSNs on this topic had well established connections. The third dimension of analysis was related to the meaning of the NSNs. The analysis of the CVC values suggested that the readings reviewed during the course changed the perception that students have about the relevance of the conceptual nodes for the NSNs. The CVC of the definers with CC has not been discussed in any article on constructive cognitive assessment (e.g., MoralesMartinez et al., 2021; Morales-Martinez, Angeles-Castellanos, et al., 2020; MoralesMartinez, Lopez-Perez, et al., 2020; Urdiales-Ibarra et al., 2018). The authors propose that the CVC value may reflect the cognitive flexibility of the schema in terms of conceptual valuation. If the definers with CC are cognitively permeable to the input of new information, they can change their relevance values through a revaluation process given the new information entering the schema and the new relationships established. Therefore, when CVC is high, the best chance of increasing the students' learning rate is by presenting novel information in the form of new conceptual nodes, new conceptual relationships, and new forms by which to interpret and use information. In addition, the GEPHI analysis indicated an essential change in the meaning attributed to learning disorders; at the beginning of the course, the students had learning, disorder, problems, and skills as central nodes, while at the end of the course, the central nodes were learning, disorder and difficulty. The definer difficulty replaced problems; this suggests that the students changed their vision towards a more proactive view of diagnosing and treating learning difficulties. However, since the centrality of skills disappeared, evaluation and diagnosis represented a focus on what is missing or damaged (difficulties and disorders), and thus, the conceptualization of diagnosis continues to be driven by the medical model of disease. Finally, the organization of initial definers suggests that the students had a general idea about assessing and diagnosing learning disorders. They established relationships between the definers in relation to their general knowledge about
http://ijlter.org/index.php/ijlter
258
development and learning. At the end of the course, the students had integrated new information nodes into their knowledge schema, refined the organization of the concepts with CC, and extended their schema with new relationships between the definers (Figure 4). For example, the definer difficulty at the beginning of the course was not central to the NSNs, and its relationships were very general. This definer was connected with concepts of a vague conceptual spectrum for the course (e.g., learning, children, school), and it was connected with only a handful of the distinctive features of learning difficulties (e.g., language, numbers, words, reading). At the end of the course, difficulty acquired greater richness in terms of its conceptual definition and conceptual relationships with other definers (Figure 5). In general, transformations in the participants’ knowledge schema in the three levels analyzed –content, structure, and organization – were apparent. So, in this work, the evidence supports the idea that constructive cognitive assessment effectively diagnoses cognitive changes due to learning. The findings in this research have implications at the theoretical level since there is currently little information available about the cognitive mechanism underlying the development of cognitive structures in academic environments. At a methodological level, empirical evidence about the effectiveness of NSNs as a cognitive approach to evaluate academic learning has been presented in this study. Furthermore, at the level of application, this study provides an alternative tool and cognitive indicators for learning that are useful for formative assessment or assessment for learning. However, since the sample in this study was very small, and there was not a control in the instructional sequence, future research should include comparison and control groups, to explore the effects of teaching strategies. Additionally, it would be very interesting to contrast the quality of organization, structure, and content of knowledge schema based on the academic levels of students. Many other variables and manipulations could be introduced to enhance our comprehension of declarative learning and the formation of cognitive knowledge structures in an academic environment (e.g., induvial characteristics, type of institution, nature of content).
7. Conclusion In sum, this study has offered empirical evidence of the cognitive changes that occur in cognitive structures of knowledge due to the academic learning process. The results indicated that constructive cognitive assessment is helpful in terms of measuring cognitive expressions of learning. For example, in this study, it was possible to identify the changes in the configuration of cognitive schemas for knowledge, changes in the conceptual content, and modifications in the relationships among the conceptual nodes that students had assimilated in their memories due to learning. Thus, cognitive assessment tools such as NSNs are an effective means for diagnosing, monitoring, and evaluating the learning process. Furthermore, the data obtained with this type of tool can be used to improve or generate new teaching strategies adapted to the cognitive characteristics of students.
http://ijlter.org/index.php/ijlter
259
Mapping the state of students’ knowledge gives valuable information about the difficulties that students have in learning material whether due to theoretical confusions, problems in semantic understanding, or difficulties with the structure of information. This kind of cognitive diagnosis will allow teachers to design learning materials and strategies that increase the clarity of the concepts presented in classes. These actions will help students adequately discriminate the conceptual categories presented and achieve significant clarity about their knowledge. However, to implement this type of proposal, it is necessary to continue exploring the benefits and limitations of using cognitive learning assessment tools and to continue the search to establish cognitive measurement parameters that are useful for improving teaching and learning processes.
8. References Arieli-Attali, M. (2013, October 20−25). Formative assessment with cognition in mind: The cognitively based assessment of, for and as learning [Paper presentation]. 39th Annual Conference: Educational Assessment 2.0: Technology in Educational Assessment, Tel Aviv, Israel. Bastian, M., Heymann, S., & Jacomy, M. (2009, May 17−20). Gephi: an open-source software for exploring and manipulating networks [Paper presentation]. Association for the Advancement of Artificial Intelligence, Third International AAAI Conference on Weblogs and Social Media, San Jose California, United States. El-Yassin, H. D. (2015). Integrated assessment in medical education. Journal of Contemporary Medical Sciences, 1(4), 36–38. http://www.jocms.org/index.php/jcms/article/view/51 Figueroa, J. G., González, E. G., & Solís, V. M. (1976). An approach to the problem of meaning: Semantic networks. Journal of Psycholinguistic Research, 5(2), 107−115. https://doi.org/10.1007/BF01067252 Figueroa-Nazuno, J. G. (2007, October 24−25). El significado de las Redes Semánticas Naturales: Y la tradición oral, 20 años después [The meaning of Natural Semantic Networks: And oral tradition, 20 years later] [Paper presentation]. Primer Simposium Internacional: Cognición y Representación del Conocimiento. Monterrey, Nuevo León, México. Lopez, E. O. (1996). Schematically Related Word Recognition (Publication No. 9613356) [Doctoral dissertation, University of Wisconsin-Madison]. ProQuest Dissertations & Theses Global. Lopez-Ramirez, E. O., & Morales-Martinez, G. E. (2019). Learning´s constructivechronometric cognitive assessment model [Unpublished manuscript]. Institute of Research on the University and Education, National Autonomous University of Mexico. Lopez, E. O., & Theios, J. (1992). Semantic analyzer of schemata organization (SASO). Behavior Research Methods, Instruments, & Computers, 24(2), 277−285. https://doi.org/10.3758/BF03203508 Lopez, E. O., Morales, G. E., Hedlefs, I., & Gonzalez, C. J. (2014). New empirical directions to evaluate online learning. International Journal of Advances in Psychology, 3(2), 40−47. https://doi.org/10.14355&ijap.2014.0302.03 Marzano, R. J., & Pickering, D. J. (1997). Dimensions of learning: Teacher's manual (2nd ed.). ASCD. Messick, S. (1984). The psychology of educational measurement. Journal of Educational Measurement, 21(3), 215–237. https://doi.org/10.1111/j.1745-3984.1984.tb01030.x Morales-Martinez, G. E. (2015). Protocolo para la recolección de conceptos objetivo y definidores centrales y diferidos (PRECODECD): Un sistema de codificación de conceptos extraídos de las redes semánticas naturales [Protocol for the collection of objective concepts
http://ijlter.org/index.php/ijlter
260
and central and deferred definers (PRECODECD): A coding system for concepts extracted from natural semantic networks] [Unpublished manuscript]. Institute of Research on the University and Education, National Autonomous University of Mexico. Morales-Martinez, G. E. (2020). Sistema de evaluación cognitiva constructiva cronométrica del aprendizaje en línea y presencial [Online and face-to-face learning´s constructivechronometric cognitive assessment system] [Manuscript submitted for publication]. Institute of Research on the University and Education, National Autonomous University of Mexico. Morales-Martinez, G. E., Ángeles-Castellanos, A. M., Ibarra-Ramírez, V. H., & ManceraRangel, M. I. (2020). Cognitive e-tools for diagnosing the state of medical knowledge in students enrolled for a second time in an anatomy course. International Journal of Learning, Teaching and Educational Research, 19(9), 341−362. https://doi.org/10.26803/ijlter.19.9.18 Morales-Martinez, G., & Lopez-Ramirez, E. (2016). Cognitive responsive e-assessment of constructive e-learning. Journal of e-Learning and Knowledge Society (Je-LKS), 12(4), 39−49. http://www.je-lks.org/ojs/index.php/Je-LKS_EN/article/view/1187 Morales-Martinez, G. E., & Lopez-Ramirez, E. O. (2018a). EVCOG Modulo 1. Configurador RSN [EVCOG Module 1. NSN Configurator] (Version 1.1.) [Computer software]. Registro Público del Derecho de Autor. 03-2018-111311554300-01 Morales-Martinez, G. E., & Lopez-Ramirez, E. O. (2018b). EVCOG Modulo 2. Capturador RSN [EVCOG Module 2. NSN Data storage] (Version 1.1.) [Computer software]. Registro Público del Derecho de Autor. 03-2018-1113115235001 Morales-Martinez, G. E., & Lopez-Ramirez, E. O. (2018c). EVCOG Modulo 3. Editor RSN [EVCOG Module 3. NSN Editor] (Version 1.1.) [Computer software]. Registro Público del Derecho de Autor. 03-2018-1113115235001 Morales-Martinez, G. E., & Lopez-Ramirez, E. O. (2018d). EVCOG Modulo 4. Análisis RSN [EVCOG Module 4. NSN Analysis] (Version 1.1.) [Computer software]. Registro Público del Derecho de Autor. 03-2018-1113115235001 Morales-Martínez, G. E., López-Ramírez, E. O., Castro-Campos, C., Villarreal-Treviño, M. G., & Gonzales-Trujillo, C. J. (2017). Cognitive analysis of meaning and acquired mental representations as an alternative measurement method technique to innovate e-assessment. European Journal of Educational Research, 6(4), 455-464. https://www.eu-jer.com/EU-JER_6_4_455_Morales-Martinez_etal.pdf Morales-Martinez, G. E., Lopez-Ramirez, E. O. & Lopez-Gonzalez, A. E. (2015). New approaches to e-cognitive assessment of e-learning. International Journal for eLearning Security (IjeLS), 5(2), 449−453. https://doi.org/10.20533/ijels.2046.4568.2015.0057 Morales-Martinez, G. E., Mezquita-Hoyos, Y. N., Gonzalez-Trujillo, C. J., Lopez-Ramirez, E. O., & Garcia-Duran, P. J. (2018). Formative e-assessment of schema acquisition in the human lexicon as a tool in adaptive online instruction. In R. Lopez-Ruiz (Ed.), From natural to artificial intelligence: Algorithms and application (pp. 69−88). IntechOpen. http://doi.org/10.5772/intechopen.81623 Morales-Martínez, G. E., López-Pérez, R. M., García-Collantes, A., & López-Ramírez, E. O. (2020). Evaluación constructiva cronométrica para evaluar el aprendizaje en línea y presencial [Constructive-chronometric evaluation as a tool to evaluate online and face-to-face learning]. Tecnología, Ciencia y Educación, 15(1), 105−124. https://www.tecnologia-ciencia educacion.com/index.php/TCE/article/view/371 Morales-Martinez, G. E., Trejo-Quintana, J., Charles-Cavazos, D. J., Mezquita-Hoyos, Y. N., & Sanchez-Monroy, M. (2021). Chronometric constructive cognitive learning evaluation model: Measuring the construction of the human cognition schema of
http://ijlter.org/index.php/ijlter
261
psychology students. International Journal of Learning, Teaching and Educational Research, 20(2), 1-21. https://doi.org/10.26803/ijlter.20.2.1 Sadeghi, K., & Rahmati, T. (2017). Integrating assessment as, for, and of learning in a largescale exam preparation course. Assessing Writing, 34, 50-61. https://doi.org/10.1016/j.asw.2017.09.003 United Nations Educational Scientific and Cultural Organization (2015). Rethinking education: Towards a global common good? https://unesdoc.unesco.org/ark:/48223/pf0000232555 Urdiales-Ibarra, M. E., Lopez-Ramirez, E. O., Castro-Campos, C., Villarreal-Treviño, M. G., & Carrillo-Colon, J. E. (2018). Biology schemata knowledge organization and meaning formation due to learning: A constructive-chronometric approach to concept mapping usability. Creative Education, 9(16), 2992−2706. https://doi.org/10.4236/ce.2018.916203 Wiliam, D. (2011). What is assessment for learning?. Studies in Educational Evaluation, 37(1), 3–14. https://doi.org/doi:10.1016/j.stueduc.2011.03.001
http://ijlter.org/index.php/ijlter
262
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 262-277, August 2021 https://doi.org/10.26803/ijlter.20.8.16 Received May 31, 2021; Revised Aug 25, 2021; Accepted Aug 30, 2021
The Effect of Resource-Based Instructions on Pre-service Biology Teachers' Attitudes towards Learning Biology Josiane Mukagihana African Centre of Excellence for Innovative Teaching and Learning Mathematics and Science (ACEITLMS), University of Rwanda-College of Education (URCE), Rwanda https://orcid.org/0000-0001-7334-331X Catherine M. Aurah Masinde Muliro University, of Science and Technology, Kenya https://orcid.org/0000-0002-0781-3202 Florien Nsanganwimana African Centre of Excellence for Innovative Teaching and Learning Mathematics and Science (ACEITLMS), University of Rwanda-College of Education (URCE), Rwanda https://orcid.org/0000-0002-3152-9893 Abstract. A positive attitude correlates to successful learning, while a negative attitude leads to poor learning. The present study aimed to compare the pre-service biology teachers' attitudes towards learning biology and assessing their attitude change before and after being taught by resource-based instructions (RBI) in Rwanda. A survey research design was used, in combination with a quasi-experimental control group and the equivalent time-series designs. Pre-service biology teachers were taught at a public university in three separate groups, with one control group and two experimental groups. Each group received a pre-attitude and a post-attitude assessment. In contrast, at a private university, they were taught as a single group, in a series of instructions, starting with the lecture method, followed by animation-based instruction and smallgroup laboratory activities, in which at each stage, the pre-service biology teachers received the same assessment. The biology Attitude Scale showed a Cronbach alpha reliability of 0.625 before its use. The data were analyzed quantitatively, and the results revealed no statistically significant difference between the public and the private pre-service teachers' attitudes before learning microbiology. Basically, the results show that pre-service biology teachers at the private University had much more positive attitudes than those in a public university. Contrary to the private university, pre-service teachers’ attitudes towards biology ©Authors This work is licensed under a Creative Common Attribution Non-Commercial No-Derivatives 4.0 International License (CC BY-NC-ND 4.0).
263
changed after instruction in all groups across all the attitudinal statements at the public university. Nevertheless, all methods used during the intervention, namely the lecture methods, animation-based instructions, and small-group laboratory activities, equally improved pre-service biology teachers’ attitudes towards learning biology. The lack of statistically significant difference among methods suggests that the preservice teachers’ attitudes towards learning biology could also be influenced by other factors including their maturity level as well as their anticipated career commitment. Keywords: attitude scale; biology; pre-service biology teachers; resourcebased instructions
1. Introduction Students' attitude towards learning is an essential cognitive behavior that contributes more to their learning outcomes, such as academic achievement, engagement with subject learning, and concept understanding (Prokop, Tuncer, et al., 2007). Hussein (2017) proved that attitude significantly improves students' intention to use instructional tools, like technology-related instructions, such as elearning. Students' attitude determines their behavioral differences towards a given subject or concepts (Haddock, 2017); therefore, understanding students' attitudinal changes after teaching and learning is essential for improving the quality of education. Studies done in Rwanda have shown that secondary-school students feel bored with learning physics; since they think that it does not relate to any real-world experiences (Mbonyiryivuze et al., 2021); moreover, some studies found that students lose interest in learning mathematics due to the instructional tools used by their teachers (Ukobizaba et al., 2020). Although attitude has been defined by many studies differently, all are of the same opinion that attitude is the expression of the inner feelings of an individual that may be perceived as a behavior of liking or disliking a particular thing (Shrigley et al., 1988, Lovelace & Brickman, 2013; Musengimana et al., 2021; Prokop, Prokop, et al., 2007; Prokop, Tuncer, et al., 2007; Tikka et al., 2000). Attitude is an individual mental state that characterizes his or her experiences of consistently responding to environmental phenomena. In its dimensions, attitude is conceptualized as being cognitive, which is the belief in an object as being affective, which reflects the favorability or the availability of an object and the behavioral intentions and the expression of intention towards the object (Kiecolt, 1988). The word attitude may be used interchangeably to refer to one's belief, opinion, or value. Furthermore, an individual's attitude can be perceived in two opposite behaviors: positive and negative, and these can be influenced by different factors (Shrigley et al., 1988). Several factors, such as the instructional methods (Mukagihana et al., 2020, 2021a), gender, students' interest, and classroom environment, were pointed out to affect students' attitudes towards scientific subjects, including chemistry (Musengimana et al., 2021).
http://ijlter.org/index.php/ijlter
264
Various instructional methods have been used to test their influence on students' attitude towards scientific subjects, and among others, co-operative learning methods, computer-assisted instruction, problem-based active learning, mediabased instruction, simulation-assisted laboratory instruction, and instructional technology and material-design course, which positively affected students, including pre-service science teachers' attitude towards science learning (Akinoǧlu & Tandoǧan, 2007; Çepni et al., 2006; McManus et al., 2003; Rabgay, 2018, Calik-Uzun, Selcen, et al., 2019, Selçuk, 2010a; Ulukök & Sari, 2016, Bektaşli, 2013). Many of these instructions are applied in the resource-supported learning environment, and they are referred to as resource-based instructions, such as computer-assisted instructions, simulation-assisted laboratory instruction, instructional technology, and material-design courses. With resource-based instructions, the teaching and learning process uses a single instruction or a variety of instructional resources; and students learn by using these resources (Butler, 2012). Resource-based instructions promote students' active participation by learning in real and tangible learning environments, allow them to interact with instructional resources through hands-on activities, and discovering knowledge and facts themselves. Hence, they enhance students' learning outcomes and their attitude changes towards learning science (Hill & Hannafin, 2001). A good number of studies focused on the students' attitude towards learning science in general (Rabgay, 2018, Calik-Uzun, Selcen, et al., 2019, Selçuk, 2010a; Ulukök & Sari, 2016, Bektaşli, 2013); but a few of these studies emphasized the pre-service biology teachers' attitude towards learning biology. Furthermore, only a few of the current studies focused on the effect of instructional methods on preservice biology teachers' attitude towards learning biology; however, the effect of resource-based instructions, especially animation-based instruction and laboratory methods through small-group laboratory activities on pre-service biology teachers' attitudes towards learning biology, was not highlighted. With such observations made in the current literature, the present study aimed to find out the pre-service biology teachers'attitudes towards learning biology in higher learning institutions in Rwanda. It also aimed to find out the effect of resource-based instructions on pre-service biology teachers' attitudes towards learning biology in higher learning institutions in Rwanda, with three main objectives: (1) to compare the pre-service biology teachers' attitude towards learning biology at private and public science teacher-training universities; (2) to assess the change in pre-service biology teachers' attitudes towards learning biology after learning via animations and small-group laboratory activities, as resource-based instructions; and (3) to test the statistically significant effect in the pre-service biology teachers' attitudes after being taught by animation-based instruction and small-group laboratory activities at private and public universities in Rwanda. The study was guided by the following research questions : 1) Do the pre-service biology teachers' attitudes towards learning biology differ between private and public universities in Rwanda?
http://ijlter.org/index.php/ijlter
265
2) How did the pre-service biology teachers' attitudes change after learning by animation-based instruction and small-group laboratory activities at private and public universities in Rwanda? 3) Is there any statistically significant effect of animation-based instruction and small-group laboratory activities as resource-based instructions on pre-service biology teachers' attitudes towards learning biology at private and public universities? The significance of the study The present study contributes significantly to the current literature by enriching the information related to the status of pre-service biology teachers' attitudes towards learning biology in higher learning institutions. It also emphasizes the role of resource-based instructions in improving pre-service biology teachers' attitudes towards learning biology. Further information would give pre-service biology teacher trainers some insight into the attitude characterizing pre-service biology teachers, thereby enabling them to decide which instructional method to apply when there is a need to improve their attitude towards learning biology. Furthermore, higher learning institutions, which train pre-service biology teachers would be better informed on the attitudes of pre-service biology teachers towards learning biology. The theoretical framework The study is grounded on reasoned action and planned behavior theories, as described by Ajzen Icek (1985). These theories assume that human behavior is sensibly based on the available information that might stimulate their feelings of liking or disliking. The theories postulate that humans react on the basis of their behavioral intention to perform, or not to perform, and then to act. The theories add that individual intention may change in time; and that they are determined by personal factors, which are the positive or negative decisions to perform a particular behavior. These factors define a person's attitude towards behavior that can be predicted from the knowledge of his or her attitude towards the object. These theories are related to this study that aims to evaluate the attitudinal changes of students towards learning biology. These attitudinal changes may be perceived as students' changes in their feeling of like or dislike of learning biology after being treated with different resource-based instructions that, in this case, act as stimuli in the study.
2. The Methodology This study used a survey design, in combination with an experimental design, whereby quasi-experimental and equivalent time series were used to check the pre-service biology teachers' attitude change after being taught by different instructional methods. The Survey design is one of the procedures in quantitative research, in which researchers may measure individuals' different aspects, such as behavior, attitudes, and opinions (Creswell, 2015). The survey was conducted before and after interventions given to groups of pre-service biology teachers. The design was used to permit the researchers to observe the changes in groups of preservice biology teachers after receiving different treatments.
http://ijlter.org/index.php/ijlter
266
2.1 Data-collection instrument and reliability A biology attitude scale developed by Russell and Hollander (1975) was adapted in this study in order to measure the pre-service biology teachers' changes in attitude towards learning biology. The scale was composed of fourteen (14) items or statements that were intended to measure the same behavior at a correlation of 0.80 to each item. The scale was rated from strongly agree (1) to strongly disagree (5); and it contained both negative and positive items. In this study, all items were administered to pre-service biology teachers; however, six (6) negative items were removed during the analysis after recognizing that pre-service biology teachers responded in the right way by positively agreeing to a positive statement and by negatively reacting to negative statements. Consequently, we recognized them as being opposite to positive items. For instance, items 2 and 3 are opposite to item 1 (see Table 1); item 6 is contrary to item 5, which states that "Biology makes me feel secure; while at the same time, it is stimulating." Items 8 and 9 are contrary to item 7: "In general, I have a good feeling towards biology." Item 12 is contrary to items 11 and 12 (see Table 1). Therefore, a total of 6 negative items were removed from the analysis; and only eight positive items were considered (items 1, 4, 5, 7, 10, 11, 13, 14). Before using the scale, its reliability was checked in the Rwandan context; and it was found to be 0.625 Cronbach alpha by the SPSS 23.0 version. This positive correlation confirmed the removal of negative-constructed items; and when considering all 14 items, the reliabilities tended to cancel each other out; and they provided a coefficient of nearly 0 in the negative direction. Table 1: Biology attitude scale (Russell & Hollander, 1975) [Note: 1 = strongly agree; 2 = Agree; 3 = Undecided; 4 = Disagree; and 5 = strongly disagree] Statements 1. Biology is very interesting to me. 2. I don't like biology, and it scares me to have to take it. 3. I am always under a terrible strain in a biology class. 4. Biology is fascinating and fun. 5. Biology makes me feel secure, and at the same time, it is stimulating. 6. Biology makes me feel uncomfortable, restless, irritable, and impatient. 7. In general, I have a good feeling towards biology. 8. When I hear the word biology, I have a feeling of dislike. 9. I approach biology with a feeling of hesitation. 10. I really like biology. 11. I have always enjoyed studying biology in school. 12. It makes me nervous to even think about doing a biology course. 13. I feel at ease in biology; and I like it very much. 14. I feel a definite positive reaction towards biology; it is enjoyable.
http://ijlter.org/index.php/ijlter
1
2
3
4
5
267
2.2 Participants and data-collection procedures This survey study was conducted at two intentionally selected Universities that train pre-service biology teachers in Rwanda; a private university, (University of Technology and Arts of Byumba (UTAB), and the University of Rwanda College of Education (URCE), a solely public university that trains pre-service teachers. At these Universities, the study was conducted after an ethical check-up, and approval from the research unit at the University of Rwanda College of education, and with both Universities' permission. Pre-service biology was introduced to the study; and they signed the consent form voluntarily. The data were collected from February to April 2021. The participants consisted of 50 pre-service biology teachers enrolled in Year (Level) two in the Faculty of Education, Department of Education in Sciences at UTAB. They formed a single group that received a pre-assessment survey; and after the group had been taught a course of biology (microbiology) by traditional methods (Lecture method), by animation-based instruction, and finally by laboratory methods through small-group laboratory activities. The teaching and learning time interval was three weeks, and after each session, they were postsurveyed by administering a Biology-Attitude Scale similar to that used in the preassessment. Thirty-four pre-service biology teachers responded positively to all the surveys, including pre-assessment, and their data were taken for analysis. At URCE, one hundred and eighty pre-service biology teachers enrolled in biology and chemistry education (BCE) Year-2; participated in the study. They were taken from a population of 528 pre-service biology teachers enrolled in the module of microbiology that was purposely selected for teaching by different resource-based instructions. They were randomly assigned to three groups, and each group was pre-assessed by a biology-attitude scale; and thereafter, it was treated at a particular time by the specific instructional method. The first group was taught by the traditional method (lecture method), the second by animation-based instruction, while the third group was treated by laboratory methods through small-group laboratory activities. After those teaching sessions, each group responded to the similar post-Biology Attitude Scale. 2.3 The Data analysis The data were entered into MS Excel 2016 and then filtered in order to track participants' full attendance throughout the intervention. The "COUNT IF" function was used to compute the number of answers, with either strongly agree, agree, undecided, disagree, or strongly disagree with each statement (for more details on how the data analysis was done, see our data deposited in Mendeley repository: "https://data.mendeley.com/datasets/bpfb5sdmyg/1"). The formula matches the answer provided by the respondent for each statement. Then, it counted all those who replied "1" as strongly agree, "2" as agree, "3" undecided, "4" Disagree, and "5" strongly disagree to each statement. After that, the software summed up the total agreements and all the disagreements to each statement. For the data collected from UR-CE, the analysis was done to both the pre-assessment and the post-assessment data of each of the three groups. The generated pre-
http://ijlter.org/index.php/ijlter
268
assessment results of the groups (control group, animation group, and lab group) and their post-assessments results were presented on a plotted cluster-column graph, and they were then compared. The same analysis was done to the data from UTAB, and the pre-assessment results, the results of the pre-service biology teachers after being taught by the lecture method, animation-based instruction, and laboratory methods through small-group laboratory activities, were compared by presenting them on a plotted cluster-column graph. To compare the private and the public pre-service biology teachers' attitudes towards learning biology and with the interventions given, we computed the averages and formed three scales by combining scale-1 and scale-2, as well as scale-3 alone, and then scale-4 and scale-5 together. We then considered the agreement rates among the pre-service teachers and the agreement rates across the interventions. We used the Chi-square test to compute the significance between the pre-service teachers and the intervention groups. This test was appropriate, as the actual count of the individual respondents was in place, instead of their scores.
3. The Results Figure 1 compares the pre-service biology teachers' attitudes towards learning biology, before implementing the resource-based instructions to pre-service teachers at public and private universities. The quick insight is that the pre-service teachers hold positive attitudes towards biology. However, those at a private university (UTAB) hold considerably more attitude than those in a public university (URCE) (see Figure 1, blue and yellow colour of the cluster-column graph). Except on statement-7 and statement-14, public universities hold more attitudes than those from private universities. In general, pre-service teachers at URCE (81%) have a better feeling towards biology than do those at UTAB (79%) [see Statement-7]. Likewise, they (81% of URCE pre-service biology teachers) "feel a definite positive reaction to biology; since it is enjoyable" in contrast to those at UTAB (74% of UTAB pre-service teachers) [see Statement-10].
http://ijlter.org/index.php/ijlter
269
97
100 90
88
85
% of respondents
80 70 60
61
68
94
81 79
80
94 84
82 85
81 74
63
50 40 30 20 10 0 STAT 1 STAT 4 STAT 5 STAT 7 STAT 10 STAT 11 STAT 13 STAT 14 UR-CE (Agreement)
UR-CE (Neutral)
UR-CE (Disagreement)
UTAB (Agreement)
UTAB (Neutral)
UTAB (Disagreement)
Figure 1: Attitude of Pre-service Biology teachers at a public and private university towards learning Microbiology. [Note: Cluster columns show the distribution of preservice biology teachers into 3 groups: an agreement, neutral, and disagreement scales. The vertical axis displays their percentage rates, while the horizontal axis displays their attitude statements.]
Biology is very interesting (97%, see Statement-1), fascinating and fun (68%, see Statement-4), and this makes the UTAB pre-service teachers feel secure, while at the same time stimulated (88%, see Statement-5) more than URCE pre-service biology teachers. Similarly, UTAB pre-service teachers "really like biology" (94%, see Statement-10), and they enjoy studying biology at school (94%, see Statement11), and they feel at ease in biology and like it very much (85%, see Statement-13) more than do the URCE pre-service teachers. Despite these variations in attitude, however, the Chi-square test shows no statistically significant difference between URCE and UTAB pre-service teachers, as regards their attitudinal statements (X2 = 48, df = 47 (1), p=0.243). Figure 2 shows the attitude results from URCE pre-service teachers before and after learning biology. It is worth mentioning that we only considered the positive statements in our analysis. Thus, eight out of 14 attitude statements are under consideration (see the detailed explanation in the methodology section). The URCE pre-service teachers' attitudes towards biology were computed in the three groups of intervention of resource-based instruction (RBI) after being taught in those modes. Figure 2 displays two sides. The side on the left is before the assessment, while the one on the right side is after the intervention. The first group is the control group or the lecture group, the second is the animation group, and the third group is the lab-based group.
http://ijlter.org/index.php/ijlter
270
The figure is a cluster-staked graph. Thus, it accumulates both agreement, neutral, and disagreement scales at the 100% rate. For instance, 59 pre-service teachers participated via the control group in the pre-assessment stage; 40 were in agreement, with one being neutral; while 18 fell into the disagreement category. Thus, 68% agreed, 2% were neutral, and 30% disagreed with the first statement [Biology is very interesting to me], and these taken together equate to 100% (see Figure 2).
CONTROL GROUP
ANIMATION GROUP
LAB GROUP
Pre assessment
Post assessment
STAT 13 STAT 10 STAT 5 STAT 1
STAT 13 STAT 10 STAT 5 STAT 1
STAT13 STAT10 STAT5 STAT 1 0%
20%
40%
60%
80%
Pre (Agreement)
Pre (Neutral)
Pre (Disagreement)
Post (Agreement)
Post (Neutral)
Post (Disagreement)
100%
Figure 2: The Attitude of Pre-service Biology teachers at a public university before and after learning Microbiology. [Note: N: 59 for the control group, N = 60 for the animation group, and N = 52 for the lab group]
The attitude change did not significantly differ between the teaching interventions (see Table 2). However, the pre-service teachers' attitudes towards biology changed after instruction in all the groups across all the attitude statements, irrespective of the use of the animation, laboratory, and lecture instructions. For instance, statement-4 [Biology is fascinating and fun] shifted from 56%, who had a positive attitude and reached 83% after being taught in the traditional mode. On the same statement, 67% of the pre-assessment teachers had a positive attitude, or they agreed with the statement; and this increased to 73% after learning through animation-based instructions. Similarly, the lab-based activities increased the positive attitudes towards biology from 60% to 79% of pre-service biology teachers (see Figure 2).
http://ijlter.org/index.php/ijlter
271
Table 2: Statistical significance between intervention groups at URCE. Pre-test Control group
Post-test X2 df p-value 1 Control group 34 30 0.281 Animation group 42 36 0.227 Laboratory group 28 24 0.260 2 Animation group Control group 34 30 0.281 Animation group 42 36 0.227 Laboratory group 28 24 0.260 3 Laboratory group Control group 35 25 0.094 Animation group 32 30 0.368 Laboratory group 27 20 0.145 Note: X2 is the Chi-square value, df is the degree of freedom, a p-value is computed, based on a 0.05 level of significance or on a 95% confidence interval .
Lecture (PostPre-assessment assessement)
Animation (Postassessement)
Lab (Postassessement)
Figure 3 shows the results of the UTAB pre-service teachers. In this case, a timeseries design was used. Thus, attitudinal change was measured quince across four stages of one group of pre-service teachers. The attitude does not seem to change from pre-assessment to another stage of intervention, such as lecture, animation, or lab-based instruction. Statement-4 and statement-5 seem to be persistent across all the interventions. The understanding of the statements might be the reason for this. STAT 14 STAT 11 STAT 7 STAT 4 STAT 13 STAT 10 STAT 5 STAT 1 STAT 14 STAT 11 STAT 7 STAT 4 STAT 13 STAT 10 STAT 5 STAT 1 0%
20%
Agreement
40% Neutral
60%
80%
100%
Disagreement
Figure 3: The Attitude of Pre-service Biology teachers at a private university before and after learning Microbiology. [Note: N = 34 both at re-assessment and continuous intervention]
http://ijlter.org/index.php/ijlter
272
Table 3 shows how the interventions made no statistically significant differences in the attitude between the pre-assessment and the pre-service biology teachers at UTAB. Neither did their attitude change from pre-assessment to lecture, animation, laboratory instruction, lecture instruction to animation, or to laboratory instruction. It changed, however, from animation to laboratory instruction at a p<0.05 (see Table 3). Table 3: Statistically significant different effects between the series of interventions at UTAB. Prior series Pre-assessment
Next series X2 df 1 Lecture instruction 26 24 Animation 27 24 instruction Laboratory 35 30 instruction 2 Lecture instruction Animation 17 16 instruction Laboratory 22 20 instruction 3 Animation instruction Laboratory 32 20 instruction Note: X2 is the Chi-square value, df is the degree of freedom, the p-value is based on a 0.05 level of significance, or on a 97% confidence interval.
p-value 0.353 0.320 0.255 0.393 0.329 0.043 computed,
4. Discussion With research question one, the study sought to identify the pre-service biology teachers' attitudes towards learning biology at a private University (UTAB) and at a public University (UR-CE) before any interventions. Although they both presented a positive attitude, and that a statistically non-significant difference in attitude towards the statements was revealed, pre-service biology teachers at the private University (UTAB) show considerably more attitude than do those in a public university (URCE). The difference in pre-service biology teachers' attitudes at private and public universities may be explained by many facts. From these results, one can consider how private and public university pre-service biology teachers are enrolled in biology subjects. Pre-service biology teachers select biology subjects within their wide choice at the private University; whereas at a public teacher-training university, students are enrolled in biology programs, based on the combination they learned at secondary schools. Those at a Public University may be enrolled in Mathematics, Biology Education, or Biology Chemistry education; while on the basis of choice, they could decide to learn subjects other than biology, such as Mathematics and Chemistry.If this happens, the pre-service biology teachers would follow the program with much effort in one subject that is interesting to him or her, but with a low attitude or negative attitude towards biology, where the subject was just presented without any individual choice. Related studies that compared students' attitudes at private and public schools or universities are still very rare; however, looking into other domains, such as the motivation towards learning biology, the study of Bernardo et al. (2015) advocated a difference in
http://ijlter.org/index.php/ijlter
273
motivational level towards learning science between public and private-school students. Related studies that compared students' attitudes at private and public schools or universities are still very rare. Apart from the statistical results, in reality, the pre-service biology teachers at the private university presented a higher positive attitude towards learning biology than did their colleagues at a public university. This implies that they might also perform differently in biology. The recommendation to the Government is to ensure that all students are oriented towards Biology education at the public University (UR-CE); and that they hold a positive attitude towards learning biology. Additionally, the orientation on the basis of the student's subject choice should be more emphasized. Although the pre-service, biology teachers at public universities presented a low level of attitude; they showed a good feeling and a more positive reaction towards biology than did those in private universities. Furthermore, they perceived biology as an enjoyable science subject. This may be explained by the fact that they learned in different physical and classroom environments, where public preservice biology teachers learn in a more diverse environment, with different biological stimuli than those in a private university. A long time ago, it was noted that the classroom environment presented more relation with the student's attitude towards a science that includes biology (Talton & Simpson, 1987). The findings relate with those of other studies, such as those of Nordqvist and Johansson (2020), Prokop, Prokop, et al. (2007), Prokop, Tuncer, et al. (2007), and Sanders and Morrison-Shetlar (2001). They noted that, in general, students have a positive attitude towards biology. Indeed, the present study's findings did not show any change of attitude, despite the instructions used. The theory of reasoned action and planned behaviour (Ajzen, 1985), which directed this study confirms the obtained results; as it concludes the final students' agreement or disagreement in terms of having a positive or negative attitude towards learning biology as a subject. We identified how the pre-service biology teachers' attitude changes after being taught by different instructional methods, at both private (UTAB) and public (URCE) universities. The data form the public university did not show any data that revealed any statistically significant difference in attitudinal change between preservice biology teachers – taught by the traditional (lecture) method, animation– based instruction, and taught by laboratory methods via small-group laboratory activities. However, instructions improved the pre-service biology teachers' attitudes differently across all the attitudinal statements. The fact that both the instructional methods improved the pre-service biology teachers' attitude to the positively considered statements may be explained by the exciting qualities of microbiological content , used during the intervention. This study agrees with our previously published results, where we found that all the resource-based interventions improved pre-service biology teachers' academic achievement in Rwanda (Mukagihana et al., 2021b). Consequently, attitude may correlate with the performance of the students. The findings deviate from what
http://ijlter.org/index.php/ijlter
274
other studies identified, where traditional methods of teaching (lecture) did not improve the students' attitude towards learning biology, such as did active instructions, like computer-aided instruction, co-operative learning methods (Rabgay, 2018; Yesilyurt et al., 2019). Although the three groups were taught by using different instructional methods at the public University (UR-CE), pre-service biology teachers learned the same content. Consequently, in each instruction, new knowledge was gained; and this added to their pre-existing attitude, thus changing the mind or beliefs of one who responded negatively to the statement before teaching, to agree with the statements positively. This makes pre-service biology teachers present a positive attitude on some statements after receiving interventions. The implication is that both lecture methods and animation-based instructions could improve preservice biology teachers' attitudes towards learning biology. However, animationbased instructions and small-group laboratory activities may do more; since they are resource-based instructions that create an active learning environment that attracts students in the teaching and learning process. The findings of the study are in line with those of other studies (Akinbadewa & Sofowora, 2020; Akinoǧlu & Tandoǧan, 2007; Koksal & Berberoglu, 2014; Sen & Oskay, 2016, (Akinbadewa & Sofowora, 2020; Akinoǧlu & Tandoǧan, 2007; Çepni et al., 2006; Koksal & Berberoglu, 2014; Rabgay, 2018). These studies tested the effect of instructional methods, like technology-based instructions and laboratory methods on students' attitude towards biology, whereby such instructional methods improved students' attitudes towards science, including biology. In particular, our study showed that even the lecture method could improve the pre-service biology teachers' attitudes. This is because pre-service teachers are mature enough; and they have to study the subject; since they will probably be teaching it after their program. Thus, the fact that they plan to teach that subject in their future career, the attitude to learning improves, despite the instructional method used to deliver it. The implication is that after learning, pre-service biology teachers knew their attitude towards biology – whether it is very interesting to them, or not. It was noted that students, when learning by the lecture method, develop their listening capacity; and they can master and maintain concepts (McManus et al., 2003); thus, it is easy for them to decide on their opinion. Conversely, the findings disagree with those of Koksal and Berberoglu (2014), who noticed that there were no improvements in students' attitude after learning via the traditional methods. Therefore animation-based instruction and smallgroup laboratory activities are recommended for improving pre-service biology teachers' attitudes towards learning biology. At the private University (UTAB), the attitude change was measured in a single group of pre-service biology teachers, who received a pre-assessment test by responding to the biology-attitude scale, and a post-assessment test, after receiving treatment through a series of instructions. Looking into the results, we
http://ijlter.org/index.php/ijlter
275
found that pre-service biology teachers maintain their positive attitude – both before and after a series of interventions. There were no significant differences in attitude change after a series of instructions, except animation to laboratory instruction. However, each series of instructions changed their agreement on statements. Small-group laboratory activities statistically improved pre-service biology teachers' attitude towards learning biology, thereby showing that those preservice biology teachers are eager to learn, understand the biological concepts, and become familiar with laboratory experiments, which they are expected to reflect during their teaching in their future career. Indeed, the very recent study by Uwamahoro et al. (2021) conducted in Rwanda has shown that laboratory activities improve students' understanding of concepts through appropriate hands-on activities. Therefore, laboratory methods should be emphasized at private universities to develop pre-service biology teachers' scientific skills and improve their attitude towards learning biology, thereby improving their performance.
5. Conclusion, limitations, and further research The present study aimed to identify how the pre-service biology teachers' attitude changed due to animation-based instruction and small group laboratory activities as resource-based instructions at private and public universities in Rwanda. The study compared their attitude towards learning biology before teaching sessions; and it revealed that both groups of pre-service biology teachers present a positive attitude towards learning biology. However, based on the differences in positive agreement towards statements and in their feelings towards biology, pre-service biology teachers at the private University (UTAB) were found to have much more positive attitudes than those in a public university (URCE). At a public university, pre-service teachers' attitudes towards biology changed after instruction in all groups across all the attitudinal statements. However, in private, the pre-service biology teachers' attitudes towards learning biology did neither change from pre-assessment, nor after all the series of interventions. There was no statistically significant difference in attitudinal change between the treatment groups at both public and private universities. The attitudinal change did not show any statistically significant differences between teaching interventions at either public or private universities. After teaching the interventions, the results revealed that both lecture methods, animation-based instructions, and laboratory methods through small-group laboratory activities equally improved pre-service biology teachers' attitudes towards learning biology. Nevertheless, via time-series design, laboratory activities showed a great attitudinal change. This unique finding reveals that preservice biology teachers are mature enough to study through any instructional method. They develop a positive attitude, since they would also teach the same subject in their future career.
http://ijlter.org/index.php/ijlter
276
Given that the present study used different designs during the interventions and a non-equivalent quasi-experimental design and equivalent time series at a public university (UR-CE) and at a private university (UTAB). we did not compare the pre-service biology teachers' attitudinal changes after the interventions at both universities. Therefore, further research should focus on the use of single and similar designs at both Universities. Furthermore, the study did not consider the gender difference in pre-service biology teachers' attitudes before and after the interventions. This information may give insight to trainers about the status of gender difference; consequently, they should be able to know which gender needs more emphasis to improve their attitude towards biology. Future research is required to highlight pre-service biology teachers' gender differences in attitude towards biology and improve their attitude towards learning and teaching biology. This is, therefore, recommended. Acknowledgments We would like to thank the Directorate of Research and Innovation at the University of Rwanda College of education (URCE) and the University of Technology and Arts of Byumba (UTAB) for granting ethical clearance and permission to conduct this study. Also, we sincerely thank the African Center of Excellence for Innovative Teaching and Learning Mathematics and Science (ACEITLMS) for the financial support provided to successfully conduct this study.
6. References Akinbadewa, B. O., & Sofowora, O. A. (2020). The effectiveness of multimedia instructional learning packages in enhancing secondary school students' attitudes towards biology. International Journal on Studies in Education, 2(2), 119–133. https://doi.org/10.46328/ijonse.19 Akinoǧlu, O., & Tandoǧan, R. Ö. (2007). The effects of problem-based active learning in science education on students' academic achievement, attitude and concept learning. Eurasia Journal of Mathematics, Science and Technology Education, 3(1), 71– 81. https://doi.org/10.12973/ejmste/75375 Bernardo, A. B. I., Ganotice, F. A., & King, R. B. (2015). Motivation gap and achievement gap between public and private high schools in the Philippines. Asia-Pacific Education Researcher, 24(4), 657–667. https://doi.org/10.1007/s40299-014-0213-2 Çepni, S., Taş, E., & Köse, S. (2006). The effects of computer-assisted material on students' cognitive levels, misconceptions and attitudes towards science. Computers and Education, 46(2), 192–205. https://doi.org/10.1016/j.compedu.2004.07.008 Hussein, Z. (2017). Leading to Intention: The role of attitude in relation to technology acceptance model in e-Learning. Procedia Computer Science, 105(December 2016), 159–164. https://doi.org/10.1016/j.procs.2017.01.196 Koksal, E. A., & Berberoglu, G. (2014). The effect of guided-inquiry instruction on 6th Grade Turkish students' achievement, science process skills, and attitudes toward science. International Journal of Science Education, 36(1), 66–78. https://doi.org/10.1080/09500693.2012.721942 Mbonyiryivuze, A., Yadav, L. L., & Amadalo, M. M. (2021). Students' attitudes towards physics in Nine Years Basic Education in Rwanda. International Journal of Evaluation and Research in Education (IJERE), 10(2), 648–659. https://doi.org/10.11591/ijere.v10i2.21173 McManus, D. O., Dunn, R., & Denig, S. J. (2003). Effects of traditional lecture versus
http://ijlter.org/index.php/ijlter
277
teacher-constructed & student constructed self-teaching instructional resources on short-term science achievement & attitudes. The American Biology Teacher, 65(2), 93–102. https://doi.org/10.2307/4451447 Nordqvist, O., & Johansson, S. (2020). Secondary school biology students' attitudes towards modern biotechnology characterised using structural equation modeling. Eurasia Journal of Mathematics, Science and Technology Education, 16(2). Prokop, P., Prokop, M., & Tunnicliffe, S. D. (2007). Is biology boring? Student attitudes toward biology. Journal of Biological Education, 42(1), 36–39. https://doi.org/10.1080/00219266.2007.9656105 Prokop, P., Tuncer, G., & Chudá, J. (2007). Slovakian students' attitudes toward biology. Eurasia Journal of Mathematics, Science and Technology Education, 3(4), 287–295. https://doi.org/10.12973/ejmste/75409 Rabgay, T. (2018). The effect of using co-operative learning method on tenth grade students' learning achievement and attitude towards biology. International Journal of Instruction, 11(2), 265–280. https://doi.org/10.12973/iji.2018.11218a Russell, J., & Hollander, S. (1975). A biology attitude scale. American Biology Teacher, 37(5), 270–273. https://doi.org/10.2307/4445229 Sanders, D. W., & Morrison-Shetlar, A. I. (2001). Student attitudes toward web-enhanced instruction in an introductory biology course. Journal of Research on Computing in Education, 33(3), 251–262. https://doi.org/10.1080/08886504.2001.10782313 Sen, S., & Oskay, O. O. (2016). The effects of 5E inquiry learning activities on achievement and attitude toward chemistry. Journal of Education and Learning, 6(1), 1. https://doi.org/10.5539/jel.v6n1p1 Talton, E. L., & Simpson, R. D. (1987). Relationships of attitude towards classroom environment with attitude towards and achievement in science among tenth grade biology students. Journal of Research in Science Teaching, 24(6), 507–525. https://doi.org/10.1002/tea.3660240602 Ukobizaba, F., Ndihokubwayo, K., & Uworwabayeho, A. (2020). Teachers' behaviours towards vital interactions that attract students' interest to learn mathematics and career development. African Journal of Educational Studies in Mathematics and Sciences, 16(1), 85–94. Uwamahoro, J., Ndihokubwayo, K., Ralph, M., & Ndayambaje, I. (2021). Physics students' conceptual understanding of geometric optics: Revisited analysis. Journal of Science Education and Technology, 30(0123456789), 1–13. https://doi.org/10.1007/s10956-021-09913-4 Yesilyurt, M., Dogan, M., Acar, M., Doğan, M., & Acar, S. M. (2019). The meta-analysis of the effect of computer-aided instruction on students' attitudes in science and mathematics. Acar, S Journal of Primary Education, 1(2), 57–69. https://dergipark.org.tr/en/pub/temelegitim/issue/44423/542243
http://ijlter.org/index.php/ijlter
278
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 278-299, August 2021 https://doi.org/10.26803/ijlter.20.8.17 Received Jul 25, 2021; Revised Aug 29, 2021; Accepted Aug 30, 2021
The Effectiveness of a Self-Questioning Strategy at Developing Academic Achievement and Critical-Thinking Skills among Secondary-School Students in Saudi Arabia Ali Tared Aldossari* Imam Abdulrahman Bin Faisal University, College of Education, Curriculum & Instruction Department, Dammam, Saudi Arabia https://orcid.org/0000-0001-6329-2387 Mohammed Msnhat Aldajani Imam Abdulrahman Bin Faisal University, College of Education, Curriculum & Instruction Department, Dammam, Saudi Arabia https://orcid.org/0000-0002-8531-437X
Abstract. This study aims to examine how effective a self-questioning strategy is at developing academic achievement and critical-thinking skills in the jurisprudence curriculum (Fiqh), followed by secondaryschool students. The research sample is taken from secondary-school students in Dammam, the Kingdom of Saudi Arabia (KSA); 64 students from two different schools were selected: 32 as a control and 32 as an experimental group. The research instruments consist of an achievement test, a test to see how critical-thinking skills are applied to the Fiqh curriculum, and a teachers’ guide to teaching the Fiqh curriculum, using the self-questioning strategy. The study recommends the use of the selfquestioning strategy when teaching the Fiqh curriculum to secondaryschool students. It also highlights the need to raise teachers’ awareness of the importance of teaching critical-thinking skills. Considering the research results and recommendations, the researchers recommend several further avenues of research related to the current research variables. Keywords: academic achievement; critical-thinking skills; curriculum; self-questioning strategy; secondary-school students
Fiqh
1. Introduction Islamic studies are a central subject at all academic levels in the Kingdom of Saudi Arabia (KSA); the curriculum guidelines state that “religious sciences are the basis *
Corresponding Author: Ali Tared Aldossari; Email: ataldossari@iau.edu.sa
©Authors This work is licensed under a Creative Common Attribution Non-Commercial No-Derivatives 4.0 International License (CC BY-NC-ND 4.0).
279
of all levels of study – primary, intermediate and secondary – and Islamic culture is the basis of all years of higher education” (Education Document, 1970). Modern methods of teaching are applied to such Islamic subjects. Abanmi (2016) states that traditional teaching strategies (such as lectures and indoctrination) are no longer accepted because they limit learner activity, passively impacting students’ cognitive achievement, limiting their skills gains, and weakening their attitudes. As such, it is now advised that teaching strategies should be reconsidered. AlMofda (2005) stresses this needs, while Abu Ajwa (2009) highlights the importance of interaction between teachers and their students, especially when teaching Islamic studies, which requires an interest in and inclination towards the subject. Abdel-Fattah (2010) observes that a self-questioning strategy helps students to develop a self-awareness of the learning process; this helps them examine their own understanding, causing them to become aware of what they are learning and the way in which they learn. Arian's study (2003) supports the idea that self-questioning is a strategy that benefits secondary-school students more than students at any other stage, because it helps them to transfer the impact of learning and develops organisational skills of the learning process. The self-questioning strategy is a modern educational strategy that synergises well with critical thinking, as it requires students to analyse and evaluate information in order to solve problems and make decisions, rather than simply requiring them to narrate the issue at hand. The development of critical-thinking skills is one of the most important contemporary educational goals; scholars in the field of educational studies believe that one of the primary aims of education is to equip every student with the skills to reach the right decision at the right time. In this regard, Magdy (2003) emphasises the contribution that training students in critical-thinking skills makes to the development of individuals who are capable of taking responsibility and making decisions, as well as expanding their mental processes.
2. Research Problem Despite the importance of academic achievement in Islamic studies and their impact on learners’ lives, many studies (Al-Balawi, 2006; Al-Matroudi, 2009; AlMousa, 2001; Al-Qarni, 2013) have indicated that there has been a decline in students’ achievements, due to the teaching methods used, as these neglect the learner and disregard the various approaches to thinking, especially critical thinking. As such, it has been noted, students’ scientific and practical skills must be developed (Wahed, 2018). Al-Surour (2005) notes that there is a strong relationship between critical-thinking skills and self-questioning, as critical-thinking skills are based on questioning (i.e., analytical and evaluative questions), leading to knowledge and understanding. Therefore, Abu Jadu and Nofal (2007) stress the idea that developing the habit of questioning is a basic requirement when developing critical-thinking skills. The current study will examine this issue by asking several pertinent research questions.
http://ijlter.org/index.php/ijlter
280
2.1. Research questions This study seeks to answer the following main question: How effective is the self-questioning strategy at developing academic achievement and critical-thinking skills studying the Fiqh curriculum for secondary-school students? The following sub-questions arise from this main question: 1. How effective is the self-questioning strategy at developing academic achievement studying the Fiqh curriculum for secondary-school students? 2. How effective is the self-questioning strategy at developing criticalthinking skills studying the Fiqh curriculum for secondary-school students? 3. What is the nature of the correlation between critical-thinking skills and academic achievement studying the Fiqh curriculum for secondary-school students? 2.2. Research hypotheses This study seeks to verify the following hypotheses: 1. There is no statistically significant difference at the level (α ≥ 0.05) between the average test scores of the experimental group (which was studying using the self-questioning strategy) and the control group (which was studying using the traditional method). 2. There is no statistically significant difference at the level (α ≥ 0.05) between the average test scores when testing the critical-thinking skills of the experimental group and the control group. 3. There is no statistically significant correlation at the level (α<0.05) between critical-thinking skills and academic achievement. 2.3. Research aims This study aims to: 1. Determine the effectiveness of the self-questioning strategy at developing academic achievement studying the Fiqh curriculum for secondary-school students. 2. Determine the effectiveness of the self-questioning strategy at developing critical-thinking skills studying the Fiqh curriculum for secondary-school students. 3. Determine the nature of the correlation between critical-thinking skills and academic achievement studying the Fiqh curriculum for secondary-school students. 2.4. Research importance 2.4.1 Theoretical importance This is the first study to investigate the effectiveness of the self-questioning strategy studying the Fiqh curriculum in the KSA. It evaluates the use of modern teaching trends, which emphasise the students’ role in the educational process and focus on developing students’ critical thinking.
http://ijlter.org/index.php/ijlter
281
2.4.2. Practical importance This study aims to develop the critical-thinking faculty of students through a selfquestioning strategy. It directs teachers to apply the self-questioning strategy in the teaching of Islamic studies and introduces a guide to the self-questioning strategy that can be applied when teaching the Fiqh curriculum.
3. Terminology used in this study The following terms are used in this study: • Self-questioning strategy: In this strategy, the student asks themselves several questions before, during and after a lesson. These questions may help develop academic achievement and critical-thinking skills. • The Fiqh curriculum: A KSA curriculum that tackles the purposes of Sharia and offers a set of legal rulings and limits (Ministry of Education, 2018). • Academic achievement: Grades that secondary-school students receive following the achievement test prepared for this purpose. • Critical thinking: A pattern of thinking that gives a learner the ability to make correct judgments (according to certain criteria) related to the issues, situations and problems they face. • Critical-thinking skills: The ability to evaluate the validity and accuracy of information considering the evidence and data available to the learner and to reach correct logical conclusions.
4. Theoretical literature and related studies Over the past century, educators have focused their research efforts on different teaching methods, examining their ability to achieve desirable educational outcomes for learners at different educational stages. On this topic, it has been said that “a successful teacher is nothing but a successful method, and those in charge of teacher training have trained their students to use different teaching methods that help them achieve the goals of teaching easily and successfully” (Bashir, 2017). 4.1. The self-questioning strategy The self-questioning strategy is an important strategy used to develop awareness. As noted, when following this strategy, a student will ask themselves several questions before, during and after a lesson. These questions will facilitate understanding and encourage the student to think about important elements, such as their main ideas, the title of their work, and their previous experience (AlAtoum et al., 2008). Nassar (2015) also observes that this strategy is one that combines cognitive and metacognitive properties when used in different ways. This is because it helps the student to focus on the details, to summarise their knowledge, and to recognise
http://ijlter.org/index.php/ijlter
282
the important elements of their study, as well as increasing an awareness of the knowledge they have gained and the processes that have been performed on it. Self-questioning is known under several different names, including self-planning, self-assessment, and self-reflection (Hamid, 2000), as well as the strategy of selfinterrogation (Attia, 2009). 4.1.1. The importance of the self-questioning strategy The self-questioning strategy is based on the student's positivity as applied to the educational process. This is because the questions that students ask build emotional and cognitive motives; thus, they become more responsible for their own learning. The importance of the self-questioning strategy can be summarised as follows: 1. It helps students to formulate any questions they have about the topic they are studying, allowing them to engage in a dialogue concerning what they know and what they would like to know. 2. It increases students’ understanding of the topic at hand, allowing more of their energy to be directed towards teamwork; thus, it makes them more efficient students. 3. Students are made to rely on themselves when constructing meaning; as such, its impact is retained for a long time. 4. Students' questions reveal their thinking style, alternative concepts, their perceptual understanding, and what they want to know. 5. Students become more sensitive to important parts of the lesson content. They evaluate their understanding of the educational material (in the sense that they become aware of what they previously did not understand) and they take remedial action by asking themselves and/or their peers questions or by engaging in additional reading (Ramadan, 2005). 4.1.2. Self-questioning strategies: steps According to Attia (2009), the self-questioning strategy contains the following steps: The teacher introduces the topic to the students, provided that it is a topic that arouses the students’ interest. 1. The students are encouraged to ask themselves questions in order to activate their metacognitive processes before they start the learning process. 2. Students read around the subject carefully, asking themselves questions that aid the planning and organisation of the learning process. 3. Students evaluate the results and take notes on their observations and performance. 4. The teacher investigates and examines the students’ responses to selfinterrogation and shows the students how to benefit from them. This is the final stage of the self-questioning process. 4.1.3. The difference between the self-questioning strategy and the “know, want, learn” strategy Due to the similarity of the steps involved, El-Ghahori (2012) believes that the selfquestioning strategy is the same as the “know, want, learn” (KWL) strategy. However, some scholars – such as Bahloul (2004) – differentiate between the two strategies, but without showing the differences. Having reviewed several studies
http://ijlter.org/index.php/ijlter
283
on this topic, the authors of this current paper have determined what these differences are: namely, that metacognitive strategies have three main dimensions: planning, organising and supervising. We have also found that, while the control aspect is more visible in the selfquestioning strategy, the organising aspect is more visible in the KWL strategy (the two strategies share the planning aspect). As such, we believe that questions asked as part of the self-questioning strategy are more profound, due to the development of the learners’ questions. 4.2. Critical thinking Critical thinking is one of the most important topics studied here, because it empowers learners with the basic skills needed as part of the learning and teaching process. 4.2.1. The definition of critical thinking Critical thinking is a careful judgment as to whether we should accept, reject or postpone a decision regarding a particular request or issue. It also assumes that we have a degree of confidence in what we accept or reject (Saadeh, 2011). Udall and Daniels (1991) define critical thinking as the ability to verify and evaluate something based on previously agreed criteria. Critical thinking is also the ability to evaluate information and examine opinions, taking into account the perspectives on the topic under study (Abu Jad & Nofal, 2007). Nabhan (2001) states that critical thinking is the apparent behaviour of an individual in a particular situation and regarding a specific subject about which a judgment or practical behaviour is required, considering the data and facts available to the individual. Finally, Qatami (2004) states that critical thinking is reasonable reflective thinking that focuses on what an individual believes or does. In summary, critical thinking can be defined as thinking patterns that give a learner the ability to make correct judgments (according to certain criteria) regarding the issues, situations and problems they face. 4.2.2. Critical thinking and education Education benefits from critical thinking, being an evaluative and judgmental process that moves through a logical thought process to solve problems. Some researchers believe that critical thinking corresponds to the higher levels of Bloom's classification; this approach has been adopted by the KSA Ministry of Education (Faqihi, 2016). 4.2.3. Critical-thinking skills There are many elements to and classifications of critical-thinking skills. This is due to the various different opinions on and definitions of the topic, the most famous of which is Watson and Glaser’s classification, which divides criticalthinking skills as follows (Al-Qahtani, 2010): 1. The skill of inference: This is the ability to distinguish between the degrees of probability concerning how right or wrong a result is, depending on the degree of its connection to certain given facts.
http://ijlter.org/index.php/ijlter
284
2. The skill of forecasting assumptions: This is the ability to examine events or facts; it is judged considering the available data or evidence. 3. The skill of deduction: This is the learner’s ability to derive the relationships between given facts, such that they can judge the extent to which a result derived from said facts is truly connected or not, regardless of the validity of the facts given or the position of the learner. 4. The skill of interpretation: This is the ability to justify or draw certain conclusions considering facts or witnessed incidents that have been accepted by the human mind. 5. The skill of evaluating discussions: This is learners’ ability to distinguish between strengths and weaknesses when judging an issue or a fact considering the available evidence. 4.2.4. The relationship between self-questioning and critical thinking Abu Jad and Nofal (2007) stress that developing a questioning ability is one of the basic requirements needed for the development of critical thinking. As Al-Surour (2005) observes, there is a strong and stable relationship between critical-thinking skills and self-questioning; this is because critical-thinking skills are based on questioning, which leads to knowledge and understanding through analytical and evaluation questions. Therefore, we should not hesitate to ask the questions that lead to knowledge of what we must do in matters of religion, worship and dealings with people. At the same time, when asking questions, we should maintain the necessary legal limits and morals. A student should be aware that knowledge is derived from questions and answers; that a good question is half the way to possessing knowledge and that the ability to ask the right question is one of the most important thinking skills. 4.3. Academic achievement Academic achievement is a basic criterion affecting most decisions related to the student, the curriculum and the educational process. Students’ progress in the educational process and their achievement of goals are recognised through academic achievement. With its various knowledge aspects, academic achievement is one of the factors shaping students’ personality; this is why educators are keen for students to obtain the maximum possible amount of knowledge through the learning process. 4.3.1. The concept of academic achievement For the purpose of this study, we define “academic achievement” as marks obtained by secondary-school students in the achievement test prepared for this research.
http://ijlter.org/index.php/ijlter
285
4.3.2. The importance of academic achievement Academic achievement is one of the most important goals of education, as it allows an awareness of the level at which students are performing, of teachers’ performance, and the appropriateness of the means and curricula used to achieve society’s educational goals. Academic achievement helps students to become aware of their abilities, as well as informing parents of the level of their child’s performance. It is one of the criteria that institutions rely on in order to move students from one educational stage or level to another. Finally, high academic achievement (in comparison to low achievement) inspires confidence in students (Adam, 2015). 4.3.3. Factors affecting academic achievement There are many factors that affect academic achievement, including the mental abilities of the student, their motivation to learn, the educational, economic and social level of the family, and teachers’ competence and sincerity. 4.4. The Fiqh curriculum and secondary-school students’ critical thinking Critical thinking may help to enlighten students about legal rulings, offering them the ability to elicit, distinguish and direct their questions and to anchor them to reality. As such, we believe that the Fiqh curriculum has a major role to play in developing critical-thinking skills, because it covers important subjects related to students’ lives, environments, homeland and nation. It contributes to building a generation whose members are useful to society; this is because it plays a major role in teaching students about legal rulings, linking them to reality and developing students’ abilities to solve problems. Secondary-school students need to acquire the various aspects of thinking skills that will enable them to generate mental evidence or clues, make conclusions, recognise the large number of lessons learned from a religious perspective, and evaluate all these with reference to appropriate criteria (Metwally, 2017). 4.5. Studies relating to the self-questioning strategy, critical thinking and academic achievement 4.5.1. Studies on the self-questioning strategy The self-questioning strategy is relatively recent; it conforms to the requirements of the current era as it helps the learner control their study. Al-Anzi’s (2013) study aims to identify the impact of the self-questioning strategy through its development of second-grade intermediate KSA students’ metacognitive skills and academic achievement in monotheism (Tawheed). Al-Tarhouni (2012) examines the effect that the self-questioning strategy has on achievement and on the development of critical-thinking skills among students of the Faculty of Arts/Department of Education at Misurata University, while Gamal (2014) evaluates the impact of the self-questioning strategy when teaching the Prophet’s biography, analysing its effect on the achievement and critical thinking of firstgrade intermediate students at Al-Azhar in Egypt.
http://ijlter.org/index.php/ijlter
286
Fallatah (2015) considers the effectiveness of self-questioning and reciprocal teaching strategies in terms of their ability to improve academic achievement and develop reflective-thinking skills among female students attending the Islamic Education Teaching Methods course at Umm Al-Qura University. Al-Shayji (2018) evaluates how effectively the self-questioning strategy develops the academic achievement and critical-thinking skills of ninth-grade intermediate students in the Islamic Fiqh unit in the State of Kuwait. Finally, Al-Qahtani (2010) examines the effectiveness of teaching science using the self-questioning strategy in terms of academic achievement and the development of reflective-thinking skills. His study uses the quasi-experimental approach, the results of which show that there are statistically significant differences attributed to the use of the self-questioning strategy (in terms of its effect on students’ achievement and the development of reflective-thinking skills) when compared to the usual traditional method. Considering this, Al-Qahtani recommends both using the self-questioning strategy at different educational stages and including certain self-questioning activities in scientific curricula. 4.5.2. Studies on critical thinking Ali (2009) considered the impact of the guided discovery and brainstorming methods in terms of their effect on the development of critical-thinking skills and the academic achievement of secondary-school students in Tabuk under the Fiqh curriculum, while Al-Anzi (2013) demonstrates the effectiveness of using the constructivist learning model to develop critical-thinking skills and academic achievement under this curriculum. Al-Shamlaty (2015) identifies the effect using probing questions has on achievement and on the development of criticalthinking skills when teaching interpretation to second-grade intermediate students. Al-Shammari (2019) examines how a learner autonomy model can be used to develop critical-thinking skills and jurisprudential curiosity under the Fiqh curriculum, considering its effect on third-grade intermediate students in Dammam. 4.5.3. Studies on academic achievement under the Fiqh curriculum Al-Ghamdi’s study (2009) evaluates the impact of the systemic approach when teaching the Fiqh curriculum, looking at its effect on the academic achievement of second-grade intermediate students in the Makkah region. Al-Qarawi (2010), using a quasi-experimental approach, considers the effect that using the multiple intelligence teaching strategy has on the achievement of third-grade intermediate students undertaking a Fiqh course. Al-Khatib (2010) shows the effect of using the V-shaped map on the acquisition of jurisprudential concepts and the formation of an integrated conceptual structure among students at Al-Hussein Bin Talal University in Jordan. Abu Shareekh (2011) investigates the effect that the multiple intelligence and “six hats” strategies have on the achievement and development of the critical-thinking skills of tenth-grade students, as well as their effect on students’ attitudes towards learning Islamic Fiqh. Al Boughbran (2013) examines the effect software has on
http://ijlter.org/index.php/ijlter
287
first-year secondary female students' achievements (at Khamis Mushait Educational District, KSA) and their attitudes to this software. Awajan (2015) considers the effectiveness of teaching according to the theory of multiple intelligence and its effect on academic achievement and the development of creative-thinking skills among fifth-grade primary female pupils studying the Islamic Fiqh unit, comparing this effect to that of the usual teaching strategy. Ali (2016) examines the effect using the cooperative learning strategy has on the achievement of seventh-grade pupils studying for a basic education in Sana’a. Faqihi (2016) investigates the effectiveness of using concept maps in terms of their ability to improve the achievement and critical thinking of second-year secondary-school students, while Al-Otaibi (2017) considers the use of a visual strategy (based on knowledge-load theory) and its effect on the achievement of second-grade secondary-school students studying the Fiqh curriculum. Finally, Al-Omari (2019) has studied the effectiveness of the inverted class, in terms of academic achievement, focusing on first-year secondary students.
5. Research procedures Our research uses the quasi-experimental approach, which is a scientific method of solving problems that is based on a real test of hypotheses concerning causeand-effect relationships. The experimental method has the highest degree of scientific control. In this study, we sought to learn and measure the impact of the independent variable, which is the self-questioning strategy, on the two dependent variables (the academic achievement and critical-thinking skills of secondary-school students). Our research design is based on two groups (see Table 1). Table 1 The academic achievement and critical-thinking skills tests (before the experiment) Experimental group
Control group
The experiment
Teaching the Prohibited Sales Unit using the selfquestioning strategy Teaching the Prohibited Sales Unit using the traditional method
The academic achievement and critical-thinking skills tests (after the experiment)
As seen in Table 1, the design of the research experiment is based on the experimental group that studied using the self-questioning strategy, and the control group that studied in the usual way, and the research applied a test of academic achievement and a test of critical-thinking skills in Fiqh curriculum before and after the experiment on both groups.
http://ijlter.org/index.php/ijlter
288
5.1. The research community The current research community consists of all secondary-school students in West Dammam in the Eastern Province of the Kingdom of Saudi Arabia that are enrolled in government schools during the first semester of the academic year 2020/2021 (8179 students). 5.2. The research sample A random sample was selected from the study community, and the researchers followed the following research procedures: 1. An inventory was made of all secondary public education schools in the city of Dammam, based on the official lists of the General Administration of Education in the Eastern Province (52 schools, including 23 schools in East Dammam, and 30 schools in West Dammam). 2. An inventory was made of all regular secondary-school students studying during the first semester of 2020/2021, according to the West Dammam Supervision Office (8179 students). 3. Two secondary schools were randomly selected from the list of schools. 4. Two classes that were studying Fiqh 1 were randomly chosen from the selected schools. 5. The Al-Qasim bin Muhammad school was chosen to represent the control group, while the Abu Sufyan bin Al-Harith school was chosen as the experimental group. 6. Six classes were studying Fiqh in the Al-Qasim bin Muhammad School, two of which were randomly selected to represent the control group, while eight classes were studying Fiqh 1 in the Abu Sufyan bin Al-Harith School, two of which were randomly selected to represent the experimental group. 5.3. The research variables 5.3.1. First (independent) variable The independent variable is the self-questioning strategy. 5.3.2. Second (dependent) variable The dependent variable is academic achievement and critical-thinking skills. 5.3.3. Third (confounding) variables • Type: The two study groups were selected from male students only, in government schools in Dammam, which are subject to the education system in the Kingdom of Saudi Arabia. • Chronological age: The students’ ages were determined using the school records; it was found that all the students studying the Fiqh 1 course are between 16–18 years of age and are studying the course for the first time. Therefore, it can be said that the two groups are equivalent in terms of the chronological age variable. • Study content: To ensure parity and equality between the two groups, in terms of the academic content under study, all elements of the academic content were taken from the Fiqh 1 book for the secondary stage: Prohibited Sales Unit.
http://ijlter.org/index.php/ijlter
289
Table 2 shows the specific topics considered as part of this study. Table 2 Topic no. 1 2 3 4 5
Subject matter Reason for prohibited earning Monopoly Usury Usury of sales Exchange contract
Topic no. 6 7 8 9
Subject matter Selling the sample Sale of Tawaruq Gambling and ambiguous sale Insurance
This unit was chosen because of its consistency and the interdependence of its contents, which make the unit suitable for an analysis of the self-questioning strategy and critical-thinking skills. 5.4. The research instruments 5.4.1. First instrument: an academic achievement test for secondary-school students The construction of an achievement test requires the following steps: 1. Determining the objective of the achievement test. 2. Preparing a table for the specifications. 3. Determining the type of test questions to be asked. 4. Formulating and arranging the test questions. 5. Preparing the test instructions. 6. Forming the answer key. 5.4.2. Second instrument: a test of critical-thinking skills for secondary-school students This test aims to determine the level of secondary-school students’ criticalthinking skills before they are developed using the self-questioning strategy. The purpose of this test is to establish a valid scale with which to measure the criticalthinking skills of students undertaking the Fiqh 1 Prohibited Sales Unit. The test consists of 40 questions, which are divided into five skills: inference; deduction; discussion evaluation; knowledge of axioms or assumptions; and interpretation. Each skill consists of 8 questions; the student gets one mark for each correct answer. Table 3: Correlation coefficients for each statement of the axis statements with the total score of the axis to which it belongs Serial
Inference
1 0.505 ** 2 0.804 ** 3 0.449 ** 4 0.769 ** 5 0.429 ** 6 0.498 ** 7 0.818 ** 8 0.559 ** ** significant at 0.01.
Correlation coefficient Knowledge of Discussion Deduction axioms or evaluation assumptions 0.736 ** 0.803 ** 0.492 ** 0.736 ** 0.859 ** 0.591 ** 0.515 ** 0.491 ** 0.730 ** 0.763 ** 0.701 ** 0.494 ** 0.658 ** 0.635 ** 0.671 ** 0.691 ** 0.536 ** 0.494 ** 0.406 ** 0.729 ** 0.433 ** 0.562 ** 0.446 ** 0.748 **
http://ijlter.org/index.php/ijlter
Interpretation 0.465 ** 0.757 ** 0.662 ** 0.409 ** 0.784 ** 0.4665 ** 0.4977 ** 0.458 **
290
The reliability of the instrument was calculated using Cronbach’s alpha. Table 4 shows the value of the reliability coefficient for each axis of the questionnaire. Table 4: Reliability coefficient Reliability coefficient 0.760 0.758 0.807 0.824 0.778 0.917
Axis Inference skill Deduction skill Discussion evaluation skill Knowledge of axioms or assumptions skill Interpretation skill The questionnaire as a whole
It is clear from Table 4 that the reliability coefficients are high, indicating that the questionnaire has a high degree of reliability.
6. Research results This study’s aim was to learn how effective the self-questioning strategy is at developing the academic achievement and critical-thinking skills of secondaryschool students studying the Fiqh curriculum for secondary-school students. Therefore, a quasi-experimental design was used that tested the control group both before and after the experiment. The researchers’ aim is to present the results of the study and its interpretation, and to discuss the results by answering the research questions and their associated hypotheses. 6.1. The first hypothesis There is no statistically significant difference at the level (α ≥ 0.05) between the average test scores of the experimental group (which was studying using the self-questioning strategy) and the control group (which was studying using the traditional method). To verify the validity of this hypothesis, the researchers used a t-test to show the differences between the two independent groups, so as to identify the significance of the differences between the average scores of the experimental group and the scores of the control group in the post application of the total score of the achievement test (Table 5). Table 5: T-test for the significance of the differences between the average scores of the experimental group and the scores of the control group in the post-application of the achievement test Arithmetic Standard Taverage* deviation value Experimental 32 63.52 19.44 2.90 Control 32 50.23 17.09 *The average has been converted to 100 marks. Group
No
http://ijlter.org/index.php/ijlter
Significance level
Comment
Eta square
0.005
Significant at 0.01
0.12
291
Table 5 shows that the value of T is significant at a level less than 0.05, which indicates that there are statistically significant differences between the average of the experimental group and the control group in the scores of the post-application of the achievement test. These differences were in the experimental group’s favour. It is also clear from the same table that the effect of ETA square on the total score of the achievement test is 0.12, which means only 12% of the total variance of the differences between the average scores of the post-application of the achievement test for the two groups. This is considered an average effect, according to Cohen's criteria (which explains the effect of the independent variable on the dependent variable), because an effect that explains about 1% of the variance indicates a small effect, about 6% indicates a medium effect and about 15% or more indicates a high effect. The results of this study agree with Omar (2008) and Al Shayji (2018), both of whom examine how effective the self-questioning strategy is at developing students’ academic achievement. The results also agree with those of Al-Qarawi (2010), Awajan (2015) and Al-Otaibi (2017), which consider the development of academic achievement under the Fiqh curriculum. These results can be attributed to the fact that the self-questioning strategy made students feel more positive during the learning process, as they are given the opportunity to test their previous knowledge and link it with their new knowledge, leading to corrected mistakes and questions generated during and after studying, and thus leading to better academic achievement. In addition, selfquestioning provided an effective learning environment that made the student the central focus of the educational process through positive participation. 6.2. The second hypothesis There is no statistically significant difference at the level (α ≥ 0.05) between the average test scores when testing the critical-thinking skills of the experimental group and the control group. To verify the validity of this hypothesis, the researchers used a t-test to show the differences between the two independent groups, so as to identify the significance of the differences between the average scores of the experimental group and the scores of the control group in the post application of the sub-skills and the total score of the critical-thinking test (Table 6). Table 6: T-test for the significance of differences between the average scores of the experimental group and the scores of the control group in the post application of the critical-thinking test Skills Inference skill Deduction skill
Group
No
Experimental Control Experimental Control Experimental
32 32 32 32 32
Arithmetic average* 71.48 58.20 58.20 48.44 66.41
http://ijlter.org/index.php/ijlter
Standard deviation 19.37 19.47 21.44 20.76 19.43
T value
Significance level
2.74
0.008
1.85
0.069
1.40
0.166
Comment Significant at 0.01 No significance
Eta square 0.11 0.05 0.03
292
Discussion evaluation skill Knowledge of axioms or assumptions skill Interpretation skill Total mark for the criticalthinking test
Control
32
59.77
18.44
Experimental
32
54.69
20.27
Control
32
42.97
21.52
Experimental Control Experimental
32 32 32
61.72 56.25 62.50
24.99 20.08 16.52
Control
32
53.20
14.20
No significance
2.24
0.029
Significant at 0.05
0.08
0.97
0.338
No significance
0.01
2.43
0.018
Significant at 0.05
0.09
* The average has been converted to 100 marks.
Table 6 shows that the T values are significant at the level of 0.05, and less for the skills of inference and knowledge of axioms or assumptions, as well as for the total score for the critical-thinking test. This indicates that there are statistically significant differences between the averages of the experimental group and the control group in the post-application scores for those skills. These differences were in the experimental group’s favour. Table 6 also shows that the values for T are not significant for the skills of deduction, discussion evaluation and interpretation, indicating that there are no statistically significant differences between the averages of the experimental group and of the control group in terms of the marks given for the post application of those skills in the critical-thinking test. This table also indicates that the ETA square for the total score of the criticalthinking test is 0.09, which means that only 9% of the total variance in the differences between the average marks given for the post-application scores in the critical-thinking test for the two groups is due to the effect of teaching using the self-questioning strategy for the two groups. This is considered an average effect according to Cohen's criteria. These results agree with those of studies that have examined the effectiveness of the self-questioning strategy at developing critical-thinking skills among students (Al-Shayji, 2018). The study of the inference and knowledge of axioms or assumptions skills shows a large function in the experimental group’s favour. “Inference” means the ability to draw conclusions based on premises, considering a set of facts and knowledge before judging their validity; this result thereby illustrates the interrelationship between the self-questioning strategy and critical-thinking skills. We can attribute this to the fact that the students in the experimental group are practising different mental activities and skills during the educational process, generating ideas and training their inference and assumption abilities. This result agrees with Ramadan (2005) and Al-Shammari (2019). Accordingly, we reject the second hypothesis. This can be attributed to the use of the self-questioning strategy to developing critical-thinking skills, because learners asks questions and guesses the answer, leading to the stimulation of the
http://ijlter.org/index.php/ijlter
293
their critical-thinking ability and helping them to retrieve relevant previous experiences. 6.3. The third hypothesis There is no statistically significant correlation at the level (α < 0.05) between critical-thinking skills and academic achievement. To answer this question, the researchers used the Pearson correlation coefficient to measure the relationship between the scores for the research sample (n = 64) in the sub-skills and the total score for the critical-thinking test, and their scores in the achievement test (Table 7). Table 7: Results for both the experimental and control groups measuring the relationship between the scores for sub-skills and the total score for the critical-thinking skills test and their scores in the achievement test (n = 64) Skills
Correlation coefficient
Significance
Inference skill
0.4864
Significant at 0.01
Deduction skill
0.5605
Significant at 0.01
0.4899
Significant at 0.01
0.5344
Significant at 0.01
Interpretation skill
0.4430
Significant at 0.01
Total mark for the critical-thinking test
0.6613
Significant at 0.01
Discussion evaluation skill Knowing axioms or assumptions skill
Description of the correlation Positive correlation Positive correlation Positive correlation Positive correlation Positive correlation Positive correlation
Table 7 shows that there are direct (positive) correlations between the skills covered by the critical-thinking test (inference, deduction, evaluation of discussions, knowledge of axioms or assumptions and interpretation) and the total score for the achievement test. This indicates that, potentially, the higher the marks achieved for the critical-thinking test by the research sample (secondaryschool students), the higher their level of academic achievement under the Fiqh curriculum. These correlations were statistically significant at a 0.01 level. It is also clear from Table 7 that there is a direct (positive) correlation between the total score for the critical-thinking test and the total score for the achievement test. This indicates that, potentially, the higher the marks achieved for the criticalthinking test by the research sample (secondary-school students), the higher their level of academic achievement under the Fiqh curriculum. These correlations were statistically significant at a 0.01 level.
http://ijlter.org/index.php/ijlter
294
Table 8: Results for the experimental group measuring the relationship between the scores for sub-skills and the total score for the critical-thinking skills test and their scores in the achievement test (n = 32) Correlation coefficient
Significance
Inference skill
0.5452
Significant at 0.01
Positive correlation
Deduction skill
0.4631
Significant at 0.01
Positive correlation
0.3896
Significant at 0.01
Positive correlation
0.5070
Significant at 0.01
Positive correlation
Interpretation skill
0.4832
Significant at 0.01
Positive correlation
Total mark for the critical-thinking test
0.6102
Significant at 0.01
Positive correlation
Skills
Discussion evaluation skill Knowing axioms or assumptions skill
Description of the correlation
Table 8 shows that there are direct (positive) correlations between the skills covered by the critical-thinking test (inference, deduction, evaluation of discussions, knowledge of axioms or assumptions and interpretation) and the total score for the achievement test. This indicates that, potentially, the higher the marks achieved for the critical-thinking test by the research sample (secondaryschool students), the higher their level of academic achievement under the Fiqh curriculum. These correlations were statistically significant at a 0.05 level or less. It is also clear that there is a direct (positive) correlation between the total score for the critical-thinking test and the total score for the achievement test. This indicates that, potentially, the higher the marks achieved for the critical-thinking test by the research sample, the higher their level of academic achievement. These correlations are statistically significant at a 0.01 level. Table 9: Results for the control group measuring the relationship between the scores for sub-skills and the total score for the critical-thinking skills test and their scores in the achievement test (n = 32) Description of the correlation Positive correlation Positive correlation Positive correlation
Skills
Correlation coefficient
Significance
Inference skill
0.2818
No significance
Deduction skill
0.6034
Significant at 0.01
0.5555
Significant at 0.01
0.4706
Significant at 0.01
Positive correlation
Interpretation skill
0.3569
Significant at 0.01
Positive correlation
Total mark of critical thinking test
0.6416
Significant at 0.01
Positive correlation
Discussion evaluation skill Knowing axioms or assumptions skill
http://ijlter.org/index.php/ijlter
295
Table 9 shows that there are direct (positive) correlations between the skills covered by the critical-thinking test (inference, deduction, evaluation of discussions, knowledge of axioms or assumptions and interpretation) and the total score for the achievement test. This indicates that, potentially, the higher the marks achieved for the critical-thinking test by the research sample (secondaryschool students), the higher their level of academic achievement under the Fiqh curriculum. These correlations are statistically significant at a 0.05 level or less, except for the inference skill, which has no statistical significance. It is also clear that there is a direct (positive) correlation between the total score for the critical-thinking test and the total score for the achievement test. This indicates that, potentially, the higher the marks achieved for the critical-thinking test by the research sample, the higher their level of academic achievement. These correlations are statistically significant at a 0.01 level. This result is consistent with Al-Anzi (2013). The researchers attribute the above results to the fact that the development of critical-thinking skills and academic achievement are highly correlated, that is, developing one side leads to the development of the other. Accordingly, we reject the third hypothesis.
7. Conclusions The study offers useful findings, in that it indicates: 1. How effective the self-questioning strategy is at improving the academic achievement of secondary-school students studying the Fiqh curriculum. 2. How effective the self-questioning strategy is at developing the criticalthinking skills of secondary-school students studying the Fiqh curriculum. 3. There is a correlation between critical-thinking skills and the academic achievement of secondary-school students studying the Fiqh curriculum. We can explain this by saying that the self-questioning strategy makes students more positive during the learning process. Additionally, students have the opportunity to test their previous knowledge and link it to their new knowledge; this leads to corrected mistakes and to the generation of new questions, leading in turn to better academic achievement and improved critical-thinking skills. The self-questioning strategy also provides an effective learning environment that makes students the focus of the educational process, facilitating the assimilation of information and making students capable of making their own decisions on different aspects of life. The researchers attribute the third finding to the fact that development of criticalthinking skills and academic achievement is highly correlated, that is, developing one side leads to the development of the other side.
8. Research recommendations and suggestions Considering the research results, we recommend the following: 1. Attention should be given to modern teaching methods (including the strategy of self-questioning) at all academic levels, especially the secondary
http://ijlter.org/index.php/ijlter
296
2.
3.
4.
5. 6.
stage. These methods should be included in the curricula of teaching colleges, thereby training student teachers in a manner that can keep up with today’s needs. The Ministry of Education, and those in charge of curricula, need to pay attention to means of developing thinking skills. Additionally, they should encourage teachers to use these approaches through continuous training courses, informing them of the most important developments in this field and thereby improving their teaching competencies. Greater awareness among teachers is necessary; this could be achieved through educational bulletins and guided reading concerning the importance of teaching critical thinking to young people. The training of thinking skills needs further attention, both in general education and among Islamic education teachers and supervisors in particular. This can be achieved by organising in-service training courses. The completion of a course related to thinking skills in general and critical thinking in particular should be a professional requirement for teachers. Students should be made the main focus of the educational process. This can be achieved by giving them a primary role in any discussion and analysis and by teaching them to search for questions, derive ideas and use them correctly.
Based on our results, and considering the recommendations, we suggest that the following future avenues of research would be worthwhile: 1. Building a programme based on self-questioning strategies that can develop all types of thinking skills. 2. Conducting research into the effect of self-questioning strategy on the development of jurisprudential skills. 3. Conducting an analytical study of the effect of critical-thinking skills on the curricula overall.
Acknowledgement The authors extend their thanks to the participating research sample and everyone who facilitated the research procedures, as this work is part of an unpublished master's thesis under the supervision of the first author in this paper and the hard work of the second author.
9. References Abanmi, F. bin A. (2016). The effect of using the flipped classroom strategy in teaching interpretation on academic achievement and attitude toward the subject, among second year secondary students. Journal of Reading and Knowledge, 172, 21–48. Abdel Fattah, A. (2010). Teaching and Learning Strategies Models and Applications. University Book House. Abdel Wahed, A. A. (2018). The effectiveness of teaching critical-thinking skills in the life skills of fourth-grade science students. Journal of Arts, Literature, Humanities and Sociology, 25, 400–415. Abdul Hamid, A. (2000). The effectiveness of certain cognitive strategies in developing some higher reading comprehension skills among first-year secondary students. Reading and Knowledge Journal, 2, 189–241.
http://ijlter.org/index.php/ijlter
297
Abu Ajwa, H. S. (2009). The effect of self-questioning strategy on developing solving chemical problems for eleventh grade students [Unpublished master’s thesis]. Gaza Islamic University. Abu Jad, Saleh & Nofal, B. (2007). Teaching Thinking: Theory and Practice. Dar Al Masirah for Publishing and Distribution. Abu Shareekh, S. T. (2011). The effect of using strategies of multiple intelligences and the six hats on the achievement and development of critical-thinking skills among the tenth-grade students in Jordan and their attitudes towards teaching Islamic Fiqh. Education Journal, 145(2), 341–375. Adam, A. K. (2015). Problems facing secondary school principals and their impact on academic achievement: An application on Khartoum State in the period from (2014–2015) [Unpublished doctoral dissertation]. University of the Holy Quran and Islamic Sciences. Al Boughbran, M. (2013). The effect of using a computerized educational software on the achievement of first-year secondary-school students in the field of Fiqh in the schools of Khamis Mushait Educational District in the Kingdom of Saudi Arabia and their attitudes towards it [Unpublished doctoral dissertation]. University of Jordan. Al-Anzi, F. bin M. (2013). The effectiveness of using the constructivist learning model in developing critical-thinking skills and academic achievement in Fiqh course for thirdgrade intermediate students [Unpublished doctoral dissertation]. Umm Al Qura University. Al-Atoum, A. Y. et al. (2008). eilm alnafs altarbawii alnazariat waltatbiq alaistadh
alduktur shafiq falah ealawna alaistadh alduktur eadnan yusuf aleatuwm alduktur eabd alnaasir dhiab aljaraah alduktur mueawiat mahmud abu ghazal [Educational Psychology Theory and Application 11th ed.]. Dar Al Masirah. Al-Balawi, H. M. (2006). The extent to which the second intermediate-grade students in Jeddah acquire the prescribed Fiqh concepts [Unpublished master’s thesis]. College of Education for Girls in Jeddah. Al-Ghamdi, F. A. (2009). The effect of using the systemic approach in teaching Fiqh on the academic achievement of students in Makkah Al-Mukarramah intermediate schools. Journal of Studies in Curricula and Educational Supervision, 1(1), 162–243. Ali, A. G. M. G. (2016). The effectiveness of the cooperative learning strategy in teaching Fiqh on the achievement of seventh-grade students of fundamental education. Specialized International Educational Journal, 50(7), 333–353. Ali, Y. bin A. K. (2009). On the path of guided discovery and brainstorming methods in developing critical-thinking skills and academic achievement in Fiqh of secondary-school students in the city of Tabuk [Unpublished master’s thesis]. Umm Al Qura University. Al-Khatib, O. S. (2010). The effect of using the V-shape map on the acquisition of jurisprudential concepts and the formation of an integrated conceptual structure among students of Al-Hussein Bin Talal University in Jordan. Journal of the Association of Arab Universities, 55, 197–230. Al-Matroudi, K. bin I. (2009). The extent to which sixth-grade primary pupils in Riyadh have acquired the prescribed Fiqh concepts. Reading and Knowledge Journal, 98, 129–169. Al-Mofda, S.S. (2005). The effect of applying cooperative learning on the achievement of second-grade secondary students of Fiqh (experimental study). Reading and Knowledge Journal, 48, 94–144. Al-Mousa, Y. M. F. (2001). The Degree of Acquisition of Basic Concepts Contained in Islamic Fiqh in the Sharia Schools Affiliated with the Palestinian Ministry of Awqaf and Religious Affairs by Students of the Seventh, Eighth and Ninth Grades [Master’s thesis, An-Najah National University].
http://ijlter.org/index.php/ijlter
298
Al-Otaibi, H. (2017). The effectiveness of the formality strategy based on the cognitive burden on the achievement of secondary school female students in the subject of Fiqh. Journal of Scientific Research in Education, 18(2), 425–443. Al-Qahtani, N. bint S. (2010). Critical-thinking skills of female students at the College of Education at the universities of King Saud University and the Emirate University. A comparative study. Journal of the Modern Education Association, 3(7), 329–406. Al-Qarawi, A. R. bin A. (2010). The effect of using the multiple intelligences strategy on the achievement of the third intermediate grade students in Fiqh course [Unpublished master’s thesis]. Imam Muhammad Bin Saud Islamic University. Al-Qarni, A. bin M. (2013). The effect of using mental maps in teaching the developed course of Fiqh on the achievement of first-grade intermediate students in Jeddah [Unpublished master’s thesis]. Umm Al Qura University. Al-Shamlaty, O. A.-Q. (2015). The effect of teaching interpretation using probing questions on the achievement and development of critical-thinking skills among secondgrade intermediate students. Journal of Human and Administrative Sciences, 7, 64– 96. Al-Shammari, T. (2019). The effectiveness of the learner independence model in developing critical-thinking skills and jurisprudential curiosity in Fiqh curriculum for third-grade intermediate students [Unpublished master’s thesis]. Imam Abdul Rahman AlFaisal University. Al-Shayji, S. (2018). The impact of self-questioning strategy on developing academic achievement and developing critical thinking among middle ninth-grade students in the Islamic Fiqh unit in the State of Kuwait. Educational Journal, 127(32), 51–88. Al-Surour, N. (2005). Teaching Thinking in the School Curriculum. Dar Wael for Publishing and Distribution. Al-Tarhouni, A. A. R. (2012). The effect of using self-questioning strategy on the achievement and development of some critical-thinking skills among students of the Department of Education, Faculty of Arts, Misurata University [Unpublished master's thesis]. University of Misurata. Arian, S. (2003). The effectiveness of the metacognition strategy in the achievement of philosophy among first-year secondary students and its impact on their attitudes towards reflective philosophical thinking. Journal of Reading and Knowledge, 20, 113–139. Attia, M. A. (2009) Metacognitive Strategies in Understanding Reading. Dar Al-Manahg for Publishing and Distribution. Awajan, W. S. M. (2015). The effectiveness of strategies based on multiple intelligences in developing academic achievement and creative thinking skills among female students of Islamic Fiqh in the fifth-grade primary. Journal of Educational and Psychological Sciences, 16(1), 197–236. Bahloul, I. A. (2004). Recent trends in metacognitive strategies in teaching reading. Journal of Reading and Knowledge, 30, 148–280. Bashir, M. T. (2017). Effective teaching strategies and ways of thinking. Researches of the First International Conference: Reading the Arab and Islamic Heritage Between the Past and the Present, 5, 459–472. Education Document (1970). Education Policy in the Kingdom of Saudi Arabia, amended in 1995. El-Ghahori, N. A. (2012). The effectiveness of self-scheduling strategy (K.W.L.H.) in developing deep understanding of physical concepts and metacognition skills for eighth-grade students in the Sultanate of Oman. Arab Studies in Education and Psychology, 1(32), 11–58.
http://ijlter.org/index.php/ijlter
299
Fallatah, R. (2015). The effectiveness of self-questioning strategy and reciprocal teaching in developing academic achievement and reflective thinking among female students of the Islamic Education Teaching Methods course at Umm Al-Qura University [Unpublished doctoral dissertation]. Umm Al-Qura University. Faqihi, F. M. A. (2016). The effectiveness of using concept maps in developing achievement and critical thinking among female students of the second-year secondary in the subject of Fiqh [Unpublished master’s thesis]. King Khalid University. Gamal, T. (2014). The effect of using self-questioning strategy in teaching the biography of the Prophet on the achievement and critical thinking of Al-Azhar first-year preparatory students. Education Journal, 158(3), 327–369. Magdy, H. A. K. (2003). Teaching Thinking: Future Strategies for the New Millennium. Dar Al Fiker Alarabi. Metwally, E-S. S. E.-S. (2017). A proposed strategy based on active learning for teaching Fiqh in developing some critical-thinking skills among Al-Azhar secondaryschool students. Arab Studies in Education and Psychology, 86, 151–181. Ministry of Education (2018). Secondary Education Guide: Curricula System. Secondary Education Development Project. Nabhan, S. (2001). A proposed program for developing critical thinking in mathematics among ninth grade students in Gaza [Unpublished doctoral dissertation]. Ain Shams University. Nassar, A. A. H. (2015). The effect of using thinking maps strategy on developing critical-thinking skills and science processes for tenth-grade students [Unpublished master’s thesis]. Islamic University, Gaza. Omar, S. M. (2008). The effectiveness of using students' self-questioning in developing cognitive achievement and self-efficacy among students studying philosophy at the secondary stage. Journal of the Educational Society for Social Studies, 17, 15. Qatami, N. (2004). Effective Teaching Skills. Dar Al-Fikr. Ramadan, H. A. M. (2005). The interaction between some metacognitive strategies and information processing levels in developing scientific concepts and critical thinking for first-year middle-school students in science. Journal of Scientific Education, 8(1), 181–236. Saadeh, J. A. (2011). maharat altafkir (mae 1997, altatbiqiatu) jawdat 'ahmad saeada [Teaching Thinking Skills.] Dar Al-Shorouk for Publishing and Distribution. Udall, A. J, & Daniels, J. E. (1991). Creating the Thoughtful Classroom: Strategies to Promote Student Thinking. Zephyr Press.
http://ijlter.org/index.php/ijlter
300
International Journal of Learning, Teaching and Educational Research Vol. 20, No. 8, pp. 300-320, August 2021 https://doi.org/10.26803/ijlter.20.8.18 Received Jul 01, 2021; Revised Aug 31, 2021; Accepted Aug 31, 2021
The Implementation of Formal Assessments in Intermediate Phase Mathematics at Primary Schools in South Africa Senzeni Sibanda Central University of Technology, Free State, South Africa https://orcid.org/0000-0002-5937-3018 Awelani M. Rambuda* Central University of Technology, Free State, South Africa https://orcid.org/0000-0002-2518-0832
Abstract. The purpose of this research was to explore the implementation of formal assessments in intermediate phase mathematics at primary schools. The research was elicited by reports that assessment methods and procedures for tackling learners’ needs had been observed to be insufficient in South African schools. The study is grounded in Piaget’s cognitive constructivism and Vygotsky’s social constructivism. The researchers conducted document analysis of teachers’ portfolios which were purposefully selected. Nine teachers– three from each of Grades 4 to 6 were sampled. The portfolios were analysed to establish whether the implementation of the formal assessments was aligned with the Curriculum and Assessment Policy Statement requirements. A checklist was used to determine teachers’ implementation of formal assessment. Measures of central tendency were used to analyse data. The results revealed that teachers were not developing the abilities of learners in handling complex mathematical procedures as per the requirements of the policy. This implies that learners lacked the ability to break down mathematical problems into different factors or constituent parts. Learners were given a test instead of a project or investigation. Hence teachers were not promoting cooperative learning which is advocated by the policy. Therefore, teachers should be assisted by knowledgeable colleagues and subject advisors in their adoption and use of assessment. There must be a close examination of the classroom observation tools that are currently being utilised. Classroom observation assists teachers to improve their assessment strategies. The Department of Basic Education should supply tablets to primary schools to promote social constructive
*
Corresponding author: Awelani M. Rambuda, arambuda@cut.ac.za
©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
301
interaction amongst teachers and learners to enhance effective teaching and learning. Keywords: assessment; cooperative learning; formal assessment; intermediate phase; mathematics
1. Introduction Chigonga (2020) asserts that assessment is the procedure employed to collect information with the intention to examine the learner’s previous understanding of the content, identify the gaps and formulate instructional decisions thereafter. In the context of this study, the intention of assessment is to enhance mathematics teaching and learning. Assessment serves a dual purpose: first, as proof of learning to indicate what the learner has achieved; second, to assist the teacher to ascertain whether the learner is performing as specified in the curriculum. A difference is made between informal assessments, such as daily monitoring of the learner’s progress, and a formal assessment, which is the systematic evaluation of learner’s progress. In this study, the main emphasis is on formal assessment because Reddy et al. (2015) assert that South African mathematics teachers lack knowledge in formal assessment. To reinforce their significance in teaching and learning enhancement, formal assessments have been continuously implemented on-line during the COVID-19 pandemic in South Africa. Teachers need to collect and evaluate learners’ understanding to come up with reasonable decisions on the way forward (Liberman et al., 2020). Whilst formal assessment is incorporated in government policy documents, there is inadequate evidence based on research to confirm whether teachers implement assessments properly and adequately. Furthermore, research indicates that, although several studies have been carried out on teacher assessment practices, there has been limited research on teachers’ assessment literacy in South Africa. Additionally, it is indicated that a greater number of teachers have inadequate proficiency in the implementation of formal assessments and that there is a lack of proper guidance and support regarding this aspect. There is thus an inadequacy of assessment knowledge and practice amongst teachers. Mathematics teaching and learning has been found to be highly insufficient in South Africa schools (Jojo, 2019). Papadikis et al. (2017) contend that the adoption of mathematics teaching theory such as Realistic Mathematics Education (RME) develops connection between mathematics and reality in kindergarten learners. Teachers and learners interact so that learners can construct reality and under complex mathematical problems. Papadikis et al. (2021) further suggest that teacher use of smart mobile devices such as tablets lead to effective teaching and meaningful learning in mathematics. Mathematics applications (apps) expose learners to real life problems which enable them to understand fundamental mathematical concepts. The comprehension of numbers is more evident in early childhood learners who use tablets (Papadikis et al., 2018). However, to check if there is meaningful mathematics learning, teachers must possess knowledge of formal assessment practices. As a result, a study on how teachers carry out assessments for teaching and learning enhancement is essential (Kanjee & Mthembu, 2015). Therefore, what is the nature of formal assessment in intermediate phase mathematics in schools? What are the most tested cognitive levels in formal assessments? Consequently, the objective of this study is to
http://ijlter.org/index.php/ijlter
302
suggest strategies mathematics teachers could adopt to improve their teaching to implement effective formal assessment. The South African Department of Basic Education (DBE) is relevant because one of the key imperatives of the Curriculum and Assessment Policy Statement (CAPS) is to implement a valuable and functional assessment program which give pertinent information to all role-players to enhance teaching and learning procedures (DBE, 2011). The preceding section provides the introduction to the research. The following sections are organised as follows: section 2 provides a literature review, section 3 presents the methodology, section 4 provides the results, section 5 provides discussion of the results, section 6 presents the conclusion and section 7 presents a list of references. The upcoming section discusses the literature review.
2. Literature review This research is framed within constructivism theory based on the constructivist perspective, an aspect that views knowledge as repeatedly formulated by human beings in everyday life situations, not just as stipulated (Donald et al., 2014). Additionally, the principal assertion of constructivism is that knowledge is not imposed by external forces; instead, it is internally constructed by an individual. Furthermore, reference is made to the key to constructivism being learner activity. The aim is to understand how teachers teach and assess learners, building from their prior knowledge. This will, in turn, lead learners to purposefully formulate new, worthwhile concepts (Lombard &Themane, 2015). Two constructivism theories and concepts related to teaching and learning of mathematics are clarified, which will inform the development of the theoretical framework of this research. These theories are Piaget’s cognitive constructivism (1953) and Vygotsky’s social constructivism (1978). Cognitive constructivism theory calls for teachers to deviate from the common practice of direct teaching and be facilitators in teaching and learning. It is a view of learning suggesting that instead of imparting knowledge, that is formulated beforehand, learners must be given opportunities to make use of their own capabilities and skills to create their own understanding (Paulsen & Dednam, 2016). Social constructivism theory is a theory concerned mainly with the way cognitive development happens from ‘the outside in’. Social constructivism theory emphasises the conception that cognitive development happens within social connections. Furthermore, reference is made to the theory that all better intellectual systems are a result of social interaction (Vygotsky, 1978). Vygotsky’s theory proposes the key role of teachers and other members in society in children’s acquisition of a particular measure of cognitive development. The theory stresses socialisation for sustained development. The mediator must perform the role of providing instructional support for the learners so that they can become independent (Donald et al., 2014). Similarly, Paulsen and Dednam (2016) mention that learning must be extended to the home and other out of school settings. This can be accomplished through learner use of tablets to develop their mathematical competence (Papadakis et al., 2016). At the same time, the teacher still has a significant role to decide to choose the most effective, wellinformed potential mediator for the learners. This can also be attained through scaffolding and student-to-student discussions (Abrie et al., 2016). However, mathematics is usually viewed as challenging to teach, even under normal
http://ijlter.org/index.php/ijlter
303
circumstances, and the situation is currently worsened by the COVID-19 pandemic (Fritz et al., 2019). Literature selected for review was derived from the gap identified, guided by the research questions. In other words, the nature of formal assessments in the intermediate phase in South African schools was precisely used to guide the literature review. 2.1. The nature of formal assessments in intermediate phase mathematics in South African schools Formal assessment consists of School Based Assessment (SBA) and end of year examinations (DBE, 2011). It is the duty of concerned teachers to monitor and control assessment tasks and record them for subsequent promotional purposes. Before being administered to learners, all formal assessment tasks must be moderated to ensure control of quality and retention of proper measures. Although the SBA element may have various formats, regarding mathematics, assignments, projects, investigations, tests, and examinations are suitable, as outlined in Table 1 below. Table 1: Minimum requirements for formal assessment in the Intermediate Phase
Continuous, SBA
End of the year Examination
Form of Assessments
Term 1
Term 2
Term 3
Tests Examinations Assignments Investigation Project Total
1
1 1
1
1
2
Term 4
1 1 2
1 2
2
Number of tasks per year 3 1 2 1 1 8 1
Weighting
75%
25%
(DBE, 2011, p. 294). As indicated in Table 1, formal assessments, in the form of SBA, have a significant role in learners passing their grades in mathematics. Formal assessments count 75% towards the final grade mark. The forms of assessment are tests and examinations, assignments, projects, and investigations. These forms of assessment did not change, even during the COVID-19 pandemic, and currently guide the program of assessment (DBE, 2021). The forms of assessment, according to requirements, are explained below. 2.1.1. Tests and examinations Tests and examinations are written individually by the learners. These tasks must be accurately set to enable a clear indication of content mastery in mathematics by the learners (DBE, 2011). Additionally, the main point is not merely to get a mark to record, but to discover what learners have learned, and on what concepts they need more practice. Borich (2014) mentions that tests precisely measure the skills which are expected to be acquired by the learners. Moreover, authentic tests must ask learners questions which will enable them to display their skills in real-life
http://ijlter.org/index.php/ijlter
304
situations. This implies that mathematics teachers must ensure that their methods of instruction in the classrooms will, in turn, enable learners to display what was taught in the real world. Borich (2014) further suggests The Test Blueprint (TTB), which must complement test objectives and guarantees that teachers include all the information crucial to a good test. TTB ensures that the test will cater for different cognitive levels of learners. The table below illustrates the blueprint for mathematics.
Total
Per cent
2. The student will discriminate addition problems.
Application
1
1
4%
2
2
8%
4
16%
6
6
24%
6
6
24%
6 25
24%
Content Outline
1. The student will discriminate the subtraction sign from the addition sign.
Comprehension
Knowledge
Table 2: Test blueprint for a unit on subtraction without borrowing
3. The student will discriminate correctly solved subtraction problems from incorrectly solved subtraction problems. 4. The student will solve correctly singledigit subtraction problems. 5. The student will solve correctly subtraction problems with double-digit numerators and single-digit denominators. 6. The student will solve correctly doubledigit subtraction problems. Total
3
4
6 18
Percent
12%
16 %
72 %
4
100%
(Borich, 2014: 382).
The information in Table 2 implies that teachers must ensure that the six cognitive levels of Bloom’s taxonomy are applied when setting tests and examinations. The next paragraph explains the assignment as a form of assessment. 2.1.2. Assignment An assignment is given to the learners as an individual task. This may be obtained from past examination papers; but whatever, it must centre on challenging content, as there is a variety of resources to refer to. It can be done at home, not under class supervision (DBE, 2011). Borich (2014) suggests that assignments must be given immediately after the lessons or activities to which they relate. Furthermore, teachers must display the assignment in their classrooms, so that learners who have missed information can always refer to the displays. The following paragraph explains the use of projects as another form of assessment.
http://ijlter.org/index.php/ijlter
305
2.1.3. Projects A project is an activity which extends learning beyond the classroom and positions it in the real world (Coombs, 1995). Moreover, projects are employed to evaluate a variety of abilities and capabilities. Projects should enable learners to implement their mathematical concepts in practical situations. Through projects, learners are expected to gather the data, analyse it thereafter, and draw conclusions (DBE, 2011, p.294). Gawe et al. (2016) point out that the project method is learner-centred learning and highly based on the constructivist principle. This gives learners an opportunity to work on their own in collecting relevant information required for the project and be able to present it thereafter. Furthermore, the project method assesses a variety of skills, at the same time integrating various activities like planning, research, data analysis, and reporting. This is essential when intermediate phase mathematics learners have data handling as a topic. Moreover, Mays et al. (2016) acknowledge that a project widens the kinds of skills needed by learners as they are assessed. Additionally, Borich (2014) confirms that project-based learning conveys to learners the significance of the learning process, aids them to set goals, and affords them opportunities to work co-operatively. This implies that teachers must ensure that skills like research and presentations are taught effectively before handing out projects for assessments. The paragraph below explains the investigation as a formal task. 2.1.4. Investigation An investigation is a formal task which can be employed to determine rules or concepts. It can include connections of patterns, arriving at conclusions, and identification of patterns. Rubrics with specific marks to be given per skill are used to assess investigations. The skills come in various forms, such as organising and recording, communicating ideas, calculations and generalising, and drawing a conclusion. Teachers must take note that all tasks must accommodate all the cognitive levels of learners. The forms of assessment used should also take the suitability of learners’ ages into consideration. Tasks should thus be designed in a manner that allows the subject content to be achieved and ensures that all the aims and objectives are attainable. Moreover, suitable marking tools, for instance, rubrics and memoranda, must be utilised (DBE, 2011). Nieuwoudt and Reyneke (2016) mention a rubric as a valuable tool in assessing learners’ responses. They assert that this is so because of the main elements that a rubric contains. Two of the three main elements are outlined below. 2.1.5. Evaluative criteria Evaluative criteria are used to distinguish between acceptable and unacceptable responses. The criteria will differ according to the skills being tested. This suggests that teachers must assist learners in developing skills through teaching and learning (Nieuwoudt &Reyneke, 2016). 2.1.6. Quality definitions Quality definitions are level descriptors which are used to specify the number of points to be earned per specific skill. Complementing the use of rubrics is the work of Elrod and Strayer (2015), which reveals that a rubric is an essential tool for teachers to monitor the learners’ work. Furthermore, it can be used as a mechanism to indicate cultural behaviours and practices in the classroom, hence
http://ijlter.org/index.php/ijlter
306
becoming an essential part of teachers’ assessment practices for learners. This is so because working with rubrics both helps teachers and learners understand competencies required and can be used to provide feedback. Formal assessment tasks must cater to different levels of cognitive development as seen in Table 3 below. Table 3: A range of cognitive levels to be catered for intermediate phase mathematics teaching and learning Cognitive levels
Description of skill to be demonstrated
Examples
Knowledge
• •
Straight recall Estimation and rounding off Identification and correct use of formula Use of mathematical facts Appropriate use of mathematical vocabulary.
1. Write down the next three numbers in the sequence: 107; 109; 111… [Grade 4]
Perform well-known procedures Simple applications and calculations Derivations from given information Identification and use of correct formula
1.Determine the value of y+5=15 [Grade 4]
(=25%)
• • •
Routine procedures (=45%)
• • • •
2. Determine the factors of 44 [Grade 5] 3. Write down the prime numbers that are factors of 36 [Grade 6].
2.Use three different techniques of calculating 59910 [Grade 5] 3. Calculate 12/4+3/12-1/3. [Grade 6]
Complex procedures
•
(=20%)
•
• •
Complex calculations and higher-order reasoning Investigations to describe rules and relationships Problems not based in real-world contexts Conceptual understanding
1. Mandy is 6 years old and Betty is 12 years old. Determine the ratio between their ages. Write the ratio in simplest form. [Grade 4] 2. Investigate the properties of rectangles and squares and identify similarities and differences. [Grade 5] 3. There are 20 sweets in the packet. William and his friends ate 2 fifths of the sweets. How many sweets are left? [Grade 6]
http://ijlter.org/index.php/ijlter
307
Problemsolving (=10%)
• •
•
Unseen non-routine problems Higher-order processing and understanding required May require breaking down into constituent parts to solve
1. The sum of three consecutive numbers is 29. Find the numbers. [Grade 4] 2. John divides a certain number by 16. He found an answer of 246. What is the number? [Grade 5] 3. Busi has a bag containing six coloured balls: 1 blue, 2 red balls and 3 yellow balls. She puts her hand in the bag and draws a ball. What is the chance that she will draw a red ball? Write the answer in the simplest fractional form [Grade 6]
(DBE, 2011: 296 Intermediate mathematics teachers should adhere to this continuum of cognitive levels when setting tests and examinations so that the unevenness of cognitive development of learners is addressed. This implies that there must be accurate moderation of assessments to guarantee that different cognitive levels of learners are catered for. The ensuing paragraph discusses the moderation of assessments. Despite the significant role meant to be played by effective implementation of formative assessment in classrooms, the literature indicates various challenges faced by teachers in this regard. Assessing learners has many facets. Additionally, the procedure of making sense of learners’ mathematical reasoning and explanation of approaches is more convoluted than can be assumed and poses challenges to teaching and learning (Suurtamm et al., 2016).
3. Methodology The study follows a quantitative approach, using a survey design. Quantitative research is an approach which utilises numerical data in a structured and empirical scheme. It makes use of a particular subgroup to derive its data; subsequently, findings are generalised to the sector that is currently under consideration (Maree & Pietersen, 2016). The aim of quantitative research is to clarify trends amongst given factors in a particular study (Ivankova et al., 2016). The survey design was employed for this research because of its appropriateness to describe current conditions in the implementation of assessments in intermediate phase mathematics teaching and learning. The researchers conducted the document analysis from the primary schools because they offer intermediate phase mathematics. A checklist (Appendix 1) was used to determine whether the requirements of formal assessment implementation were met by intermediate phase mathematics teachers.
http://ijlter.org/index.php/ijlter
308
3.1. Format of the checklist Killen (2015) points out that checklists are useful for assessment in situations where a very specific set of objective judgements needs to be made about learner performance – in this case, in the implementation of formal assessments in intermediate phase mathematics teaching and learning. The checklist utilised in this study was constructed from the information provided in the Curriculum and Assessment Policy Statement Grades 4-6 mathematics (DBE, 2011, pp. 294-296). The checklist comprised three sections, namely: • Section A: Biographical data with three items. • Section B: Minimum Requirements for Formal Assessment: Intermediate Phase Mathematics with seven items. • Section C: Tests Cognitive Levels with Description Skills to be demonstrated with sixteen items. Document analysis was conducted in the school environment. Ethical issues were considered by the researchers. Gasa and Mafora (2015) emphasise that if the information is not openly accessible, written consent for the utilisation of the data ought to be obtained from the possessors. The researchers obtained permission through the consent form which was annexed to the letter addressed to the principals and intermediate phase teachers. Document analysis was explained to the participants before it was conducted. Purposive sampling, which is classified under a non-probability sampling method, was utilised for document analysis. Maree and Pietersen (2016) point out that this method of sampling is applied with a clearly defined aim in mind – in this instance, with the purpose of obtaining information on how formal assessments are implemented in intermediate phase mathematics teaching and learning. Nine teachers were sampled for document analysis, three from each of Grades 4 to 6. Their portfolios, which included learners’ formal tasks for the year 2018, were analysed. The checklist’s Cronbach’s alpha coefficient was 0.83, which suggested a moderate reliability for the scale. 3.2. Biographical details of the respondents The following table represents the biographical the data of the teachers whose formal assessment records were analysed. Table 4: Biographical details of the teachers whose formal assessment records were analysed N=9 Personal The % Respondents % Total data According to Category A1. Gender
A2. Grade
A3. School Quintile
http://ijlter.org/index.php/ijlter
Male Female
4 (44.4%) 5 (55.6%)
4 5 6 1
3 (33.3%) 3 (33.3%) 3 (33.3%) 2 (22.2%)
100
2 3 5
1 (11.2%) 3 (33.3%) 3 (33.3%)
100
100
309
Analysis of the data in Table 4 indicates that three (44.4%) formal assessment records of male teachers were analysed, whilst five (55.6%) formal assessment records of female teachers were examined. In terms of representation according to the gender of the teachers who agreed that their formal assessment records be analysed, there was no significant difference. Further analysis of the data in the table reveals that formal assessment records of all the three grades in the intermediate phase were analysed. However, representation according to schools’ quintile was not evenly spread. Analysis of the data in the table discloses that 22.2% of the analysed formal assessment records were from quintile 1 schools, 11.1% were from quintile 2 schools, 33.3% were from quintile 3 schools, and the remaining 33.3% were from quintile 5 schools. The researcher did not analyse documents from quintile 4 schools because all teachers from this quintile who were approached were not willing for their formal assessment records to be analysed. They volunteered to participate only in the interviews. The results are sufficient, as 80% of the schools in quintile 5 in the Lejweleputswa district were represented, which gives a clear indication of how formal assessments are implemented. The ensuing table presents, and gives an analysis of, the data obtained for the minimum requirements for formal assessment in intermediate phase mathematics as stipulated in the CAPS document.
4. Results 4.1. The minimum requirements for formal assessment in intermediate phase mathematics Table 5: Minimum requirements for Formal Assessment N=9 Forms of Assessment Mean Median Standard Deviation School-based Assessment (SBA)
B1
Tests
2.11
2.00
0.33
B2
Midyear Examination
2.00
2.00
0.00
75%
B3
Assignment
1.67
2.00
0.50
B4 B5 B6 B7
Investigation Project Total End of the year Examination
1.11 1.44 1.11 2.00
1.00 1.00 1.00 2.00
0.33 0.53 0.33 0.00
End of the year Examination 25%
Analysis of the data in Table 5 reveals that teachers fully administer examinations (M=2.00, MD=2.00, SD=0.00). Statistics show that the data is symmetrical, since the skewness measure is zero. Furthermore, all formal assessment records that were analysed indicated that teachers fully administer examinations as stipulated in the CAPS document. This is because the standard deviation is zero, which implies that there is no deviation from the mean. The mean value of 2 indicates that the implementation of examinations had been achieved as per the rubric of the checklist. Another revelation is that tests were wholly administered (M=2.11, MD=2.00, SD=0.33). Statistics show that the data is positively skewed, since the
http://ijlter.org/index.php/ijlter
310
mean is higher than the median. Resultantly, the mean value indicates that the implementation of the test was fully achieved according to the CAPS requirements. With reference to assignments (M=1.67, MD=2.00, SD=0.50), statistics show that this form of assessment is moderately achieved as it is not fully done according to the CAPS requirements. The data is negatively skewed, since the mean is smaller than the median. Investigation as a form of formal assessment is not done accurately as per the CAPS requirements (M=1.11, MD=1.00, SD=0.33). The mean value of 1.11 indicates that the implementation of investigation was not achieved as per the rubric of the checklist. This implies that some of the teachers whose records were analysed are not giving their learners investigative tasks. This implies that teachers are not promoting critical and creative thinking in their learners as suggested in CAPS. Statistics show that the data is positively skewed, since the mean is higher than the median. Additionally, statistics show that the project as a form of assessment is not achieved at all (M=1.44, MD=1.00, SD=0.53). The mean of 1.44 is an explanation of underachievement as per the rubric of the checklist. This implies that teachers are not providing learners with opportunities through which they can express their competencies of solving complex issues in daily occurrences. By virtue of assignment, investigation, and project not being achieved, the total tasks per year is resultantly not achieved (M=1.11, MD=1.00, SD=0.33). The mean value reveals that the total number of tasks per year was not met. Statistics show that the data is positively skewed, since the mean is higher than the median. The following table serves to present and analyse cognitive levels and abilities tested in learners. 4.2. Cognitive levels and abilities tested in learners Table 6 indicates levels and description of skills which should be demonstrated by Grades 4 to 6 learners in mathematics. Analysis of the data in Table 4.26 below shows that there is greatest achievement in the testing of cognitive skills such as straight recall (M=2.11, MD=2.00, SD=0.33). The mean value of 2.11 is a clear indication of achievement as per the checklist rubric. Statistics show that the data for this item is positively skewed, since the mean is higher than the median. Furthermore, testing of cognitive skills such as performing well-known procedures, simple applications and calculations, unseen-non routine procedures, and breaking down problems into constituent parts are fully mastered as stipulated in the CAPS document (M=2.00, MD=2.00, SD=0.00). This implies that these skills are taught effectively, which in turn makes the broad aims of the subject achievable (DBE, 2011: 295). The standard deviation is zero, which implies that there is no deviation from the mean. The mean value of 2 indicates that the testing of cognitive levels and abilities of learners has been achieved as per the rubric of the checklist. Statistics stipulate that the data for these skills is symmetrical, since their skewness measure is zero. The following table presents and analyses the data on testing of cognitive levels. Regarding the testing of cognitive skills, such as estimation and appropriate rounding off and use of mathematical facts, the statistical results are similar, which indicate that they were moderately achieved (M=1.89, MD=2.00, SD=0.33). Their standard deviations are not very far from the mean. The implication,
http://ijlter.org/index.php/ijlter
311
therefore, is that knowledge as a cognitive level skill is tested as stipulated in the CAPS document, hence making the aims and objectives of the subject achievable. However, with reference to the testing of cognitive skills, such as problem solving and investigations to describe rules and relationships, their statistical results are similar, which indicates that there is high underachievement (M=1.11, MD=1.00, SD=0.33). The mean of 1.11 for these skills reveals that their testing is not being done according to the CAPS requirements as per checklist rubric. This implies that teachers are not exposing learners to complex procedures which are designed to improve their higher-order reasoning. Moreover, the learners do not have opportunities to solve unseen non-routine procedures – as a result, this impedes their conceptual understanding of the subject. The data for these statistics is positively skewed, since the means are higher than their medians. Table 6: Testing of cognitive levels with description of skills Checklist items
Mean
Median
N=9 Standard Deviation
C1
Estimation and appropriate rounding off of numbers.
1.89
2.00
0.33
C2
Straight recall.
2.11
2.00
0.33
C3
Identification and direct use of correct formula. Use of mathematical facts. Appropriate use of mathematical vocabulary. Perform well known procedures. Simple applications and calculations, which might involve many steps. Derivation from given information may be involved. Identification and use after changing the subject of correct formula, generally similar to those encountered in class. Problems involving complex calculations and/or higher order reasoning. Investigations to describe rules and relationships; there is often not an obvious route to the solution. Problems not based on real world context could involve making significant connections between different representations. Conceptual understanding. Unseen, non-routine problems (which are not necessarily difficult). Higher order understanding and processes are often involved. Might require the ability to break the problem down into its constituent parts.
1.78
2.00
0.33
1.89 1.33
2.00 1.00
0.33 0.50
2.00 2.00
2.00 2.00
0.00 0.00
1.44
1.00
0.52
1.67
2.00
0.50
1.11
1.00
0.33
1.11
1.00
0.33
1.33
1.00
0.50
1.89 2.00
2.00 2.00
0.33 0.00
1.67
2.00
0.50
2.00
2.00
0.00
C4 C5 C6 C7 C8 C9
C10 C11
C12
C13 C14 C15 C16
http://ijlter.org/index.php/ijlter
312
The data shows that testing of the appropriate use of mathematical vocabulary is not achieved (M=1.33, MD=1.00, SD=0.50). The mean of 1.33 indicates that the testing of this skill does not meet the CAPS requirements as per the checklist rubric. This denotes that teachers are not developing the correct use of the language of mathematics, as stated under the specific skills which must be developed in mathematics (DBE, 2011:8). This, in turn, affects the achievability of aims and objectives of the subject. Ultimately, the data shows that derivation from given information as a cognitive skill is not tested (M=1.44, MD=1.00, SD=0.52). The mean of 1.44 indicates that this skill is not tested according to the CAPS requirements as per the checklist rubric. This implies that teachers are not exposing learners to different formulas applicable in mathematics, as stated under routine procedures as a cognitive skill to be taught. This means that learners cannot identify and use other mathematical formulas they may encounter – other than those used or taught in their classrooms. The following table presents and analyses the data on the most tested cognitive levels in formal assessments. Table 7: The most tested cognitive levels in formal assessments N=9 Cognitive Levels
Mean
Median
Standard Deviation
D1 Knowledge
1.80
1.80
0.24
D2 Routine procedures D3 Complex procedures D4 Problem-solving
1.78 1.36 1.89
1.75 1.25 2.00
0.63 0.22 0.17
Analysis of the data in Table 7 shows that the cognitive level that is most developed and tested in learners is problem-solving, although it is moderately tested (M=1.89, MD=2.00, SD=0.17). The data is negatively skewed, because the mean is lower than the median. The mean of 1.89 confirms a moderate achievement according to CAPS requirements as per checklist rubric. This finding means that learners can moderately solve non-routine problems which are not necessarily difficult, which might lead to their understanding of word sums. The standard deviation is nearer to the mean, indicating that the documents analysed revealed similar information. The second most developed and tested cognitive level is knowledge, which is also moderately implemented in formal assessment (M=1.80, MD=1.80, SD=0.24). The mean of 1.80 indicates that this cognitive level is moderately achieved according to the stipulated requirements in the CAPS and as per checklist rubric. This suggests that learners moderately round off and recall some of the mathematical facts. Statistics indicate that the data is normally distributed, since the mean is equal to the median. The third cognitive level which is promoted and tested in the learners is routine procedures. This cognitive level is also moderately tested (M=1.78, MD= 1.75, SD=0.63). The implication of this finding is that learners are taught well-known procedures and other simpler calculations as stipulated in the CAPS document. Statistics indicate that the data is positively skewed, since the mean is higher than the median. However, the data reveals that development and testing of complex procedures in learners is not being done (M=1.36, MD=1.25, SD= 0.17). This mean indicates that teachers are neglecting the development of
http://ijlter.org/index.php/ijlter
313
learners in this cognitive level. Therefore, teachers are not developing the abilities of learners in handling complex mathematical procedures as per the requirements of CAPS. This implies that learners lack the ability to break down mathematical problems into different factors or constituent parts. In summary, document analysis shows that minimum requirements for formal assessments in intermediate phase mathematics are not being met as stipulated in the CAPS document.
5. Discussion Startlingly, it has been proven that learners were not actively engaged in the implementation of some of the assessment requirements in intermediate phase mathematics teaching and learning. A quantitative analysis of the checklist shows that investigations and projects are not implemented as forms of assessment in mathematics – rather, learners were given a test instead of a project or investigation. Resultantly, learners lack critical and creative thinking, which must be triggered by active engagement and exploration. This finding implies that teachers are not promoting cooperative learning, which is advocated by the CAPS. Cooperative learning is a set of instructional strategies in which learners work in mixed ability groups to reach specific cognitive and social development objectives (Eggen & Kauchak, 2016). Additionally, co-operative learning provides learners with an opportunity of working together and makes certain that every member of the group has a chance to participate. Moreover, it encourages learners to act as learning resources for one another (Gawe et al., 2016). Vygotsky (1978) also underpins this idea through the role of social interaction. In mathematics teaching and learning, learners can work together on projects – for example in data handling projects – collecting, organising, representing, analysing, interpreting, and reporting the data. They can also work together, regarding space and shape, to construct 3-D shapes using mathematics apps as suggested by Papadakis et al. (2018). Borich (2014) discusses some of the outcomes of co-operative learning, which are attitudes and values, pro-social behaviour, alternative perspectives and viewpoints, and higher thought and processes. People’s values and attitudes are modelled by interacting with society through the exchange of information. Learners can achieve this by working in groups or in pairs, sharing ideas and exchanging information. This plays a crucial role in shaping their values and attitudes, which, in turn, they need to deepen their independent thought (Borich, 2014). This idea is endorsed by Vygotsky (1978) who mentions the significance of social interaction in cognitive development. Classrooms are now a significant medium in which to reinforce pro-social behaviours, as a result of the high volume of working parents or guardians. Therefore, teachers must plan for and implement co-operative learning programmes to bring learners together. This implies that teachers must come up with tasks and activities which will promote working together on the part of learners (Borich, 2014). Similarly, teachers must create learning experiences which give learners opportunities for working cooperatively in interesting, challenging, and open-ended tasks such as projects and investigations (Killen, 2015). Cooperative learning furnishes the context in which several views and ideas can be exchanged (Borich, 2014). This is closely linked to participative learning, in which learners are motivated to state their views on the subject matter. It rests on the premise that learning takes place when
http://ijlter.org/index.php/ijlter
314
negative criticism does not exist in class (Vakalisa, 2016). This is applicable when learners work together in projects and assignments which develop critical thinking skills which improve mathematics competence. Furthermore, cooperative learning is associated with outstanding academic achievement of learners. It improves learners’ critical-thinking and problem-solving skills. Higher thought process cannot occur without an amalgamation of attitudes and values, prosocial behaviour, viewpoints, and integrated identity. This implies that teachers must come up with tasks and activities which will stimulate learners’ higher thought processes – higher-order thinking must be stimulated by complex thinking tasks. This, in turn, implies that teachers must engage learners in, for example, research and problem-solving skills, which encourage high-order thinking (Borich, 2014). Acclaiming the idea of the aforementioned outcomes are the ideas of Gawe et al. (2016), who emphasise the benefits of co-operative learning. They mention learner achievement and social consequences as some of the noticeable benefits. Outstanding achievement has been reported in the classrooms where co-operative learning takes place. Furthermore, higher-order concepts can be taught effectively through co-operative learning. As stated by Gawe et al. (2016), “The expression that ‘two heads are better than one’ suggests the superiority of ideas that emerge when more than one person is engaged in a project…some of the complex tasks that learners are given to investigate” (p. 267). Additionally, co-operative learning can contribute to integrating learners into networks of peer social relationships which, in turn, assist them with constructive conflict resolution leading to academic performance. Therefore, teachers must ensure that every learner has a chance to participate in a group (Gawe et al., 2016). Sustaining this idea is Vygotsyk’s (1978) work, through mediation and the Zone of Proximal Development (ZPD), where cognitive mediation will take place. Consequently, learners’ thinking competencies can be lessened if not directed to comprehend on Bloom’s level of thinking (Jacobs, 2016). The emphasis of the level of understanding is underpinned by one of mathematics’ specific aims – to establish a profound understanding of concepts to have a logic of mathematics as a subject (DBE, 2011). This aim is not achieved according to this study. Additionally, the checklist data confirm the issue of language challenges, which affects testing of cognitive skills. Teachers are not making use of appropriate mathematical language when testing learners for formal assessments.
6. Conclusion The research has contextual limitations because it was conducted in primary schools with different circumstances of working conditions; therefore, the results cannot be generalised because schools differ in contextual factors. The findings of the study confirm that formal assessments are inadequately implemented. Mathematics is usually viewed as a challenging subject to teach, even under normal circumstances, and the situation is currently worsened by the COVID-19 pandemic. Therefore, teachers should be assisted by knowledgeable colleagues and subject advisors in adopting and using assessment. Professional development must be a significant element that seeks to establish the constructive use of formative assessment. Three aspects which relate to assessments are the nature of tasks and materials to support teachers’ use of formative assessments;
http://ijlter.org/index.php/ijlter
315
professional development that supports changes in teaching practice; and classroom observations with a formative assessment focus. There must be a close examination of the classroom observation tools that are utilised to enable teachers to ascertain the capacity of the tool to give valid feedback on formal assessment tasks. Furthermore, classroom observation assists teachers to improve their assessment strategies when they are given feedback. Feedback from classroom observations supports teachers in identifying areas of formal assessments in which they need to improve to enhance the teaching and learning of mathematics. Teachers need to promote practical learning, which can be achieved using portfolios and oral presentations to evoke deeper understanding of mathematical concepts. Concept mapping, linking connections involving related mathematical ideas, must be utilised to achieve improvements in critical and creative thinking in investigations and problem-solving tasks. Moreover, teachers are encouraged to adapt the instructional method to promote the utilisation of assessment to ascertain learners’ misconceptions. Subsequent teaching and learning alternatives must be drawn from these misconceptions and, consequently, accord learners who misinterpreted the concept(s) another chance of achievement (Chigonga, 2020). Although, South Africa is a developing country, with several socioeconomic problems, the Department of Basic Education should supply tablets to schools to enhance the teaching and learning of mathematics. Research indicates that the use of tablets in mathematics teaching and learning improves competence in early childhood education.
7. References Abrie, M., Blom, N., & Fraser, B. (2016). Theoretical foundations. In M. Jacobs, N. C. G. Vakalisa, & N. Gawe (Eds.), Teaching-Learning Dynamics. (pp. 37). Pearson Education. Borich, G. D. (2014). Effective teaching methods: Research-based practice. Pearson. Chigonga, B. (2020). Formative assessment in mathematics education in the twenty-first century, theorizing STEM education in the 21st century. IntechOpen, 1-10. http://dx.doi.org/10.5772/intechopen Coombs, B. (1995). Successful teaching: A practical handbook. Heinemann. Department of Basic Education, (2011). Curriculum and assessment policy statement, grades 4-6 mathematics. Government Printers. https://www.education.gov.za/ Portals/0/CD/National%20Curriculum%20Statements%20and%20Vocational/ CAPS%20IP%20%20MATHEMATICS%20GR%204-6%20web.pdf?ver=2015-0127-161430-553 Department of Basic Education, (2021). Intermediate phase annual teaching plans (ATPs), mathematics. Government Printers. https://www.education.gov.za/ Curriculum/IntermediatePhaseATPs2021.aspx Donald, D., Lazarus, S., & Moolla, N. (2014). Educational psychology in social context: Eco systematic applications in South Africa. ABC Press. Eggen, P. D, & Kauchak, D. P. (2016). Educational psychology: Windows on classrooms. Pearson. Elrod, M., & Strayer, J. F. (2015). Using an observational rubric to facilitate change in undergraduate classroom norms. In C. Suurtamm & A. McDuffie (Eds.), Annual Perspectives in Mathematics Education 2015: Assessment to Enhance Teaching and Learning (pp. 87-96). National Council of Teachers of Mathematics.
http://ijlter.org/index.php/ijlter
316
Fritz, A., Haase, V. G., & Rasanen, P. (2019). International handbook of mathematical learning difficulties: From the laboratory to the classroom. https://doi.org/10.1007/978-3-31997148-3 Gasa, V., & Marofa, P. (2015). Using secondary sources of data. In C. Okeke & M. van Wyk (Eds.), Educational research: An African approach. (pp. 355-370). Oxford University Press. Gawe, N., Jacobs, M., & Vakalisa, N.C.G. (2016). Learner-centred methods. In M. Jacobs, N. C. G. Vakalisa & N. Gawe N. (Eds.), Teaching-learning dynamics. (pp. 250-281). Pearson Education. Ivankova, N.V., Creswell, J.W., & Plano Clark, V.L. (2016). Foundations and approaches to mixed-methods research. In K. Maree (Ed.), First steps in research. (pp. 306336). Van Schaik. Jacobs, M. (2016). Aims and objectives. In M. Jacobs M., N. C. G. Vakalisa & N. Gawe (Eds.), Teaching-learning dynamics. (pp. 105-137). Pearson Education. Jojo, Z. (2019). Mathematics education system in South Africa. IntechOpen, https//doi.org/10.5772/intechopen.85325 Kanjee, A., & Mthembu, J. (2015). Assessment literacy of foundation phase teachers: An exploratory study. South African Journal of Childhood Education, 5(1), 142168. https://doi.org/10.4102/sajce.v5i1.354 Killen, R. (2015). Teaching strategies for quality teaching and learning. Juta. Lombard, K., & Themane, M. (2015). Contextualising teaching practice as a component of teacher education. In M. J. Taole (Ed.), Teaching practice, perspectives and frameworks. (pp. 11-24). Van Schaik. Liberman, J. Levinin, J., & Luna-Bazaldua, D. (2020). Are students still learning during COVID-19? formative assessment can provide the Answer. World Bank Blogs. https://blogs.worldbank.org/education/are-students-still-learning-duringcovid-19 formative-assessment-can-provide-answer Maree, K., & Pietersen, J. (2016). The quantitative research process. In K. Maree (Ed.), First steps in research. (pp. 162-172). Van Schaik. Mays, T., Grosser, M., & de Jagger, L. (2016). Getting practical: A guide to teaching and learning. Oxford University Press. Nieuwoudt, S., & Reyneke, M. (2016). Assessment. In M. Jacobs, N. C. G. Vakalisa & N. Gawe (Eds.), Teaching-learning dynamics. Pearson Education. Papadakis, S., Kalogiannakis, M., & Zaranis, N. (2016). Comparing tablets and PCs in teaching mathematics: An attempt to improve mathematics competence in early childhood education. Preschool and Primary Education, 4(2), 241-253. https://doi.org/10.12681/ppej.8779 Papadakis, S., Kalogiannakis, M., & Zaranis, N. (2018). The effectiveness of computer and tablet assisted intervention in early childhood students’ understanding of numbers. An empirical study conducted in Greece. Education and Information Technologies, 23(5), 1849–1871. https://www.learntechlib.org/p/191703/ Paulsen, R., & Dednam, A. (2016). Challenges in Mathematics: Mathematical Literacy and Numeracy. In E. Landsberg, D. Kruger & E. Swart (Eds.), Addressing barriers to learning: A South African perspective. (pp. 241-262). Van Schaik. Piaget, J. (1953). The origin of intelligence in the child. Routledge and Kegan Paul. Reddy, C., Le Grange, L, Beets, P., & Lundie, S. (2015). Quality assessment in South African schools. Juta.
http://ijlter.org/index.php/ijlter
317
Suurtamm, C., Thompson, D., Kim, R., Moreno, L., Sayac, N., Schukajlow, S., Silver, E., Ufer, S., & Vos, P. (2016). Assessment in mathematics education: Large scale assessment and classroom assessment. Springer. Vakalisa, N. C. G. (2016). Participative teaching. In M. Jacobs, N. C. G. Vakalisa & N. Gawe (Eds.), Teaching-learning dynamics. (pp. 39-71). Pearson Education. Vygotsky, L. (1978). Mind in society: The development of higher mental processes. Harvard University Press. William, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3-14. https://doi.org/10.1016/j.stueduc.2011.03.001
http://ijlter.org/index.php/ijlter
318
Appendix 1 Checklist for Formal Assessments Checklist Number SECTION A: Biographical Data A1. Indicate gender of the teacher. Male 1 Female 2 A2. Grade: 4 5 1 2 3 A3. School quintile 1 2 3 4
6 (choose only one grade per checklist)
5
T 2
T 3
1
1
1
1
B5.Project
http://ijlter.org/index.php/ijlter
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
B4.Investigation
B6.Total B7. Examination
1 3
B2.Examinations B3.Assignment
End of the year Examination
T 4
Outstanding Achievement 3
SBA
T 1
Number of tasks per year
Achieved 2
B1.Tests
Minimum requirements per term
Weighting
Forms of assessment
Not Achieved 1
SECTION B: Minimum Requirements for Formal Assessment: Intermediate Phase Mathematics
1
2
1
1
1 2
2
2
75%
1 2
8 1
25%
319 For office use only Key 1 Not achieved 2 Achieved 3 Outstanding achievement
Cognitive levels
Description of skills to be demonstrated
Not achieved
Achieved
Outstanding achievement
SECTION C: Tests Cognitive Levels
1
2
3
Knowledge (=25%)
C1. Estimation and appropriate rounding off of numbers C2. Straight recall
1
2
3
C3. Identification and direct use of correct formula C4. Use of mathematical facts C5. Appropriate use of mathematical vocabulary C6. Perform wellknown procedures C7. Simple applications and calculations, which might involve many steps C8. Deriviation from given information may be involved C9. identification and use after changing the subject) of correct formula generally similar to those encountered in class C10.Problems involving complex calculations and/or higher order reasoning C11. Investigations to describe rules and relationships- there is
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
Routine procedures (=45%)
Complex procedures (=20%)
http://ijlter.org/index.php/ijlter
Comments
320
Problem solving (=10%)
often not an obvious route to the solution C12.Problems not based on real world context-could involve making significant connections between different representations C13.Conceptual understanding C14.Unseen, nonroutine problems (which are not necessarily difficult) C15.Higher order understanding and processes are often involved C16.Might require the ability to break the problem down into its constituent parts
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
Consent Form I, the undersigned hereby agree to participate in the research on the Implementation of Formal Assessments in the Intermediate Phase Mathematics as foundation of teaching and learning enhancement in Lejweleputswa district. ___________________________ Signature of the Educator
http://ijlter.org/index.php/ijlter
___________________ Date
PUBLISHER Society for Research and Knowledge Management Port Louis Republic of Mauritius www.ijlter.org
International Journal of Learning, Teaching and Educational Research
The International Journal of Learning, Teaching and Educational Research is an open-access journal which has been established for the disChief Editor Dr. Antonio Silva Sprock, Universidad Central de semination of state-of-the-art knowledge in the Venezuela, Venezuela, Bolivarian Republic of field of education, learning and teaching. IJLTER welcomes research articles from academics, edEditorial Board ucators, teachers, trainers and other practitionDr. Fatima Zohra Belkhir-Benmostefa ers on all aspects of education to publish high Dr. Giorgio Poletti quality peer-reviewed papers. Papers for publiDr. Chi Man Tsui cation in the International Journal of Learning, Dr. Fitri Suraya Mohamad Teaching and Educational Research are selected Dr. Hernando Lintag Berna through precise peer-review to ensure quality, Dr. Charanjit Kaur Swaran Singh originality, appropriateness, significance and Dr. Abu Bakar readability. Authors are solicited to contribute Dr. Eglantina Hysa to this journal by submitting articles that illusDr. Mo'en Salman Alnasraween trate research results, projects, original surveys Dr. Hermayawati Hermayawati and case studies that describe significant adDr. Selma Kara vances in the fields of education, training, eDr. Michael B. Cahapay learning, etc. Authors are invited to submit paDr. Bunmi Isaiah Omodan pers to this journal through the ONLINE submisDr. Vassiliki Pliogou sion system. Submissions must be original and Dr. Meera Subramanian should not have been published previously or Dr. Muhammad Kristiawan be under consideration for publication while Dr. Wahyu Widada being evaluated by IJLTER. Dr. Som Pal Baliyan Dr. Reem Khalid Abu-Shawish Dr. Froilan Delute Mobo Dr. Mohamed Ali Elkot Dr. Anabelie Villa Valdez Mr. Teody Lester Verdeflor Panela