Vol 11 no 1 april 2015 by ijlter.org

IJLTER.ORG

p-ISSN: 1694-2493 e-ISSN: 1694-2116

International Journal of Learning, Teaching And Educational Research

Vol.11 No.1

PUBLISHER London Consulting Ltd District of Flacq Republic of Mauritius www.ijlter.org

International Journal of Learning, Teaching and Educational Research

The International Journal of Learning, Teaching and Educational Research is an open-access journal which has been established for the disChief Editor Dr. Antonio Silva Sprock, Universidad Central de semination of state-of-the-art knowledge in the Venezuela, Venezuela, Bolivarian Republic of field of education, learning and teaching. IJLTER welcomes research articles from academics, edEditorial Board ucators, teachers, trainers and other practitionProf. Cecilia Junio Sabio ers on all aspects of education to publish high Prof. Judith Serah K. Achoka quality peer-reviewed papers. Papers for publiProf. Mojeed Kolawole Akinsola Dr Jonathan Glazzard cation in the International Journal of Learning, Dr Marius Costel Esi Teaching and Educational Research are selected Dr Katarzyna Peoples through precise peer-review to ensure quality, Dr Christopher David Thompson originality, appropriateness, significance and Dr Arif Sikander readability. Authors are solicited to contribute Dr Jelena Zascerinska to this journal by submitting articles that illusDr Gabor Kiss trate research results, projects, original surveys Dr Trish Julie Rooney Dr Esteban Vázquez-Cano and case studies that describe significant adDr Barry Chametzky vances in the fields of education, training, eDr Giorgio Poletti learning, etc. Authors are invited to submit paDr Chi Man Tsui pers to this journal through the ONLINE submisDr Alexander Franco sion system. Submissions must be original and Dr Habil Beata Stachowiak should not have been published previously or Dr Afsaneh Sharif be under consideration for publication while Dr Ronel Callaghan Dr Haim Shaked being evaluated by IJLTER. Dr Edith Uzoma Umeh Dr Amel Thafer Alshehry Dr Gail Dianna Caruth Dr Menelaos Emmanouel Sarris Dr Anabelie Villa Valdez Dr Özcan Özyurt Assistant Professor Dr Selma Kara Associate Professor Dr Habila Elisha Zuya

VOLUME 11

NUMBER 1

April 2015

Table of Contents Using Natural Language Processing Technology to Analyze Teachers’ Written Feedback on Chinese Students’ English Essays ........................................................................................................................................................................ 1 Ming Liu, Weiwei Xu, Qiuxia Ran and Yawen Li Using Particle Swarm Optimization Approach for Student Engagement Measurement............................................ 12 Ming Liu, Yuqi Wang, Hua Liu, Shujun Wu and Chang Li A Study of the Development of Courseware and Students’ Learning Effectiveness in Primary Education: Using Three Teaching Techniques as an Example ....................................................................................................................... 22 Fang-Chun Ou A Comparative Examination of Teacher Candidates’ Professional Practicum Experiences in Two Program Models ................................................................................................................................................................................................. 36 Nancy Maynes, Anna-Liisa Mottonen, Glynn Sharpe and Tracey Curwen A Study of Formative Assessment Strategies in Teachers‘ School-Based In-Service Training ................................... 53 Eva Nyberg and Mona Holmqvist Olander Designing and using interactive e-books in Vietnam ..................................................................................................... 75 Ngoc-Giang Nguyen Impact Investigation of using a Digital Literacy Technology on a Module: Case Study of Tophat .......................... 99 Xue Zhou and Stella-Maris Orim Implementation of the 2006 Education Amendment Act on Indigenous Languages in Zimbabwe: A Case of the Shangaan Medium in Cluster 2 Primary Schools in the Chiredzi District ................................................................. 117 Webster Kododo and Sparky Zanga The Concept of In Situ Lecturing ...................................................................................................................................... 128 Joachim R. R. Ritter and Ellen Gottschämmer

The Mathematics Problem and Mastery Learning for First-Year, Undergraduate STEM Students ...................... 141 Layna Groen, Mary Coupland, Tim Langtry, Julia Memar, Beverley Moore and Jason Stanley Teaching Culture through Language: Exploring Metaphor and Metonymy in Chinese Characters ..................... 161 Hu, Ying-Hsueh Coaches‟ Perceptions of how Coaching Behavior affects Athletes: An Analysis of their Position on Basic Assumptions in the Coaching Role ................................................................................................................................. 180 F. Moen, R. Giske and R. Høigaard Regional Educational Development Research and School Improvement: A Systematic Literature Review of Research ............................................................................................................................................................................... 200 Associate Professor Lena Boström The Value-Added Assessment of Higher Education learning: The case of Nagoya University of Commerce and Business in Japan ............................................................................................................................................................... 212 Hiroshi Ito Surname and Nobuo Kawazoe

International Journal of Learning, Teaching and Educational Research Vol. 11, No. 1, pp. 1-11, April 2015.

Using Natural Language Processing Technology to Analyze Teachers’ Written Feedback on Chinese Students’ English Essays Ming Liu, Weiwei Xu, Qiuxia Ran and Yawen Li Southwest University Beibei District, Chongqing, China

Abstract. Writing an essay is a very important skill for students to master, but a difficult task for them to overcome. It is particularly true for English as Second Language (ESL) students in China. It would be very useful if students can receive timely and effective feedback about their writing. In order to build an automatic feedback system, we need to understand the relationship between textual features and human teacher feedback, and how well those features were used for predicting feedback rating. In this study, we analyzed 105 Chinese English majors’ essays with teachers’ feedback and used Coh-Metrix, a computational linguistic tool, to extract features from their writing. The study results showed some feedback was moderately correlated to some textual features (e.g. text easability cohesion and lexical diversity were related to coherence feedback) and those feedback are more predictable, such as spelling, grammar, supporting ideas and coherence. This finding has important implications for building automated writing feedback tool. Keywords: Writing Feedback, Text Analysis, Natural Language Processing.

1. Introduction With the coming of the 21st century and the globalization of English, English essay writing, as one of the four basic skills of language learning, has become a more and more important skill. It not only requires some basic writing skill, such as spelling and grammar, but also asks some high competency of writing, such as coherence, structure and reasoning. Thus, it is also a difficult task to overcome. It is particularly so in China. Statistics show that the number of college students in China has soared to twenty-six million in 2013 (Bureau of Statistis of China, 2013), accounting for the largest proportion of ESL learners worldwide. Since 1987, the writing test has become one important aspect of the College English testing in China. As for college students in China, college English has been an obligatory course to take. In a typical English course, © 2015 The authors and IJLTER.ORG. All rights reserved.

students have to do 2-3 essay writing assignments and take 1 essay writing test in order to pass national English tests, such as College English Test (CET) 4 or Test for English-Major (TEM) 4. Essay writing is the last part of these tests. Novice writers need feedback to develop their writing skills; however, providing timely and meaningful feedback is time-consuming and expensive. Since the early 1980s, researchers have investigated feedback on students’ writing (Brannon & Knoblauch, 1982). These study results showed that written feedback provided a potential value in motivating students to revise their draft and improving their writing (Leki, 1991). As a result, written feedback is the most popular method among various feedback delivery modes (oral feedback, audiotaped and writing conference) that teachers use to interact and communicate with students. Straub (Straub, 2000) suggested that the effective teacher feedback should be written in complete sentences, avoid abstract, technical language and abbreviations, relate their comments back to specific words and paragraphs from the students’ text, by viewing students’ writing seriously, as part of a real exchange. In addition, an increasing number of studies have also been conducted to see whether certain types of feedback are more likely than others to help ESL students improve the accuracy of their writing, such as direct and indirect feedback (Lee, 2004). Direct or explicit feedback occurs when the teacher identifies an error and provides the correct form, while indirect strategies refer to situations when the teacher indicates that an error has been made but does not provide a correction, thereby leaving the student to diagnose and correct it. With the advanced development of information technology and natural language processing techniques, various numbers of automatic essay scoring (AES) systems have been proposed. Haswell (Haswell, 2006) reviewed systems for automated feedback tracing back to the 1950s. These systems focused more on assessment of end products, and less on providing formative feedback (Shermis & Burstein, 2003; Williams & Dreher, 2004) The Writer Workshop (Anderson, 2005) and Editor (Thiesmeyer & Theismeyer, 1990) both focus on grammar and style. Sourcer’s Apprentice Intelligent Feedback system (SAIF) (Britt, Wiemer-Hastings, Larson, & Perfetti, 2004) is a computer assisted essay writing tool used to detect plagiarism, uncited quotations, lack of citations, and limited content integration problems. The Glosser system (Villalon, Kearney, Calvo, & Reimann, 2008) aims to support reflection in writing through trigger questions. It uses text mining algorithms to help learners think about issues such as coherence, topics, and concept visualization. However, Glosser only provides generic trigger questions. Liu et al. (Liu, Calvo, & Rus, 2014; Liu, Calvo, & Rus, 2010) investigated an automatic trigger question generation system which could support critical review writing. The aim of this study is to investigate the frequent type of feedback used by human teachers and the relationship between the feedback and the textual features extracted by using the natural language processing techniques.

The rest of this paper is constructed as follows: Section 2 presents related work on feedback classification. Section 3 describes the study and discusses the results. Finally, Section 4 concludes this paper.

Table 1: Criterion Category

Criterion Category Grammar

Usage

Mechanics

Style

Organization

Examples Fragments, Run-on Sentences Subject-verb agreement, Ill-formed verbs Pronoun Error, Missing Possessive Error Wrong article, Missing article Confusing words, Wrong form of word Preposition Error Spelling, Capitalize Proper Nouns Missing Question mark, Missing final punctuation Missing Apostophe, Missing Comma Repetition of words, Inappropriate words or phrases Too many short sentences, Too many long sentences Background, Thesis, Main-point

Supporting ideas, Conclusion

2. Related Work Recent development in natural language processing techniques has made it possible for researchers to develop a wide range of sophisticated techniques that facilitate text analysis. Some tools, such as Coh-Metrix (Graesser, McNamara, Louwerse, & Cai, 2004), LIWC (Pennebaker & Francis, 1999) and Gramulator (Rufenacht, McCarthy, & Lamkin, 2011), are useful in this respect, and have certainly contributed to ESL knowledge (S．A. Crossley & McNamara, 2012). Coh-Metrix is a powerful computational tool that provides over 100 indices of cohesion, syntactical complexity, connectives and other descriptive information about content (Graesser et al., 2004). Coh-Metrix has extensively been used to analyze the overall quality of writing (S．A. Crossley & McNamara, 2012) and one important aspect of writing quality, such as coherence (Scott a. Crossley & McNamara, 2011a). For example, Crossley and McNamara found that computational indices related to text structure, semantic coherence, lexical sophistication, and grammatical complexity best explain human judgments of text coherence. This study focused on using Coh-Metrix to analyze more aspects of writing quality including, Supporting Ideas, Conclusion and Sentence Diversity. The AES systems, such as Criterion (Burstein, Chodorow, & Leacock, 2004), can provide feedback on some aspects of writing including grammar, usage, mechanics, style, organization, development, lexical complexity and prompt© 2015 The authors and IJLTER.ORG. All rights reserved.

specific vocabulary usage (See Table 1). The Criterion categories are more relevant to our case since we aim to generate corrective feedback on different aspects of ESL student writing.

3. Study We conducted an empirical study in analyzing Chinese ESL college student essays with teachers’ comments and the relationship between the teacher feedback and textual features. Section 3.1 describes the annotation process, where each essay is scored in different aspect, such as Grammar, Spelling, Coherence, Organization and Supporting Ideas. Section 3.2 shows the textual feature extraction process. Section 3.3 illustrates the relationship between the textual features and each feedback category, while section 3.4 examines the predictive strength of the features in explaining the score variance in the each feedback score.

3.1 Proposed Feedback Taxonomy

Table 2: Feedback Frequency and Pearson Correlations between Raters Feedback Category Grammar Spelling Word Count Sentence Diversity Conclusion Supporting Ideas Coherence Chinglish Expression Organization

Frequency 48 12 24 40 44 98 40 24 89

r .824 .504 .707 .454 .747 .632 .716 .352 .534

Our dataset containing 105 English majors’ essays with teachers’ feedback was collected from a large university in China. Two experienced English teachers volunteered to rate the quality of the essays. They had at least five years of teaching composition course for English majors. Their first task was to identify the most frequent feedback type adapted from the standardized rubric used for grading college English. 9 frequent feedback categories were found, including Grammar, Spelling, Word Count, Sentence Diversity, Conclusion, Supporting Ideas, Organization, Coherence and Chinglish (See Appendix I). Table 2 shows that Supporting Ideas and Organization categories were more frequent than others, while Spelling and Chinglish Expression and word count were less frequent. We observed some feedback categories were similar to the Criterion categories, such as Grammar, Spelling and Supporting Ideas. But, the Chinglish Expression and Conclusion categories only appeared in our dataset. The teachers’ second task was to give a score to each feedback category regarding to the rubric (See Appendix I) on a scale of 3. 1 means negative © 2015 The authors and IJLTER.ORG. All rights reserved.

feedback on the category while 3 means positive feedback on the category. The Correlations between the raters are located in Table 2. The raters had the highest correlations for judgments of Grammar, Word Count, Conclusion and Supporting Ideas and the lowest correlations for Chinglish and Sentence Diversity. For further analysis, the dataset was randomly divided into training set (n=70) and testing set (n=35). A training set was used to identify which of the textual features most highly correlated with each feedback score. Moreover, the training set was used to train a multiple regression model to examine the amount of variance explained by each writing feature. The model was then applied to a test set to calculate the accuracy of the analysis.

3.2 Textual Feature Extraction We used Coh-Metrix 3.0, which could retrieve 108 scores of textual features. More information can be found on the website (http://cohmetrix.Memphisedu/cohmetrixpr/index.html).

Descriptive indices: It includes the number of paragraphs, number of sentences, number of words, number of syllables in words, mean length of paragraphs etc. Cohesion: Cohesion is a key aspect of understanding language discourse structure and how connections within a text influence cohesion and text comprehension(Kintsch & van Dijk, 1978). Coh-Metrix employs referential cohesion including noun overlap, argument overlap, stem overlap, and LSAbased semantic overlap. Sentence Complexity: The grammatical structure of a text is also an important indicator of human evaluations of text quality. Difficult syntactic constructions (syntactic complexity) include the use of embedded constituents, and are often dense, ambiguous, or Ungrammatical (Graesser et al., 2004). Syntactic complexity is also informed by the density of particular syntactic patterns, word types and phrase types. Lexical sophistication: Lexical sophistication refers to the writer’s use of advanced vocabulary and word choice to convey ideas. Lexical sophistication is captured by assessing the type and amount of information provided by the words in a text. Words are assessed in terms of rarity (frequency), abstractness (concreteness), evocation of sensory images (imagability), salience (familiarity), and number of associations (meaningfulness). Words can also vary in the number of senses they contain (polysemy) or levels they have in a conceptual hierarchy (hypernymy). Moreover, we propose and extract 8 new features that are not available in CohMetrix. These features refer to characteristics of ESL learners’ writing style and reflect on the importance of the introduction section, conclusion section and mechanics in errors including spelling errors and grammatical errors. In the database, each essay is stored as a plain text, where each line is a paragraph. We © 2015 The authors and IJLTER.ORG. All rights reserved.

use Java API to extract the first line and last line text, as introduction and conclusion section respectively. For checking spelling errors, an open source spelling error checker, called LanguageTool (http://www.languagetool.org/), is employed to scan each word. For checking grammatical errors, the Link Grammar Parser (Lafferty, Sleator, & Temperley, 1992) is used to check the grammar of a sentence based on natural language processing technology. If the link grammar could not generate links (relations between pairs of words) after parsing a sentence, this sentence would be considered as ungrammatical. Number of words in Introduction: the total number of words in the first paragraph considered as the introduction section. Number of words in Conclusion: the total number of words in the last paragraph considered as the conclusion section. Introduction Portion: the ratios of number of words in introduction to the total number of words in the document. Conclusion Portion: the ratios of number of words in conclusion to the total number of words in the document. Spelling errors: the number of spelling errors. We employ an open source spelling error checker called LanguageTool (http://www.languagetool.org/), which is part of the OpenOffice suite. Grammatical errors: the number of sentences with grammatical errors. We use the Link Grammar Parser (Lafferty et al., 1992) to check the grammar of a sentence, which is also widely used in ESL context. Percentage of spelling errors: the ratios of the number of word spelling errors to the total number of words in the document. Percentage of grammatical errors: the ratios of the number of sentence with grammatical errors to the total number of sentences in the document. Therefore, there are totally 116 features extracted from each essay.

3.3 Pearson Correlation Table 3: Correlations between Textural Features Scores and Raters’ feedback scores Feedback Category Features R

Chinglish

Coherence Conclusion

Supporting Ideas

Grammar Sentence Variety Spelling Organization Word Count

Gerund incidence paragraph length first person singular pronoun incidence Text Easability Cohesion Lexical diversity Conclusion Portion Lexical diversity Intentional verbs incidence Adjective incidence CELEX Log minimum frequency for content words Grammar errors Hypernymy for verbs Standard deviation of Sentence length Spelling Errors Number of paragraphs Word count

.415 .459 .493

P value <0.05 <0.05 <0.01

.433 .402 .477 .394 .496 .503 .541

<0.05 <0.05 <0.05 <0.05 <0.05 <0.05 <0.01

-.606 .506 .413

<0.01 <0.01 <0.05

-.617 .507 .666

<0.05 <0.01 <0.01

Based on the system producing feature scores and the human annotators’ score on each category, we used IBM SPSS for evaluating the Pearson correlation between textual features and each category. Over 30 textual features demonstrated significant correlations with the human ratings of each feedback category. Table 3 shows the Chinglish was more related to the number of Gerund used, the paragraph length and the first person singular pronoun incidence. The Coherence was correlated to Text Easability PC Deep cohesion, consistent with Crossley and McNamara’s study result (S. Crossley & McNamara, 2010). As expected, the Conclusion was more related to the features of Conclusion Portion and Lexical Diversity. We have not defined specific features which can detect the Supporting Ideas. However, some features, such as Intentional verbs and Adjective incidence, have shown their moderate correlations with the category of Supporting Ideas. As we had expected, the Grammar and Spelling were negatively related to the features of grammar error and spelling error. The Word Count was correlated to the number of words in an essay. Organization was correlated to the number of paragraphs since the essays with only 1 or 2 paragraphs were given lower scores by human annotators since they did not have a clear essay structure, introduction, body and conclusion. Crossley and MacNamara (Scott a. Crossley & McNamara, 2011b) got the similar study results, where six features including the total number of paragraphs were significant predictors in the regression to the raters’ organization evaluations.

3.3 Test Set Model We used the training set to train a regression model for each feedback category and evaluated the model in testing set. Table 4 shows the performance of each regression model for predicting essay feedback ratings. It has been found that Grammar (r2=.881) and Spelling feedback (r2=.886) were easier for prediction, since some textual features were highly related to those feedbacks. It also demonstrated that the combination of the textual features accounted for 88.1% of the variance in the grammar evaluation of the 35 essays comprising the test set. On the other hand, organization and conclusion were difficult to predict since r2=.223 and r2=.380 respectively since the textual features were not correlated to those feedback ratings. Table 4: Linear Regression Analysis to Predict Essay Feedback Ratings in Testing Set Feedback R R2 S.E. Chinglish .764 .584 .349 Expression Coherence .790 .624 .472 Conclusion .616 .380 .486 Supporting .745 .555 .407 Ideas Grammar .939 .881 .260 Sentence .735 .540 .423 Variety Spelling .941 .886 .242 Organization .475 .223 .473 Word Count .756 .572 .535 Notes: S.E. is standard error

4. Conclusion Human teachers’ written feedback is very useful for students to revise their draft and improve writing. A great number of researches has been conducted to investigate the theoretical foundation of feedback in terms of feedback mode, feedback strategies and feedback classification. With the development of information technologies, automated essay scoring tools have been proposed, which can extract textual features and generate corrective feedback on the traits of writing including grammar, usage, style, mechanics and organization. However, these AES systems are mainly designed for international ESL students, who take TOFEL test. Those students can only represent a small portion of ESL students, because they obviously possess a higher English competency. Thus, we conducted an empirical study to investigate the frequent feedback types and examine the feasibility of using existing natural language processing tools to automatically measure the feedback. In the study, we collected 105 essays written by English majors and some teachers’ comments at a large university in China. Two English teachers first found 9 frequent feedback categories based on the teachers’ comments. Some feedback categories are consistent with the Criterion category. Then, they gave a © 2015 The authors and IJLTER.ORG. All rights reserved.

score on a scale of 1 to 3 to each feedback category of each student essay. The study results showed that the feedback had moderate correlations with some features extracted by using Coh-Metrix, a computational writing analysis tool, and some proposed new features. For example, coherence feedback was highly related to Text Easability Cohesion and Lexical diversity, while Supporting Ideas was related to Intentional verbs incidence and Adjective incidence. Moreover, it has been found that some feedback, such as supporting ideas, coherence, grammar and spelling, were more predictable. It indicated the feasibility of using existing NLP tools to measure the quality of feedback. Our future work will examine teachers’ comments in detail and collect nonEnglish major student essays for analysis. In addition, we will focus on building an automatic essay feedback generation system. Specifically, we will investigate the feedback generation mechanism by using association rule mining algorithms. In addition, we will look at how to incorporate effective feedback strategies, such as formative feedback theory, into feedback generation templates.

Acknowledgment The authors would like to thank those teachers and student participants. This work is partially supported by Chongqing Social Science Planning Fund Program under grant No. 2014BS123, Fundamental Research Funds for the Central Universities under grant No. SWU114005, No. XDJK2014A002 and No. XDJK2014C141 in China.

Appendix A Table 5: Nine Traits Rubric for Essay Writing Category Organization

Supporting Ideas

Grammar

Scoring 1 Rudiment of organization apparent, but may be illogical, ineffective or different to understand the sequencing of ideas 2 Satisfactory organization of sections, but the sequencing of paragraphs within sections may be problematic. 3 Effective method of organization for both section and for paragraphs within sections. 1 Minimal use of examples and facts to support the writer’s idea. 2 using some examples and facts to discuss strengths/weakness of some opinions, but may have difficulties (1) choosing appropriate facts; (2) sufficiently explaining those facts; (3) connecting them to present thing. 3 Effective supports the strengths and weakness of one’s opinion; Generally effective use of choice of examples and facts, although some material may be extraneous or not adequately explained 1 Uses simple sentence constructions, but there are still numerous errors (greater than 7).

Sentence Variety

Coherence

Word Count

Conclusion

Spelling

Chinglish Expression

2 Uses simple sentence with minor errors (between 5-7). 3 Uses complex sentence with minor errors (less than 5). 1 Little complex sentences or longer sentences (less than 2) are used 2 Moderate number of complex sentences or longer sentences (between 2 and 4) are used 3 A Effective use of complex sentence construction or longer sentence (greater than 4) 1 Some apparent sequencing of sentences within paragraphs, relying primarily on a limited set of cohesive devices (e.g. first, second, third) and basic connection words (e.g. however, also, because). However, there may be frequent points in which the reader has difficulties understanding sequencing of ideas. 2 Writer sequences ideas, relying primarily on a limited set of cohesive devices; some errors or unclear transitions, but they do not significantly impair understanding of the text. 3 Coherent and logical sequencing of ideas, using a wider range of cohesive devices (e.g. pronominalization, passive, etc;) only minor and occasional errors. 1 Less than 50 words 2 Between 50 and 150 words 3 Around 200 words 1 No conclusion key words found; Conclusion is inappropriate; No conclusion 2 briefly summarized some points 3 It stresses the importance of the thesis statement, gives the essay a sense of completeness. 1 greater than 3 2 within 1 and 3 3 no spelling error 1 greater than 5 2 within 3 and 5 3 less than 2

References Anderson, J. (2005). Mechanically Inclined:Building Grammar, Usage, and Style into Writer's Workshop. Brannon, L., & Knoblauch, C. H. (1982). On students' rights to their own texts: A model of teacher response. College Composition and Communication, 33, 157-166. Britt, M. A., Wiemer-Hastings, P., Larson, A. A., & Perfetti, C. A. (2004). Using Intelligent Feedback to Improve Sourcing and Integration in Students' Essays. Int. J. Artif. Intell. Ed., 14, 359-374. Bureau of Statistis of China, N. (2013). China Statistical YearBook. Burstein, J., Chodorow, M., & Leacock, C. (2004). Automated essay evaluation: The Criterion online writing service. AI Magazine, 25, 27. doi: 10.1002/rcm.5057 Crossley, S., & McNamara, D. (2010). Cohesion, coherence, and expert evaluations of writing proficiency. The 32nd Annual Conference of the Cognitive Science Society. Austin: TX.

Crossley, S. a., & McNamara, D. S. (2011a). Text Coherence and Judgments of Essay Quality: Models of Quality and Coherence. The 33rd Annual Conference of the Cognitive Science Society. Crossley, S. a., & McNamara, D. S. (2011b). Understanding expert ratings of essay quality: Coh-Metrix analyses of first and second language writing. International Journal of Continuing Engineering Education and Life-Long Learning, 21, 170. doi: 10.1504/IJCEELL.2011.040197 Crossley, S. A., & McNamara, D. S. (2012). Predicting second language writing proficiency： The role of cohesion， readability， and lexical difficulty. Journal of Research in Reading, 35, 115-135. Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-metrix: analysis of text on cohesion and language. Behavior research methods, instruments, & computers, 36, 193-202. Haswell, R. (2006). The complexities of responding to student writing; or, looking for shortcuts via the road of excess. Across the Disciplines, 3. Kintsch, W., & van Dijk, T. (1978). Towards a model of text comprehension and production. Psychological Review, 85, 363-394. Lafferty, J., Sleator, D., & Temperley, D. (1992). Grammatical Trigrams: A Probabilistic Model of Link Grammar. Paper presented at the Proceedings of the AAAI Conference on Probabilistic Approaches to Natural Language. Lee, I. (2004). Error correction in L2 secondary writing classrooms: The case of Hong Kong. Journal of Second Language Writing, 13, 285-312. doi: 10.1016/j.jslw.2004.08.001 Leki, I. (1991). The preferences of ESL students for error correction in college-level writing classes. Foreign Language Annals, 24, 203-218. Liu, M., Calvo, R., & Rus, V. (2014). Automatic Generation and Ranking of Questions for Critical Review. Educational Technology & Society, 17, 333-346. Liu, M., Calvo, R. A., & Rus, V. (2010). Automatic Question Generation for Literature Review Writing Support. Carnegie Mellon University, USA: Springer's Lecture Notes in Computer Science Pennebaker, J. W., & Francis, M. E. (1999). Linguistic inquiry and word count (LIWC). Rufenacht, R. M., McCarthy, P. M., & Lamkin, T. A. (2011). Fairy Tales and ESL Texts: An Analysis of Linguistic Features Using the Gramulator. Proceedings of the Twenty-Fourth International Florida Artificial Intelligence Research Society Conference. Shermis, M. D., & Burstein, J. (2003). Automated essay scoring: A cross-disciplinary perspective. 16. The student, the text, and the classroom context: A case study of teacher response, 7 2355 (2000). Thiesmeyer, E. C., & Theismeyer, J. E. (1990). Editor:A System for Checking Usage, Mechanics, Vocabulary, and Structure. Villalon, J., Kearney, P., Calvo, R. A., & Reimann, P. (2008). Glosser: Enhanced Feedback for Student Writing Tasks. Williams, R., & Dreher, H. (2004). Automatically Grading Essays with Markit©. Issues in Informing Science and Information Technology, 1, 693-700.

International Journal of Learning, Teaching and Educational Research Vol. 11, No. 1, pp. 12-21, April 2015

Using Particle Swarm Optimization Approach for Student Engagement Measurement Ming Liu, Yuqi Wang, Hua Liu, Shujun Wu and Chang Li Southwest University Beibei, Chongqing, China

Abstract. Measuring Student Engagement is a difficult task. Previous research has used a cloud-based writing platform, Google Docs, which can store a number of document revisions with timestamps. Engagement measurement algorithm has taken the advantages of each timestamp in a revision and calculated how much time the student spent on a writing task. However, the parameters passed to the algorithm were fixed and hard to determine, for example, how much time means fully engaged or partially engaged. In this paper, we proposed a new student engagement measurement algorithm based on a computational intelligence approach, Particle Swarm Optimization technique, to find the optimized parameters for the engagement measurement algorithm. In the study, the proposed algorithm measures the engagement of two groups of students in two different writing activities (long-term and short term writing activities) carried out in our cloud-based writing platform. The study results show that the correlations between the engagement measurement and student selfreport are high. In addition, it indicates that this approach is robust to measure student engagement in both long-term and short term activities. Keywords: Student Engagement Measurement, Advanced Educational Technologies, Particle Swarm Optimization.

Introduction Student engagement plays an important role in a learning activity. Studies (Fredricks, Blumenfeld, & Paris, 2004) show that a student who is engaged and intrinsically motivated in a task is more likely to learn from an activity and models of school engagement identify three core dimensions: behavioral, cognitive and emotional engagement. ‘Behavioral engagement’, which is the focus of the present study, refers to student participation in school related activities and involvement in any learning tasks such as those being done online (Fredricks et al., 2004). ‘Cognitive engagement’ refers to motivation, thoughtfulness and willingness to make an effort to comprehend ideas and © 2015 The authors and IJLTER.ORG. All rights reserved.

master new skills. ‘Emotional engagement’ includes emotions and interest, such as affective reactions in the classroom towards teachers. These three aspects are interrelated and helpful to understand engagement as a whole. The measurement of behavioral engagement is more obvious because behavioral patterns can be defined, observed and interpreted. Traditionally, student engagement is measured by teachers’ observation (Bulger, Mayer, Almeroth, & Blau, 2008; Martin, 2007). But, this approach is time consuming and subjective. In the era of ‘big’ data, a large amount of student data about their behavior being harnessed to improve learning interactions and to personalize the learning experience can be collected by the system (Tanes, Arnold, Selzer King, & Remnet, 2011). For instance, when a student participates in an activity that is technology mediated, a detailed collection of behavioral events can be recorded. Computer keystroke-logging (Leijten & Van Waes, 2013) or screen capturing (Latif, 2008) allow a detailed account of the behavior of a writer including actions such as starting a new paragraph or deleting a text portion and these are all considered indicators of behavioral engagement. Thus, new computer technology permits the observation and identification of learning events, which can then be examined in relation to other indices of engagement. However, these technologies require specialized setups and often hardware. In the recent year, with the development of the cloud-based online writing platform, such as Google Doc or Wiki, it is possible to capture student’s writing behavior easily by utilizing document revision history (Cole, 2009; Liu et al., 2013). However, the engagement measurement algorithm requires so many predefined parameters, such as the time threshold for full engagement or for partial engagement. Previously, the thresholds are determined by educational experts, which is too subjective. If the thresholds are set too high or too low, it would affect the accuracy of engagement measurement and effect of engagement visualization. Particle swarm optimization (PSO) is a population-based metaheuristics used for stimulating social behaviour such as fish school to a promising position (S. W. Lin, Ying, Chen, & Lee, 2008). PSO is a subset of swarm intelligence which was occurred in the late 1980s to relate to cellular robotic systems, where a number of agents in an environment interact based on local rules. Over the past years, particle swarm optimization technique has lately been illustrated to have the ability to solve complex problems, such as automatic group composition(Y.-T. Lin, Huang, & Cheng, 2010), e-learning problems(Huang, Huang, & Cheng, 2008), automatic test sheets generation (Yin, Chang, Hwang, Hwang, & Chan, 2006). These studies suggested that swarm intelligence is useful for providing high scalability and robust computation. In our study, we use PSO to optimize the engagement measurement algorithm.

Behavioural Engagement Studies of behavioural engagement in learning environments typically use evidence collected by human observers, such as teachers or students (Lane, 2009; Martin, 2007). For example, using scales such as the Student Engagement © 2015 The authors and IJLTER.ORG. All rights reserved.

Walkthrough Checklist, observers such as administrators, instructional supervisors or teachers, have examined the degree to which students exhibit engagement in the classroom, by measuring behaviors such as positive body language, consistency of focus, spoken participation (Jones, 2009). The observer ratings are then compared to simultaneous and anonymous ratings by students of their level of engagement according to the extent to which the work is interesting and challenging, and the degree to which they understand why and what they are learning. Jones (2009) have defined the models of general engagement including behavioral, emotional and cognitive engagement as consisting of three dimensions; intensity, consistency and breadth. Intensity relates to the level of engagement of each student. Consistency refers to how long students remain engaged at high levels throughout the class period and breadth refers to how broadly the class as a whole is engaged. Measuring dimensions of engagement allows teachers to provide differentiated feedback. For example, if the engagement intensity is low, teachers can focus on adding rigor and relevance to expectations and lessons. To date, most of the research on student engagement has occurred in classrooms (Sheldon & Biddle, 1998), yet researchers are increasingly exploring learning theories in web-based activities (Chena, Lambertb, & Guidryb, 2010), social software (2009), smart interactive devices (Blasco-Arcas, Buil, HernándezOrtega, & Sese, 2013) and virtual environments (Bouta, Retalis, & Paraskeva, 2012). ‘Clickers’ (Blasco-Arcas et al., 2013) allowed students to quickly answer questions presented in class. Responses can be anonymized or identified and software programs are usually used to summarize responses and present visualizations in the form of charts. Technology-based tools such as Wiki technology (2009) have been used to support learning engagement. Cole (2009) tested Wikis in a third year undergraduate course to examine the degree to which they supported student knowledge construction, peer interaction and group work. However given the optional nature of this form of technology in the course, students did not contribute to the Wiki as was intended. Thus focus groups were used to examine barriers to uptake rather than the effects of Wikis on student engagement per se. However, a limitation of previous studies is that they have not addressed how to automatically track and analyze student behaviour patterns and present them in a way that is understandable. Given the difficulties identified by previous studies (2009) related to student use of webbased techniques the present study was conducted within a laboratory environment rather than as part of a course.

Engagement Visualization and Measurement

Figure 1: Line-based Visualization: green lines with different thickness show that a user has done several intensive writing in the drafting process. Graphs are copied from (Liu, Calvo, & Pardo, 2013).

Engagement is critical to the success of learning activities such as writing, and can be promoted with appropriate feedback. Tracer is a learning analytic system (Liu et al., 2013) which derives behavioral engagement measures and creates visualizations of behavioral patterns of students writing on a cloud-based application. Figure 1 shows that the Line-based Visualization uses a line to connect the points and the thickness of a line indicates the intensity of the userâ&#x20AC;&#x2122;s behavior during a period of time. This information is derived from Intensitybased engagement measurement algorithm (IbA), where a series represents a line and its weight represents a line thickness. Therefore, the whole graph is made of lines. The weighting process is defined as follows: 1. A hashmap is predefined, where each entry contains a time threshold and a corresponding weight value. For example, (0.5h, 0.8) indicates that the time threshold is 0.5h and its corresponding weight is 0.8. 2. If the duration between neighboring events is less than the shortest time threshold, we assign that corresponding weight to the series. For example, in one month project proposal writing assignment, the following combinations/hashmap: (0.5h, 1), (1h, 0.8), (3h, 0.4) and (12h, 0.2) is considered based empirical experience. For example, if the duration of an activity is 2 hours, we assigned 0.4 as a weight to the series because 3h is the shortest time defined in the hashmap that is longer than 2h. Thus the total engagement score is calculated as the following weighted sum:

Engagement=

n i si

â&#x2C6;&#x2014; wi

(1)

where i is the index of a series, Si is the duration of the series i and Wi is the weight assigned to i.

Particle Swarm Optimization PSO looks through a collection of individual solutions called particles that update iteratively. Each particle at iteration t can be represented by a D-dimensional state đ?&#x2018;Ą đ?&#x2018;Ą đ?&#x2018;Ą vector as đ?&#x2018;Ľđ?&#x2018;&#x2013;đ?&#x2018;Ą = đ?&#x2018;Ľđ?&#x2018;&#x2013;1 , đ?&#x2018;Ľđ?&#x2018;&#x2013;2 , â&#x20AC;Ś , đ?&#x2018;Ľđ?&#x2018;&#x2013;đ??ˇ . Then, to obtain the optimal solution, we define Dđ?&#x2018;Ą dimensional velocity vectors đ?&#x2018;&#x2030;đ?&#x2018;&#x2013;đ?&#x2018;Ą = đ?&#x2018;&#x2030;đ?&#x2018;&#x2013;1đ?&#x2018;Ą , đ?&#x2018;&#x2030;đ?&#x2018;&#x2013;2đ?&#x2018;Ą , â&#x20AC;Ś , đ?&#x2018;&#x2030;đ?&#x2018;&#x2013;đ??ˇ for each particle and determined by its own best previous experience, denoted as pbest, and the best experience of all the particles, denoted as gbest. Particles change velocity based on the pbest and gbest as follows: Vidt ď&#x20AC;˝ Vidt ď&#x20AC;1 ď&#x20AC;Ť c1r1 ď&#x20AC;¨ pbestidt ď&#x20AC; X idt ď&#x20AC;Š ď&#x20AC;Ť c2r2 ď&#x20AC;¨ gbestidt ď&#x20AC; X idt ď&#x20AC;Š

,d=1,2,3â&#x20AC;ŚD

(2)

Where đ?&#x2018;?1 đ?&#x2018;&#x17D;đ?&#x2018;&#x203A;đ?&#x2018;&#x2018; đ?&#x2018;?2 are the learning factors which are commonly set to 2 and đ?&#x2018;&#x;1 , đ?&#x2018;&#x;2 are random numbers distributed uniformly in the range [0, 1]. Then, each particle updates to a new potential answer based on the velocity as: đ?&#x2018;Ą+1 đ?&#x2018;Ą đ?&#x2018;&#x2039;đ?&#x2018;&#x2013;đ?&#x2018;&#x2018; = đ?&#x2018;&#x2039;đ?&#x2018;&#x2013;đ?&#x2018;&#x2018; + đ?&#x2018;&#x2030;đ?&#x2018;&#x2013;đ?&#x2018;&#x2018;đ?&#x2018;Ą

(3)

When the iteration number reaches a pre-determined maximum iteration number, the update process is terminated and the best individual of the last generation is the final solution to the target problem.

PSO enhanced Engagement Measurement Algorithm In this section, we describe the proposed PSO-EM algorithm for predicting the total time a student spent on the writing task. The aim of this study is to optimize the accuracy of the engagement prediction by estimating the best values of an engagement measurement function parameters described above. We used the Matlab to implement this algorithm. The evaluation matrix for SVR is MSE (mean square error). 1

MSE = đ?&#x2018;&#x203A;

đ?&#x2018;&#x203A; đ?&#x2018;&#x2013;=1(đ?&#x2018;&#x201C;

đ?&#x2018;Ľđ?&#x2018;&#x2013; â&#x2C6;&#x2019; đ?&#x2018;Śđ?&#x2018;&#x2013; )2

(4)

MSE is a common evaluation measurement for numeric value prediction, which has been adapted in education (Tang & Yin, 2012). In our study, PSO starts with 20-randomly chosen particles and looks for the best particle iteratively. Each particle is a 6-dimensional vector including three time thresholds and three weights represents a candidate solution. The engagement measurement algorithm is constructed for each candidate solution to estimate its performance. The procedure describing proposed PSO-SVR approach is as follows.

Function PSO-EM () { Initializing PSO with 20 particles and each engagement measurement algorithm with each particle. Evaluating the fitness (MSE) of each particle. For each iteration in 200 For each particle in 20 Calculating the particle velocity and updating the particle Calculating the fitness of the particle by passing the parameters to engagementMeasurement() Comparing the fitness values and updating the local best and global best particle. End End . }

Study In order to evaluate the feasibility of the proposed engagement measurement algorithm, we have conducted a study, where 120 students were writing an individual document in a web-based writing system. This system is developed based on etherpad (http://etherpad.org/), which is an online real-time text editor, letting authors to write a text document, and look all the revision history of the document. Each document revision history has been recorded in a textual database. We need to extract the timestamp of each revision as an input to the engagement algorithm.

Participants and Procedure A total of 120 university students participated in this study. The participants’ age ranged from 20 to 30 years (M: 25, SD: 5) and there were 61 males and 59 females. Those student participants came from different disciplines, including computer engineering and education. They had no prior knowledge of the system and did not participated in any previous related study. We arranged a separate one hour writing activity for 60 education majors (writing a personal best travel experience) while one month writing activity (writing a project proposal) for 60 engineering students. We conducted this study in a controlled environment so that each participant could only write in our system (see Figure 2), thus avoiding the ‘copy-and-paste’ issues. Once the writing activity was finished, each participant was asked to estimate their engagement time in the writing session. The dataset was divided into the training set (n=30) and testing set (n=30) for each activity. We used the training set to train the parameters of the engagement algorithm and testing set to evaluate the performance of the algorithm.

Figure 2: the user interface in the online writing system

Results The correlation among participants and engagement measurement functions is presented in Table 1. This study results show that correlations between the proposed engagement algorithm (PSO-EM) and human are highly correlated (r=.73 and r=.81) in both writing activities. This algorithm outperformed IbA which has moderate correlation (r=.49 and r=.59) with student self-report (Human). We also observed that the student engagement time in the one-hour writing activity is more predictable than in the one month writing activity, because the one-hour writing activity produced less document revisions.

Table 1: Correlation of engagement time One month writing PSO-EM

IbA

PSO-EM

IbA

0.67

Human Self-Report

0.73

0.49

One hour writing

Human

PSO-EM

IbA

Human

0.69

0.81

0.59

After 200 iterations, PSO-EM converges. Table 2 shows that PSO-EM algorithm (MSE:15.88 in one hour;MSE:31.89 in one month) gets lower MSE scores than traditional IbA (MSE:16.13 in one hour;MSE:64.95 in one month) in both writing tasks (one hour and one month writing tasks). Table 2: Performance of PSO-EM Algorithm and Its best parameters. T1 means 1 Time Threshold1 Parameter while W1 means weight1 Parameter

One Hour

One Month

Evaluation Measure

Parameters

Writing Task IbA PSO -EM ibA PSO -EM

MSE

0.5 m

1.0 m

2.0 m

0.33

0.66

16.13

3.30m

4.20 m

5.12m

1.09

2.34

2.89

15.88

0.5h

1.0h

2.0h

0.33

0.66

64.95

3.3h

4.20h

5.12h

1.09

2.34

2.89

31.89

In the one hour writing task, PSO-EM finds the best parameters for this dataset include Threshold1 as 3.30, Threshold2 as 4.20 and Threshold3 as 5.12 minute, and Weight1 as 1.09, Weight2 as 2.34 and Weight 3 as 2.89. In addition, in the one month writing task, the best parameters for threshold are different from those parameters in one hour writing task and the unit is hour. This result indicates that the PSO-EM algorithm is robust to automatically adjust its parameter values based on the dataset or the nature of the task. It also suggests that PSO-EM outperformed the traditional method.

Conclusion and Future Work In this paper, we introduce a novel algorithm, called PSO-EM for engagement measurement, particularly student engagement in a writing activity. This algorithm is based on a computational intelligence approach, called Particle Swarm Intelligence, to find the best parameters for engagement measurement algorithm. Our study result indicates that this algorithm outperformed the traditional engagement measurement method and can automatically adjust the function parameters based on the writing task. We also found that the short-time writing activity (one-week) was more predictable than the long-time writing activity (one-month), since the short-time writing activity produced less revision data for analysis. However, PSO-EM can still perform well in complex revision data due to its robust capability. Our future work will focus on generating real time visualizations based on the engagement algorithm to support individual and collaborative writing.

Acknowledgements This work is partially supported by Chongqing Social Science Planning Fund Program under grant No. 2014BS123, Fundamental Research Funds for the Central Universities under grant No. XDJK2014A002 and No. XDJK2014C141 and No. SWU114005 in China.

References Blasco-Arcas, L., Buil, I., HernĂĄndez-Ortega, B., & Sese, F. J. (2013). Using clickers in class. The role of interactivity, active collaborative learning and engagement in learning performance. Computer & Education, 62, 102110. Bouta, H., Retalis, S., & Paraskeva, F. (2012). Utilising a collaborative macroscript to enhance student engagement: A mixed method study in a 3D virtual environment. Computers & Education, 58(1), 501-517. Bulger, M. E., Mayer, R. E., Almeroth, K. C., & Blau, S. D. (2008). Measuring Learner Engagement in Computer-Equipped College Classrooms. Journal of Educational Multimedia and Hypermedia, 17(2), 129-143. Chena, P.-S. D., Lambertb, A. D., & Guidryb, K. R. (2010). Engaging online learners: The impact of Web-based learning technology on college student engagement. Computers & Education, 54(4), 1222-1232. Cole, M. (2009). Using Wiki technology to support student engagement: Lessons from the trenches. Computer & Education, 52(1), 141-146. Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School Engagement: Potential of the Concept, State of the Evidence. Review of Educational Research, 74(1), 59-109.

Huang, T. C., Huang, Y. M., & Cheng, S. C. (2008). Automatic and interactive elearning auxiliary material generation utilizing particle swarm optimization. Expert Systems with Applications, 35, 2113-2122. Jones, R. D. (2009). Student Engagement: Teacher Handbook. Rexford:NY: International Center for Leadership in Education. Lane, E. (2009). Clickers: can a simple technology increase student engagement in the classroom? Paper presented at the International Conference on Information Communication Technologies in Education, Corfu, Greece. Latif, M. M. A. (2008). A state-of-the-art review of the real-time computer-aided study of the writing process. International Journal of English Studies, 8, 2950. Leijten, M., & Van Waes, L. (2013). Keystroke Logging in Writing Research: Using Inputlog to Analyze and Visualize Writing Processes. Written Communication, 30, 358-392. doi: 10.1177/0741088313491692 Lin, S. W., Ying, K. C., Chen, S. C., & Lee, Z. J. (2008). Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Systems with Applications, 35, 1817- 1824. Lin, Y.-T., Huang, Y.-M., & Cheng, S.-C. (2010). An automatic group composition system for composing collaborative learning groups using enhanced particle swarm optimization. Computers & Education, 55, 1483-1493. doi: 10.1016/j.compedu.2010.06.014 Liu, M., Calvo, R. A., & Pardo, A. (2013). Tracer: A tool to measure student engagement in writing activities. Paper presented at the the 13th IEEE International Conference on Advanced Learning Technologies, Beijing, China. Martin, A. J. (2007). Examining a multidimensional model of student motivation and engagement using a construct validation approach. British Journal of Educational Psychology, 77, 413-440. Sheldon, K. M., & Biddle, B. J. (1998). Standards, accountability, and school reform: Perils and pitfalls. Teachers College Record(100), 164-180. Tanes, Z., Arnold, K. E., Selzer King, A., & Remnet, M. A. (2011). Using Signals for appropriate feedback: Perceptions and practices. Computers & Education, 57(4), 2414-2422. Tang, H.-W. V., & YIN, M.-S. (2012). Forecasting performance of grey prediction for education expenditure and school enrollment. Economics of Education Review, 31(4), 452-462. Yin, P. Y., Chang, K. C., Hwang, G. J., Hwang, G. H., & Chan, Y. (2006). A particle swarm optimization approach to composing serial test sheets for multiple assessment criteria. Journal of Educational Technology & Society, 9, 3-15.

International Journal of Learning, Teaching and Educational Research Vol. 11, No. 1, pp. 22-35, April 2015

A Study of the Development of Courseware and Students’ Learning Effectiveness in Primary Education: Using Three Teaching Techniques as an Example Fang-Chun Ou Overseas Chinese University Taichung, Taiwan Abstract. As Taiwan has carried on its educational reform, many problems have emerged over the past ten years. These issues have to be solved as soon as feasible. Specifically, primary education is facing severity in competition and stern challenges in a fast globalizing world. This study aims to explore Taiwan’s English education so as to find out new approaches to revision and innovation. In Taiwan, most students have to learn English since elementary school. English teachers usually adopt different methods to teach students so as to achieve teaching excellence. Three groups of primary school students participated in a study with three teaching methods involved for learning English as a foreign language (EFL). TPR (Total Physical Response) was employed with the first group, giving instruction and then students responding with body movement. CLT (Communicative Language Teaching) was adopted with the second group, which emphasized interaction and communication genuinely. Conventional teaching method was used with the third group, in which students learned from what teachers taught in class. The pre- and post-test were carried out to investigate which teaching method was the most significant. The present study indicates under controlled conditions that TPR & CLT, proven beneficial in TPR & CLT context, can yield a positive outcome. In contrast, the traditional teaching method has the least progress among the three teaching methods. In addition, the findings of the study support that the participants enhanced in the vocabulary and picture matching of the posttest. The result of this study could be a good demonstration for teachers to provide more options in English learning. Through the © 2015 The author and IJLTER.ORG. All rights reserved.

curriculum, teachers could promote the ability to devise a flexible variety of activities in order to stimulate pupilsâ&#x20AC;&#x2122; learning as well as make them better interested in English. This study has set up a great value for other similar researches and should be replicated with students at various English proficiency levels. Keywords: TPR (Total physical response), CLT (Communicative language teaching), Conventional teaching method, EFL (English as a foreign language).

Introduction English is one of the indispensable languages in modern society, which can be employed to do a great deal of trades between countries as well as spoken to interact with foreigners. It is also an essential bridge that connects people from variety of events. In response to the requirement of international society, strengthening English ability has become an important issue of education. Moreover, with English learning, learners can blend into social and cultural activities in English-speaking countries in good time. Language learners should understand and respect multiculturalism in order to be cosmopolite. Nowadays, being capable of speaking English fluently has become one of the basic requirements in the global village. The purpose of English teaching and learning is to build up learnersâ&#x20AC;&#x2122; ability of communication, increase the motivation and interest of English learning, and develop a global perspective. Additionally, language learners are expected to enhance the ability of handling international matters and conflicts. It has been a quite normal phenomenon cultivating English capability since a very young age, particularly in Taiwan. In line with the government policy to improve international competitiveness, MOE (Minister of Education) stipulates English teaching and learning should be implemented in Grade1-9 Curriculum. According to MOE, the teaching methods should be active and interactive. The content of teaching material should be related to daily life, practical and interesting. By means of diverse teaching materials and activities participation, the four skills including reading, listening, speaking, and writing can be built up gradually, and then be put into practice. ÂŠ 2015 The author and IJLTER.ORG. All rights reserved.

In order to break the myth of grades, the aims of 12-year compulsory education are leading students toward creative learning on the initiative, having knowledge from the learning process, experiencing pleasure, cultivating their own characteristics, and communicating with English. However, the major problem of English education is teaching too much and too difficult. Teachers usually make students remember vocabularies and grammars compulsorily. Therefore, the mechanical drills kill studentsâ&#x20AC;&#x2122; learning motivation toward English. As a consequence, the policies of 12-year compulsory education are set up to teach efficiently, blend information technology into teaching, and encourage students to think and express creatively. In light of this, English ability and practicality are more important than they used to. Additionally, in 2010, Attar and Chopra pinpoint the teaching methodology and approach should keep changing in order to meet the needs of language learning. Namely, how to design effective teaching modes and cultivate students' communicative competence have become the major concerns in English teaching and research. Tracking back to the early period, English teaching mostly put emphasis on Grammar Translation Method and Audio-Lingual Method. The traditional teaching method is drill-oriented, which introduces and practices language knowledge and skills in details. Worse still, students tend to be bored and punctilious gradually. Until 1994, Ministry of Education started highlighting Communication Language Teaching, which aims at meaningful interactions, language skills, genuine material, language ability development, and English communication under different social situations properly. As a consequence, designing diverse teaching techniques as well as appealing activities and courseware should be taken into consideration so as to benefit students by increasing achievement and learning outcomes.

Research Questions 1. Does the intervention in the use of teaching methods help improve elementary school studentsâ&#x20AC;&#x2122; English proficiency? 2. Which type of question (vocabulary, picture matching, and reading comprehension) was influenced most after exposed to these three teaching methods?

Literature Review The advantages of Communicative Language Teaching have been proved and employed successfully in ESL/EFL (English as a Second Language/ English as a Foreign Language) classrooms around the world (Kelch, 2011). Chang (2011) explores to compare the feasibility of Grammar Translation Method and Communicative Language Teaching in English grammar teaching as well as tries to find out which on is more appropriate in Taiwan. In his study, the result shows students benefit more from grammar instruction with Grammar Translation Method (GTM) adoption. With contrast to GTM, Communicative Approach focuses on fluency rather than accuracy. The teacher corrects errors immediately if the scope of the classroom activity is accuracy, but if the scope of the activity is fluency the errors will be corrected later on. As a result, combining both methods might be the best way to improve circumstances in English grammar teaching. Wei (2010) reviews the advantages of Communicative Language Teaching method and analyzes the obstacles of implementation in EFL classroom context. In his study, it provides guidelines for compromising CLT with the conventional teaching approach. Additionally, it recommends some techniques and principles for English teaching implementation in EFL environment.

Teacher Training The main purpose of language education is to enhance the quality of teachers as well as the quality of education. The English teachers should possess professional knowledge related to ELT (English Language Teaching) and be capable of employing varieties of teaching methods. Regarding mid- and long-term teacher training (MOE, 1999), MOE encourages normal universities to establish departments of English education. Besides, school should provide English subgroups, English minors or second specialty students a twenty-credit course of ELT.

Teaching Methods TPR (Total Physical Response) was originally developed by James Asher. In the 1960s, TPR makes good use of physical movements and associates with the theoretical framework of mother tongue. Most importantly, teachers can check young learnersâ&#x20AC;&#x2122; comprehension through their reactions linked to body movements, which reinforce their comprehension ability. ÂŠ 2015 The author and IJLTER.ORG. All rights reserved.

CLT (Communicative Language

Teaching) emphasizes interaction

and

communication in classrooms. In fact, it was a response to Chomsky’s theory (Chomsky, 1965). Chomsky showed linguistic competence is not the mastery of structures, but communication competence in real situation. Teachers should create a wide variety of authentic situations for students to interact with their classmates. Then students have the opportunity to share their individual experience in target language. Most importantly, students gain more self-confidence through practices and keep enthusiastic toward language learning.

Methodology Subjects The target subjects were an unselected convenience sample. Thirty 5th and 6th elementary school students voluntarily participated in this study. They were asked to take the identical pre- and post-test to evaluate the appropriateness of three different teaching techniques (TPR, CLT, conventional teaching) in different classroom settings.

Course Material The researchers created an innovative story that students have never read before. In addition, ten sentences and vocabulary cards were made to emphasize grammar instructions and practices.

Instruction and Testing Procedure Three groups of subjects were administered the pretest to obtain initial scores of the students’ English proficiency. There are three parts in the test. Part one is multiple-choice questions of vocabularies, part two is matching correct pictures according to the story, and the last part is reading comprehension. The actual instruction lasted three hours with three different teaching methods adopted in three different classroom settings, respectively. After the instruction, a posttest was implemented to investigate the differences among the three different teaching methods. The students completed both the pre- and posttest as the requirement. All subjects were given the same test used in pre-test as a post-test.

vocabulary, five are picture matching, and the remaining five are reading comprehension. The result of the test focuses on which teaching technique was the most suitable for primary school students. Table 1 (Test Question Distribution)

Question Categories

Numbers

Percentage

Vocabulary

50 %

Picture matching

25 %

Reading comprehension

25 %

Results The means and standard deviations of the pre-test and post-test scores for the conventional teaching method were presented in Table 2. Table 2 Descriptive Statistics of Pretest and Posttest (Conventional)

N=20 Conventional

Pretest

11.055

Posttest

6.770

A paired-samples T test was conducted to evaluate whether the conventional teaching method increases studentsâ&#x20AC;&#x2122; scores. The results indicated the mean scores for posttest (M= 57, SD= 6.770) was not significantly greater than the mean scores for pretest (M= 25, SD= 11.055), t(9) = -8.677, p= .12 (Table 3). The results revealed there is no effect of the conventional teaching method

adoption. Table 3 Results of Paired Samples T Test

Pair 1

Mean

Conventional Pretest-posttest

Std.

Sig.

-8.677

.12

Deviation -32.50

11.844

The means and standard deviations of the pre-test and post-test scores for Total Physical Response method were presented in Table 4. Table 4 Descriptive Statistics of Pretest and Posttest (TPR)

N=20 TPR

Pretest

7.45356

Posttest

10.28753

A paired-samples T test was conducted to evaluate whether TPR increases students’ scores. The results indicated the mean scores of posttest (M= 76, SD= 10.28753) was significantly greater than the mean scores of pretest (M= 25, SD= 7.45356), t(9) = -11.057, p= .000 (Table 5). The results confirmed the effectiveness and appropriateness of the total physical response adoption. Table 5 Results of Paired Samples T Test

Pair 2

Mean

TPR

Std.

Sig.

-11.057

.000

Deviation

Pretest-posttest

-51.50

14.72903

The means and standard deviations of the pre- and post-test scores of the communicative language teaching method were presented in Table 6. Table 6 Descriptive Statistics of Pretest and Posttest (CLT)

N=20 CLT

Pretest

15.12907

Posttest

5.77350

A paired-samples T test was conducted to evaluate whether the communicative language teaching increases students’ scores. The results indicated the mean scores for posttest (M= 90, SD= 5.77350) was significantly greater than the mean scores for pretest (M= 32, SD= 15.12907), t(9) = -16.900, p= .000 (Table 7). The results confirmed the effect and appropriateness of the communicative language teaching adoption. Table 7 Results of Paired Samples T Test

Pair 3

Mean

CLT Pretest-posttest

Std.

Sig.

-16.900

.000

Deviation -58.00

10.85255

The second question of the present study was the following “Which type of question (vocabulary, matching, and reading comprehension) was influenced most after exposed to these three teaching techniques?” A multivariate analysis of variance (ANOVA) was performed on the data with the three scores (scores of vocabulary questions, matching questions, and comprehension questions) used as dependent variables and Group as the independent variable. The three dependent variable scores were calculated by subtracting test scores of each question type obtained at the beginning of the instruction (pre-test scores) from © 2015 The author and IJLTER.ORG. All rights reserved.

those obtained at the completion of the instruction (post-test scores). The ANOVA for the Group main effect was found to be significant, F (6,50)= 25.515 (Wilks’ Λ = .061), p < .001. As a result, the univariate ANOVAs on each dependent variable were conducted as follow-up tests to the MANOVA. Using Bonferroni method, each ANOVA was tested at the .0167 level (.05/3). There was a significance in the vocabulary question scores, F (2, 27) = 63.224, p < .001, eta squared = .824. The difference in the picture matching questions scores was significant as well, F (2, 27) = 8.113, p = < .001, eta squared = .375. The difference in the reading comprehension questions scores was nonsignificant, F (2, 27) = 25.317, p= .159, eta squared = .652. (Table 8) Table 8 Results of Comprehension Difference Scores by Question Types

Note: adjusted Alpha = 0.0167

Findings The first important finding of this study suggests that the teaching methods, TPR and CLT do enhance elementary students’ English proficiency. The present study demonstrates under controlled conditions that TPR& CLT, proven beneficial in TPR & CLT context, can yield a positive outcome. In contrast, traditional teaching method has the least progress among the three teaching methods. Moreover, the research evidence indicates that explicit, overt physical movements can greatly increase the positive outcome of instruction. To students who just listen to teachers and repeat after them do not possess much comprehension because they do not really understand the context of the course, © 2015 The author and IJLTER.ORG. All rights reserved.

nor do they know how to apply them to the real life. The teaching methods, TPR and CLT can help students become more confident and have more involvement in class. Overall, the findings of the study support that the participants enhanced in the vocabulary part (requiring respondents to select the best word according to the picture) and picture matching (requiring respondents to choose the best sentence describes the pictures) of the posttest. The second important finding of this study deals with the question that which type of test questions was influenced most after the instruction of these three teaching methods. It was found that the participants in this study in fact did tend to use the physical movements to link the meaning of the vocabularies. Besides, the pictures cards do assist them to have better understanding of plots of the story.

Discussion The first results show students achieve better improvement in TPR and CLT classrooms. The reasons are provided as follow. Firstly, during the instruction of TPR, instructors gave a lesson in target language, and students responded with whole body actions. Students were not forced to speak, and instructors waited until students acquire enough language input through listening comprehension, then they would speak out without any fear. Namely, language learning should not involve any stress and the lively interaction could impress the physical response upon studentsâ&#x20AC;&#x2122; mind. Secondly, during the instruction with CLT teaching method, students were taught the story along with picture cards, and they were asked to communicate with instructors. By means of these, more interactions were expected. As a result, students could keep the story in mind easier and more efficient. Lastly, during the instruction with the conventional teaching method, instructors taught by simply reading aloud the story lines and made explicit translation. Compared to TPR and CLT, the conventional teaching method was not lively that the students only sat tediously and sometimes did not catch what were taught thoroughly. The second results indicate that students achieve better toward vocabularies and picture matching than reading comprehension. The reasons are explained in ÂŠ 2015 The author and IJLTER.ORG. All rights reserved.

detail.

Vocabulary Vocabulary picture cards were created to employ during the instruction. Students saw the picture at the first glance and were encouraged to guess the meaning of the vocabulary. Then, vocabulary card was revealed and students were requested to repeat after the instructors and sounded it out. Moreover, an exciting game was designed for students to play in class. As a consequence, students learned through the action and memorized novel words more easily and efficiently.

Picture Matching The story picture cards were used to associate and connect the pictures with the content. Through viewing picture cards, students found the key words from story lines, which enhanced their visual-mental correspondence. While having an exam, students were easier to reason the story and match the right pictures.

Reading Comprehension Instructors invented the story taught in class, and it has never been heard before. Although students learned with the visual aid of picture cards and some exciting games were set up especially for them, most of the students still had difficulties reading as well as comprehending long paragraphs. As a consequence, while having a test, students expressed they guessed instead of answering conscientiously. When it comes to TPR method, some recommendations are provided as follow. First of all, realia is a good choice. Teachers can make good use of objects from the real life to make the instruction more clearly and attract more attentions. In addition to real objects, picture cards and posters are helpful as well. In fact, students are able to associate the images of picture cards with new vocabularies easily, which makes them have less pressure when memorizing new words. Secondly, physical movement is strongly recommended. In class, the actions demonstrated by instructors make the commands or instructions more meaningful and clear. Moreover, students, especially young children, have more interests in learning when they leave their seats and do some actions around. Thirdly, instead of using a long sentence to direct studentsâ&#x20AC;&#x2122; behaviors, teachers can use combinations of commands. For instance, teachers give one command ÂŠ 2015 The author and IJLTER.ORG. All rights reserved.

first, and then do the action spontaneously. Gradually, when students are familiar with the commands, teachers can add more commands at one time. However, do not add more than three commands at one time because students might get confused while receiving the signals. Most importantly, teachers can observe students’ comprehension easily and directly. If students can correctly do the action after the command, then they do really comprehend what teachers teach in class, which makes them feel confident and self-achieved. With regard to CLT, there are some suggestions provided as follow. Situational Language Teaching (SLT) can motivate students’ interests in learning. When teachers introduce a new target language in words or phrases, instead of translating them into students’ native language, teachers can demonstrate the lessons through the use of realia, pictures or pantomime. Teachers may also use intonation, rhythm, and concert pseudo-passiveness to get students’ attention and motivate their interests in the lesson. Initially, students are really dependent on their teachers. After teachers’ questions, students tend to make themselves understand first, and then they are encouraged to answer in front of the whole class. Gradually, with more practices, they may be more independent and have greater security. Meanwhile, students can also listen to other’s opinions, and learn from each other little by little. In fact, the interaction goes both ways, from teachers to students and from students to teachers. Although students might make mistakes, teachers usually employ various techniques to get students to self-correct. Namely, the feeling of security is enhanced by many opportunities of the cooperative interactions with their fellows and teachers. By means of this, teachers evaluate not only students’ accuracy, but also their fluency. Teachers act as advisors or co-communicator. Rardin (1988) mentioned language learning is neither student-centered, nor teacher-centered, but rather teacher-student centered. The CLT method makes students feel proud to use the knowledge to express in different languages. Two reasons are provided to explain why these three teaching methods were chosen in the first place. First, TPR and CLT are the most popular teaching methods adopted in educational institutions. Most instructors consider students’ interest in learning foreign languages is the priority. When students feel interested in English, they will feel more comfortable and easy to communicate with others by using a foreign language. Next, the traditional teaching method is © 2015 The author and IJLTER.ORG. All rights reserved.

still employed now and then. Under the circumstance, most language learners deem memorizing vocabularies a rather tough task; needless to say, speaking English causes pressure and anxiety. Worse still, it surely lessens learners’ motivation toward English learning.

Teacher Training During the past decade, the communicative language teaching approach has been recommended especially for language teachers because of the essential and emphasis of language use in foreign/second language classrooms (Mangubhai et al., 2005). In addition, Li and Yu (2001) have identified the communicative language teaching method has improved communicative ability of language learners in which the conventional teaching approach has been demonstrated unsuccessfully. However, due to the lack of sufficient teacher training in CLT, teachers usually do not know how to implement CLT as well as do not possess confidence in English speaking capabilities to carry out the communicative approach (Butler, 2011). Specifically, most language teachers lack of this kind of training and they are often afraid of “losing face” or feel embarrassed when making errors or when they are not capable of answering students’ questions promptly (Park, 2012). In light of the significance, Carrier (2003) points out the different teaching approaches should be demonstrated and highlighted through direct explanation, explicit teacher modeling, and extensive feedback in teacher training programs in terms of the implementation in language classrooms. Specifically, in the environment of English as a foreign language in Taiwan, the supply of language input and practice opportunities are insufficient for the learners to become immersed. Therefore, teachers should value process-oriented instruction more highly than content-oriented or grammar-oriented instruction because it is beneficial for students to become independent learners. The language teacher should also bear in mind that elementary school children are not mature enough to take full responsibilities for their own language learning. Therefore, children’s proficiency levels and their cognitive maturity would determine the types of activities (strictly-controlled ones, semi-guided ones, or free communicative ones) the teacher puts into practice in a communicative classroom.

Limitation The sample size is small, which causes the effect of the experiments was not statistically significant, so the results cannot be completely generalized to young EFL learners from other areas. In addition, time duration of each class is two hours. Within this short period of instruction, it was at time difficult for the instructors to circle around the classroom while the activities were conducted since the instruction involved observing the class and providing assistance. Consequently, language learners would benefit from the instruction with sufficient guiding period.

Pedagogical Implication The result of this study could be a good demonstration for teachers to provide more options in English learning. Through the curriculum, teachers could promote the ability to devise a flexible variety of activities in order to stimulate pupils’ learning as well as make them better interested in English. This study has set up a great value for other similar researches and should be replicated with students at various English proficiency levels. For instance, in addition to TPR and CLT, The Direct Method, Community Language Learning, and Reciprocal teaching are strongly recommended as the integrated teaching method to promote the teaching process. This study explores Taiwan’s education to find out new approaches to revision and innovation. According to Jarvis and Atsilarat (2004), new teaching approaches have been addressed so as to diversify the approaches in existence to accomplish

global

innovation.

As for the future

investigation, more

breakthroughs in curriculum and instruction need to be put into consideration, in order to gain an overall picture of the optimal outcomes of education.

References 教育部 (1999).

國小英語師資培育檢核相關報導。 Retrieved

from

http://content.edu.tw/junior/english/scedu/rimage/r04.htm. 教育部 (1999). 培訓國小英語師資完整計畫方案。 Retrieved from http:// npl.ly.gov.tw/npl/report/880517/14.pdf. Attar, M. & S. S. Chopra (2010). “Task-Based Language Teaching in India”. MJAL 2:4. © 2015 The author and IJLTER.ORG. All rights reserved.

Butler, Y. G. (2011). The implementation of communicative and task-based language teaching in the Asia-Pacific region. Annual Review of Applied Linguistics, 31, 36-57. Carrier, K.A. (2003). NNS teacher in Western-based TESOL programs. ELT Journal, (3), 57- 242. Chang , Shih-Chuan. (2011) “A Contrastive Study of Grammar Translation Method and Communicative Approach in Teaching English Grammar” English Language Teaching, Vol. 4, No. 2. Published by Canadian Center of Science and Education. Chomsky, N. (1965). Aspects of the Theory of Syntax. Cambridge, MA: MIT Press. Jarvis, H & Atsilarat, S. (2004). Shifting paradigms: from a communicative to a context-based approach. Asian EFL Journal, (6), 4-8. Kelch, K. (2011). Curriculum development in English language teaching: Innovations and

challenges

for

the

Asian

context.

International

Journal

Organizational Innovation (Online), 3(3), 22-42. Li, D. (1998). It's always more difficult than you plan and imagine: Teachers' perceived difficulties in introducing the communicative approach in South Korea. TESOL Quarterly, 32 (2), 677-703. Mangubhai, F., Marland, P., Dashwood, A., & Son, J. B. (2005). Similarities and differences in teachers' and researchers' conceptions of communicative language teaching: Does the use of an educational model cast a better light? Language Teaching Research, 9(1), 51-86. Park, S. M. (2012). Communicative English Language Teaching in Korea. Humanising language teaching, 14(6), 1-6. Wei, H. (2010). Communicative Language Teaching in the Chinese Environment. USChina

Education

Review,

7(6),

78-82.

Retrieved

March

15,

2011,

from

http://eric.ed.gov/PDFS/ED511286.pdf. Yu, L. (2001). Communicative Language Teaching in China: Progress and Resistance. TESOL Quarterly, 35(1), 194-198

International Journal of Learning, Teaching and Educational Research Vol. 11, No. 1, pp. 36-52, April 2015

A Comparative Examination of Teacher Candidates’ Professional Practicum Experiences in Two Program Models Nancy Maynes, Anna-Liisa Mottonen, Glynn Sharpe and Tracey Curwen Nipissing University, North Bay, Ontario

Abstract. This paper reports on one aspect of a larger study, examining the relationship between teacher candidates’ self-reports of knowledge and confidence related to many key areas of professional practice. Survey information was provided by concurrent and consecutive bachelor of education students. Perceptions of professional gains through the practicum were examined. Students who are studying education through a concurrent program feel that they have acquired significantly more professional background about teaching through practicum experiences than students acquiring a comparable degree though a consecutive route. As the practical applied knowledge that students acquire through practicum experiences is essential for teacher development, this finding is relevant, especially as each of these programs is undergoing structural changes as a reflection of new provincial directions about teacher education. The results of this study demonstrate that the amount and placement over time of practicum provided in a teacher’s pre-service program matters to the level of professional expertise they feel that they have acquired overall. Keywords: practicum, consecutive education programs, concurrent education programs.

Introduction This paper reports on a study regarding whether or not pre-service teacher candidates feel knowledgeable and confident in the acquisition of skills they need to teach in their own classrooms at the completion of their respective teacher preparation programs. The study contrasted responses from teacher candidates who completed their teacher preparation programs in different models. One group graduated through an eight month program, involving 13 weeks of classroom practicum time; the second group graduated with a 5 year concurrent education degree, including 19 weeks of classroom practicum. The focus of this study is on teacher candidates’ perceptions of what is gained through practicum experiences in the classroom. We investigated how effective

in some tasks new teachers perceive themselves to be as a direct result of what they have learned through practicum experiences.

Background Theories may provide the knowledge that teacher candidates require to work effectively with students in the classroom. However, without opportunities to apply these theories to practice during practicum time, candidates may lack the necessary confidence to address new contexts with equal effectiveness, and they may lack the pedagogical content knowledge to determine strategy efficacy as they encounter new situations early in their career. Practicum time in a teacher education program is typically designed as a professional internship of short duration, strategically placed in the teacher candidatesâ&#x20AC;&#x2122; professional program. The practicum allows the teacher candidate to try out ideas that they have learned in courses in the context of a classroom where a certified teacher can act as a mentor for them. However, not all teacher preparation programs provide the same amount of classroom practicum experience for teacher candidates. In the jurisdiction where this study took place, teacher candidates are required by their accreditation body to acquire a minimum of 12 weeks of successful practicum experience. Success in the practicum is assessed by the professional judgment of the mentor teacher, who is referred to as an associate teacher (AT) in this jurisdiction. In this study, however, two paths to acquiring the professional teacher accreditation are examined in relation to the perceived impact of the practicum on knowledge and confidence of the new teacher. Students acquiring their accreditation through a consecutive program route in this jurisdiction engage in 13 weeks of practicum (i.e., one week more than required by the local accreditation body), while those who acquire their accreditation through the concurrent program route acquire 19 weeks of practicum (i.e., 7 weeks more than required by the local accreditation body). Additionally, the 19 practicum weeks in the concurrent program are distributed across the 5 years of the program, while the 13 weeks of the consecutive degree route are spread across 8 months. While we acknowledge that the quality of the practicum experience each teacher candidate may experience can be vastly different due to many circumstances, our study focuses solely on examining perceptions related to how the length and placement of the experience may have an instructional impact. As teacher candidates, prospective teachers enter the professional arena through practicum experiences; however, they are often unequally exposed to many learning opportunities (Beck, Kosnik & Rowsell, 2007). It is logical to assume that more time in a practicum context would allow more exposure to a greater variety of learning opportunities. Many of the learning opportunities that a pre-service teacher candidate may have during any practicum may be wholly dependent on the skills and resources of the teachers to whom they are assigned for their practicum. Additional practicum time may allow new teachers to have otherwise unavailable exposure to strategies utilized by experienced teachers, and they may lack contextualized opportunities to apply their course-based knowledge in contexts that would allow the teacher candidate to develop

confidence in their ability to use these strategies if they have little or no time to see them in operation and to adapt theoretical ideas to pragmatic contexts. Therefore, the current study provides us with a benchmark of current reports of knowledge and confidence acquired through practicum experiences on which to base program design decisions for this aspect of teacher preparation. Additionally, in the jurisdiction where this study is taking place, the government has recently made significant changes to accreditation criteria, which will come into effect in fall of 2015. In response to the demands for new program designs in the accreditation program for teacher certification in this jurisdiction, many accrediting institutions are considering the elimination of the concurrent program route and retaining the single option of a 2-year consecutive program. This study may shed some light on the efficacy of this decision as it relates to decreased opportunities for longer program embedded practica. Teacher preparation programs include a combination of course work in a university setting, and internship style practicum placements in classroom settings. In the jurisdiction where this study was completed, practicum placements are arranged in any of 52 school boards in the province. Teacher candidates are able to identify any three of these school boards as areas where they might ultimately apply for a teaching position. Then, program placement officers approach school boards to arrange the number of placments required in their area. Usually, school boards have employees who are then responsible for placing the teacher candidate in a specific classroom for a specific placement block. As this university offers two routes to the completion of the same bachelor of education (B.Ed.) degree, with two approaches to the placement and differences in the total amount of time provided for the practicum, we identfied the need to compare teacher candidatesâ&#x20AC;&#x2122; perceptions of the relative value of these differences in providing them with the skills and strategies needed to support their developing professional skills to prepare to be successful with the role of teacher. The skills that were identified for this aspect of the larger study were selected because, while some theory for each skill can be provided in the context of their courses, each skill could reasonably be expected to develop more fully if teacher candidates had contextualized opportunities in schools to use these skills and to consider the impact of their practices in relation to the outcomes they achieved. Six skills were identified by researchers in this category of professional practice. They include: the ability to manage a classroom; the knowledge and confidence to interact with parents; the knowledge and confidence to interact with school and board administrators; the ability to manage difficult student behaviours; the ability to deal with difficult situations; and the knowledge and confidence to address the learning needs of all children.

Literature Review During the past 15 years there has been a considerable amount of intensive investigation into the value and learning afforded to teacher candidates whose professional preparation program provides opportunities for them to hone their theoretical course knowledge by participating in classroom placements, usually referred to as practicum experiences, or collectively as practica. While we were able to find many studies related to the perceived value of teacher practicum experiences, there seems to be an absence in the professional literature regarding investigations of the relative perceived value of different approaches to providing the practicum experience and the perceived value of different amounts of practicum experience. It seems reasonable to assume that more time in a classroom practicum placement is likely to provide more opportunities for the teacher candidate to gain a wider variety of professional skills, but there is a dirth of literature about existing programs to support this contention. Much of the existing research literature about teacher practicum placements addresses perceptions of how effective this experience is as a contributor to the overall professional preparation of a new teacher. A study by Brouwer & Korthagen (2005) confirmed the role of the practicum in the overall development of competent teachers. While both classroom theory and practicum experiences were found to be contributors to a new teacherâ&#x20AC;&#x2122;s development, the practicum in a school context was more influential than the course components of the teacher education program on the development of teaching competence. However, the nature of the practicum has also been found to matter when teacher competency are the desired outcome. In a study by Beck, Kosnik, and Rowsell (2007), researchers identified the need for more focus in the practicum on practical issues related to the daily tasks of functioning in a classroom. In this study, teacher candidates identified six characteristics or skills needed to be provided and developed in their preparation programs to prepare them to teach, including: theoretical understanding, practical knowledge and skills, comprehensive program planning ability, knowledge of what must be done in the first few weeks of school, understanding and skill in assessment and evaluation, and knowledge of how to implement effective group work. It is interesting to note that five of these six characteristics relate to implementation practices that might be expected to develop in teacher candidates during their practicum placements, even though the participants in the study also identified the need to have theoretical understanding. It seems clear from this study that prospective teachers recognize and value the theoretical aspects of the preparation program to help them understand what they should do, but they value the practical experiences of the practicum to show them how and when to do these things. The Brouwer and Korthagen (2005) study also demonstrated that by gradually increasing student teaching activity complexity, by increasing cooperation among students (triads of student teachers), cooperating teachers, and university supervisors, and by alternating between student teaching and college (in-class) sessions, teacher education programs allowed student teachers to relate theory and practice. This need for balance between the course theory and the practicum experiences is supported

by the research of Ng, Nicholas, and Williams (2010). This research revealed that pre-service teacher beliefs may also be influenced by placement experiences, suggesting that placements may be an important factor in shaping beliefs about teaching and teaching efficacy. Also, this study argued that teacher education programs should strive to improve pre-service teachers’ teaching efficacy, since efficacy leads to improvements in teaching ability. Schultz (2005) provides support for the concept of day-to-day problem solving capacity development through practicum learning. The study highlighted the need for teacher preparation to support new teacher inquiry to help teacher candidates use problem solving approaches when they face the day-to-day challenges in a classroom. However, other research shows that the type of school where a practicum takes place influences the learning that a teacher candidate acquires from the practicum. Results of a study by Ronfeldt (2012) demonstrated that teachers who had field placements at easier-to-staff schools were more capable of improving students’ test scores and also more likely to remain teaching in challenging city schools during their first five teaching years. Thus, according to the results of this study, teacher education programs should consider assigning pre-service teachers to field placements at easier-to-staff schools. This study also emphasized the importance of identifying what and how pre-service teachers learn at easier-to-staff schools. The authors argued that it may be that these schools have many characteristics of overall effective schools where good teaching and learning flourish, such as high quality administration and support, professional staff relations and collegiality, and more experienced teachers (Ronfeldt, 2012). It seems logical that exposure to such contexts would influence a teacher candidate’s learning about how to teach well. The research literature about practicum experiences is also very clear about two other key findings. First, the structure of a practicum matters to what can be learned from it. Second, but far from less important, is the nature of the relationship between the teacher candidate and the classroom teacher who hosts their practicum placement is critically important to how successful that placement will ultimately be, as measured by the teacher candidate’s perceptions of their learning in a classroom context. In the context of this study, practicum placements have some provincially mandated expectations about the amount of placement required for certification. However, there is no requirement outside of that minimum time that identifies how or where a placement is structured. Therefore, teacher candidates may be in a position to take advantage of opportunities that have been shown to have considerable professional learning value. For example, in a study by Bryan and Sprague (1997), teacher candidates reported their perceptions of the value of having practicum opportunities to teach in a foreign country. First, the placement was helpful with respect to initial hiring by helping teacher candidates to obtain a teaching job and to improve the experience of initial teaching interviews. Second, retention was positively affected by the foreign

country placement, as demonstrated by the 80% retention rate compared to the 50% retention rate characteristic of typical populations of new teachers. Third, attitudes toward pupils were improved by the placements in foreign countries, in that students reported developing more respect for individual differences among students. Fourth, practicum experiences in foreign countries improved attitudes regarding second languages. Teacher candidates’ appreciation for and patience with speakers of second languages increased. Fifth, overseas experiences increased curriculum choices, as teacher candidates were able to use curriculum units they had previously used overseas. Sixth, overseas experiences increased teaching flexibility, since teacher candidates often had to create interesting lessons with limited materials. Seventh, overseas experiences increased the variety of teaching strategies used by teacher candidates, since they had to learn to understand different learning styles and adapt their teaching to them (Bryan & Sprague, 1997). The perceptions that these foreign experiences helped teacher candidates manage learning for a diverse group of learners is related to the present study’s investigation of perceptions of the ability to manage classrooms, manage difficult behaviours, deal with difficult situations, and address the learning needs of all children. Similar perceived advantages of foreign practicum placements are also evident in several other studies (Grierson & Denton, 2013; Maynes et al., 2012; 2013). The Grierson study documents how participating in an international practicum in rural Kenya supported the development of global perspectives in Canadian teacher candidates. Through analysis of data gathered through a post-practicum focus group, individual interviews, and written reflections, the participants demonstrated how they came to recognize the significant assets of those who live in this developing country, in lieu of perceiving their differences as deficits. Relating their insights to the literature reveals how their experiences have prepared these teacher candidates to provide global education and to differentiate instruction in response to the needs, interests, and background experiences of diverse student populations. The findings of this study provide support for international field experiences that include cultural immersion and collaborative reflection, and document how the provision of this practicum in association with the Canadian NGO Free the Children provided these opportunities and had the potential to increase participants’ teaching confidence and competence. Since both sets of participants in the present study had opportunities to participate in international practica, their skill sets and personal perceptions of knowledge and confidence may have been influenced by these professional experiences, as well as by the timing and focus of other practica. Another structural variation in a teacher candidate’s professional practicum experience that has been found to have a strong impact on professional preparation is participation in practicum experiences with a peer. In a 2003 study that compared teacher candidates’ perceptions of practicum experiences when they were alone in the classroom practicum, with a second group who were partnered with a peer in the same classroom, they found that teacher candidates preferred the pairing situation for its perceived advantages to their

learning (Bullough at al., 2003). In the paired context, teacher candidates reported more opportunities and experience with creating lessons that were more innovative; improved classroom management; and the paired participants felt that the students in their classrooms learned more material and learned it more quickly. When the pairing is done between a more advanced teacher candidate (i.e., in a later year of a multi-year preparation program) and a teacher candidate newer to the professional preparation context (Grierson at al., 2011) the newer teacher candidates report positive learning through the pairing and mentoring which they received from more experienced teacher candidates when they participated in a paired practicum peer mentorship program. The majority of these participants perceived the mentorship program enhanced their confidence and professional growth, although some challenges were also acknowledged. Clearly pairing teacher candidates in their practicum, for different purposes, has perceived learning advantages for the participants. The nature of the relationship between the teacher candidate and the classroom teacher who hosts their practicum placement has been found to be critically important to the perception of the teacher candidate about how valuable their placement has been to their professional preparation to teach. A study by Korthagen, Loughran, & Russell (2006) attempted to identify central principles that can be used to create teacher education programs and practices that address teacher candidates’ and teacher educators’ expectations, needs, and practices accruing from a teacher preparation program. By analyzing three pre-service teacher education programs (one from each of The Netherlands, Canada, and Australia), the researchers identified seven principles of practice for teacher candidate learning and for guiding change and improvement in teacher education programs. The seven principles identify requirements for enhancing the process of learning about teaching. These seven principles include the perceptions that: learning about teaching involves continuous conflict and competition among various demands; knowledge about how to teach should be perceived as a subject that is yet to be created, rather than as an already created subject; learning about teaching means that the teacher’s focus must be on the learner, not the curriculum; teacher candidate research enhances learning about teaching and teacher candidates can guide their own professional development by conducting research on their own teaching practice; learning about teaching requires that those learning to teach work closely with peers, in horizontal, not vertical relationships; meaningful relationships must exist among the schools, universities, and teacher candidates to promote learning about teaching; and, to enhance learning about teaching, teacher educators should model the teaching and learning approaches used in the teacher education program in their own practice. All of the principles are based on learning from experience. It is interesting to note that at least five of these seven principles of an effective teacher education program can only be developed fully within the context of a successful practicum component of the program. Furthermore, the researchers note that the principles are interconnected, and improvements to all of them consecutively will be the most effective approach to create programs that address the expectations, needs and practices most suitable for both the new

teachers and their students. The principles reflect three major program (change) components: perceptions of knowledge and learning that guide teacher educator practices, program structure and specific practices, and staff and organization quality (Korthagen, Loughran, & Russell, 2006). Each of these principles also connects to the nature of the practicum and to its role in extending the theoretical learning of a course into the situated learning of practice in the classroom. Other studies that highlight the critical nature of the relationship between the hosting teacher in the classroom and the teacher candidate help to identify the specific behaviours that support successful practicum experiences. Beck and Kosnik (2002) identify seven components of the relationship between the classroom teacher (often referred to as the associate teacher or AT) and the teacher candidate including: the provision of emotional support by the AT; having a peer relationship characterized by mutual respect (as opposed to a supervisory one) with the AT; opportunities for ongoing collaboration with the AT in planning but independence in teaching a lesson; room to be flexible with the content and the methods they use to teach; feedback from the AT; opportunities to observe good teaching by the AT; and a demanding but not excessive workload while the teacher candidate is on a practicum placement. This study highlights the complexities and the necessity of providing opportunities for teacher candidates to develop their skills to interact productively with other professionals and to deal with difficult situations in a school context. It does not, however, address how such skills are developed in teacher candidates. The need for the development of such positive relationships is supported by the work of Evelein, Korthagen, and Brekelmans (2008) in another study which found that new teachers have much lower measures of need fulfillment in the early stages of their careers than more experienced teachers. This difference might be attributed to more skill, and therefore more success, in dealing with day-to-day interactions that form the basis of classroom implementation. This study is supported by the work of Ferrier-Kerr (2009) who found that the professional relationships between the AT and the teacher candidates are based on several factors, including: personal connections; interpretation or understanding of respective roles (of the AT and the teacher candidate), the ATâ&#x20AC;&#x2122;s style of supervision; and, engagement in reflective practice. Grundoff (2011) studied first year teachersâ&#x20AC;&#x2122; perceptions of how their practicum experiences helped them prepare for early career teaching. Findings supported the importance of the practicum in developing contextualized skills of the profession but found that some practicum features supported skill development while other features hindered development. Participants in this qualitative study found that the practicum had many differences from the reality of actually teaching in their own classrooms. Differences that facilitated the transition of a teacher candidate into the role of teacher included feeling like they were part of the school community once they were teachers; they felt respected by other

teachers and by students. They also valued having time to develop relationships with the children. They enjoyed the increased sense of autonomy. However, they also found some differences between the practicum and their professional teaching roles that they felt had hindered their transition into their professional teaching roles. Differences that hindered the transition included a feeling of shock and anxiety upon beginning to teach due to a discrepancy between expectations and reality, which mostly had an impact on teachers for their first few weeks of teaching. Participantsâ&#x20AC;&#x2122; transition was disrupted in two areas due to the mismatch. First, new teachers did not have a clear understanding of what they had to do at the beginning of the year. Second, new teachers did not recognize the size and scope of teaching. Practicum placements were limited to practicing and developing skills within the context of the classroom. As new teachers, they were overwhelmed by their responsibilities outside of the classroom. As a result, student teachers underestimated the range and amount of work, and were frequently tired as new teachers. Findings suggested that teacher preparation programs should include a practicum at the beginning of the year so that student teachers can experience what needs to be done firsthand. Also, the authors argued that practicum roles, relationships, and sites should be re-evaluated. Specifically three key recommendations resulted from this study, including: practicum should be viewed as an opportunity for collaboration so classroom teachers should have a larger role in teacher education programs, and schools rather than classrooms should be recognized as placement sites. By providing teacher candidates with a school-situated practicum, the candidates can gain exposure to a variety of teaching styles and also have embedded opportunities to experience the out-ofclassroom aspects of this professional role during practicum times. Furthermore, the authors of the study reported that, with this structure governing practice teaching, teacher candidates could experience more interaction with administrators by being required to attend meetings and perhaps participate in more interaction with parents. Each of these prior research studies informed the selection of questions we used to structure the current study. While the current body of literature about the importance, nature, location, relationships, and structure of the practicum as a component of a teacher preparation program has been examined, the comparative perceptions of these characteristics across programs routes toward a B.Ed. degree in the local jurisdiction does not appear to have been studied.

Method Participants. Participants in this study were from both the consecutive and the concurrent programs at three campuses from one Northern Ontario University, during the 2011-2012 acadenic year. A total of 212 respondents (25 males,186 females, 1 gender not reported) completed the survey and were included in the study. Respondentsâ&#x20AC;&#x2122; ages ranged between 18 and 58 years old (M = 23.18, SD = 4.91). Respondents were completing or had completed a consecutive teacher

preparation program (n = 81) or were completing or had completed a concurrent teacher preparation program (n = 131). Demographic data were collected to identify the details of each respondent’s program route and the stage of completion of their teacher preparation. Of the 131 concurrent education respondents, 31 were in the final year or recently graduated from their teacher education program. Twenty-one respondents had previously graduated. Only data from those students completing the final year of their B.Ed. degree or recently having completed this degree were included in this aspect of the study. Research Questions. Therefore, the 2 key questions for this aspect of the larger study are: 1. Do the additional 6 weeks of practicum experience make a perceived difference in the level of knowledge and confidence of the teacher candidate?; and 2) Do student teachers perceive that the distribution across time, of the practicum experiences, influences their knowledge and confidence as teachers? Measures. Demographics. Several types of demographic data were collected in this survey to support comparisons across groups. Data about age, gender (0 = male, 1 = female), the participants’ current status in the education program (i.e., concurrent or consecutive program; their year of graduation from consecutive or concurrent), were collected for descriptive information and to allow investigation of relationships between demographics and dependent variables. Knowledge and Confidence. A total of 6 questions developed by the researchers was used to assess teaching knowledge and confidence which participants attributed to having related practicum experiences. Each of the 6 questions focused on pre-service teachers’ perceptions of how the practicum may have helped them develop abilities to address common classroom tasks. Questions were responded to on a 5 point scale from 0 = definitely not to 4 = definitely. The 6 questions were summed to obtain an overall total score that could range between 0 and 30; higher scores indicated the perception that more knowledge and confidence was gained from the practicum experience. Internal consistency was calculated for the following six items related to the question “How well do you think your practicum placements have prepared you to”: manage a classroom, interact with parents, interact with administrators, manage difficult behaviours, deal with difficult situations, and address the learning needs of all children? Cronbach’s alpha demonstrated the measure (6 questions) to be highly consistent (α = .89). Procedure. An invitation to participate in a comprehensive study of pre-service teachers’ confidence and knowledge was posted on an existing Facebook group designed to promote professional support amongst teacher candidates. A brief description of the purpose of the study was provided. This site included a link to the participant information letter. Those who were interested in the study as

potential participants followed the link to the information sheet which provided all informaton necessary for informed consent. Potential participants could agree to continue or could exit the program, after reading the introductory letter and examining the informed consent form. Completion of the questionnaire indicated each respondent’s agreement to participate in the study. One reminder of the opportunity to participate in the survey research was posted on the Facebook site approximately one month after the study was first advertised. Data collection was completed over a two month period. Completion of the entire questionnaire required approximately 15 minutes. Only those questions related to perceptions of the value of learning as a direct result of the practicum experiences were analysed for this subcomponent of the larger knowledge and confidence study.

Results Independent samples t-tests were conducted to compare participants from the consecutive and concurrent education programs on their average responses to each of the six survey questions. For the question, “How well do you think your practicum placements have prepared you to manage a classroom?” results demonstrated significant differences between consecutive and concurrent program participants, t(207) = 2.018, p = 0.045, with concurrent program participants (M = 3.30, SD = 0.89) scoring, on average, higher than consecutive program participants (M = 3.05, SD = 0.82). For the question, “How well do you think your practicum placements have prepared you to interact with parents?” results did not demonstrate significant differences between consecutive and concurrent program participants, t(206) = 0.568, p = 0.571, with concurrent program participants (M = 1.45, SD = 1.30) scoring comparably to the consecutive program participants (M = 1.35, SD = 1.24). For the question, “How well do you think your practicum placements have prepared you to interact with administrators?” results did not demonstrate significant differences between consecutive and concurrent program participants, t(207) = -1.429, p = 0.154, with concurrent program participants (M = 2.20, SD = 1.24) scoring comparably to the consecutive program participants (M = 1.95, SD = 1.25). For the question, “How well do you think your practicum placements have prepared you to manage difficult behaviours?” results demonstrated significant differences between consecutive and concurrent program participants, t(207) = 2.205, p = 0.029, with concurrent program participants (M = 2.98, SD = 1.06) scoring, on average, higher than consecutive program participants (M = 2.65, SD = 0.98).

For the question, “How well do you think your practicum placements have prepared you to deal with difficult situations?” results demonstrated significant differences between consecutive and concurrent program participants, t(207) = 2.265, p = 0.025, with concurrent program participants (M = 2.80, SD = 1.08) scoring, on average, higher than consecutive program participants (M = 2.46, SD = 1.08). For the question, “How well do you think your practicum placements have prepared you to address the learning needs of all children?” results demonstrated significant differences between consecutive and concurrent program participants, t(207) = -2.489, p = 0.014, with concurrent program participants (M = 3.27, SD = 0.81) scoring, on average, higher than consecutive program participants (M = 2.96, SD = 0.93). These results are displayed in Table 1. Table 1 Comparison of Concurrent and Consecutive Education Program Participants’ Responses to the Individual Items Related to the Question: How well do you think your practicum placements have prepared you to:

Education Program Concurrent M SD

Consecutive M SD

How well do you think your practicum placements have prepared you to: manage a classroom? 3.30 2.018* interact with parents? 1.45 0.568 interact with administrators? 2.20 1.429 manage difficult behaviours? 2.98 2.205* deal with difficult situations? 2.80 2.265* address the learning needs of all children? 3.27 2.489* *p<0.05

0.89

3.05

0.82

1.30

1.35

1.24

1.95

1.25

1.06

2.65

0.98

1.08

2.46

1.08

0.81

2.96

0.93

When the six questions were combined to create a total score, results demonstrated a significant difference between the participants from the concurrent and consecutive education programs on average responses to the overall question: How well do you think your practicum placements have prepared you to…?, (t(207)=-2.186, p=0.030). Specifically, participants from the concurrent education program (M=2.67, SD=0.86) scored higher, on average,

compared to participants from the consecutive education program (M=2.40, SD=0.84). The significant group differences related to 4 of the 6 indicators studied in this survey; while there were differences in the average overall scores on the remaining 2 indicators, those differences were not found to be significant. In other words, responses from concurrent program participants indicated that, generally, they thought they were better prepared by their practicum placements, at least compared to consecutive program participants, whose responses generally indicated that they thought they were less well prepared by their practicum placements. Next, we considered if the number of different classroom placements that teacher candidates experienced on average could account for any differences in their perceptions of the learning value of their practicum placements. The combined participant group (consecutive and concurrent students together) completed practicum placements in a minimum of 2 and a maximum of 4 different classrooms. The average number of classrooms experienced by this group of survey participants was 3.39. An ANOVA was used to compare the average responses on the 6 survey questions in relation to the number of different classroom practicum placements each person had experienced. There were no significant differences on any of the 6 areas of this set of questions that are attributable to the different numbers of classrooms teacher candidates had experienced during their practicum placements. In other words, the number of practicum placement classrooms had no effect on how well the teacher candidates thought the practicum placements prepared them to: manage a classroom, interact with parents, interact with administrators, manage difficult behaviours, deal with difficult situations, or address the learning needs of all children.

Discussion This cross-sectional study was an attempt to understand the knowledge and confidence of current and recently graduated faculty of education students regarding their perceptions of how well prepared they felt to handle complex interactions that are required in a classroom. Results indicate that teacher candidates in this program are feeling well prepared to handle some of the interaction tasks that will be required of them as teachers but feel severely underprepared in other areas. Both groups of teacher candidates feel fairly knowledgeable and confident in their ability to manage the day-to-day operation of a classroom and to address the learning needs of all children. Both groups in this study reported feeling they had achieved considerable knowledge and confidence in both of these areas through their practicum experiences, reporting 3.30 (concurrent) and 3.05 (consecutive) on measures related to managing a classroom and 3.27 (concurrent) and 2.96 (consecutive) on their ability to address the learning needs of all children. Since each of these abilities is a crucial aspect of a teacherâ&#x20AC;&#x2122;s role in the classroom, these results are encouraging, although significantly more so among concurrent students.

However, the remaining four measures of this aspect of the larger study are a cause for concern about the efficacy of practicum experiences as they are currently structured. While survey participants felt that they had developed some knowledge and confidence to manage difficult behaviours in the classrooms (2.98 concurrent; 2.65 consecutive), participants felt less knowledgeable and confident in their ability to deal with difficult situations, interact with administrators, and interact with parents (Table 1). In fact, our results suggest that as teacher candidates move further and further away from interaction with students and face the need to address situations with greater complexity or which involve other adults in the school environment, they feel less prepared to do so confidently. Preparation to interact with parents is an area that is very weak for both concurrent and consecutive teacher candidates, indicating that the practicum experiences as they are currently structured, provide them with insufficient exposure to these situations to allow them to develop knowledge and confidence in this area. In this jurisdiction, the practicum placement for both concurrent and consecutive teacher candidates is evaluated by associate teachers and by faculty advisors. The evaluations are structured by criteria for assessment, which include foci that reflect the standards of the profession in the province. Among these standards is a category of professional behaviours titled “Leadership in Learning Communities”. The Ontario College of Teachers, the accreditation body for the teacher preparation institutions in the province, explains this aspect of the standard as follows (http://www.oct.ca/public/professionalstandards/standards-of-practice; Accessed January 28, 2014): Leadership in Learning Communities  Members promote and participate in the creation of collaborative, safe and supportive learning communities. They recognize their shared responsibilities and their leadership roles in order to facilitate student success. Members maintain and uphold the principles of the ethical standards in these learning communities. In most jurisdictions, this standard forms the basis of assessment of the teacher candidates’ professional interactions in the school context. The standards are broken down into observable behaviours that are assessed by both the associate teacher and the faculty advisor at set times during the teacher preparation programs, which differ across the concurrent and consecutive program routes. Each Faculty of Education has internal control over how they define details within each of the standards of practice. In the jurisdiction where this study took place, these standards are associated with two professional behaviours: 1) collaborating with others to create a learning community; and 2) assuming professional responsibility (the planning binder, duties, meetings, punctuality, and initiative). It may be that these behavior descriptions are too vague to draw the attention of the teacher candidate, the associate teacher, or the faculty advisor to the specific types of knowledge and skills addressed in this survey. It is often said that what is assessed gets attention and that adage may apply in © 2015 The authors and IJLTER.ORG. All rights reserved.

this instance. If we were specific about the types of interactions that need development through a practicum experience, we might expect that teacher candidates could more fully develop knowledge and skills with classroom management, interaction with parents, interaction with administrators, managing difficult behaviours, dealing with difficult situations, and addressing the learning needs of all children. There might be value to consider a more targeted approach to teaching practica than may exist in some jurisdictions at present. If newly revised teacher education programs in this jurisdiction will allow for 18 weeks of practicum experience before a teacher is accredited, those weeks could be focused on the acquisition of specific skills that can only be learned in context, rather than on the general application of theoretical skills and knowledge that have been taught in course work. For example, later in the program, practicum opportunities might include several chances for the teacher candidate to interact with parents (e.g., through school committees, parent-teacher interviews, special events which they may arrange to involve parents, etc.). Following these experiences, associate teachers and faculty advisors would have a wealth of assessment information about the teacher candidateâ&#x20AC;&#x2122;s skills in working with parents, which can be used to provide guidance for further professional growth. The findings in this study are timely as Ontarioâ&#x20AC;&#x2122;s provincial jurisdiction is currently undergoing significant program review to extend the program of teacher preparation to align with requirements in most other Canadian provinces. While the jurisdiction has a reputation of providing strong teacher education programs, an additional year of preparation will allow for additional course and practicum time. This combination should provide additional opportunities for teacher candidates to acquire both theoretical knowledge through further course work and pedagogical content knowledge through practicum work. However, to ensure that the combination of theoretical and practical knowledge is acquired and that, therefore teacher candidates have an opportunity to develop stronger pedagogical content knowledge, we believe that targeted practicum experiences hold potential. During a targeted practicum experience, teacher candidates would focus on developing strength and confidence in one set of professional competencies during each teaching block, rather than being diversified across the usual focus on five different sets of competencies. Potentially, this could allow teacher candidates the freedom to explore, and use, a variety of interaction strategies, thereby having greater opportunities to develop knowledge and confidence in this area. Since schools are increasingly focusing on a professional team approach and making efforts to partner with parents in ensuring the success of the child, this approach to a targeted practicum would provide a focus on learning supported by parents and other professionals. Such an endeavor would seen to be invaluable as program review efforts progress.

Conclusions This study was part of a larger study, which investigated the knowledge and confidence, which is perceived by teacher candidates as a result of many aspects of their teacher preparation programs, including the practicum components. Due to the length of the larger study, investigation into the practicum aspects of the study focused on only 6 measures. It may be of value to examine the practicum experiences in either or both of these participant populations more thoroughly through more detailed questions and to triangulate survey responses through other forms of data. While we cannot over extend the interpretation of our data since it reflected responses from only 212 teacher candidates. While this is a solid basis for some conclusions, this number of participants represents only about 20 percent of the total teacher candidate population from this university in the study year. It may be that a larger participant group would reveal different trends. As this jurisdiction undertakes significant structural changes in its teacher preparation program and the practicum time involved in the consecutive program increases from 13 to 18 weeks, over two years, it is expected that we may see a close in the gap in perceptions of knowledge and confidence resulting from practicum experiences in the two groups of teacher candidates (i.e., consecutive and concurrent). Additionally, work is presently underway to develop more targeted skill development during practica than has previously been used at this university. It will be important to monitor these experiences closely as this new program delivery model is implemented.

References Beck, C., & Kosnik, C. (2002). Components of a good practicum placement: Student teacher perceptions. Teacher Education Quarterly, Spring, 81-98. Beck, C., Kosnik, C., & Rowsell, J. (2007). Preparation for the first year of teaching: Beginning teachersâ&#x20AC;&#x2122; views about their needs. The New Educator, 3, 51-73. doi:10.1080/15476880601141581 Brouwer, N., & Korthagen, F. (2005). Can teacher education make a difference? American Educational Research Journal, 42(1), 153-224. Bryan, S.L., & Sprague, M.M. (1997). The effect of overseas internships on early teaching experiences. Clearing House, 70(4), 199-201. Bullough, R.V., Young, J., Birrell, J.R., Clark, D.C., Egan, M.W., Erickson, L., Frankovich, M.,Brunetti, J., & Welling, M. (2003). Teaching with a peer: A comparison of two models of student teaching. Teaching and Teacher Education, 19, 57-73. Evelein, F., Korthagen, F., & Brekelmans, M. (2008). Fulfilment of the basic psychological needs of student teachers during their first teaching experiences. Teaching and Teacher Education, 24, 1137-1148. doi:10.1016/j.tate.2007.09.001 Ferrier-Kerr, J.L. (2009). Establishing professional relationships in practicum settings. Teaching and Teacher Education, 25, 790-797. doi:10.1016/j.tate.2009.01.001 Grierson, A., & Denton, R. (2013). Preparing Canadian Teachers for Diversity: The Impact of an International Practicum in Rural Kenya. In L. Thomas (Ed.), What is Canadian about Teacher Education in Canada? Multiple Perspectives on Canadian Teacher Education in the Twenty-First Century (pp 187-210). Sherbrooke, PQ:

Canadian Association for Teacher Education/Association Canadienne pour la formation à l’enseignement. Grierson, A., Cantalini-Williams, M, Wideman-Johnston, T., & Tedesco, S. (2011). Building scaffolds in the field: The benefits and challenges of teacher candidate peer mentorship. Brock Education, 20(2), 85-103. Grudnoff, L. (2011). Rethinking the practicum: Limitations and possibilities. Asia-Pacific Journal of Teacher Education, 39(3), 223-234. doi:10.1080/1359866X.2011.588308 Korthagen, F., Loughran, J., & Russell, T. (2006). Developing fundamental principles for teacher education programs and practices. Teaching and Teacher Education, 22, 1020-1041.doi:10.1016/j.tate.2006.04.022 Maynes, N., Allison, J., & Julien-Schultz, L. (2013). An Examination of Longevity of Impact of an International Practicum Experience on Teachers’ Beliefs and Practices Four Years Later, International Education Studies.6 (4). Maynes, N., Allison, J., & Julien-Schultz, L. (2012). International practica experiences as events that of influence in a teacher candidates’ development. McGill Journal of Education, 47 (1), 69-90. Ng, W., Nicholas, H., & Williams, A. (2010). School experience influences on pre- service teachers’ evolving beliefs about effective teaching. Teaching and Teacher Education, 26, 278-289. doi:10.1016/j.tate.2009.03.010 Ronfeldt, M. (2012). Where should students teachers learn to teach? Effects of field placement school characteristics on teacher retention and effectiveness. Educational Evaluation and Policy Analysis, 34(1), 3-26. doi:10.3102/0162373711420865 Schulz, R. (2005). The practicum: More than practice. Canadian Journal of Education, 28(1&2),147-167.

International Journal of Learning, Teaching and Educational Research Vol. 11, No. 1, pp. 53-74, April 2015

A Study of Formative Assessment Strategies in Teachers‘ School-Based In-Service Training Eva Nyberg and Mona Holmqvist Olander Department of Pedagogical, Curricular and Professional Studies University of Gothenburg Gothenburg, Sweden

Abstract. The aim of this study was to explore a team of teachers‘ (n=4) use of theoretically based formative assessment strategies within the course of a learning study. The thematic analysis is based on video observations of teachers‘ discussions during planning meetings, teaching in the classroom and evaluation meetings. The subject-specific content focused on learning about fractions, specifically the concepts of double and half, in three groups of six- to seven-year-old students (n=51 in total). An iterative process was used in which the teachers in the study used video recordings as a tool for analyzing their work in the classroom. The thematic analysis shows that the use of a general learning theory – variation theory – strengthens the effect of the teachers‘ formative assessment. Without explicit use of the assumptions from the theoretical framework, the formative assessment strategies had only a minor impact on students‘ learning outcomes. Keywords: Formative assessment, Classroom study, Elementary school, Variation theory; Learning study.

1. Introduction There have been several attempts to develop teachers‘ formative assessment by means of in-service training. A range of studies concerning programs for inservice training aiming at developing teachers‘ competence in performing formative assessment have been carried out, with various outcomes (e.g., Bennet 2011, Phelan, Choi, Vendlinski, Baker & Herman, 2011). It thus seems rather difficult to transform formal training about formative assessment into classroom practice. Wiliam (2006) claims that ―tools for formative assessment will only improve formative assessment practices if teachers can integrate them into their regular classroom activities‖ (p. 287). School-based in-service training could therefore be one way to develop teachers‘ abilities to use formative assessment to increase the students‘ learning outcomes. This school-based research project involved a team of teachers who had previously participated in an in-service training course on formative assessment, and the focus was on their use of

formative strategies in their classrooms during an iterative process in which the researchers were partly involved. Formative assessment can be approached in several different ways. In this study, the in-service training was aimed at encouraging the teachers to base their formative assessment actions on theoretical conjectures. Our hypothesis is that formative assessment strategies such as feedback become more powerful if they are based on a theory of learning, and we therefore explore the effect of applying the principles of variation theory (e.g., Marton, 2014) during planning, implementation and evaluation of teaching and learning, in order to increase student learning. Research has shown that formative assessment often lacks a theory of action, which makes it difficult to evaluate and understand the mechanisms causing the intended effects: Unless we understand the mechanisms responsible for change, we won‘t know if the effects are due to those mechanisms or to irrelevant factors. We also won‘t be able to predict the conditions, or population groups, for which the formative assessment is likely to work (Bennet, 2011, p. 14). Bennet (2011) also argues that a teacher‘s hypothesis about a students‘ understanding is dependent on the teacher‘s cognitive model, which can help the teacher to evaluate a student‘s understanding. Without a theory of learning, there is a risk that the teachers carrying out the formative assessment will fail to identify the underlying mechanism for how the learning takes place. This is also stressed by Black and Wiliam (2009, 2012) and Wiliam (2009), who argue that to know what feedback to give to students, the teacher needs both a theoretical model of how students learn and the ability to apply this theoretical understanding in a specific context. In light of this, this research project aims to further study the way in which the theoretical assumptions of variation theory can be used as guiding tools for teachers to assess their students formatively. In formative assessment, the data gathered during assessment is used to inform decisions regarding modifications and adaptations of the teaching to meet the learning needs of the students (e.g., Bennet, 2011; Black & Wiliam, 2009). This includes using evidence about difficulties collected in one group of students ―to modify instruction for another group of students at some point in the future‖ (Wiliam, 2009, p. 26). This means that it is only if the result from the assessment is used to inform teaching and learning that it can be said to have a formative function. Phelan et al. (2011) produced a study pointing out the difficulties in identifying what constitutes effective formative assessment. They studied teachers and students in a formative assessment intervention, aimed at improving student performance in mathematics. Surprisingly, the authors did not find any significant differences between the treatment group and the control group; the hypothesis that the improvement would be greater in the treatment group—in which formative assessment was used—remained unverified. Although the extensive research overview by Black and Wiliam (1998) delivered © 2015 The authors and IJLTER.ORG. All rights reserved.

the clear message that more formative assessments in school can lead to great improvements in student learning, the above-mentioned study by Phelan et al. (2011) emphasizes that it is not simply the case that any formative assessment tool leads to an improvement in student performance. The teachers‘ competence in performing the formative assessment is most likely crucial for the outcome, and therefore teacher variables need to be more closely analyzed. What matters seems to be how the formative assessment is carried out, i.e., what is noted by the teacher and the way in which this is addressed during the instruction. This means that effective formative assessment is constituted by both knowledge of the content in question and knowledge of what it takes to learn this specific content, in line with Bennet (2011), as well as Black and Wiliam (2009). This is the reason why we have added a theoretical framework to strengthen the teachers‘ knowledge about learning during the in-service training. In this article, we have taken into consideration the fact that to be formative, the instruction needs to be specific at a micro-level in relation to what is elicited through the assessment. This involves determining in what way the content is offered in relation to how it is experienced by the students and doing this according to theoretically based assumptions about what it takes to learn. The teachers in our study were guided by the variation theory of learning (Lo & Marton, 2012; Marton & Booth, 1997; Marton, 2014; Lo, 2013; Holmqvist, Gustavsson & Wernberg, 2008; Holmqvist, 2011), to which they were introduced by the researchers participating in the interventions during the introduction to the in-service project. The concept of variation in variation theory does not refer to varying methods, but rather to varying the features of the content that have not been previously discerned by the students. The assumptions are that to discern new aspects of the object of learning, these have to vary against an invariant background consisting of the aspects already known. Variation theory will be more thoroughly described below.

2. Aim and research questions The aim of this study was to explore a team of teachers‘ (n=4) use of theoretically based formative assessment/feedback strategies during a school-based inservice training in a primary school. The subject-specific content focused on learning about fractions, specifically the concepts of double and half, in a group of six- to seven-year-old students. The research questions were: 1. In what way do the teachers take advantage of the variation theory of learning in their formative feedback to the students? 2. What impact does the theoretically based formative assessment have on the teachers‘ way of constructing lessons in new groups of students?

3. The iterative in-service training process In this section we describe the tools used by the teachers in their in-service training: variation theory (theoretical framework) and the learning study process (an iterative process including video recordings).

3.1 Variation theory as a guiding principle Educational learning theories often describe learning in terms of conditions for learning. For example, a theory might state that learning takes place in an interaction between people, or as an internal construction by the individual, or through a combination of both. However, these theories do not usually offer explicit guidance on how to develop domain-specific learning or how to use the theories in practice. Variation theory is built on the research field of phenomenography, which originated in the 1970s and is concerned with studying qualitatively different ways of experiencing the same phenomena (Marton, 1981; Marton & Booth, 1997; Marton, 2014). Variation theory is thus useful in providing a way to analyze what it takes to learn. It is also used as a guiding principle in lesson planning (Holmqvist, Gustavsson & Wernberg, 2008; Kullberg, 2010; Lo & Marton, 2012). Its point of departure is non-dualistic, meaning that the content to be learned cannot be separated from the learner‘s understanding of it (Lo & Marton, 2012). Discernment, simultaneity and variation are cornerstones in variation theory. To learn, one has to discern what one has previously not been aware of and relate it to what one already knows. By varying aspects of the content that have not been previously discerned by the learner against an invariant background of aspects that are already known, new aspects become discernible. These patterns of variation are taken into consideration during planning (the intended object of learning) and implementation of the lesson (the enacted object of learning), and when analyzing the learning outcomes shown by the students‘ test results (the lived object of learning). One important point is that the pattern of variation will differ depending on the students‘ previous knowledge; it is based not only on what it takes to learn in relation to specific content but also on what it takes to learn specific content in relation to what is already known by the learner (Pang & Marton, 2013). The differences in improvement are thus not due solely to the design, method or other strategies used – for example formative assessment techniques as such (Phelan et al., 2011) – as the students‘ perspective has to be addressed in an adequate way if the full power of formative assessment is to be exploited. The design of the instruction aims to make visible those aspects that are necessary for further learning, called critical aspects. In variation theory, this refers to aspects of the content that have not yet been discerned by the learner but have to be discerned in order to develop further knowledge. As soon as the aspect is discerned, it stops being critical. If a lesson offers only aspects of the content that are not critical for learning, for example aspects that the learner has already discerned, then no learning will take place. 3.2 Learning study as a school developmental model The learning study is a model developed by researchers from Hong Kong and Sweden (e.g., Lo, Pong & Chik, 2005; Holmqvist, 2010; Marton, 2003; Marton & Tsui, 2004), inspired by the Japanese lesson study (Lewis, Perry & Murata, 2006). It entails a systematic and cyclical process consisting of a number of stages, starting with the choice and study of the object of learning to be addressed and taught. In this iterative model, the design of new lessons is based on the analysis of earlier lessons, with the aim of further developing students‘ previous learning outcomes. The object of learning is initially chosen by the teachers and defined © 2015 The authors and IJLTER.ORG. All rights reserved.

in detail after a screening, an interview or test to assess students‘ preknowledge, which identifies the aspects that may be critical for learning. A pretest is thereafter constructed based on the results of the initial screening and guided by the chosen theory of learning, in this case, variation theory. The teachers and researchers collaborate in planning the first lesson, which is taught by one of the teachers in the group. The lesson is video recorded. During the subsequent meeting the lesson is analyzed by the teachers and the researchers using both the video recording and the results of the pretest and posttest. The focus in these meetings is the learning outcomes, which provide the basis for the development of a new lesson to be taught to another group of students. If the learning study also has a research aim, an analysis of the whole process is made to report the results of the study. Learning studies, which involve teachers and researchers in collaboration (Marton & Tsui, 2004), have been implemented in 120 schools in Hong Kong in a project called ‗Variation for the improvement of teaching and learning project‘ (VITAL). The project has been evaluated (Elliott & Yu, 2008) showing the benefits for both student and teacher learning (Elliott & Yu, 2013). On the other hand, research on the collaboration between teachers and researchers shows tensions of several different kinds: ―outsiders‘ versus insiders‘ perspectives; academic versus grounded knowledge; unclear hierarchical statuses; and diverse and conflicting agenda‖ (Adamson & Walker, 2011). Empirical studies on the implementation of learning studies in other parts of the world, in this case Sweden, have also found them to be effective both for students and for professional development among teachers (Holmqvist, 2006, 2011). The first research project where learning studies were implemented in Sweden was funded by the Swedish Research Council (Holmqvist, 2002), and 18 learning studies in three different school subjects were studied (Gustavsson, 2008; Wernberg, 2009; Kullberg, 2010). The results show an increased learning outcome both in the short-term and the long-term (Holmqvist, 2011). In a similar model for school development, the lesson study (Stigler & Hiebert, 2009), teaching is developed through a similar cyclical process. The main difference is that learning studies are guided both by a theory of learning and a focus on content-specific research into students‘ understandings of the specific object of learning (Holmqvist, Gustavsson & Wernberg, 2007, p. 189). The focus in learning studies is on the learners‘ understanding of the content, while the focus in lesson studies is mainly on the improvement of the lesson itself or on other issues needing to be improved in the classroom. The results of an evaluation of the Swedish learning studies show that the teachers see learning studies as timeconsuming and difficult to work with continuously, despite the high student learning outcomes (Olteanu & Lennerstad, 2011). Lewis (2015) points out the lesson study as a so-called improvement science, which theorizes the need for two different types of knowledge sciences: a system of basic knowledge from the discipline of education and a system of profound knowledge. The learning study process, in which a combination of theoretically based conjectures and the teachers‘ deep knowledge about teaching is needed, also seems to fulfill the requirements for being labeled as improvement science.

3.3 Formative assessment in the learning study model In a learning study, the aim of the assessment is to make in-depth microanalyses of the students‘ learning in order to identify what the critical aspects of the content in question are in relation to the learners, and how these aspects can be made discernible in a powerful way. Having identifying the critical aspects, the teachers use and try out this information in their formative assessment—that is, they use it to guide their teaching in the classroom. Our hypothesis, therefore, is that formative assessment informed by variation theory enables teachers to pinpoint the aspects that should be focused on in the learning situation and to change this focus depending on the learners‘ needs. James and Pedder (2006), when discussing results from a project including a survey of 1000 teachers, assert that programs of professional development ―should be focused on classrooms and classroom practice‖ (p. 39). One of their conclusions is that both individual and social processes are ―important conditions for the promotion of assessment for learning in classrooms‖ (James & Pedder 2006, p. 39). They put forward the concept of ―research lessons‖ as described by Stigler and Hiebert (1999) as one possible approach to developing assessment practices in the classroom. Pedder and James (2012) further stress the importance of collaborative, classroom-based professional learning ―for fostering effective assessment for learning‖ (p. 41). We agree with these claims, and consider the learning study as a school-development process in line with the findings about powerful professional development. By the use of a theoretical perspective on learning, such as variation theory, the formative assessment in the learning study helps the teachers to understand what needs to be changed in instruction or which feedback is relevant in order to increase students‘ learning, using the theory as a guiding tool.

4. Procedure of the Study 4.1 Method The analysis of the teachers‘ strategic use of variation theory is qualitative and based on video recorded meetings and interventions as well as observations. A thematic analysis was made (Boyatzis, 1998) based on several readings of the material (which was transcribed verbatim), as well as watching and re-watching of the video recorded meetings and lessons. The students‘ summative assessments were used as a triangulation to strengthen the observations and find out whether the teachers‘ use of theoretical assumptions was reflected in students‘ learning outcomes. 4.2 Context The study took place during an in-service training project, which was conducted in a school district in a rural area close to a small town in the south of Sweden. In the Swedish school system, all classes are mixed with regard to both gender and abilities. Students spend 9 years in compulsory school, from age 7 to 16. The first time the children receive grades is at the age of 12, i.e., in grade 6. All classes include children of the same age and during the first 6 years, the classes normally consist of groups of 15-25 children.

The content of the lessons in the Swedish school system is governed by a national curriculum (Swedish National Agency for Education, 2011), but the design of instruction and methods used is left to the teachers. The education to become a primary school teacher consists of four years of academic teaching studies and vocational training at university level. 4.3 Participants and implementation Two researchers and four teachers from two different primary schools participated in the study, which took place over one semester. 51 seven-year-old students (24 girls and 27 boys) were involved in the study. The students‘ learning process and learning outcomes are not, however, in focus in this paper; the findings about the students form another part of the project (Holmqvist & Nyberg, 2014). Three of the teachers were each in charge of one lesson, including pretest, research lesson and posttest. One teacher, who was responsible for developing mathematics instruction in the school district, participated only in the planning meetings. Before the first meeting, as a part of the overall in-service project in the school district, the teachers were introduced to variation theory and the learning study model in a lecture by one of the researchers. The teachers were also given a book about the theoretical background and concept of the learning study (Holmqvist et al., 2006). The researchers‘ deeper knowledge of variation theory, however, was important throughout the discussions. The object of learning chosen by this group of teachers was the ability to double and halve numbers. This topic was chosen based on the teachers‘ previous observations that children had difficulty learning how to perform these operations with numbers, but not with concrete objects. The teachers had previously been introduced to formative assessment. The lessons and the planning meetings were video recorded. Before each new lesson, the recording of the previous lesson was analyzed, and experiences from that lesson were evaluated and discussed, including the results of the pretest and posttest. Three lessons were conducted, meaning that there were four meetings between the teachers and researchers (see Fig. 1). During the first meeting, the pretest was constructed and the first lesson was jointly planned. During the second and third meetings, the previous lessons were evaluated with respect to learning outcomes, and the coming lessons were planned on the basis of this evaluation. During the fourth meeting, the last lesson and the results from the tests were discussed, as was the outcome of the learning study as a whole.

PM 1 Week 1

RESEARCH LESSON 1 Week 1, 1st group of students, 1st teacher

PM 2 Week 2

RESEARCH LESSON 2 Week 2, 2nd group of students, 2nd teacher

PM 3 Week 3

RESEARCH LESSON 3 Week 3, 3rd group of students, 3rd teacher

EM Week 4

Figure 1: The four-week time-line of the learning study. PM=Planning meeting, EM=Evaluation meeting.

The length of intervention for each group of students was one lesson comprising approximately 15 minutes of pretest, 30 minutes of instruction, and 15 minutes of posttest. The lessons were planned and evaluated on the basis of variation theory. 4.4 Data The data collected during the school-based study consists of video recorded meetings (3), video recorded lessons (3) and one final observed meeting. Participating observations: Four planning meetings with the teachers were conducted during the four weeks of the learning study. Each meeting took place at the schools where the teachers worked. The meetings lasted 2 hours. Videotaped lessons: Three lessons were videotaped. Each lesson lasted 1 hour. Group 1 (Cycle 1) consisted of 24 students, group 2 (Cycle 2) consisted of 13 students and group 3 (Cycle 3) consisted of 14 students. 4.5 Analyses The video recordings from the lessons (n=3) and planning meetings (n=4) were transcribed verbatim. The data was analyzed as an exploratory single case study (Stake, 2006; Yin, 2009) at a fine-grained level (Phelan et al., 2011). For the qualitative analysis of the teachers‘ use of theoretically based formative assessment strategies and reasoning, a thematic analysis was used as a first step (Boyatzis, 1998). The analysis was based upon how the core concepts of variation theory – contrast, simultaneity and variation (Holmqvist, 2011; Kullberg, 2010; Marton, 2014; Lo, 2012) - were used by the teachers, and in what respects they were used in the planning, implementation and evaluation of each lesson. Thereafter a more detailed and specific analysis was made across the three lessons including the planning and evaluation meetings, during which the different data sources were compared and analyzed in parallel. 4.6 Ethical considerations The study followed the ethical considerations described by the Swedish Research Council (Hermerén, 2011). All participants, including parents, teachers, school leaders and children, were continuously informed about the aims of the study, the use of the data and their rights to confidentiality and to withdraw from participation. The parents also signed an agreement to let their children participate in the study and to enable the researchers to use the data.

5. Results The analysis describes the different ways the teachers‘ formative assessment strategies are expressed during the study. The analysis ends up with three themes regarding the teachers‘ development of formative assessment strategies guided by the theoretical assumptions. The first theme describes how the teachers, through their formative assessment, gradually developed insight about what critical aspects are and how they can be used to increase the students‘ understanding of halving and doubling (Theme A). The second theme highlights another example of how the teachers‘ formative assessment led to increased evidence-based insight into the students‘ understanding during the course of the © 2015 The authors and IJLTER.ORG. All rights reserved.

learning study (Theme B), by taking into consideration the connection between the object of learning and the learners pre-knowledge (non-dualism). The third theme addresses the teachers‘ feedback during their respective lessons, focusing mainly on how the teachers used variation theory (Theme C) in their feedback to the students. 5.1 Theme A: Critical aspects of the content The knowledge about what critical aspects are and how they can be dealt within instruction is the main focus in this theme. The teachers found that to understand the concept of doubling, the students had to discern variation in amounts at the same time as discerning the invariant aspect of doubling—that is, taking the same amount ―one more time‖ and adding it to the original amount. This section presents the analysis of how this was addressed and handled during the course of the learning study. 5.1.1. The first meeting During the first planning meeting, the test questions and the design of the first lesson were discussed and constructed. The design was based on the results from the screening carried out by the teachers themselves, the teachers‘ previous experiences from teaching this content, the researchers‘ knowledge of common obstacles for students, and the researchers‘ extensive experience of using variation theory for test construction and lesson planning. The various aspects the object of learning were analyzed, to determine which aspects the students had already discerned and which ones they needed to discern in order to develop their knowledge. During the screening, when one of the students was asked by the teacher to double the number four, the child answered five (i.e., four plus one). This student evidently interpreted ―add one more time‖ as ―add one (the number 1)‖. During the planning meeting before the first lesson, one of the teachers confirmed that this was a critical aspect of the object of learning. Excerpt 1, first planning meeting (T1 = Teacher 1): T1: Yes, we have many children who cling to that; they don‘t get doubling, for them it is … plus one. We tried previously when we taught this particular topic not to start with one, because we thought that was what made them stick to this, but there are those who still hold on to ―plus one‖ even if we start by asking them to double three. The teachers and the researchers consequently set up the hypothesis that one important part of the instruction might be to avoid the expression ―one more time‖, which the students could interpret to mean ―plus one‖. Instead, the expression ―the same amount one more time‖ was to be used during the first research lesson to make it possible for the students to discern that ―one more time‖ is not the same as ―one more‖. For the students who had not already noticed the difference, this might be a critical aspect.

One of the concepts in the theoretical framework used here is about how to make aspects of the content discernible for the students. In this case the concept of simultaneity was used. The first lesson was therefore planned to allow the students to apply doubling and halving at the same time, but without having the original number of objects on their desks. The decision not to let the students see the objects representing the original number was based on previous experience of students making a mechanical visual doubling, without really understanding the concept of doubling, e.g., by adding the same amount once again, which results in an understanding of double as ―the same as‖; doubling four then gives four instead of eight, as the student added four to the original amount. 5.1.2 Second meeting The first lesson was evaluated in the second meeting. The analysis during the meeting of the pretest and posttest focused on the theoretical concept of simultaneity. However, in their analysis of the test results, the teachers found no significant improvement regarding mean points. The students (n=24) scored 3.9 (out of 10 maximum points) in the pretest and 4.7 in the posttest (p=0.073). The analysis of the test revealed that after instruction, the students still had problems discerning what doubling means. In one of the test questions the students were asked to draw double or half of a particular number of objects, having chosen the original number themselves. This was as difficult for them after instruction as before instruction. It seemed that the students had not yet learned to separate the original number from the concept of doubling or understood that doubling and halving was not related to a specific number but involved the same general processes regardless of what number used. To allow the students to separate the rules for doubling and halving from the object being processed, the group of teachers and researchers decided that simultaneity was to be used. By contrasting halving and doubling, the aim was to enable the student to separate the original amount from half the amount as well as double the amount. The aim of looking at halving and doubling with different numbers was to help the students to generalize and discern the general idea of halving and doubling instead of e.g., adding one to the original amount in all cases. The review of the video recording of the lesson during the meeting revealed another way of understanding doubling that the group had not previously encountered, in which doubling an amount was taken to mean adding two, instead of taking the original amount once and once again. The information ―take the original amount twice‖ was understood as +2. The contrast between the original amount and the doubled amount had thus not been explicit enough and therefore the difficulty remained. Excerpt 2, second planning meeting (R1 = Researcher 1): R1: We said it had to do with ―one more time‖, adding or subtracting one, but here they seem to come to the conclusion to add two because they understand ―take the same amount twice‖ to mean ―add two‖. You responded to that well, but we will take this to the next lesson because we didn‘t know it before.

The review during the meeting of the video recording of the first lesson clearly showed that the teacher had followed the agreed plan based on the microanalysis, that is, using the expression ―the same thing one more time‖ rather than ―the same thing twice‖: Excerpt 3, first lesson (T1 = Teacher 1): T1: Now, you are to add twice as many as six. I put six pieces here. I have now added six one time. If we want to have double the amount, we must have this six and then I have to add another six pieces. One conclusion after this discussion was that it was very important to be even more explicit and use the phrase ―the same amount as the first and then the same amount one more time‖ instead of ―the same amount twice‖. Another discussion concerned how to design the task to help the children distinguish between the original amount and the new amount. While watching the video recording, the teacher who had taught the first lesson reflected on whether one reason for the students‘ confusion might be that the students did not have the initial number of objects in front of them while working with their tasks. Excerpt 4, second planning meeting (T1 = Teacher 1, R1 = Researcher 1): T1: So, the question is whether it is best to put the objects away or whether it would have been better to let them have the original number of objects in front of them and let them do it again, so to speak … do it next to … so that both were there to make it possible for them to compare … R1: Maybe it would be even clearer to have a borderline in between … so: ―This is what we had from the beginning, and now we are going to put down double that amount, so this is what should be there now‖ … We would then not get the problem of their just adding the same amount … to be clearer that they should not just list … but to understand doubling, you must understand that you have a new amount that is twice as large as the other. Here, the teacher who had given the first lesson wondered whether it might have been better for the students‘ learning if they had had the original amount that they were asked to double or halve in front of them so that they would be able to contrast this with the new amount. This discussion concerned how the children could be helped to separate the original amount from the solution, avoiding just adding the same amount as the initial and adding this to the ‗doubled‘ amount (double of four experienced as plus four). In the end, the group agreed upon increasing the contrast between the amounts, and the plan for the second lesson was therefore to work with the original number of objects and its double or half simultaneously but separated from each other. To make © 2015 The authors and IJLTER.ORG. All rights reserved.

this relation clear, it was decided that the original amount should be placed on one side of a border and the doubled or halved amount should be placed on the other. In this way, the original number of objects and the halved or doubled amounts could be contrasted and discernible simultaneously. This should be done by both the teacher and the students while they solved tasks during the lesson, and the initial number of objects should be explicitly contrasted with the new amount, in accordance with the assumptions of variation theory. 5.1.3 Third planning meeting After the second lesson, during the third planning meeting, the analysis of the pretest and posttest indicated that the second group of students had understood the content better than the first group. The analysis of the pretest and posttest for this group of students (n=13) showed significant differences between pretest and posttest. Mean scores increased from 6.5 to 8.2 (p=0.015), thus indicating increased learning, in contrast to the first group of students, where, as mentioned above, no significant differences were found. The increased mean scores indicated that the design chosen for this lesson had been successful. However, the tasks in the test that allowed the original number of objects to be chosen by the students themselves were still problematic in this second group of students. This third meeting therefore included discussions about various ways to improve the students‘ abilities to solve these tasks. The teacher who taught the second lesson (T2), the teacher who would teach the third lesson (T3), and one of the researchers (R1) discussed the issue of simultaneously contrasting different amounts when working with the concepts of doubling and halving in class. Excerpt 5, third planning meeting (T2 = Teacher 2, T3 = Teacher 3, R1 = Researcher 1): R1: We talked last time about having the original amount, and then half and twice that amount. This change, I think, would be interesting because you would have the example of both half and twice the same original amount. … You put the original amount there at the same time as you tell them to put down half and double that amount. T3: However, why have you put the ruler along there? (Points at R1‘s paper, on which a ruler has been laid down to divide the paper into two areas.) T2: To make them see the point with … the original amount. R1: It would have been good … if you had … It may be a paper or something like that which you just copy … and then they put down the same original amount as you as the teacher have, in the middle. … So, if you make copies of a handout you can mark HALF and DOUBLE … there … if you want to emphasize that. © 2015 The authors and IJLTER.ORG. All rights reserved.

T2: Then, you go on with the simultaneity the whole time … in everything you do, so you don‘t … it is quicker too. During this third meeting, the group decided that simultaneity, not only regarding the initial amount and half or double but all three, should be used in the subsequent lesson for both the original numbers and their doubles and halves. The students would work with double and half of the original amount at the same time, meaning that even more simultaneity would be involved in the teacher‘s instruction. The original amount, half the amount, and double the amount would be contrasted with each other simultaneously. Another important conclusion was that the original amount should remain unchanged in the middle of the children‘s worksheet, and the teacher should say ―the same thing again and once more‖ to explain the doubled amount. This would be emphasized by the use of one more borderline than in the second lesson—that is, the paper used for the exercise would be divided into three fields, instead of two, with the initial amount placed in the middle, and the doubled and halved amounts on either side, separated by lines. The pretest and posttest associated with the third and last research lesson did not show improvements over the second lesson. The mean score in this group of students (n=14) was 6.1 in the pretest and 7.4 in the posttest, a difference that was not significant (p=0.162). This can be compared to the previous research lesson (lesson 2), after which the mean posttest result had increased significantly (from 6.5 to 8.2 points). The tasks in which the students themselves chose the original number to double or halve were still problematic, as were other tasks dealing with halving and doubling a predetermined amount. This result illustrates how small changes in instruction make differences in what the students can learn. It also shows how, even if the teachers in their joint planning of the lesson had found a way to adjust the instruction to the students‘ knowledge, these plans were not always actually understood and implemented by the teacher giving the lesson. To understand the result, the teachers and the researchers reviewed the videotape of the lesson. This showed that the exercise where the intention had been to use simultaneity to clarify the difference between the original amount and the doubled and halved amounts, was performed mostly by the teacher, but not by the students during their work in the classroom. The change planned for this lesson, to use simultaneous contrast between all the amounts to make the difference even more explicit, was thus only partly carried out by the teacher in charge of the third lesson. The formative assessment used during the lesson was therefore not informed by the theoretical assumptions discussed during the planning meeting. One example was the decreased number of examples of amounts used during the teacher-led instruction, which only included one number (6). Another issue was when the teacher, demonstrating the examples, twice placed the objects in the wrong area on the overhead projector (placing the doubled amount in the area for half). She corrected the error when it was pointed out by a student, but did not explain to the class what had been wrong or why she changed the placement of the objects. © 2015 The authors and IJLTER.ORG. All rights reserved.

This oversight might give students the impression that this placement is randomly chosen and not a deliberate content-based choice. In conclusion, the third lesson could not verify the group‘s hypothesis that increased contrast between the different amounts would help students to discern the concepts of double and half, since simultaneity was not used as was agreed upon. The theory was thus not used as planned and the students‘ learning outcomes did not improve. However, the teachers became even more aware of how small changes might have a great impact on students‘ performance and in what way theoretical assumptions can be handled in the classroom. 5.2 Theme B: The undivided learning space One of the assumptions of variation theory is that the learner and what is going to be learnt cannot be separated from each other. This means that the way the learner experiences the object of learning is unique and this has to be taken into consideration in a teaching situation. Finding powerful ways to develop student learning requires both deep knowledge about the content being taught and deep insight in the students‘ pre-knowledge. Otherwise it becomes difficult to design powerful learning situations. It is clear from the analysis of the planning meetings that the teachers‘ insights into the students‘ understanding of double and half increased during the study, both as a result of the repeated analyses of the pretests and posttests, and as a result of the discussions based on the video recordings of the lessons. One example concerns the teachers‘ opinions about the children‘s abilities to understand the concept of ―half‖. The teacher who performed the first research lesson (T1) declared, during the first planning meeting, that the problem children had with understanding doubling and halving was not the concept of half, but the use of the expression ―half as many‖. Excerpt 6, first planning meeting (T1 = Teacher 1): T1: The difficulty with ―half‖ is not really the concept of ―half‖. As long as you stick to saying ―half‖, then you can do it. Because these things, they can divide them. What makes it complicated, as we have understood it, is when you say half as many. For that, of course, makes it difficult because there should be fewer and yet you say many. Analysis of the pretest and posttest results associated with the first lesson showed that before the lesson, more students gave a correct answer to the questions asking for ―twice as much‖ than to the questions asking for ―half as many‖. After the lesson, more students were better able to answer questions about halving, but still not as well as they could answer questions about doubling. Five items in the pretest and posttest concerned doubling, and five concerned halving. The students‘ mean scores on the pretest were 0.45 for the items on doubling and 0.33 for those on halving, while the corresponding scores on the posttest were 0.51 and 0.44. However, despite this data, at the second planning meeting, the teacher quoted above still believed that the concept of ―half‖ was easier for students.

Excerpt 7 (T1 = Teacher 1, T2 = Teacher 2, T4=Teacher 4, R1 = Researcher 1): T1: However, it feels like half is so much easier. R1: Yes, but we thought last time that half was harder … half as many. T2: However, I think you learn to take ―half‖ before you learn how to ―double‖, that is … to share with a sibling or with a friend … they have known this since before … R1: However, they‘re not better at ―half‖ than ―double‖. T2: Aren‘t they? R1: They do worse at half in all the items. T4: It‘s the b-alternative, which … R1: Yes, exactly, and the b-alternative is the most difficult straight through, except for the question with the squares. T2: Halving is more difficult than doubling. T1: The question is whether it would have been easier if they had had this stuff [in front of them], then I think certainly they can take half of it, but only if they can actually see it … R1: Yes, but they cannot either. They have the stuff there, but still … T1: However, don‘t they get it more right when they have to split it in half? This example shows that eliciting the students‘ understanding through finegrained analysis of the scores on diagnostic test questions gradually changed the teachers‘ view of what the students found hard to understand. Before the first test was conducted (during the first research lesson), the teachers assumed that halving was easier for the students to understand than doubling, but when the tests revealed that this was not the case, their opinion was slowly altered. The excerpts indicate, however, that their initial opinion was quite persistent and difficult to change; they were not easily convinced of the contrary even if the test results showed this. The design of instruction thus risks focusing on aspects that are not problematic for the students and neglecting the aspects that are critical. The test was designed on the basis of variation theory, and by contrasting halving with doubling, it was possible to compare the students‘ knowledge of these concepts. The results of the tests informed the teacher‘s formative assessment, as long as the teachers accepted what the tests really said about the © 2015 The authors and IJLTER.ORG. All rights reserved.

students‘ knowledge and let go of their own beliefs. As exemplified in the exchange quoted above, it was however difficult for the teachers to take the students‘ perspectives when these perspectives contradicted the teachers‘ own beliefs. Previous research results have indicated that formative assessment might not always have an impact on students‘ learning outcomes, and if teachers‘ instruction depends on their own beliefs instead of students‘ knowledge, it is obvious that the impact of formative assessment is weaker than if they take the students‘ knowledge into consideration. 5.3 Theme C: Theoretically guided formative feedback In this theme, the teachers‘ formative feedback in the classroom is in focus. The analysis of the teachers‘ feedback about their respective lessons focuses on how they used variation theory to understand and respond to the students‘ understanding of the content. In spite of the teacher‘s obvious engagement with teaching in the first lesson and in the evaluation of this lesson, the subsequent analysis of the video recording showed that the lesson rarely included direct feedback to the students. The teacher listened to the students‘ answers, but seldom commented on whether the answers were correct. Excerpt 8, first lesson (T1 = Teacher 1, S1 = Student): S1: We had six, then we thought that half of them was all of them. T1: You thought half was all and so you took them all away. So, we have some different answers. Let us see what the others thought. What did you come up with, [student name]? (Turns to another group.) After listening to the first student, the teacher turned directly to the next group without revealing the correct answer, and this answer was not revealed until summing up the instruction at the end of the lesson. This lack of feedback was not highlighted or discussed when the group evaluated the first lesson during the second planning meeting, so we do not know whether it was common for this teacher or not; it is possible that the teacher conducted the lesson differently from how she usually would due to her awareness of being filmed. Nevertheless, it is evident that the students were left in doubt several times. According to the videotapes, the teacher who conducted the second lesson gave the students more detailed feedback regarding critical aspects than did the first teacher. It is probable that the discussion during the evaluation of the first lesson had an impact on this teacher‘s awareness of which aspects of the concepts of doubling and halving the children might not have discerned, which may have led her to challenge the students‘ answers with a more developed formative assessment than the first teacher was able to. In the example below, the teacher worked formatively with a student who was asked to show half of eight. The teacher clearly contrasted the doubled and halved amounts, comparing them

with each other to help the students discern the difference between double and half, taking advantage of the experiences from the previous lesson. Excerpt 9, second lesson (T2 = Teacher 2, S = student): T2: Was he asked to put down half or double the amount? S: Half! T2: Half. Is there the half of the amount on that side here? (Points to the tray and one of the piles with four chestnuts.) (The student nods.) T2: How many are here? (Points to the tray and one of the piles with four chestnuts.) S: Four! T2: And how many are here? (Points to the tray and the other pile with four chestnuts.) S: Four! T2: However, that‘s just as much! Where is half the amount? Here, the teacher made a serious effort to understand which aspects the student had experienced and which the student had not yet discerned. She evidently used knowledge gained from the analysis of the first lesson about how the aspects might be discerned and how to use patterns of variation to make aspects discernible. She noticed that the student had taken four chestnuts from the original amount, resulting in two piles with the same amount of nuts, i.e. four chestnuts in each pile. As she wants the student to discern both eight and half of eight (four), she puts a question to make it possible for the student to discern the difference. Because the student had taken four items from the original pile of eight items when halving, the distinction between four and eight was not visible anymore; the child ended up with two piles with equal amounts instead. Based on such mistakes, to draw attention to the difference between the original amount and half this amount, the teacher repeatedly used simultaneous contrast in her instruction. There were, however, also examples during the lesson in which the teacher did not give a clear indication of whether a student‘s answer was correct. During the third research lesson, the students were given some direct feedback during their performance of the tasks, but not after, and often this feedback neither confirmed nor rejected the students‘ answers. The lack of correspondence between the intended design of the third lesson and the teacher‘s actual performance of it could be one reason for this lack of © 2015 The authors and IJLTER.ORG. All rights reserved.

constructive feedback. The communication between teacher and students was mainly based on giving instruction at a procedural level: Excerpt 10, third lesson (T3 = Teacher 3): T3: You also think that 4+4 is 8. Have a look here … I have 8 pieces from the beginning. That is my quantity, my amount that I am not allowed to touch. If I am going to put down the double amount, I have to put down as many as I have here (points at the 8 pieces in the middle area) + the same amount. This means I have to put down 8+8, and this means? The feedback in the third lesson thus ends with the teachers telling the students the correct way to solve the problem, a process, instead of taking into consideration how the students might have understood the problem and what aspects had been discernible.

6. Discussion and Conclusions This study explored teachers‘ development of formative assessment strategies guided by variation theory during a classroom-based professional development project. The result of the analysis found three themes for how the teachers used theoretically informed strategies for assessment in the classroom in their planning, conducting, and evaluating of lessons. These three themes demonstrate how formative assessment contributed to the development of the teaching: theme A concerns the critical aspects for learning the specific content; theme B focuses on how teachers understand their students‘ knowledge about the content; and theme C brings A and B together by exploring the teachers‘ formative feedback in the classroom. The results indicate that throughout the study, the teachers became increasingly aware of how even small changes in teaching could affect the students‘ learning and how to use this knowledge in further instruction. For example, after finding out that the students tried to find a definite figure to use when doubling or halving, such as in all cases choosing to add one, two, or another number to the initial amount instead of seeing the relationship between the original amount and what to add, the teachers used different ways of explaining the concepts to the students. Two of the teachers realized the importance of varying the initial amount, based on the theoretical assumptions, and varying this aspect made it easier for the students to understand that the number added changes when the original amount changes. However, the third teacher was not aware of the importance of this because she had not accepted the common theoretical base and thus used this knowledge neither in her instructions nor in her feedback to the students, resulting in no significant increase in learning. The learning outcomes of the students‘ tests strengthened the other two teachers‘ analyses, and they became aware that even small changes in instruction might have an impact on student learning, such as what examples to use in the teaching (type and number of different objects), how these are to be used in the teaching and exercises (simultaneously contrasted or not), or what expressions to use when explaining the tasks (―same amount twice‖ or not). Eliciting and clarifying students‘ ideas and understandings at a fine-grained level based on theoretical assumptions thus © 2015 The authors and IJLTER.ORG. All rights reserved.

seems to have helped the teachers to understand the difficulties students may have in comprehending the object of learning, and to give relevant feedback to the students. The teachers‘ formative assessment, which increased over the course of the learning study, was informed by the results of the students‘ tests and the video recordings of the lessons, both of which proved to be powerful tools for the teachers‘ assessment of the students‘ learning and understanding. Thus, there is evidence within the study that the learning study design encourages teachers to work formatively and that the formative assessment is strengthened by the theoretical framework used. This is in line with the argument put forward by Black and Wiliam (2009, 2012) and Wiliam (2009) that in order to be able to give relevant feedback to a student, the teacher needs a theory of how students learn and the ability to apply this theoretical understanding in a specific context. This is also in line with the work of James and Pedder (2006) and Pedder and James (2012), who suggest that the concept of ‗research lesson‘ could be a useful strategy to promote assessment for learning in classrooms. The evaluation of the last lesson showed that the simultaneity of double, half, and the original amount had not been presented in the third lesson as intended. During the planning meeting before this lesson, the teachers developed a new design, which was then not implemented during instruction. It was also clear through subsequent analysis that direct feedback to the students differed between the teachers and the groups; the teacher conducting the second lesson gave more frequent and more specific feedback than the other teachers by explicitly using the concept of simultaneous contrast. It is therefore possible that the improvement in scores after the second lesson was due to the combination of increased feedback with a lesson design that followed the assumptions of variation theory with regard to determining which aspects of the content should be in focus and which should be kept in the background. On the basis of this, we can conclude that the outcome of the learning study as a whole might have been even better had the joint evaluation of the lessons focused more strongly on the teachers‘ feedback (or lack thereof) to the students, both individually and to the class as a whole, along with discussing the effectiveness of the lesson design, as revealed by the students‘ test results and comments and behavior during the lessons.

7. Limitations of the Study The small scale of the study—four meetings with four teachers over a limited period of time and with a limited number of students—means that it is not possible to make any generalizations from the results. However, these limitations made it possible to perform an in-depth study, giving some insights into how a theory of learning can guide teachers in their planning, implementation and evaluation of a content-specific topic. Since the study is class-room based we consider it however to have rather high ecological validity (Brewer, 2000), especially regarding the use of variation theory in designing, implementing and evaluating teaching, and the impact this is likely to have on

the opportunities for children in similar situations and settings to develop knowledge about doubling and halving.

8. Acknowledgments We would like to thank the participating teachers, who generously shared their time and teaching, the Learning Design research team (LeaD) at Kristianstad University, for their encouragement and support; and the Department of Pedagogical, Curricular and Professional Studies, University of Gothenburg, Sweden, for partly financing this study. We are also grateful for valuable comments provided by the reviewers and for the thorough language review carried out by Dr. Catherine MacHale, which increased the clarity of the paper.

9. References Adamson, B., & Walker, E. (2011). Messy collaboration. Learning from a learning study. Teaching and Teacher Education, 27, 29-36. Bennet, R. E. (2011). Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice, 18: 1, 5 – 25. Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in education: Principles, Policy and Practice, 5(1), 7–73. Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5–31. Black, P., & Wiliam, D. (2012). Developing a theory of formative assessment. In J. Gardner (Ed.), Assessment and learning (pp. 206–229). (2nd ed.) London: Sage Publications. Boyatzis, R.E. (1998). Transforming qualitative information: Thematic analysis and code development. London: SAGE. Brewer, M. (2000). Research Design and Issues of Validity. In H.T. Reis & C.M. Judd (Eds.), Handbook of research methods in social and personality psychology (pp. 3-16). Cambridge, U.K.: University Press. Elliott, J., & Yu, C. (2008) ―Learning Studies as an Educational Change Strategy in Hong Kong: An independent evaluation of the ‗Variation for the Improvement of Teaching and Learning‘(VITAL) Project‖, funded by the Dept for Education and Manpower, Hong Kong. Published by the Centre for Learning Studies, HK Inst Educ. http://www.ied.edu.hk/cls/resources.htm Elliott, J., & Yu, C . (2013) Learning Studies in Hong Kong Schools: A Summary Evaluation Report on ‗The Variation for the Improvement of Teaching and Learning‘ (VITAL) Project, Education and Didactique, 7(2), 147-163. Gustavsson, L. (2008). Att bli bättre lärare. Hur undervisningsinnehållets behandling blir till samtalsämne lärare emellan. [To become better teachers. How the handling of the lesson content becomes a topic of conversations between teachers.] Umeå: University of Umeå. Hermerén, G. (2011). Good research practice. Stockholm: The Swedish Research Council. Holmqvist, M. (2002). Lärandets pedagogik. Forskningsansökan till Vetenskapsrådet. [The Pedagogy of Learning. Research application to the Swedish Research Council.] Dnr 721-2002-3386 Holmqvist, M., Gustavsson, L., & A. (2007). Generative learning: learning beyond the learning situation. Educational Action Research, 15(2), 181-208. Holmqvist, M., Gustavsson, L. & Wernberg, A. (2008) Variation Theory – An Organizing Principle to Guide Design Research in Education. In Kelly, A.E., Lesh, R., &. Baek J. (eds) Handbook of design research methods in education, p 111-130. New York: Routledge. © 2015 The authors and IJLTER.ORG. All rights reserved.

Holmqvist, M. (2011). Teachers‘ learning in a learning study. Instructional Science, 39(4), 497-511. Holmqvist Olander, M., & Nyberg, E. (2014). Learning study guided by variation theory: Exemplified by children learning to halve and double whole numbers. Journal of Research in Childhood Education . 28(2), 238-260. James, M., & Pedder, D. (2006). Professional learning as a condition for assessment for learning. In J. Gardner (Ed.), Assessment and learning (pp. 27–43). London: Sage Publications. Kullberg, A. (2010). What is taught and what is learned: Professional insights gained and shared by teachers of mathematics. Diss. Göteborg : Göteborgs universitet, 2010. Göteborg. http://gupea.ub.gu.se/hdl.handle/2077/22180 Lo, Mun Ling (2012). Variation theory and the improvement of teaching and learning Göteborg: Acta universitatis Gothoburgensis. http://gupea.ub.gu.se/handle/2077/29645¨ Lewis, C. (2015). What is improvement science? Do we need it in education? Educational Researcher, 44(1), 54-61. Lewis, C., Perry, R., & Murata, A. (2006). How should research contribute to instructional improvement? The case of lesson study. Educational Researcher, 31(3), 3-14. Lo, M.L., Pong, W.Y., & Chik, P.M. (Eds.) (2005). For each and everyone: Catering for individual differences through learning study. Hong Kong: Hong Kong University Press. Lo, M. L., & Marton, F. (2012). Towards a science of the art of teaching: Using variation theory as a guiding principle of pedagogical design. International Journal for Lesson and Learning Studies, 1, 7–22. Doi: 10.1108/20468251211179678 Marton, F. (1981). Phenomenography - describing conceptions of the world around us. Instructional Science, 10, 177-200. Marton, F. (2003). Learning Study - pedagogisk utveckling direkt i klassrummet. [Learning Study—pedagogical development directly in the classroom]. In Carlgren, I. (Ed.). Forskning av denna världen: praxisnära forskning inom utbildningsvetenskap [Research of this world-practice-oriented research in educational sciences] (pp. 41–46). Stockholm: Vetenskapsrådet [The Swedish Research Council]. Marton, F. (2014). Necessary conditions of learning. New York, N Y: Routledge Marton, F., & Tsui, A. (2004). Classroom discourse and the space of learning. Mahwah, N.J.: Lawrence Erlbaum Associates. Marton, F., & Booth, S. (1997). Learning and awareness. Mahwah, N.J.: Lawrence Erlbaum Associates. Olteanu, C., & Lennerstad, H. (2011). Lesson study och Learning study samt IKT i matematikundervisningen: en utvärdering av matematiksatsningen [Lesson Study and Learning Study and ICT in mathematics education: An evaluation of the mathematics initiative]. Stockholm: Fritzes. Pang, M. F., & Marton, F. (2013). Interaction between the learners‘ initial grasp of the object of learning and the learning resource afforded. Instructional Science, 41(6), pp 1065-1082. Pedder, D., & James, M. (2012) Professional learning as a condition for assessment for learning‘. In J. Gardner (Ed), Assessment and Learning (pp. 33-48), (2nd ed.). London: Sage Publications. Phelan, J., Choi, K., Vendlinski, T., Baker, E., & Herman, J. (2011). Differential Improvement in Student Understanding of Mathematical Principles Following Formative Assessment Intervention. The Journal of Educational Research, Vol 104, pp. 330-339.

Stigler, J.W., & Hiebert, J. (1999). The teaching gap: best ideas from the world's teachers for improving education in the classroom. New York: Free Press. Stigler, J. W., & Hiebert, J. (2009). The teaching gap: Best ideas from the world’s teachers for improving education in the classroom. New York: Free Press. Swedish National Agency for Education (2011). Curriculum for the compulsory school system, the pre-school class and the leisure-time centre 2011. Stockholm: Swedish National Agency for Education [Skolverket]. Wernberg, A. (2009). Lärandets objekt: Vad elever förväntas lära sig, vad görs möjligt för dem att lära och vad de faktiskt lär sig under lektionerna [The Object of Learning: What students are expected to learn, what is possible for them to learn and what they actually learn in class]. Umeå: University of Umeå. Wiliam, D. (2006). Formative Assessment: Getting the Focus Right. Educational Assessment, 11:3-4, 283-289. Wiliam, D. (2009). An integrative summary of the research literature and implications for a new theory of formative assessment. In H. Andrade & G. J. Cizek (Eds.), Handbook of Formative Assessment (p.18-40). New York: Routledge. Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation 37, 3–14.

International Journal of Learning, Teaching and Educational Research Vol. 11, No. 1, pp. 75 – 98, April 2015

Designing and using interactive e-books in Vietnam Ngoc-Giang Nguyen Doctor’s degree student of The Vietnam Institute of Educational Sciences 101 Tran Hung Dao, Hanoi, Vietnam Abstract. E-books are now our imperative need. Because of their strong points, e-books help the process of learning a lot. E-books allow us to teach an infinite number of students without depending on location and studying time. In addition, E-books improve independent thoughts, creative thoughts and informatics thoughts for students. Keywords: e-books, e-learning, technology.

1. Introduction In recent years, the research, design, and produce tendencies of ebooks have been developing a lot. An e-book called Class-book produced by Vietnamese Education Publishing House has justified the above thing. Students learning with the e-book can avoid carrying books cumbersomely. If students buy this ebook, they can use it in all 12 years of learning. Because e-books can interact in two dimensions with customization ability, multimedia content, they help students learn enthusiastically. In addition, e-books have other functions that ordinary books do not have, such as feedback, relating documents, adapting to the fast changing world, etc. Therefore, e-books were and will be the first selection in innovating methods, equipment and learning contents. Although e-books are important, being researched and developed strongly, some of us still do not know what e-books are, how many kinds of e-books there are, what characteristics e-books have, what the differences between the learning with e-books and the learning with e-learning are. Thus, we will refer to these things in the article. Moreover, we have also built an interactive geometry ebook, which has many strong points of technology, and is better than current ebooks. We have also designed tools that allow us to divide learners, ecommunication, etc. in order to help the process of learning better.

2. Research content 2.1. The conception on e-books According to Wikipedia, an electronic book (digital book, e-edition) is a booklength publication in digital form, consisting of text, images, or both, readable on computers or other electronic devices. Although sometimes defined as “an electronic version of a printed book”, many e-books exist without any printed © 2015 The author and IJLTER.ORG. All rights reserved.

equivalent. Comercially produced and sold e-books are usually intended to be read on dedicated e-book readers, however, almost any sophisticated electronic device that features a controllable viewing screen, including computers, tablet computers, and smartphones can also be used to read e-books. (Wikipedia) The electronic counterpart of a printed book, which can be viewed on a destop computer, laptop, smartphone or e-book reader. When traveling, a large number of e-books can be stored in portable units, dramatically eliminating weight and volume compared to paper. Electronic bookmarks make refering easier, and ebook readers may allow the user to annotate pages. (Encyclopedia) An e-book is the presentation of electronic files on a digital screen. Although the term e-book has implied its information expressed in a book form, it also shows that the book content is expressed on some device. These are two basic characteristics to recognize the difference between an e-book and a traditional book. Static writing and static image are typical of a book in general, including traditional books and e-books. However, an e-book also shows animated images and sound that a traditional book does not have. The files of an e-book can be provided as sound recorded units (disc), or can be downloaded from an information store by e-readers (including websites of the Internet). (Vst.vista.gov.vn) In our opinion, an e-book is a book software product having an interface and functions that act on the Internet or without the Internet. An e-book can be viewed on personal computers, e-readers, tablet computers, or smart-phones. An e-book can integrate many kinds of advanced and modern technology of informatics and media in order to serve information transmitting, learning, and studying best.

2.2. The characteristics of e-books According to us, there are four characteristics of e-books now, as follows: First, e-books are book software products, so they have mechanical structures as normal books. They have e-book pages. We can access their pages randomly as normal book pages. Second, e-books are book software products acting on the Internet or without the Internet. That is, e-books act on both on-line and off-line well (acting on off-line will restrict some functions of e-books). Indeed, when we refer to this thing, we want to refer two ideas. The first is that e-books act on the Internet. From this, we can design e-books having outstanding functions due to the Internet. That is, e-books can educate an infinite number of students, and educate without depending on location and learning time. Students can learn with many teachers at the same time and at various locations. All students who are not of the same skin color, country, nation, etc. can still learn. E-books are equipped artificially intelligent which can give predictions and feedback of tests, self-study exercises, ÂŠ 2015 The author and IJLTER.ORG. All rights reserved.

etc. immediately. E-books allow us to insert animated images, videos, etc. in order to make lessons deep and understandable. In addition, e-books allow us to organize learning acts that improve the activeness of students by using interactive tools and functions of e-books. The second is that e-books act on the Internet. From this, we can design e-books as normal packaged software. If we design like this, there are not the typical functions acting on the Internet such as the tele-learning function, the function of live images and sound transmitting, etc. This idea is the basic characteristic when we distinguish learning with the help of e-learning from learning with the help of e-book. E-learning without the Internet does not act but e-books without the Internet still act normally. Third, e-books always accompany with e-readers. Thus, besides transmittable content, e-books must focus on e-reader devices. These two factors are two close faces of e-books. The e-reader devices are diversified. E-books can be viewed on personal computers (PCs, laptops), e-readers (Kindle, Nook, Sony, Reader, etc.), tablet computers (Multiform: iPad, Android Tablets (Galaxy Tab, Kindle Fire, etc.), Surface; Specialized: Kno, Class-book) or smart-phones. Some learning forms (ex. e-learning) do not care about information transmittable devices. The devices of e-learning are often personal computers. Fourth, e-books have the abilities to integrate the forms of advanced and modern technologies and the media in order to serve the best information transmitting. When we refer to this thing, we focus on the continuous development of ebooks. If the technologies of e-books are advanced, so are e-books. Some technologies of e-books now are forums, e-dictionaries, text editing, hyperlink, and hypertext functions, etc. E-books have e-draft notebooks to help us draw figures, find out the solution to problems, verify, prove, express the visualization, etc. move and act on animated figures. E-books allow us to mark tests automatically and objectively, allow learners to do exams in a limited time, and decentralize marking exams to teacher. E-books give specific self-learning tasks to each student and allow us to divide students with an infinite number. After students do divided problems, e-books will give the results, a tree chart as well as a table of studentsâ&#x20AC;&#x2122; answers intermediately with just one click. E-books allow us to e-meet, e-teach, e-exchange, to share texts, sound, images, videos, PowerPoint, MP3, and MP4 files. E-books can also save the activities of users in e-documents. E-books allow users to improve the best activeness of learning and studying to develop independent thoughts and motivate users to learn knowledge for themselves. They also give users an impulse to inner motivation and creative desires. Some technologies which now we do not have yet and researchers are studying are artificial technologies. We need to equip e-books with an intelligent brain to allow them to self-learn, self-develop, and from that, they can solve problems given by other people or by themselves and solve new ones. In addition, e-books do not yet have remote technologies such as voice

technology, sound technology, etc. In the future, if these technologies are performed, e-books will develop greatly.

2.3. The classification of e-books According to us, there are four kinds of e-books now, as follows: Static e-books are digital text files as PDF files. PDF files are e-books with simple formats allowing readers to have access easily. The weak point of this kind is that it has no sound and graphics, and its structures are simple. It does not have many informatics functions except an only informatics transmitting function. Its strong point is that it has small capicity, so we can email it easily. Dynamic e-books without changing content are softwares that are the same as other learning software using programming language, such as Flash programming language. This kind is only good for the English subject and multiple choice software of other subjects because their interfaces are nice and their interactions are high. However, their weak points are that they do not allow users to change the contents and the contents given are only the contents in a particular area. These e-books require users to have high programming levels and nice and varied graphic design abilities. Dynamic e-books changing content are the same as e-books that are designed with PowerPoint software. E-books of this kind can integrate sound, videos and images and their strongest point is that they allow users to add, correct, change the contents of e-books as well as allow users to copy data. The weak point is that their images and interface are not as nice as dynamic e-books without changing content. Interactive e-books are book software acting on the Internet or without the Internet that integrate many kinds of advanced and modern technology of informatics and media. The e-books are designed in order to serve information transmitting, learning and studying well as well as remedy the weak points of traditional information transmitting, learning and studying. The e-books show all the strong points of the above e-books such as nice and eye-catching interfaces and e-information sending with real time. In addition, they have other strong points that the above e-books do not have. They have two structures which are the full structure and the empty structure. The full structure allows teachers to provide information for students with a pedagogic intension. The empty structure allows users to build materials. Students can access the content and the process of learning. Interactive e-books have clear strong points as mentioned above. As a result, this kind of e-books improves the best activeness of users. The weak point is that it has high designed cost.

Of all the above e-books, interactive e-books now have the best ability to support learning in general and learning mathematics in particular at secondary schools according to the orientation of innovating learning methods.

2.4. The comparison between e-books and traditional books According to us, the difference between e-books and traditional books is as follow: Table 1. The comparison between e-books and traditional books

Traditional books

E-books

Strong points

- They are ready to use.

- They are small and light according to the exploitable equipment, without depending on the quantity of e-books.

- They do not need sub-equipment. - Their cost is rather cheap. Weak points

- They can adapt to a fast changing world.

- The new contents can be updated - They are cumbersome and heavy easily and timely. when we carry a lot of books. - The contents are multi-media. - The contents are easily outdated in the context of a fast changing world, - They connect related documents: text especially in science and technology books, reference books, dictionary areas. books, and notebooks. - The contents are restricted due to the - They interact with two dimensions fees of their print. with customization ability for students. - The contents are not easy to reuse - They allow users to feedback. among related books. - They allow users to approach other - They do not contain multi-media documents intermediately. functions (video, sound, or interaction.) Weak points - Each document is single and does not - The cost is expensive (due to the need

have a connection. - It’s difficult documents.

of sub-equipment.) to

read

related - They need to use electricity. - The longevity depends on durability of user’s equipment.

the

2.5. The comparison between e-learning and e-book According to us, the difference between e-learning and e-book is as follow: Table 2. The comparison between e-learning and e-book

Factors Resources and learning location

e-learning

e-book

- The materials mainly - The materials mainly depend on documents depend on documents and graphic techniques. and graphic techniques. - The contents can - The contents can change according to the change or are packaged decentralization of the without changing. administrator. - It happens in an - It happens in an imaginary class or imaginary class or an happens every time and imaginary laboratory. everywhere.

Showed interface

Showed equipment

The interaction between

- It happens Internet.

the - It happens on the Internet or without the Internet.

- It does not focus on showed equipment. Showed equipment of elearning usually is personal computers.

- It focuses on showed equipment. Showed equipment of e-books is personal computers, tablets, e-readers or smart-phones.

- It is via e-mail, chat, e- - It is via e-mail, chat, e-

teacher and students

The interaction between students â&#x20AC;&#x201C; students

To assign a task and a check

forum, or e-class.

- The information is communicated passively, and sometimes not globally

- The information is communicated intermediately and globally according to the equipment.

Students interact Students interact directly via e-mail, chat, directly via e-mail, chat, or e-forum. or e-forum. - Space and time are limited.

- Space and time are not limited.

- The interaction depends on the consciousness of each member of the group.

- The interaction is 1-1. The effectiveness depends on each attended member.

- In the classroom

- Everywhere

The time predetermined limited.

is - At every time and the and time is not limited.

- The check depends on - The check depends on each individual test. criteria. - The tests have a limited - The tests have a fixed number of questions, number of questions. depending on the competence of students - It is based mostly on text. - It is based on audio and animation. - We can just assign our tasks if we have the - We can still assign our Internet. tasks without connecting to the Internet. To set an exam and to assess students

- Setting an exam is - Setting an exam is based on theory and based on the reality and

practical problems

situated problems.

- Giving marks is - Giving marks is based according to teacherâ&#x20AC;&#x2122;s on the competence of marking scheme. each student. - Tests are based on - Tests are not on printed printed documents. documents, so not costly. - We can just set an exam and assess students if we have access to the Internet. Feedback information

2.6. Building the eduvietnam.edu.vn

- We can still set an exam and assess students without having access to the Internet.

- It is almost not They are both synchronous. The synchronous and not feedback happens after synchronous. The some time. feedback is immediate and better. - The feedback happens mostly according to - The feedback happens group. according to individual, group, or divided learners.

interactive

e-book

the

address:

http://e-

The structure of the e-book includes the following parts: 1. E-book title: The e-book title is The interactive geometry e-book. The bar menu includes following items: 2. E-forum: E-forum is the Adobe Connect Enterprise software embedded in the e-book. E-forum allows us to organize conferences or e-conferences with a small or large size or we can apply E-forum to e-teach, etc. After clicking the E-forum button, the interface of the e-forum is as follows:

Share : this button is in the middle of the screen, allowing us to control the signal of input images.

Attendees: this item allows us to see and control the list of the television bridges connecting to the conference room. In this conference room system, we have three major object groups: - Host: the chairman and the major controller having the extreme right in the room. - Presenter: the lecturer, the person who is allowed and decentralized by the chairman to use the functions of the conference room, for example, using webcam, sharing documents, etc. - Participants: guests, the people attending a meeting, who have no power, but listen, or watch. They can use other functions when they are decentralized.

Chat: This item allows us to chat (to talk) and control.

This interface is of the conference room; however, the interface is not standard because we still see this interface in the Tab of the web browser. If we want to display full screen, we click on the item “Help” at the right corner, whose mark is yellow. Next, we choose the last item „Install Adobe Connect Add-in‟. After that, we choose „Yes‟ to continue to install Add-in. Note that we should use Mozilla Firefox, IE web browser because the installation of Add-in is simple and causes less mistakes than Chrome or Co Rom++….

Now, let’s study the most basic parts and functions of the e-forum: a. Images: this item allows us to Switch on or Switch off and control the signals of Webcam or Camera. In order to switch on the images we click on the black square Start My Webcam in the middle of the screen, in the frame Video. After clicking on this square, the screen displays the announcement: Camera and Micophone Access. We click on the item Allow and the screen displays the window Preview with camera images or Webcam. In order to swich on the image signals, we click the button Start Sharing. After clicking on this button, we will see the image signals. This time, the official signals of images will be transmitted to all of the television places. In order to switch off the image signals or to pause the images, we click on the Stop My Webcam button at the top right corner of the image frame. b. Sound: This item allows us to switch on or switch off the microphone or adjust the sensitiveness of the microphone. We choose Microphone and Test loudspeaker: this step is forced when we log in the conference room. This step is performed only one time and we do not need to do it again. This step allows us to choose the input signal of the microphone (if we have many input signals) and to test the loudspeaker as well as the sensitiveness of the microphone. In order to do this, we click on Meeting/Audio Setup Wizard on the menu bar. After that, we click on Next to continue the progress of installation.

Click on the Play Sound button to test the loudspeaker. If the signal of the loudspeaker is good, the blue signal runs along as shown above. Click on Next to install.

After clicking on Next, we see a new table with a list of microphone equipment available on the computer. We choose each item and test the microphone by choosing the input signals and then click on Next. To continue, we click on Record to see whether the microphone works or not. If the microphone is good, a blue signal runs along. Finally, we click on Next/Finish to finish the process of sound installation.

On the menu bar, we click on the symbol and then the down arrow and choose Connect My Audio. After completing the connection, the green symbol appears like this: . We can speak now. If we want to adjust better, we click on the down arrow beside the green microphone symbol: -

Mute My Microphone: allows us to pause the microphone. The microphone is mute when we click on this button. Disconnect My Microphone: is to stop the microphone. Adjust Microphone Volumeâ&#x20AC;Ś: Adjust the ssensitivity of microphone.

Note: whenever we do not use the microphone to speak, we have to switch it off by clicking on the button Mute My Microphone or when microphone symbol is green, we click on the green symbol, which means clicking on Mute My Microphone. The target of this function is to avoid the phenomena of howls and

echoes caused by switching on too many microphones among television bridges at the same time. At one time, there is only one television place allowed to switch on its microphone and speak. The others must switch off their microphones. Next, we adjust the sensitivity of the microphone suitably. We should not to choose the high sensitivity. c. resource share : this function allows us to share files such as still images, flash video files, PowerPoint files, the backgrounds of the computer (desktop) or the applications that are running on the computer, etc.

If we click on Share My Screen, the system has three specific items as follows: Desktop (allowing us to share all the applications which are running on the computer to other television places), Application (Sharing the allowed applications to suitable television places while the other applications are invisible), Windows (allowing us to share the applications of the operating system.) d. Raise hand: is to ask for the chairman’s permission to speak. We click on the menu bar, and then we click on the symbol (man icon) or click on the down arrow and choose “Raise Hand”. e. Discussion (Chat): is the same as the chat function in Yahoo Messenger. 3. E-dictionary: allows us to look up the meaning of English – Vietnamese mathematical words. 4. E-check: E-books allow us to test and assess students the same as the traditional way. First, teacher can test and assess students by observation. Teacher observes students by the function E-forum. From that, teacher gives the exact remarks about acts, gestures, and altitudes of students. Second, teacher can test and assess student by oral. This technique is effective. This technique assesses the expressed abilities of students such as listening, speaking, explaining, etc. Teacher can understand and help each student in the process of knowledge acquirement. Teacher asks students questions and listen to their answers via e-forum. Teacher and students must have computers and webcam to make the oral. Third, students can self-check. Students self-assess in order to develop the responsibility, the self-assessment, the independence, the confidence, and the creation of themselves. The self-check is done by stocktaking notes, ranking scales, and the answers. The self-check of students will remedy the weak points of inferiority. Fourth, teacher can test and assess students by the performance of students. Teacher allows students to use the functions of e-book to give solutions, to perform software as well as to give stocktaking notes or ranking scales on e-book. The assessment is easy and convenient. Fifth, testing © 2015 The author and IJLTER.ORG. All rights reserved.

and assessment can be done by collecting studentsâ&#x20AC;&#x2122; products. E-books save the traces of students on them automatically. If we use e-books, the saving is easy. Teacher uses e-books to set up files of students. Finally, testing and assessment can be done by using questions and exercises. Questions and exercises can be used to assess learning, for example, if we define the levels of students, the compilation needs to satisfy these following requests: +) Questions and exercises must be suitable for the program, the standard knowledge of Ministry of Education and Training, and the levels of students. +) Questions and exercises must be stated exactly and clearly in order for students to understand exactly. +) Besides the questions and exercises oriented to the basic requests, we need to prepare questions, deeper exercises requiring wider knowledge to encourage active thoughts. +) Assessing results is not only giving marks, but also giving comments on the contents, forms and learning methods as well as giving aid and plans to help students. The function e-forum is designed to test and assess using questions and exercises. The use of e-forum has these strong points as follows: +) Because e-forum decentralizes giving marks to many teachers, we can partly avoid subjective factor. E-check allows us to check, assess each student. We do not need a lot of efforts for checking and assessing writing exams. We only spend once entering data and use it for the next times. +) Besides the strong points of objective testing such as independence of the subjective ideas of teacher, assessing broad knowledge, covering all desired contents in a short time, and assessing the fast thought abilities of students, ebooks also allow us to self-assess automatically. When students finish the exam, we will know the result immediately. Using questions and objective exercises to test students does not require a lot of efforts. We only enter the data once and use it for the next times. +) We can combine writing form with oral form to test students easily. The interface of an objective test is as follows:

The interface of a writing exam is as follows:

The check and assessment in general as well as e-check and e-assessment in particular show that the e-check and e-assessment have more strong points than the traditional check and assessment such as more objective, more convenient, independent of the number of students, learning place, learning environment, and learning time. Moreover, the storage is better. The choice of questions and answers is also as convenient as the traditional one. 5. E-documents: are PDF files, HTML files as well as video files and others of the Internet. E-files provide a document resource to discovery learning. 6. Forum: allows us to communicate and discuss strong points, weak points of ebooks as well as gives the remedial solution to weak points. 7. Instruction: instructs in detail the use of e-book. 8. Introduction: introduces the principles and targets of e-books. 9. Connection: connects with the administrator of e-books. 10. Entertainment: includes intellectual games. Vertical menu includes the following items: 11. E-notebook: E-notebook is the major part of the interactive e-book. Its content performs two functions. The first is to organize learning acts according to discovery method. The second is to organize learning acts according to divided method. In addition, E-notebook has an e-rough notebook (the GeoGebra software is embedded in the e-notebook) allowing us to draw figures, predict the result of problem, etc. a. Discovery learning Discovery learning includes the following factors: +) A system of questions about the aware problem and solving problems through discussion. +) With the help and instructions of teacher, students give the way of approach, survey problems and collect data. +) Students solve problems by individual or group. +) Finally, students exploit and develop obtained problems, perform manipulations of science thoughts. From these factors, we see that discovery learning is divided into three various kinds:

+) Type 1: is called Guided discovery learning. Teacher gives problems and their solutions. Students find ways to explain. +) Type 2: is called Modified discovery learning. Teacher gives problems. Students give the solutions. +) Type 3: is called Free discovery learning. Students give problems and the solutions by themselves. (Nguyen, 2007) b. The levels of discovery learning There are many ways of approaching the levels of discovery learning. We can divide the levels of discovery learning based on the content and the way of approaching the content of a problem. However, we can also divide the levels of discovery learning according to the learning level of students. Weak students The discovery level of weak students is the simplest. Weak students only need to remember the information and events without needing to understand them. The discovery learning acts of weak students are usually: identification, definition, name calling, writing, expressing, etc. Average students The level of the discovery of average students is higher than of weak students. Average students have to not only understand and explain the given information but also apply the information to new situations and events and solve the given problems. The discovery learning acts of average students are usually: classification, comparison, ordering, reasoning, distinguishing, applying, changing, collecting, etc. rather good students The level of the discovery learning of rather good students is higher than the two above levels. Rather good students have abilities to divide information into many components in order to know the inner relations and their structures. In addition, students must have competence to connect pieces of information in order to create new ideas, generalize information and infer corollaries. The discovery learning acts of rather good students are usually: analyzing, explaining, connecting, asking questions, reasoning, collecting, combining, constructing, designing, developing, integrating, organizing, etc. Good students The level of discovery learning of good students is the highest. Good students must have competence to identify and evaluate information, problems, things, phenomena following a specific target. Good students should know how to create problems and solve them. The discovery acts of good students are usually: affirming, connecting, assessing, explaining, finding out the mistakes, predicting, creating, etc. Applying e-books to teach planar geometry according to the organization of discovery acts E-books allow us to teach according to the organization of discovery acts effectively with an infinite number of students. Teacher easily organizes discovery acts thanks to the page classification design of e-books. This thing is

illustrated by teaching The transformation in the first teaching period Introduction to transformation of the Vietnamese advanced geometric text book 11th. Example 1 With a given line a and hyperbola ( H ), for each point M on ( H ), let M ' be a point symmetric to the point M with respect to a . Give remarks on the positional points M ' when M moves on ( H ). Table 3. The discovery learning of the symmetry with the help of e-books Levels Learning Question content The corresponding figure aptitude of a student 1 Very Give remarks good on the positional point M ' when M is on the hyperbola. 2

Good

Give remarks on the positional M' points when points M are on the hyperbola.

Rather good

Give remarks on the positional M' points when points M are on the hyperbola.

Average

Click on the Interaction button to predict the locus. Find the locus of points M '.

Weak

Find the locus of points M ' .

After organizing discovery acts, teacher has students give remarks: The symmetry on line a is a transformation that maps each point M onto point M ' symmetric to the point M with respect to a. Symbols and terms The symmetry on line a is usually called S a . The symmetry on a line is simply called the symmetry. Line a is called the axis of the symmetry, or simply called symmetry axis. d. The process of divided learning Step 1: define learning strategy The strategy here is the divided learning one. Step 2: build the process of the divided learning Build the process of solving problems according to the divided strategy so that its steps are as close to an algorithm as possible. Step 3: predict and classify the mistakes of students Define the expressions and causes of each mistake in order that we set up a feedback mechanism. Students can make some familiar mistakes such as calculating mistakes, logical mistakes, etc. And from that, teacher gives remedial ways as well as supplemental instructions for students. Step 4: divide the activities of students Divide the process of learning into knowledge doses to form the program structure and connect these knowledge doses. The doses are divided into small ones so that they can predict the thoughts of students happening at these doses based on the answer results of students. Dividing knowledge doses is related to the classification of the mistakes of students. The learning effectiveness is much higher if the process of divided learning is suitable to the thought level of each student. Teacher needs to give chosen tests and students will be instructed based on suitable reasoning branches. Good and pretty good students will follow the shortest branch while average and bad students will follow the longer one. Step 5: write the chart of divided learning E-books automatically give the chart of divided learning for each student as well as the statistical table of correct and incorrect choices immediately. This thing helps students to self-check their mistakes in solving problems. e. Divided learning in the environment of e-books Divided learning in the environment of e-books allows us to remedy the weak points of traditional learning. It allows us to educate an infinite number of students, not depending on learning space, learning time as well as learning place. Divided learning in the environment of e-books happens at every time, everywhere. Students are provided with feedback as well as instructions immediately. After students finish their choices, they will get a statistical chart as well as a statistical table of their choices. This thing helps students to check

their mistakes easily, and from that, students can remedy the mistakes in the next problem. Example 2 With two given fixed points B, C on the circle (O ; R) and a point A moving on this circle, prove that the orthocenter of ABC is on a fixed circle. The divided solution in the interactive e-book is as follows: Knowledge (TT1) Figure According to the hypothesis, B, C are fixed on the circle (O ; R) . Thus, BC is fixed. The directions of line d are a set of lines that are parallel or coincident with line d. Hence, the direction of line (A) AH is (B) AB is (C) BH is (D) OH is invariant invariant invariant invariant

Branch 1 (Branch ( A )) If students choose answer ( A ), this are correct. They continue to deal with knowledge (TT2). Branch 2 (Branch ( B )) If students choose the answer ( B ), this is incorrect. The system of the e-book gives the complement instruction Hd and the situations of the mistakes of students: The complement instruction Hd  A moves on the circle, so the line connecting a fixed point with point A always has changeable directions.  A moves on the circle, so the orthocenter H of triangle ABC is also movable. Thus, the line connecting a fixed point with point H always has changeable directions. The invariant line is the line … Figure ( A2 ) AB . ( B2 ) ( C2 ) AO . ( D2 ) AC . Since AB always makes BC a constant angle.

AH . Since AH always makes BC a constant angle.

Since AO always makes BC a constant angle.

Since AC always makes BC a constant angle.

 If students choose answer ( B2 ), this is correct. They continue to deal with the knowledge (TT2).  If students choose the answers ( A2 ), ( C2 ), or ( D2 ), they come back to the knowledge (TT1) to relearn. Branch 3 (Branch ( C )) If students choose answer ( C ), this is incorrect. The system of the e-book gives the complement instruction Hd and the situations of the mistakes of students:

The complement instruction Hd  A moves on the circle, so the line connecting a fixed point with point A always has changeable directions.  A moves on the circle, so the orthocenter H of triangle ABC is also movable. Thus, the line connecting a fixed point with point H always has changeable directions. The invariant line is line … Figure ( A3 ) BH . ( B3 ) CH . ( C3 ) BA . ( D3 ) AH . Since BH Since CH Since OH Since AH always always always always makes makes makes makes a BC a BC a BC a BC constant constant constant constant angle angle. angle. angle.  If students choose answer ( D3 ), this is correct. They continue to deal with the knowledge (TT2).  If students choose the answers ( A3 ), ( B3 ), or ( C3 ), they come back to the knowledge (TT1) to relearn. Branch 4 (Branch ( D )) If students choose answer ( D ), this is incorrect. The system of the e-book gives the complement instruction Hd and the situations of the mistakes of students: The complement instruction Hd  A moves on the circle, so the line connecting a fixed point with point A always has changeable directions.  A moves on the circle, so the orthocenter H of triangle ABC is also movable. Thus the line connecting a fixed point with point H always has changeable directions. The invariant line is line … Figure ( A4 ) OA . ( B4 ) BH . ( C4 ) CH . ( D4 ) AH . Since OA always makes BC a constant angle.

Since BH always makes BC a constant angle.

Since CH always makes BC a constant angle.

Since AH always makes BC a constant angle.

 If students choose the answer ( D4 ), this is correct. They continue to deal with the knowledge (TT2).  If students choose answers ( A4 ), ( B4 ), or ( C4 ), they come back to the knowledge (TT1) to relearn. Knowledge (TT2) Since the directions of line AH are invariant, we use a

 translation of directions AH . Because there are the only points A, H relevant to directions AH , we  prove that AH is a constant vector. In order to prove AH is constant, we prove AH equals to a constant segment. According to the hypothesis, B, C are fixed,

Figure

so we will prove that AH equals to B ' B or C ' C and B ' B , C ' C also have directions AH . Hence, if we construct a parallelogram AHBB ' , BB ' is a diameter of circle (O) circumscribing triangle ABC. Since (A) line (B) line (C) line B ' B (D) B ' B is B' A is is B ', H , O perpendicul perpendicul perpendicula are -ar to BC -ar to AC -r to BC collinear. (because (because (because B ' B // AH B ' A // BH B ' B // AH ) ) ) and line B' A is perpendicula r to AC (because B ' A // BH ) Branch 1 (Branch ( C )) If students choose answer ( C ), this is correct. They continue to deal with the knowledge (TT3). Branch 2 (Branch ( A )) If students choose answer ( A ), this is incorrect. The system of the e-book gives the complement instruction Hd and the situations of the mistakes of students: The complement instruction Hd

   If B ' AC  B ' BC  900 , B ' C is a diameter of circle ( O ) circumscribing triangle ABC. Figure line B ' B  BC , so ( A1 ) ( B1 ) ( C1 ) B ' AEB is ( D1 )   B ' ACB B ' AC  B ' BC  900 B ' BFA is a a (since line B ' A is paralleloalso perpendicular gram ( F to AC ) is the foot of the altitude dropped A from to BC ).  If students choose answer ( A1 ),

parallelogram is a ( E is the foot pralleloof the altitude gram dropped from B to AC ).

this is correct. They continue to deal with the

knowledge (TT3).  If students choose answers ( B1 ), ( C1 ), or ( D1 ), they come back to the knowledge (TT2) to relearn. Branch 3 (Branch ( B )) If students choose answer ( B ), this is incorrect. The system of e-books gives the complement instruction Hd and the situations of the mistakes of students: The complement instruction Hd

   If B ' AC  B ' BC  900 , B ' C is a diameter of circle ( O ) circumscribing triangle ABC. Figure line B ' A  AC it follows ( A2 ) ( B2 ) ( C2 ) ( D2 )   B ' AC  B ' BC  900 B ' AOH B ' AEB is B ' AFB is is a a a (since line B ' B is parllelprallelogrparllelogralso perpendicular ogram. am ( E is am ( F is to BC ) the foot of the foot of the altitude the altitude dropped dropped from B to from A to AC ). BC )  If students choose answer ( A2 ), this is correct. They continue to deal with the

knowledge (TT3).  If students choose answers ( B2 ), ( C2 ), or ( D2 ), they come back to the knowledge (TT2) to relearn. Branch 4 (Branch ( D )) If students choose answer ( D ), this is incorrect. The system of the e-book gives the complement instruction Hd and the situations of the mistakes of students: The complement instruction Hd

   If B ' AC  B ' BC  900 , B ' C is a diameter of circle ( O ) circumscribing triangle ABC. Figure That B ', H , O are collinear is a wrong statement because ( A4 ) B ', D, E are collinear

( B4 )

( C4 ) B ', O, C are ( D4 ) ( B ', D, O B ', D, F collinear   are B ' AC  B ' BC  900 are colline- ) collinear ar

 If students choose answer ( C4 ), this is correct. They continue to deal with the knowledge (TT3).  If students choose answers ( A4 ), ( B4 ), or ( D4 ), they come back to the knowledge (TT2) to relearn. Knowledge (TT3) Thus, B ' is a fixed point on circle ( O ) ( B ' is the intersection point of the fixed line CO with O )) and //  B ' B  AH . Translating to vector B ' B , we have:

Figure

(A) H  F

(B) A  H

(D) E  O

 If students choose answer ( B ), this is correct. They continue to deal with the knowledge (TT4).  If students choose answers ( A ), ( C ), or ( D ), they come back to the knowledge (TT3) to relearn. Knowledge (TT4) Figure Because A moves on circle ( O ), the locus of points H (when A moves on circle ( O )) belongs to circle ( O ' ) being the image of circle ( O ) through



vector B ' B.

After answers have been chosen, the e-book automatically gives the branches of the user’s choice. Its interface is as follows:

The statistical table interface of answers is as follows:

12. Self-learn task: the e-book gives the self-learn request and self-learn task to each student. This function is similar to the function e-check.

13. E-portfolio: allows us to record the whole learning process of users. 14. The functions Register, Password change, information are the same as normal webs.

3. Results and discussion We delivered survey forms to 117 teachers of upper secondary schools in Ho Chi Minh city, Viet Nam in order to check the application of the interactive e-book in learning mathematics. The result shows that 70 teachers (59.83%) think that it is very necessary; 37 teachers (31.62%) think that it is necessary; 10 teachers (8.55%) think that it is not necessary yet; nobody thinks that it is not necessary. It shows that teachers are interested in using interactive e-book in teaching. The chart 1.1. The attitude of teacher towards the application of the interactive e-book in teaching The attitude of teacher towarsd the application of the interactive e-book in learning

8.55% 0% Very necessary

31.62%

Necessary

59.83%

Not necessary yet Not neceassry

We also surveyed the target of using interactive e-book in lessons. The result is as follows: Chart 1.2: The target of using interactive e-book in teaching

Percentage % 70

59.83

62.39

55.56

60 50 40

25.64

29.91

Percentage %

20 10 0

To ilustrate To stimulate To instruct according the interest student to to lesson of learning discover knowledge

To practise To the skills of strengthen applying e- the practice book

Thus, the majority of teachers in the survey asserted that they had used interactive e-books to stimulate the interest of students. However, there were still many teachers using the application of the interactive e-book to illustrate their lessons (70 teachers (59.83%)). Whereas the most important target of using interactive e-books is to help students find out knowledge. There were only 65 ÂŠ 2015 The author and IJLTER.ORG. All rights reserved.

(55.56%) teachers applying this. Besides, very few teachers (35 teachers (29.91%)) were interested in applying the interactive e-book to form knowledge, to practice skills, and to strengthen the practice of students. By observing, attending the lesson together with interviewing, we realize that there are a lot of reasons that make teachers fail to use the interactive e-book. Some teachers do not know how to use the interactive e-book. Some other teachers supposed that they were not fully equipped with informatics, so they were puzzled about teaching lessons. We also delivered survey forms to 251 students on the application of the interactive e-book in learning. The result is as follows: Chart 1.3. The effect of the interactive e-book on students

Percentage % 69.72 63.35

53.79

59.36

64.94

50 40 30

Percentage %

20 10 0

To increase the interest of learning

To help student to study

To motivate students to create initiatively

To help To help students to students to be self-learn confident

Chart 1.3 shows that students are interested in using interactive e-books in learning. There are 159 students feeling interested in learning (63.65%) and especially, 175 students (69.72%) feel that this e-book helps them to study. The interactive e-book improves studentsâ&#x20AC;&#x2122; self-learning and self-study. Students feel more confident when they learn with the help of the interactive e-book.

4. Conclusion E-books have many strong points than other kinds of education, which are easy to use and to learn everywhere, every-time. E-books can act on-line or off-line, which allows us to e-educate, to e-exchange, to transmit images, sound, MP3, and MP4 files, or to interact, etc. In addition, e-books can be designed completely to improve the activeness of students, the classification of students, the individual learning of students in order to help the process of learning develop best. Finally, e-books allow us to educate an infinite number of students without depending on location and learning time, so they are good for learning.

References Cumaoglu G., Sacici E, & Torun K. (2013) E-Book versus Printed Materials: Preferences of University Students, Contemporary educational technology, 4(2), 121-135. Leslie Czechowski (2011), Problems with E-books: Suggestions for Publishers, Journal of the Medical Library Association, 99, 3, pp 181â&#x20AC;&#x201C;182.

Dao, T. L., & Nguyen, V. H. (2010). Learning mathematical concept and theorem according to the divided strategy in e-learning environment, Vietnamese Educational Journal, (244), 33-34, 51. Encyclopedia. Retrieved from http://www.pcmag.com/encyclopedia/term/42214/ebook Hibbard, L. E. (2014), Ebooks: An Alternative to Paper Books for Online Students?, International Journal of Learning, Teaching and Educational Research, 46-56. Dracine Hodges, Cyndi Preston, and Marsha J. Hamilton (2010), Resolving the Challenge of E-Books, Collection Management, Vol. 35, No. 3 & 4, pp.196–200. Hanho Jeong (2010), A comparison of the influence of electronic books and paper books on reading comprehension, eye fatigue, and perception. Retrieved from file:///C:/Users/ADMIN/Downloads/Students_who_read_print_b ooks_have_a_better_reading_comprehension_of_the_text_and_prefer_paper _books_over_e_books%20(1).pdf. Le, T. T. T (2013). Some activities of forming evaluated competence of learning results for students at primary educated department, Vietnamese Educational Journal, (323), 39-40. Nguyen, V. H. (2007). Applying guided discovery teaching in the process of teaching mathematics at upper secondary school. Journal of Vietnamese Educational Science, 28-29. Nguyen C. T., Nguyen K., Le K. B., & Vu V. T. (2004), Learning and teaching the way of learning, Vietnamese Pedagogy University Publishing House. Nguyen C. T., Nguyen K., Vu V. T, & Bui T. (1998), The process of teaching – self-learning, Education Publishing House. Owen V., Tiessen R., Weir L, DesRoches D., & Noel W. (2008), E-Books in Research Libraries: Issues of Access and Use. Retrieved from http://carlabrc.ca/uploads/pdfs/copyright/carl_e-book_report-e.pdf. Tran T., Dang X. C., Nguyen V. H., & Nguyen D. N. (2011), The application of informatics in teaching mathematics at secondary school, Education Publishing House. Tran V., & Le Q. H. (2006), The discovery of grade 10th geometry with the Geometer’s Sketchpad, Eduaction Publishing House. Tran Vui., & Le Q. H. (2007), The discovery of grade 11th geometry with the Geometer’s Sketchpad, Eduaction Publishing House. Walters W. H. (2013), E-books in Academic Libraries: Challenges for Acquisition and Collection Management, Libraries and the Academy, Vol. 13, No. 2, pp. 187– 211. Moet.gov.vn. Retrieved from http://www.moet.gov.vn/?page=9.6 Vst.vista.gov.vn. Re trieved from http://vst.vista.gov.vn/home/database/an_pham_die n_tu/MagazineName.2004-06-09.1932/2004/2004_00004/MItem.2004-09-30.4740 /MArticle.2004-09-30.5116/marticle_view. Wikipedia. Retrieved from http://en.wikipedia.org/wiki/E-book.

International Journal of Learning, Teaching and Educational Research Vol. 11, No. 1, pp. 99-116, April 2015

Impact Investigation of using a Digital Literacy Technology on a Module: Case Study of Tophat Xue Zhou and Stella-Maris Orim Coventry University West Midlands, England Abstract. With digital technologies replacing the previous source of entertainment and communication and becoming the primary means by which we are informed and entertained, there is a need to be informed and competent with the relevant skills. Also, there is a greater need to use the digital devices that are employed by students using these technologies, which tend to keep them captivated at all times. The aim of this study is twofold: to enhance student performance by improving the approach to feedback and enhance student engagement by improving student class participation with the use of their „disruptive‟ devices. We hypothesize that student engagement with the digital devices lead to better and overall student performance. The areas identified for research were in relation to student feedback and engagement through the encouragement of their participation. The approach adopted in this research was the evaluation of the use of Tophat as a tool in the creation of an enhanced student-centred learning experience by creating an active learning environment. The strategy was to adhere to the usual idea of student tutorial and after an hour the first research activity started. This involved the use of the disruptive devices (mobile phones, iPads and Android devices) in a constructive manner. At the end of the session, Tophat was used to provide additional feedback and prepare them for their coursework. This was achieved through the use of „Tournament‟ to find out each week those that engaged and performed well. The results showed that 74.8% of the students are of the view that the use of Tophat enhanced their engagement in the module and 71.9% students perceived that higher level of feedback was received through Tophat. The impact of digital technology in higher education was discussed in this study. Keywords: Tophat; Technology enhance learning; Digital classroom; Students‟ engagement; feedback.

Introduction The effect of the use of digital literacy technology on students has become a topical area for research, which is not surprising as education in itself has to meet the students in their own arena. It is an acceptable requirement that

100

institutions of learning should engage in the development of creativity and innovation in students (Kärkkäinen & Stéphan 2013). However, this is a tallorder which is difficult to achieve when the educational systems still function traditionally and are losing the students to digital media; even in the classrooms. It is required that higher institutions should engage with providing students with skills for innovation, however, there is the question of the type of teaching, learning and assessment that can help students learn and acquire skills for creativity and innovation. Result from the research carried out by Sander (2000) suggested that students preferred to be taught by interactive lectures and groupbased activities. Their least preferred teaching and learning methods were traditional lectures, role-play and oral presentations. This is similar to the experiences with students in the classroom today. Coursework assessment preference was for essays, research projects and problems/exercises. Students’ engagement Engagement Student academic engagement involves the willing commitment of the students to the course of their academic pursuits. Kuh (2009) stated broadly that student engagement is reflected in the amount of time and effort students put into achieving college outcomes. This involves the participation in the achievement of the required learning outcomes before they come into the classroom, in the classroom and after they leave the classroom. Students‟ engagement could be enhanced and influenced by instructors‟ pedagogical choices and practices (Lane and Shelton 2001). Such good practices provide students with prompt feedback, encourage active learning and communicate high expectations, encourage interaction between students and faculty, cooperation among students and respect different talents and ways of learning (Chickering and Gammon 1987). Performance The academic performance of students is viewed as a measure of the students‟ ability to show that they have achieved the learning outcomes in a particular course. This can be measured in myriad ways such as: attendance monitoring, observation, interview, tracking their online engagement with course content and participation, self-reporting. Also, there are other traditional means of checking performance based on the achievement of learning outcomes: essays, oral and poster presentation, critical reviews, discussions, examination and test. Digital classroom in higher education In the past, traditional ways of teaching and learning were upheld in the higher education institutions. This was viewed as a process of transmitting content to the students and comprised of the lecturers deciding on the topics, teaching and assessment methods (Biggs and Tang 2010). However, in today's larger classes with diversified students, many lecturers could encounter major difficulties in sustaining academic standards. In relation to these difficulties Biggs (1999) states that they can be overcome when all components of teaching and learning are aligned constructively. This will be based on the premise that learning objectives

101

clearly define what is required from students. This teaching context of constructive alignment encourages students to carry out the learning activities which will help them attain the required understanding (Biggs and Tang 2010). Students are thus encouraged to engage in learning activities that are relevant in achieving these learning outcomes. Biggs and Tang (2015) explained that in this context, teaching is not topic-based, as is traditional teaching, but focuses on what students are intended to do after they have learned the curriculum topics. As a result, several efforts have been placed in developing activities that could engage students while enhancing the attainment of learning outcomes. Since the modern classroom is faced with several challenges with student engagement as one of the key issue, there have been efforts in the use of students‟ digital devices in fostering engagement while enhancing learning. Digital tools in education Due to several technological innovations and the dependence of students on these innovations, the higher education is undergoing significant changes in their methods of teaching, learning and assessment. This is because, as Prensky (2001:1) puts it, „today‟s students are no longer the people our educational system was designed to teach‟. Prensky (2001) further explained his view that due to the ubiquitous environment and the sheer volume of students‟ interaction with it, they process and think through information differently from their predecessors. He referred to them as „native speakers‟ of the digital language. This places a lot of requirement on the educational institutions that cater for the needs of these students. To meet these needs, the setting of the lecture hall remains the convention in most higher institutions, but, these are being enhanced by the integration of new tools, techniques and pedagogies (McAleese et al. 2013) which these „native speakers‟ are conversant with. This integration has necessitated studies in relation to the best use of innovative technologies in higher institutions. A reflection on what Ihde (1993) calls the „active relational pair‟ presents a view on the ways in which mobile devices have become absorbed into human social networking practices. Robinson and Hullinger (2008) also found that asynchronous instructional technology encourage students to achieve higher order thinking skills such as evaluation, analysis, synthesis, judgement, and application of knowledge. In corroboration with this view, Merchant (2012) observes that the mobile phone, with Twitter, Facebook and YouTube is heavily marketed by a range of providers due to human reliance in their everyday lives. This is due to the fact that most customers rely heavily on the use of these applications in their daily social lives. Also in line with meeting the needs of the digital age by innovative ways to engage students through the use of various applications, games and tools in the classroom, Wilson and McManimon (2014) corroborated with McAleese et al. (2013). However, they argue that best practice session is utilizing

102

the cloud as a tool to bridge the learning gap by providing more useful instruments for the enhancement of teaching styles. Student classroom response systems, which allow the instructor to gauge students‟ understanding of a given topic in real time, have been widely used in the higher education. The results showed that such fast response system can improve students‟ in-class attention, attendance, interests and involvement in the learning process, but also allow lecturer alter the content according to the students response (Bruff 2009; Terrion and Aceti 2012). Clicker as one of student response system has been proved to forester student engagement and success in the module (Hoeskstra 2008; Kaleta and Joosten 2007) It is arguable that the integration of these new technologies and approaches to education are already having a clear and positive impact on higher education provision as seen in the findings of McAleese et al. (2013). They established that this integration presents the opportunity to design the methods of teaching to individual students‟ needs, advancing learning analytics which lead to quicker feedback on students‟ performance. These technologies afford opportunities to learn anytime, anywhere and from anyone, provided the learner is motivated. This flexibility is crucial for nontraditional learners whom Prensky (2001) refer to as „digital natives‟. However, besides obvious benefits to the students, McAleese et al. (2013) articulated other benefits in relation to the institutions and governments. These are: a change in the approach to continuing professional development and lifelong learning; an important tool to governments in ensuring a diversity of provision within higher education systems to meet the needs of all learners and provision of the platform for reaching international markets which will complement existing developments in cross-border education. Also, there is the benefit identified in relation to the facilitation of greater collaboration with both global and local partners. Regardless of these identified benefits, there are some identified disadvantages to the move towards digitalising classrooms. In this regard, Merchant (2012: 770) stated that „as ownership and access to smartphones has spread into the teenage years, their place in institutions of formal education has been marked by contention‟. This is a view which is widely held in a number of conventional institutions of learning. The view that mobiles have no place in the classroom has recently been contested by educators, such as Parry (2011), who suggest that mobile learning and literacies should play an important role in education. Parry argues that it is crucial to recognise that mobile computing power radically changes the classrooms and more importantly, the „spaces that students inhabit and the conversations they have outside of our teaching‟ (2011: 17). Woodcock et al. (2012: 80) observed that Technology is increasingly being introduced into the classroom, primarily through student-led activities and typically through the use of smartphones. Some of these activities are webbased and involve the use of applications such as QuizIt, Socrative and Tophat. These activities have been found to be useful in a lot of cases, and Liburd and

103

Christensen (2013) suggest that technology, particularly web 2.0, can help increase the depth of learning by increasing interaction, critical thinking, and collaboration. However, there are identified issues in relation to the adoption of innovative teaching approaches. One of such is students‟ willingness to accept changes from the traditional way of teaching and learning. Besides the difficulty of some lecturers in adopting these innovative approaches, they are also a bit apprehensive about how students would respond to them. Brown et al. (2013:80) found that students are „in fact able and willing to use a classroom response and engagement system in order to increase engagement‟. Additionally, findings from their research suggest that students have an overall desire to use technology in the classroom. Brown et al. (2013:81) found in their study that contrary to perceived fears of lecturers that students may not all have smart devices in the lecture room, which can use for the innovative interactions, all the 413 respondents in their survey had „at least one device that would be capable of responding via text message or through a browser‟. Several devices and innovative tools and software have been introduced to different classroom settings. The results have varied slightly, but what all these results have in common is a positive disposition of most of the students to the innovative practices. Some have used clickers as a means of enhancing students‟ classroom engagement. Park and Farag (2015) explored the use of addressing clicker in a legal studies course. Findings from their studies suggest that both lecturers and students are more engaged with the course material and in the process of teaching and learning. He claims that clickers can be used to break-up the monotony of lecture, assess student understanding of material and difficult concepts, and identify areas of student misunderstanding and confusion. This can give the lecturer an idea of where to focus on. Also, they suggested that the use of clickers give every student, even those who are uncomfortable participating in class, an opportunity to provide input (Farag 2015). Common tools used in these innovative approaches include iPad, smartphones, laptops etc. Ravishankar et al. (2014) investigated the potential impact of the iPad/Tablet and its applications on teaching and learning in the area of electrical engineering. Their findings suggest that these devices may transform the teaching into a collaborative and interactive way. An example of innovative software is the android classroom response system. Karakostas et al (2014) carried out a pilot study on the use of QuizIt which is a new prototype real-time response system for Android mobile devices meant to enhance active learning methods and assess students' understanding. Their results from a questionnaire-based evaluation show that the students were quite positive about the use of QuizIt as a supporting system to their laboratory course. Some others have used Socrative as an online Student Response System to increase in-class student engagement. Dervan (2014) investigated the use of Socrative cloud-based (Internet) Student Response System (SRS) in improving student engagement and the learning experience, compared with the traditional

104

lecture setting. The findings from his study were positive suggesting that the use of Socrative as an online Student Response System increased the in-class engagement of students. Overview of Tophat In the light of the argument by Parry (2011), the use of several tools and techniques have been explored in educating students (Park and Farag 2015, Ravishankar et al. 2014, Karakostas et al 2014) but there is not much recorded on the use of Tophat. A traditional classroom is based largely on lectures, but with the use of innovative techniques and tools, students are encouraged to participate in the teaching and learning process. In most cases, they are encouraged to take ownership of their learning. With the use of Tophat (Tophat.com) engagement and participation is encouraged by stimulating the students through discussion start-ups, asking questions, and gauging understanding with the students‟ own devices such as iPad, smartphones and laptops. The software makers boast of providing a command centre where one can run and manage all aspects of the classroom from one platform. Research Purpose Besides the issue of the appropriate type of teaching, learning and assessment, there is also the issue with the students‟ learning styles which are diverse. An attempt at developing strategies that will focus on the different students‟ learning styles would result to student engagement and likely lead to enhanced student performance. Krause et al. (2008) argues that it is imperative to develop a broader understanding of engagement as a process with several dimensions. Since technology continues to be increasingly used by educational institutions (Becta 2009), this implies the need for appropriating pedagogical and education tools in supporting the enhancement of the quality of student experience. This need has necessitated several researchers to carry out studies on pedagogy, digital literacy technologies, student engagement and performance. However, there has not been any study on the impact of the use of TOPHAT as a digital literacy technology tool on students‟ engagement and feedback provision. Consequently, this study will focus on the following questions: RQ1: Does student class participation improve with the use of their „disruptive‟ devices on the Tophat platform? RQ2: What is the perceived impact of the use of TOPHAT on students‟ engagement in the module? RQ3: Does Tophat increase the amount of formative feedback received by students?

105

Methods Participants The participants were postgraduate international students in Engineering Management at Coventry University in UK. A total of 187 students enrolled in the course however, 103 students (55%) participated in the survey. Design and Procedure Students who enrolled in engineering management course were invited to participant in this study. Students were informed that their consent entailed the ability to withdraw out of the study at any time without punishment. The module lasted for eight weeks, as part of the assessment; students need to complete the coursework by applying the skills acquired over the period of the module. As part of the module requirements, students were directed on how to set up Tophat accounts during the lecture in the first week of the module. At the end of fifty minutes lecture session, students were asked to use their digital devices (e.g. smart phones, IPad, laptop) to access different questions, tournament and discussion forum, which were designed to monitor class attendance, test their understanding of lecture contents and serve as revision questions for the piece of coursework. The Tophat accounts were administered by the two researchers. University student email addresses were used to send the students invitation to the platform. Educational activities on the Tophat platform were set up by the researchers and these included questions design, attendance register, tournament etc. Instruments An online survey was designed and administered through Tophat to measure the students‟ perceptions of the impact of Tophat on their module engagement, teaching and learning experiences. Two sets of questions were designed and released to the students though their digital devices. The first set was for testing the students‟ understanding of the module content, while the second set was for investigating students‟ opinions and learning experiences. In the first set of designed questions, different types were used, which included multiple choices, word, numeric, sorting problem, matching problem and „click on target‟ questions. The second set comprised 4 Likert-scale question using four-point agreement level from 1 (Strongly disagree) to 4 (Strongly agree) and qualitative word questions. Analysis Descriptive statistics analysis in the form of percentages and means were used to analyse the demographic data of the participants. The students‟ gender, age and technology level were assessed. Four Likert scale survey questions focused on the impact of Tophat on students‟ engagement, understanding of module content, feedback, coursework feed-forward. The students were also asked to evaluate the use of Tophat and its impact on their engagement and feedback

106

received on the module. Additionally, students‟ preferred question type was observed based on their response. The researchers had access to the data of students‟ class attendance, correctness of responses, participation weight. T-test and ANOVA test analysis were carried out to assess the differences in students‟ engagement and understanding for the different gender and age groups respectively. PASW (SPSS) Statistics Version 17 was used to analyse the data exported from Tophat. To evaluate the degree of students engagement in the module through Tophat, thematic analysis technique was utilised to determine the efficiency of Tophat in enhancing students‟ engagement and feedforward for the integrated coursework.

Results The presentation of the results is mapped onto the structure of the earlier section “analysis”. The data showed that reliability cronbach‟s alpha (α) was 0.796, indicating acceptable reliability. Description of survey data The age of 103 participants ranged from 19 to 45 years, with 29% female and 71% male, which were distributed as follows: Under 21 years old (6.8%), 22 – 25years old (70%), 26 – 30 years old (12.6%), 31-35 years old (5.8%), over 36 years old (4.8%). The finding showed that the average score of participant self-appraisal of their technological competency was seven out of ten. The means and standard deviations of the responses to the four Likert scale questions addressing the students‟ perception of Tophat are provided in Table 1. Students‟ perceptions of the effects of Tophat were distributed as follows: enhanced their engagement (74.8%), enhance understanding of the topics (67%), enhance level of feedback (71.9%) and feedforward to complete their coursework (61.1%). Table 1: Descriptive statistics for Likert-scale questions Construct

Mean

Disagree

Agree

2.825

Strongly Disagree 6.8%

1. The use of Tophat has enhanced my engagement. 2. The use of Tophat has enhanced my understanding of the topics 3. The use of Tophat on this module has enhanced the level of feedback 4. The use of Tophat on this module for feedback has impacted positively on my ability to complete my coursework.

60.2%

Strongly Agree 14.6%

Standard Deviation 0.75961

18.4%

2.737

7.8%

25.2%

52.4%

14.6%

0.80393

2.835

5.8%

22.3%

54.4%

17.5%

0.78086

2.670

6.8%

32%

48.5%

12.6%

0.78451

107

Experience with Tophat based on Gender Two gender groups exist; male and female. The t-test assesses whether the mean results of the two gender groups are statistically different. Table 2 below shows the results of the t-test which examined the impact of gender groups on the scales of Technological skill, Engagement in the module, understanding of the module content, level of feedback, and coursework feed-forward. No significant difference was found between male and female students‟ response to those components. Table 2: Demographic differences by gender group (n=103) Variable Technological skill level Engagement Understanding of module Feedback Coursework Feedforward

Mean Male Female 7.1233 7.3000 2.8904 2.6667 2.7671 2.6667 2.8630 2.7667 2.7123 2.5667

Sig.

Multiples Differences

0.193 1.860 0.330 0.322 0.731

0.662 0.176 0.567 0.572 0.395

None None None None None

Experiences with Tophat based on Age The variable of age was divided into five groups: below 21; 22 to 25; 26 to 30; 31 to 35 and over 36. An ANOVA test was conducted to test how age influences different students‟ perceptions of Tophat. The results of the ANOVA test are shown in Table 3. The result showed that there is no significant difference among different age group of students‟ perception of Tophat. Table 3: Demographic differences by age group (n=103) Variable

8.429

B (2225) 7.039

Mean C D (26(3130) 35) 7.333 7.250

3.143 3.000 2.857 2.857

2.831 2.740 2.870 2.714

2.667 2.583 2.583 2.250

A (<21) Technological skill level Engagement Understanding Feedback Coursework Feedforward

2.750 2.750 2.750 2.750

Sig.

7.000

0.936

0.446

None

2.667 2.667 3.000 2.667

0.470 0.295 0.388 1.032

0.757 0.881 0.817 0.394

None None None None

E (≥36)

Multiples Differences

Thematic analysis of students’ engagement The qualitative data collected was analysed thematically with the initial codes selected based on the data. These codes were the overriding concepts from the data, which were presented as negative and positive views (Table 4 and 5, column 3), where applicable. Presented in Table 4 – 7, are quotes (column 2),

108

which are representative of the students‟ views, as depicted in the codes (column 1). In relation to the perception of the students‟ on the way Tophat affected their engagement on the Module, several views were aired. Some of these quotes are presented in Table 4 below and were coded broadly as impacting their: participation; provision of game-based learning, a learning community; emphasis on learning outcome and revision. A few of these comments presented in table 4 are: It has impacted my engagement in this module positively in the sense that it makes the course look easy by revising what has been taught in the tutorials with all the students which makes it more interesting, interactive and well understood through the feedback given after any questions asked. Tournament is an interesting feature and makes us more engaged to the Tophat Although a few participating students did not respond to this qualitative question, 47% of the entire sample believed it impacted on their engagement while 12% felt otherwise. A few of their quotes were: It was a total waste of time. I don't learn anything from it I learned from class lectures not from Top Hat In terms of the impact of Tophat on feedback provision, students‟ perceptions are presented in Table 5. The quotations are coded into six main parts: Clarity, feed-forward, quality, timeliness, approach and accuracy. Some comments by the students which are presented in table 5.

We get feedbacks immediately after each session Tophat helps to track our progress throughout the module; it has a user friendly interface However, 10% of the entire sample believed that quality of feedback should not be delivered on Tophat platform. A few of their quotes were:

I can't really say something about it that depends on evaluator if he/she thinks Tophat is of more value to this course but for me feedback should not be based on Tophat. Not much. As sometimes I don't understand why the answer is wrong. Students were also asked their most and least favourite feature of Tophat. Their quotations are listed in Table 6 and Table 7. According to students‟ response, the use friendly interface, game-based learning (tournament) and instant feedback encourage students to involve in the learning process. However, students also expressed their concerns in relation to the use of Tophat, such as: predominantly use a surface learning approach, low quality of the feedback, instability of technical support (wifi, devices, Tophat did not integrate with the commonly used education platform – Moodle).

109

Table 4: Summary analysis of the impact of Tophat on students‟ engagement Q: Would you say Tophat has impacted on your engagement in this module? Codes Positive (48) Negative (12) Participation  Easy for sign, it has the code for each lecture.  Nothing. I it has helped because I cannot be in the class mean when while I'm not in the class I touch this is the time I must to Game-based  Tournament is an interesting feature and learning makes us more engaged to the Tophat Learning community

 

Easy to communication Active learning



Emphasis on learning outcome



Being aware of the activity after every class, made me concentrate fully to the lectures in preparation of the activity. more engaged better understanding of terms through multiple choice layout Provided a guide line for the study skills I can find the key point of lecture



It has impacted my engagement in this module positively in the sense that it makes the course look easy by revising what has been taught in the tutorials with all the students which makes it more interesting, interactive and well understood through the feedback given after any questions asked. I remember the material after answering the fast question Improved understanding and rehearsal To remind me to finish my work on time By getting knowledge from MCQ It can help me to understand the module better It helps, but most of understanding is coming from class.



  

Revision



     



I learned from class lectures not from Top Hat It was a total waste of time. I don't learn anything from it Not impacted Help but not necessary

110

Table 5: Summary analysis of the impact of Tophat on feedback provision Q: How would you say Tophat has impact on the level of feedback in this module? Codes Positive (52) Negative (18) Clarity  Making me understand the meaning of  Not much. As answer. But, I have a suggestion that I think sometimes I don't we can finish Tophat at home understand why the answer is  To see how much students learn from wrong. lecture  It help me to familiarize with research Feed  development of my understanding forward  I don't know how to say that but its helped me through questions  it increased my eagerness in learning Quality  It quite level of feedback.  Not much feedback received  Increased my satisfaction some, but very little.  I'm not sure that Tophat impact in  high impact feedback part  Very good to get latest information  Satisfyingly great Timeliness  It has good impact because feedbacks are always given on time  We get feedbacks immediately after each session  The best thing with the Tophat that we get the feedback immediately  Get the feedback easy and quickly  It had been perfect, proving prompt n accurate feedback Approach  This way of feedback was a better way  Not significantly  Top Hat helps to track our progress  I can't really say throughout the module , it has a user something about it friendly interface that depends on evaluator if  Convenience he/she thinks  Easy way to deliver the feedback Tophat is of more  It's a good way to give feedback, helpful value to this  Provided a very complete feedback system course but for me  It has made my learning experience more feedback should fluid not be based on Tophat.  The feedback for the module has been very Accuracy accurate and good  It has impact on the level of feedback because of its interactivity amongst students  It will help by getting different ideas from different students

111

Table 6: Summary analysis of the students‟ dislike of Tophat Q: What did you like about the use of Tophat in enhancing your learning experience? Codes from the Actual quotations data Interactive  The fact that it made accessing information about the course approach online interactive.  Looked forward to the engagement after every class  engaged me with e learning Revision, feedback  Seems like a really good learning tool and feed-forward to  Choice question the coursework  Help me refresher course. Friendly interface.  It can help me to enhance my understand of course and if I have any question that I can ask there Tournament  More interactive than writing on paper and creates competition amongst fellow students  Increased competitiveness  Competition  Competitive with others at the end. Convenient  Was able to use device anywhere  It could be convincible approach but during in dissertation time.  Everyone can answer  Can use with phone Easy to use and  No feedback on answered questions user friendly  Easy to use  Different types of questions in a good learning way Test the  It made me question my understanding of the course because understanding, I'm unable to answer questions in time summaries the  It helps to understands the notes from the lectures learning outcome  Improve understanding of lecture  Giving me some choices, then ask which is correct.  Asking questions about topics thought in class served as a refresher

112

Table 7: Summary analysis of the students‟ dislike of Tophat Q: What did you not like about the use of Tophat in enhancing your learning experience? Codes from the Actual quotations data Surface learning,  No feedback on answered questions in-depth learning Technological infrastructure (WiFi, devices, and classroom, integrated with Moodle)

 

   

Time to response

Fully explanation on how to use it and the purpose of using it



    



Tophat is superfluous, why do not use our Moodle? The fact that the questions are done live in lectures. Sometimes I did not have my tablet or laptop with me and it is annoying to complete questions on a mobile device Sometimes no signal Sometimes it gets reset, keeps showing the same question, what I experienced during the questionnaire network problem I didn't like its complex procedure, everybody can't afford android phones and this software is working on assumption that everybody has android phones and that also in working condition every time, please keep learning procedure simple, so that we learn what is required not the technology for its usage. Not very useful. When I can use this program, the time, question, answer are limited. On other time, is just a picture on my iPad and nothing in it Not enough time cope the word answers Not enough time to complete all the questions Access through phone which is difficult to type A bit complication to register The fact that it wasn't introduced in the beginning of the course, If It was introduced earlier, I feel it would have really enhanced my learning experience Never use it before Open questions are not clear enough Not fully prepared and used when the class was done so to motivation from the class was lacking I should use it more and know an effective way to get use it It doesn't work well on my phone and the time frame before the next question pops in is too short have experienced difficulties to use it and don't know where to find feedback for my answers A bit hard to understand at the beginning how it works

113

Discussion RQ1: Does student class participation improve with the use of their „disruptive‟ devices on the Tophat platform? The data demonstrated that the students‟ participation in the module with the use of Tophat on their digital devices increased from week to week. User friendly interface and active learning approach resulted to increase in the student excitement and motivation to participate in the module activities. Variation in the types of questions used served to encourage student participation and held their attention for a longer period of time. On the other hand, some challenges were identified. Some of these were that it took around three weeks to encourage all the students to register on Tophat. Also, the students struggled with the technological infrastructure; these were in relation to access to internet connection in that classroom (wifi), affordability of smart mobile devices; proficiency in the use of these devices; desire of working on a familiar existing platform (Moodle). When the students did get involved with the activities, the data suggested that some of them (11%) were of the view that the time given for them to respond to the questions was too short. RQ2: What is the perceived impact of the use of TOPHAT on students‟ engagement? Quantitative analysis showed that digital attendance register through Tophat improved students‟ classroom attendance. The quick test at the end of each session gives students a clear idea of the expected learning outcome and the results informs them of their sense of mastery of the topic covered. The availability of the revision questions with correct answer also allows students to reinforce their learning, conduct constructive revision and further study. Furthermore, the use of tournament fosters students‟ behavioral engagement and affective engagement as competition with peers was found to increase students‟ enthusiasm and motivation towards the module. Discussion forum generated learning communities, in which students interacted with lecturers and peers easily through their digital devices. This close interaction between lecturer and students on the platform was found to improve students‟ critical thinking, knowledge acquisition and overall learning experiences. Most importantly, Tophat provide a platform that allows students who are shy and deliberate to express themselves. Such anonymous voting and discussion system maximize students‟ psychological adjustment and knowledge attainment. RQ3: Does Tophat increase the amount of formative feedback received by students? Both quantitative and qualitative results showed that the use of Tophat as part of the module plays an important role in providing instant and clear feedback. Easy access to the feedback could help students develop their understanding of the knowledge and feed-forward to complete their integrated coursework. However, using Tophat as a means of focusing on providing prompt and summative feedback results to a lack of detailed explanation and as such limits @2015 The authors and IJLTER.ORG. All rights reserved.

114

students‟ in-depth knowledge exploration. As most of the management module request students to think outside the box, vague feedback may restrict students‟ creativity and ability to formulate a broad understanding.

Conclusion This study aims to determine whether the use of Tophat could enhance students‟ engagement and provision of feedback/feedforward in a higher education module. The findings showed that the integrated features of Tophat such as tournament and quiz increase students‟ engagement across all the age range and gender. Prompt feedback received by the students enables the process of their revision and ability to apply the knowledge of concept acquired (feedforward). These results are important for educational practitioners as there is university emphasis on a move towards the application of digital technology in teaching, learning and assessment. The impact of Tophat was found to transform disruptive digital devices to efficient tools for pedagogical interventions. Furthermore, in relation to the classroom application, the students‟ digital devices were used to constructively impact the lecture sessions. They were found to engage with the learning through the activities set up on the Tophat platform. The implication from the findings of this research is that the use of digital devices as an innovative tool can enhance students‟ learning experiences by providing instant and quality feedback.

Limitation and Future work Not all the students that participated in the study had sufficient understanding of the use of the technological infrastructure. The Tophat system was not used repeatedly as a means of teaching and engaging the students fully during the lectures, but at the end of the 50minutes lectures. Students were provided with revision questions, which they attempted after lecture session. These questions were designed to engage them for 30 minutes and feed-forward for their coursework. These questions could have been more comprehensive and engaged them for a longer period of time. The research focused on one master level module, further study can be expanded on undergraduate level.

Reference Biggs, J. (1999) What the Student Does: teaching for enhanced learning. Higher Education Research & Development, 18(1), 57-75. Biggs, J. & Catherine, T. (2010). Applying constructive alignment to outcomes-based teaching and learning. Training Material for “Quality Teaching for Learning in Higher Education” Workshop for Master Trainers, Ministry of Higher Education, Kuala Lumpur. Biggs, J., & Tang, C. (2015). Constructive alignment: An outcomes-based approach to teaching anatomy. In L. K. Chan, & W. Pawlina (Eds.), Teaching anatomy: A practical guide (pp. 31-38). Switzerland: Springer International Publishing. Brown, E. A., Nicholas, J., Thomas, & Lisa, Y. T. (2014). Students‟ willingness to use response and engagement technology in the classroom. Journal of Hospitality, Leisure, Sport & Tourism Education, 15, 80-85.

115

Bruff, D. (2009). Teaching with Classroom Response Systems: Creating Active Learning Environments. San Francisco, CA: Jossey-Bass. Chickering, A.W., & Gamson, Z.F. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 39(7), 3-6. Dervan, P. (2014). "Increasing in-class student engagement using Socrative (an online Student Response System)." AISHE-J: The All Ireland Journal of Teaching and Learning in Higher Education. 6(2), 1977-1983 Hoekstra, A. (2008).Vibrant student voices: exploring effects of the use of clickers in large college courses. Learning, Media and Technology.33, 329–341. Ihde, D. (1993). Postphenomenology: essays in the postmodern context. Evanston: Northwestern University Press. Jones, I., & Day, C. (2009). Harnessing technology: New modes of technology-enhanced learning action research. Becta report Online available from: <http://www.sero.co.uk/assets/capital/ht_new_modes_action_research.pdf> . Kaleta, R. & Joosten, T. (2007). Student response systems: a University of Wisconsin system study of clickers. EDUCAUSE Center for Applied Research: Research Bulletin, 10 (1), 1-12. Karakostas, A., Adam, D., Kioutsiouki, D., & Demetriadis, S. (2014, November). A pilot study of QuizIt: The new android classroom response system. In Interactive Mobile Communication Technologies and Learning, 2014 International Conference on (pp. 147-151). IEEE. Kärkkäinen, K., & Vincent-Lancrin, S. (2013). Sparking Innovation in STEM Education with Technology and Collaboration. Krause, K.L.; Hamish, C. (2008). Students‟ engagement in first‐year university. Assessment & Evaluation in Higher Education, 33(5), 493-505. Kuh, G. D. (2008). Advising for student success. In V. N. Gordon, W. R. Habley & T. J. Grites (Eds.), Academic advising: A comprehensive handbook (2nd ed., pp. 6884). San Francisco, CA: Jossey-Bass. Lane, D. R. & Michael W.S. (2001). The centrality of communication education in classroom computer‐mediated‐communication: Toward a practical and evaluative pedagogy. Communication Education, 50(3), 241-255. Liburd, J. J. & Inger-Marie F. C. (2013). Using web 2.0 in higher tourism education. Journal of Hospitality, Leisure, Sport & Tourism Education,12 (1), 99-108. McAleese, M., et al. (2013). Report to the European Commission on Improving the quality of teaching and learning in Europe’s higher education institutions. Luxembourg: Publication Office of the European Union. Wilson, L., & McManimon, S. (2014, October). Bringing Back the ICEAGE: Interactive Cloud-based Engagement Activities Globalizing Education!. In World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education. 2014(1), 2065-2068. Merchant, G. (2012). Mobile practices in everyday life: Popular digital technologies and schooling revisited. British Journal of Educational Technology, 43(5), 770-782. Park, J. D., & Farag, J. D. (2015). Transforming the Legal Studies Classroom: Clickers and Engagement. Journal of Legal Studies Education, 32. Parry, D. (2011). Mobile perspectives: on teaching mobile literacy. British Journal of Educational Technology, 43(5),770-782. Prensky, M. (2001). Digital natives, digital immigrants part 1. On the horizon, 9(5), 1-6. Ravishankar, J., Epps, J.; Ladouceur, F., Eaton, R., & Ambikairajah, E. (2014). Using iPads/Tablets as a teaching tool: Strategies for an electrical engineering

116

classroom. In Teaching, Assessment and Learning, 2014 International Conference on (pp. 246-251). IEEE. Robinson, C. C., & Hullinger, H. (2008). New benchmarks in higher education: Student engagement in online learning. Journal of Education for Business, 84(2), 101-109. Sander, P., Stevenson, K., King, M., & Coates, D. (2000). University students' expectations of teaching. Studies in Higher education, 25(3), 309-323. Terrion, J. L., & Aceti, V. (2012). Perceptions of the effects of clicker technology on student learning and engagement: a study of freshmen Chemistry students. Research in Learning Technology, 20. Woodcock, B., Middleton, A., & Nortcliffe, A. (2012). Considering the Smartphone Learner: an investigation into student interest in the use of personal technology to enhance their learning. Student Engagement and Experience Journal, 1(1), 1-15.

117

International Journal of Learning, Teaching and Educational Research Vol. 11, No. 1, pp. 117-127, April 2015

Implementation of the 2006 Education Amendment Act on Indigenous Languages in Zimbabwe: A Case of the Shangaan Medium in Cluster 2 Primary Schools in the Chiredzi District Webster Kododo Great Zimbabwe University, Masvingo, Zimbabwe Sparky Zanga St. Josephs Tongoona High School, Jerera, Masvingo, Zimbabwe

Abstract. This research investigated the extent to which the 2006 Education Amendment Act was being implemented in selected Zimbabwean schools. The amendments were intended to usher into education most of the formerly neglected indigenous languages with the hope of propping up learner performance in schools. The research sought to establish the attitudes of teachers, school heads, pupils and parents towards the use of Shangaan as the medium of instruction in schools where the language is commonly used as L1 in the community. Attitudes of language users were seen to be a significant factor in education language innovation. The research recognizes that there is a gap between policy enactment and policy implementation. Policy implementation will either succeed or fail depending on the attitudes of implementers and the user community. Keywords: Shangaan, Shona, indigenous, attitudes, user community, Zimbabwe.

Background Like most of the African states, Zimbabwe is a multilingual country. As such, choice of language for use in the promotion of literacy and basic education for citizens has been debated for quite some time. This has been due to the view that the learner should be educated for his/her own benefit and ultimately for the benefit of the society (McNab 1989). Therefore, in an attempt to strike a balance between these two ends, Zimbabwe embarked on language policy innovation in 2006 (Education Amendment Act 2006). After independence in 1980, Zimbabwe had fashioned its first Education Act in 1987 meant to address the perceived (by some) negative dominance of the English Language where only Shona and Ndebele (the two main indigenous languages) were allocated inferior status

118

with the rest of the indigenous languages having no recognised role in education (Chimhundu 1984). Chimhundu (in Roy-Campbell & Gwete 2000; see also Royneland 1997) sadly notes that in the 1987 Education Act, the Zimbabwean government had failed to honour the proposal presented by the Minority Languages Committee in 1985 that in areas where there were dominant, specified indigenous languages, these should be taught in addition to Shona and Ndebele. The proposal was in response to debates and experiments on suitability of L1/L2 as medium of instruction in schools. The debate had been exacerbated by the UNESCO (1953) claim that L1 is the best medium for infant education. Owing to heightened debate and accusatory complaints to the effect that poor performance in schools were partly a result of the use of an L2 as medium of instruction in education, the Zimbabwean government embarked on language innovation that culminated in the 2006 Education Amendment Act. In the amendments, a new section on language use replaced the old Section 62, Chapter 25:4. Part of the new section reads; 1. Subject to this section, all the three main languages of Zimbabwe, namely, Shona, Ndebele and English shall be taught on an equal time basis in all schools up to Form Two level, 2. In areas where indigenous languages other than those mentioned in Section (1) are spoken, the Minister may authorise the teaching of such languages in schools in addition to those specified in Section (1), 3. The Minister may authorize the teaching of foreign languages in schools, 4. Prior to Form One, any of the languages referred to in Sections (1) & (2) may be used as the medium of instruction, depending upon which language is more commonly spoken or better understood by the pupils. One can note that pronouncing policy in education is one thing but implementing that policy is another. In spite of good intentions in policy formulation, there can be various factors that may affect the implementation of the said policy. One of such factors is language attitudes. As such, these researchers got inspired to investigate attitudes of language users towards the implementation of the language policy amendments in the sampled school communities. Attitudes have the capacity to affect policy implementation (Kadodo et al 2012). Research elsewhere shows that language choice for individuals tends to be influenced by culture, politics and economics (Diamond 1993). Language attitudes raises the question of what language users prefer when confronted by an array of competing interests ranging from social to economic? Choices are arrived at after serious balancing acts for individual users be it learners, parents, teachers, school managers or education managers. The dilemma implied here makes it necessary to investigate and ascertain language users‟ choices and the reasons they attach for such language choices. The implementation of the language policy as directed by the 2006 Education Amendment Act in Zimbabwe is subject to users‟ attitudes.

Research Question What are the language attitudes of pupils, parents, teachers and school heads at

119

Nandi, Mwenje and Nyahanga primary schools towards the use of Shangaan as medium of instruction as provided for in the 2006 Language Amendment Act?

Conceptual framework 1. Language and education Language is one of the most essential asserts gifted to humans, indeed a miracle that defines their existence (Aitchinson 2008). Being a key to communication, language can bring human beings together as much as it can set them apart. One primary cause of division among communities is differentiation of languages roles in education. Defining language, Finocchiaro (in Brown 1987:5) says that it is “a system of arbitrary … symbols which permit …people in a given culture, or other people who have learnt the system of that culture, to communicate or to interact”. Thus, apart from both the vocal and visual, “a language has a dual character… [as] a means of communication and a carrier of culture” (RoyCampbell & Gwete 2000:7). Durckeim (in Blackledge & Hunt 1985) defines education as an influence exercised by a generation on those not yet ready for social life [or those who wield power over those without]. In other words, communicating through language is one of the channels through which particular norms and values of a society can be transmitted from one generation to the next. Undermining the language of a people, therefore, devalues their dignity and leads to, unfortunately, a painfully slow death of such languages (Open Space 2008; waThiongo 1993). As McNab (1989:11) notes, “education is perceived as the terrain for excellence where language related inequalities and discrimination are manifested”. In other words, education must navigate through this terrain of language use to ensure that all cultural groups are catered for. Language is one of the significant factors in education that may lead to either the educability or miss-educability of learners. It can have a telling effect on the achievement of learners defining the quality of (or lack of) learning and teaching in educational institutions. It is on this basis that some researchers advocate for mother-tongue education seen as more effective for mastery of educational concepts (See Open Space 2008; Adegbija 1994; Bamgbose 1991; Mupande 2006; Brock-Utne 1993). This research argues that every language is (or can be) an effective and efficient tool for its users in education so long those users have firm control of that language. Firm control of language implies speaker capacity in both the linguistic and social nuances of the given language. The negation and relegation of a language from an education system in a country is tantamount to excluding the speaker community from national and developmental activities. For this reason, the United Nations propagated the universal declaration of linguistic rights (Open Space 2008) as paragon for the existence of even the so called minority communities. Language is a kaleidoscope that unlocks various meanings of existence for its users. Where learners, participants or community members are in firm control of the means of engagement (language as one key), they are able to display their abilities and contributions. Undermining a people‟s language is equally undermining their confidence, ability and contributions. Use of unfamiliar language leads to bad results (Prah 2000). This research was guided by these beliefs regarding the intricate relationship between language

120

and education, thus seeking to find out to what extent the 2006 Language Amendment Act was being implemented in selected schools. One can note that language users‟ attitudes are key to the implementation of language innovation (Kadodo et al 2012). This research sought to examine language attitudes of pupils, parents, teachers and school heads at Nandi, Mwenje and Nyahanga primary schools towards the use of Shangaan as medium of instruction as stated in the 2006 Language Amendment Act. 2. Attitudes to policy An attitude is “an organized predisposition to think, feel, perceive, and behave toward referent or cognitive object … an enduring structure of beliefs that predisposes the individual to behave selectively toward attitude referent” (Taylor et al., 1997: 130; Ajzen, 1988:4; Kerlinger, 1986: 453; Kosslyn & Rosenberg, 2006: 738). In fact, attitudes are the “very general evaluations that people hold of themselves, other people, objects and issues” (Tesser, 1995: 196). Beyond the basic functions of language, the roles that we subtly assign to various languages at our disposal reflect what attitudes we hold of each of these languages (Adler & Rodman 2001). The undertones in this move are the transfer, in dosages, of the said attitudes (positive or negative) to the unsuspecting language users. This is done through subtle insinuations that learning through language A leads to employment and better life whilst learning through language B may lead to lack of this and that. This method is borrowed from most colonised worlds where it has been successfully used to shape oppressed people‟s preferences. What we think about a language (cognitive attitude), what we feel about it (affective attitude) and what we actually do with that language (behavioural attitude) (Taylor et al 1997; Child 1993) are clear attitudinal demonstrations of the values we attach to each of the languages that are at our disposal. Consequently, the said attitudes will shape how we use language in the various activities of our lives. In the same manner, this also has visible influence regarding whether language policy innovation will or will not be successfully implemented. These researchers note that issues of language use are always bounded in power struggles as demonstrated during the colonial processes in various parts of the world. In the then Rhodesia (Zimbabwe now), there was incessant tutelage that English was „the language‟ whilst the local ones were of no consequence. As Diamonds (1993) at http://www.pre.org (accessed 20/08/2011) notes, “when a people have been told for many years that their cultures [so their languages too – our emphasis] are worthless, they come to believe it”. Consequently, this created positive attitudes (in most colonised people) towards colonisers‟ languages whilst conversely creating negative ones for indigenous languages that had been disempowered by lack of economic rewards for them (Kadodo et al 2012). Ironically, at independence, there has been unequal empowerment of indigenous languages in Zimbabwe. The 1987 Education Act tended to raise some languages (notably Shona and Ndebele) to national languages at the exclusion of the rest of other local languages. Regrettable to say is that the same power struggles hitherto stated regarding the colonial times came to haunt the language use in an independent state. Linguistic imperialism, hence likewise

121

cultural oppression, can be practiced even in democracies with the pretext of national unity which can be used to impose some languages over others. This, in our view, is no different from the ways used by former colonisers. Clearly, this explains how Shangaan, like other indigenous languages, was left in the cold for a number of years in an independent Zimbabwe without any meaningful role in education. It was not until the „noise‟ was getting louder and louder for recognition of other indigenous languages that the 2006 Education Amendment Act was gazetted. However, it is one thing to gazette policy but quite another to implement it. It was the intention of this research to establish the extent to which the Shangaan medium was being used (2006 Education Amendment Act) in selected schools in a Shangaan speaking community given the long history of linguistic and cultural oppression both in the pre and post independent Zimbabwe. The question begging answer is, „what measures were put in place to incentivise its use for users to have embarked on counter-attitudinal processes?‟

Factors that may raise or impede language policy implementation Successful implementation of language policy is dependent on various factors which explain why some policies fail but others succeeding. Policy without stringent implementation measures has slim chances of succeeding. In short, for policy to succeed, the operating environment must be permissive and supportive creating a „want‟ in the users to see that policy working. One key factor for language policy to work is the job market that should act as an incentive for the said policy to be favourable to the user community (Open Space 2008). Unless there are promises for the products of that policy being employable, then that policy may not be supported by the user community. Whatever language is economically incentivised will tend to attract positive attitudes from the user community. In close proximity is the role of media (both print and electronic) which can play a supportive role or become the devil‟s advocate (Ndinde & Kadodo 2014). Media is key in shaping people‟s attitudes by the way how it will campaign for or against a view or language. For instance, the nature of programmes and the language employed to air them could have an influence on people‟s language preferences. The points noted above are, in turn, dependent on the commitment of the leadership of a country (Roy-Campbedll & Gwete 2000). A language policy gazetted by leadership not politic enough will hardly succeed. The policy itself must be outlined in succinct language leaving no room for speculation or debate as to the meaning of statements. In other words, tentative and speculative language must be avoided so that each instruction is understood for what it is. An astute leadership will first ensure that relevant teaching staff and appropriate teaching materials are in place prior to gazetting education policy. It is no point pronouncing an education policy when appropriate preparations have not been done because that is tantamount to pronouncing its failure before implementation starts. Banda‟s 1968 Chewa only medium in Malawi and Ratsiraka‟s 1972 Malagachisation in Madagasca are cases in point. In short, it is important to look at both facilitating and debilitating factors within the operating environment so as to take corrective measures for policy implementation to succeed.

122

Methodology The research was guided by the descriptive survey design in an effort to understand the attitudes of pupils, teachers, headmasters and parents at the three selected schools towards implementation of the 2006 language innovation. The descriptive survey was seen as suitable for measuring user‟s attitudes (Chikoko & Mhloyi 1985) regarding the implementation of the 2006 language innovation. This research employed mixed methods (Maree 2010) where both qualitative and quantitative data were collected. Use of the mixed methods helped in the triangulation to increase reliability. Questionnaire, interview and observation data collection methods were employed. These collection methods were chosen for their versatility in data collection of users‟ language preferences. The questionnaire is a self-report instrument that guarantees anonymity of research participants (Best & Khan 1993) thus increasing chances for participants to reveal their deep-seated feelings regarding attitude referent. On the other hand, face-to-face interviews increased rapport with participants allowing elicitation (Brenner 2006) of information. These researchers also felt they needed to observe a couple of lessons at the research site to ascertain visible reactions of learners when what language (or combination of languages) was/were used. This was a useful method for learning learner behaviours (Sapsford & Jupp 2006) that betrayed learners‟ language preferences.

Selection of participants Purposive and random sampling techniques were used in sample selection. Whereas the three school heads (of the three schools) were purposively selected, ninety pupils, thirty teachers (from the three schools) and ninety parents (of learners at the three schools) were randomly selected using the lottery method.

Data collection procedures Four sets of short questionnaires were designed (heads‟, teachers‟, pupils‟ & parents‟) and these were distributed and collected by the researchers. Participants were allowed ample time to complete these each on his/her own. Interviews were also organised and carried out by these researchers (the three school heads, ten teachers and ten parents) whereupon the data was recorded in field notes. Six lesson observations were done by the researchers to get a close feel of what was happening at classroom level. Data was also collected in form of field notes on what transpired. The data collected was organised into contingent tables for analysis with interview data thematically factored into the discussion.

123

Results and discussion Table 1: Participants’ language choices

Question theme Preferred medium for education

Language Heads choice English 3(100%)

Teachers Pupils

Parents

18(60%)

63(70%)

52(58%)

Shona

10(33%)

15(17%)

18(20%)

-cannot understand English

Shangaan

2(7%)

12(13%)

20(22%)

6(20%)

15(17%)

12(13%)

-understand Shangaan -teachers & learners understand it

English

3(100%)

14(47%)

63(70%)

70(78%)

-language used in exams & for employment

Shangaan

10(33%)

12(13%)

8(9%)

-we can understand it

Best medium Shona for education

Learners‟ Shangaan favourite English language Shona Language used English by teachers to explain concepts Shangaan

Shona Learners‟ L1

Summaries of reasons -pass exams; -most books are in English

16(18%) 45(50%) 29(32%) 8(27%)

-it is the language of examinations

6(20%)

-pupils offer correct answers

16(53%)

-pupils participate

English Shona Shangaan

1(1%) 40(44%) 49(55%)

The „global‟ picture of results from the presented data in Table 1 above and Table 2 below reveal the respondents‟ attitudes to the three languages that are at their disposal. In Table 1, 100% of participating school heads, 60% of teachers, 70% of learners and 58% of parents preferred that English continue to be the

124

medium of instruction in their schools. Key to their decisions as revealed in their reasons are issues to do with the current situation where examinations for all content subjects are in English as well as employment opportunities that still use the English Language as gate-keeper. After all, English remains the granary for key information and technology that teachers and learners require for their teaching/learning activities in content subjects. For that reason, it becomes logical for them to prefer a language medium that they perceive as giving learners better life opportunities and that language in their view is English. In spite the fact that 55% were L1 Shangaan and 44% were Shona L1 speakers, the majority of learners were very pragmatic thus, preferred English because in their view it promises employment opportunities. There were other issues that were cited by teachers during interviews where they felt that Shangaan as an education medium did not have the terminology range to adequately convey the scientific and commercial „worlds‟. In conjunction with this perceived problem is the non-availability of reading materials even for Shangaan itself as a subject, let alone reading materials written in Shangaan for content areas. These researchers, however, note that languages can grow depending on what range of uses we assign them. Likewise, Shangaan can develop to cover the range that the users so desire it to cover so long we avoid the pitfalls of purism where we want languages to be what they were centuries ago. Regarding materials and manpower development, these are issues that can be resolved say in phases provided the policy-makers, implementers and user communities are all agreed on the necessity for such move. Simply put, policy development should not be conceived in the top-down form which, more than often, leads to tissue rejection (Obanya 1987). There is need for extensive positive consultations. As implied above, respondents also noted that none of the teachers on the ground at the time of data gathering was ever trained to teach in Shangaan. This was seen to be a handicap to the implementation of the 2006 policy. This, coupled with the reality of lack of materials on the ground, questions the sincerity of policy-makers in gazetting the 2006 Education Amendment Act. The ground surely was not „flattened‟ for indigenous languages to be used as media of instruction in schools. Scales have always, and sadly remains so, in favour of the English Language. There has not been any attempt to incentivise the indigenous languages as a measure for users to commit counter-attitudinal processes and develop positive attitudes towards them. Table 2 below give results of observations made by these researchers when they attended some of the lessons in the research schools.

125

Table 2: Observed effects of language use in 6 lessons

Areas & behaviours noted Les son No Subject taught

Code Pupil’s visual switching expression /mixing Religious & More English Low Look-warm Moral Ed. than Shona mostly Mathematics English None Indifferent

Mathematics

More Shona & High Shangaan than English

Social Studies

Mathematics

Environmenta l Science

English but High with more Shona & Shangaan English & a Low little bit of Shona English & a Moderate little bit of Shangaan

Language used

Participation level Moderate Very low

Enthusiastic & Very high for confident Shangaan, high for Shona & lower for English Enthusiastic & Very high in excited either Shona or Shangaan than English Look-warm Low mostly Look-warm

Moderate

Lesson observations, as shown in Table 2, allowed these researchers a small window to observe language use and learner responsiveness in language ecology. The researchers note that overall teachers maintained English Language as the focal but with varying degrees of permissibility of Shangaan and Shona. Apart from Lesson 2 as shown in the table where the teacher enforced all English, the other 5 lessons had varying degrees of code switching/mixing. The pattern emerging from the lessons seemed to drive to a conclusion that the more permissible the teacher was to code switching/mixing (Lessons 3 & 4 in Table 2) the more the classroom activities were liberated allowing learner hype and participation (Freire 1972). As Freire notes, such environment leads to the traditional teachers‟ and learners‟ roles mutating allowing the teacher and learner each to be both an educator and learner at the same time. This would create classroom partnerships where learners are carefully and subtly moulded into mature critical learners. On the contrary, the teaching/learning environment in Lesson 2 presented a contrast to Lessons 3 and 4. It is not unreasonable, in the case of Lesson 2, to be worried that the teacher may be tempted to „tell‟ the learners his/her own knowledge than creating enabling environment for learners to engage issues leading to their own learning. Else

126

how are these learners engaging issues in this handicapped and ominously quiet environment? Based on the observations above, we conclude that the question of what language of instruction in education matters. Looking at the sets of findings in this research one notices some internal contradictions. Significant to note is that leaners seem to enjoy, and possibly learn more meaningfully when they are in control of the language of education. This should, supposedly, see them opt for their L1 as medium of instruction. However, when learners consciously make a choice they still opt for a language they may not have efficiency in. The matrix in the maze is that their conscious choice of language is driven by what they yen for in their future lives, more of a wish-driven choice. This is what holds sway to their language attitudes. Unfortunately, a number of education systems today have become so mechanistic and examination-driven that teachers can just coach learners in their quiet environments to pass the national examinations in spite of their low proficiency. Is this not possibly the reason why the industry sector is perennially complaining of raw graduates from some of our institutions?

Conclusions Based on the findings of this research one concludes that choice of language of education is not always a rational process but is more often emotional. Notwithstanding that learners may not be efficient in a language they may still opt for that language as medium of instruction owing to their perceived life opportunities. For that reason, before legislating any language policy there is need to ascertain users‟ language attitudes. If these are not in tandem with the proposed language we would rather incentivise the intended language for users and implementers to prefer such language. This research also concludes that contingent planning should precede gazetting of any education policy. This is possible in situations where there are open and well-intended consultations. The fact that this was not meaningfully done as precursor to gazetting of the 2006 Education Amendment Act in Zimbabwe, the innovation has not been embraced by the user community and therefore has not succeeded.

Declaration The researchers wish to declare that there was no research grant attached to this research by any organization.

References Adegbija, E. (1994). Langauage Attitudes in Sub-Saharan Africa: A Sociolinguistic Overview. Clevedon: Multilingual Matters. Adler, R.B. & Rodman, C. (2001). Communication. London: Routledge. Aitchinson, J. (2008). The Articulate Mammal: An Intorduction to Psycholinguistics 5e (5th ed.). London: Routledge. Ajzen, I. (1988). Attitudes, Personality and Behaviour. Milton Keyness: OUP. Bamgbose, A. (1991). Language and the Nation. The Language Question in Sub-Saharan Africa. Edinburgh: EUP. Best, J.W. & Khan, J.V. (1993). Research in Education (7 th ed). Boston: Allen & Bacon. Blackledge, D. & Hunt, B. (1985). Sociological Interpretations of Education. London: Groomhelm. Brenner, M.E. (2006). “Interviewing in Educational Research” in American Educational

127

Association, Handbook of Complementary Methods in Education, pp. 357370. Brock-Utne, B. (1993). Education in Africa, Vol. 1 Report No. 3. Stockholm: Bindern. Brown, H.D. (1987). Principles of Language Learning and Teaching (3 rd ed.). Englewood Cliffs: Prentice Hall. Chikoko & Mhloyi (1985) at htt://www.tandfonline.com (accessed 06/10/11) Child, D. (1993). Psychology and the Teacher. London: Holt Rinehart. Chimhundu, H. (1984). “Towards a Policy on the Teaching of African Languages in Zimbabwe” Paper Presented at the Linguistic Conference of SADAC Universities, Malawi, 19-21 November 1984. Diamond (1993) at http://www.prel.org (accessed 20/08/11) Freire, P. (1972). The Pedagogy of the Oppressed. New York: Herder & Herder. Kadodo, W., Kadodo. M., Bhala, T. & Bhebe, C. (2012). “The influence of teachers‟ and students‟ attitudes towards the use of Shona as medium of instruction in secondary schools” in International Journal of English and Literature, February 2012, 3(2), pp32-39. Kerlinger, F.N. (1986). Foundations of Educational Research (3 rd ed.). New York: Holt, Rinehart & Winston. Kosslyn, S.M. & Rosenberg, R.S. (2006). Psychology in Context (9th ed.). Delhi: Pearson Education Inc. Maree, K. (ed.) (2010). First steps in research. Pretoria: Van Schaik Publishers. Mupande, I. (2006). Silent Voices: Indigenous Languages in Zimbabwe. Harare: Weaver Press. NcNab, C. (1989). Language Policy and Language Practice: Implications and Dilemmas in Ethiopian Education. Stockholm: University of Stockholm. Ndinde, S. & Kadodo, W. (2014). “The role of community-based information centres in development: Lessons for rural Zimbabwe” in International Journal of Learning, Teaching and Educational Research, 2(1), pp. 44-53, February 2014. Obanya, P. (1987). A Curriculum Development‟s Manual. Switzerland: WCOTP. Open Space (2000). Language Inclusion Participation and Expression, 2(1), November 2008, Johannesburg: Open Society Institute for Southern Africa. Prah, K.K. (ed.) (2000). Between Distinction and Extinction: The Harmonization and Standardization of African Languages. Cape Town: Centre for Advanced Studies of African Societies (CASAS). Roy-Campbell, Z.M. & Gwete, W.B. (2000). Language Policy and Planning. Harare: University of Zimbabwe. Royneland, U. (ed.) (1997). Language Contacts and Language Conflict. Voldo College, The Iva Aasen Institute: University of Oslo. Sapsford, R. & Jupp, V. (eds.) (2006). Data Collection and Analysis. Los Angels: Sage. Taylor, S.E., Peplau, L.A. & Sears, D.D. (1997) Social Psychology (9 th ed.). New Jersey: Prentice Hall. Tesser, A. (1995). Advanced Social Psychology. New York: McGraw-Hill Inc. The 1987 Education Act. The Government of Zimbabwe. The 2006 Education Amendment Act. The Government of Zimbabwe UNESCO (1953). The Use of Vernacular Languages in Education. Paris: UNESCO. waThiongo, N. (1993). Decolonizing the Mind. Harare: ZPH.

128

International Journal of Learning, Teaching and Educational Research Vol. 11, No. 1, pp. 128-140, April 2015

The Concept of In Situ Lecturing Joachim R. R. Ritter and Ellen Gottschämmer Karlsruhe Institute of Technology, Geophysical Institute Karlsruhe, Germany

Abstract. Traditional teaching at university level takes place mainly in lecture rooms without any direct linkage to the real subject. Thus even during well taught lectures the human, in our case students’, multisense receptivity is stimulated only in a limited way. This means that just a part of the actual subject matter can be transferred. Therefore, students will not learn to observe the whole range of the circumstances and environmental parameters involved in a specific subject. This problem arises especially in natural sciences and, partly, engineering teaching in which the environmental setting is often a key to successfully understand complex processes, proportions and scales as well as human (counter) actions. In our concept “in situ lecturing” we teach at the place which is being studied, hence “in situ”. In situ lecturing is a valuable pedagogical concept to develop student’s understanding of basic concepts, to enable them to transfer concepts and theory to local conditions, to train practical skills and to promote a comprehensive understanding of processes. This approach is achieved by an integrative combination of pre-courses and practicals in the classroom, followed by in situ lectures, practicals and seminars as well as a final reporting. Examples are presented from geophysics but these may be transferred to many other disciplines. Keywords: university teaching; field work, teaching method.

Introduction The curricula of most degree programmes in natural sciences and engineering predominantly involve classroom lectures, practicals for solving exercises, seminars with student presentations and, rarely, excursions or actual fieldwork. Classroom lectures, practicals and seminars account for the by far most part of the higher education. Classroom lecturing is vital for setting a solid base for primary skills such as understanding of theoretical concepts (e.g. mathematics), first principles (e.g. basic physics), taxonomy and laboratory working techniques. However, classroom lectures are mostly passive events for students with a one way communication, although there are many concepts to stimulate the audience (e.g. Laws, 1991, Powell, 2003, Reiber, 2006). There are also many applied courses which concentrate on specific themes, e.g. monitoring of environmental parameters, assessment of hazardous natural processes, construction of buildings, instruments and machines etc. Teaching in some © 2014 The authors and IJLTER.ORG. All rights reserved.

129

subjects requires indispensably out of classroom instruction, e.g., geosciences (for a review see Butler, 2008 and references therein). In a subsequent professional career such activities require skills which cannot be trained adequately by only classroom lecturing (Lonergan & Andresen, 1988, Kastens et al., 2009). During lectures the students should learn and understand the necessary theoretical and conceptual basis of a subject. Yet, when examples are displayed in the classroom, students experience only a limited aspect of a study or situation which is presented in a slide or movie. Often a detailed and comprehensive description of a situation or object and its environmental circumstances and parameters is missing (even the lecturer may not know under which circumstances a photo or film was taken). The true scale of objects is often not clear to students, if they do not experience the real objects themselves. In addition, visual material for class room teaching is often biased by wellchosen factors or circumstances: optimum light conditions to illuminate an object, undisturbed environmental conditions etc. Students should participate actively during practicals where they present their solutions to questions and exercises distributed by the lecturer and where they apply the principles and concepts which were taught during the lectures. Such solutions are mainly developed outside the classroom, sometimes within a small group of students. In the majority of practicals, the students are given real data e.g. measurement curves such as seismograms, rock samples, electronic modules etc. to work on. Such material is often carefully selected to avoid complications due to noisy data, unclear interpretational options etc. The problem of scale arises also in many subjects, e.g. a complete bridge cannot be examined in the lab during an engineering course. Seminars are conducted in such a way that students learn to present, discuss and think about case studies from articles in the literature (which they read typically in the late evening just before the seminar is be held). Again, links to a realistic situation are given in parts only and experience of the real world is missed. During many excursions the students are carried from one point of interest to the next one without any active participation. Sometimes even the connection between basic theoretical principles and objects of an excursion are poorly explained and, in this way, the link between them remains obscure. Students may also be confused if there are different lecturers in the classroom and during the excursion who use different ways and concepts to describe the same object or process. Only rarely is actual fieldwork is done by students (e.g. mapping, collecting, measuring in the field or assembling of an instrument). However, well prepared lecturing outside the classroom trains students for their professional career and widens their perspective (Hursh & Borzak, 1979) due to the inherent interdisciplinary nature of outside teaching (Claiborne et al., 2014). In the following section we propose ways to better link theoretical concepts, passive and active learning and real situations by adding to the theory given in ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

130

lectures to include practicals and seminars where the subject is actually based. This concept can be compared to the idea of learning a language in the country where it is actually spoken. In that case, the language will not only be adopted during the language classes, but also during the rest of the day. In a similar way, during an in situ class, students are confronted with the matter all day long. This stimulates the students to think about the subject in more detail. Such an approach leads to questions (and answers) which would not have been asked (and answered) during a typical classroom lecture. Thus, students get a much deeper and more comprehensive understanding of the subject. An example can be seen in Figure 1, where students give in situ a seminar on the flanks of Stromboli volcano, Italy. The background of our examples is described in Box 1.

Figure 1: Student giving an in situ presentation (step E) about volcanic processes at the slope of Stromboli volcano, Italy. The used screen is the backside of a poster which was displayed earlier for another student presentation. A notebook and a beamer with battery power are utilised for the projection in the open field; volcanic ash plumes from small eruptions could be observed from this place during a pleasant evening lecture (left picture courtesy of J. Käufl).

Concept and Examples The term “in situ” is the Latin expression for “in the original place” or short “in place”. We use in situ to express that teaching is done outside the classroom at a place which is directly linked with the subject matter. Therefore, in situ can be related to a lecture, practicals, exercises or a seminar with student presentations. Our experience from in situ lecturing at bachelor’s and master’s degree level shows that a balanced combination of lecture presentations, practicals with exercises and seminar-type elements is a meaningful way to educate students in many disciplines. Of course, the subject or theme must be suitable for successful in situ lecturing. Geosciences are obviously very suitable for in situ lecturing, because the study objects such as volcanoes, mines, observatories etc. are obvious targets which should be studied outside the classroom (Thompson, 1982, Butler, 2008). Education in many other disciplines may also benefit from in situ lecturing, e.g. biology (zoos, wilderness), physics (particle accelerators), history (historic sites) and many others.

131

Table 1: Overview of the concept of in situ lecturing with its main elements A - F.

Step

Type

Main Functions

preparatory lecture

 introduction of first principles  theoretical basis  outline and organisation of in situ part

preparatory student work

 reading of relevant literature  preparation of presentation, e.g. for classroom presentation or in situ seminars  preparation of handouts or lecture notes for fellow students  organisation of in situ part

in situ lecture

 repetition of first principles and basic concepts  presentation of subject matter with direct link to the subject

in situ practical

 solving exercises directly linked with the subject  observation, identification and description of elements related to the real subject  onsite training

in situ seminar



post-trip documentation

students:  writing a final report  reflecting on the subject and preparation for exam

presentations directly related with the subject, in the field or during evening seminars

lecturers:  reflecting on the course for further improvement  reflecting on students’ comments

Figure 2: Students discussing and working on a short-time exercise during a preparatory lecture (step A). Such exercises increase their attention to the lecture and help the students immediately reflect the lecture subject. © 2014 The authors and IJLTER.ORG. All rights reserved.

132

Our pedagogical concept for integrated in situ teaching includes the following elements or steps (Table 1): A) preparatory classroom lectures on basic concepts and theoretical work, B) preparatory student work which includes the repetition of the basic concepts, reading of relevant literature, preparation of classroom presentations, in situ seminars and lectures, as well as getting familiar with the specifics (e.g., geology, geophysics, eruption style, vegetation, accessibility) of the location of the field study, C) in situ lectures in order to reinforce the theoretical background and to transfer the basic principles to real situations, D) in situ practicals to actively apply the basic theory learnt during the lectures, within the actual situation, E) in situ seminars with presentations by the students to explain observations in the field, review the background, explain the local history or add details as well as discuss uncertainties and other problems related to the present in situ state of the subject, and F) post-trip documentation. The succession A) â&#x20AC;&#x201C; F) is arranged in order to achieve a high transfer of knowledge and skills to the students. Especially the active parts foster the training of competences with a special focus on real professional situations and working environments. Details and examples for steps A) to F) include: A) Preparatory classroom lecturing: higher education must include instruction in the solid theoretical background of the subject matter. Thus our geophysics teaching includes the presentation of basic physical concepts and equations. These theoretical parts are taught best in a traditional class course where one can concentrate on physics theory and the related mathematics but it should also include practical applications, case studies and quantitative parameter descriptions. For instance during a course on induced seismicity (see Box 1), we explained the theory of tectonic stress with applications to Earth materials and to earthquake fault zones as well as the influence of stress changes generated by humans. In order to overcome their passive status, we introduce student activities such as brief exercises (Figure 2) or pop quizzes or provocative questions which they solve in small teams. By these means, the students are continually motivated. (Powell, 2003, Handelsman et al., 2004). The preparatory classroom lecturing includes also a first introduction to the sites of the in situ lecturing as well as organisational issues.

Figure 3: Students giving a poster presentation on borehole logging methods during preparatory student work (step B). Each student had to prepare and explain a part of the presentation to get familiar with the subject matter discussed in the field. ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

133

Figure 4: Local field engineer (left) during an in situ lecture (step C) in a quarry with volcanic rocks, Vogelsberg region, Germany.

B) Preparatory student work: active participation of students is a key element for understanding the subject matter. For the preparation of the in situ part we place a special emphasis on reading the relevant literature. We select mainly articles in scientific journals to force the students to read original science reports and get familiar with scientific approaches and the appropriate language style. Reading well-selected journal articles helps students deepen their knowledge and improve their understanding of the theoretical concepts. Students also learn typical linguistic expressions and the appropriate use of terms used in their fields. Besides, our German students are thus forced to train their skills in English, the main language in science. Another focus is on the region of the in situ part â&#x20AC;&#x201C; this means getting familiar with the geography, regional and local geologic framework as well as geophysical field studies and models in our case. For an in situ lecture course on the volcanoes in Southern Italy (see Box 1), each student had to write a chapter of the excursion guide book based on a literature study and prepare a presentation to be given later in situ (Figures 1 & 8). For a course on the volcanic complex of Vogelsberg volcano, Germany, the students had to prepare both, a short chapter for the lecture notes, and a 10-15 minutelong poster presentation about methods used in borehole logging. The posters were presented in the preparatory classroom lecture before the in situ part (Figure 3). Additionally, the students prepared a presentation given at an in situ seminar, either in the field, or during an evening seminar held in the hotel (Figures 9-10). C) In situ lectures: in situ means that university lecturers or external experts present the learning matter at the specific place where the subject is relevant. In situ lectures can be done in several ways and at different kinds of locations (Figures 4-6). Mobile equipment such as a notebook and a beamer can be used in a hotel, on a ship or even in the open field (Figure 1). Concepts presented ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

134

earlier during the preparatory classroom lecturing can be repeated and directly linked with the real world. Of course the main point should be the direct inclusion of the local phenomena (e.g. a geoscientific site, major machinery or instrument, a buildingâ&#x20AC;Ś). A classic example is the explanation of rock types and the genesis of these rocks and their usage in a quarry (Figure 4). One may also visit a seismic reflection field survey and explain the implementation of recording arrays, the data acquisition and preliminary data interpretation. Such a lecture can be given also by an external expert, e.g. a field engineer of a company, a local geology expert etc. Generally, parts of the in situ lecturing can be done by local experts who have specialised local expertise (Figure 5), specific experience with a production machine or research infrastructure (Figure 6) etc. It is useful to explain to such external lecturers in advance what is the aim of the lecture, what is the state of knowledge of the students and what the students could do as possible practicals.

Figure 5: In situ lecture (step C) at about 700 m depth inside a potash mine. A group of geophysics students is instructed by a local geologist.

D) In situ practicals: during in situ practicals students have the opportunity to experience things that cannot be done in a class room (Figure 7). This may be the exploration of the underground using geophysical equipment, the handling of real production machinery or a realistic field study e.g. for hazard assessment. Especially in geoscience training, it is important to learn techniques for successful fieldwork (Kastens et al., 2009). Compared to laboratory studies, in the field, one has to cope with completely different working conditions: rough, partly extreme weather conditions, varying light conditions (which can cause a different appearance of rocks or minerals for example), identification of often hidden objects (e.g. a mineral vein underneath a vegetation cover), systematic spatial mapping or collection of data or a working environment with an unknown cultural and language background (e.g. archaeological work in a remote area in the Himalayas). It is important that students learn such realistic situations in situ and gather first experience for a later professional career (Kastens et al., 2009). ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

135

E) In situ seminars: students present a seminar talk in situ while incorporating the study object directly in their presentation. As media posters or computer presentations can be taken along and displayed at the hotel or in the open field (Figures 8-10). Subsequent to the preparation of such a presentation, the students are well prepared for the in situ subject, because they are forced to actually read the required literature and to think about the main relevant topics for the presentation. The shorter the allowed time for the presentation, the higher the pressure for focussing on the main facts and background information. This is a valuable exercise for students to learn to concentrate on the basic principles.

Figure 6: In situ lecture (step C) in the museum of historical seismometers of the University of Strasbourg, France. The main principles of seismometry are explained to the students and the operating mode of the mechanical components can be demonstrated at real objects.

F) Post-trip documentation: After the in situ phase, students should write up what they saw, experienced and learnt in a final report. Due to our experience, we recommend giving the students a clear limit for the length of the final report (the shorter the better: students learn to concentrate on relevant parts and lecturers are protected from reading numerous endless essays) and we tell them which main points should be covered. Individual final reports may be combined to manuscript-like lecture notes which cover different aspects of the subject matter and may serve e.g. as preparation material for an examination or for a future in situ course.

136

Box 1: Case Studies Our in situ lecturing experience is based on the following geophysical topics and examples: a: (Geo-)Physical Volcanology and Hazard Assessment: volcanic eruptions are a major hazard to society in many regions of our planet. We travelled to the volcanic islands of Lipari, Vulcano and Stromboli in the Tyrrhenian Sea and to Mt. Vesuvius (Italy). In these places we studied volcanic edifices, volcanic rocks, volcanic activity, monitoring concepts and monitoring instruments. We determined and estimated hazards related with the volcanic activity including the identification of vulnerabilities and the estimation of values at risks. b: Geophysical Exploration of Volcanic Fields: deep geophysical exploration (down to several hundred kilometres depth) of volcanic regions is necessary to understand the origin and history of magmatic processes inside the Earth and for understanding future volcanic activity. We went to the Eifel volcanic field in Germany to explain such approaches. Geophysical models were presented and discussed for a thorough understanding of the deep magmatic processes underneath the Eifel. In addition many surface expressions of volcanology can be studied including rock types, gas emissions or current vulnerable infrastructure. c: Induced Seismicity: induced seismicity embraces man-made earthquakes and related processes which are mainly due to mining operations and water injections into the Earth. We travelled to geothermal power plants, deep mines and water reservoirs which are potential sources of induced seismicity. Monitoring concepts were explained to the students as well as the destroying impact of induced seismicity to infrastructure. d: Historic Seismicity and its Use for Seismic Hazard Analysis: historic seismicity deals with earthquakes and their impact on society and nature in the past (mainly the time before instrumental seismicity started at around 1900). We explained the relevance of historic seismicity for estimating the hazard and risk by future earthquakes in the preparatory course and then visited a museum with historic seismic instruments as well as a town near to our university which suffered from destructive earthquakes in the past. e: Geophysical Investigations at a Complex Miocene Volcanic Structure: this lecture deals with questions such as how can we use geophysical exploration and measurements in order to investigate a complex volcanic structure. We went to the extinct Vogelsberg volcanic complex near Frankfurt, Germany and visited quarries, geotopes and borehole sites. There we explained the use of geophysical measurements which are needed to understand the volcanic structure at depth. In the preparatory lectures, students worked on poster presentations about borehole logging methods, and in the field we discussed the results of geophysical measurements conducted in the region. Students conducted geomagnetic measurements and had to identify anomalies of the local magnetic field. ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

137

Discussion We introduce an extensive concept in which a part of the traditional classroom teaching is transferred to the actual places where the subject matter can be studied directly and under realistic conditions. Compared to other teaching approaches such as excursions or fieldwork, we propose to lecture and practise comprehensively in situ (Table 1). Whereas students often follow passively the visited sites during an excursion, we motivate and force them to actively contribute to the lecturing. This active contribution goes beyond typical fieldwork, as the students are involved e.g. in preparing the lecture notes or giving presentations. Basic concepts and skills are taught and learnt ahead of the in situ part during preparatory classroom activities. During the in situ part, lectures are done to repeat these basics and deepen the studentsâ&#x20AC;&#x2122; knowledge. In addition new learning matter is introduced by including a direct link to local specialities, some of which can never be presented in a realistic manner inside a classroom (Figures 4-7). Active application of the freshly trained skills will admit students an even deeper insight to the subject during the in situ practicals and in situ seminars. This comprehensive learning cycle helps students acquire a wide range of competences, even exceeding the main subject. When the complete preparatory and in situ lecturing is prepared and executed by the same lecturer or lecturer team, the subject matter can be presented in a coherent way to the students. This avoids confusing students due to different descriptions or parameter abbreviations of the same object as it can happen when different lecturers use their own teaching material.

Figure 7: Students exploring hot fumaroles inside the Fossa crater on Vulcano island, Italy. The students measured the temperature of the emissive gas and liquid sulphur (up to 270 Â°C), analysed rock samples and identified endangered infrastructure during the in situ practical (step D).

138

The in situ lecturing is suited for bachelor’s as well as master’s level teaching, we also applied it to mixed groups (from first year bachelor’s to second year master’s students). Of course, exercises and themes for seminar presentations should be adjusted individually to the different students according to their background and experience. The in situ lecturing requires significant input and active contribution from the students: both, the preparatory and in situ phases include practicals which can be quite time consuming. Especially the preparation of the in situ seminar presentation, including the generation of hand-out material for fellow students, may take some time. An example for a work plan can be outlined as follows: we plan about 40-60 working hours for the active preparatory phase and 10-20 hours for the preparatory lecturing. Depending on the subject and site, the in situ part can last another 30 hours (3 days) to 120 hours (12 days). For the final report about 20-40 hours may be required. This corresponds to an overall work load of 200-300 hours or 7-10 credit points of the European Credit Transfer System (ECTS, with 1 ECTS credit point equivalent to 30 hours of student work). An important point is to make clear to the students what is expected from them. Especially presentation material (posters, handouts, computer presentations …) prepared for the in situ part must be done thoroughly by the students, because missing background material may not be available during travel. If the students prepare material for in situ lecturing, then flaws must be avoided as it may be also difficult to conduct revisions during travel. The students can be included in the organisation of the in situ part in order to learn the organisational side of their subject. For instance students may organise the travel to a starting point of the in situ part. We told our students that the in situ part of a volcanism-related lecture series will start at the port of Naples (Italy) at a specific pier, day and time. It was their own responsibility of get to this place in time which is about 1000 km away from their usual classroom.

Figure 8: Students giving an in situ seminar presentation on the mechanism and volcanic hazard of the 1944 lava flow at Mt. Vesuvius, Italy. The group stands on this specific lava flow and the city of Naples with one million inhabitants is seen in the background (step E). © 2014 The authors and IJLTER.ORG. All rights reserved.

139

Figure 9: Students giving an in situ poster presentation (step E) on a magnetic anomaly at the Geiselstein volcanic intrusion, Vogelsberg region, Germany. The group stands on top of the magmatic rocks which were also studied with geomagnetic measurements during an in situ practical (step D).

Most students are highly motivated during our in situ lecturing. The students’ seminar presentations, hand-out material and their final reports were prepared predominantly in an excellent way. The average grades assigned to the in situ courses were better compared to other classroom lectures. We interpret this positive outcome to be the result of high motivation due to our concept of in situ lecturing. Likewise our in situ courses were evaluated very positively by the students within the regular anonymous evaluation procedure which is conducted at our university (KIT) (Craanen, 2010). The overall grade of the students’ evaluation was always very high (between 1 and 1.5 with 1 as the best grade on a scale between 1 as excellent and 5 as deficient). Furthermore, the students provided helpful comments to improve this kind of lecturing method. Students commended for example “that they liked to talk to local experts”, “that they could go to sites which are not publicly accessible”, “that they were demonstrated the subject in accordance with practical needs” or “that they could evidently realise the relationship between theory and real measurement instruments”. We are encouraged by this positive feedback to further conduct and develop in situ lecturing as well as recommend this concept to other lecturers.

Acknowledgements Our development of the in situ courses benefitted much from the response of the students who gave valuable comments in their official evaluation sheets for the courses. In two cases our geophysics lectures were complimented with a valuable geology part by Geologierat Bernd Schmidt (Mainz) during the preparatory and the in situ phases. Such interdisciplinary input improved the quality of the teaching. Prof. Norman Harthill (Karlsruhe) kindly helped improve the manuscript. The in situ lecturing in Southern Italy was financially supported by a teaching grant (Fakultätslehrpreis) to E. G. © 2014 The authors and IJLTER.ORG. All rights reserved.

140

References Butler, R. (2008). Teaching Geoscience through Fieldwork. GEES Subject Centre, Learning and Teaching Guide, http://www-new1.heacademy.ac.uk/ assets/Documents/subjects/gees/GEES_guides_rb_teaching_geoscience.pdf (accessed 1 March 2015). Claiborne, L., Morrell, J., Bandy, J., & Bruff, D. (2014). Teaching outside the classroom, http://cft.vanderbilt.edu/guides-sub-pages/teaching-outside-the-classroom (accessed 1 March 2015). Craanen, M. (2010). Fakultätsübergreifendes Monitoring der Veranstaltungsqualität am Karlsruher Institut für Technologie (KIT) [Title in English: Inter-faculty monitoring of teaching quality at KIT]. Qualität in der Wissenschaft (QiW) Zeitschrift für Qualitätsentwicklung in Forschung, Studium und Administration, 4 (1/2010), 2-11. Handelsman, J., Ebert-May, D., Beichner, R., Bruns, P., Chang, A., DeHaan, R., Gentile, J., Lauffer, S., Stewart, J., Tilghman, S. M., & Wood, W. B. (2004). Scientific Teaching. Science, 304 (5670), 521-522. Hursh, B. A., & Borzak, L. (1979). Toward cognitive development through field studies. Journal of Higher Education, 50, 63-78. Kastens, K. A., Manduca, C. A., Cervato, C., Frodeman, R., Goodwin, C., Liben, L. S., Mogk, D. W., Spangler, T. C., Stillings, N. A., & Titus, S. (2009). How geoscientists think and learn. Eos, Transactions American Geophysical Union, 90 (31), 265-266. Laws, P. (1991). Calculus-based physics without lectures. Physics Today, 44 (12), 24-31. Lonergan, N., & Andresen, L. W. (1988). Field-based Education: Some Theoretical Considerations. Higher Education Research and Development, 7, 63-77. Powell, K. (2003). Science education: Spare me the lecture. Nature, 425, 234-236. Reiber, K. (2006). Wissen – Können – Handeln. Ein Kompetenzmodell für lernorientiertes Lehren [Title in English: Knowledge – Ability – Action: A conceptual model for learning-oriented teaching]. Tübinger Beiträge zur Hochschuldidaktik, 2/1. Thompson, D. B. (1982). On discerning the purposes of geological fieldwork. Geology Teaching, 7, 59-65.

141

International Journal of Learning, Teaching and Educational Research Vol. 11, No. 1, pp. 141-160, April 2015

The Mathematics Problem and Mastery Learning for First-Year, Undergraduate STEM Students Layna Groen, Mary Coupland, Tim Langtry, Julia Memar, Beverley Moore and Jason Stanley School of Mathematical and Physical Sciences, University of Technology Sydney, NSW Australia Abstract. In the 2014 academic year Mastery Learning was implemented in four first-year mathematics subjects in an effort to address a lack of preparedness and poor outcomes of increasing numbers of undergraduate students in science, engineering and mathematics programs. This followed partial success with the use of diagnostic testing and pre-teaching, active learning, and a greater emphasis on problem solving in context - under-prepared students were still more likely to fail the pre-teaching subject and to struggle with subsequent mathematics subjects. This paper describes the learning design used, and the outcomes achieved, with implementing Mastery Learning â&#x20AC;&#x201C; the positive: improved academic success, time management, and attitudes towards learning and Mathematics, an increased sense of independence, confidence and retention of content, and reduced stress and anxiety; and the negative: students having a sense of being taught how to pass a test rather than having a deeper understanding of the content. It will be seen that this negative is a consequence of a small but important difference in implementation. Keywords: The Mathematics Problem, Mastery Learning, first-year undergraduate experience, competency-based assessment.

1. The Challenge â&#x20AC;&#x201C; the Mathematics Problem First-year undergraduate mathematics education of students in science, technology, engineering and mathematics (STEM) programs currently face many challenges. In Australia, failure rates are unacceptably high for some intake cohorts (Groen, Beames, Coupland, Stanley & Bush, 2013), and attrition is higher than desirable. These problems can be traced back to high school where around 40 per cent of junior secondary mathematics classes are taught without a qualified mathematics teacher (McKenzie, Rowley, Weldon & Murphy, 2011).

142

Additionally, enrolments in elementary 1 mathematics subjects are increasing (Barrington & Evans, 2014). This is a problem as hard prerequisites for entry to most STEM programs in most Australian universities do not exist, though advanced mathematics is usually „recommended‟ and intermediate mathematics is „assumed‟. These recommendations and assumptions are often ignored, and students enter university with elementary mathematics or no mathematics at the senior high school level. In 2013, students with these backgrounds amounted to 56% of the University of Technology Sydney (UTS) science intake, 31% of the engineering intake and 28% of the mathematics intake. (Information technology students have no mathematics subjects in the cores of their programs.) The lack of preparedness and declining enrolments, collectively referred to as the Mathematics Problem (Hawkes & Savage, 2000), are problems world-wide (Hoyt & Sorensen, 2001; Smith, 2004; Luk, 2005; Heck & van Gastel, 2006; Brandel, Hemmi & Thunberg, 2008; Rylands & Coady 2009; Varsavsky, 2010). More than a decade ago, UTS, and many other Australian universities, introduced diagnostic testing of assumed knowledge (at UTS called the Readiness Survey). The diagnostic tests worked (and work) in conjunction with the pre-teaching subject, at UTS, Foundation Mathematics. Students failing the Readiness Survey take Foundation Mathematics prior to their first core mathematics subject to ensure they have the assumed knowledge of their program. Over this same period, small changes to the first-year undergraduate mathematics curriculum were made, active learning 2 was incorporated into learning designs of all first-year mathematics subjects, and changes were made to assessment schemes to reflect the more diverse learning activities undertaken. There was also an increase in emphasis on problem-based learning. Though improvements in pass rates were observed, unfortunately, under-prepared students were still more likely to fail Foundation Mathematics and later subjects. Our response to a lack of mathematical preparedness of some STEM students needed to be revisited.

2. A Solution – Mastery Learning 2.1 Background Mastery Learning endorses the belief that all students can learn and achieve the same level of content mastery when provided with the appropriate learning conditions (including time) (Bloom, 1971). Mastery is defined in terms of a subject‟s objectives and achievement of a prescribed level of performance, or competency, in (criterion-referenced) tests. This level of performance is usually 75 or 80% of the marks available for the „mastery‟ test. There is little or no delay between marking and feedback, and students failing to meet the mastery level Using the classification system of Barrington and Brown (2005), for New South Wales high school mathematics subjects, Mathematics Extensions 1 and 2 are classified as “advanced”. Mathematics (2 unit) is “intermediate” and General Mathematics is “elementary”. 2 See Prince (2004) for a brief description of active learning and problem-based learning. 1

143

are provided with remedial activities before a second attempt at achieving mastery. The process may be repeated (theoretically, as many times as required to demonstrate mastery, but practically, and in the current implementation, three times). Mastery Learning is not a recent innovation, but it is ideally suited to undergraduate mathematics education given its frequently hierarchical structure. It is particularly suited in the case of under-preparedness as it has the capacity to deal with individual learner differences. The research literature indicates positive effects of mastery learning on students, especially in the areas of achievement, attitudes toward learning, and the retention of content (Guskey & Pigott, 1988; Kulik, Kulik, & Bangert-Drowns, 1990; Anderson, 1994; Trigg, 2013). Why then isn‟t the use of Mastery Learning widespread? The answers lie in its intensive resource use and the (usual) requirement to allow students to work at their own pace. The issue associated with resource use is now very effectively addressed by online learning systems where immediate marking and feedback are available, remedial activities can be flagged and supported, and textbooks are not merely ebooks. The second issue is one that requires a more creative approach to learning activity (including assessment) scheduling. The implementation chosen for UTS first-year mathematics students utilised both these answers. Mastery Learning was trialled at UTS in the first (Autumn) semester of 2013. The results were promising enough to suggest a further trial of two subjects in Autumn semester of 2014. Analysis of the results of these two subjects confirmed Mastery Learning as a solution to the problems facing first-year STEM students in their mathematics subjects, and Mastery Learning was implemented in another two subjects in second (Spring) semester of 2014. This paper examines the success, or otherwise, of this Mastery Learning initiative.

2.2 Implementation After subdividing the subject curriculum into learning units and further into a logical sequence of smaller objectives, learning materials, instructional strategies and activities were identified, sequenced and executed over the teaching period. Criterion-referenced tests were administered. These were supervised, online, summative tests of just under an hour‟s duration, undertaken approximately two weeks after the completion of a unit. They assessed the „fundamental‟ knowledge and skills objectives of the unit, that is, the knowledge and skills that provide the basis for further development. In the UTS implementation, „mastery‟ was set at 80% of the marks available on the mastery tests. Students were provided with multiple opportunities for formative assessment prior to the mastery tests. Marking and feedback for both the formative and the summative assessments were provided online immediately. Students used the feedback to feed-forward, engaging in remedial activities individually or collaboratively (where necessary). Students were then given a further formative assessment before being given a second opportunity to sit the test, the „secondchance‟ test - same concepts, different questions (Bloom, 1971). Students already

144

exhibiting mastery, could choose to undertake the second-chance test. The best mark of the attempts was used in determining the final mark for the subject. While there was slight variation between subjects as to the finer details of the implementation, four mastery tests were scheduled over the course of the semester. Each test was worth 16% of the final mark for the subject. The final exam was then worth 36% of the final mark. Passing at mastery level in all four mastery tests ensured that students achieved a Pass in the subject. Perfect scores ensured an upper range Pass was achieved. Students achieving mastery in all tests could sit the final examination to improve their result beyond the Pass grade. Students were required to earn 13/16 on at least one of their attempts of each mastery test. Students not achieving mastery on the first attempt were given a choice of remediation activities to participate in. This remediation was conducted outside scheduled class time. Where mastery was not demonstrated on the second attempt, more structured activities were made available (primarily small group and peer-assisted learning). The third attempt was conducted at the end of the semester, allowing students additional time to acquire the knowledge and skills they needed. (The third attempt was only available to students who had not already demonstrated mastery.) To facilitate a successful third attempt, the last third of the semester consisted of enrichment activities which were examined in the final exam. The emphases in these enrichment activities were the application of modelling and problem-solving using the tools developed in the earlier units. These enrichment activities also provided students who were yet to demonstrate mastery with the opportunity for reinforcement and further exposure to content in context. Students requiring a third attempt (approximately 5% of a class, on average) were not eligible to sit the final exam. This demonstrates the primary trade-off made to implement Mastery Learning in semesters of fixed length, and is similar to some of the implementations reported by Twigg (2013). UTS is not the only Australian university to implement Mastery Learning, the University of Canberra implemented Mastery Learning in 2014. Twigg (2013) reports on a number of US tertiary institutions that are also using Mastery Learning as part of a learning design called the Emporium Model.

2.3 The subjects The subject Foundation Mathematics is offered to any UTS student, but primarily targets students who fail the Readiness Survey. This quick, online diagnostic test is only compulsory for engineering students. Engineering students who fail the test are enrolled in Foundation Mathematics. For other students failing the survey, enrolment in Foundation Mathematics is recommended. Success of this combination has been mixed. For example, failure rates in Foundation Mathematics by engineering students over the 20122013 semesters averaged around 59%. Mastery Learning in this subject was introduced in the Autumn semester of the 2014 academic year.

145

The standard program for engineering students includes Mathematical Modelling 1, followed by Mathematical Modelling 2. Mastery Learning was introduced into Mathematical Modelling 2 in Autumn semester of 2014, while it was introduced into Mathematical Modelling 1 in Spring semester. This was a consequence of the fact that the Autumn Mathematical Modelling 2 cohort is smallest in first semester, and a final decision regarding the roll-out of Mastery Learning to other subjects had yet to be made. Science students undertake Mathematical Modelling for Science, followed by Mathematics and Statistics for Science. Mastery Learning in these subjects was introduced in the 2015 academic year (following success in the other subjects). Mathematics students undertake Introduction to Linear Dynamical Systems, followed by Introduction to Analysis and Multivariable Calculus. Mastery Learning was introduced into Introduction to Multivariable Calculus in Spring semester 2014, while it was introduced into Introduction to Linear Dynamical Systems in Autumn 2015. These sequences of subjects are broadly similar in content, learning and assessment design, and learning environment. Students in information technology, biological, medical and environmental sciences have no core first year mathematics in their programs, though some take Foundation Mathematics. Active learning was introduced into first-year mathematics subjects approximately ten years ago. In these subjects this takes the form of interactive activities as well as collaborative learning in what were once traditional lectures. Minor changes were also made to assessment schemes at this time to encourage participation in all learning activities. However, for most first-year mathematics subjects, the majority of marks were still allocated to the traditional closed-book final exam. Weights for this varied, but the typical weight for the final exam was 65% of the marks available for the subject. (As previously mentioned, this changed with the introduction of Mastery Learning.)

3. Data Data was collected concerning program, tertiary entrance rank (ATAR), Mathematics subject(s) studied in high school and mark(s) in the Higher School Certificate (HSC) or other background, as well as final mark and grade in Foundation Mathematics, Mathematical Modelling 1, Mathematical Modelling 2, Introduction to Linear Dynamical Systems and Introduction to Analysis and Multivariable Calculus for the 2012-2014 academic years (six semesters). Information about the sample sizes can be found in Table 1. Table 1 â&#x20AC;&#x201C; Size of the database for Foundation Mathematics (FM), Mathematical Modelling 1 (MM1) and Introduction to Analysis and Multivariable Calculus (IAC) Subject

Background (2012, 2013, 2014) Adv. Semester Ext. 2

Int. Ext. 1

Elem.

Math. Gen. Math.

No (HSC) senior math.

Tertiary studies

Other*

146

Autumn

3, 6, 9

9, 18, 24

41, 31, 50

21, 34, 63

1, 3, 7

217, 146, 141

22, 23, 27

Spring

2, 0, 3

5, 8, 3

7, 18

8, 9, 13, 9

0, 0, 3

128, 92, 86

18, 19, 31

MM1

Spring

1, 1, 3

11, 3, 12

21, 26, 17

5, 10, 5

0, 0, 0

77, 68, 72 23, 30, 33

IAC

Spring

25, 22, 16

36, 34, 19

15, 20, 13

0, 0, 1

0, 0, 0

19, 13, 10 4, 6, 4

The “Other” category includes students with partially complete qualification, and students where no information on their background could be found. The class sizes of the six semesters of Mathematical Modelling 2 were 286, 560, 235, 574, 274 and 483 respectively. Students are assigned grades based on their final marks – A Fail (Z) is 0% to 49%, a Pass (P) is 50% to 64%, a Credit (C) is 65% to 74%, a Distinction (D) is 75% to 84%, and a High Distinction (H) is 85% to 100%. To examine qualitative aspects of the student experience, responses to openended questions on the end-of-semester student surveys (Subject Feedback Surveys) were examined. Representative responses were included in this paper. Focus groups were also conducted by staff not involved with the teaching or administration of the subjects included in the study. Students self-selected to participate, in line with ethics approval. Groups consisted of up to 10 students, and responses to six set questions (on attitudes, confidence, stress and assessment structure and nature) and a concluding open-ended, catch-all, question.

4. Methodology Mixed methods were used to assess the impact of Mastery Learning quantitative techniques were used to assess student achievement, though students‟ perceptions of this achievement were also examined. Qualitative techniques were used to assess the impact of Mastery Learning on things such as confidence, anxiety, attitudes, and behaviour. In order to examine the statistical significance of any improvements in mean final marks that may be consequent to the implementation of Mastery Learning, one-sided t-tests comparing sample mean final marks, under the assumption of unequal variances, were used. Where the mark distribution wasn‟t normally distributed, a chi-square test was used (Levine, Berensen & Krehbiel 2008). Pairwise comparisons of semester results from 2012 with 2014 and 2013 with 2014 were undertaken. A 5% level of significance was used for all statistical tests. Where the level of significance was something other than this, p-values are reported. Autumn outcomes were compared with Autumn outcomes, and

147

Spring outcomes with Spring outcomes (representing the different intakes into the programs). For pairwise comparisons of failure rates z-tests were used (Stat Trek, 2015). Where normality couldn‟t be assumed, chi-square tests were used. For cohorts where the expected numbers of failures or successes were too small (<5), Fisher Exact tests (McDonald, 2014), as implemented at Preacher (2015), were used. Comparison of medians was also undertaken as a means of examining the impact of any skewness in the mark distributions. For pairwise comparison of medians, z-tests or the Wilcoxon Rank Sum Test were used (where an underlying normal distribution could be assumed), otherwise a Mann-Whitney U-test was used. These were implemented using the Mann-Whitney U-test Calculator of Stangroom (2015) and the implementation of the Wilcoxon Rank Sum Test in Excel using Zaiontz (2015). These techniques were selected in preference to transforming the data using a logarithmic transformation as some of the cohort sizes were too small to assume the transformed variables would be normally distributed.

5. The outcomes 5.1 Quantitative measures 5.1.1 All programs 35010 Foundation Mathematics. In Table 2 we can see an overall reduction in failure rates for the Autumn semester cohorts – the improvement between the Autumn 2012 rate and the Autumn 2014 rate is significant at the 10% level (p = 0.087), and the improvement between the Autumn 2013 rate and the Autumn 2014 rate is significant at the 1% level (p = 0.00002). There is also reduction in the failure rate for the target background cohort, General Mathematics students. Using Fisher‟s Exact Tests to compare Autumn 2012 with Autumn 2014, the reduction in failure rate is significant at the 10% level (p = 0.092), while the reduction between Autumn 2013 and Autumn 2014 is not significant (p = 0.219). A one-sided t-test assuming unequal variances on the mean overall final mark finds that the improvement in mark is significant at the 5% level when Autumn 2013 is compared with Autumn 2014, but is not significant when Autumn 2012 is compared with Autumn 2014. These results are duplicated for the mean final mark for the Mathematics and General Mathematics cohorts. Median final marks also improve for nearly all background cohorts with Mastery Learning. For the cohorts of primary interest, the General Mathematics and Mathematics cohorts, the improvement in median using the z–test for the General Mathematics cohort when Autumn 2012 is compared to Autumn 2014 is significant at the 10% level (p = 0.051), and the increase in median when Autumn 2013 is compared to Autumn 2014 is significant at the 1% level (p = 0.00082). For the Mathematics cohort, the Autumn 2012 comparison is not significant, while the Autumn 2013 comparison is significant at the 1% level (p = 0.00036).

148

Table 2 - Autumn semester results by high school background Background

Autumn 2012

Autumn 2013

Autumn 2014

mean

median %fail (of)

mean

median %fail (of)

mean

median %fail (of)

Ext. 2

97.0

0% (3)

77.50

83.5

0% (7)

87.00

0% (8)

Ext. 1

85.11

0% (9)

66.78

0% (15)

80.71

92.5

8% (24)

Math.

73.59

33% (12)

56.00

25% (24)

72.88

14% (50)

General Math.

49.57

48% (21)

30.97

45% (11)

52.97

28% (63)

No HSC math.

55.5

0% (3)

Overall

21.16 % (314)

28.81 % (261)

18.75 % (321)

In Table 3 we see the results for the Spring offering â&#x20AC;&#x201C; the reduction in failure rates on pairwise comparisons overall are significant (p = 0.00002 and 0.027 respectively). We can also see improvement in the failure rates of students with backgrounds in Mathematics and General Mathematics. The improvement in failure rates for students with a Mathematics background is significant (p = 0.004) when Spring 2012 is compared with Spring 2014 (using the Fisher Exact Test). However, when Spring 2013 is compared with Spring 2014, the improvement is not significant. The comparisons of the failure rates for students with a General Mathematics background between Spring 2012 and Spring 2014, and Spring 2013 and Spring 2014 are both statistically significant (p = 0.009 and 0.049 respectively). Here we again see favourable outcomes for the main target group (students with a General Mathematics background). Using a one-sided t-test with unequal variances on the mean overall Spring final mark, we find that this improvement is significant at the 5% level when Spring 2012 is compared with Spring 2014, but is not significant when Spring 2013 is compared with Spring 2014. For the students with Mathematics backgrounds, the increase in mean and median were significant when Spring 2012 - Spring 2014 comparison, but not when Spring 2013 was compared to Spring 2014. For the General Mathematics cohort, the increases in the mean for both comparisons were significant, but there were no significant improvements in the medians.

149

Table 3 - Spring semester results by high school background Background

Spring 2012

Spring 2013

Spring 2014

mean

median %fail (of)

mean

median %fail (of)

mean

median %fail (of)

Ext. 2

0% (2)

0% (3)

Ext. 1

0% (5)

0% (8)

0% (3)

Math.

63% (8)

13% (8)

6% (18)

General Math.

63% (8)

38% (13)

0% (9)

No HSC maths

100% (2)

Overall

21.43 % (169)

12.21 % (140)

9.61% (153)

To ensure that significant improvement in most of the statistics measuring achievement could not be attributed to more capable students electing to take General Mathematics, ATAR was examined for the Autumn and Spring cohorts. For the Autumn comparisons, mean ATARs were 74, 80 and 67, with the Autumn 2012 to Autumn 2014 reduction significant at the 10% level, while the Autumn 2013 to Autumn 2014 reduction was significant at the 1% level (using a t-test with equal variances). For both pairwise Spring semester comparisons, there was no significant difference at the 5% level between the mean ATARs. We can conclude that there is no evidence in the mean ATARs of both the Autumn and Spring cohorts that suggest more capable students are electing to do General Mathematics, and hence that there is no significant increase in capability of the 2014 cohorts indicated. Foundation Mathematics data were also broken down by program. In Table 4 we can see that though numbers of Engineering students in the Spring offering of Foundation Mathematics have increased significantly, the failure rate for this cohort is now 0%. Comparisons of the failure rates for Engineering students reveal that the reduction in failure rates for both Autumn and Spring cohorts for all years compared are significant at the 5% level â&#x20AC;&#x201C; in three of the four comparisons the improvements in failure rates are significant at the 1% level. Science students, also show a 67% decrease in failure rate in the Autumn cohort and a 50% decrease in failure rates in the Spring cohort. Comparisons of the failure rates for Science students reveals that the reductions in failure rates for both Autumn and Spring 2013 cohorts are significant at the 5% level, while Fisherâ&#x20AC;&#x;s Exact test yields no significant reduction in comparing the Autumn 2012

150

cohort (p = 0.123), though in comparing Spring 2014 with Spring 2012 the improvement is significant (p = 0.047). Students in Information Technology programs, have the lowest failure rates. This can be explained by the fact that they have higher participation rates in intermediate and advanced mathematics subjects at high school. Table 4 - Failure rates by STEM course Year

2012

2013

2014

Semester

Autumn (of)

Spring (of)

Autumn (of)

Spring (of)

Autumn (of)

Spring (of)

Science

0.2 (25)

0.333 (6)

0.633 (30)

0.364 (11)

0.205 (39)

0.105 (19)

Engineering 0.697 (76)

0.760 (46)

0.580 (69)

0.317 (41)

0.377 (69)

0 (102)

Information 0.05 (20) Technology

0.091 (33)

0.043 (23)

0 (12)

0.25 (4)

0 (13)

5.1.2 Engineering programs. 33130 Mathematical Modelling 1. Using a one-sided t-test assuming unequal variances (and a 5% significance level) on the mean overall Spring final mark, we see the improvement is significant when Spring 2012 is compared with Spring 2014, but is not significant when Spring 2013 is compared with Spring 2014. These results are repeated for the mean final mark for the Mathematics cohorts. The analysis couldnâ&#x20AC;&#x;t be conducted for the General Mathematics cohorts as the sample sizes were too small. Overall, we can see an approximately 50% reduction in the failure rate overall compared with previous Spring semester results. This result is significant at the 5% level for both the Spring 2012 and Spring 2013 comparisons to Spring 2014. Considering background cohorts, for those students with a Mathematics background using Fisherâ&#x20AC;&#x;s Exact test yields a significant reduction in failure rate at the 5% level when the Spring 2014 Mathematics cohort is compared to the Spring 2012 cohort (p = 0.0005), but the reduction is not significant when the Spring 2014 cohort is compared to the Spring 2013 cohort (p = 0.185) . Other background cohorts from the target groups are too small to perform significance tests. The failure rates for students who had undertaken Foundation Mathematics prior to taking Mathematical Modelling 1 has seen a reduction from 25% down to 6% for a similar sized cohort. Pairwise comparisons using a z-test demonstrates that both reductions are significant at the 5% level (p = 0.00004 and 0.015 respectively). No strong conclusions can be drawn about the failure rates

151

for those students who did HSC mathematics and who did not take Foundation Mathematics as the background cohorts are too small. However, when students with other backgrounds are included, we see a reduction in failure rate from 20% to 12%. Pairwise comparisons using a z-test demonstrates that only the reduction from 2012 to 2014 is significant at the 5% level (p = 0.00002), while the reduction from 2013 to 2014 is not significant (p = 0.116). Using a one-sided t-test with unequal variances on the mean overall Spring final marks, finds that this improvement is significant at the 5% level when Spring 2012 is compared with Spring 2014, but is not significant when Spring 2013 is compared with Spring 2014. For the cohort that hadnâ&#x20AC;&#x;t undertaken Foundation Mathematics (the top groupings of Table 5), we see that a one-sided t-test with unequal variances demonstrates that there is a significant improvement (Îą=0.05) between Spring 2012 and Spring 2014 mean final marks. However, the increase in mean marks for the Spring 2013 and Spring 2014 cohorts is not significant. Similar results hold for the comparison of mean final marks for students who did undertake Foundation Mathematics prior to enrolling in Mathematical Modelling 1. In comparing medians little can be said about individual background cohorts as the samples sizes are quite small. In comparing the overall Foundation Mathematics and non-Foundation Mathematics cohorts, and the subject overall cohort, the increases in median are significant at the 5% level when Spring 2012 is compared to Spring 2014 (p = 0.027, p = 0.00076, p = 0.00012). When the Spring 2013 median is compared with the Spring 2014 median for the non-Foundation Mathematics cohort and for the subject overall, we find significant improvements (p = 0.015 and p = 0.03673 respectively). However there is no significant improvement for this yearly comparison for the cohort who had previously taken Foundation Mathematics.

Table 5 - Spring semester results by background Spring 2012

Spring 2013

Spring 2014

mean

median %fail (of)

mean

median %fail (of)

mean

median %fail (of)

Ext. 2

100% (1)

0% (1)

33% (3)

Ext. 1

55% (11)

0% (3)

8% (12)

Math.

57% (21)

19% (26)

6% (17)

No FM

152

General Math.

100% (5)

30% (10)

40% (5)

Cohort overall

41% (117)

20% (121)

12% (114)

Ext. 2

0% (1)

0% (3)

0% (1)

Ext. 1

0% (0)

0% (3)

0% (1)

Math.

0.5 (6)

20% (5)

0% (1)

General Math.

54% (13)

22% (9)

7% (15)

No HSC Maths

0% (0/1)

Cohort overall

44% (43)

25% (48)

6% (49)

Total Overall

41% (138)

21% (138)

10% (142)

With FM

33230 Mathematical Modelling 2. For the Autumn 2014 Mathematical Modelling 2 cohort in Table 6 we see an increase in mean and median final mark, and associated reduction in failure rate for students who achieved a Pass in Mathematical Modelling 1. The improvements in overall mean final mark for both Autumn semester comparisons are significant at the 5% level, and are in fact significant at the 1% level, with the implementation of Mastery Learning. The improvements in median for both Autumn comparisons were significant at the 1% level using the Wilcoxon Rank Sum Test. For the cohorts who achieved a Pass grade or better in Mathematical Modelling 1, it is not universally true that the mean and median final marks improved. The improvements in mean and median final marks are significant (at the 1% level) for the two, year comparisons of students who obtained Passes in Mathematical Modelling 1, and when Credits and Distinctions for Autumn 2012 are compared with Autumn 2014, but are not significant at the 5% level for Credits and Distinctions when Autumn 2013 results are compared with Autumn 2014 results. The reduction in failure rates overall for both Autumn comparisons are significant at the 1% level. For students who received a Pass in Mathematical

153

Modelling 1, both comparisons to Autumn 2014 resulted in significant reductions at the 1% level. For students with Credit grades, the Autumn 2012 comparison to Autumn 2014 was significant at the 1% level, though the Autumn 2013 comparison was not (p = 0.105). For students with Distinction grades, the Autumn 2014 reduction in failure rate was not significant at the 5% level (p = 0.0571) when compared to Autumn 2012, nor was the Autumn 2013 comparison (p = 0.270). This improvement in outcomes for students earning a Pass grade is further support for a Mastery Learning approach to first-year Mathematics subjects for Engineering students. Table 6 - Autumn semester Mathematical Modelling 2 results by Mathematical Modelling 1 result MM1 grade

Autumn 2012

Autumn 2013

Autumn 2014

mean

median %fail (of)

mean

median %fail (of)

mean

median %fail (of)

Pass

72% (74)

35% (93)

20% (88)

Credit

35% (46)

16% (32)

0% (20)

Dist.

32% (28)

16% (19)

6% (16)

High Dist.

0% (16)

0% (7)

0% (9)

Overall 47

45% (286)

35% (235)

11% (274)

A similar pattern can be seen in the Spring Mathematical Modelling 2 failure rates (Table 7). Again, there is not universal improvement in means and median final mark for all cohorts, though the improvements are significant for the Pass and Credit cohorts from Mathematical Modelling 1. Comparisons of means overall (under an assumption of unequal variances), medians, as well as a comparison of failure rates, find that improvements in overall outcomes are significant (Îą=0.01) for all Spring pairwise comparisons. Comparisons of mean final mark (unequal variance) and failure rate were also undertaken for each Mathematical Modelling 1 grade cohort for Spring classes. The improvements for the Mathematical Modelling 1 Pass and Credit cohorts are significant for all years compared (means, medians and failure rates at the 1% level). For the Spring Distinction cohort only the 2013 to 2014 failure rate showed significant improvement, though this was at the 10% level.

154

Table 7 - Spring semester Mathematical Modelling 2 results by Mathematical Modelling 1 result MM1 grade

Spring 2012

Spring 2013

Spring 2014

mean

median %fail (of)

mean

median %fail (of)

mean

median %fail (of)

Pass

47% (222)

43% (206)

6% (140)

Credit

15% (124)

16% (109)

2% (84)

Dist.

7% (91)

9% (90)

2% (100)

High Dist.

2% (51)

0% (78)

1% (89)

Overall 56

27% (560)

26% (574)

4% (483)

Overall then, the improvements in achievement with Mastery Learning are very encouraging and would appear to ameliorate the previous observation that students who obtained a Pass grade in Mathematical Modelling 1 were not as likely to achieve a Pass or higher in Mathematical Modelling 2.

5.1.3 Mathematics programs. 35102 Introduction to Analysis and Multivariable Calculus. Table 8 presents the mean, median and failure rate of the subject Introduction to Analysis and Multivariable Calculus based on mark in the prerequisite subject, Introduction to Linear Dynamical Systems. Using a one-sided t-test with unequal variances and a significance level of 5% on the mean of the overall Spring final marks, we find that this reduction is not significant when Spring 2012 is compared to Spring 2014. It is also true that the small increase in median is also not significant for this comparison. However, the Spring 2013 improvement in mean is significant at the 10% level, while the improvement in median is significant at the 1% level. Little can be said about the Credit, Distinction and High Distinction cohorts from Introduction to Linear Dynamical Systems (the prerequisite subject) because of the small size of the cohorts. However, there is a statistically significant increase (at the 10% level) in mean final mark for student who achieved a Pass in Introduction to Linear Dynamical Systems for both pairs of years compared.

155

There is a slight reduction in the failure rate of Pass students, though this is not significant at the 5% level (p = 0.270 and 0.391 respectively). The reductions in failure rates overall are not significant (p = 0.221 and 0.187 respectively). Table 8 - Spring semester Introduction to Analysis and Multivariable Calculus results by Introduction to Linear Dynamical Systems result 35101 grade

Spring 2012

Spring 2013

Spring 2014

mean

median %fail (of)

mean

median %fail (of)

mean

median %fail (of)

Pass

50.62

33% (39)

50.21

26% (38)

51.14

24% (41)

Credit

8% (13)

60.79

14% (14)

50% (2)

Dist.

79.5

0% (8)

66.29

14% (7)

64.25

77.5

25% (4)

High Dist.

90.6

0% (8)

79.5

80.5

12.5% 91 (8)

0% (2)

32% (99)

33% (95)

26% (63)

Overall 55

Results in Table 9 (results by background) exhibit an improvement in median result, along with an approximately 25% reduction in failure rate for students with a Mathematics background. Statistical analysis could not be undertaken on the Extension 2, Mathematics and General Mathematics cohorts due to the small sizes of these cohorts (Stat Trek, 2015). There is a significant (Îą=0.01) improvement in mean and median final mark in both year comparisons for students with an Extension 1 background. There is a significant improvement at the 1% level in failure rate for this cohort when Spring 2012 and Spring 2013 are compared to Spring 2014 using a Chi-square test. Table 9 - Spring semester results by high school background Background

Spring 2012

Spring 2013

Spring 2014

mean

median %fail (of)

mean

median %fail (of)

mean

median %fail (of)

Ext. 2

24% (25)

27% (22)

38% (16)

Ext. 1

56% (36)

29% (34)

16% (19)

156

Math.

13% (15)

40% (20)

31% (13)

General Math.

0% (1)

5.2 Qualitative outcomes - student perspectives Accommodate individual differences. Mastery Learning by its very nature seeks to accommodate differences in background and time students require to develop mastery of learning outcomes. Smaller, targeted tests with immediate feedback, learning support and multiple attempts to demonstrate mastery are more accommodating than one-chance testing. The majority of students report liking the mastery tests: I like the mastery testing scheme. I think it‟s fair, and I particularly like that [the lecturer] gives a pool of questions that can be repeated and practiced. (SFS – Mathematical Modelling 1 Spring 2014) Achievement. By providing timely feedback, supporting remediation and allowing multiple test attempts, students are able to identify and address their weaknesses, leading to an overall improvement in academic performance: I thoroughly enjoyed the subjects re-structure using the mastery tests. I previously failed the subject ... The mastery tests have enabled me to consistently and thoroughly cover the material better preparing me for my … future career as an engineer. (SFS – Mathematical Modelling 1 Spring 2014) Having the mastery testing system was really helpful … I found that being able to view your results and having second attempts encouraged me to study and learn much more. (SFS – Introduction to Analysis and Multivariable Calculus Spring 2014) Attitudes towards Mathematics and Learning. The motivating characteristics of Mastery Learning and applied problem solving resulted in an improvement in attitude towards mathematics: … the Mastery Tests … helped me ease into [the subject and] … I have a greater appreciation for maths now. (Focus group – Foundation Maths Autumn 2014) The second chance in the test really helped me to learn where I had made my mistakes and to learn from them and to come back and achieve a great mark the second time around. (SFS – Mathematical Modelling 1 Spring 2014) Retention of content. Mastery tests foster knowledge and skills retention: … the mastery tests … helped retain the concepts longer. (SFS Foundation Mathematics Autumn 2014) [Proprietary online learning system] is a fantastic tool, the instantaneous feedback and the ability to read the text online with worked examples is great. The ability to continually practice the

157

alternate problems solidifies the knowledge gained. (SFS – Mathematical Modelling 2 Spring 2014) Reduced stress and anxiety. Mastery Learning can reduce stress and anxiety by having more frequent, lower stakes tests that students can also sit multiple times. Final exam stress is also reduced – students know that they are walking into the exam knowing that they have already passed the subject: I liked the fact that … the final exam wasn't as stressful knowing that you have passed the subject after passing the mastery tests. (SFS – Mathematical Modelling 2 Spring 2014) I enjoyed doing the mastery tests as it helped me to stay on top of the work and made it much more relaxed when preparing for the final exam (SFS – Introduction to Analysis and Multivariable Calculus Spring 2014) However, a small number of students felt that there was too much testing and pressure: Just thinking about getting 80% on a mastery test is too [much] pressure. (SFS – Mathematical Modelling 2 Spring 2014) Improved time management. Through a structured approach to the timing of assessment tasks, with Mastery Learning students learn to manage their time more effectively, and so cramming for the final exam is also discouraged: The mastery tests were a great way in keeping up to date with content. (SFS – Mathematical Modelling 2 Spring 2014) The mastery tests made you learn continuously instead of cramming at the end of the semester. (SFS Foundation Mathematics Autumn 2014) Increased independence. Mastery Learning encourages self-correction and independence: I liked … the Mastery tests which ensured that your learning and understanding of the subject matter were reinforced and that you kept up to date with your work throughout the semester. The online [proprietary learning system] was an excellent resource for study at home. (SFS – Introduction to Analysis and Multivariable Calculus Spring 2014) Increased confidence. Mastery Learning can build confidence: The „2nd chance‟ class test. … It is not the same test but similar and this does wonders to a student‟s confidence … (SFS Mathematical Modelling for Science Autumn 2013) The assessment of 4 mastery test was good - made me feel more confident approaching finals. (SFS – Mathematical Modelling 2 Spring 2014) The above aspects combine to empower students so that they are not just achieving better marks and retaining what they have learned but are also

158

developing skills and attributes that can not only be translated to the rest of their studies but to their later professional life. For some students the experience was less positive – they felt that they were learning how to do the tests not master fundamentals of the subject: I was able to pass because all I had to do was learn how to do a specific set of questions for each mastery test. (SFS – Mathematical Modelling 2 Spring 2014) As a consequence of this feedback from students in Autumn 2015 more variation in questions was included in Mathematical Modelling 2. Online testing. There are significant advantages to using online testing for the mastery tests. These include: [Proprietary online learning system] is a fantastic learning tool … I find that my ability to learn through tools like this is greatly enhanced due to the instantaneous feedback and the practice alternate problems. (SFS – Mathematical Modelling 2 Spring 2014) However, online testing was not universally popular, especially at the beginning of the semesters when students were learning how to navigate the environment: I did not like the way assessments were conducted online. I spent far too long trying to work out how to enter the answers on the computer correctly rather than focusing on the actual material. (SFS – Foundation Mathematics Spring 2014) The … [online] assessment need[s] to be more reliable, the system is unpredictable. (SFS – Mathematical Modelling 2 Spring 2014)

6. Conclusion The aim of the paper was to contribute to the current debate on the ways to address the lack of preparedness of some first-year students of STEM programs (the Mathematics Problem). At the University of Technology Sydney, a form of Mastery Learning has shown itself to address the lack of preparedness successfully for many STEM students as well as for first-year mathematics students overall – it has advantages over one-chance testing and heavily weighted final exams. These advantages include improved academic performance. Students also report increased independence and confidence, and improved time management and retention of content. For many students the learning experience is positive with less stress and anxiety. While some students report poor experiences with the online learning and testing environment, most appreciate the central role it plays in facilitating Mastery Learning. The poorer experience of students in Mathematical Modelling 2 in 2014 resulted in the finetuning of content in the sequences of formative and summative assessments. Overall, Mastery Learning appears to afford a sustainable solution to the increasing lack of mathematical preparedness of some students in STEM programs.

159

Acknowledgements Some funding for this project was received under the UTS First Year Experience Grants. We would like to thank Dr Kathy Egea for conducting and organising the transcription of the focus groups. Thanks also go to Stephen Bush for his assistance with Figure 1 and his encouragement and interest.

References Anderson, S. A. (1994). Synthesis of research on mastery learning, US Department of Education, Office of Education, Research and Improvement, Educational Resources Information Center (ERIC). Retreived 13 October 2014 from http://files.eric.ed.gov/fulltext/ED382567.pdf Barrington, F. & Brown, P. (2005). Comparison of year 12 pre-tertiary mathematics subjects in Australia 2004-2005, Australian Mathematical Sciences Institute/ International Center of Excellence for Education in Mathematics (AMSI/ICE-EM) report. Barrington, F. & Evans, M. (2014) Participation in Year 12 Mathematics 2004 – 2013, Retrieved 14 October 2014 from http://amsi.org.au/publications/participation-year12-mathematics-2004-2013/ Bloom, B. S. (1971). Mastery learning, In J. H. Block (Ed.), Mastery learning: Theory and practice. Brandel G, Hemmi K & Thunberg H (2008), The widening gap - a Swedish perspective, Mathematics Education Research Journal 20, 38-56. Groen, L., Beames, S. Y., Coupland, M. P., Stanley, P., & Bush, S. (2013). 'Are science students ready for university mathematics?', Proceedings of the Australian Conference of Science and Mathematics Education (2013), University of Sydney, Sydney. Guskey, T. R. & Pigott, T. D. (1988). Research on group-based mastery learning programs: A meta-analysis, Journal of Educational Research 81, 197–216. Hawkes, T. & Savage, M. (eds.) (2000). Measuring the Mathematics Problem (London: Engineering Council). Retrieved 28 October 2014 from www.engc.org.uk/aboutus/publications.aspx Heck, A. & van Gastel, L. (2006). Mathematics on the threshold, International Journal of Mathematical Education in Science and Technology, 8 (15), 925-945. Hoyt, J. E. & Sorensen, C. T., (2001). High School preparation, placement testing, and college remediation, Journal of Developmental Education 25, 26-34. Kulik, C. C., Kulik, J. A. & Bangert-Drowns, R. L., (1990). Effectiveness of mastery learning programs: A meta-analysis, Review of Educational Research 60, 265–299. Levine, D. M., Berenson, M. L. & Krehbiel, T. C., (2008). Statistics for Managers Using Microsoft Excel, Retrieved 11 March 2015 from: http://www.prenhall.com/behindthebook/0136149901/pdf/Levine_CH12.pdf Luk, H. S., (2005). The gap between secondary school and university mathematics, International Journal of Mathematical Education in Science and Technology 36, 161174. McKenzie, P., Rowley, G., Weldon, P. R. & Murphy, M., (2011). Staff in Australia‟s schools 2010: main report on the survey. Retrieved 24 October 2014 from http://research.acer.edu.au/tll_misc/14

160

McDonald, J. H., (2014). Handbook of Biological Statistics, Retrieved 11 March 2015 from: http://www.biostathandbook.com/fishers.html Preacher, K. J., (2015). Calculation for Fisherâ&#x20AC;&#x;s Exact Test. Retrieved 11 March 2015 from: http://quantpsy.org/fisher/fisher.htm Prince, M., (2004). Does Active Learning Work? A Review of the Research [Electronic version], Journal of Engineering Education, 93, 223-231. Rylands, L. J., & Coady, C. (2009). Performance of students with weak mathematics in first-year mathematics and science, International Journal of Mathematical Education in Science and Technology 40 (6) 741-753. Smith, A., (2004). Making Mathematics Count (London: HM Stationery Office). Retrieved 28 October 2014 from www.mathsinquiry.org.uk/report/MathsInquiryFinalReport.pdf Stangroom, J., (2015). Mann-Whitney U-test Calculator, Social Sciences Statistics. Retrieved 13 March 2015 from http://www.socscistatistics.com/tests/mannwhitney/Default2.aspx Stat Trek.com, (2015). Hypothesis Test: Difference Between Proportions. Retrieved 9 March 2015 from http://stattrek.com/hypothesis-test/difference-inproportions.aspx Twigg, C. A., (2013). Improving Learning and Reducing Costs: Outcomes from Changing the Equation [Electronic version]. Change: The Magazine of Higher Learning, 45, (July-August 2013) Varsavsky, C., (2010). Chances of success in and engagement with mathematics for students who enter university with a weak mathematics background, International Journal of Mathematical Education in Science and Technology 41 (8) 1037-1049. Zaiontz, C., (2014). Wilcoxon Rank Sum Test for Independent Samples, Real Statistics Using Excel. Retrieved 16 March 2015 from http://www.real-statistics.com/nonparametric-tests/wilcoxon-rank-sum-test/

161

International Journal of Learning, Teaching and Educational Research Vol. 11, No. 1, pp. 161-179, April 2015

Teaching Culture through Language: Exploring Metaphor and Metonymy in Chinese Characters Hu, Ying-Hsueh Tamkang University, English Department New Taipei City, Taiwan

Abstract. Learners of Mandarin Chinese often find reading and writing Chinese characters extremely challenging. This study proposes a holistic approach that is anchored in the theoretic framework of Cognitive Linguistics to teaching Chinese characters for reading/writing by explicitly heightening learners‘ awareness of the cultural knowledge encoded in the radicals and characters. Radicals are keys to learning characters as they are a vital clue to the meaning of a word and help compose compound characters. Traditional ways of organizing and teaching radicals are in accordance with the number of strokes they have. However, the proposed approach organizes radicals by way of concepts that reflect the folk categories speakers of Chinese share. Learners were also taught how conceptual metaphor and metonymy motivate the formation of radicals and compound words. Twenty-nine international students at a university in Taiwan participated in the study. A survey was administered after 6 weeks of treatment yielding results that are favorable to the approach. It was also found that such an approach may not suit all learners depending on their prior knowledge. Keywords: Chinese radicals, semantic cues, cognitive linguistics, metaphor and metonymy, language and culture. .

1 Introduction The Sapir-Whorf Hypothesis claims that one‘s language, depending on whether it‘s the strong or weak hypothesis we are referring to, shapes or influences our world view. After the hypothesis was formulated in the early 20th century, a series of heated debates ensued, which have been followed by countless research and experiments to refute or support the hypothesis. Evidence so far has suggested that both the lexicon and the grammatical structure of a language do seem to influence certain key conceptualizations such as color, space, time, gender, and the event structure of various motions (Casasanto & Boroditsky, 2007). The evidence, in turn, supports the notion that language is also the product and manifestation of human conceptualization faculties that have been influenced by the physical, social and cultural environment humans live in.

162

This interrelationship is best encapsulated in the Embodiment Hypothesis proposed by Lakoff and Johnson (1980, 1999). The hypothesis posits that primary human cognition, which is mostly concrete, is anchored in embodied experiences such as using one‘s body to navigate space, motions, balance, and the senses of enclosure that create boundaries and dimensions. These in turn give rise to more abstract cognitive concepts such as time, causality, and container schemas that help in the comprehension of complex phenomena. In this light, language does not merely passively label objects or abstract concepts humans have, but participates in creating concepts at the same time. This understanding has formed the backbone of cognitive linguistics (CL) as it is known today (Dirk, 2006, pp. 1-20). This linguistic insight opens up an exciting avenue for foreign language teaching and learning, particularly with respect to the role of culture, which comprises the values and beliefs a speech community shares. It also highlights the necessity of teaching language and culture simultaneously. One fruitful area is the vocabulary itself. Works on concept transformation in the naming of lexical items establish that the lexicalization process of a given language should consist of that which is ―tantamount to category formation at the level of a whole culture.‖ (Györi, 1998, p.99) In other words, the formation of a cultural category inevitably involves linguistic coding, as there is no other way for conceptual categories to spread in a culture and for it to become explicitly part of cognitive structures of the individual members of that culture. In this light, a closer look at the 214 radicals that structure over thousands of Chinese characters frequently used today reveal a rich conceptual system of categorization. It groups experiences of various interactions with the natural, social and cultural worlds ancient Chinese lived in. This conceptual system, as Lakoff (1987) and Johnson (1987) argue, converges on cognitive mechanisms of prototype, image schema, metaphor and metonymy. They in turn helped create more vocabulary through these radicals to form numerous compound characters and words as the language continued to evolve. Such insight forms the basis of a Chinese e-learning course, CRILL (Chinese Radical Integrated Language Learning) the researcher developed, which aims to explore its pedagogical validity. The course consists of 15 units introducing 15 basic radicals denoting three groups of concepts encoded in Chinese vocabulary and idioms: body parts, natural phenomena and plants. Its design was originally to raise the awareness of Chinese social/cultural values for adult beginners learning Chinese as a Foreign Language (CFL) so as to increase language retention, cultural understanding, and enthusiasm. To explore the last two aspects, a small, preliminary study was launched with a five open question survey designed and distributed to 29 subjects of pre-intermediate level after over 6 weeks of training between 2012 and 2013. A survey was designed to address the following main questions: 1) Can a Chinese language course based on illuminating the metonymic and metaphorical concepts in the radicals and characters motivate the learning of reading/writing for pre-intermediate learners? 2) Can such a design enhance and motivate cross-cultural understanding? And finally 3) Does a learner‘s language background affect their perception of pedagogical efficacy and thus enthusiasm in such a course? For example, would Japanese learners or heritage learners who © 2015 The author and IJLTER.ORG. All rights reserved.

163

had learned Chinese characters in school or at home find CRILL as useful as speakers of Indo-European linguistic heritage? The findings of the study may have significant pedagogical implications for teaching CFL, particularly in terms of writing, reading and cultural learning. This study also provides some insight into the merit of holistic teaching by expounding on the metonymic and metaphorical concepts encoded in lexical items, and thus lending increasing support to the practical application of CL in modern language classrooms.

2 Background of the Study Embodied Cognition and Cognitive Linguistics The Embodiment Hypothesis proposed by Lakoff and Johnson (1980, 1999) claims that ―Reason is not disembodied, as the tradition has largely held, but arises from the nature of our brains, bodies, and bodily experience.‖ (Lakoff and Johnson, 1999, p.4) Hence, the functioning of our bodies is crucial for the structure of our conceptual system. Our conceptual system is, they argue, mirrored in language patterns, as in systematic use of metaphors. This view gives rise to their Conceptual Metaphor Theory (CMT) which has been extensively adopted by cognitive linguists to investigate a wide range of issues from lexicon meanings such as polysemy (Sweetser,1990) to grammar patterns (Talmy, 1988). Some linguists argue that metaphorical concepts may have emerged from metonymic ones (Barcelona, 2000; Radden, 2000). Because of this connection , metaphor and metonymy are often intertwined to form ―metaphtonymy‖ (Goossens, 1990). Consider this example : ―She could read my mind ,‖ given by Ruiz de Mendoza Ibáñez, (2000, p. 121). He explains that ―read someone‘s mind‖ combines a metaphor of MIND IS A BOOK and the metonymy of MIND STANDS FOR THOUGHT, giving rise to the eventual understanding of ―She understands me.‖ Metaphors and metonymies are often found on phrasal or sentential level; however, they also help form lexical items. Metaphors and Lexicons Each language family utilizes different ways of encoding concepts in the lexical items, for example, Proto-Indo-European languages (PIE) employ morphemes which are mainly consisted of roots, prefixes, and suffixes, and the extension of meaning can be achieved through compounding, derivation, borrowing, the creation of neologisms, acronyms, etc. An equally efficient if not more productive way of extending lexical items, as Dirven (1985, p. 96) points out, is through the processes of metaphor. He uses the term ‗metaphor‘ in its broad sense which also includes metonymy. Györi‘s (1996, 1998) work on concept transformation of naming lexical items in the course of their semantic changes in several major European languages illustrates these processes, which, he argues, are deeply anchored in culture. These studies of diachronic semantic changes strongly suggest that a word structure not only encodes semantic but also conceptual information. The primary motivation for these changes, as argued by Györi, is functional because it is based on a speech community‘s adaptation to its environment, which is not merely biological but, more importantly, a socio-cultural one. © 2015 The author and IJLTER.ORG. All rights reserved.

164

Although Chinese has an entirely different writing system, it also extends its lexicon following similar rules as those of PIE. A Chinese character consists of one or more components put together in various ways in a typically square-shape format. As in most PIE, it is not an arbitrary process how certain components are combined to form new words or new meanings. Based on a printed posthumous work of a Chinese scholar Xu Shen (86 BC), the Shou Wen Jie Zi, (―Explains Simple Characters and Compounds‖) that was published in cir. 121 AD, there were10516 characters arranged under 534 to 544 primitive symbols which are the origin of the 214 radicals used today. The most common way of forming characters is to combine a radical component that stands for meaning and a component that stands for sound. This phono-semantic principle created nearly 95% of commonly used characters in modern Chinese (Dictionary of Chinese Character Information, 1988). This high rate reveals the significance of radicals in Chinese characters. Most of them are of pictograms, which as indicated in Shou Wen Jie Zi, can be divided into 1) Cosmology and Geology, 2) Plants, 3) Zoology, 4) Human Body Parts, 5.) Artifacts and Other Man-made Objects, 6) Clothing, and 7) Housing and Shelters. The radicals found in the current frequent words from both Taiwan and mainland China, of course, exceed these pictograms. Some radicals may fall under other categories not mentioned in Shou Wen Jie Zi such as Colors and Shapes, and different scholars may come up with slightly different groupings. (Zhou, 2012). Such groupings do not represent arbitrary divisions of the world; they in fact converge on the cognitive capacities of the human mind. These concepts are all based on cognitive salient prototypes the speakers of a community; these are folk categories rather than scientific ones (Ungerer & Schmid, 1996, p.19). The radical, together with other components of a character, through metaphor or metonymy, give rise to the meaning of the character. Take the radical which is also a character, 心 xin, as an example. It is a pictogram of a human heart. The semantic analysis of the word (Hu and Fong, 2010) supports a prevalent metaphor that is HEART IS THE SEAT OF EMOTION, which in turn gives rise to numerous characters denoting various emotions. Consider these words: 怒 ―anger‖,悶 ―pent-up anger, depressed, ‖愁 ―sadness‖, and 恨 ―hate‖ as well as 憤 ―anger‖ (radical of the last two words is 忄, a stylized 心), which are all composed of a semantic part and a phonetic part. However, the phonetic part is often necessary for meaning to emerge (Ma, 1997). The phonetic part, 奴 means ―slave‖, so together with the metonymic concept of XIN STANDS FOR EMOTION, it requires the understanding of this concept, SOMEONE IS NOT IN CONTROL OF HIS/HER EMSOTION, through a metonymic process so as to arrive at the interpretation of anger. Another case in point is the word 恨 hèn. It consists of 艮 which means ―tough, leathery, or blunt‖ for sound and 忄 for meaning, and it is necessary to blend these two concepts to arrive at the understanding that TOUGH EMOTION IS HATE. A further example that the phonetic component of such compounds is necessary for meaning to emerge is, for example, when the phonetic part 艮 gěn (the sound hèn might have evolved from gěn) was used to compose new words. A case in point is 狠 hěn, ―cruel,‖(compared with 恨 hèn) where the radical 犭 stands for four-legged © 2015 The author and IJLTER.ORG. All rights reserved.

165

animals. Underlying the overall meaning is the metaphor: HUMANS ARE ANIMALS, and the understanding that four-legged animals are usually fierce, so when this feature is mapped onto a human, we come to the understanding of ―cruelty.‖ These examples also transpire the process of lexicalization of written Chinese through metaphors and metonymies. Radicals Teaching in TCFL Several studies have examined the role of radicals in teaching writing and reading Chinese characters to SL and FL learners of Chinese (cf. Shen, 2000, Shen and Ke, 2007, Wang and Koda, 2013). They have invariably demonstrated the effectiveness of teaching radicals through the semantic cues embedded in them to help recognize and retain words over more traditional methods such as rote learning. It was also established that semantic cues work best when the meaning of the word has direct connection with the radical; these characters benefit most from explicit teaching of semantic cues. (Dunlap et al. 2011) In short, there are characters that are more transparent than others in terms of the predictability of the radical. The less transparent characters are those whose radicals are not directly related to their overall meaning. For example, Dunlap et al investigated a cluster of words that have the radical 禾 ‗grain, rice plant,‘ and they listed some characters that are not directly related to ‗grain‘ which include 稅 ‗tax‘, 稱 ‗to weigh, to call‘, and 稍 ‗a little bit‘; they are supposedly more difficult to learn and recall. However, as Zhou‘s (2013) study of the radical/character 土 ‗earth, soil‘ in Shou Wen Jie Zhi demonstrates, it was through the principles of CMT (and semantic field) that words related to 土 had emerged. Hence, on a closer inspection, applying CMT in examining the etymology of those words of 禾, one would find that they are still related to rice grain through various degrees of metaphorical and metonymic extension. For instance, 稅 is a combination of 禾 and 兑 ‗exchange‘, so considering the importance of agriculture in ancient Chinese society, using rice grain for tax payment was probably practiced in those days. In this case, 禾 stands for money or commodity used to pay for taxes—a metonymic principle made sense in a cultural context. Shen (2004) espouses deep learning which is using semantic cues in teaching CFL; therefore, it would be of interest to investigate the role of metaphor and metonymy in even deeper learning. The challenge is how to make metaphorical and metonymic clues accessible to learners so that they can become teachable and learnable? The study discussed below explores this issue so as to answer the research questions raised earlier.

3 The Study Experiment Material: CRILL Based on the theoretical framework of embodied cognition, folk categories, metaphor, and metonymy as discussed above, an on-line, self-learning, asynchronized course CRILL (Chinese Radicals Incorporated Language Learning) for learning the Chinese writing system and culture for English speakers was designed between 2008 and 2010. Since September 2010, it has been made accessible to students who enrolled at the university in northern Taiwan © 2015 The author and IJLTER.ORG. All rights reserved.

166

where the researcher works. It is accessible through an intranet platform, Campus (http://campus.viainno.com/campus/Homepage.aspx), that was developed and maintained by the digital learning center of the university who was also a partner in the development of CRILL. CRILL was designed to introduce Chinese culture through the writing system to novice adult learners in college. As most FL learners find Chinese writing difficult, they tend to find learning it rather daunting. Even while learning they find the experience frustrating as there are few rules for them to follow. CRILL therefore aims to help overcome these experiences by illustrating the universal as well as certain cultural-specific concepts in the formation of Chinese characters. Hence, characters are not grouped together by the number of strokes or sound. In tandem with the folk taxonomy found in the category system of Chinese radicals discussed previously, the characters are organized in accordance with the concepts the radicals share, starting from universal ones such as human body parts, to concepts such as nature and plants which can be regional and culturally specific, having been derived from Chinese geography and climate. In view of this principle, there are fifteen units with the first seven units dealing with human body parts (outer and inner organs), followed by four units with nature, and the last four with plants. The figure (Figure 1) below shows the table of contents of CRILL as found on the website.

Figure 1 Page of Contents for Unit 3 in CRILL

Under the heading ―Body‖, learners will find the radicals for eye 目, nose 耳, mouth ロ, hand 手, foot 足, heart 心, and flesh 肉/(月?), whereas radicals for sun 日, moon 月, mountain 山 and water 水/氵 are under ―Nature‖, and radicals for bamboo 竹, wood 木, grass 草, and rice 米 are under ―Plant‖. These radicals were

167

chosen as they represent the most common concepts and thus have generated a rich vocabulary in Chinese with many of it suitable for beginners. Each unit comprises of 12 to 13 sections. The following figure (Figure 2) shows one of the units, Unit 3 – Foot, as an example. The menu on the left gives learners a clear idea what the contents are.

Figure 2 Lead-in Activities for Unit 3 in CRILL

Each unit is divided into three phases of learning: lead-in activities (Section 1 to 5), core learning materials (6-10), and post-learning exercises (11-13). As can be seen, the lead-in activities include 1) a list of learning goals, 2) warm-up activities that ask learners to think iconically about a body part or a natural/artificial object, 3) matching pictograms with the radical/character of the unit, 4) animation of the evolution of the radical/character, and 5) the recognition of the radical/character among various characters. The figure below (Figure 2) shows what learners see when one clicks on Section 4 for the historical evolution of 足 (the foot) and 走 (to walk), two radicals which are characters as well for the same body part, ―foot‖, in Unit 3.

168

Figure 3 Evolution of Chinese Characters of 足 and 走

The second phase of learning (Sections 6 to 10) involves several compound words as well as fixed expressions that are commonly associated with the radical(s) of the unit. These phrases, with some being polysemous, are a mixture of concrete and abstract meanings so that learners can see the role metaphor and metonymy play in meaning extension. For example, in Unit 3 as seen in Figure 2, two radicals which are characters as well are introduced:足 and 走 with the former representing the physical body part, foot, while the latter represents the motions that are in tandem with the foot. In short, the same body part gives rise to two related concepts represented by two slightly different icons. When 足 (the physical foot) functions as a radical that helps create further semantic items, it is written as 𧾷, which can be seen in many motion verbs that involve various actions involving the foot such as 踢 (to kick), 跑 (to run), 蹲 (to squat), and 跳 (to jump). Nouns such as 路 (the road), 跡 (track or trace) are semantic items extended from various interactions of the foot with certain objects. All these words are phono-semantic compounds encoding concepts that are, according to CM theory, metonymic, namely BODY PART STANDS FOR ACTION, and ACTION STANDS FOR CONCEPT. They are relatively concrete. However, 足 is polysemous, like most words in all languages. One of its senses which is more abstract in fact means ―satisfied‖, deriving from the metaphor BODY IS A CONTAINER, so expressions such 足够 (enough, sufficient) and 滿足 (satisfied, content) capture this metaphorical sense. Together with these phrases, there are also sentence patterns and sentence building activities included in these sections that help to provide some kind of context for association. Both phrases and sentences have all been controlled in terms of frequency and familiarity for the beginner‘s level. The final phase consists of post-learning exercises which usually uses songs, poems, or nursery rhymes that are associated with the radical/character of the unit (Section 11). Section 12 provides exercises with feedback for learners to gauge their own learning outcome. Finally, each unit ends with an idiom that contains the radical/character of the unit with a story explaining the origin of the idiom.

169

Procedure In order to ascertain the perceived efficacy of CRILL and answer the questions raised earlier, a survey consisting of five open-ended questions was designed and distributed from 2012 to 2013 to twenty-nine international students who were studying at the university where the researcher works. They all enrolled in the Fall-semester course entitled ―Cross-cultural Learning‖ the researcher offered. CRILL was an integral component, among other course materials, of the course syllabus, and it was assigned as self-study homework over six weeks each semester after the mid-term exam. Prior to that, participants had been taught about the concepts of metonymy and metaphor existing in all languages and they were assigned to specific tasks in identifying those found in their own language. They would present their findings in class so that they could also make some cross-cultural comparisons. Following that, the organization of Chinese radicals and the metaphorical and metonymic clues in Chinese radicals/characters were incorporated. These exercises were meant to prepare them for CRILL. Once they started with CRILL, they could decide when and how long they wanted to spend on CRILL. They were encouraged to raise questions in class should they have encountered any issues during their self-study. The website is equipped with a log recording the frequency and time they actually spent on CRILL, although this data was not taken into account in the final analyses. At the end of the course, which lasted sixteen teaching weeks in total, participants would take a test on various course materials and CRILL. At the end of this test, a five open-question questionnaire was administered to investigate the efficacy of CRILL. By completing these questions, participants would receive extra points for the test. Participants The students that enrolled in the ―Cross-cultural Learning‖ course from 2012 to 2013 came to Taiwan either as exchange students staying six months to one year, or as international students pursuing an undergraduate degree at the university. Their Chinese proficiency would be considered pre-intermediate at the time of enrolment, although speaking Chinese fluently was not a prerequisite for attending the course since the course was mostly conducted in English. Nonetheless, they all had previous Chinese writing experience before the course started. Their writing experience differed according to the region they grew up in and the language they speak at home. Among the twenty-nine students, thirteen of them speak Indo-European languages coming from Europe and the Americas (N=13), nine of them speak Japanese coming from Japan (N=9), while seven of them come from other Asian countries (OACs, N=7) including Korea, Malaysia, and Vietnam. Such grouping is of particular interest when considering that Japanese learners learn Kanji –a script based on Chinese characters – at a young age. Would Japanese learners also find the radical based learning, as put forward in CRILL, beneficial to them? In short, would early and long exposure to Chinese writing make it easier or more difficult in understanding the explicit knowledge for the formation of Chinese characters compared with speakers of different

170

linguistic backgrounds? Although the number of participants for each group in this study was few, it still gives us some preliminary indications. Students’ Perceptions: Questionnaire In order to address the research questions for the study, a questionnaire with the following five questions were designed and administered: A. What did you not know about Chinese language before starting with CRILL? B. What did you not know about Chinese culture before starting with CRILL? C. What do you like most about CRILL, for example, in what ways has it been helpful for your learning/reviewing? D. What do you NOT like about CRILL? In what ways has it been confusing and hindered your learning/reviewing? And E. Any suggestions you may have in order to make this platform better from a learner‘s point of view? All questions address the research questions one and two, namely whether the design of CRILL can benefit and motivate learners learning Chinese characters and culture, with questions D and E focusing on their critical comments on CRILL. As for research question three, regarding learners‘ language background and their evaluation of CRILL, data elicited from the five questions (A, B, C, D, and E) in the questionnaire were further analyzed according to the participants‘ region of origin. The responses for each question were categorized and coded to be calculated in terms of percentage, so as to yield an overview of the participants‘ experience and evaluation of CRILL. For question A regarding what they hadn‘t known about Chinese language before they started with CRILL, participants‘ answers were able to be grouped into the following three codes: 1. No familiarity with the radicals, 2. Having familiarity with the radicals, and 3. Having some familiarity with the radicals. For question B, addressing what they hadn‘t known about Chinese culture before they started with CRILL, the coding was as follows: 1. Culture and idioms, 2. Culture and characters, 3. Neither of the above, 4. Festivals/culture, and 5. None. For Question C, which asked what they liked most about CRILL, for example, in what ways it was helpful for their learning/reviewing, their responses were categorized into: 1. Sentence/grammar, 2. Characters, 3. Idioms, and 4. Games/songs. Responses from participants for question D and E received similar coding as they address related issues. Hence, the coding for question D is as follows: 1.Too easy, 2.Confusing translation, pinyin, or pronunciation, 3. Repetition, 4. Insufficient feedback, reading and composition input, 5. Insufficient examples (for the lexical items and sentence patterns taught in each unit), 6. Silly, 7. Technical issues, and 8. No problems. Similarly, the coding for question E were: 1.Too easy-should have more levels, 2. Correct or Confusing translation/pinyin/pronunciation, 3. Reduce repetition, 4. Give feedback; more reading and composition input, 5. Create more linguistic examples, 6. No songs, 7. Improve technical issues, 8. No problems, and 9. No change. With these codes, it was possible to measure some tendency in terms of percentage.

171

Analysis and Results Overall results. Firstly, most of the participants (66%) were unfamiliar with how Chinese radicals are organized before they started with CRILL, despite the fact that they had all learned Chinese for several years. 10% of them reported some familiarity, while 24% good familiarity. This finding is further corroborated by the results of question B, in which nearly half of the subjects (45%) did not have any idea about the connection between culture and the formation of characters. This number could be higher when some subjects (7%) who claimed not to have any idea about the cultural meanings contained within idioms or characters before starting CRILL are also included. Nearly one third of them had no knowledge of the connection between festivals and culture (31%), and less than a quarter of them did not know the connection between culture and idioms (17%). Although most participants were not familiar with the way radicals and characters were presented in CRILL, they liked and considered such design helpful to their learning with 69 percent of the participants expressing positive perception to its design, in contrast to other components (See table 1). Table 1 Overall Results for Question C: What do you like most about CRILL?

Coding N Percentage 1 29 11% 2 29 69% 3 29 9% 4 29 11% Note: 1= Sentence/grammar 2=Characters 3=Idioms 4=Games/songs The results of questions D and E (Tables 2 and 3) provide some further insight into participants‘ criticisms and suggestions. The results of question D (Table 2) show that the level of difficulty in materials ranks as the highest complaint (28%), followed by ―inconsistent translation and pronunciation‖ (17%). ―No feedback /reading/composition input‖ and ―insufficient examples for the lexical items and sentence patterns‖ (10% for each respectively) were also among the major complaints with ―technical issues‖ being another one (10%). A small number of participants did find the design somewhat ―boring and childish‖, specifically referring to the songs and nursery rhyme parts (7% and 3% respectively). However, 10 percent of them found no major problems in the design of CRILL. As for suggestions for improvement (Table 3), participants ranked having ―more feedback/reading/writing practices‖ (29%) as the most important, followed by having ―technical issues corrected‖ (24%). A small number of them (7%) gave the suggestion of ―adding higher levels to existing CRILL curricula‖ in the future. Participants considered ―reducing repetition and improving consistent translation and pronunciation‖ more important than ―adding higher levels‖ (17% and 14% respectively). A very small number of participants (3%) would have liked to see ―some more linguistic examples‖ to be added to either fixed expressions or sentence patterns.

172

Table 2 Overall Results for Questions D: What do you NOT like about CRILL?

Coding N Percentage 1 29 28% 2 29 17% 3 29 7% 4 29 10% 5 29 10% 6 29 3% 7 29 10% 8 29 17% Note: 1=Too easy 2=Confusing translation/pinyin/pronunciation 3=Repetition 4=Insufficient feedback, reading and composition input 5= Insufficient examples 6=Silly 7=Technical issues 8=No problems Table 3 Results of Question E: Suggestions to improve CRILL

Coding N Percentage 1 29 7% 2 29 14% 3 29 17% 4 29 29% 5 29 3% 6 29 0% 7 29 24% 8 29 3% 9 29 3% Note: 1=Have more level 2=Correct confusing translation/pinyin/pronunciation 3=Reduce repetition 4=Give feedback; more reading and composition input 5= Create more linguistic examples 6= No silly songs 7=Improve technical issues 8=No changes Results by Group. When the results presented above are broken down into region, with participants‘ language background taken into consideration, the individual picture for each region resembles somewhat that of the overall results. However, there are some minute differences which can help answer research question three, regarding whether participants‘ language background would affect their evaluation of CRILL and their motivation to learn Chinese. Firstly, although Japanese learners are exposed to Chinese characters in Kanji at an early age, the number of participants unfamiliar with the metonymic and metaphorical nature of Chinese character formation and the cognitive categorization of radicals is nearly the same as those speaking European languages such as French, German, and Spanish (69% vs. 67%). Even the participants from other Asian countries (OACs), some of whom were heritage learners, reported a lack of knowledge in a percentage similar to those from Europe/Americas (57% vs. 69%). However, a small number of participants from Europe/Americas and OACs said they were very familiar with the principles behind radicals and characters (15% and 14% respectively) before they started © 2015 The author and IJLTER.ORG. All rights reserved.

173

with CRILL, whereas none from Japan claimed so. On the other hand, about one third of the participants from Japan (33%) said that they were somewhat familiar with radicals/characters, slightly more than those who claimed so in the OACs group (28%). Only 15% percent of the participants from Europe/Americas claimed to be somewhat familiar with the radicals and characters. In terms of cultural knowledge encoded in the radicals/characters, most European language speakers (69%) and a good number of participants from Japan (44%) claimed what they learned in CRILL was new to them. However, if we also include the 15% of European languages speakers who claimed not to be familiar with the connections between idioms and characters with culture, the number of participants who were ignorant of the connection between language and culture in this group is much higher than that of the Japan group. Only a small number (14%) from OACs claimed they were unaware of the connection between culture and radicals/characters. Furthermore, about half of the participants from Japan and OACs said they were not familiar with the cultural meanings of the festivals introduced in CRILL, while a fairly small number of participants from Europe/Americas reported such a lack of knowledge (7.5%). Not surprisingly, all participants from Europe/Americas (100%) reported a favorable perception of the presentation of radicals/characters in CRILL and found this design beneficial to learning, followed by a high percentage of participants from OACs who said so (85%). About half of the participants from Japan (55%) also liked and were motivated by such a design, with the other half split between grammar practice (22%) and games/songs (33%). A relatively smaller number of participants from the OACs group liked and were motivated by grammar practice (14%) and games/songs (14%), while a very small number to none of the participants from Europe/Americas considered these two components beneficial and motivating (7.5% and 0% respectively). As for criticism, most of the Europe/America and Japan groups thought the skill-level taught in CRILL was too easy for them (38% and 33% respectively), whereas none of the participants from OACs said so. Similarly, the OACs group had the highest number of participants who did not find any major issue with the methodology of CRILL (42%). About a quarter of the participants in the Japan group (22%) also found no serious issues with CRILL. However, none of the participants in the Europe/Americas group expressed such an evaluation. When we consider the ranking of the criticisms by group, it becomes clear that both the Japan and OACs groups had different emphases on what was missing when compared to the Europe/Americas group. The latter considered ―not having feedback for exercises and no reading/composition input‖ (coding 4) a big drawback, whereas the former two groups did not share this criticism at all. They instead thought that the ―number of examples for the phrases and sentence patterns were insufficient‖ (coding 5) and more of a hindrance to their learning. Lastly, all three groups believed that there should be ―more feedback with their exercises with additional reading/composition input and practices‖ (coding 4) along with ―improving technical malfunctions‖ (coding 7). They also seemed to

174

agree that ―repetition needed to be reduced‖ (coding 3). The suggestions for other issues are less unanimous.

4 Discussion Benefit and Motivation Although the analyses presented here are confounded by the small sample size of subjects and the preliminary nature of this study, some tentative observations can still be drawn. Firstly, in answering the research question one and two, when the overall data without group division being examined, it is safe to argue that the cognitive approach to teaching Chinese radicals/characters explicating some metonymic and metaphorical principles of word formation can benefit and motivate the learning of Chinese writing and culture. Participants‘ overall perception indicates that CRILL was very favorably received, particularly with respect to how radicals/characters were organized and introduced, even though such learning approaches were new or partially new to them. These results lend a positive support to the pedagogical application of integrating cognitive linguistic theory into teaching Chinese as a foreign language. Cognitive approaches that incorporate metonymic and metaphorical knowledge with a touch of etymology (Boers et al., 2007) in language teaching has been found to be beneficial and motivating in the learning of English vocabulary (Boers, 2004; Boers and Lindstromberg, 2009), idioms (Hu & Ho, 2009), prepositions (Hu & Fong, 2010; Tyler and Evans, 2004), and phrasal verbs (Abreu & Vieira, 2009; Yasuda, 2010). There have been even studies and research in FL and SLA that call for the overall promotion of metaphorical competence in foreign language learning (Littlemore & Low, 2006). There is substantial empirical evidence in support of the efficacy of explicating the semantic cues in the radicals for Chinese characters recognition and retention (Shen and Ke, 2007, Wang and Koda, 2013). However, none of these studies examined the role of metaphor and metonymy in word meaning, formation of compound words, and polysemy. There are also very few extensive teaching materials based on semantic cue nor cognitive approach in the field of TCFL. In light of this, the design of CRILL is making an important step in the direction of developing efficient approaches and teaching materials to teaching the Chinese writing system. Similar to semantic cues, CRILL seeks to guide learners with patterns and principles so that learning to write and read is not an arbitrary and mundane task. Although semantic cues are useful in recognizing and predicting the meaning of characters, they work best in transparent characters. However, the approach applied in CRILL went further by incorporating metaphorical and metonymic clues, so that hopefully less transparent characters, compound words, and polysemy can be better explained, thus, recognized and retained. Above all, cognitive approach highlights the rich cultural background encoded in the characters and compound words/phrases. If teaching a language is concerned with passing on cultural knowledge at the same time, CRILL is certainly more satisfying in this respect and the results of the study arguably support this assertion.

175

Language Background In addressing whether the linguistic background or any previous knowledge in the cognitive nature of Chinese radicals/characters could motivate and benefit learners or not, the results suggest that Indo-European speakers from Europe and the Americas tended to enjoy the cognitive method provided in CRILL more than Japanese speakers. Even speakers from OACs considered this approach more positively than those from Japan. This finding came as a surprise when the number of participants in each group who had not had any knowledge of Chinese radicals before their participation of the CRILL program is fairly comparable. According to the Japanese participants, they had to learn Kanji from a young age but had never been taught explicitly about the cognitive principles involved in the composition of characters. Despite a lack of prior knowledge, they did not find this approach as engaging and motivating as their counterparts in the Europe/Americas group and OACs. At this point it is difficult to determine whether the difference in attitude is because of a language issue, that is, the familiarity with Kanji which could give participants the impression that these cognitive clues encoded in the radicals/characters are not that challenging or interesting. This observation is supported by their own admission in class to the researcher that they found Chinese (character) writing relatively easy, while learners from Europe/Americas found the opposite. The latter considered speaking easier than writing, whereas the former regarded speaking harder to master. There could be other factors that contributed to the difference. As CRILL is in English, and the classroom language was also English, with learners from Europe/Americas having higher English proficiency than those from Japan and OACs, there was a possibility that learners from the former group enjoyed the approach more because they understood more. The other factor could be learning style, with Japanese students considered to be more closure-oriented (Hansen & Stanfield, 1981: Joy & Kolb, 2007; Oxford, 1990), and thus, it was possible that they enjoyed holistic teaching less. Criticism of CRILL Judging from participantsâ&#x20AC;&#x2DC; criticism and suggestions, they tended not to be satisfied with general on-line language learning and the technical issues associated with it. Research has shown in several cases, the efficacy of on-line learning over traditional face-to-face classrooms, at least in higher education settings (Xu & Jagger, 2013). However, failure does occur when inadequately equipped e-learning systems are implemented (Hara & Kling, 2000; Zhang, et al 2004). The results from question D and E reflect some of the key challenges many on-line language learning tools are facing nowadays: feedback and technical issues. For language learners using an interactive, self-learning, and asynchronized on-line learning tool, it is frustrating when they are unable to check their own input. Although CRILL is equipped with some feedback mechanism for character writing and vocabulary as well as grammar practice, the technology involved is fairly basic and breaks down occasionally due to the

176

limitation of available technology and funding. These issues can certainly create frustration. Despite these issues, it is encouraging to learn that participantsâ&#x20AC;&#x2DC; overall perception of CRILL in terms of radicals/characters and how they compose other characters and vocabulary was positive. Participants also enjoyed the incorporated cultural learning. In light of these indications, the design of CRILL has achieved its primary goal which was to illustrate Chinese cultural beliefs and values through its writing system as an introductory course to beginners. It, however, may fall short as a full-fledged language learning program. Limitations and Implications of the Study As the sample size of each group is not always comparable, with OACs comprising the smallest group (N=7), it was difficult at times to determine whether the difference observed was due to individual preference or affected by other variables such as language, learning style and culture. Despite the fact that in some cases, there seem to be a difference between participants from the Europe/Americas and those from Japan, it was quite difficult to measure any effect from the OACs group as their number was too small. Therefore, any observation regarding this region is tentative. Furthermore, the study could not provide a definite answer to the question of the efficacy of CRILL as pre- and post-test were not administered. It also did not examine how the CRILL approach can facilitate greater learning in writing and reading Chinese characters. The study, at most, examined the perceptions of the participants based on self-reporting, and it is also not clear whether the learning approach adopted in CRILL is more effective than other approaches as there were no control groups. Better experiment designs are undoubtedly required for any future research. The pedagogical implications for this study are crucial for effective learning approaches and on-line tools in the future. In fact, a new Chinese on-line learning website has been under development that has sought to rectify the shortcomings of CRILL while also continuing to develop its positive features. These undertakings hope to demonstrate the importance of combining sound theories with viable practices in language teaching and learning.

5 Conclusion This study set out to explore the viability of a teaching approach based on the linguistic insight gained from CL in recent years. The results so far can establish its overall merit in the field of TCF by providing a holistic view of the Chinese writing system and in what way it is deeply rooted in the social and cultural worlds of the people in the Chinese speaking communities. The study demonstrates that understanding metaphor and metonymy in lexicon extension can not only enhance the learning of Chinese characters but also promote the understanding of the social and cultural knowledge encoded in them. Such conclusions certainly require caution as it was found that this approach may suit learners differently. The learner differences could be partly individual or partly cultural. As CRILL was originally designed with American and European ÂŠ 2015 The author and IJLTER.ORG. All rights reserved.

177

learners in mind, it is of great interest to find that the Japanese learners in this study were not as motivated by the approach as their European and American counterparts. This finding is of value for any future development of teaching materials and pedagogy.

Acknowledgments This design and study of CRILL was funded by the Ministry of Science and Technology (MST), Taiwan (99-2631-S-032-001). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author and do not necessarily reflect the views of MST.

References Abreu, A. S., & Vieira, S. B. (2009). Learning phrasal verbs through image schemas: A new approach. Retrieved, from http://ssrn.com/abstract=1491689 Barcelona, A. (2000). On the plausibility of claiming a metonymic motivation for conceptual metaphor. In A. Barcelona (Eds.), Metaphor and metonymy at the crossroads (pp. 31-58). Berlin: Mouton de Gruyter. Boers, F. (2004). Expanding learners’ vocabulary through metaphor awareness: What expansion, what learners, what vocabulary?. In M. Achard & S. Niemeier (Eds.), Cognitive linguistics, second language acquisition, and foreign language teaching (pp. 211-232). Berlin, New York: Mouton de Gruyter. Boers, F., Eyckmans, J., & Stengers, H. (2007). Presenting figurative idioms with a touch of etymology: More than mere mnemonics? Language Teaching Research, 11, 43-62. Boers, F., & Lindstromberg, S. (2009). Optimizing a lexical approach to instructed second language acquisition. Basingstoke: Palgrave Macmillian. Casasanto, D., & Boroditsky, L. (2007). Time in the mind: Using space to think about time, Cognition, 106(2), 579-593. doi:10.1016/j.cognition.2007.03.004 Dictionary of Chinese Character Information [汉字信息字典] (1988). Shanghai, China: Science Press [科学出版社]. Dirk, G. (Ed.). (2006). Cognitive linguistics: Basic readings. Berlin: Mouton de Gruyter. Dirven, R. (1985). Metaphor as a basic means for extending the lexicon. In W. Paprotte & R. Dirven (Eds.), The ubiquity of metaphor: Metaphor in language and thought (pp. 85-119). Amsterdam: John Benjamins Publishing Company. Dunlap, S., Perfetti, C. A., & Liu, Y. (2011). Learning vocabulary in Chinese as a foreign language: Effects of explicit instruction and semantic cue of reliability. Retrieved from: http://www.pitt.edu/~perfetti/PDF/DunlapLearningVocabulary.pdf Goossens, L. (1990). Metaphtonymy: The interaction of metaphor and metonymy in expressions for linguistic action, Cognitive Linguistics, 1(3), 323-340. Györi, G. (1996). Historical aspects of categorisation. In E. H. Casad (Eds.), Cognitive linguistics in the redwoods (pp. 175-206). Berlin, New York: Mouton de Gruyter. Györi, G. (1998). Semantic change, semantic theory and linguistic relativity. Duisburg: L.A.U.D. Hansen, J., & Stanfield, J. B. (1981). The relationship of field dependent independent cognitive style to foreign language learning achievement. Language Learning, 31, 349-369. Hara, N., & King, R. (2000). Students' distress with a Web-based distance education course: an ethnographic study of participants' experiences. Information, Communication and Society, 3(4), 557–579. Hu, Y. H., & Fong, Y. Y. (2010). Obstacles to CM- guide L2 idiom interpretation. In S. Knop, F. Boers & A. Rycker (Eds.), Fostering language teaching efficiency through cognitive linguistics (pp. 293-316). New York: Mouton de Gruyter.

178

Hu, Y. H., & Ho, Y. C. (2009). Prepositions we live by: Implications of the polysemy network in teaching English prepositions in and on. In B. Lewandowska-Tomaszczyk & K. Dziwirek (Eds.), Studies in cognitive Corpus linguistics (pp. 336-370). Frankfurt: Peter Lang Verlagsgruppe. Johnson, M. (1987). The body in the mind. Chicago: The University of Chicago Press. Joy, S., & Kolb, D. (2007). Are there cultural differences in learning style? International Journal of Intercultural Relations, 33(1), 69-85. Lakoff, G. (1987). Women, fire and dangerous things. Chicago: University of Chicago Press Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: University of Chicago Press. Lakoff, G., & Johnson, M. (1999). Philosophy in the flesh: The embodied mind and its challenge to western thought. New York: Basic Books. Littlemore, J., & Low, G. (2006). Metaphorical competence, second language learning, and communicative language ability. Applied Linguistics, 27(2), 268-294. doi: 10.1093/applin/aml004 Ma, K. Y. (1997). The relation between the writing system and the use of metaphor in English and Chinese (Unpublished master‘s thesis). University of North Dakota, Grand Forks, ND. Oxford, R. L. (1990). Missing link: Evidence from research on language learning styles and strategies. In J. E. Alatis (Ed.), Linguistics, language teaching acquisition: The interdependence of theory, practice and research (pp. 438-460). Georgetown University Round Table on Language and Linguistics, 1990. Radden, G. (2000). How metonymic is metaphor. In A. Barcelona (Eds.), Metaphor and metonymy at the crossroads (pp. 59-78). Berlin: Mouton de Gruyter. Ruiz de Mendoza, F. J. (2000). The role of mapping and domains in understanding metonymy. In A. Barcelona (Eds.), Metaphor and metonymy at the crossroads (pp. 109-132). Berlin: Mouton de Gruyster. Shen, H. H. (2000). Radical knowledge and character learning among learners of Chinese as a foreign language. Linguistic Studies, June, 85-93. Shen, H. H. (2004). Level of cognitive processing: Effects on character learning among non-native learners of Chinese as a foreign language, Language and Education, 18, 167-182. Shen, H. H., & Ke, C. (2007). Radical Awareness and Word Acquisition Among Nonnative Speaker of Chinese. The Modern Language Journal, 91(1), 97-111. Sweetser, E. (1990). From etymology to pragmatics: Metaphorical and cultural aspects of semantic structure. Cambridge: C.U.P. Talmy, L. 1988. Force dynamics in language and cognition. Cognitive Science, 12, 49-100. Tyler, A., & Evans, V. (2004). Applying cognitive linguistics to pedagogical grammar: The case of over. In M. Achard & S. Niemeier (Eds.). Cognitive linguistics, second language acquisition, and foreign language teaching, (pp. 257-280). Berlin: Mouton de Gruyter. Ungerer, F., & Schmid, H-J. (1996). An introduction to cognitive linguistics. London: Longman. Wang, J., & Koda, K. (2013). Does partial radical information help in the learning of Chinese characters? In E. Voss, S-J D. Tai & Z. Li (Eds.), Selected proceedings of the 2011 second language research forum: Converging theory and practice (pp. 162-172). Somerville, MA: Cascadilla Proceedings Project. Xu, D., & Jaggars, S. S. (2013). The impact of online learning on students‘ course : Evidence from a large community and technical college system. Economics of Education Review, 37, 46-57. Xu, S. (121 AD). Shou wen jie zi [Text in Chinese]. Yasuda, S. (2010). Learning phrasal verbs through conceptual metaphors: A case of japanese EFL learners. TESOL Quarterly, 44(2), 250-273. © 2015 The author and IJLTER.ORG. All rights reserved.

179

Zhang, D., Zhao, J., Zhou, L., & Nunamaker, J. (2004), Can e-learning replace classroom learning? Communications of the ACM, 45(5), 75-79 Zhou, L. J. (2012, May). The teaching of Chinese characters—A fun way of learning characters to decode Shou wen jie zhi. Retrieved from: http://mandarin.nccu.edu.tw/data/teacher/pdf [Text in Chinese]. Zhou, Y. L. (2013, May). Understanding the ontology of the radical/character 土 (Soil/Earth) in Shou wen jie zhi through conceptual metaphor theory and semantic field theory. Paper presented at the 15th Symposium on Chinese Philology in Central Taiwan 第十五屆中區文字學學術研討會, Chinese Cultural University, Taipei. [Text in Chinese].

180

International Journal of Learning, Teaching and Educational Research Vol. 11, No. 1, pp. 180-199, April 2015

Coaches‟ Perceptions of how Coaching Behavior affects Athletes: An Analysis of their Position on Basic Assumptions in the Coaching Role F. Moen Norwegian University of Science and Technology, Trondheim, Norway

R. Giske University of Stavanger, Norway

R. Høigaard University of Agder, Norway Abstract. This study explores coaches‟ beliefs about what they think their athletes expect from them as coaches in sport. A sample of 36 different statements representing different opinions about coach behaviours and how coach behavior affects athletes‟ motivation, performance, focus, and emotions, was presented to 23 Norwegian coaches working in high schools specialized for elite sports. The participants were coaches in various sport disciplines and were asked to consider and rank-order the statements by using a Q sorting procedure. The authors discuss their analysis from a Q methodical factor analysis. In general, the coaches share some common viewpoints that are represented in two different factors (consensus). Each factor represents congruence views about expectations in the role as a coach in sport. The dominant view (factor A) is that coaches believe that their athletes expect involvement leadership, whereas servant leadership was dominant in factor B; a view that only a few of the coaches shared. Keywords: coaching styles, coaching behavior, athletes, sport.

1. Introduction The question as to what coaching behaviour is constructive in order to develop the athlete in sport has occupied researchers and practitioners for several decades, and the influence of the coach on the athletes is well documented (Abraham, Collins, & Martindale, 2006; Blom, Watson II, & Spadaro, 2010; Côté & Gilbert, 2009; Myers, Chase, Beauchamp & Jackson, 2010). When a coach emphasizes training and instruction, and gives positive feedback that recognizes and rewards good performance, athletes are more satisfied with their leadership behaviour (Chelladurai, 2007). Similarly, a study performed by Moen, Høigaard, and Peters (2014) found that athletes who were most satisfied with their performance

181

progress evaluated their coaches‟ leadership behavior in accordance with the guidelines described by Chelladurai (2007) above. From a coaching perspective athletes‟ performance, behaviour, motivation, and emotional states within sport is affected of coaches‟ behaviour (Jowett & Cockerill, 2002; Lyle, 1999; Mageau & Vallerand, 2003; Strachen, Cote & Deakin, 2011). For an athlete to reach their potential the amount of time spent on deliberate practice is of vital importance (Ericsson, 2009) as well as the ability to manage or carry out the extensive amount of practice motivation is necessary (Deci & Ryan, 2002; Ericsson, Krampe, & Tesch-Römer, 1993). Lyle (2002) suggests that performance coaching embrace an attempt to control contributory variables and this perspective the coaches‟ role is to adapt high quality exercise, based on the athlete‟s physically, mentally and stage of social development. The ability to create a positive coaching process demands knowledge about the sport (e.g. technical, tactical, physiological, and psychological), the learning process, and the athletes‟ abilities and potential (Jones, 2006). Furthermore, good communications skills and the ability to establish a „productive‟ coach-athlete relationship are also of vital importance (Jowett & Cockerill, 2003). Moen (2012) claims that the ability to ask open-ended questions and listen to the athlete is essential in the coaching process in order to optimally stimulate an athlete‟s intrinsic motivation. This can be considered as the origin of an optimal athlete-coach relationship based on mutuality (Jowett & Meek, 2000; Olympiou, Jowett, & Duda, 2008). Lyle (2002) argues that it is not the individual coach, but the coaching process that is unique, and that the coaching role is determined by the nature of the coaching process, which takes place in a personal and social space. Furthermore, the values, idiosyncrasies and personal qualities of the coach are reflected in the interpersonal behaviour and the engagement in the coaching process. This is in line with Horn‟s (2008) working model of coaching effectiveness, which emphasizes that coaches‟ expectancies, values, beliefs, and goals influence a coach‟s behaviour. Previous research on teachers‟ beliefs has shown that beliefs may influence their perceptions and pedagogy in action (Nespor, 1987; Rokeach, 1976). Rokeach (1976) claims that all beliefs are potential predictors to action, and that both action and beliefs have an influence on one another (Haney, Lumpe, Czerniak, & Engan, 2002). Both teaching and coaching behaviour reflect values, which may be considered as means to evaluate the experience. Personal value judgements about what is appropriate, good, bad, and worthy coaching behaviours affect the coaching process and reflect core elements in a coaching philosophy (Lyle, 1999; 2002; Wilcox & Trudel, 1998). Thus, an examination of beliefs about coaching behaviours delve into the heart of understanding coaches actions and may contribute to better grasp the coaching practice (Cassidy, Jones, & Potrac, 2009). Taken together, a coach‟s main aim is to stimulate the athlete to develop his or her talent and capabilities by affecting the variables that have an impact on

182

performance (Jowett & Poczwardowski, 2007). Coaches‟ beliefs and values precursor practice and according to Cassidy et al., (2009), it is essential to know because it may contribute to explain why coaches do as they do. In the present study we focus on understanding coaches‟ perception of how their behavior effect an athlete. More specifically we want to explore the coaches‟ opinion on how and perhaps which coaching behavior affects the athlete‟s motivation, emotion, performance, and focus. We are particularly interested in identifying coaches whom express similar subjective views in order to unveil patterns among the participants. In the present work the Q methodology as a tool for studying the coaches‟ beliefs was used. Even though this research strategy has been applied studying teacher beliefs (Lim, 2009; Thorsen, 2009; Øverland, Thorsen, & Størkesen, 2012) for at least two decades, only few study in sports science have utilized this approach (Moen, 2012; Moen & Garland, 2012; Moen & Kvalsund, 2014; Moen & Kvalsund, 2013; Moen & Verburg, 2012). Thus, a second aim was to clarify if Q methodology is useful for studying coaches beliefs related to their actions as coaches.

2. Method A Q methodology was chosen because this methodology in general investigates subjectivity related to a defined topic (Brown, 1980). Subjectivity in all forms, including beliefs, views, experiences and opinions, are investigated in Q methodology (Brown, 1996). The methodological approach is completed through five tasks: 1) selecting participants, 2) defining a concourse, 3) developing a Q sample, 4) completing a q sorting, and 5) completing data analysis (Brown, 1996; Moen & Garland, 2012; Watts & Stenner, 2012). 2.1 Participants The data in this study was collected from 23 Norwegian coaches (mean 46 yrs., range 26 – 64 yrs.). Their average education was 4 years at the University level with an average of 19 years practicing as a coach. The coaches were recruited from one high school specialized for various sport disciplines (e.g. cross country skiing, biathlon, track and field, football, volleyball, and handball). This particular high school was selected because of its long experience with developing youth athletes into top international athletes. The coaches work with athletes ranging from 16 to 19 years old with performance levels varying from national top level to national top regional level. 2.2 The Concourse Based on literature, theories, and research within the coaching field in sport (Chelladurai, 2007; Côté & Gilbert, 2009; Jowett & Cockerill, 2002) we developed a concourse that consisted of a list of about 80 statements that covered different

183

beliefs about the current research question (Stephenson, 1986). The statements were written from an athlete‟s point of view: “My coach does not have to be open for questions.” The concourse was then reduced into a meaningful Q sample in order to create a balanced sample for stimulating the Q-sorters (coaches) to use the subjective statements (sample) to rank-order them self-referentially and draw a picture of their own self-conceived view on the topic (McKeown & Thomas, 1988). 2.3 The Q sample In the present study, two main themes (what Stephenson, 1950, calls effects) emerged in the concourse, coach behaviour and effect. Within the theme coach behaviour three sub-themes (what Stephenson, 1950 calls levels) seemed to be relevant: the coach‟s decision making style, the coach‟s motivational tendencies, and the coach‟s instructional behaviour (see Table 1). Within the theme effect four subthemes emerged: the athlete‟s motivations, focus, performance and emotions (Table 1). Table 1 The design of the statements based on coaching behaviour and effect

Levels Coaching behaviour

a. coach‟s decision making style

b. coach‟s motivational tendencies

c. coach‟s instructional behavior

Effect

d. athlete‟s motivation

e. athlete‟s performance

f. athlete‟s focus

g. athlete‟s emotions

The next step is to combine each of the three sub themes of coach behavior with each possible effect (Moen & Garland, 2012). Each possible combination becomes a categorical cell and it results in 3 x 4 cells (coach behavior x effects), as shown in Table 2. Table 2 The combination of levels in the design

Combination of levels Coaching behaviour

Effect

Statement No

1, 13, 25

14, 26

15, 27

16, 28

17, 29

18, 30

19, 31

20, 32

9, 21, 33

10, 22, 34

11, 23, 35

12, 24, 36

184

The authours decided to use three statements that most clearly represented the viewpoints from the concourse to represent each combination of cells. The final Q sample resulted in 36 statements (3 x 12) that represent the different combinations of cells showed in Table 1 and 2 (see Appendix). The statements in each cell are interrelated and represent the viewpoint of that cell, but each cell obtains both negative, neutral and positive statements. This is done to ensure reflections related to the particular viewpoint representing each cell. As shown in Table 2, the first statements in each cell were allocated numbers from 1 to 12, the second statements were given numbers from 13-24, and the third statements were given numbers from 25-36. In this way, it will be more challenging for the Q sorter (the coach) to understand very clearly how the system is built up (Moen & Garland, 2012). 2.4 The Q sort The coaches were invited to voluntarily participate in the study and they were all gathered in a classroom at this specific school for about 45 minutes. They were asked to provide their names, age, participated sports, education level and experience as a coach onto an individual scoreboard. The general data was gathered to explore if coaches from similar sport background or education shared similar views about their coaching practice in order to better understand the emerging factors. The coaches were given a specific condition to reflect upon and were then asked to take their time to read through all the statements presented to each of them in the Q sample (36 cards with the different statements from the Q sample). The coaches were asked to consider what they believed were expected coaching behavior from themselves as coaches. They were asked to rank-order the statements in a scoreboard ranging from a score of +5 for “most strongly agree” to -5 for “most strongly disagree” under the so-called forced quasi-normal distribution of the statements, as shown Figure 1 below (Brown, 1980, p. 197-198).

most

very

strongly

disagree

-5

-4

-3

disagree

-2

disagree

-1

neutral

agree

strongly

very

most

agree

strongly

agree

Figure 1. The scoreboard used in Q methodology

185

The coach is however free to place each statement from the Q sample anywhere within the distribution, but the participant is forced to keep to the distribution form in order to make all the necessary nuanced evaluations of the 36 different statements (Kvalsund, 1998). The statements that are placed on both extreme ends of the scoreboard, ±5 and ±4, are normally the statements that the participants have a strong connection with. Statements placed in the middle of the scoreboard are normally statements they have a more nautral connection with (Moen & Garland, 2012). 2.5 Q factor analysis After the all the scoreboards from the coaches who participated in the investigation were collected, each Q sort (each coach‟s score on the different statements in the Q sample that is placed on the scoreboard) is entered into a program that is tailored for Q methodology called PQMethod (Schmolck, 2002). PQMethod uses factor analysis to analyse congruent scores from different coaches into factors (Allgood & Svennungsen, 2008; McKeown & Thomas, 1988; Moen & Garland, 2012; Rhoads, 2007). The aim is not to generalize the findings, but to find and qualitatively explore the viewpoints related to the different factors that emerge from the factor analysis. Thus, it must not be mistaken by the quantitative factor analysis (Brown, 1980). For any n Q sorts, the correlations from the Centroid factor analysis produce a matrix of 23 x 23 cells (n x n). A high correlation coefficient indicates that two Q sorts (two scoreboards from two coaches) are sorted more or less the same way. The next step is to use a Varimax factor analysis to analyse the correlation matrix in order to find possible factors across the Q sorts. Different numbers of factors were tested for extraction, but the initial factor analysis showed that two main factors emerged: one factor had an eigenvalue (EV) of 9.6 counting for 42 % of the variance, whereas the other extracted factor had an EV of 1.54, counting for 7 % of the variance. The most influential factor(s) is the one with the highest EV, and EV is used to decide how many factors that are going to be extracted in the analysis. If a factor has an EV higher than 1 that factor is defined as a significant factor (Brown, 1980; Kvalsund, 1998). After experimenting with various alternatives by Varimax rotation of factors, the authors decided to consider a hand rotation of factors based upon an unrotated two-factor solution from the Centroid factor analysis. The main argument for using this strategy was that the initial analysis revealed a high correlation between the factors from the Varimax rotation. Factors that are highly correlated indicate that there is probably only one main factor with which virtually all participants are associated. After studying the statements that represented the different factors, a two-factor solution was chosen. From using the unrotated

186

factors as the final solution the overarching consensus of the factors reveals, since the Varimax rotation is spreading the consensus across the rotated factors, which causes them to be highly correlated.

3. Results The two factors that were discovered in this study are the two categories of beliefs related to the research question among the coaches who participated in this study (Brown, 2002). Thus, the emerging factors are created and influenced by the coaches who load on this particular factor. Q methodology uses an estimate developed by Brown (1980) to decide how high a factor loading needs to be to contribute to a factor or not (Pett, Lackey, & Sullivan, 2003, p. 208). A factor loading on a minimum of .41 was estimated in this study to decide if a Q sort (a coachâ&#x20AC;&#x;s individual scoreboard) contributed to a factor (Brown, 1980; Kvalsund, 1998). The factor matrix in Table 3 shows that factor A has 16 pure cases (sorts that load only on one factor) and 21 loadings when mixed cased are included. Factor B has 2 pure cases and 7 cases when mixed cases are included. Table 3 The Matrix of Rotated Factors and their Loadings

Q sort

Factor A

Factor B

0.64X

0.00

0.50X

0.06

0.69X

-0.28

0.84X

0.04

0.36

0.44X

0.81X

-0.14

0.64X

-0.09

0.55X

-0.35

0.79X

-0.07

0.47X

0.26

0.69X

-0.32

0.30

-0.46X

0.55X

-0.11

0.72X

-0.51X

0.72X

-0.51X

0.74X

-0.44X

187

0.64X

-0.22

0.50X

0.23

0.65X

-0.39

0.41X

-0.50X

0.53X

-0.61X

0.64X

0.02

0.42X

-0.02

Pure cases

Mixed cases

% variance explained

Note: X= significant factor loading. Factor loadings with bold faces are pure cases loading on a factor, and loadings with italic faces are mixed cases loading on more than one factor. As seen in Table 3 all sorts that significantly load on factor A are positive, while only one of the sorts that load on factor B is positive. Six of the significant loadings on factor B are negative. The analysis found a negative significant correlation between factors A and B (-.65). The statements on both extreme ends of the scoreboard, ±5 and ±4, are the statements that involves the most reflected beliefs among the coaches. Therefore, our analysis in this study focuses on the statements on both extreme ends of the scoreboard (Brown, 1980). Based on characteristic statement, distinguishing statement and consensus statement we labelled factor A: Involvement leadership and factor, B: Servant leadership. 3.1 Factor A: Involvement leadership The most extreme statements loading on factor A on the positive side (+5 and +4) emphasize the importance of involvement of athletes to affect their motivation, emotions (such as curiosity and interest), and performance (statement number 1, 2 and 4). The most extreme statements on the negative side (-5 and -4) also highlight the importance of involvement to affect an athlete‟s emotions (such as commitment), and focus (statement number 16 and 27) (see Table 5). Finally, feedback and social support are also emphasized in order to affect performance (statement number 18). A democratic coaching behavior is when athletes are invited to participate in coaching decisions together with the coach (Chelladurai, 1989). The involvement leadership factor represents the views of 16 of the participant coaches. A comparison with the respondents who not loaded on factor A and the general variables on each scoreboard that documented the

188

coaches coaching experience and education level, shows no differences in coaching experience or education level between the two groups of coaches. The strong loading on factor A seems to reflect that coaches consider stimulating athlete involvement as a core element in their coaching practice. Following Chelladurai´s (1989, 1990) description of democratic behavior, the loading on factor A seem to be associated with this dimension affording participation in athletes‟ decisions in their sport (item 16). However, the views in factor A also represent a fundamental pedagogical perspective emphasized that athlete involvement are a major source influencing athlete motivation, curiosity and performance. Vygotsky (1978) suggest that all learning and development is based on social activity and interaction with tools. Coaching effort directed towards stimulating athlete involvement can be considered as measure that gives opportunities to reciprocate reflection and enable the coach and the athlete to think and understand each other. Mediation is a key concept in Vygotsky‟s theory, and the role of the mediator (coach) is to clarify what kind of involvement is effective or not in order to enhance performance of the individual (Kozulin, 2003). According to Kozulin (2003) human meditation seems to be too numerous and context dependent to allow a simple classification, but belongs to interactive activity. The participants loading on factor A is a strong disagreement suggesting that neither feedback nor social support are crucial for their performance in sport indicating that this coaching behavior is a kind of mediating involvement that enhancing performance (statement 18). Table 4 Distinguished Statements Loading on Factor A; Involvement leadership

Number

Statement

Strength

My motivation for training increases when my coach involves me.

I become curious and interested if my coach involves me in matters concerning my training.

If I‟m involved in the process concerning my training I perform better.

I become stressful if my coach involves me in important matters regarding my -4 training.

Neither feedback nor social support are crucial for my performances in sport.

I become uncommitted if my coach includes me in decisions regarding my -5 sport.

Note: Included mixed cases, 21 coaches loaded on factor A.

-4

189

3.2 Factor B: Servant leadership - caring and non-task controlling Viewpoints loading on factor B were held of two (pure cases) of the participant coaches. Compared with the other respondents‟ general variables (sports, experience and education level) they are both involved in individual sport (athletic) with a long coaching and teaching experience, but with no difference in educational background. On the positive side (see Table 6), they emphasize the importance of not being met by instructive behavior from the coach (statement number 24 and 33). The coaches who are loading factor B believe that clear instructions negatively affect motivation and involvement negatively affects focus (statement number 27). Therefore, this coaching behavior is undesirable. The most extreme statement on the negative side further emphasize that there is no need for clear instructions to develop the athlete‟s performance (statement number 10). The lack of faith in instruction as a pedagogical measure appear to be a main characteristic in factor B, which contradicts that a basic element in coaching is guided improvement the long term development of athletes. However, textbooks in motor learning and physical education have argued that instructions can be ineffective and the effect of instruction may depend on the skill and the performance level of the athlete (Schmidt & Wrisberg, 2008; Siedentop & Tannhill, 2000). An analysis of the general variables to the respondents loading on factor B shows that they are from athletics. In some of the athletics exercises like distance running, technical skill instruction may be consider excessive especially if the athlete has developed and automatized a rather efficient running technic. Coaching these athletes is more about creating a training program with recommended training volume (training frequency, duration and intensity). The two other psychological statements representing factor B negatively emphasizes that either a close relationship with a coach or involvement are needed to affect emotions such as curiosity and focus (statement number 28 and 31). However, coaches disagree with these statements indicating that the respondents view a close relationship and involvement from the coach as suitable. Jowett (2005) claims that coach athlete relationship can be described in a) prizewinning and b) helpful/caring relationship dimensions. The two dimensions are interrelated and the prizewinning category has effective and ineffective as sub-dimension, while the helpful/caring relationship has successful and unsuccessful subcategories. The belief that instruction is not efficient, which represent one belief loading one factor B may be categorized in the ineffective prizewinning dimension in Jowett‟s (2005) taxonomy. The belief that closeness and athlete involvement are desirable can be considered as a basic prerequisite developing a successful relationship, which is a category in the helpful/caring dimension in the taxonomy. According to Stone, Russell, and Patterson (2003, 2004) servant leaders gain influence in a non-traditional manner that derives from servant hood itself. They

190

allow extraordinary freedom for followers to exercise their own abilities and place much higher degree of trust than would be the case in any leadership style required by the leader to be directive. The coaches loading on factor B seem therefore to have elements of a servant leadership. Table 5 Distinguished Statements Loading on Factor B

Number

Statement

Strength

I am losing my curiosity when my coach gives me clear instructions.

I become stressful if my coach involves me in important matters +4 regarding my training.

If I‟m told exactly what to do I lose my motivation.

I’m calm and steady regardless of a close relationship with my coach or not.

-4

I’m curious regardless of involvement or not from my coach.

-4

Clear instructions regarding what I am supposed to do develop my -5 performances.

Note. Included mixed cases, 20 cases loaded on factor B.

4. Discussion The 23 coaches who participated in this investigation were instructed to sort 36 statements about different views on coaching behaviours, and rank the statements on a scoreboard ranging from +5 to -5 regarding what they believe are expected behavior in their roles as coaches in sport. Based on their experience they were asked to reflect on the content of the statements and prioritized them in accordance with their own personal view. The results show that 21 out of the 23 coaches, (when mixed sorts are included) loaded significant positive on Factor A: Involvement leadership (Table 3). This factor counts for 38% of the variance. Seven coaches (when mixed sorts are included) loaded on factor B: Servant leadership (Table 3). However, only one coach loaded significant positive on factor B and the rest of the loadings were negative. After analysing the two different factors it is clear that the factors represent individual viewpoints that clearly separate them from each other. The negative correlation between the factors confirms this as well as the scores on each statement representing the two factors (Appendix). In the discussions below these two factors will therefore be treated based on their typical individual viewpoints.

191

4.1 Involvement leadership The theory of transformational leadership was developed by Bass (1985) and he proposes that transformational leaders display certain characteristics such as showing concern and care for each subordinate, and stimulating them intellectually to think about old problems in an innovative way. In sport, Charbonneau, Barling, and Kelloway (2001) showed that intrinsic motivation mediate the relationship between transformational leadership and sport performance, suggesting that transformational leadership may enhance intrinsic interest in the task. The following strong statement loading on factor A ”I become curious and interested if my coach involves me in matters concerning my training” (4) and “Neither feedback nor social support are crucial for my performances in sport” (-4) seems to be beliefs that correspond with elements of transformational leadership. A belief in athlete involvement in the training process seems to be a prerequisite to stimulate athletes intellectually, and provide a foundation for innovations. A fundamental belief in involvement is also a precondition for stimulating athlete autonomy and competence, which are cornerstones in self-determination theory (Ryan & Deci, 2002). A basic assumption in self-determination theory is that intrinsic motivation is affected by the extent to which the fundamental human needs for competence, autonomy, and relatedness are fulfilled or satisfied. According to Amorose and Anderson-Butcher (2007) there are motivational benefits of autonomy-supportive coaching behavior (Amorose & Horn, 2000). The respondents loading on factor A strongly disagree that feedback and social support are crucial for performance in sport indicating that these coaching behavior are important for performance enhancement. This finding also seems to be in line with transformational leadership, which emphasizes caring and concern for subordinates or athletes. Interestingly, 21 out of the 23 coaches in this study loaded on factor A when mixed sorts are included. There are five mixed sorts across the two factors and they are all loading negatively on factor B. Thus, the views representing factor A are strong among the coaches in this study and factor B seems to represent a contrasting view compared with factor A; since their statements are sorted completely different (see Appendix and the different scores for each statement on factor A and B). Viewpoints representing factor A in this study do not solely confirm earlier research on coaching behaviours (Chelladurai, et al., 1988; Horne & Carron, 1985; Moen & Sandstad, 2013; Riemer & Chelladurai, 1995). This research shows that coaching behaviour associated with training and instruction, positive feedback, and social support are highly correlated with athletes‟ satisfaction and their intrinsic motivation. Viewpoints that are associated with positive feedback and social support are represented in factor A, but viewpoints associated with training and instructions are not. This is rather surprising, while instructive behaviour from a coach, especially with junior athletes, might be necessary to

192

meet the quality that is needed to enhance the athlete‟s performance level (Ericsson, 2009). Importantly, exercise that is needed to improve an athlete‟s performance levels is not found to be a playful enjoyment (Ericsson, et al., 1993). Developing an athlete‟s level of performance is an effortful endeavour that takes engagement, curiosity and inspiration. Interestingly, the coaches that are loading on factor A in this study believe that coaching behaviour promoting involvement has an affect on these emotions. The belief in the involvement of athletes during the training process can be considered as a basic value and a prerequisite to empower the athlete to become more independent and take ownership of the learning process. According to Jowett (2007) is an interdependent relationship between coach and athlete described in terms of closeness, commitment and complementarity. A belief in athlete involvement in the training process, as well as an opinion of the necessity of social support and feedback in the learning process seems to be an important precondition to promote a positive and healthy coach-athlete relationship. However, athlete involvement is also a necessary precondition establishing a more democratic leadership. One can ask if the emphasis on athlete involvement and social feedback among these coaches on the one side, together with the absence of determined behavior such as criticism and instruction on the other, are too friendly of nature for coaches who are working to improve a junior athlete‟s levels of performance. A recent study found that junior athletes expect a paradoxical mixture of humility (involvement, positive feedback, a personal relationship and social support) and determinate behavior (criticism and instructions) from their coaches (Moen & Sandstad, 2013). 4.2 Servant leadership - caring and non-task controlling The most psychologically significant statements representing factor B seem to represent the view that an athlete does not need instructions from a coach to affect motivation, performance and emotions. Two of the most psychological negative statements representing factor B (statements number 28 and 31) are understood as concerns among the coaches loading on this factor. The consensus representing factor B seems that an athlete is expected to act independently of the coach and that the coach does not expect to take too much responsibility regarding the athlete. Thus, the way coaches loaded factor B suggests that they believe their role is expected to be in the background and not being involved too much in the work with the athlete and the athlete is expected to take responsibility him- or herself. Werthner & Trudel (2006) suggest that elite coaches learning can be understood in terms of mediated, unmediated and internal processes. Where in a mediated learning situation a coach directs the athlete, opposed to no coaches present in unmediated situations. Thus, the

193

athlete must take the initiative and the responsibility for choosing what to learn in unmediated learning situations. These concepts may also be applicable concerning athlete learning and coaches loading on factor A seems to believe strongly in learning these young athletes is a mediating process, while coaches loading on factor B believe this is an unmediated process. 4.3 Conclusion, implications and future research The aims of this study were to explore coaches‟ opinion about how and eventually which coaching behavior affects the athletes. Furthermore, discover patterns among the participants and identify coaches who express similar subjective views, and clarify if Q methodology is suitable for studying coaching beliefs. The present findings show that the coaches share some common viewpoints represented in two different factors (consensus). Each factor represents congruent views related to the role of coaching expectations in sport. The dominant view (factor A) is that coaches believe that their athletes expect involvement leadership, whereas servant leadership was dominant in factor B, a view that only a few of the coaches shared. The coaching role in sport is relatively free with great opportunities for the individual to create and design it based on personal convictions (Heinemann, 1983). Thorsen (2009) argues that Q methodology stimulates participants‟ awareness of their own position related to the research question. Based on findings from studies in cognitive therapy, which shows that attitudes, beliefs and expectations can shape peoples reality and behavior, we assume that the finding in this study is important for coaching in sport (David, Lynn & Ellis, 2010). We often assume that coaches‟ values are observable in their behavior or whish that it should be. However empirical examination shows that the connection is not as straightforward as the coaching literature would have us to believe (Lyle, 1999). This is because little account is taken of the contextual pressure, which also influences coaching behavior (Cassidy, Jones, & Potrac, 2009). These beliefs seem to be easy to say and difficult to keep, and the link between coaches‟ beliefs‟ and their actions has rarely been empirically investigated (Lyle, 1999). Further research should illuminate the development of these beliefs and clarify how coach education influences each factor. The findings in the present study show that a Q methodology approach can be a helpful in the exploration of coaching beliefs 4.4 Limitations Q studies are not designed to generalize results to larger populations or to determine causal relationship between variables or estimate prevalence (Øverland, Thorsen, & Størksen, 2012). Generally, Q methodology explores subjective views and this was the intent in studying coaches‟ perceptions of

194

significant beliefs. Qualitative research has occasionally been criticized for modest quality (Mays & Pope, 2000), but clear guidelines for conducting a Q study should accommodate this critic. Brown, (1980) argue that Q methodology is a less biased than questionnaires, where the content can be predetermined, and interviews where it is up to the researcher to categorize the results.

Acknowledgments The authors would like to thank the coaches for their cooperation, enthusiasm, and most of all, their participation in this study.

References Abraham, A., Collins, D., & Martindale, R. (2006). The coaching schematic: Validation through expert coach consensus. Journal of Sport Sciences 24, 549-564. Allgood, E., & Svennungsen, H. O. (2008). Toward an articulation of trauma using creative arts and Q methodology. Journal of Human Subjectivity, 6, 5-24. Amorose, A. J., & Anderson-Butcher, D. (2007). Autonomy-supportive coaching and self-determined motivation in high school and college athletes: a test of selfdetermination theory. Psychology of Sport and Exercise, 8, 654-670. Amorose, A. J., & Horn, T. S. (2000). Intrinsic motivation: Relationships with collegiate athletes gender, scholarship status and perceptions of their coachesâ&#x20AC;&#x; behavior. Journal of Sport & Exercise Psychology, 22, 63-84. Bass, B. M. (1985). Leadership and performance beyond expectations. New York: Free Press. Blom, L. C., Watson II, J. C., & Spadaro, N. (2010). The impact of a coaching intervention on the coach-athlete dyad and athlete sport experience. Athletic Insight- The Online Journal of Sport Psychology, 12. Retrieved from http://www.athleticinsight.com/ Vol12Iss3/Feature.htm. Brown, S. R. (1980). Political subjectivity, applications of Q methodology in political science. Yale University. Brown, S. R. (1996). Q methodology and qualitative research. Qualitative Health Research, 6, 561-567. Brown, S. R. (2002). Subjective behaviour analysis. Operant Subjectivity 25, 145-163. Cassidy, T., Jones, R., & Potrac, P. (2009). Understanding sports coaching: The social, cultural and pedagogical foundations of coaching practice (2nd ed.). London: Routledge. Charbonneau, D., Barling, J., & Kelloway, E. K. (2001). Transformational leadership and sports performance: The mediating role of intrinsic motivation. Journal of Applied Social Psychology, 1, 1521- 1534. Chelladurai, P. (1989). Manual for the leadership scale for sports. Unpublished manuscript. The University of Western Ontorio. Chelladurai, P. (1990). Leadership in sports: A review. International Journal of Sport Psychology, 21, 328-354. Chelladurai, P. (2007). Leadership in sports. In G. Tenenbaum., & R. C. Eklund (Eds.), The

195

handbook of sport psychology (3rd edition, pp. 113-135). New York: Wiley. Chelladurai, P., Imamura, H., Yamaguchi, Y., Oimnuma, Y., & Miyauchi, T. (1988). Sport leadership in a cross-national setting: the case of Japanese and Canadian university athletes. Journal of Sport & Exercise Psychology, 10, 374-389. Côté, J., & Gilbert, W. (2009). An integrative definition of coaching effectiveness and expertise. International Journal of Sport Science and Coaching, 4, 307-232. David, D., Lynn, S., & Ellis. A. (2010). Rational and irrational beliefs. Implications for research, theory, and practice. NY: Oxford University Press. Deci, E. L., & Ryan, R. M. (2002). Handbook of self-determination research. Rochester, New York: University of Rochester Press. Ericsson, K. A. (2009). Development of professional expertise: Toward measurement of expert performance and design of optimal learning environments. Cambridge, UK: Cambridge University Press. Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100, 363–406. Haney, J. J., Lumpe, A. T., Czernaik, C. M., & Egan, V. (2002). From beliefs to actions: The beliefs and actions of teachers implementing change. Journal of Science Teacher Education, 13, 171-187. Heinemann, K. (1980). Introduction to the sociology of sport. Schorndorf: Hofmann Verlag. Horn, T. S. (2008). Coaching effectiveness in the sports domain. In T. S. Horn (Ed.) Advances in sport psychology (pp. 309-354). Champaign, IL: Human Kinetics. Horne, T., & Carron A. V. (1985) Compatibility to coach-athlete relationships. Journal of Sport Psychology 7, 137-149. Jones, R. L. (2006). The sports coach as educator: re-conceptualising sports coaching. Routledge: Taylor & Francis Group, London. Jowett, S. (2005). The coach-athlete partnership. The Psychologist, 18, 412-415. Jowett, S. (2007). Interdependence analysis and 3+1 Cs in the coach-athlete relationship. In S. Jowett, & D. Lavallee (Eds.), Social psychology in sport (pp.15-28). Champaign, IL: Human Kinetics. Jowett, S., & Cockerill, I. M. (2002) Incompatibility in the coach–athlete relationship. In I. M. Cockerill (Ed.), Solutions in sport psychology. (pp. 16–31) London: Thompson Learning. Jowett, S., & Cockerill, I. M. (2003). Olympic medalists‟ perspective of the athlete-coach relationship. Psychology of Sport and Exercise, 4, 313-331. Jowett, S., & Meek, G. A (2000). The coach–athlete relationship in married couples: An exploratory content analysis. The Sport Psychologist, 14, 157–175. Jowett, S., & Poczwardowski, A. (2007). Understanding the Coach-Athlete Relationship. In S. Jowett, & D. Lavallee (Eds.), Social Psychology in Sport (pp.3-14). Champaing IL, Human Kinetics. Kozulin, A. (2003). Psychological tools and mediated learning. In A. Kozulin, B. Gindis, V. Ageyev & S. Miller (Eds.), Vygotsky's educational theory in cultural context (pp. 15- 38). Cambridge: Cambridge University Press.

196

Kvalsund, R. (1998). A theory of the person. Department of Education Faculty of Social Science and Technology Management. The Norwegian University Of Science And Technology Trondheim. Lim, C. (2009). Understanding Singaporean preschool teachers‟ beliefs about literacy development: four different perspectives. Teaching and Teacher Education, 26, 215-224. Lyle, J. (1999). Coaching philosophy and coaching behaviour. In J. L. Cross, & J. Lyle (Eds.), The coaching process: Principles and practice for sport (pp. 25–46). Oxford: Butterworth-Heineman. Lyle, J. (2002). Sports coaching concepts. London: Routledge. Mageau, G. A., & Vallerand, R. J. (2003). The coach-athlete relationship: A motivational model. Journal of Sports Sciences, 21, 883-904. Mays, N., & Pope, C. (2000). Qualtative research in health care: Assessing quality in qualitative research. BMJ, 320, 50-52. McKeown, B., & Thomas, D. (1988). Q Methodology. London: Sage. Moen, F. (2012). Subjective beliefs among coaches about how relational factors affect intrinsic motivation, responsibility and development in Sport. Athletic Insight Journal, 3, 219- 235. Moen, F., & K, Garland (2012). Subjective beliefs among Sport Coaches about Communication during practice. Scandinavian Sport Studies Forum (SSSF), 3, 121-142. Moen, F., & R. Kvalsund (2013). Subjective beliefs among sport coaches about communication during coach-athlete conversations. Athletic Insight Journal, 5, 229-250. Moen, F., & R. Kvalsund (2014). Subjective beliefs among sport coaches about communication during practice and coach-athlete conversations . Operant Subjectivity, 37, 1–19 Moen, F., & Verburg, E. (2012). Subjective beliefs among athletes about how relational factors affect intrinsic motivation, responsibility and development in sport. The International Journal of Coaching and Science, 1,81-100. Moen, F., & Sandstad, H. (2013). Subjective expectations among athletes about coaches in sport. Journal of Human Subjectivity, 11, 135-155. Moen, F., Høygaard, R., & Peters, D. M. (2014). Athletes’ performance progress in relation to perceived coach-leadership behavior. The International Journal of Coaching Science, 8, 67-79. Myers, N. D., Chase, M. A., Beauchamp, M. R., & Jackson, B. (2010). The coaching competency scale II – High School Teams. Educational and Psychological Measurement, 70, 477-494. Nespor, J. (1987). The role of beliefs in the practice of teaching. Journal of Curriculum Studies, 19, 317-28. Olympiou, A., Jowett, S., & Duda, J. L. (2008). The psychological interface between the coach-created motivational climate and the coach-athlete relationship in team

197

sports. The Sport Psychologist, 22, 423-438. Pett, M. A., Lackey, N. R. & Sullivan, J. J. (2003). Making Sense of Factor Analysis: The Use of Factor Analysis for Instrument Development in Health Care Research. Thousand Oaks, CA: Sage. Rhoads, J. C. (2007). Q methodology. In N. J. Salkind & K. Rasmussen (Eds.), Encyclopedia of measurement and statistics (pp. 799-802). Thousand Oaks, CA, London, & New Delhi: Sage. Riemer, H. A., & Chelladurai, P. (1995). Leadership and satisfaction in athletics. Journal of Sport and Exercise Psychology, 17, 276-293. Rokeach, M. (1973). The nature of human values. New York: The Free Press. Rokeach, M. (1976). Beliefs, attitudes, and values: A theory of organization and change. San Francisco: Jossey-Bass Publishers. Schmidt, R. A. & Wrisberg, C. A. (2008). Motor Learning and Performance, (4th ed.). Champaign, IL: Human Kinetics. Schmolck,

(2002).

PQMethod

download

mirror

retrieved

from

http://www.qmethod.org/Tutorials/pqmethod.htm Siedentop, D., & Tannehill, D. (2000). Developing teaching skills in physical education (4th ed.). Palo Alto, CA: Mayfield. Stephenson, W. (1950). A statistical approach to typology: The study of trait universes. Journal of Clinical Psychology. 6, 26-38. Stephenson, W. (1986). Protoconcurcus: The concourse theory of communication. Operant Subjectivity, 9, 37-58, 73-96. Stone, A. G., Russell, F. R., & Patterson, K. (2003). Transformational versus servant leadership: A difference in leadership focus. Proceedings of the Servant Leadership Research

Roundtable.

Retrieved

July

15,

2004,

from

http://www.regent.edu/acad/cls/2003ServantLeadershipRoundtable/ Stone, A. G., Russell, R. F., & Patterson, K. (2004). Transformational versus servant leadership: A difference in leader focus. Leadership & Organization Development Journal, 25, 349-361. Strachan, L., Côté, J., & Deakin, J. (2011).

A new view:

Exploring positive youth

development in elite sport contexts. Qualitative Research in Sport, Exercise, and Health, 3, 9-32. Thorsen, A. A. (2009). Teachers' priorities and beliefs: A venture into beliefs, methodologies, and insights. Department of Education, University of Stavanger, UiS, Doctoral dissertation. Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press. Watts, S., & Stenner, P. (2012). Doing Q methodological research. CA: Sage. Werthner, P. & Trudel, P. (2006). A new theoretical perspective for understanding how coaches learn to coach. The Sport Psychologists, 20, 196-210. Wilcox, S. & Trudel, P. (1998). Constructing the coaching principles and beliefs of youth ice hockey coach. AVANTE, 4, 39-66.

198

Øverland, K., Thorsen, A. A., & Størksen, I. (2012). The beliefs of daycare staff and teachers regarding children of divorce: A Q methodological study. Teaching and Teacher education, 28, 312-323.

Appendix A Factors Statements

My motivation for training increases when my coach involves me.

-3

If I‟m involved in the process concerning my training I perform better.

-3

My coach does not have to be open for questions.

-1

I become curious and interested if my coach involves me in matters

-2

-1

-5

-3

-1

15. My coach needs to consult me if I‟m supposed to have an effective focus.

16. I become uncommitted if my coach includes me in decisions regarding

-5

17. My motivation increases when I receive positive feedback.

-2

18. Neither feedback nor social support is crucial for my performances in

-4

19. A close and personal relationship with my coach makes me stressful.

-3

20. My curiosity is best stimulated when the relationship with my coach is

-1

-2

concerning my training. 5.

My motivation increases when my coach is concerned about my personal well-being.

If I‟m supposed to achieve good performances my coach needs to focus on my personal welfare.

My situation becomes less stressful when my coach contributes in personal affairs.

A personal and close relationship with my coach makes me enthusiastic concerning my training.

My motivation increases when I‟m told exactly what to do.

10. Clear instructions regarding what I am supposed to do develop my performances 11. I keep my focus if the coach intervenes in training and explain what is right and wrong. 12. I become curious if my coach gives me clear instructions about what I need to do. 13. My motivation increases when my coach takes decisions that concern me. 14. My performances are not good when my coach denies complying with my opinions.

my sport.

sport.

not too close and personal. 21. I lose my engagement when I‟m observed by my coach and receive no feedback.

199

22. I perform at my best when I have to clarify my own task for training.

23. I become insecure if a coach does not tell me exactly what to do.

-1

24. I am losing my curiosity when my coach gives me clear instructions.

-3

25. My motivation decreases when my coach needs my approval in

-2

-4

28. I‟m curious regardless of involvement or not from my coach.

-4

29. My motivation increases when my coach does not have focus on

-1

-4

-2

33. If I‟m told exactly what to do I lose my motivation.

-2

34. I perform at my best when my coach just observes what I do during

-1

35. I lose my focus when it is too much instructions.

-1

36. It is easier to be curious when the coach is more in the background.

important matters concerning my training. 26. I‟m not able to perform if my coach often asks me for approvals in important matters. 27. I become stressful if my coach involves me in important matters regarding my training.

personal issues. 30. In order to develop my performances I also need critical feedback from my coach. 31. I‟m calm and steady regardless of a close relationship with my coach or not. 32. Whether my coach is concerned about personal issues or not do not affect my curiosity.

training.

* Translated from Norwegian to English by the authors.

200

International Journal of Learning, Teaching and Educational Research Vol. 11, No. 1, pp. 200-211, April 2015

Regional Educational Development Research and School Improvement: A Systematic Literature Review of Research Associate Professor Lena Boström Departement of Education, Mid Sweden University

Abstract. With the background of regional educational problems in some counties in Sweden, in the form of lower educational level and school achievements an integrative and systematic literature review (SLR) has been implemented in order to examine regional educational development in combination with school development worldwide during the last 50 years. The literature review shows an internationally sprawling view of the content and object of the study as well as a few international studies on the relationship between regional development and school improvement in mainstream school. Different regions, countries, and continents show various problems within this focus. The study shows different aspects; networks and knowledge-based clusters as guidelines; political decentralization and its repercussions on student learning; technical solutions; the importance of vocational schools in rural areas; tax reduction to compensate rural areas; the importance of a majority/national language and rural versus urban values. Regional development policies aimed at stimulating regional industrial competitiveness and innovativeness, educational attainment, and creative occupations emphasize that creative professionals are strongly related to regional income. The most important implication common worldwide, are concerns pertaining to urbanization, depopulation of rural areas, the need to highlight regional development factors, and a belief in education, especially involving mainstream schools.

Keywords: literature review, mainstream schools, regional educational development, school improvement.

Introduction Two counties in Mid-Sweden, Jämtland and Västernorrland, have identified regional educational problems. These counties have lower educational levels and school achievements compared to other parts of Sweden (Skolverket, 2014) and have a relatively large proportion of an out-migration of well-educated individuals from the region (Statistiska centralbyrån, 2014). The national school results have, in some parts of the region, steadily declined for nine years; other regional areas also need to be improved (Skolverket, 2014). An integrative and © 2015 The author and IJLTER.ORG. All rights reserved.

201

systematic literature review (SLR) concerning Swedish (and Nordic) research has been carried out in order to get an overall picture of the best possible knowledge in this area. This review showed that there has been little debate, and the sources were mostly descriptive studies. In addition, one-third of the sources were official reports. The discussions, debates, and investments pertaining to universities and education programs‘ role in the Swedish rural areas began in the mid-1980s. Even then, the role of universities for rural and Sweden‘s prosperity was discussed in light of increased global competition. There are not any research studies or debates to discuss the grade levels through the whole school system. Studies about school improvement and the impacts on students and regional development are also missing. Young people‘s and women‘s voices are absent (Boström, 2015). The concern for the lack of research on ‗regional youth‘ is raised by (Svensson, 2006), with the words, “When rural areas are studied, the focus is outside the adolescent themes and the adolescents studied, the focus is outside the rural perspective‖ (p. 24). A key question for a practitioner teaching in school or higher education today could be as follows: How can we make sense of current regional educational research and debates in order to implement the results into teaching and courses? (Svensson, 2006). In other words, there is a research gap regarding the role of school (mainstream) and regional development. The purpose of this overview is therefore to gather international knowledge about the object of knowledge. The aim of the study was to deepen the knowledge of previous research on regional educational development and school improvement worldwide as well as to capture and describe the phenomenon of variation in different contexts.

Previous Research The past decade may have witnessed considerable expansion and development within the field of regional educational research (From & Olofsson, 2014). Good education systems, ranging from preschools to universities, are vital for development. This mutual relationship is described and emphasizes that initiatives in the classroom or department are influenced by the surrounding context of the school, the district, and the nation (Hernandez & Goodson, 2010; Veugelers & Ziljsra, 2010). It is important not only from a regional perspective, but also in a national and international context. The importance of higher education for a region‘s growth has increasingly attracted international attention in many countries, including Russia, Britain, Canada, Turkey, China, and Australia, and on all continents. If countries want to be globally competitive, regional innovation systems need to be strengthened. In order to achieve this, cooperation between higher educational institutions, public authorities, and the business sector becomes vital. Regional engagement, academic excellence, and research are complementary activities. In order to allow innovative efforts in peripheral regions to be as productive as they are in core areas, they need to be complemented by huge investments and human capital (Crescenzi, 2005). The role of human capital and the creative class, as well as technology factors, sharpen regional development. For regions to compete, evolve, and survive, education is crucial, and there are clear links between © 2015 The author and IJLTER.ORG. All rights reserved.

202

education and regional development (From & Olofsson, 2014). A further aspect is the concept of regional management, which seems to play an important role in developing regional headquarters into dynamic competence centers (Ambos & Schlegermilch, 2010). Even in Sweden, the regional educational development in cooperation with academic attention is problematized and researched (e.g., Persson et al., 2003). The majority of the research in Sweden during the last 40 years has focused on higher education, regional projects, and collaboration between industries and universities. Only a few studies have touched on the younger generation‘s situation (Boström, 2015). An attempt to connect to regional development in the school context is the research project ―The best regional education systems in the world,‖ which is the partnership of one university, huge companies, and three selected schools in the region of Mid-Sweden. The intent behind the project is to make it possible for a region like Mid-Sweden to develop in a positive way for citizens companies, and the public sector, and the underlying assumption is that the regional educational system needs to contribute to regional changes regarding economic growth as well as the public sector (Mittuniversitetet, 2012). An important background factor is that these countries have lower educational levels and school achievements compared with other parts of Sweden, and they have a relatively large proportion of out-migration of well-educated individuals from the region. Therefore, the aim of this study is to identify regional educational-development research and regional school improvements across a variety of contexts through a systematic review of associated literature published between 1983 and 2014. Such a review contextualizes and is an international background for the research project. The concepts in the study are based on research into both regional educational development and regional school development By virtue of the foregoing, the purpose of this study was, from an international perspective, to identify and classify the published research on regional educational development and school improvements. The objectives were to (a) identify patterns and trends in the research, (b) describe and compare the published findings, and (c) point to a future research agenda.

Theoretical Framework The study departs from regional educational development and school improvement theory. Regional educational development is a useful analytical tool as it can mirror the relation between regional development and education. There are correlations between level of education and regional development. To succeed in regional development, a highly skilled workforce is required where education is a cornerstone (e.g. Florida, Mellander & Stolarick, 2010; Tomaney & Wray, 2011). Universities were perceived of utmost importance with regard to regional development (Westlund, 2004), but regional development is also a social change and transformation (Berglund & Johansson, 2007). To go from stability and recognition to the new and unfamiliar is described in the following words: ―It is the combination of the new and the traditional providing innovative opportunities for regional development and economic growth, which requires interaction and communication‖ (From & Olofsson, 2013, p. 35). © 2015 The author and IJLTER.ORG. All rights reserved.

203

The core of the definition of school improvement is described as “… a systematic, on-going, and supported efforts to make principals and teachers in schools more capable of achieving the national and local objectives in the school” (Björkman, 2008, p. 33), with references to P. Dalin, 1994, and W. van Velzen et al., 1985, and implies a particular form of changes in a certain direction to make something better. Goodland, 1994, believes that school improvement needs a focus as a whole in order to be successful. Culture and structure both have to be consciously processed in order to achieve effective and sustainable change (Hargreaves & Fink, 2008). School improvement theory must, according Höög and Johansson, 2014, bear cornerstones and different focuses concerning structure, culture, processes, and leadership. These elements collaborate, interact, and have repercussions on each other in order to change and adapt the structure and long term consistent processes which in turn affect the school‘s leadership by providing dynamics and conflict. In summary, the theoretical frameworks are set around school improvement comprising structure in the context of regional development, including culture, processes, and leadership, which requires communication and interaction with both new and traditional activities.

Aim and Methods The aim of this study was to deepen the knowledge of previous research on regional educational development and school improvements worldwide. This study is based on an integrative and systematic literature review (SLR), inspired by Polit and Beck‘s (2008) model for a qualitative, thematic, content analysis. Integrative literature reviews offer a means for researchers to find and assess what is known about a particular topic, with the aim of finding a solution to a particular problem or suggesting directions for future research. It is described as being the broadest category of research reviews, combining the findings of a range of different research designs including those of qualitative and quantitative studies (Evans & Waring, 2012). To analyze the contents of the various texts, a descriptive content analysis was performed. The contents of the results of the studies were examined methodically and progressively while interpreting texts to find prevalent phenomena. Distinctive categories were identified and then narrowed down to sub-categories. Traditional content analyses can be divided into three steps: selection of focus texts, encoding of the texts, and interpretation of the results (Auhiva, 2008). The situational context was taken into account, so the ―maximum variation sampling‖ was achieved (Franzosi, 2008). Maintaining scientific integrity involves great attention to validity during the phase of the integrative review, and not defining the operational definitions too narrowly or too broadly. The reviewer must balance the definitions and methods review constantly during the research process. The integrative literature review has many benefits to the scholarly reviewer, such as identifying gaps in current research and the need for future research, bridging related areas of work, and bringing focus to central issues in an area (Cooper, 1998). A thematic integrative study design was chosen to obtain a holistic understanding of the subject. This © 2015 The author and IJLTER.ORG. All rights reserved.

204

approach, in accordance with the method, allows researchers to combine data from theoretical literature as well as empirical literature (Whittemore & Knafl, 2005 ), which also makes the analysis more complex and difficult. The purpose could therefore be answered by a wider range of sources. Strategy and Inclusion Criteria Employing progressively detailed inclusion criteria, the systematic review of the literature involved multiple phases: an initial identification of potential studies via an advanced search of PRIMO; subsequent screening of abstracts and titles; hand searching of seven books, in-reference lists, and bibliographies of all retrieved papers, 15 book chapters, and five reports, ten books, five dissertations, ten debate articles on the theme; and an in-depth review of 190 full articles along with a detailed thematic analysis of 33 full articles, five chapters, one dissertation, two reports and 7 books. An automated advanced search of this database was conducted using increasingly refined search criteria to identify those potentially relevant studies for use in the review (see Figure 1).

Figure 1. Summary of Method of Review

Keywords used were: regional educational development*, school improvement*, school efficiency*, regional school development*, and school effectiveness*, with different combinations. The literature search was limited to 1980â&#x20AC;&#x201C;2014 and to publications written in English. To contextualize the findings of the review, an overview of the current debates and conclusions within regional educational development and school improvement internationally were provided, including a consideration of the relationships between regions, education, school improvements, and success factors. The authors identified applicable articles and verified their inclusion in the search results in order to ensure the sensitivity of the search strategy. References from included studies were also assessed.

205

Structured around two levels of analysis, a summary and discussion of the outcomes of the review are presented. This involved a general overview concerning the distribution; research foci; and associated theoretical framework, methods, and design of the articles reviewed; followed by a thematic analysis of regional educational development research in relation to school improvement from an international perspective. The sources were reviewed and evaluated concerning content, scientific quality and relevance. Data processing, quality, sorting, categorization, and examination of the items were made using assessment documentation. Finally, the content was thematized. Results and Analysis This section outlines the outcomes of the review. It is structured around two levels of analysis: distribution of themes, years, methodological approaches, assessed relevance, and the sources and languages of the articles reviewed, and was followed by a thematic analysis and discussion of the data in relation to the nature of an impact on regional educational development and school improvement. Only 48 articles remained for the analysis. The reason for this may be that regional developments in mainstream school do not have a specific focus worldwide. Distribution Figure 1, above, shows the 48 sources that were included in the review and Table 1, below, specifies the themes. There were six themes: Political aspects, Social Capital & Networks, Higher education, General Education, Values, Languages & Cultures, and one named ―Others.‖ The sources were categorized as one entity. Table 1. Distribution of themes among the sources.

Themes Politics Networks & Social Capital Higher Education General Education Values Languages & Cultural Others

N = 48 14 8

Percent 29,2% 16,7%

7 6 5 4

14.6% 12.5% 10,4% 8.3%

8.3%

Two dominant themes are political aspects and social capital and networks (almost half of the content). The theme ―Others‖ are technical solutions, women‘s roles and dichotomization, major universities, and small colleges. The majority of the sources were published from 2001 and forward, which were a majority of the articles (see Table 2). In other words, interest in the subject has grown rapidly in the last fifteen years.

206

Table 2. Distribution of 48 sources reviewed by year.

Years 1980–1985 1986–1990 1991–1995 1996–2000 2001–2005 2006–2010 2011–

Sources N= 48 1 0 2 4 10 20 11

Percent 2.1% 4.2% 8.3% 20,8% 41,7% 22,9%

The geographic distribution shows that almost half of all the sources are published in European countries, and approximately one-fourth come from the US/Canada (see Table 3). This means that the research has been conducted in Western countries. Table 3. Continents among the sources.

Continent Europe North America Asia Africa South America Australia International

Sources N= 48 25 11 5 3 2 1 1

Percent 52,1% 22,8% 10,4% 6.3% 4,2% 2,1% 2,1%

Thus, this area of research seems to be a typical western phenomenon. Concerning methodology approach, there is an almost total domination of qualitative approaches, 54.2% (Table 4) followed by quantitative studies (27.1%). The studies which do not clearly have a methodological approach usually have a descriptive, historical approach. Table 4: Distribution of methodological approach.

Methodological approach Qualitative Approaches Mixed Method Design Quantitative Approaches Not Reported/Unclear

Sources N= 48 26

Percent

8.3

27.1 %

14.4

54.2 %

207

The qualitative approaches are highly varied from case studies to focus group interviews and observations. A small percentage uses quantitative approaches or mixed methods. When it comes to data collecting strategies, most of the sources have one strategy, but some have 2 to 4 strategies. The strategies are many and various, from video-taping to self-assessment reports to content analyses. We have not been able to classify them, as they are too sprawling. There are also problems with being able to classify analysis levels. Thematic Analysis In this SLR, it is possible to observe a lack of empirical research involving the combination of regional educational development and school improvement, especially for mainstream schools (cf. Boström 2015). Research is published, but it focuses mostly on one aspect (school improvement) or the other (regional educational development) or each, but not so much the combination. Political aspects seem to be the most dominant theme with it described in almost onefourth of the contents. The content of policy moves at different levels, political control, labor market reforms, decentralization, tax reduction to compensate rural areas, and ideological debates about the central cities versus regions. However, there are no concrete descriptions or visions of how the different proposals could have repercussions on school improvement. The quote below will illustrate the general approach: Education today is characterized by two opposite tendencies: —a centripetal tendency … a centrifugal tendency … these tendencies map out the main vectors of the development of innovative activity in regional systems of education. (Larina, 2006, p. 31)

Another focus is Social Capital & Networks, preferably in combination with the development of a ―creative class‖ and a diversity that can develop business ideas and promote entrepreneurial learning. The importance of interaction between people trained in creative professions so that they can create new products/services and generate jobs, new technology, and tolerance for the regions as described by Florida et al. (2010): ―Canadian regional development is shaped by the 3Ts of technology, talent, and tolerance. Talent in the form of human capital and the creative class is strongly associated with regional income‖ (p. 31). The power of building networks within the regions is highlighted as an opportunity to recruit students of different origins and to raise and support multi-cultural awareness and social support. Thus, capacity, knowledge, new ideas, and norms will be created (Cheong, Wing, & Alan, 2009). Through building networks within the region, there are opportunities to recruit students of different origins, and to raise and support multi-cultural awareness and social support. Thus, capacity, knowledge, new ideas, and norms that are created (Cheong, Wing & Yen, 2010) and socio-economic cohesion may be facilitated (Pachura, 2010). Cluster strategies can also act as a bridge between the political rhetoric and the observed reality (Doloreux & Shearmur, 2006). Higher education is also mentioned as a cornerstone in the study object, but more in connection with the importance of new research. General education is also mentioned as important, and more accurately, the quality of schools (Ava© 2015 The author and IJLTER.ORG. All rights reserved.

208

los, 1996) and opportunities available through curricula (Rahimia et al., 2010). Values are about the mental images that prevail and who prevail. Dichotomization of urban and rural standards is evident above all in Swedish Research (Svensson 2006; 2008; 2014). Cultural and linguistic aspects in certain regions are language-related changes that can enhance or hinder regional development (Daoud, 2010), and attitudes, as well as behaviors that may influence or be influenced by culture (Karahanna, Evaristo, & Srite, 2005), such as settings for the native language in school (Huguet & Llurda, 2001).

Conclusions In summary, what appear to be common worldwide is the concerns pertaining to urbanization, depopulation of rural areas; the need to highlight regional development factors; as well as a belief in education. One result of this study is that there are surprisingly few international studies on the relationship between regional development and school improvement. Approximately 70% of all the sources are published in Western countries, Europe and the USA/Canada. This does not mean that the problems are greater there than in the rest of the world. Perhaps instead, there are more research resources in these countries compared to the rest of the world. The literature review shows that the study object was observed more and more during the millennium. This may indicate that it will be noticed increasingly more henceforth. More than half of all resources have a qualitative approach, which means that the texts are descriptive. Needed are the long-term, consistent empirical studies—preferably interventions studies—with a mixed methods design. Different regions, countries, and continents show various problems within this focus. For example, regional educational development does not mean the same thing in West Africa compared to northern Canada. Countries and continents have different economic, structural, and cultural conditions and orientations. Education in this context typically refers to higher education, while factors that involve mainstream schools are not illustrated to any greater extent. The literature review shows an internationally sprawling view of the content and object of this study. It involves very different aspects, such as: 



 

Political issues, such as policy (Mc Dade & Spring,2005), ideological (Cf Laurina, 2006; Quiang, 2011) or economic governance, the effects of decentralization (Toi, 2010) and its repercussions on schools and student learning and ideological solutions; Social capitals and network structures (Pachura, 2010), such as knowledgebased c1usters as a guideline for regional development policies aimed at stimulating regional industrial competitiveness and innovativeness, with a focus on the importance to create eeducational achievement and creative occupations (Florida et al,, 2010) and emphasize that creative professionals are strongly related to regional income. Educational focus, mainly the connections between regions and universities. Cultural and linguistic aspects and values within society and school systems. Particularly interesting for the project V-brus is dichotomization between urban and rural values (Svensson 2006; 2010; 2013)

209

Regarding the Swedish research in relation to international research, there are three similarities, but otherwise they have pretty big differences. The emphasis in education connected to regional development is on higher education and not so much concerned with mainstream school. There are concerns about urbanization and beliefs in education both worldwide and in Sweden. The Swedish review shows examples of regional initiatives. The differences are that there is much more emphasis on different cultural and political aspects from an international perspective. The international literature is the national contingent, the Swedish is more regionally-focused, and the international review shows the importance of network clusters and the ability to inspire innovation through a ―new‖ creative class. If this new creative class should be created, developed, and have influence in the regions, a political strong will is required for school improvement in combination with regional development and research on a broad front. But from another perspective one might also ask about the logics and rurality and political rhetoric about the Swedish North (cf Nilsson & Lundgren, 2015).

References Ahuvia, A. (2008). Traditional, interpretative and reception based content analyses: Improving the ability of content analysis to address issues of pragmatic and theoretical concern. In R. Franzosi (ed.), Content Analysis: Vol. 1. (pp. 183–202). London: SAGE. Ambos, B., & Schlegelmilch, S. (YEAR). The new role of regional management. Vienna: Palgrave Macmillian. Avalos, B. (1996). Education for global/regional competitiveness: Chilean policies and reform in secondary education. Compare, 26(2), 217–32. Berglund, K., & Johansson, A. (2007). Entrepreneurship, discourses and conscientization in processes of regional development. Entrepreneurship & Regional Development: An International Journal, 19(6), 499–525. doi:10.1080/08985620701671833 Björkman, C. (2008). Internal capacities for school improvement: principals’ views in Swedish secondary schools (Doctoral dissertation). Umeå University, Umeå. Boström, L. (2015). Regional educational development research in Sweden: A literature review of research over half a century (Unpublished paper, in progress). Mid Sweden University Cheong, C., Wing, N., & Yen, C. (2011). Development of a regional education hub: the case of Hong Kong. International Journal of Educational Management, 25(5), 474 – 493. doi:10.1108/09513541111146378 Cooper, H. (1998). Synthesizing research: A guide for literature reviews. Thousand Oaks, CA: SAGE. Crescneszi, R. (2005). Innovation and regional growth in enlarged Europe: The role of local innovative capabilities, peripherality, and education. Growth and Change, 36(4), 471–507. Dalin, P. (1994). Skolutveckling. teori. Bok 2. [School development theory. Book 2]. Stockholm: Liber Utbildning AB. Daoud, M. (2006) The Language Situation in Tunisia. Current Issues in Language Planning, 2(1), 1–52. doi:10.1080/14664200108668018 Doloreux, D., & Shearnur, R. (2006). Regional development in sparsely populated areas: The case of Quebec‘s missing maritime cluster. Canadian Journal of Regional Science, 29(2), 195–220.

210

Evans, C., & Waring, M. (2012). Applications of styles in educational instruction and assessment. In L. F. Zhang, R. J. Sternberg, & S. Rayner (Eds.), The handbook of intellectual styles (pp. 297–330). New York: Springer. Florida, R., Mellander, C., & Stolarick, K. (2010). Talent, technology and tolerance in Canadian regional development. The Canadian Geographer, 54(3), 277–304. Franzosi, R. (2008). Content analysis: Objective, systematic, and quantitative description of content. In R. Franzosi (Ed.), Content analysis, Vol. 1 (pp. xxi–xli). London: SAGE. From, J., & Olofsson, A. (2013). Kunskapsekonomi och regional utveckling. [Knowledge economy and regional development]. In Y. Fiedrichs, M. Gawell, M. & J. Wincent (Eds.), Samhällsentreprenörskap för lokal utveckling (pp. 17–42). Sundsvall: Mittuniversitetet. Goodland, J.(1994). The national network for educational renewal. Phi Delta Kappan, 75(8), 35–42. Hargreaves, A., & Fink, D. (2008). Distributed leadership: democracy or delivery? Journal of Educational Administration, 46(2), 229–240. doi:10.1108/09578230810863280 Hernandez, F., & Goodson, I. F. (2010). Social geographies of educational changes: Drawing a map for curious and dissatisfied travellers. In F. Hernandez & I.F. Goodson (Eds.), Social and Geographies of Educational Changes (pp. xi–xxi). Dordrecht: Kluwer Academic Publisher. Höög, J., & Johansson, O. (2014). Framgångsrika skolor: mer om struktur, kultur, ledarskap. [Successful schools: more about the structure, culture, leadership]. Lund: Studentlitteratur. Huguet, A., & Llurda, E. (2001). Language attitudes of school children in two Catalan/Spanish bilingual communities. International Journal of Bilingual Education and Bilingualism, 4(4), 267–282. doi:10.1080/13670050108667732 Karahanna, E., Evaristo R., & Srite, M. (2005). Levels of culture and individual behavior: An integrative perspective. MIS Quarterly . Vol.30(3), pp.679-704 Larina, V. P. (2006). The innovative activity of schools in a regional system of education. Russian Education and Society, 48(3), 31–42. Lemar, S. (2006). Developing collaboration between schools and labour market in vocational education training—A fruitful or futile ambition? conference presentation. Luo, W., & Chen, S. (2010). Fiscal decentralization and public education provision in China. Canadian Social Science. 6(4), 28–41. Marmolejo, F., &. Pukka, J. (2006). Supporting the contribution of higher education to regional development: Lesson learned from an OECD review of 14 regions through 12 countries. Paper presented at the UNESCO Forum on Higher Education, [Location]. McDade, B., & Spring, A. (2005). The ‗new generation of African entrepreneurs‘: Networking to change the climate for business and private sector-led development. Entrepreneurship & Regional Development, 17, 17–42. McFarlane, C. (2010). The comparative city: Knowledge, learning, urbanism. International Journal of Urban and Regional Research, 34(4), 725–42 doi:10.1111/j.1468-2427.2010.00917.x Mittuniversitetet (2012). Projektplan Världens bästa regionala utbildningssystem. Nilsson , B. & Lundgren, A.S. (2015). Logics of rurality: Political rhetoric about the Swedish North. Journal of Rural Studies, 37, pp85-95. Pachura, P. (2010). Regional cohesion: Effectiveness of network structures. Berlin: Springer. Persson, L. O., Sätre Åhlander, A. M., & Westlund, H. (2003). Rural communities facing challenges and conflicts. In L. O. Persson, A. M. Sätre Åhlander, & H. Westlund (Eds.), Local responses to global changes (pp. 1–9). Stockholm: National Institute for Working Life. © 2015 The author and IJLTER.ORG. All rights reserved.

211

Polit, D., & Beck, C. (2008). Nursing research: Generating and assessing. Philadelphia: Lippincott, Williams & Wilkins. Rahimia, A., Ahmad, S., Borujenib, M., Esfahanic, A., & Liaghatdard, H. (2010). Curriculum mapping: a strategy for effective participation of faculty members in curriculum development. Procedia - Social and Behavioral Sciences 9, 2069–2073. Santema, M. (1997). Regional Development and the Tasks of vocational Education and Training Professionals. Journal of European Industrial, Training. 21(6), 229–237. Shi, L. & Wei-qing, C. (2010). Fiscal decentralization and public education. Canadian Social Science, 6(4), 28–41. Skolverket. (2014). SALSA/SIRIS. Retrieved from http://www.skolverket.se/statistikoch-utvardering/statistik-i-databaser. 2014-02-02 Statistiska Centralbyrån (2014). Utflyttande högskoleutbildade från Jämtlands och Västernorrlands län. [Out-migration of college-educated in Jämtlands and Västernorrlands counties compared to the whole of Sweden]. Retrieved from http://www.scb.se/sv_/Hitta-statistik/, 2014-02-02 Svensson, L. (2006). Vinna och försvinna? Drivkrafter bakom ungdomars utflyttning från mindre orter (Doctoral dissertation). Linköpings universitet, [Location]. Svensson, L. (2013). We don‘t want you to join us if you don‘t leave us! Economic and Social Aspects of Peripheral Regions Special Issue, 4(99), 46–52. Svensson, L. (2014). Nu styr vi upp stan. In M. Vallström (Ed.), När verkligheten inte stämmer med kartan. Lokala förutsättningar för hållbar utveckling. Lund: Nordic Academic Press. Toi, A. (2010). An empirical study of the effects of decentralization (From, 2014) in Indonesian junior secondary education. Educational Research for Policy and Practice, 52(3), 107–125. Tomaney, J., & Wray, F. (2011). The university and the region: An Australian perspective. International Journal of Urban and Regional Research, 35(5), 913–31. doi:10.1111/j.1468-2427.2011.01020.x van Velzen, W., Miles, M. B., Ekholm, M., Hameyer, U., & Robin, D. (1985). Making school improvement work. A conceptual guide to practice. Leuwen Acco: OECDpublication. Veugelers, W., &. Ziljstra, H. (2010). Networks of schools and constructing citizenship in secondary education. . In F. Hernandez & I.F. Goodson (Eds.), Social Geographies of Educational Changes (pp. 65–78). Dordrecht: Kluwer Academic Publishers. Westlund, H. (2004). Regionala effekter av högre utbildning, högskolor och universitet: en kunskapsöversikt. [Regional effects of higher education, colleges and universities: a systematic review]. Östersund: Institutet för tillväxtpolitiska studier; A2004:002 Whittemore, R., & Knafl, K. (2005). The integrative review: updated methodology. Journal of Advanced Nursing, 52(5), 546–553.

212

International Journal of Learning, Teaching and Educational Research Vol. 11, No. 1, pp. 212-226, April 2015

The Value-Added Assessment of Higher Education learning: The case of Nagoya University of Commerce and Business in Japan Hiroshi Ito Surname and Nobuo Kawazoe Nagoya University of Commerce and Business Nisshin, Japan

Abstract. Assessment of higher education learning has been considered increasingly important. One of the current trends in this field is the value-added assessmenthow much students learn during a certain period of time at university. In the United States, for example, Arum and Roksa (2011) conducted a large-scale assessment on second-year university students’ learning with the Collegiate Learning Assessment (CLA) to examine how much university students improved generic skills during the first two years of higher education. Findings suggested that they did not improve much. The researchers concluded that the poor result was attributed to the fact that American university students on average study only 12 hours a week. In Japan, the situation may be even worse as Japanese university students on average study 3.5 hours, much less than their cohorts in the United States. However, studies on the value-added assessment of Japanese university students’ learning are scarce. With the Progress Report on Generic Skills (PROG), an assessment tool similar to CLA, as well as interviews with students who took PROG, this study quantitatively examines how much students improved generic skills at a Japanese university during the first two years of higher education. The findings show that as was the case of the US peers, Japanese university students in this study did not improve their generic skills very much in the first two years of higher education. This study also qualitatively explores possible reasons for such results. The findings also show that the researched students on average studied only 40 minutes a week. This study suggests offering more courses with active learning approaches to intrinsically motivate students in order for them to spare more time for learning. Keywords: higher education; learning assessment; PROG

213

Introduction Assessment of higher education learning has been considered increasingly important (Clouder et al., 2012; Kushimoto, 2010; Sambell et al., 2012). Yet, measuring how much students learn has been a challenge for stakeholders involved in higher education assessment (Hardison and Vilamovska, 2009). While the grade point average (GPA) has traditionally been used for measuring students’ academic performance at university, it is not considered a reliable indicator for learning because grading varies according to institutes, instructors, and other factors (Shavelson 2009). For instance, “As” in one institute and in another institute (or from one instructor and another instructor) may not have the same academic value. The implication of GPA from a certain year to another is also difficult to interpret, as they do not measure the same components. Even if a student’s GPA was “A” in the first year and “B” in the second year, it is infeasible to determine that his or her learning deteriorated because the level of academic content may differ in the first year and the second year. In this context, various kinds of assessment tools have been created to measure university students’ learning. This paper will review these assessment tools and then quantitatively and qualitatively analyze how much students improved generic skills at a Japanese university during the first two years of higher education.

Literature Review Learning assessment tools In the 1980s, tests such as the College Outcome Measures Program (COMP), the Academic Profile, and the Collegiate Assessment of Academic Proficiency were developed in order to measure the outcomes of general education programs. Especially, COMP drew attention from stakeholders in higher education assessment because it evaluated value-added learning: how much students learned during a certain period of time. COMP has two different forms: the Objective Test (multiple-choice questions) and the Composite Examination (multiple-choice questions, essays, and speeches). The Objective Test comprises of multiple-choice questions and the Composite Examination comprises of multiple-choice questions, essays, and speeches. Despite its popularity, COMP has been criticized for its inability to measure communication skills as well as critical thinking skills. In the late 1980s, the College Basic Academic Subjects Examination (College BASE) was introduced. Apart from the subject content areas (i.e., English, mathematics, science, social studies), the College BASE assesses generic skills: interpretive reasoning, strategic reasoning, and adaptive reasoning. The College BASE has three forms: 1) the long form with content areas, 2) the short form with English and mathematics, and 3) an institutional-matrix form. As is the case with © 2015 The authors and IJLTER.ORG. All rights reserved.

214

COMP, however, a study revealed that the test assesses only a fraction of generic skills (Pike, 2011). It is in this context that CLA has emerged as one of the most popular assessment tools of generic skills in higher education in the United States (Klein et al., 2007). Other parts of the world now recognize CLA because the Organization for Economic Cooperation and Development (OECD) has been developing Assessment of Higher Education Learning Outcome (AHELO) based on CLA (Douglas et al., 2012). CLA is an open-ended, value-added, performance assessment tool that measures generic skills such as critical thinking, analytical reasoning, problem solving, and written communication through writing tasks, make-an-argument tasks, critique-an-argument tasks, and realistic performance tasks (Council for Aid to Education, 2013). In the United States, Arum and Roksa (2011) examined the CLA scores of 2,322 students at 24 universities over two years between the beginning of their first year in 2005 and the end of their second year in 2007. The study indicates that undergraduate students on average improved generic skills by 7%. While there are no universal standards for learning in higher education, they argue, students’ gains in academic performance were low. They concluded that the poor result was attributed to the fact that the US college students on average study only 12 hours a week.

Studying hours of university students in the US and Japan According to the National Survey of Student Engagement (2011), even 20 hours of studying time per week is not sufficient to fully prepare students for class. Apparently, both instructors and students have “If you don't bother me, I won’t bother you” attitudes. That is, Kuh (2003) explains, instructors do not make students work hard so that they do not have to grade many papers or exams. Peters (2011) also argues that university instructors are responsible for students’ low study habits. While students make as little effort as possible for their course work, university instructors tend to minimize course preparation time. The situation, however, may be even worse in Japanese higher education as Japanese university students study much less than their peers in the United States. As shown in Table 1, nearly 70% of Japanese university students study less than five hours a week.

215

0 hours 1-5 hours 6-10 hours 11-15 hours 16-20 hours 21-25 hours 26- hours

Japanese university students 9.7% 57.1% 18.4% 7.3% 3.2% 1.9% 2.4%

US university students 0.3% 15.3% 26% 22.3% 16.8% 9.8% 9.5%

Table 1. Studying hours per week University Management and Policy Research Center/National Survey of Student Engagement (Tsuji, 2013)

Tsuji (2013) estimates that the average number of hours spent studying by Japanese university students is only 3.5 hours. He explains that while Japanese university students are no less intelligent than students in other nations, their analytical reasoning and problem-solving skills are not well developed due to their lack of studying time. MacVeigh (2002) echoes Tsuji, stating that many Japanese university students are unable to think and write critically and logically and describes Japanese higher education as “a nationwide educational failure” (p.4). Tsuji (2013) argues that there exists what he calls “(p. 77) spiral” between Japanese industry and higher education: companies’ human resources personnel believe that university students do not study and they dedicate themselves to their part time jobs and/or circles/clubs, but companies’ human resources personnel want students to study to acquire generic skills that lead to employability. University students claim that human resources personnel do not consider GPAs for job applications. Students thus spend more time on part time jobs and circles/clubs because they believe that it is more important for their future employability. Instructors are afraid that if they made students study hard, students would evaluate them poorly in the course evaluations. It is therefore better for instructors to give “whatever” lectures without sufficient preparation and spend more time on their research. Students then complain that instructors do not teach them well and spend even more time on part time jobs and circles/clubs instead of studying.

216

Human Resources Personnel: we want university students to study and acquire generic skills for employability ! Students: Human Resources Personnel don't consider GPA for job application. What is the point of studying hard?

" Students: Instructors don't teach us well. The classes are boring.

$ Instructors: If we make students study hard, they will be harsh on us in the course evaluation. We give so-so

Figure 1. Negative Spiral (Tsuji, 2013)

In 2011, Tsuji (2013), as part of his NPO project, conducted surveys with 2,000 senior students in 28 departments at nine prestigious Japanese universities such as Waseda, Keio, Hitotsubashi, and Sophia. The student participants reported that only four out of approximate forty courses that they took at university helped them learn to think. A student, for example, reported that “Instructors didn’t ask us any questions. At the end of the semester, we were only given a one-page report for evaluation.” Another student claimed that “Professors just read textbooks in front of us.” Tsuji’s study indicates that Japanese university students are not in an environment where they are encouraged to study and develop their generic and employability skills. However, how can we know how much students learn at university in Japan? One answer is to employ learning assessment tools such as CLA and PROG.

PROG This study employs PROG in an attempt to measure students learning. PROG examines two sets of generic skills: literacy and competency. This usage of the terms literacy and competency can be confusing as the elements of literacy and competency overlap. For instance, the Organization for Economic Cooperation and Development (OECD)’s definition of literacy—using tools interactively (e.g. language, technology)—is one of its Definition and Selection of Competencies (DeSeCo)’s key competencies. As Matsushita (2010) puts it, these tools include non-cognitive elements such as social and emotional elements that are part of competency (See Figure 2).

217

DeSeCo's Competencies

Literacy

PROG's Literacy and Competency

Competencies

Literacy

Competency

Figure 2. DeSeCo’s Competencies vs PROG’s Literacy and Competency Based on (Matsushita, 2012)

In Japan, however, literacy is separated from competency (Matsushita, 2010). Despite weaknesses in the terminology, this paper follows PROG’s usage of the terms literacy and competency but uses them in italics to differentiate them from OECD’s usage. In the PROG test, 45 minutes are allocated for the Literacy section and 40 minutes for the competency section. Literacy is composed of data collection, data analysis, problem solving, and conceptual thinking skills. Critical thinking skills, which are considered important generic skills, are partially integrated into the data analysis skills. PROG’s literacy assessment also involves a few short essays to measure written communication and other skills. These elements are similar to what CLA examines. Competency is composed of skills in general communication, collaboration, networking, leadership, negotiation, and stress management as well as problem solving that is also included in literacy. According to Kawaijuku and Riasec (2015), problem-solving skills in literacy are tested on whether students can solve problems logically while problem-solving skills in competency are tested to what extent students solve problems as young professionals do (as described later). There are some notable differences between CLA and PROG. CLA is composed of open-ended essays while PROG is based on a combination of short essays and multiple-choice questions. Also, CLA is designed to produce results at the institutional level such as school average scores while PROG is designed to produce results at the individual level, which is scores for each individual who takes the exam. PROG also provides feedback sheets after the test with suggestions of how to improve generic skills. The PROG score ranges from 1 to 7 for both literacy and competency. Score 4 is the level desired to be reached by the end of the first year of university. Students with this score are expected to be able to adequately understand and rephrase information from documents and graphs. Score 7 is the level desired to be reached by the time of graduation. Students with this score are expected to be © 2015 The authors and IJLTER.ORG. All rights reserved.

218

able to organize data and demonstrate information derived from the data in academic writing and graphs. Students at this level are able to establish arguments logically (Riasec, 2012). The following problem is an example of a question that intends to test conceptual thinking skills in PROG’s literacy section (Riasec, 2012). You are going to do a group presentation about globalization for a university seminar. Each group has 10 minutes, including questions and comments. Each group can decide the aspects of globalization on which to focus. You have 20 days until the presentation. Please make a plan, showing the process of how you would proceed with the preparation, taking the following elements into consideration: a. b. c. d. e. f. g. h.

Preparing a presentation Collecting information and selecting ideas Deciding a group theme Analyzing information Deciding the content Practicing and modification Reviewing the presentation Deciding the roles

Students are expected to draw a flowchart that illustrates what they would do and when they would do it (Riasec, 2012). PROG has a few short essay questions that are similar to CLA’s make-an-argument prompt. For instance, students are asked to read questions and answer in writing. The following is an example of such as question. A university student, who travelled to South Korea the other day, said that while young Koreans did not understand Japanese, those who were close to 80 years old whom s/he met were fluent in Japanese. Why do you think elderly Koreans are able to speak Japanese fluently? Briefly write down the reason(s).

While a student could write a creative story, for example, that the Koreans had lived in Japan in their youth and learned Japanese, they are expected to write a short essay based on his/her knowledge of Japanese colonial education in Korea between 1910 and 1945. How does PROG address competency? The following question is an example from the competency section (Riasec, 2012). You are a project leader of developing a new product at a company. One of your subordinates came up with an interesting idea about a new © 2015 The authors and IJLTER.ORG. All rights reserved.

219

product. However, it is difficult to develop the idea into a product. In order to do so, it is necessary to deal with X, a major company with a marketing network, and Y, a venture enterprise with innovative technology. However, your company has not dealt with them in the past. What would you do? Choose one of the following options. A. I would tell my subordinate that it is difficult to develop his idea into a product though his idea is interesting B. I would encourage my subordinate to think of a realistic method of developing his idea into the product C. I would tell my subordinate that we would keep the idea and wait for an opportunity to arise D. I would think of how to communicate with X and Y through acquaintances E. I would contact Y immediately to see if we could collaborate with each other to develop the idea into a product Arguably, there is no right or wrong answer for this question. How does PROG score competency then? Kawaijuku and Riasec (2013) explain that they administered the test to the young business leaders that are currently active in society and collected sample data. They then analyzed the patterns of this group’s answers to each question. That is, PROG attempts to measure students’ competency by comparing their answers with the young professional leaders’ answers and how similar students’ answers are to those of the young leaders. Similar answers to young leaders’ answers score higher in competency while dissimilar answers score lower. According to Kawaijuku and Riasec (2014), as of April 2014, more than 100,000 university students took PROG. The average scores of PROG are 3.89 in literacy and 3.22 in competency. Approximately 63% of the test takers were first year students, 13.3% second year students, 19.7% third year students, and 3% fourth year students.

Year First year students Second year students Third year students Fourth year students Total

Literacy 3.82 3.97 3.97 4.28 3.89

Competency 3.20 3.16 3.23 3.58 3.22

Table 2. PROG Scores

As Table 1 shows, students’ scores in literacy improve through the entire course of university; however, the results show that there exists no improvement from the second to third year and then a remarkable improvement from the third to the fourth year. In competency, scores deteriorated slightly from the second to third year but improved from the third to fourth year. Although the number of © 2015 The authors and IJLTER.ORG. All rights reserved.

220

test takers is smaller than in other years, fourth year students substantially improved in both literacy and competency. It may be important to note, however, that Japanese fourth year students hardly go to university due to job hunting. Their improvement in generic skills, therefore, may not be attributed to university education, as described in the next section. NUCB Education: Developing generic skills toward learning goals The current research examines the learning of the first two years of education at the Nagoya University of Commerce and Business (NUCB). This university has participated in a national project entitled Improving Higher Education for Industrial Needs. In the context of improving university students’ employability to meet industrial needs, NUCB started focusing on generic skills and thus employing PROG to assess students’ generic skills. Given that this study examines how much students improved generic skills in the first two years of NUCB education, it may be important to explain the first year experience (FYE) program called the Vision Planning Seminar (VPS) and the following year seminars at this university. The purpose of VPS is to help the first year students acquire generic skills and envision the professional careers based on the assumption that if they can envision their futures in the early stage of university life, they should be able to set goals and work toward acquiring skills necessary to achieve those goals. As shown in the Table 1, students are explicitly expected to acquire generic skills such as critical thinking, analytical reasoning, problem-solving and writing. Critical Thinking

Analytical Reasoning ë ì Developing Generic Skills í î

í Writing

Communication ê î Discussion Presentation

Problem Solving

Figure 3. Purpose of NUCB Education: Seminars and Generic Skills The table constructed based on the skills measured by CLA (Council for Aid to Education, 2013)

The generic skills acquired in VPS feed into the second year to fourth year seminars and for writing the bachelor’s thesis. NUCB has set eight learning goals (LGs) to be achieved before graduation. LGs are assessed through the following skills: 1. 2. 3. 4. 5.

Establishing a concrete, important, and feasible research theme Acquiring academic knowledge and applying it to solve problems Arguing convincing conclusions through proper processes Expressing arguments through organized structure in writing Creating and performing presentations

221

6. Communicating messages clearly and effectively 7. Utilizing information technology 8. Thinking and acting globally Students are expected to demonstrate the first four of these eight skills in their bachelor’s thesis and the last four in their second to fourth year seminars. These skills are generic skills as well. For instance, the skills to set a research topic or apply knowledge entail critical thinking, analytical reasoning, problem solving and writing. After all, generic skills overlap with basic research skills in many respects.

Methodology The current research examines the PROG scores of 45 NUCB students who took PROG tests twice, first in April or May 2013 and second in December 2014, and analyzes the difference between these scores. That is, this study focuses on how much NUCB students learn in the first two years of university. One of the limitations of this study is its low sample size, which may hinder the generalization of the result. Yet, this type of value-added assessment is still rare in Japan and serves as an exploratory study. Some may also point out that the research period of two years is insufficient for this type of longitudinal valueadded study. As Arum and Roksa (2011) affirm, however, “most of the gains in generic skills occur in the first two years of college…seniors do not spend much more time studying than freshmen” (p. 36-37). Although Kawaijuku and Riasec’s (2014) study shows that Japanese university students substantially improved generic skills in the fourth year, the majority of Japanese university students do not attend regular courses in their fourth year and thus their improvement in generics skills are more likely attributed to their own study for the Synthetic Personality Inventory (SPI), an aptitude test for the selection of personnel, or related exams to seek employment, not university course work. Thus, the first two years of university learning is a reasonable indicator for the overall learning at university. Signed Wilcoxon’s rank sum test was conducted to examine whether there was a significant difference in the medians of the PROG scores between April/May 2013 and December 2014 (p<0.05) In selecting a statistical hypothesis test, we conducted the Lilliefors test based on the Kolmogorov-Smirnov test, which can determine whether sample data is normally distributed and found that p-value was less than 0.05 for all sample data, which shows that the sample data do not seem to be normally distributed. We thus selected Wilcoxon’s rank sum over matched sample t-test that examines the means of sample data difference. The analysis of PROG scores is supplemented by interviews with students who took PROG. The interview questions explore the following issues: how many hours a week students study how many hours a week students work and how © 2015 The authors and IJLTER.ORG. All rights reserved.

222

they have found courses at NUCB. The result of the current research might also be supplemented in the future by another longitudinal research of the same students at the end of their fourth year.

Results L1 C1 L2 C2

N 45 45 45 45

Means 3.09 2.91 3.16 3.20

SD 1.427 1.379 2.225 1.375

Min. 1 1 1 1

Max. 6 6 7 6

Table 3. Signed Wilcoxon’s rank sum test: Descriptive statistics *L1: Literacy score in the first year; L2: Literacy score in the second year *C1: Competency score in the first year; C2: Competency score in the second year

While students’ PROG scores improved by 0.07 (2.27%) in literacy and improved by 0.29 (9.97%) in competency, the difference of the medians between L1 and L2 was not statistically significant at any critical value (p-value=0.394). The difference of the medians of the medians between C1 and C2 was not statistically significant at p<0.05 but significant at p<0.10 (p-value=0.057). Given that Arum and Roksa (2011) consider 7% improvement in CLA for two years “not much” in the US context, the PROG results of Japanese students can also be interpreted as “not much.”

L2-L1 Ranks

Negative Positive

Ranks Total C2-C1 Ranks Ranks Total

Ties Negative Positive

N 17a 17b 11c 45

Mean Rank 16.59 18.41

Sum of Ranks 282.00 313.00

13d 20e 12f 45

15.23 18.15

198.00 363.00

Ties

Table 4. Signed Wilcoxon’s rank sum test: Rank a. L2<L1, b. L2>L1, c. L2=L1, d. C2<C1, e. C2>C1, f. C2=C1 In literacy, 17 students scored worse in the second test than in the first test while 17 students improved their scores. In competency, 20 students improved their score while 12 worsened it. Overall, NUCB students little improved generic skills for the first two years of higher education, though the results are not statistically significant and further studies at a larger scale are required to generalize the results. © 2015 The authors and IJLTER.ORG. All rights reserved.

223

With regard to interview results, while on average students spent 13.5 hours working, they spent only 40 minutes studying per week. The studying hours of NUCB students mark below the average studying hours of Japanese university students, which are 3.5 hours, as indicated in the literature review section. The maximum studying hours of NUCB students per week were three hours while one third of the students reported that they did not study at all. Three-quarters of students reported that they have encountered courses that they found interesting such as marketing, management and statistics. However, the number of courses that students found interesting is limited to a few out of many others. Some students claimed: “There is not much difference in content between high schools and universities,” “The university courses are boring,” “I always sleep in class,” “High school teachers teach better than university professors,” “One class has too many students,” and so forth. One student also lamented that she had never received any feedback from professors regarding her assignments.

Discussion This study indicates that in terms of PROG scores, NUCB students little improved generic skills during the first two years of higher education. Literacy did not improve at any significance level. While it is possible that NUCB students have made a statistically significant improvement in competency at p<0.10, though not at p<0.05, they might have acquired competency-related skills through interactions with others off campus (e.g., part-time jobs). Apart from descriptive interpretations of PROG results, there may also exist two possible reasons/interpretations of the result: 1) PROG may not measure generic skills that NUCB intends to develop (or NUCB may not develop generic skills that PROG measures) and 2) (some) students make different levels of effort during the first and the second PROG tests. Regarding the issue of incompatibility of measuring and developing different generic skills between PROG and NUCB, PROG, for example, does not measure ethical standards or global perspectives (Akihiro Tanabe, executive operating officer of Riasec, personal communication, February 18, 2015), both of which are nowadays considered generic skills and are elements of NUCB mission components. At the same time, while NUCB has set LGs to improve generic skills and measure them through students’ self-evaluation and rubrics developed by NUCB’s Assurance of Learning Committee, the university has been unable to provide evidence that it has developed generic skills comprehensively and objectively, as PROG results partially indicate. With regard to a plausible chance that some students make different levels of effort at the first and the second PROG tests, given that PROG results do not reflect on their GPAs or anything else, students may not motivated to try their hardest

224

when taking the PROG test. At any event, however, the sampled NUCB students study little as the study shows. NUCB makes institutional efforts to increase students’ studying time by establishing the self-study extra (SSE) point system. This system means that if students hours studying in the library, they earn academic benefits. If they spend 100 hours studying, for example, they can improve their grades by one level in five subjects, for example, from B to A. NUCB also sets strict standards for term-exams (one-fourth to one-third of test takers fail and must take reexams to pass the course). Yet, NUCB students do not study much, as the surveys suggest. Offering more courses that interest students is essential to intrinsically motivate them to learn. Employing active learning approaches may be recommended. As some scholars argue, active learning approaches help students learn to acquire generic skills (Barkley, 2010; Harper & Quaye, 2009; Ito, 2014c). A study examining an NUCB course employing active learning approaches also suggests that it has contributed to improving generic skills such as: agility, adaptability, and initiative; problem-solving skills; and curiosity and imagination (Ito, 2015). This paper does not suggest that NUCB should completely redesign its curriculum to focus on improving generic skills that PROG measures; nevertheless, the university needs to better demonstrate how generic skills, which it intends to develop as expressed in its mission statement and LGs, are nurtured. Making and using rubrics to measure these skills is one option. NUCB’s AOL has indeed developed rubrics to measure skills to achieve learning goals, though the committee struggles with implementation because rubrics can require substantial amount of time and effort to make and use. Administering a test that measures NUCB generic skills (e.g., global perspective) is another option. Riasec, the company that administers PROG, has flexibly dealt with measuring generic skills that PROG cannot measure. In order to measure students’ global perspectives, for example, the company has engaged 735 professionals (aged between 25 and 49), who worked for global companies and managed foreign subordinates, to take the competency portion of the exam. These data can be used to make comparisons with students to know how similar these students are with global professionals: in other words, how global their perspectives are (Kawaijuku & Riasec, 2014). Although this paper discusses the case of a particular Japanese university, the issue of developing and measuring generic skills are applicable to other contexts. Any university has to identify generic skills needed by their students and its institutional role in promoting generic skills. It then needs to develop or implement an effective set of tools to measure students’ generic skills.

225

Conclusion This study quantitatively examined how much students improved generic skills at a Japanese university during the first two years of higher education and qualitatively explored possible reasons for such results. The findings show that Japanese university students did not improve their generic skills by much during the first two years of higher education, arguably because students study little during this time. This study suggests that offering more courses with active learning approaches to intrinsically motivate students to spare more time for learning can contribute to improving generic skills.

Acknowledgement We are indebted to Mr. Hirotaka Nishio at NUCB for his support in this research. We are also grateful to Mr. Ezra Anton Greene at University of British Columbia for editing this paper.

References Arum, R. and Roksa, J. (2011). Academically adrift: Limited learning on college campuses. Chicago and London. Barkley, E. F. (2010). Student engagement technique: A handbook for college faculty. New York: Jossey-Bass Clouder, L., Broughan, C., Jewell, S. & Steventon, G. (2012). Improving Student Engagement and Development through Assessment. London and New York: Routledge. Douglas, J., Thomson, G. & Zhao, C. (2012). The Learning Outcomes Race: the Value of Self-Reported Gains in Large Research Universities. Higher Education 64(3): 317335. Hardison, C. M. and Vilamovska, A. (2009). The Collegiate Learning Assessment: Setting standards for performance at a college or university. Santa Monica: Rand Education. Ito, H. (2014a). Assessing an assessment tool of higher education: The case of PROG in Japan. International Journal of Evaluation and Research in Education 3(1): 1-10. Ito, H. (2014b). Whatâ&#x20AC;&#x2122;s wrong with learning for the Exam? An assessment-based approach for student engagement. Journal of Education and Learning 3(2): 145-152. Ito, H. (2014c). Shaping the First-Year Experience: Assessment of the Vision Planning Seminar at the Nagoya University of Commerce and Business. International Journal of Higher Education 3(3): 1-9. Kawaijuku and Riasec. (2013). PROG: Progress Report on Generic Skills. Tokyo: Kawaijuku and Riasec. Kuh, G. D. (2012). What we are learning about student engagement. Change 35: 28. Kushimoto, T. (2012). Outcome Assessment and its role in self-reviews of undergraduate education: in the context of Japanese Higher Education Reforms Since 1990s. Higher Education 59: 589-598. McIntyre, J., Todd, N., Huijser, H., and Tehan, G. (2012). Building pathways to academic success. A Practical Report. The International Journal of the First Year in Higher Education 3(1): 109-118. McVeigh, B. J. (2002). Japanese higher education as myth. New York and London: An East Gate Book. National Survey of Student Engagement. (2011). How much time college students spend studying varies by major and corresponds to faculty expectations, survey finds. ÂŠ 2015 The authors and IJLTER.ORG. All rights reserved.

226

Available at: http://nsse.iub.edu/NSSE_2011_Results/pdf/NSSE_2011_Press_Release.pdf. Peters, R. A. (2011). Enhancing academic achievement by identifying and minimizing the impediments to active learning. Public Administration Quarterly 35(4): 466-493. Pike, G. R. (2011). Assessing the Generic Outcomes of College: Selections from assessment measures. San Francisco: Jossey-Bass. Riasec. (2012). Measuring Generic Skills. Tokyo: Riasec. Sambell, K., McDowell, L. and Montgomery, C. (2012). Assessment for Learning in Higher Education. London and New York: Routledge. Shavelson, R. (2009). Measuring College Learning Responsibility: Accountability in a New Era. Palo Alto: Stanford University Press. Tsuji, T. (2013). Nagze nihon no daigakusei wa sekai de ichiban benkyo shinai no ka [Why Japanese university students study the least in the world]. Tokyo: Toyo keizai shinbunsya.