Vol 16 no 8 august 2017 by ijlter.org

IJLTER.ORG

p-ISSN: 1694-2493 e-ISSN: 1694-2116

International Journal of Learning, Teaching And Educational Research

Vol.16 No.8

PUBLISHER London Consulting Ltd District of Flacq Republic of Mauritius www.ijlter.org

International Journal of Learning, Teaching and Educational Research

The International Journal of Learning, Teaching and Educational Research is an open-access journal which has been established for the disChief Editor Dr. Antonio Silva Sprock, Universidad Central de semination of state-of-the-art knowledge in the Venezuela, Venezuela, Bolivarian Republic of field of education, learning and teaching. IJLTER welcomes research articles from academics, edEditorial Board ucators, teachers, trainers and other practitionProf. Cecilia Junio Sabio ers on all aspects of education to publish high Prof. Judith Serah K. Achoka quality peer-reviewed papers. Papers for publiProf. Mojeed Kolawole Akinsola Dr Jonathan Glazzard cation in the International Journal of Learning, Dr Marius Costel Esi Teaching and Educational Research are selected Dr Katarzyna Peoples through precise peer-review to ensure quality, Dr Christopher David Thompson originality, appropriateness, significance and Dr Arif Sikander readability. Authors are solicited to contribute Dr Jelena Zascerinska to this journal by submitting articles that illusDr Gabor Kiss trate research results, projects, original surveys Dr Trish Julie Rooney Dr Esteban Vázquez-Cano and case studies that describe significant adDr Barry Chametzky vances in the fields of education, training, eDr Giorgio Poletti learning, etc. Authors are invited to submit paDr Chi Man Tsui pers to this journal through the ONLINE submisDr Alexander Franco sion system. Submissions must be original and Dr Habil Beata Stachowiak should not have been published previously or Dr Afsaneh Sharif be under consideration for publication while Dr Ronel Callaghan Dr Haim Shaked being evaluated by IJLTER. Dr Edith Uzoma Umeh Dr Amel Thafer Alshehry Dr Gail Dianna Caruth Dr Menelaos Emmanouel Sarris Dr Anabelie Villa Valdez Dr Özcan Özyurt Assistant Professor Dr Selma Kara Associate Professor Dr Habila Elisha Zuya

VOLUME 16

NUMBER 8

August 2017

Table of Contents Learning from each other: Dialogical Argumentation in an Online Environment ........................................................ 1 Anita Chadha and RenĂŠe B. Van Vechten A Qualitative Examination of Factors for Success in a Content-Based English Language Learner Classroom ....... 18 Janet Delgado, Ed.D and Lorraine T. Benuto , Ph.D. Radio Wave Errors: Students Mistaking Radio Transverse Electromagnetic Light Waves as Longitudinal Sound Waves ..................................................................................................................................................................................... 37 A. E. Tabor-Morris, Ph.D., T. M. Briles, Ed.D. and R. Schiele, B.S. Impact of Teaching Attitudes and Behaviors for Learning on the Reading Achievement of Students Falling Behind .................................................................................................................................................................................... 51 Michael E. Bernard Balancing Reflection and Validity in Health Profession Studentsâ&#x20AC;&#x2DC; Self-Assessment ................................................... 65 Sherri Melrose Innovative Teaching with Use of an Art Work ................................................................................................................. 77 Marios Koutsoukos and Iosif Fragoulis

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 8, pp. 1-17, August 2017

Learning from each other: Dialogical Argumentation in an Online Environment Anita Chadha University of Houston Downtown, Houston, Texas, U.S. Renée B. Van Vechten University of Redlands Redlands, California, U.S. Abstract. This research builds upon past work exploring how an online academic website can provide a learning environment in which students engage in dialogic argumentation by voicing their diverse perspectives, challenging their peers through counterarguments, and articulating their positional differences. Drawing from two semesters of data from an academic website populated by three classes, we analyze 375 peer-topeer responses for their argumentative interactions. Using a mixed methods approach, we find statistically significant evidence that argumentative interactions lead to deeper engagement across the classes. This study concludes that online discussions—a form of computer mediated communication (CMC)—are an innovative means to advance e-learning, a concern for educators across disciplines. Keywords: Argumentation; Online deliberation; Online teaching; Dialogic argumentation; Computer-mediated communication

Introduction: Learning from each other: Dialogical Argumentation in an Online Environment Online learning environments now proliferate in our digital age and researchers have observed that in online, networked environments, learning can occur through an egalitarian process in which participants generate, challenge, reflect upon, and defend ideas, thereby constructing meaning through a social process (Rowntree, 1995; see also Chu et al., 2017; Cooper, 2001; Gordon & Connor, 2001; Wilson 2001). Also known as computer-mediated communication (CMC), webbased, interactive technologies are particularly well-suited to creative collaboration among active participants (Lee & McLoughlin, 2007). The CMC environment influences interaction due in part to visual anonymity and the absence of nonverbal cues. As Herring (1993) argues ―they provide for the possibility that individuals can participate on the same terms as others, that is, more or less anonymously, with the emphasis being on the content, rather than on the form of the message or the identity of the sender‖ (p. 1). With greater focus on the written message produced through asynchronous means, students

can participate in discussions on the same terms as others, without respect to geographic distances (Lane, 1994) or due to personal disabilities (Collins, 2014; Lane, 1994). Relative anonymity can also encourage users to be more expressive and form relationships with others rapidly (Schouten et al., 2009). Researchers have demonstrated that learning through CMC transpires through an individual process of critical reflection, a process of testing one’s ideas while being challenged, reconsidering one’s experiences and ideas in light of new information, and then reconciling differences. The argumentative process also involves synthesizing information and anticipating and responding to opposition, all of which are particularly conducive to learning (Jacoby, 2009; Blount, 2006; Bloom et al., 1956). In short, communicators learn through arguing with each other (Dehler & Porras-Hernandez, 1998), and dialoguing in ways that contains elements of argumentation also represents an opportunity to learn actively (Bender, 2003). As Socrates might have asserted, active argumentation channels learning. In this paper, we investigate online discussion forums created for and by undergraduate students enrolled in American Politics courses from three campuses, assessing their interactions for patterns of dialogical argumentation. In the experiment, students were given a weekly prompt about a contemporary issue in American politics, and participants created individual statements that, inevitably, reflected various levels of intellectual engagement with the material. From generalizations to fairly thoughtful and well-constructed essay-like answers that evidence deep, critical reflection, the content of those discussion posts provided the data for our study. First we identify various forms of interaction, and present a model for analyzing the content of those website discussion posts, testing whether students engage the learning process when they argue with each other in online discussion forums. Literature Review Argumentation, according to Toulmin (1958), is a process whereby an individual or group, wanting to be taken seriously, tries to convince the others that the assertions being made are acceptable, meritorious, or valid, and there is abundant evidence that this argumentative process has great worth as a learning tool (Clark & Sampson 2008; Schwarz & DeGroot, 2007; Clark & Sampson 2007). Through it, students’ understanding of challenging concepts can increase (Andriessen et al., 2013) and their ability to reason productively also can improve (Kuhn et al., 1997; Kuhn & Udell, 2003; Bell, 2004). Argumentation sets the scene for changes in people’s views because of the knowledge building and transformation of ideas that can occur through this process, leading to learning. This is a process of critical reasoning, and at its core is the idea of change in thinking. Change occurs because the arguer convinces, or a respondent critically reflects upon an idea and updates or refines an existing concept or belief. In any case, argumentation involves opposition, a process that some have characterized as occurring within a dialectic, whereby a position statement is made and justification is given, a counterargument is made and claims are questioned or examined, and a reply or rebuttal to the counterargument is supplied the dispute may ultimately be resolved into a conclusion (Toulmin 1958; Clark & Sampson 2007; 2008).

Arguments include supports such as warrants, backing, and qualifiers. In an online setting where students confront a discussion question and puzzle through its implications, the process generally parallels basic argumentative phases described by Toulmin (1958): online, they might might raise questions, challenge a premise, add new information, and/or anticipate responses. In essence, through dialoguing with each other via e-collaboration, students have the opportunity to engage in a process of argumentation that enables learning. In dialogical argumentation, these being arguments carried out through written or verbal dialogue, participants negotiate their divergences and reconstruct their perspectives in a social context. In other words, arguments by participants are sensitive, as Leitão (2000) argues, ―to specific demands of argumentative situations‖ (p. 336). In this way, ―macro‖ meets ―micro,‖ as macro-level factors influence what is essentially a micro-level process of decision making. Hakkinen (2013) points out this interactive relationship: ―these processes are intertwined…in a way that is not reducible to one level only‖ (p. 550). For example, a person might respond in a certain way because of how s/he has internalized shared norms about proper conduct, or the collective understanding about the purposes of an argument (Resnick et al., 1993). Likewise, personal attitudes such as openness to change or expectations about compromise (Coirier & Golder, 1993), as well as personal characteristics such as race, can influence how arguments unfold. Measurable change, therefore, takes place at the individual level as well as the social or group level. These changes might be of any magnitude; wholesale change is not required for an argument to be successful. As Leitão (2000) points out, in a discussion in which opposing views are justified and recognized, shifts in perspective occur across a continuum, ranging from subtle (qualifying a position) to complete reversal in stance. Much research on online argumentation draws on Toulmin’s initial work on argumentation (1958). For example, Clark & Sampson (2007) note that ―analytic frameworks focus on many different aspects of argumentation including argument structure, epistemic types of reasoning, conceptual normativity, quality of warrants, number of warrants, logical coherence of claims with warrants, argumentation sequences, patterns of participation, conceptual trajectories, and the process of consensus building which can be applied across disciplines.‖ (p. 275).1 They examine how students engage six major components of arguments: claims (assertions about what exists or what values people hold); data (facts or statements used as evidence to support the claim); warrants (statements that explain the relationship of the data to the claim); qualifiers (special conditions under which the claim holds true); backings (underlying assumptions); and rebuttals (exceptional conditions capable of defeating or rebutting the warranted conclusion. The context, combined with the type of project, often determine which components are necessary for a successful argument.

As Clark et al. point out that the pedagogical goal of an online project, class, or environment determine it use “for students to learn from argumentation (e.g., develop a more in-depth understanding of the content that is being discussed),” whereas the hierarchical analytic framework is better suited for analyzing online environments where students are learning “how to engage in argumentation (e.g., proposing, justifying, challenging ideas)” (2007: 352).

Several scholars argue that these components can be combined into more parsimonious models with fewer categories (Stegmann et al., 2012; Kollar et al., 2007; Means & Voss, 1996; Stegmann et al, 2007). Thus, the quality of each component depends on the validity and content of the argumentative claims, but how they are ultimately judged is discipline- or domain-specific. In order to test how students are in fact engaging in academic argumentation for the purposes of learning in online discussion forums, we turn to Clark & Sampson (2007) and (Erduran et al., 2004), who incorporate Toulmin’s framework to evaluate the presence, type, and quality of each element within online group dialogue. As Clark and Sampson (2007) explain, ―argumentative phrases are categorized based on their operational purpose: (a) opposing a claim, (b) elaborating on a claim, (c) reinforcing a claim with additional data and/or warrants, (d) advancing claims, and (e) adding qualifications‖ (p. 255). In our study, we combine and then organize these categories into progressively complex combinations in order to create a rubric by which to the judge the quality of an argument, whereby ―quality‖ refers to the structure rather than the normative content of the argument. This approach allows coded phrases to be aggregated and evaluated for their argumentative strength, and we adapt this method in the first part of our analysis. We also turn to scholars who have developed a variety of analytical approaches, tools, and frameworks for evaluating qualitative argumentative dialogues generated in pursuit of different educational goals in different subjects (physics, mathematics, linguistics, social sciences), and through various modalities (faceto-face, online chatting). These methods for analyzing online dialogues include qualitative, quantitative, and mixed methods approaches. Likewise, we use a mixed methods approach for our analysis, first by coding the discussion forums and then testing the content analysis quantitatively. These methods have been used successfully in past research (Chadha, 2017a; Chadha, 2017b; Chadha, 2017c; Van Vechten and Chadha 2013). Before we elaborate upon this model, however, we first describe the nature and source of our data: a website designed around discussion threads. The Collaborative Website Overview Data are drawn from a collaborative, cross-campus website project that involved students enrolled during the two spring semesters of 2012 and 2013 as shown in Table 1. In spring 2012, students from two campuses participated in the website, for a total of 79 students. In spring 2013, a total of 81 students from three campuses participated, including 21 from an upper level class, and 60 from two introductory American Politics courses on other campuses. Except for the upper level course, courses contained mostly freshmen and sophomores, and virtually all were unfamiliar with the use of online courses requiring argumentation.

Table 1 Courses and Participants

Spring 2012

Total # of peer-topeer responses

Spring 2013

Campus

Course Title

American Politics

Political Science Capstone

American Politics

Number of Students % of Course Grade

15%

13%

15%

13%

10%

N=375 entries

Methods The collaborative website was organized around asynchronous discussion forums that students developed through their online participation. Our research focused on the discussion forum entries recorded by the 160 students during the two spring semesters, and also questions that our students answered on pre- and post- surveys. Our approach included both qualitative and quantitative elements. First, we performed a content analysis of the 375 postings produced by the students, and then tested the data through linear regression. Comparability across classes. To minimize differences among courses, the professors agreed to three syllabi requirements that were distributed to all students. First, the students were required to respond to a minimum of eight instructor-posed questions and respond to their peers a minimum of eight times, for a total of 16 posted responses over the course of the semester. Second, they were required to use a minimum of 75 words in each response. Third, each professor assigned a grade for these activities that represented between 10 to 15 percent of the course grade for this collaborative activity. Participation was voluntary, and students could opt for an alternative assignment. During each semester a total of 14 weekly discussion questions were posed, covering variety of contemporary and enduring issues in American Politics. The number of responses varied with the type of question, whereby ―hot button,‖ controversial issues received the most attention. For this analysis, we selected discussion questions to represent a cross-section of the type of questions asked, as shown in Table 2. With the exception of laying the ground rules for civil discourse in the general guidelines that were distributed by each professor at the start of the semester, it should be noted that the professors did not intervene in the forums. Typically the students had one week to think about and post their replies. Data collection. Our data collection began with the selection of discussion question forums for analysis. In the past nine years of work in this area we have found that controversial civil rights subjects with a moral dimension often elicit the strongest responses and provoke the liveliest arguments; whether to site a Muslim mosque near Ground Zero in Manhattan or to allow a fundamentalist Christian group to protest against gay rights at a soldier’s funeral were two that

elicited heavy back-and-forth dialogue, for example. Questions that contain links to articles also seem to draw more thoughtful responses. Alternatively, when students are asked to consider slightly more abstract or theoretical issues, or are asked to supply a personal judgment to questions such as, ―What is presidential greatness?‖ they offer assertions but rarely engage in vigorous debate or challenge each other. Students seem more unwilling to challenge each other when opinions prevail over argumentative elements (most seem to assume a ―judge-not-lest-ye-be-judged‖ position). Peer-to-peer interaction is also biased in favor of agreement (Chadha, 2017b; Chadha, 2017c; Van Vechten & Chadha 2013; Van Vechten, 2013). We chose discussion question forums (DQs) from two semesters that would represent different types of queries, both controversial and theoretical, and include high numbers of posts. For the sake of comparison, we also included one question that was nearly identical in both semesters (gay marriage). The selected topics are included in Table 2 where the ―Responses‖ constitute peer-to-peer responses and the ―Posts‖ refers to responses to the discussion question. Table 2: Discussion Question Forums Selected for Analysis Spring 2012 and Spring 2013

Semester

Week Posted

Discussion Questions

Spring 2012, n=79

4 5 7 2

Relevance of a presidential candidate’s personal life Federal government support for colleges & universities Free speech and right to privacy Gay marriage Right to lie Gun control

4 10 11 12

Government’s role Regulating food Political representation Gay marriage

Spring 2013, n=81

TOTAL:

Peer Responses / Posts* 44/ 100 30/ 76 34/ 79 58/ 119 39/ 91 26/ 76 34/ 72 37/ 88 32/ 69 41/ 91 375/ 861

As Table 2 shows, there were a total of 375 responses and 861 posts during the spring semesters of 2012 and 2013. It is important to note that not every student is represented in a given forum; because students are required to respond to a question plus post a reply to another student, the total number of replies reflects about two posts per student. A typical discussion forum includes responses from roughly two-thirds of the website’s student population. To ensure consistency and reliability of interpretation, only one author coded the data. Operationalizing the variables with the framework. The analytic framework that Clark and Sampson (2007, 2008) developed to evaluate dialogic interactions in the hard sciences forms the basis for our analysis of argumentative quality, as shown in Table 3. We focus on the type of interaction, not content, to determine

―quality.‖ Clark and Sampson’s model depicts six levels that represent progressively more sophisticated forms of argumentation typically presented in the ―hard‖ sciences. In their model, each higher step represents higher-level reasoning that involves more intellectually demanding components of the process, such as providing backing for claims in rebuttals. At bottom is an absence of argumentation, and at the highest level are extended arguments that include at least one rebuttal. Our analysis focuses on interactive argumentative dialogue in the social sciences rather than on factual claim/counterclaim exchanges that typify hard sciences discourse. In our adaptation, we propose that the quality of the dialogue should be judged on the range, type, nature, and frequency of argumentative elements contained in peer-to-peer responses, as shown in Table 3. For our model we created a more parsimonious hierarchy of four types (instead of six), whereby each type represents progressively more sophisticated levels of argumentation as shown alongside the Clark & Sampson model. Coding each phrase within a posting for argumentative elements, or variables, within each online response allowed us to distinguish four levels of dialogical argumentation. It should be noted that a complete statement or posting could contain any number of these different elements. Table 3 Dialogical Argumentation Typology

Clark & Sampson (2007) Levels Characteristics Argumentation 5

 Our (2013) Model of Type Characteristics/elements of dialogical argumentation

Rebuttals and at least one rebuttal that challenges the grounds used to support a claim Rebuttals that challenge the thesis of a claim but does not include a rebuttal that challenges the grounds used to support a claim Claims or counterclaims with grounds but only a single rebuttal that challenges the claim Claims or counterclaims with grounds but no rebuttals Simple claim versus counter-claim with no grounds or rebuttals Non-oppositional

Rebuttals that Challenge and Dispute peers’ claims on the grounds used to support those claims, using warrants, claims and counterclaims Rebuttals that Correct and Clarify a Position with peers on the grounds used to support a claim, using qualifiers, claims, or counterclaims

Agreement/Disagreement With and/or Repetition of peer’s argument, but Adds to Argument by providing more information, such as facts or backing of claims; no grounds or rebuttals

Contains unsupported generalizations

In our model, Type 0 would include a response consisting mainly of unsupported generalizations: sweeping statements or opinions offered without any

supporting logic. Virtually no substantive or meaningful information was offered. Type 1 responses mostly contained echoes or repetition of a peer’s claims, but the argument was advanced minimally through the inclusion of a new perspective, angle, or information. We coded for whether they added new, non-normative information that expanded the discussion (as opposed to providing emotionally-charged, normative, ―should‖ directives or claims), entries that also might have taken the form of ―teaching‖ a new angle or offering a new perspective. We also looked at whether a student simply agreed with a peer, disagreed (a more challenging position), or did both in their responses. Type 2 responses encompassed clarifications, meaning that positions were clarified through qualifiers and/or counterclaims are rebutted. The arguer may have offered an analogy, considered new angles, sharpened the position, and so forth; in essence, the aim was to rebut a counterclaim by adding new information or adding qualifiers. In Type 2 responses the author might also have corrected a peer by adding new information, or pressed a peer to reconsider a claim. However, no direct challenges to opposing claims were offered. At the highest level of argumentation, Type 3, the arguer offered direct rebuttals or challenges to peers that included warrants or qualifiers intended to push deeper thinking about a point that was made. There was also an attempt to dispute or argue, by disputing a claim and questioning its validity or veracity. Each of these levels evidences progression of thought that promotes learning. Descriptive measures for these interactive components are presented in Table 4. Table 4 Elements of Dialogic Argumentation (Spring 2012 and Spring 2013) N= 375 Combin AGREED CORRECT ADD CLARIFY CHALDISPUTE -ations and/or INFO LENGE DISAGREE D

No Yes

39.5% 60.5%

22.4% 77.6%

47.7% 52.3 %

52.3% 47.7 %

23.5% 76.5 %

25.6% 74.4 %

Table 4 shows that over half (60.5%) of peer-to-peer responses contained direct engagement with a peer, which took the form of agreement and/or disagreement. Well over half of the posts (77.6%) included corrections, meaning that they provided factual information in an attempt to clear up a misconception. Another 52.3% added or provided additional information to support their responses, and 47.7% clarified their responses with specific information or by articulating a different perspective. A similarly high percentage (76.5%) challenged each other, and another 74.4% disputed (or directly argued) by supplying supporting evidence or reasoning for their claims. We were also interested in measuring whether students could use these different elements in combination, which would be a sign that students were more deeply engaged and on the path to actually learning through their interactions. In our view, generalized replies that required little thought, expressed emotional reactions, contained unsupported generalizations, and contributed nothing new to the discussion could be distinguished from those in which students were pushing themselves to consider new angles and reconsider their own issue stances.

To measure depth of academic engagement, we created an index based on five elements. First, we scored student entries for assertions that were reflective, deliberative or critical (reflective), the functional equivalent of claims, and/or qualifiers, and/or rebuttals. Second, we coded for whether the post included an honest question that created further deliberation among students (rather than a rhetorical one), such as when a student asked a peer to think about another aspect of an issue (honestq). Third, we looked ―backings to claims‖ that took shape in two forms: in references to authorities, such as an assigned text or the professor’s teachings (classtext); or in links or references to outside media or sources such as an article, video clip, or other online materials supporting any assertions the student is making (media). Fifth, we coded for length (short, medium, or long based on the number of words), as a proxy for effort to articulate an argument. Students who wrote virtual essays, for example, clearly achieved a different level of critical thought than those who merely offered an opinion that was expressed in a few lines. Our composite variable, ―depth of academic engagement‖ (or more simply, depth of engagement) represents a sum of the scores for these five elements. Therefore, a post that evidences deep engagement would incorporate all five elements: reflective + honestq + class text + media materials + length. These results are presented below. Research Questions and Hypotheses. We were interested in how seriously students engage with each other in online discussions, and whether they argued with each other and wrestled with the material in gently provocative ways that could change a person’s mind or produce a new position. More specifically, we wanted to know whether the computer-mediated communication process of dialogical argumentation could foster academic learning. Building on Clark and Sampson’s work (2007; 2008), we hypothesized (H1) that the most sophisticated levels of argumentation would be least common, in that students would challenge and dispute each other (Type 3 responses) less often than they would correct and clarify their positions to each other (Type 2), and that the majority of students would reach a basic level of engagement by agreeing and/or disagreeing with each other (Type 1). We also hypothesized (H2) that Type 0 responses would be less prevalent than Type 1 responses, given our clear guidelines about length of posts and our expectations that they would reflect on their answers before recording their thoughts. Thus, we expected the greatest number of posts to be Type 1, representing ―entry-level‖ engagement with the learning process. Furthermore, we hypothesized that students who ―dove deeply‖ into the process by incorporating links to other materials or producing lengthy posts would also be more likely to argue at higher levels of sophistication. Findings and Discussion Type of Arguments. Noting first that a student’s post could contain different argumentative elements of argument, we mined the responses for progressively more sophisticated combinations that would allow us to categorize them by type. We found that almost one-third (30.1%) of posts included the most advanced ―Type 3‖ combination of arguments: these incorporated challenges and disputations, and pushed ahead the discussion with new, engaging points or questions. A larger percentage (56.5%) included Type 2 combinations, which

encompassed corrections and clarifications. The largest percentage (68.5%) included Type 1 interactions where the students agreed and/or disagreed with each other, or added a new point or information. A much larger portion, 77.4%, contained unsupported generalizations, as reported in Table 5. Table 5 Dialogical Argumentation Types: Spring 2012 & Spring 2013 (N=375)

Type 3 Type 2 Type 1

Percent of Interactive Posts 30.1% 56.5% 68.5%

Type 0

77.4%

Combined Elements of Argumentation Challenge + Dispute Correct + Clarify Agreement/Disagreement + Offer Info Information + Unsupported generalizations

The results in Table 5 support our main hypothesis (H1), such that fewer than a third of all peer-to-peer responses contained the most sophisticated arguments, while non-oppositional statements of opinion were among the most common types found among the responses. More difficult arguments were indeed less common. Contrary to our expectations (H2), however, Type 0 responses were more common than Type 1, which provides some evidence that students were contributing to the discussions without investing much thought. At the lowest level of engagement, Type 0, students typically made sweeping claims or generalizations lodged in ―common wisdom,‖ yet remained civil. This example of a Type 0 response comes from a spring 2013 dialogue about the utility of banning sugary drinks and taxing fatty foods: “I do think it’s a nice thought however ultimately I just feel that people should just do a better job of taking care of themselves and be better role models for the youth. If you set good examples kids will look up to you and what you do.” In this example, the student backs her opinion by a broad generalization. This exmplifies Type 0 responses in which information relevant to the thread might be included, but unsupported generalizations render it unhelpful for advancing an argument from which students can learn, either through practice or the act of considering their peers’ arguments. In Type 1 interactions, students disagree at least mildly with their peers (often they combine disagreement with agreement), and they continue to advance an argument by offering a new perspective, angle, or information, even if only briefly. No direct challenges are made. In the following excerpt from a discussion forum about government regulation of food from the Spring 2013 semester, Student Y responds to Student X by not only repeating X’s claim, but also by supplying his own reasoning, which effectively adds a new point (childhood diseases) to the dialogue: Student X initial post: Bloomberg's attempt to ban the large sugary drink is a good idea because most of America is unhealthy and obese. Banning the large sugary drink is a good health decision. Now the question, Is it really a debate? No, this should not be a debate and the large sugary drink should not be banned. Banning the large drink will not stop people from drinking large amount of soda, it will only have them purchase two drinks instead of one which will equal to or

more to the size of one large drink. People should be allowed to purchase their own size of drink. If customers are interested in their health, restaurants have posters up on the wall which has the amount of calories on items sold Student Y’s response to Student X: I agree because these days we now have children with diabetes and obesity. The educational system has already attempted to help the obesity problem by offering healthier food options. Maybe allowing the government to help with the obesity problem will aid the obesity and diabetes issue that we have present. Here, Student Y adds to the discussion with this concise point about government’s responses to childhood obesity, but does so without challenging his peer directly and without providing data or qualifiers for the assertion about obesity. Type 1 responses thus contribute in some small way to the general argument, and over two-thirds, 68.5%, did so. Over half the interactions (56.5%) were of Type 2: a student would clarify his or her position, and/or rebut a counterclaim, sometimes correcting a peer by adding new information. Warrants and backings in the form of reasoning and examples were common, indicating that the author was engaging the learning process in a more rigorous way. Type 2 is exemplified by this thoughtful reply to another student’s post, which the author does not question: Even though i am proud of Bloomberg for trying to help make New York's citizen's healthier, i do not think potentially banning soda size is the main health problem. There are many reason why. For example, just banning the soda size alone will not stop people from drinking more soda. In fact, this will have the complete opposite reaction. Once people hear their soda size is being cut down, it will only make them want to buy more soda to make up for the loss in size, meaning they will buy more cans or bottles of sodas until they are satisfied. I agree with [Margo] that restaurants should make the public more aware of the ingredients rather than the calorie count of food items. If people know about what is in their food, then they will have a better idea of how to control what they eat, making healthier food choices. Let us take the fast food chain "Subway" for example. They give healthy food choices and make their customers aware of the calorie and sugar content which helps us all to make better food choices. When it comes to decisions about food, at the end of the day each person is responsible for their own choices and will have to bear the consequences or gain the benefits according to how they choose. Type 2 posts, therefore, include a correction or counterclaim (information that corrects a peer’s claim) and a clarification of one’s own position, usually through additional data, warrants, or qualifiers. We interpret this type of post as moderate engagement of the learning process. Type 3 responses include a direct challenge to a peer’s statement or premise, and key parts of a dispute are present as real dialogical argumentation unfolds. Again, 30.1% fell into this category, as they combined elements of argumentation (warrants, claims, counterclaims) that enabled the author to clarify, challenge, and argue thoughtfully. This kind of argument is demonstrated here: I don't think that you managed to capture the entirety of my argument. All you managed to do was call me bigoted; and you support your argument by saying that the beliefs of many don't mean anything (as in your case for Religion) when your opinion on gay marriage is at its foundation just a belief. The phrase of separate but equal was meant to be in respect to the Church and State. I'll admit

that it was a poor choice of words given the civil rights history of the U.S. I respect your point of view here, but you completely overlooked mine. We regard this type of response as paving the way for others to reconsider their views, including the target of the post, as well as other students who might read the exchange. Students at this level are fully engaged in argument, trying to convince others that his claims are meritorious and valid. Depth of Engagement. We also assessed depth of academic engagement quantitatively. First we created an index for depth of engagement by scoring the responses for the presence of five various elements (as described in the methods section): overall reflectiveness, asking honest questions, including references to the class or textbook, inserting links to outside media or materials, and length (scored one to three)2. A response containing none of these elements would be scored zero; a response reflecting deep engagement would incorporate all five elements. Actual scores ranged from zero to five, and most of the 375 responses clustered around the mid-range—what we might call ―moderately engaged,‖ as Table 6 shows. Table 6 Frequency and Percent of Academic Engagement Scores

Score Frequency Percent Cumulative Percent 0 <1 .03% .03% 1 43 11.47% 11.5% 2 145 38.7% 50.2% 3 98 26.1% 76.3% 4 66 17.6% 93.9% 5 23 6.1% 100% N 375 100% According to Table 6, about one out of ten responses demonstrated engagement at the lowest level; it’s unlikely that the author learned anything new or that peers gleaned meaningful information from these posts. Over twothirds (64.8%) were moderately engaged, having scored at least a two or a three on our scale. At levels four and five, students are now invested in the learning process, sharing materials and new ideas, prodding each other to question further, or providing links to interesting articles that could shed further light on the issue at hand. Almost a quarter (23.7%) appeared to be deeply engaged. Finally, we wanted to know if a student’s use of argumentative elements could predict how ―reflective‖ his or her response was. We scored each response for overall reflectiveness: did the student generally seem to be thoughtful, or was the response a knee-jerk, ―let’s-get-this-over-with‖ response? Using a dummy

Considered as a single variable (it is otherwise included in the “depth” score), length is another indicator of students’ engagement through discussions with each other leading to in dialogical argumentation. Students were required to post at least 75 words, and found that posts on average exceeded the minimum at a mean of 96 words, but with a rather large standard deviation (56 words). Viewed another way, in both semesters most students (73%) posted what we coded as “medium” length posts, meaning 50-150 words. At the lower end, 15% of all students posted far less than the required minimum (0-50 words), and the remainder (11.5%) far exceeded the minimum by posting at least double what was required (151+ words). 2

variable for ―reflective,‖ we analyzed the data through a series of simple linear regressions to estimate the relative weight of each element of argumentation. Interestingly, each term was found to be statistically significant (p<.001). A summary of results is displayed in Table 7. Table 7 Summary of Regression Analyses

Model

Unstandardized Coefficients B 1.603 -.226 .819 .214 .002 .716

(Constant) Correct Clarify my position Challenge views Dispute Adding information Agreeing and .194 disagreeing *p< .001.

Standardized Coefficients Std. Error .108 .130 .098 .159 .157 .108 .067

Beta -.085* .367* .082* .001* .321* .154*

This finding tells us that students who employ argumentative elements also tend to be more reflective in their answers; they invest more in their learning when they argue. Overall, the regression analysis confirms what we found through our content analysis, providing further evidence that dialogical argumentation occurs across three different types of argument—that is, Types 1 to 3. This is the strongest evidence that websites designed for academic purposes can produce virtual learning environments. Related Variables While we found statistically significant evidence that students engage each other in the process of argumentation, we continued to explore other questions that formed during this process. Would a question about gay marriage that was posed a year apart produce noticeable differences in argumentation? We found this not to be the case. The student populations from two semesters took similar approaches to answering questions, a conclusion confirmed by the ―depth of engagement‖ patterns, which were roughly parallel across two semesters as shown in Figure 1.

Gay Marriage (2013) Gay Marriage (2012) 0

Percent L-5

L-4

L-3

L-2

L-1

Figure 1: Depth of Engagement Scores for Discussion Questions about Gay Marriage In Spring 2012 and Spring 2013

We also explored whether the wording of the discussion question was correlated with the sophistication of argumentation. We found that question prompts that attract the most replies are current-events-oriented and contain links to outside materials such as news articles. Theoretical questions attract the fewest responses (even when they are required), as well as the most limited branching among students; there is plenty of room for students to think critically and post reflective statements, but they argue less with each other over definitions or applications of abstract concepts when these are the actual topic of discussion. In spring 2012 and spring 2013 the discussion questions that attracted the fewest replies were about government’s role (n=72) and the nature of political representation (n=69), compared to 119 and 91 replies about gay marriage. A final note concerns the way that students interacted asynchronously to create ―back-and-forth‖ dialogues. Most of the peer-to-peer responses (84%) involved one single reply rather than a sustained series of responses; 11.5% extended to two responses; 4.5% involved three or more responses. However, some of those exchanges involved several persons, and branching was common. The spring 2012 forum on gay marriage provides good examples of this. Almost half (47%) of the posts in this forum were actual interchanges between or among students. Similarly, a spring 2012 forum that asked students to weigh privacy against the government’s need to collect private information attracted 79 replies, 44% of which were ―branches‖ that included three or more people. Clearly students are engaging each other through this format, though just under half are participating in actual ―dialogues‖ involving more than two people.

Conclusions The purpose of our shared academic website was to provide a space for undergraduate students from different campuses to interact and to promote thoughtful discussion and learning through asynchronous discussion-based forums. We hoped our students would learn about the issues and their own positions through dialogical argumentation. This inquiry into the nature of online student dialogue uncovered statistically significant evidence that students did just that: they engaged the learning process through arguing with each other, asynchronously through discussion forums, in the spring 2012 and 2013 semesters. Concrete elements of argumentation were visible in students’

responses to each other, namely in the way that they countered each other’s claims, clarified their own positions, offered new perspectives and information, questioned each other, and challenged one other to account for their views. It should be noted that the process does not encompass all students equally; not everyone argued, and not every student was invested in the site. However, based on the totality of evidence, we conclude that the process was a valuable learning tool for those who did engage. Students are more likely to engage in activities when they feel their contribution is valued by others. As students reported in open-ended survey responses at the end of the semester, the iterative nature of the online exchanges tended to foster an online community (59% in spring 2012 agreed that the website made them ―feel as if they were part of a larger political community‖), which begins with following basic rules for civil discourse. The overwhelming majority (84%) also felt that the discussion on the site increased their interest in political issues and prompted them (84%) to find more information about these issues. Specifically designed educational portals such as ours can simultaneously promote engaged e-learning and a sense of community. Definitively, CMC is an effective means to engage students in meaningful academic exchanges, regardless of discipline. In using a digital portal designed to support interactive e-learning and by concentrating on students’ interaction, we have shown that argumentation involving students across geographic boundaries can lead to productive ―conversations‖ that prod students into thinking reflectively in an environment conditioned by academic instruction. It’s clear that online educational portals possess great potential to encourage critical thinking and learning. The ingredients for knowledge construction and cognitive development are threaded into discussion forums, and when enlivened through argumentation, learning can take place.

References Andriessen, J., Baker, M., & Suthers, D. (Eds.). (2013). Arguing to learn: Confronting cognitions in computer-supported collaborative learning environments. Springer Science & Business Media 1. 33-56. Bell, P. (2004). Promoting students’ argument construction and collaborative debate in the science classroom. Internet environments for science education, 3. 115-143. Bender, T. (2012). Discussion-based online teaching to enhance student learning: Theory, practice and assessment. Stylus Publishing, LLC. Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives, handbook I: The cognitive domain. Blount, A. G. (2006). Critical reflection for public life: How reflective practice helps students become politically engaged. Journal of Political Science Education, 2(3), 271-283. Chadha, A. (2017a). Learning to Learn: Lessons from a Collaboration. Journal of the Scholarship of Teaching and Learning, 17(3), 34-47. Chadha, A. (2017b). E-Learning Opens Doors to the Online Community: Lessons from a Longitudinal Study. National social science association, 45.

Chadha, A. (2017c). Comparing Student Reflectiveness in Online Discussion Forums across Modes of Instruction and Levels of Courses. Journal of educators online 14 (2), 1-19. Chu, H. C., Chen, J. M., & Tsai, C. L. (2017). Effects of an online formative peertutoring approach on students’ learning behaviors, performance and cognitive load in mathematics. Interactive Learning Environments, 25(2), 203–219. doi:10.1080/10494820.2016.1276085 Clark, D. B., & Sampson, V. D. (2007). Personally‐ seeded discussions to scaffold online argumentation. International Journal of Science Education, 29(3), 253277. Clark, D. B., & Sampson, V. D. (2008). Assessing dialogic argumentation in online environments to relate structure, grounds, and conceptual quality. Journal of Research in Science Teaching, 45: 293-321. Coirier, P., & Golder, C. (1993). Writing argumentative text: A developmental study of the acquisition of supporting structures. European journal of psychology of education, 8(2), 169-181. Collins, L. (2014). How online learning enables personalized learning. Getting Smart. Retrieved fromhttp://gettingsmart.com/2014/03/online-learningenables-personalized-learninginterview-dr-lisa-collins/ Cooper, J. (2001). Peer learning in law: using a group journal. In D. Boud., R. Cohen, & J. Sampson (Eds.), Peer learning in higher education: Learning from & with each other (pp. 115–127). London, UK: Stylus Publishing Inc. Dehler, C., & Porras-Hernandez, L. H. (1998). Using Computer Mediated Communication (CMC) to Promote Experiential Learning in Graduate Studies. Educational Technology, 38(3), 52-55. Erduran, S., Simon, S., & Osborne, J. (2004). Tapping into argumentation: Developments in the application of Toulmin's argument pattern for studying science discourse. Science education, 88(6), 915-933. Gordon, R. & Connor, R. (2011). Team based learning in management education. In D. Boud, R. Cohen. R., J. & Sampson (Eds.), Peer learning in higher education: Learning from & with each other (pp. 85–98). London, UK: Stylus Publishing Inc. Hakkinen, P. (2013). Multiphase method for analyzing online discussions. Journal of Computer Assisted Learning, 29(6), 547-555. Herring, Susan. (1993). Gender and democracy in computer-mediated communication. Electronic Journal of Communication, 3(2), 3–31. doi:10.1016/B978-0-12-415040-9.50121-4 Jacoby, B. (2009). Civic engagement in higher education. Concepts and practices. Jossey-Bass, San Francisco. Kollar, I., Fischer, F., & Slotta, J. D. (2007). Internal and external scripts in computer-supported collaborative inquiry learning. Learning and Instruction, 17(6), 708-721. Kuhn, D., Shaw, V., & Felton, M. (1997). Effects of dyadic interaction on argumentative reasoning. Cognition and instruction, 15(3), 287-315.

Kuhn, D., & Udell, W. (2003). The development of argument skills. Child development, 74(5), 1245-1260. Lane, D. R. (1994). Computer-mediated communication in the classroom: Asset or liability. In Workshop presented. Lee, M. J., & McLoughlin, C. (2007). Teaching and learning in the Web 2.0 era: Empowering students through learner-generated content. International journal of instructional technology and distance learning, 4(10), 21-34. Leitão, S. (2000). The potential of argument in knowledge building. Human development, 43(6), 332-360. Means, M. L., & Voss, J. F. (1996). Who reasons well? Two studies of informal reasoning among children of different grade, ability, and knowledge levels. Cognition and instruction, 14(2), 139-178. Resnick, L. B., Salmon, M., Zeitz, C. M., Wathen, S. H., & Holowchak, M. (1993). Reasoning in conversation. Cognition and instruction, 11(3-4), 347-364. Rowntree, D. (1995). Teaching and learning online: a correspondence education for the 21st century? British Journal of Educational Technology, 26(3), 205215. Schouten, A. P., Valkenburg, P. M., & Peter, J. (2009). An experimental test of processes underlying self-disclosure in computer-mediated communication. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 3(2). Schwarz, B. B., & De Groot, R. (2007). Argumentation in a changing world. International Journal of Computer-Supported Collaborative Learning, 2(2), 297313. Stegmann, K., Weinberger, A., & Fischer, F. (2007). Facilitating argumentative knowledge construction with computer-supported collaboration scripts. International journal of computer-supported collaborative learning, 2(4), 421447 Stegmann, K., Wecker, C., Weinberger, A., & Fischer, F. (2012). Collaborative argumentation and cognitive elaboration in a computer-supported collaborative learning environment. Instructional Science, 40(2), 297-323. Toulmin, S. (1958). The uses of argument. Cambridge: Cambridge University Press. Van Vechten, R. B. (2013). He Said/She Said: Tracking Gender Differences in Online Academic Discussions. Paper presented at the APSA teaching and learning conference. Van Vechten, R. B. and Chadha, A. (2013). How students talk to each other: Findings from an academic social networking project. Teaching civic engagement: From student to active citizen, 167-188. Wilson, J. T. (2001) Project management teams: A model of best practice in design. In D. Boud., R. Cohen, & J. Sampson (Eds.), Peer learning in higher education: Learning from & with each other (pp. 99–114). London, UK: Stylus Publishing Inc.

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 8, pp. 18-36, August 2017

A Qualitative Examination of Factors for Success in a Content-Based English Language Learner Classroom Janet Delgado, Ed.D. The School District of the City of York Lorraine T. Benuto , Ph.D. University of Nevada, Reno Department of Psychology Abstract. The dramatic increase of English Language Learners (ELLs) created a sense of urgency across school districts who struggle with efficiently educating students in a manner that facilitates the acquisition of English for ELLs and ensures that testing standards across content areas are met. Content-based sheltered instruction can provide quality education while maintaining the integrity of effective English language practices to a large number of ELLs simultaneously. The purpose of this qualitative multiple-case study was to identify and understand the essential attributes and relationships that contribute to the successfulness of content-based sheltered instruction elementary classrooms for English learners in an urban school district in southern Pennsylvania (this school has successfully and effectively accelerated their ELLs' levels of second language acquisition). Utilizing an Appreciative Inquiry Approach, the attributes that maximize the instruction of ELLs across four content-based, sheltered instruction ELL classrooms within one K-8 school were identified. Results revealed that these classrooms were successful when strong interrelationships were evident between language and content learning, efficient organizational structures, as well as a focus on the celebration of culture.

Introduction Many public school districts across the country require additional resources to educate students who arrive daily from other U.S. cities and countries around the world (GĂĄndara, Maxwell-Jolly & Rumberger, 2008). The vast differences that exist between students across socioeconomic, cultural, and linguistic dynamics contribute to students' levels of knowledge upon entering school (Fayden, 2011). These factors create a sense of urgency to accelerate English mastery so as to narrow the achievement gap, meet testing standards (Cosentino de Cohen & Chu Clewell, 2007), and avoid the ramifications of the Elementary and Secondary Education Act (U.S. Department of Education, 2009).

Unfortunately, this exacerbates an already impoverished learning environment for many (Fayden, 2011). Culturally and linguistically diverse (CLD), or English language learners (ELLs), are among the largest and most disadvantaged subgroups in many school districts (Fortuny, Capps, Simms & Chaudry, 2009). Approximately 17 million culturally and linguistically diverse children live in the U.S. (Migration Policy Institute, 2012). These learners come from home environments where a language other than English is spoken and they are acquiring language and literacy skills in English, regardless of their birth origin (Herrera, 2010). Studying language proficiency is important although other factors (e.g., socioeconomic circumstances) can impact how students perform on high-stakes testing (Fortuny et al., 2009). It is common for members of school organizations to select pedagogical approaches that can directly affect change within students' rate of second language acquisition (Ramos & Krashen, 2011; Starnes, 2010). Unfortunately many ELLs receive content instruction from educators who are not prepared to address their second language development needs (Echevarría, Short & Powers, 2008; Gándara, et al., 2008). Content-based, sheltered classrooms can provide a superior alternative to traditional English-only methods (Thomas & Collier, 2002) as these classrooms are led by dually certified ELL and content educators who deliver grade-level material and focus on English acquisition simultaneously (Genzuk, 2011). In most cases, they incorporate the Sheltered Instruction Observation Protocol (SIOP) model (Echevarría, Vogt & Short, 2017) as they demonstrate cultural responsiveness and allow native language and English dialogue in a low-anxiety language environment (Herrera, 2010; Taylor, 2010). This pedogogical design was influenced by Thomas and Collier's seminal work, known as the Prism Model (1997), which facilitates the simultaneous development in the academic, cognitive, linguistic and sociocultural domains of ELLs in their native and target languages within their academic environments. Thomas and Collier (1997) created the Prism model for language acquisition from a large-scale study and the prism model considers multiple areas of linguistic, academic, cognitive and sociocultural development (Thomas & Collier, 1997; 2002). When fostered simultaneously, these areas can determine the academic success for English language learners. In particular, facilitators within schools must encourage the development of language and culture in both the English learners’ native and target languages to provide supportive sociocultural environments for students (Thomas & Collier, 1997). Once all the learners' domains are addressed within the school, second language acquisition and acculturation can take place and students can prosper academically (Jang & Jimenez, 2011). Thomas and Collier (2002) were able to reaffirm their positive longitudinal results several years later, supporting the importance of developing the domains presented in the Prism Model for language acquisition within schools. Although Thomas and Collier (1997; 2002) stressed the importance of bilingual education, they proposed models of language instruction in English only that correlated closely to the areas of the Prism Model of second language acquisition. Specifically content-based ESL programs, featuring sheltered instruction, proved to be the most effective alternative to bilingual education

(Clark, 2009) when compared to other forms of subtractive schooling (Garza & Crawford, 2005), such as English immersion and/or pull-out services. Contentbased, sheltered instruction ELL classrooms operate through an instructional delivery approach that is focused on teaching English through content via explicit and scaffolded language experiences to assist students in reaching grade-level content expectations in English (Echevarría, Richards-Tutor, Canges & Francis, 2011). Many school districts experience rapid increases in the enrollment of English learners and are required to implement English-only instructional practices to educate them (Calderón, Slavin & Sanchez, 2011). However, school districts must offer English language instruction that amalgamates the learners’ academic needs with appropriate personnel and resources without compromising the second language development of English language learners in the school setting (Herrera, 2016; Thomas & Collier, 1997). One such means by which school districts can achieve this is via the implementation of contentbased sheltered instruction ELL classrooms (Short, Fidelman & Louguit, 2012). Content-based sheltered instruction classrooms provide learning environments that allow learners to acquire English as they benefit from grade-level content and language instruction designed specifically for ELLs. Not only do they accelerate the learners’ rate of English acquisition, they make it possible for educators to differentiate their instruction as their goal is to narrow the achievement gap between English language learners and their monolingual peers within their school districts as quickly as possible (Genzuk, 2011). However, this sheltered instruction ELL classroom model is underutilized because of the negative perceptions that allude to the segregation of students (Gándara & Orfield, 2010), as well as the perceived overwhelming concentration on English language skills in these classrooms (Clark, 2009). Thus the problem addressed in this study was that the increasing need for educating large populations of ELLs efficiently continues to be a national challenge (García, Jensen & Scribner, 2009) and while content-based sheltered instruction was effective, it was unclear what factors contribute to the successfulness of contentbased sheltered instruction elementary classrooms for English The purpose of this qualitative multiple-case study was to identify the essential attributes and relationships (Yin, 2014) that contributed to the successfulness of content-based, sheltered instruction elementary classrooms for English language learners in an urban school district in southern Pennsylvania. Because data was collected from multiple sources in multiple forms (and for the purpose of organization and clarity), the information about the materials and participants is consolidated here under this section.

Methodology Research tools Individual Interviews. Semi-structured individual interviews were conducted with administrators. The interview questions were pilot tested with one administrator and one district support personnel. The interview questions were revised based on the feedback provided. Participants in the individual interview portion of this study consisted of three administrators, including one

ELL reading specialist. They were selected because of their administrative roles, expertise and support (Lauckner, Patterson & Krupa, 2012) at the onset of the school’s implementation of sheltered ELL classrooms for a duration of at least five years. Additional study participants included seven educators certified in both elementary education and ELL. A variety of viewpoints based on teaching experiences were represented as three of the seven educators had taught as ELL pull-out teachers, as well as grade-level content teachers. Research tools for Observations. Four classroom observations were conducted and data was collected using the Danielson’s A Framework for Teaching protocol (2007) and the SIOP Protocol (Echevarría, Vogt, et al., 2017). See Table 1 for information about participants. Table 1 Classroom Observation Characteristics Classrooms

Grade

Content Area

Group

ELL Service

ELLs Class

Classroom 1

6th

math

full class

integrated

57%

Classroom 2

3rd

science

full class

integrated

68%

Classroom 3

1st

language arts

full class

integrated

63%

Classroom 4

3rd

language arts

full class

integrated

68%

A Framework for Teaching protocol (Danielson, 2007). The classroom environment scale from the Danielson’s A Framework for Teaching protocol (2007) is a professional practice performance scale utilized by administrators to evaluate teachers' performance. It was used to examine sociocultural attributes within each of the ELL classrooms to provide a context for the observations. Specifically, Doman 2 focused on classroom culture and climate, procedures and physical environment. SIOP Protocol (Echevarría, Vogt, et al., 2017). The instructional content of each classroom was observed and data was measured accessing a tool specifically created for ELL sheltered instruction (Short, Echevarría, et al., 2011). The SIOP protocol evaluates the categories of lesson preparation, building background, comprehensible input, learning strategies, interaction opportunities, application experiences and assessment options (Echevarría, Vogt, et al., 2017) within sheltered instruction lessons. The SIOP observation tool (Echevarría, Vogt, et al., 2017) provided rich data that was already present during content instruction within each elementary ELL classroom. The observation tool was utilized during English language arts or a specific content area lesson of the classroom teachers’ choice as it increased the opportunity for the educators to demonstrate a variety of practices implemented across content areas. This valid and reliable instrument was organized as a 5-point Likert scale (Trochim & Donnelly, 2008) and demonstrated the degree to which educators followed the lesson preparation and delivery protocol with fidelity. At the

conclusion of each observation, a brief post-observation interview with the teacher was conducted to clarify and/or extend conversation regarding what was observed during the session. Focus Groups. Six students from grades three or six were also included via focus groups (see Table 2 for demographic details), so that the data collected represented varied perspectives (Yin, 2014) of content-based, sheltered instruction ELL classrooms (see Table 3 for a breakdown of demographics). At the outset of the focus group interview, the purpose and protocol was introduced after initial questions about educator experiences were discussed in a whole group session format. The protocol established the procedures for creating a visual representation of their group's conversations, an Affinity diagram. The Affinity diagram is created by members of a group (Abilla, 2010), and provides a visual structure that organizes a large amount of data into themes based on their connections and relationships (National School Reform Faculty, 2012). Table 2 Student Focus Group Demographics Student

Sex

Grade

Yrs. at Home School Language(s)

Family Origin

Student 1

male

three

Spanish

Puerto Rico/ Dominican Republic

Student 2

female

three

Spanish/English

Puerto Rico

Student 3

male

three

Khmer/English

Cambodia

Student 4

female

three

Spanish

Puerto Rico

Student 5

male

six

Spanish/English

Puerto Rico

Student 6

male

six

Spanish

Puerto Rico

Data Collection An Appreciative Inquiry (AI) approach is a method of inquiry that focuses on the positive attributes of an organization (Gaddis & Williams, 2009). The main components included in an AI approach are discovery, dream, design and delivery/destiny (Bushe, 2011) and these provided a procedural focus for this study inquiry (Bushe, 2011; Cantore & Cooperrider, 2013; Stevenson, 2011). Data Processing and Analysis A case study database was necessary to capture large amounts of language data across multiple settings that visually represented commonalities in the data and assisted in the compilation of each case record (Patton, 2002) utilizing Atlas.ti7. Adhering to an inductive process (Schutt, 2012); code identification, revision, as well as data reduction (Johnson, Christensen & Turner, 2014) was necessary to focus a large amount of language data into manageable components (Miles, Huberman & Saldana, 2013). To determine elements of significance and relationships across the multiple sources of data,

additional analysis led to the identification of the context of each assigned code frequency (Miles et al., 2013). Table 3 demonstrates the distribution of the frequency of codes by data source and was utilized as the first layer of analysis (Friese, 2012) as to arrive at themes that exemplify the successfulness of contentbased, sheltered instruction ELL classrooms at Wilson K-8 School. Table 3 Code Families by Data Source Code Family

Individual Interviews

Classroom Observations

Focus Groups

Total

Academic Language

123

Class Community

School Characteristics

School Culture and Climate

School Supports and Resources

Student Achievement

Instructional Practices

212

315

Teacher Behavior and Beliefs

Totals

296

411

176

883

Results RQ1. The first research question was: How are content-based, sheltered instruction classrooms maximizing the instruction of ELLs in an elementary setting with large populations of ELLs? . The data from the four content-based, sheltered classroom observations revealed that multiple explicit instructional practices within positive and engaging classroom communities, in conjunction, maximized the instruction of ELLs. These classrooms consistently provided safe and supportive conditions for learning by all students, including English language learners, so as to accelerate their progress academically and linguistically (Jang & Jimenez, 2011). Evidence of successful student talk and active engagement were described within each classroom observation. Academic/language instruction was infused within all of the described gradelevel content lessons. However, additional data was needed to identify specific practices and strategies as evident within the lessons, as well as from the perspectives of the students themselves. The following two sub-questions led to more precise information as the triangulation of the data (Shutt, 2012; Yin, 2014) from multiple sources i.e., the SIOP observation protocol (EchevarrĂa, Vogt, et al., 2017) and the student focus group interviews. SQ1. The first sub-question derived from research question 1 was: What instructional practices take place in content-based, sheltered instruction ELL classroom that makes them effective for English language learners? Two current and retired administrators were asked what positive attributes were evident, relating to instructional practices, when they entered and/or observed content-based,

sheltered instruction ELL classrooms on a regular basis. Although a minimum of eight specific examples were provided by each participant, they discussed topics and referred to examples of instructional practices throughout the interviews. When the language data was compiled from the individual interviews, the overarching categories of instructional practices most identified are displayed in Table 4. Table 4 Categories of Instructional Practices Witnessed in ELL Classrooms Instructional Practices

Frequency

Explicit academic/language instruction

Bilingual support

Differentiated instruction and participation

Levels of support throughout lessons

Multisensory teaching and learning

Explicit academic/language instruction, bilingual support, differentiated instruction and participation, levels of support throughout lessons and multisensory teaching and learning were the instructional practices identified as effective for the English language learners in the ELL classrooms. These observation teaching behaviors were documented on the SIOP to indicate the elements of effective sheltered instruction for each observed classroom and were reported. Although all categories were identified within each lesson, the following, in particular supported the instructional practices mentioned above: comprehensible input, strategies, interaction, practice/application and delivery. These categories measured the effectiveness in content delivery, the variety of multisensory experiences, explicit strategies, as well as student/peer engagement and support and were noted on the Likert scale (1-not evident to 4highly evident) as “3-evident or 4-highly evident” within each lesson observed at Wilson K-8 School. The data triangulated, or analyzed from multiple sources, was necessary to ensure validity of the results (Yin, 2014) throughout the qualitative study. Particularly to respond to this sub-question, it was accomplished via the classroom observations and the results from the SIOP protocol. The results support the effective practices of explicit academic/language instruction, bilingual support, differentiated instruction and participation, levels of support throughout lessons and multisensory teaching and learning as essential to the success of content-based, sheltered instruction ELL classrooms. SQ2. From the perspectives of learners, what strategies increase their learning and their use of academic language? The English language learner participants at Wilson K-8 identified several practices that increase their learning and use of academic language within content-based, sheltered instruction ELL classrooms (see Tables 5 and 6 for details). Students confirmed that working collaboratively within peer configurations (partners or groups) naturally elevates the levels of student

engagement and support during explicit content and strategy instruction. Additionally, the classroom observation data supports the ELLs' statements and expanded on the examples of academic language techniques used in the classroom, such as cognate instruction and teacher elaboration of vocabulary and student responses. In fact, within the classroom observation data, the instructional practices and academic language code families displayed two strongest occurrences: teacher practices and academic language. Table 6 indicates the specific codes within the academic language family displayed during the four non-participatory classroom observations of the ELL classrooms. The trend was reaffirmed that in order to increase content learning, as well as academic language, intentional opportunities for student engagement, experimentation and discussion among peers were essential in the successful operation of content-based, sheltered instruction ELL classrooms. Table 5 Instructional Practices that Increase Learning and Academic Language Instructional Practices

Student Response Frequency

Collaborative peer groups (partners included)

Explicit content and strategy instruction

Multisensory teaching and learning

Student engagement in lessons

Students receive sufficient support

Table 6 Academic Language Code List Components of Academic Language

Frequency Classroom Observations

within

Explicit academic and language Instruction (cognates 11 included) English language practice and experimentation

Teacher elaboration of vocabulary and student 37 responses Vocabulary instruction

Q2. How are successful content-based, sheltered instruction ELL classrooms supported within their school? Data, specific to this question, was analyzed from two sources within the multiple-case study. Data was collected from individual interviews and the teacher focus group interview. Both sources were necessary as to reveal the evidence derived from multifaceted perspectives through triangulation (Lauckner et al., 2012). Within the individual interviews,

as well as the teacher focus groups, the participants were asked to reflect on the unique attributes of the school and in what ways are the ELL classrooms supported from the school/district level. Ultimately, they both affirmed the necessity of support available of varying degrees within the school structure. The individual interview participants who consisted of past and current personnel within administrative positions at Wilson K-8 School, as well as the teachers involved in the teacher focus group interview, presented similar elements considered as essential academic support systems within the school. Flexibility within scheduling and curriculum allows for additional personnel to assist with small group and/or explicit instruction based on the students' individual needs, as well as the administration encouraging teacher discretion on the depth and breadth of content/language delivery necessary for each classroom within a flexible schedule and curriculum. Both sources reported that the self-contained student configuration created an entire system of support within content-based, sheltered instruction ELL classrooms. Although the specific school's climate and climate was mentioned as a support system that overlapped the essential components, this area will be discussed in greater detail within its corresponding sub-question. SQ3. How are educators maintaining a culturally sensitive environment that promotes the efforts of ELLs? The language data compiled from interviews, classroom observations and focus groups offered insight on how educators maintain an environment that facilitates the development of their ELLs among their peers within content-based, sheltered instruction classrooms. The triangulation of the data (Yin, 2014) indicated that the grade-level ELL teachers and their students consistently work on ensuring a positive, classroom community as to encourage all learners to feel they are part of a unit and that they are capable of excelling in academics similar to their peers. Although the code frequencies across the language data were mostly found in the classroom community and teacher’s behaviors and beliefs code families, instructional considerations also contributed to their successful environments. Data collected from classroom observations and from their teachers, in particular, indicated that a sense of community was created when the students realized that collaborative peer groups were expected. Encouragement and the respect of differences in individuals’ cultures, personality traits and language skill sets were essential to build the class environment. “We are a family. Helping out comes naturally to many of our students.” From the evidence analyzed from the language data from the multiple sources, as well as from the Danielson's classroom observation tool (2007), educators maintain a culturally sensitive environment that promotes the efforts of ELLs within content-based, sheltered instruction ELL classrooms. They achieve this by delivering sound, effective instructional opportunities that intentionally demonstrate to the English language learners that they are capable of academic achievement similar to that of their peers. These opportunities must also be provided within the context of positive classroom environments for all students. SQ4. In what ways can school climate and culture support contentbased, sheltered instruction classrooms? The school’s climate and culture is

categorized as caring, supportive and knowledgeable, according to the individual interview participants. Although 10 out of the 21 faculty members worked at the school for at least seven years, the current administrator observed the school’s unique qualities within her first year working at the school. “We have a nurturing culture here at Wilson K-8. Teachers here go above and beyond the four walls to support our students. I would say it is way more than what I have seen at other buildings that I have worked in. Teachers support the students who have language needs. The teachers are experienced in reading students where they are and move them forward. There is compassion for what students are going through in their everyday lives.” Table 7 demonstrates the holistic distribution of occurrences within the school culture family code across the three sources of data. One interview participant noted that as a result of the school community, the relationships between teachers, students and their families were why many ELL families request building transfers to remain or to enter this school specifically from within the school district. Table 7 Components of the School Community School Culture Primary Codes

Overall Frequency

Community among staff

Described as family

Parent participation and events

School community

Teacher and student academic advancement

The belief that all individuals can learn and work toward academic advancement among students and teachers contributed to the successful climate and culture of Wilson K-8. Not only was bilingualism viewed as a positive attribute within students to accelerate students along academically, the teachers at Wilson K-8, were also viewed as professional learners. The teachers were knowledgeable as over 50% were ELL certified, as well as within their content area. However, the teachers were learning effective strategies from each other. One administrator explained professional development opportunities that occurred among colleagues. “The ELL classrooms often, many of them, became models I could refer teachers to go visit. If they wanted to see how a particular teaching/learning strategy was used, I could send them to observe a classroom….We had a lot of strong teaching models, and teachers were learners. They didn’t stop learning. They learned from each other in professional learning communities (PLCs). In their PLCs, they could learn from each other, they could observe each other, and that was one of the strongest benefits that I think we had in our building because there was such a wealth of good teaching in that building and people were willing to try new things.” climate

Data from this multiple-case study provides the manners in which school and culture support content-based, sheltered instruction ELL

classrooms. As a result of their close professional relationships, the teachers and other personnel at the school assist each other so that they can support the academic advancement of their students. Because they are knowledgeable and compassionate about the students' family and cultural biographies, they problem-solve together to provide what ELLs and their families need to achieve personally and academically. Inductive reasoning and analysis was conducted to identify the themes present across the various sources of data within the multiple-case study. Six themes emerged through memo-ing and examining the frequency and correlations between codes and within code families. These themes were identified below (see Table 8) and are presented as equally vital to the success of content-based, sheltered instruction ELL classrooms. Although the following themes are representative and validated throughout all of the questions posed during the multiple-case study, the themes are discussed in order of the main research questions and their corresponding sub-questions. The overarching concepts all relate to the members of the organization understanding their population and addressing their studentsâ&#x20AC;&#x2122; needs within their academic, linguistic and sociocultural development as suggested by Thomas & Collier (1997; 2002). Table 8 Emerging Themes Classroom configurations in language and learning Explicit instruction by knowledgeable personnel Student engagement for students of all language levels Academic and structural flexibility Celebration of culture, language, and learning Functionality of its members as a support unit

Discussion This descriptive multiple-case study was an analysis of factors present in successful content-based, sheltered instruction ELL classrooms within an urban K-8 school educating high populations of English language learners at the elementary level. Data collected from individual interviews, classroom observations and focus groups addressed the research questions. The findings contribute to the affirmations presented within the theoretical framework of the Prism model of language acquisition within schools (Thomas & Collier, 1997; 2002). Although the selected school did not demonstrate effective bilingual education practices as recommended in the Prism model; the content-based, sheltered instruction ELL classrooms were established in efforts to move students academically within similar language communities. They exhibit components that support students academically, linguistically and cognitively within socioculturally supportive environments (Herrera, 2016; Thomas & Collier, 1997; 2002). The implications from the findings of this study are discussed below.

Classroom configurations in language and learning. The results form this study support the notion that content-based, sheltered instruction ELL classrooms maximize instruction by creating specially designed classroom environments in which educators can provide explicit instructional practices that unify academic and language objectives throughout all grade appropriate content delivery necessary for English language learners (Clark, 2009; Short & Echevarría, 2016; Genzuk, 2011). They address the importance of solid core instruction within Tier 1 of the Response to Intervention (RTI) framework (Echevarría & Hasbrouck, 2009) through the implementation of explicit instruction utilizing the SIOP protocol designed specifically for these learners (Echevarría, Vogt, et al., 2017). Findings from this study indicate that classroom configurations within content-based, sheltered instruction ELL classrooms may reduce the amount of ELLs in need of intervention at a more intense level as their core instruction is designed specifically for their learner populations and not of the traditional structure intended for the majority culture. These findings are consistent with the extant research (Chang, 2008; Gándara et al., 2008). Explicit instruction by knowledgeable personnel. Explicit academic/language instruction, bilingual support, differentiated instruction and participation, levels of support throughout lessons (Echevarría, Frey & Fisher, 2015) and multisensory teaching and learning were identified as the effective instructional practices provided by ELL certified grade-level teachers across content-based, sheltered instruction ELL classrooms (Chang, 2008) observed during the study. Administrators stated there was a specific advantage to the high number of certified grade-level ELL teachers with these skills so as to provide consistency of effective instruction from year to year from kindergarten to sixth grade within one school. Multiple years within the program was viewed as vital to accelerating students academically and linguistically (Cosentino de Cohen & Chu Clewell, 2007) as language proficiency is multi-directional and unique for each student (Herrera, 2016). Student engagement for students of all language levels. From the learners' perspectives, students confirmed that working with partners and/or groups engages and provides natural support during explicit content, strategy, and/or language instruction (Swain, 2000). Regardless of English language levels, all students benefit from additional time to negotiate for meaning (Short, Echevarría, et al., 2011), experimenting with language within their own zones of learning (Min, 2006), as well as working through new academic content with others. By facilitating total student participation within lessons (which is recommended by Herrera [2016] and Himmele and Himmele [2012]) ELLs feel empowered and a sense of belonging within their learning communities (Washburn, 2008). Academic and structural flexibility. Flexibility within the school organization became the overarching theme for how the classrooms in the current study operate successfully within their school communities. Successful content-based, sheltered instruction ELL classrooms are supported within the school organization by facilitating academic and structural flexibility within all aspects so as to tailor their instruction to the specific needs of their ELLs. Not only is there educational autonomy with regards to essential content and scheduling, educators make decisions to create appropriate matches between the

ELL service types available and the linguistic, academic and acculturation experiences of the students. Echevarría and Hasbrouck (2009) placed a high priority on the ability for schools to be able to make decisions based on their ELLs’ individual cultural biographies (Herrera, 2016) within the core structure of the school. Celebration of culture, language, and learning. The student participants embraced their sense of home within their content-based, sheltered instruction ELL classrooms and among the school. In order to achieve this, educators must maintain culturally sensitive environments that promote the efforts of ELLs by facilitating instructional opportunities that reassure ELLs that they can see themselves in their learning (Taylor, 2010). Particularly within the contexts of positive classroom environments, English language learners’ levels of language proficiency in L1 and L2 and their own life experiences (Dong, 2013) are showcased and utilized intentionally throughout content lessons. Although sociocultural theory remains prevalent within this theme, Cummin’s contextual interaction theory (1996) justified the connection between the relationships of successful language input, a supportive affective environment, as well as the status associated within culture and language among peers and teachers (Lavadenz, 2011). Content-based, sheltered instruction ELL classrooms become a center for individual motivation through the celebration of culture, language and learning. Functionality of its members as a support unit. The school climate and culture at Wilson K-8 supports the success within content-based, sheltered instruction ELL classrooms by establishing a cohesiveness that is understood and accepted by all (Rodriguez, Ringler, O’Neal & Bunn, 2009). A high level of support was noted as one of the key elements of success for these classrooms as the strong-knit relationships between among administrators, teachers, students and their families maintain a positive learning environment that flexes and problem-solves to understand their learners (Washburn, 2008)). Social capital theory substantiates the benefits of human relationships and the power to achieve more through collective actions (Bourdieu, 1985). Essentially, the functionality of the schools' members as a support unit provides the context and conditions for excellence among elementary level English language learners at this school.

Future Directions The first recommendation for educational application is for schools to expand on the variety of ELL service options available within a school within increasing student populations of ELLs. This is necessary so that knowledgeable, certified educators can appropriately match students of different cultural biographies with optimal instructional and school climate conditions specific to their needs (Pray & Monhardt, 2009). Content-based, sheltered instruction self-contained and ELL integrated classrooms can provide classroom environments that balance the academic, linguistic and sociocultural needs of specific students as supported by the Prism model (Thomas & Collier 1997; 2002). Many of the administrators and teachers who participated in this study stated the importance of placing elementary students in classrooms that

allow for explicit instruction and classroom engagement experiences that match the specific academic, language and sociocultural needs of the students (Jang & Jimenez, 2011; Murry, 2012). However, continuous professional development opportunities that include the most recent research-based strategies within explicit instruction, second language acquisition and instruction and the SIOP model (Echevarría, Short, et al., 2008 must remain at the forefront as schools in need must establish an effective educational framework for the influx of English language learners (Genzuk, 2011; Reeves, 2009; Taylor, 2010 ). The second recommendation is that a team, consisting of a school administrator and ELL personnel could be assembled within the schools. Their focus would be to assist with the academic, structural and sociocultural flexibility necessary to appropriately screen and monitor the large populations of ELLs (Brown & Sanford, 2011) through collaboration, instructional planning, and/or direct student support in efforts to continue to make measurable gains in English language proficiency (Anderson & Dufford-Melendez, 2011). Ultimately, the ELL support team could work alongside a school's RTI team (Brown & Sanford, 2011; Calderón, Slavin & Sanchez, 2011; Echevarría & Hasbrouck, 2009) to maintain solid core instruction while increasing the sociocultural experiences by way of strong interrelationships between teachers, students and their families (Good et al., 2011). Recommendations for further research. Future studies could be performed to expand on the factors for success within content-based, sheltered instruction ELL classrooms. The first recommendation could be to replicate this study, but data from observations could be collected throughout a longer period of time during one school year as it would provide longitudinal data that could represent a more accurate depiction (Cozby, 2014) of the effective instructional practices and classroom environments of each classroom. Additionally, an increase in the number of ELL classrooms observed could also contribute data in the same manner within the multiple-case study. The second recommendation for further research could add an additional student focus group interview session of current students who have participated in content-based, sheltered instruction ELL classrooms, but have provisionally exited out of ELL services as they have accomplished academic and linguistic proficiency at their grade-level. This could increase the depth into the students' perspectives of their experiences (regarding instructional practices and their classroom environments within their previous content-based, sheltered instruction ELL classrooms at Wilson K-8 School). Most importantly, they could offer additional insight into the program's success overall as the students participated since the beginning of their journey towards English language proficiency within content-based, sheltered instruction ELL classrooms within the schoolwide model (Kang, 2010; Rodriguez et al., 2009). Finally, the last recommendation could examine the difference between the mission of this school's content-based, sheltered instruction ELL classroom model and their classroom configurations to include integrated and selfcontained and establish clear definitions of the expectations and achievements when comparing the existing English-only models. Further research analysis of existing ELL classrooms to the classrooms highlighted within this study may

lead to additional contributing factors for success that may have been overlooked in this study.

Conclusions A qualitative multiple-case study was conducted to identify the essential attributes and relationships (Yin, 2014) that contributed to the successfulness of content-based, sheltered instruction elementary classrooms. Minimal research exists of ELL classroom models with similar characteristics addressing the needs of large populations of ELLs efficiently and utilizing English as the language of instruction within one school. In addition to the urging pressures of upcoming achievement standards (Alberti, 2013; Schmidt & Burroughs, 2013) and new teacher evaluation tools (Danielson, 2012; Marzano, 2012), school districts across the nation struggle to identify effective solutions to educate new, large populations of ELLs (Guccione, 2011) effectively in the quickest amount of time without providing bilingual education. The findings of this study establish attributes of successful content-based, sheltered instruction ELL classrooms that can serve as a viable method for many school districts in need of solutions. Six themes emerged from the triangulation of the data collected from interviews, observations and focus groups which answered the research questions: 1) classroom configurations in language and learning; 2) explicit instruction by knowledgeable personnel; 3) student engagement for students of all language levels; 4) academic and structural flexibility; 5) celebration of culture, language and learning; and 6) functionality of its members as a support unit. The implications were organized and presented by the themes that arose from the findings of the multiple-case study. However, these relationships indicated within the study mirror the areas of development necessary for English language proficiency in schools as indicated by Thomas and Collier (1997; 2002). Thus, the fundamental implication of this qualitative, multiple-case study is that all attributes and relationships must interact and depend on each other so as to create successful content-based, sheltered instruction ELL classrooms for its students. The two recommendations for application based on the findings of the study have been discussed through the perspective of improving the quality of services English language learners receive in school districts with rising ELL populations, in particular. School districts must develop their ELL programs so several models are available within each school with a large population of ELLs in the elementary grades. Because ELLs obtain their own unique cultural biographies (Murry, 2012) that include academic experiences in their native language, the type of instructional service and classroom environment must also vary so as to meet these learners' needs efficiently. An ELL team of professionals within each building can assist with these student cases and offer professional development, instructional strategies, as well as hands-on support in the classroom. Further investigation of successful content-based, sheltered instruction ELL classrooms is suggested to increase the validity and reliability of the results of this study. It is recommended that the study be conducted on a larger scale to include classroom observation within content-based, sheltered instruction ELL

classrooms multiple times throughout the school year. Longitudinal research could provide a deeper holistic perspective of the instructional practices and positive classroom elements. Future researchers may wish to include students who have provisionally exited the ELL program and attend classes in mainstream grade-level classrooms in an additional student focus group interview session. Finally, a study examining the attributes and expectations of other ELL classrooms within the existing literature compared to this study’s findings would allow school districts to select content-based, sheltered instruction classrooms as described in the study as the ideal ELL service model when bilingual education is not available to address many ELLs at once. Ultimately, school districts may invest additional resources and ethical support for the learners on the path to academic and English language success within their school organizations.

References Abilla, P. (2010). Affinity diagram. Retrieved from http://www.shmula.com Alberti, S. (2013). Making the shifts. Educational Leadership, 70(4), 24-27. Anderson, K.S. & Dufford-Melendez, K. (2011). Title III accountability policies and outcomes for K-12: Annual measureable achievement objectives for English language learners in Southeast Region states. (REL-No. 105). Washington, D.C.: U.S. Department of Education, Institute of Education Services, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/edlabs. Bourdieu, P. (1985). The forms of capital. In J.G. Richardson, Handbook of Theory and Research for the Sociology of Education, (pp. 241-258). New York: Greenwood. Brown, J.E. & Sanford, A. (2011). RTI for English language learners: Appropriately using screening and progress monitoring tools to improve instructional outcomes. Washington, D.C.: National Center on Response to Intervention. Bushe, G.R. (2011). Appreciative inquiry: Theory and critique. In Boje, D. Burnes, B. and Hassard, J. The Routledge Companion to Organizational Change, 87103, Oxford, UK: Routledge. Calderón, M., Slavin, R. & Sanchez, M. (2011). Effective instruction for English learners. The Future of Children, 21(1), 103-127. Cantore, S. P. & Cooperrider, D.L. (2013). Positive psychology and appreciative inquiry. The Wiley-Blackwell Handbook of the Psychology of Leadership, Change, and Organizational Development, 267-287. doi: 10.1002/9781118326404.ch13 Chang, M. (2008). Teacher instructional practices and language minority students: A longitudinal model. The Journal of Educational Research, 102(2), 83-97, 160. Clark, K. (2009). The case for structured English immersion. Educational Leadership, 66(7), 42-46. Cosentino de Cohen, C. & Chu Clewell, B. (2007). Putting English language learners on the educational map: The No Child Left Behind Act

implemented. Education in Focus. Washington, D.C.: The Urban Institute. Cozby, P.C. (2014). Methods in Behavioral Research. New York: McGraw Hill. Danielson, C. (2012). Observing classroom practice. Educational Leadership, 70(3), 32-37. Danielson, C. (2007). Enhancing professional practice: A framework for teaching. Alexandria, VA: Association for Supervision and Curriculum Development (ASCD). Dong, Y.R. (2013). The Bridge of Knowledge. Educational Leadership, 71(4), 30-36. Echevarría, J., Frey, N.S. & Fisher, D. (2015). What it takes for English learners to succeed. Educational Leadership, 72(6), 22-26. Echevarría, J. & Hasbrouck, J. (2009). Response to intervention and English learners. CREATE: Center for Research on the Educational Achievement and Teaching of English Language Learners Washington, D.C.: Institute of Education Services. Echevarría, J., Richards-Tutor, C., Canges, R. & Francis, D. (2011). Using the SIOP model to promote the acquisition of language and science concepts with English learners. Bilingual Research Journal, 34(3), 334-351. Echevarría, J., Short, D. and Powers, K. (2008). Making content comprehensible for non-native speakers of English: The SIOP model. The International Journal of Learning, 14(11), 41-49. Echevarría, J., Vogt, M. & Short, D.J. (2017). Making content comprehensible for English learners: The SIOP Model. New York: Pearson Education. Fayden, T. (2011). Forming the forms of higher intelligence: Further than standards. NABE News, 33(1), 16-20. Fortuny, K., Capps, R., Simms, M. & Chaudry, A. (2009). Children of immigrants: National and State Characteristics. Perspectives on low-income working families. Washington, D.C.: The Urban Institute. Friese, S. (2012). Qualitative data analysis with ATLAS.ti. London: Sage. Gaddis, S.E. & Williams, C. (2009). See yourself in 4-D: How to use appreciative inquiry to ignite positive change. Retrieved from http://www.communicationsdoctor.com/articles/seeyourselfin4-D.pdf Gándara, P. & Orfield, G. (2010). A return to the Mexican room: The segregation of Arizona’s English learners. Civil Rights Project, University of California, Los Angeles, CA. Retrieved from ERIC database. (ED 511322) Gándara, P., Maxwell-Jolly, J. & Rumberger, R. (2008). Resource Needs for English Learners: Getting Down to Policy Recommendations. UC Santa Barbara: Linguistic Minority Research Institute. Garza, A. V., & Crawford, L. (2005). Hegemonic multiculturalism: English immersion, ideology, and subtractive schooling. Bilingual Research Journal, 29(3), 599-619,728. Genzuk, M. (2011). Specially designed academic instruction in English (SDAIE) for language minority students. Center for Multilingual, Multicultural Research Digital Papers Series. Center for Multilingual, Multicultural Research, University of Southern California. Retrieved from http://www.usc.edu/dept/education/CMMR/DigitalPapers/SDAIE_ Genzuk.pdf

Good et al., 2011). Guccione, L.M. (2011). Integrating literacy and inquiry for English learners. The Reading Teacher, 64(8), 567-577. Herrera, S.G. (2016). Biography-driven culturally responsive teaching. New York: Teachers College Press. Himmele, P. & Himmele, W. (2012). How to know what students knows. Educational Leadership,70(1), 58-62. Jang, E.Y. & Jimenez, R.T. (2011). A sociocultural perspective on second language learner strategies: Focus on the impact of social context. Theory into Practice, 50, 141-148. doi: 10.1080/00405841.2011.558443 Johnson, R.B. Christensen, L.B. & Turner, L.A. (2014). Research methods, design and analysis. Saddle River, NJ: Pearson. Kang, A. (2010). The contribution of L1 CALP and L2 proficiency to successful CBI experience. English Teaching, 65(4), 3-39. Lauckner, H., Patterson, M., & Krupa, T. (2012). Using constructivist case study methodology to understand common developmental process: Proposed methodological questions to guide the research process. Qualitative Report, 17, 1-22. Lavadenz, M. (2011). From theory to practice for teachers of English learners. CATESOL Journal, 22(1), 18-47. Marzano, R.J. (2012). The two purposes of teacher evaluation. Educational Leadership, 70(3), 14-19. Miles, M.B., Huberman, A.M. & Saldana, J. (2013). Qualitative data analysis: A methods sourcebook. Thousand Oaks, CA: Sage. Min, G. (2006). Vygotsky's sociocultural theory and the role of input and output in second language acquisition. CELEA Journal, 29(4), 87-92. Murry, K. (2012). Cognitive development, global learning, and academic progress: Promoting teacher readiness for CLD students and families. Journal of Curriculum and Instruction, 6(1), 11-24. National School Reform Faculty (2012). Appreciative inquiry: A protocol to support professional visitation. Retrieved from http://www.nsrfharmony.org Patton, M.Q. (2015). Qualitative research and evaluation methods: Integratinge theory and practice. Thousand Oaks, CA: Sage. Pray, L. & Monhardt, R. (2009). Sheltered instruction techniques for ELLs: Ways to adapt lessons to meet the academic needs of English language learners. Science and Children, 46(7), 34-38. Ramos, F. & Krashen, S. (2011). Success without bilingual education?: Acquisition of academic language in English through comprehensible input. NABE News, 33(4), 13-14. Reeves, D.B. (2009). The value of culture. Educational Leadership. 66(7), 87-89. Rodriguez, D., Ringler, M., O'Neal, D. & Bunn, K. (2009). English language learners' perceptions of school environment. Journal of Research in Childhood Education, 23(4), 513-526. Schmidt, W.H. & Burroughs, N.A. (2013). How the Common Core boosts quality and equality. Educational Leadership, 70(4), 54-58. Short, D.J. & EchevarrĂa, J. (2016). Developing academic language with the SIOP

Model. Boston: Pearson. Short, D. J., Echevarría, J., & Richards-Tutor, C. (2011). Research on academic literacy development in sheltered instruction classrooms. Language Teaching Research, 15(3), 363-380. doi:10.1177/1362168811401155 Short, D.J., Fidelman, C.G., & Lougit, M. (2012). Developing academic language in English language learners through sheltered instruction. TESOL Quarterly, 46(2), 334-361. Shutt, R.K. (2012). Investigating the social world: The process and practice of research. Thousand Oaks, CA: Sage. Starnes, B.A. (2010). Rethinking diversity. Kappan, 92(1), 74-75. Stevenson, H. (2011). Appreciative inquiry: Tapping into the river of positive possibilities. Cleveland Consulting Group. Novelty, OH. Retrieved from http://www.clevelandconsultinggroup.com/articles/appreciativeinquiry.php Swain, M. (2000). The output hypothesis and beyond: Mediating acquisition through collaborative dialogue. In J. Lantolf, Sociocultural theory and second language learning, (pp. 97-114). Oxford: Oxford University United Press. Taylor, R.W. (2010). The role of teacher education programs in creating culturally competent teachers: A moral imperative for ensuring the academic success of diverse student populations. Multicultural Education, 17(3), 24-28. Thomas, W.P. & Collier, V. (2002). A national study of school effectiveness for language minority students’ long-term academic achievement. Center for Research on Education, Diversity and Excellence, Santa Cruz, CA. Retrieved from ERIC database. (ED 475048) Thomas, W.P. & Collier, V. (1997). School effectiveness for language minority students. National Clearinghouse for Bilingual Education, Washington, D.C. Retrieved from ERIC database. (ED 436087) Trochim, W.M.K. & Donnelly, J.P. (2008). The research methods knowledge base. Mason, OH: Atomic Dog. Washburn, G.N. (2008). Alone, confused and frustrated: Developing empathy and strategies for working with English language learners. The Clearing House, 81(6), 247-250. Retrieved from http://search./proquest.com/docview/196844858.accountid=28180 Yin, R.K. (2014). Case study research: Design and methods. Los Angeles: Sage.

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 8, pp. 37-50, August 2017

Radio Wave Errors: Students Mistaking Radio Transverse Electromagnetic Light Waves as Longitudinal Sound Waves A. E. Tabor-Morris, Ph.D., T. M. Briles, Ed.D. and R. Schiele, B.S. Georgian Court University Lakewood, New Jersey, USA Abstract. Commonly anecdotally noted among physics instructors is that students often misidentify radio-waves as sound waves, not as part of the electromagnetic light-energy spectrum. To highlight the prevalence of this error, a pilot survey, whose results are presented here, was made of a total of 225 high school physics students from four high schools in New Jersey in the USA, taken immediately after students had covered both sound and electromagnetic radiation. Note that although the study is made in one locality, there is likelihood that the same data would be obtained in any introductory physics classroom and future studies are suggested. This survey suggests that a majority of students appear to still incorrectly conclude that 'radio waves' are sound, even after instruction otherwise. This is perhaps reinforced by students' sensory illusion interpretation which might be articulated as: "I hear a radio, I experience 'radio broadcasts' as sound, so if 'radio signals' are 'radio waves', they are hence sound waves". The survey results were also sought to see if students who responded that "radio waves are sound" in this study, more consistently answered other related questions that used that assumption - that is, once they made that decision, did they stick with it, even when not consecutively asked the questions in the survey? Or is it possible that aspects of questioning or syllabi can mislead students? To help teachers assist students in properly identifying radio waves some ideas are suggested, particularly directly challenging students to realize that this is a frequent misunderstanding. Keywords: Radio Waves; Physics Education; Astronomy Education.

Introduction ‗Are radio waves sound waves or part of the electromagnetic spectrum?‘ and ‗are they hence longitudinal or transverse waves?‘ These are common questions from physics teachers to their students when studying radio waves. Yet, anecdotally, students often get these questions wrong. Why do physics teachers on both the secondary level and in higher education often refer to difficulties in teaching the electromagnetic spectrum with a sense of frustration,

especially regarding students‘ frequent misidentification of radio waves as longitudinal sound waves and not as transverse electromagnetic waves after being taught otherwise? Are there conceptualizations that students are making regarding the physical phenomenon of radio transmissions that make it difficult to communicate the correct physics of radio signals to students? Are more stringent warnings to students of difficulties needed when teaching sound waves and electromagnetism? The purpose here is to document the prevalence of the existence of this difficulty and possible implications of students‘ misidentification of radio waves as sound instead of as electromagnetic waves in the radio frequency range, an invisible portion of the light spectrum. For this pilot study, the students completed a paper-pencil instrument to show their understanding of this material. Also, how strong is the possibility that students may carry through their conclusions that radio waves as sound or light to other questions? It is also the first published study to attempt to quantify the prevalence of this error, and also to point out the difficulty of producing testing questions that do not mislead students via pointing out the weakness of the survey tool used. Note that due to its fundamental nature, we use the term light (as is done in many physics textbooks) to mean both the visible portion of the electromagnetic spectrum and also invisible portions of the electromagnetic spectrum such as radio waves, microwaves, infrared, ultraviolet rays, X-rays, and gamma rays. The choice to confine the definition of light to just visible light appears to be increasingly viewed as abstruse and unusual, hence the terminology choice made here.

Background The need for, and difficulties of, students to understand the nature of radiation, radio waves, sound waves and radio signals is noted in both the physics and astronomy communities (Berger, 2015; Barder et al., 2005; Landt, 2015; McGuinnes & Oliver, 1998; Plotz & Hopf, 2016; Newmann & Hopf, 2012; Rego & Peralta, 2006). Radio waves are introduced in most physics textbooks during the discussion of waves and then again after sound when electromagnetic radiation is introduced. Astronomy textbooks do so when they introduce the concept of light. Physics textbook author Paul Hewitt generated a thought problem on this important misunderstanding in his ―Figuring Physics‖ series available as a Next Time Question (Hewitt, 2007). Other suggestions and helps in addressing teaching radio waves are available in Perkins et al. (2006), Wise (2006), DeVries (2001), and Finkelstein et al. (2006). The question remains of why would students think of radio waves, transmissions from one radio to another radio tower as sound waves? It should be noted that when one thinks of a radio, one thinks of the sound it produces – such as listening to a car radio or a radio station on one‘s cell phone. Students might then easily misinterpret radio waves as being the same as radio signals. A radio signal does change forms: from sound to electrical signal inputted to a transmitting antenna, to electromagnetic (EM) ‗radio waves‘ that travel to a detecting radio antenna, which in turn produces electrical signals that are converted to sound output that is heard, where the receiving ‗radio‘ itself is acting like a translator box and producing a sound wave. If one is simply

considering the beginning and end product, it only makes sense to consider radio signals as sound waves. In that sense, physics learning that asks the question – does the physics make sense in everyday life based on what is sensed and are students naïvely putting in a situation of observation to be deceived by sensory illusions and correlation to everyday experiences (Tabor-Morris, 2015; Caramazza et al., 1981) which can mislead if all aspects of the radio signal are not considered? Also the intricacy of signal changing forms may be only briefly addressed in class, perhaps because physics teachers tend to try to stick to the basics – such as saying ‗radio waves transmit signal‘ and ignore the other aspects of radio signals such as signals through electronics before and after the radio waves are sent, and the fact that the original signal was a sound wave and the final product is a sound wave, such as music listened to on a radio (Lazebnik, 2002). In addition, other examples given in class may lend to the misconception. For example, some radio telescope dishes, even in current astronomy, have been construed to resemble ears and are sometimes even referred to as ―listening to outer-space‖. Others, perhaps unfortunately, have been identified by the colloquial term of ―horns‖, such as the ―horn antenna‖ used by Penzias and Wilson in the 1960s to map radio signals from the Milky Way leading to the discovery of the Cosmic Microwave Background.(APS website online). However, is it possible that some other explanations and demonstrations are reinforcing this misinterpretation rather than clarifying it? Education research indicates also that errors are often actualized not only from direct input but by organizing and reorganizing which initially may be fragmented (―knowledge in pieces‖) and re-evaluating misconceptions such as ―naïve theories‖ (Tuminaro & Redish, 2007; Bao & Redish, 2006; Disessa & Sherin, 1998, Carmazza et al., 1981; Etkina & Ruibal-Villasenor, 2008). Wave types might be considered in class in different contexts (such as a discussion of sound and then later light) that are never fully connected for students.

Purpose and methodology of survey The source, nature, and consequence of the described errors is explored in this article, which reports on a pilot study (via a paper survey) to physics students in several high schools. A short seven question multiple choice survey of students was prepared and distributed in the classroom by the high school physics teacher. The reasoning for the questions in the pilot survey was based on frequent multiple-choice questions types similar to those often asked/tested on light and sound to high school physics students with the goal to obtain data on student responses on this subject, following physics education research models (Ding & Liu, 2012). The objective was to test the idea that students may have trends in how they answer these multiple-choice questions. The survey was reviewed and approved by the University‘s six-scientist Institutional Research Review Board (IRRB) prior to dissemination and was also evaluated by an outside evaluator from the Social Work Department and one from the School of Education (with pre-college teaching experience) at our University for face and content validity in addition to bias. This study was intended as a first look at this problem and outside expertise was not sought in this initial study but future survey tools will be subject to more rigorous

scrutiny. Methodology in the selection of the high school students taking physics was based on availability of the test subjects via the cooperation of local high schools which included parental permission as all test subjects were minors. The high schools were surveyed after students had covered electromagnetic radiation in their spring semester (April), as verified by a survey to their teachers. Data was collected with a total of 225 students (that is, N=225, where N is a variable equal to the number of subjects) participating from 4 high schools (mix of public and private institutions) with a total of 7 different classes who participated in this survey. No record of who the teachers of these classes was made, nor if any of these teachers taught multiple courses that were surveyed. All the students in this study were from the same age group strata (high school juniors and seniors of ages 16-18 years old) with the data taken in only one particular year, in 2012; hence, this study is only representative of a single cohort based on age, year, and demographic location. Distribution to other cohorts was never initiated due to inherent concerns in the study as will be highlighted. The demographics of the students in the study were such that all were from central/southern New Jersey and were in basically the same economic strata: the average student was from a middle-class family not under financial stress. The students were taking a physics course from one of all levels of physics including general, college preparatory, honors and advanced placement (AP) and all were juniors or seniors in high school; no Physics First (freshman) classes were surveyed as these schools did not have that program. Students were not asked personal questions regarding family, wealth, gender identity, ethnicity, minority status or whether or not their parents were college graduates. Each student completed a paper survey in their physics classroom. The survey was administered in the selected high school physics classrooms by the high school physics teachers. These surveys were returned to the researchers and tabulated. This study could not be further stratified since the surveys were delivered back from the schools bundled without differentiation between classes; hence, the experiences due to a variety of physics class levels or teachers (who might teach various levels) could not be segregated for further analysis, although individual class sizes were small; and hence, statistically insignificant as stand-alone units. This surveyâ&#x20AC;&#x2DC;s results are presented mostly in a qualitatively-descriptive manner and the discussion of the survey study is meant to serve as a pilot for a possible larger group sampling, although the sample size was adequate, within common and appropriate apriori parameters (Apriori Calculator online). Even given the limitations of this study, some interesting trends can be noted. Details of the survey instrument are given in the next section along with the results.

Survey and results Table 1 is a summary presentation of all the results obtained in this study. Note that consideration of whether high school students would take such a survey seriously should be made (Kalat, 2010). We did make this consideration. Firstly, the survey was distributed in studentsâ&#x20AC;&#x2DC; physics class by their physics teacher, a place where students would be expected to answer questions thoughtfully, being in the classroom or laboratory where they had

learned the material. Secondly, all students answered all the questions in the survey, indicating that they took it seriously. Thirdly, Question #5, an easy stand-alone question, had a high percentage of correct answers. It was a question embedded deep in the survey so unlikely to be prompted by a teacher, and had an overall 92% correct percentage and only one class (High School 3) with only one notable lower percentage (72%), possibly not having covered this or emphasizing it less. Conclusions likely can be drawn that students took the survey seriously, noting that distractor answers in that question â&#x20AC;&#x201C; an energon is a make-believe particle, a pion likely would not (yet) have been discussed in a high school physics class, nor would a phonon. Table 1: Physics questions asked high school students (with percentages normalized to N=225) Note: numbers may not add to 100% as numbers were rounded.

# 1

(N=225)

High School 1 Percentage Correct (N=104)

Light is a ___ wave. a.) longitudinal b.) transverse

33% 67%

41% 59%

45% 55%

12% 88%

Radio waves are a form of sound waves. a.) true b.) false

60% 40%

57% 43%

48% 51%

64% 36%

70% 30%

Electromagnetic waves have zero __. a.) mass b.) wavelength c.) energy d.) frequency e.) velocity

83% 4% 3% 10% 0

77% 5% 5% 13% 1%

88% 5% 2% 5% 0

63% 0 9% 27% 0

91% 3% 0 6% 0

14%

18%

16%

21%

25%

30%

27%

11%

14%

16%

18%

Question Text (correct answer is underlined)

The electromagnetic spectrum, in order from lowest energy to highest. a.) x-ray,microwave, infrared, visible, ultraviolet b.) visible, infrared, ultraviolet, microwave, x-ray c.) x-ray, infrared, microwave,

Overall Percentage Correct

High School 2 Percentage Correct (N=43)

High School 3 Percentage Correct (N=11)

High School 4 Percentage Correct (N=67)

ultraviolet, visible d.) microwave, infrared, visible, ultraviolet, x-ray e.) microwave, x-ray, ultraviolet, visible, infrared

46%

37%

25%

80%

36%

Another name for a bundle of light is a a.) phonon b.) photon c.) proton d.) pion e.) energon

1% 92% 3% 2% 2%

0 92% 2% 2% 4%

2% 93% 2% 2% 0

0% 72% 9% 9% 0

0 95% 5% 0 0

Radio waves travel in air at a speed that is __ the speed of SOUND. a.) slower than b.) the same as c.) faster than

20% 44% 36%

22% 39% 39%

25% 35% 40%

27% 36% 36%

13% 57% 30%

Radio waves travel in air at a speed that is __ the speed of LIGHT. a.) slower than b.) the same as c.) faster than

71% 21% 8%

71% 18% 11%

65% 25% 5%

72% 18% 9%

70% 22% 8%

Note that a glaring 8% of the population surveyed said incorrectly that radio waves could travel faster than the speed of light, when in most high school classes, the speed of light is expected to be proclaimed to new learners as the universal speed limit. However, it should be noted that no collection of syllabi of the students was collected and hence no correlation made to what students were presented with which may have missing items from those usually expected, at least in this study.

The introductory survey question and implications It was desirable that the survey be short since the aim was to sample studentsâ&#x20AC;&#x2DC; knowledge and not have them realize that their answers were incorrect and to go back and change them. The survey was meant as an expression of student knowledge, and not a learning experience. The first question was

considered a ‗control question‘, to see if students remembered light was a transverse wave, and to determine what the overall rate of student correct answers would be to a technical course-based question. 1.) Light is a ________ wave. A) longitudinal –INCORRECT B) transverse - CORRECT Overall average results in student responses was tabulated with 32% incorrect and 68% answering correctly. However, before proceeding, let us also take a look at the performance of individual schools (See Fig. 1).

Figure 1: Question #1 results (“Light is a longitudinal/transverse wave”): Percentage correct vs. high schools surveyed.

Except for one school, the spread of answers was similar. The glaringly large percentage correct (at nearly 90%) on Question #1 for High School 4 (likely two classes but maybe more, at N=67), but lower scores on later questions may indicate that possibilities: 1.) topic of light had possibly just recently been covered; but possibly radio waves were not covered or at least not in much depth, 2.) possibly that the teacher(s) at this school is (are) doing a better job overall in achieving better results in student learning on that topic, and/or 3.) teacher(s) may have prompted the answer to Question #1 to the students, for example, such as by saying ―doing this survey

remember how we covered…,‖ or some other similar, even more outright, prompting. However, if the answer had ‗just been given‘, the return rate would likely have been 100%. It is a topic for future studies. Having an initial question to buffer was not intentional but in retrospect seemed good practice, especially as teachers would be unlikely to prompt more than one question. Note that it might be difficult to get a third party to administer the survey. Excluding the results from High School 4, the percentage rate of correct answers was a more moderate 58% correct and more consistent, within ±3 percentage points. Still this is a discouraging statistic, considering that all students in this study would have recently covered this topic. However, the unfamiliarity of the words (‗longitudinal‘ and ‗transverse‘) and students only recently being introduced to them, and not seeing them often might lend to less accuracy in answering the question.

Rate of students incorrectly identifying radio waves as sound The questions of most interest for this article in which physics students incorrectly identify radio waves as sound are Questions 2, 6, and 7. Logically, a percentage of students answering Questions 2 incorrectly might also tend to answer Question #6, and Question #7 incorrectly but consistently (that radio waves were sound). Figure 2 highlights these answers per high school. Question #2: Radio waves are a form of sound waves. True: N=134 (60%) incorrect False: N=91 (40%) correct

Figure 2: Percentage correct VS. Questions regarding Radio Waves (as light)

Averaged results showed 60% of these high school physics students were identifying radio waves as sound. However, there also exists the possibility that students are mistaking ―radio waves‖ for ―radio signal‖, as mentioned previously, which would, as heard from a radio, indeed be sound (Lazebnik, 2002). Also, out of those 134 incorrect answers, 82 also answered Question #6 that ―radio waves travel at the same speed as sound‖ – a percentage rate of 82/134X100= 61% of those students who specifically answered Question #2 that ‗radio waves were sound‘ who were sticking with that idea that radio waves travel at the speed of sound in Question #6, higher than the overall percentage at 44%. Of the 82 who answered that radio waves were sound in Question #2 and Question #6, there were 73 who answered in Question #7 that ―radio waves travelled less than the speed of light‖. Hence 89% of those seemed to be sticking to the idea that radio is sound. That is 32% of the entire population (N=225) surveyed, a possible indication that this portion of the students were again following through on the idea that radio is sound. This seems to be defined as a

―naïve theories‖ as discussed in the background section (Tuminaro & Redish, 2007; Bao & Redish, 2006; Disessa & Sherin, 1998, Carmazza et al., 1981; Etkina & Ruibal-Villasenor, 2008). Students may, once committed, not realize that they are being deceived by what could be termed a sensory illusion (Tabor-Morris, 2015) and/or be over-correlating their experience of everyday experiences in a way that is not physical (Caramazza et al., 1981).

Students correctly answering and following though that radio waves are light Of the 91 students who answered Question #2 correctly that radio waves were light, 34 also correctly answered Question #7 that ―radio waves travel at the speed equal to the speed of light‖. So, it appears 34/91X100= 37% of those students followed through with their idea that radio waves are light into that question. The index of refraction effect on the speed of light would be negligible, something some students might (or might not) have considered, in which then, assuming students follow through, would raise the percentage of correct answers to – that is, including a and b as correct – from 37% to (89 answering either a or b) a total of 97%. However, it is unclear that students would be thinking along those lines. This is clearly an area for future research and survey modification. Correlating this back to Question #6, if students were following the logic that radio is light, they would have answered that radio waves travel faster than sound. Of the students who answered that radio waves travel at the speed of light (only), 28 or 82% indicated that radio waves travel faster than sound (assumedly at the speed of light, as they answer in the next question). This is 12% of the entire student population surveyed, a lower overall percentage of those who appeared to follow through on the logic for radio waves as sound.

Possible changes to questions in survey in future research Question 6 and 7 might have the ―in air‖ portion of the phrase also put at the end of the sentence in both cases so that any confusion regarding the effects of index of refraction could be eliminated. There also exists the unexplored possibility that some students may be identifying radio waves, instead of electromagnetic waves, but instead as AC-type electrical signals. Question #2 could be expanded to include (c.) electrical signal and/or a Question #8 could be added that distinguishes students‘ thoughts about which is fastest could be added: Which is fastest? Speed of a.) sound in air, b.) light in vacuum, c.) electricity in metal, expounding on past problems seeking to be addressed. Noting that electricity travels faster than sound but slower than light in vacuum (Halliday, 2007), this question might allow students to better qualify radio waves or at least distinguish them from other phenomenon. Student attitudes among a particular cohort, with participants representing a single strata and age but inspires the research question whether the results could be applied more broadly to physics, such as on the undergraduate level. Giving the same tests to the same class twice within a short period of time is used to judge the stability of the testing (Engelhardt, 2009). This was not completed in this case

as this was always intended to be a pilot study and questions were not Leichardt gray-scale, and therefore less likely to be answered differently if asked consecutively to the same audience, with the assumption being that learning is stable, which would be a separate study. Cohort and longitudinal studies with pre- and post-tests are also suggested for the future. In addition, the responses of students in southern New Jersey (mostly suburban) may or may not be representative of all students nationally or internationally. There is also less racial and/or economic diversity than in many other populations in the United States and elsewhere. However, given that the survey was taken in a region of the country that was somewhat culturally homogenous and overall non-stressed financially, these stressors and differentiators most likely were not any significant factor in, for example, student-to-student interactions. For this study, a cross-sectional sampling across many age groups could not be made since most people take a physics class in a short period/time of their life only, although college students could also be studied in the future and may be of interest due to their further maturity and possibility of having taken a physics course in high school. A longitudinal study was also not possible for this group to investigate if their knowledge and attitudes carried though to future classwork. This was due to the fact that the students were not identified as individuals and are in high school only a short period of time before graduation. Hence after high school graduation students and their responses would not be traceable as either individuals or a unit, since limited communication is maintained by high school graduates with their alumni schools. In the future, a larger sampling of students could be made and several years of data from students who are juniors/seniors in those years. Also, students on the college level were not tested but would be of interest.

Conclusion and Suggestions Many students have difficulty correctly identifying radio waves as transverse light waves. The primary purpose of this study was to highlight that the problem exists on a statistically significant level and is worthy of further study and active remediation by teachers. Based on the results of the survey-tool in this pilot study, a large percentage of students appear to conclude that, since they experience radio broadcasts as sound, then sound waves are the actual transmission of radio signals, at least more so than that for light. That ―things are not always what they seem‖ is something scientists are used to analyzing. How can teachers prepare students for conditions when what seems initially obvious is more subtle? Teachers‘ awareness and addressing this problem directly may be essential to assisting student learning. Note that it could be easy to see that students might interpret the term "radio waves" as waves emitted by a (handheld) radio - which would be sound, but would be more concerning is if the misconception extended to students believing that radio broadcasts (from base) are transmitted as sound (and somehow amplified by a radio). A future survey could address this issue. For the purposes here, it is suggested that teachers alert students to this often made error is one possible method to help remediate student confusion.

Can teachers better identify to students that a radio is a ―translator box‖ that translates radio signals into sound? Questioning students on the steps for radio communication might also be helpful in sorting out their understanding of the process. Also helpful may be identifying to students that radio waves are electromagnetic radiation in a range invisible to the eye, in a sense, ―invisible light‖ with better distinction between radio signals and radio waves? Radio stations are continually broadcasting radio waves on many frequencies (―We are now awash in radio waves but do you hear anything? No.‖). Cosmic sources such as the sun create what is interpreted by radios as static. People simply cannot sense the radio waves using our bodily senses. In addition, there are also indications that students need to take in physics terms multiple times for physics jargon to be absorbed into the students‘ vocabulary and understanding. In addition, repetition of phrasing may be necessary for students to be able to distinguish scientific and everyday meanings for the same words. For example, the survey results suggest that terms such as ‗longitudinal‘ and ‗transverse‘ need to be reviewed multiple times by teachers for students, but that term-sticking is achievable as seen in the survey with the term ‗photon‘, not a term used in everyday language. Fragmenting of knowledge, as addressed in the background (Tuminaro & Redish, 2007; Bao & Redish, 2006; Disessa & Sherin, 1998, Carmazza et al., 1981; Etkina & RuibalVillasenor, 2008), may also be occurring from the time students are first presented with wave types until they study light. Frequent quizzes that keep students on track might be very valuable during the study of sound and light, including questions that tie past concepts with present instruction. Future studies beyond this pilot study are suggested. This would entail revaluating the questions for possible bias as well as content. Another item for future studies would be to check syllabi between classes data was taken for to assure consistency of topics. Another issue may lie in the fact that students may not even be able to identify radio transmission towers. Why? Many may not even experience ―radio‖ except streaming over the internet. While examples of transmission of cell phones and cell phone towers, might be more effective for this current younger generation, as many are cellular phone device users, most of these transmissions fall into the microwave region. Microwaves constitute another invisible portion of light‘s electromagnetic spectrum and are perhaps difficult to address without discussing microwave ovens, another common appliance to students. Confusion can ensue regarding applications of these as well: relaying communications versus heating of food. This would merit another study. Confusions in astronomy and physics with sound and light, such as the use of terms ‗horn telescopes‘ mentioned in the background should also be considered with care. Consequences of not understanding the nature of radio waves would also include students being unable to ascertain secondary effects such as absorption and polarization as well as applications of communications such as AM and FM. While some important questions were addressed in this study, a number of others are still unanswered, such as the construction of questions that will not ―lead‖ students to answers, but also will not confuse them.

Acknowledgments Gratefully acknowledged is the opportunity afforded at the American Physical Society (APS) National Meeting, Baltimore, Maryland March 2013 National Meeting to present preliminary analysis of the data in this study.

References Bao, L. & Redish, E. F. (2006). Model analysis: Representing and assessing the dynamics of student learning, Physical Review Special Topics-Physics Education Research, 2(1), 010103. Bardar, E., Prather, E., Brecher K., Slater T. (2005). The Need for a Light and Spectroscopy Concept Inventory for Assessing Innovations in Introductory Astronomy Survey Courses, Astron. Educ. Rev. 4 (2) 20, doi: http://dx.doi.org/10.3847/AER2005018 Berger, A., (2015). Interviews with Upper-Level Undergraduates about Representations of Electromagnetic Plane Waves, presented at the Physics Education Research Conference 2015, College Park, MD, doi: http://www.compadre.org/Repository/document/ServeFile.cfm?ID=13836&D ocID=4254 Caramazza A., McCloskey M. & Green B. (1981). Naïve beliefs in ―sophisticated‖ subjects: Misconceptions about trajectories of objects, Cognition, 9(2), 117-123. DeVries, M. (2001). The history of industrial research laboratories as a resource for teaching about science-technology relationships, Research in Science Education, 31(1), 15-28. Ding L. & Liu X. (2012). Getting Started with Quantitative Methods in Physics Education Research online at: <http://www.compadre.org/per/items/detail.cfm?ID=12601> [Accessed 25 July 2017] Disessa A. A. & Sherin BL (1998). What changes in conceptual change? International journal of science education, 20(10), 1155-1191. Engelhardt P. V. (2009). An Introduction to Classical Test Theory as Applied to Conceptual Multiple-choice Tests in Getting Started in PER, edited by C. Henderson and K. A. Harper, American Association of Physics Teachers, College Park, MD Reviews in PER Vol. 2, doi: http://www.percentral.org/items/detail.cfm?ID=8807 Etkina, E., Karelina, A. & Ruibal-Villasenor M (2008). How long does it take? A study of student acquisition of scientific abilities, Physical Review Special Topics-Physics Education Research, 4(2), 020108. Finkelstein, N., Adams, W., Keller, C., Perkins, K. & Wieman, C., (2006). High-tech tools for teaching physics: The physics education technology project Journal of Online Learning and Teaching 2(3) Halliday, S. (2007). Introduction. In Science and Technology in the Age of Hawthorne, Melville, Twain, and James, Palgrave Macmillan US., 1-15. Hewitt, P., (2007) Next Time Questions- Radio Waves, online (originally published in The Physics Teacher journal), doi: http://www.arborsci.com/next-timequestions Kalat, J. (2010). Introduction to psychology, Independence, KY: Cengage Learning, 37-38. Landt, J., (2005). The history of RFID IEEE potentials, 24(4), 8-11. Lazebnik, Y. (2002). Can a biologist fix a radio?—Or, what I learned while studying apoptosis, Cancer Cell, 2(3), 179-182. McGinnis, J. & Oliver, J. S. (1998). Teaching about Sound: A Select Historical Examination of Research, Science & Education, 7(4), 381-401.

Neumann, S. & Hopf, M. (2012). Students‘ conceptions about ‗radiation‘: Results from an explorative interview study of 9th grade students, Journal of Science Education and Technology 21(6) 826-834. Perkins, K., Adams, W., Dubson, M., Finkelstein, N., Reid, S., Wieman, C. & LeMaster, R. (2006). PhET: Interactive simulations for teaching and learning physics, The Physics Teacher, 44(1) 18-23. Plotz, T. & Hopf, M. (2016). Students misconceptions about invisible radiation. In Electronic Proceedings of the ESERA 2015 Conference, Science Education Research: Engaging learners for a sustainable future, 95-100. Rego, F. & Peralta, L. (2006). Portuguese students' knowledge of radiation physics, Physics Education, 41(3), 259. Tabor-Morris, A. E. (2015). Thinking in terms of sensors: personification of self as an object in physics problem solving, Physics Education, 50(2), 203-209. Tuminaro, J. and Redish, E. F. (2007). Elements of a cognitive model of physics problem solving: Epistemic games. Physical Review Special Topics-Physics Education Research 3.2 020101 Wise, K. C. (2006). Can You Hear Them Now?: Investigating Radio Waves Science Activities. Classroom Projects and Curriculum Ideas, 43(3), 23-30. APS Website: The Large Horn Antenna and the Discovery of Cosmic Microwave Background Radiation, doi: www.aps.org/programs/outreach/history/historicsites/penziaswilson.cfm A Priori Sample Size Calculator, doi: http://www.danielsoper.com/statcalc/calculator.aspx?id=1

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 8, pp. 51-64, August 2017

Impact of Teaching Attitudes and Behaviors for Learning on the Reading Achievement of Students Falling Behind Michael E. Bernard Melbourne Graduate School of Education University of Melbourne

Abstract. This research evaluates the impact of a teacher professional development program, â&#x20AC;&#x153;Attitudes and Behaviors for Learningâ&#x20AC;? (AB4L) on the achievement of students struggling with reading. Two primary school teachers from two economically disadvantaged schools received three half-day training sessions in practices to teach students positive attitudes and behaviors for learning, which they implemented during literacy classes. 98 students were taught the AB4L program while 86 students were not. Results include: (a) Student- and teacher-rated learning behaviors were positive correlated with objective reading performance; (b) A significant benefit of AB4L on the reading performance of those students who scored in the lower 50 percent of their class on a reading comprehension survey; (c) Students in classes where AB4L was implemented who showed improvements in reading comprehension also showed increases in behaviors for learning. A recommendation is that teacher professional development programs should incorporate positive attitudes and behaviors for learning. Keywords: Reading Achievement; Reading Improvement; Reading Teaching; High Risk Students; Student Learning Behavior.

Introduction This research seeks to illuminate the extent to which primary-age students who have fallen behind their classmates in reading display delays in the development of attitudes and behaviours for learning that have been found to contribute to student engagement and achievement. A second question addressed in the present research is whether a professional development teacher training program that up-skills teachers in ways to present and strengthen student attitudes and behaviours for learning during classroom literacy lessons results in an improvement in the achievement of students struggling with reading. As exemplified by the Department of Education and Training investment in the National Partnerships for Low SES Schools. Literacy and Numeracy and Improving Teacher Quality (Australian Department of Education and Training, 2014), educational policy continues to explore innovative and effective ways to

assist schools in helping students meet basic literacy standards and to close the achievement gap for the disadvantaged. This study adds to the „best practice‟ literature on the teaching of reading and the link between students‟ attitudes and social and emotional learning skills which are vital for student engagement and the development of their literacy skills (Bernard, 2011). Student characteristics fundamental to engagement and achievement (e.g., Lee, 2014) have been termed learning-to-learn skills (Barnett, et. al., 1996), academic self-regulation skills (Zimmerman, 1990), learning behaviors (McDermott, 1999), academic enablers (DiPerna & Elliott, 1999), approaches to learning (Rock & Pollack, 2002), social and emotional learning competencies (Collaborative for Social and Emotional Learning) and student dispositions (Hattie, 2013). Specific learning behaviors include goal setting, self-monitoring, time planning, social skills including seeking help when needed, engagement, confidence, persistence, self-talk for managing frustration, flexible methods for learning as well as positive attitudes towards learning including high selfefficacy beliefs and intrinsic interest in learning (McDermott, 1999). Research into the characteristics of students with difficulties in reading reveals delays in the development of self-regulatory learning behaviors such as setting and achieving learning goals, monitoring success, the failure to use selftalk to manage anxiety and frustration of completing difficult learning tasks as well as a range of negative attitudes towards themselves and learning (e.g., Vaughn, & Broadman, 2007). Research has examined gender differences in students‟ use of behaviors for learning with the advantage being demonstrated by girls. For example, Duckworth and Seligman (2006) discovered gender differences in favour of girls in self-discipline and self-control. Gender differences in behaviors for learning may depend on the academic domain (e.g., Pokay & Blumenfeld, 1990). Research continues to accumulate demonstrating the effects of noncognitive and linguistic competencies on student achievement (e.g., Durlak, et. al., 2011). Of particular relevance to this research is a study (Ashdown & Bernard, 2012) that investigated the effect on reading achievement of a social and emotional learning skills curriculum designed to teach positive attitudes and behaviors for learning and well-being. The lessons were designed to teach young children confidence, persistence, organization and resilience including a range of positive attitudes (e.g., optimism, self-acceptance, internal locus of control for learning). The lessons were taught three times a week, supported by a variety of additional social and emotional teaching practices. The results indicated that the program increased reading achievement for the lower achieving grade 1 students. Bernard (2006) proposed that it is time that we teach social-emotional competence for learning as well as we teach academic competence. Five teaching practices are contained in the Attitudes and Behaviours for Learning (AB4L) (Bernard & Milne, 2016) professional development program that was evaluated in this research. Practice 1. Prepare Students to Begin Literacy Lesson with a Positive Mindset. For many years (e.g., see Bloom, 1976 “Human Characteristics and School Learning”) researchers have identified student attitudinal dispositions towards school and specific classes as they begin a learning task as a major factor in their achievement. Hattie‟s (2013) meta-

analysis of over 800 studies on student achievement reported the effect size of students‟ disposition to learn as .61. Practice 2. Share with Students the Goals of the Literacy Lesson, Have Them Set Goals, Monitor Progress and Revise Learning Methods and Behavior and Practice 3. Communicate Behavior-Specific Feedback for Learning. Hattie‟s (2013) meta-analysis also revealed a large effect size of teachers helping students set goals for learning and providing positive and negative feedback to students on their achievement as well as on their use of learning strategies (also, see Locke & Latham, 2002; Schunk, 2003). Practice 4. Identify and Discuss Behaviors for Learning. Research shows student learning behaviors contribute to school readiness, literacy and mathematics outcomes (e.g., DiPerna, Volpe & Elliott, 2002; Fantuzzo, Perry, & McDermott, 2004; Green & Francis, 1988; McDermott, 1984; McWayne, Fantuzzo, & McDermott, 2004). Practice 5. Discuss Positive (and Negative) Self-Talk for Learning. The selfregulatory nature of inner speech and self-talk (Vygotsky, 1934/86) has been found to assist students in guiding their thinking and learning (Kross, et. al., 2014; Winsler & Naglieri, 2003; Winsler, Manfra & Diaz, 2006; see Hardy‟s 2006 review of self-talk literature). Self-talk is a major component of cognitivebehavior therapy, a „best practice‟ intervention for young people with emotional and behavioral problems that interfere with their learning (Bernard, 2006). Moreover, students who acquire self-regulatory skills experience improved academic achievement and increased self-efficacy (e.g., Zimmerman & Schunk, 2001). The present project posed the following research questions. 1. Are students‟ behaviors for learning associated with their reading performance? 2. Are there gender differences in behaviors for learning? 3. Will the AB4L program have a positive effect on students‟ behaviors for learning? 4. Will the AB4L program have a positive impact on students‟ reading performance? 5. Will students in the bottom 50 percent of their class in reading performance who receive the AB4L program show greater improvement in their reading than students in the bottom 50 percent of their class in reading performance who do not receive the program? 6. Will the AB4L program have a different impact on the behaviors for learning and reading performance of boys versus girls? 7. Do students who show improvements in their behaviors for learning show concomitant changes in their reading performance?

Method Participants The study used a pre-post treatment-control quasi-experimental design to evaluate the effectiveness of AB4L. Two school principals located in a rural community in Victoria agreed to have the program implemented in their schools. In School A, the students in two composite grade 3/4 classes were chosen to receive the AB4L program while in School B, the students in two composite grade 5/6 classes received the AB4L program. For purposes of comparison, in School A, the students in two composite grade 5/6 classes did

not receive the AB4L program while in School B, students in two composite grade 3/4 classes did not receive the AB4L program while students in two composite grade 5/6 classes received the AB4L program. The total number of students receiving the program was 98 (51% female) while the total number of students not receiving the program was 86 (45% female). Measures Teacher Ratings: Learning Behaviors Scale (LBS). The LBS (McDermott et al., 1999) is a standardized 29-item teacher-completed rating scale. Items are rated on a 3-point scale (0 = doesn't apply, 1 = sometimes applies, 2 = most often applies) indicating the presence of the behavior over the past two months. Scales were scored as the mean of the items. The measure included an overall score and four subscales measuring motivation, attitude, persistence, and strategy. Internal consistency coefficients are high for the overall (.89-.92) and subscale scores (.70.87) both overall and for age, gender, and ethnic subsamples, and stability coefficients across a 2-week interval were strong (.91-.94; McDermott, 1999). Student Ratings: Student Learning Behaviors Survey (SLBS). A new rating scale, the Student Learning Behaviors Survey, was developed for this study and was designed to measure student self-perceptions of attitudes and behaviors associated with their engagement during literacy instruction (reading and writing). Items were rated on a 2-point scale (0 = disagree, 1 = agree). Questions were developed by the first author of this study that examined student self-perception of their confidence, persistence, goal orientation, teamwork, disorganization and worry associated with literacy instruction. The initial survey of 18 items asked students to agree or disagree with a series of questions; for example, “I get easily tired when I read or write,” “I distract others during reading or writing time.” See Table 2 for complete item text. The SLBS had a mix of positively and negatively worded items. A maximum likelihood factor analysis was conducted on the preintervention responses of all participating students to the initial set of 18 items. Examination of the scree plot suggested one main factor, and a second smaller factor; variance explained by the first 8 unrotated components was 23.6, 10.9, 7.3, 6.6, 6.2, 5.7, 5.3, 4.9. The two items that loaded highly on the second factor (“17. I could do a lot better in my reading”; and “18. I could do a lot better in my writing”) were removed as they appeared to combine both perceptions of low ability with a perceived ability to do better. After removal, the scree plot showed clear support for a large first factor with variance explained by the first 8 unrotated components of 23.6, 8.4, 7.3, 6.4, 5.9, 5.3, 5.1, and 4.2. While it would be possible to attempt to further explore subscales on this measure, the scale composed of the first factor reflected the most reliable and systematic source of variance, and provides a parsimonious representation of overall positive attitudes and behaviors regarding reading and writing. The retained items and their factor loadings are shown in Table 1. The test was scored as the mean of the 16 items after item reversal. Cronbach‟s alpha reliability for the Student SLBS was .79 at Time 1 and .73 at Time 2. The correlation between pre and post intervention scores was r = .61.

Table 1: Item Loadings for 16 Retained Items of the Social Learning and Behavior Scale _______________________________________________________________________ Item Loading _______________________________________________________________________ 1. I get easily tired when I read or write. (R) -.36 2. I sometimes forget to bring to class things I need to learn (pencils, paper, book). (R) -.33 3. I distract others during reading or writing time (R) -.43 4. At the beginning of a lesson (reading, writing), I set a goal for what I want to learn. .20 5. I put up my hand to answer a difficult question (reading, writing). .33 6. I like to read. .47 7. I like to write. .33 8. I worry a lot about my schoolwork (reading, writing). (R) -.38 9. When reading or writing gets really hard, I can give up before getting it done properly. (R) -.73 10. I can do schoolwork that is hard to do (reading, writing). .41 11. When I do not understand something (reading, writing), I give up easily. (R) -.74 12. I am a good listener when working in my reading or writing groups. .49 13. I help others when they do not understand something (reading, writing). .43 14. I get distracted when I am doing my reading and writing. (R) -.43 15. It takes a long time for me to settle down to do my reading and writing. (R) -.47 16. I lose confidence when reading or writing. (R) -.62 _______________________________________________________________________ Note. Reversed items are indicated by (R). Loadings are based on a one factor maximum likelihood factor analysis. Objective Reading Performance. The Victorian Curriculum and Assessment Authority‟s On Demand Computer Adaptive Reading Test (2006, 2010) was used to assess students‟ level of reading comprehension and, specifically, the extent of development of reading comprehension competence over the three and a half month period of this evaluation project. This 30-item test presents 10 sub-test packets of three reading items to students. Each item is designed to test a specific skill associated with reading comprehension. Some examples of reading comprehension skills assessed include analyze imagery in a text, analyze plot in a text, analyze point of view in a text and analyze setting in a text. The test provides a standard score that corresponds to grade-level performance relative to AusVELS standards. Across the whole sample reading performance scores were correlated r = .79 across the two time points. Attitudes and Behaviors for Learning Program The AB4L program provides teachers with explicit instruction in the use of five practices that can be employed throughout a reading lesson to teach students various attitudes and behaviors for learning (see Table 2). Teachers were trained to integrate the five AB4L practices throughout the different components of a literacy lesson (before the lesson begins, during whole class, teacher-led instruction, during small group/dyadic/individual work, at end of literacy session –reflection on learning, assignment of literacy homework).

Table 2: Teaching Practices for Integrating Attitudes and Behaviors for Learning into Different Components of Literacy Instruction _______________________________________________________________________ 1. Prepare Students to Begin Literacy Lesson with a Positive Mindset. At the beginning of a literacy lesson, help students maintain a positive focus by reviewing different positive attitudes. 2. Share with Students the Goals of the Literacy Lesson, Have Them Set Goals, Monitor Progress and Revise Learning Methods and Behavior. Regularly, ask students to set goals (what they want to learn; mark they will receive). Spell out the different concepts and skills/strategies that will be taught in the literacy lesson. At the end of the class, have students reflect on goal attainment. Based on this feedback, encourage students to modify their approach to learning. 3. Communicate Behavior-Specific Feedback for Learning. Acknowledge individual and groups of students who display different „behavior for learning‟ by providing feedback that names/describes the behavior and attitude they have demonstrated in a literacy activity 4. Identify and Discuss Behaviors for Learning. Discuss different behaviors that students should practice/use that can help them to be self-managing and engaged during a literacy activity. 5. Discuss Positive and Negative Self-Talk for Learning. Describe and model positive and negative self-talk that that students can use to remain calm when feeling frustrated or overwhelmed by a learning activity. _______________________________________________________________________ Procedure During the two weeks before the first of three teacher-training sessions, collection of evaluation data occurred. All teachers (those to receive training in the AB4L program; those who did not receive training) completed the Learning Behaviors Scale for each of their students. Teachers had all students complete the Student Learning Behaviors Survey. All students also completed the On Demand Computer Adaptive Reading Test. The teacher training sessions took place over a three- and a half-month period. The sessions were conducted by an experienced classroom teacher/literacy coordinator. Each session took approximately three hours. Evaluation Data Collected (2 weeks before commencement) Week 1. Teacher training Session1. Week 3. Classroom observation of teachers by trainer. Week 6. Teacher training Session 2. Week 8. Classroom observation of teachers by trainer. Week 11. Teacher training Session 3. Week 13. Classroom observation of teachers by trainer. Evaluation Data Collected (2 weeks after commencement) During weeks 3, 8, and 13, the trainer conducted a classroom observation of each participating teacher as the teacher taught a literacy lesson. The purpose of the observation was for the trainer to determine the extent to which the teacher was implementing AB4L. After each observation, the trainer would

summarize in an email the findings in terms of each teacherâ&#x20AC;&#x;s strengths and areas for improvement. To address the issue of potential bias of having the developer of the AB4L program involved in its evaluation, the program was implemented and surveys administered by a literacy coordinator of a nearby school whose livelihood did not depend on finding significant effects of the AB4L program. Additionally, the second author of this research paper who conducted all aspects of data entry and statistical analysis had no prior familiarity with the program and has no vested interest in the results.

Results Correlations at Pre-Intervention Pearson correlations using pre-intervention measures indicated that objective reading performance was positively correlated with both student-rated learning behavior (r = .34, p < .01) and teacher-rated learning behavior (r = .45, p < .01). All subscales of teacher-rated learning behavior were correlated with objective reading performance (correlations ranged from r = .32 to r = .53). Finally, student-rated learning behavior was positively correlated with teacher ratings (r = .53, p < .01). With regards to gender differences, an independent groups t-tests indicated that teachers rated the learning behavior of girls more highly than boys (d = 0.46, p = .003). However, no significant differences were obtained for student-rated behavior (d = 0.24, p = .14) or objective reading performance (d = 0.10, p = .53). Effect of the Teaching Intervention (AB4L) Means and standard deviations for pre- and post-intervention scores are shown in Table 3. Before assessing the effect of the intervention, we first examined assumptions. There were no significant differences between the control group and intervention group at pre-intervention on any of the outcome measures. However, although not a significant difference, the intervention group did score about a third to a half standard deviation lower on learning behaviors at baseline. Standard deviations were similar across time points and groups. In terms of normality, objective reading performance (skew at preintervention = 0.30) had minimal skew, whereas student-rated learning behaviors (skew at pre-intervention = -0.67) and teacher-rated learning behaviors (skew at pre-intervention = -0.87) were negatively skewed reflecting a tail of particularly poor performers. Given the moderate to large sample size, the inferential tests used are robust to the presence of this mild skewness.

Table 3. Means and Standard Deviations for Students Who Did and Did Not Receive the AB4L program at Pre-Test and Post-Test on Main Outcome Variables Pre-Intervention Control

Post-Intervention Interven.

Control

IntInterven.

Variable (scale range) M (SD)

M (SD)

SLBS (0 - 1)

0.75 (0.21)

0.70 (0.22)

0.75 (0.18)

0.83 (0.17)

Reading (1 - 10)

3.12 (1.19)

3.18 (1.11)

3.26 (1.26)

3.44 (1.06)

LBS Total (0 - 2)

1.58 (0.39)

1.43(0.49)

1.75 (0.29)

1.73 (0.39)

LBS Strategy (0 = 2)

1.74 (0.36)

1.62 (0.45)

1.82 (0.28)

1.80 (0.38)

LBS Motivation (0 - 2) 1.55 (0.47)

1.26 (0.57)

1.71 (0.34)

1.70 (0.44)

LBS Attitude (0 - 2)

1.63 (0.42)

1.51 (0.54)

1.83 (0.26)

1.79 (0.38)

LBS Persistence (0 - 2) 1.48 (0.51)

1.27 (0.65)

1.62 (0.43)

1.66 (0.51)

Note. SLBS = Student Learning Behavior Scale (Student-Rated); LBS = Learning Behavior Scale (Teacher Rated). Control group did not receive the AB4L intervention, whereas the Intervention group received the AB4L intervention. To assess the effect of the teaching intervention, a gain-score approach was adopted. This involved first computing change scores for each outcome measure calculated as post-intervention score minus pre-intervention score. Then, independent groups t-tests were performed on these change scores with intervention group as the independent variable. This approach is statistically equivalent to examining the interaction effect in 2 by 2 mixed ANOVA (Knapp & Schafer, 2009). To quantify the size of the difference in improvement, the effect size measure discussed in Morris (2008) was used. This is the pre-post equivalent of standardized mean difference (i.e., Cohen's d) and represents the standardized mean difference in change scores. Students who received the program showed significantly greater increases compared to the control group in teacher-rated learning behaviors (d = 0.30, p = .03), student-rated learning behaviors (d = 0.55, p < .001), but no significantly greater increase in objective reading performance (d = 0.11, p = .32). Because the research uses a quasi-experimental design, we also examined whether the effects were maintained using an ANCOVA approach to assessing the effect of the intervention (for a discussion of this issue, see Knapp & Schafer, 2009). This involved running a linear model predicting time 2 outcome scores from condition (intervention or control) and covarying for time 1 outcome scores. Using this alternate approach to assessing the effect of the intervention, the effect of student rated learning behaviors was still highly significant (p < .001), and the effect on objective performance was still non-significant (p = .26). However, the effect on teacher-rated learning behaviors changed somewhat whereby the effect was only statistically significant for persistence (p = .03), and was non significant for total (p = .25), strategy (p = .09), motive (p = .12), and attitude (p = .58). This difference in results between the ANCOVA and change

score approach is an instance of Lord's paradox. It presumably arises for teacherrated behavior, because scores were slightly lower, albeit non-significantly, in the intervention group at pre-intervention. This makes it less likely that the intervention group will have larger covariate adjusted scores, thus making it harder to get a significant ANCOVA. Alternatively, under some mechanisms of change, it makes it somewhat easier to get a significant change score. To assess whether the intervention was particularly effective for lower performing students, a further analysis was conducted of students in the lower half of their class in their reading scores at pre-intervention. Using the above mentioned t-test on change scores, for these students, there was significantly greater increases in objective reading performance in the intervention group (Pre-Test, M = 2.34, SD = 0.74; Post-Test, M = 2.88, SD = 0.65) relative to the control group (Pre-Test, M = 2.07, SD = 0.51; Post-Test, M = 2.29, SD = 0.62), d = 0.42, p = .03. To assess gender differences in the effect of the AB4L program, an ANOVA was performed examining the gender by intervention group interaction on change scores for teacher-rated behavior, student-rated behavior, and objective reading performance. There was no statistically significant evidence for differential effects of the AB4L program for boys and girls (all p's > .05). Correlation of Improvements in Rated Behavior and Objective Performance To assess whether students who show improvements in their learning behaviors show concomitant changes in objective reading performance, correlations between change scores separately for the two conditions were calculated (see Table 4). Change scores were calculated as post-intervention minus pre-intervention scores, such that a positive change score indicates that the student showed higher scores after the intervention (e.g., better learning behaviors or improved objective reading performance). Correlations of change scores of teacher-rated behavior and student-rated behavior with change scores for objective reading performance were positive and significant in the intervention group but not the control group. Thus, it can be seen that for students receiving the AB4L Program, those who showed increases in their student-rated and teacher-rated learning behaviors tended to show improvements in objective reading performance. However, a test of significant differences between independent correlations using Fisherâ&#x20AC;&#x;s r to z transformation was performed (for formulas, see Cohen et. al., 2003) indicated that the differences between control and intervention group correlations were not statistically significant.

Table 4. Correlation of Change Scores on Teacher and Student Rated Learning Behavior with Change Scores on Reading Performance for Intervention and Control Groups

Change in SLBS Change in LBS Total Change in LBS Strategy Change in LBS Motivation Change in LBS Attitude Change in LBS Persist

Change in Objective Reading Performance Control Intervention .03 .22 .10 .27 .06 .24 .07 .27 .10 .24 .03 .13

Note. Values in the table are Pearson's correlations between change scores (postintervention minus pre-intervention) calculated separately for control and reading intervention groups. For example, a positive value indicates that increases in self- or teacher-rated behaviors are correlated with increases in objective reading performance. SLBS = Student Learning Behavior Scale (Student-Rated); LBS = Learning Behavior Scale (Teacher Rated). Correlations larger than .22 are statistically significant at the .05 level and are shown in bold. Discussion This investigation examined the extent to which schools, especially those with high proportions of students from socially and economically disadvantaged backgrounds, should concern themselves with ensuring that the set of student characteristics referred to as attitudes and behaviors for learning should be an essential aspect of literacy teaching practice. Baseline Correlations and Gender Differences Consistent with previous research (e.g., Rock & Pollack, 2002), objective reading comprehension was correlated with teacher and student ratings of learning behaviors. These correlations may reflect both the benefits of the learning behaviors as well as a general sense of efficacy in performance that may be induced by the teaching environment that included opportunities for feedback and peer comparison. Teachers rated girls higher in behaviors for learning than they did boys confirming previous findings of gender differences (e.g., Duckworth & Seligman, 2006; Schaeffer, 2004). Of interest is that when student self-perceptions were examined, no gender differences were found. This inconsistency with teacher ratings may be due to a tendency of boys to provide unrealistic ratings of their learning behaviors resulting in the elevation of scores on the Student Learning Behaviors Scale. One implication of this finding is that teachers may need to be more explicit in providing boys with feedback concerning their use and non-use of various attitudes and behaviors for learning. Effect of Intervention The AB4L program had a positive impact on the behaviors for learning of students who received the program. This finding that behaviors for learning are teachable is supported by extensive previous research (McDermott, Leigh, & Perry, 2002; McDermott, Mordell & Stoltzfus, 2001). A novel aspect of AB4L is

that it is not a stand-alone program taught where students are taught foundational positive attitudes and learning behaviors apart from academic instruction. Integrating the teaching of attitudes and behaviors for learning as a part of literacy instruction is likely to produce a much stronger effect than a program taught on its‟ own. While the AB4L program benefitted students‟ behaviors for learning, it did not show the same overall effects on reading performance for all students. There are multiple influences on students‟ reading competences and achievement and while the enhancement of student learning behavior places students in a better position to profit from instruction, we also know that prerequisite, background knowledge is a major factor in predicting and explaining levels of achievement (e.g., Wang, Haertel &Wahlberg, 1993). Perhaps, participating students‟ pre-requisite reading comprehension skills were so under-developed that improvements in a reading comprehension test (Reading in Demand) was not possible in such a short period of time. Alternatively, it is the case that there are students in the two low SES schools in the sample who are reading near grade level expectations. It may be that these students have reasonably well-developed learning behaviors despite the low SES index of the school. For these students, it may be the case that the benefits of AB4L on reading performance may be seen in the long-term. Results reveal a significant benefit of AB4L on the reading performance of those students who scored in the lower 50 percent of their class on the reading comprehension survey used in this project. Comments from teachers indicated a shift in focus of class concern from reducing negative behavior to advancing positive behavior and an increase in whole-class student interest wanting to be successful. It may be the case that this shift in classroom culture along with the explicit teaching of positive attitudes and behaviors for learning had the most impact on the disengaged, under-achieving students. The finding of equal benefit of impact of AB4L on boys and girls is an important finding especially for the education of boys. It appears that the explicit teaching practices employed in AB4L where students are asked to practice ways of thinking and learning behaviors to use during classroom instruction combined with behavior-specific feedback equally suits the learning styles of boys and girls. Correlated Changes Of some significance is the finding that students in classes where AB4L was implemented who showed improvements in their reading comprehension also showed increases in their behaviors for learning. This would suggest that as many have argued that behaviors for learning are, indeed, mediating factors in the chain of influence leading to academic competence and achievement (e.g., McDermott, et. al., 2001). Additionally, this evidence of correlated changes in reading achievement and behaviors for learning suggests that the positive impact of AB4L was a specific effect of explicit teaching of attitudes and behaviors for learning and the teaching practices employed rather than solely a general effect of teachers being more positive.

Limitations First, the study used a quasi-experimental design that made use of existing classes. Thus, baseline differences between groups and differences between teacher effectiveness may have influenced the results. Nonetheless, the use of a pre-testing in the current study and the focus on change scores provides some control. Second, the AB4L program only had a significant effect on objective reading performance of students in the lower 50% of reading performance. Benefits of the AB4L program for students competent in reading as well as attitudes and behaviors for learning cannot be ascertained from the present analyses. These finding cannot be generalized to students who have reading challenges but do not attend economically disadvantaged schools. A third limitation of the study is that it is impossible to discern which of the different teaching practices focused on positive attitudes and behaviors for learning were the most powerful. Conclusion Based on these findings and previous research, student characteristics and their role in academic development and achievement needs to be in the center of the radar screen of education reform efforts to improve reading of students falling behind. Teacher preparation and professional development programs as well as the coaching and mentoring of principals and teachers should incorporate positive attitudes and behaviors for learning, especially as additional instructional support students who are most likely to be at risk for educational failure as well as those who are under-achieving in literacy and numeracy.

References Ashdown, D.M, & Bernard, M.E. (2012). Can explicit instruction in social and emotional learning skills benefit the social-emotional development, well-being, and academic achievement of young children? Early Childhood Education Journal, 11, 397-405. Australian Government Department of Education and Training (2014). National Partnerships for Low SES Schools. Literacy and Numeracy and Improving Teacher Quality. https://www.education.gov.au/national-partnerships-low-ses-schoolsliteracy-and-numeracy. Barnett, D. W., Bauer, A. M., Ehrhardt, K. E., Lentz, F. E., & Stollar, S. A. (1996). Keystone targets for changes: Planning for widespread positive consequences. School Psychology Quarterly, 11, 95-117. Bernard, M.E. (2006). Itâ&#x20AC;&#x;s time we teach social-emotional competence as well as we teach academic competence. Reading and Writing Quarterly, 22, 103-119. Bernard, M.E. (2011). The link between studentsâ&#x20AC;&#x; social and emotional characteristics and the development of literacy and numeracy skills. Research Monograph #08. The Victorian Literacy and Numeracy Secretariat, State Government of Victoria. Bernard, M.E., & Milne, M. (2016). Attitudes and behaviours for learning: A teacher professional development program for raising student engagement and achievement in Literacy and numeracy (2nd. Ed.). pp. 70. Manuscript submitted for publication. Bloom, B.S. (1976). Human characteristics and school learning. New York: McGraw-Hill. Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd edition). Mahwah, NJ: Erlbaum. Collaborative for Social and Emotional Learning, www.casel.org.

DiPerna, J.C., & Elliott, S.N. (1999). The development and validation of the Academic Competence Evaluation Scales. Journal of Psychoeducational Assessment, 17, 207– 225. DiPerna, J.C., Volpe, R.J., & Elliott, S. (2002). A model of academic enablers and elementary reading/language arts achievement. School Psychology Review, 31, 298-312. Duckworth, A., & Seligman, M. (2006). Self-discipline gives girls the edge: Gender in self-discipline, grades and achievement test scores. Journal of Educational Psychology, 98, 198-208. Durlak, J., Weissberg, R., Dymnicki, A., Taylor, R. & Schellinger, K. (2011). The impact of enhancing students‟ social and emotional learning: A meta- analysis of achoolbased universal interventions. Child Development, 82, 474- 501. Fantuzzo, J., Perry, M.A., & McDermott, P. (2004). Preschool approaches to learning and their relationship to other relevant classroom competencies for low-income children. School Psychology Quarterly, 19, 212-230. Green, L. F., & Francis, J. M. (1988). Children's learning skills at the infant and junior stages: A follow-on study. British Journal of Educational Psychology, 58, 120-126. Hardy. J. (2006). Speaking clearly: A critical review of the self-talk literature. Psychology of Sport and Exercise, 7, 81-97. Hattie, J. (2013). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London: Routledge. Knapp, T. R., & Schafer, W. D. (2009). From Gain Score t to ANCOVA F (and vice versa). Practical Assessment, Research & Evaluation, 14(6), 1-7. Kross, E., Bruehlman-Senecal, E., Park, J., Burson, A., Dougherty, A., Shablack, H., Bremner, R., Moser, J., & Ayduk, O. (2014). Self-talk as a regulatory mechanism: How you do it matters. Journal of Personality and Social Psychology, 106, 304-24. Lee, J. (2014). The relationship between student engagement and academic performance: Is it a myth or reality? Journal of Educational Research, 107, 177-185. http://dx.doi.org/10.1080/00220671.2013.807491 Locke, E.A., & Latham, G.P. (2002). Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist, 57, 705-717. McDermott, P. A. (1984). Comparative functions of preschool learning style and IQ in predicting future academic performance. Contemporary Educational Psychology. 9. 38-47. McDermott, P. A. (1999). National scales of differential learning behaviors among American children and adolescents. School Psychology Review, 28, 280-291. McDermott, P. A., Leigh, N. M., & Perry, M. A. (2002). Development and validation of the Preschool Learning Behaviors Scale. Psychology in the Schools. 39, 353-365. McDermott, P. A., Green, L. F., Francis, J. M., & Stott, D. H. (1999). Learning Behaviors Scale. Philadelphia: Edumetric and Clinical Science. McDermott, P.A., Mordell, M., & Stoltzfus, J.C. (2001). The organization of school performance in American school discipline, motivation, verbal learning and nonverbal learning. Journal of Educational Psychology, 93, 65-76. McWayne, C. M., Fantuzzo, J. W., & McDermott, P, A. (2004). Preschool competency in context: An investigation of the unique contribution of child competencies to early academic success. Developmental Psychology, 40, 633-645. Morris, S. B. (2008). Estimating effect sizes from pretest-posttest-control group designs. Organizational Research Methods, 11, 364-386. Pokay, P., & Blumenfeld, P.C. (1990). Predicting achievement early and late in the semester: The role of motivation and use of learning strategies. Journal of Educational Psychology, 82, 41-50. http://dx.doi.org/10.1037/00220663.82.1.41

Rock, D.A., & Pollack, J.M. (2002). Early childhood longitudinal study – kindergarten class of 1998-99 (ECLS-K), psychometric report for kindergarten through first grade. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Educational Statistics. Schaefer, B.A. (2004). A demographic survey of learning behaviors among American students. School Psychology Review, 33, 481-497. Schunk, D.H. (2003). Self-efficacy for reading and writing: Influence of modeling, goal setting, and self-evaluation. Reading and Writing Quarterly, 19, 159-172. The Victorian Curriculum and Assessment Authority‟s On Demand Computer Adaptive Reading Test (2006, 2010). www.vcaa.vic.edu.au/Pages/prep10/ondemand/index.aspx Vaughn, S., & Broadman, A. (2007) Teaching reading comprehension to students with learning difficulties. New York: Guilford Press. Vygotsky, L. (1934/1986). Thought and language. Cambridge, MA: MIT Press. Wang, M.C., Haertel, G.D., & Walberg, H.J. (1993). Toward a knowledge base for school learning. Review of Educational Research, 63, 249-294. Winsler, A., Manfra, L., & Diaz, R. M. (2006). “Should I Let Them Talk?” : Private speech and task performance among preschool children with and without behavior problems. Early Childhood Research Quarterly, 22, 215- 231. Winsler, A., & Naglieri, J. (2003). Overt and covert verbal problem-solving strategies: Developmental trends in use, awareness, and relations with task performance in children Aged 5-17. Child Development, 74, 659-678. Zimmerman, B.J. (1990). Self-regulated learning and academic achievement: An overview. Educational Psychologist, 25, 3-17. Zimmerman, B.J., & Schunk, D.H. (Eds.) (2001). Self-regulated learning and academic achievement. Mahwah, NJ: Erlbaum.

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 8, pp. 65-76, August 2017

Balancing Reflection and Validity in Health Profession Students‘ Self-Assessment Sherri Melrose Athabasca University Alberta, Canada

Abstract. Students and practitioners in self-regulating health professions are expected to engage in reflective, valid self-assessment activities. However, self-assessment processes can be flawed. People may have a limited understanding of the critical thinking needed to reflect on their performance and they may over-estimate or under-estimate their abilities. This article highlights educational approaches that can help students achieve a balance of reflecting critically and developing more accurate self-assessments. Considerations involved in defining selfassessment are identified. Explanations of how integrating reflection requires critical thinking; information from both internal and external sources; and incidental learning are provided. Suggestions for addressing validity by recognizing that inaccuracies exist; knowing that people‘s history with academic success can impact their selfassessments; and creating links to affective outcomes are offered. Emphasis is placed on viewing self-assessment as a formative learning activity that is introduced early and consistently in health education programs. Keywords: health profession students‘ self-assessment; reflection in selfassessment; validity in self-assessment

Introduction Self-assessment, a necessary skill for lifelong learning, requires people to identify standards to apply to their work, and then to make judgements about the extent to which they have met these standards (Boud, 1991; 1995). For practitioners in self-regulating health professions, self-assessment activities are an integral aspect of both their pre-service programs and their ongoing inservice professional development (Eva & Regehr, 2005). Novice practitioners enter their profession with a stronger ability to assess and develop the competencies they need when they have become familiar with assessing their own progress during their education (Boud, & Falchikov, 2006; KajanderUnkuri, Meretoja, Katajisto, Saarikoski, Salminen et al., 2013; Linn, Arostegui & Zeppa, 1975; Passi & Southgate, 2016). Supporting learners towards developing

their capacity to self-assess has been identified as the ‗missing link‘ needed to ensure that future health professionals are truly reflective, self-regulating practitioners (Redwood, Winning & Townsend, 2010). However, the self-assessment process can be flawed (Melrose, Park & Perry, 2015). For the most part, people overrate themselves and assess their progress as above average (Davis, Mazmanian, Fordis, Van Harrison, Thorpe, et al., 2006; Dunning, Heath & Suls, 2004; Mort & Hansen, 2010; Pisklakov, Rimal & McGuirt, 2014). They often identify areas of weakness inaccurately (Regehr & Eva, 2006). People can overestimate their performance and misjudge the skills they believe they have mastered (Baxter & Norman, 2011; Galbraith, Hawkins & Holmboe, 2008). Students who are least able to self-assess accurately often also demonstrate limited abilities in other areas of study (Austin & Gregory, 2007; Colthart et al., 2008). The phenomenon of less able people over-assessing their ability and more able people underestimating themselves is known as the Dunning-Kruger effect (Ehrlinger, Johnson, Banner, Dunning & Kruger, 2008; Kruger & Dunning, 1999). Consequently, the validity of self-assessment as an accurate measurement of student learning has been questioned (Falchikov & Boud, 1989; Gordon, 1991; Lundquist, Shogbon, Momary & Rogers, 2013; Ward, Gruppen & Regehr, 2002). Balancing the merit of reflection in self-assessment with questions about the validity of health profession students‘ self-assessment is not straightforward. Existing research has focused on evaluation studies and most of this work has been directed to physicians‘ learning. However, health professionals from a variety of different settings are expected to engage in self-assessment in their learning and in their practice. Increasing understanding of self-assessment among all members of health care teams can make an important difference in helping learners grow into self-regulated professionals. Geared to a multidisciplinary audience, this article provides an overview of how selfassessment can be defined, how reflection can be integrated into the process and how issues of validity can be addressed.

Toward a Definition of Self-Assessment Assessment provides information about how people are progressing in relation to objectives, goals and outcomes. In health professions programs, assessments usually include standardized measurement tools as well as inferences about what individuals do in relation to what they know (Melrose, Park & Perry, 2015). In clinical practice settings, specific times at both mid-term (formative) and end of course (summative) are designated for discussing student progress. Self, peer and educator assessment may be included in these discussions. Formative evaluations are diagnostic, ongoing and focused on both what students are currently doing well and areas where they need to improve in future. A final grade is seldom included in formative evaluations as the goal of the activity is to improve student performance (Melrose, Park & Perry, 2015).

Self-assessment, particularly when integrated into formative evaluations, can be construed as a learning activity and not merely a grading activity. When viewed as a learning activity, self-assessment invites students to actively participate in and reflect on their own learning (Boud & Falchikov, 1989). It helps students recognize desired goals, gather evidence about their present position and come to an understanding about ways they can close the gap between the two (Black & William, 1998). Instead of simply relying on teachers to evaluate their progress, opportunities for self-assessment encourage students to think critically about the quality of their studies (Andrade & Valtcheva, 2009). Self-assessment assists students to create links among tasks they are presently engaged in, outcomes expected by their profession, outcomes they expect for themselves and future tasks they will engage in (Bourke, 2016). In essence, self-assessment can be conceptualized as a formative, educational, developmental, self-monitoring activity that draws upon both internal and external data, standards, and resources to inform and judge one‘s performance (Sargeant 2008).

Integrating Reflection Reflection has a dynamic relationship with self-assessment. As Mann (2010) so eloquently stated: ―To be effective at self-assessment requires skills in critical reflection; to be effective in reflection, self-assessment skills are required‖ (p.311). However, the purposes and goals of reflection are different from those of self-assessment. Reflection is a process of personal self-understanding that can lead to significant discoveries and insights, while self-assessment involves using predetermined performance criteria to determine insights, strengths and needed improvements (Desjarlais & Smith, 2011). Reflective processes are often retrospective; they do not necessarily involve others or externally imposed performance criteria; and they may not include expectations of improvement. It is important to acknowledge that selfassessment skills are not limited to engaging in reflective activities. However, reflection, particularly critical reflection, plays a foundational role in health profession students‘ self-assessment. Critical Reflection Reflection that can be considered critical and therefore of most use in self-assessment goes beyond simply looking back on experiences. Theorists have extended our understanding of the complex reasoning that is involved. Advocating for the use of reflection as an active and deliberate problem-solving process, John Dewey (1933) believed reflection should include recalling an event and then questioning why things happened as they did. Donald Schön (1983), theorized that reflective practice includes both ‗reflectionin-action‘ (intuitively drawing on previous experiences to resolve situations while they are occurring) and ‗reflection-on-action‘ (thinking about an event that has taken place and considering what could be changed in future). Steven Brookfield (1995; 1988) asserted that reflective practice also requires people to become aware of and question their assumptions and their ways of interpreting

information. Jack Mezirow (1998) explained how critical reflection requires people to examine the way they perceive events and then transform their thinking in order to find new ways of making meaning. In nursing, Christopher Johns (2017) proposed a structured model calling for practitioners to reflect on experiences by both "looking in" to examine their thoughts and emotions and "looking out" to understand external factors influencing the situation. The complexities of thinking critically and engaging in reflective practice may seem overwhelming to health profession students, particularly those at a beginning stage of their program. One approach that can help students strengthen their self-refection skills and to grow as reflective practitioners is to introduce reflective activities early (Falchikov & Boud, 1989; Kanthan & Senger, 2011; Mann, Gordon & MacLeod, 2009). Tools such as reflective journals can provide opportunities for developing reflective practice skills (Constantinou & Kuys, 2013; Koh, Wong & Lee, 2014; Lew & Schmidt, 2011). While reflective journals are usually written products, Tulgar (2017) notes how reflections can also be captured through Smartphone audio or video self-recordings. Similarly, students can use social media applications to create reflective journals (Dabbagh & Kitsantas, 2012). In clinical practice settings, educators can intentionally invite students to begin any discussion of their performance with self-reflection and self-analysis (Melrose, Park & Perry, 2015). When students are consistently required to engage in critical reflection throughout their programs, the process becomes increasingly familiar. Extending students‘ critical reflection skills to strengthen their self-assessment skills involves building in opportunities to cast students‘ own thinking against predetermined outcomes. When students are performing new clinical tasks, it is not unexpected that their capacity to self-assess is also less accurate. However, later in their programs, self-assessment accuracy improves (Blanch-Hartigan, 2011; Fitzgerald, White & Gruppen, 2003). Therefore, just as providing supplemental opportunities to practice clinical skills can be helpful, providing opportunities to practice self-assessment can also be helpful. The climate within these practice opportunities should be supportive and non-punitive (Asadoorian & Batty, 2005). Integrating information from external sources Self-assessment skills also involve integrating information from external sources. For health profession students, the educators who evaluate them (faculty, instructors, tutors, mentors, preceptors and practitioners) are key external sources. Given the power and influence these educators have over students‘ progress in their chosen profession, feedback from educators is a critical element that undergirds the selfassessment process. Explicit, formative feedback lets students know how their educators perceive their performance. In turn, these perceptions can clarify criteria expected for good performance; they can stimulate learners to identify strengths and weaknesses; and they can help learners focus their efforts productively (Sitzmann, Ely, Brown & Bauer, 2010). Self-assessment that does

not integrate educator feedback is incomplete (Motycka, Rose, Ried & Brazeau, 2010). In many instances, feedback can be difficult to ‗hear,‘ and can leave students feeling distressed; doubtful about their abilities; unmotivated; and reluctant to persevere with their studies (Mann, 2010). Students may view even the most well intended educator comments as a potential intolerance for their mistakes and an indication that they lack knowledge, leaving them reluctant to seek out and act on feedback (Mann, 2010). Efforts to ameliorate these difficulties can include regular meeting times; educators sharing their anecdotal notes or ongoing records of student progress; and providing specific time-limited strategies for task improvement (Melrose, Park & Perry, 2015). Further, opportunities where students routinely exchange assessment feedback with their peers can help make the process less intimidating. Feedback exchanges, where students apply the same assessment criteria as educators, can be organized as pair-share and small group activities. When feasible, these sessions could involve students in decisions about the assessment criteria being used; the origin and relevance of the assessment criteria; and practice priorities that may impact the criteria. Incidental learning Affirming learning that students have achieved which does not relate to predetermined goals is a valuable but often neglected aspect of reflection. Incidental learning, also called surprise, unexpected or unintended learning, is learning that occurs as a by-product of doing something else (Marsick & Watkins, 1990; 2001). Incidental learning can emerge from observing others; from discussions with people in the environment; as a consequence of making mistakes; and from being required to adapt to or accept situations (Kerka, 2000). Creating space for students to share and celebrate incidental learning within their self-assessments can highlight accomplishments that may otherwise go unnoticed. To draw out incidental learning, educators can pose questions such as ―What surprised you when …?‖ or ―Talk about what happened that you didn‘t expect when …‖ (Melrose, Park & Perry, 2015). In sum, integrating reflection into self-assessment can begin by simply reflecting and seeking to gain new personal insights. Developing the skill further can include critical reflection, which involves thinking deeply about ways of solving problems that are occurring or have occurred. Critical reflection requires people to change their thinking and consider new ideas. When the process of reflection becomes especially valuable to self-assessment is when these internal processes are coupled with the integration of information from external sources. For health profession students, feedback from educators and peers is a primary external source. A balance of internal, external and incidental information is needed when students seek to assess their performance in relation to the standards, criteria and competencies required by their profession. In the next section, common concerns related to the validity of students‘ self-assessments are discussed.

Addressing Validity As previously mentioned, the Dunning-Kruger effect (Dunning & Kruger, 1999) where less able people over-state their ability and more able people under-state their ability, has influenced people‘s views about the validity of self-assessment. Questions are often posed about whether self-assessment activities provide accurate, dependable and truthful representations of students‘ abilities. Recognize that inaccuracies exist It is important to recognize that inaccuracies in students‘ self-assessments exist in many health professions. Research evidence indicates that students‘ self-assessments frequently differ from educator assessments. Comparing classroom test scores, Brown and Harris (2013) found only weakly positive correlations between educator ratings and students‘ selfassessed ratings; between actual test scores and self-estimates of performance; and between educator and student judgments when the same rubric was used. In simulated emergency situations, Baxter and Norman (2011) found nursing students‘ self-assessments were significantly inaccurate in comparison with educators‘ observations of their performance. Similarly, in peer simulation situations, Sanderson, Kearney Kissell and Salisbury (2016) found dental hygiene students‘ self-assessments were also significantly inaccurate in comparison to those of their educators. Measuring communication skills, Gude, Finset, Anvik, Bærheim, Fasmer et al. (2017) also reported a lack of concordance between medical students‘ own and their educators‘ assessment. Clearly, consistently achieving congruence between student and educator assessment may not always be possible. In these instances, conceptualizing student self-assessment as a formative developmental learning activity can be helpful. Approaches such as video and verbal feedback have been found to enhance the accuracy of students‘ self-assessments (Colthart et al., 2008; Hulsman & van der Vloodt, 2015; Volino & Das, 2014). Providing easy online access to self-administered tests with answers has the potential to provide students with accurate information about their level of knowledge (Miller, 2008). Reviewing a collection of work, such as a portfolio, capstone project or reflection summary, rather than just single instances of student performance can provide a wider view of how students are meeting competencies (Gadbury-Amyot, Woldt & Siruta-Austin, 2015). Implementing self-assessment activities in contexts where the emphasis is on mastery goals (achieving competence in practice) rather than performance goals (achieving immediate competence completing a task) can also contribute to more accurate self-assessment (Butler, 2011). Know the impact of a history with academic success Students accepted into health profession programs often have a strong history of academic success. If students are used to performing well in learning situations and have consistently received positive feedback, they are likely to feel confident in their abilities. In turn, they may have a view of themselves as above average. When asked to self-assess, their thinking may be based on potential or ideal performance more than their actual performance (Evans, McKenna & Oliver, 2001).

From this perspective, the lack of congruence between student and educator assessments can be viewed as an opportunity to support studentsâ&#x20AC;&#x2DC; positive selfconcept and self-worth. Rather than emphasizing inaccuracy, educators can prompt students to identify steps they have taken in the past to achieve success, and then encourage them to apply these steps to their present learning situation. On the other hand, students who do not have a strong history of academic success may be unaware of inaccuracies in their self-assessment or they may be reluctant to disclose them. If students perceive their learning environment as overwhelming, they may not know where to begin identifying what they do not know. In health care environments, where professionals are accountable to the public for providing safe competent patient care, students may not feel that it is acceptable to admit weakness. In these instances, once again, rather than emphasizing inaccuracy, educators can highlight the links between accurate practitioner self-assessment (which includes admitting to not knowing and then seeking out needed information) and patient safety (Sujata, Oliveras & Edson, 2001). Create links to affective outcomes A further consideration influencing the validity of self-assessment is the distinction between cognitive and affective learning outcomes. Cognitive learning outcomes are more factually based, may relate to a particular course of study, and are associated with external sources such an exam grade or educator rating (Sitzmann, Ely, Brown & Bauer, 2010). Affective learning outcomes are related to internal sources, extend beyond a specific course or learning event and they include feeling satisfied, motivated, able to carry out tasks and willing to apply and use knowledge gained (Sitzmann et.al.). A meta-analysis of evaluation studies revealed that construct validity of selfassessment was strongly correlated with affective outcomes (particularly satisfaction and motivation) and only weakly correlated with cognitive outcomes (Sitzmann et. al.). Given this correlation, self-assessment activities linked to affective outcomes have a greater chance of yielding a more accurate measurement result. Practitioners from different disciplines and practice areas all need selfassessment skills that help develop their thinking beyond the boundaries of a single course or learning event (He & Canty, 2013; Mann, 2010). Therefore, when addressing validity in self-assessment, connections between self-assessment activities and the nature of the outcomes being measured is an important consideration. Knowing the inherent difficulty in quantifying success with affective achievements, addressing validity in self-assessment must be grounded in a commitment to designing activities that are suitable for measuring broad outcomes, mastery goals and critically reflective thinking.

Conclusion Self-regulating health professionals must be able to assess what they know; what they don‘t know in relation to what they are expected to know; and what they need to learn in order to provide safe competent care. Self-assessment is a learned skill and one that can be best developed through early, consistent and supportive activities during pre-service educational programs. Conceptualizing self-assessment as a formative learning activity offers a perspective where students and educators focus on improving performance rather than simply grading competencies. A balance of reflection that taps into critical thinking and accurate representations of students‘ abilities is needed for self-assessment to be viewed as valid. Achieving this balance between reflection and validity is complex. In order to integrate reflection into their self-assessments, students must think critically and find new ways to solve problems and find meaning. They must analyze their performance in relation to pre-determined outcomes. They must also extend their own thinking to include feedback received from educators, peers and other external sources. Further, they must take incidental or surprise learning into account. Journals, either written or audio/video recorded are useful tools for developing reflective thinking. Inviting students to self-assess at the beginning of educator-student conversations; in pair-share discussions; and in small group conferences can provide valuable practice opportunities. The validity of student self-assessment is often questioned because students‘ views of their abilities can be very different from those of their educators. Less able students over-estimate their ability and more able students underestimate their ability. Questions about validity can begin to be addressed by first recognizing that inaccuracies exist. Providing video feedback; self-administered online tests with answers; and reviewing a collection of work instead of a single instance can help students‘ self-assess more accurately. A history of either success or limited academic success impacts congruence between student and educator assessments. Reminding students of how they achieved success in the past provides useful guidance. Emphasizing how disclosing areas of weakness can lead to increased patient safety offers meaningful rationale for moving forward. Finally, stronger validity can be achieved when links are created between selfassessment activities and broad affective outcomes related to feeling satisfied with knowledge that has been gained and feeling motivated to apply and use that knowledge. Continuing to find ways to balance reflection and validity in self-assessment is both a challenge and an opportunity for students and educators in the health professions.

References Andrade, H., & Valtcheva, A. (2009). Promoting learning and achievement through selfassessment. Theory into Practice, 48(1), 12-19. http://dx.doi.org/10.1080/00405840802577544. Asadoorian, J. & Batty, H. (2005). An evidence-based model of effective self-assessment for directing professional learning. Journal of Dental Education, 69(12), 1315-1323. http://www.jdentaled.org/content/69/12/1315.full Austin, Z. & Gregory, P. (2007). Evaluating the accuracy of pharmacy students‘ selfassessment skills. American Journal of Pharmaceutical Education, 71(5), 1–8. Baxter, P., & Norman, G. (2011). Self-Assessment or self-deception? A negative association between self-assessment and performance. Journal of Advanced Nursing, 67(11), 2406–2413. doi:10.1111/j.1365-2648.2011.05658.x Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2), 139-148. Blanch-Hartigan, D. (2011). Medical students‘ self-assessment of performance: Results from three meta-analyses. Patient Education and Counseling, 84(1), 3 – 9. DOI: http://dx.doi.org/10.1016/j.pec.2010.06.037 Boud, D. (1991). HERDSA Green Guide No 5. Implementing student self-assessment (Second ed.). Campbelltown: The Higher Education Research and Development Society of Australasia (HERDSA). Boud, D. (1995). Enhancing learning through self-assessment. London. Routledge Falmer. Boud, D. & Falchikov, N. (1989). Quantitative studies of self-assessment in higher education: A critical analysis of findings. Higher Education, 18, 529-549. Boud, D. & Falchikov, N. (2006). Aligning assessment with long-term learning. Assessment and Evaluation in Higher Education, 31, 399–413. DOI:10.1080/02602930600679050. http://www.jhsph.edu/departments/population-family-and-reproductivehealth/_docs/teaching-resources/cla-01-aligning-assessment-with-long-termlearning.pdf Bourke, R. (2016). Liberating the learner through self-assessment. Cambridge Journal of Education, 46(1), 97-111. DOI: 10.1080/0305764X.2015.1015963 Brookfield, S. (1995). Becoming a critically reflective teacher. San Francisco: Jossey Bass. Brookfield, S. (1998). Critically reflective practice. Journal of Continuing Education in the Health Professions, 18, 197–205. DOI:10.1002/chp.1340180402 Brown, G. T. L., & Harris, L. R. (2013). Student self-assessment. In J. H. McMillan (Ed.). The SAGE handbook of research on classroom assessment (pp. 367-393). Thousand Oaks, CA: Sage. Butler, R. (2011). Are positive illusions about academic competence always adaptive, under all circumstances: New results and future directions. International Journal of Educational Research, 50(4), 251-256. DOI: 10.1016/j.ijer.2011.08.006 Colthart, I., Bagnall, G., Evans, A., Allbutt, H., Haig, A., Illing, J. & McKinstry, B. (2008). The effectiveness of self-assessment on the identification of learning needs, learner activity, and impact on clinical practice: BEME Guide No. 10. Medical Teacher, 30,124–145. Constantinou, M., & Kuys, S. (2013). Physiotherapy students find guided journals useful to develop reflective thinking and practice during their first clinical placement: A qualitative study. Physiotherapy, 99(1), 49–55. DOI:10.1016/j.physio.2011.12.002 Dabbagh, N., & Kitsantas, A. (2012). Personal learning environments, social media, and self-regulated learning: A natural formula for connecting formal and informal learning. Internet and Higher Education, 15(1), 3-8. DOI:10.1016/j.iheduc.2011.06.002

Davis, D., Mazmanian, P., Fordis, M., Van Harrison, R., Thorpe, K. et al. (2006). Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. The Journal of the American Medical Association, 296(9), 1094– 1102. Desjarlais, M. & Smith, P. (2011). A comparative analysis of reflection and selfassessment. International Journal of Process Education, 3(1), 3-18. http://www.processeducation.org/ijpe/2011/reflectionh.pdf Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the educative process. New York: D.C. Heath and Company. Dunning, D., Heath, C. & Suls, J. M. (2004). Flawed self-assessment: Implications for health, education, and the workplace. Psychological Science in the Public Interest, 5(3), 69-106. https://faculty-gsb.stanford.edu/heath/documents/PSPI%20%20Biased%20Self%20Views.pdf Ehrlinger, J., Johnson, K., Banner, M., Dunning, D. & Kruger, J. (2008). Why the unskilled are unaware: Further explorations of (absent) self-insight among the incompetent. Organizational Behavior and Human Decision Processes, 105 (1), 98– 121.doi:10.1016/j.obhdp.2007.05.002 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2702783/ Eva, A. & Regehr, G. (2005). Self-assessment in the health professions: a reformulation and research agenda. Academic Medicine, 80(10 Suppl):S46-54. Evans, A., McKenna, C. & Oliver, M. (2001). Towards better understanding of selfassessment in oral and maxillofacial surgery. Medical Education, 35(11),1077. Falchikov, N. & Boud, D. (1989). Student self-assessment in higher education: A metaanalysis. Review of Educational Research, 59, 395–430. Fitzgerald, J., White, C., & Gruppen, L. (2003). A longitudinal study of self-assessment accuracy. Medical Education, 37(7), 645-649. Gadbury-Amyot, C., Woldt, J. Siruta-Austin, K. (2015). Self-assessment: A review of the literature and pedagogical strategies for its promotion in dental education. Journal of Dental Hygiene, 89(6), 357-364 Galbraith R., Hawkins, R. & Holmboe, E. (2008). Making self-assessment more effective. Journal of Continuing Education in the Health Professions, 28(1), 20–24. Gordon, M.J. (1991). A review of the validity and accuracy of self-assessments in health professions training. Academic Medicine, 66, 762–769. Gude, T., Finset, A. Anvik, T., Bærheim, A., Fasmer, O. et al. (2017). BMC Medical Education, 17,107.DOI 10.1186/s12909-017-0943-y He, X., & Canty, A. (2013). A comparison of the efficacy of test-driven learning versus self-assessment learning. The Journal of Chiropractic Education, 27(2), 110–115. http://doi.org/10.7899/JCE-13-6 Hulsman, R. & van der Vloodt, J. (2015). Self-evaluation and peer-feedback of medical students‘ communication skills using a web-based video annotation system. Exploring content and specificity, Patient Education and Counseling, 98(3), 356-363 http://dx.doi.org/10.1016/j.pec.2014.11.007. Johns, C. (2017). Becoming a reflective practitioner (5th Ed.). Oxford, UK: Wiley-Blackwell Kajander-Unkuri, S., Meretoja, R., Katajisto, J., Saarikoski, M., Salminen, L., et al. (2013). Self-assessed level of competence of graduating nursing students and factors related to it. Nurse Education Today, 34(5), 795 – 801. Kanthan, R. & Senger, J. (2011). An appraisal of students' awareness of "self-reflection" in a first-year pathology course of undergraduate medical/dental education. BMC Medical Education, 11, 67. https://doi.org/10.1186/1472-6920-11-67 Kerka, S. (2000). Incidental learning. Trends and Issues Alert 18. Columbus OH: ERIC Clearinghouse on Adult, Career and Vocational Education. http://www.calproonline.org/eric/docs/tia00086.pdf

Koh, Y., Wong, M. & Lee, J. (2014). Medical students‘ reflective writing about a taskbased learning experience on public health communication. Medical Teacher, 36(2), 121–9. DOI: 10.3109/0142159X.2013.849329. Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one‘s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121-1134. https://www.ncbi.nlm.nih.gov/pubmed/10626367 Lew, M. & Schmidt, H. (2011). Self-reflection and academic performance: Is there a relationship? Advances in Health Sciences Education, 16(4), 529–545. http://doi.org/10.1007/s10459-011-9298-z Linn, B., Arostegui, M. & Zeppa, R. (1975). Performance rating scale for peer and self assessment. British Journal of Medical Education, 9(2), 98–101. Lundquist, L. M., Shogbon, A. O., Momary, K. M. & Rogers, H. K. (2013). A comparison of students‘ self-assessments with faculty evaluations of their communication skills. American Journal of Pharmaceutical Education, 77(4), 72. http://doi.org/10.5688/ajpe77472 Mann, K. (2010). Self-assessment: The complex process of determining ―how we are doing‖—A perspective from medical education. Academy of Management Learning & Education, 9(2), 305–313. Mann, K., Gordon, J. & MacLeod, A. (2009). Reflection and reflective practice in health professions education: A systematic review. Advances in Health Science Education, 14, 595–621. DOI 10.1007/s10459-007-9090-2 Marsick, V. & Watkins, K. (1990). Informal and incidental learning in the workplace. London & New York: Routledge. Marsick, V. & Watkins, K. (2001). Informal and incidental learning. New Directions for Adult and Continuing Education, 89, 25–34. Melrose, S., Park, C. & Perry, B. (2015). Creative clinical teaching in the health professions. Retrieved from http://epub-fhd.athabascau.ca/clinical-teaching/ Mezirow, J. (1998). On critical reflection. Adult Education Quarterly, 48, 185-198. http://dx.doi.org/10.1177/074171369804800305 Miller, P. (2008). Self-assessment: The disconnect between research and rhetoric. Physiotherapy Canada 60(2), 117–124. DOI: 10.3138/physio.60.2.117 Mort, J. & Hansen, D. (2010). First-year pharmacy students‘ self-assessment of communication skills and the impact of video review. American Journal of Pharmaceutical Education, 74(5), 1–7. Motycka, C., Rose, R., Ried, D. & Brazeau, G. (2010). Self-assessment in pharmacy and health science education and professional practice. American Journal of Pharmaceutical Education, 74 (5) Article 85. Passi, V. & Southgate, L. (2016). Revisiting 'Performance rating scale for peer and self assessment'. Medical Education, 50(3),267-270. DOI: 10.1111/medu.12798 Pisklakov, S., Rimal, J. & McGuirt, S. (2014). Role of self-evaluation and self-assessment in medical student and resident education. British Journal of Education, Society and Behavioral Science, 4(1), 1–9. Redwood, C., Winning, T. & Townsend, G. (2010). The missing link: Self-assessment and continuing professional development. Australian Dental Journal, 55, 15–19. DOI:10.1111/j.1834-7819.2009.01177.x http://onlinelibrary.wiley.com/doi/10.1111/j.1834-7819.2009.01177.x/full Regehr, G. & Eva, K. (2006). Self-assessment, self-direction, and the self-regulating professional Clinical Orthopaedics and Related Research, 449, 34–38. http://innovationlabs.com/r3p_public/rtr3/pre/pre-read/Selfassessment.Regher.Eva.2006.pdf

Sanderson, T., Kearney, R., Kissell, D. & Salisbury, J. (2016). Evaluating student selfassessment through video-recorded patient simulations. Journal of Dental Hygiene, 90(4), 257-262. Sargeant J. (2008). Toward a common understanding of self-assessment. Journal of Continuing Education in Health Professions. 28, 1–4. DOI: 10.1002/chp.148. https://www.ncbi.nlm.nih.gov/pubmed/18366124 Schön, D. (1983). The reflective practitioner: How professionals think in action. London: Temple Smith. Sitzmann, T., Ely, K., Brown, K. & Bauer, K. (2010). Self-assessment of knowledge: A cognitive learning or affective measure? Academy of Management Learning & Education, 9(2), 169-191. Sujata, B., Oliveras, E. & Edson, W. (2001). How can self-assessment improve the quality of healthcare? Operations Research Issue Paper 2(4). Published for the U.S. Agency for International Development(USAID) by the Quality Assurance (QA)Project, Chevy Chase, MD and JHPIEGO Corporation, Baltimore, MD. Tulgar, A. (2017). Selfie@ssessment as an alternative form of self-assessment at undergraduate level in higher education. Journal of Language and Linguistic Studies, 13(1), 321-335. Volino, L. & Das, R. (2014). Video review in self-assessment of pharmacy students‘ communication skills. Journal of Education and Learning, 3(4), 26-36. doi:10.5539/jel.v3n4p26. http://dx.doi.org/10.5539/jel.v3n4p26 Ward, M., Gruppen, L. & Regehr, G. (2002). Measuring self-assessment: Current state of the art. Advances in Health Sciences Education Theory and Practice, 7, 63–80.

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 8, pp. 77-84, August 2017

Innovative Teaching with Use of an Art Work Marios Koutsoukos ASPETE, School of Pedagogical and Technological Education, Kozani, Greece Iosif Fragoulis ASPETE, School of Pedagogical and Technological Education, Patra, Greece

Abstract. This study presents the use of an art work in the teaching of a course entitled “Environment and Agriculture” in the last grade of a Vocational Lyceum. More specifically, the painting of Dutch artist Jan van Goyen entitled “A windmill by a river” was used as a teaching tool in an innovative approach of teaching the topic of wind energy as a renewable natural resource. The purpose of this approach was to enhance students’ critical judgment concerning renewable sources of energy, such as wind energy, fostering at the same time transformative learning, as students become aware and critical of their own initial assumptions and develop new perspectives. Taking into consideration and applying David Perkin’s theory about using art works in teaching, students were able to strengthen their creative thinking, getting to know the various aspects of wind energy through emotional observation and thorough study of an art work. Keywords: Teaching; Use of art; Observation; Critical judgement

Introduction Several academics and scholars, such as Gardner, Eisner, Perkins, and Kokkos, have occasionally expressed the view that education through the employment of artwork contributes to the learning process, and it also helps students develop a variety of skills, including critical thinking, cognitive development, creativity, discovery of new knowledge and expression of emotions (Gardner, 1990; Perkins, 1994; Eisner, 2002; Kokkos, 2011; Phillips & Fragoulis, 2012). Therefore, artworks can prove to be a precious treasure that can be utilised appropriately in educational practices, serving a wide range of objectives (Barnes, 2015). In recent years, the use of visual arts in education has also been extended to the teaching of environmental sciences. According to research in the international

bibliography, this practice can result in impressive learning outcomes (Inwood & Taylor, 2012; Rosenthal, 2003; Neperud, 1997). For instance, a painting depicting a natural landscape can function as a stimulus for introducing an interesting teaching approach to an environmental education course. More specifically, a painting portraying a river or a lake can serve as the starting point for teaching water resource management.

Use of art in education Over the past few years, both formal and non-formal educational approaches have been developing based on the use of art for educational purposes. Education through the use of art involves the introduction of artworks in the teaching process – the ones related to the subject of teaching in the context of processing a learning objective. Through the processing of these artworks, the meaning they carry gets revealed, and then it can be used as a pre-text for undertaking a deeper approach towards the subjects under consideration (Barnes, 2015; Kokkos, 2011; Efland, 2002). In Greece, the first systematic efforts to train the teachers of formal education, regarding the use of art in education, were implemented during the school year 2011-2012, as part of the implementation of the “Major Edification Program” project. The Hellenic Open University has also made a substantial contribution to this field, since the Academic year 2012-2013, in the framework of the Postgraduate Programs “Studies in Education and Adult Education”. Similar initiatives have been assumed by the Adult Education Scientific Association for the edification of teachers and adult educators in the utilization of works of art in the teaching practice.

Models of approaching works of art in the educational process Artwork approach models provide a theoretical and methodological framework, according to which the student, gradually approaching a work of art, enters areas of deeper understanding by activating his critical and reflective spirit (Kokkos, 2011). Four models are described in the relevant literature. Those are the models of Feldman, Broudy, Anderson and Perkins. According to Feldman (1967), the approach of an artwork is realized through four interrelated phases: description, analysis, interpretation and evaluation. The Broudy (1972) model, which is related to that of Feldman, emphasizes the field of activating the aesthetic observations of an artwork (e.g., searching for shapes, volumes, ways to connect them), as well as the identification of the expressiveness that the artwork radiates. Anderson’s (1993) model includes the following reflective processes: initial reaction, description, interpretation, evaluation. It presents several similarities to the Feldman model, but differs from that in the first phase, which in this model is described as the initial reaction. Within the context of this model, and particularly during the first phase, the student proceeds to formulate an instinctive reaction to the observed artwork. In the aforementioned models, the analysis of an artwork is based on the identification and observation of a subject that is gradually approached, starting from its instantly perceived features and,

finally, unveiling its “inner-deeper” characteristics, that is, the deeper meaning of the work.

The Perkin’s artwork analysis model According to Perkins, the approach of works of art is done in such a way so that the observer is able to distinguish: a) the key actions that trigger the reflective spirit, b) the possible questions that can cause the reflective effect, c) the teaching approaches one needs to implement in order to provoke reflection. These three elements form a comprehensive methodology for the reflective observation of works of art. Perkins’ model is considered more comprehensive in comparison to other models that have been proposed from time to time, as it can apply to the approach of various artworks, requiring only minimal modifications (Kokkos, 2011; Phillips & Fragoulis, 2012). As part of the artwork observation process suggested by Perkins, the observer adopts a documented view at the observation. Thus, students are not confined only to value judgments, such as “I like it” or “I dislike it” but, by observing specific elements of the artwork, they methodically develop an argument that strengthens their opinion on the art work (Kokkos, 2011). In addition, thoughtful looking at art appears to have an instrumental value as it provides an excellent setting for the development of better thinking (Perkins, 1994). Perkins argues that, through this process, students are not only able to enrich their aesthetic experience, but also, at the same time, cultivate a critical-reflective approach towards the events taking place within and outside the educational framework (Perkins, 1994). Perkins' model consists of four phases that are correlated and can be summarized as: a) giving looking time, b) making looking broad and adventurous, c) making looking clear and deep and d) making looking organized. More specifically, in the first phase students are given time to observe carefully an artwork, at first without attempting to understand or evaluate it. Next, during the second phase, students try to approach the artwork bearing in mind the question “what is that the artist would like us to observe” (Phillips & Fragoulis, 2012). Proceeding to the third phase, analytical and deeper observation takes place, as students in collaboration with their teacher attempt to answer to several questions which arose during previous observation, as well as to interpret meanings and draw conclusions. Finally, during the fourth phase, students holistically approach the art work by taking advantage the observational experience of previous stages.

Teaching Scenario The main and general purpose of this educational approach was to develop students' critical thinking and reflection on issues related to the use of alternative, environmentally friendly types of energy, as well as to transform their initial assumptions regarding their importance. With regard to the expected learning outcomes deriving from the use of this specific educational practice, the students, after processing the artwork, had to be able to:  express awareness on issues related to the exploitation of mild and environmentally friendly forms of energy, 

develop a holistic approach to individual environmental issues, particularly regarding the use of wind energy,



demonstrate critical thinking on environmental issues,



apply the stages of the Perkins’ model to the artwork approach,



adopt a positive attitude towards the creative exploitation of important works of art for the purpose of approaching environmental issues.

The painting “A windmill by a river”, by the Dutch painter Jan Van Goyen, was created in 1642 and is currently kept at the National Gallery of London. This depicts a landscape of the Dutch province, with a gray and hazy sky covering ¾ of the painting, while in the lower right side of the composition, emerges a windmill at the river’s edge (Beck, 1977). The windmills, a “trademark” of the Dutch countryside, were largely used for pumping and supplying water from rivers and lakes, as well as for the grinding of grains, at the time the artwork was created (Beck, 1977). Since the days of Van Goyen and up to the present day, wind energy, generated by the exploitation of the winds, constitutes a mild and environmentally friendly form of energy, as it does not burden the natural resources. Within this context, and in the teaching of the “Renewable Natural Resources” thematic sector of the course “Environment and Agriculture”, the use of this particular artwork as a teaching tool was attempted for the development of critical thinking. Perkins’ artwork observation model, as described in the relevant literature, (Perkins, 1994; Kokkos, 2011) was used as a teaching tool. This particular teaching approach was implemented over the course of 3 teaching hours, including 2 intermediate breaks. More specifically, the phases used were as follows: Phase 1: Time for observation, Phase 2: Open and adventurous observation, Phase 3: Analytical and deeper observation, Phase 4: Overview of the process. More specifically, in the first phase, students were given the necessary time to observe the artwork and express their initial spontaneous estimates. This way, the teacher invited students to observe the artwork, focusing on the colors, the

figures, the landscape, and the windmill. Afterwards, following the expression of the initial spontaneous remarks regarding the artwork, the educator used the collaborative method and invited the students to split into groups and write in captions what they noticed during their initial contact with the work of art. Then, using the brainstorming technique, the students were asked to capture and write down concepts, ideas and views, regarding their first visual contact with the artwork. Moreover, through the use of visual intelligence, the teacher invited the students to make use of the previous knowledge they might have on this artwork, in order to stimulate their perceptiveness, while outlining the natural landscape depicted in the painting. Subsequently, the students were asked to distance themselves from the artwork for a little while, averting their gaze off it for a short period of time. After a while, returning to the observation of the painting, the teacher posed a question regarding the position of the windmill in the depicted landscape, and the students, divided into working groups, were invited to circle the interesting features of the artwork and formulate further questions regarding the painting. Moving on to second phase, that of open and adventurous observation, the teacher attempted to stimulate the students’ attention, asking them why is the sky illustrated cloudy and full of winds, as well as why is the windmill positioned next to the river. The teacher went on with the pursuit of the spirit, the symbols, and the meanings, exploring whether the view of this specific artwork stimulates some feelings to the students and whether there are certain messages behind the artwork’s symbolisms. Another important activity of the second phase was the alteration of the observation scale, where the teacher focused on the sky, the positioning of the windmill, as well as the river, and asked of the students to circle and study a specific area in the artwork, effectively inviting them to identify the central theme in relation to the regional one. At this point, the space-time placement of the artwork was carried out, as the teacher provided the historical details of the painting, which was created by Van Goyen in 1642 and illustrates a typical landscape of the Dutch countryside during that time, where windmills were used for pumping and supplying water from rivers and lakes as well as for grain milling. Entering the third phase of the analytical observation, the teacher posed the question “What message does the creator of the artwork wish to convey, by placing the windmill in this particular landscape?”. At the same time, the students, working in workgroups, were invited to respond to key concerns, for example: “Why is the cloudy and windy sky prevailing in ¾ of the painting? How is the windmill near the river connected with the rest of the landscape? How is the wind energy, generated by the windmill, being utilized?”. Moreover, through the use of active and participatory techniques, such as discussion and brainstorming, the teacher, in the context of focused intellectual change activity, asked from the students to isolate parts of the artwork or modify them

accordingly. At this point, questions, such as the following, arose: “If the windmill was absent from the landscape, would that alter the structure of the painting?” and “Were the sky not cloudy and full of winds, would the message of the artwork be altered?”. Delving deeper into the third phase of the analytical observation, the teacher invited the students to cover some elements of the artwork, using their hands, such as the windmill or the river. Then the teacher asked them to observe what exactly would happen, in terms of the artwork’s quality, if these elements were, in fact, absent. This way, the students were led to the discovery of the significance of a specific object while, simultaneously, answering the following questions: “Is the presence of the windmill near the river important?” and “How is the windmill connected to the use of wind energy and the pumping of water from the adjacent river?”. Subsequently, the teacher, making use of additional sources (e.g., internet, encyclopedias, or related books), invited the students to compare this particular painting to other artworks by the same artist, created during the same time period and utilizing a similar subject, and encouraged them to identify the similarities and the differences between them and reflect on why these similarities and differences exist. Upon completion of the 3rd Phase, arises the critical question: “How can environmentally friendly forms of energy, such as wind energy, be exploited?”. The fourth and final phase of the Perkins artwork observation model concerned the review of the whole process. More specifically, the teacher invited the students to split into workgroups of 4-5 people and answer the question: “What is the link between the artwork and the critical question?”. At first, the students were asked to record the thoughts they had before approaching the visual artwork in question, and then to write down the thoughts they have now, after the process of approaching the artwork. Table 1: The art work used as a teaching tool.

Painting title

A windmill by a river shown in figure 1 below

Artist

Jan van Goyen (1596 – 1656)

Characteristics

29,4cm Χ 36,3 cm

Year created

1642

Web

http://www.nationalgallery.org.uk/paintings/ja n-van-goyen-a-windmill-by-a-river

Museum

National Gallery, London

Figure 1: A windmill by a river

Conclusion The evaluation process carried out after the implementation of the method revealed that, through the systematic observation of the work of art, the students: a) acquired knowledge and expressed awareness regarding issues related to the exploitation of mild forms of energy, b) developed a contemplative attitude towards environmental issues, c) acknowledged the possibilities offered by the utilization of works of art to the approach of environmental issues, d) became familiar with the methodology of approaching artwork as a tool for developing critical thinking, e) adopted a positive attitude towards the creative exploitation of important works of art in the approach of environmental education issues, and, f) developed their creative thinking and broadened their perception by using their imagination and ingenuity. In conclusion, through the use of an artwork, activation of the students was attempted, in order to actively engage them in the learning process and the examination of data and parameters of a subject. Jan Van Goyen’s painting, “A windmill by a river”, has been a useful teaching tool, utilized in the context of an effective and innovative approach to the environmental issue of renewable natural resources and, more specifically, wind energy. This process has, on the one hand, intrigued the interest of the students, who participated with warmth and excitement, and, on the other hand, made the teaching approach of these specific cognitive objects more experiential, interactive and lively for the teacher. As far as practical implications are concerned, the case study presented in this article can offer interesting insights for teachers, educators and researchers interfering with use of art works in education. The teaching approach suggested in this article could be adopted by educators who wish to teach individual modules of environmental education in an experiential and innovative way. In

this direction, the present study enriches the relevant literature and at the same time, provides ideas for the use of art works in the teaching of environmental sciences.

Acknowledgements The authors would like to thank the students who participated to the present research, giving a useful feedback to this innovative teaching procedure. In addition, the authors would like to thank the editor and the reviewers of this journal for their valuable comments on an earlier version of this paper.

References Anderson, T. (1993). Defining and structuring art criticism for education. Studies in Art Education, A Journal of Issues and Research, 34, 199-208. Barnes, R. (2015). Teaching art to young children. Third Edition. New York: Routledge Beck, H.U. (1977). Jan Van Goyen, 1596-1656: Poet of the Dutch Landscape. London: Alan Jacobs Gallery. Broudy, S. (1972). The Enlightened Cherishing: An Essay on Aesthetic Education. Urbana, IL: University of Illinois Press and Kappa Delta Pi. Burnham, R. & Kai-Kee, E. (2011). Teaching in the Art Museum. Interpretation as Experience. Los Angeles: The J.Paul Getty Museum Efland, A. (2002). Art and Cognition: Integrating the Visual Arts in the Curriculum. New York: Teachers College Columbia University. Eisner, E. (2002). The Arts and the Creation of Mind. New Haven and London: Yale University Press. Feldman, E. (1967). Arts as Image and Idea. New Jersey: Prentice Hall. Fragoulis, I. (2003). Life Long Learning and Environmental Education. Teaching Approaches. Athens: Papazisis (in Greek) Fragoulis, I. (2014). Local History-Educational Material for Life Long Learning Centers. Athens: Ministry of Education (in Greek) Gardner, H. (1990). Art Education and Human Development. California, Los Angeles: The Getty Education Institute for the Arts. Inwood, H. & Taylor, R. (2012). Creative Approaches to Environmental Learning: Two Perspectives on Teaching Environmental Art Education, International Electronic Journal of Environmental Education, 2(1), 65-75 Perkins, D. (1994). The Intelligent Eye: Learning to Think by Looking at Art. California, Los Angeles: The Getty Education Institute for the Arts. Kleiner, F.S. (2015). Gardnerâ&#x20AC;&#x2122;s art through the ages. 15th Edition. The Western Perpesctive. Boston: Cengage Learning Kokkos, A. (2011). Transformative Learning through Aesthetic Experience Towards a Comprehensive Method. Journal of Transformative Education, 8(3), 155-177 Neperud, R. (1997). Art, ecology and art education: Practices and linkages. Art Education, 50(6), 14-20. Phillips, N. & Fragoulis, I. (2012). The use of art in the Teaching Practice for Developing Communication Skills in adult, International Education Studies, (5)2, 132-138. Rosenthal, A. (2003). Teaching systems theory and practice through environmental art. Ethics and the Environment, 8(1), 153-168.