Why aren’t teachers making use of online video in the classroom? G. Powell, M. Santoso, M. Abuaish, K. MacAllister, E. Guo
Simon Fraser University 2009
1
Abstract A survey of actively practicing teachers in the Lower Mainland of British Columbia (N=199) with a response rate of 41% (N=81) was carried out to investigate why teachers are not using Internet video in the classroom. Of those who responded, 40% (N=32) reported that they were not using online video (OLV) in instruction. An analysis of the results showed that there were some demographic differences between users and non-users by grade taught, and that the most prevalent reasons for not using OLV were technical problems, a lack of equipment and a lack of skills. The reasons for not using OLV are reported and explored, with implications and recommendations for policy makers.
Introduction Learning from Video in the Classroom Film strips with audio have been a part of classroom instruction for well over half a century, bringing informative and engaging experiences to students around the world. More recently video tape and DVD formats have made it possible to pause, rewind, review and advance frameby-frame in order to foster a deeper understanding of physical phenomena and encourage discussion and critical thinking. Video can be used to expose students to the reality of historical events, such as the fall of the Berlin Wall or the lead up to the Gulf War, and to better understand phenomena that are of a spatial or temporal scale beyond perception, such as the interactions of molecules, the formation of black holes or the movement of continents. In post-secondary environments, Dequeker and Jaspaert (1998) note that video case studies have been a central component of medical diagnosis training for over 30 years and that video can be a highly effective and salient method of developing clinical reasoning skills. Video does have limitations Despite the benefits of video, it is not a replacement for reality. DeLoache and Korac (2003) found that for young children, fidelity to reality is important in developing a rich understanding of the world around them. The most memorable and transferable learning experiences are direct ones. In their review of research on video-based learning by very young children, DeLoache and Korac show that children learn significantly better from direct experience. The difference between medical students and very young children in learning from video is based on what Richard Mayer (2005) refers to as the Fidelity Principle (in the 4C/ID model of multimedia learning), which states that the more experience with the subject matter students have, the more they can benefit from high-fidelity representations like video as opposed to line drawings and other low-fidelity representations. One can conclude from this that video is most appropriate in situations that offer no alternative direct experience opportunities, and that learners benefit when
2
they can relate what they are viewing to their own direct experience - a guideline that suggests teachers should check for understanding at regular intervals during a video to ensure students are attending to relevant details. It should also be noted that video in the classroom can be abused by teachers with sub-optimal results. Hobbs (2006) used surveys and observations of 130 secondary-level teachers over the course of six years to better understand the uses of video in the classroom. Her study found that teachers sometimes use video to "fill time, keep students quiet, as a break from learning or as a reward for good behavior." She also found that many teachers do not use the technique of "pause, rewind, review". Some teachers were found to disengage while students were viewing video rather than actively engaging in the video with them. Using this medium as an attentional hook or to control student behavior are not optimal uses of video and may not benefit learners. Appropriate use of video brings benefits Lawson et al. (2006 and 2007) found that when used appropriately, students benefit significantly from the use of video in the classroom. In these studies, students who wrote answers to guiding questions out-performed students who simply took notes, suggesting that teachers need to make an effort to ensure that students attend to the relevant aspects of the video and remain engaged throughout. Far from being a passive activity, good use of of this medium in the classroom needs to be an active process for both teacher and student in order to be effective. When used appropriately, video can benefit learners. Advantages of Online Video (OLV) The advent of OLV has brought further advantages, as identified by Fill & Ottewill (2006): • • •
•
•
Control - Learners can more easily, and individually replay, stop, rewind, scan and search video in order to analyze or understand it better. Chunking - OLV tends to be parsed (and meta-tagged) into smaller, more easily digested pieces that are more readily searched and referenced. Synthesis - Students and teachers can weave OLV into their projects by linking and embedding, allowing for interpretation and context. Online videos can also be cut, edited and 'mashed up' to create entirely new productions, which can then be shared and reinterpreted again. Additionally, video repositories like YouTube and Google Video provide tools teachers and students can use to annotate clips (see: http://www.youtube.com/t/annotations_about), allowing for questions, commentary and emphasis. Speed - OLV can be accessed, deployed and transmitted more rapidly than VHS, DVD or other physical formats. Information is also available in a timely fashion - video captured on a cell phone during a political protest can be watched by people on the other side of the globe while the event is still taking place. Revision - Once a video has been created, it is easy to post a new version that is immediately available to all. This is far less costly in time and resources than re-burning DVDs, recording new video tape and distributing physical copies.
3
•
Motivation - By allowing students and teachers to become active participants in the creation, editing, distribution, annotation and revision of video (rather than simply being passive viewers), they become more engaged in the learning process.
OLV matures Over the last decade, OLV has blossomed into a medium more rich and diverse than cable television, providing teachers with millions of videos on demand that can be accessed immediately without any additional cost to school budgets. Obtaining videotape or DVD disks through district library services can be comparatively slow and limited, with turn around time of several days between ordering and delivery. Because resource purchasing may happen at specific times during the year, traditional video sources may not address up-to-the minute developments or issues. The quality of the audio and video stream is almost on par with traditional television broadcasts, and a new class of high definition video offers the highest quality images available anywhere. What is online video (OLV) and how is it being used? According to Rose and Lenski, the definition of Internet/online video (OLV) is “video content streamed over the Internet” (2005, p. 3). As a medium, OLV can assist learners in understanding complex concepts in science, mathematics and engineering as well as provide relevance through the display of actual historical events, debates and speeches. Mullen and Wedwick explain how useful YouTube is and how easy it is to find archived videos on the YouTube site. By using keywords and phrases to locate desired clips, one can find “the videos of the 1969 Apollo II landing, an interview with J.K. Rowling, a Vietnam War documentary or the video of a tour of Anne Frank’s Secret Annex (2008, p. 67)." In one example, they share the experience of using OLV in the classroom to help middle school students understand the definition of the vocabulary word nostalgia. “Their young age made it hard for them to feel nostalgia about the past. Rebecca asked them to think about children’s shows they may have watched when they were younger. Many of them remembered watching the PBS television show, The Big Comfy Couch, and Rebecca was able to quickly pull up an episode clip to watch. They began reminiscing about their preschool years and formed an authentic understanding of the term nostalgia at the same time ” (p. 67). Video resources can also be tailored and then streamed to meet targeted instructional needs. Members of the Cognition and Technology Group at Vanderbilt University claimed that using video specifically designed with embedded problems in a mathematics classroom allows students to learn in a “contextualized” way and to use their valuable knowledge as a tool to accomplish tasks in new situations or to solve real life problems (CGTV, 1992a, pp. 65-67). Hsin-Yih (2000) conducted research with grade five students using video-based anchored instruction in a mathematics classroom. She stated, “with the video, the students can watch the story and experience the situation more vividly, interacting with the embedded data more easily” (p. 60). Her study found that in this specific context, students had “a positive change of attitudes
4
toward mathematics and Anchored Instruction” (p. 66). Students also improved their thinking skills and were able to solve complex mathematics problems (p. 67). Many educators are making the same kinds of claims for OLV use in their own classrooms and are actively seeking ways to use this resource (Keddie, 2008). Video (whether it is on VCR, DVD or online) can inform and inspire learning in ways that traditional lectures cannot. It provides more visceral examples and memorable associations. OLV has the advantage of being available anywhere, anytime without being limited by the number of copies available in the school or district library. Because teachers in British Columbia have a mandate to incorporate technology into their instruction (BC Ministry of Education, 2005), using OLV (along with the many added features such as linking and embedding, sharing links with colleagues and students, and annotating clips) is rapidly becoming an essential teaching strategy at a time when nearly all teachers have access to the Internet in their classrooms. As well, powerful new search engines are being developed that focus exclusively on video content (i.e. blinkx.com) and in some cases also provide transcripts of the audio. Why is OLV not being used by all educators? Given the potential benefits of OLV, one would expect that every teacher with access would be exploring its application in their own practice. In reality, there are many teachers who are not. Why would teachers with access to the technology, particularly those who make use of conventional video, not use OLV? Perhaps their subject area does not lend itself to this technology. Perhaps they are not aware of the benefits or do not know how to apply OLV in their instruction. Perhaps they believe that there is no relevant or useful content available, or they lack the hardware or software. For some, it could be technical issues or a function of district policies that restrict access due to security filtering or firewalls. This study seeks to find the answers to these questions by surveying a number of teachers from Elementary, Secondary and postsecondary environments.
Research Purpose This study was undertaken in an effort to understand why teachers are not using OLV in the classroom. A review of teacher websites suggests a range of reasons such as lack of access or lack of content. Mullen and Wedwick discuss the fact that some teachers find that YouTube is banned in many school districts because “there are highly inappropriate videos available on the site” (2008, p. 68). It is anticipated that the possible reasons will cluster into the following categories: • Lack of awareness on the part of teachers of applicable content to their practice • Lack of technical skills or experience • Technical problems with or unavailability of hardware, software or networking • Pedagogical beliefs about the utility of the technology
5
This study intends to shed light on the reasons why OLV is not being used by surveying using a convenience sampling of teachers from Elementary, Secondary and post-secondary institutions in the Lower Mainland of British Columbia. While it is assumed that the reasons for not using OLV are shared among all levels, an examination of the demographic breakdown of the respondents will show if this can be confirmed. While teachers at all three levels face specific age and developmentally-related challenges (i.e. the differences between 6 year old students and university undergrads), there are still the constant, across-all-levels issues of access to technology, strategies for technology integration, the lack of technical expertise, sourcing of subject-specific content and instructor disposition towards OLV. Based on the results of this survey, a set of recommendations will be produced that will address the issues raised, with a view to benefiting both teachers and students in the process.
Research Questions and Approach Despite the benefits to learning, cost savings and widespread availability of OLV for classroom use, many teachers are not using the technology. The central research questions are: 1. 2.
Why are teachers not making use of OLV in the classroom? What are the defining characteristics (such as age, gender, and years of experience) of teachers who are not making use of OLV in the classroom?
This study is firmly anchored in an objectivist view: if the right questions are asked, it is possible to learn which elements influence the adoption of OLV in the classroom. It is predicted that there will be a correlation between age and use of OLV; it is expected that younger teachers, having a greater comfort level with internet technologies, are more likely to use OLV than older teachers. This would suggest a trend in which OLV use increases over time as older teachers retire and younger teachers enter the profession. A strong relationship between age and OLV use may suggest that more effort needs to be made to assist older teachers in seeing the benefits and obtaining the skills they need to take advantage of this resource. Given equal access to education and technology, it is expected that gender will have no predictive power over OLV use in the classroom. Will the stereotype of the gadget-loving male or the techno-phobic female emerge, or have these roles from the 20th century disappeared? There may also be differences between the grade level taught and the amount of use, with a positive correlation showing that the older the students, the more likely their teacher is to incorporate OLV into their classroom experience.
Methods Instrument
6
Surveying was used to gather the data because it enabled the researchers to query a wider range and greater number of teachers than would have been possible using in-person interviews, given the constraint of having to complete the project in a single semester. This method integrated effectively into the busy schedules and diverse locations of the teachers to be interviewed, and presented participants with a consistent set of questions as opposed to the variability that might have come from differences in interviewer tone of voice, non-verbal cues or wording. The survey (see Appendix A) starts with a demographic section containing four questions about the teacher’s age, gender, grade level taught and subject areas. This data was gathered in order to determine if there are any relationships between non-use of OLV and teacher demographics. This demographic section is followed by two questions that ask (a) whether or not the participant uses OLV in the classroom and (b) if so, how often. These two questions are intended to distinguish between teachers who already use OLV and those who never or rarely have, as it is the latter group that is being targeted in this research. (Participants who already use OLV in the classroom are directed to the final question as all of the intervening items are worded for those who are non-users.) The third and fifth items are open-ended questions that allow participants to express their reasons for not using Internet video in the classroom and their beliefs about the educational benefit of OLV. This gave the participants the opportunity to express their attitudes and opinions before being asked any questions that could bias subsequent answers. This was followed by a bank of fourteen 5-point Likert scale questions ranging from strongly agree to strongly disagree. The bank of 5-point Likert scale questions ranging from 'Strongly agree' (1) to 'Strongly disagree' (5) was designed to line up with the suspected reasons (as stated in the Research Purpose section) for why teachers are not using OLV. Pilot testing the survey After the survey was constructed, a pilot run was conducted to determine the suitability and clarity of the question items. Three instructors were selected : one high school science teacher and two elementary teachers (one at the primary level and one working in the Grade 4-7 range). The three teachers were given a pen and paper version of the survey to complete. Once they had worked through the instrument, a short interview was conducted with each participant. They were asked what they thought the questions meant in order to confirm that there were no misunderstandings. They were also asked if they felt the survey was too long, too repetitious, or missing important aspects of the issue. The feedback was very positive. All the participants were able to explain what they thought the questions meant, and their understanding was the same as the intent of the survey designers. All three wanted to see a general: “any final comments you wish to make” section at the end of the survey to pick up any additional thinking or feedback. Two of the respondents wanted to see some additional subject areas itemized in the demographic section. (In addition, the survey was shown informally to two other teaching acquaintances of the researchers to solicit additional reactions.) This feedback was used to refine the layout of the
7
survey. Limitations of the survey method This survey was conceived, designed, conducted and analyzed in a single semester by a group of five graduate students enrolled in a university course, which limited the amount of time available to complete the research. This resulted in the use of convenience sampling from a limited area. As a consequence, this survey should be seen as a pilot in need of confirmation by a larger, province-wide sample. Reliability According to Cresswell, in order for any particular instrument to be considered reliable, "individual scores from an instrument should be nearly the same or stable on repeated administrations of the instrument, and that they should be free from sources of measurement error, and consistent." (2008, p. 646). Reliability is a concern for researchers because if the results are not stable, or similar, then the data cannot be trusted. He goes on to say that there are three factors that could result in unreliable data collection : ambiguous questions, a variation in test administration procedures and participants who are not in the right state of mind (i.e. tired, or nervous.) (2008, p. 169). In order to maximize the level of reliability of the data collection methods, the researchers sought to address Cresswell's three factors as follows: • ambiguity: the questions were piloted with a number of teachers. This checked their understanding of the meaning of each question to eliminate any that seemed ambiguous. • test administration: since the survey was administered online (with a print option), there was no opportunity for the researcher to "vary" the presentation of the questions. Each participant received the same questions, devoid of additional elements such as tone of voice, rephrased items, or undue emphasis. • participant readiness: since respondents were not required to complete the survey at a specific time, or in a designated location, the data collection happened when the participants were ready. They could choose to complete the survey online, at their convenience, or print a hard copy and hand write their responses. The survey questions were created to align with the four suspected reasons for why teachers may not be using OLV. This allowed for an internal consistency check. Creswell explains that: “[s]cores from an instrument are reliable and accurate if an individual’s scores are internally consistent across the items on the instrument. If someone completes items at the beginning of the instrument one way, then they should answer the questions later in a similar way” (2005, p. 164).
8
In addition, participants began the survey by answering a "free-write" response regarding why they do not use OLV in instruction. This was intended to show if there is a strong concordance between their anecdotal responses and their choices in the Likert scale questions as further evidence of internal consistency. Validity Cresswell defines validity as meaning "that researchers can draw meaningful and justifiable inferences from scores about a sample or population." (2008, p. 649). This means considering the following: are the questions being asked meaningful to the participants? Is their understanding of the question the same as the researcher who designed it? Are the questions measuring what the researchers think they are measuring? Obviously, validity is a key requirement. A reliable survey is meaningless if the results do not lend themselves to justifiable inferences. The survey addresses the question of "face validity." Because the questions clearly address the four issues being targeted, there is little "interpretation" of the responses required in order to answer them. Participants are asked directly about their use of technology, their attitudes towards it, their competence in using it and their access to it. This concern was verified in the pilot process, ensuring that the final version of the survey asks questions the researchers believe they are targeting. "Content validity" is also addressed since there is an adequate number of questions that line up with the four key areas. There are however some concerns related to the external validity of the survey data. How generalizable can the findings be? Because the participants were selected using a convenience sampling rather than a random sample, it is possible to draw conclusions about the participant data, but not about the teaching population in general. Since the response rate to the survey was only 41%, there are limitations to how the conclusions can be applied. Response bias likely affected the completion rate: non-users may have less technical facility generally, and may have opted not to respond to an on-line survey format, and non-users may have been less likely to feel strongly enough about OLV to go to the trouble of responding.
Participants A convenience sample of teachers from schools across the Lower Mainland of British Columbia (N=199) was conducted, with a response rate of 41% (N=82). One of the respondents was excluded because s/he did not answer any of the questions, leaving 81 participants who were included in the analysis. Of those, there were 32 OLV non-users and 49 OLV users. See the Analysis and Results sections for comparisons, distributions and differences.
9
Figure 1. Counts of participants by grade showing relative proportions of users to non-users
10
Figure 2. Counts of participants by age showing relative proportions of users to non-users
11
Figure 3. Counts of participants by gender showing relative proportions of users to non-users
Figure 4. Counts of participants by subject showing relative proportions of users to non-users
12
Each of the project group members invited teacher groups they worked with/or have a connection with, in keeping with a "convenience sampling" approach. Participants were drawn from six high schools, three elementary schools, and two post secondary schools, as well as teaching staff from an ER at a local hospital, and Chemistry and Physics instructors from a local university. All participants resided in the Lower Mainland at the time of the survey and were chosen because they represented a good cross section of subjects and levels. By filling out the survey, instructors consented to becoming part of this study. Data collection procedure Two versions of the survey were created: a paper version for manual distribution or download, and an online version. The question order and wording were identical in both forms to help ensure consistency of response. Teachers were contacted and invited to participate in this study. Teachers who are currently using OLV answered questions 1, 2, and 23 while those who are non-users answered all questions. Because the focus was on those instructors who are not users of this technology, and their reasons for not doing so, this showed what percentage of the respondents were non users. Questions 1 and 2 gave a count of the total number, while collecting detailed data only from the non-users. The researchers invited teachers to participate using a combination of email and/or face-to-face distribution of the paper-based survey. Emails sent with the URL of the final version of the survey targeted groups on June 20th. Follow-up emails were sent with a "thank you/reminder" 3 days later (23th). Final reminder was sent on June 30th.
13
The data is primarily collected through an online survey tool provided by SFU[1]. The online survey also included a link on the survey page to a printable version for teachers who prefer this method of response. The completed print version could be handed back in person to a researcher, faxed or mailed. Data from the paper version were input into the online survey version manually. Data re-coding Survey data was downloaded from a secure university server as a text delimited file. This was imported into Microsoft Excel and the SAS-JMP statistical software and then converted to .XLS and .JMP files respectively. After conversion, text items were re-coded as numeric whole numbers as shown in the table on the left.
Text Coding, Re-coding and the Addition of New Reasons First round of text coding After the data was collected, the total number of responses to the three open-ended questions on the survey were as follows: • 31 participants answered question #3, which asked for reasons why the teacher was not using OLV in the classroom. • 28 participants responded to question #5, which asked teachers why they think that OLV might benefit learners in the classroom. • 11 participants gave final comments (question #23). Coding of the text-based items was performed by two researchers separately. Text was separated into discrete statements by parsing comments at conjunctions (e.g.: and, but) and punctuation (e.g.: period, comma, hyphen). Based on that, there were 56 statements from question #3, 32 statements from question #5, and 22 statements from question #23. The resulting 110 statements were coded according to their reasons for not using OLV, grouping items by the following constructs: • 0 = Not a reason (a filler statement)
14
• • • • •
1 = Lack of teacher awareness of the possible benefits of OLV 2 = Technical problems with OLV 3 = Lack of technical skills or experience with OLV 4 = Negative pedagogical beliefs about OLV 5 = Other (reasons not part of the initial four constructs.)
The inter-rater agreement for this coding was 70%. After a review of the disparities in rater coding, a consensus was reached the remaining 30%. Second round of coding: The Addition of New Reasons In the first round of text coding, the "other" reasons for not using OLV emerged. These reasons were not anticipated by the study designers. The comments labeled "other" in the first round of coding were then examined to see if there were any common areas of concern. The three themes that emerged were issues around trust, time and policy. They were coded as follows: • Other 5 = Trust in the reliability of the material available through OLV • Other 6 = Not having the time to use OLV • Other 7 = Policy issues Deriving Cronbach's alpha values Originally, the researchers generated a set of four main reasons for why OLV might not be used in instruction: 1. Lack of awareness of the benefits of using OLV 2. Lack of technical skills or experience 3. Technical problems with or unavailability of hardware, software or networking 4. Pedagogical beliefs about the utility of the technology These were used to create the set of agreement statement stems for the Likert-scale questions, which were grouped by construct as follows: • Questions {14, 16R, 19} (α = 0.31), seek insight into the lack of awareness of the possible benefits or applications in their practice. • Questions {9R, 10R, 11R, 13, 15} (α = 0.87) seek to underscore technical problems with or unavailability of hardware, software or network. • Questions {6R, 7R, 8R, 21R, 22R} (α = 0.60) seek to underscore the lack of technical skills or experience. • Questions {12, 17, 18, 20} (α = 0.64) seek backup of pedagogical beliefs about the utility of the technology. Cronbach's alpha values for some of the reason groupings fell below 0.70 indicating that these were not unified concepts or ideas. Further item analysis revealed that some of the concepts failed to overlap, and rather stood on their own as reasons - not matching the response patterns of any other reason. By going through and analyzing which questions were answered in a consistent cluster, a reconfiguration of clustered reasons resulted in groups with alpha values above 0.70 or unrelated reasons that stood on their own. The resulting eleven possible reasons why teachers are
15
not using OLV are listed below with reliability coefficients (Cronbach's alpha), α where appropriate. Items that were later reversed in analysis are indicated by an 'R' next to the survey question number:
Text Code 1
Reason, (Cronbach's alpha)
Survey Questions
Lack of of awareness 4R: I think that video content (from any source, including VHS, DVD or Internet) can benefit learners in the classroom. of the benefits, 18: I don't see a benefit to using online video in my subject area. (α = 0.87)
2
Technical problems, (α = 0.87)
9R: I think my classroom computer equipment is adequate for making use of online video. 10R: I think my classroom computer equipment is reliable and works properly. 11R: I think my school internet connection is reliable. 13: I am frustrated by technical difficulties when trying to make use of online video. 15: I would like to use online video in the classroom, but I don't have the equipment.
3
Lack of technical skills or experience, (α = 0.86)
6R: I am confident in my ability to use Internet for browsing web sites. 7R: I am confident in my ability to locate online video content. 8R: I know how to use computer equipment in the classroom to view online video. 16: I would like to use online video in the classroom, but I don't know where to find good content.
4
Not having the time or 12. I would like to use online video but I just don't have enough class time. being too busy
5
Concern over 14: I am concerned about students viewing inappropriate online video content. inappropriate content
6
lack of equipment
15: I would like to use online video in the classroom, but I don't have the equipment.
7
Lack of interest
17R: I would like to learn more about using online video in instruction.
8
Lack of content
19: There isn't any useful online video content that applies to my subject area.
9
Preference for older formats
20: Video is pedagogically useful, but I prefer more traditional VHS or DVD formats to OLV.
Note that question #15 is part of reason 2, as it is a technical problem, but it is also analysed separately, as it is also an issue worth examining in isolation. The analysis of text-based openended responses (see the First text coding process, above) revealed two additional constructs that were not considered in the development of the Likert scale questions.
10
Lack of trust in the reliability of the technology or the information available
No questions were asked in the survey to assess this, but it was found in the text responses.
16
Policy issues that
11 prevent use of OLV in No questions were asked in the survey to assess this, but it was found in the text responses. the classroom
Limitations introduced by new reasons The final 11 reasons used in the analysis include many items based on a single question. In a replication of this study, several questions would need to be devised for each reason, and questions would need to be added to address the final two reasons that emerged from the text analysis. This larger number of reasons serves to enrich the results with a wider variety of issues than originally identified. Third round of text coding Using the new set of 11 reasons, another coding process was completed by the same two researchers. They used the same 110 statements as those in the first coding process, with statements that were not considered to be reasons coded as a '0'. The inter-rater agreement for this coding was 81.8%. Of the 110 statements, all of the answers to question #5 described benefits of using OLV rather than reasons for not using it, so all of these were coded as '0', which removed 32 statements from the original 110, leaving 78. Six of the statements from questions #3 and #23 were also coded as '0', leaving a total of 72 statements made by teachers who do not use OLV in the classroom that describe the reasons why they do not. Counts of each of the reasons appear in Figure 5 in the Analysis section.
Analysis Text responses Counts were compiled for each of the 72 reasons provided by teachers for why they were not using OLV in the classroom.
17
Figure 5. Counts of reasons provided in text-based responses
Analysis of users vs. non-users based on demographic groups The first analysis shown in the results section breaks all teachers included in the study into users and non-users, then looked for differences between these two groups based on age, gender and grade taught. Both OLV user and non-user demographic data was compared for differences in the relative proportions of users vs. non-users. Results are reported as contingency tables with χ² and Pearson values (See Results section). Percentage differences were calculated for the Discussion section that examined reasons why there might be differences between groups by grade, in particular why so few post-secondary teachers are users. Analysis of reasons for not using OLV All users were removed, leaving only non-users in the analysis (N=32). For each, a distribution was generated showing responses, and percentages were calculated to show differences in counts between agreement and disagreement. This was done to reveal the number and proportion of non-users who identified with each reason. 1. Adding (Strongly Agree + Agree) = Agreement count 2. Adding (Strongly Disagree + Disagree) = Disagreement count 3. Dividing (Agreement / total non-users) = Percent in agreement 4. Dividing (Disagreement / total non-users) = Percent in disagreement
18
Results Age as a difference between users and non-users of OLV Comparisons by age of users and non-users in a contingency table resulted in an overall χ² Pearson value of 1.04, p < 0.78, meaning that any differences between these two groups by age are likely due to chance. In this sample, teachers in their 20's (50% users) were only slightly less likely than teachers in their 30's (61% users), 40's (69% users) and 50's (61% users) to be users of OLV. Note that in these data tables, the upper left cell serves as a legend to the calculated cells. The first two calculated values are Count and Total%. Data Table 1: Age by users and non-users
Gender as a difference between users and non-users of OLV Comparisons by gender of users and non-users in a contingency table resulted in an overall χ² Pearson value of 0.29, p < 0.59, meaning that any differences between these two groups by gender are likely due to chance. In this sample, female teachers (63% users) were only slightly more likely than male teachers (57% users) to be users of OLV.
Data Table 2: Gender by users and non-users
19
Grade taught as a difference between users and non-users of OLV Comparisons by grade level taught of users and non-users in a contingency table had only 2 primary teachers, both of whom were users (see left Data Table 3 below). A second comparison was done with the primary teachers removed (see right Data Table 3 below), resulting in an overall Ď&#x2021;² Pearson value of 17.02, p < .001. Figure 6. Usage proportions by grade
The low likelihood of this difference being due to chance was due to the fact that about 75% of both intermediate and secondary teachers were users, while post-secondary teachers showed the opposite proportion, with only 25% users. Post-secondary teachers surveyed were less likely to use OLV than teachers at other grade levels.
Data Table 3: Grade taught by users and non-users
Because post-secondary teachers were more likely to be non-users, a further analysis was done to compare the subjects taught by post-secondary users and non-users. The 24 post-secondary teachers included 1 history teacher, 1 math, 12 medical, 8 science and 2 'other'. Proportionally, 25% of medical teachers were users compared with only 12.5% of science teachers. Distributions of reasons for not using OLV in the classroom For each of the 11 reasons queried in the survey, non-users (N=32) responses are shown along with the percentage who agreed (agree + strongly agree) and disagreed (disagree + strongly
20
disagree) with each statement. Agreement distribution of non-user reasons
Percentage agreement
72%
Disagree or strongly disagree
23%
1
Are neutral
3%
Agree or strongly agree Figure 7: Lack of awareness of the benefits as a reason
28%
Disagree or strongly disagree
28%
2
Are neutral
44%
Agree or strongly agree Figure 8: Technical difficulties as a reason
53%
Disagree or strongly disagree
31%
3
Are neutral
16%
Agree or strongly agree Figure 9: Lack of skills as a reasons
21
44%
Disagree or strongly disagree
34%
4
Are neutral
22%
Agree or strongly agree Figure 10: Lack of time as a reason
53%
Disagree or strongly disagree
19%
5
Are neutral
28%
Agree or strongly agree Figure 11: Concern over inappropriate content as a reason
34%
Disagree or strongly disagree
22%
6
Are neutral
44%
Agree or strongly agree Figure 12: Lack of equipment as a reason
22
69%
Disagree or strongly disagree
22%
7
Are neutral
9%
Agree or strongly agree Figure 13: Lack of interest as a reson
63%
Disagree or strongly disagree
28%
8
Are neutral
9%
Agree or strongly agree Figure 14: Lack of content as a reason
75%
Disagree or strongly disagree
25%
9
Are neutral
0%
Agree or strongly agree Figure 15: Preference for older formats as a reason
23
Ranking of reasons for not using OLV Both the Likert-type questions and the text responses were ranked by how frequently the various reasons were given for not using OLV in the classroom:
Percent agreement ranking of Ranking of text responses by the percent reasonsgiven in Likert responses of total reponses given (% of all responses, N=72)
(% of non-users who agree or strongly agree with the reason)
1. Technical problems (19%) 1. Technical problems (44%) 2. Lack of skills (18%) 2. Lack of equipment (44%) 3. Not seeing the benefits (15%) 3. Inappropriate content concerns (28%) 4. Not having the time (13%) 4. Not having the time (22%) 5. Lack of equipment (11%) 5. Lack of skills (16%) 6. Policy issues (11%) 6. Lack of content (9%) 7. Lack of content (7%) 7. Lack of interest in learning more (9%) 8. Lack of trust (4%) 8. Not seeing the benefits (3%) 9. Preference for older formats (<1%) 9. Preference for older formats (0%) 10. Inappropriate content concerns (<1%) 10. Lack of trust - NOT TESTED FOR 11. Lack of interest in learning more (<1%) 11. Policy issues - NOT TESTED FOR Data Table 4: Rankings of reasons for not using OLV by frequncy of occurrence
Technical problems was the highest-ranked reason for not using OLV in the classroom in both the text and Likert responses. Not having the time was ranked fourth in both lists. If the text reasons for not using OLV in the classroom are given equal weight as the Likert reasons, the following rank-ordered list is the result: The most prevalent reasons for not using OLV given by non-users, aggregated from all responses: 1. Technical problems 2. Lack of equipment 3. Lack of skills 4. Lack of time 5. Not seeing the benefits of using OLV 6. Inappropriate content concerns 7. Lack of content 8. Policy issues 9. Lack of interest 10. Lack of trust 11. Preference for older formats The aggregate ranking above is intended to answer the question of why teachers are not using OLV in the classroom, and to show the relative importance of each of the reasons. The top 5
24
reasons in the list account for 75% of all reasons given in all responses.
Discussion The two research questions posed by this study are 1) Why are teachers not using OLV and 2) What are the defining characteristics (such as age, gender, and years of experience) of teachers who are not making use of OLV in the classroom? Reasons for non-use of OLV The Likert-scale questions were intended to find answers to the first research question, "Why are teachers not making use of OLV in the classroom?" The following statements summarize the responses reported in the results section for non-users (N=32): • • • • • • • • •
72% see the benefit in using OLV, while only 3% do not. 44% are experiencing technical difficulties, while 28% are not. 53% feel that they have the skills to use OLV, while 16% do not. 22% don't have the time to make use of OLV, while 44% do. 28% are concerned about students viewing inappropriate content, 53% are not. 44% lack the equipment, while 34% feel their equipment is adequate. 69% are interested in learning more about OLV, while 9% are not. 9% believe there is a lack of content in their subject area, while 63% do not. 75% prefer OLV to older formats, and none of the non-users surveyed did not.
Figure 16: Counts and relative proportions of agreement to each of the reasons found in Likert questions
Thus, we can say that in general, regarding OLV use, most non-users do see a benefit, prefer the
25
format and would like to know more. Most non-users also believe there is adequate content available in their subject area. About half of non-users either lack the equipment or are experiencing technical difficulties and about half feel they have the skills to use OLV. This issue of "technical difficulties" was raised by respondents in the written responses to question 3. They reported such issues as: difficulties using OLV when moving from one class to another, old equipment that does not function reliably taking too long to load or breaking down frequently, a slow or interrupted stream of OLV content, the lack of a reliable internet connection, problems with file size, incompatibilities between the hardware and software installed on different computers and a lack of technical support. This wide range of concerns points to the need for instruction and technical support for teachers in the field. The issues of reliability and trust are also worth mentioning as they are closely intertwined. While the results show a trend of interest in using OLV and belief in its pedagogical benefits, at the same time, some teachers wrote in their comments that they hesitate to use it because it may not be reliable. How can they be guaranteed that the video clip will be there when the class needs it? Teachers may be tempted to use traditional media as such forms are seen as more dependable. One might assume that if the teachers had more confidence in the school's equipment, internet connection, technical support and their own skills, a different pattern of OLV use might start to emerge. A question of demographics In addressing the second research question, among the participants in this study, the overall distributions between users and non-users for both age and gender showed no differences that could not be attributed to chance (see Results, Data Table 1). There was a trend observed that showed a 19% increase between teachers in their 20's, and those in their 40's. The sample was too small for this trend to be generalizable, but if this pattern held for a larger sample, it may be that over the 20 year period between 20's and 40's, teachers gain experience and as a result, Figure 17: Percent of users by age group
are more likely to have experimented with and learned about the application of technologies in their practice, making them more likely to use OLV. The 8% drop-off in users between the 40's and 50's age group could be due to older teachers being slightly more set in their ways, however, there were no teachers in the sample who answered agree or disagree to question #20, claiming to prefer older formats, so it appeared as though all of the teachers included in the analysis (N=81) would prefer to use OLV, regardless of age, gender or grade taught. These patterns could
26
be explored in a further study to find out if younger teachers are less likely to use OLV in the classroom, and if there is any effect of age on being less likely to adapt new technologies into their practice. There was a difference between users and nonusers by grade level taught (see Results), particularly between secondary teachers, 74% of whom were users, and post-secondary teachers, only 25% of whom used OLV in the classroom. Due to the small sample size and the convenience sample that included postsecondary teachers who were 83% medical and science teachers, this sample is probably not representative of all post-secondary teachers, Figure 18: Subjects taught by post-secondary teachers surveyed, users and non-users (N=24).
but this difference is worth investigating in further studies. Responses of secondary and postsecondary teachers to question #19, "There isn't any useful content..." showed that 100% of secondary teachers included in the analysis (N=43) either disagreed or strongly disagreed with that reason, indicating that they all believed that there was useful content, while only 54% of post-secondary teachers included in the analysis (N=24) either disagreed or strongly disagreed, meaning that just under half of post-secondary teachers believed there was useful content in their subject area.This is surprising as most of the post secondary teachers were medical and science teachers, fields that are rich of visual diagnosis and procedures. One factor not considered in the original survey is that standard lecture format dominates in colleges and universities, and that OLV would more likely be assigned as homework than to be used in the classroom. Other factors that could have contributed to this trend are a lack of trust in OLV content that is not typically peer reviewed for academic purposes, concerns around copyright issues and post secondary institution policies regarding open source content. These possible motivations were not addressed directly in the survey. Rather, it was the open-ended responses that raised these "postsecondary" concerns. As a consequence, it is recommended that this subgroup be further investigated in future studies to better understand their particular point of view regarding OLV use in their fields. Future studies could examine the use of online video in post-secondary institutions with a larger, wider sample to determine areas where non-use is highest and to investigate why this is the case.
Conclusion, recommendations and directions for further research On-line video can bring many benefits to the 21st century classroom. OLV (along with the many added features such as linking and embedding, sharing links with colleagues and students, and
27
annotating clips) has the potential to be an essential teaching tool at a time when nearly all teachers have access to the Internet in their classrooms. The Ministry of Education mandate to integrate technology into instruction suggests the need for teachers to be skilled users of this medium. The findings of this preliminary study are limited by the small number of teachers surveyed and the restricted number of school districts participating. While there was little to no effect of gender and age on OLV use among the teachers surveyed, a larger sample size is needed confirm this result in the general teaching population. The fact that the primary method of surveying was electronic is another limiting factor. Future surveys should be widely distributed, with printed forms being the default format. This would likely generate a better response rate, especially in reaching non-technologically experienced teachers. The key areas that need further exploration are the limiting factors of skill deficit, technical difficulties and equipment availability: 45% believe they lack skills, 45% cited technical difficulties, 42% lacked the equipment. If these percentages hold true for the larger population of teachers, addressing these issues through training and policy could substantially increase the use of OLV in the classroom. A smaller number (27%) expressed concern over students viewing inappropriate content. This can be dealt with by in-servicing teachers on how to find appropriate material. This would also address the concern expressed by some teachers (9%) that there is a lack of content available in their subject area. Recommendation 1: Better inform teachers The Ministry of Education or a consortium of districts could take the lead in this area by distributing information to schools (via web or print) outlining the benefits of OLV to teachers. This should include examples relevant to individual subject areas. District pro-d committees could be another avenue of information distribution. For the 57% of teachers who would like to use OLV but do not have the time, practical tips on how to save time when using OLV should be made available. Information on where to find reliable sources of content for every major subject area could be distributed to schools via school teacher-librarian networks. For teachers concerned about students viewing inappropriate content, methods of screening out irrelevant video, as well as information on talking to students about inappropriate content could also be included. Districts could offer additional information sessions for parents and teachers on how to deal with questionable content. Recommendation 2: Perform a training needs analysis and offer technical skills training Making technical skills training available to teachers through online resources that could include video-based instruction, live web-based seminars or classroom workshops on professional development days would address many of the issues caused by a lack of technical skills or with
28
technical difficulties often experienced by teachers. It is recommended that a follow-up survey be conducted with a wider sample of instructors to ascertain exactly where the skill gaps are and to identify the specific technical difficulties that teachers commonly face. This would inform a needs analysis that could be used to develop an effective training program. For example, something as simple as knowing how to download a flash video clip as a backup in case the online version is not accessible would make a difference to teacher confidence. For technical problems, a searchable wiki-based resource or discussion board could be created where teachers could post problems they are experiencing. Responses could be posted by peers and technical experts across the province. It is also possible to create an online decision support system that could be used to help troubleshoot and diagnose problems in the field as they happening. Recommendation 3: Provide access to low-cost equipment that would enable use of OLV For the 42% of teachers who lack the equipment to make use of OLV, surveys should be conducted with a larger sample to determine exactly what those equipment needs are. Schools in British Columbia have access to programs that give away free computers or can take advantage of economies of scale by making equipment purchases for multiple districts through the Ministry of Education. Districts could also be made aware of free open-source software they can use for OLV in the classroom.
Directions for further research: This study is best understood as a pilot project whose aim was to look at OLV use trends in a very specific population of teachers. The trends identified deserve a more in-depth investigation, with a larger, more diverse sample of instructors. The researchers intend to present the findings of this project to the Ministry of Education of British Columbia and to undertake a larger study exploring this issue once funding is secured.
29
References BC Ministry of Education. (2005). Retrieved June 2, 2009, from http://www.bced.gov.bc.ca/ technology/6-9.htm Cognition and Technology Group at Vanderbilt. (1992a). The Jasper experiment: an exploration of issues in learning and instructional design. Educational Technology Research and Development, 40(1), 65 â&#x20AC;&#x201C; 80. Creswell, J., W. (2005). Educational research : planning, conducting, and evaluating quantitative and qualitative research (2nd Ed). Upper Saddle River, N.J. : Merrill. p. 164. Creswell, J., W. (2008). Educational research : planning, conducting, and evaluating quantitative and qualitative research (3rd Ed). Upper Saddle River, N.J. : Merrill. p. 169, 646, 649. DeLoache, J.S. & Korac, N. (2003). Video-based learning by very young children. Developmental Science, 6(3), 245-246. Dequeker, J. & Jaspaert, R. (1998). Teaching problem-solving and clinical reasoning: 20 years experience with video-supported small-group learning. Medical Education, 32(1), 384-389. Fill, K. & Ottewill, R. (2006). Sink or Swim: taking advantage of developments in video streaming. Innovations in Education and Teaching International, 43(4), 397-408. Hobbs, R. (2006). Non-optimal uses of video in the classroom. Learning, Media and Technology, 31(1), 35-50. Hsin-Yih, C. S. (2000). Using video-based anchored instruction to enhance learning: Taiwan's experience. British Journal of Educational Technology, 31(1), 57. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=3157737&site=ehost-live Keddie, J. (2008) TEFLclips. Retrieved June 2, 2009, from http://www.teflclips.com Lawson, T.J., Bodle, J.H. & Houlette, M.A. (2006). Guiding Questions Enhance Student Learning From Educational Videos. Teaching of Psychology, 33(1), 31-33. Lawson, T.J., Bodle, J.H. & McDonough, T.A. (2007). Techniques for Increasing Student Learning From Educational Videos: Notes Versus Guiding Questions. Teaching of Psychology, 34(2), 90-93. Mayer, Richard E. (2005). The Cambridge Handbook of Multimedia Learning. Cambridge, U.K. New York : University of Cambridge. Mullen, R., et. al., Avoiding the Digital Abyss: Getting Started in the Classroom with YouTube,
30
Digital Stories, and Blogs [Part of a special issue: Integrating Technology into the Classroom]. The Clearing House v. 82 no. 2 (November/December 2008) p. 66-9. Rose, B. & Lenski, J. (2005). Internet and multimedia 2005: the on-demand media consumer. Arbitron Inc./Edison Media Research. Retrieved June 1, 2009 from http://internet.arbitron.com/ downloads/IM2005Study.pdf Thorndike, R., M. (2005). Measurement and evaluation in psychology and education (7th Ed). Upper Saddle River, N.J. : Pearson Education. Chap 5.
31
Appendix A The survey form used in this study:
ONLINE VIDEO IN THE CLASSROOM SURVEY The researchers conducting this survey are graduate students enrolled in the Educational Technology and Learning Design program at Simon Fraser University. The objective is to gather information on teacher use of online video (OLV) in the classroom. You are invited to participate by responding to the following questions, which should take no longer than 15-20 minutes. Your responses are completely anonymous and data collected will be used only for this class assignment and stored on a password-protected server accessible only by the students conducting this survey. Data will be destroyed at the end of the project. There is no way to link your response to your identity. Your participation is voluntary and you can choose to stop or withdraw at any time. Completing this survey implies that you agree to have your responses included in the analysis. If you have questions or concerns at any time about the study or the procedures, you may contact our instructor, Alyssa Wise at 778-782-8046 or by e-mail at afw3@sfu.ca. NOTE: The term ‘Online Video’ (OLV) includes video content streamed over the Internet as well as downloads made from selected Internet video content and presented off-line. Some examples of OLV sources are YouTube, Google video, MSN video, and TeacherTube. Your age: □ 20 to 29 □ 30 to 39 □ 40 to 49 □ 50 to 59 □ 60 to 69 □ 70 or over
Gender: □ Female □ Male
The grade level you teach: □ Primary (K to 3) □ Intermediate (4 to 7) □ Secondary (8 to 12) □ Post-secondary
Your general subject area(s): □ Drafting or Trades □ Economics or Consumer Ed. □ Fine arts, Drama or Music □ History or Social Studies □ Languages or Literature □ Mathematics/Stats/Geometry □ Medical/Health □ Physical Ed. or Sports □ Sciences (Phys/Bio/Chem…) □ Other
Please indicate whether or not the following statements are true of your teaching practice. 1) I currently use online video in the classroom □ Yes (continue to question2) □ No (continue to question 3) 2) If you currently use online video in the classroom, how often do you use it?
32
□ Several times a day
□ Daily
□ Weekly
□ Monthly
□ Less than once a month
(If you have answered “Less than once a month”, continue to question 3.) Otherwise, you can skip to the final question in this survey, Question 23.
33
3) If you are currently not using online video in the classroom, explain why this is the case
Please indicate your level of agreement or disagreement with the following statement by selecting the one answer that most closely matches your opinion. 4) I think that video content (from any source, including VHS, DVD or Internet) can benefit learners in the classroom.
□
□
□
□
□
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
5) Explain why you think this
34
Please indicate your level of agreement or disagreement with the following statements by selecting the one answer
that most closely matches your opinion. 6) I am confident in my ability to use Internet for browsing web sites. □ Strongly Agree
□ Agree
□ Neutral
□ Disagree
□ Strongly Disagree
□ Disagree
□ Strongly Disagree
7) I am confident in my ability to locate online video content. □ Strongly Agree
□ Agree
□ Neutral
8) I know how to use computer equipment in the classroom to view online video. □ Strongly Agree
□ Agree
□ Neutral
□ Disagree
□ Strongly Disagree
9) I think my classroom computer equipment is adequate for making use of online video □ Strongly Agree
□ Agree
□ Neutral
□ Disagree
□ Strongly Disagree
10) I think my classroom computer equipment is reliable and works properly □ Strongly Agree
□ Agree
□ Neutral
□ Disagree
□ Strongly Disagree
□ Disagree
□ Strongly Disagree
11) I think my school internet connection is reliable □ Strongly Agree
□ Agree
□ Neutral
12) I would like to use online video but I just don’t have enough class time □ Strongly Agree
□ Agree
□ Neutral
□ Disagree
□ Strongly Disagree
13) I am frustrated by technical difficulties when trying to make use of online video □ Strongly Agree
□ Agree
□ Neutral
□ Disagree
□ Strongly Disagree
14) I am concerned about students viewing inappropriate online video content □ Strongly Agree
□ Agree
□ Neutral
□ Disagree
□ Strongly Disagree
15) I would like to use online video in the classroom, but I don’t have the equipment. □ Strongly Agree
□ Agree
□ Neutral
□ Disagree
□ Strongly Disagree
16) I would like to use online video in the classroom, but I don’t know where to find good content. □ Strongly Agree
□ Agree
□ Neutral
□ Disagree
□ Strongly Disagree
35
17) I would like to learn more about using online video in instruction □ Strongly Agree
□ Agree
□ Neutral
□ Disagree
□ Strongly Disagree
□ Disagree
□ Strongly Disagree
18) I don’t see a benefit to using online video in my subject area. □ Strongly Agree
□ Agree
□ Neutral
19) There isn’t any useful online video content that applies to my subject area □ Strongly Agree
□ Agree
□ Neutral
□ Disagree
□ Strongly Disagree
20) Video is pedagogically useful, but I prefer more traditional VHS or DVD formats to OLV □ Strongly Agree
□ Agree
□ Neutral
□ Disagree
□ Strongly Disagree
21) If you currently use online video at home during your personal time, how often is this? □
□
□
□
□
Several times a day
Daily
Weekly
Monthly
Less than once a month
22) Have you ever uploaded a video for others to watch? If so, how often have you done this? □
□
□
□
□
More than 10 times
6 to 10 times
2 to 5 times
Once
Never
36
23) Do you have any final comments?
Thank you for taking the time to participate in this survey.
You can fax your completed survey to: Attention: Online Video Survey 604-277-0741 or Mail your completed survey to: Online Video Survey 5368 Cambie Street Vancouver BC V5Z 2Z8 [1] http://websurvey.sfu.ca/survey/37945928
[1] http://www.bced.gov.bc.ca/technology/6-9.htm, and http://www.bced.gov.bc.ca/irp/drafts/ela812_draft.pdf
37
Appendix B Alphabetical list of free, reliable, trustworthy, educational video content: Education Television
http://education-tube.com/
Geography at the Movies
http://www.gatm.org.uk/
GEOSET
http://geoset.group.shef.ac.uk/
History Classroom
http://www.history.com/content/classroom/
Periodic Table of Videos
http://www.periodicvideos.com/
MIT Video Gateway
http://watch.mit.edu/
MIT World
http://mitworld.mit.edu/browse/
Nobel Media
http://nobelprize.org/nobelmedia/
PBS Nova Archive
http://www.pbs.org/wgbh/nova/
Research Channel
http://www.researchchannel.org/prog/
Science Daily Videos
http://www.sciencedaily.com/videos/
Sqooltube
http://sqooltube.com/
Teacher Tube
http://www.teachertube.com/
TED Talks
http://www.ted.com/
The People's Archive
http://www.peoplesarchive.com/
Tuva Project
http://research.microsoft.com/apps/tools/tuva/
Vega Science Trust Archive
http://www.vega.org.uk/video/
Video Lectures
http://videolectures.net/
National Film Board
http://www.nfb.ca/
38