Anytown_Qualitative by Scott Warren

Abstract This paper examines the qualitative findings from a mixed-methods comparison study of the use of an online multi-user virtual environment called Anytown which supplemented face-to-face writing instruction in a fourth grade classroom to determine implications for the design of such environments and the reported impact of this design on students and teacher. Taking a case-based approach, this study examined the experiences of 44 students in two classrooms in order to detect major differences between the participation of students in a class in which face-to-face instruction was their only means of practicing and receiving feedback on their writing and a second class which provided students with an additional eight hours of opportunity to practice descriptive writing within the Anytown multi-user virtual environment. The main findings suggested that several elements must be present in the design of digital learning environments stemming from literature on using games and learning as a means of encouraging on-task behaviors. Further, it was noted that with a social constructivist design, specific forms of scaffolding that emerge from game context should be developed within the system to encourage student peer cooperation and use of system affordances while reducing reliance on faceto-face direction-giving. (Keywords: online, MUVE, on-task behavior, games)

This research was supported in part by a CAREER Grant from the National Science Foundation, and directly by the National Science Foundation Grants #9980081 and #0092831.

Introduction After years of theorizing that the use of video games and simulations should have positive impacts on student learning (Aldrich, 2003; Cassell & Jenkins, 2000; Dickey, 2007; Jenkins, Squire, & Tan, 2003b; Prensky, 2001), research into their use in elementary and middle school settings has begun to support these claims with data. The use of Multi-user Virtual Environments (MUVEs) such as Quest Atlantis and River City have been correlated with improved content area learning in science as measured by items taken from state standardized tests. The results of studies examining game-like learning environments such as Indiana Universityâ&#x20AC;&#x2122;s Quest Atlantis and Harvardâ&#x20AC;&#x2122;s River City have correlated use of them with increased motivation to learn (Dede, Ketelhut, & Ruess, 2006; Tuzun, 2004). The Anytown MUVE was developed as a means to supplement existing instruction and provide a space within which students could experience a choice-based, multi-path narrative, interact with digital characters in a dynamic manner, and practice descriptive and persuasive writing in the immersive manner most often found in either the real world itself or video games, detailed simulations, and authentic problem-based, constructivist learning environments. Anytown, modeled as a small town in a generic, rural Midwest state, is one world among several within the Quest Atlantis (QA) system which has the primary goal of engaging students in science inquiry learning. The main difference between Anytown and much of the rest of the QA is that it focuses on student descriptive writing skills. While elements of science inquiry are present within the narrative context and interactive activities of the unit, students in Anytown role-played as cub reporters at the small town newspaper and were required to engage in investigative

reporting. Situating learners within a writing role that they could conceivably engage in as working professionals later in life was expected to increase student engagement. Creating relevant learning tasks has been found to be effective with other forms of anchored instruction (Bransford, Vye, Bateman, Brophy, & Roselli, 2003; Vanderbilt, 1990, 1993). Figure 1 presents Anytownâ&#x20AC;&#x2122;s Main Street, which acts as an interactive space for locating details, evidence, and clues to be included in student writing.

Figure 1. Main Street in Anytown.

Events that learners investigated include arson at a local landmark building, vandalism of historic locations, and strange occurrences in a local park.

While a more complete description of the design of Anytown and the quantitative results measuring learning gains during its implementation are reported elsewhere (Warren, Barab, & Dondlinger, In Press; Warren & Dondlinger, In Press), this article focuses on reporting the findings of a qualitative, case study approach and seeks to contextualize those results within the body of research on games for learning as well as the full scope of research on the Anytown MUVE.

Research on games for learning While the evidence is slowly building that complex digital environments such as video games, digital simulations and MUVEs can positively impact student time on task and attitude towards learning subject matter (Barab et al., 2006; Jenkins, Squire, & Tan, 2003a; Squire, 2006; Squire, Giovanetto, Devane, & Durga, 2005; Steinkuehler, 2004; Tuzun, 2004), the question as to whether media and software is actually impacting student learning remains (Clark, 1991; Kozma, 1991). Much of the research literature related to using games for learning that has employed empirical methods has not shown significant differences when using this form of environment in comparison with direct instruction (Hays, 2005). Further, the complexity of these digital systems makes it very difficult to parse out what pedagogical element is actually responsible for learning gains, whether it be the interaction among student, instructor, system, system agents, pedagogical tools, or the epistemic stance that drove the development of the curriculum. A recent review of 160 mathematics and reading software products conducted with the support of the United States Department of Education (2007) found no evidence that any digital product

impacted student learning at a statistically significant level. While the educational technology community questions both the methodology and validity of the study, the cost to benefit ratio involved with these complex systems given the myriad confounding variables and varied research designs is valid and troubling (Nagel, 2007). Nevertheless, what is at the heart of game systems that appears to be correlated with a willingness to engage for long periods are activities that allow for play. Play has been hypothesized to be one of the most fundamental forms of human activity and means of learning (Baudrillard, 1994; Derrida, 1997; Wittgenstein, 1968). It allows for the mindâ&#x20AC;&#x2122;s exploration of the rules and consequences of engaging with or breaking them. In some instances, this play is subtle, with the learner testing their finger against the blade of the rule and discovering it is unpleasant. In others, players examine the rules by slamming into them physically, learning their harsh consequences concurrently. Vygotsky (1978) observed that children at play encounter a number of the rules to which they submit freely as part of the act of play, a theory supported by the research of Barab and Jackson (2006). A graphic depiction of the view of the conception of play that guided the design of Anytown can be seen in Figure 2.

NOTE:

= moments of play or interplay

Figure 2. Conception of play as interaction between experience and internal rules From Warren & Jones (in press)

However, the design of games or game-like systems by educators and researchers is largely unguided by professionals in the field of commercial game design, and scientific analysis of these complex systems is yet incomplete. An exception is Squireâ&#x20AC;&#x2122;s (2004; 2005; 2006) work with the real-time strategy game Civilization III, an analysis of a curriculum designed around a commercially produced video game rather than the design of an entirely new interactive system. Despite a number of books on the subject of game design, commercial companies have a vested interest in keeping the secrets of what makes a game successful held in proprietary patents, which limits the transparency of their process for educators interested in developing similarly engaging products. As a result, instructional designers seeking to leverage the purported benefits of the use of games, simulations, and digital learning environments have an incomplete picture of

either the processes or costs associated with effectively designing these digital learning spaces. The Anytown MUVE was a successful design intervention in terms of its principal design goals: reducing the amount of teacher time spent answering directional or procedural questions, increasing the amount of voluntary student writing, and improving student scores on a standardized writing assessment. The quantitative methods used to measure these results are described more fully elsewhere (Warren et al., in press), but are summarized in Table 1.

Table 1. Quantitative results from Anytown study two

Teacher time spent answering directional or procedural questions Student voluntary writing Student scores on standardized writing assessment

Treatment M = 12.118, SD = 6.6951

Comparison M = 28.413, SD = 3.9033

Differences t (15) =5.947, p = .043

M = 1.0870, SD = .288

M = .000, SD = .000

t (40) = -16.410, p =.006 F (1, 40) = 4.32

However, because of the complexity of the variables introduced by the use of such a designed digital environment, the results of these scores indicates that the Anytown MUVE was correlated with improved writing and elements of it are likely to have played a role in these differences. The quantitative data fails to indicate what elements within the MUVE or within the classroom environments, both comparison and treatment, might

have facilitated these improvements. Moreover, these data do not suggest how the designed environment could be improved to better serve both student and teacher experience. The purpose of this analysis is to apply qualitative, case study methods to contextualize the quantitative findings and better reveal what worked well and what could be improved. The principal research questions for this analysis follow:

1. Did embedded pedagogical elements reduce teacher time answering directional or procedural questions? 2. Did the motivating elements (compelling narrative tied to game challenges and rewards) increase student motivation, the factor expected to impact voluntary writing? 3. Did the immersive context and/or more meaningful guidance and feedback from the teacher impact student writing achievement?

The rationale for employing these qualitative methods follows Denzin & Lincoln’s (2003) prescription for a method that “seeks answers to questions that stress how (sic) social experience is created and given meaning” (p. 13). This case study was used to examine “a phenomenon of some sort occurring in a bounded context” (Miles & Huberman, 1994 , p.25). Jonassen and Hernandez-Serrano have described case based reasoning as “entail(ing) the elicitation, analysis, and inclusion of stories as a primary form of instructional support” (2002, p. 65). The studied phenomenon in question was

student and teacher experience with the Anytown digital learning environment as compared to more traditional classroom instruction.

Research methods Qualitative methods tend to look at qualities, characteristics, and attitudes inherent in a system. The use of observations and interviews as primary methods of data collection allows qualitative researchers to make sense of a situation or of what happened in a system prior to an intervention, and what is happening since that intervention (Denzin & Lincoln, 2003; Gall, Borg, & Gall, 1996). This examination of the Anytown system took place as a formative review as well as a summative evaluation of the interactions, benefits, and detriments stemming from the implementation of this intervention.

Participants and setting The research setting was an elementary school in a small, Midwestern city near a large, land-grant research university. Participants included 44 students in two 4th grade classrooms and their teachers. Both teachers reported being strong users of face-to-face problem-based learning environments in their classroom instruction. One classroom functioned as a comparison; the other classroom supplemented traditional instruction with the Anytown technology-supported learning environment as a treatment condition. Students were quasi-randomly selected by the schoolâ&#x20AC;&#x2122;s computer system for assignment to their respective classes in two months prior to the study.

Data Collection Methods Data collection occurred over a four-week period using observation, interviews, and video recording methods. A total of 20 periods of instruction were documented: ten periods

in the comparison classroom and ten periods in the treatment condition. Five researchers collected and coded the field notes, interview transcripts, and transcripts of recordings for each class period involved in the study. Field notes. At least three observers trained in qualitative methods took notes in each classroom over the course of the collection period. In the treatment classroom, observers focused on documenting student interaction with the in-system embedded scaffolds, learner willingness to complete tasks that included goals nested within the game narrative, and out-of-system face-to-face discussions related to the narrative context or the explicit rules of the learning system. During laboratory and classroom instruction periods, researchers observed the teachers in both the comparison and treatment group and recorded instances of teacher procedural or directional questions answering behaviors in their field notes. Researchers then developed a coding scheme to classify utterances documented in the field notes. Codes were triangulated and member checked across observers. The coding scheme agreed upon by the five observers is detailed in Table 2.

Table 2. Field note coding scheme Text Color Light blue Dark blue Red Light green Pink Light Gray - (25%) Yellow Violet ( ) = text tag

Code Peer help (ph) Voluntary activity (Quests, outside school work) (va) Frustration point, problem, or question (fp) Informal assessment Hypothesis (nascent or otherwise, includes claims) (h) Clarifying question (cq) Teacher redirect to system (tr) Other tension (ot)

Analysis began with a large number of possible codes identified by each member, and each was then folded into larger categories that were agreed upon by the group. Each category is related to student and teacher discourse. Informal and semi-structured interviews. Researchers also conducted semistructured and informal interviews with teachers and students in the comparison and treatment both during the data collection period as events occurred that warranted interview and at the conclusion of the collection period. These interviews provided a means of documenting student and teacher learning and teaching experiences and to verify codes generated by other data collection methods. Audio and video recording. Within the naturalistic setting of the computer lab, treatment classroom, and comparison classroom, MP3 audio recordings of student activity were made and transcribed. Examining the treatment teacherâ&#x20AC;&#x2122;s normal classroom writing instruction allowed a comparison between the teachers to determine the degree of similarity between them. Both sets of teaching activities could then be compared with learning activity and instruction in the Anytown supplemental treatment sessions. All

audio and video data was collected throughout the period as a means of capturing typical student interaction outside the digital environment and during face-to-face, teacher-led activities in the treatment condition. Data Analysis Methods Using case-based interpretive approach (Gall, Borg, & Gall, 1996; Robson, 2002), field notes and transcripts were coded line by line and integrated with reflections and reviewing comments. Another researcher on the team then reviewed the codes, comments, and reflections to check for validity and agreement regarding each. Once the validity of the codes was established, they were then folded into larger categories. A selection of three full class periods of coded field notes, interview transcripts, and transcripts of recordings for each class were analyzed. This selection represents seven hours and thirty-two minutes of transcript text and field notes related to the sixteen hours of observation. The study analyzed 1,982 lines of text, which included 23,968 words and 111,371 characters. The codes that emerged surrounded instructional issues related to the teacher, students, directions, researcher interactions, and power relationships among all participants. These were represented by 22 codes that addressed the interactions of participants, researchers, and system. A description of the codes used to produce these results is presented in Table 3.

Table 3. Coding scheme for teacher and student speech acts within transcripts/field notes Indicating marker

Coded Meaning

Treatment Number of lines/ (Percentage)

Comparison Number of lines/ (Percentage)

343 (33.6%)

537 (55.8%)

Black text Light blue text

Non-relevant speech act Directional question asking

Sea green text

Technical problems

104 (10%)

7 (.7%)

Plum text Gray text Dark red text Orange text Dark yellow text Blue-gray text Red text Lavender text Tan text Turquoise text Rose text

Within system directions Clarifying question Interface help Rule-setting Student or teacher frustration School system tension Direct teaching Student information offered Empathy expression Grammar question/instruction Teacher informational statement

81 (8%) 222 (22%) 97 (10%) 203 (20%) 71 (7%) 32 (3%) 337 (33%) 51 (5%) 23 (2%) 141 (14%) 281 (28%)

2 (.2%) 101 (11%) 4 (.4%) 104 (11%) 182 (19%) 29 (3%) 491 (51% 149 (15%) 12 (1%) 412 (43%) 291 (30%)

Lime text

Teacher informal assessment of work

318 (31%)

212 (22%)

Medium blue highlight Red highlight Bright green highlight

Teacher check for understanding

189 (19%)

229 (29%)

Typical interchange Whole class direction

62 (6%) 147 (14%)

67 (7%) 287 (28%)

Pink highlight Light blue highlight

Teacher/researcher discussion Peer help

77 (8%) 37 (4%)

0 (0%) 117 (12%)

Yellow highlight Gray highlight

Teacher redirect to system Teacher redirect from off-task

492 (48%) 312 (31%)

13 (1%) 368 (38%)

*Reported percent is the number of lines as a percent of total lines

Findings Analysis of this qualitative data revealed both successes and challenges with using a learning game or simulation in a classroom. Despite the challenges, the qualitative findings support the interpretation that the quantitative results can be attributed to the design. However, contextualizing the results within the qualitative responses of participants allows us to draw further conclusions for how to improve future implementations of the design.

Embedded pedagogical elements Intentionally designed pedagogical agents, both characters and objects, were embedded in the 3D space and were meant to complement the ill-structured problems posed by the narrative. Some of these agents created designed points of frustration that forced student reliance on the system and their peers in order to solve these problems. Others functioned as gate-keeping mechanisms to prevent learners from skipping vital stages of the in-system writing instruction or critical pieces of supporting evidence necessary to their investigation. Both types of pedagogical agents were intended to empower students to direct their own learning and to free the teacher from answering directional and procedural questions. A review of the video, audio, and field notes for both classes indicated that students in the Anytown treatment class worked on task-related activities longer and more often than did those in the comparison classroom, and the comparison teacher, Ms. Cedar, spent much more time giving directional guidance. Analysis of the interactions between students and Ms. Cedar suggests that most of the power in the comparison classroom was concentrated in the hands of the teacher. Feeling powerless to direct their own learning to more motivating tasks because of strictures placed on them by the teacher, students engaged in transgressive behaviors as they sought to take back power by what means were available to them. These behaviors included talking or whispering to peers, drawing unrelated pictures, kicking their feet, throwing objects, talking to the researcher, asking irrelevant questions, and putting their heads on their desks. Although they were in the minority, a few students remained engaged with their learning tasks

throughout each period they were observed. Notably, these few were the strongest writers, most advanced readers, and highest achievers in the class. Review of student interactions with the teacher and peers indicated that their power in the classroom was connected to a positive rapport with the teacher, motivation stemming from achievement and conforming to the teacher declared norms. While the treatment teacher, Mrs. Teak, did spend time answering directional and procedural questions early in the intervention, she eventually weaned students of their reliance on her by redirecting students to each other, to the system affordances, and to peers in order to solve the problem. For example:

Ms. Teak: Did you ask me something? Eric: Yes. I wanted to know how to pick things up, but now I just want to know how to pick up the gas can. How you get the gas can out, but somebody already answered it. What do we have to do first? Ms. Teak: Ask Robert or Isabelle.

In this instance, the teacher avoided directing the student to the correct answer and intentionally redirected him to his peers instead. In this way, she did not assert her power over the student in the learning context and instead empowered him to solve his own problem through peer negotiation, which is a key feature of problem-based learning environments (Jonassen, 1999). Had the teacher reclaimed her power to direct student learning behaviors, she would have likely stymied the affordances of the learning environment or reinforced his existing dependence on her as arbiter of information.

However, several students were stymied by collecting the Gas Can clue, a designed frustration point preventing student advancement without successful completion of a sequence of learning activities preceding this specific game task. Retrieving this pedagogical object (as shown in Figure 3) was required in order gather the evidence necessary to complete the Burning Cabin Mystery Quest (depicted earlier in Figure 1). A secondary purpose of this designed frustration point was to compel student reliance on their peers.

Figure 3. The Gas Can, a designed frustration point for enacting cognitive challenge

Student frustration with the object was revealed in the in-system chat feature. More than half of the student dialogue in the chat board either requested help with getting the gas can or offered responses to these requests. The following was a typical interchange:

Kenny: Can you help me with picking up the Gasoline Tank? Isabelle: pick it up by clicking on it Kenny: How do you pick it up? Kenny: The Gasoline tank? Jimmy: i dont know try aging (sic)

The frustration surrounding gaining the Gas Can item generated more discussion than all other computer-mediated discourse within the system. The audio recordings during these class periods revealed that student face-to-face discourse also revolved around solving this specific problem. The designed frustration point appeared to function in the way that the designer intended, forcing students to rely on each other and the system for information.

Motivating challenges and rewards One of the affordances of using digital media for instruction that has been reported by Glazewski et al (2004), Brush and Saye (2003), and Goldberg et al, (2003) is that students remain engaged longer than in more traditional forms of classroom instruction. In digital games, the motivational elements are tied to the narrative plotline and include challenges and rewards that players earn through their successful interactions with the environment. As part of the design of the Anytown, we sought to include many similar forms of extrinsic rewards to help motivate writing. During the study, we sought to examine what emerged as motivating rewards in both the comparison and treatment classrooms.

Motivation in two classrooms. In the comparison classroom, Ms. Cedar often challenged students to engage with the subject matter through direct questions related to the topic under discussion; however, it was often the same students who repeatedly volunteered to answer and appeared to have high levels of intrinsic motivation to achieve at classroom tasks. These six students had also completed reading the entire Quest Atlantis book as part of a small circle of readers and were consistently the highest achievers in class, an observation in keeping with the assertion that ties to narrative structures motivate engagement. This group was rewarded by being allowed to read in a small group and were provided with independent time with the teacher to discuss the book. This extrinsic motivation was maintained in their future reading activities, which were similarly remunerated. As a reward however, it appeared to mainly motivate this small subsection of students and the remainder of the students did not participate. In the Anytown treatment, students reported feeling unmotivated when the teacher restricted their free choice activity in the 3-D environment that would have allowed them to complete optional game tasks that encouraged exploration and note taking. This lack of motivation was reported most during the third and fourth periods when the teacher implemented a policy that required students to raise their hands and wait for her to review their work on-screen prior to submitting it to the Anytown system for review by her later. This policy was due to a miscommunication between the designer and teacher as to the purpose and functions of the system. Ms. Teak reported that she felt that this further oversight was necessary because she did not fully understand how the design was intended to function or her role in it.

In order to overcome this obstacle, the designer/researcher was compelled to intervene and ask the teacher to desist from requiring this review so that the system could function as intended and the study could proceed. Once the teacher removed this requirement, the system functioned as intended and students reported feeling empowered to explore and choose their own writing and game activities in the space. Consequently, the curricular, instructional, and motivational elements seemed to function as intended. Students in the treatment class were motivated to complete a total of 26 voluntary writing activities. Transgressive behavior. The opportunities within Anytown for transgressive student behavior are also of note; a few students excelled at exploiting these. For example, students told Ms. Teak, peers, and researchers that they were intrigued by the “Dark Lakes” and “Raintree Manse” thematic mini-games. These areas were intended as rewards for completing academic tasks or to provide additional experiences that students could write about in non-descriptive forms such as poems and fictional stories. Participant students revealed that their intrigue was because of the unique ambiance of these spaces resulting from their appearing out of place in the town and that they were further engaged by obstacles intended to keep them out until they had completed prerequisite Quests. These locations took four male students--Stanley, Sawyer, Eric, and Walter--away from their main task fairly often. This behavior most often took place in locations that were important to later Quest learning activities, but had no relevance to those that the boys were working on at the time. Their wish to engage in activities not yet open to them by the built-in system of rewards pulled them off-task as they attempted to exert their own power and “game” the system. The term “gaming the system” refers to

behaviors that are either in violation of the systemâ&#x20AC;&#x2122;s intended constraints or that undermine established game rules (Baker, et al, 2008). Although intriguing locations intended to reward players prompted undesirable behavior from some players, this behavior confirms the motivating quality of such design elements.

Immersive visual experience Given their immersive visual richness, graphical virtual environments have obvious applications for science inquiry. They can simulate practices such as observation and experimentation without constant replenishment of costly supplies or learner exposure to the hazards inherent in laboratory settings. The impact that such environments may have on writing, however, is less intuitive. Nevertheless, the designers speculated that the immersive quality of a 3-D MUVE could have a positive impact on a learnerâ&#x20AC;&#x2122;s ability to visualize their writing topics and therefore provide richer descriptive detail in their essays and other writing assessments. Further, it was hypothesized that if they engaged in a form of experiential learning about a mysterious environment that the details of the events, places, and people would remain more firmly in their minds and then would transfer to more vivid, detailed descriptive writing. The comparison classroom teacher employed problem-based and constructivist methods to engage students in generating solutions to ill-structured problems in a face-toface (F2F) setting rather than the immersive 3-D MUVE, Anytown. However, the F2F environment was restricted mainly to the immediate classroom with a few opportunities to visit the library, computer lab, and other institutional spaces. By necessity, this would

be more physically static and less expansive than the game world we would provide the students in which they could practice their writing. As previously noted, the more intriguing areas of Anytown enticed some students too much, compelling them to “game the system” in order to gain access to areas prohibited until prior Quests were completed. One student, Johnny, spent a lot of time exploring areas that were designed to be off-limits until prerequisite tasks had been completed. However, Ms. Teak asked questions of Johnny to involve him in verbally describing his experience as a means of leveraging those details to complete his assigned writing task, showing interest in his unique experiences and working to have him identify relevant details. Johnny later used this discussion as he wrote about his explorations in his Quest response:

“I liked this eccentric building that I was in. I can describe it for you. It looked like some kind of really old building. It had some stairs. I went down and I came to a tunnel, and there were red lights on the wall.”

Given that one of the goals of the use of this game system was to engage students in descriptive writing, Johnny’s use of adjectives and visual descriptions of his environment was a positive development. The teacher later reported that this student had not turned in a single piece of writing during the first two months of school leading up to the use of the Anytown system. Eric was another student who found the immersive qualities of the game compelling. He sought to enter the mine that serves as an entrance to the “Dark Lakes”

ghost story area by repeatedly bumping the walls that enclose this area. This behavior ensured him little progress on his mandatory writing activities. However, Eric was able to overcome the obstacle and gain access to the prohibited place, a feat that gave him a sense of power over the system. Although this behavior also caused some friction between Eric and his teacher, he was unwavering in his queries about the purpose of the Dark Lakes mini-game area. It was only when Ms. Teak asked to see the extent of his work on his assigned Quest that Eric relented and returned to working on that assignment so that he could get to the Dark Lakes area later. While Ericâ&#x20AC;&#x2122;s behavior was not what the designers intended, the immersive intrigue of the environment did entice Eric to finish his work. Implications The large number of qualitative data sources led the researchers to several implications stemming from the use of this interactive game environment. These related both to the Anytown environment specifically and more generally to the areas of games, simulations, and other interactive digital designs. Because the content area focus of the unit was literacy practices of elementary schools students, several conclusions focus on how best to leverage these digital environments to increase student engagement with reading and writing. Further, the difficulties encountered with communicating the designerâ&#x20AC;&#x2122;s intention for the system properly to the users and the consequences of that challenge also influence these findings.

Innovation in classroom practices Any new curriculum will require the development of new classroom practices and the creation of training techniques at the university and in-service levels that encourage teachers to engage with emerging technologies in ways that prepare them for meaningful integration with content. Further research should examine if teachers recognize when their administrative duties, such as answering repeated procedural and directional questions, take them away from teaching. If teachers engage in increased instructional discourse with students, do they spend more time providing specific feedback on student writing? Do they increase their discourse with students in the computer lab related to the characteristics of good writing such as proper spelling, punctuation and capitalization? If the teacher does not leverage the increased time they have in the computer lab to improve student learning as a result of a digital environment, then the innovation is not helpful. If future research shows that teachers are engaging in significantly increased instruction, feedback on student work, challenging poor knowledge constructions, and informal assessment of student learning, then the environment allows the teacher to engage in teaching behaviors associated with improvements in learning (Webb, Nemer, & Ing, 2006; Webb, Nemer, Kersting, Ing, & Forrest, 2004).

Teacher versus system control As noted earlier, the perceived conflict between a teacherâ&#x20AC;&#x2122;s need for control and the need for the learning environment to function as designed indicated the necessity for clearer communication with the teacher prior to the implementation of such a game-like environment. Although the treatment teacher was provided with a six-page manual that

outlined the Anytown environment, the activities, and the teacher role, accompanied by a one hour, face-to-face training session with the researcher, this may not have been enough. When the teacher asserted her control over students by preventing voluntary activities and requiring teacher review of all quests before submission stymied the design intentions of the researcher. Encouraging teachers to experience the student’s point of view prior to implementation and to use technologies in an immersive way should result in better technology integration. However, such preparation goes beyond mere familiarization with a technology product. Ensuring that teachers are comfortable with their role in problem-based or constructivist learning environments is imperative. Preparation programs should provide additional background about what the teacher might expect in terms of student cognitive challenge, the student-centered design of the environment, and what kinds of teaching behaviors would defeat the purposes of the system. This is in keeping with the results of Dede’s work (2006) which recommended that additional teacher professional development would make the River City science and history-based MUVE more effective.

Technology and literacy Both the findings of this study and those of Steinkuehler’s (2004) work with the game Lineage II indicate relationships between student engagement and digital learning environments as means of creating intrinsically motivating writing practices for K-12 students. Krashen’s (2004) review of research related to reading and writing noted that increased reading practice is often correlated with improvements in student writing even

more so than increased writing practice. Future studies should examine whether this form of digital environment can motivate students to engage in regular reading. Also of interest is linking reading to the design of learning environments that can be prepared using those principles of engagement already employed by game companies, digital and analog, and by successful teachers to improve student willingness to read. Existing games such as Blizzardâ&#x20AC;&#x2122;s World of Warcraft (Dickey, 2007) already provide paper books that complement the game experience, providing additional information and stories about characters that might be experienced in the game. In fact, pre-orders of the World of Warcraft software were accompanied by copies of the first novel in a trilogy that could be used by players to help provide a narrative frame for the world in which they would play while linking the new game to the past story of Warcraft III, the previous title in the series. This helped to draw players into a new story without risk because it complemented their daily experience within the game.

Future Research It is the hope of the researchers that several major lines will be explored through future research: those involved with Anytown and those that focus on the use of games, game-like learning environments in general, and student problem-solving within gameinfused problem-based learning environments. In terms of the Anytown environment, the role of peer teaching and support appeared to be an important part of both student successes on learning tasks within the space and in overcoming major difficulties with the game elements of the design. While this is important, further research is needed to fully understand the role that peer support plays in learning using such an environment.

26 Beyond peer support and the role of game incentives in general for motivation, it is also important to understand how such learning objects can be designed effectively within such an environment. Anytown contained over 150 such objects that students could earn or otherwise receive; however, students used or referenced only a fraction of these during their time interacting in the space. What still must be explored is which objects were motivating to students and what the qualities were present that made them so. Also, questions must be addressed related to which objects have high and low impact on learning or engagement with a learning game system, so that future designers have a sense of what must be present in an effective learning object that is contextualized by a game space.

References Aldrich, C. (2003). Simulations and the future of learning. San Francisco: Pfeiffer. Barab, S. A., & Jackson, C. (2006). From Plato's Republic to Quest Atlantis: The role of the philosopher-king [Electronic Version]. Technology Humanities Education Narrative. Retrieved 1/20/2006 from http://www.thenjournal.org/feature/78/. Barab, S. A., Warren, S. J., Zuiker, S., Hickey, D., Ingram-Goble, A., & Dodge, T. (2006, April 7-11, 2006). Transfer of Learning in Complex Learning Environments. Paper presented at the American Educational Research Association Annual Meeting, San Francisco, CA. Baudrillard, J. (1994). Simulacra and Simulation (The Body, in Theory: Histories of Cultural Materialism). Ann Arbor: University of Michigan Press. Bransford, J., Vye, N., Bateman, H., Brophy, S., & Roselli, B. (2003). Vanderbilt's AMIGO Project: Knowledge of How People Learn enters cyberspace. Retrieved

4/20/2004, 2004, from http://www.vanth.org/mmedia/vanth0103/vanth0103cd/papers/AmigoWFig.pdf Brush, T., & Saye, J. (2003). The Effects of Multimedia-Supported Problem-Based Historical Inquiry on Student Engagement, Empathy, and Historical Reasoning. Paper presented at the American Educational Research Association. Cassell, J., & Jenkins, H. (Eds.). (2000). From Barbie to Mortal Kombat. Cambridge, MA: The MIT Press. Clark, R. E. (1991). When Researchers Swim Upstream: Reflections on an Unpopular Argument About Learning from Media. Educational Technology(2). Dede, C. (2006). MUVEEs Project. Cambridge, MA: Harvard University. Dede, C., Ketelhut, D., & Ruess, K. (2006). Designing for motivation and usability in a museum-based multi-user virtual environment. Retrieved 3/11/2006, 2006, from http://www.gse.harvard.edu/~dedech/muvees/documents/AELppr.pdf Derrida, J. (1997). Of grammatology (G. C. Spivak, Trans. Corrected ed.). Baltimore, MD: The Johns Hopkins University Press. Dickey, M. D. (2007). Game design and learning: A conjectural analysis of how massively multiple online role-playing games (MMORPGs) foster intrinsic motivation. Educational Technology Research & Development, 55(3), 253-273. Education, U. S. D. o. (2007). Effectiveness of reading and mathematics software products: Findings from the first students cohort. Retrieved. from. Gall, M. D., Borg, W. R., & Gall, J. P. (1996). Educational Research: An introduction (6th ed. Vol. I). White Plains, NY: Longman Publishers.

Glazewski, K., & Clark, D. (2004). Supporting scientific inquiry in online learning environments. Unpublished manuscript, West Lafayette, Indiana. Goldberg, A., Russell, M., & Cook, A. (2003). The effect of computers on student writing: A meta-analysis of studies from 1992 to 2002. The Journal of Technology, Learning, and Assessment, 2(1). Hays, R. T. (2005). The effectiveness of instructional games: A literature review and discussion (Technical Report No. 2005-004). Orlando, FL: Naval Air Warfare Center Training Systems Division. Jenkins, H., Squire, K., & Tan, P. (2003a). Entering the education arcade. Computers in Entertainment 1(1), 17. Jenkins, H., Squire, K., & Tan, P. (2003b). Entering the education arcade. . Computers in Entertainment 1(1), 17. Jonassen, D. H. (1999). Designing constructivist learning environments. In C. M. Reigeluth (Ed.), Instructional Design Theories and Models: A new paradigm of instructional theory (Vol. 2, pp. 215-239). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers. Kozma, R. (1991). Learning with media. Review of Educational Research, 61(2), 179211. Krashen, S. D. (2004). The power of reading : insights from the research (2nd ed.). Westport, Conn.; Portsmouth, NH: Libraries Unlimited; Heinemann. Miles, M., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage Publications, Inc.

Nagel, D. (2007). DOE software study: Are the data flawed? [Electronic Version]. Technological Horizons in Education (THE) Journal Retrieved April 19, 2007 from http://www.thejournal.com/articles/20463. Prensky, M. (2001). Digital game-based learning. New York: McGraw-Hill. Robson, C. (2002). Real world research. Malden, MA: Blackwell Publishing. Squire, K. (2004). Replaying history. Unpublished dissertation, Indiana UniversityBloomington, Bloomington, IN. Squire, K. (2006). From Content to Context: Videogames as Designed Experience. Educational Researcher, 35(8), 19-29. Squire, K., Giovanetto, L., Devane, B., & Durga, S. (2005). From users to designers: Building a self-organizing game-based learning environment. Tech Trends, 49(5), 34-42, 74. Steinkuehler, C. (2004). The literacy practices of massively multiplayer online gaming. Paper presented at the American Educational Research Association, San Diego, CA. Tuzun, H. (2004). Motivating learners in educational computer games. Unpublished Dissertation, Indiana University, Bloomington, IN. Vanderbilt, C. a. T. G. a. (1990). Anchored instruction and its relationship to situated cognition. Educational Researcher, 19(6), 2-10. Vanderbilt, C. a. T. G. a. (1993). Anchored instruction and situated cognition revisited. Educational Technology, 33(3), 52-70. Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.

Warren, S., Barab, S., & Dondlinger, M. (in press). A MUVE towards PBL writing: Effects of a digital learning environment designed to improve elementary student writing. Journal of Research on Technology in Education. Warren, S. J., & Dondlinger, M. J. (in press). Designing Games for Learning. In R. Fertig (Ed.), Handbook of Research on Effective Electronic Gaming in Education. Hershey, PA: Idea Group Reference. Warren, S. J., & Jones, J. G. (in press). Supporting teachers, researchers, and emerging technologies: Evaluating games and simulations intended for learning Connexions. Webb, N., Nemer, K., & Ing, M. (2006). Small-group reflections: Parallels between teacher discourse and student behavior in peer-directed groups. The Journal of the Learning Sciences, 15(1), 63-119. Webb, N., Nemer, K., Kersting, N., Ing, M., & Forrest, J. (2004). The effects of teacher discourse on student behavior and learning in peer-directed groups (Spencer Foundation Grant Report). Los Angeles: CSE/CRESST/University of California, Los Angeles. Wittgenstein, L. (1968). Philosophical investigations (G. E. M. Anscombe, Trans. 2nd ed.). Oxford: Blackwell.

Acknowledgements: The authors would like to thank several people for their hard work in support of the design of Anytown and many hours spent collecting data for this study. These include: Heather Warren, Dan Hickey, Steve Zuiker, Adam Ingram-Goble, Inna Kouper, Susan Herring, Eun-ju Kwon, Robert Appelman, and Mitzi Lewison.