Proceedings fo the 7th European Conference on Games Based Learning ECGBL 2013 by ACPIL

Proceedings of the 7th European Conference on Games-Based Learning Instituto Superior de Engenharia do Porto, Portugal 3-4 October 2013

Volume One

Edited by Dr Carlos Vaz de Carvalho and Dr Paula Escudeiro Instituto Superior de Engenharia do Porto Portugal A conference managed by ACPI, UK www.academic-conferences.org

The Proceedings of The 7th European Conference on Games Based Learning Hosted by Instituto Superior de Engenharia do Porto (ISEP) Portugal Volume One 3-4 October 2013 Edited by Dr Paula Escudeiro Programme Chair and Carlos Vaz de Carvalho Conference Chair

Copyright The Authors, 2013. All Rights Reserved. No reproduction, copy or transmission may be made without written permission from the individual authors. Papers have been double-blind peer reviewed before final submission to the conference. Initially, paper abstracts were read and selected by the conference panel for submission as possible papers for the conference. Many thanks to the reviewers who helped ensure the quality of the full papers. These Conference Proceedings have been submitted to Thomson ISI for indexing. Please note that the process of indexing can take up to a year to complete. Further copies of this book and previous yearâ&#x20AC;&#x2122;s proceedings can be purchased from http://academic-bookshop.com E-Book ISBN: 978-1-909507-65-4 E-Book ISSN: 2049-100X Book version ISBN: 978-1-909507-63-0 Book Version ISSN: 2049-0992 CD Version ISBN: 978-1-909507-66-1 CD Version ISSN: 2049-1018 The Electronic version of the Proceedings is available to download at ISSUU.com. You will need to sign up to become an ISSUU user (no cost involved) and follow the link to http://issuu.com Published by Academic Conferences and Publishing International Limited Reading UK 44-118-972-4148 www.academic-publishing.org

Contents Paper Title

Author(s)

Page No.

Preface

vii

Committee

viii

Biographys

Teachers’ Beliefs About Game Based Learning: A Comparative Study of Pedagogy, Curriculum and Practice in Italy, Turkey and the UK

Yasemin Allsop, Ebru Yeniman Yildirim and Marina Screpanti

Using Gamification to Animate a Virtual Community

António Andrade and Carlos Vaz de Carvalho

SIMaging the CITY: The Educational use of Simulation Video Games for Disadvantaged Youth

Massimiliano Andreoletti and Gianna Cappello

“The Chest That Longs to be Moved”: A Serious Game for the Greek Muslim Minority Children

Alexandra Androussou, Evangelia Kourti and Nelly Askouni

Transformational Play; Using 3D Game-Based Narratives to Immerse Students in Literacy Learning

Anna Arici and Sasha Barab

Approaches to Collaborative Game-Making for Fostering 21st Century Skills

Susan Bermingham, Nathalie Charlier, Francesca Dagnino, James Duggan, Jeffrey Earp, Kristian Kiili, Evelien Luts, Lien van der Stock and Nicola Whitton

Best Practices for Deploying Digital Games for Personal Empowerment and Social Inclusion

Lizzy Bleumers, Ilse Mariën, Jan Van Looy, James Stewart, Dana Schurmans and Anissa All

Investigating the Relationship Between School Performance and the Abilities to Play Mind Games

Rosa Maria Bottino, Michela Ott and Mauro Tavella

Experience With Digital Game-Based Embodied Learning: The Road to Create a Framework for Physically Interactive Digital Games

Carsten Busch, Florian Conrad, Robert Meyer and Martin Steinicke

Toward Improvement of Serious Game Reliability

Thibault Carron, Fabrice Kordon, Jean-Marc Labat, Isabelle Mounier and Amel Yessad

The Effects of Gamification on Student Attendance and Team Performance in a Third-Year Undergraduate Game Production Module

Hope Caton and Darrel Greenhill

Game-Based Learning in Health Sciences Education

Nathalie Charlier, Evelien Luts and Lien Van Der Stock

Specification and Design of a Generalized Assessment Engine for GBL Applications

Yaëlle Chaudy, Thomas Connolly and Thomas Hainey

105

Safer Internet: Enhancing Good Practices on the Internet Through Games Based Learning for Greek Elementary School Students

Vasiliki Choleva, Loukas Koutsikos Simeon Zourelidis, Vlassios Filis, Dimitris Metafas and Charalampos Patrikakis

115

Using Game Mechanics to Measure What Students Learn

Jill Denner, Linda Werner, Shannon Campe and Eloy Ortiz

123

Combining Game Based Learning With Content and Language Integrated Learning Approaches: A Case Study Utilizing QR Codes and Google Earth in a Geography-Based Game

Kyriaki Dourda, Tharrenos Bratitsis, Eleni Griva and Penelope Papadopoulou

130

Paper Title

Author(s)

Page No.

The Design and Evaluation of a Multiplayer Serious Game for Pharmacy Students

Maciej Dudzinski , Darrel Greenhill , Reem Kayyali , Shereen Nabhani , Nada Philip , Hope Caton , Sonya Ishtiaq and Francis Gatsinzi

140

Cheating and Creativity in Pervasive Games in Learning Contexts

Stine Ejsing-Duun, Thorkild Hanghøj and Helle Skovbjerg Karoff

149

Supporting Teachers in the Process of Adoption of Game Based Learning Pedagogy

Valérie Emin-Martinez and Muriel Ney

156

Cognitive Walkthrough for Learning Through Game Mechanics

David Farrell and David Moffat

163

Global Math: Development of Online Platform for Mathematical Thinking Games

Toru Fujimoto, Keiichi Nishimura, Kaoru Takahashi, Masahiro Yachi, Kiyoshi Takahashi and Yuhei Yamauchi

172

What Can Play Theory Tell us About Computer Games for Young Children?

Georgy Gerkushenko and Svetlana Sokolova

179

Role Game Playing as a Platform for Creative and Collaborative Learning

Lisa Gjedde

190

Development and Evaluation of a Generic E-CLIL Web2.0 Games Engine

Thomas Hainey and Thomas Connolly

198

Designing Games to Disseminate Research Findings

Claire Hamshire, Rachel Forsyth and Nicola Whitton

208

Facilitating Teacher Students’ Innovation Competence through Problem-Based Game Design Processes

Thorkild Hanghøj and Sia Hovmand Sørensen

216

Deploying Serious Games for Management in Higher Education: Lessons Learned and Good Practices

Jannicke Baalsrud Hauge, Francesco Bellotti, Rob Nadolski, Michael Kickmeier-Rust, Riccardo Berta and Maira Carvalho

225

Neuroeducational Research in the Design and use of Games-Based Teaching

Wayne Holmes, Paul Howard-Jones, Erico Tanimoto, Carol Jones, Skevi Demetriou, Owen Morgan, Philip Perkins and Neil Davies

235

Playing and Learning: An iPad Game Development Case Study

Jennifer Jenson and Rachel Muehrer

244

An Overview of Game Console Motion Sensor Technologies Exploited for Education

Marina Kandroudi and Tharrenos Bratitsis

252

Picking the Right Interface for Engaging Physical Activity Into Game Based Learning

Helle Skovbjerg Karoff, Gunver Majgaard, Lars Elbæk and Mona Have Sørensen

261

Playing and Gaming – Studied in an Informal Learning Setting

Helle Skovbjerg Karoff, Stine Ejsing-Duun and Thorkild Hanghøj

268

Game Based Learning in Mathematics: Teachers' Support by a Flexible Tool

Aikaterini Katmada, Apostolos Mavridis, Thrasyvoulos Tsiatsos

275

Learning Analytics with Games Based Learning

Harri Ketamo

284

Gamification and Intelligent Feedback Mechanisms for a Division Learning Tool

Michael Kickmeier-Rust and Dietrich Albert

290

Developing Games for Health Impact: Case Brains vs Zombies

Kristian Kiili, Manuel Ninaus, Mikko Koskela, M Tuomi and Antero Lindstedt

297

Meleon - a Casual Mobile Game Supporting Immersion and Reflection in Learning

Luise Klein

305

Paper Title

Author(s)

Page No.

The Literature Race - NFC Based Mixed Reality Game

Antti Koivisto, Harri Ketamo, Eero Hammais and Juho Salli

314

Bringing Game Achievements and Community Achievements Together

Johannes Konert, Nico Gerwien, Stefan Göbel and Ralf Steinmetz

319

Modeling the Player, Learner and Personality: Independency of the Models of Bartle, Kolb and NEO-FFI (Big5) and the Implications for Game Based Learning

Johannes Konert, Stefan Göbel and Ralf Steinmetz

329

Raising Awareness on Archaeology: A Multiplayer Game-Based Approach With Mixed Reality

Mathieu Loiseau, Élise Lavoué, Jean-Charles Marty and Sébastien George

336

Scientific Discovery Games for Authentic Science Education

Rikke Magnussen, Sidse Damgaard Hansen, Tilo Planke, Jacob Friis Sherson

344

Creating Games in the Classroom – From Native Gamers to Reflective Designers

Gunver Majgaard

352

A Holistic Framework for the Development of an Educational Game Aiming to Teach Computer Programming

Christos Malliarakis, Maya Satratzemi and Stelios Xinogalos

359

Examining Early Childhood Education Students’ Attitudes Toward Educational Computer Games in Kindergarten

Dionissios Manessis

369

Integrating Non-Virtual Electronic Activities in Game-Based Learning Environments

Jean-Charles Marty, Thibault Carron, Stéphane Talbot, Gregory Houzet and Philippe Pernelle

378

From « Haute-Couture » to « Ready-to-Wear »: Typology of Serious Games Implementation Strategies in Higher Education

Hélène Michel

386

Motivation and Manipulation: A Gamification Approach to Influencing Undergraduate Attitudes in Computing

Nicholas Mitchell, Nicky Danino and Lesley May

394

Sit Down to Table and Confess who you are! Design of an Epistemic Game for Nutritional Education at Secondary School

Réjane Monod-Ansaldi, Eric Sanchez, Daniel Devallois, Thomas Abad, Pierre Bénech, Anne Brondex, Isabelle Mazzella, Sandrine Miranda, Claudie Richet and Céline Recurt

401

Learning in Context Through Games: Towards a new Typology

Alex Moseley

409

Let the Students Construct Their own fun And Knowledge - Learning to Program by Building Computer Games

Peter Mozelius, Olga Shabalina, Christos Malliarakis, , Florica Tomos, Chris Miller and David Turner

418

Towards Understanding the Instructional Value of Real-Time Continuous Feedback From the use of Simulation Games

Mathews Nkhoma, Jaime Calbeto, Narumon Sriratanaviriyakul, Thu Yein Win, Quyen Ha Tran and Thanh Kim Cao

427

Learning Math as you Play: Comparing Arithmetic Performance Enhancement Induced by Game Play and Paper Exercises

Elena Patricia Nuñez Castellar, Anissa All and Jan Van Looy

434

Serious Game Adaptive Learning Systems

Chinedu Obikwelu and Janet Read

442

Combatting Social Isolation and Cognitive Decline: Play a Physical or Digital Game?

Daire Ó Broin and Ross Palmer

450

iii

Paper Title

Author(s)

Page No.

Sports Games’ Role for Learning Health Knowledge

Kelly O’Hara, Dulce Esteves, Rui Brás, Ricardo Rodrigues, Paulo Pinheiro and Marco Rodrigues

458

A Multi-Agent Architecture for Collaborative Serious Game Applied to Crisis Management Training: Improving Adaptability of non Played Characters

M’hammed Ali Oulhaci, Erwan Tranvouez, Sébastien Fournier and Bernard Espinasse

465

Nuclear Mayhem – a Pervasive Game Designed to Support Learning

Trygve Pløhn

475

StartUp_EU: Using Game-Based Learning and Web 2.0 Technologies to Teach Entrepreneurship to Secondary Education Students

Aristidis Protopsaltis, Thomas Hainey, Spiros Borosis, Thomas Connolly, Jesus Copado and Sonia Hezner

484

Measuring Effects of Reflection on Learning: A Physiological Study

Wen Qi, Dominique Verpoorten and Wim Westera

495

Evaluation of Adaptive Serious Games using Playtraces and Aggregated Play Data

Christian Reuter, Florian Mehm, Stefan Göbel and Ralf Steinmetz

504

Learning Effectiveness of Management Simulation Game Manahra

Petr Smutný, Jakub Procházka and Martin Vaculík

512

Using the Master Copy - Adding Educational Content to Commercial Video Games

Heinrich Söbke, Thomas Bröker and Oliver Kornadt

521

An Application of Adaptive Games-Based Learning Based on Learning Style to Teach SQL

Mario Soflano, Thomas Connolly and Thomas Hainey

531

Can Moral Sensitivity be Enhanced by Game Play?

Gunilla Svingby

539

Digital Educational Games: Adopting Pedagogical Agent to Infer Leaner‘s Motivation and Emotional State

Ogar Ofut Tumenayu and Olga Shabalina

546

Adapting the Complexity Level of a Serious Game to the Proficiency of Players

Herre van Oostendorp, Erik van der Spek and Jeroen Linssen

553

Designing Casual Serious Games in Science

Ayelet Weizman

561

Designing a Collaborative Serious Game for Team Building Using Minecraft

Viktor Wendel, Michael Gutjahr, Philipp Battenberg, Roman Ness, Sebastian Fahnenschreiber, Stefan Göbel and Ralf Steinmetz

569

Application of the Principles of Gamification to Facilitate Acquisition of Self-Management Skills in Young People With Long-Term Medical Conditions

Andrew Wilson and Janet McDonagh

579

Development of an Implementation Framework for Games-Based Construction Learning Using Scratch in Primary Education

Amanda Wilson, Thomas Hainey and Thomas Connolly

587

Game Literacy Revisited: Developing Critical Play in Schools

Rafael Marques de Albuquerque and Shaaron Ainsworth

599

A Systematic Literature Review of Methodology Used to Measure Effectiveness in Digital GameBased Learning

Anissa All, Elena Patricia Nuñez Castellar and Jan Van Looy

607

Investigating Collaborative Games to Teach Mathematics-Based Problem Solving in the Classroom

Reem Al-Washmi, Gail Hopkins and Peter Blanchfield

617

PHD papers

Paper Title

Author(s)

Page No.

Training Flexible and Adaptive Arithmetic Problem Solving Skills Through Exploration With Numbers: The Development of NumberNavigation Game

Boglárka Brezovszky, Erno Lehtinen, Jake McMullen, Gabriela Rodriguez and Koen Veermans

626

Trials to Assess Team-Based Mixed-reality (TBMR) Games in HE

John Denholm and Sara de Freitas

635

Understanding ‘Game-Ness’ Within the ® SCRABBLE Family of English Word Games

Paridhi Gupta

645

Interactive Story as a Motivator Element in an Educational Video Game

José Rafael López-Arcos, Francisco Luis Gutiérrez Vela, Natalia Padilla-Zea and Patricia Paderewski

656

A Domain Ontology of Game Theory Applied to Game Based Learning

Yemna Mejbri, Maha Khemaja and Rafik Braham

666

Puzzle-Based Games as a Metaphor for Designing in Situ Learning Activities

Javier Melero, Patricia Santos, Davinia HernándezLeo and Josep Blat

674

Supporting and Facilitating Collaborative Learning in Serious Games

Kimmo Oksanen and Raija Hämäläinen

683

Playing for the Future - Examining Gameplay, Narrative and fun in Games-Based Training

Mark O’Rourke

691

Towards Game Based Learning Design Process Based on Semantic Service Oriented Architecture (SSOA)

Kaouther Raies, Maha Khemaja and Rafik Brahamm

698

Using Games for Learning, From the Students’ Perspectives

Aishah Abdul Razak and Thomas Connolly

706

Incidental Learning in a World of Warcraft Guild, a Case Study

Gabriela Rodríguez

714

In Search for the Right Measure: Assessing Types of Developed Knowledge While Using a Gamified Web Toolkit

Martin Ruskov, Paul Ekblom and Angela Sasse

722

The Influence of Digital Games on Learning Reading: A Closer Look

Mas Idayu Md Sabri, Peter Blanchfield and Gail Hopkins

730

The Mediatization of Digital Games for Learning – a Dual Rub-Off Effect

Helga Sigurdardottir and Robin Munkvold

740

Efficacy of Reward Allotment on Children’s Motivation and Learning

Zhenhua Xu, Earl Woodruff and Bodong Chen

748

Applying Ideas From Intelligent Tutoring Systems for Teaching Programming in Game Based Learning

Matej Zapušek and Jože Rugelj

756

Cultivating Preschoolers Creativity Using Guided Interaction With Problem Solving Computer Games

Georgios Fessakis and Dimitrios Lappas

763

Haptic Physics Simulation

Luciano Santos and Carlos Vaz de Carvalho

771

Evaluating the Embedding of Games Based Learning in a Computing Subject at University

Emilia Todorova and David Moffat

776

Masters

WIP Papers

Paper Title

Author(s)

Page No.

A Design Approach for Implementing 3D Educational Collaborative Virtual Environments on Virtual World Platforms

Rosa Reis, Benjamin Fonsecaand Paula Escudeiro

785

EMOTE: Embodied-Perceptive Tutors for Empathy-Based Learning in a Game Environment

Sofia Serholt, Wolmet Barendregt, Tiago Ribeiro, Ginevra Castellano, Ana Paiva, Arvid Kappas, Ruth Aylett and Fernando Nabais

790

Exploring Learning Effects During Virtual Sports Through Biomechanical Analysis (a Work in Progress)

Pooya Soltani and Jo達o Paulo Vilas-Boas

793

siLang: Culturally Oriented Language Skill Development in Line With Workplace Needs

Hariklia Tsalapatas, Olivier Heidmann, Rene Alimisi and Elias Houstis

797

Developing Ethical Decision Making Skill of Novice Volunteers in Natural Disaster Response

Didin Wahyudin, Shinobu Hasegawa and Tina Dahlan

800

Preface These proceedings represent the work of researchers participating in the 7th European Conference on GamesBased Learning, which is being organised and hosted this year by the Instituto Superior de Engenharia do Porto (ISEP), Porto, Portugal. The Conference Chair is Dr Carlos Vaz de Carvalho and the Programme Chair is Dr Paula Escudeiro, both from the Instituto Superior de Engenharia do Porto, Porto, Portugal. The conference will be opened with a keynote from Dr. Baltasar Fernández-Manjón, from Complutense University of Madrid, Spain, on the topic of Learning Analytics Applied to Serious Games. The opening keynote address on the second day is by Paulo Gomes, Game Director and Producer at BIGMOON STUDIOS. The Conference is a valuable platform for individuals to present their research findings, display their work in progress and discuss conceptual advances in many different areas and specialties within Games-Based Learning. It also offers the opportunity for like minded individuals to meet, discuss and share knowledge. ECGBL continues to evolve and develop, and the wide range of papers and topics will ensure an interesting twoday confercence. In addition to the main streams of the conference, there are mini tracks focusing on the areas of Multi-User Virtual Environments, Content and Assessment Integration, User Profiling and Barriers and Opportunities for the introduction of GBL in Educational Settings. In addition to the presentations of research the conference this year has introduced a competition to provide an opportunity for educational game designers and creators to participate in the conference and demonstrate their game design and development skills in an international competition. This competition has been sponsored by SEGAN – Serious Games Network. With an initial submission of more than 50 games, 24 finalists will present their games at the conference. Prizes will be awarded to the three games judged to demonstrate the best quality and originality of game play itself and the positioning and articulation of the game’s contribution to the educational domain. With an initial submission of 179 abstracts, after the double blind peer review process, there are 71 research papers, 18 PhD research papers, 3 Masters research papers and 5 work-in-progress paperspublished in these Conference Proceedings. These papers represent research more than 30 countries, including Algeria, Australia, Austria, Belgium, Brazil, Canada, Czech Republic, Denmark, Finland, France, Germany, Greece, Hong Kong, India, Ireland, Israel, Italy, Japan, The Netherlands Norway, Portugal, Russia, Slovenia, Spain, Sweden, Switzerland, Tunisia, UK, USA and Vietnam. We hope that you have an enjoyable conference. Dr Paula Escudeiro Programme Chair and Carlos Vaz de Carvalho Conference Chair October 2013

vii

Conference Committee ECGBL Conference Director Professor Thomas M Connolly, University of the West of Scotland, UK Conference Executive: Dr Carlos Vaz de Carvalho, Instituto Superior de Engenharia do Porto, Portugal Dr Paula Escudeiro, Instituto Superior de Engenharia do Porto, Portugal Dulce Mota, Instituto Superior de Engenharia do Porto (ISEP), Porto, Portugal Isabel Azevedo, Instituto Superior de Engenharia do Porto (ISEP), Porto, Portugal António Castro, Instituto Superior de Engenharia do Porto (ISEP), Porto, Portugal Ana Barata, Instituto Superior de Engenharia do Porto (ISEP), Porto, Portugal Bertil Marques, Instituto Superior de Engenharia do Porto (ISEP), Porto, Portugal Rosa Reis, Instituto Superior de Engenharia do Porto (ISEP), Porto, Portugal Mini Track Chairs Prof. Dr Wilfried Admiraal, Leiden University, The Netherlands Dr Kristian Kiili, Tampere University of Technology, Finland Prof Konstantinos Kalemis, National and Kapodistrian University of Athens, Dr Thomas Hainey, University of the West of Scotland, UK Dr Jordi Sánchez-Navarro, Open University of Catalonia, Barcelona, Spain Dr Daniel Aranda, Open University of Catalonia, Barcelona, Spain Dr Stefan Göbel, Technical University of Darmstadt, Germany Viktor Wendel, Technical University of Darmstadt, Germany Johannes Konert, Technical University of Darmstadt, Germany Committee Members The 2013 conference programme committee consists of key people in the games based learning community, both from the UK and overseas. The following people have confirmed their participation: Dr Wilfried Admiraal (Leiden University, Leiden, The Netherlands); Dr. Minoo Alemi (Sharif University of Technology, Iran); Anissa All (iMinds-MICT-Ghent University, Belgi); Daniel Aranda (Universitat Oberta de Catalunya, Spain); Nikolaos Avouris (University of Patras, Greece); Isabel Azevedo ( Instituto Superior de Engenharia do Porto (ISEP), Portugal); Ana Barata ( Instituto Superior de Engenharia do Porto (ISEP), Portugal);Dr. Wolmet Barendregt (Gothenburg University, department of applied IT, Sweden); Francesco Bellotti (University of Genoa, Italy,); Mary Bendixen-Noe (Ohio State University, USA);Dr Tobias Bevc (University of Augsburg, Germany);Dr Bani Bhattacharya (IIT Kharagpur, India); Dr Peter Blanchfield (School of Computer Science, University of Nottingham, UK); Natasha Boskic (The University of British Columbia, Canada);Dr. Rosa Maria Bottino (Istituto Tecnologie Didattiche - Consiglio nazionale Ricerche, Italy); Hadya Boufera (University Of Mascara, Algeria); Philip Bourke (LIT-Tipperary, Ireland);Dr Liz Boyle (University of the West of Scotland, UK); Dr Tharrenos Bratitsis (University of Western Macedonia, Greece);Prof Anthony Brooks (Aalborg University, Denmark);Prof David Brown (Nottingham Trent University, UK);Prof. Dr.-Ing. Carsten Busch (University of Applied Sciences HTW-Berlin, Germany,); Dr George Caridakis (University of the Aegean / NTUA, Greece,); Dr Thibault Carron (Université de Savoie, Chambéry, France); Rommert Casimir (Tilburg University, The Netherlands); António Castro ( Instituto Superior de Engenharia do Porto (ISEP), Portugal);Dr Erik Champion (Massey University, New Zealand);Prof Maiga Chang (Athabasca University, Canada); Dimitris Charalambis ( University of Athens, Greece);Dr Darryl Charles (University of Ulster, UK); Nathalie Charlier (Catholic University of Leuven, Belgium);Dr Yam San Chee (Nanyang Technological University, Singapore);Dr. Ming-Puu Chen (National Taiwan Normal University, Taiwan,); Satyadhyan Chickerur (M S Ramaiah Institute of Technology, India);Professor Thomas Connolly (University of West of Scotland, UK); Tamer Darwish (Brunel University, UK); Ioannis Darzentas (University of Aegean, Greece);Dr Sara De Freitas (Birkbeck College University of london, UK);Dr Sophia Delidaki (Hellenic American Educational Foundation, Greece,);Dr Ioannis Deliyannis (Ionian University, Greece,);Dr. Muhammet Demirbilek (Suleyman Demirel University, Turkey);Dr David Edgar (Glasgow Caledonian University, UK); Patrick Felicia (Waterford Institute of Technology, Ireland); Georgios Fesakis (University of the Aegean, Greece);Dr. Brynjar Foss (University of Stavanger, Norway);Dr Christos Gatzidis (Bournemouth University, UK); Dr Sebastien George (INSA Lyon, France); Panagiotis Georgiadis (University of Athens, Greece); Andreas Giannakoulopoulos (Ionian University, Greece);Dr Stefan Goebel (Technical University Darmstadt, Germany); Pedro Pablo Gomez-Martin (Universidad Complutense, Madrid, Spain); Cleo Gougoulis (Peloponnesian Folklore Foundation, Greece);Dr Dimitris Gouscos (University of Athens, Greece); Maria Grigoriadou ( University of Athens, Greece);Dr David Guralnick (Kaleidoscope Learning, New York, USA);Dr Thomas Hainey (University of the West of Scotland, UK); Paul Hollins (The University of Bolton, United Kingdom);Dr Birgitte Holm Sorensen (Aalborg University, Copenhagen, Denmark);Professor Rozhan Idrus (Universiti Sains Malaysia, Malaysia);Dr Jose Ignacio Imaz (University of the Basque Country, UPV-EHU, Spain); Jeffrey Jacobson (Carnegie Museum viii

of Natural History, Pittsburgh, Pennsylvania, USA); Ruben Jans (Limburg Catholic University College, Belgium); Runa Jesmin (Global Heart Forum, UK);Dr Larry Jones Esan (London Academy Business School, UK); Alexandros Kakouris (University of Athens, Greece); Fragiskos Kalavassis (University of the Aegean, Greece);Prof Konstantinos Kalemis (National Centre of Local Government and Administration, Greece); Dr Michail Kalogiannakis (University of Crete, Faculty of Education, Crete); Dr Anastasios Karakostas (Aristotle University of Thessaloniki, Greece); Dr Elisabeth Katzlinger-Felhofer (Johannes Kepler University, Linz, Austria);Dr Harri Ketamo (Satakunta University of Applied Sciences, Finland);Dr Kristian Kiili (Tampere University of Technology, Pori, Finland); Evangelia Kourti (University of Athens, Greece); Rolf Kretschmann (University of stuttgart, Germany);Dr Timo Lainema (University of Turku, Finland);Prof Miguel Leitao (ISEP, Portugal);Dr. Ximena Lopez Campino (Initium, Italy,); Carrie Lui (James Cook University, Australia);Dr Hamish Macleod (University of Edinburgh, UK);Dr. Rikke Magnussen (Danish school of education, Aarhus university/Steno Health Promotion Center, Denmark); Emanuela Marchetti (Aalborg University Esbjerg, Denmark,); Bertil Marques (Instituto Superior de Engenharia do Porto (ISEP), Portugal);Dr Jean-Charles Marty (LIRIS lab, Lyon, France); Stephanos Mavromoustakos (European University Cyprus, Cyprus); Florian Mehm (Technische Universität Darmstadt, Germany); Michail Meimaris (University of Athens, Greece); Bente Meyer (The Danish University of Education, Denmark); Prof Florence Michau (Grenoble Institute of Technology, France);Dr Christine Michel (INSA-Lyon, France);Dr Jonathan Moizer (University of Plymouth, UK); Assoc Prof Begona Montero-Feta Universitat Politecnica de Valencia Dr Adam Moore (Trinity College, Ireland); Alexander Moseley (University of Leicester, UK); Dulce Mota ( Instituto Superior de Engenharia do Porto (ISEP), Portugal); Constantinos Mourlas (University of Athens, Greece); Peter Mozelius (Stockholm University, Department of Computer and Systems Sciences, Sweden); Karen Neville (University College Cork, Ireland);Dr Piotr Nowakowski (John Paul II Catholic University of Lublin, Poland); Kimmo Oksanen (Finnish Institute for Educational Research, University of Jyväskylä, Finland);Dr John O'Mullane (University College Cork, Ireland);Dr. Michela Ott (Institute Educational Technology, Italy); Dimitra Panagouli (Hellenic American Educational Foundation, Greece); George Papakonstantinou (University of Thessaly, Greece); Agis Papantoniou (Multimedia Laboratory of the School of Electrical and Computer Engineering (ECE) of the National Technical University of Athens (NTUA). , Greece);Dr Marina Papastergiou (University of Thessaly, Greece); Paul Peachey (University of Glamorgan, Treforest, UK); Gilbert Peffer (CIMNE, Spain);Dr Neil Peirce (Trinity College Dublin, Ireland);Dr Eva Petersson Brooks (Aalborg University Esbjerg, Denmark); Elias Pimenidis (University of East London, UK);Professor Selwyn Piramuthu (University of Florida, Gainesville, USA);Prof. Dr. Maja Pivec (FH JOANNEUM University of Applied Sciences, Austria); Angeliki Poylymenakou (Athens University of Economics & Business, Greece); Dr Aristidis Protopsaltis (Institut für Lern-Innovation Friedrich-Alexander-Universität, Germany); Rosa Reis ( Instituto Superior de Engenharia do Porto (ISEP), Portugal);Prof Dr Bernd Remmele (WHL Graduate School of Business and Economics Lahr, Germany); Vyzantinos Repantis (Psychico College, Hellenic-American Educational Foundation, Greece,); Simos Retalis (University of Piraeus, Greece);Dr Pauline Rooney (Dublin Institute of Technology, Ireland);Dr Eleni Rossiou (University of Macedonia,Thessaloniki, Greece);Dr Maria Roussou (makebelieve design & consulting, Greece);Dr Florin Salajan (North Dakota State University , Canada); Jordi Sanchez Navarro (Universitat Oberta de Catalunya, Spain); Manthos Santorineos (School of Fine Arts in Athens, Greece);Dr Olga Shabalina (Volograd State Technical University, Russia); Samir Shah (Penn State University, USA);Dr Markus Siepermann (Technische Universität Dortmund, Germany); Helga Sigurdardottir (Nord Trøndelag University College and the Norwegian University of Science and Technology, Norway);Dr Gavin Sim (University of Central Lancashire, England);Dr. JulieAnn Sime (Lancaster University, UK);Dr Chrysanthi Skoumpourdi (University of the Aegean, Greece,);Prof Venustiano Soancatl (Universidad del Istmo, Mexico); Elsebeth Sorensen (University of Aarhus, Denmark);Dr Mark Stansfield (University of West of Scotland, UK); Martin Steinicke (University of Applied Sciences HTW Berlin, Germany);Dr. Arnab Sylvester (Coventry University, UK);Dr Sabin Tabirca (University College Cork, Ireland, ); Uday Trivedi (R.C. Technical Institute, India);Dr. Thrasyvoulos Tsiatsos (Aristotle University of Thessaloniki, Greece,);Dr Chuang Tsung-Yen (National University of Tainan, Taiwan); Richard Tunstall (University of Leeds, UK);Dr Andrea Valente (Aalborg University Esbjerg, Denmark);Dr Peter Van Rosmalen (CELSTEC / Open University of the Netherlands, The Netherlands);Dr Linda Van Ryneveld (Tshwane University of Technology, Pretoria, South Africa);Dr Carlos Vaz de Carvalho ( Instituto Superior de Engenharia do Porto (ISEP), Portugal);Dr Ayelet Weizman (Snunit center for the advancement of web-based learning, the Hebrew University, Israel); Viktor Wendel (Technical University Darmstadt, Germany); Nicola Whitton (Manchester Metropolitan University, UK); Dorothy Williams (Robert Gordon University , UK); Andrew Wilson (Birmingham City University, United Kingdom); Amanda Wilson (University of the West of Scotland, Scotland);Dr. Aljona Zorina (ESCP Europe, France)

Biographies Conference Director Professor Thomas M Connolly is the original instigator of this conference in 2007, Thomas Connolly is a Professor in the School of Computing at the University of the West of Scotland, having managed the Department of Computing and Information Systems for several years. Thomas worked for over 15 years in industry as a Manager and Technical Director in international software houses before entering academia. His specialisms are games-based learning, online learning and database systems. He has developed three fully online MSc programmes and developed and leads the undergraduate BSc Computer Games Technology programme. He is co-author of the highly successful academic textbooks Database Systems (now in its 4th edition) and Database Solutions (in its 2nd edition). He is a reviewer for several international journals and has been on the committee for various international conferences. He is a member of CPHC (Council of Professors and Heads of Computing) and member of the Higher Education Academy.

Conference Chair Dr Carlos Vaz de Carvalho has a PhD in Information Systems and Technology. He is a Professor at the Engineering School of the Porto Polytechnic (ISEP) and the Director of the R&D group GILT (Graphics, Interaction and Learning Technologies). He was e-Learning Director (2001-2005) of ISEP and Director of the Distance Learning Unit of the Porto Polytechnic (1997-2000). He has published over 100 references on the subject including several books.

Programme Chair

Dr Paula Escudeiro is a Professor at IPP-ISEP with vast experience in project supervision and evaluation, accumulated for the past 21 years. She has a PhD on Informatics/Information Systems on Education and prior experience on software industry related to the development of educational software. She is the director of the Multimedia Laboratory at ISEP and belongs to the Graphics, Interaction and Learning Technologies research center (GILT).

Keynote Speakers Dr Baltasar Fernández-Manjón is a full professor in the Facultad de Informatica at the Complutense University of Madrid (2001-2006) and the Vice Dean of Research and Foreign Relationships at the Computer Science School at UCM (2006-2010). He is IEEE Senior Member and in 2010-2011 he has been Visiting Associate Professor at Harvard University and Visiting Scientist at LCS-MGH. He received a Bachelor in Physics (major in Computer Science) and a PhD in Physics from the UCM. He is member of the IFIP Working Group 3.3 "Research on the Educational uses of Communication and Information Tecnlogies" and of the Spanish Technical Committee for E-learning Standarization (AENOR CTN71/SC36 "Tecnologías de la información para el aprendizaje"). His main research interests are e-learning technologies, educational uses of serious games, application of educational standards, and user-modelling. Dr Paulo J. Gomes is the Game Director and Producer at BIGMOON STUDIOS and he produced games such ‘WRC3’, ‘MotoGP13’, ‘Jagged Alliance: Back in Action’, ‘Trapped Dead Lockdown’ and many others. Has more than 20 years of experience in software development and project management. He’s credited in more them 15 videogames published worldwide on PS3, Xbox360, Wii, PC, Linux, Mac and Mobile. Paulo has a PhD in Computer Science, MBA and he is a Multimedia Professor at Portucalense University.

Mini Track Chairs Prof. Dr Wilfried Admiraal is a full professor of Educational Sciences and chair of the research program Teaching and Teacher Learning of Leiden University Graduate School of Teaching. His research interest combines Educational Sciences, Social Psychology and technology. He published journal articles on mobile game-based learning and game creation by students. Dr Daniel Aranda is a Senior Lecturer in the Department of Information and Communication Studies at the Open University of Catalonia. He is researching on how young people use digital technologies. He works in the research group SPIDER (Smarter People through Interactive Digital Entertainment Resources), Communication & New Media (at the Internet Interdisciplinary Institute / IN3) and eCo (research and innovation in e-learning, information and communication), at the eLearn Center (UOC).

Dr Stefan Göbel holds a PhD in computer science from TUD and has long-term experience in Graphic Information Systems, Interactive Digital Storytelling, Edutainment applications and Serious Games. After five years work as researcher at Fraunhofer Institute for Computer Graphics, from 2002 to 2008 he was heading the Digital Storytelling group at the Computer Graphics Center in Darmstadt. In late 2008 he moved to TUD and is heading the prospering Serious Gaming group at the Multimedia Communications Lab. Dr. Göbel is author of numerous papers and member of different program committees such as ACM Multimedia, ICME, Edutainment, Foundations on Digital Games, Serious Games Conference and serves as jury member of the Serious Games Award. Dr. Thomas Hainey is a Lecturer in Computer Games Technology and Serious Games Researcher at the University of the West of Scotland. He teachers an honours level course in serious games and is primarily interested in the empirical evaluation of games-based learning applications and how to integrate assessment into games-based learning applications. He has a number of publications in this area. Dr Kristian Kiili works as a Senior Research Fellow and an Adjunct Professor at the Tampere University of Technology in which he heads the Advanced MultimediaCenter research laboratory. His current research focuses on game-based learning, user generated game content, game design, and educational exertion games. He is presently involved in two European initiatives: the Game and Learning Alliance (GALA NoE) and Making Games in Collaboration for Learning (MAGICAL). Results received from his research has been published in several scientific publications as well as applied in commercial products Johannes Konert finished his diploma in Computer Science and accompanying studies in cultural studies at the Karlsruhe Institute of Technology (KIT) with a thesis proposing a web-based knowledge management system for the integration of workflow and learnflow. After three years working on the foundation and development of the online social network friendcafe as CEO and senior developer he joined the research group at Multimedia Communication Lab (KOM) at Technische Universität Darmstadt to focus on Serious Games and Social Networks. He became a Ph.D. student of the DFG Research Training Group “Feedbackbased Quality Management in E-learning”. In his research he focusses on the development of solutions to use Social Media concepts for knowledge transfer between peers in Serious Games. Prof Konstantinos Kalemis is an Instructor at the National Centre for Public Administration and Local Government (E.K.D.D.A.) in Adult Education and Lifelong Learning and assigned at the Dept of Primary Education in National and Kapodistrian University of Athens. He has authored a large number of scientific articles, studies and papers in Educational Congress and Seminars. His interests focus on the introduction of New Technologies as an alternative teaching process and the design of new curriculum plans for the open and d-Learning. His research interests also include the education of immigrant ethnic minorities focusing on the gifted and talented students and aim to advance the theory and technology of natural language and knowledge processing, especially semantic analysis that bridges the gap between language and knowledge, by the novel use of both machine learning and inference methods. Dr Jordi Sánchez-Navarro is a Senior Lecturer in the Department of Information and Communication Studies at the Open University of Catalonia (Universitat Oberta de Catalunya).His research revolves around innovation in entertainment and how this interacts with the new practices of cultural consumption in the contemporary media landscape, and how they impact on education. Dr Thrasyvoulos Tsiatsos is currently Assistant Professor in the Department of Informatics of Aristotle University of Thessaloniki. He obtained his Diploma, his Master's Degree and his PhD from the Computer Engineering and Informatics Department of Patras University (Greece). His research interests include Networked Virtual Learning Environments, Computer Uses in Education, Evaluation methods of Internet Learning Environments and Open and Distance Education using Multimedia and Internet Technologies. He has published more than 120 papers in Journals and in well-known refereed conferences and he is coauthor in 3 books. He has been a PC member and referee in various international journals and conferences and participated in more than 20 R&D projects. Also he is member Technical Chamber of Greece. Viktor Wendel received his degree in Computer Science from the Julius-Maximilians-University of Würzburg in 2009. Since November 2009, he is working as a research assistant at the MultimediaCommunications-Lab at the Technical University of Darmstadt. Research topics are Game Mastering in Multiplayer Serious Games, and Collaborative Learning. Further, he is an editor for ACM SIGMM Records.

Biographies of Presenting Authors Aishah Abdul Razak received her MSc. in Information Technology from Multimedia University, Malaysia. She is now pursuing her PhD in the School of Computing at the University of the West of Scotland. Her research interest is in games-based learning for primary school children. Wilfried Admiraal is full professor of Educational Sciences and chair of the research program Teaching and Teacher Learning of Leiden University Graduate School of Teaching. His research interest combines Educational Sciences, Social Psychology and technology. He published journal articles on mobile game-based learning and game creation by students Rafael Marques de Albuquerque graduated as Bachelor (2008) and Master (2011) in Graphic Design in the Federal University of Santa Catarina (Brazil) and is currently carrying out his PhD in Education in the Learning Sciences Research Institute of the University of Nottingham (UK). His research interests are digital games, learning and school, especially game literacy. Rene Alimisi has a rich engineering background and thorough knowledge of the field of Information Communication Technology in Education. She holds a master with distinction in ICT in Education (IOE, University of London). She has more than 4 years experience within the area of Lifelong Learning European Projects and teaching experience in well- known institutions in Greece and UK. Anissa All works as a junior researcher at IBBT-MICT (Ghent University) since July 2011. Since January 2013, Anissa is working on a PhD through a IWT grant (Flemish agency for Innovation by Science and Technology). In this PhD research, she will develop a standard procedure to measure effectiveness of serious games aimed at cognitive learning outcomes. Yasemin Allsop has been working as an ICT coordinator in primary Schools across London for almost 10 years. MPhil/PhD student at Goldsmiths, University of London. Focus is on children’s learning and cognitive development through digital game making activities. Interested in the role of teachers when teaching with digital games and game design. Reem Al-Washmi is a second-year PhD student at the University of Nottingham, in the School of Computer Science. Her PhD is entitled “Collaborative games-based learning to support problem solving in UK KS2 Mathematics”. Her long term interests include the study of problems that children face in mathematics and the use of technology to overcome these. António Andrade has a Degree in Communication Design from the Faculty of Arts of the University of Porto. Currently he is working on his Computer Science MSc at the School of Engineering of the Porto Polytechnic (ISEP), researching on Virtual Communities of Practice. Massimiliano Andreoletti Professor of Educational gaming and animation at the Catholic University of the Sacred Heart of Milan. Researcher and author of several essays and articles in the Media Education and Educational Technologies (internet, video games, mobile, tablet/pad, cloud). He’s still a videogamer and father of a child of five years. Alexandra Androussou is a Associate Professor in Teaching Methodology at National and Kapodistrian University of Athens, Greece. Her research and writing focus on teaching practices, teacher education and education of minority groups. She also deals with the development of educational materials for children in electronic and conventional form and she produces educational materials for teachers such as the website www.kleidiakaiantikleidia.net Anna Arici is a Learning Scientist with the Center for Games & Impact, at Arizona State University, where she designs and researches game-based learning environments for educational and social challenges, change, and sustainable impact. Additionally, as director of Quest2Teach, she creates game-infused learning curricula and gamification systems for pre-service teachers to prepare and support highly effective educators. Jannicke Baalsrud Hauge is research scientist at Bremer Institut für Produktion und Logistik(BIBA). She is teaching decision making in SC at the University of Bremen and Jacobs University. Main interest: Serious games, TEL, use of ICT in logistics. Responsible for many WPs in EU and national projects on ICT applications, logistics and TEL. Authored 100+ papers. Sasha Barab is a Professor in the Teachers College at Arizona State University where he co-founded and serves as the Executive Director of the Center for Games & Impact. Dr. Barab is an internationally recognized learning scientist who has researched, designed, and published extensively on the challenges and opportunities of using games for impact.

xii

Lizzy Bleumers graduated as a Master in Psychology and obtained a postgraduate degree in user-centered design. She now conducts user research at iMinds-SMIT (Vrije Universiteit Brussel) within the area of play, learning and participation. She was recently involved in a European project, which explored how digital games can be part of empowerment and inclusion initiatives. Dr. Rosa Maria Bottino Director of the Institute for Educational Technology of the Italian National Research Centre (ITDCNR). Author of more than 100 scientific publications both in national and international journals, books and conference proceedings. Dr. Bottino participated in both national and European projects and Networks of Exellence in Technology Enhanced Learning, including GALA NoE on serious games Cyril Brom is an assistant professor at Faculty of Mathematics and Physics of Charles University in Prague. His research interest is in serious games, modelling behaviour and episodic memory of virtual human-like characters, in level of detail AI, and in computational biology. Hope Caton lectures Game Creation Processes at Kingston University, London, where she also leads inKUbator, a multidisciplinary games development studio Hope founded in 2010 with Dr Darrel Greenhill. In addition to investigating the effects of introducing gamification in the classroom, Hope’s areas of research include developing serious games for health and education. Nathalie Charlier is an assistant professor at the Faculty of Pharmaceutical Sciences and co-ordinator of the Teacher Training in Health Sciences Education at the KU Leuven, Belgium. She obtained a BSc and MSc in Pharmaceutical Sciences and her PhD in Medical Sciences. Her current research interests are (i) game-based learning in health science education and (ii) the use of new technologies in education. Yaëlle Chaudy researcher at the University of the West of Scotland. She obtained an MSc in computing from INSA Lyon (Institut National des Sciences Appliquées) and a bachelor in French as a Second Language from Stendhal University in Grenoble. Interested in both computing and education, she is now studying the integration of assessment in GBL applications. John Denholm In final stages of PhD at Serious Games Institute, Coventry, researching into value of educational games. MSc from Imperial College, London and held several senior positions in major UK companies, mainly corporate planning and development of strategic business models. He has lectured on Business, Project Management and Finance courses at Birmingham City and Coventry Universities and supervises Master’s students at Warwick and Manchester Universities. Jill Denner, PhD, is a Senior Research Scientist at Education, Training, Research, a non-profit organization in California where she studies how students learn while creating computer games. She has written numerous peer-reviewed articles, and coedited: “Beyond Barbie and Mortal Kombat: New Perspectives on Gender and Gaming,” published by MIT Press in 2008. Kyriaki Dourda is a post-graduate student at the Early Childhood Education Department, at the University of Western Macedonia, Greece. She has graduated from the School of English Language and Literature at the Aristotle University of Thessaloniki. Her research interests include: Learning and Teaching Modern languages, GBL, CLIL, Language learning strategies. Ronald Dyer has held senior positions in the area of technology strategy, transformation and change management for performance improvement in the US & Trinidad & Tobago. He is a final year candidate for the Doctorate in Business Administration at Grenoble Ecole de Management, France, focused on research on serious game. Stine Ejsing-Duun is interested in the relation between technology, perception and cognition. Her ambition is to describe how technologies allow us to transcend ourselves. Her research has been connected to games, play and playful processes in various areas. Her present studies are within the fields of learning and art. Lars Elbæk (Ph.D.) is associate professor at University of Southern Denmark, and director of the research group ‘PE Pedagogy and Sports Psychology’. He has worked with interaction design and use of IT in children and youth sport and physical education through a number of years. Lars Elbæk is currently working partly at the Play and Learning – Kids n’ Tweens Lifestyle EU founded project. www.kidsntweens.dk Valérie Emin, PhD, is a researcher at the Institut Français de l'Éducation, member of S2HEP Laboratory. She coordinates since 2008 a research project on pedagogical scenario design in science and technique discipline. Her current research topics are "Pedagogical scenarios design" and "Game based learning". She's an associate member of GALA european network of excellence.

xiii

David Farrell is a lecturer in game design at Glasgow Caledonian University with interests in player centric design and Serious Games. He was the lead designer and developer of the e-Bug Serious Games and his research focuses on improving the integration of good game design practice with well grounded pedagogy. Toru Fujimoto is an assistant professor at the Center for Research and Development of Higher Education, The University of Tokyo. He completed his graduate study at the Pennsylvania State University (Ph.D. in Instructional Systems). His research focus is on the design of learning environments using digital games and social media. Georgy Gerkushenko conducts research in the area of social informatics and e-learning since 2000. He gained PhD degree on creating electronic educational resources in 2004 and MBA degree on “Chief Information Officer” in 2009. Currently he is a senior lecturer of CAD department at Technical University in Volgograd, Russia. Main research interests are personal learning environment, education social networking and electronic educational resources. Dr. Lisa Gjedde is Professor with special responsibilities at Aalborg University in Copenhagen at the Dept. for Learning and Philosophy, director of the Research Center for Creative and Immersive Learning Environments: reCreate. Dr. Stefan Göbel holds a PhD in computer science from TUD and has long-term experience in Graphic Information Systems, Interactive Digital Storytelling, Edutainment applications and Serious Games. Since 2008 he is heading the Serious Gaming group at the Multimedia Communications Lab of Technische Universität Darmstadt. Paridhi Gupta is a PhD research student in the department of School of Design at the Hong Kong Polytechnic University (Hong Kong). Her research investigation focuses on interactive play and games within English Language subject classrooms in Hong Kong local Primary schools. She has a Master’s Degree in visual communication from IDC, IIT Mumbai. Dr. Thomas Hainey is a Lecturer in Computer Games Technology and Serious Games Researcher at the University of the West of Scotland. He teachers an honours level course in serious games and is primarily interested in the empirical evaluation of games-based learning applications and how to integrate assessment into games-based learning applications. He has a number of publications in this area. Claire Hamshire has worked at Manchester Metropolitan University (MMU) since 2003; initially as a Senior Lecturer in Physiotherapy and from 2008 as a Senior Learning and Teaching Fellow. This role combines faculty teaching with a cross institutional contribution to technology and games-based innovation. She was awarded a Higher Education Academy National Teaching Fellowship in 2012 Thorkild Hanghøj is an Associate Professor at the ResearchLab: IT, Learning and Design (ILD), Aalborg University, Copenhagen. He holds a PhD on the playful knowledge in educational gaming. Research areas include: game-based teaching, games and Mother Tongue Education, and problem-based game design. Shinobu Hasegawa received his B.S., M.S., and Ph.D. degrees in systems science from Osaka University in 1998, 2000, and 2002. He is now an associate professor in Center for Graduate Education Initiative, Japan Advanced Institute of Science and Technology. His research areas include support for Web-based learning, game-based learning, language learning, and community based learning. Wayne Holmes is currently Head of Education for the games-based learning company zondle. Previously, he was a teacher and the Head of Research for an education charity. He has an MA in Philosophy, an MSc (Oxon) in Education, and is completing his DPhil (doctorate) in Education (researching games-based learning) at the University of Oxford. Robyn Hromek is a practicing educational psychologist working in Australian schools and an Honorary Associate of the The University of Sydney, Australia. She has created a set of 15 board games to teach social and emotional skills to children and young people and has spoken at numerous international conferences on games-based learning. Jennifer Jenson is Professor and Director for the Institute of Research on Learning Technologies at York University, Canada. She has published on games, game design, gender and game play and digital games in education Helle Skovbjerg Karoff (PhD) is Assistant Professor at Aalborg University/Copenhagen and a member of ILD: IT and Learning Design. Helle´s main research field is play and interaction with technology, especially questions of the dynamics of play, for example through danger, movement and sociality. Harri Ketamo, PhD founder/chief scientist, Eedu ltd. and Adjunct Professor, Tampere University of Technology. Specialized in Complex Adaptive Systems, Cognitive Psychology of Learning, Neural Computing and Educational Technology. Was Direcxiv

tor of Education, Satakunta University of Applied Sciences and CEO & founder GameMiner ltd, company focusing on Data Mining/game AI. Published international/peer-reviewed research articles; presentations on studies in international forums. Several awards and nominations related to R&D activities. Michael Kickmeier-Rust holds a PhD in cognitive psychology and he is an experienced project manager and software developer. His research and development activities focus primarily on technology-enhanced learning, in particular intelligent, adaptive educational systems and human-computer interaction. Since 2010 Michael is with the Knowledge Management Institute at Graz University of Technology. Kristian Kiili works as an adjunct professor at Tampere University of Technology. His research focuses on game based learning, exergaming, and game design issues. Results received from his studies has been published in several scientific publications as well as applied in commercial e-learning products. Luise Klein obtained a MSc. degree in Digital Media from the University of Applied Sciences Bremerhaven. Her interests are in enabling learning with and about media and technology, especially in informal playful settings. She also develops her game-based learning and mobile learning applications. Antti Koivisto is a Ph.D. student at the Tampere University of Technology in Pori, Finland. He currently works at Satakunta University of Applied Sciences as a researcher and at Eedu Ltd as a game developer. His research interests are exergames. He is currently researching how games suit for elderly and mentally disabled people. Johannes Konert finished his diploma in Computer Science at the Karlsruhe Institute of Technology (KIT). After three years work on the foundation and development of the online social network friendcafe, in June 2010 he joined the research group at Multimedia Communication Lab (KOM) at Technische Universität Darmstadt to focus on Serious Games and Social Networks. Evangelia Kourti is an associate Professor of Social Psychology specializing in communication at the National and Kapodistrian University of Athens, Greece. Her research interests cover the scientific fields of communication, media and children and the psychology of cyberspace. Loukas Koutsikos holds a Master Degree (MSc) in "ICT for Education", from the National Kapodistrian University of Athens. He holds a Bsc of Electrical Engineering Educator from the Higher School of Pedagogical and Technological Education. He works in Secondary Vocational Education and has participated in various programs dealing with the implementation of Educational Technology. Dimitrios Lappas has graduated from the Hellenic Military Academy in 2005. He also has a Bachelor's Degree from Pre-school Education and Educational Design Department of the University of the Aegean. He is currently a postgraduate student in “Models of Designing and Planning Educational Units”, Master's and PhD Degree program at the University of the Aegean. José Rafael López-Arcos Member of the GEDES research group in the Department of Computer Languages and Systems at the University of Granada. His research focuses on the integration of storytelling into educational video games. Rikke Magnussen associate professor at ResearchLab: ICT and Design for Learning, Aalborg University Copenhagen. Main research interest is how game-based technology can open for innovation in science education. Part of numerous national and international learning game development and research projects for over ten years and published extensively on subject of game's potentials in science education. Gunver Majgaard (PhD) is Associate Professor at The Maersk Mc-Kinney Moller Institute, University of Southern Denmark. Her research interests are design of digital educational tools; human computer interaction; participatory design processes; learning processes; didactical design; program and curriculum development. She has developed the engineering program "Learning and Experience Technology". Christos Malliarakis is a teacher of Computer Science in Mandoulides Schools, a large private primary/secondary school in Thessaloniki, Greece. He holds a BSc and an MSc in Informatics from the Computer Science Department of the Aristotle University of Thessaloniki and he is undergoing his PhD research in Game Based Learning on Computer Programming since May 2011. Dionissios M. Manessis holds a M.Sc. in ICT for education, from the University of Athens, Greece. He is now a Ph.D. student at the department of Early Childhood Education of the University of Athens. His research interests include the use of digital games in Early Childhood Education and students’ attitudes towards Statistics. xv

Jean-Charles Marty associate professor at LIRIS lab in Lyon (France). Research interests are in observation of collaborative activities, through traces of these activities. Research results are applied to Technology Enhanced Learning, and in particular to learning game environments. Participates to several projects in this field (Learning Adventure, Learning Games Factory, Serious Lab for Innovation, Pegase). Organized an international school on Game-Based Learning in June 2011. Apostolos Mavridis is a PhD candidate on the subject of “Game Based Learning” in the Department of Informatics, Aristotle University of Thessaloniki, Greece. He holds a BSc in Computer Science and an MSc in ICTE (Information and Communication Technology in Education). Mas Idayu Md Sabri is a PhD student at the University of Nottingham. She is currently on study leave from her employment as a lecturer at the University of Malaya, Malaysia. She obtained her BComp Science from University of Malaya, majoring in Software Engineering and obtained her MSc Multimedia Technology from the University of Bath. Her research interests are multimedia technology, edutainment, and interactive learning. Emna Mejbri She obtained the master degree in computer sciences, from the University of Kairouan, Tunisia in 2011. Currently, she is a phD student at the National School of Computer Sciences of Mannouba, Tunisia. Her main research interest is in the area of Learning and Games development. Javier Melero received both his Engineering degree in Computer Science (2008) and Master in Information, Communication and Audiovisual Media Technologies (2009) from the Universitat Pompeu Fabra (UPF), Spain. Since 2006, he has been involved in European and National projects in the field of TEL. His main research focus is about designing technology-supported puzzle-based games. Hélène Michel is a Senior Professor in Grenoble Ecole de Management (France). With a specialization in Innovation Management, she started working on Serious Games in 2003. Her research focuses on the strategic approach of serious games and on their performance’s evaluation. Alex Moseley is an Educational Designer and University Teaching Fellow at the University of Leicester, with long experience of course design and development in higher education. His research areas are in games-based learning, student engagement and effective research skills, and he designs games for education and museum contexts. Peter Mozelius has since 1999 been employed as a teacher for the Stockholm University and the Royal Institute of Technology at the Department of Computer and Systems Sciences (DSV) in Kista, Sweden. He is currently working as an IT-Pedagogue and researcher. His research interests are in the fields of ICT4D, game-based learning and software engineering. Robin Munkvold has been teaching software design at Nord-Trondelag University College (Norway) since 1999. The last five years he has been Program Director within the field of Digital Games and Media Technology. He has published several papers on subjects regarding online learning and ICT as a tool for supporting different pedagogical approaches. Rob Nadolski is assistant professor in TEL at CELSTEC at the Open University of the Netherlands. Main interests: competencebased education, serious games, especially enhancing learner support facilities by exploiting newest technologies. Involved in design and project management of e-learning applications for acquiring complex cognitive skills and research on such applications. Has participated in European and national projects. Elena Núñez Castellar received her degree in Psychology in 2006. In 2011 she obtained her degree of Doctor in Psychology from the Ghent University. During her PhD she got broad experience with research methods in cognitive neuroscience, namely EEG (electro-encephalography) and reaction times research. Since 2012 she joined the research group for Media & ICT (iMinds-MICT). Chinedu Obikwelu is a PhD research student with the ChiCI research group, University of Central Lancashire, he is currently researching the scaffolding mechanism in serious games with emphasis on adaptive scaffolding. He has worked in both the educational and IT sector as a teacher and an IT Support Officer respectively. Daire Ó Broin holds a Ph.D. in Computer Science from TCD, which focussed on approaches to developing the conditions of flow. He has been a lecturer at IT Carlow since 2008, where he teaches on the Computer Games Development programme . His research interests include increasing engagement and intrinsic motivation in games and learning. Kelly O’Hara (Ph.D. Sport Science) is professor at Beira Interior University, Portugal, and a researcher at Sports Sciences, Health Sciences and Human Development Centre. Her research interests are develop game based-learning environments by

xvi

integrating, sports and health, and tennis training. She has published several papers, book chapters, and she is reviewer in international journals. Kimmo Oksanen (Lic.Ed.) is doctoral student at the Finnish Institute for Educational Research (FIER), University of Jyväskylä. He is working on his doctoral thesis about supporting and evaluating collaborative learning in a game context. His research interests include game experience, game design and collaborative learning. Mark O'Rourke is an Educational Advisor with the Curriculum Innovation Unit at Victoria University, Melbourne, Australia. He has worked as a VET researcher, Multimedia Lecturer, Program Manager, Head of School, and Chair Academic Board. Mark's research activity focuses on games-based learning and he is a Fulbright Professional Scholar having undertaken research at USC in LA. M’hamme Ali Oulhaci is a PhD student at LSIS laboratory Aix-Marseille University; his works include behaviors simulation, multi-agents systems, and learners’ assessment in serious games. He got his master at Paris Dauphine Unversity. Contact him at LSIS, Domaine universitaire de Saint Jérôme Avenue Escadrille Normandie Niemen 13397 MARSEILLE Cedex 20. Trygve Pløhn works as a lecturerer and a researcher at the Nord-Trøndelag University College. He obtained his MSc in Software Development, Information Technology from the IT University of Copenhagen in 2007. He is a PhD Candidate at the Norwegian University of Science and Technology. His main research interest is within Pervasive Games and Serious Games. Jakub Procházka, Ph.D. is an assistant professor at the Department of Corporate Economy and at the Department of Psychology, Masaryk University, Brno, Czech Republic. His current research focuses on psychology of leadership and leadership effectiveness. He teaches interactive courses in the field of organizational and work psychology. Dr Aristidis Protopsaltis is a Senior Researcher at the Institut für Lern-Innovation at Friedrich-Alexander-Universität ErlangenNürnberg. His background is in Cognitive Science, Serious Games, ICT and Education. He is involved in a number of European projects with focus on education, e-learning and Serious Games. He has published numerous peer-reviewed conference and journal papers. Wen Qi researcher at CELSTEC Open Universiteit. PhD in Men Machine Interaction. Research interests are in Virtual Environments, Serious Games (for learning) and Human Computer Interaction. He has worked in different research projects in both academia and industry. Those research projects were sponsored by US, European and national funding agencies. He is now active in gaming based learning. Rosa Reis teaches at IPP-ISEP, Computer Engineering Department. MSc on Information Systems in Education and PhD student on Informatics at University UTAD-Tras-os-Montes and Alto Douro, Vila Real. Researcher at GILT-Graphic Interaction & Learning Technologies. Researches application of techniques of software engineering in design of educational collaborative virtual environments. Involved in several National and European research projects, presently regular reviewer of several conferences and scientific journals. Christian Reuter studied Computer Science at TU Darmstadt and finished his Master Thesis about the “Development and Realization of Methods and Concepts for Multiplayer Adventures” in 2011 before he then joined the Multimedia Communication Lab. His research focus includes the Authoring-Platform “StoryTec”, especially its extension for multiplayer serious games. Tiago Ribeiro is an eclectic researcher seeking harmony between arts and technology. He has been collaborating internationally on research projects like LIREC and EMOTE, and also with CMU, focusing especially on non-verbal expression in robots both through animation and sound. He is currently in an early stage of obtaining his PhD, in which he pursues artist-oriented intelligent robot animation. Gabriela Rodríguez has a B.A. degree in Spanish Literature and a M.A. in Education. She is currently a PhD student at Turku University’s Faculty of Education, where she is part of a research group designing a mathematical Serious Game as a tool to develop flexible and adaptive use of arithmetic strategies amongst upper elementary children. Martin Ruskov is currently a PhD student at the UCL Deptartment of Computer Science. He has previously worked in the areas of interactive storytelling and authoring tools for multimedia publishing. For his PhD Martin explores how to develop effective serious games and measure the learning happening with their use.

xvii

Eric Sanchez is associate professor at the Ecole Normale Supérieure de Lyon, head of EducTice, a research team of the French Institute for Education and adjunct professor at the University of Sherbrooke, QC (Canada). His research work concerns the uses of digital technologies for educational purposes (e-learning, simulation, serious games). Luciano Santos has a graduate degree in informatics engineering from the Engineering School of the Porto Polytechnic (ISEP) in 2011, and is currently undertaking a Master’s degree in graphics and multimedia systems at ISEP. Ángel Serrano-Laguna, MSc, works for the Complutense University of Madrid as a researcher in the e-UCM e-learning group as well as being a PhD student. His current research interests are educational video games, learning analytics and the eAdventure project. He has published 8 academic papers related to these topics. Helga Sigurdardottir is a PhD candidate at the Nord-Trøndelag University College. She is attending the PhD program in Interdisciplinary Culture Studies at The Norwegian University of Science and Technology, Trondheim, Norway. Helga has a bachelor degree in Social Anthropology, a Master in Education (Program Evaluation) and a Teacher Certificate from The University of Iceland. Petr Smutný is assistant professor at the Department of Corporate Economy of Faculty of Economics and Administration, Masaryk University, Brno, Czech Rep. His current research focuses on managerial simulation games effectiveness and leadership effectiveness. He teaches several courses using game based approach. Currently he is vice-dean for external affairs of the faculty. Heinrich Söbke researcher in “Intelligent Learning” programme (www.intelligentes-lernen.de) at Bauhaus-Universität Weimar. Focuses on game based learning, where his background in computer science enables him to transfer software design principles into technical design of video games. Ws visiting scholar in Department of Curriculum & Instruction at University of Wisconsin, Madison, when worked on development of educational games at Morgridge Institute for Research. Mario Soflano is a researcher at University of the West of Scotland. His background education is in computer science. His main interests are computer games technology, games design, educational technology, web development, adaptive system and mobile games / software development. Pooya Soltani has a M.Sc. degree in Exercise Physiology from Shiraz University, Iran. He is now a PhD candidate of Sport Science at University of Porto. He’s interested in characterizing Exergames and their effects in three domains of physiology, biomechanics and psychology. He’s working at Porto Biomechanics Laboratory under supervision of Professor João Paulo Vilas-Boas. Narumon Sriratanaviriyakul (Cherry) is Senior Lecturer in Centre of Commerce and Management at RMIT University Vietnam and has 8 years of teaching experience in higher education in international universities. Her research interests include gamebased learning, online social network, case study methods, and technology in education. Martin Steinicke earned his BSc. and MSc. in Business Informatics at University of Applied Science HTW-Berlin. He works in the research project „Innovationsdramaturgie nach dem Heldenprinzip“ headed by Professor Carsten Busch and teaches DGBL. His work centers on game based learning in the business context and information & knowledge diffusion in social networks. Gunilla Svingby is a Professor of Education, at Malmö University, Sweden. I was professor at Lund University, Gothenburg, Oslo University, and Tromso University. Some research projects: A computer game on ethics as a learning environment, Continuous assessment and dynamic examination, Professional competence with simulations in teacher education, Learning by mobile games, Mathematics for the digital generation. Emilia Todorova has recently graduated in BSc Information Systems Development from Glasgow Caledonian University. She has worked on projects involving the Bologna Process, Quality Assurance and Education Policy. Her research interests are on improving learning and teaching, using technology within higher education and quality assurance in the EHEA. Ofut, Ogar Tumenayu: obtained his bachelor degree in Computer science from Cross River University of Technology, Calabar, Nigeria. He is currently studying for his master degree in Volgograd State Technical University Volgograd, Russian Federation. His scientific research is in field of Design and implementation of adaptive Educational Games. Herre van Oostendorp is Associate Professor Human-Media Interaction at the Institute of Information and Computing Sciences, Utrecht University. His research activities are on the domain of Human-Computer Interaction. He is a specialist on the areas of web navigation, hypertext comprehension, usability engineering and cognitive principles in serious game design xviii

Didin Wahyudin is a lecturer in Indonesia University of Education, Bandung. He received Master degree in Game Technology from Bandung Institute of Technology, Indonesia. He has experienced as a first responder in many disasters. Currently, He is a PhD student at School of Information Science JAIST Japan focusing on research of Mobile Game Based Learning. Ayelet Weizman Director of science education at Snunit center for the advancement of web-based learning, located at the Hebrew University of Jerusalem. Designing educational games and interactive learning and teaching materials in science on several websites. PhD in Planetary Sciences from Tel Aviv University and post doctorate studies in science education at Michigan State University. Viktor Wendel received his degree in Computer Science from the Julius-Maximilians-University of Würzburg in 2009. Since November 2009, he is working as a research assistant at the Multimedia-Communications-Lab at the Technical University of Darmstadt. Research topics are Game Mastering in Multiplayer Serious Games, and Collaborative Learning. Further, he is an editor for ACM SIGMM Records. Dr Nicola Whitton is a Research Fellow at Manchester Metropolitan University, specializing in the innovative use of learning technologies in Higher Education. Her particular interest is in the design and use of computer games with adult learners and she is the authors of Digital Games for Learning, a practical guide to educational game development. Amanda Wilson is a research student at the University of the West of Scotland. Her research focuses on how games based construction learning can be implemented into the curriculum within primary education in Scotland using Scratch. Andrew Sean Wilson worked in biomedical research for last twenty years. Interested in use of technology in medical research particularly in the management of musculoskeletal diseases. Designed and developed educational computer programs to help patients and practising doctors gain better understanding of how to manage these diseases. Sees game based learning as another way of assisting in this. Amel Yessad is PhD in computer science. Currently, she is an associate professor in the team MOCAH–LIP6 of the University Pierre et Marie Curie. Her research focuses on knowledge engineering, technology enhanced learning, and serious games. Dr Yessad is involved in several serious game projects. Ebru Yeniman Yildirim is a senior lecturer and Head of Computer Technology & Programming at Uludag University, Bursa, Turkey. She has written books on computer programming and managed many large-scale EU projects on Vocational Training. She is interested in e-learning, how new technologies impact on the teacher’s role in the classroom and game based learning. Matej Zapušek is employed as teaching assistant for computer science courses at University of Ljubljana, Faculty of Education. He is also a PhD student at University of Ljubljana, Faculty of computer and information science. His main field of interest is researching the possibilities for teaching introductory programming with intelligent tutoring systems. Symeon Zourelidis is a postgraduate student of the M.Sc. program "ICT in Education", in the National Kapodistrian University of Athens. He works as a director in primary school. He participates in teacher training course for the use and the application of ICT in the classroom. Research interest focuses on new technologies in education.

xix

Teachers’ Beliefs About Game Based Learning: A Comparative Study of Pedagogy, Curriculum and Practice in Italy, Turkey and the UK Yasemin Allsop1, Ebru Yeniman Yildirim2 and Marina Screpanti 3 1 Wilbury Primary School, London, UK 2 Uludag University, Bursa, Turkey 3 Istituto comprensivo n.3 Chieti, Italy yallsop@msn.com yeniman@uludag.edu.tr marina.screpanti@virgilio.it Abstract: Digital games are more popular than ever among children and young adults (Prensky 2001; Gee 2003; Fromme 2003; Oblinger 2004). Recent reviews show that children spend hours playing video games either on their console or digitally online. Educators started to see the power of this new medium and explore ways to use digital games to support learning within schools. Incorporating digital games into classrooms, however, has been a challenging task for many reasons; According to Jessel (2012) “Innovation arising from new technologies makes a variety of demands upon the role of the teacher”. The question is; are the teachers ready for these demands, as the new technologies transform their role continually? This study aims to give a comparative account of teachers’ views of their role when teaching using digital games in primary classrooms. Additionally it investigated the interrelation between game based learning, curriculum, pedagogy and practice. This study presents the views of teachers from Turkey, Italy and the UK. In‐depth interviews and an online survey were used to find out the teachers perceptions of game based learning and how this impacts on their roles as a teacher. The research also analyzed the interview findings to understand the dynamics between curriculum design, learning culture and practice when implementing game based learning. The research found that there is a strong link between how learning is designed to incorporate digital games, the theories and strategies that have been used to deliver the curriculum and how this manifests itself in practice within the classroom. The research also showed that teachers are aware that their roles when using new technologies in education has changed, however, because of the lack of necessary training they are not clear on how to adopt these changes. In some countries the curriculum was flexible enough to accommodate game based learning, however, in some without a radical reform this would not be possible. The mass difference between country specific curriculum, pedagogy and practice highlights the need for a flexible model or approach of embedding digital games into primary classrooms. Keywords: game based learning, digital game design, teachers’ perceptions, curriculum, pedagogy, teacher’s role in GBL.

1. Introduction The recent review into educational value of digital games suggests that games facilitate learning and provide opportunities for developing transferrable skills such as; problem solving, critical thinking and collaboration (Allsop 2012; Squire 2003; Kirriemuir and McFarlane 2004), hence the implementation of digital games into primary classrooms is still at the beginning phase. It is evident that many children spend hours playing digital games and researchers continue to investigate the educational potential of learning with this medium, thus teachers are still not fully clear about their role in the GBL environment. This may be due to a lack of education authorities establishing clear policies for both learning with games and game making in education in relation to teacher’s role, or not giving enough time to teachers to get familiar with the mechanics of the digital games. Another reason could be not training teachers in pedagogical knowledge that they need for teaching with digital games, as it transforms the learning space into a dynamic lab, which may require adoption of different teaching strategies and classroom management skills. According to Jessel (2012), “Innovation arising from new technologies makes a variety of demands upon the role of the teacher”. He continues, “At another level, the introduction of innovation makes major demands upon teachers' pedagogical, professional and managerial skills.” Surely, using traditional methods of teaching will not fully support teachers to utilise the potential of learning with digital games. As the new technologies constantly evolve, maybe the focus point should be moved from the instrument itself to developing a model for teachers to learn to evaluate each medium in terms of what can be achieved in practice and which strategies need to be adopted.

Yasemin Allsop, Ebru Yeniman Yildirim and Marina Screpanti

2. Research aims The main goal of this research is to provide a broad review of teachers’ perception on the use of computer games in primary schools in Italy, Turkey and the UK. Additionally, to find out the key factors which impact on teachers’ attitudes to using digital games in teaching. It also aims to explore what works well in supporting teachers to embed digital games into their teaching practice through investigating the interrelation between pedagogy, curriculum and practice.

3. A dynamic alliance: Curriculum, pedagogy and practice Before exploring the interrelation between the dynamic trio; curriculum, pedagogy and practice, we need to understand the term ’pedagogy’ within the context of education. Mortimore (1999) describes pedagogy as ‘any conscious action by one person designed to enhance learning in another’, Hanks et al (1986) refers it as the ‘principles, practice or profession of teaching’. It is clear that pedagogy includes ‘teaching’, ‘learning space’, ‘content’ and ‘methods’. Therefore, pedagogy can be seen as the umbrella concept shaping the strands of curriculum and practice. In other words, how curricula content manifests into knowledge and skills in the classroom, mainly shaped by how it has been taught in practice. The issue is that pedagogical approaches to education are not necessarily detached from cultural traditions and beliefs, thus transforming the education systems to accommodate game based learning is a more complex task than just re‐arranging a classroom space. Pepin (2010) suggests that the cultural traditions and philosophical beliefs of countries determine the principles that national curriculum is designed upon, along with teachers’ pedagogies in schools. Consequently, the content and aim of the curriculum itself, places expectations on teachers. In many education systems where curriculum is designed to evaluate learning through test scores, teachers use pedagogy to serve and meet this purpose rather than focusing on how to develop learning. This does not only limits the teachers methods to lead teaching, but also the notion of meeting different learning needs of students which in most cases results as a failure in education. In England, ICT was placed in the National Curriculum for England (1999) for children from the ages 5‐16 as an individual subject and also as a tool for teaching and learning. Although the position of digital games in education was not defined, the potential of new technologies for developing thinking skills was emphasised. In the new Draft National Curriculum for England, which will be active from 2014, the term ICT replaced by Computing, and the writing, designing and testing programs included as part of the study programme. The focus is to teach pupils how digital systems are designed and programmed, then allowing them to apply this knowledge to solve problems, by designing solutions and creating products. Although it is a pleasing outcome to have programming as part of the curriculum, it places demands on schools, whereby they now need to plan how to deliver these attainment targets as lessons. Certainly game design activities can be used to meet these aims, however in most cases, game design activities were limited to after school clubs, mostly run by enthusiastic volunteer teachers, therefore, preparing the whole school to meet this new phase may present challenges. Including programming in the curriculum, does not guarantee it will be practiced in schools. As mentioned before, having new technologies as part of the curriculum, places demands on teachers, which requires training in both pedagogy and technology. Budget cuts in the UK has impacted on the level of training services provided by Local Education Authorities and City Learning Centres, which were established to offer ICT based learning opportunities for schools and for the wider community. The current review of GBL (Games Based Learning) in the UK shows that although there are many teachers interested in game based learning, the use of digital games in the classroom is seldom. A lack of teacher subject‐knowledge, not enough training opportunities, technical problems, cost, e‐safety concerns and learners not necessarily seeing a link between games and learning are seen as the barriers to the use of digital games in education by teachers (Becta 2010). On the other hand, when we look at the Curriculum for Excellence in Scotland (2009), the scenario is quite different. It clearly states that computer games develop problem solving skills and collaboration. Along with this, the Scottish Centre for Games and Learning provides digital resources to support teachers and the wider school community, which widely encourages the use of games in education. Learning and Teaching Scotland have run many projects, which have focused on the use of commercial games in schools to stimulate and engage learners. It is quite clear that the shared mission between schools, curriculum, researchers and constant communication were key to the successful implementation of digital games in schools in Scotland. One good point about both curriculums is, giving teachers the flexibility to use different methods and

Yasemin Allsop, Ebru Yeniman Yildirim and Marina Screpanti instruments to teach and also the opportunity to be creative about it. This again, presents the importance of the teacher’s role in bringing new technologies into a classroom. In Italy, although games and its benefits are mentioned in the curriculum (la normativa italian dal 2007), this does not necessarily point at digital games. In general, the use of ICT in education to develop the knowledge and skills of pupils in this digital age was included in the curriculum under the Computer Science and Technology title. The role of ICT as a tool for learning was well defined for primary levels. According to the document, by using ICT for learning, pupils will learn to present and share their work; they will also use educational games for communication and collaboration. Teachers in Italy, are very interested in using new technologies including digital games in their teaching, however they don’t feel confident enough to use them. One interesting point is, the inclusion of ICT in initial teacher training in terms of both technical skills and pedagogical approaches. Although there are training opportunities for in service teachers, these are not compulsory. There are no requirements for teachers to have ICT skills. Innovative teachers are described as, those who adopt active learning methodologies and use new digital technologies to meet student’s needs and learning goals. The integration of ICT into cross‐curricular subjects especially for learning languages and basic skills in maths and literacy is still a work in progress. Some teachers suggest that e‐twinning had a great impact on this as it encouraged them to use new technologies for collaborative work. In Turkey, the General Directorate of Educational Technologies department, which is part of the Education Ministry, is responsible for the implementation of new technologies in education for the whole country. As part of the FATIH Project, the Turkish government provided schools with the main technical equipment such as; computers, Internet access, IWB and tablets. Although ICT is included in the curriculum, which is prepared at a national level, a recent review showed that it is not successfully integrated into the curriculum in schools. ICT is being taught as a specific subject in primary schools and as an optional subject at secondary level. Basic IT skills are being taught at universities, as part of teacher training and a basic IT skills certificate is a requirement for gaining a teaching qualification. In‐service training for current teachers is provided constantly, however this is usually aimed at technical knowledge, rather than pedagogies and the cross‐curricular use of technology. In Turkey, where the evaluation of learning is based on test scores and the content of the lessons, provided by the central education authority, there is almost no flexibility for teachers to try using different tools and strategies in their teaching. Teachers don’t just have to understand the changes that new technologies bring upon them, but also the demands of constant educational reforms by the Education Ministry. Although there are a number of universities working along with the Education ministry and supporting them in making policies, teachers and learners are not necessarily included in the communication circle, which may impact on their attitude to new technology. Investing heavily in technology, but not in training teachers to use new technologies including digital games and game design in education purposefully, can be noted as the main reason for teachers not using digital games in teaching.

4. Literature review The recent literacy review of teachers’ perceptions of game based learning, digital technologies in general presents diverse results. Gaffney (2010) explored the factors and design principals in technology adoption. He concluded, “Actions of governments, education authorities, schools, teachers and students are aligned and integrated through the implementation process to increase teacher use of such resources for the benefit of students”. This points to the importance of collaboration and communication between the stakeholders of education. We would add the universities and teacher training institutes into this list too, as they also play an important role in not only providing teacher candidates with training on current practices, but also developing the theoretical framework which involves teachers, learners and policy makers. Egenfeldt‐Nielsen (2011) used an online teacher survey to find out about game based learning in schools. The participants were from Denmark, Finland, Norway, Portugal and the USA; Denmark having the highest number with 185 participants of the 275 respondents in total. The results were very similar for each country, where most of the teachers were having a little or average experience with games. 60% of the teachers used computer games in education with these computer games providing some variety in teaching, increasing learner’s levels of engagement and opportunities for differentiating teaching being seen as the main reasons for using games in education. Technical issues such as cost of games, lack of teacher’s subject knowledge, problems with evaluating learning were listed as the foremost barriers towards teaching with games.

Yasemin Allsop, Ebru Yeniman Yildirim and Marina Screpanti Razak et al (2012) investigated the teachers’ views on the approach of digital games‐based learning within the curriculum for excellence, in primary schools across Renfrewshire in Scotland. They used an online survey and 49 primary schools were included in the research. They had 62 responses from teachers. 100% of the teachers were female with a mean age of 32.8. They also interviewed teachers who had been identified from the survey. 39% of the teachers stated that they used only computer games, whereas 50% noted that they never used computer games or game creation tools. 3% of the teachers cited that they used only game creation tools and 8% said that they used both computer games and game making tools. Most of the games that were used by the teachers were categorised as free online maths and language games. As a conclusion the research showed that digital game based learning, game design in specific was not widely used in primary schools in Renfrewshire, Scotland. The Teaching with Games report by Futurelab (2013) investigated the use of commercial off–the‐ shelf computer games in formal education both in the UK and the International arena through literature reviews from 2006 and onwards. The study found that the motivational power of games was the main reason for teachers to use games in class. Although 59% of all teachers stated that they would use games in the future, 37% noted that they wouldn’t use games in education because games had little or no educational value. The barriers to the use of games in schools were; issues with equipment, difficulty in assessing learning, not having games relevant to the curriculum and games having no educational value. One interesting finding of this report was that 72% of teachers never played games outside of the school environment.

5. Methodology Although a qualitative method was adopted for the preliminary data, this study employs both quantitative and qualitative approaches including; a teacher survey and in‐depth interviews to understand the context of the teachers’ beliefs about game based learning.

6. Ethics In qualitative research, informed consent needs to be sought and may be withdrawn at any time, and it is also important to include direct talk regarding the continued willingness to participate (Cassell, 1982). The participants were given information about the aims of this research project and were aware that they may withdraw from the project any time.

7. Data analysis Online Survey An online survey containing 10 closed questions was used for the data collection. The survey link was shared on social networks and professional groups in the UK and Italy. The link for the survey in Turkish was sent to the Education Directorate in various cities in Turkey, which was placed on their intranet with local schools. There were 46 responses from the UK, 49 from Turkey and 43 from Italy, totalling 138 altogether. 42% of the teachers were aged between 31‐40, 31% were below 30, 15% between 41‐50 and 12% were between 51‐60. The gender distribution was; 66% female and 34% male. When we looked at the teachers’ personal experience of game playing, the UK teachers had more experience than others (89%), whereas 43% of Turkish teachers had no experience of gaming. Experience of teaching with games also showed a similar result, where only 11% of UK teachers had no experience of teaching with games; this was followed by 39% of Italian teachers and 43% of Turkish teachers. However when asked about their experience of teaching game design, all three countries had similar results, almost 70% of UK, Turkish and Italian teachers stated as they had no experience of teaching game design. Digital games were mostly used for teaching Mathematics, literacy (English, Turkish and Italian), science and languages. A majority of the teachers in all three countries cited that they would use digital games for teaching in the future, around 10% were undecided and about 12% of Italian and Turkish teachers said they would not use digital games for teaching. Figure 1 presents the most popular reasons for using digital games in the classroom. Games have motivational power, games improve learning (In a specific subject; maths, literacy, language), encourages creativity,

Yasemin Allsop, Ebru Yeniman Yildirim and Marina Screpanti develops problem solving/ critical thinking skills, provides opportunities for collaborative working were listed as the main reasons by teachers.

Figure 1: Reasons for considering using digital games in the classroom for educational purposes When asked about the barriers to the use of digital games in the class (Figure 2), again the result was very similar in all three countries. Access to equipment in the classroom, teachers subject knowledge, relevance to the curriculum, the lack of schools ICT capability were selected by most teachers. Interestingly, only around 8% of the teachers saw evaluating learning as a barrier. One interesting outcome was the way that teachers saw their role in teaching with games and teaching game design. Where 65% of the teachers stated that class teachers should teach game playing, 60% of the total teachers said a specialist teacher should teach game design. This may be related to a lack of subject knowledge or not having enough experience with game design. It would be interesting to see how this would change in the UK, as the new Computing curriculum will be based on programming.

Figure 2: Barriers to use digital games for educational purposes in the classroom enviromentr

Yasemin Allsop, Ebru Yeniman Yildirim and Marina Screpanti In‐ depth Interviews Face to face In‐depth interviews were carried out with five teachers from each country. One of the research team members carried out 3 of the 5 interviews in the school where she works. She has been teaching game design in her school. Some of the teachers interviewed had had opportunities to watch her teaching on many occasions. The interview data was grouped into modes of personal experience of gaming, teaching with digital games, the advantages of GBL, barriers to GBL, impacts on learning and the teachers’ role. Figure 3 shows these modes. The data analysis presents that teachers’ personal experience with gaming differs in each country, where UK teachers talk about playing with games from early ages, teachers from Turkey had little or no experience at all. Boredom was the main reason for many teachers playing digital games. In all three countries digital games were used for teaching the main curriculum subjects; maths, literacy, science and languages. Motivation, differentiation, fun were listed as the main advantages of GBL. Learning without realising was also mentioned by teachers from their countries. When discussing the barriers to GBL, again teachers from all three countries talked about a lack of subject knowledge, not enough training, difficulties in the classroom and behaviour management were quoted. Teachers cited many positive and negative impacts of GBL on pupils learning. Where some teachers suggested that GBL develops problem solving, creativity, collaborative skills, thinking skills, visual‐spatial skills, some shared their worries of not meeting the learning objective, technology killing the creativity and developing shallow skills rather than in‐depth ones. One teacher from the UK was quoted as “I think, it should support teacher led teaching rather than replacing it; It can become a lazy way of teaching if you are not careful. It needs to be targeted carefully. I am worried that it may take over traditional methods of children researching things, using books and reading. Rather than, sometimes, this not so much games as much, but sometimes they read on the Internet, they read just a snip of it, but when they use a book, they read more. You know, on the Internet, everything is so easy. There is a strong argument that, children should be taught traditional ways of researching. And they do copy. There is no way of checking that, they go to child friendly sites”. One interesting point was that the teachers made links between how learning manifests in the classroom and how digital games are taught. In other words, their awareness of the relationship between the pedagogy and practice. “It is hard to say how digital games would impact learning. Outcome would very much depend on the person teaching it and their teaching approach. Because in my experience, it is really easy to get lost and carried away with digital games in the classroom. Kids can be / get absorbed, just a fact that it is quite a fun and dynamic lesson, and not actually taking learning objectives from it. If it is structured in a coherent way and appropriate strategies use then it can be very useful”. Many teachers from all three countries mentioned the ‘learning without realising’ mode. Teachers thought that this had a very positive impact on children’s attitudes to lessons as it made schools look like a less formal place. They quoted: “….. it doesn’t seem like you are learning, like school, it is not like write this down, copy this down. They enjoy it. It is entertaining. I don’t think they see it as learning, where we know that even though you are playing that game, you are learning how to do that maths activity, where they think they are just playing games. They think we have been nice”. “…..It keeps them interested, they all want to come and touch the IWB, it makes them pay attention, they are learning but they don’t even realize they are learning”.

Yasemin Allsop, Ebru Yeniman Yildirim and Marina Screpanti

Italy

Turkey

Games Played

Commodore Amiga Game boy PlayStation Nintendo Puzzles Quizzes Logic games Super Mario Games on Mobile phone Guitar Hero Tablet Boredom

Spectrum Console Monkey Island Civilization

Playing with kids Online gaming Logic games Crosswords Boredom

Reasons for playing games

Teaching with digital games

Advantages of Teaching with digital games

Maths Mental‐oral starter Plenary IWB Reward Tutorial Visual Repetition Differentiation Independent Increased confident Engagement Learning without realizing Reinforce Assessment Fun Entertaining

Barriers to teaching using digital games

Fitting it in Meeting Learning Objectives Distraction for learning Kids off task Recording Kids over excited Managing kids

Impacts of GBL on children’s learning

Not creative Shallow skills Not meeting learning objectives Thinking for themselves

Teachers’ role

Facilitator Lead children Questioning Lazy way of teaching Training needs

Generational divide Stand back more Observe more Director of a lesson‐ not controller Subject knowledge Experience

Figure 3: In‐depth interview analysis

Leisure time Boredom Professional Developing brain power purposes Playing with own kids Curiosity Maths Sometimes English Usually at the end of a Science lesson Brain game Maths History English Geography Revision Motivation Motivation Learning by doing Interested Fun Active learners Differentiation Inclusive Collaboration Fun Technical skills Entertaining Learning fast Learning without realizing Stimulating Imagination Instant feedback Challenging Digital divide Time Lack of teachers Technical equipment understanding Lack of teachers knowledge Teachers attitude Preparation Parents attitude Fitting in Lack of technological equipment Fitting in Logical skills Problem solving Memory Cognitive skills Emotional skills Thinking skills Social skills Visual‐spatial skills Creativity Creativity Closing digital divide Addictive ICT skills Reasoning Critical thinking Attitude to school New strategies Involving all learners Teaching differently Active learning Moving around Student centered learning Action in teaching More training

Yasemin Allsop, Ebru Yeniman Yildirim and Marina Screpanti One interesting finding was the teachers’ knowledge and understanding of their role and the new strategies to use when teaching with games and teaching game design. Terms such as ’facilitator’, ‘teaching differently’, ‘active learning’, and ‘stand back more’ were widely used. The teachers were aware that they needed to stand back more and act as a coach or a guide in the classroom. They also discussed the importance of questioning rather than just showing learners what to do. One teacher who watched one of the members of the research team teaching game design was quoted as saying; “From what I can gather, watching others and you, it seems like you are much more of a director of the lesson more than a controller if that makes sense. You are pushing the kids towards the right direction; you gave them license to run as far as they can, if they get it wrong or they need support you sort of wheel them in. It seems to me you give kids more license to learn, which I think is a very positive thing to do”. This also emphasises the importance of team teaching or teachers having an opportunity to watch others teaching with games or game design. This is especially useful when modelling how pedagogy is put in practice through teaching strategies in the classroom. Teachers see their new role as more active and interactive when teaching with games or game design. “ … Because teaching with games means teaching in a totally different way. No teacher sat at a teachers desk and pupils listening, but movement, action in teaching”, stated a teacher from Italy. Many Turkish teachers talked about how they use digital games to change students’ attitudes towards school by making learning fun. One teacher in particular mentioned that using games in teaching changes the way learners perceive the teachers. The teacher was quoted as saying: “It changes the way students see you, you can be a hero in their eyes’”. Another teacher stated that digital games are like a second language to kids. Furthermore it was added that using games in teaching allows teachers to re‐shape their communication with the learners, which then impacts on their behaviour and attitude to learning as a whole. Teachers in all three countries had a similar view almost in all the topics. Only one teacher talked about the negative impacts of teaching with games on the teacher rather than the learner.

8. Discussion and conclusion Our research data indicates that teachers are interested in teaching with digital games and most of them see digital games as an effective educational tool. Their use of games in teaching varied in each country and even between the teachers within the same country. Not having a clear framework on GBL within the curriculum to guide them in the classroom, lack of subject knowledge and not knowing how to adopt new pedagogical approaches stopped them from using games in teaching and it also impacted on their view of teaching with games. Most of the time digital games were seen as a reward or a tool with which to achieve a specific curriculum target in a specific subject. Although some teachers mentioned the impacts of games on developing transferable skills such as; problem solving, critical thinking, collaboration and creativity, there is no indication that the teachers know how to design the GBL space to achieve this. It can be suggested that teachers need to be trained in the best methods of teaching with games through a simplified pedagogy. The reason we use the term ‘simplified pedagogy’ is because trying to teach teachers detailed theories of GBL is time consuming and is not always useful, unless they are modelled in the practice. Therefore training them in pedagogy using practical strategies such as; questioning, classroom and behaviour management, classroom design as a learning space, and planning lessons using GBL can be more effective. Many teachers were worried about behaviour management and monitoring children when using digital games for teaching. It is evident that GBL transforms the way a classroom is arranged as a learning space physically and also the way teachers manage it. Surely if students were told to be quiet, it is not realistic to expect them to develop any communication or collaborative skills. This can also have an impact on how students perceive learning with games. It can be fun or boring, depending on how it has been used. Another important finding was that in all three of these countries, it appears that the link between the policy makers, research institutes and schools seems to be either unclear or doesn’t exist. Although there are organisations providing reports on game based learning in schools and presenting the issues to focus on, this

Yasemin Allsop, Ebru Yeniman Yildirim and Marina Screpanti message is not necessarily reaching out to the schools and more importantly the teachers or policy makers. One interesting point is that teacher training, both in‐service and initial teacher education in terms of using new technologies including GBL, and pedagogies to support these, has not been seen as a priority in any country mentioned here. In other words teachers were left alone to work out how to teach with games. This is a worrying outcome, as the teachers’ attitude to digital games is negative and if they see digital games with no educational value, they will never include it in their teaching. One teacher described this as ‘enlarging the generational divide’. Another valuable facet is how country specific curriculum design affects the way teachers use digital games in practice. The national policies on the use of digital games in education differ considerably not only from country to country but also from school to school, where the decision is left to school managers or in many cases the teachers individually. Interestingly in some countries although technology is available, game based learning still did not made its way into the classroom. Therefore it will be useful to focus on developing flexible models of GBL spaces. In this study we aimed to find out about teachers beliefs of game based learning and how this related to curriculum, pedagogy and their practice. Where the curriculum, content of lessons was developed by policy makers and theories discussed by researchers, it is inevitable that teachers are confused about their direction in this cycle. It is vital that research institutes focusing on GBL, establish a clear and constant two‐way communication with teachers to develop GBL in primary schools. They should not only aim to train teachers but also start listening to them, which will feed into their research. This will give an active role to teachers in developing GBL theory and practice, which may help them to keep up‐to‐date with the latest findings in the area. In a similar approach, policy makers should have clear aims and instructions on integrating digital games into classroom. They should involve the research institutes and also teachers from the classrooms directly when writing policies on GBL. We don’t think that having research institutes views on teachers’ perception will provide an in depth account of teachers understanding and practice of teaching with games. Finally in order to support teachers in various countries for developing GBL practice, further comparative studies are required and more often. This will provide us with a very valuable result; the change in teachers thinking on GBL over time.

References Allsop, Y (2012) “Exploring the Educational Value of Children's Game Authoring Practices: A Primary School Case Study”, Proceedings of the European Conference on Games Based Learning, p21. Becta (2010) “Harnessing Technology School Survey: 2010”, [Online] http://dera.ioe.ac.uk/1544/1/ becta_2010_htss_report.pdf Cassell, J. (1982) “Harms, benefits wrongs and rights in fieldwork”. In J. Seiber (Ed.), The Ethics of Social research: Fieldwork, Regulation and Publication, Springer verlag, New York. DfES (1999) The National Curriculum for England and Wales, DfES, London. DfE ( 2013) “Computing Programmes of Study Key Stages 1‐4” [online], http://media.education.gov.uk/assets/files/pdf/c/computing%2004‐02‐13_001.pdf Egenfeldt‐Nielsen, S. (2011) “International survey of the experience and perceptions of teachers” in Egenfeldt‐Nielsen, S., Sørensen, B. H. and Meyer, B. (eds.) Serious Games in Education – a Global Perspective, Aarhus University Press: Aarhus Fromme, J. (2003) “Computer games as a part of children’s culture”, The International Journal of Computer Game Research, 3(1). Gaffney, M. (2010) “Enhancing teachers' take‐up of digital content: Factors and design principles in technology adoption”, [online], http://www.ndlrn.edu.au/verve/_resources/Enhancing_Teacher_Takeup_of_Digital_Content_Report.PDF Gee, J.P. (2003) “From video games, learning about learning”, [online] http://chronicle.com/article/From‐Video‐Games‐ Learning/8730/ Hanks, P., McLeod, W. and Urdang, L. (Eds)(1986) Collins dictionary of the English language, Collins, London & Glasgow. Jessel, J. (2012) “Social, cultural and cognitive processes and new technologies in education” in Miglino, O., Nigrelli, M. L., & Sica, L. S. Role‐games, computer simulations, robots and augmented reality as new learning technologies: A guide for teacher educators and trainers, Liguori Editore, Napoli. Kirriemuir, J., and McFarlane, A. E. (2004) “Literature review in games”, [online], http://archive.futurelab.org.uk/resources/documents/lit_reviews/Games_Review.pdf Mortimore, P. (Ed.) (1999) Understanding Pedagogy and its Impact on Learning, Paul Chapman Publishing, London Oblinger, D. (2004) “The Next Generation of Educational Engagement”. Journal of Interactive Media in Education, 2004 (8) Squire, K. (2003) “Video games in education”, International Journal of Intelligent Simulations and Gaming, 2(1), 49‐62.

Yasemin Allsop, Ebru Yeniman Yildirim and Marina Screpanti Pepin, B. (2010) “How educational systems and cultures mediate teacher knowledge: teacher 'listening' in English, French and German classrooms” (p. 119‐138), in Ruthven, K. and Rowlands, T. (eds) Mathematical knowledge in teaching, Springer, Dordrecht. Perrotta, C., Featherstone, G., Aston, H. and Houghton, E. (2013) “Game‐based Learning: Latest Evidence and Future Directions” NFER (FUTURELAB), Slough, [online], http://www.nfer.ac.uk/nfer/publications/GAME01/GAME01.pdf Prensky, M. (2001) “Digital natives, digital immigrants”, [online], http://www.marcprensky.com/writing/prensky%20‐ %20digital%20natives,%20digital%20immigrants%20‐%20part1.pdf Razak, A. A., Connolly, T. M., Baxter, G. J., Hainey, T., Wilson, A. (2012). "The Use of Games‐based Learning at Primary Education Level within the Curriculum for Excellence: A Combined Result of Two Regional Teacher Surveys", VI European Conference on Games‐based Learning (ECGBL), Cork, Ireland, October.

Using Gamification to Animate a Virtual Community António Andrade and Carlos Vaz de Carvalho GILT ‐ Graphics, Interaction and Learning Technologies, ISEP, Porto, Portugal 1110030@isep.ipp.pt cmc@isep.ipp.pt Abstract: A Community of Practice (CoP) is a group of individuals who willingly come together because they have common interests in a specific area and want to develop their skills and competences by collaborating with other members and sharing their experience. CoPs have been applied to diverse environments, including organizations, education, associations and the social sector, as well as the governmental institutions and international development. A community of practice may emerge from both bottom‐up or top‐down initiatives and it will exist as long as the members believe they have something to contribute to it, or gain from it. The widespread access to personal computers and to the internet made the virtual environment very interesting for communities of practice, providing collaboration tools and both synchronous and asynchronous forms of communication. However, some differences and drawbacks in the interaction between individuals are also registered in the translation to the virtual environment. Keeping these communities alive depends a lot on the motivation of members, on their commitment and will to participate. Those are not easy to attain and maintain, especially on the online environment. Different approaches have been implemented to animate a community (be it real or virtual) and ensure a high level of communication and experience sharing. Of course, providing a rich and valuable knowledge repository is crucial, but the question is, how to turn passive community members into active participants? In this paper we present three communities of practice with completely different domains and communities (footwear industry professionals, trainers and trainees; innovation and marketing students, teachers and experts; and serious games practitioners and researchers). We compare the approaches that were used to animate the individuals to participate and to be involved in those communities. In particular, we present the results of using gamification as a form of motivating participants. Keywords: communities of practice, gamification, serious games, collaboration, participation, motivation

1. Background A Community of Practice (CoP) is a virtually unified or physically collocated body of individuals who willingly come together with or for a common set of traits or interests that revolve around a specific topic or set of topics, and who wish to learn about or to help evolve and mature such interests through collaborative efforts. In short, it is a group of people who share a common interest in a particular domain or area and created the community with the specific goal of gaining knowledge related to that field through the process of sharing information and experience within the group (Wenger, 1998). As such, a community of practice is often organically created, bottom‐up, with as many objectives as members of that community, although institutionalized, top‐down, approaches are also possible. One of the most famous examples of such community happened within the Xerox company, where co‐workers spontaneously organized a kind of internal support group and knowledge base, making common problem solving easier for other colleagues (Orr, 1986). Eventually the company saw the value of such structure and created the Eureka project (Brown and Duguid, 2000), a formal company approach to generalize this model, in fact recognizing its validity. CoPs have been applied to diverse environments, including organizations, education, associations and the social sector, as well as the governmental institutions or for the international development. Typical activities engaged inside a community of practice relate to problem solving; information request; experience seeking; asset reuse; coordination and synergy; development discussion and knowledge mapping (Wenger, 2006). The structural characteristics of a community of practice are (Wenger, McDermott and Snyder, 2002):

Domain: the domain of knowledge is the common ground that gathers members and community activities. However it is important to remember that each member will have a different level of expertise on the domain, spawning from the amateur to the specialist;

Community: the community refers not only the isolated members’ characteristics but mostly the fabric of relationships and interaction norms established between them in the context of the community of practice. Of course, a strong sense of community or belonging will foster interaction.

António Andrade and Carlos Vaz de Carvalho

Practice: the practice of a community represents the amassed and shared products and activities in the specific domain, i.e., the core knowledge produced by interaction between members. Earlier Wenger (1998) would define this as a shared repertoire.

The requirements for a healthy community of practice can be grouped under three main topics:

Social presence: the management of a community of practice often faces many barriers that inhibit individuals from engaging in knowledge exchange. Some of the reasons for these barriers are egos and personal attacks, overwhelming CoPs, and time constraints (Wasko and Faraj, 2000). Thus, communicating with others within a community of practice involves creating social presence (Tu, 2002);

Motivation: the will to share knowledge is critical to the success of a CoP (Ardichvili, Page and Wentling, 2002);

Collaboration: collaboration is essential to ensure that communities of practice thrive. More seasoned colleagues and a higher educational level tend to foster a more collaborative culture (Simon and Sveiby, 2002).

A community of practice can exist as long as the members believe they have something to contribute to it, or gain from it. CoP membership changes and members may take on new roles within the community as interests and needs arise. CoPs are social structures which widely promote informal learning. Informal learning is recognized today as a fundamental part of an individual’s learning path. The term is an umbrella for all activities that somehow lead to the acquisition of knowledge and competences but happening outside schools or other training institutions (which are referred to as the formal system) (European Union, 2012). The range of informal learning contexts is considerable and can be broken into different levels of learner awareness. Learning by socialization, for instance, is obviously not as intentional and systematic as self‐driven learning (Shugurensky, 2000). The widespread access to the internet popularized a new paradigm, often referred to as the web 2.0. Amongst other things the web 2.0 is characterized by being given more emphasis to the relationships established between users and being given focus on the diversity of content generated by each user. To support this, there was also a technical shift in the background creating new collaboration tools (such as wikis) (Leino and Ovaska, 2008). This evolution in the information technology field made it very interesting for the creation and rooting of Virtual Communities of Practice (Lai et al., 2006). Since Virtual Communities of Practice (VCoP) add to traditional co‐located CoPs the ease of asynchronous interaction (Gray, 2004), members are no longer restricted to being in the same physical space or even time zone to engage in common activities, reducing, for instance, the sense of isolation in professionals who work alone (Wenger, White and Smith, 2009). Furthermore, the new tools available to use through the internet allow not only for easier organization and consumption of the shared repository but also enable easier collaboration. Being dependent on technological support, VCoPs tend to result from top‐down approaches (due to maintenance and development requirements) and are generally prone to a slower development pace (Lai et al., 2006). Although increasingly less, this also means that access can be conditioned to members with difficulties finding hardware or network resources or who are uncomfortable with their computer skills (Gray, 2004). Finally, it is also important to acknowledge that online communities do not provide the richness of identitary hints we find in face‐to‐face interaction. Personality traits are often emphasized or attenuated, which may also lead to online newcomers’ insecurity. In this paper we present three communities of practice, in completely different domains (footwear industry professionals, trainers and trainees; innovation and marketing students, teachers and experts; and serious games practitioners and researchers). We compare the approaches that were used to animate the individuals to participate and to be involved in those communities. In particular, we present the results of using gamification as a form of motivating participants.

António Andrade and Carlos Vaz de Carvalho

2. Communities of practice The subject of this ongoing research is deploying and maintaining platforms for three European Lifelong Learning Program projects’ VCoPs. Having distinct targets and goals, these communities also demonstrate the diversity of applications for VCoPs which we will present below. Technically, these communities are supported by open source software that was expanded to fit the community requirements. Thus, Elgg 1 has been our framework of choice for all the social aspects of the VCoPs, integrating it with the Moodle 2 learning management system when a more formal structure is needed.

2.1 TIED Shoe The TIED Shoe project is a Transfer and Innovation LEONARDO project which aims to create a virtual training center for the development of the footwear industry. Its main goals are to (TIED Shoe, 2012):

provide a training center to share the best practices in footwear design;

improve and upgrade competences and skills of VET (vocational education and training) colleges and schools;

extend the common educational qualifications and accreditation of skills and knowledge for professionals in the footwear industry.

The TIED Shoe VCoP was built in two layers. The first should welcome every interested participant, keeping the interaction open, while the second layer should support e‐learning courses. Ideally, transition between layers should be seamless, allowing discussion to spawn from one to the other. The first layer, an open social portal, provides tools for spontaneous publication and sharing: blogs and microblogs, the ability to upload file attachments and organize website bookmarks as well as the ability to create groups and group discussions. Networking between members is supported by customizable profiles as well as the ability to “friend” other users and private message. Public discussion in the form of comments on submitted contents is also provided. The second layer is intended for assigned students only, supporting e‐learning courses (“New Design Tools”, “Materials”, “Innovation”, “Internationalization” and “Entrepreneurship”). A different system – Moodle – was employed here but the transition from the social portal to the e‐learning platform and back was made seamless by the implementation of a single sign‐on system.

2.2 EMIC The European Marketing and Innovation Centers (EMIC) is another European Lifelong Learning Project (LLP). It targets entrepreneurs and marketers (also students or innovation professionals) and, in practice, its general idea is to create a network of national MIC (Marketing & Innovation Centers). EMIC specific goals are to (EMIC, 2012):

create an observatory for national good practices;

good practice implementation in companies;

set up new forms of training for Marketing & Innovation;

support students in the translation to the professional world;

support student research.

Similarly to the TIED Shoe VCoP, two seamless layers were implemented: one for the main social portal and another for the private e‐learning courses. However, the student support objective required the ability to relate enterprises and students looking for jobs or internships. Thus we expanded the social portal to include these two new roles: ‘student’ and ‘enterprise’. The attribution of these roles is manually moderated. 1

https://elgg.org/ https://moodle.org/

António Andrade and Carlos Vaz de Carvalho As a user with a ‘student’ role, one is able to manage a new private profile section which contains a resume and a file upload field to attach a complete curriculum. As a ‘student’, one can also browse the placement proposals submitted by ‘enterprise’ users and register his interest on the placement. Notice that, by design, the resume and curriculum section of a ‘student’s profile are only visible by ‘enterprises’ to which the ‘student’ has registered interest. As a user with an ‘enterprise’ role, one is able to create ‘placement’ objects. These can be job or internship openings and are only visible to ‘student’ users. Once ‘students’ submit their interest to one of these placements, the corresponding ‘enterprise’ user is able to browse the list of interested users, as well as consult their curriculum. From this section, the ‘enterprise’ can contact the ‘student’ by private message, being the final selection process left open. Finally, local MICs were also provided with a tool which enables them to design and carry out online surveys. This allowed them to easily identify local practices and trends, as planned for the project’s observatory. To this 3 end another open source tool was coupled to the VCoP: the LimeSurvey system.

2.3 SEGAN Games are believed to be a potential learning medium. Their enjoyable context and interactivity enhance retention, cooperation and competition skills, strengthen social competences and the fun factor can continuously feed motivation into the learning process (Vaz de Carvalho and Fernandez‐Manjon, 2013). However, looking at the current impact that can be observed from the use of games and simulations as informal medium or optional course support material, serious games have not been explored to their full potential. The Serious Game Network, or SEGAN, is a CoP funded in the scope of the LLP program with the intent of systematizing “the European approaches to serious games, combining theory, research and practice in a way that promotes Europe as the leader in this field” (SEGAN, 2012). In fact, more than half hundred projects funded by the European Commission under this thematic have been realized to date and this convergence seems now fundamental to increase the awareness of the benefits and impacts of serious games. The SEGAN community is mainly composed of academic researchers, game producers and Serious Games (SG) users but it is also open to any individual interested in the domain of SG and their implementation. SEGAN practice consists of the compilation of serious games resources in the online platform supporting the community as well as the open discussion of related topics, working towards annual publications on the design, development, delivery and evaluation of serious games. Face‐to‐face events are also part of the activities, namely an annual conference and summer school. Monthly open webinars are another important part of the community practice. At this stage, the SEGAN VCoP portal focuses on allowing the discussion between members as well as the sharing and categorization of resources (game and tool bookmarks, video and blog posts, events, etc). Of course, the social features we have seen on both previous platforms are also present on SEGAN.

3. Motivation and gamification The main issue we identify at this point in the VCoPs we are developing is a general lack of regular and spontaneous participation throughout the community fabric. Part of this is naturally due to their young development stage. However, we sense it is also correlated with a more general motivation issue. According to a 2009 research, 43% of the European internet users were then considered as “non‐participative” (Riu and Jokisalo, 2009). The golden rule to attract new members to a VCoP is providing them value (Lai et al., 2006). Assuming this is being accomplished (by at least the core members), other ways to promote the dynamics and the activity of the community are required to ensure it becomes alive and stays that way. 3

https://www.limesurvey.org/

António Andrade and Carlos Vaz de Carvalho At this point it seems important to analyze what can constitute motivation to participate. Trevor Moore identified some motivational categories for the participation in VCoPs: Table 1: Motivational categories and their correlating expressions (Moore, 2007) Motivational Category Altruism

Correlating Expressions used by Past Researchers benevolence, charity, concern for community, public duty, social support

Belonging Collaboration Egoism

an attempt to combat loneliness, taking pleasure in sense of community the assisted articulation of ideas, development of insight, refined thinking personal gain, generation of employment, portfolio‐building

Egotism Emotional Support Empathy

attention‐getting, bragging rights, peer recognition an emotional connection compassion, understanding, a willingness to selflessly help others

Knowledge Power Reciprocity

seeking information, self‐efficacy influence, ownership moral obligation, fairness

Reputation Self‐Esteem Self‐Expression

social standing, status respect, positive reinforcement, esteem support catharsis, expression of emotion, offering opinions

Wisdom

learning, challenge, creative thinking

Games are another interaction experience greatly dependent on motivation. However, they obviously seem to get a more spontaneous adoption. In fact, they demonstrate a motivational category that Moore did not include in his survey: fun (Prensky, 2002). However, there’s more to games than fun: to Csíkszentmihályi (1990) games produce a flow, that is, a mental state of completely focused motivation. Thus, we felt challenged to experiment whether one could rely on community gamification as a way to foster and maintain activity. One of the most common gamification definitions explains it as the process of applying game mechanics to an interface as means to engage users (Zichermann and Cunningham, 2011). This may consist of defining explicit motivational elements such as user points, levels and leaderboards, achievements and badges (Werbach and Hunter, 2012), or yet virtual currencies redeemable for goods or perks. The most common critique to this process points the risk of turning game‐like interaction into an end in itself which creates no implicit motivation, hence undermining content quality and missing out on the experiential and storytelling dimension of a product. In other words, it interprets rewards in a strictly behaviorist way (Deterding, 2010). In fact, helping to make sense of a non‐game context, by overcoming this issue, is probably the main objective for gamification (Nicholson, 2012). Another significant critique arose inside the SEGAN community pointing leaderboards as inhibiters of newcomers’ participation. For Zichermann (2012), creating social context is crucial when gamifying a system, producing opportunities for users “to engage with and make new friends”. Of course, team play, collaborative action and altruism, as well as unexpected or non‐traditional socializing, may also serve this goal. Despite some criticism and skepticism, company success stories abound (Zichermann, 2011) (2013) and VCoPs 4 such as StackExchange have clearly benefited from such process before. The main question seems to boil down to the way gamification is implemented in the community. Different frameworks have been developed for the design of a gamified system. Werbach and Hunter (2012), Marczewski (2012) and Duffy (2012) all suggest a sequence of questions the designer should follow to get directions. Chou (2013) proposes another framework based on what he calls the Octalysis, a chart of eight axes where core drives are related to game mechanics. Firstly, it is important to know the system to be gamified and who its audience is. Next we need to define goals: what user behaviors do we want to induce and what actions indicate success? For instance, we could 4

http://stackexchange.com/

António Andrade and Carlos Vaz de Carvalho focus on content quality over quantity. Rewards should of course be defined according to this priority. This is probably the most important step, forcing the designer to have a clear idea of the gamification target. The designer then has to find out which mechanics to implement, which extrinsic and intrinsic motivational elements to create and how will the user receive feedback on its actions. Finally, selecting the criteria for analytics and monitoring that data is important not only for user rewards but to continuously assess and/or validate the implemented strategy. Stressing the importance of both knowing the audience and carefully selecting gamification elements, Duffy (2012) suggests that using attainable achievements as alternative to cumulative user points may be friendlier to newcomers. The author also recommends awarding MVP (Most Valuable Player) status to a select number of users: those will act as role models and, using special perks, may help curate and shape the community.

4. The SEGAN experiment The SEGAN VCoP started off as a Facebook 5 group. This had a few advantages such as the ease of use, the integration with such a popular leisure channel and the tendency for viral membership. Overall, this made participation in the group a natural flow in member’s daily routines. However, using Facebook as the basis of the community also had negative implications which could harm its sustainability in the long term. On the top of the list were: the lack of administrative and organizational control; issues related to content ownership; the complete lack of independency from a community identitary point of view; and, most of all, the single and dynamic flow of information which lead to members missing some important information. Once the self‐hosted community platform was ready, the migration was made in a way where users would see their previously created contents mapped to the new platform. Despite this, only part of the users decided to follow the community to the new platform and, with the exception of days surrounding webinars and meetings, the visits and participation dropped. This seemed to be the best timing to apply some gamification aspects to the SEGAN community. Many of the common gamification techniques seemed to make sense in SEGAN’s case and would potentially inject some motivating fun factor. Content quality was a priority to SEGAN but before that it was important to engage members. Since trying not to overwhelm newcomers was another important factor taken into account, it was decided to use both badges and user experience point (XP) leaderboards. As a general rule, XP tends to value quantity (and long term engagement) while achievement badges value quality. Visit recency and frequency are commonly accepted engagement measurements (Zichermann and Cunningham, 2011). Thus for each day the user logs into the platform he gets 1 XP. However, if a user manages to log in for thirty consecutive days, he/she will get the “Enthusiast” badge and the respective XP prize. Contributing contents is also rewarded. A blog post is awarded 10 XP, the double of the prize for creating a bookmark, which generally does not create much value by itself. Later we understood that interesting, ongoing discussions are probably one of the best ways to keep members coming back to a topic or blog post. As such, comments should be rewarded at least as much as the original topic. If a user happens to create a blog post which receives more than twenty five comments she would be awarded the “Debate Starter” badge. Ideally this duality between achievements and XP allows for multiple ways to feel recognition inside the community. Content quality is provided by peer review in the simple form of up and down votes. Again, both the voter and the content author are awarded XP and eligible for specific achievement badges. Perks for MVPs and prizes are also planned to be implemented. The data resulting from the community gamification is used throughout the platform to expose interesting users and content. The leaderboard (which stresses monthly gain over all time totals) and badges pages allow to assess overall community performance. Each user’s profile is also enhanced with their total XP and badge 5

https://www.facebook.com/

António Andrade and Carlos Vaz de Carvalho listing. The “Top Rated Content” and “Top Influencers” (users who created most textual content, such as blogs and comments) blocks are also products of this process.

5. Conclusions and further work From the initial gamifying process some interesting ideas rose up. We learned that despite our focus on content creation while setting the point awarding rules, to foster interaction between members through gamification one should balance not only the points awarded between the topic creator and the commentators but the quality of the discussion, as perceived by the community (e.g. using votes). Some users reacted quite strongly to this process, arguing that leaderboards presence was too overwhelming. Others tried to push the game and make it to the top of the leaderboard as easily as possible. A few, curious of these updates, participated a little more than usual for a couple of days. Negative points and/or loosing part of the accumulated points were another suggestion to keep fading users engaged in the long term. Ultimately, it is interesting to notice that although the gamification implementation received a rather cold reception in the community, activity also seems to have thickened, at least temporarily. All in all, the SEGAN community is now a little more active than the other two, non‐gamified, VCoPs. Hence we believe there are some ways the gamification process can be improved:

Leaderboards should be made seamless and more useful, i.e., serve as a reputation and identitary distinctiveness;

Gamification point awards should be more proportional to the community perceived value. This may take us to experiment with a more dynamic award system, at least partially based on a value supply‐and‐ demand economic approach.

References Ardichvili, A., Page, V. and Wentling, T. (2002) Motivation and Barriers to Participation In Virtual Knowledge‐Sharing Communities Of Practice. Brown, J.S. and Duguid, P. (2000) 'Balancing Act: How to Capture Knowledge Without Killing It.', Harvard Business Review, vol. 78, May/June, pp. 73‐80. Chou, Y.‐k. (2013) Octalysis: Complete Gamification Framework , 30 April, [Online], Available: HYPERLINK "http://www.yukaichou.com/gamification‐examples/octalysis‐complete‐gamification‐framework/" [30 April 2013]. Csíkszentmihályi, M. (1990) Flow: The Psychology of Optimal Experience, New York: Harpers Perennial. Deterding, S. (2010) Pawned. Gamification and its Discontents, September, [Online], Available: HYPERLINK "http://www.slideshare.net/dings/pawned‐gamification‐and‐its‐discontents" Duffy, S. (2012) 4 Tips for Keeping Your Gamified Community Motivated, April, [Online], Available: HYPERLINK "http://mashable.com/2012/04/24/tips‐motivating‐gamified‐community/" / . EMIC (2012) EMIC Objectives, [Online], Available: HYPERLINK "http://emic.ismai.pt/objectives/" European Union (2012) Learning for All, 02 Outubro, [Online], Available: HYPERLINK "http://ec.europa.eu/education/lifelong‐learning‐policy/adult_en.htm" [Outubro 2012]. Gray, B. (2004) 'Informal Learning in an Online Community of Practice', Journal of Distance Education, vol. 19, no. 1, Spring, pp. 20‐35. Kelly, T. (2012) Everything You'll Need To Know About Gamification, November, [Online], Available: HYPERLINK "http://techcrunch.com/2012/11/17/everything‐youll‐ever‐need‐to‐know‐about‐gamification/" Lai, K.W., Pratt, K., Anderson, M. and Stigter, J. (2006) Literature Review and Synthesis: Online Communities of Practice, Dunedin, New Zealand. Leino, S. and Ovaska, J. (2008) A Survey on Web 2.0. Marczewski, A. (2012) A Simple Gamification Framework / Cheat Sheet, [Online], Available: HYPERLINK "http://marczewski.me.uk/gamification‐framework/" [4 April 2013]. Moore, T. (2007) Understanding Member Motivation for Contributing to Different Types of Virtual Communities: A Proposed Framework. Nicholson, S. (2012) 'Strategies for meaningful gamification: Concepts behind transformative play and participatory museums', Meaningful Play 2012, Lansing, Michingan. Orr, J.E. (1986) 'Narratives at work: story telling as cooperative diagnostic activity', CSCW '86 Proceedings of the 1986 ACM conference on Computer‐supported cooperative work, Nova Iorque, 62‐72. Prensky, M. (2002) 'The Motivation of Gameplay, or, the REAL 21st century learning revolution', On The Horizon, vol. 10, no. 1. Riu, E. and Jokisalo, A. (2009) Informal learning in the era of Web 2.0. SEGAN (2012) About SEGAN, November, [Online], Available: HYPERLINK "http://seriousgamesnet.eu/community/pages/view/1990/about‐segan"

António Andrade and Carlos Vaz de Carvalho Shugurensky, D. (2000) The Forms of Informal Learning: Towards a Conceptualization of the Field. Simon, R. and Sveiby, K.‐E. (2002) 'Collaborative climate and effectiveness of knowledge work ‐ an empirical study', Journal of Knowledge Management, vol. 6, pp. 420‐433. TIED Shoe (2012) TIED Shoe Summary, 18 November, [Online], Available: HYPERLINK "http://tied‐shoe.eu/en/" Tu, C.‐H. (2002) 'The management of social presence in an online learning environment', International Journal on E‐ learning, April‐June, pp. 34‐45. Vaz de Carvalho, C. and Fernandez‐Manjon, B. (2013) 'Welcome message from the Editors‐in‐Chief', in Vaz de Carvalho, C. and Fernandez‐Manjon, B. (ed.) EAI Endorsed Transactions on Game‐Based Learning, ICST. Wasko, M. and Faraj, S. (2000) '"It is what one does": why people participate and help others in electronic communities of practice', Journal of Strategic Information Systems, vol. 9, pp. 155‐173. Wenger, E. (1998) Communities of Practice: Learning, Meaning, and Identity, Cambridge University Press. Wenger, E. (2006) Communities of practice, a brief introduction, June, [Online], Available: HYPERLINK "http://www.ewenger.com/theory/index.htm" . Wenger, E., McDermott, R. and Snyder, W.M. (2002) Cultivating Communities of Practice. Wenger, E., White, N. and Smith, J.D. (2009) Digital Habitats: stewarding technology for communities. Werbach, K. and Hunter, D. (2012) For the Win: How Game Thinking Can Revolutionize Your Business, Wharton Digital Press. Zichermann, G. (2011) 7 Winning Examples of Game Mechanics in Action, July, [Online], Available: HYPERLINK "http://mashable.com/2011/07/06/7‐winning‐examples‐of‐game‐mechanics‐in‐action/" Zichermann, G. (2012) Getting Three Fs in Gamification, 19 January, [Online], Available: HYPERLINK "http://www.gamification.co/2012/01/19/getting‐three‐fs‐in‐gamification/" [4 April 2013]. Zichermann, G. (2013) Gamification: The Hard Truths, 23 January, [Online], Available: HYPERLINK "http://www.huffingtonpost.com/gabe‐zichermann/gamification_b_2516376.html" [25 Mar 2013]. Zichermann, G. and Cunningham, C. (2011) Gamification by Design, O'Reilly.

SIMaging the CITY: The Educational use of Simulation Video Games for Disadvantaged Youth 1

Massimiliano Andreoletti1 and Gianna Cappello2 1 Department of Pedagogy, Catholic University Sacro Cuore, Milano, Italy 2 Department of Culture and Society, Palermo University, Italy massimiliano.andreoletti@unicatt.it gianna.cappello@unipa.it Abstract. In this paper we argue that the intermediaries supporting individual and community social empowerment (families, schools, youth and health services, both public and private, cultural and social entrepreneurs) could use video games so as improve the effectiveness of their interventions. We report the initial findings of an ongoing action‐research project which aims at experimenting the educational use of a simulation video game (SimCity) in a youth club (Centro Tau) located in a highly disadvantaged and mafia‐bound area of Palermo (Italy). From these findings, it appears that, by SIMaging their ideal CITY, the Centro Tau youth have started to think about “civic” issues on a very concrete and practical level. Despite the strong affective ties they show towards their daily living context, they have lucidly identified its negative aspects and confronted themselves on the choices and solutions necessary for reinventing it as a better urban setting inspired by a vision for sustainable development. It also appears that gaming may represent for them an important and powerful opportunity, in a way a kind of “training ground”, for experiencing collective action, peer‐based learning and self‐ esteem. Keywords: simulation video game, civic empowerment, disadvantaged youth

1. Introduction With ever‐greater force our society expresses the pedagogical urgency to provide adequate responses to the educational needs that people may express during their life. The increased demand for moments of training and self‐training pushes for identifying the new needs of the 21st century subjects, and simultaneously, to look for solutions that fit the new communicative styles and make use of the solutions that the media, especially the digital ones, can offer. This is even more necessary in those situations where cultural, social and economic gaps heavily reduce the horizons of possibilities for individual and collective empowerment offered to marginalized groups.

2. Game‐based learning for social inclusion: A sociological view In the 21st century, where digital media are pervasively changing the way we work, consume, socialize and live, those who address at various levels and contexts the needs of the most disadvantaged people may find new ways of action in the use of video games, despite the techno‐panic complaints, traditionally dominating public discourse, according to which video games are in fact part of the problem rather than a possible solution. We argue here that the intermediaries supporting individual and community empowerment (families, schools, youth and health services offered by public and private institutions, cultural and social entrepreneurs) could use video games so as improve the effectiveness of their interventions. Computer simulations, the motivating force of gameplay, the ubiquitous nature of mobile phones, the connective potential of social networks are increasingly proving effective in developing learning by creating interest, motivation, satisfaction and loyalty (Cappello, 2009; 2012). A gamifying approach, that is the application of game mechanics and game thinking to non‐game environments, is increasingly being experimented in various fields such as like customer engagement, education and training, business, the military (Herger, 2012; Deterding et al., 2011) 2 .

1 Although the paper was discussed and developed jointly, Andreoletti actually wrote the paragraphs 2 and 3, and Cappello the paragraphs

1 and 4. 2 UNAOC (United Nations Alliance of Civilizations), a branch of UNESCO, in partnership with Learning Games Network and MIT‐Education

Arcade, has recently launched a contest in order to promote the creation of apps/games for intercultural dialogue and human rights (http://www.unaoc.org/create/). The winning applications for 2012 are available for download from: http://www.unaoc.org/create/finalists/. Among the winners are apps/games allowing users to improve their knowledge of Arab culture (Ibn Batuta), to experience the cultural diversity through the eyes of children (Touchable Earth) and to become aware of the global water scarcity (Get Water!) and the importance of critical thinking in journalism (Reality).

Massimiliano Andreoletti and Gianna Cappello Following these developments, we aimed at experimenting the opportunities and challenges of video games as a tool for social inclusion of the people, namely youth, living in at‐risk urban areas 3 . Playful, creative and motivating learning strategies based on video games may ensure that the young people living in these areas are not left behind by failing more traditional educational and pro‐inclusion approaches. Learning via video games may challenge them with the technological, organizational, planning and operational skills they usually associate with ‘their’ media uses. It also increases their motivation to learn, self‐esteem and peer relationships; develops numeracy/literacy/digital literacy skills; reduces marginalization, stigmatization, ‘gang antagonism’ and ‘gang feuds’; supports active participation in an e‐inclusive society where digital media are used as tools for individual and collective empowerment (Haché, A. et al., 2010). These goals may be achieved by using both special‐purpose (serious) games specifically, developed for a purpose beyond entertainment, and commercial off‐the‐shelf (COTS) games 4 . In order to increase the systematic use of video games in these contexts, three priorities are to be addressed (Stewart et al., 2013) 5 :

negative stereotypes against video games need to be dismantled through robust scientific evidence demonstrating the positive impact they can have at various levels and settings. Therefore, protocols for the assessment and evaluation of the activities need to be developed and tested within the framework of action‐research projects;

professional training for teachers, educators, health and social care workers is pivotal for reaching a critical mass of video game users in these contexts. Training of software and video games designers is also crucial in order to give them the specific competence necessary for addressing social‐inclusion issues;

from these two priorities, the third one follows, that is the bridging between the research field on the one hand, and the production, distribution and use of video games in support of disadvantaged people, on the other. It is necessary to create a multidisciplinary and multidimensional “ecosystem” where research is being conducted closely in contact with the video games industry, the end users and the intermediaries supporting individual and community empowerment (families, schools, youth and health services offered by public and private institutions, cultural and social entrepreneurs, etc.).

These priorities stem from the fact that social exclusion is a multidimensional problem caused by a series of discriminatory factors, often reinforcing each other. E‐inclusion cannot be in any way considered as a solution in and of itself. In fact, it may even be part of the problem as it may end up exacerbating existing inequalities. A vicious circle may in fact originates insofar as social exclusion leads to digital exclusion, which in turn reinforces social exclusion. E‐inclusive policies and actions must be then conceived in alignment with other policies devoted to wellbeing, health, education, employment, crime prevention and post‐detention rehabilitation, migration, etc., an alignment which directly involves a range of stakeholders: governments, schools, families, universities, the digital media industry as well as non‐profit organizations.

3. The psycho‐pedagogical significance of simulation video game In the last ten years the sectors of the academy which are interested in educational processes have shown a gradual interest toward the world of video games. This openness stems not only from the level of diffusion now reached by the medium, but especially from the analysis of the potentiality that it may have in both formal and informal educational contexts (Gee, 2007; Prensky, 2007, Egenfeldt‐Nielsen, 2007). Potentiality that is corroborated mainly in the capacity of the video game to place the subject in situation, constantly asking him/her to be an active part in the construction of meaning and sense, offering him/her the opportunity to reflect on the process that s/he carried out, to allow him/her the possibility of making a mistake without being judged or evaluated, to experiment with solutions that are not otherwise feasible, to implement his/her desires and needs. 3 Adopting the European Union definition, social inclusion is the “process which ensures that those at risk of poverty and social exclusion

gain the opportunities and resources necessary to participate fully in economic, social and cultural life and to enjoy a standard of living and well‐being that is considered normal in the society in which they live. It ensures that they have a greater participation in decision making which affects their lives and access to their fundamental rights” (Joint Report by the Commission and the Council on Social Inclusion (2004). Retrieved from: http://ec.europa.eu/employment_ social/soc‐prot/soc‐incl/joint_rep_en.htm. 4 For a discussion about this distinction, see Stewart, 2012. 5 Some of the case studies studied by Stewart and his team are: InLiving (http://www.inliving.co.uk/about‐us/), a role‐playing game promoting citizenship and participation at community level, policy awareness, and learning about government budgeting; Choices and Voices (http://www.choicesandvoices.com/), short role‐playing games for pupils to prevent violent extremism, promote community cohesion, team work, understanding on social and economic inequality.

Massimiliano Andreoletti and Gianna Cappello The close relationship that has developed between the operational dimension and the educational practice opens the way to design and implement interventions on the territory that may become real occasions of empowerment for the involved people. However, the need to build paths that refer in a constant way to both a personal and a group reflection on the relationship between theory, meant as a “set of hypotheses to explain a given phenomenon or an order of phenomena”, and reality, seen as the outside world that surrounds every person, often collides with the existing difficulties to locate the interconnections binding together people and facts and eventually understand the relation, central to every learning activity, between ourselves and the world around us. Such is the condition of those people living at the margins of society who are not able to recognize their educational needs and, in many cases, reject any given support and opportunity. A solution, which may better meet the needs of these subjects, comes from the techniques of social and cultural animation traditionally gravitating around playfulness and entertainment. The motivational dimension, given by the use of games, is further amplified by digital and electronic solutions that make see video games not only an entertainment’s mechanism strongly used by adolescents and young people, but above all a very useful pedagogical support that enables involved people to immerse themselves within significant simulations. The possibility to visualize in multimodal terms the concrete results of one’s own simulation activities expresses “versatility, ability to innovate, lateral and divergent thinking, flexibility, curiosity, propensity to the exploration of new solutions” (Anolli and Mantovani, 2011). By now the possibility to access easily and cheaply video games that allow the simulation of complex and articulated systems, according to a logic based on the interest and the personal pleasure of the subjects, enables formative agencies operating on a territory to satisfy adequately personal and collective needs. The educational effectiveness that video games are gradually showing in the educational context also thanks to sound empirical research (Rosa, 2012), originates precisely from this ability to reproduce more or less complex systems. In fact, at a general level, all video games can be meant as simulators, however the genre of simulative video games was specifically designed to be a simulation of phenomena and systems at different levels of details and complexity. In the psychological field, the potentiality of simulation games can be seen on two particular areas, given that they are “the central engine of our mind, since they enable us to have a general and local, dynamic and flexible representation of the various aspects of external and internal reality” (Anolli and Mantovani, 2011):

the world (external reality): the simulation is extremely functional with regards to the cognitive processes of reality because it allows people to bind the object (the world) and the method (the theory), facilitating them to identify the possible links that exist between the parts, to construct new models that do not exist yet, to immediately check solutions within the safe environment of the game. At a first level, the simulation allows to know the world, or parts of it, by facilitating multiple types of users in approaching external reality. On a second level, it is meant as an environment in which one can (re)build the world with different degrees of resolution, from elementary to accurate, and effective results when addressing possible or imaginary worlds: “what does not exist yet but can exist and become real given certain conditions. Addressing possible worlds, the simulation, in addition to being a ‘recreation’ of what is already there in the reality, is also the exploration of what could be elsewhere” (Anolli and Mantovani, 2011). On a third level, it plays a fundamental role in the context of creativity, because “through the simulation the human mind is able to create new combinations never considered before thanks to unexpected combinations and unforeseen associations (perhaps unpredictable and unthinkable until then)” (Anolli and Mantovani, 2011). Simulation allows to locate new hypotheses, to anticipate reality, to create things not yet existing. It hence represents “a great support to strengthen the innovative capacity of human beings, promoting a significant increase in the association and generation of new ideas” (Anolli and Mantovani, 2011);

the person (internal reality): the ability to reflect on one’s own work and imagine one’s own future, highlights how the use of simulation environments can be extremely effective in educational activities: “the knowledge of the past and the future, either near or far, is always mixed with the knowledge of the present” (James, 1890). The potentiality of simulation video games is particularly significant in the educational activities enacted in contexts of social disadvantage. The possibility of visualizing a future different from the one envisaged in these contexts allows the involved people to represent various forms of their selves, i.e., “configurations of themselves in the prospected future which may be desired if positive or feared if negative” (Oyserman and James, 2009). These forms are “possible selves”. The use of the simulation video games with adolescents and young people, who live in contexts of social

Massimiliano Andreoletti and Gianna Cappello disadvantage, may strongly support the entire educational action, since “the expectation and the planning of the future of themselves tend to improve the wellbeing of present life, since, especially in western culture, they are based on the hope that a change for the better is possible as a result of the malleability of his own self” (Klein and Zajac, 2009). The role of simulation video games is not only fulfilled in the future, but achieves important effects also on present daily action, especially when they “provide detailed images connected with strategies that bind the present situation with future states” (Anolli and Mantovani, 2011).

4. SIMaging the CITY: Designing the intervention model 4.1 Foundations The decision to choose the video game medium was based on the assumption that it “allows you to develop different paths of analysis and reflection that, starting from the simulated world within the video game, come to an end in the reality and vice versa” (Andreoletti and Ragosta, 2013). The video game becomes then a bridge between the real and the virtual world allowing people to reflect on the specific characteristics of each world in personal ways, to identify which can be the common elements between the two, and to analyse which templates of one world can be applied on to the other world. The video game is here meant as “an environment in which you can play and where the technology is at the same time the instrument that conveys the game activity, but especially a world with an added value in which the subject has the opportunity to explore, experiment with, manage, interact and communicate with high levels of autonomy, interaction, presence, immersion and imagination” (Andreoletti, 2012). The use of a simulator follows a series of conditions that validate its use avoiding personal and utilitarian dispersions and drifts. The choice of the title must be dictated by the possibility given to the players to experience an open and significant problem, while the design must allow the free expression of their own desires, the reasoned experimentation of their own choices, and the constant confrontation with reality, governed by a shared reflection among all those participating in the activity. From a conceptual/methodological point of view, the underlying model is that of the experiential learning (Pfeiffer and Jones, 1985; Pfeiffer and Ballew, 1988), which allows to pass from a reflection on an experience, also simulated, to an application in the real world through a cyclic process made of five stages (experience, communication, analysis, generalization and implementation). Under the guidance of the educator‐mediator, the group becomes the place where reality is read, analysed, modelled and then rebuilt inside of the video game. Here ideas, desires and needs can take shape, and then be brought again on the group, shared with the other players and then generalized to other situations of the real world. The educational activity can therefore be meant as the result of three different levels of intervention inscribed one into the other (see figure 1).

Level 2 ‐ Group Modelling Sharing Reflection Generalization

Level 3 ‐ Video game Experimentation Realization Analysis

Level 1 ‐ Reality Reading Analysis Application

Figure 1: Levels of use of the video game within educational activities Given this conceptual/methodological framework, we decided to choose a commercial video game, and not a serious game, because we wanted to offer a playful experience which was as close as possible to what the user is normally used to, especially in term of graphics and gameplay. The need to use an up‐of‐date version available at a reasonable price made us choose SimCity 4, a city builder management simulation video game.

Massimiliano Andreoletti and Gianna Cappello Within a city builder, the video gamer has the tasks and the prerogatives of a mayor managing a town through administrative functions and according to the resources available and the specific realities of the territory. However, the use of SimCity must not be perceived as the simple reduplication of the mayor’s activity, as the game is in fact meant to be an environment that allows the player to reflect not only on the management issues of a more or less complex virtual city, but also on the reality in which everyone is immersed. When employed in an educational context, players become aware of their being citizens at both a micro level (the district, the city) and at a macro level (the country, the continent, the planet), going from the role of simple spectator of what is happening around him/her with no ability/possibility to intervene, to the role of a critical and active actor, an involved citizen who thinks and works to implement significant changes in the society in which s/he lives. The simulation video game is indeed a very effective solution to create these educational opportunities. SimCity can be likened to an aquarium: it reproduces only the optimal conditions of reality, by omitting all those situations that would cause a rapid destruction in a real ocean. The player/designer inserts in this empty container only the elements (plants, fish, and ornaments) that s/he want to. The level of experimentation concerning his/her own desires and needs is practically unlimited in time and space. The management of the educational activity must lead the player throughout a process that initially allows him/her to express himself/herself (in terms of needs, affections, emotions about the surrounding reality), trying to make him/her avoid the drift of simply entertaining or unproductive playing activity. Eventually, the player is led to reflect on realities that gradually move away from his/her own (needs and emotions about a remote reality), making him/her experience a more multicultural vision.

4.2 Structure Our educational project, divided in four operational modules, offers a model applicable to different contexts and situations, and implementable in whole or in part, depending on the specific needs:

approaching the simulated city: exploring and mastering the game. The objective here is to explore and to experiment with the game possibilities (the game interface, the ways of shifting/orientation/construction, the menu structure, etc.);

analysing and rebuilding your own city: (video)play with reality. With this module we want to give participants a global and structured vision of their surrounding environment, as the majority of people lives and attends only a portion of their reality. Starting from this consideration, through this module participants become aware of the relationship between the real world (the lived city) and the virtual world (the built city), reproducing it again within the simulation game;

rethinking your own city: (video)play with fantasy. The playful activity is here meant as a space in which participants are be able to express dreams, desires, and aspirations: it’s the personal rethinking of the real city by inserting what they would like to be there and removing what they would not want to. This activity allows participants to reflect in depth on themselves and to check what are the consequences of their choices;

watching ‘other’ cities: (video)play with the rest of the world. This module allows participants to experience expressions and manifestations of human life distant from their daily lives. As such, they ‘live’ the conditions that different existing societies and cultures have in their places of origin (Andreoletti and Ragosta, 2013).

5. SIMaging the CITY: Experimenting the model 5.1 The context, the methodology, the activities The very first idea to experiment the use of a simulation video game for disadvantaged people in an informal educational context came to us from participating to the European project “Gamepaddle” 6 . As Italian partners of the project, representing both our universities and MED (the Italian Association for Media Education), we decided to implement our experimental activities in a youth club in Palermo (Centro Tau) whose educators are already quite active in using media in their activities. Centro Tau is located in a highly disadvantaged area of 6 The project was funded within the “Youth in Action” EU Program. For details: http://www.gamepaddle.eu

Massimiliano Andreoletti and Gianna Cappello the city (the La Zisa neighbourhood) with a heavy impact of both micro and organized criminality (mafia) and a long‐standing lack of social, cultural and recreational initiatives/infrastructures. Schools are here daily confronted with high levels of dispersion and dropping out, and therefore the role of youth clubs is crucial in offering complementary (if not alternative) educational and cultural activities to these youth. Activities started in early March 2012 (with a short training of the Centro Tau educators) and went on till June 2012 7 ; a total of 15 youth were involved (11 males and 4 females), aged 14‐18 8 . The methodology included participant observation sessions, focus groups and questionnaires. After a preliminary discussion with the educators and a first meeting with the youth, we decided to make some changes to the initial model of intervention (as described in the previous paragraph) in order to better adapt it to the actual context of implementation and to the youth involved. Therefore, after a first activity to approach the simulated city (see phase 1 of the model), three other sets of activities followed: 1. analyse and reconstruct your actual city. The youth were asked to produce a paper map (a poster) of the La Zisa neighbourhood with the aim of having them look around and think about (both individually and collectively) their living context. They were asked to identify the places that had negative and positive connotations for them. Accompanied by one of the educators, they went around the neighbourhood making interviews and taking photos which were afterwards glued to a big poster together with drawings, web‐ retrieved pictures and written comments. As a result of these activities, each kid created a personal portfolio with all the materials collected. The final poster represents a choral portrait of the La Zisa picturing the most important locations of the youth’ daily lived experience; 2. playing video games. Before going into the final phase, an intermediate activity was carried out in order to increase the youth’ participation and interest, and also have an idea of the needs and motivations that lead them to playing video games. Through a peer discussion, they made a selection of favourite video games which were then used for a contest where 7 teams, each consisting of 2 youth, where to defy each other. The list of the video games selected clearly reflects the male predominance within the group as it includes sports games (Need for Speed. Carbon, Fifa 2012, Pro Evolution Soccer 2012, etc.) and action games (The Godfather, Grand Theft Auto. San Andreas, Assassin’s Creed II, etc.). Parallel to the contest, sessions of SimCity playing were also held so that they could start familiarizing with it. After some initial enthusiasm, the youth increasingly abandoned the contest session and, rather surprisingly, appeared more interested in “messing around” (Ito et. al. 2010) with SimCity; 3. creating your ideal city was the third and final phase when youth were asked to create their ideal city. Throughout the sessions, playing was constantly accompanied by an intense sharing of opinions and suggestions about the difficulties encountered as well as the strategies and solutions adopted. Prompted by us to discuss and report about the ideal cities they were developing, they were keen to confront them with the strengths and weaknesses of the La Zisa, making a wish‐list of all the elements that in their opinion should be included in their ideal city (more green areas, recreational structures, meeting places for young people, big and modern sports facilities, working places).

5.2 Some initial findings and temporary conclusions Admittedly, in order to better assess and evaluate our model, further experimentation is needed. A second set of experimental activities is to start in early June 2013 involving youth from both Centro Tau and another youth club located in a different at‐risk area of Palermo (Brancaccio). As suggested by the educators of the Centro Tau, the youth who have already been part of the first experimentation at the Centro Tau will play an important mediating and legitimating role as they represent a key component for motivating and involving the youth from Brancaccio. By examining the qualitative data collected during this first experimentation, we can draw some initial findings and conclusions which are going to orient our future experimentation. 7 As it typically occurs with action‐research, the team includes educators working at Centro Tau and academics: Massimiliano Andreoletti

and Anna Ragosta (Catholic University, Milan, MED), Gianna Cappello, Marcello Marinisi and Annalisa Castronovo (University of Palermo, MED), Ignazio Rosato, Giovanni Bonsignore and Daniela Bellomonte (Centro Tau). 8 In order to have a more gender‐balanced group, we did attempt to involve more girls attending the Centro Tau but apparently video gaming did not seem to be an appealing activity to them.

Massimiliano Andreoletti and Gianna Cappello One expected outcome is that, by using SimCity to deconstruct their living context and then reinvent it as an ideal one, the youth from Centro Tau were able to express needs and desires about it in a quite unusual and effective way. As a matter of fact, as the educators explained to us, it is quite difficult to have these kids sit and talk about their living context, especially when it comes to identify those negative elements which might have some kind of tie to their private lives (for example, the corners where mafia’s drug peddling occurs, or the frequent unauthorized constructions, or else the widespread practice of littering anywhere). Undoubtedly, La Zisa is and remains a “home” to them (as they quite proudly put it during one focus group), a familiar and comfortable place they are tightly bound to, a place they strongly identify with, to praise and protect, especially when they want to challenge the “crime‐like” representations expressed in public discourses about Palermo at‐risk areas. Yet, as we could draw from the activities they carried out during phase one, by increasingly involving them in concrete activities (taking pictures, making drawings, writing comments to be collected on the poster) rather than discussions and debates, they were able to come to terms with their mixed and contradictory feelings about their living context, identifying and, more importantly, sharing among themselves the negative and positive aspects of it. Another interesting, and yet quite unexpected finding, concerns the gaming experience itself. As said, during phase two, in order to motivate and involve them, and also have an idea of their relationships to video games, we proposed them a gaming contest which, after some initial enthusiasm, was soon abandoned. As it emerged from discussions with the educators as well as from a focus group we had with the youth, a possible reason for this is due to the fact that playing video games (especially the COTS games they chose for the contest) was seen as a totally free and spontaneous activity, scarcely confinable to some kind of organized and supervised control, such as the contest. Playing SimCity was instead seen as a follow up of the previous activity (the production of the poster) and a step to the final phase they were eager to get that is the construction of their ideal city 9 . Ultimately, the failure of the contest confirms a quite recurrent evidence in the scientific literature about the pros and cons of using COTS games for pro‐social goals. While on the one hand they can count of communities of practices whose members have developed a strong sense of expertise and self‐confidence as well as high levels of motivation, interest and mutual help, on the other hand the long learning curve required to be fully part of these communities cannot be easily recreated within some kind of pre‐determined and supervised learning context. As Stewart writes, “COTS games are developed and designed to be played over and over again and are built according to highly complex semiotic systems that contain many variables to endure a nearly unlimited play time” (Stewart 2012), and these systems cannot be recreated “in vitro”, as with our contest. COTS games may be a problem for the educators (or the teachers) too who cannot be so easily involved, as it was in fact the case with the educators of the Centro Tau during the contest. The long learning curve of COTS games, as Stewart goes on to say, “makes them less attractive for teachers or others. Teachers need a certain amount of time to master the game themselves, before they can apply the game in a learning context. It implies that the digital divide surmounts.” Apparently, less COTS‐games literate educators or teachers will be less able to use them in a pedagogically sound manner. Unlike the second one, the third and final phase fully met our expectations. Participants showed a great involvement in expressing their needs and desires about their ideal living context through the digital simulation. Even it with different degrees of complexity, all of them managed to build and manage their ideal city. Albeit working on individual simulations, a lot of intense peer discussion and mutual help went on with regards, for example, to what was to be included or removed from the cities 10 . As they often commented, by working concretely on the building and management of a city, they had the possibility to think about and express their opinion on the “civic” issues, feeling all the time highly involved, both at an individual and collective level. In fact, building the ideal city was considered more a collective endeavour rather than an individual one. Those who developed the most creative and effective solutions were appointed by the group as “best mayors” and were often requested – by both the educators and the peers – to give advice and suggestions. Following Ito et al. (2010), we believe that the most important learning outcomes of this kind of gaming activity cannot be reduced to an issue of transfer of knowledge or skills. In a context like Centro Tau, what really counts is that such knowledge and skills are developed according to a logic of intense social 9 For this reason we have decided to remove the contest activity in our future experimentation. 10 The most interesting debates occurred when it came to allocate resources and space for certain particular buildings: a police station, for example. None of the youth had thought about it and when the educators suggested it, some promptly agreed while others appeared much more disinterested. Gender was also a condition for discussion. Having decided to include a huge shopping mall, girls were mocked by the boy for wasting their resources on it. The inclusion of free and modern sports facilities as well as a large “piazza” where all people could meet and have a walk with friends and families, were instead a choice all youth made enthusiastically.

Massimiliano Andreoletti and Gianna Cappello interaction, effective communication and problem solving, addressing one specific goal. For these kids, gaming may represent an important and powerful opportunity for experiencing collective action and self‐esteem. Indeed, as Thomas and Brown (2007) argue, it may even function as training ground for future collaborative forms of work and social action. Therefore, while waiting for these findings to be further verified and possibly advanced by our future experimentation, we can tentatively conclude that when youth have the opportunity to pursue projects based on their own interests (as video games are), and to share them within a network of peers with similar investments, the result is highly active and motivating forms of learning.

References Andreoletti, M. (2012), Videogioco. In Aglieri, M. and Ardizzone, P. (eds.), Realtà educative. Milano, Unicopli. Andreoletti, M. and Ragosta, A. (2013), S’IMpara con i videogiochi. In Parola, A. and Rosa, A. and Giannatelli, R. (eds.), Media, linguaggi, creatività. Un curricolo di Media Education per la scuola secondaria di primo grado. Erickson, Trento. Anolli, L. and Mantovani, F. (2011), Come funziona la nostra mente. Apprendimento, simulazione e Serious Games. Il Mulino, Bologna. Cappello, G. (2009), Nascosti nella luce. Media, minori e media education. FrancoAngeli, Milano. Cappello, G. (2012), Ritorno al futuro. Miti e realtà dei nativi digitali. Aracne, Roma. Deterding, S. et al. (2011), “From game design elements to gamefulness: Defining “gamification””. Proceedings of the 15th International Academic MindTrek Conference, pp 9–15. Retrieved from: http://85.214.46.140/niklas/bach/MindTrek_Gamification_PrinterReady_110806_SDE_accepted_LEN_changes_1.pdf Egenfeldt‐Nielsen, S. (2007), Educational potential of computer games. Continuum, New York. Gee, J. P. (2007), What video games have to teach us about learning and literacy. Palgrave Macmillan, New York. Haché, A. et al. (2010), “Using ICT to reengage and foster the socio‐economic inclusion of youth at risk of social exclusion, marginalized young people and intermediaries working with them”. Retrieved from: http://youth‐partnership‐ eu.coe.int/ youth‐partnership/ekcyp/research_europe.html. Herger, M. (2012), “Gamification. Facts & Figures”. Retrieved from: http://enterprise‐gamification. com/index.php/en/facts. Klein, W. M. P. and Zajac, L. E. (2009), Imaging a rosy future: The psychology of optimism. In Markman, K. D. and Klein, W. M. P. and Suhr, J. A. (2009), Handbook of imagination and mental simulation. Psychology Press, New York. Ito, M. et al. (2010), Hanging out, Messing around, Geeking out. Kids Living and Learning with New Media, The MIT Press, Cambridge MA. James, W. (1890), The Principles of Psychology, II volume. Little Brown, Boston. Oyserman, D. and James, L. (2009), Possible selves: From content to process. In Markman, K. D. and Klein, W. M. P. and Suhr, J. A. (2009), Handbook of imagination and mental simulation. Psychology Press, New York. Pfeiffer, J. W. and Jones, J. E. (1985), Reference guide to handbooks and annuals. University Associates Publishers, San Diego. Pfeiffer, J. W. and Ballew, A. C. (1988), Using case studies, simulations, and games in human resource development. University Associates Publishers, San Diego. Prensky, M. (2007), Digital Game‐Based Learning. Paragon House, St Paul. Rosa, A. (2012), I videogiochi come palestra di sperimentazione valoriale. In Felini, D. (ed.): Video Game Education. Studi e percorsi di formazione. Unicopli, Milano. Stewart, J. (ed.) (2012), “State of Play of Digital Games for Empowerment and Inclusion. A Review of the Literature and Empirical Cases”. Retrieved from: http://is.jrc.ec.europa.eu/pages/EAP/ eInclusion/games.html. Stewart, J. et al. (2013), Exploring the Potential Impact of Digital Games for Empowerment of Groups at Risk of Social and Economic Exclusion: Opportunities, Challenges and Possible Actions”. Retrieved from: http://is.jrc.ec.europa.eu/ pages/EAP/eInclusion/games.html. Thomas, D. and Brown, J.S. (2007), “Why Virtual Worlds Can Matter”. Retrieved from: http://www.johnseelybrown.com/needvirtualworlds.pdf.

“The Chest That Longs to be Moved”: A Serious Game for the Greek Muslim Minority Children Alexandra Androussou, Evangelia Kourti and Nelly Askouni National and Kapodistrian University of Athens, Athens, Greece alandr@ecd.uoa.gr ekourti@ecd.uoa.gr naskouni@ecd.uoa.gr Abstract: This paper refers to the creation and the use of a serious game produced within a large‐scale educational project, aiming at the social inclusion of Muslim minority students in Western Thrace in Greece. The game represents an educational challenge both at political and social level as it has to meet the particular characteristics of the target population it is addressing. Muslim minority's children face massive under achievement at the primary school and high drop‐out rates from compulsory education. The game was designed as a basic learning tool in educational activities outside the school in order to enhance the use of the Greek language, taking into consideration the needs of this specific population. It is intended for students between the ages of 8 and 12. It is played in group with the help of specially trained teacher‐animators in the 14 “Support Centers” (2 central, 8 peripheral and 4 itinerant) created for the needs of the educational project attended by 2.800 students. The socio‐cultural context in which this game was created and the underlying pedagogical theories in which is grounded, concerning the content and the technical specifications of the game are presented. A specific example concerning the use of this game, in a small isolated mountain village, is analyzed in order to illustrate how the very form of the game together with its contents mobilized children’s interest and led them to produce different types of texts (written and audiovisual) and by this to improve their communication skills in Greek language. It is argued that through the game’s educational activities children became more open to the outside world but also developed a stronger identity as new ways of thinking emerged as to their identity and their position in society. It is emphasized that the political dimension of the pedagogical choices in the design of a serious game aimed at vulnerable social groups is critical for the success of the game’s goals. Keywords: serious game, language learning, empowerment, identity, Muslim minority, intercultural education

1. Introduction This paper refers to the creation and the practice of a serious game which was produced within a large‐scale educational project, aiming at the social inclusion of Muslim minority students in Western Thrace, entitled “The Education of Muslim minority Children”. 1 It is a highly contextualized serious game, as it takes into consideration the educational needs of this specific population. In this perspective, although it could be considered useful only for this population, its pedagogical conception could be of interest to anyone creating educational games for minority groups in different formal or informal educational settings. For this purpose the social‐ cultural context in which the game was created and the underpinning pedagogical theories, concerning the content and the technical specifications of the game, will be presented. In order to contribute to the ongoing discussion around the possibilities of using serious games for learning (Breuer and Bente 2010) a specific example concerning the use of this game will be analysed.

2. The social cultural context "The Chest that Longs to be Moved" [O sevdas tou sedoukiou] is a computer serious game designed for a special educational and social context. It was created as part of the educational project mentionned above to be used as a basic learning material in educational activities outside the school, in order to primarily enhance the knowledge of Greek by the Muslim minority children in Thrace, a province of North‐Eastern Greece bordering Bulgaria and Turkey. Τhe history of this area has been quite turbulent and in order to appreciate the complexity of the implementation of this project (Dragonas and Frangoudaki 2006) ‐and thus the way the game was designed ‐ one should have in mind the social and historical framework linked to and affected by the Greek‐Turkish relations (Anagnostou 1999; Yiagcioglu 2004). Although Greek citizens, most of the Muslim minority students have a Turkish ethnic identity and their education is regulated by the 1923 Lausanne Treaty concerning the rights of both the Muslim minority in Thrace and the Orthodox Greeks living in Istanbul. 1

www.museduc.gr

Alexandra Androussou, Evangelia Kourti and Nelly Askouni Minority students and their families live separately from the Greek majority in ethnically and linguistically homogeneous settlements, where the majority of women do not speak Greek. Consequently, most children speak Greek only during school hours. Almost the entire minority population belongs to the two lower social strata: farmers and manual workers, exceeding the national average by far. At the same time they have a very low educational level (Askouni 2008). In the beginning of this program, the intervention team had to face enormous problems: geographical and social isolation of the minority, children who did not speak Greek and attended special minority schools, inadequate textbooks, teachers totally unprepared to teach in this context, and of course political obstacles (Frangoudaki 2008). The great challenge for this project was to “translate” these critical conditions into a specific intervention strategy. The intervention had to be holistic, including research, development of educational material, transformative actions within the school structure (curriculum changes, innovative approaches to pedagogy and teacher in‐service training), as well as educational activities within the community (Androussou, Askouni, Dragonas, Frangoudaki and Plexoussaki 2011). The pedagogical rationale adopted for the development of the new educational material was based on the principles of critical pedagogy (Freire 1973; Apple 1982; McLaren 1992) and transformative pedagogy (Cummins 2004). In developing the educational materials, an open‐ended methodology was adopted. In this perspective, literacy was viewed as a social practice (Baynham 1995) whereby learners comprehend, interpret and acquire linguistic and communicative competences, and thus can use language in both oral and written form in a wide range of contexts. The emphasis was placed on children who acquire skills in the use and interpretation of all possible meaning sources (visual, linguistic, technological etc) as well as language production. In this context, actions and interventions outside school were also organized. The Program set up 2 central, 8 peripheral, and 4 itinerant “Support Centers” (their acronym in Greek is KESPEM). Their aim was mainly to provide compensatory classes that would enhance minority children’s command of Greek and improve their school performance. The classes were to be conducted during the school year, but also during the summer 2 vacations. The computer serious game we will present was created for the needs of these classes. 3 According to the main scope of the program it addresses language, culture and identity issues.

3. Rationale and design of the game As minority children in Thrace are mainly bilingual (Turkish‐Greek) or trilingual (Pomak‐Turkish‐Greek), teaching a second language (in our case Greek) raises, apart from the matter of appropriate teaching methods, issues involving relations of hierarchy between languages, and hence power relations in force in the particular social and cultural context (Cummins, 2004). Given that the main objective of the game was learning Greek, it was designed to provide the basis for establishing a framework within which each student could develop at his/her own pace and personal experiences, his/her basic learning ability (Cope and Kalantzis 2000). The pedagogical rationale of the game was also aimed at strengthening the students' identity (Cummins 1996), allowing for the release of their communicative skills. Thus, the themes of the texts of the game move from the familiar/what they know, to the unfamiliar/what they don’t know.The two main words chosen for the name of the game, "sevdas” and “sendouki", exist in both languages (Greek and Turkish). In Greek, they are loan words from Turkish and used in every day life in both languages. "Sevdas" means a complaint, and "sendouki" means chest, trunk. These two words refer directly to Turkish and therefore to the identity of the minority. In the political framework described above, the choice of these words is obviously symbolic. It is clearly a message of respect of the minority identity. 2

Other activities include: pedagogical and creative activities for pre‐school children, creative workshops for children and adolescents, Greek language classes for parents of minority pupils, seminars to "familiarize" primary and secondary education teachers with the Turkish language. 3 The following persons collaborated in the production of “The Chest that Longs to be moved”: Scientific advisor, responsible for pedagogical design and production: Alexandra Androussou. Computer design: Μ. Kyrkos. Illustrations: Th. Tibilis. Documentation texts, Educational activities: A. Dimitriou, V. Lagopoulou. Edting: M. Zografaki. Narration: K. Sidiropoulos, M. Sontaki. (see also http://museduc.gr/en/ηλεκτρονικα‐εκπαιδευτικα‐παιχνιδια/ο‐σεβντάς‐του‐σεντουκιού)

Alexandra Androussou, Evangelia Kourti and Nelly Askouni The fact that nowadays most young people are exposed to and use modern information and communication technologies, personal computers, gaming consoles and the Internet, played a crucial role in our decision to design a computer serious game. ICT has become part of young people's everyday culture and they have been socialized with digital media. The idea was that technology can act as an incentive but also give opportunities to children from low economic and educational strata to become familiar with the use of new tools, and in this way strengthen their identity. Computer games, according to Gee (2003), constitute great tools to support educational processes that introduce children to literacy. As he argues, computer games may be used in the educational process because they create a new charming learning environment that combines entertainment and education that motivate children in learning. As most of the serious games, this game is used for more than just mere entertainment (Susi, Johanesson and Backlund 2007) and belongs to the vast majority of serious games which are aimed at learning and education (Breuer and Bente 2010). In that sense, according to Michael and Chens’ classification (2006), where serious games are defined as games that teach, train and educate, it is part of the educational games category. This game is not only designed as a learning tool, but also as motivator and/or generator of interest. It doesn't only take into account contextualization, personalization and choice, but also combines entertainment and learning in a way that the children do not experience the learning part as something external to the game (Breuer and Bente 2010).

4. Description of the game In designing the game first we had to overcome a number of practical problems related to the specific population: the format, the tools that had to be used, the skills that had to be developed before children could have access to the opening screen. We had to take into consideration the technological characteristics of the computers available to run the game and the computer literacy of the students. Most of them had not even seen a computer. They did not know how to turn it on, how to use the mouse or what a double click means. The design had therefore to be very simple. It consisted in very simple procedures in Flash programm version 6, with consecutive animations and a very simple interface in order to be understandable. The game is designed in 2D for single players and when more children are present we use “hot‐ seat”. Each child plays the game and then the teacher/animator organises the pedagogical activities for the whole group. The presence 4 and role of a teacher/animator in this context is important. He/she acts as facilitator and mediator: He/she organizes the group playing on the computer, animates the process and observes the rules of the game. The game has 6 different levels with increasing difficulty in Greek per level. It is multi‐modal, includes many sources (texts, images, and games) and constantly refers to other external sources (books, maps, images, dictionaries). Each level consists of games (memory mappings, grouping words), texts and activities (writing, language games, mathematical operations, visual, and combination of these forms) on the texts. There are a total of 89 texts, with their respective activities, and 89 games, presented alternately to the flow of the game. th At the end of each level children "earn" a wheel of the chest. When the group wins the 6 wheel the chest is ready to travel again. While this is a computer game, it uses also other sources of knowledge connecting the virtual world with the world of the traditional book. It is a journey through space and time, a kind of interdisciplinary electronic library which deals with history, geography, social and civic education, art and everyday life. It alternates between individual and collective work, constantly providing opportunities for text production, short and/or longer. The game supports an interdisciplinary approach to knowledge not only in content but in form: it supports children in their efforts to combine information from various sources (electronic and print) and to approach information from different perspectives. It is designed to be played in groups. This choice is based on the principle of social learning. Rather than compete between them, children learn to cooperate and coexist in order to be able to play the game. The pedagogical principle of 'learning to learn', where knowledge is not meant as sterile information but the result of a complex process, a journey between experience and new information (Giordan 1998), was applied. As far as the language is concerned the game’s texts had to be written in simple language but deliberately 4

The teachers involved in this action are trained systematically from the scientific responsible of the KESPEM to whom they send daily assessments electronically and receive feedback about their work.

Alexandra Androussou, Evangelia Kourti and Nelly Askouni more demanding than the children’s level in Greek. The purpose was that the texts should not be understood by the children as typical 'language' school texts for working on vocabulary, grammar, etc., but to prompt questions by the children and trigger their curiosity to search for further information and become more open to the world. As Freire and Macedo (1987: 37) note «the reading of the world precedes the reading of the word and the reading of the word requires continuously reading the world ...». Although the texts are intellectually demanding, they are based on the experiences of children, thus adopting the perspective of situated learning (Gee 2004). They are not intended to make children speak Greek fluently, but to make them feel that they can also express their own experiences, anxieties, desires in Greek. Script and content of the game The scenario of the game is discovered by the children‐players while opening the first screen: in a summer 5 camp on the island of Samothrace, a group of children discover an abandoned chest. They open it and they find a letter where the chest tells its story. It is a chest full of objects and memories of a long journey that starts at the end of the 19th century, passes through the Balkans, through the Mediterranean, arrives in Alexandria, then to Marseilles, travels to Turkey, goes to Greece, migrates to some European countries and ends in Samothrace. During this journey the chest changes owners (traders, sailors, immigrants and refugees). Its story represents different types of migration in the modern Greek history (diaspora, labor migration, refugees due to civil war, population exchanges). Passing through Germany the chest changes owners, it passes from an immigrant of the majority to one of the minority from Thrace. In the letter, the chest explains that it wants to move and travel again in the world but has lost its 6 wheels. Players are challenged to find the wheels to help it travel again. Every wheel corresponds to a different language level. The texts of the game are short, 350‐500 words. Depending on the level, players face different difficulties. Each text is accompanied by a small vocabulary at the end. The texts refer equally on very familiar situations to minority children such as local festivities and rituals (e.g. the celebration of cherry, the kourbani), monuments of Thrace (e.g. the Komotini Imaret) and on very unfamiliar topics as rivers of Africa (e.g. Okavango), monuments and capitals of Europe (e.g the Mermaid in Copenhagen, the Eiffel Tower in Paris), great historical personalities (e.g. E. Venizelos, Gandhi, Martin Luther King). (see annexe, screenshot i) All texts have a historical dimension, placing each subject in historical, social and political context and are not merely of encyclopedic character. The texts are first displayed on the screen in the specific geographic area conerned, on a world map. This process along with the display of the dates a text refers to, in the form of a fast clock that runs backwards in time (from now to the specific date), situates visually the information in its historical and social context. There is no text that is decontextualized. In the flow of the game, the text is first read by the children as a group on the screen and heard from the computer’s speakers. Then, the children have to answer questions which are not typical comprehension ones. In order to answer these questions they need to elaborate the information contained in the text through multiple pedagogical activities, such as role playing, constructions etc. They are also asked to seek information from different sources and make a synthesis. Once they have finished working on a text they have access to a game which they have to play on the computer in order to advance to the next text. The images chosen for these games are relevant to the children’s interests in everyday life (e.g. football players, popular singers etc.). They are invited to find the same (e.g. two hidden identical pictures), (see annexe, screenshot ii) to classify into categories (e.g. rivers, lakes), to match equivalents (e.g. capitals countries flags with countries) (see annexe screenshot iii). Switching texts and games creates a treaty‐based learning pleasure and joy (Resnick 2011). Children have great fun when they unexpectedly come across images from their experience as playing children and not as "students".

5. Elements of evaluation of the game The evaluation of a serious game can be done in various ways. In order to have a complete picture, it is necessary to carry out a systematic and long‐term field research which will include interviews of students‐ players, interviews of teacher‐facilitators, observation in the classroom and analysis of the children's productions. Given the political and ideological dimensions of this field and the sensitive balances to be 5

Samothrace is a Greek island in the northern Aegean Sea well known to the children as it is situated within the Evros regional unit of Thrace.

Alexandra Androussou, Evangelia Kourti and Nelly Askouni followed by the intervention group, 6 it was not possible at this stage to make a survey interviewing children or to have a systematic observation by an external evaluator. As already mentioned, the pedagogical and political challenge of this intervention was to create relationships of trust between the members of the minority group and the intervention team, in order to be allowed to access these small, closed and conservative communities. For this reason, in this phase of the PEM, we were reduced to an action research which consists in analyzing the productions of children together with the daily records kept by the teachers of each group, where they mark the learning objectives for the group, how they drew up the educational process, the results obtained and the evaluation of the teacher. The combined analysis of first the daily evaluations of teachers, which include indications for the particular framework in which they worked, indicate the starting point of each group and inform on the educational process, and secondly, the productions of children (texts produced on the computer, ppt, maquettes, videos etc.) enable us to see to which degree the game achieved its two main objectives: strengthening of Greek and empowerment of the children's identity. Action research is still on‐ going, as the program is still running. “The Chest that Longs to be Moved” for the year 2012 ‐ 2013 has been used by 2,000 children aged 7 to 12 years who participated in the activities of KESPEM, both stationary and itinerant. Its use has resulted in a large diversity of productions in various media (paper, computer, videos, maquettes). In this presentation we will limit ourselves to the analysis of a specific production in a small and remote village in Thrace, where the game was played in an itinerant KESPEM. This example could illustrate how the principles of the game effectively worked. The conclusions of this analysis have been confirmed in a more recent publication (Androussou, in press).

6. Playing with the Chest in Plagia village Plagia is a small mountain village with 293 inhabitants, situated 45 km from Komotini, the capital of the prefecture of Rodopi. The van takes one hour to reach the village because of the mountain road. In the village there is only one cafe and a small grocery store. Most of the people are involved in tobacco growing and some few in farming. The elementary school has 19 minority children. 10 of them, aged 7‐12 years, are attending the classes of the itinerant KESPEM systematically, the older boys work with their parents after the school hours. As in many mountain villages, few minority children have visited Komotini and most of them have never been outside their area (only two have traveled once to Edirne in Turkey for shopping). They speak only Turkish and hear / use Greek only during the school hours and the two‐hour courses of itinerant KESPEM, which visites their village once a week. This village participates in the itinerant KESPEM courses since October 2011. At that time students' level of Greek was very low. The teacher/animator could have only a rudimentary exchange with the children in Greek. On this, an important role was played by the driver of the van, who is a member of the minority and participates often in the educational process as a mediator in translating from Turkish to Greek. During the first year, the teacher worked with the children on the first level activities of the computer game and the children won the first wheel. This year she began the second level of the game and noticed a clear progress and a wish by the children for creative activities. One of these activities was the creation of the video “Athens TV.” From November 2012 until February 2013, the children worked on four texts of the second level concerning the city of Athens, in particular Omonoia Square, the subway, the Acropolis Museum and the old mosque (now a museum).When the children finished working on these texts, the teacher proposed them to make a written synthesis of what they had learned about Athens. The children proposed to make a maquette of Athens instead. Moreover they asked to make a video presenting a television programm about Athens in an imaginary TV channel, which they called “Athens TV”. They intended to show the video to all the students and to the program monitoring comittee in Athens. The implementation of their project required the following steps: first they read again the four texts, then they wrote down what impressed them more and finally they chose what to put in the maquette. The construction of the maquette was done collectively during one course and involved seven children. Each child decided what to construct. He/she decided also to write a text about his/her construction. Their sources were pictures from the internet and a short video of the Acropolis Museum 6

For this issue see Askouni and Plexoussaki (2010) .

Alexandra Androussou, Evangelia Kourti and Nelly Askouni website (a visit simulation to the museum). Then the children decided how to produce the video and assigned roles for its realisation. One of them had to shoot the video and the others to present something about Athens. The presenters had also to write a text to be read in front of the camera. The video was shot in their classroom and the children appeared one after the other. On a desk, there was their maquette of the city of Athens. On the front side of the desk there is was sign indicating “Athens TV” and next to it, a Greek flag, which had been drawn by the children in the wrong sense. On the right side of the screen there was a map of Greece. The whole setting referred to the newscast. Each child presented one of the buildings of the maquette and read the small text he/she had prepared, with more or less fluency in Greek. They presented general information on Athens and the Acropolis Museum, the old mosque (now a museum), the subway, a block of flats in Athens, a hospital, a bakery and the Parthenon. The goals of the game achieved through this activity can be summarised as follows:

The game as motivation for learning: the texts functioned as an incentive to learn how to search for more information in the extracurriculum books of the library of the itinerant KESPEM and/or the internet. This was reinforced by the teacher/animator who created a permissive communication framework, did not restrict children’s imagination or penalize their errors (e.g. inverted flag, spelling errors, speech and writing errors, etc.), as would happen in the formal school, while promoting their playfulness and creativity.

The game as motivation to improve the use of Greek: in order to play this game chlildren are obliged to use Greek at different degrees of difficulty. Greek language becomes thus not only functional but also attractive, as it allows ‐ as is the case of Turkish‐ to create and imagine. Moreover, it introduces them to new worlds in a playful way, and to the discovery of new languages through the experience of technology and media (computer, video, internet).

The game as a means of creativity: the game allows the students‐players to transform and to explore in different ways the knowledge acquired through the texts (role playing, maquettes, videos etc). In this way, they are familiarized with the creative process as it is shown in what they constructed. The maquette of the city of Athens, can be considered as a metaphor of the virtual space in which they were introduced via the Internet, and their video production as an evidence of the knowledge acquired and their ability to create through technology.

Through the example of this activity one gains an insight into how this computer game, and the games in general, with appropriately designed technology, can set the creative and communication skills of children in motion and allow them to access a symbolic space where the unfamiliar can become familiar, and thus part of children’s identity.

7. Conclusion "The Chest that longs to be moved" functions as the basic canvas on which the relationship of children with learning processes is established. It is the link between the intervention team of the project and the children. Although the game is demanding because it uses only Greek, it becomes for the children a virtual environment that includes their experiences, their history, their own space. They can recognize themselves in the texts proposed and feel that their identity is respected. In this way Greek seems to become for them a more friendly language to which perhaps it is worth to be receptive. Beyond the improvement of Greek, the game aims to build trust and mutual acceptance in order to help children escaping from the "closed” world of a rural isolated community and be informed about and “participate” in the broader national and international community. In this perspective the game can also be considered as this "imaginary" space where there are bridges between the cultural elements of the minority and the majority allowing for the setting up of multiple identities, which are not yet visible in the public sphere. Up to the present day, the experience of the application of this specific game makes clear that the political dimension of the pedagogical choices is fundamental. It is obvious that the success of a serious educational game ‐especially when it targets socially vulnerable groups‐ depends on its design, which has to take into consideration more than just the rules of a “good” game. Games are not "neutral" educational tools. The pedagogical dimension adopted is always part of a wider political perspective (Freire 1998). In such specific situations ‐but also more widely‐ recognizing the diversity of the voices of learners allows the production of educational material and the development of processes that do not lead to the exclusion of the other. In this

Alexandra Androussou, Evangelia Kourti and Nelly Askouni perspective, serious games as educational materials are not only tools for learning but also means for the right to express the minorities’ voices.

Annex 1

Annex 2

Annex 3

Alexandra Androussou, Evangelia Kourti and Nelly Askouni

References Anagnostou, D. (1999) Oppositional and Integrative Ethnicities: Regional Political Economy, Turkish Muslim Mobilization and Identity Transformation in Southeastern Europe, Doctoral Dissertation, Cornell University. Androussou, A. (in press) Itinéraires identitaires des enfants de la minorité musulmane en Thrace par l’intermédiaire d’un jeu sérieux, Revue Synergies du Gerflint. Androussou, A., Askouni, N., Dragonas, Th., Frangoudaki, A and Plexoussaki, E. (2011) “Educational and Political Challenges in Reforming the Education of the Muslim Minority in Thrace, Greece”, The International Journal of Learning, Vol17, No. 11, pp. 227‐238. Apple, M. (1982) Education and Power. Routledge & Kegan Paul, London. Askouni, N. (2006) Minority Education in Thrace: The Process from Marginality to Social Integration, Alexandria, Athens (in Greek). Askouni N. and Plexoussaki E. (2010) Enquête auprès des familles appartenant à une minorité. Conflits politiques, distance culturelle et questions de méthode, in Tillard Bernadette, Robin Monique (dir.). La recherche au domicile des familles. Démarche et savoir‐faire. Paris : L'Harmattan, pp.97‐117. Baynham, M. (1995) Literacy Practices, Longman, London. Breuer, J. and Bente, G. (2010) “Why so serious? On the Relation of Serious Games and Learning”, Eludamos. Journal for Computer Game Culture. Vol 4, No. 1, pp. 7‐24. Cope, B. and Kalantzis, Μ., (2000) Multiliteracies. Literacy learning and the Design of Social Futures, MacMillan, South Yarra. Cummins, J. (1996) Negotiating Identities: Education for Empowerment in a Diverse Society, California Association for Bilingual Children, Los Angeles. Cummins, J. (2004) Language, Power and Pedagogy: Bilingual Children in the Crossfire, Multilingual Matters, Clevedon. Dragonas, T. and Frangoudaki, A. (2006) “Educating the Muslim Minority in Western Thrace”, Islam and Christian‐Muslim Relations, vol 17, No. 1, pp. 21‐41. Frangoudaki, A. (2008) Thrace is Changing: A Commentary on Prospects and Obstacles. In Dragonas,T. and Frangoudaki, A. (Eds.) Addition not Subtraction, Multiplication not Division: Reforming the Education of the Minority in Thrace. Metaihmio, Athens (in Greek). Freire, P. (1973) Pedagogy of the Oppressed, Seabury Press, New York. Freire, P. (1998) Teachers as cultural workers: letters to those who dare teach, Westview Press, Boulder, CO. Freire P. and Macedo P. (1987) Literacy: Reading the Word and the World, Routledge, London. Gee, P.J. (2003) What video games have to teach us about learning and literacy, Palgrave Macmillan, New York. Gee, P.J. (2004) Situated Language and Learning. A critique of traditional schooling, Routledge, New York. Giordan, A. (1998) Apprendre!, Belin, Paris. Kalantzis, M. and Cope, B. (1999) “Multicultural Education: Transforming the Mainstream”, in May, S. (ed.) Critical Multiculturalism: Rethinking Multicultural and Antiracist Education. Falmer Press, London. Mc Laren, P. (ed.) (1992) Postmodernism, Postcolonialism and Pedagogy. Albert Park Victoria, James Nicholas Publishers, Australia. Michael, D. and Chen, S. (2006) Serious Games: Games That Educate, Train and Inform, Thomson, Boston. Resnick, M. “Edutainment? No Thanks. I Prefer Playful Learning”, [online], MIT Media Laboratory DERGARTEN, http://llk.media.mit.edu/papers/edutainment.pdf Susi, T., Johanesson, M. and Backlund, P. (2007) Serious Games ‐ An Overview (Technical Report), University of Skövde, Skövde, Sweden. Yiagcioglu, D. (2004) From Deterioration to Improvement in Western Thrace: a Political System Analysis of a Triadic Ethnic Conflict, Doctoral Dissertation, George Mason University.

Transformational Play; Using 3D Game‐Based Narratives to Immerse Students in Literacy Learning Anna Arici and Sasha Barab Center for Games & Impact, Mary Lou Fulton Teacher College, Arizona State University (ASU), USA anna.arici@asu.edu sasha.barab@asu.edu Abstract: The philosopher and educator John Dewey (1938) supported a transactive view of schooling, where learners are active change agents rather than passive observers, and through their actions and consequences, they transform the problem into a known. Modern technologies now make his vision a reality, putting learners as active protagonists in their own learning, taking on authentic roles via avatars, and seeing the consequences of their actions played out in a 3D immersive world. The strength of this kind of game‐based learning is what we call Transformational Play; a 3‐fold theory that positions the person with intentionality, the content with legitimacy, and the context with consequentiality. Grounding this theory in context, we designed an educational 3D role playing game (RPG), school curriculum, and large‐scale comparison study in 18 seventh‐grade classrooms (N=450). This study demonstrates the positive impact of game‐based learning in a compelling population of disadvantaged students (Latino, Native American, poverty), who have new access to rich technology, as part of a 1:1 laptop initiative. These students, many of whom are second‐language learners, showed significant gains in literacy, persuasive writing and engagement in 2.5 weeks of gameplay in a 3D immersive narrative based on Mary Shelley’s Frankenstein. In this 3D curriculum, “The Doctor’s Cure”, students take on the role of an investigative reporter via their avatar, and complete a series of missions to uncover a moral dilemma involving Dr. Frankenstein’s work. As reporters, students actively collect evidence through interviews, build logical arguments to support their theses, submit these to an in‐game logic machine for evaluation, and get feedback about the alignment between their evidence and reasoning. Additional game tools and scaffolds allow students to act ‘a head above’ their current literacy capabilities (Vygotsky, 1978), while teachers play and provide feedback in their game‐character role as the ‘Editor’. With the goal of having equally engaging and novel experiences for both conditions, the control condition used the graphic novel curriculum ‘Frankenstein’. Measures of engagement (based on Csikszentmihalyi’s Flow, 1996) showed both conditions rated the experiences equally engaging. Further, we designed teacher‐led activities for the graphic novel to closely parallel the scaffolds in the game curriculum, and both conditions wrote and revised persuasive pieces. Despite these similarities, interesting and significant differences emerged. Students in both conditions showed significant learning gains on lower‐ level items identifying basic components of persuasive writing. However, the game‐based students scored significantly greater on the higher‐level task, requiring students to craft and compose their own persuasive essay from the ground up. Further differences emerged in the engagement measures and observational field notes. Qualitative analyses were used to unpack the quantitative findings, which illuminated the strength of in‐game tools for creating a fluency in these literary practices. The findings support the theory of Transformational Play and its potential for the classroom; that students can be scaffolded via games to engage personally and meaningfully in complex learning, that is experientially consequential and personally transformative.

Keywords: 3D RPG, schools, literacy, engagement, empirical study, transformational play

1. Introduction

Anna Arici and Sasha Barab The philosopher John Dewey (1938) supported a transactive view of schooling, where learners aren’t passive observers examining objects, but instead are active change agents, who through their actions and consequences, transform the problem into a known. Dewey launched the Laboratory School in Chicago in 1896 designed to have students actively ‘learn by doing’, breaking them out of traditional rows and rote learning, and allowing them to take on a role in their classroom community, to perform tasks, and to be productive. The Laboratory School was clearly ahead of its time theoretically, but failed to gain momentum due to the difficulty in creating individualized roles and tasks for students in the classroom. The basic theoretical underpinnings have stood the test of time, however, and many modern educators have struggled with how to make learning more student‐centered. Technology has finally caught up to Dewey’s vision, and we now have tools which make it possible to individualize learning and provide authentic tasks and roles to students, yet many schools still resemble those of 100 years ago. Enter any modern middle or secondary school and you will find students seated in rows, as passive recipients of teacher‐led learning, with technology that is all too often used in archaic ways (i.e., whiteboards used to list the day’s schedule, or laptops used mainly for typewriting and printing). We believe the point of modern technology in the classroom, and specifically well‐designed games, is to change education by removing the traditional boundaries that inhibit learning and perpetuate bad practices. Game‐based learning frees the learner to actively pursue experiences through sense‐making, exploration, testing assumptions, and seeing the consequentiality of their actions. Further, games give the learner a motive and motivation for pursuing this expertise, with immediate feedback on their strategies, strengths, and their own evolving identity (Gee, 2003; Shaffer, 2007; Squire, 2006; Steinkeuhler, 2006). As games gain popularity and acceptance in schools, it is important to differentiate between types of games, including the distinction between exogenous and endogenous games (Malone and Lepper, 1987,). Exogenous means ‘to originate externally’, and refers to games which form a wrapper or organization around academic content, as a means to making it more motivating and enjoyable, but without being related to the academic content. MathBlaster is a popular exogenous game, which is motivating for practicing mathematical problems, but the game mechanics of shooting aliens are unrelated to the math skills involved. Conversely, endogenous games incorporate learning and practicing skills as a strategy to accomplish game goals, and have a deep connection to the domain content by integrating relevant practices of the learning environment into the structure of the game (Halverson, 2005). Endogenous games have been shown to engage students in complex problem solving and meaningful transactive learning, however, many schools prefer exogenous games for their quick accessibility to skill practice, reviews, and its streamlined nature. While there is value in practice, well designed virtual games can go further to position the learner in meaningful situations, with the ability to take on authentic roles, and the agency to make decisions and see their outcomes, while redefining the learner’s sense of self through what they can do and who they can become. If properly designed, games can provide the problems, tools, people, experiences, perspectives, and consequences to ensure that learners develop rich content understanding (Arici, 2009, Barab, Gresalfi, & Arici, 2009).

2. Transformational play

Transformational Play is the theory by which we illuminate the strengths of game‐based learning, which positions the person with intentionality, the content with legitimacy, and the context with consequentiality (Barab, Gresalfi, Ingram‐Goble 2010). Learners who play transformationally become protagonists who use the knowledge, skills, and concepts of the educational content to first make sense of a situation and then make choices that actually transform the play space and the player—they are able to see how that space changed because of their own efforts. Such play is transformational in that it changes the context in which play is occurring, at the same transforming the player and his or her potential to interact with the world.

Anna Arici and Sasha Barab When creating learning environments to bring about transformational play, we attempt to position these elements in the following ways:

Person With Intentionality (positioning players as protagonists with the responsibility of making choices that advance the unfolding story line in the game)

Content With Legitimacy (positioning the understanding and application of academic concepts as necessary if players are to resolve the game‐world dilemmas successfully)

Context With Consequentiality (positioning contexts as modifiable through player choices, thus illuminating the consequences and providing meaning to players’ decisions)

3. Atlantis remixed: The doctor’s cure Based in this theory, and to further investigate its underpinnings, we created a 3D immersive role‐play game for learning literacy skills, called “The Doctor’s Cure”. This game is embedded within the Atlantis Remixed (ARX) Project, an international learning and teaching project that uses 3D virtual environments to immerse children, ages 9‐16, in educational tasks (the second generation of Quest Atlantis). Through interactions with in‐game mentors (non‐player characters, or NPCs) and by using in‐game tools to engage the academic content, the students are given the scaffolds and affordances necessary to take on the role of an expert in an authentic task and make influential decisions, which they see played out in their virtual world. “The Doctor’s Cure” (TDC) is a literacy game based on Mary Shelley’s novel Frankenstein, and set in a gothic world, where students take on the on the role of an investigative reporter via their avatar, and complete a series of missions to uncover a moral dilemma involving Dr. Frankenstein’s work. As reporters, students actively collect evidence through interviews and investigations, build logical arguments to support their theses, submit these to an in‐game logic machine for evaluation, and get feedback about the alignment between their evidence and reasoning.

Players are intentionally positioned as agents of change whose purpose is to help the village of Ingolstadt decide if they should allow "Dr. Frank" to keep looking for a cure in spite of his questionable research methods. Players soon learn that persuasive writing is a necessary tool to resolve the game’s narrative conflict. As the game progresses, players experience how their choices and use of persuasive writing dramatically change Ingolstadt, its citizens, and even the students’ own identity as a writer and leader.

3.1 In‐game tools In‐game tools provide support in the interrogation of texts, as well as a model for testing the logic of their argument, and immediate feedback in the process. One of the pedagogical scaffolds in the game is the ‘Lens of Lumination’ tool, which allows students to examine texts above their current reading level. The Lens of Illumination is designed to help the player engage in meaning making by “illuminating” the relevant claims in the documents, so players can decide if they want to collect that evidence. Once players have gathered what they think is a good collection of evidence to support their thesis, they visit Scoop Perry, and use his Persuasive Argumentation Tool (PAT), which displays all the evidence they have collected thus far. They then drag‐and‐drop their evidence to match their claims, and create a chain of logic to support their thesis. The immediate feedback and flexibility of the tool allows players to move elements around easily, repeat and test various logic orders. Once sound, players proceed with the game and write their

Anna Arici and Sasha Barab persuasive essay for the town newspaper, to convince the townspeople of their view. These tools are a small part of the much larger narrative.

3.2 Technology This project is based in an existing commercial 3D engine (Unity) to leverage a robust feature set with the possibility of cross platform deployment, and allow us to create native applications for both Windows and Mac platforms found in various schools. This gives the opportunity in the future to offer versions for tablets, other mobile devices and the web.

4. Study overview To test the theory of Transformational Play, we implemented The Doctor’s Cure game across a school district of 7th grade Literacy classes. This study compared a 3D gaming curriculum and context with that of a similar curriculum based in the graphic novel Frankenstein. The comparison study was a quasi‐experimental design of intact classes, with 8 classes assigned the control and 8 assigned the experimental conditions—about 450 total students with just over 400 completing both the pretest and the posttest.

4.1 Participants This school district, located in the southwestern United States and bordering on Mexico, is a fairly large district with more than 18,000 students, many disadvantaged, and have a demographic breakdown that includes 95% minority groups (87% being Hispanic), 83% low socio‐economic status, and 20% English Language Learners. Additionally, the community recently approved over $25 million dollars in bonds for technology upgrades, bringing one‐to‐one laptops to all students. Participants were 12‐13 years old.

Anna Arici and Sasha Barab

4.2 Measures Measures included traditional instruments, quantitative assessments, as well as ethnographic techniques. Learning gains were measured by a pretest and posttest, which were identical for both conditions, and counterbalanced within conditions. Questions varied from lower level multiple choice, to short answer, to a final essay writing question, where all students crafted their own persuasive essay with a given prompt. A rubric was created to analyze all open‐ended responses and two raters scored a subset of tests with an interrater reliability of alpha = .81. To quantitatively assess the engagement of the learner, a student self‐ report measure was gathered. This Engagement Questionnaire was based on that of Csíkszentmihályi’s (1990) study with ‘flow’, where he interrupted students involved in various activities to respond to their current state of engagement, motivation and challenge in the task at hand.

5. Results 5.1 Learning data A repeated measures ANOVA on the multiple choice/short answer test revealed a significant main effect for testing time, F(1,402)=46.25, p=.000, a non‐significant main‐effect for condition, F(1,402)=1.72, p=.191, and a non‐significant interaction, F(1,402)=.07, p=.793. Follow‐up analyses indicate that both the control, t(190)=5.11, p=.000, and the experimental conditions, t(205)=4.93, p=.000, improved significantly from pre‐ post, but there were no significant differences between groups with respect to the amount improved (see Figure 1a). In summary, while both the control (PreM = 8.24, PreSD = 4.09; PostM = 9.60, PostSD = 4.30) and experimental groups (PreM = 8.60, PreSD = 3.82; PostM = 9.91, PostSD = 4.17) demonstrated statistically significant learning gains on the multiple choice/short answer tasks with small/medium effect sizes (Con ES=.33, Ex ES=.33), there were no differences for group gains. See Figure 1, left side. A repeated measures ANOVA on the persuasive essay task revealed a significant main effect for testing time, F(1,353)=14.47, p=.000, a non‐significant main‐effect for condition, F(1,353)=.87, p=.350, and a significant interaction, F(1,353)=18.32, p=.000. Follow‐up analyses indicate that while the control, t(167)=.36, p=.72, did not have statistically significant learning gains, the experimental condition, t(187)=5.73, p=.000, improved significantly from pre‐post on the essay writing task (see Figure 1b). In contrast, while the experimental condition had statistically significant learning gains on the persuasive essay task from pretest (M = 3.65, SD = 1.24) to posttest (M = 4.27, SD = 1.30) with a moderate effect size (ES=.51), there was not a statistically significant improvement from pretest (M = 4.09, SD = 1.20) to posttest (M = 4.06, SD = 1.25) for the control condition. See Figure 1, right side.

Figure 1: Learning Gains. Left side is comparison of groups over time on multiple choice/short answer. On the right side is comparison of groups over time on essay prompt These data show that both the comparison (graphic novel) group and the experimental (game) group had significant learning gains from pre to post test on low‐level concepts related to literacy and persuasive argumentation. That is, both groups were able to identify and describe the basic elements of persuasive writing equally well. However, when looking at the final essay composition, requiring the highest levels of

Anna Arici and Sasha Barab thinking, application, synthesis and evaluation, the students in the game condition students scored significantly higher in their compositions. These findings lend support for Transformational Play, in that students who were actively engaged as first‐ person protagonists in the game narrative, with an authentic role and purpose for writing persuasively, did better in their writing than did those who had similar assignments based around a passive narrative in a graphic novel. Further, the game students had the contextual tools embedded in the virtual world to help them make sense of the evidence and create meaningful logic models. Students later demonstrated they were able to write persuasively even in the absence of these tools on the posttest, suggesting near transfer of these skills to a non‐game context.

5.2 Engagement data We collected engagement data using a modified version of Cziksentmihalyi (1993) flow’s instrument that consisted of 7 Likert‐type questions (e.g., did you enjoy what you were doing, was the activity challenging, were you succeeding at what you were doing) and had .77 internal consistent using Cronbach’s Alpha. Results showed that there were statistically significant differences between groups, t(125)=5.41, p=.000, with the experimental group (M = 3.86, SD = .62) outscoring the control group (M = 3.29, SD = .58) with a large effect size (ES=.95). Additionally, a Chi Square carried out on data in which students were asked: “What was your main reason for doing the task?” Results were significant, with the distribution being statistically different from chance, X2(121)=35.67, p < .001. In fact, 74% (49/66) of students in the treatment condition attributed that choice to “because I’m interested in the task,” as opposed to 22% (12/55) of the control. In contrast, 75% (43/55) of the control chose either “to get a grade” or “my teacher told me to,” while only 23% (17/66) of the treatment condition chose this option. These findings are summarized in Table 1. Table 1: Engagement Data. When students were asked “What is your main reason for doing this task?” a significant number of game‐based students attributed it to being intrinsically interested in the task, rather than doing it for an external grade or by teacher direction Attribution

Treatment (Game)

‘I’m Interested in task’

49 (74%)

Control (Graphic Novel) 12 (22%)

‘Teacher told me to do it’

2 (3%)

14 (26%)

‘To get a good grade’

15 (23%)

28 (51%)

‘Other’

0 (0%)

1 (1%)

5.3 Interview data Interviews and written feedback were collected from the students throughout the implementation, and at its conclusion. When asked to describe their experiences in this unit, these are comments that are representative of the most commonly offered from students. First, there were several reactions that were similar for both the Game and the Graphic Novel students. Both groups described the experience as:

‘Challenging’, ‘Engaging’, ‘Fun’, ‘I am learning a lot’.

‘It will help improve my writing’,

‘This is something I’d like to continue on my own’.

"When the story ended, I was shocked because I wanted to read more" (Graphic Novel)

"Once I finished the game I was like, "Aw, man!" I wish there was another part. It was cool and fun to play." (Game condition)

In fact, both conditions continued to access the games and novels after the study ended, and switched to participate in the activity they were not originally assigned. Next, there were some reactions that were more isolated to only the Game Condition students. The following are representative reactions to the game that differed from the graphic novel condition, with categorical interpretations in parentheses:

Anna Arici and Sasha Barab

“I’m surprised that they let us play this game at school!” (engagement)

‘The game involved a lot of reading and writing.” (challenge)

“I look forward to see what is happening next, and how it will turn out.” (consequentiality)

“My opinion about who is right and who is wrong has changed over time.” (multiple perspectives)

“I’m investigating the other side of the debate now.” (active participant, multiple perspectives)

"The people agree and disagree with the Doctor. It will be interesting to see how they act after my article is published!" (embodied role, multiple perspectives)

“"I like it because it gives you some feedback and you can go back to fix it." (engagement)

“The game helps me be much more persuasive, so I can convince the people [in the game] about what I’ve found [as an investigative reporter].” (embodied role, engagement)

“I’m looking forward to writing my essay so I can convince people that they should support the doctor!” (strong opinions, motivated to articulate)

Some felt strongly that the doctor should not experiment on the creature because he has rights. Others felt that the ends justified the means, and the cost of the creature was worth saving the thousands of others from the plague. Others took middle ground, suggesting alternative ways to find a cure for the plague.

Game classes were incredibly on‐task, and had little conversation going on that wasn’t related to the game. Many students were up and helping others, cooperating in game play, and celebrating when they were successful.

6. Qualitative lessons The quantitative and interview data are encouraging in that they demonstrate the learning value of the designs. Additional lessons learned were seen over time from our observational data, field notes, and ethnographic methods that further support and explain the quantitative findings, and lead to lessons learned at the level of students and of teachers, as they both made sense of this new game curriculum.

6.1 Students This initial work was based in Transformational Play theory, which involves positioning students as agents‐of‐ change who use real‐world knowledge, skills, and concepts to make sense of a situation and then make choices that actually transform the play space and themselves; creating a place in which what you know is directly related to what you are able to do and, ultimately, who you become. Toward that end, some core lessons include: 6.1.1 Engaging problem At the core of game‐based learning as leveraged in this study is that students find the problem and learning situation engaging. In The Doctor’s Cure, we set them up as a reporter who had to determine if Dr. Frank should be allowed continue with his ethically questionable experiments on the “creature.” Most kids liked this role, and talked about themselves as having the fate of the virtual town in their hands. 6.1.2 Pedagogical scaffolds The pedagogical scaffolds and in‐game tools worked very well in the Doctors Cure, as can be seen in the data. Students reported how much they loved the argumentation tool, and how it provided feedback on the quality of their evidence, and teachers felt similarly. In fact, teachers felt that this was the largest contributor to student success in argumentation. The ability for students to drag and drop ideas alleviated much of the language barrier present in this population of students, and they could focus on just the logic model itself. Similarly, the lens of illumination supported them in identifying the important arguments in the complex texts, although students appeared less engaged with the glasses than the argumentation tool—possibly because the glasses helped the player locate the ONE right answer, where the argumentation tool gave players feedback on their entire personal argument.

Anna Arici and Sasha Barab 6.1.3 Consequential feedback Clearly, one of the most engaging aspects of the experience for students was the consequential feedback they received around their choices. When players made a choice or performed a series of actions and the game responded, they would literally “hoot” out loud, or laugh and look around at other computers with a smile on their face. We saw numerous examples of students discussing different outcomes with each other in the classroom, hallway, or in some cases we heard about students still discussing choices at home. The feedback that teachers provided on the reports was less enthusiastically received, and we would have liked to see more rich discussion and interest in this feedback—although it was often not framed with having in‐game meaning or consequentiality. As the unit progressed, a number of students’ enthusiasm started to wane. We need to figure out better ways of providing reinforcement and successes to keep students engaged for longer periods.

6.2 Teachers Our theory of change around using games for impact, positions teachers as central to the learning process, with much of our design intended to support them by establishing a rich and engaging storyline for student learning and providing them meaningful places to provide feedback to students. 6.2.1 Professional development Teachers needed to be trained in how to use the system technically, but also in how to provide valuable feedback on student work, how to use embedded data in the system to support meaningful class discussions, and what is one’s role and responsibilities for successfully implementing a game‐infused curriculum. More generally, teachers needed ongoing support to engage the system successfully. Simply training them before implementation was inadequate, and we found that prompting teachers to review student work, to probe student understanding, and to provide just‐in‐time lectures was necessary. 6.2.2 Teacher dashboard

Consistent with our notion of games as a service, we wanted to provide a teacher toolkit that allows easy management of classes. Teachers really appreciated the Teacher Dashboard interface, saying it made it easy to register students, monitor student activity, and review student work. Further, some teachers were able to use it to identify decision biases across students to support meaningful in‐class discussions. One can think of the toolkit as a living dashboard that provides teachers up‐to‐the‐minute data, with seven tabs that teachers can access (roster, progress, reports, summaries, evidence, choices, resources), and a visual representation of each student’s progress. Teachers reported liking this immensely, however, very few teachers could share examples of how they used this in the class to monitor and guide student progress. Additional pre‐game training should focus on the ability to pair students needing help with another student further ahead, or pull students at similar progress points together for just‐in‐time small group lessons, and maximize the data driven decision making this tool offers.

Anna Arici and Sasha Barab 6.2.3 Teacher feedback

The Teacher Dashboard also includes a way for teachers to provide rich written feedback to all student submissions of work from the game, using a full text editor, and speaking from the role of the Editor, ‘Scoop’, in the game. Students receive this feedback in the game itself, embedded as something written to them from Scoop. While we were hoping to engage teachers in the rich narrative and playfulness of the game, we didn’t witness great examples of teacher roleplay. We found some teachers’ feedback to students when reviewing reports were more focused on grammar and spelling, rather than broader literacy concepts of persuasive argumentation. They also had difficulty “staying in character” by providing the feedback from the perspective of the in‐game editor, instead they used their own voice. More modeling and framing of these roles needs to be included in future training. While the Teacher Dashboard does allow teachers to manage the implementation, it also requires many new teacher responsibilities, and at no point does it validate teachers for their hard work, allow them to connect with other teachers, and most problematically provide a “public” showing of the quality of student work and their feedback. Clearly, building in more social network and validation (e.g., badges) for teachers is a priority for upcoming releases.

7. In conclusion The Doctor’s Cure was created and grounded in the theory of Transformational Play, leveraging the three interconnected elements of person, content, and context in immersive gameplay. This study explored the nuances of learning and engagement, when those three elements are supported, demonstrating significant learning gains at higher levels of authentic practice. Additionally, levels of engagement were increased as students took on the role of the protagonist in the story, and saw the consequences of their decisions, rather than reading about someone else’s experiences in a graphic novel. Qualitative findings uncovered additional strengths, such as the in‐game tools for providing practice and fluency in working through difficult logic models and complex texts. However, teachers need additional support to shift their pedagogy into a game‐based approach.

8. Extending impact and future direction Our broader design road map has a strong vision for extending the experience of these bounded games to larger impact, using what we refer to as Big “G” infrastructure. The Doctor’s Cure game is an example of a specific bounded experience, or Small “g” game, which is self‐contained and completable, pre‐optimized to introduce and cover a particular lesson. While these games provide rich immersive learning experiences, we need to extend learning beyond these into the community and real‐world applications via Big “G” games; an open‐ended infrastructure that seamlessly integrates the small “g” games into a larger, flexible ‘meta‐game’ structure and affinity space to foster user‐driven extensions and adaptations in support of real‐world goals and outcomes.

Acknowledgements We would like to acknowledge the many people and teams necessary to envision, design and implement a game curriculum of this scale. This would not be possible without the hard work of the team at E‐line Media’s Game Design Studio in Phoenix, The interns at Center for Games & Impact, Janis Watson, Michelle Capriles‐ Escobedo, Alex Cope, Dr. Manuel Isquierdo, Dr. Edwin Dawson, our cooperating school district teachers and administration, and hundreds of eager gamer‐students who played our games in school. This study was supported through generous support from the Bill & Melinda Gates Foundation.

Anna Arici and Sasha Barab

References Arici, A. (2009). Meeting kids at their own game: A comparison of learning and engagement in traditional and 3D MUVE educational‐gaming contexts. Unpublished doctoral dissertation, Indiana University, Bloomington. Barab, S., Gresalfi, M. and Arici, A. (2009) “Transformational play: Why educators should care about games”. Educational Leadership, 67(1), 76–80. Barab, S., Gresalfi, M., and Ingram‐Goble, A. (2010) “Transformational play: using games to position person, content , and context”, Educational Researcher. v39 i7. 525‐536. Csíkszentmihályi, M. (1996) Creativity: Flow and the Psychology of Discovery and Invention, New York: Harper Perennial, ISBN 0‐06‐092820‐4. Dewey, J. (1938). Experience & Education. New York, NY: Kappa Delta Pi. ISBN 0‐684‐83828‐1. Gee, J. P. (2003) What video games have to teach us about learning and literacy. New York: Palgrave. Halverson, R. (2005) What can K‐12 school leaders learn from video games and gaming? Innovate. 1(6). Malone, T. W., & Lepper, M. R. (1987). “Making learning fun: A taxonomy of intrinsic motivations for learning”, In R. E. Snow & M. J. Farr (Eds.), Aptitude, learning, and instruction: Vol. 3. Cognitive and affective process analysis (pp. 223‐ 253). Hillsdale, NJ: Erlbaum. Shaffer, D. W. (2007) How computer games help children learn. New York: Palgrave Macmillan. Squire, K. (2006) “From content to context: Videogames as designed experience”. Educational Researcher, 35(8), 19–29. Steinkuehler, C. A. (2006) “Massively multiplayer online video gaming as participation in a discourse”. Mind, Culture, and Activity, 13(1), 38–52. Vygotsky, L.S. (1978). Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University

Approaches to Collaborative Game‐Making for Fostering 21st Century Skills Susan Bermingham1, Nathalie Charlier2, Francesca Dagnino3, James Duggan1, Jeffrey Earp3, Kristian Kiili4, Evelien Luts2, Lien van der Stock2 and Nicola Whitton1 1 Manchester Metropolitan University, Manchester, UK 2 KU Leuven, Belgium 3 Istituto per le Tecnologie, CNR, Italy 4 Tampere University of Technology, Pori, Finland s.h.bermingham@mmu.ac.uk Nathalie.Charlier@pharm.kuleuven.be dagnino@itd.cnr.it j.duggan@mmu.ac.uk jeff@itd.cnr.it kristian.kiili@tut.fi Lien.VanDerStock@pharm.kuleuven.be n.whitton@mmu.ac.uk evelien.luts@pharm.kleuven.be Abstract: Many examples exist of the effective use of digital games for learning, both in the classroom and informally, for developing subject knowledge, skills (cognitive, (psycho)motor and psychodynamic), attitudes and behaviours. However, educational games are often limited in scope to the topic of the game itself and position learners as ‘players’ in the game space, rather than giving them control over the gaming environment. In fact, the increasing body of research literature suggests that making games could better address the needs of learners than just playing existing learning games. Collaborative game‐making provides a model in which learners can work together to create something that is meaningful for them, giving them input into both the process and product, and facilitating the development of a range of 21CS (21CS), such as digital literacy. Intuitive digital game‐making tools have become increasingly available in recent years, allowing students to directly access game‐making environments and support the growth in use of collaborative game‐making learning activities in schools. Making Games in Collaboration for Learning (MAGICAL) is an EU‐funded project that aims to explore the use of collaborative game‐making as a pedagogic model. It seeks to establish whether, and in what ways, the approach can support collaboration, problem‐solving, creativity and digital literacy skills. This paper starts by considering the literature on digital game‐making, particularly highlighting the benefits, drawbacks and research gaps. It then goes on to describe the MAGICAL project in more detail, particularly focusing on the way in which the 21CS can be defined, communicated to learners, and assessed. Next, the different approaches to collaborative game‐making in the classroom are discussed. The paper concludes by highlighting lessons learned from the project so far, and presenting recommendations for collaborative game‐making in the classroom. Keywords: collaborative learning, 21CS, collaborative game‐making, digital games

1. Introduction There is much evidence that digital games can be an effective tool for learning and engagement (e.g. Connolly et al, 2012; Petrotta et al, 2013), both in the classroom and informally, for developing subject knowledge, skills, attitudes and behaviours. However, educational games are often limited in scope to the topic of the game itself and position learners as ‘players’ in the game space, rather than giving them control over the gaming environment. In fact, the increasing body of research literature suggests that making games could better address the needs of learners than playing games. Collaborative game‐making provides a model in which learners can work together to create something that is meaningful for them, facilitating the development of a range of 21st Century Skills (21CS). Making Games in Collaboration for Learning (MAGICAL) is an EU‐funded project that aims to explore the use of collaborative game‐making as a pedagogic model. It seeks to establish whether, and in what ways, the approach can support collaboration, problem‐solving, creativity and digital literacy skills.

Susan Bermingham et al. This first considers the literature on digital game‐making, and then goes on to describe the MAGICAL project in more detail, particularly focusing on the way in which the 21CS can be defined, communicated to learners, and assessed. Next, different approaches to collaborative game‐making are discussed, and the paper concludes by highlighting lessons learned from the project so far, and presenting recommendations for collaborative game‐ making in the classroom.

2. Collaborative game‐making Researchers have proposed that the design and development of games could better address the needs of learners than simply playing existing games (Brennan & Resnick, 2012; Robertson, 2012). For example, Vos and colleagues (2011) described how making games, rather than playing games, better supported fifth and sixth graders’ use of deep learning strategies. The pedagogic idea of learning by making games assumes that the construction of games helps learners to reformulate their understandings of the subject and express their personal ideas and feelings about both the subject of the game and the games constructed (Kafai, 2006). Game‐making can also support the development of 21st century competencies like creative problem solving, collaboration, ICT literacy, systems thinking, and positively affect engagement in STEM subjects (e.g. Zimmerman, 2007; Clark & Sheridan, 2010). However, research has focused mainly on the motivational and adoption issues of the approach (e.g. Robertson & Howell, 2008, Owston, et al., 2009, Triantafyllakos et al., 2010). The design and development of games is creative teamwork, which is assumed to support reflective thinking and co‐construction of knowledge (Roschelle et al., 2000). However, a review of existing literature revealed that the social aspect of game‐making activities has been neglected. Collaborative game‐making is mainly considered in relation to helping other game makers, communicating ideas with others, and designing for others. Much collaborative game‐making research has been very narrow and superficial in nature. ‘Designing for others’ has been often utilized as a pedagogic strategy in game‐making studies (e.g. Kafai, Ching & Marshall, 1997; Baytak & Land, 2010; Owston et al. 2008). This demands that the students take the perspectives of the target group into account, which in turn may support the development of flexibility in students’ thinking (Kafai, 1997), enhance communication skills, and design thinking. For example, Kafai and colleagues (1997) conducted a study in which fifth and sixth graders made games for younger students. In the study students had to use a language that younger students would understand that encouraged older students to use their own words instead of just cut‐and‐pasting. Kangas (2010) showed that game playing and computer game‐making provided children (aged 7 to 12) with opportunities to practice their work‐group skills. Designing games can lead to an enhanced sense of classroom community, which encourages students to ask and questions and provide help for others (Baytak & Land, 2010) as well as to share tips and alternative ways of doing things in the game‐making environment (Robertson & Nicholson, 2007). Thus, game‐making activities provide opportunities to learn to be a better communicator and support the development of reading and writing skills as well as the use of spoken language and visual communication aids. Owston and colleagues (2008) compared fourth grade students’ reading and writing skills when they either developed or played trivia games. The development group outperformed player group in logical sentence construction but no differences were found in development of basic reading skills; but this is unsurprising because the measurement used favoured the development group. Nevertheless, there is some evidence that girls might benefit the use of game authoring activities that emphasize skills in writing narratives (Carbonaro et al. 2008; Robertson, 2012). For example, Robertson (2012) found that girls spent more time on writing conversations and utilized peer feedback more eagerly than boys. Creativity also can be successfully promoted by making games (Eow et al., 2010; Kangas, 2010). For example, Robertson and Howells (2008) argued that user‐generated game content can empower learners by enabling them to express their creativity. Providing resources such as different kinds of thinking tools and co‐creation approaches were found to be essential in promoting creativity (Kangas, 2010). Developing a game, whether individually or in collaboration with peers, is a very complex task and generally requires strong ICT skills. Tools to manage and support game‐making activities range from professional tools to those designed for children. In previous research, several different kinds of game‐making tools have been used, such as Adventure Author (Robertson, 2012), Gamemaker (Baytak & Land, 2010), Gamestar Mechanic

Susan Bermingham et al. (Torres, 2009), Scratch (Brennan & Resnick, 2012) and Neverwinter Nights (Robertson & Howells, 2008). Game‐making activities can vary a lot and tools need to be selected based on learning objectives and the skill levels of students. Scratch is a good example of a visual programming language designed for children who have no previous programming experience (Resnick, et al., 2009). The use of a visual programming language like Scratch develops students’ computational thinking skills (Brennan & Resnick, 2012; Denner et al., 2012). Other reported benefits related to ICT skills are systems thinking (Torres, 2009), programming knowledge (Kafai et al., 1997), interactive story authoring (Carbonaro et al., 2008) and storytelling (Robertson, 2012). However, only few studies reported gains in visual design (Robertson, 2012) or audience awareness skills (Robertson, 2012).

3. The MAGICAL project: Game‐making and 21st century skills The potential that collaborative game‐making presents as an innovative, student‐centred approach to learning is especially interesting for 21st century skills (21CS). This is the focus of a European project called Making Games in Collaboration for Learning (MAGICAL) 1 . The project’s mission is twofold: to identify, implement and share sound methods and tools for collaborative game‐making activities; and to investigate the impact, if any, on school students’ 21CS. MAGICAL foresees collaborative game‐making activities in primary and lower secondary school. These involve use of a digital environment for making and playing games, with support from specially trained teachers. The highly complex and dynamic nature of group‐based game‐making (and indeed of 21CS) poses considerable research challenges, calling for a pragmatic approach. Accordingly, MAGICAL is seeking to identify and monitor manifestations of 21CS and to investigate these as part of a case‐study approach. A literature review of 21CS carried out in the framework of the project (Dagnino et al., 2012) revealed the lack of consensus on what skills it covers or how best to classify them. Most authors adopt 21CS as an overarching concept covering a range of meta‐abilities and soft skills, social skills, attitudes, and aspects of self‐awareness and self‐reflection. These are complex in themselves, making a shared definition of 21CS that much harder to reach (Kickmeier‐Rust & Dietrich, 2012). The literature review also gathered a number of 21CS classifications and frameworks. Voogt & Pareja Roblin (2010) carried out a meta‐analysis that mapped six prominent 21CS frameworks and distilled these into a common 21CS meta‐framework comprising:

collaboration

communication

ICT literacy

social/cultural skills & citizenship

creativity

critical thinking

problem solving

productivity/development of quality products.

This is a high‐level reference framework comprising semantically loaded concepts which, at a lower level, present a number of overlaps, intersections and interdependencies. Nevertheless, it is a useful basis for building shared understanding. The limited literature on learning via digital game making does not devote much specific attention to 21CS (Earp et al, in press). Most empirical studies are short‐term and involve small sample sizes (e.g. Vos, Meijden & Denessen, 2010; Games, 2008). While some positive findings are reported, there is a tendency to rely heavily on conceptual research. A need for robust empirical research clearly exists, especially regarding the inherently 1

Co‐funded under the European Commission's Lifelong Learning Programme, transversal programme: KA3 (ICT) Multilateral Projects 519006‐LLP‐1‐2011‐1‐IT‐KA3‐KA3MP. http://www.magical‐project.net/

Susan Bermingham et al. social nature of game‐making activities, with their multiple skills and roles. Furthermore, only a few papers focus on assessment of learning processes and outcomes in game‐making contexts (Brennan & Resnick, 2012; Resnick et al., 2009) and assessment methods in game‐making remain unclear. MAGICAL’s efforts in these respects focus on some of the skills in Voogt & Pareja Roblin’s 21CS meta‐framework. Collaboration. MAGICAL proposes game‐making as a collaborative activity that involves multiple design and authoring roles. This is the project’s main innovation, as the few game‐making studies that consider collaboration do so in terms of peer review of individually produced games (Robertson, 2012). To support team‐based game‐making, MAGICAL is developing a special fantasy‐themed authoring environment called MAGOS in which the various functions and mechanics for making a game are represented as magical powers. These are distributed among up to four player/authors who initially have different areas of responsibility, e.g. game level design, implementation of physics, creation of visuals, integration of sound effects and music. The possibility to exchange ‘powers’ as required during real‐time editing (supported by online chat) is intended to encourage and facilitate cooperation, negotiation and, potentially, collaboration. Attention will focus on how collaborative processes unfold both in synchronous face‐to‐face mode and online, whether synchronously or asynchronously. Other important aspects here are the role of teacher mediation and the impact of MAGOS’ affordances, especially those designed and integrated specifically for the purpose of supporting collaboration. Creativity and problem solving. Game‐making brings a wealth of opportunity for addressing creativity in its various forms and manifestations (Eow et al., 2010; Frossard et al., 2011). MAGICAL adopts a holistic approach to creativity, considering it in terms of human qualities (Hague & Williamson, 2009), processes (Frossard et al., 2011), and the artefacts produced (Cropley, 2001; Amabile 1996; Plucker & Beghetto 2004; Villalba 2008). Drawing broadly on these perspectives, creativity can be considered to be manifest in the generation and implementation of meaningful, engaging and novel solutions in appropriate formats and for particular audiences. Accordingly, MAGICAL’s focus on creativity stretches beyond the aesthetics of learner‐produced games to embrace aspects like novel ideas, approaches and solutions, the capacity to draw original or unforeseen connections, and unorthodox or imaginative ways of dealing with limitations. A closely related 21CS that could well emerge from ‘learning by doing’ with collaborative game‐making is problem solving, especially in what Bransford & Stein (1993) define as the ‘problem‐solving cycle’: recognising and defining a problem, reflecting on and developing a solution strategy, gathering and analysing ideas, considering/trying alternatives, evaluating the implemented solution. These can be related to the cognitive processes that Mayer and Wittrock (2006) identify in problem solving, namely representing, planning, executing, and self‐regulating (evaluation of self‐efficacy and consequent adjustment). As a learning by doing approach, collaborative game‐making could offer learners opportunities to activate these processes within the problem‐solving cycle, especially when design and development is carried out in iterative phases. MAGICAL also considers ‘problem posing’, i.e. the capacity to design games and game situations that are challenging but solvable. Problem posing reflects attitudes like designing‐for‐others (Kafai, Ching & Marshall, 1997; Baytak & Land, 2010; Owston et al. 2008) and systems (or design) thinking, i.e. appreciation of system complexity and awareness of how individual elements interrelate within the whole (Hayes & Games, 2008). ICT literacy. In the field of education and digital technologies, 21CS often fall under headings such as digital competences, digital literacy, and ICT literacy (Council of the European Union, 2006; Ferrari, 2012; NETS Project & Brooks‐Young, 2007). The 21CS meta‐framework mentioned above distinguishes between the very broad concept of information literacy and the more specific area of ICT literacy. For project purposes, MAGICAL focuses on fluency and fruitfulness in technology use; application of existing ICT knowledge; and critical, iterative production of digital artefacts and application of design thinking. Communication, critical thinking and productivity are not investigated in MAGICAL as distinct 21CS in their own right but are considered within treatment of the other 21CS. Together with social/cultural skills & citizenship, these may be the subject of future investigation.

Susan Bermingham et al.

4. Collaborative game‐making in the classroom Making games requires a range of creative skills including problem finding, problem solving, evaluation and communication (Robertson & Nicholson, 2007). Because of its complexity, teachers who wish to implement game‐making in their classrooms might benefit from theoretical models supporting their teaching methods. Resnick’s (2007) spiral model for creative thinking, for instance, describes an iterative process of imagine, create, play, share and reflect. Learners imagine a new idea, use the technology to create it, play with their creation, and share it after completion with their friends/peers. By reflecting on their experiences of working on this project, they are able to generate new ideas, and so the cycle continues. A similar model has been suggested by Robertson & Nicholson (2007) who describe six creative stages of game design: exploration, idea generation, game design, game implementation, game testing and evaluation. These steps can be taken in order, but it is common for designers to return to previous stages as their ideas evolve. In addition to creative skills, an important aspect of games literacy is the ability to critically review games (Buckingham & Burn, 2007). Besides assessing existing games, in the context of classroom game‐making, this involves the ability to self‐ and peer‐evaluate games. Roberston (2012) noted that “the learner must decide whether the story is compelling, whether the visual design of the game is attractive, whether the player’s abilities are a good match for the level of challenge, whether the player is given sufficient autonomy as well as clear goals, and also check that the behaviour of the game is as intended under all possible conditions” (p. 386). Collaboration is an important aspect of constructing or making games. In a review of literature on collaborative game‐making activities in the classroom, only one study (Denner et al., 2005) was found in which games build by teams of at least two learners. In most studies, learners constructed their games individually and collaboration between learners was restricted to the evaluation process; in an intermediate or final phase of the game‐making process peers are asked to review or provide feedback for improvement to the designer. In other studies, exploring collaboration in a game‐based learning environment, collaboration was restricted to game‐playing, not game‐making. Denner and colleagues (2005) describe the Girls Creating Games (GCG) program, which is an after‐school and summer program for sixth‐ through eighth‐grade girls designed to address the barriers to girls' active participation in information technology. Students were given the role of designer by teaching them to program an interactive computer game. The program that Denner and colleagues designed uses four simultaneous strategies that build on key recommendations for overcoming barriers to and providing supports for girls' participation in technology. Three of these (identity‐forming activities excluded) might be directly applicable to collaborative game‐making: Game design and production. The participants were guided through five game‐building steps: requirements specification, user interface design, prototyping, prototype testing, and implementation. At first, they designed their games on paper from one scene to the next, which they then used as guides for their work on the computer. Pair programming. The participants worked in pairs, sharing time as ‘driver’ and ‘navigator’. The driver handles mouse and keyboard, while the navigator assists the driver by for instance reading from supporting documentation or providing social support for difficult tasks. Pair programming allows for guidance and constant review of keyed data by the navigator to identify errors, and it provides a structure for peer support through collaborative learning (Williams and Kessler, 2000). To prepare the participants in the GCG program, prior activities were organized away from the computer to help them develop their communication skills. Challenging stereotypes. Given the specific goal of the program to stimulate girls’ interest in IT, female technology students and professionals were brought in to challenge geek stereotypes by at once being tech‐ savvy and having well‐rounded lives. They provided both instrumental and psychosocial support.

5. Lessons learned from the MAGICAL project During the MAGICAL project, the project team has developed and evaluated a training programme for teachers and student teachers on the use of game‐making for developing transferrable skills. As the MAGOS environment described in section 3 has been in development during the initial phase of training, a range of different approaches to game‐building were used to test the training materials.

Susan Bermingham et al.

The creation non‐digital games;

Paper prototyping of digital games and apps;

Game development in pairs using the Sploder single‐user development environment.

Engagement was high in the workshop on designing and making a game and the training packages provided the stimulus to learn more. There is an apparent chasm between the training, which is designed to offer hands‐on experiences as a learner, and the jump to real classroom environments, with feedback such as: ‘how do I do this with my pupils?’, ‘how to translate this into a school environment?’, and ‘some ideas like building a game in a group seemed too complex for children to do in an hour’. The digital game based workshop using Sploder was well‐received, and trainees particularly enjoyed the show‐ and‐tell aspect of the session as they were invited to play others’ games. However, the link between the game authoring tool and curriculum content was for many too vague and the lack of confidence of some trainees was also evident. Trainees are on a packed training programme and are desperate for classroom strategies that they can ‘take off the shelf’ and use tomorrow. Timing of the training sessions was in the week before the commencement of the trainees main teaching placement, this may have added to trainee teachers unease with making links to the classroom as they lacked information about their school, the curriculum, and the pupils. From the literature discussed in this paper, and the experiences during the project, the following recommendations are provided. For collaborative game‐making activities in a learning environment:

It is necessary to give the design team plenty of time to play with the game‐making software and encounter some of the difficulties with it before moving on to design activities.

Before using game‐making software, design teams can be offered the possibility to design their game on paper first. By doing so, distraction due to software problems or lack of skills is limited.

When working on the same computer, each member of the team should be assigned a specific role, for instance driver or navigator. This might even be important when team members work together on different computers.

To support teachers to use collaborative game‐making:

Ensure that teachers are provided with lots of examples of how to use a game‐making environment in their own specific contexts.

Provide examples (e.g. via podcast or video) of game‐making in action.

It is important to provide detailed information on how to plan a lesson around game‐making, so a bank of concrete examples would be useful.

Some students will be confident with the game‐making software straight away, but others may need more support while they get used to the software.

The timing of the training is crucial to ensure that trainees have the opportunity to put what they have learned into practice quickly.

Acknowledgements (ITD) MAGICAL has been funded with support from the European Commission. This publication [communication] reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

References Amabile T. (1996). Creativity in context: Update to the social psychology of creativity. Boulder, CO: Westview Press. Barajas, Trifonova, Delli Veneri, Frossard, Mellini (Eds.). (2011) Games and Creativity in Education and Training. Naples: Fridericiana Editrice Universitaria. Baytak, A. & Land, S. M. (2010). A case study of educational game design by kids and for kids. Procedia Social and Behavioral Sciences, 2, 5242‐5246.

Susan Bermingham et al. Buckingham, D., & Burn, A. (2007). Game literacy in theory and practice. Journal of Educational Multimedia and Hypermedia, 16(3), 323‐349. Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. In Proceedings of the 2012 annual meeting of the American Educational Research Association, Vancouver, Canada. Carbonaro, M., Cutumisu, M., Duff, H., Gillis, S., Onuczko, C., Siegel, J., Schaeffer, J., Schumacher, A., Szafron, D., & Waugh, K. (2008). Interactive story authoring: A viable form of creative expression for the classroom. Computers & Education, 51, 687–707. Clark, K. & Sheridan, K. (2010) Game design through mentoring and collaboration. Journal of Educational Multimedia and Hypermedia. 19(2), 125‐145. Connolly, T., Boyle, E. a., MacArthur, E., Hainey, T. & Boyle, J.M. (2012) A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59 (2), 661–686. Cropley, A. J. (2001). Creativity in Education and Learning. London: Kogan Page. Denner, J., Werner, L., Bean, S. and Campe, S. (2005). The Girls Creating Games program: strategies for engaging middle‐ school girls in information technology. A Journal of Women Studies, 26(1), 90‐98. Denner, J., Werner, L., & Ortiz, E. (2012) Computer games created by middle school girls: Can they be used to measure understanding of computer science concepts? Computers & Education, 58, 240–249 Earp J., Dagnino, F.M., Kiili C., Kiili K., Tuomi, P., Whitton N. (In press). Learner Collaboration in Digital Game‐making: An Emerging Trend. Submitted to Proceedings of ATEE Winter Conference 2013. Eow, Y. L., Ali, W. Z. b. W., Mahmud, R. b., & Baki, R. (2010). Computer games development and appreciative learning approach in enhancing students’ creative perception. Computers & Education, 54(1), 146‐161. European Commission. (2007). Key Competences for Lifelong Learning — A European Reference Framework. Luxembourg: Office for Official Publications of the European Communities. Dagnino, F.M., Earp J. &Ott M. (2012) Investigating the "MAGICAL" Effects of Game Building on the Development of 21CS. In Proceedings of the in 5th International Conference of Education, Research and Innovations (ICERI), 5778‐ 5785. Madrid, Spain. Frossard. F., Barajas M., Alcaraz‐Domínguez S., Trifonova A. & Quintana J. (2011). GBL Design for Enhancing Creativity in the Classroom In M. Barajas, A. Trifonova, A. Delli Veneri, F. Frossard & B. Mellini (Eds.) Proceedings of Games and Creativity in Education and Training Conference (GACET 11). Napoli: Fridericiana Editrice Universitaria. Games, I. A. (2008) Three Dialogs: a framework for the analysis and assessment of twenty‐first‐century literacy practices, and its use in the context of game design within Gamestar Mechanic. E‐Learning, 5(4), 396‐417. Hayes, E., & Games, I. A. (2008). Making computer games and design thinking: A review of current software and strategies. Games and Culture, 3 (1) 309‐332. Kafai, Y. B., Ching, C. C., & Marshall, S. (1997). Children as designers of educational multimedia software. Computers & Education, 29 (2&3), 117‐126. Kafai, Y. B. (2006) Playing and Making Games for Learning: Instructionist and Constructionist Perspectives for Game Studies. Games and Culture, 1, (1), 36–40. Kangas, M. (2010). Creative and playful learning: Learning through game co‐creation and games in playful learning environment. Thinking Skills and Creativity, 5, 1‐15. Kickmeier‐Rust, M.D. & Albert, D. A (2012). Domain Model for Smart 21CS Training in Game‐Based Virtual Worlds. In Proceedings of the IEEE 12th International Conference on Advanced Learning Technologies (ICALT 2012), 680–681. Rome: IEEE. Mayer, R. E., & Wittrock, R. C. (2006). Problem solving. In P. A. Alexander & P. H. Winne (Eds.), Handbook of educational psychology (2nd ed.), 287–304. Mahwah, NJ: Erlbaum. NETS Project & Brooks‐Young S. (2007) National Educational Technology Standards (NETS) for Students, Second Edition. Washington, DC: International Society for Technology in Education. Owston, R., Wideman, H., Ronda, N. S. & Brow, C. (2008). Computer game development as a literacy activity. Computers & Education, 5, 977–989. Perrotta, C., Featherstone, G., Aston, H. & Houghton, E. (2013). Game‐based learning : latest evidence and future directions. Slough: NFER. Plucker, J.A. & Beghetto, R.A. (2004) Why creativity is domain general, why it looks domain specific, and why the distinction doesn’t matter. In R. J. Sternberg, E. L. Grigorenko, and J. L. Singer (Eds) Creativity: From potential to realization, 153– 68, Washington, DC: American Psychological Association. Resnick, M., Maloney, J., Monroy‐Hernandez, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B. and Kafai, Y. (2009) Scratch: Programming for All. Communications of the ACM, 52(11), 60–67. Robertson, J. and Nicholson, K. (2007). Adventure Author: a learning environment to support creative design. In Proceedings of the 6th international conference on Interaction design and children (IDC '07), 37–44. New York, NY: ACM. Robertson, J. (2012). Making games in the classroom: Benefits and gender concerns. Computers & Education, 59, 385–398. Robertson, J. & Howells, C. (2008). Computer game design: Opportunities for successful learning. Computers & Education, 50, 559–578. Roschelle, J., Pea, R., Hoadley, C., Gordin, D., Means, B. (2001) Changing How and What Children Learn in School with Computer‐Based Technologies. The Future of Children, 10(2), 76‐101.

Susan Bermingham et al. Torres, R. J. (2009) Using Gamestar Mechanic within a nodal learning ecology to learn systems thinking: A worked example. International Journal of Learning and Media, 1(2). Accessed from http://ijlm.net/knowinganddoing/10.1162/ijlm.2009.0016 on 14 May 2013. Triantafyllakos, G., Palaigeorgiou, G., & Tsoukalas, I. A. (2011). Designing educational software with students through collaborative design games: The We!Design&Play framework. Computers & Education, 56(1), 227‐242. Vos, N, van der Meijden, H., & Denessen, E. (2011). .Effects of constructing versus playing an educational game on student motivation and deep learning strategy use. Computers & Education, 56, 127–137 Villalba, E. (2009). Computer‐based Assessment and the Measurement of Creativity in Education. In Scheuermann & J. Björnsson (Eds.) The Transition to Computer‐Based Assessment: New Approaches to Skills Assessment and Implications for Large‐scale Testing, 29‐39. Luxembourg: Office for Official Publications of the European Communities. Williams, L.A. & Kessler, R.R. (2000). All I Really Need to Know About Pair Programming I Learned in Kindergarten. Communications of the ACM, 43, 108‐14. Zimmerman, E. (2007) Gaming Literacy: game design as a model for literacy in the 21st century. Harvard Interactive Media Review, 1(1), 30‐35.

Best Practices for Deploying Digital Games for Personal Empowerment and Social Inclusion Lizzy Bleumers1, Ilse Mariën1, Jan Van Looy2, James Stewart3, Dana Schurmans1 and Anissa All2 1 iMinds‐SMIT‐VUB, Brussels, Belgium 2 iMinds‐MICT‐UGent, Ghent, Belgium 3 JRC‐IPTS, Seville, Spain lbleumer@vub.ac.be imarien@vub.ac.be j.vanlooy@ugent.be james.stewart@ec.europa.eu dana.schurmans@vub.ac.be anissa.all@ugent.be Abstract: Digital games are being increasingly used in initiatives to promote personal empowerment and social inclusion (PESI) of disadvantaged groups through learning and participation. There is a lack of knowledge regarding best practices, however. The literature on game‐based learning insufficiently addresses the process and context of game‐based practice and the diversity of contexts and intermediaries involved in PESI work. This paper takes an important step in addressing this knowledge gap using literature review, case studies, and expert consultation. Based on our findings, we formulate a set of best practices for different stakeholders who wish to set up a project using digital games for PESI. The seven cases in point are projects that represent various application domains of empowerment and inclusion. Case studies were conducted using documentation and interviews, covering background and business case, game format/technology, user groups, usage context, and impact assessment. They provide insight into each case’s strengths and weaknesses, allowing a meta‐analysis of the important features and challenges of using digital games for PESI. This analysis was extended and validated through discussion at two expert workshops. Our study shows that a substantial challenge lies in selecting or designing a digital game that strikes a balance between enjoyment, learning and usability for the given use context. The particular needs of the target group and those that help implement the digital game require a highly specific approach. Projects benefit from letting both intermediaries and target groups contribute to the game design and use context. Furthermore, there is a need for multi‐dimensional support to facilitate the use and development of game‐based practice. Integrating game use in the operation of formal and informal intermediary support organisations increases the chances at reaching, teaching and empowering those at risk of exclusion. The teachers, caregivers and counsellors involved in the implementation of a game‐based approach, in turn can be helped through documentation and training, in combination with structural support. Keywords: game‐based learning, empowerment, inclusion, digital games, best practices

1. Introduction Empowerment, digital and social inclusion are important strategic policy goals of several EU policy agendas and initiatives. In light of these goals, different stakeholders are seeking to understand the opportunities of innovative ways of learning and participation as possible pathways to improve employability, health, well‐ being and civic engagement. Digital games, which are already being advocated as learning tools and which have become part of a participatory culture, seem promising candidates for empowerment and inclusion initiatives. The recognition of the dual nature of play as instrumental and fun (Schouten, 2011) has encouraged various stakeholders to consider digital games as an engaging means to induce change (knowledge and skill acquisition, attitudinal, behavioural or social change). In 2011, the Digital Games for Empowerment and Inclusion (DGEI) project was set up to identify opportunities and challenges for those seeking to harness the potential of digital games for empowerment and inclusion and to make recommendations for research and policy (Bleumers et al, 2012; Stewart and Misuraca, 2012; Stewart et al, 2013).

1.1 Key concepts In this study, we adopt a common interpretation of personal empowerment as a community‐supported process of individual change “whereby individuals achieve increasing control of various aspects of their lives

Lizzy Bleumers et al. and participate in the community with dignity” (Lord and Hutchison, 1993: 4). Social inclusion is closely related to personal empowerment, focusing more on increased participation or re‐integration in society (Notley and Foth, 2008; Wright and Wadhwa, 2010). This participation is multi‐dimensional and encompasses production, political, social, consumption and savings activity (Atkinson, 1998; Selwyn, 2002).

1.2 Empowerment and inclusion through game‐based learning and participation Digital games are used in a variety of sectors and for diverse purposes beyond entertainment (Sawyer & Smith, 2008). With regard to social inclusion, games are already being used in three domains: supporting disengaged and disadvantaged learners and enhancing employability, promoting health and well‐being, and fostering civic participation and community engagement. The common assumption underlying most of these initiatives is that games are motivational, learning and participatory tools. The way these tools are put to use varies whereby roughly three approaches can be distinguished: (1) using games that were specifically developed for learning and/or participation (special‐ purpose); (2) harnessing learning and participation in well‐designed commercial off‐the‐shelf games (COTS); (3) fostering learning and participation by (co‐)creating and modifying digital games. We need to be cautious of uncritical accounts of game‐based empowerment and inclusion, however. While we see that there is access to, interest in and usage of digital games among at‐risk youth in particular (Royle and Colfer, 2010; Karabanow and Naylor, 2010), game play in itself is a skill that requires mastery in which support might be needed (Jenkins, 2009). Intrinsic motivation to play games is not a given. Furthermore, interest in digital games may be highly specific (Ortiz, 2009). Ito and Bittanti (2010) have noted the presence of generational and socio‐economic divides in the more committed forms of game‐play that form pathways to interest‐driven learning.

1.3 Addressing knowledge gaps Reviewing academic research on game‐based learning and participation, we see a prevalence of studies addressing the usage and impact of games in formal learning settings such as schools and universities (e.g. Squire and Barab, 2004; Khaled, 2011). PESI interventions often take place outside mainstream education, however, in contexts where learning takes place through more general activities or even unintentionally. Here we find that there is a dearth of scientific study and evidence on the outcomes and conditions of game use, and the dynamics of innovation in game development and use. We also note a focus in the literature on the content and characteristics as deterministic of the effectiveness of a game. While acknowledging this as an important factor, there are many other factors that determine the impact of the use of a game on a population, such as training, availability of assessment tools, business model, institutional support and barrier, which appear to be handled and understood far less in academic literature. In this paper, we address these knowledge gaps. Through case studies, combined with literature review and expert interviews, and through expert workshops, we explore issues of use and impact, but also cover aspects of the implementation approach and processes such as the role of the intermediaries (e.g. social workers).

2. Case studies We conducted an analysis of game‐based PESI cases. The characterization of each of the cases allowed us to identify the main drivers and barriers encountered by the involved stakeholders.

2.1 Methodology Seven projects were selected that met a range of set criteria. First, cases needed to represent various application domains of empowerment and inclusion (namely, education and employability, well‐being, and civic and community engagement). Secondly, we looked specifically for well‐documented cases, providing us with the necessary information to do an analysis. Cases were also required to demonstrate interaction among various actors surrounding game usage (such as game developers and social support organisations) as our analysis was geared towards understanding game‐based PESI practices and their dynamic context. In this sense, our case studies are distinct from (game) content analysis. We did not conduct an in‐depth analysis of the games’ content, nor did we focus on the games per se. Finally, cases ideally also demonstrated some form of impact. Impact assessment practices were of interest to us both from a research perspective (i.e. methods

Lizzy Bleumers et al. used) as well as from an evidence‐based policy perspective (i.e. what is being done and where is support needed). In addition, cases were chosen to achieve diversity in terms of type of digital game usage (special‐purpose, commercial off‐the‐shelf, and game making), geographical origin, targeted group (general public, university staff, ill teenagers, etc.) funding (public, private, mixed), hardware and gameplay design. Finally, cases were not required to represent best practice, as insight in encountered obstacles would also be informative. Selected cases were based on the following games: 1. PING (Grin Multimedia): Adventure game aimed at raising awareness about poverty and social inclusion among young people (13 – 18y) 2. InLiving (Grassroots Learning): Role‐playing game aimed at promoting independent living among young (prospective) tenants (16‐25y) 3. At‐Risk (Kognito Interactive): Avatar‐based simulation game that enables faculty staff to identify and refer students in psychological and mental distress 4. Choices and Voices (PlayGen): Short role‐playing games that help school children between 12 and 18 years old to explore issues that might lead to tension and violence 5. Starbright world (Schematic, Userplane, & Starlight Children’s Foundation): Online social network that allows chronically ill youngsters (13‐20y) to express themselves and exchange experiences 6. CivWorld (Firaxis): Multi‐player strategy game for Facebook users that facilitates learning about Western History and encourages strategic thinking 7. Gamestar Mechanic (E‐Line Media & Institute of Play): Platform for playing, creating and sharing games directed at 8 to 14 year olds and their teachers via an online teacher community Each case represents a unique aspect of PESI. Together, the cases illustrate the diversity of the field, covering poverty awareness (case 1), independent living (case 2), depression and suicide prevention (case 3), violence prevention (case 4), community support for chronically ill (case 5), history education and strategic thinking (case 6), digital media literacy and motivating STEM (Science Technology Engineering Math) learning (case 7). To investigate these cases, an informational outline was drawn based on the literature and specific needs for this study. Areas of interest were project background and organisational structure (consortium, developer, funding), description (objectives, target audience, game format, technology, use context), and impact (effectiveness and reach). Next, we completed the outline for each case based on publicly available information (i.e. game websites, other online documentation, academic literature; see Bleumers et al, 2012) complemented by expert interviews when needed (n=3). Table 1 illustrates the diversity that characterizes the seven reviewed cases in terms of project background (i.e. initiation and partners involved), the context to which the games are introduced, and the way in which impact was assessed. Initiators who lay the groundwork for game‐based PESI initiatives vary from case to case. Intermediary organisations, game developers, researchers and foundations all appear as initiators. In most cases, partnerships are built to secure sufficient funding (i.e. a mix of private funding, grants and sponsorships), to obtain additional expertise and resources, and to distribute the games. Whereas some cases exclusively focus on school‐based use, the majority target a broader usage context including after‐school programmes and home usage. The most marked differences are evident for impact assessment. Cases differ in terms of the extent to which they were assessed, data collection methods (e.g. in‐game vs. out‐of‐game assessment), timing of assessment (e.g. longitudinal vs. single time‐point) and operationalization of impact (e.g. number of participants, game experience and/or learning). The last column of Table 1 shows the main drivers and barriers that we derived from the case studies. These key determining factors point us towards a first set of best practices for game‐based PESI initiatives: 1. Balance learning outcomes and a fun gaming experience: A positive game experience is crucial to ensure take‐up. Positive game experiences result in a higher level of perceived learning, and a higher level of motivation to participate. 2. Adopt a user‐driven approach to create game content, game play and to define valid outcomes: Multi‐ stakeholder involvement, including intermediaries and end‐users, throughout the design process taps into and

Lizzy Bleumers et al. acknowledges stakeholders’ various forms of expertise. Customizable game‐based approaches allow users to tailor the game experience to their local, particular needs. 3. Include a clear plan for publishing, marketing and distributing the game‐based approach: People at‐risk are often reached through intermediary organisations that guide and contextualize the use of digital games, promoting usage and the attainment of empowerment goals. Therefore, to ensure deployment and a sustainable impact, a detailed plan is needed on how to make the game accessible to them and to support them in implementing it (e.g. public prevention and educational programmes). In the case of Gamestar Mechanic, for instance, the involvement of a publisher provided partners with a much‐needed publishing strategy, while another partner created a guide for helping teachers implement the game in their classes. 4. Integrate assessment mechanisms in the game and for the overall project: Built‐in progress and assessment tools can support learning through personalized feedback. For example, a drawback of Choices and Voices is the lack of knowledge about its users’ actual learning curve and behavioural change. In addition, research evidence showing that digital games contribute to PESI can improve the attitude towards using them and hence, lead to an increase in funding, deployment and use.

2.2 Findings Table 1: Overview of the seven cases

5. Starbright World

4. Choices & Voices

3. At‐Risk

2. InLiving

1. PING

Project background Initiation: foundation and research institute Poverty organisations & schools Game developer Government Initiation: neighbour‐hood housing org. partnered with mobile game developer Business service provider, housing org. Government

Use context Primarily developed for classroom usage Can also be played at home (online or downloadable for free)

Impact assessment Pre‐release field‐tests Online data analytics Longitudinal data by way of pre‐, post and follow‐up surveys

Key factors + Teacher toolkit + Attention to fun aspect + Extensive user research ‐ Lack of marketing

Formal education system Informal social structures, (e.g. social housing)

In‐game questionnaire system

+ Participatory design + Embedded in (in)formal intermediary support structures ‐ Maintenance: game developer went bust

Initiation: games and simulations developer partnered with mental health association Health service providers Government

Formal learning context, namely high schools, colleges and universities

Built‐in progress and assessment tools Longitudinal data by way of pre‐, post and follow‐up surveys

+ Customization enables aligning with local needs + Assessment built‐in ‐ Top‐down approach focused on access ‐ Limited play time

Initiation: Game developer co‐designed game with Police unit University Government Local schools

Formal educational context, i.e. classroom

Qualitative impact assessment of experience and perceived usefulness Structured group discussions

+ Multi‐stakeholder + Teachers as guides + Integration in educational curricula ‐ Lack of specific data on actual learning & change

Initiation: Children’s foundation supported by industry Interactive agency Integrated messaging platform provider

Informal context: hospital, at‐home, anyplace anywhere

Pre‐ and posttest Pilot with 9 replicated single system designs

+ Multi‐dimensional platform + Programme recognition + Extensive user research + Collaboration with e.g. hospitals for distribution

7. Gamestar Mechanic 6. CivWorld

Lizzy Bleumers et al. Initiation: Game company Middle school students

At home At school In after‐school programmes

None (anecdotal, user led)

+ Exploit commercial distribution + Community of practice ‐ Lack of assessment ‐ Simplified game system = less play time, less system thinking

Initiation: researcher & game developer Foundation Designer & Publisher of game‐based learning products & services

At school In after‐school programmes In community centres or libraries

Discourse‐based design + Sustainable publishing ethnography strategy + Focus on transferrable digital skills

3. Expert workshops Two expert workshops were organised to identify challenges and opportunities for game‐based PESI initiatives and to pinpoint priority issues for this sector.

3.1 Methodology Experts were selected based on literature review and snowball sampling, as there is no clear, pre‐existing community of experts working in the field of digital games and social inclusion. We invited both gaming experts and experts in PESI application domains such as migrant integration and remedial education from academic, industry and NGOs, ensuring a geographic and gender balance (Stewart et al, 2013). The procedures followed in both workshops were modified versions of the Technology Foresight expert panel methods (JRC‐IPTS, 2007). In fields where there is little documented practice or clear research pathways, this methodology enables identification of the state of knowledge, and primary issues, challenges and opportunities through interactive discussions with experts (Scapola and Miles, 2006). It supports building consensus and helps validate a research policy agenda in a short period of time. In each workshop, we introduced the project goals, the policy interest, and the project findings. Each participant presented his or her work briefly after which a common vision(s) of digital games for empowerment and inclusion was developed, and challenges and opportunities for meeting those visions were identified. The first workshop (in January 2012, 34 participants) involved three exercises. In the first exercise, a role‐ playing brainstorming technique was applied whereby participants explored two social inclusion scenarios from the perspectives of different stakeholders (i.e. the use of a digital game to instruct first‐time tenants, and another on migrant integration support). This laid the groundwork for a second exercise, during which participants brainstormed about challenges and opportunities regarding the use of digital games for empowerment and inclusion. Resulting issues were then visually arranged by volunteers from the group, and subsequently reanalysed by the research team to identify clusters of issues. In the third exercise, participants worked in groups to delineate a 'Big Issue' to be tackled. In addition to articulating further opportunities of using digital games in PESI projects, they developed narratives and/or storyboards for a particular use case. As part of this exercise, they were encouraged to identify key intermediaries and their motivations, resources, and structural situation. They reflected on commercial and policy implications, and finally established time frames in which these scenarios could occur and the risks that would be involved. The second workshop (in October 2012, 27 participants) aimed to validate earlier findings and build consensus as to priorities and potential actions between stakeholders from industry, practice, research and policy. Drawing from earlier project findings and participants’ experience as presented and discussed during this workshop, participants selected by vote the three priority issues out of seven identified, and reflected on potential actions to address these issues, who would be involved in those actions, and their timeline.

Lizzy Bleumers et al.

3.2 Findings During the first workshop, experts identified three key societal challenges where integrating digital games could benefit PESI projects: support for migrant integration, for marginalised young people, and for improving the quality of life of older people. They identified over a hundred opportunities and challenges. Table 2 highlights key findings in this respect. Table 2: Highlighted opportunities and challenges for game‐based PESI projects as identified in Workshop 1 Opportunities Potential of new mass market technologies to reach excluded populations, particularly through mobile devices High and growing acceptance of games and gaming, in some target groups such as young people Strength of games to support customized learning through multiple mechanisms Strong potential of games‐based approaches to re‐ engage disengaged learners, inside and outside of formal education Games allowing people to experience failure safely Exploitation of commercial off‐the‐shelf games and usage of game‐making techniques Using games and gaming to improve efficiency and effectiveness of intermediaries

Challenges Low awareness and negative stereotypes of game use Lack of knowledge, skills and institutional support among PESI stakeholders Low quality and/or sustainability of many game‐based PESI projects, often due to funding scheme, lack of expertise, or lack of team‐based design The lack of human and institutional capacity to develop game‐based PESI projects and distribute special‐purpose game products Lack of impact assessment tools and lack of evidence for effectiveness Knowledge gaps in how games can be used effectively and in how to navigate the context of use Lack of people trained in the development and use of digital games

Good practice was identified as programmes and projects that address the challenges listed in Table 2, in particular by: (1) providing impact assessment and assessment tools to support decision makers and users to recognise the value of game‐based approaches, and (2) engaging professional intermediaries such as teachers and social workers in the design, implementation and support of game‐based PESI initiatives in a sustainable form, rather than through short term, one‐off research‐based projects. Significantly, stand‐alone special‐ purpose games were never seen as the solution to the issues in the workshop scenarios. Rather, games and play elements were seen as tools for improving services linked to employers, social services and informal education services. Drawing on this, the second workshop identified the following priority issues: (1) the need to address the image of games and challenge stereotypes, (2) the lack of evidence of impact conditions for effective use of game‐based approaches and the importance of developing methods for gathering evidence and demonstrating impact, and (3) the low level of development of the supply sector and high barriers to distribution and use. These workshops provided many pointers towards issues that need to be addressed by good practice in game‐ based PESI development and use, particularly around building sustainable production and use models, engaging intermediaries in design, providing evidence and training, and tackling structural barriers to game use.

4. Best practices Based on the findings discussed in the earlier sections, we can provide an outline of best practice in game‐ based PESI initiatives. What appears essential for the successful usage of games for empowerment and inclusion, overall, is a multi‐stakeholder, project‐based approach. We will first discuss this approach and then posit a set of requirements that can serve as guidelines for these projects from funding to follow‐up.

Lizzy Bleumers et al.

4.1 Project‐based and multi‐stakeholder approach Social exclusion can be understood as a multi‐dimensional and dynamic process, situated in a community context, encompassing civic, economic, social and interpersonal disintegration (Phipps, 2000). To appropriately and effectively address the complex nature of this issue, an integrated set of various forms of expertise, tools and activities is needed. As a consequence, the usage of digital games should be conceived of as being one part of a multi‐stakeholder and project‐based approach, where the success of the project critically depends on each facet of the project context, not merely the qualities of the game. The cases we reviewed clearly illustrate this. Central to PESI projects are the people at risk themselves. While it may seem logical to see at‐risk groups primarily as adopters, i.e. the target or end‐users whose situation one seeks to improve, this is a reductive interpretation. It fails to acknowledge the possibility that people at risk can shape PESI projects by acting as representatives of their group throughout the project. To ensure participation of those at risk, intermediary organisations play a major role. Through the trust relationship intermediaries maintain with at‐risk groups, they are well aware of the struggles, skills and interests of this audience. Intermediaries can signal the demand for (game‐based) PESI initiatives, or act as domain experts in these projects. They are gatekeepers that can introduce a game‐based initiative to people that might otherwise be hard to reach, as well as guide, motivate and facilitate the empowerment of participants in these initiatives. To achieve their fullest potential, intermediary organisations themselves need support, as they often lack relevant assets and capabilities such as knowledge on how to implement games in their everyday practice, game design and publishing expertise, financial resources, and so on. An exchange of resources can be made possible by arranging partnerships between these organisations and game developers, publishers, research and funding organisations. Whereas game developers have the means to create engaging games, researchers can support evidence‐based game design. They also have the expertise to monitor and evaluate PESI initiatives. This has the added benefit of not needing to place intermediaries in the role of assessor, potentially compromising the relationship they have with their target group. Actors with publishing expertise can provide the know‐how needed to market and scale up a local PESI initiative. Funding organisations can contribute to a budget not only for game development, but also for manuals on how to play and deploy the games, impact assessment, and project follow‐up. This requires that each of these components were already considered in the project plan, which we will discuss in the following section.

4.2 Requirements for game‐based PESI projects Identifying the key components to the successful implementation of game‐based PESI initiatives is a work‐in‐ progress. Here we propose a set of requirements, drawn from our study’s findings. Successful game‐based PESI projects address concrete needs of both end users and professional intermediaries. User research (incl. domain analysis and UX research) prior to game development facilitates such needs alignment. In addition, creating a game concept and outline should take place in close collaboration with these actors. Finally, foreseeing customizability of the game‐based approach enables users to adapt it further to their preferences later on. By giving this stage in the development process sufficient attention, drastic and costly changes during actual production can be avoided. It is advisable to make a highly detailed game design document and development plan so as to keep the production time low. While a positive game experience is important for take‐up, it does not guarantee successful diffusion of the game. Again strong partnerships are needed whereby experts in effective publishing cooperate with intermediaries who work with at‐risk groups to adequately approach people at risk. A single‐shot strategy is inadvisable. A broad, multichannel communication strategy stretching across different use settings and over a period of time is likely to be more effective. To ensure the game‐based PESI initiative is not abandoned, additional requirements have to be taken into account. A sustainable approach requires that the quality of the game‐based platform and related services are assured and that updates can be made initialized when user needs change. Integrating the game as a tool, together with technical, pedagogic and/or agogic support, to make existing intermediary services more

Lizzy Bleumers et al. effective, will also contribute to the longevity of the initiative. Finally, assessment tools (built‐in and external to the game) need to be available so that stakeholders can be informed of usability and playability of the digital game as well as short‐ and long‐term impact on learning, empowerment and social inclusion. Clearly these requirements cannot be met without a sound financial plan that takes into account all aspects of setting up and carrying out game‐based projects including research, creation, marketing, implementation and follow‐up. Public‐private partnerships can be set up to secure mixed funding so that high‐quality and sustainable projects can be delivered. However, it is difficult to justify funding for game‐based approaches in the absence of evidence of what works and how. For evidence on the meaning and influence of game‐based PESI projects to grow, multi‐level impact assessment will need to become standard. Assessment tools, research support and platforms for sharing project experiences can facilitate this process.

5. Final reflection: Knowledge transfer and service scaling Having defined requirements that are likely to contribute to the effectiveness of game‐based PESI projects, the question arises how successful projects can be scaled up and transfer knowledge to other initiatives. To conclude, we briefly summarize the opportunities discussed in Stewart et al (2013: 117‐120): (1) institutional actors can demonstrate their work, (2) developers and publishers can distribute packaged games including training material in multiple language versions, re‐customize games and practices and foster a community of players and game‐makers, (3) intermediary organisations can share good practice and build up local expertise and finally, (4) individuals can provide training.

Acknowledgements This paper is based on the results of an exploratory and interdisciplinary study on Digital Games for Empowerment and Inclusion (DGEI) which was jointly funded by DG CNECT and JRC‐IPTS under the Administrative Agreement AA INFSO/SMART 2011/0054 – JRC 32397 ‐ 2011. The Digital Society Department at iMinds (former IBBT) were subcontracted by IPTS to develop the state of play review of digital games for empowerment and Inclusion that makes up a substantial part of this analysis. The authors thank the experts interviewed for the case studies: Mitra Memarzia (free‐lance artist and educator), Brian Alspach (E‐line Media) and Seann Dikkers (University of Wisconsin, Gaming Matter) and the participants in the expert workshops that were held in Seville and Brussels. They also acknowledge the contribution of their colleagues in organising the workshops, developing and reviewing the reports produced throughout the DGEI study (see http://is.jrc.ec.europa.eu/pages/EAP/eInclusion/games.html#publications)

References Atkinson, A.B. (1998) “Social Exclusion, Poverty and Unemployment”, in Atkinson, A.B. and Hills, J. (Eds.), Exclusion, Employment and Opportunity, CASE paper 4, London School of Economics, London, pp 1–20. At‐risk (2008) Kognito Interactive, [Browser game], http://www.kognito.com/products/faculty. Bleumers, L., All, A., Mariën, I., Schurmans, D., Van Looy, J., Jacobs, A., and Willaert, K., De Grove, F. and Stewart J. (Ed.) (2012) State of Play of Digital Games for Empowerment and Inclusion: A Review of the Literature and Empirical Cases, JCR Technical report No. 25652 EN, Publications Office of the European Union, Luxembourg. CivWorld (2011) Firaxis, [Social network game], http://www.facebook.com/civworld. Gamestar Mechanic (2010) E‐line Media, [Browser game], http://www.gamestarmechanic.com. JRC‐IPTS (2007) “Online Foresight Guide to Expert Panels”, [online], JRC‐IPTS, Sevilla, http://forlearn.jrc.ec.europa.eu/guide/4_methodology/meth_expert‐panel.htm. InLiving (2008) Grassroots Learning, [Mobile game], http://inliving.aniac.co.uk. Ito, M. and Bittanti, M. (2010) “Gaming”, Hanging Out, Messing Around, Geeking Out: Kids Living and Learning with New Media, MIT Press, Cambridge, MA, USA, pp 195–242. Jenkins, H. (2009) Confronting the challenges of participatory culture, John D. and Catherine T. MacArthur Foundation Reports on Digital Media and Learning, MIT Press, Boston, MA, USA. Karabanow, J. and Naylor, T.D. (2010) “‘Being hooked up’: Exploring the experiences of street youth and information technologies”, The International Journal of the Humanities, Vol 5, No. 3, pp 253–260. Khaled, R. (2011) “Equality = Inequality: Probing Equality‐Centric Design and Development Methodologies”, in Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P. and Winckler, M. (Eds.), Human‐Computer Interaction – INTERACT 2011, Lecture Notes in Computer Science, Springer Berlin Heidelberg, pp 405–421. Choices and Voices (2009) PlayGen, [Browser game], http://playgen.com/choicesandvoices/. Lord, J. and Hutchison, P. (1993) “The process of empowerment: Implications for theory and practice”, Canadian Journal of Community Mental Health, Vol 12, No. 1, pp 5–21.

Lizzy Bleumers et al. Notley, T. and Foth, M. (2008) “Extending Australia’s digital divide policy: an examination of the value of social inclusion and social capital policy frameworks’”, Australian Social Policy, Vol 7, pp 87–110. Ortiz, J.A. (2009) “Re‐gaming the digital divide: Broadband, MMOGS and US Latinos”, iConference 2010 Proceedings, University of Illinois, Urbana‐Champaign, IL, USA, 7p. Phipps, L. (2000) “New communication technologies ‐ A conduit for social inclusion”, Information, Communication & Society, Vol 3, No. 1, pp 39–68. Poverty Is Not a Game (2010) Grin Multimedia, [PC Game and Browser game], http://www.povertyisnotagame.com. Royle, K. and Colfer, S. (2010) The breadth and scope of computer games in learning: Applications to 14 to 19 learners with a specific focus on applicability to those who are classified as Not in Employment, Education or Training (NEET) (Research report), Centre for Developmental and Applied Research in Education (CeDARE) & BECTA. Sawyer B. and Smith P. (2008) “Serious Games Taxonomy”, Presentation at Serious Games Summit GDC, San Francisco, CA, February, http://www.dmill.com/presentations/serious‐games‐taxonomy‐2008.pdf. Scapola F. and Miles I. (2006) Eliciting Experts' knowledge: A comparison of two methods. Technological Forecasting and Social Change, 73, pp 679‐704. Schouten, B. (2011) The Role of Play, Inaugural lecture, Eindhoven University of Technology. Selwyn, N. (2002) Defining the “Digital Divide”: Developing a Theoretical Understanding of Inequalities in the Information Age (Occasional Paper No. 49), Cardiff University, Cardiff. Squire, K.D. and Barab, S. (2004) “Replaying History: Engaging Urban Underserved Students in Learning World History through Computer Simulation Games”, Proceedings of the Sixth International Society of the Learning Sciences, Santa Monica, CA, pp 505–512. Starbright world (2006) Schematic, Userplane, & Starlight Children’s Foundation, [Social network site] [Browser game], http://www.starbrightworld.org. Stewart, J. and Misuraca, G. (2012) A Roadmap for Action on Digital Games for Empowerment and Inclusion in Europe, DGEI Project Deliverable, JRC‐IPTS, Sevilla. Stewart, J., Bleumers, L., All, A., Mariën, I., Schurmans, D., Van Looy, J., Jacobs, A., Willaert, K., De Grove, F., Misuraca, G. and Centeno. C (2013) The Potential of Digital Games for Empowerment and Social Inclusion of Groups at Risk of Social and Economic Exclusion: Evidence and Opportunity for Policy, JRC Scientific and Policy Reports No. EUR 25900 EN, Publications Office of the European Union, Luxembourg. Wright, D. and Wadhwa, K. (2010) “Mainstreaming the e‐excluded in Europe: strategies, good practices and some ethical issues”, Ethics and Information Technology, Vol 12, No. 2, pp 139–156.

Investigating the Relationship Between School Performance and the Abilities to Play Mind Games Rosa Maria Bottino, Michela Ott and Mauro Tavella ITD‐CNR, Genova, Italy bottino@itd.cnr.it ott@itd.cnr.it tavella@itd.cnr.it Abstract: Is there any relationship between school performance and the ability to play digital mind games? This paper tries to answer this key question and in doing so, it draws on a long‐term research experiment conducted in primary schools and dealing with the use of mainstream mind games (namely those games that deeply require the enactment of thinking and reasoning skills and are almost independent from knowledge/competence in specific school subjects). It reports on an experiment involving 60 Italian primary school children, which was based on the use of the LOGIVALI Test, a game‐based standardized test assessing primary school pupils reasoning abilities. The games adopted in the experiment were five digital mind games (mostly public domain products) falling into the category of “mini‐games”; some of them were the computerized versions of well‐known board games (e.g. battleship, master mind, domino). The main characteristic common to all the adopted games was that they do not require specific prerequisites in curricular school subjects, beyond very basic literacy and, most importantly, do not imply the possession of specific mathematical skills. During the experiment, the possible relationships between gaming and learning performance of primary school students were investigated; a strong correlation between the students’ possession of the reasoning skills necessary to successfully play with mind games and their school performance was found. These considerations corroborate the hypothesis that games exercise a set of specific reasoning abilities that are “transversal” to most curricular activities. The targeted experiment also showed that the great majority of students (including low achievers), independently from the level of their school performance, are very attentive and engaged in game‐based learning tasks. These findings, together with the results of other experiments carried out by the authors in different frameworks (but with the same target population and with the same games), concretely support the idea that early interventions to support the development of reasoning abilities carried out by means of engaging and motivating game‐based activities can positively impact on students school performance. In a proactive perspective, the obtained results corroborate the idea that a carefully designed, teacher‐ driven and well‐focused use of specific mind games can contribute to sustain and foster students’ reasoning and problem solving skills and that these skills may have, in the long run, a positive impact on the students’ global school achievement. Keywords: mind games, transversal skills, game‐enhanced learning, technology enhanced learning, primary education

1. Introduction Digital games are broadly regarded as technologies offering a high potential to foster and support learning (Sandford et al, 2006; Prensky, 2001; Hong et al, 2009; De Freitas & Oliver, 2006; Pivec, 2007). Game‐based learning refers to teaching‐learning actions carried out in formal and/or informal educational settings by adopting games. It encompasses the use of both games designed expressly for fulfilling learning objectives (educational games) and "mainstream games" ‐‐ i.e. those games that are not developed for education (i.e. for fun) when used to pursue learning objectives (Kirriemuir and McFarlane, 2004 p.19). To date a wide number of significant research studies have been carried out that look, from different perspectives, at the actual relationships among different types of digital games and specific learning objectives to be met (Mc Farlane et al, 2002; Mitchell & Savill‐Smith, 2004). This paper focuses on mind games, namely those games (elsewhere called brainteasers or puzzles Kebritchi et al., 2010; Milovanović et al, 2009) that deeply require the enactment of thinking and reasoning skills. The paper draws on a research experiment dealing with the use of mainstream mind games in formal educational settings with the aim of fostering primary school pupils’ reasoning abilities. Such abilities are defined as “transversal” or “key” and it is widely recognized that they underpin the majority of learning tasks, thus sensibly contributing to enhance global academic achievement (Prensky, 2005; Schiffler, 2006; Carbonaro et al, 2010). The use of digital mind games to support the development/enhancement of transversal abilities and, in particular, those involving reasoning and logical thinking, is still scarcely explored (Rohde & Thompson, 2007) but some authors claim that the use of such games can contribute to enhance school achievement (Robertson & Miller, 2009; Franco et al, 2011). In the following, drawing on a field experiment involving 60 Italian primary school children we investigate the relationships between the possession of the reasoning abilities required to solve mind games and the school performance at primary school level.

Rosa Maria Bottino, Michela Ott and Mauro Tavella

2. Background This paper reports and discusses the results of a field experiment conducted in twenty Italian primary schools classes (60 children in 4th and 5th grades) in the Lombardia region. Actually, the experiment was based on the use of the LOGIVALI (LOGIcal thinking eVALuatIon) Test, a game‐based norm referenced test assessing the reasoning abilities of primary school pupils (Bottino et al, 2010). In the following before presenting in details the experiment and the related methodology, we briefly summarize the basic concepts underpinning the LOGIVALI test.

2.1 The Logivali test The LOGIVALI test is a game‐based, norm‐referenced test that follows a custom set‐up, specific methodology to investigate and assess the possession of some specific logical and reasoning abilities. The test is grounded on the use of five digital mind games (Bottino et al, 2010) that fall into the category of “mini‐games”, that is “games that take less than an hour to complete” (Prensky, 2005). The five games adopted in the LOGIVALI test (Table 1) do not require specific prerequisites in curricular school subjects, beyond very basic literacy and, most importantly, do not imply the possession of specific mathematical skills; some of the games are the computerized versions of well‐known board games (e.g. battleship, mastermind and domino). Table 1: LOGIVALI games: screenshots and short description

Pathological

Tree Tent

GMC Master Mind

The Pathological game is a puzzle game consisting of marbles and wheels. The marbles are of different colours and roll along paths. Each wheel has four slots which are to be filled in with marbles of the same colour: to do so, the user clicks on the wheel to rotate it. When the wheel is completed, the marbles vanish and the wheel turns dark. Pathological is organized in progressive difficulty levels that correspond to different schemas. In order to complete each schema it is necessary to take into consideration not only the colour of the ball that is coming but also the colours of the next ones (left‐hand side of the screenshot). http://pathological.sourceforge.net TreeTent is a single‐player puzzle game in which the goal is to work out where the computer has positioned a certain number of tents on a board where a certain number of trees appears. A tent can only be found in a cell horizontally or vertically adjacent to a tree, tents are never placed adjacent to each other, and for each tree only one tent can be positioned on the board. The numbers along the right and lower sides provide a clue as to the number of tents in that row or column. The player can make inferences on the content of each square by placing either a tent or grass (light green colour) in the cell. When this is done, a small pellet appears in the square. Clicking on the tick in the toolbar validates the player’s inferences: if the guess is correct the small pellet disappears, otherwise the system provides an error warning. http://www.yoogi.com/treetent.htm The goal of MasterMind is to guess a sequence of coloured pegs that the computer has selected at random. The player start making guesses by filling the holes at the bottom row of the left‐hand column with coloured pegs chosen from the range available in the top‐left corner. Each time a row is completed, the program gives feedback on the attempt in the right‐hand column: a black peg means that the player has correctly positioned a peg of the right colour, while a white one means that a peg of the right colour has been chosen but is not in the right place. The feedback provided by the program does not reveal which individual peg colours/positions are correct within each attempt. At each attempt, the player need to process all the feedback received up to a given point in the game in order to decide what to do next. http://gmoerth.freeservers.com/mm/index.htm

Rosa Maria Bottino, Michela Ott and Mauro Tavella Tetravex is a domino‐like game which is played with tiles that are divided into four triangles each marked with a number from 0 to 9. The goal of the game is to drag the tiles in the right‐hand board into the left‐hand board and position them so that any two adjacent tile numbers are the same. http://live.gnome.org/GnomeGames

Tetravex

Hexip

Hexip is single‐player game whose goal is to find the position of ships within the hexagon‐shaped board (dark boxes contain ships, light boxes are empty). The game provides information on the number of boxes occupied by ships both on the horizontal and diagonal rows of the board (the numbers outside the hexagon). The player can make inferences on the content of each box by colouring it either with light colour or with dark colour, in this case a small pellet appears in the box. Clicking on the tick in the toolbar validates the player’s inferences: when an attempt has been validated, the small pellet disappears if the inferred content is correct; otherwise the system provides an error warning. http://www.yoogi.com/hexip.htm

The overall administration methodology of the LOGIVALI Test includes the following steps:

Teachers explain the games to the students.

Students play individually with five digital mind games; each game is used twice, in two different one‐hour playing sessions to be held in the school computer lab.

Teachers monitor students at work during the playing sessions, but abstain from intervening with suggestions and help.

After completing the two playing sessions with each game, each student individually takes a specific test on that game (sub‐test). Tests are administered by the teachers who also are in charge of making students aware of the fact that no curricular evaluation is foreseen for the tests. Each sub‐test is composed of eight exercises, each containing multiple items in the form of multiple‐choice questions (Serradell‐Lopez, 2010) or, when possible, practical drills (e.g. “fill in the schema with the needed moves”).

The test takes into account a limited number (six) of specific reasoning abilities identified as “crucial” (Bottino et al., 2007) although they are, obviously, only a subset of the abilities required to deal with the games at hand. The six abilities investigated through the LOGIVALI Test are:

Ability 1 “knowing the rules of the game”: to know the rules underlying a given game and to be able to apply them in concrete game situations.

Ability 2 “first level reasoning”: to be able to make an inference taking into consideration a single given datum.

Ability 3 “second level reasoning”: to be able to make an inference taking into consideration two given information or game constraints.

Ability 4 “third level reasoning”: to be able to make an inference taking into consideration more than two given information and game constraints.

Ability 5 “managing uncertainty”: to be able to establish whether the data available at a given moment of the game are sufficient to decide whether a certain guess is correct or not.

Ability 6 “operatively applying reasoning abilities”: to be able to implement into the game the results of own reasoning (actions should follow thoughts).

As to the ability of making inferences, in the LOGIVALI test, it is not considered a unique monolithic skill but it is regarded and treated as an ability varying in difficulty according to the number of different data, information

Rosa Maria Bottino, Michela Ott and Mauro Tavella and constraints to be dealt with. In particular, the test differentiates the abilities to make inferences according to the need for taking into account one, two, and three or more data (abilities 2, 3 and 4). On the basis of the results obtained with the involved sample population (around 500 primary school students) the test was standardized and the reference norms were defined on a percentile ranking basis. In (Bottino et Al., 2010) the whole LOGIVALI test validation and standardization procedure is presented. During such procedure, it emerged that the number of correct answers progressively decreased in the three stages: Ability 2 (80% of correct answers); Ability 3 (63% of correct answers); Ability 4 (46% of correct answers). This confirmed that the three abilities should be considered as increasingly complex abilities for the target population.

3. Methodology As mentioned above, for the purpose of the present study 60 pupils out of twenty Italian primary schools classes (4th and 5th grades) in the Lombardia region were selected and monitored, while individually playing with digital mind games. The students were classified in three groups according to their school achievement (school actual results based on the scores obtained, plus teachers’ judgments):

group A‐high achievers;

group B‐medium achievers;

group C‐low achievers.

The target group of 60 students comprised three students per class: one for each group A, B and C. The LOGIVALI test was administered during school hours to all the students in each classroom and, in each class, the students chosen as representatives of the three groups, were individually monitored by teachers. Detailed results were recorded (score obtained by each student subdivided by sub‐test and by entailed ability, plus attribution of the position in the “percentile” scale) and carefully analyzed. In order to monitor students at work, teachers were provided with “made on‐purpose” monitoring sheets where they were asked to record relevant data on each student’s behavior, in particular on their autonomy, engagement and attention. The available data for each student thus consisted in:

Scores at the LOGIVALI test (global score and individual scores in each single sub‐test)

Percentile analysis of performance

Qualitative data on: autonomy, engagement and attention

For the scope of this study, all the data available through the LOGIVALI test were analyzed and compared with each student’s data on school performance. It was thus possible to make some inferences on the possible correlations between students’ performance in playing/solving mind games and that attributed in school tasks. In order to obtain a complete view of each student’s profile, behavioral aspects (derived from monitoring sheets analysis) were also considered and the correlations with the above mentioned performances were further studied. The research approach adopted in this study basically followed the paradigms of “mixed research methodology” (Burke et al., 2007; Burke & Onwuegbuzie, 2004); in fact, a combined use was made of both quantitative and qualitative approaches. Actually, as said above, the performance of each student at the LOGIVALI test was assessed according to the percentile ranks determined by the test; subsequently, the obtained quantitative data on performance at the LOGIVALI test were compared with available data on each student’s level of school achievement. Data recorded by teachers in the monitoring sheets and referring to students’ autonomy, engagement and attention were, instead, analyzed by employing qualitative research methods, in particular the “direct observation” research method (Herbert, 1995).

Rosa Maria Bottino, Michela Ott and Mauro Tavella

4. Results The ultimate aim of the conducted experiment was that of understanding if, at early school levels, digital mind games can be used to elicit and support the development of those reasoning abilities that underpin learning tasks that students are likely to face throughout all their school careers. This hypothesis, if confirmed, could pave the way for well‐targeted and effective interventions using games to sustain the development of reasoning skills, starting from the early stages of school education. Having in mind this ultimate objective, in the following we outline the key results of the experiment that shed light on:

the actual relationship between the students’ performance at the LOGIVALI test and their school achievement;

the students’ attitudes during game play.

4.1 The relationship between school achievement and ability to play mind games The results of the test lead to the key finding that: A substantial consistency exists between school achievement and the ability of playing mind games, as demonstrated by the students’ performance at the game‐based LOGIVALI Test. In particular: Looking at the global performances at games (overall results of the LOGIVALI test, encompassing the results of the five sub‐tests, we see that performances of high and low achievers are very different ( in favour of high achievers) while the medium group is situated half way between the two Results proposed in Figure 1, where dark parts stand for low performances (actually 2 levels of low performances: poor and fair) and light parts for high performances (actually 2 levels of high performances: very good and good) show that:

Only students in group A (high achievers) show prevailing high performances at the LOGIVALI test (light parts in the figure) with respect to the other two categories where dark parts are prevailing; more than half of the students of this group perform at the two highest levels.

The very majority of the students in group C (low achievers) perform at low levels (fair and poor); most performances of these students are at the lowest level.

Performances of students in group B (medium achievers) are comparable to those of students in group C if we consider dark and light parts but performances at the fair level prevail on performance at poor level.

Figure 1: Performances of the three groups of students at the LOGIVALI Test As to the ability of recalling the game rules (Ability 1) the performance of the three groups are scaled. As shown in Figure 2, very good performances decrease sensibly from Group A to group C. If we look at the two big blocks (dark and light parts) we see that high achievers perform far better with respect to the other two groups, whose performance is still comparable, but with a prevalence of very poor performance for low achievers. This is true despite what emerged from the LOGIVALI Test standardization procedure, namely that

Rosa Maria Bottino, Michela Ott and Mauro Tavella Ability 1 is the simplest one among the abilities tested by the LOGIVALI test and that it involves lower cognitive load. Ability 1

Figure 2: Performances of the three groups at the ability of recalling the game rules As to the inferential abilities (core abilities 2, 3 and 4 of the LOGIVALI test) we see that performances of high, medium and low achievers are always scaled (with the prevalence of good performances for high achievers and of poor performances for low achievers). This emerges from Figure 3 where the results at the three abilities of the test related to the capacity to make inferences on the basis of given data (namely Ability 2, 3 and 4) are presented. Ability 2

Ability 3

Ability 4

Figure 3: Performances of the three groups at the three “core” abilities considered, regarding inference The figure shows that in all the three abilities poor performances prevail in group C in all the three abilities and vice versa good performances prevail in group A while medium achievers are always in between the two. While looking at Figure 3 nevertheless we must consider that the three graphs should be regarded singularly and no direct comparison among the three graphs representing the three different abilities can be done. The structure of the available data doesn’t allow us to directly compare the performances of the three groups of students in the different abilities: our data are, in fact, based on the calculation of the percentiles which varies in the different abilities (treated as independent) as a consequence of the different number of correct answers given by the sample population. Despite this, if we look singularly at the considered reasoning abilities, we see that high achievers always perform better than the other two groups. The emerging trend is the same for all the six considered abilities, despite reasonable differences linked to the intrinsic level of difficulty of each one of them, as illustrated below. As to the ability of “managing uncertainty" (Ability 5) performances of students in Group C are lower with respect to those of students in the other two groups; only Group C show genuinely poor performances while high and medium achievers performances are comparable .

Rosa Maria Bottino, Michela Ott and Mauro Tavella This ability, namely the ability to be able to establish whether the data available at a given moment of the game are sufficient to decide whether a certain guess is correct or not, clearly discriminates among the three groups: it appears to be very difficult for low achievers (Figure 4), so that only very few of them perform well. Both the other two groups, instead, appear to be more confident with the task.

Ability 5

Figure 4: Performances of the three groups at the ability of “managing uncertainty" As to the ability of "operatively applying reasoning abilities": (Ability 6), performances of students in Group A are higher with respect to those of students in the other two groups. Differences between group B and Group C are found as well, in favour of Group B. This ability implies "operatively applying reasoning abilities" namely to be able to implement into the game the results of own reasoning (in the logic that actions should follow thoughts). Here we need to observe that Group A performance is far higher with respect to the one of the other two groups (Figure 5): this could allow arguing that students in this group are more able to “put into practice what they have learnt” (and this attitude/ability could be responsible for their good achievement in curricular activities). Ability 6

Figure 5: Performances of the three groups at the ability of “"operatively applying reasoning abilities"

4.2 Affective aspects of mind game playing: attention, engagement, autonomy During the experiment, issues related to children’s behaviour such as engagement, attention and motivation were also investigated. This was done by means of made on‐purpose monitoring sheets where the teachers and the members of the research group, while observing students at work, carefully recorded data on their attitudes considering a number of common specific indicators, to guarantee comparability. The first attitude considered was autonomy; in this respect, as shown in Figure 6, it was noticed that, while students in the high and medium achievers group were substantially autonomous most of those in the low achievers group frequently required help and advice although only few of them were considered non‐ autonomous at all.

Rosa Maria Bottino, Michela Ott and Mauro Tavella

Figure 6: Level of autonomy shown by pupils in the three groups The examination of the monitoring sheets showed also that all pupils, were fundamentally attentive and engaged while playing (Figure.7); this was true also for a consistent number of low achievers, irrespective of their actual gaming ability and capacity to reach the solution.

Figure 7: Level of attention and engagement shown by pupils in the three groups

These data are confirmed by a parallel investigation conducted, with the same modality and methodology, on a sample of 27 pupils identified by teachers for low academic achievement and learning difficulties. This study (Dagnino et al, 2013) revealed that the 70% of the group had a good level of attention while up to the 83% of the children showed a good level of engagement in the proposed tasks. These data confirm that a considerable amount of children of the target age are attentive and engaged in the interaction with mind games, despite their level of achievement, even those that need adequate support by educators to reach the game solution.

5. Discussion The final aim of the described experiment was to investigate whether digital mind games can be used successfully to elicit and support the development of those “transversal” reasoning abilities that affect global school achievement. To this aim, evidence was found that:

A substantial consistency exists between primary school students’ school achievement and the ability of playing digital mind games.

The great majority of students, independently from their level of school performance, are basically very attentive and engaged in game‐based learning tasks.

Rosa Maria Bottino, Michela Ott and Mauro Tavella As a matter of fact, from the conducted experiment it emerged that:

The performances at games are scaled and reflect the results of high, medium and low achievers.

The ability itself of recalling games’ rules is more present in high achievers and progressively at a lesser extent in the other groups of students.

Inference tasks are well performed by high achievers but represent a sensible obstacle for low achievers (mainly when inferences need processing more numerous data).

The ability of verifying the work done by also managing uncertainty and establishing whether the data available at a given moment of the game are sufficient to decide whether a certain guess is correct or not is peripheral for the group of low achievers.

The ability to put into practice what learnt during exercises is almost a prerogative of high achievers

The level of autonomy in playing games is good for both high and medium achievers while low achievers need some external help.

Digital games attract all students’ attention and stimulate their engagement and motivation.

Since the experiment was based on games that do not require knowledge/competences specific to curricular school subjects (e.g. arithmetic, linguistic), the obtained results suggests that playing with mind games entails transversal skills that are involved also in curricular tasks and activities (e.g. making inferences, verifying the work done, recalling rules, etc.). Furthermore, digital games are suitable tools to exercise such skills in that they are well accepted by students and able to stimulate their positive attitude towards learning. The actual effectiveness of game‐based training programmes in the field of logical abilities is substantiated also by the findings of a previous research experience conducted by the authors (Bottino et al, 2007) that showed a positive impact of the gaming activity on both school performance and learning behaviour. The experiment, in fact, had evidenced that a group of students who had undergone a game‐ based reasoning and logical training with games for a long period (three school years) performed better at a national math test with respect to a control group. The teachers involved in the experiment had also reported a positive impact of the gaming activities on the general learning behaviour of the students involved in the experiment. Based on all these results, we could reasonably draw the conclusion that mind games could be adopted profitably in early educational programmes targeting the development and the enhancement of “transversal” reasoning skills and that they may have, in turn, a positive impact on the students’ global school achievement.

6. Conclusions and future work This paper is based on an in‐field experiment conducted in primary schools and based on the use of digital mind games. The experiment has showed that a strong correlation exists between school achievement and the ability to play digital mind games even if these games do not require knowledge/competences specific to curricular school subjects. Moreover the experiment has showed that during gaming activities all children were attentive and engaged, irrespective of the difficulties they encountered. These findings, in the light of results from related experiments, suggest that, at early school levels, mind games can be well accepted, suitable and effective tools to trigger and foster those reasoning abilities that are brought into play in curriculum‐wide learning experiences. As a consequence it can be argued that early interventions in the field of reasoning abilities carried out by means of game‐based activities can positively impact on students school performance (mainly in the areas of maths and logical reasoning). The reported experiments were, nevertheless, exclusively targeted to primary schools; further in‐depth investigations seem to be required in order to be allowed to generalize and extend the emerging considerations and assumptions. In particular, the question whether the abilities and skills acquired at this stage can actually be transferred to other school levels and/or curricular areas, remains open and asks for further related long‐term experiments.

Rosa Maria Bottino, Michela Ott and Mauro Tavella

References Bottino R. M., Ott M., Tavella M. and Benigno V.: Can digital Mind Games be Used to Investigate Children’s Reasoning Abilities? In: Meyer B. (ed.) Proceedings of the 4th ECGBL Conference on Games Based Learning Copenhagen, pp. 31‐ 39. Academic Conferences (2010) Bottino, R.M., Ferlino, L., Ott, M., Tavella, M.: Developing strategic and reasoning abilities with computer games at primary school level. Computers & Education, 49 (4), 1272‐1286 (2007) Bottino, R.M., Ott, M.: Mind games, reasoning skills, and the primary school curriculum: hints from a field experiment. Learning Media & Technology, 31 (4), 359‐375 (2006) Burke, R. J., Onwuegbuzie, A. J. (2004). Mixed Methods Research: A research paradigm whose time has come, Educational Researcher, 33(7), 14‐26 Burke, R. Johnson, Onwuegbuzie, A. J. & Turner, L. A. (2007). Toward a Definition of Mixed Methods Research, Journal of Mixed Methods Research, 1, 112‐133 Carbonaro, M., Szafron, D., Cutumisu, M. and Shaeffer, J.: Computer‐game construction: a gender neutral attractor to Computing Science. Computers & Education, 55 (3), 1098‐1111 (2010). Dagnino F.M, Ballauri M., Benigno V., Caponetto I., Pesenti E. (2013). Reasoning abilities in primary school: A pilot study on poor achievers vs. normal achievers in computer game tasks. Learning and Individual Differences 23 pp.213‐217 De Freitas, S., Oliver, M.: How can exploratory learning with games and simulations within the curriculum be most effectively evaluated?. Computers & Education 46, 249‐264 (2006). Franco C., Mañas, I., Cangas A.J., and Gallego J.: Exploring the Effects of a Mindfulness Program for Students of Secondary School. International Journal of Knowledge Society Research (IJKSR): 2 (1), 14‐28 (2011) Herbert J., 1970. Direct observation as a research technique. Psychology in the Schools, 7 (2), 127‐138 Hong, J.C., Cheng C.L., Hwang, M.Y., Lee, C.K., Chang, H.Y.: Assessing the educational values of digital games. Journal of Computer Assisted Learning, 25, 423‐437 (2009). Kebritchi M., Hirumi A., Bai H.: The effects of modern mathematics computer games on mathematics achievement and class motivation. Computers & Education 55 (2), 427‐443 (2010) Kirriemuir J., McFarlane A.: Literature Review in Games and Learning. Report 8: Futurelab Series. (2004). http://archive.futurelab.org.uk/resources/documents/lit_reviews/Games_Review.pdf Mc Farlane, A., Sparrowhawk A. & Heald, Y.: Report on the educational use of games: an exploration by TEEM of the contribution which games can make to the education process. Teem, Cambridge, (2002) Milovanović, M., Minović, M., Kovačević, I., Minović, J., Starčević, D.: Effectiveness of Game‐Based Learning: Influence of Cognitive Style. In: Lytras, M.D., Ordonez de Pablos, P., Damiani, E. Avison, D., Naeve, A., Horner, D. G. (eds.) Best Practices for the Knowledge Society. Knowledge, Learning, Development and Technology for All. Communications in Computer and Information 49 pp. 87‐‐96. Science Springer, Heidelberg. (2009) Mitchell, A. and Savill‐Smith, C.: The use of computer and video games for learning. A review of the literature. Learning and Skills Development Agency, Ultralab, M‐learning (2004) Pivec, M.: Play and learn: potentials of game‐based learning. British Journal of Educational Technology, 38, 387‐393 (2007). Prensky, M.: Digital game‐based learning. McGraw‐Hill, New York (2001) Prensky, M.: Educational Games Complexity Matters: Mini‐games are Trivial ‐ but “Complex” Games Are Not. An important Way for Teachers, Parents and Others to Look At Educational Computer and Video Games, Educational Technology, 45 (4), 1‐15 (2005) Robertson, D., Miller, D., Learning gains from using games consoles in primary classrooms: a randomized controlled study. Procedia Social and Behavioral Sciences, 1 (1), 1641‐1644 (2009) Rohde, T. E., Thompson, L.A., Predicting academic achievement with cognitive ability Intelligence 35 (1), 83‐92 (2007) Sandford R., Ulicsak M., Facer K., Rudd T., Teaching with Games. Futurelab report. http://archive.futurelab.org.uk/projects/teaching‐with‐games. (2006). Last accessed 10/05/2013 Schiffler, A.: A heuristic taxonomy of computer games, http://www.ferzkopp.net/joomla/content/view/77/15/ (2006) Serradell‐Lopez, E. Lara, P., Castillo, D. And Gonzales, I.: Developing Professional knowledge and Confidence in Higher Education. International Journal of Knowledge Society Research IJKSR, 1 (4), 32‐41 (2010)

Experience With Digital Game‐Based Embodied Learning: The Road to Create a Framework for Physically Interactive Digital Games Carsten Busch, Florian Conrad, Robert Meyer and Martin Steinicke Creative Media Research Group, University of Applied Sciences HTW‐Berlin, Germany carsten.busch@htw‐berlin.de florian.conrad@htw‐berlin.de robert.meyer@htw‐berlin.de martin.steinicke@htw‐berlin.de Abstract: Over the past years, we have been researching various approaches to digital game‐based learning in the field of change and innovation management. Broadening the range of possible applications while consolidating methodical underpinnings, we have subsequently narrowed down our findings into the description of three specific treatments. This paper focusses on one of the applied treatments, namely to make participants go through a game sequence or interact with a digitally enhanced setup (e.g. play‐acting with motion capturing and real‐time rendering of a virtual character) to engage learners in embodied and experience driven learning. We present our experience starting with commercial of the shelf physically interactive digital games, followed by two examples of self‐made stand‐alone Kinect games that have been developed for use in team and leadership trainings. The latter will be introduced describing their goals, the resulting game design as well as lessons learned. Starting from the experience with such settings in project “HELD” as well as applications of embodied digital learning and physically interactive game‐based learning by others led us to the belief that there is a need for a framework that enables educational game and interaction designers to develop digital embodied settings without the need of (re)coding the Kinect management code as well as a number of other features relevant for education and training settings (e.g. control app, QR player identification and performance tracking). To further foster the easy development of physically interactive digital games and simulations, or digital aesthetic performances the framework integrates with the Unity game engine, thus enabling both rapid prototyping and quality games. First tests seem very promising with playable game prototypes developed in less than three days. To gather more feedback on real‐life applications using digital embodied learning we plan to offer the introduced framework free of charge for non‐profit applications. Keywords: physically interactive digital play, embodied learning, team and leadership training, systems and core‐ mechanics based learning

1. Introduction At HTW Berlin a comparatively wide definition of digital game‐based learning is applied (Bodrow et al, 2011). That covers three more or less distinct scenarios (Busch et al, 2012a): To begin with, learning by playing a digital game with or without individual reflection or group discussion. This covers learning in formal as well as informal – e.g. “stealth learning” (De Freitas & Maharg, 2011) or “interest‐driven learning” (Squire, 2011:19‐ 22) – settings. In our work we have explored this scenario focussing on “story‐based repetition and reflection” on the one hand and on “system‐based learning” on the other (Busch et al, 2012a). The second scenario is learning by designing and creating (e.g. Marlow, 2012) or modifying (e.g. Squire, 2011:174f; Monterrat et al., 2012) digital games. Wherein the produced game or modification might be created to generate either entertainment or, as in our case, a learning experience for the prospective player (Busch et al, 2013). The third scenario covered by our definition is learning framed by playful interaction with digital media or game components as in our digital aesthetic learning approach, using for example the “Spore Creature Creator” in change‐ and management‐workshops (Busch et al, 2012b).

2. Embodied system‐focused game‐based learning As mentioned, one approach that fits into scenario one, is learning by experiencing and discussing the narrative of a digital game. A contrasting one is based on the notion of games as systems which can be considered from three complementary perspectives (Busch et al, 2012a):

depicting real or imaginary systems (Salen & Zimmerman, 2004:422f)

as (complex) systems themselves (Salen & Zimmerman, 2004:156)

as a temporary system that encompasses the game and its players, their actions and the social space surrounding them (Salen & Zimmerman, 2004:471)

Carsten Busch et al. Considering perspective one, games can be used to explore the represented system, its procedural properties and causal relationships, consciously or unconsciously engaging with its procedural rhetoric (Bogost, 2007). Depending on the context and learning goals, the game itself does not necessarily need to depict the target system very accurately. Indeed, in some cases, not at all. When focusing on training skills or discussing behavioural patterns, for example, it may suffice that the game triggers relevant reactions or patterns through its core‐mechanics and rule‐set (perspective two). These can then be discussed and brought into relation to the learning goals after or between game sequences. Including perspective three into this line of thought, enables learning arrangements that foster or change the learning experience afforded by a given digital game by adding ad‐hoc rules, requirements or goals prior to a game sequence (Busch et al, 2013b). This is an inherently social process, though, that may be restricted to face‐to‐face workshops. Since our research was focussed on such workshops it has proven to be a promising approach (Busch et al, 2013a). While commercial off‐the‐shelf digital games (COTS‐DGs) with conventional input and output devices can certainly be used in such workshop‐settings, the relatively short duration of workshops in conjunction with oftentimes low gaming‐ experience of participants favours less complex and time‐consuming games as teaching tools. These games should ideally (Busch et al, 2011):

be easy to learn and have an intuitive interface to reduce the threshold for non‐gamers

be fun, challenging and engaging for a wide range of participants

be entertaining and easy to follow for spectators to allow watching and discussing their results

Furthermore research highlights the importance of embodied experience and emotions for learning and creativity (Hannaford, 2008). Thus we focussed on digital games that use motion‐controlled interfaces – such as Microsoft's Kinect – to enable easy access and large projections to create a social space including both the players and the audience (Busch et al, 2012a), the latter consisting of those workshop participants currently not playing but actively spectating. To test and evaluate this approach we first focussed on COTS‐DGs that might be used out of the box to challenge and improve a number of skills relevant to business workshops, targeting for example communication, collaboration and leadership skills using the two‐player rafting game in “Kinect Adventures” (Busch et al, 2011). Additionally we identified Kinect games that can be used to provoke a specific emotion or a situational awareness which mirrors feelings in business settings – such as performing in the digital game “Dance Central”. While we found the application of the identified games to be very effective, the number off such games did not evolve as hoped. Contrasting felt potential for such embodied game interfaces in both media and academia few new concepts where introduced. While some games did add Kinect support this often comes as one or two separate modes that do not feel well integrated into the whole gaming‐experience (e.g. the rail shooter mode in “Harry Potter and the Deathly Hallows – Part 1”). Games that focus on the Kinect as the primary input device and go beyond the early game concepts (mini‐games, sports or dance games) have not been successful (e.g. “Steel Battalion: Heavy Armor”) or were delayed to be featured on the next console generation (“Ryse”). With notable exceptions being "Puss in Boots” and “The Gunstringer”. Thus the number of usable COTS Kinect games is still insufficient for varying themes and workshops. Indeed, to date Microsoft provides “[m]ore than 200 ready‐to‐go activity plans” for usage of COTS Kinect games in educational settings (Microsoft, 2013). But the connection between game‐mechanic and learning goals often seem to be a bit far‐ fetched (e.g. playing “20.000 Leaks” in “Kinect Adventures” then discussing water disasters as a follow‐up action) and only a single activity is marked “for higher education”. A further reason why we started to develop custom motion controlled games was both the possibility to target specific learning goals by designing core‐mechanics that evoke relevant behavioural patterns or perceptions to be experienced and discussed, as well as the option to integrate some of the candidate ad‐hoc rules, requirements or game goals that might be devised by workshop participants into the games in advance, enabling more controlled and repeatable experiments. One last note on the typical sequences when using the embodied system‐focused digital game‐based learning approach. As described in (Busch et al, 2012a) in workshop‐settings we generally give a very short introduction to the used game covering only the mode of interaction and the oftentimes very loosely defined goals of the respective game. After that, cycles of game‐play and discussion are initiated. The play sequences typically cover less than two minutes while the discussions span between three and 15 minutes, sometimes even

Carsten Busch et al. longer. The discussions cover at least three aspects. Firstly, asking the players about their experience: What worked out well? What did not work? How did both players feel in their preliminarily agreed upon role or modus operandi? Reported perceptions are immediately compared with the perceptions of the audiences, quite often revealing interesting disaccords which fuel the discussion. This typically introduces the next stage where connections to working life experiences are discussed, and insights and knowledge are shared within the group. The last stage then focuses on who shall play next and which setting, strategy or ad‐hoc rule shall be applied.

Figure 1: Setting for embodied system‐based learning in workshops (game “KinAct”)

3. The road to iBox Following we describe in 3.1 and 3.2 two examples of self‐made stand‐alone Kinect games that have been developed for use in team and leadership trainings. These will be introduced describing their goals, the resulting game design and most importantly the lessons learned. Our experience with such applications of embodied digital learning as well as the interest in but sparseness of physically interactive game‐based learning applications by others led us to the belief that there is a need for a framework that enables educational game and interaction designers to develop digital embodied learning games and interventions fast and efficiently. The developed framework will be described in 3.3.

3.1 Developing team coordination and communication skills with “Kinect Tetris” One riposte that we frequently encountered in our workshops with COTS‐DGs like the rafting game in “Kinect Adventures” was that the game would require little or sparse communication for a leadership strategy to be successful, due to the simplicity of its core‐mechanics combined with its fast pace. Thus having or creating a strong bias toward some strategies while making others unfeasible. But having a strong biased model is actually an advantage in this case, enabling discussion of the model and corresponding real‐life situations which fosters learning (Squire, 2011:22‐26). Nonetheless this frequently encountered argument highlighted the potential in system‐focused game‐based learning to let workshop participants experience different systems of core‐mechanics and rule‐sets, showcasing how systemic environments shape the behaviour of interacting individuals and favour some behavioural strategies over others. Therefore we needed a game that would be comparable to the rafting game in that it can be used in the same setting with the same approach but requiring more coordination and communication as well as optionally negating successful strategies of the former. The task was given to students taking the digital game‐based learning course (Busch et al, 2103a) taught by one of the authors. In a brainstorming session a Tetris clone for two players was identified as promising due to the fact that as a classical game recognition might ease mastering the core‐mechanics. Furthermore the manipulation of blocks is pretty straightforward but especially the placing requires and enables more coordination than steering the boat in the “Kinect Adventures” rafting game. The game supports

Carsten Busch et al. two input modes. In the first players manipulate the falling blocks by taking postures that loosely resemble the form of those (see Figure 2).

Figure 2: Kinect Tetris: “L”‐block, “L”‐like “move”‐command, “S”‐like “rotate”‐command, “S”‐block These postures are not only used for moving and rotating, though, but also for a new game‐mechanic the “wishing for blocks“. When both players take the same postures concurrently instead of moving/rotating it, the falling block changes into the shape embodied by both players. This leads to a doubling of meaning for the postures and thus increases complexity of the rule‐set which needs to be handled by the players as a team. When they do not coordinate their behaviour a falling “I”‐shape might not be moved but accidentally changed to an “L”‐block instead – ruining all the careful stacking done before. The “wishing for blocks“‐mechanic actually introduces a “degenerate strategy” (Salen & Zimmermann, 2004:241) into the game. When players realize this they may create only “easy” blocks like squares or “I”s and stack these – gaining a high score until the game finally becomes too fast to handle. Another way players can seemingly sabotage the game is in that they do not cooperate and communicate at all but rather that only one player actively plays the game while the other stands still. This often results in better performance due to the fact, that neither time is spend on coding, transmitting and decoding information nor do misunderstandings or conflicting strategies hinder overall performance. Indeed these degenerate strategies are not consequences of a flawed game design – in contrast to entertainment game (Salen & Zimmermann, 2004:241) – but intentional. They enable on the one hand the exploration and discussion of the game and real‐life/business systems and on the other hand lead to the introduction of game‐mechanics that change the up to this point encountered game experience. For example depending whether the “wish for 'I's”‐ or the single‐player‐strategy was used, the game can be replayed with a specified amount of “wishing for blocks ammunition” or a “scale” game‐mechanic that requires players to balance their inputs. Both rule‐set adaptations enable a changed gameplay experience and thus offer new strategies as well as insights. A complete description of the game, its application as well as results would go beyond the scope of this paper. Of importance here is firstly that the game was successful in respect of its goals and well received by workshop participants, which led to the aim of creating further games to target themes of interest for leadership and team workshops. Secondly the usage of (de)activatable game‐mechanics was both very fruitful and well received but the manipulation of the game menu via mouse and keyboard required unnecessary movement of the workshop moderator creating ruptures in the workshop‐flow. Thirdly tracking the performance of teams by simply listing their points was feasible but did not capture other aspects of gameplay and did not allow for an analysis of individual performance over multiple runs, especially when mixing teams. Finally faulty postures where often attributed to a flawed optical recognition by the Kinect sensor instead of being accepted as mistakes of oneself, slowing learning of the core‐input‐mechanics. After a number of applications of the game in workshops the possibility to observe ones posture via a mirroring device was identified as a potential solution in future games.

3.2 Experiencing feedback in team and leadership workshops with “KinAct” The success of the “Kinect Tetris” game to evoke relevant behavioural patterns and strategies as a seed for meaningful discussions strengthened our belief in the promise of embodied system‐focused game‐based learning in business workshops. A main goal in designing “Kinect Tetris” was to create a game that would enable leadership and communication experiences differing from those in the rafting game of “Kinect Adventures”. Having accomplished this, we wanted to broaden the approach to other leadership and team related learning goals and concepts. Again, the task was given to students taking the digital game‐based learning course (Busch et al, 2103a) taught by one of the authors. In brainstorming sessions the themes of cooperation versus conflict/competition as well as the influence of feedback on performance and its

Carsten Busch et al. perception where identified as promising, see e.g. (Ashford & Tsui, 1991; Kaymaz, 2011). The resulting game “KinAct” was then refined by one of the students in her bachelor thesis (Günther, 2013) and tested by her and one of the authors both in a pre‐test with students as well as in a workshop setting with a team from the university´s IT service department. In the game, each player needs to mirror the postures that are displayed in their “lane” before the manikin reaches the bottom (see figure 1). Successful and timely mimicry is rewarded with points while failure earns a red “X”. If one of the two players accrues three of those negative points the game ends for both players’ and a screen displaying the players individual points as well as their aggregated score is shown. To foster team behaviour it is possible to enable the “team mode”, to increase the possibility of conflicts a “conflict mode” is available. In the latter, occasionally a manikins descends in the middle (“bonus”) lane and only one player may occupy this lane to mimic it. The successful player gains a high number of points and all her negative points are cleared. Contrary to this, in the case of the “team mode” both players need to team up to mirror a combined form displayed in the bonus lane, sharing the rewards if successful. To enable participants to experience and discuss feedback in the – admittedly only comparatively (Frasca, 2007:73‐75) – safe environment of a game in the workshop context (Busch et al., 2013b), the game settings enable an amplification of positive and negative feedback for each player independently. Important to note here is that we understand “feedback” in the way it is commonly used in game studies (Salen & Zimmerman, 2004:337), management literature (Ashford & Tsui, 1991) and Flow Theory (Csikszentmihalyi, 2010:83f): As information perceived by or delivered to a recipient that relates to the negative and/or positive consequences or outcome of their actions. Which is a core requirement for active digital game‐based learning (Meier & Seufert, 2003:14). Thus amplifying feedback is achieved in KinAct by increasing the audio and visual representation of outcome information.

Figure 3: KinAct: standard (green, normal and red manikin) and amplified visual Feedback As a consequence of our experience with the “Kinect Tetris” game, the manipulation of the games menus and settings was gesture‐based similar to COTS Kinect games. But we found that while this makes for a better player experience in an entertainment or presentation context, as the player does not need to move to the PC to restart a game or change modes, it is detrimental for the workshop context, indeed. Since the workshop moderator now needs to either take the place of one player or describe to them what they should do step by step. As a consequence we integrated an optional mouse‐mode so when introducing the game to the audience the workshop moderator could control the game by hand movement while later on she can do that using the PC interface. Nonetheless, one problem was not fixed by this workaround. Both the usage of the “Kinect Tetris” game as well as the team and conflict modes in “KinAct” rely on the moderator reacting to the players` progress and strategies by introducing a given game‐mechanic countering their actions or enabling a different challenge. The important part here is the introduction. If the players use the degenerate “wishing strategy” in “Kinect Tetris” the moderator introduces the ammunition mechanic by explaining it in the first place and then letting them experience the change. If players feel they cannot really cooperate in “KinAct” the moderator activates the team or conflict mode, explains it to the workshop‐ participants and then starts the next run. In contrast to this, key insights through the feedback mechanic in “KinAct” rely on the (to some degree) unbiased experiencing of those. Will a player feel motivated if she only receives strongly amplified feedback? Will players understand their mistakes when receiving no amplified negative feedback? But seeing the changes and further options available in the setting screen of the game before playing may influence the expectations and thus the experience of players. To counter this keyboard shortcuts were introduced for specific combinations of core‐mechanics. While that solves the elucidated problem, it is still not

Carsten Busch et al. an optimal fit as it reduces the overall flexibility of the embodied system‐focused game‐based learning approach. Furthermore, the game did discriminate between player and team score, thus enabling better tracking of individual performance over multiple runs. Nonetheless, these scores need to be noted by participants or the workshop moderator and are otherwise lost if a new game begins. This led to problems if the participants restarted the game unauthorized or the application was accidentally exited. To fix the problem of failure acceptance and related learning problems described in 3.1, KinAct implements a camera view situated on the outer bounds of the screen (see figure 1), so participants can check their body posture while playing the game. While this reduced problems compared to Kinect Tetris some beta‐users and workshop participants were still observed to question the capability of the optical sensors while obviously failing to adopt the correct posture. A reason for this may be the time‐pressure created by the game that requires players to focus on the displayed manikin symbols instead of the seemingly game‐external camera view panel.

3.3 The iBox framework for Kinect games and embodied aesthetic learning setups Experience with both creating and applying digital games using the Kinect as an input device shows the potential of their usage in different learning contexts. While our primary target has been business workshops, on various public presentations especially teachers from primary and secondary schools commented that they would like to use such games in their lessons. The idea of using Kinect for educational purposes and even digital game‐based learning did appear almost instantly after its release by Microsoft. Nonetheless, the number of games still seems to be rather small, even after two years. E.g. (KinectEducation, 2013) lists merely five “Apps” (only two of these being games). Oftentimes, these games are (both in terms of presentation and game‐mechanics) rather simple and based on a behaviouristic drill‐and‐practice approach.

Figure 4: Main components of the iBox framework described in this paper A reason for the lack of high quality games may be the technological hurdle of mastering the Kinect management code and the development of a game from scratch. Starting from our own need to streamline development and reuse code in a consistent manner we felt the need for a technological framework that would enable designers to focus on the game‐play instead of managing the Kinect sensory data. Under guidance of two of the authors a group of master students was set to the task of developing a solution for this while keeping the lessons learned from previous games for workshops in mind. The result – code name “iBox” (i3.cx, 2013) – fits not only our needs but offers educational game designers a number of options and functionalities (figure 4) that accelerate development of Kinect games or embodied versions of digital aesthetic interventions (see Busch, et al., 2013a). Firstly, iBox works with Unity3D thus omitting the need development from scratch by integrating into a powerful game engine that enables rapid prototyping as well as high quality games. iBox maps the Kinect (or Asus Xtion) data‐streams onto skeleton information of currently up to eight avatars in Unity thus stripping away the need to implement the interpretation of Kinect data. Furthermore, this may reduce the aforementioned problems with failure acceptance of players, due to the fact that the avatar‐body gives real‐ time feedback about the posture of the player instead of the binary “matches posture or not” feedback in “Kinect Tetris” and “KinAct”.

Carsten Busch et al. Secondly, iBox works in a distributed setting (Busch et al, 2012c). This enables the usage of multiple Kinect sensors to capture extensive 3D data for embodied digital aesthetic interventions or for games with core‐ mechanics that lead to situations where one player might conceal the other(s) – a boundary condition that led to rather stationary game designs in “Kinect Tetris” and “KinAct”. Additionally, the networked approach enables the creation of games that connect players from different locations. Thus participants could play in different rooms, with remote peers from another city or even country, and (using media facades) across various cities in a Public Gaming context (Conrad, 2009; Busch & Conrad, 2012). Thirdly, iBox enables tracking players independently of their position, in contrast to KinAct and Kinect Tetris where players may switch roles by switching places. Furthermore the framework integrates both a QR code generator and reader to enable player identification and statistics over multiple runs of a game or even a series of workshops. These statistics are pushed to a given web‐service for review or to display development of the player. Fourthly, games based on iBox can be controlled via the “iBox Manager” app available in the iTunes store (figure 5). The app displays defined interface elements in Unity, so that a workshop moderator or teacher may for example manipulate the camera view of the game or (de)activate core‐mechanics and rule‐sets during a game or prior a new run. This solves the problems with managing the games settings described in 3.1 and 3.2. Furthermore, the app displays general and individual game statistics during and after a run which can be hidden from players. So the teacher or moderator can identify relevant findings in the first place and subsequently target these in the discussion without unintentionally influencing participants/students with displayed information. In addition, the app offers a “replay” mode that enables the moderator to view and show a “recording” of the game to foster discussion or (dis)prove claims. Indeed the “replay” is not a simple video but a recording of the game‐state over time, thus enabling manipulation of the simulation in retrospect – e.g. changing the camera angle (implemented) or simulating “what if” scenarios (planned).

Figure 5: IBox Manager app displaying core‐mechanics options and statistics during a game Last but not least, iBox comes with an interface that supports the integration of further hardware both as input and output devices. Input devices might be used for both game‐play relevant interaction or to capture user data for analytical purposes. The latter was exemplarily implemented using a glove with an integrated pulse sensor transmitting captured data which is displayed by the iBox Manager app as a pulse curve and can be used in combination with the replay functionality. Additionally, the iBox framework´s device interface affords the usage of physiological interfaces (Jercic et al., 2012) to both control the game or as a game‐ mechanic.

4. Conclusion While embodied digital game‐based learning has been gaining more and more attention from school teachers, academics and professional coaches alike, there is a lack of quality educational games for this field. With the iBox framework we are trying to encourage game makers and educators to create high quality games by providing them with a powerful tool that streamlines the development process and already implements many

Carsten Busch et al. features that we have found to be specifically useful for this kind of application. First tests have proven the iBox’ potential and we are looking forward to getting more user feedback by releasing the framework for free non‐commercial use.

Acknowledgements “Kinect Tetris” was developed under the guidance of Martin Steinicke by Manuel Adameit, Gregor Altstädt, Tino Baumgart, Robert Kasparson and Tobias Wehrum. “KinAct” was developed under the guidance of Martin Steinicke by Malte Bünz, Kerstin Günther and Timo Welde. It was developed further by Kerstin Günther in her Bachelor‐Thesis under the guidance of Carsten Busch and Martin Steinicke. The iBox framework was developed under the guidance of Carsten Busch and Martin Steinicke by Caroline Hilprecht, Robert Meyer, Eugen Rats and Nathanael Siering.

References Ashford,S.J. & Tsui,A.S.(1991) Self‐Regulation for Managerial Effectiveness: The Role of Active Feedback Seeking. The Academy of Management Journal Vol. 34(2),pp.251‐280 Bodrow,W., Busch,C. & Steinicke,M.(2011) Digital Game Based Learning. Proceedings of the International Conference on E‐ Learning and the Knowledge Society Bogost,I.(2007) Persuasive Games: The Expressive Power of Videogames,MIT Press,pp.28 Busch,C.; Conrad,F. & Steinicke,M.(2011) Using Digital Games to Teach the Hero's Journey as a Model for Change and Innovation Management, Proceedings 5th European Conference on Games Based Learning,pp.740‐744 Busch,C. & Conrad,F.(2012) Public Gaming – Augmenting Public Spaces with Digital Games. In: Pop, S.; Stalder,U.; Tscherteu,G.; Struppek,M.(2012) Urban Media Cultures,AV Edition,pp.152‐155 Busch,C.; Conrad,F. & Steinicke,M.(2012a) Digital Games and the Hero's Journey in Change and Innovation Management Workshops. Proceedings 6th European Conference on Games Based Learning,pp.72‐81 Busch,C.; Conrad,F. & Steinicke,M.(2012b) Digital Aesthetic Learning: Using In‐Game Artefact‐Creation to facilitate Learning th in Change and Innovation Workshops. Proceedings of the 18 International Conference on Technology Supported Learning & Training (Online Educa) Busch,C.; Kramer,A.; Meyer,R. & Steinicke,M.(2012c) Angewandte Consumer Electronics in Lehre und Entwicklung. Proceedings WCI2012 Wireless Communication and Information,pp.5‐15 Busch,C.; Conrad,F. & Steinicke,M. (2013a). Digital Games and the Hero's Journey in Management Workshops and Tertiary Education. Electronic Journal of e‐Learning,11(1). Busch,C.; Conrad,F. & Steinicke,M.(2013b) Coaching mit digitalen Spielen. In Inderst,R.T. & Just,P. (2013) Build 'em Up ‐ Shoot 'em Down: Körperlichkeit in digitalen Spielen.pp.274‐289 Conrad,F.(2009) Public Gaming – Exploring the Potentials of Public User Interfaces in Pervasive Games. Bachelor‐Thesis HTW‐Berlin Csikszentmihalyi,M.(2010) Flow: Das Geheimnis des Glücks. Klett‐Cotta; Edition:15. De Freitas,S. & Maharg,P.(2011) Digital Games and Learning. Continuum International Publishing. p.113 Frasca,G.(2007) Play the Message – Play, Game and Videogame Rhetoric. Ph.D. Dissertation, IT‐University of Copenhagen Günther,K.(2013) Digital Game‐based Learning im Bereich Teammanagement: Weiterentwicklung und Evaluierung eines Kinect‐Spiels zur Verbesserung der Kommunikation. Bachelor‐Thesis HTW‐Berlin Hannaford,C.(2008) Bewegung – das Tor zum Lernen. VAK Verlags GmbH. Edition:7 i3.cx (2013) iBox Knowledge Center retrieved 23.04.2013 from www.i3.cx Jercic,P.; Astor,P.J.; Adam,M.T.P.; Hilborn,O.; Schaaff,K.; Lindley,C.; Sennersten,C. & Eriksson,J. (2012) A serious game using physiological interfaces for emotion regulation training in the context of financial decision‐making. Proceedings of the European Conference of Information Systems (ECIS) Kaymaz,K.(2011) Performance Feedback: Individual Based Reflections and the Effect on Motivation. Business and Economics Research Journal. Volume 2(4).pp.115‐134 KinectEducation.com(2013) Apps. retrieved 20.04.2013 from apps.kinecteducation.com Marlow,C.(2012) Making Games and Environmental Design: Revealing Landscape Architecture, Proceedings 6th European Conference on Games Based Learning,pp.309‐316 Meier,C. & Seufert,S.(2003) Game‐based Learning: Erfahrungen mit und Perspektiven für digitale Lernspiele in der betrieblichen Bildung. Handbuch E‐Learning (2003). Fachverlag deutscher Wirtschaftsdienst. Microsoft(2013) Microsoft in Education – School activity plans. retrieved 22.04.2013 from www.microsoft.com/education/en‐us/products/Pages/kinect.aspx#3 Monterrat,B.; Lavoué,E. & George,S.(2012) Learning Game 2.0: Support for Game Modding as a Learning Activity, Proceedings 6th European Conference on Games Based Learning,pp.340‐347 OpenSoundControl.org(2013): Introduction to OSC. retrieved 21.04.2013 from opensoundcontrol.org/introduction‐osc Salen,K. & Zimmerman,E.(2004) Rules of Play: Game Design Fundamentals, Mit Press Squire,K.(2011) Video Games and Learning: Teaching and Participatory Culture in the Digital Age Teachers College Press

Toward Improvement of Serious Game Reliability Thibault Carron, Fabrice Kordon, Jean‐Marc Labat, Isabelle Mounier and Amel Yessad LIP6, CNRS UMR 1606, Université Pierre & Marie Curie, Paris, France Thibault.Carron@lip6.fr Fabrice.Kordon@lip6.ff Jean‐Marc.Labat@lip6.fr Isabelle.Mounier@lip6.fr Amel.Yessad@lip6.fr Abstract: Serious games are complex software applications resulting from a costly and complex engineering process, involving multiple stakeholders (domain experts, teachers, game designers, designers, programmers, testers, etc.). In addition, the serious games implying multiple learners‐players are dynamic systems that evolve over time and implement complex interactions between objects and players. Traditionally, once a serious game is developed, testing activities are conducted by humans who explore the possible executions of the game’s scenario to detect bugs. The non‐deterministic and dynamic nature of multi‐player serious games enforces the complexity of testing activities. Indeed, exploring all possible execution paths manually is impracticable humanly due to their large number. Moreover, the test can detect some bugs, but cannot verify some properties of serious games such as the scenario allows a learner to acquire all the knowledge, that the winner is necessarily one who has achieved all the learning objectives or the scenario does not lead to deadlock situations between learners. This type of properties has to be verified at the design stage of serious games' development. We propose a framework enabling a formal modelling and an automatic verification of serious game's scenario at the design stage of development process. We use Symmetric Petri nets as a modelling language and choose to verify properties by means of model checking. Petri nets are a mathematical notation suitable for the modelling of concurrent and dynamic systems. Due to the dynamic nature of serious game’s scenario, we selected a particular Petri net model: Symmetric Petri net. Model checking is a powerful way to verify systems; it provides automatically a complete proof of correctness, or explains, via a counter‐example, why a system’s property is not correct. This paper discusses how this framework is used to verify the serious game properties before the programming stage begins. In order to concretise our discourse, we apply our approach on a scenario of a serious game and present how design's properties are expressed and verified thanks to the formal framework. Keywords: Serious Game, software engineering, Serious Game verification, model checking, petri nets

1. Introduction Context Serious games can be defined as ``(digital) games used for purposes other than mere entertainment'' (Susi et al., 2007). They are a way to help people to acquire domain knowledge and develop skills. Particularly, we share the definition of Fabricatore (2000): ``a serious game is a virtual environment and a gaming experience in which the contents that we want to teach can be naturally embedded with some contextual relevance in terms of the game‐playing [...]''. Serious games are complex software applications resulting from a costly and complex engineering process, involving multiple stakeholders (domain experts, game designers, designers, programmers, testers, etc.). In addition, the serious games implying multiple players are dynamic systems that evolve over time and implement complex interactions between objects and players. Once a serious game is developed, testing activities are conducted by humans who explore the possible executions of the game to detect bugs. Problem The non‐deterministic and dynamic nature of multi‐player serious games enforces the complexity of testing activities. Indeed, exploring all possible execution paths manually is impossible due to their large number. Also, multi‐player serious games belong to the class of system for which it is well known that testing activities are not sufficient to re‐enforce reliability. Moreover, testing activities do not allow verifying specification properties and are intrinsically performed too late because they require the game to be implemented first; thus, detected problems are costly to correct. Contribution

Thibault Carron et al. To avoid complex testing procedures and preserve serious game reliability, we propose to perform formal verification of serious games at the design stage. Our objective is to ensure that a serious game satisfies properties that are extremely difficult to assess by means of tests only. Once the verification has been performed on an abstract specification, development starts from a validated design. This paper presents a verification approach enabling automatic verification of serious game properties. Among the available techniques, we chose Petri nets to formally specify the serious game and model checking to verify properties. Petri nets are a mathematical notation suitable for the modelling of concurrent and dynamic systems (Jensen et al., 2009). Due to the dynamic nature of serious games, we selected a particular Petri net model: Symmetric net with bags. Model checking is a powerful way to verify systems; it automatically provides complete proof of correctness, or explains, via a counter‐example, why a system is not correct (Bérard et al., 2001). The paper presents our methodological approach that is illustrated on a case study based on a real serious game as proof of the concept. Section 2 presents relevant properties for serious games. Section 3 details our approach. Then, we apply it to the case study in section 4 before section 5 presents some related work before concluding and presenting some perspectives.

2. Relevant properties for Serious Games Our work aims at automatically verifying (using model checking) stated properties on serious games at the design stage. We classify expected properties along two axes (see table 1). The first one deals with the relationship between a property and a game that can be:

Game‐independent: it is relevant for any serious game,

Game‐dependent: it is specific to a given serious game and not applicable to others.

The second axis only involves the type of property (and later, the algorithms to be used for verification):

Invariant properties are always verified in the game,

Reachability properties must be verified in a game state that can be reached from the initial state,

Temporal properties, expressed using a temporal logic like CTL or LTL, define causal relations between some classes of states in the game.

Table 1 provides examples of game properties. Property patterns can be defined for game‐independent properties, and then filled with conditions describing specificities of a given game. This is the case for the game‐independent reachability properties where only lose or win conditions need to be specified. Game‐ dependent properties need to be defined according to the rules and the gameplay of the game modelled. Table 1: Classification of properties (invariants, reachability and temporal) Invariant Reachability Temporal

Game‐independent It is always possible to perform an action before the game ends The player can win (respectively lose) the game The player must perform at least one action before winning or loosing

Game‐dependent The player can always call for help "The player can reach the virtual lab" or "The player can get skills to kill the monster" The player can not complete the level as long as he does not have the competence C

According to the type of formula, a verification scheme can be elaborated. Temporal properties require the designer to write a temporal logic formula concerning the causal relation between the identified states. Other properties only require the definition of a logic formula without any temporal connector. Usually, temporal properties are more likely to be game‐dependent than others.

3. Verification framework Today, the video game industry uses human testers to detect bugs in games. Obviously, this method is costly and unreliable. In order to cut development costs and increase serious game reliability, serious game specifications have to be verified prior to implementation. Indeed, testing is used to verify properties of serious games, such as the ones presented in section 2. However, testing is not sufficient to verify properties

Thibault Carron et al. of serious games. In this section, we advocate the use of formal verification as a better way to proceed. To do so, we propose a generic pattern model describing a wide range of serious games.

3.1 Generic pattern for Serious Games Our research focuses on multi‐player serious games where scenarios are composed of activities, often presented to players as challenges. An activity requires a player to have acquired some skills and some virtual objects and provides him with new skills and/or virtual objects, depending on his performance. Thus, only players having the required skills and virtual objects (vo) may perform an activity. Figure 1 shows a diagram relating activities in a game. Activities can be performed in sequence (e.g. Act1, then Act2), in parallel (e.g. Act2 and Act3) or with some exclusion (e.g. Act6 requires players 1 and 2, thus, if it is completed, player 1 cannot perform Act4 anymore).

Figure 1: Example of a scenario of a serious game

3.2 Verification method Verification for software systems is commonly classified into three classes: simulation, algebraic methods and model checking. All can be applied to a design model, once the behaviour of the system is appropriately specified. Simulation is not well adapted when we want to cover the whole execution space. Algebraic methods are difficult to operate and require highly skilled and experienced engineers. Model checking (Clarke et al., 1999), is well adapted to finite systems, despite an intrinsic combinatorial explosion problem: it is based on an exhaustive investigation of the system's state space and is fully automated. We advocate that model checking is best suited for serious games. It is a good compromise between the accuracy of provided diagnostics and the automation/cost of the procedure because:

it provides automatic verification of properties,

it is more reliable than simulation (as well as tests on the final product with human testers),

it requires little expertise in logical reasoning,

it always terminates (with sufficient memory resources and when we consider finite systems) with a yes or no (then providing a useful counterexample) answer.

3.3 Symmetric Petri Nets with Bags (SNB) Among the multiple variants of Petri Nets, we chose Coloured nets that are necessary to get a reasonable sized specification, thanks to the use of colours to model data. Next, within the large variety of coloured Petri Nets, we selected Symmetric Nets with Bags (Haddad et al., 2009) where tokens can hold bags of colours. They support optimized model checking techniques (Colange et al., 2011). Moreover, the notion of bags is relevant to modelling some dynamic aspects that are typical of serious games in a much simpler way than with most other coloured Petri nets. We provide here an informal presentation of Symmetric Nets and use them to model the generic pattern of serious game we presented.

Thibault Carron et al. 3.3.1 Informal definition and example A petri net is a bipartite graph composed of places (circles), that represent resources (e.g., the current state of a player in the game) and transitions (rectangles) that represent actions and consume resources to produce new ones. Some guards ([conditions] written near a rectangle) can be added to transitions The SNB of Figure 2 models a serious game activity. Place beforeActivity holds players and their context: skills and virtual objects (stored in bags). Here, only a set (not a bag) is required for skills, which is denoted by the keyword unique in the declaration of variables BS, BS1 and BSp. The initial marking M in place beforeActivity contains one token per player (identified by p) associated with its skills and virtual objects (sets BS and BV respectively). Place activityDesc holds the required skills and virtual objects for each activity. The initial marking M' in place activityDesc contains a token per activity (identified by a) associated with its prerequisite (BS1 and BV1) and the information needed to compute the consequence of the activity on the player (in terms of BSp and BVp).

Figure 2: Modelling a game activity in SNB Each activity begins (firing of transition start) only when players' skills and virtual objects include the one of its prerequisite. Then, the activity may end in failure (transition looseA) or in successfully (transition winA). Functions fwin and floose represent the evolution of skills and virtual objects at the end of the activity (dropped in place beforeActivity). The SNB shown in figure 2 allows us to model with a very abstract and concise manner a serious game scenario. This powerful expressiveness permits us to have the whole scenario on a “small” graph (useful for automatic execution) but for a better understanding, it is possible to imagine it “deployed”: one for each activity as we will illustrate in the fourth section. 3.3.2 Interest of SNB The SNB are appropriate to model this type of system for three main reasons. First, their capacity to structure data in tokens with sets and multisets (bags) allows to capture the dynamic part of serious games well: here, the number of skills and virtual objects that varies (and can be empty). Second, they provide an easy modelling of operations such as union or inclusion tests in transition guards. This allows for a more compact specification. Third, SNB preserve the use of symmetry‐based techniques allowing efficient state space analysis (Colange et al., 2011) that is of particular interest for the formal analysis of serious games. The model in Figure 2 is exactly the one that is verified (once max values defined), it is not mandatory to instantiate it per activity and player.

4. Application to a case study As a proof of concept, we apply our verification framework to the serious game Nuxil constructed on a Game Based Learning Management System called Learning Adventure (LA). LA is a 3D multi‐player environment with lakes, mountains and hills where activities can be performed. Players can move within this environment, performing activities in order to acquire skills and virtual objects. We present here the Nuxil’s automatic

Thibault Carron et al. verification during the design stage. At the end of the development, the Nuxil’s scenario has been ecologically tested in an institute of technology by 60 students (four groups of 15). In the game Nuxil, players explore the environment and have to use linux commands (e.g. copy (cp), move (mv), edit file permissions (chmod), etc.). The objective is to illustrate linux commands through both visual and interactive environment. For instance, the command mv is used by players in order to move virtual objects between game areas and the command chmod provides permissions for opening a chest that is represent a locked computer file.

4.1 The case study Nuxill activities allow players to acquire skills (e.g. mastering the file management commands) and win virtual objects (e.g. the key of a chest). Activities are proposed to players, depending on their skills and their virtual objects. In this article, we selected three activities to illustrate our approach; their inputs and outputs are presented in table 2. We deliberately distinguish virtual objects voi and skills ski in the description of an activity ai. The former are related to the gaming and the second to the learning.

4.2 Modeling the case study To verify Nuxil, we first instantiated the generic pattern of figure 2 into the model of figure 3 with Activities={a1, a2, a3}, Skills={sk1, sk2, sk3} and VirtualObjects={vo1, vo2, vo3, vo4, vo5, vo6}. For example, a player must have at least the virtual objects vo2, vo3, and vo6 in order to perform the activity a2. We consider that initial marking M of place beforeActivity is such that for each player p, M contains the token {<p, {}, {vo1, vo2}}. The initial marking M' of place activityDesc is {<a1, {}, {vo1}, {sk1}, {vo1}>, {<a2, {}, {vo2,vo3,vo6}, {sk2}, {vo4, vo5}>}, {<a3, {sk1}, {vo4}, {sk3}, {}>}. This SNB models how players acquire skills and virtual objects. When a player loses an activity, its skills and virtual objects sets are not modified. When he wins one, he loses the virtual objects needed to perform the activity and wins the ones produced by the activity. New skills increase its skills set. Table 2: Description of some activities in Nuxil Activities copy (a1)

Activity input (preconditions) "basic commands" area (vo1)

chmod (a2)

"file permissions" area (vo2), chest closed (vo3) "PNJ wizard (vo6) chest opened (vo4), file commands (sk1)

documentation (a3)

Activity output (postconditions) “basic commands" area (vo1), file commands (sk1) "advanced commands" area (vo5), chest opened (vo4), file permissions (sk2) linux architecture (sk3)

The Nuxil scenario allowed us to simplify the generic model by merging places beforeActivity, afterActivity and winner. We assume the game ends once a player obtains all skills. At this stage, no new activity should start.

Figure 3: Part of the Nuxil game formal model and an instantiation of it on the right

Thibault Carron et al. For example (see Figure 3), let’s imagine that the “copy” activity has been achieved. The player has in Skill Bag : sk1 (file command skill: the upper star on the figure 3 ) and in his BV (virtual objects bag): 3 virtual objects (two access keys and a closed chest). The PNJ will then propose the “chmod” activity and in case of a success, the player will get in addition a new skill (sk2) and two virtual objects: vo5 (“advanced commands" area) and vo4 (chest opened ) as rewards as explained on the table 2 (vo2 will be removed). In case of failure, we are back in the initial state but it is possible to get specific virtual objects in that case for remediation purpose.

4.3 Verification Since the formal representation of the serious game allows the construction of the reachability/quotient graph, i.e., the state space of the game, we can verify invariant, reachability or temporal properties. We present two groups of properties. The first one concerns game deadlocks that are linked to the identification of the wining states. The second one concerns the success of learning process, i.e. a player may always increases his skills. The properties are informally expressed but they can be stated as temporal logic formulas (Bérard et al., 2001) that can be model‐checked. 4.3.1 WinningProperty Let us first define the property winner(p) stating that a player p won the game. We call WinningProperty the property identifying the end of the game. A player wins the game if he holds all the required skills. Therefore, winner(p) is true if: there is a token <p,{BS},{BV}> in the place beforeActivity such that cardinality(BS)=maxSkills. In other words, a token in place beforeActivity is such that the bag of skills BS contains all skills. The game can end when all players (or one) win(s) all the activities. In the first case: WinningProperty = for all p in Players, winner(p). In the second case: WinningProperty = there is at least one p in Players, winner(p). If we want to model a game where only a player can win (i.e. once a player wins, the others cannot begin a new activity) we have to change the guard of transition start in the SNB. This transition can be fired only if the marking of place beforeActivity does not contain a token <p, {BS}, {BV}> such that winner(p) is true. 4.3.2 DeadlockProperty When deadlocks appear, the property WinningProperty becomes not satisfied. In these case, some executions where no player can win are possible. If we want to verify that an ending state is always reachable (i.e. it is always possible to finish the game with a winner), we have to verify that a state WinningProperty is always reachable from the initial state of the game. Such a property is a temporal logic property since it has to be verified for each execution. 4.3.3 Learning process property We want to verify that a player always has the possibility to increase his skills (until he wins the game). Such a property is a temporal logic property since it is necessary to compare the states along each execution. We call increaseSkillsProperty the associated property. We define first the increaseStrictly(s,s',p) property where s and s' are two states of the game and p a player. increaseStrictly(s,s',p) is true if the bag of skills of player p at state s is strictly included in the bag of skills of player p at state s'. Then, increaseSkillsProperty = for all p in Players, for all reachable state s, set of skills is equal to Skills or all the possible executions lead to a state s' such that increaseStrictly(s,s',p). This formula allows verifying by model checking that the serious game always improves the player’s skills.

Thibault Carron et al.

5. Related work Petri nets are used in both academia and industry to model concurrent systems since they are well adapted to this class of problem. However, only a few studies address the use of Petri nets and model checkers for games. Moreover, in most cases, Petri nets are used to analyse game scenarios in order to adapt them to the player. (Araujo et al., 2009) discuss the applicability of Petri Nets to model game systems and game flows compared with other languages such as UML. The work presented by (Yessad et al., 2010) uses place/transition Petri nets to assess the progress of players in games once they are developed. Other studies focus on the analysis of game scenarios at the design stage. For instance, the ``Zero game studio'' group (Lindley 2002) uses causal graphs to model game scenarios. The work presented in (Champagnat, 2005) defines a set safety and liveness properties of games that should be verified in the game scenarios before their implementation. In the domain of Technology‐Enhanced Learning, Petri nets are used to capture the semantics of the learning process and its specificities. In particular, Hierarchical Petri nets are used by (He et al., 2007) to model desirable properties. The objective is to help designers to design and optimize e‐ learning processes. We consider these researches to be close to ours except that our research allows verifying patterns of properties related to both the learning aspects and the behaviour of the game. This is a possible thanks to a generic model of serious games, capturing most games and modelling learning skills as well as learners and game objects. In addition, SNB provides an efficient way to model games and together with efficient model‐ checking‐based analysis. We claim our solution is more adapted to the dynamic and complex nature of serious games.

6. Conclusion We presented a formal verification of serious games at the design stage. It relies on Symmetric nets with Bags and the use of model checking to verify automatically behavioural properties of serious games. Our objective is to reduce cost and complexity of serious games elaboration by enabling early error detection (at design stage). One interesting point of our approach is to provide a procedure helping engineers to elaborate the design of their serious game. In particular, we propose a classification of properties that are relevant in that domain. It is then possible to infer from these patterns an efficient verification procedure involving the appropriate model checkers (i.e. the one that implements the most efficient algorithms for a given property pattern). Another important point is the use of Symmetric Petri nets with bags that better tackle the combinatorial explosion problem intrinsic to the model checking of complex systems. We applied our approach to a real case study for assessment purposes. Even if this case study remains small, it shows encouraging results. This work is part of a project aiming at designing efficient formal verification based procedures for the design of serious games. The formal model, once it is verified, could be a basis for an automated implementation of a serious game execution engine. In the long term, this could decrease the time to implementation as well as to cut a large part of its costs. Future Work A trend is to exploit the formal specification to extract relevant scenarios for testing purposes. Subsequently, human tester would have directives to follow during their testing work. Another trend is to define transformation rules in order to construct semi‐automatically Petri nets from some models of scenarios more user friendly such as eAdventure (http://e‐adventure.e‐ucm.es), LEGADEE (http://liris.cnrs.fr/legadee/), etc.

References Araujo, M. and Roque, L. (2009) Modeling Games with Petri Nets, Breaking New Ground: Innovation in Games, Play, Practice and Theory: Proceedings of the 2009 Digital Games Research Association Conference, London. Bérard B. et al (2001) Systems and Software Verification. Model‐Checking Techniques and Tools. Springer, 2001.

Thibault Carron et al. Champagnat, R., Prigent, A. and Estraillier, P. (2005) Scenario building based on formal methods and adaptative execution, International Simulation and gaming association. Clarke, E., Grumberg, O. and Peled, D. (1999) Model checking. MIT Press. Colange, M. et al. (2011), Crocodile: a Symbolic/Symbolic tool for the analysis of Symmetric Nets with Bag, 32nd International Conference on Petri Nets and Other Models of Concurrency, LNCS, vol. 6709. Springer, pp. 338–347. Fabricatore, C. (2000) Learning and videogames: an unexploited synergy, AECT National Convention ‐ a recap. Secaucus, NJ : Springer Science + Business Media. He, F. and Le, J. (2007) Hierarchical Petri‐nets model for the design of e‐learning system, Proceedings of the 2nd international conference on Technologies for e‐learning and digital entertainment. Lindley, C. A. (2002) The gameplay gestalt, narrative, and interactive storytelling, Proceedings of the Computer Games and Digital Cultures Conference, pp. 6–8. Haddad, S. et al. (2009) Efficient State‐Based Analysis by Introducing Bags in Petri Net Color Domains, 28th American Control Conference (ACC’09). IEEE Press, pp. 5018–5025. Jensen, K. and Kristensen, L. (2009) Coloured Petri Nets : Modelling and Validation of Concurrent Systems, Springer. Susi, T., Johannesson, M. and Backlund, P. (2007) SeriousGames: An Overview (technical report)., Skövde, Sweden: University of Skövde. Yessad, A. et al. (2010) Using the Petri nets for the learner assessment in serious games, ICWL, pp. 339–348.

The Effects of Gamification on Student Attendance and Team Performance in a Third‐Year Undergraduate Game Production Module Hope Caton and Darrel Greenhill School of Computing and Information Systems, Faculty of Science, Engineering and Computing, Kingston University, London, UK h.caton@Kingston.ac.uk d.greenhill@Kingston.ac.uk Abstract: This paper investigates the effects of a gamified awards and penalties framework on a third‐year undergraduate game production module which has a predominantly male demographic. Students work in teams of three game programmers and two artists to develop a computer game prototype, applying their game‐programming knowledge and game theory to complete the project. ‘Gamification’ harnesses the reward and penalty game mechanics and apply it to real‐world problems, such as, the motivational challenges that can be a stumbling block to many student team projects. Achieving an award is a framework for incorporating competition‐based learning into the classroom, while the issuing of penalties is a system for encouraging attendance. Penalty cards issued for absence (and other infractions) affected grades. However, the receiving of an award was not connected to the assessment. The benefits and drawbacks of students collaborating on team projects have been well studied. From the student's point of view, the main drawback of team‐ based learning is most commonly unequal contribution. Using game theory as a basis for establishing a system of awards and penalties, this paper offers a gamified framework to keep students equally contributing to team efforts. This paper asks three questions: 1) Does the awards/penalties framework improve attendance? 2) If yes, does improved attendance result in improved assessments? 3) Does the framework improve cohesion and performance in student teams? This paper presents quantitative evidence to answer the first two and offers speculative comments on the third. Initial results suggest that the awards and penalties framework improves attendance and increases student performance and overall grade. Speculatively, the framework appears to be effective in increasing motivation. Informal student commentary indicates that while motivation is not improved across the cohort, those that are motivated contribute significantly more time and effort to the project. Awards proved successful in improving completion of previously resisted tasks and, if timed correctly, can attract students to attend a class they would otherwise choose to miss. Keywords: gamification, class awards, attendance, free riders, team‐based projects

1. Introduction The Game Production module is a one‐term, 15‐credit third‐year undergraduate module. The final assessment is to produce a computer game prototype and its corresponding game design document from an original concept. The game prototype project was a collaboration between Games Technology and Graphics Technology students who worked in teams to complete the assignment. However, the module was compulsory for technology students and optional for the others. Student assessments were made up of two components: a team mark worth 70% and an individual mark worth 30%. The team component was assessed by the lecturer and the module leader at the end of term at a team demonstration of the game prototype. Individual assessments were based on a self‐reflective report on the prototype development which included an informal peer assessment. The class was taught by an award‐winning lecturer using a Constructivist approach and a blend of Collaborative and Project‐based learning techniques. Students collaborated in teams of 4‐8 and were responsible for organising themselves to complete the project. Lectures were kept to a minimum. Instead teams were encouraged to communicate and creatively collaborate in order to solve problems and complete tasks. Module learning outcomes included developing employability skills in line with expectations of the computer games industry. In as far as possible, the class environment aimed to replicate a computer games production studio, using industry standard methodologies and expecting professional behaviours from students, including regular attendance and punctuality. Students were taught the AGILE project management system which, in line with Project‐based learning, they were permitted to modify and adapt to the needs of their particular team.

Hope Caton and Darrel Greenhill

2. Literature review Gamification (noun) is defined as: ‘the application of typical elements of game playing (e.g. point scoring, competition with others, rules of play) to other areas of activity, typically as an online marketing technique to encourage engagement with a product or service: gamification is exciting because it promises to make the hard stuff in life fun.’ Gamify (verb) gamifies, gamifying, gamified (http://oxforddictionaries.com/definition/english/gamification). In Karl Kapp’s recent book about gamifying learning, game‐thinking is defined as the most important part of gamification: 'thinking about an everyday experience...and converting it into an activity that has elements of competition, cooperation, exploration and storytelling' (Kapp 2012:11). Games exert a pull on human emotions because their design revolves mainly around the concept of success and failure. 'Games are built on reward structures: win the game and get the prize' (Kapp 2012:89). Reward structures keep players motivated to repeat even boring tasks by appealing to the human need for positive feedback. Upon completion of a task, a player will experience positive emotions. If a reward is given, these good feelings are amplified. The player receives immediate recognition of success: points, awards, power‐ups and collectables. Conversely, when players fail they feel negative emotions like stress and anxiety. The player wants to avoid these emotions, but desires to win the reward so he/she plays again. Motivation can be categorised as having two parts: intrinsic and extrinsic (Kapp 2012:93). Intrinsic motivation is primarily driven from within the learner, while extrinsic motivation is a result of external stimulus. Awards motivate intrinsically by stimulating the desire to win, penalties motivate extrinsically through introducing an externally applied punishment. Students demonstrated statistically significant higher levels of intrinsic motivation in the game‐based environment (Vos, Meijden, Dennessen 2010). Studies have identified three phenomena that repeatedly occur in team projects: the 'Free Rider' and resulting 'Sucker' effect (Kerr & Bruun 1993); and 'Social Loafing' (Karau & Williams, 1993) wherein people exert less effort to achieve a goal when working in a group than if working alone. Once the shirking begins, it can prove contagious to other students in the team. For example, the Free Rider, who doesn't contribute because they assume that other team members will do their work for them, gives rise to the Sucker wherein a fully performing team member lowers his/her efforts in response to the Free Rider. The effect of increasing resentments among other fully‐contributing members lowering their own efforts, often manifests through absence. Marburger (2001) found the 'percentage of students absent from class gradually increased as the semester progressed.' When absences from class occur at a low attendance point in the term it can impact negatively on team morale and performance. The challenge is to develop strategies to keep students equally contributing and thereby minimise the contagion of the Free Rider to other team members. Rampant absenteeism in university classes continues to be an issue for academia since Romer’s 1993 study revealed that, on average, only 66% of undergraduates were in attendance at any given university lecture or class (Romer 1993). More recently, an attendance study conducted at Glamorgan university found the combined average attendance of 748 Humanities students across 22 modules was less than 50% (Newman‐ Ford, Fitzgibbon, Lloyd and Thomas 2008). The problem persists in ‘universities across countries, universities and disciplines… Yet there is little evidence of university or governmental policy on it’ (Cleary‐Holdforth 2007). Student absence is particularly damaging for team projects for reasons already mentioned especially as the pattern of attendance is that it declines throughout the term (Marburger 2001). Therefore, just as an extra effort or push is required to complete a team‐based project, attendance is at its lowest ebb. Is the sole solution to enforce mandatory attendance? Luca Stanca (2006) argues attendance should 'definitely not' be mandatory, but at the same time agrees that 'steps should be taken to combat absenteeism because of the negative effect on learning'. Romer recommended further research into when mandatory attendance is appropriate and experimentation with enforcement frameworks. Incentives such as points for class participation, or unannounced quizzes that contribute to the final grade, have been shown improve attendance (Devadoss and Foltz 1996). Devadoss also observed classes taught by lecturers who had won teaching awards or who used an interactive teaching style were better attended.

Hope Caton and Darrel Greenhill Marburger measured the effect of enforcing mandatory attendance on two classes over the same time period during the 2002 and 2003 autumn semesters. In the two‐year study, classes were taught by the same instructor and met in the same time slot, 'therefore any differences in absenteeism could be traced to the enforced attendance policy' (Marburger 2006). The 2002 group was told attendance would not affect their grade while the 2003 were told absence would impact their grade. Summary data revealed that students under the mandatory attendance policy were, on average, absent from 11.6 % of the classes, while in the voluntary group, students were absent from 20.7% of the classes. 'Students who were absent during a class period were 9 ‐ 14% more likely to respond incorrectly to a question pertaining to material covered in their absences than were students who were present' (Marburger 2006). This method has been applied in the following way in this paper: In both years and for both groups, the lecturer kept daily attendance records of student absence. The link between attendance and achievement has been proved. Devadoss and Foltz (2001) calculated a student who attends all classes is likely to achieve, on average, 0.45 point higher grade than a student attending half the classes. A Durham university study linked first‐year student academic improvement with attendance to a minimum of 70% of classes. The study revealed that ‘if a student does not attend at least 80% of the time then their chance of failure is greatly increased’ (Colby 2004:11). It is important therefore for academics to be conscious of student attendance. ‘Immediate intervention following an instance of student absence, rather than after three or more consecutive absences, may help to combat this trend and ultimately help to improve students’ academic attainment’ (Newman‐Ford. et al). Issuing quick penalties for student absence may prove effective in halting the student’s slide into absenteeism.

3. Learning strategies The following learning strategies were used during the teaching of this module. Collaborative, Constructivist and Project‐based learning strategies were used for both groups. Competition‐based and game‐based learning were added in the trial year with the implementation of the awards and penalties framework. Collaborative‐based Learning (CBL) This is a methodology that focuses on activities that maximise the collaboration among students, either in couples or small groups, to improve learning by the exchange of information and knowledge among students (Burguillo 2010). Competition‐based Learning (CnBL) Here learning is achieved through a competition, but the learning result is independent of the student's score in that competition. It is an approach that caters for diverse learning styles and individual differences. CnBL is not to be confused with Competitive‐based Learning, wherein the learning is dependent upon the results of the competition. Constructivist learning Constructivist learning approaches underline the idea of an active, producing student in a situation where knowledge is not transmitted to the student, say via lectures, but constructed through activity or social interaction (Vos, et al 2011). Constructivist theories ask for different learning environments that are complex, realistic and meaningful. Such an environment contributes to student motivation to learn and invokes a high level of active engagement. Game‐based Learning (GBL) This method engages students as players in learning activities often by means of serious games which are designed to promote active student participation and interaction as the centre of the experience, as opposed to games developed purely for their entertainment value. When comparing the motivation of students who learned in the game‐based learning environment to those who learned in the traditional school environment.

Hope Caton and Darrel Greenhill Project‐based Learning (PjBL) This approach provides complex tasks based on challenging questions or problems that involve the students' problem solving, decision making, investigative skills, and reflections that are also supported by a tutor that provides facilitation. Class projects are intended to bring a deep learning in issues related with their education. In this method, the tutor specifies the greater task but the students self‐organise and self‐delegate the steps towards completion of the task. Burguillo's paper introduces a framework for using game theory tournaments as a base to implement CnBL, together with other learning techniques, in order to increase student motivation and performance. His results indicate 'the use of friendly competitions provide a strong motivation that helps increase student performance.' (2010) However, it is not clear to what extent CnBL improves student motivation and performance in team‐based projects. Team project‐based learning is one of the most 'commonly used methods to activate interactions among students' (Lee and Lim 2012). It is increasingly used as a teaching and learning method in higher education because it promotes higher learning skills including cooperative ability, critical reasoning, creative thinking, responsibility and communication. The so‐called 'soft skills', essential to employability. Yet current research does not address the problem of declining attendance and its effects on collaborative learning. When a particular student's expected contribution is not completed, or is submitted late it can have a disastrous effect on team morale. Papers have been written about strategies to address the difficulties for the lecturer in assessing individual contribution in student team project, like Peer Assessment, (Pond, Coates & Palermo 2010; Lee, H‐J and Lim, C 2012). Peer assessment strategies and frameworks aid the lecturer when assessing individual grades on a team project, but do little to solve the problems of team performance in real time, during the term. It is worthwhile developing frameworks to improve engagement and attendance on team projects in order to improve the student experience.

4. Research methodology The first component of this study quantitatively measured the effects of an enforced attendance policy on absenteeism. The second component compared the individual student marks, and team marks in the trial group against the individual and team marks of the control group. In the third component, team morale and motivation was speculatively reflected upon by the lecturer, with quotes from three students. The efficacy of the awards and penalties scheme on class attendance was evaluated by comparing student absence rates in the semester when the scheme was used, with those of the same semester in the previous year when the scheme was not used. The two years are referred to below as the 'control year' and the 'trial year'. This methodology is valid only if the teaching environment, the class assignment and assessment process was the same in both years. In this study the following parameters were the same:

The lecturer and module leader were the same in both years.

The assignment set for the team project was fundamentally the same in both years, though the individual component was modified from the keeping of weekly minutes of team meetings in the first year, to recording weekly work logs in the second year.

Assessment of the team project was based solely on course work presented and submitted at the end of term and was the same in both years. However, the method of individual assessment was modified in the second year to include the awards/penalties scheme and also a peer assessment component.

Accurate records of student attendance in class were kept in both years. In the control year students signed themselves in on a paper class register; in the trial year attendance was recorded and monitored by the lecturer.

There were 62 students registered in the control year (2012), 74 in the trial year (2013).

Hope Caton and Darrel Greenhill

5. Awards and penalties framework Awards were issued to the entire team while penalties were issued to individuals. The system of awards and penalties was disseminated via a class management document uploaded onto study space with the module guide and assignment description. It was also discussed and explained in class. The framework responds to the challenge to invent a system that would allow the class lecturer to respond quickly to student absence which had, in previous semesters, been observed to damage team morale. A system of yellow and red penalty cards to students for infractions such as absences or non‐completion of assigned tasks was discussed and approved by the faculty. Yellow cards were issued first, as a warning to the student and carried no marks deduction. If there was no improvement, a red card was issued and 25 marks deducted (out of 100) from the team assessment. To promote student ‘ownership’ of the framework, they were given a vote as to the extent of the marks deduction for a red card. Penalties were designed to also be a mechanism for students to request the lecturer issue penalty cards to their teammates. The goal here was to empower students to penalise ‘free riders’ and thereby mitigate the ‘sucker’ spiral. However, to establish a framework of penalties without awards would be contrary to game design principles. Awards are an essential part of any gamified system and therefore had to be included in the framework, but could not form part of the assessment as they were not included in the module descriptor or the learning outcomes. Awards were given in the following categories: Best Original Concept, Best Design, Best Programming, Most Fun Game, Best Team Cohesion, Game of the Year. The aim was to encourage team participation and boost morale by introducing friendly competition between the teams. Shortlisted projects were displayed on board in a much‐travelled corridor and all students were encouraged to vote. Winning teams had their pictures taken and displayed on the board as well as being promoted on the Kingston University website. One award, the Best Concept, was given out in week nine, just before the Easter break, and the rest were given at the completion of the project. A penalty card was issued by the class lecturer to the student via email. Cards were issued for any infraction that would damage team morale including: absence, continuous lateness, failure to complete tasks, failure to communicate with team members, failure to meet deadlines. Cards were issued by the class tutor via a direct email to the student. In the event of dispute between students and the class tutor, the final decision was made by the module leader and field leader.

6. Results In total, 18 students in the trial group were issued with penalty cards. In 13 out of the 18 cases, receiving a yellow card did result in improved attendance and participation. The five students who did not improve were issued with red cards and suffered a deduction of 25 marks from the team mark. 1. Did attendance improve? Attendance was used as the first measure of engagement and participation. Data sets for the two years were constructed using official class lists and records of attendance kept by the lecturer. The average attendance for the control year was 76%. In the trial year, attendance was consistently higher, averaging at 83%. The attendance was considerably higher in week 9, at a normally low period in term attendance patterns.

Figure 1: Comparison of average weekly student attendance during the term 2. Did performance improve?

Hope Caton and Darrel Greenhill Students who had withdrawn from the subject by the end of the second week were excluded from the analysis because these students did not seriously attempt the subject. To produce a measure of academic performance for this paper that was not directly affected by the awards/penalties scheme, any marks deducted for red cards were added back in to the final grade. Results show an increase in the percentage of students achieving a first or upper‐second class grade both on an individual comparison and when compared in teams. Table 1: Student attendance for the control and trial groups Control Group 2012

Date

Week

Class size

Number absent

Number present

30‐Jan‐12

65%

5‐Feb‐12

73%

13‐Feb‐12

87%

20‐Feb‐12

85%

27‐Feb‐12

78%

5‐Mar‐12

75%

12‐Mar‐12

75%

19‐Mar‐12

73%

26‐Mar‐12

68%

23‐Apr‐12

83%

30‐Apr‐12

69%

Average

75%

Trial Group 2013

Date

Week

class size

Number absent

Number present

28‐Jan‐13

81%

2‐Feb‐13

97%

11‐Feb‐13

84%

18‐Feb‐13

88%

25‐Feb‐13

82%

4‐Mar‐13

84%

11‐Mar‐13

77%

18‐Mar‐13

76%

15‐Apr‐13

84%

22‐Apr‐13

86%

29‐Apr‐13

74%

Average

83%

Figure 2: Shows the comparative individual mark distribution for the two groups

Hope Caton and Darrel Greenhill

Figure 3: Shows the comparative team mark distribution for the two groups Table 2: Showing the number of students in the class who achieved a particular mark Control group 2012

Trial group 2013

Mark

Students

Percentage

Mark

Students

Percentage

0‐9

4.84%

0‐9

1.35%

10‐19

0.00%

10‐19

0.00%

20‐29

0.00%

20‐29

0.00%

30‐39

0.00%

30‐39

5.41%

40‐49

17.74%

40‐49

18.92%

50‐59

19.35%

50‐59

14.86%

60‐69

25.81%

60‐69

28.38%

70‐79

20.97%

70‐79

24.32%

80‐89

11.29%

80‐89

6.76%

90‐99

0.00%

90‐99

0.00%

100.00%

3. Did team cohesion improve? The lecturer noticed a marked increase in class energy and enthusiasm in the trial group over the control group. Not everyone was motivated by competition for awards, but those who were motivated, were very motivated, spending hours in the lab working on the game project. Two students who were observed spending considerable additional time in the games lab were asked if the awards affected their motivation. Their responses were: ‘I think about things more, like in the middle of the night,’ said Mike Watts. His teammate, Konrad Jablonski agreed: ‘We push harder to make it better.’ The Best Original Game Concept award was instituted in order to improve participation in a class assignment resisted by students in the previous year: the game concept storyboard. In the module, student teams were tasked with creating a storyboard to illustrate their game levels, goals and score systems. In the control year only 2 out of 12 teams completed the task. In the trail year 10 out of 12 teams completed concept storyboards and these were done to a better standard. The Game Concept award was given out in week 9, on a day that would normally have poor attendance because it was at the end of term and just before a break yet the class attendance was above average. The graph (Figure 1) shows a marked improvement in the trial year over the control year that cannot be attributed to mandatory attendance because absences were increasing throughout the term regardless of the penalties. It was the chance of winning an award that motivated students to attend, and possibly curiosity played a part in attracting them. The effect of winning an award is motivational to the team to continue to work hard to maintain momentum and keep the standards high. The leader of the team who won the Best Concept award, Rob Dupre, offered this comment: ‘Winning the Best Concept award has definitely driven the group to produce work that fits the expectations that comes with winning a competition. The award has had a positive influence on the overall

Hope Caton and Darrel Greenhill motivation of the team. Personally the goal has always been to produce a piece of work that was of a quality suitable for showing at a game developer conference. Winning the Best Concept award made that opportunity more realistic: this further drove me to produce work of a high quality. Although never used within the team; the additional idea of subtracting marks from those people who failed to complete requested tasks certainly acted as an incentive for those who were close to the mark.’ Overall, it was noted by the lecturer and the module leader that games developed in the trial year were more sophisticated, had better design, playability and were completed to a more professional standard than in the control year. We believe this demonstrates that gamification improved team cohesion.

7. Discussion This study responds to the challenge to educators to 'identify and implement measures that will increase class attendance' (Devadoss 2001) by providing a framework of awards and penalties for application in team‐based learning environments. Figure 1 shows improved attendance in week 9, a marked contrast to the previous year. It was in this class the Best Game Concept Award was announced. Though attendance was higher in the trial group than in the control group, penalties were not, on their own, able to halt the slide in attendance that normally occurs as the term progresses. However, when the reward component was activated and the announcement of an award was scheduled for what was in the control group a high‐absence class, attendance improved by 16%: 69% attended class 9 in the control group, 84% in the trial. The framework does require more of the lecturer’s time. Lecturer monitoring and recording of attendance takes up, on average, 30 minutes of class time, However, this record proved useful as a reference tool when explaining to disappointed students the reasons for their low grades. Judging which team should receive which award required the collection and collating of votes from colleagues and students, which required additional effort on the part of the lecturer and a small additional budget to pay for prizes and a celebratory cake. However, deciding which team was to receive which award proved more difficult than anticipated. Votes were provided by staff and students and a clear winner did not always emerge. It was interesting to observe the various methods students used to beat the attendance penalty system. Arriving to class late and leaving early was the most common, which necessitated the recording of punctuality. To mitigate this, a 9.30 project meeting was scheduled for each class. If the student had not arrived by 9.30, he/she was marked late, with penalties issued for repeated infractions. Some students disputed the penalties, while others took it in good grace. Rarely were more than two emails required to settle disputes. In one case, the penalty was revoked. In the 2013‐2014 academic year, the one‐term Game Production module will be replaced by a two‐term module, Game Creation Processes, taught by the same lecturer and module leader. The awards and penalties framework will be modified and awards will be given out more frequently throughout the academic year to provide attendance ‘power‐ups’. In the new two‐term module, there will need to be a strategy for how to follow‐up a red card if a student continues to offend. Perhaps students who have been issued a red card will be removed from the team and placed in a team led by a lecturer or teaching assistant who may be able to exert more authority. The student would start again with a clean slate in terms of yellow and red cards, however the marks deduction would still be applied. In 13 out of 18 cases, the issuing of a yellow card was enough to improve student attendance, without requiring a corresponding deduction of marks. In this manner, the yellow penalty card could be said to function like a formative assessment. Further research could investigate the efficacy of awards and penalties for use in a more formative assessment.

References Burguillo, J. (2010). Using game theory and Competition‐based Learning to stimulate student motivation and performance. Computers & Education 55, (2010) 566‐575 Cleary‐Holdforth (2007) http://level3.dit.ie/html/issue5/cleary‐holdforth/cleary_holdforth.pdf (Accessed August 27, 2013)

Hope Caton and Darrel Greenhill Colby, J. (2004) Attendance and Attainment, 5th annual conference of the information and computer sciences – Learning and Teaching Support Network (ICS‐LTSN), 31 August‐2 September, University of Ulster. http:www.ics.heacademay.ac.uk/italics/Vol4‐2/ITALIX.pdf Devadoss, S. and Foltz J. (1996). Evaluation of factors influencing student class attendance and performance, American Journal of Agricultural Economics 78, 499‐507 Kapp, K. (2012) The Gamification of Learning and Instruction: Game‐Based Methods and Strategies for Training and Education. San Francisco: John Wiley & Sons Karau, S.and Williams K (1993). Social Loafing: A Meta‐Analytic Review and Theoretical Integration. Journal of Personality and Social Psychology, 65(4), 681‐706 Kerr, N.L. and Bruun, S.E. (1083). Dispensability of member effort and group motivation losses: Free Rider effects. . Journal of Personality and Social Psychology, 44, 78‐94 Lee, H‐J and Lim, C. (2012). Peer Evaluation in Blended Team Project‐based Learning: What do students find important? Educational Technology & Society, 15 (4) 214‐224 Lee, J. and Hammer, J. (2011). Gamification in education: what, how, Why Bother? Definitions and uses. Academic exchange quarterly, 15(2) 1‐5. Marburger, D. (2001). Absenteeism and Undergraduate Exam Performance. Journal of Economic Education 32(2) 99‐109 Marburger, D. (2006). Does Mandatory Attendance Improve Student Performance? Journal of Economic Education, 37(2) 148‐155 Newman‐Ford, L. Fitzgibbon, K. Lloyd, S. Thomas, S. (2008). A large‐scale investigation into the relationship between attendance and attainment: a study using an innovative, electronic attendance monitoring system. Studies in Higher Education 33(6) 699‐717 Oxford Dictionary"gamification". Oxford Dictionaries. Oxford University Press. http://oxforddictionaries.com/definition/english/gamification (accessed August 26, 2013) Pond, K. Coates, D. and Palermo, O. (2007) Student Experiences of Peer Review Marking of Team Projects. International Journal of Management Education 6(2) 30‐43 Romer, D. (1993) Do Students Go to Class? Should They? The Journal of Economic Perspectives 7(3) 167‐174 Rodgers J. (2002) Encouraging tutorial attendance at university did not improve performance. Australian Economic Papers September 256‐266 Stanca, L. (2006) The Effects of Attendance on Academic Performance: Panel Data Evidence for Introductory Microeconomics. Journal of Economic Education, 37(3) 251:266 Vos, N. Meijden, H. Denessen, E. (2010) Effects of constructing versus playing an educational game on student motivation and deep learning strategy use. Computers and Education 56 (2011) 127‐137

Game‐Based Learning in Health Sciences Education Nathalie Charlier, Evelien Luts and Lien Van Der Stock Teacher training in health sciences education, KU Leuven, Leuven, Belgium Nathalie.Charlier@pharm.kuleuven.be Evelien.Luts@pharm.kuleuven.be Lien.Vanderstock@pharm.kuleuven.be Abstract: The paper discusses the use of games specifically in the domain of health sciences, both in secondary and higher education. From a preliminary review we will present both traditional and digital games used to improve and/or assess young people’s knowledge in relation to health sciences. In addition, we will discuss three studies we have set up to investigate the effectiveness of a designed board game to teach and assess first aid competencies of secondary school and university students. Keywords: game‐based learning, game‐based assessment, health sciences

1. Introduction The concept of game‐based learning – or the use of games in learning and teaching – has been growing for many years now. Games that encompass curricular objectives are believed to hold the potential to render learning more learner‐centered, easier, more enjoyable, more interesting, and, thus, more effective (Papastergiou, 2009). During play students are actively solving meaningful problems (Antonietti & Cantoia, 2000; Price & Rogers, 2004). This aligns with Piaget’s (Piaget, 1952) understanding of knowledge construction. Despite their learning potential and the considerable academic interest in games, however, the uptake in a formal educational context remains limited (Kenny & McDaniel, 2011). Many top scholars have reported on the potential of harnessing the popularity of video games in particular (often called serious games or educational games) for engaging children and helping them to learn difficult concepts (e.g. Gee, 2005; Prensky, 2001; Squire, 2002). What is lacking, however, is concrete empirical data to support or refute these theoretical claims. Educational games seem to be effective in enhancing motivation and increasing student interest in subject matter, yet the extent to which this translates into more effective learning is less clear. In this preliminary review we include (quasi‐)experimental studies investigating whether using a traditional or digital game in a health sciences course in secondary or higher education can improve knowledge. The aim of the paper is to provide insight in the use of games in health sciences education (HSE) with a view to identifying the potential benefits of implementing games as educational tools into HSE courses. We will first start with a preliminary review and end with the discussion of three studies we have set up around our own developed board game.

2. Review In health sciences, the aim of most games is knowledge accumulation. Simulations, on the other hand, are mostly used for skills training. Neither simulations nor skills training are discussed in this review.

2.1 Method and inclusion/exclusion criteria A literature search was undertaken between February and May 2013 in the following international online bibliographic databases: ERIC, PubMed and ScienceDirect. The search string used was: “health education” AND game AND (teaching OR learning OR education). The use of discipline‐specific terms such as ‘nurses’, ‘students, nursing’, ‘medical’, amongst others and/or terms related to specific courses such as ‘anatomy’, ‘morphology’, ‘first aid’ amongst others yielded further resources. Searches were limited to articles published in journals and conference proceedings. Eligible papers were those that reported studies:

focusing on the use of a specific educational game or games, in comparison to another teaching/learning method such as a didactic lecture format.

Nathalie Charlier, Evelien Luts and Lien Van Der Stock

focusing on teaching and learning activities in health sciences for students in secondary and higher (for instance nursing and medical) education (aged 12 years or older).

based on systematic reviews, randomized control trials, experimental pretest/posttest control group design or quasi‐experimental structure.

using games such as card or paper‐based, board games as well as computer‐based games.

Papers were excluded if they were:

studies related to professional development (of for instance registered nurses).

directed towards health promotion (such as smoke prevention, healthy living, nutrition, prevention of alcohol abuse).

aimed at rehabilitation centers or schools for the disabled (and thus focusing on rehabilitation).

aimed at adolescents suffering from a mental or physical condition/disease.

evaluations of games using a tool such as a questionnaire or survey, as these did not allow comparisons with didactic teaching.

not available in English.

2.2 Results A total of 16 research papers were included in the preliminary review. Most studies found are executed in medical courses/topics at university level: breast imaging, pediatrics, cancer management, immunology, hepatitis, pulmonary physiology, ectopic pregnancy, and viral exanthema. 2.2.1 Overview Studies are inconclusive regarding the effect of games on improving young people’s knowledge. Several researchers have investigated the use of games to support classroom learning and found positive results in the experimental group compared to a control group (Cowen & Tesh, 2002; Khan et al., 2011; Vahed, 2008). Other studies demonstrated improved grades in a pre‐posttest set‐up without the use of a control group (Da Rosa et al., 2006; Eckert et al., 2004; Fukuchi et al., 2000). Other researchers reported similar positive effects of games, although their results were self‐reported by the students instead of measured (Boctor, 2013; Colombo et al., 1998; Moy et al., 2000; Ogershok & Cottrell, 2004; Patel, 2008; Roubidoux et al., 2002). While the studies above proved favorably towards the implementation of a game, other studies demonstrated no difference in knowledge accumulation between experimental and control groups (Annetta et al., 2009; O’Leary et al., 2005; Sward et al., 2008). The study of Rondon et al. (2013) reported similar results between experimental group and control group in terms of short‐term gains, but the traditional lecture proved to be more effective to improve students’ short‐ and long‐term knowledge retention. 2.2.2 In detail Roubidoux and colleagues (2002) developed an interactive computer game for teaching breast imaging to medical students. Case scenarios and questions were incorporated into the game. Students’ self‐reported answers on a survey show that they think the game provides additional reinforcement of learning beyond that of the handout or lecture. Also for medical students, Sward and colleagues (2008) designed a Web‐based game to assess their pediatric knowledge. The game is played in teams of two to four students, progressing around the board by “rolling” electronic dice and answering questions. The experimental group did not differ from the control group on content mastery. Fukuchi and colleagues (2000) developed the Oncology Game, a similar game as Sward and colleagues (2008), in order to teach medical students that cancer management is multidisciplinary. This is an interactive, computer‐assisted board game in which each team of two students is randomly assigned to two (out of 16) patient scenarios. The team has to obtain the best treatment for the patients by advancing through surgical, medical, and radiation oncology clinics. At the posttest, students demonstrated a significant change in the total number of questions answered correctly and a significant improvement in performance after playing the game. Another board game, although not computer‐assisted,

Nathalie Charlier, Evelien Luts and Lien Van Der Stock was designed by Ogershok & Cottrell (2004) to further student learning in the field of pediatric medicine. Players advance on the board by answering questions correctly. Results were not measured, but students and residents self‐reported that the game contributed to student learning. Colombo and colleagues (1998) developed a card game for medical students, named The Cellular and Humoral Immunology Game. The main aim is to answer questions correctly. Although there was no control group included in the study, results pointed out that most students declared that the game helped them to understand the subject. A similar card game, also on the topic of immunology, was developed by Eckert and colleagues (2004). The T‐lymphocyte and B‐lymphocyte self‐tolerance game is a card game used as a method for teaching immune self‐tolerance in medical students. The game consists of small cardboard pieces with illustrations and statements about self‐tolerance mechanisms. Students were instructed to associate the name of the mechanism with the corresponding cardboard piece correctly. Although there was no control group to compare, as in the study of Colombo and colleagues (1998), students’ grades improved significantly. Da Rosa and colleagues (2006) conducted a comparable study on the topic of hepatitis, using a card game in which students had to match explanatory cards with the respective clinical case. Students’ grades significantly improved after playing the game. As with other studies (Eckert et al., 2004; Colombo et al., 1998), no control group was included in this card game. Besides computer games, Web‐based games, board games and card games, researchers developed games for medical students based on television shows. Moy and colleagues (2000) turned Who wants to be a millionaire into Who wants to be a physician. The game was made for students to review material previously presented in class and included questions on the topic of pulmonary physiology. The researchers did not use a control group, but the participating students reported that basic information was integrated to understand broader concepts and mechanisms and that the thematic organization helped to develop higher levels of thinking. Both the research teams of O’Leary (2005) and Khan (2011) developed Jeopardy‐style games. O’Leary and colleagues (2005) designed a game about ectopic pregnancy in which students get points for each correct answer to a question. The game contained five categories and had cards with increasing dollar values containing questions in a correspondingly increasing degree of difficulty and complexity. Immediately after the class, both the lecture (control) group and the experimental group showed almost identical scores on the pre‐ and posttest comparison. Khan and colleagues (2011) designed a Jeopardy‐style game on the topic viral exanthema for medical students. The game included nine categories on the topic of viral exanthema. As in the research of O’Leary (2005), both the lecture (control) group as the experimental group showed significant but comparable improvement in their knowledge on the pre‐ and posttest comparison. Follow‐up test, after two months, however showed that the retention of knowledge was significantly better in the experimental group. Jeopardy‐style games are not only used in medical courses. Boctor (2013) designed Nursopardy, a game specifically for nursing students. The game was developed to reinforce fundamentals of nursing material, aiding students’ preparation for their final exam, including eight categories. The results demonstrate that students found the game to be a reinforcement of material learned, helped them review fundamentals of nursing information, and helped them to learn new information. A limitation in the study was however the absence of a control group. Jungman (1991) and Cowen & Tesh (2002) also developed games for nursing students, on the topics of respectively anatomy and pediatric cardiovascular dysfunction. Jungman (1991) used a control group to compare to an experimental group playing the board game ANATOMANIA. To move around the board, students have to answer questions about anatomy and physiology. If students answer correctly, they receive money, and whoever collects the most money, wins. The experimental group and the control group did not differ significantly with respect to pretest‐posttest scores. Cowen & Tesh (2002) also used a control group in their study. Results pointed out that the posttest scores in the experimental group were significantly higher than those in the control group. Apart from the games for medical and nursing students, researchers have designed games for university students of different health sciences: Speech‐language and hearing science, dental technology, and pharmacy. Rondon et al. (2013) designed a game for Speech‐ Language and Hearing Science students following the Anatomy and Physiology class. The researchers used a multiple‐choice quiz as a computer game‐based learning method. Only when considering the anatomy questions section, students scored better in the post‐ test than the control group. The control group, consisting of students that received the traditional lecture,

Nathalie Charlier, Evelien Luts and Lien Van Der Stock performed better in both post‐test and long‐term post‐test when considering the anatomy and physiology questions. This result, saying a traditional lecture seems to be more effective to improve students’ long‐term knowledge retention, is contrary to the results of the study by Khan and colleagues (2011), pointing out that the experimental group had better follow‐up scores. Vahed (2008) designed the Tooth Morphology Board Game to promote literacy and improve the students’ ability to retain with understanding the content area of Tooth Morphology. Results pointed out that the test scores on the course were better for the years in which the game was implemented, compared to year where it was not. In a study of Patel (2008), both instructor and students developed games to discuss patient care plans in a pharmacy course. Compared to lecture material, students felt that games did increase their knowledge, but were undecided on whether games improved their test scores. Almost half of the students felt that the games did not help them remember or understand test material. Only one study tested a game in a secondary school (Annetta et al., 2009). In this study, a Multiplayer Educational Gaming Application was developed by the biology teacher to cover key genetics concepts. Contrary to most results found in studies in universities, this computer‐based game did not demonstrate a greater understanding of the genetics concepts presented.

3. First aid and the board game “Land in nood” The second part of this paper focuses on implementation of a board game that we have developed to teach and assess first aid competencies of secondary school and university (teacher training) students. Since most cardiac arrests occur outside the hospital setting, it is, in most cases, the general public who will be responsible for providing initial basic life support (BLS) (Van de Velde et al., 2009). First aid (FA) training, including cardiopulmonary resuscitation (CPR), in the school curriculum is therefore essential, because it maximizes the number of potential first aiders in the community. A major challenge in skills education and first aid in particular is the assessment of competency (knowledge, skills and attitudes (KSA)). Testing technical skills require methods and assessment instruments that are somewhat different than those used for cognitive skills (such as paper‐and‐pencil tests. For instance, complex manikins are used to realistically simulate clinical cases, but learners are restricted to conducting physical examinations other than those for which the manikins are designed for (Gordon, 2001). Specially trained actors ‐referred to as “Standardized Patient” (SP)‐ portray patients with particular health concerns and are able to answer the full spectrum of questions about their condition (Eagles, 2001). However, because of the high costs for training, students are not exposed to a large number of cases and the encounters are often only used for summative assessment and not as formative learning activities (Hubal et al., 2000). As a consequence, alternative assessment methods that are economically and logistically feasible, need to be explored. Given the absence of an existing game that is in line with the first aid curriculum, we developed our own board game. “Land in nood” was designed in line with a standard work written by emergency medicine experts. Several trials testing the game and the pedagogical method were conducted beforehand to optimize the playfulness of the game, and the presentation of the content. This latter is presented by means of question cards targeting low level of knowledge (by means of true/false or short answer questions) and high level of knowledge (by means of open essay questions), as well as questions demanding a (simple or complex) skill demonstration. At this moment, we created a possibility to replace the question cards by an android app. However, we will not discuss the use of the app, given the preliminary development stage of this tool. Here, we will focus on three of the experimental studies we have set up:

a study in secondary education using the board game to teach new learning content,

two studies, one in secondary and the other in higher education, evaluating the validity of the board game as assessment instrument.

More detailed information can be found in our recently published research papers (Charlier, 2011; Charlier & De Fraine, 2013).

100

Nathalie Charlier, Evelien Luts and Lien Van Der Stock

3.1 The board game The game is played by groups of three to four competing students or student teams. The game board represents a landscape of a developing country built by the players as the game progresses. The game starts with a single terrain triangular tile face up and 69 others shuffled face down for the players to draw from. On each turn a player draws a new terrain tile and places it adjacent to the tiles of the progressing landscape in such a way that it extends features on the tiles it abuts: swamps must connect to swamps, fields to fields, seas to seas and bushes to bushes (Fig. 1).

Figure 1: Top view example of the game showing five tiles with the following features: swamps (pale brown), fields (dark brown), seas (blue), bushes (green) and building lots (white square). Other features that are shown in clockwise order from the left: four piles of question cards (green, blue, red and brown), two piles of the triangular tiles, a box with building blocks and some colored building blocks such as first aid posts and the tiles representing natural disasters After placing the new tile, that player chooses a blue, red, green or brown card with a true/false, short answer, performance and open essay question respectively. This question is read aloud by a competitive peer player. If the answer or performance is deemed as being correct by the peers, the player keeps the question card. If not, the card is placed aside. As soon as a player has collected a blue, a red and a green card, he is allowed to station a first aid post. Building is only allowed on a specific feature on the tile marked by a white square. If this feature is present on the newly placed tile, the player may opt to station a first aid post in exchange for the three collected cards. In a later round the player can transform his first aid post to a hospital in exchange for a brown question card. To overcome concerns about peer evaluation reliability and validity, the final decision had to be made via consensus by three peers, increasing the objectivity of the peer assessment. The outcome of the game (number of first aid posts and hospitals) was not used as assessment score, due to a factor of luck (i.e. the chance of drawing a tile with a white square) and sabotage (by natural disaster). Instead a summative individual score was generated by comparing the amount of individually collected question cards (representing the level of acquired knowledge) to the number of play rounds. A more detailed description of the game rules and course is described in our previous study (Charlier, 2011).

3.2 Game‐based learning as a vehicle to teach first aid content: A randomized experiment The goal of this study was to evaluate the learning effectiveness of our board game for acquiring first aid knowledge. We compared this to a traditional approach in the form of an interactive lecture giving a PowerPoint presentation, encompassing identical learning objectives and content but lacking the gaming aspect.

101

Nathalie Charlier, Evelien Luts and Lien Van Der Stock 3.2.1 Participants The study was carried out in general secondary education (science programme): a total of 120 students from 4 class groups of the 8th grade of a Belgian public school (middle SES, mainly Caucasian) participated. Fifty‐five (46%) students were female and 65 were male. The intervention took place over an 8‐week period. 3.2.2 Measurement of knowledge To analyze a learning curve, we measured the first aid knowledge of the students prior to the intervention (prior‐knowledge test), immediately after (post‐test) and 8 weeks later (short‐term retention test). We developed a paper‐and‐pencil test which consisted of 10 multiple choice questions. To assure comparability between the three measurement occasions 3 questions were identical, 2 questions were identical in content but were rephrased and 5 questions were different in content but similar in level of difficulty. A scoring key for all items of the written tests was designed in advance. 3.2.3 Results Compared to the prior‐knowledge‐test scores, we observed ‐ immediately after the intervention ‐ significant increases in knowledge scores as a result of both game play and the lecture. However, when we compare the game group with the traditional method group, we found a smaller learning gain in the game group. Two months after the intervention, the mean retention score of both groups decreased significantly compared to the post test results. The retention test results remained however significantly higher than the initial knowledge prior to the course. Although the lecture group obtained a significant higher learning gain compared to the game group immediately after the intervention, two months later we observed in this group a higher loss in knowledge. Our results demonstrate that – although self‐instruction by the board game alone was sufficient to teach first aid knowledge to secondary school students – the traditional lecture was in this case more effective in increasing student knowledge of first aid.

3.3 Game‐based assessment: Making assessment fun The goal of these studies was to evaluate the validity of our board game as assessment‐instrument of first aid knowledge (secondary education) and skills (higher education). Due to practical and logistical reasons we limited our assessment in secondary education to knowledge only (demonstration of skills was excluded). 3.3.1 Participants Study 1: A total of 303 secondary (grade 11 and 12) and post‐secondary (level 5) school students (237 women and 66 men) of 11 schools in Belgium participated. The mean age was 19.80 years old (SD=6.63). All students were enrolled in a first‐aid (FA) related course being part of their school curriculum, i.e. vocational education related to the health sciences (such as nursing, child care, pharmacy assistant). The full study took place over one and the same school year. Study 2: the group consisted of 55 students taking the one‐year preservice teacher training programme during or after their Master’s degree. The course, a FA training module (including BLS), is part of a compulsory, one‐ semester course (4 ECTS). Summative assessment – the focus of this study – took place during the last of four sessions. 3.3.2 The intervention In both studies, we investigated prior knowledge by setting up a pre‐test at the start of the FA course. This paper‐and‐pencil test consisted of two true/false, four short answer and three essay questions. The following weeks the FA module was given by their regular teacher/professor, who has given a FA course at least once before this study. The content of all FA courses was in line with the attainment targets and was derived from the standard work of the Flemish (Belgian) Red Cross. Within each of the two studies, we randomized the students of each class into two groups at the end of the FA course. In study 1, one group received a game‐based assessment (163 students) while the other one was

102

Nathalie Charlier, Evelien Luts and Lien Van Der Stock subjected to a traditional test (140 students). Assessment was organized within the same class time but in two separate rooms. In study 2, we assessed both groups by both assessment methods by means of a cross‐over design (Charlier, 2011). The assessments were scored by the peers. Assessment in the game format occurred while playing the game. Assessment in the traditional test format occurred at the end of the exam. All copies were collected and randomly redistributed among the students. Each student assessed the written test of a peer using a mark of 0 or 1 for each item, which was consistent with the scoring procedure in the game. 3.3.3 Results In both studies, mean scores of the game‐based assessment did not significantly differ from those of the traditional test, suggesting that the board game can serve as a valid alternative assessment instrument both in secondary and higher education. In study 1, we found no significant in learning gain between boys and girls, nor in the game‐based assessment nor in the traditional test. Also, girls in the game‐based group obtained no significant different posttest score than girls in the control group. We observed the same for boys. Several factors may favor the choice of the board game over the traditional exam. While traditional tests are most likely to induce a high degree of stress/anxiety resulting in a poor performance, games can be fun, motivating, and challenging and therefore able to dispel some fear of examinations (Desrochers et al., 2007). In this study, winning the game is not necessarily equivalent to an excellent performance in the assessment. Players who don’t like playing games and who have no insight or strategy could have performed poorly in building their first aid posts and hospitals, resulting in a low game result. To avoid this we opted to disconnect the results of the game play from the assessment score. Furthermore, using a game format creates the opportunity of assessing large‐size classes simultaneously. Peer assessment might be an alternative to overcome costly and time‐consuming assessment methods. To overcome concerns of reliability, observation matrices, extensively used during training, served as skills assessment instruments. Also a final decision was made by at least two peers to increase accuracy.

4. Conclusion Although games in learning and teaching have been studied for many years now, the empirical research on the effectiveness of educational games is fragmented and results remain inconclusive. The literature includes research on different tasks, age groups, and types of games, making it even more difficult to compare results and draw general conclusions. In addition, generalizations from research on the effectiveness of one game in one learning area for one group of learners cannot be made to all games in all learning areas for all learners. From our own experience with the board game in first aid, we suggest that when teaching concepts for the first time a lecture method is recommended, whereas, a game has a potential benefit in stimulating initial interest, or as a review method for previous instruction. For assessment purposes, we demonstrated that the board game assessment resulted in comparable exam scores compared to a traditional paper‐and‐pencil test. Given the positive and enthusiastic atmosphere that we observed during the board game assessment, we would recommend teachers to try out a game in the assessment process, since it might reduce the fear of examinations (Desrochers et al., 2007).

References Annetta, L. a., Minogue, J., Holmes, S. Y., & Cheng, M.‐T. (2009). Investigating the impact of video games on high school students’ engagement and learning about genetics. Computers & Education, 53(1), 74–85. Boctor, L. (2013). Active‐learning strategies: The use of a game to reinforce learning in nursing education. A case study. Nurse education in practice, 13(2), 96–100. Antonietti, A., & Cantoia, M. (2000). To see a painting versus to walk in a painting: An experiment on sense‐making through virtual reality. Computers and Education, 34(3), 213‐223. Charlier, N. (2011). Game‐based assessment of first aid and resuscitation skills. Resuscitation, 82(4), 442‐6. Charlier, N., & De Fraine, B. (2013). Game‐based learning as a vehicle to teach first aid content: a randomized experiment. Journal of School Health, in press.

103

Nathalie Charlier, Evelien Luts and Lien Van Der Stock Colombo, D., Fritsch, A., Ordovas, K. G., Spode, A., & Scroferneker, M. L. (1998). Playing with cellular and humoral immunity. Biochemical Education, 26, 20–21. Cowen, K. J., & Tesh, A. S. (2002). Effects of Gaming on Nursing Students’ Knowledge of Pediatric. Journal of Nursing Education, 41(11), 507–509. Da Rosa, A. C. M., Moreno, F. D. L., Mezzomo, K. M., & Scroferneker, M. L. (2006). Viral hepatitis: An alternative teaching method. Education for health (Abingdon, England), 19(1), 14–21. Desrochers, M. N., Pusateri Jr., M. J., & Fink, H. C. (2007). Game assessment: fun as well as effective. Assessment and Evaluation in Higher Education, 32, 527‐53. Eagles, J. M., Calder, S. A., Nicoll, K. S., & Sclare, P. D. (2001). Using simulated patients in education about alcohol misuse. Academic Medicine, 76, 395. Eckert, G. U., Da Rosa, A. C. M., Busnello, R. G., Melchior, R., Masiero, P. R., & Scroferneker, M. L. (2004). Learning from panel boards: T‐lymphocyte and B‐lymphocyte self‐tolerance game. Medical teacher, 26(6), 521–4. Fukuchi, S. G., Offutt, L. a, Sacks, J., & Mann, B. D. (2000). Teaching a multidisciplinary approach to cancer treatment during surgical clerkship via an interactive board game. American journal of surgery, 179(4), 337–40. Gee, J. P. (2005). Learning by design: Good video games as learning machines. E‐Learning, 2 (1), 5–16. Gordon, J. A., Wilkerson, W. M., Shaffer, D. W., & Armstrong, E. G. (2001). Practicing medicine without risk: students’ and educators’ responses to high‐fidelity patient simulation. Academic Medicine, 76, 469. Hubal, R. C., Kizakevich, P. N., Guinn, C. I., Merino, K. D., & West, S. L. (2000). The virtual standardized patient. Stud Health Technol Inform, 70, 133‐8. Kenny, R. F., & McDaniel, R. (2011). The role teachers’ expectations and value assessments of video games play in their adopting and integrating them into their classrooms. British Journal of Educational Technology, 42(2), 197‐213. Khan, M. N., Telmesani, A., Alkhotani, A., Elzouki, A., Edrees, B., & Alsulimani, M. H. (2011). Comparison of jeopardy game format versus traditional lecture format as a teaching methodology in medical education. Saudi medical journal, 32(11), 1172–6. Moy, J. R., Rodenbaugh, D. W., Collins, H. L., & Dicarlo, S. E. (2000). Who wants to be a physician? Reviewing pulmonary physiology. Advances in physiology education, 24(1), 30–37. Ogershok, P. R., & Cottrell, S. (2004). The pediatric board game. Medical teacher, 26(6), 514–7. O’Leary, S., Diepenhorst, L., Churley‐Strom, R., & Magrane, D. (2005). Educational games in an obstetrics and gynecology core curriculum. American journal of obstetrics and gynecology, 193(5), 1848–51. Papastergiou, M. (2009). Digital game‐based learning in high school computer science education: impact on educational effectiveness and student motivation. Computers and Education, 52, 1‐12. Patel, J. (2008). Using game format in small group classes for pharmacotherapeutics case studies. American journal of pharmaceutical education, 72(1), 21. Piaget, J. (1952). Play, dreams and imitation in childhood. Journal of Consulting Psychology, 16(5), 413‐414. Prensky, M. (2001). Digital game‐based learning . New York: McGraw‐Hill. Price, S., & Rogers, Y. (2004). Let’s get physical: The learning benefits of interacting in digitally augmented physical spaces. Computers and Education, 43(1–2), 137‐151. Rondon, S., Sassi, F. C., & Furquim de Andrade, C. R. (2013). Computer game‐based and traditional learning method: a comparison regarding students’ knowledge retention. BMC medical education, 13(1), 30. Roubidoux, M. A, Chapman, C. M., & Piontek, M. E. (2002). Development and evaluation of an interactive Web‐based breast imaging game for medical students. Academic radiology, 9(10), 1169–78. Squire, K. (2002). Cultural framing of computer/video games. International journal of computer game research, 2 (1). Sward, K. A, Richardson, S., Kendrick, J., & Maloney, C. (2008). Use of a Web‐based game to teach pediatric content to medical students. Ambulatory pediatrics : the official journal of the Ambulatory Pediatric Association, 8(6), 354–9. Vahed, A. (2008). The Tooth Morphology Board Game : An Innovative Strategy in Tutoring Dental Technology Learners in Combating Rote Learning, 467–479. Van de Velde, S., Heselmans, A., Roex, A., et al. (2009). Effectiveness of non‐resuscitative first aid training in laypersons: a systematic review. Ann Emerg Med, 54(3), 447‐457.

104

Specification and Design of a Generalized Assessment Engine for GBL Applications Yaëlle Chaudy, Thomas Connolly and Thomas Hainey University of the West of Scotland, Paisley, Renfrewshire, UK Yaelle.chaudy@uws.ac.uk Thomas.connolly@uws.ac.uk Thomas.hainey@uws.ac.uk Abstract: The interest towards the introduction of Games‐Based Learning (GBL) in education is increasing. Using computer games to assist the learning process offers a wide range of possibilities inconceivable in a traditional classroom. Assessment of the learner during game‐play represents a key challenge for GBL. This task is time consuming and requires both technical and educational knowledge. However, careful consideration of the integration of assessment in GBL is crucial as assessment has a very important role in teaching and learning; it is essential for teachers to assess how much their students have achieved the learning goals of a lesson and learners rely on assessment to receive feedback on their work. This paper discusses the specification of a generalized assessment engine that could be integrated into any GBL application. This paper first reviews the literature on traditional assessment and approaches to assessment in GBL. Next, the paper analyses the characteristics of a range of existing GBL applications. Using these characteristics, the literature on assessment and the outline GBL assessment model proposed by Hainey et al. (2012), a refined assessment model is presented. Finally, based on the refined assessment model this paper proposes a specification for a generalised assessment engine, provides an outline design and discusses the implementation of this design. Keywords: games‐based learning, assessment, assessment integration, assessment engine, assessment model

1. Introduction A teaching and learning process is incomplete without assessment. Indeed, assessment is essential to determine whether the teachers achieved their teaching goal and the learners achieved their learning goals. For GBL application developers, the integration of assessment raises several issues. Firstly, careful specification of the learning goals is essential; a developer needs to define what learning outcomes are being assessed. These learning outcomes can be as simple as “translation of colours in French” or “multiplication tables”, and as complex as “logical reasoning” or “team work”. Secondly, it is important to determine what form the assessment will take, whether it will be a multiple choice quiz or a pervasive assessment throughout the game‐ play. Then the developer will need to choose the type of assessment to be performed in the game: formative and/or summative. Finally, feedback should be provided; this may be positive, coming as a confirmation of the learner’s action or answer; negative, correcting an inappropriate response or it might also be triggered by a lack of motivation or a difficulty with the game rules. Feedback could also be external to the game, coming at the end as a form of debriefing. For each of these issues, this paper will develop the associated educational concepts, introducing learning outcomes, formative and summative assessment, and feedback. Next, and based on a review of the literature on the integration of assessment and feedback in GBL, and on an analysis of various existing GBL applications, we will propose a refined assessment model for Hainey et al.’s (2012) framework and will discuss the specification of a generalised assessment engine that can be used within GBL applications.

2. Assessment and its integration in GBL 2.1 Learning outcomes Kennedy, Hyland and Ryan (2007) propose two different approaches to education. Firstly, the traditional teacher‐centred approach where the teacher defines the content and plans and assesses the lesson. Secondly, the student‐centred approach that is based on learning outcomes. The European Qualifications Framework (EQF) (European Commission, 2008) promotes the latter approach and defines learning outcomes as statements of what a learner knows, understands and is able to do on completion of a learning process. They are defined in three terms:

Knowledge, the outcome of the assimilation of information through learning. Knowledge is the body of facts, principles, theories and practices that are related to a field of work or study. In the EQF, knowledge is described as theoretical and/or factual;

105

Yaëlle Chaudy, Thomas Connolly and Thomas Hainey

Skill, the ability to apply knowledge and use know‐how to complete tasks and solve problems. In the EQF, skills are described as cognitive or practical;

Competence, the proven ability to use knowledge, skills and personal, social and/or methodological abilities, in work or study situations and in professional and personal development. In the EQF, competence is described in terms of responsibility and autonomy.

Similar, albeit less explicit, definitions can be found in the literature (e.g. Bingham, 1999; Gosling and Moon, 2001; Donnelly and Fitzmaurice, 2005). It is highly beneficial for any teaching/learning process, including GBL, to carefully define the learning outcomes before working on the content.

2.2 Summative and formative assessment Assessment can be classified in two categories, summative and formative. Garrison and Ehringhaus (2007), state that “Summative Assessments are given periodically to determine at a particular point in time what students know and do not know” and that “Formative Assessment informs both teachers and students about student understanding at a point when timely adjustments can be made”. The main difference is, therefore, the aim of the assessment. The first approach is essentially for grading purposes and validation of knowledge while the second one contributes actively to the learning process proposing ways to improve. Robert Stake (cited in Scriven 1991, 19) proposes a culinary metaphor: “When the cook tastes the soup, that’s formative and when the guest tastes the soup, that’s summative”.

2.3 Feedback Feedback is crucial to the assessment process; it plays an important part in formative assessment and contributes to the learner’s motivation. Fleming and Levie (1993) introduce and illustrate five types of feedback options. They stress that the option has to be chosen according to the level of the learner and the stage of the learning:

Confirmation: Simple statement of the correctness of the answer is given: “Your answer was incorrect”.

Corrective: Confirmation of correctness and expected answer are given: “Your answer was incorrect. The correct answer was ‘Jefferson’”.

Explanatory: Confirmation of correctness, explanation and expected answer are given: “Your answer was incorrect because Carter was from Georgia. Of all those listed only Jefferson called Virginia ‘home’”.

Diagnostic: Confirmation of correctness and activities to undertake to improve are given: “Your answer was incorrect. Your choice of ‘Carter’ suggests some extra instruction on the home‐states of past presidents might be helpful.”

Elaborative: Confirmation of correctness and elaboration are given: “Your answer, ‘Jefferson’, was correct. The University of Virginia, a campus rich with Jeffersonian architecture and writings, is sometimes referred to as Thomas Jefferson's school.”

Hattie and Timperley (2007) propose a model feedback which introduces three questions effective feedback should answer: “Where am I going? (What are the goals?), How am I going? (What progress is being made toward the goal?) and Where to next? (What activities need to be undertaken to make better progress?)”.

2.4 Assessment integration in GBL Games‐based learning applications integrate the assessment process in many different ways. Some may choose to embed quizzes into the game whereas others monitor every action the learners perform to draw probabilistic conclusions on their achievement of the learning goals. Resulting from our last literature review (Hainey et al., 2013), we introduce six main approaches for assessment integration in GBL applications:

Monitoring of states: Monitoring the states of a game‐play allows the system to perform both summative and formative assessment by associating those states to a probability of achieving a learning goal. States can be as general as ‘level completed’ or ‘answer given’ but they can also show a more detailed knowledge of the game‐play with ‘location visited’, ‘non‐player character met’ or ‘content accessed’.

Quests: The assessment of learning is part of the game as a particular quest type. Each quest, when completed, can then be assessed. A non‐player character can give hints if the task is too complicated. The assessment quests can be ‘search the Internet’, ‘create content’ or ‘match description’.

106

Yaëlle Chaudy, Thomas Connolly and Thomas Hainey

Use of an assessment model or profile: The assessment is based on an existing assessment model or profile. The models commonly used are: Gagné’s nine events of instruction (Gagné, 1965) and IMS Question & Test Interoperability (Martínez‐Ortiz et al. 2006).

Non‐invasive assessment: The assessment is undertaken without the stress inherent to the assessment as the student is unaware assessment is taking place. In many cases, the game also adapts according to the student’s performances (e.g. triggering an intervention from a non‐player character). The pervasive assessment can be performed using, for example, Bayesian networks (Shute, Masduki and Donmez, 2010) or Petri nets (Thomas, Labat, Muratet and Yessad, 2012).

Quizzes: Quizzes are integrated at various stages of the game. The quizzes can be of various types, for example, multiple choice, true/false and fill in the blanks.

Peer assessment: This includes informal peer assessment in a forum, for instance, or giving a special status for some of the players enabling them to provide the others with feedback.

3. Analysis of existing GBL applications The first analysis undertaken focused on GBL applications that perform the integration of assessment with quests or quizzes. Ten web‐based games from various projects have been analysed to find common characteristics and to study their choices according to the notions introduced in Section 2. The aim of the analysis is to identify the commonalities and highlight the differences between these games. Table 1 summarises the analysis of the GBL applications investigated. In Section 5 of this paper, a specification of a generalised assessment engine will be given based on these games; the third example, Startup challenge 3, has been chosen to be used as the illustration. This game is short and simple; it will provide an ideal base to easily illustrate the rules described. However, the engine is generalisable to any of the applications stated here. Table 1 : Analysis of existing GBL applications Startup: Challenge 1 (startup.neues‐lernen.de 2009) The player hires the team selecting appropriate roles, skills and candidates. Startup: Challenge 2 (startup.neues‐lernen.de 2009) The player builds a path from an initial word to a final one using association of ideas. Startup: Challenge 3 (startup.neues‐lernen.de 2009) The player asks potential buyers questions and sells them appropriate products. Startup: Challenge 5 (startup.neues‐lernen.de 2009) The player completes a project’s to do list giving tasks to different people. Startup: Challenge 6 (startup.neues‐lernen.de 2009) The player divides his advertisement budget according to a population.

Learning Outcomes Competence: Hiring a team Skills: Data extraction from project specification Skill: Guessing words Knowledge: Most common letters in the English language Competence: Selling a product Skill: Interpretation of clients’ answers

Formative / Summative Summative: Score displayed at the end, calculated based on final team chosen

Feedback No

Summative: Score displayed throughout the game‐play, updated after every choice the player makes

Confirmation Feedback when too many mistakes: “Failed, choose another path” No

Competence: Tasks distribution Knowledge: Different roles and tasks to be done Competence: Dividing effectively the budget Skill: Analysis of the population

Summative: Score displayed at the end, based on the associations “person / task”

Summative: Score displayed at the end, based on the division of budget

CHERMUG (Boyle et al. 2012) The player performs a quantitative and a qualitative study.

Competence: Performing a study Knowledge: Terminology of a quantitative / qualitative study

Formative: Each answer selected is confirmed or corrected and further explanation is given

Confirmation to elaboration Expected answer not given

107

Summative: Score displayed at the end, based on the pairs “client / product”

Yaëlle Chaudy, Thomas Connolly and Thomas Hainey EnerCities (Knol and De Vries 2011) The player builds his own sustainable city. 3rd world farmer (3rdworldfarmer.com 2006) The player controls a family of third‐world farmers. Climate Challenge (bbc.co.uk 2007) The player, president, makes political decisions to stop global warming in 100 years. Stop Disaster (stopdisastersgame.org 2007) The player is responsible for a village; he has to reduce the damage caused by a disaster about to happen.

Learning Outcomes Competence: Building a sustainable city Skill: Management of resources Competence: Surviving and making the farm and family grow. Competence: Stopping global warming Skills: Political decisions and strategies Competence: Preparing a village for a disaster Skill: Dividing a budget to build shelters, hospitals, etc.

Formative / Summative Both summative and formative: A lot of information is displayed during the game‐play Score updated in real time Mostly summative due to the nature of the game; there is no correct answer. Score based on wealth, health, education, etc. Formative: For every turn, the decisions made change the goals (e.g. CO2, water, economy, popularity). Score based on these goals

Feedback Feedback is given if the level of a resource is low or if a goal is achieved

Summative: At the end of the game‐play the disaster occurs and the final result is shown, based on measures taken by the player

Elaborative feedback (e.g. why building a hospital was a good idea)

The newly calculated score represents a feedback

The new values for the goals represent a feedback

4. Proposed assessment model Taking into account the different approaches to assessment and the implementations of assessment in the literature review we have updated the preliminary model and framework proposed by Hainey et al. (2012) for assessment integration for serious games based on the Input/Process/Output Game Model (Garris, Ahlers and Driskell, 2002). Table 2 provides a description of each of the phases in the model. Figure 1 shows our proposed assessment integration model for serious games. Table 2: Description of the phases Phase

Planning

Design

Game‐ play

Debriefing

Description Specification of the overall learning outcomes Specification of the instructional content in terms of what particular instructional content should be integrated to correspond with the overall learning outcomes. Specification of the overall assessment strategy. Should the overall strategy be to facilitate formative assessment, summative assessment or a combination of both? What information, from the game‐ play, should be used for assessment? The design phase involves integrating the pedagogical content with the assessment involving subject matter experts with the game characteristics and possibly technical standards. The implementation of the assessment and how this should be displayed e.g. text, graphics, animation; integration of different quest types, utilisation of adaptation and assessment models in middleware, micro‐ adaptive non‐invasive assessment of competencies, quizzes, after action reviews and the monitoring and upgrading of states. One issue that should be addressed at this stage is to what degree formative assessment should be implemented. During the course of the game‐play learners will use their judgement, reflect and behave in reaction to the content and the assessment. Embedded formative assessment is integrated into the game and is therefore inside the game‐play cycle and possibly also summative assessment at periodic optional stages. External formative and summative assessment can be optionally performed outside of the game cycle. Formative assessment can take the form of constructive feedback from peers and instructors, and summative assessment can be paper‐based tests. Summative assessment would not be as frequent and would take place at key points. Both types of assessment could possibly alter the game to adapt to the player’s performances. When the game is complete participants can reflect during a debriefing period with peers and instructors. External summative assessment could optionally take place as at the end of a learning unit. On completion of the cycle, the learning objectives should be achieved.

108

Yaëlle Chaudy, Thomas Connolly and Thomas Hainey

Figure 1: Preliminary assessment integration model for GBL applications

5. A generalised assessment engine It quickly became clear that designing and developing a generalised assessment engine for use in GBL applications would represent a useful contribution to the area. Indeed, it would allow GBL application developers to save a considerable amount of time and cost, not only implementing the assessment process into their game but also thinking it through. One of the goals of the assessment engine is to provide game developers with a tool for assessment as well as with guidelines on how to perform the best assessment for their game. Importantly, an assessment engine would also allow the separation of the GBL application and its assessment, making the whole process more flexible and quicker and easier to change the score calculation of the game. After the analysis presented in Section 3, various patterns emerged about how the assessment is performed in existing GBL applications. The literature review (Hainey et al. 2013) and the summary in Section 2, provide starting points on how an ideal assessment should be carried out. The assessment engine specification relies on the modified framework proposed in Section 4 and will include the following:

109

Yaëlle Chaudy, Thomas Connolly and Thomas Hainey

A Domain‐Specific Language (DSL) to be used as an assessment configuration descriptor.

A generalised assessment engine with algorithm and DSL parser.

A communication protocol between the GBL application and the engine.

5.1 A DSL for the assessment configuration file 5.1.1 What is a DSL and why use one? A DSL is a programming language defined for a specific purpose. It has its own semantic and syntactic rules. Using a DSL for the description of the game assessment would enable developers to quickly and easily define their assessment without having to implement it with a general‐purpose programming language (GPL). In their paper, Mernik, Heering and Sloane (2005) outline the key issues for the development of a DSL and present situations where introducing a DSL is beneficial. They highlight the difficulty of DSL development as it requires knowledge in both domain and programming languages. Out of the patterns discussed where DSL is successfully used, the most relevant to our study is “task automation”. This pattern presents the introduction of a DSL as a way to automating common and repetitive tasks. In the scope of this paper, defining a generalised assessment engine that would make the assessment process quicker and more effective, development of a DSL seems to be an advantageous option. It would embed the domain (assessment in GBL) knowledge into the programming language; expertise of assessment integration would be situated in the engine itself and no longer in the game. The GBL application developer would still have to define the game’s assessment but could do so in an easier and quicker way, using our DSL to write an assessment configuration file. 5.1.2 What model for the DSL? We are at the early stages of the specification and design of the DSL, the preliminary language we propose is based on the educational concepts presented in Section 2, the model in Section 4 and the systematic literature review published in a previous paper (Hainey et al., 2013). The feature diagram presented in Figure 2 exposes the semantics of the preliminary DSL, values in bold with a grey background are terminal names of the language. Further description of the various components follows, along with an example of the language in use. The DSL itself can be seen as a guideline to the assessment process. Its intention is to encourage the GBL application developers to include, in their games, details of the assessment they might have omitted if they were implementing it themselves. However, in order to make the DSL more usable, a number of the components of the language were made optional. They are indeed omitted in numerous GBL applications as shown in the analysis in Section 3.

Figure 2: Feature model of the DSL

110

Yaëlle Chaudy, Thomas Connolly and Thomas Hainey Learning Goals. It is crucial for the objects of the assessment to be part of the configuration; learning goals are, therefore, the first component of the language. The name (id) of a goal is the only mandatory parameter but a developer can choose to give a description of the learning goal (text) and a starting value (integer, default value is zero). The final score is, by default, the sum of the values of the learning goals but can also be defined explicitly. All learning goals are fully defined in this component and their names will be available for use as a reference in the following blocks. Feedback. Section 2 states the importance of feedback in the learning/teaching process; it is natural to see them appear in the language as a component. Besides a name (id) and a message (text), a feedback can have a type (default value is neutral) and can be final (default value is false), meaning that when the feedback is triggered, the game ends. A piece of feedback is only defined once here and its name is used as a reference in the following blocks. Actions. This component contains the logic of the assessment and this is where we declare how the values of learning goals are updated. The user defines the meaningful actions of the game from an assessment point of view. He/she gives them a name (id), parameters if needed and, then, decides how many points should be added to a particular goal (by default the first one) according to the value of the parameters. The terminal name “others” can be used to define every parameter value not declared in the configuration file. The Reactions block allows the user to specify rules for sending feedback according to the parameters received with the action. If the feedback is specific to an action, the names of the parameters can be used in angle brackets and will be replaced by the corresponding value when the feedback is sent. Feedback Model. In this last part, the general feedback model of the game is defined, this time the trigger of the feedback is not an action but a broader concept such as the reaching of a limit (superior or inferior) for a learning goal value or a significant inactivity. The developer defines the meaningful concepts for the game and associates the feedback to be sent. Figure 3 represents the assessment configuration file that could be defined for the Startup_EU challenge 3 presented in Section 3. We intentionally added feedback in order to illustrate the totality of the DSL.

5.2 The assessment engine and the communication 5.2.1 Parsing of the DSL In order to use the assessment configuration file written in our DSL, the assessment engine needs to parse it and generate the appropriate classes and instances. Firstly, two classes, Feedback and LearningGoal are created and the first two blocks of the configuration file are used to instantiate the appropriate objects. In our example, we would have only one object of type LearningGoal and five objects Feedback. The lists of objects created are attached to the main class as parameters and two methods are created to search them: getGoalByName and getFeedbackByName. This main class has one more parameter, the final score of the game. According to the description defined in the configuration, a method, updateScore, is created. In our example we would have the score taking the value of the learning goal association as it is the only goal. Then, for every action defined in the DSL, a method is created with its name and parameter(s). This method tests the parameter(s), comparing them with the value in the configuration, and updates the goal value(s) and final score accordingly. The last method created, sendFeedback, is responsible for sending feedback to the GBL application. Figure 4 shows the class diagram generated after parsing the previous configuration file (Figure 3). 5.2.2 Architecture of the solution In terms of architecture, the two main options for implementation are a library and a web service. We have chosen the latter option for various reasons. Firstly, a web service offers more flexibility in terms of programming language used for the GBL application development; we do not have to restrict the language as a web service can be called from any type of application. Secondly, and on a longer‐term basis, it offers the possibility of creating a repository in which all the sessions of game‐play could be recorded. This would enable data mining for detection of issues with the game and with learning and patterns of learning/game‐play amongst players globally, potentially for all games that use the assessment engine.

111

Yaëlle Chaudy, Thomas Connolly and Thomas Hainey

Figure 3: Example of assessment configuration file using the DSL

Figure 4: Class diagram generated after parsing of the configuration file The architecture is built by invocation of the game creator as illustrated in Figure 5. At the creation of the game, the developer will define the configuration file and send it to the engine. The engine will then parse the configuration file and generate a web service capable of processing a specific comunication protocol to perform the game’s assessment. The engine also allocates resources for the game repository. The engine will then return the id the game should use to invoke the correct web service. Once the id is received, the game can be implemented using the web service and communication protocol.

112

Yaëlle Chaudy, Thomas Connolly and Thomas Hainey

Figure 5: Overview diagram ‐ creation of the game During the game‐play, the GBL application will communicate with the web service created. It will first request a session id for that particular game‐play, using start(). Then, it will send messages for every meaningful action. If defined, it can receive feedback from the engine at any stage of the game‐play. Finally, it will receive the notification of the end of the game. The sequence diagram, Figure 6, shows the communication protocol used with our example. All the information extracted from the game‐play is stored in the repository created in stage 1.

Figure 6: Sequence diagram of the communication game/engine

6. Conclusion This paper has summarised the important concepts of assessment in education and discussed the integration of assessment in GBL based on a previous literature review. From an analysis of existing GBL applications, and a refined assessment integration model, we have proposed a preliminary specification for a generalised assessment engine that could be integrated into any GBL application. The solution presented relies on a domain‐specific language (DSL) for the configuration of the game assessment, an engine able to parse the DSL and compute the assessment and a communication protocol between these two elements. The design of the

113

Yaëlle Chaudy, Thomas Connolly and Thomas Hainey DSL and the communication protocol are presented and the architecture of the engine, a web service, is described. Future work will include a refinement of the design and implementation of the engine, as well as a deployment and test on existing games and later the creation of a game using the engine from the start.

Acknowledgements This work has been co‐funded by the EU under the FP7, in the Games and Learning Alliance (GaLA) Network of Excellence, Grant Agreement nr. 258169 and the EU Lifelong Learning Programme under contract 518060‐LLP‐ I‐2011‐1‐UK‐COMENIUS‐CMP (StartUp_EU ‐ Be a High Tech Entrepreneur).

References 3rdworldfarmer.com (2006). 3rd World Farmer: A simulation to make you think. [online] Available at: http://www.3rdworldfarmer.com/ [Accessed: 24 Apr 2013]. Bbc.co.uk (2007). BBC ‐ Science & Nature ‐ Climate Challenge. [online] Available at: http://www.bbc.co.uk/sn/hottopics/climatechange/climate_challenge/ [Accessed: 24 Apr 2013]. Bingham, J. (1999). Guide to Developing Learning Outcomes. The Learning and Teaching Institute Sheffield Hallam University, Sheffield: Sheffield Hallam University Boyle, E., van Rosmalen, P., MacArthur, E., Connolly, T. M., Hainey, T., Johnston, B., Moreno Ger, P., Manjón, B. F., Kärki, A., Pennanen, T., Manea, M., and Starr, K. (2012). Cognitive Task Analysis (CTA) in the Continuing/ Higher Education th Methods Using Games (CHERMUG) Project. 6 European Conference on Games‐based Learning (ECGBL). 4 – 5 October 2012, Cork, Ireland. Donnelly, R and Fitzmaurice, M. (2005). Designing Modules for Learning. In: Emerging Issues in the Practice of University Learning and Teaching, O’Neill, G et al. Dublin: AISHE. European Commission. (2008). The European Qualifications Framework for Lifelong Learning (EQF). Luxembourg: Office for Official Publications of the European Communities. Fleming, M. and Levie, W. (1993). Instructional message design: principles from the behavioral and cognitive sciences. 2nd ed. Englewood Cliffs NJ: Educational Technology Publications, p.219‐222. Gagné, R. M. (1965). The conditions of learning and theory of instruction New York, NY: Holt, Rinehart & Winston. Garris, R., Ahlers, R. and Driskell, J.E. (2002). Games, motivation, and learning: A research and practice model. Simulation & Gaming, 33(4), 441–467. Garrison, C., & Ehringhaus, M. (2007). Formative and summative assessments in the classroom. National Middle School Association. Gosling, D. and Moon, J. (2001). How to use Learning Outcomes and Assessment Criteria. London: SEEC Office Hainey, T., Connolly, T.M., Baxter, G.J., Boyle, L. and Beeby, R. (2012). Assessment Integration in Games‐based Learning: A Preliminary Review of the Literature, paper presented at 6th European Conference on Games Based Learning, Cork, Ireland, 4‐5 October 2012. Hainey, T., Connolly, T.M., Chaudy, Y., Boyle, L., Beeby, R. and Soflano, M. (2013). Assessment integration in serious games. In Psychology, Pedagogy and Assessment in Serious Games. (In press) Hattie, J. and Timperley, H. (2007). The power of feedback. Review of Educational Research, (77), p.81‐112. IMS Question & Test Interoperability (QTI) Specification (2012). IMS Global Question and Test Interoperability (QTI) Specification. [Online] Available at: http://www.imsglobal.org/question/ [Accessed: 17 Apr 2013]. Kennedy, D., Hyland, Á., & Ryan, N. (2007). Writing and using learning outcomes: a practical guide. Cork, Ireland: University College Cork. Knol, E., & De Vries, P. (2011). EnerCities‐A Serious Game to Stimulate Sustainability and Energy Conservation: Preliminary Results. eLearning Papers, (25). Martínez‐Ortiz, I., Moreno‐Ger, P., Sierra, J.L., and Fernández Manjón B. (2006). <e‐QTI>: A Reusable Assessment Engine. Advances in Web Based Learning – ICWL 2006, Lecture Notes in Computer Science, 4181 p. 134‐145. Mernik, M., Heering, J. and Sloane A.M. (2005). When and how to develop domain‐specific languages. ACM Computing Surveys (CSUR), 37 (4), p.316‐344. Nyquist, J. B. (2003). The benefits of reconstructing feedback as a larger system of formative assessment: A meta‐analysis. Unpublished Master of Science thesis, Vanderbilt University. Scriven, M. 1991. Beyond formative and summative evaluation. In Evaluation and education: At quarter century. 90th yearbook of the National Society for the Study of Education, edited by M. W. McLaughlin and D. C. Phillips, 18‐64. Chicago: University of Chicago Press. Shute, V.J., Masduki, I. and Donmez, O. (2010). Conceptual Framework for Modelling, Assessing and Supporting Competencies within Game Environments. In Tech., Inst., Cognition and Learning, Vol. 8, pp. 137–161 Startup.neues‐lernen.de (2009). StartUp_EU. [online] Available at: http://www.startup.neues‐lernen.de/ [Accessed: 25 Apr 2013]. Stopdisastersgame.org (2007). Stop Disasters. [online] Available at: http://www.stopdisastersgame.org/ [Accessed: 24 Apr 2013]. Thomas, P., Labat, J‐M., Muratet, M., and Yessad, A. (2012). How to Evaluate Competencies in Game‐Based Learning Systems Automatically? In Lecture Notes in Computer Science, Volume 7315, Intelligent Tutoring System.

114

Safer Internet: Enhancing Good Practices on the Internet Through Games Based Learning for Greek Elementary School Students Vasiliki Choleva1, Loukas Koutsikos1 Simeon Zourelidis1, Vlassios Filis2, Dimitris Metafas3 and Charalampos Patrikakis3 1 ICT in Education, National and Kapodistrian University of Athens, Athens, Greece 2 Department of Informatics & Telecommunications, National and Kapodistrian University of Athens, Athens, Greece 3 Department of Electronics, Technological Education Institute of Piraeus, Piraeus, Greece vikyhol@hotmail.com loukaskoutsikos@yahoo.gr zourelid@otenet.gr v‐Filis@yahoo.com dmetafas@teipir.gr bpatr@teipir.gr Abstract: The Internet today has become an integral part of children’s and young people’s lives. They are the biggest user groups of online and mobile technologies all over the world. Children of Elementary School are often, because of their age, unprotected against traps on the Internet, such as cyber bullying, cyber stalking or sharing their personal information online. Today's Education and especially the Elementary School system should be considered as an ally as far as safer Internet issues are concerned. This paper, presents the implementation, by elementary school students, of a game about the ways of the Internet. The specific game was developed by the students themselves through Kodu, which is a visual programming tool especially designed for introducing children to programming principles. The aforementioned were held as part of their participation in an official innovating Educational Program entitled: “Safer Internet: Connect with Respect”. Seventeen students (eleven boys and six girls), guided by their teacher, produced a game scenario about the dangers of the Internet and ways to avoid them. This educational framework introduces children to the safety of the Internet through the excitement of creating technology. Keywords: online interaction, cyber bulling, safer Internet, visual programming

1. Introduction and motivation Types of media where everyday people can publish and subscribe to what others publish have changed the real world. The explosive growth of social media, smart phones and digital devices transforms children’s daily life at home and at school. An increasing number of children under the age of 13, use tablets and smart phones, download applications or join Facebook, usually unprotected and without their parents’ permission. Cyber bullying has become more common in society, particularly among young people. There are many incidents of young people who have been lured into meeting someone, previously unknown to them whom, who they contacted initially online and subsequently been harmed. Among the educational community there is a need to propose a way to educate children‐ particularly at young ages‐to take care, detect and avoid the risks associated with being online and using the Internet. An educational game created by children for children is the best way to promote the self‐motivation, cooperation, active participation of the students and the development of their creativity and imagination, in order to guide them into an Internet journey with safety and awareness. Greek Curriculum suggests using Educational Visual Programming Environments such as Logo ‐ like Programming Environments (Easy logo, Turtle Art,) or others like Scratch, Kodu and Game Maker which provide attractive interface and develop programming skills by designing and building their own game. Kodu is the visual programming tool that has been selected as it has a simple and icon based language and fully supports the various disciplines and the objectives of ICT literacy in the modern Curriculum. It also introduces students to the creative side of programming and as Matthew MacLaurin, Kodu’s lead designer, says “the core of Kodu is simulation design; the programming is secondary to that” (Lerner, 2009).

115

Vasiliki Choleva et al.

2. Educational approach Social networks have proved to be dangerous, especially for the children and the adolescents. Modern research has proved that a big percentage of these, does not hesitate to give personal information in the internet that can remain always there and anyone can have access. Even though the vast majority of the students use the Internet at their homes, at their private space, the parents can’t effectively control them. Children constitute the generation of digital technology and most of the times their knowledge is superior to their parents. School plays an important role in informing, educating, guiding and advising students on safety issues in the use of the Internet. School should inform students about the dangers that they face from the leak of their personal data, the electronic markets and the fake web sites which aim at misguiding them. School should “teach” them not to believe everything or anyone on the Internet. The teachers owe to inform, to educate and to make students capable of facing the dangers that derive from the use of the Internet. This educational role shouldn’t involve theoretical lessons which are rather boring but activities, projects or educational software which are interesting and motivating. The ever‐increasing technological changes offer new opportunities in the educational process and allow the use of new means of knowledge representation and the learner’s child’s participation in the learning process. Learners need attractive and interactive media and tools, so that their involvement with the Information Technology makes the learning experience more exciting. This new approach emphasizes creative pedagogy and requires student’s active participation by designing and building animations, stories or their own games. Children of young ages (mainly elementary school children), whose cognitive skills are being formed, develop their creative abilities as they think, design their own scenario, create and play their own game. Children respond to the gaming culture (Salcito, 2009) and as teachers, we could inspire students through some core concepts of gaming, like collaboration, creativity, action or extensive involvement (Prensky, 2007), for learning. Above all, these learner‐centered pedagogical approaches through modern Educational Programming Languages (pSkills, 2010) focus on developing the programming skills through learning activities, which are attractive to students and prepare them to apply their skills in later courses of programming, as they move on to more advanced platforms. Besides, the mission of games with visual programming language is not to teach children how to write code, but how to think like programmers (Wilson, 2009).

3. Educational framework The educational framework aims to introduce children of elementary schools to the ideas of the Internet and the way access to it and its services in a responsible way by creating a game for them, for their friends and for their classmates about the risks of being online and using the Internet. The teaching subject was “The potential risks of the internet” and the visual programming tool Kodu was selected for teaching the subject in real class conditions, on a case study with sixth grade elementary school students.

3.1 Methodology As modern learning models require the active and constructive participation of the learners, the educational intervention was based on the following axes:

Cooperative learning as the teaching method

Assure Learning Model Lesson into the teaching process.

116

Vasiliki Choleva et al. Assure is an Instructional Designed Model that is based on Gagne’s Events of Instruction and is intended to ensure effective teaching using Technology and Tools in the classroom (Smaldino & al., 2005). Assure identifies six major steps in an instructional planning process as it shows at the table below. Table 1: Assure lesson plan template into the teaching process 6th Grade (7th school of Ag. Paraskevi) 17 students; 11boys/6 girls Cooperative learning skills Programming skills Development of creativity and imagination Learning of composing stories Creation of a digital game based on students’ scenario Security over the internet (privacy protections, security, user education, informed consent and delighted users) Cooperative learning Kodu programming environment Introduction to Kodu programming environment (installation, tool palette, samples of other games, examples of the code) Creating Groups Designing Kodu world (characters and objects) Programming Characters Direct observation Evaluation form of the teaching process (completed by the students) Questionnaires (completed by the participants after the implementation) Presentation and Feedback (introduce the game to students, parents and teachers on the last day of the school year)

Analyze learners State standards and objectives

Select instructional methods, media and materials Utilize technology, media and materials

Require learner participation

Evaluate and revise

3.2 The educational intervention The educational intervention began in December 2012 and was completed in April 2013. Inspired by the Jesse Schell's "The Art of Game Design: A book of lenses" the teacher and the students discussed the topic of the "Safer Internet". From this discussion the students came up with the idea of a "labyrinth" which non‐surprisingly was expressing their feeling about the topic. Following a spiral approach, the teacher was guiding the students in designing the elements of the game, the game mechanics and the goals such as to be concrete, achievable and rewarding, while the students were enriching their "labyrinth" theme. The game managed to generate a strong experience exactly because its theme was designed according to the students' perspective and captured their feeling. Though the process of game design was based on the prior planning of the general game concept, the students were allowed to express themselves and also to comment on the ideas introduced before proceeding with the implementation. This approach allowed for a better communication of the links between the virtual ‐ electronic world objects (Kodus, lightnings) to the real world situations and corresponding hazards and safety tools (viruses, protection software). By the 5th of February (Safer Internet Day) students had finished the posters and the final scenario of the game and the selection of the storyboard took place. They had finished the list of the ideas to improve their game. They also had used the Kodu Game Lab Community in order to observe how other schools have used Kodu Programming Environment in their learning program.

117

Vasiliki Choleva et al.

Figure 1: The final game scenario

Figure 2: Some characters of the Labyrinth

Figure 3: Safer internet day‐posters They started to create the game world and were enhanced to program the characters and the objects.

Figure 4: Examples of programming characters of the game After the game was implemented, the students were called to create some accompanying material (in the form of a video) explaining the rationale, their views and thoughts on security over the Internet, and a guide to other students for the rules of the game “The Labyrinth”.

4. The description of the game The design and implementation of the final outcome that was created, the digital game named: “The Labyrinth”, derived from the principles that were reported at the section of the Methodology. The game world is a labyrinth whose corridors represent the chaotic Internet connections. The player is a student who needs to use the Internet in order to write his school assignment. At the entrance of the Labyrinth stands his own home while his school is at the exit. The player can handle “tools” that are in his way

118

Vasiliki Choleva et al. out of the labyrinth. Those “tools” are representations of the means intertwined with the actual Internet world (such as the blue lighting that is used as a firewall against the Red Kodus malware).

Figure 5: An overview of the game world The following table lists the characters introduced in the game, as well as their representations. Table 1: The characters that appear in “The Labyrinth” Active Characters (A.C.)

Representations

The Yellow Kodu (Main Character)

School student who surfs over the Net to seek for information in order to write his essay Crackers of minor hazard

The Red Kodus The Black Kodus

Crackers of major hazard

The Stick‐Guards

Recurring characters acting as player’s assistants against the dangers of the Net.

Discs

Packages of Information on the Net (e.g. Images, Video and Documents)

The Lightings

Internet Protection tools (firewall)

The Mines & Rover Robots

Hazardous internet threats ‐ Malware (Viruses, Trojans, Worms)

When the game starts the hero stands invisible at the entrance of the labyrinth, just next to his home. At that time the main character (Yellow Kodu) remains invisible just for 2 seconds and becomes visible just after entering the labyrinth. The main idea is to provoke the real life emotion of entering the Internet world through the lure of a labyrinth. While moving onwards, through the labyrinth, he finds some useful packages of information (images, videos and documents) for his school paper and faces the various threats of the game. When the user navigates to unsafe websites (by bumping onto the Red and Black Kodus), those unknown internet users ask for his

119

Vasiliki Choleva et al. personal data in order to let him download all the useful information he has found or in order to offer delightful presents. When the hero passes over them and downloads the useful data without offering any personal information they reveal their real purposes as they let viruses (mines), Trojans and worms (Rover robots). The player has to get the necessary Internet protection programs (lightings) or ask for his teacher’s help (Stick ‐ Guard), in order to repulse those attacks.

Figure 6: The Black Kodu asks for personal Figure 7: The Teacher helped our Hero As far as the Black Kodus are concerned, it’s worth mentioning that they represent the worst enemy that the hero encounters, as they are armed with viruses, Trojans and worms. In order to face their attacks, the player has to ask for the assistance of the Stick‐ Guard Teacher. This idea is an initial children’s thought, as they planned the game aiming to mark a basic Internet navigation principle; avoid sharing personal information and data with unknown users of the Net, as well as address to a “significant other” (Andersen et.al, 2002), such as the teacher or a parent for assistance. Merits to be noted, that as the player moves further inside the labyrinth the difficulty of the enemies increases. Furthermore, the gained points depend on the difficulty of each enemy as the player gains only 100 points for the easy Red Kodus, but 200 points for the more difficult Black ones. More bonus points are gained when the player:

Uses the firewall internet protection program (50 points), and

Downloads successfully the useful packages of information (getting the discs‐ 50 points)

Figures 8: Firewall (Blue Lighting) and teacher against internet threats The player wins on approaching his school, after having acquired all the necessary information for his essay. If the player gets to his school lacking useful packages of information, he is asked to return “online” and deal with outstanding issues. On the other hand the player can lose in two cases: A. By the end of seven minutes, and B. On the depletion of health after a large number of hazardous attacks with malware by the Red and Black Kodus.

120

Vasiliki Choleva et al.

Figure 9: The winner message

Figure 10: The game over message

5. Evaluation and future work In order to evaluate the effect that our approach had, after the completion of the implementation of the game, having allowed the students to work on the concepts (and links to game features) introduced in our approach as regards safety over the internet, we have processed the collected answers. The results indicate that all the children (100%) stated that they wouldn’t share personal information, such as name, age or home address to other unknown users online, 76% of them wouldn’t take part in online contests if they weren’t sure about their reliability and 71% would ask the opinion of a parent or their teacher when they find themselves in a state like this. On the other hand as far as the Kodu programming environment is concerned the students (88%) stated that they liked the Greek version, asked for more characters such as animals and weapons (82%) and 76% would like to be taught in school as accompanying material for the various school disciplines (e.g. mathematics, history, ICT). Last but not least almost all the children that took part in this survey (94% positive review) indicated that they enjoyed working together on the design of the game. Our intention is to further evaluate the work done, and to test the general effect that our proposal could have in the future use of skills and experience learned by the students, outside the specific scope of internet safety. This includes the reuse of concepts and practices in the design of other games or even using the concept of games to approach other educational goals.

6. Conclusions The education community needs the creation of supporting structures. These structures should aim at the continuous briefing of students on the secure use of the Internet and the possible dangers. The teachers should offer not only the necessary knowledge but also the attitude towards the safer use of the Internet. This attitude can be developed through the inter‐thematic approach of various cognitive domains where students will cooperate in challenging “real life” simulations and activities and will use their fantasy, their creativity and their curiosity. In this way, learning will become appealing and effective, just as it happens when the children use or create digital games. The paper focuses on the development of digital applications by utilizing the potentials of the visual programming tool, Kodu, in order to propose some good practices for teaching young children “surfing” over the Internet with security, in a pleasant and more effective way. The response of the children that took part in this project is related to the wider effort in the Greek society to prevent cyber bullying or other kind of mistreatment against minor users of the Internet. Such an example is the television and radio campaign named “Safer Internet” held by the Greek Center for Safe Internet. The overall education intervention that was carried out aims to indicate the value of Game Based Learning as a good practice that can be applied in the classroom, as well as to share the experience of new opportunities for learners, into the teaching process.

121

Vasiliki Choleva et al. As Kafai (2009) has stated: game making activities offer an entry point for young gamers into the digital culture, not just as consumers but also as producers. Thus, we have to offer to learners the excitement to create technology.

References Andersen, S., Chen, S. and Miranda, R. (2002) Significance Others and the Self. Self and Identity, Vol 1, pp 159‐168 Gee, J.P. (2005) Why Video Games Are Good for your Soul: Pleasure and Learning, Common Ground, Australia nd Jonassen, D. (2000) Computers as Mind tools for Schools Engaging Critical Thinking, 2 edition, Prentice Hall, Ohio. Kafai, Y.B. (2006) Playing and Making Games for Learning: Instructionist and Constructionist Perspectives for Game Studies. Games and Culture, Vol 1, No. 1, pp 36‐40. Lerner, E. (2009) “Kodu doesn’t have realistic graphics, huge explosions, or even a way to win. But it just might change the way we think about the world”, [on line], http://seedmagazine.com/content/article/serious_fun/P1/ Morrison, G. R. and Lowther, D. L. (2005). Integrating computer technology into the classroom, 3rd edition, Prentice Hall, New Jersey. Morrison, G. R., Ross, S. M., Kalman H. K. and Kemp, J. E. (2011). Designing Effective Instruction, 6th edition, John Wiley & Sons, New Jersey. Prensky, M. (2007) Digital Game‐Based Learning, Paragon House. pSkills (2010) “Programming Skills Development in Secondary Education by means of Modern Educational Programming Languages”, [on line], http://pskills.ced.tuc.gr Roblyer, M. D. (2006) Integrating educational technology into teaching, (4th ed), Prentice Hall, New Jersey. Salcito, A. (2009) “Integrating Kodu and gaming into the classroom”, [on line], http://blogs.msdn.com/b/microsoftuseducation/archive/2009/07/09/integrating‐kodu‐and‐gaming‐in‐the‐ classroom.aspx Smaldino, S., Russel, J.,Heinich, R. and Molenda, M. (2005) Instructional technology and media for learning, Prentice Hall, New Jersey. Wilson, C. (2009) “Logo on Steroids: The new video game Kodu will teach you (or your kid) about programming. It’s also actually fun”, [on line], http://www.slate.com/id/2222546/

122

Using Game Mechanics to Measure What Students Learn Jill Denner1, Linda Werner2, Shannon Campe1 and Eloy Ortiz1 1 Research Department, ETR (Education, Training, Research), Scotts Valley, California, USA 2 Computer Science Department, University of California, Santa Cruz, California, USA jilld@etr.org linda@soe.ucsc.edu shannonc@etr.org eloyo@etr.org Abstract: Despite the growing popularity of teaching children to program games, little is known about the benefits for learning. Making a game involves formulating complex problems, designing systems, and understanding human behavior, but these constructs have proven difficult to measure. In addition, studies of what children learn often ignore the social context in which game programming occurs. In this article, we propose that game mechanics can be used as a window into how the children are thinking and we describe a strategy for using them to analyze students’ games. We describe how the game mechanics categories were identified, and the results of the game analysis, including variation in the mechanics used by students working alone or with a partner. The study involved sixty 10‐14 year old students in the US who spent 10 hours learning to use the Alice programming environment, and 10 hours designing and creating their games, alone or with a partner. Forty games were coded for five game mechanics that require the programmers to think in ways that are dynamic, time dependent, or complex. The results suggest that students were most likely to include mechanics that engage the player and programmer in thinking about dynamic systems, and least likely to include reasoning that resulted in a conditional change in game state based on time. Working with a partner resulted in a broader range of mechanics, which suggests a deeper understanding of how to formulate problems, design systems to represent them, and consider the interaction of the player with that system. The findings contribute to efforts to assess what novice programmers learn by creating games, and suggest that the analysis of game mechanics is a useful strategy for assessing the range of complex problem solving during game design and programming. The findings can also contribute to efforts to create developmentally appropriate instructional approaches that engage students in complex problem solving. Keywords: creating games, children, assessment, complex problem solving, game mechanics

1. Introduction The field of games and learning has exploded in the last decade, but most of this work focuses on game play, rather than on the creation of games. A growing number of freely available and novice‐friendly game authoring tools has led to increased interest in the educational benefits of computer game programming (CGP). Despite this interest, little is known about what novices learn from CGP, due in part to assessment‐ related challenges. In this article, we describe a strategy for assessing whether CGP can engage students in computational thinking, which involves formulating as well as solving complex problems. st The ability to solve complex problems (CPS) is a characteristic found in most lists of essential 21 century skills (Computer Science Teachers Association 2011). CPS involves tasks that are dynamic (each action changes the environment), time dependent, and complex (require a collection of decisions that determine later ones) (Quesada, Kintsch, & Gomez 2005). Efforts to study these features have focused on how students attempt to solve problems, but we argue that the design of a complex problem is particularly relevant to computer‐based CPS. Designing and programming a game is what Jonassen (2000) has described as a ‘design problem;’ it is ill‐ structured, which means the student defines the goal, the solution path, and how to evaluate the solution. The task of programming a game offers the opportunity for students to engage in CPS tasks, but it also allows them to create complex problems (for the game player). We examine CPS in the context of middle school students making computer games. The “problem” is the situation that the game creator has formulated for the player to deal with. For example, games can include situations that are dynamic, where the task environment is a system that is influenced by and influences the player’s actions. In addition, games can include features that are time dependent; for example, the challenge of a game is increased by a time limit in which the tasks must be completed. Finally, games can be complex in that a decision or series of decisions during game play determine the available game play decisions and outcomes.

123

Jill Denner et al.

1.1 Computational thinking To assess how students design complex, computer‐based problems, we draw on recent research on computational thinking (CT). Wing (2006) stated that “CT involves solving problems, designing systems, and understanding human behavior, by drawing on the concepts fundamental to computer science” (p. 33). But CT is not only about solving problems, it is about designing or formulating problems (Barr and Stephenson 2011) including creating models and simulations, and programming computer games (Lee et al. 2011). In this article, we extend this work to look at aspects of CT that involve formulating complex problems, designing systems, and understanding human behavior. The aspect of CT that involves engaging students in the formulation of complex problems is situated within the theoretical perspective of constructionism. In this view, learning is the process of knowledge construction in the context of social and cultural participation (Kafai 2006). Learning about complexity requires students to be actively engaged in design and modeling activities (Hmelo, Holton, & Kolodner 2000); when making a game, students define the narrative, goals, rules and choices to be negotiated by the player. The programmer must not only know how to add code, they must also consider the design features‐‐how the player will interact with the game (Peppler & Kafai 2007), the outcomes of player action, what feedback will result, and the goal of the game (e.g., how the player will win or lose). This process of creating interactivity requires problem posing and testing (rather than just problem solving), a process that is linked to increased flexibility in thinking, problem solving skills, and conceptual understanding (Smith & Cypher 1999; Silver 1994). The second aspect of CT in game programming involves designing and thinking in terms of systems, which is a key aspect of complex problem solving (Fisher, Greiff, & Funke 2012). Systems thinking requires knowledge of how individual parts work, as well as an understanding of how they interact to form a whole system (Shute, Masduki, & Donmez 2010; INCOSE 2006). Students must first imagine the system, then put the pieces together in a way that embodies the rules or laws (a logic) for how the system will run, including opportunities for the player to interact since the user is an integral piece of the system they are building. The third aspect of CT involves understanding human behavior. In the field of Game Studies, the term “game mechanics” is used to describe how the player interacts with the game rules, including the sets of rules that make games interactive and fun (or not) to play. Game mechanics are the actions, behaviors, and control mechanisms that are available to the player (Hunicke, LeBlanc, & Zubeck 2004); engaging with game mechanics moves the game play along (Sicart 2008). The game designer must engage in complex problem solving to create rules, interactions between the rules, and the mechanisms through which the player interacts with the game. In this article, we focus on CT, an aspect of CPS that includes the coordination of multiple interconnected features by automating a system that others can play. We propose that computer games made by children provide a window into how they are thinking because making a game requires not only programming, but the ability to design and coordinate the pieces of a system, and to consider the perspectives of the people who will play the game. The games provide evidence of the extent to which students were able to formulate a problem and build a model of that problem as a system of interconnected, complex pieces rather than as isolated elements.

1.2 Game mechanics as indicators of CPS Building on prior research, we have identified several game mechanics that require player thinking that is either dynamic, time dependent, or complex. The five mechanics are: player controlled movement, feedback, conditional change in game state, multiple outcomes, and an entertaining player experience. The inclusion of these mechanics reflects the game creators’ understanding of the importance of the close relationship between the game software or program and the player’s experience with this program. The breadth and depth of the game creator’s consideration of the player is an indication of his or her level of CPS. We include more detail about these five indicators below. The game creator must consider the ways in which the player’s actions create a dynamic system that can evolve independently of the creator’s intentions. For example, there may be opportunities for player‐ controlled movement of the camera, an object, or a character that is linked to advancing game play. In addition, the creator can make a responsive system, which gives visual or auditory feedback in response to

124

Jill Denner et al.

player action, which requires an understanding of how the system works (Jacobson & Wilensky 2006). For feedback to work, the game creator must make the challenge or goal of the game clear, and coordinate a change in game state that comes in response to the player’s actions. Game programming creates opportunities for the game creator to reason about time dependence, such as the order in which the player negotiates tasks. For example, the inclusion of a conditional change in a game state requires the player to negotiate multiple interrelated decisions (each dependent on the previous one) to play the game. It may mean that the player must complete a series of tasks (e.g., find the waterfall, fill the tank with water) before they can move on in the game (e.g., extinguish house fires), or click on all the fish before time runs out. This feature creates tension because the player must race the clock, find a hidden object, or avoid an enemy in order to reach their goal. Players are motivated to act because they want to release that tension by achieving a particular goal (Sicart 2008). Finally, game creators can create complex tasks for the player. Games with only one outcome (e.g., the player ends up in the same situation regardless of what they do) require less complex problem solving. However, in games with multiple outcomes, the coordination of these different parts of the system requires identifying and creating different, interconnected paths. In addition, game designers also make choices about whether and how to evoke entertaining or fun player experiences, by creating opportunities for the player to achieve a new level, including humor, or causing something to happen in the game that challenges the player’s expectations (Sicart 2008), and by using dialogue or a change of scenes or perspective to move the game play along. Few studies of complex problem solving consider the role of social interaction. There are clear benefits of social interaction for students’ cognitive development (Howe, Tolmie, Greer, & Mackenzie 1995; Rogoff 1991), performance (Barron 2000; Eseryel et al. 2012; McDowell, Werner, Bullock, & Fernald 2006), and persistence (Uribe, Klein, & Sullivan 2003). Partners may encourage each other to summarize and explain what they know, respond to immediate feedback, take time to work through what they do not understand, and ask questions‐‐ all high level thinking skills that improve performance (Palincsar & Brown 1984). An in‐depth examination of the social and cultural practices that influence CPS is beyond the scope of this study, so instead we will explore whether CPS looks different in the games made by students working alone or with a partner. In summary, this article describes a strategy for assessing the complex problem solving that is required to design and program a playable computer game and the results of that assessment, including variation across games made by students working alone or with a partner.

2. Methods Students in four technology elective classes at a public middle school in California used Alice 2.2, a drag‐and‐ drop 3D programming environment with pre‐defined objects and operations (Dann, Cooper, & Pausch 2009). They spent ten hours learning to use Alice by following step‐by‐step written instructions, and 10 hours programming games that were supposed to include: a goal, interactivity, and instructions. Participants were between 10‐14 years old (mean=12); 37% female, 45% white, 37% Latino/a, and 73% spoke English or primarily English at home. There was a range of parent education: 27% had mothers had a high school degree or less, and 38% had mothers who completed a university degree. The classes were randomly assigned to a pair programming or solo programming condition. The analysis focuses on 40 games by 60 students (20 by pairs, and 20 by solos) using an artifact analysis approach that is similar to that done by others of Alice programs (Rodger et al. 2009), of Scratch programs (Maloney et al. 2008), and of Kodu programs (Stolee & Fristoe 2011). Undergraduate computer science students were trained to score games in three of the five game mechanics: player‐controlled movement, conditional change, and an entertaining player experience. Each game was analyzed by at least two people, and discrepancies were discussed and resolved. The two additional mechanics (feedback and multiple outcomes) were scored by the first and third authors. The first author independently coded 20% of the games and the one discrepancy was resolved by discussion. The coding categories are described in Table 1.

125

Jill Denner et al.

Table 1: Game mechanics Game Mechanic Player‐controlled movement

Feedback

Conditional change in game state Multiple outcomes Player experience

Definition and Examples Player controls an object that moves the game play along. Use arrow keys to find hidden objects Use letter keys to navigate a vehicle through a set of rings Game reaction as a result of player action. Click on the objects and they disappear When the character gets close to an object, it speaks Player must negotiate multiple interrelated tasks within game. Collect objects before the timer runs out Move an object while avoiding an obstacle The game progresses toward different endings. Opportunity to win or lose Evokes a fun, active experience through humor, changes in scenes and/or dialogue (not including instructions) that moves the game play along.

A key aspect of these mechanics is that they all are designed to evoke player actions that move game play along. For example, games had a conditional change in game state when the creator programmed something to happen as a result of a timer running out, or if the proximity function within a ‘while’ or ‘while/when something is true’ loop results in a player losing when they get too close to an enemy. The inclusion of multiple outcomes is designed to make the game progress toward more than one possible ending. Games were coded as having a win state even if there was no feedback that they had won, as long as the directions and goal were clear (e.g., the player was told that if they move the bunny across the forest and avoid the hawks they will “save the bunny” but no “win” message appears when the bunny disappears from sight).

3. Results In this section, we will describe what the data reveal about students’ complex problem solving, including which of the five game mechanics was most common, and which were difficult for students to make execute successfully. Then we will describe the CPS scores (a combination of the five mechanics). As Table 2 shows, the dynamic systems mechanics (feedback and player‐controlled movement) were most common. Games made by pairs had more mechanics (65 across the 20 games), than games made by solos (56 mechanics across the 20 games). One mechanic was more common in games made by students working alone: player‐controlled movement, and the least common mechanic overall was conditional change in game state. Table 2: Number of games with each mechanic Game Mechanic Pairs Solos Total Player‐controlled movement 10 17 27 Feedback 20 15 35 Conditional change in game state 8 7 15 Multiple outcomes 14 10 24 Player experience 13 7 20 Total 65 56 121

To generate an overall CPS score, games were assigned a 1 for each mechanic that was present and executable at least once, and a zero if it was either absent or did not work correctly. The number of indicators coded as “present” were summed to make a CPS score that ranged from 1‐5; a score of five suggests the creators engaged in a broader range of CPS. As shown in Table 3, there was great variation in the number of mechanics used in each game. Only four games (all made by pairs) had all five of the game mechanics, and two games had only one (both made by solos). The average for all games was 3.03 mechanics. Table 3: Number of games with each complex problem solving score CPS score Pairs Solos Total 1 0 2 2 2 7 8 15 3 4 2 6 4 5 8 13 5 4 0 4 Average 3.3 2.8 3.0

126

Jill Denner et al.

Table 4 provides examples of what each of these game mechanics looked like in a selection of games, whose scores range from 3‐5. The first row contains the game title, and a description of the goal of the game. Table 4: Examples of mechanics in four games

Game Mechanic Player‐ controlled movement Feedback

Conditional change in game state Multiple outcomes

Player experience

M808 Super Tank Battle: find and blow up cars before timer runs out. Move tank through city. When the cars are clicked on, text says BOOM, fire animation appears and printed counter increments. Must locate and destroy all 7 cars before timer runs out. If player destroys all 7 cars in time, there is a “win” message. If not, there is a “lose” message. None

Defeat the Dragon: answer questions to save the forest and princess None

Jet through Rings: navigate a jet through the rings

Fairy Game: escape the shark by finding the right fairy

Move jet through rings.

Move the male fairy to run over dinosaurs.

Characters talk and dragon shrinks after a response entered.

Pressing key ‘C’ gives player a cockpit view; counter increments when pass through rings.

Text indicates if right or wrong when clicks on fairies; dinosaurs disappear when they get close.

None

Must move jet through all the rings.

“Win,” “lose,” or “thanks for playing” (if player chooses the “not ready” button in the beginning). Dialogue between characters.

If jet goes through all the rings, the player passes. If not, they are asked to repeat the game. None

Must guess the correct character in time, or restart. If player clicks on the right fairy, they go to the next scene, if it is the wrong fairy, there is a “try again” message. Scenes change, dialogue including hints.

To illustrate these game mechanics, we describe Fire Truck Frenzy, one of the four games that included every type of mechanic. The goal is to extinguish house fires within a given amount of time; the player uses the arrow keys to move the fire engine to a waterfall, load it up with water, and then steer it to each house. The player experience is enhanced by text displayed when the engine is close enough to the waterfall (i.e., the engine says “I’m full!”) and when it is close to a house (i.e., “I’m empty again”) which prompts the player to return to the waterfall for more water. This game includes conditional logic that requires the player to take certain actions in a particular order, within a certain amount of time that eventually leads to a win or lose (i.e., a change in game state). If the player extinguishes all the house fires in the set amount of time, a congratulatory message appears. If the player does not extinguish all the fires before the timer runs out, all the houses disappear, meaning they burned up (i.e., the player loses by not successfully completing the game goal).

4. Discussion This article contributes to efforts to describe and assess what children learn by programming computer games. We show the viability of using game mechanics to measure complex problem solving. The findings suggest that students most frequently used mechanics with pieces involving feedback so that a player’s actions result in a dynamically changing environment, and required player‐controlled movement to move the game forward. Creating these mechanics requires students to engage in aspects of computational thinking that include using abstraction and algorithmic thinking, while coordinating a system of interconnected features that engage and respond to human behavior (Lu & Fletcher 2009). This assessment approach can also identify the reasoning processes that were difficult for the game programmers. Like others have found, the students were less likely to effectively integrate dynamic, non‐linear effects of agents in a complex system (Jacobson & Wilensky 2006) such as mechanics involving a conditional change in game state with multiple, interconnected pieces, and those in which the order or the amount of time taken for a player’s tasks are important. While making any game can engage students in abstract reasoning, the findings suggest that a limited number are creating and making use of different and interconnected levels of abstraction, another key feature of CT (Lu & Fletcher 2009).

127

Jill Denner et al.

In this article, we proposed an overall CPS score, but there are limitations to looking at the data in this way. To illustrate, we provide two interpretations for why only 60% of the games had a win/lose state, which is described as a typical feature of a game (Juul 2003). One interpretation is that it is difficult for 12 year old students to create a problem where attempts to resolve it move the player to one of multiple, interdependent outcomes. Another interpretation is that not having multiple outcomes (e.g., no way to win) was not a lack of CPS, but rather a choice. For example, in a study of the Storytelling version of Alice, Kelleher, Pausch, and Kiesler (2007) found that some students prefer to make a game that tells a story instead of one that involves competition and winning or losing. The findings show the importance of considering social context when assessing what students learn from CGP. Working with a partner resulted in a broader range of mechanics, which suggests a deeper understanding of how to formulate problems, design systems to represent them, while also considering the potential human behaviors of the players. This is consistent with a large body of research that shows how social interaction on open ended tasks promotes new ways of thinking and problem solving (Rogoff 1991). The most common game mechanic for students working alone (player‐controlled movement) involves event programming, which in Alice requires the least amount of coordination across the pieces of the programming environment. This is because Alice scaffolds the creation of the event handler (with automatic creation of underlying code) and facilitates understanding of this mechanism by showing the code in a separate window. Limitations of this study include the constraints of the Alice programming interface. Eseryel, Guo, and Law (2012) state that complex problem solving requires students to have an accurate “mental model they build to depict the cause‐and‐effect relationships among the factors affecting the complex problem situation….they need a good interface to lead them to identify the problem, collect necessary and all the resources that they need” (p. 263). In this study, we only analyzed games made with the Alice 2.2 programming environment because they were made in our second year of program implementation, after our instructional materials had improved and the teachers were more experienced. However, we expect that the games made using Storytelling Alice will be more likely to include a fun player experience because it is much easier to change scenes than in Alice 2.2, and there are more objects with built‐in methods that can be used for humorous situations, such as robots, clowns, and a school lunch lady. In addition, we chose to exclude games that had limited interactivity or too many bugs to be playable. Therefore, our results are not intended to indicate what middle school students are capable of; instead, they are intended to illustrate a strategy for assessing CPS. The findings have implications for assessment and instruction. The game mechanics categories may prove useful when analyzing games created in other programming languages to understand what students learn. While the underlying programming language constructs from which the mechanics are built are dependent on the language or environment (Denner, Werner, & Ortiz 2012; Werner, Campe, Denner 2012), some of the mechanics may be generalizable to other applications. A second potential contribution of the analytic approach described in this paper is for instruction. Educators want developmentally appropriate instructional approaches that engage students in complex problem solving; structuring game design around these categories could address this need.

Acknowledgements This research is funded by a grant from the US National Science Foundation 0909733.

References Barr, V. and Stephenson, C. (2011) “Bringing Computational Thinking to K‐12: What is Involved and What is the Role of the Computer Science Education Community?”, ACM Inroads, Vol 2, No. 1, pp 48‐54. Barron, B. (2000) “Problem solving in video‐based microworlds: Collaborative and individual outcomes of high achieving sixth grade students.”, Journal of Educational Psychology, Vol 92, pp 391‐398. Computer Science Teachers Association (2011) “National Standards for K–12 Computer Science”, [online], http://csta.acm.org/Curriculum/sub/K12Standards.html Denner, J. Werner, L. and Ortiz, E. (2012) “Computer Games Created by Middle school Girls: Can They be Used to Measure Understanding of Computer Science Concepts?”, Computers and Education, Vol 58, No. 1, pp 240‐249. Eseryel, D., Guo, Y. and Law, V. (2012) Interactivity Design and Assessment Framework for Educational Games to Promote Motivation and Complex Problem‐solving Skills. In D. Ifenthaler, D. Eseryel, and X. Ge (Eds.), Assessment in Game‐ based Learning: Foundations, Innovations, and Perspectives, pp 257‐286. Springer, New York. Hmelo, C.E., Holton, D.L. and Kolodner, J.L. (2000) “Designing to Learn about Complex Systems”, The Journal of the Learning Sciences, Vol 9, pp 247‐298.

128

Jill Denner et al.

Howe, C., Tolmie, A., Greer, K. and Mackenzie, M. (1995) “Peer Collaboration and Conceptual Growth in Physics: Task Influences on Children’s Understanding of Heating and Cooling”, Cognition and Instruction, Vol 13, pp 483‐503. Hunicke, R., LeBlanc, M., and Zubek, R. (2004) “MDA: A Formal Approach to Game Design and Game Research”, Proceedings of the Challenges in Game AI Workshop, Nineteenth National Conference on Artificial Intelligence. INCOSE (2006) “A Consensus of the INCOSE Fellows”, [online], International Council on Systems Engineering, www.incose.org/practice/fellowconsensus.aspx. Jacobson, M.J. and Wilensky, U. (2006) “Complex Systems in Education: Scientific and Educational Importance and Implications for the Learning Sciences”, The Journal of the Learning Sciences, Vol 15, pp 11‐34. Jonassen, D.H. (2000) “Toward a Design Theory of Problem Solving”, Educational Technology Research & Development, Vol 48, pp 63‐85. Juul, J. (2003) “The Game, the Player, the World: Looking for a Heart of Gameness”, Paper read at Level Up: Digital Games Research Conference Proceedings, Marinka Copier & Joost Raessens (Eds.), 30‐‐45 Utrecht: Utrecht University. Kafai, Y.B. (2006) “Constructionism”, In R.K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences, pp 35‐46. Cambridge University Press, New York. Kelleher, C., Pausch, R. and Kiesler, S. (2007) “Storytelling Alice Motivates Middle School Girls to Learn Computer Programming”, Paper read at the SIGCHI conference on Human factors in computing systems. ACM, New York, NY, USA, 1455‐1464. Lee, I., Martin, F., Denner, J., Coulter, B., Allan, W., Erickson, J., Malyn‐Smith, J., & Werner, L. (2011) “Computational Thinking for Youth in Practice”, ACM Inroads, Vol 2, No. 1, pp 32‐37. Lu, J.J. and Fletcher, G.H.L. (2009) Thinking about Computational Thinking. Paper read at SIGCSE, Chattanooga, TN. Maloney, J., Peppler, K., Kafai, Y., Resnick, M. and Rusk, N. (2008) “Programming by Choice: Urban Youth Learning Programming with Scratch”, Paper read at SIGCSE, Portland, Oregon, March. McDowell, C., Werner, L., Bullock, H. and Fernald, J. (2006) “Pair Programming Improves Student Retention, Confidence, and Program Quality”, Communications of the ACM, Vol 49, No. 8, pp 90‐95. Palincsar, A.S. and Brown, A.L. (1984) “Reciprocal Teaching of Comprehension‐fostering and Monitoring Activities”, Cognition and Instruction, Vol 1, pp 117‐175. Peppler, K. and Kafai, Y. B. (2007) “What Videogame Making can Teach us about Literacy and Learning: Alternative Pathways into Participatory Culture”, In Akira Baba (Ed.), Situated Play: Proceedings of the Third International Conference of the Digital Games Research Association (DiGRA) pp 369‐376. Tokyo, Japan: The University of Tokyo. Quesada, J., Kintsch, W. and Gomez, E. (2005) “Complex Problem‐solving: A Field in Search of a Definition?”, Theoretical Issues in Ergonomics Science, Vol 6, pp 5‐33. Rodger, S., Hayes, J., Lezin, G., Qin, H., Nelson, D., Tucker, R., Lopez, M., Cooper, S., Dann,W. and Slater, D. (2009) “Engaging Middle School Teachers and Students with Alice in a Diverse Set of Subjects”. In Proceedings of the 40th ACM technical symposium on Computer science education. ACM, New York, NY, USA, pp 271‐275. Rogoff, B. (1991) “Social Interaction as Apprenticeship in Thinking: Guided Participation in Spatial Planning”, In L.B. Resnick, J.M. Levine, and S.D. Teasley (Eds.), Perspectives on socially shared cognition, pp 349‐364. American Psychological Association, Washington DC. Shute, V. J., Masduki, I. and Donmez, O. (2010) “Conceptual Framework for Modeling, Assessing, and Supporting Competencies within Game Environments”, Technology, Instruction, Cognition, and Learning, Vol 8, No. 2, pp 137‐ 161. Sicart, M. (2008) “Defining Game Mechanics”, The International Journal of Computer Game Research, Vol 8. Silver, E.A. (1994) “On Mathematical Problem Posing”, For the Learning of Mathematics, Vol 14, No. 1, pp 19‐28. Smith, D.C. and Cypher, A. (1999) “Making Programming Easier for Children”, In A. Druin, The design of children's technology, pp 201‐222. Morgan Kaufman, San Francisco. Stolee, K. and Fristoe, T. (2011) “Expressing Computer Science Concepts through Kodu Game Lab”, In Proceedings of the 42nd ACM technical symposium on Computer science education. ACM, New York, NY, USA, pp 99‐104. Uribe, D., Klein, J.D. and Sullivan, H. (2003) “The Effect of Computer‐mediated Collaborative Learning on Solving Ill‐defined Problems”, Educational Technology Research & Development, Vol 51, pp 5‐19. Werner, L., Campe, S. and Denner, J. (2012) “Children Learning Computer Science Concepts via Alice Game‐programming”, Paper read at SIGCSE, Raleigh, N. Carolina, USA. Wing, J.M. (2006) “Computational Thinking”, Communications of the ACM, Vol 49, pp 33‐35.

129

Combining Game Based Learning With Content and Language Integrated Learning Approaches: A Case Study Utilizing QR Codes and Google Earth in a Geography‐Based Game Kyriaki Dourda, Tharrenos Bratitsis, Eleni Griva and Penelope Papadopoulou Early Childhood Education Department, University of Western Macedonia, Florina, Greece kdourda@gmail.com bratitsis@uowm.gr egriva@uowm.gr ppapadopoulou@uowm.gr Abstract: In this paper the GBL educational approach is combined with that of Content and Language Integrated Learning (CLIL) within the context of an educational geography computer game, utilizing QR Codes and Google Earth for teaching English Language to Greek Primary School students. This integration provides a motivational and cognitive basis for language learning, since it represents a meaningful, contextualized activity and on the other hand, gives students the chance to expand their cognitive skills and use more sophisticated language. The proposed game was utilized in the context of a pilot case study which immersed 11 to 12‐year‐old students in problem solving challenges regarding the use of geography in realistic contexts. Its purpose is not only to develop content knowledge but also to observe and enhance the learning strategies that students use while learning a foreign language. In attempting to solve these problems, students were engaged in eight‐week collaborative work, involving six levels of gameplay by following hints, provided by QR codes images. The findings of this case study suggest how foreign language learning can successfully take place within a geography game‐based learning environment, and they underscore the efficacy of approaching GBL in terms of performance. Students’ performance was evaluated through knowledge tests and various complex tasks throughout the game play, involving writing, reading and oral skills. In general, students showed positive attitudes towards the game and the post‐test results have significant differences compared to those of the pre‐test, in terms of vocabulary acquisition in the foreign language and geography knowledge. The results also showed that the collaboration required by this game, allowed the students to interact in a controlled environment, where they undertook roles and responsibilities. To this end, the findings will make an important contribution to the empirical evidence of GBL particularly with regards to its application in primary education. Keywords: QR codes, Google Earth, CLIL, language learning, GBL

1. Introduction The present case study is a follow up of a previous research proposal, in which an idea of combining the educational approaches of GBL and CLIL as a basis for foreign language learning was described (Dourda et al., 2012). GBL, which supports and facilitates the learning process in a more comfortable environment, and CLIL, which is believed to help students improve specific language terminology by centering on meaning, were effectively implemented in the context of the present case study (Anderson et al, 2008; Dalton‐Puffer & Smit, 2007). In this paper, the first results of the study are presented, focusing especially on language learning aspects, such as vocabulary and reading skills as well as the learning strategies, followed by the students. In addition, content knowledge and collaboration also move into focus. The main focus of the study was English Language Teaching, through an interdisciplinary approach which involves gaming, geography, problem solving and critical thinking. A detective game was designed and utilized as means of introducing geography related content in order to provide hints for the detection of a criminal. The case study was conducted with the participation of 17 students from a public primary school, in Greece. Thus, 11 to 12‐year‐old students were introduced to geographical information, as well as to post task activities, in a foreign language, while attempting to solve a mystery, by combining data within a game context. The aim of the study was to explore the teaching potential deriving from the combination of GBL and CLIL approaches in order to produce an authentic, meaningful and enjoyable learning context. The paper is structured as follows; initially the theoretical background is presented, discussing the GBL and CLIL teaching approaches within the Foreign Language teaching context. Then, the research methodology and results are presented, before the concluding discussion.

130

Kyriaki Dourda et al.

2. Theoretical background 2.1 GBL in foreign language (FL) learning context As researchers have continuously urged foreign language educators to seek alternatives to traditional instruction, during the last decade the GBL educational approach keeps increasing in the foreign language learning context, utilizing the ability of games to make language education entertaining and to provide learning environments that contextualize knowledge and immersive experiences for learners (Anderson et al, 2008; Meyer, 2009). GBL in particular embodies the philosophy of “learning through a grammar of doing and being” and provides students with opportunities for authentic learning (Squire, 2006). Through various kinds of software, GBL assists students in language learning, while developing their problem solving and critical thinking skills through engagement and iterative feedback that are crucial to the learning process (Donmus, 2010). Moreover, GBL approach to foreign language learning provides more effective learning compared to traditional methods, develops positive attitudes in students and increases the retention process (Donmus, 2010). GBL also provides the opportunity for Content‐based Learning (CBL), as “games are not necessarily about memorizing or providing correct answers”, but rather about the comprehension of the content provided in the game and the application of various learning strategies (Sørensen & Meyer, 2007).

2.2 Content and language integrated learning (CLIL) in FL context According to Eurydice (2006) the acronym CLIL is used to describe that school situation whereby “a foreign language is the vehicle to teach certain subjects in the curriculum other than the language lessons themselves”. The CLIL approach serves as a tool for the promotion of the foreign language teaching and has been praised on many different grounds. First of all, the primacy of meaning over form is considered to have positive effects on the affective dimension, reducing target language anxiety and increasing interest and motivation in the learners (Dalton‐Puffer & Smit, 2007). Secondly, it is thought that CLIL is effective, because by being able to comprehend and reason about a content in a foreign language, students have the chance to improve themselves in specific language terminology and generally to expand their cognitive skills (Lasagabaster, 2008). Furthermore, learning a foreign language through content provides an opportunity to teach academic tasks and higher order thinking skills which is not only beneficial for foreign language students, but also necessary for their overall success in school (Troncale, 2002). What is at issue in this paper is clearly the role of reading and lexicon in language teaching, which seem to benefit more from the CLIL approach. Tsai & Shang (2010) have shown that utilizing CLIL instruction enhances students’ reading comprehension as well as critical thinking ability. The greatest gain in terms of the language system, however, is undoubtedly produced in the lexicon, as through studying content subjects in the foreign language CLIL learners obtain larger vocabularies of subject‐specific terms (Dalton‐Puffer, 2002).

2.3 Foreign language learning strategies Gallo‐Crail & Zerwekh (2002), define learning strategies as “the special thoughts or behaviors that individuals use to help comprehend, learn, or retain new information”. For them, learning strategies are important to language learning because they enhance students’ own learning, and students use them for active, conscious, and purposeful self‐regulation of learning According to Oxford (2003) learning strategies can be classified into six groups: cognitive, metacognitive, memory‐related, compensatory, affective, and social. The main focus of the present case study is on the cognitive strategies (which relate to how students think about their learning and enable them to manipulate the language material in direct ways), the memory‐related strategies (which relate to how students remember language), the social strategies (which involve learning by interaction with others) and compensatory strategies (which enable students to make up for limited knowledge). Moreover, both reading and vocabulary are considered to be fundamental to foreign language competence and often constitute the greatest difficulty for learners. This study aims at examining their use of reading and vocabulary learning strategies (and whether an optimum way to deploy learning strategy training exists). With

131

Kyriaki Dourda et al. a special focus on a hypertext approach to foreign language reading, content knowledge can be acquired gradually and incrementally through repeated exposures to the various tasks in the game context. This can lead to richer content‐specific vocabulary use (efficiency that can be achieved by simultaneous twofold learning‐reading and vocabulary), and the more learner‐based nature of the foreign language acquisition. For instance, by exposing learners to an amount of unknown geography‐related vocabulary, the aim is to attempt to enhance the strategy of inferring word meaning from context, which seems to be ineffectively used by the learners. Besides, Coady (1993) emphasizes the importance of vocabulary building as an integral part of reading, viewing it as a strategic skill, necessary to be included in reading instruction (Kusumarasdyati, 2006).

3. Case study The purpose of the current case study was to improve sixth‐grade primary school students’ geographical knowledge and English vocabulary through a combination of GBL and CLIL settings. Moreover, an attempt was made to enhance and identify the learning strategies that students use while learning a foreign language, since their effective use is believed to be “one of the most important skills that students need to master in order to achieve success in language learning” (Gallo‐Crail & Zerwekh, 2002). Taking into account the aforementioned theoretical grounds, a game combining the benefits of GBL and the CLIL methodology was designed, in order to create a learning environment for acquiring geography‐related content, using the English language as a medium. The totality of the game‐play involved interacting directly with language, mostly through reading and writing. The expected outcome of this case study was for the students to develop the ability to identify information in order to solve a problem, identify resources to be used for gathering information (provided by the QR Codes), make decisions, analyze and present solutions in a written form. Students were also encouraged to engage positively in the learning process by directing their own learning, by being active, reflective and critical learners, by extending learning beyond the presented situation into new areas whereby some transfer of skills, abilities, knowledge and strategies may take place.

3.1 Game description This section of the paper focuses entirely on a more thorough description of the game itself, as the didactical concepts, the design principles and the game’s technological platform have already been analyzed (Dourda et al., 2012) “Whodunit” (for "Who done [did] it?") is a plot‐driven, web‐based detective game, suitable for 11‐12 years old students. The game is structured around six stations/levels (landmarks around the globe) in the form of missions, which comprise hypermedia learning material and relevant questions of progressive difficulty levels. In each mission, the students have to solve a number of problems, presented to them as open‐ended questions. The learning material encompasses websites with texts and images, relevant to both geography and a detective story. To successfully complete the game, students have to accomplish all six missions and to collect all clues that the suspect leaves behind. They are initially placed in their hometown (realistic context), where they undertake the role of a detective, following the traces of a notorious criminal, Mr. X. Within each level, the detective should correctly answer all the questions ‐ sent to the Chief’s e‐mail or written on paper ‐ posed to them, gaining the corresponding points in order to obtain a special key, granting them the right to proceed to the next level. The final goal is gather all the clues and the necessary information about Mr.X’s identity, thus arresting him by travelling at the location he is hiding in. The character design and narrative environments of the game were constructed in such a way as to foster students’ intrinsic motivation and sustain their persistent participation in the game playing process. The game is entirely web‐based and consists of three different sites – the Diary, the Chief’s office and the Internet. The Diary is the place where the detective writes information about Mr. X’s moves, gathers important evidence about his identity and describes the geographical location and the natives that inhabit this location (Figure 1). The Chief’s office is the place where the detective informs the Chief via letters, e‐mails, telegrams, post cards etc. about Mr. X’s whereabouts, provides the evidence gathered and the clue for the next location to be investigated (Figure 2).

132

Kyriaki Dourda et al.

Figure 1: Screenshot: Detective’s Diary – Investigation in the Amazon Rainforest

Figure 2: Screenshot: Chief’s Office – Letter to the Chief The Internet refers to the site where the detective searches for information about the geographical location he/she is going to travel next (Figure 3). Each time the students investigate a specific location, they take the opportunity to refine their search by gathering information about the geographical position, the climate, the local people as well as the flora and fauna of each location. The main goal of the game is to follow the track of Mr. X around the world, provided by clues hidden in QR Codes. This allows the students to “travel”, visiting famous geographical sites/locations and world attractions: the Sahara Desert, the Amazon Rainforest, the Grand Canyon, the Great Barrier Reef, the Himalayas and the Caspian Sea (Figure 4).

133

Kyriaki Dourda et al.

Figure 3: Screenshot: Internet – information about the Sahara Desert

Figure 4: Screenshot: Himalayas Mountain Range ‐ QR code hint pinned on Google Earth The location clue discloses the whereabouts of Mr. X, implying his next trace around the world. When the information is clear to the students, they can search the location on Google Earth and find the pinned QR Code that leads them to the appropriate website with further information. However, if the information is indirect and the students are unable to recognize their next destination, the QR‐Code hints help them find the precise location. In the various locations, students need to conduct some detective work in order to unveil the trace of Mr. X. Through investigating each location and with the help of the natives, important information is collected and listed in the clue log (Diary). The narrative frame of the game invites engages the students in a challenging, English speaking world. In this sense, English is learned through a way that is not unfamiliar to the students, but allows them to participate on a confident basis in the game. As described in section 2.2, by conducting all the prescribed activity in the foreign language, students are able to obtain larger vocabularies of subject‐ specific terms display “greater fluency, quantity and creativity and show the kind of higher risk‐taking inclination often associated with good language learners” (Dalton‐Puffer, 2002). The game provides a familiar and motivating context for the learning activity.

134

Kyriaki Dourda et al. Moreover, students can visit their journals, created for the game, to check and review the cases and the various evidences. By writing in a journal students will be able to assess their own activities, to see how they are doing and to evaluate their decisions and actions (metacognitive awareness) (Figure 5). Students/players level up by solving the mystery cases, investigating more clues, and hunting for Mr. X around the world. Additional rewards will be offered after accomplishing various tasks.

Figure 5: Screenshot: An example of a team’s journal

3.2 Research methodology The presented case study immersed seventeen students (11 to 12‐year‐old) attending a Greek Public Primary School in problem solving challenges. Students, who were chosen randomly, were engaged in eight‐week collaborative work, involving six levels of gameplay. First a survey of both students’ digital habits and learning preferences was conducted to establish a contextual baseline on which to place the study. The survey indicated that all the students were computer proficient and frequent users of computer games. Moreover, for the purposes of the study a knowledge test ‐ consisted of 30 multiple‐choice questions to measure students’ performance in geography and English vocabulary ‐ was conducted by the researcher according to the requirements of the educational game. The knowledge test was completed twice, once before the implementation of the game and once after it had been completed. The researcher used the same pre‐ and post‐tests to ensure the reliability of the test in terms of format, content and cognitive levels. After the implementation of the game, a satisfaction/feedback questionnaire was also distributed to the participants. After the pre‐test, the researcher introduced the game to the students and asked them to form groups of three (one team only consisted of two members). Each team member undertook a role, such as the manager, the computer user, and the journal keeper. The students were informed that the game process involved 45 minutes of instruction and one hour of playing. Instruction consisted of multimedia presentations (Prezi), in order for them to be introduced to each game level’s content and familiarized with the relevant geographical knowledge and unknown English vocabulary, necessary for them to proceed with the game. The instructive session was implemented in a game‐like format that seemed to motivate the students. After each instruction, students accessed the game online in the school computer laboratory. By completing each game level, the students were asked to respond to various complex tasks whose purpose was to assess and evaluate their content and language knowledge. Students’ evaluation began after each game session, including a variety of complex tasks that emerged students to write (on paper or online) e‐ mails/letters and conduct reports, describing their experience from each mission around the world, either by the detective’s or the indigenous inhabitant’s perspective. Other tasks also required from the students to record a video of themselves talking in English about their progress in the game, and even communicate

135

Kyriaki Dourda et al. through a message application on a mobile phone with the Chief (Figure 6). In that way, it was possible to obtain immediate data about their performance in Geography, their improvement in English vocabulary as well as reading and writing skills and, moreover, the various language learning strategies they used.

Figure 6: Screenshot: An example of a post task (evaluation)

3.3 Research hypothesis The research hypothesis is that the implementation of such a teaching approach will have the following beneficial influences on the students’: (a) geography knowledge, (b) English vocabulary and reading skills, (c) the use of foreign language learning strategies, (d) collaboration, (e) satisfaction with the game.

4. Results Through the game the playing process became a “serious” activity and the students achieved both the game and the learning objectives. The results are based both on acquired quantitative and qualitative data. Student’s familiarity with the computer (games), their satisfaction with the game and their language learning preferences were collected through questionnaires (quantitative data). Summative performance in the game collected through the pre‐ and post‐tests (quantitative data). Student’s use of learning strategies was obtained through observation, the researcher’s and students’ journal, video recording as well as through the various tasks the students achieved after each gameplay (game logs).

4.1 Student learning outcome The first research question examined the difference of students’ performance in the pre‐ and post‐test in terms of content knowledge. The post‐test scores revealed a significant improvement for all the students as their scores were higher (Figure 7). Specifically, the improvement of the students’ scores for the pre‐test to the post‐test was about 30% (average value). No significant difference was found among the participants, in correlation with their familiarity with computers, as they were all computer proficient. As far as the second research question is concerned, qualitative data obtained through observation, journals, video‐recording and evaluation tasks, showed that students’ vocabulary was improved. The researcher also noted the fact that students completed the post‐test without asking for help in order to understand the meaning of some questions or answers, as opposed to the pre‐test, where they faced difficulties due to the number of unknown words. Apart from that, students’ reading skills also improved through their continuous exposure to texts in the game. After the first two levels they started scanning for information, inferring word meaning from the context and as they progressed in the game they overcame any hindrance in the form of unfamiliar words.

4.2 Student language learning strategies The study also focused on identifying the learning strategies used during unintentional vocabulary learning. Moreover an attempt was made to assess the relationship between strategy use and vocabulary performance. As far as vocabulary learning is concerned, results indicate that certain learning strategies are more effective in

136

Kyriaki Dourda et al. acquiring new vocabulary words and that students have preferences in the strategies they use to learn these words in the foreign language (Figure 7). Data was collected through questionnaires, video‐recording and researcher’s journal.

Figure 6: Total correct answers per student in pre‐ and post‐test

Figure 7: The language learning strategies students used in the study More specifically, all the students preferred and used the memory strategies throughout the game. Students tended to retrieve information and understand unknown vocabulary via creating mental linkages, associating unknown with known words and using their imagination. Cognitive strategies were used by the 65% of the students (11 out of 17 students), and mainly by skimming and scanning in order to find information in the texts, translating them into their mother tongue, using resources (e.g. Google, online dictionaries) and taking notes in their journal. Social strategies were preferred by the 77% of the students (13 out of 17 students), as most of them were motivated to cooperate with their peers in order to achieve a common goal in completing the game successfully. They were also often asking questions for clarification or even correction both to their peers and the researcher. Compensation strategies, which were mainly noted in the researcher’s journal and video recordings, were widely used by all the students in order to overcome their limited knowledge in English. Students preferred to use gestures or facial expressions, to coin words and use a circumlocution or a synonym. Moreover, they were using clues in order to guess the meaning of what they were reading and frequently asking for help either by their peers or the researcher, as well as often switching to their mother tongue. However, data is still under analysis and as the results indicate that students used more than one learning strategy during the game process, it would be interesting to examine in which cases exactly and how often each strategy was used.

4.3 Student collaboration Allowing students to collaborate with peers is another issue, explored in this study. As students worked in groups, they were actively responsible for their own learning processes. Information retrieved from observation, the researcher’s journal and video‐recordings showed that students engaged in collaborative

137

Kyriaki Dourda et al. learning. The game process engenders teamwork, communication and collaborative spirits among the students. None of the teams identified a team leader, but instead all members acquired a role, such as that of the computer operator, the manager and the journal keeper.

4.4 Student satisfaction Furthermore, after the interventions, students’ views on the application they had used were elicited through a feedback/satisfaction questionnaire. All students reported satisfaction and enjoyment from their engagement with the game and commented that they were able to learn through playing. Over 60% of the students commented that the game improved their knowledge in English and Geography and almost 80% found the game challenging, interesting and exciting. In addition, 10 out of 17 students reported collaboration and teamwork as one of the most positive aspects of the game process. Moreover, according to researcher’s journal, throughout the whole gameplay, students were drawn to the game, especially to its attractive illustrations.

5. Conclusion This paper has suggested a game‐based concept for teaching a foreign language based on a case study implemented in a Greek primary school. The results suggest that the use of foreign language learning strategies as well as the geography‐related content were facilitated and improved. In addition, reading skills, lexicon, motivation and collaboration were enhanced. The reason that this game was selected for the described approach is the fact that it includes authentic material with a meaningful purpose, no confusing graphical interface or complicated control schemes. Instead it utilizes technological tools (such as Google Earth, QR Codes and websites) that can be easily manipulated by educators without any specific technological knowledge. Moreover, it can be applied in all levels of education in order to teach a variety of subjects and enhance a range of learning skills. Therefore it comprises an ideal educational tool to be implemented in a range of different learning contexts and used for a variety of teaching purposes. Thus this paper can also operate as guide for educators, willing to apply such approaches and techniques in their classes but are not very technically competent. The core idea is a game approach, based on an idea which is close to the students’ interests, enriched with authentic and meaningful content, within the context of the subject to be taught. Furthermore, an important issue that is implied through this paper and should be taken into serious consideration is the urgent need to change teaching methods in order to enhance the skills that future citizens will need in a digital society. According to Prensky (2005) teachers should learn to use computer games as tools that provide engaging and effective learning experiences for students. However, teachers with little experience in the use of computer games are reluctant to use them (Gros, 2007). For this reason, it is important to design guides that can explain the merits of games to teaching staff and enable them to use them in a way that is oriented far more towards the acquisition of the knowledge required by the school curriculum (Gros, 2007). Future work includes the transformation of the aforementioned game into a more interactive one, perfectly suited for additional speaking and listening activities and grammar‐focused activities through the implementation of post‐playing tasks designed around the content of the game. Moreover, as recently GBL has also been proposed for adult education, a new form of interactive content is worthy of exploration for learning purposes.

References Anderson, T.A.F., Reynolds, B.L., Yeh, X. and Huang, G. (2008) "Video Games in the English as a Foreign Language Classroom", Second IEEE International Conference on Digital Game and Intelligent Toy Enhanced Learning, pp.188‐ 192. Dalton‐Puffer, C. (2002) “Content and language integrated learning in Austrian classrooms: applied linguistics takes a look”, VIEWS, Vol. 11, No. 1, pp 4‐26. Dalton‐Puffer, C. and Smit, U. (2007) “Introduction”, In C. Dalton‐Puffer and U. Smit (eds.), Empirical Perspectives on CLIL Classroom Discourse, Franktfurt, Vienna: Peter Lang., pp 7‐23, [online], Available: http://www.univie.ac.at/Anglistik/Dalton/SEW07/Dalton‐Puffer%20&%20Smit%202007.pdf (accessed on 17/04/12). Donmus, V. (2010) “The use of social networks in educational computer‐game based foreign language learning”, Journal of Procedia – Social and Behavioral Sciences, Vol 9, pp 1497‐1503. Dourda, K., Bratitsis, T., Griva, E. & Papadopoulou, P. (2012). "Combining Game Based Learning with Content and Language Integrated Learning Approaches: A Research Proposal Utilizing QR Codes and Google Earth in a Geography‐Based

138

Kyriaki Dourda et al. Game". Proceedings of the 6th European Conference on Games Based Learning, ed. Dr Patrick F., Waterford Institute of Technology, Cork, Ireland, p.115. Eurydice Report (2006) Content and Language Integrated Learning (CLIL) at school in Europe, European Commission, [online], Available: http://www.eurydice.org/index.html (accessed on 17/04/12). Gallo‐Crail, R., and Zerwekh, R. (2002) “Language learning and the Internet: Student strategies in vocabulary acquisition”, In C. A. Spreen (Ed.), New technologies and language learning: Cases in the less commonly taught languages (Technical Report #25; pp. 55–79), Honolulu, HI: University of Hawai‘i, Second Language Teaching & Curriculum Center. Gros, B. (2007) “Digital Games in Education: The Design of Games‐Based Learning Environments”, Journal of Research on Technology in Education, Vol 40, No. 1, pp 23‐38. Kusumarasdyati (2006) “Vocabulary Strategies in Reading: Verbal Reports of Good Comprehenders”, Paper presented at The Australian Association for Research in Education Conference, Adelaide, 2006, [online], Available: http://pdffinder.net/Vocabulary‐Strategies‐in‐Reading:‐Verbal‐Reports‐of‐Good‐Comprehenders.html (accessed on 17/04/12). Lasagabaster, D. (2008) “Foreign Language Competence in Content and Language Integrated Courses”, The Open Applied Linguistics Journal, 2008, Vol 1, pp 31‐42. Meyer, B. (2009) “Designing serious games for foreign language education in a global perspective”, in A Méndez‐Vilas, J Mesa González, J Mesa González & A Solano Martín (eds.), Research, Reflections and Innovations in Integrating ICT in Education, Formatex, Vol. 2, pp. 715‐719. Oxford, R. L. (2003) “Language learning styles and strategies: An Overview”, Heinle & Heinle Thompson International, pp 1‐ 25, [online], Available: http://web.ntpu.edu.tw/~language/workshop/read2.pdf (accessed on 17/04/12). Prensky, M. (2005) “Engage or Enrage me”. What today’s learners demand, [online], Available: http://net.educause.edu/ir/library/pdf/erm0553.pdf (accessed on 17/04/12). Sørensen, B.H. and Meyer, B. (2009) “Serious Games in language learning and teaching – a theoretical perspective”, Proceedings of the 3rd European Conference on Games‐Based Learning, ed. / Maja Pivec, pp. 263‐270. Squire, K. D. (2006). From content to context: Videogames as designed experience. Educational Researcher, 35(8), 19‐29. doi: 10.3102/0013189x035008019 Troncale, N. (2002) “Content‐Based Instruction, Cooperative Learning, and CALP Instruction: Addressing the Whole Education of 7‐12 ESL Students”, [online], Available: http://journals.tc‐ library.org/index.php/tesol/article/viewFile/19/24, (accessed on 17/04/12). Tsai, Y. and Shang, H. (2010) “The impact of content‐based language instruction on EFL students' reading performance”, Asian Social Science, Vol. 6, No. 3, pp 77‐85.

139

The Design and Evaluation of a Multiplayer Serious Game for Pharmacy Students Maciej Dudzinski 2, Darrel Greenhill 2, Reem Kayyali 1, Shereen Nabhani 1, Nada Philip 2, Hope Caton 2, Sonya Ishtiaq 1 and Francis Gatsinzi 1 1 School of Pharmacy and Chemistry, Faculty of Science, Kingston University, London, UK 2 School of Computing and Information Systems, Faculty of Science, Kingston University, UK k0926455@kingston.ac.uk Abstract: Educational computer games are increasingly being used in higher education and offer the potential of greater engagement, improved results and simpler, centralised updating of teaching material. However the evidence for the usefulness of such technologies is not yet conclusive. Consequently there is a need for improved design and evaluation of educational games. The aim of this study is to identify a successful game design for a multiplayer serious game to be used in learning. The design is being developed and evaluated through the creation of a game called ‘Pharmacy Challenge’ to allow small groups of pharmacy students at Kingston University (KU) to simultaneously revise certain aspects of the pharmacy curriculum in timed quiz‐based challenges. The game is a web application with both single and multiplayer modes that can be run from a web browser on phones, tablet devices and PCs. All activities performed by players including time of access, time to answer and questions answered can be stored in data logs for future analysis. A pre‐intervention survey conducted on students’ perceptions on educational gaming informed the design of the game, which indicated that most students tend to play games on mobile devices. The game was then trialled on a group of around 60 mostly female students on a module running on years 3 and 4 of the pharmacy course over a week long period which could be played at any time of the day. Following the trial a post‐intervention survey was used to assess the students’ perception of the game. Students found the game interesting, stimulating and helpful and they identified its potential to motivate them and to facilitate their learning Positive responses indicate that games can be a valuable addition to pharmacy curriculum. The successful introduction of the game into the pharmacy curriculum demonstrates the value of education games in learning and student engagement. Keywords: serious games, mobile learning, pharmacy students, educational games, multiplayer game, web game

1. Introduction Serious games or educational games are nowadays a trending topic in the educational sector. They provide new ways for schools to motivate their pupils and drive them towards better grades. (Kapp, 2012; Cuenca‐ Lopez, et.al. 2010) The rise in the web and mobile market in the recent years allows for creation of prototypes that would let pupils stay connected while studying and share learning experience with their peers. Currently educators have broader set of tools when it comes to creating educational games. (Huang, et.al. 2013; International Communication Union 2012) This study focused on the use of web games in order to facilitate learning of pharmacy students at Kingston University (KU). It is a part of the one year MSc by Research programme and involves a collaboration of students and academics from both computing and pharmacy faculties. The intention of the authors was to create and evaluate a prototype of a game that would allow students to revise certain aspects of pharmacy curriculum. They could do that on their own, or choose to play with their friends. The goal was to make it usable on any platform with an access to the browser. The questions used in the game would be based on the British National Formulary (BNF), which is a drug formulary listing all prescribed medicines available in the UK. In order to find the answer to the multiple choice questions (MCQ) students would have to search the BNF. In that way the game would help students search the BNF more quickly and efficiently, as well as prepare them for future exams and professional practice. Open timed exams using the BNF as a source of reference is one of the key exams that pharmacy students need to pass within their degree and later to enable their registration as pharmacists. The study involved various stages and was designed according to research guidelines.

140

Maciej Dudzinski et al.

Conceptual Model What?

Outcomes Implementation ‐ Stability – bug free game ‐ Scalability to add new questions and expand the game ‐ Compatibility with many devices, platforms, browsers ‐ Sustainability – self running, no need for administration work

Question: “Can educational games improve pharmacy students’ performance and information retention?”

How? Strategy: ‐ Carry out a literature review on the subject of educational games research ‐ Prepare a game design ‐ Develop a game prototype ‐ Test the prototype on a small group of students ‐ Improve the prototype and release it for the evaluation period ‐ Gather both qualitative and quantitative data through pre- , post- questionnaires and knowledge quizzes, semistructured interviews and gameplay data logs ‐ Analyse the results

Service ‐ ‐ ‐ ‐

Functional – provides useful information and explanations Effective – helps to revise Easy to use – clear instructions, intuitive navigation and good user interface Available at any time or in any place

User ‐

‐ ‐ ‐ ‐

Performance Improvement in terms of knowledge gain, faster decision making and more correct answers Higher motivation to study Confidence boost Satisfaction from fun activity i.e. play Satisfaction from social activity

2. Literature review The literature review identified 40 articles that focused on such topics as the concept of play and games, gamification, perceptions on games, gender differences in gaming, the concept of flow, learning theories, previous related studies, game design and research guidelines. Early findings suggest that students want to see that their knowledge is relevant and can be applied in practical situations. They point out the importance of effective feedback as a drive to successful learning. (Cuenca‐ Lopez, Martin‐Caceres. 2010) There is a growing interest in educational gaming research. The current perceptions on games are mostly favourable across Europe. There are mixed views coming from the member of academia, but the trends are slowly changing and games are considered an ‘important part of intellectual development and socialisation of the individual’. To counter the negative views on gaming, it is important to emphasise the educational value of e‐learning and establish a connection between the application and its long term effects on students’ academic performance. (Kapp, 2012; Hwang, Wu, 2012; Huang, et.al. 2013) There are certain gender differences when it comes to views on games and genres preferences. Males are spending more times on games than females and they use media for entertainment purposes mostly, while females mostly for communication and schoolwork. (Liu, 2011; Huang, et.al. 2013) Moreover, males prefer action, adventure, racing and role playing games, while females prefer social, logic, quiz, and simulation and arcade games. (Liu, 2011; Appel, 2012) There are also differences regarding competiveness, anxiety levels and mental load. Females tend to report higher anxiety levels and mental load when playing a competitive game. In general players are reporting higher anxiety and mental load when the time is limited, thus it is advisable to

141

Maciej Dudzinski et al. extend decision time in order to let the players use their skills best. (Hwang, Hong, Cheng, Peng, Wu, 2012; Appel, 2012; Huang, et.al. 2013) A concept of flow is described with regard of full immersion into an act of play that brings higher concentration and full attention of the individual. Such a state might be beneficial for education games and game designers should aim to induce this effect on their players. Such a game should be easy to use, providing continuous stimulation, maintain players’ interest with varying output while being mostly fun. (Robles, Gonzales‐ Barahona, 2012; Ryu, Parsons, 2012) There are many studies involving the use of games in the educational sector. Most studies focus on motivations, perceptions and attitudes of students regarding educational games. Less focus was put on specific fields where such games can be used and practical experiments that could be evaluated to test effectiveness of such games. (Hwang, Wu, 2012) Regarding the educational games in the pharmacy area, there were few attempts with medical games identified with one pharmacy card game that could be considered a prototype to a proper computer game. (Hwang, Wu, 2012; Hwang, Sung, Hung, et.al. 2012; Akl, Pretorius, Erdley, et.al. 2010) Regarding the game design guidelines, it was noted that the game should provide measurable feedback to the player and allow room for failure in order to induce ‘trial‐and‐error’ approach. Also the feedback should be frequent and targeted to allow student to draw conclusions from the actions taken. It is possible to include optional hints for the players to assist them with difficult tasks. To avoid cluttering the screen they could be provided in textual, visual, or auditory forms. (Cuenca‐Lopez, Martin‐Caceres. 2010; Melero, Hernandez‐Leo, 2012; Wu, et.al. 2012) Current findings don’t prove, or disprove the case of games effectiveness in the classroom. (Akl, Pretorius, Erdley, et.al. 2010; Young, et.al. 2012) There is a need for better designed studies that would accurately measure changes in students’ performance and document correctly this data, using various data sources and using relevant control groups for comparisons. An inclusion of time coded data logs is necessary in order to analyse students’ behaviours in multiplayer games. (Akl, Pretorius, Erdley, et.al. 2010; Young, et.al. 2012; Chueng, Hew, 2009; Wu, et.al. 2010) Furthermore, it is important to note that games should be an optional form of study and shouldn’t replace traditional lectures. (Robles, Gonzales‐Barahona, 2012) Game Design The game is an MCQ quiz challenge that allows pharmacy students to play on their own, or with other students. Students can play the game by visiting the game’s website and logging in using an anonymous name. A player is automatically allocated to the empty slot and can start playing the game. At the start of the game the player is given 50 points that can be used to bet on the answers. The mechanics are similar to those of Blackjack as each player can double the bet, or lose it depending on their answer. Each round the player is given an MCQ question with 5 possible answers. The player has to choose the right answer and select the amount of points to bet. The player has the maximum of 3 minutes to answer the question. After the bets are placed the outcome of the round is presented and scores updated. After each round, the player is notified about the section in the BNF booklet where the right answer can be found. The player can bet any amount of points to a maximum of 500 points. More points are awarded if the player answers the question under 30 seconds and if they answer the question under 60 seconds a smaller bonus is added to the overall score. Players can communicate with each other using the chat box that is visible in the top‐left corner of Figure 1. Each player has a random avatar assigned to help them distinguish themselves from each other. All questions are stored in a MySQL database on the university server. Original questions were first written in an XML format and then inserted into a database using a PHP script. When player requests the question a JQuery request is called to the appropriate PHP script that draws random question from the server and returns it to the player encoding the data into the JSON object that can be read by the client with JavaScript.

142

Maciej Dudzinski et al. Client side logic is controlled by JavaScript files with an addition of JQuery library. They handle the communication with the server and they call PHP scripts. To maintain connection with the database a “Longpolling” technique is used by which an open connection is maintained with the server for 60 seconds and the server’s database is probed for any changes. When a change in any state is identified, new data packet is sent to the players in order to synchronise them. If there is no change in the game state during the polling period, the client is notified of the fact and the old connection is closed and new one is established again by the client. The front end of the game is handled by the HTML and CSS files, which provide consistent output among devices and browsers. Certain fixes were required in order to accommodate the game for all major platforms, and more attention was required in order to make it usable on mobile platforms. To help manipulate changes on the screen, JQuery function were used in order to inject HTML code to the website and provide dynamic experience.

Figure 1: Screen shot from the prototype

3. Methodology The study was divided into 3 sections, each section listed was ethically approved by KU ethics committee: Pre‐intervention research and planning

Completing a literature review in order to identify good practice, related work, as well as research guidelines and students perceptions. Only the most recent journals were taken into consideration and the keywords used were Serious Games, Educational Games, Pharmacy Education, Mobile Games, E‐Learning.

Needs assessment questionnaire was distributed among all years of pharmacy students at KU in order to gather their views on educational games, playing games and their learning preferences. Their answers were analysed and used to design the game.

The preliminary data helped to shape the look and functionality of the game and the prototype was created and evaluated by a small group of students using a semi‐structured interview. All their answers were recorded and analysed to further improve the game.

Evaluation period

Before the game was released to the students a BNF knowledge quiz was taken by a sample of 71 students. It was an open book MCQ quiz that was distributed in classroom. Those students would take similar quiz after playing the game over a 1 week period. The data collected would indicate of any changes in their performance.

The game was released to the pharmacy students from year 3 and 4 for a one week testing period. During that time the students could access the game at any time, whether at school, or home and could play it as

143

Maciej Dudzinski et al. much as they wanted to. Each session played was registered in the data logs to further analyse students’ behaviour. Post‐intervention perceptions

After the evaluation period the students from year 3 and 4 were given a post perception questionnaire that gathered their opinions on the game and its effect on their learning.

The initial sample of 71 students took the post BNF knowledge quiz.

All data gathered will be analysed and used in order to produce an MSc thesis.

4. Evaluation results Pre‐intervention questionnaire A quantitative pre‐intervention questionnaire was distributed to a sample of 442 Pharmacy students (year 1‐4) at Kingston University and 251 responses were gathered (response rate of 56.8%). The mean age was 23 years old and the group was formed of 36% male and 64% female students. The questionnaire consisted of 20 questions that probed students about their educational preferences, as well as game features preferences, their perceptions on educational games and their background. Questions regarding game preferences, games usage and study patterns:

Most students prefer to study individually (62.9%, n=158) and only a few (14.6%, n=39) prefer to study in Groups of 3‐4, and (13.8%, n=31) like to study in Pairs.

A majority of students prefer to keep their game score anonymous during a multiplayer session (69.7%, n=175). Those that were willing to share their score were in a minority (27.1%, n=68).

There were mixed views regarding usability of the educational games. 41.8% (n=105) of students said that they ‘Seem useful’, 19.5% (n=49), of students said that they ‘Never played any’ and 17.9% (n=45) said that they are ‘Not sure’.

Regarding the usage of mobile applications for learning ;55.8% (n=140) of students used them and 38.6% (n=97) said that they never used any.

When asked what device they play games mostly on the students answered mobile devices (42.2%, n=106) and PC (33.1%, n=83). Other devices such as Consoles (10.4%, n=26) and Tablets (5.2%, n=13) were in a minority.

The favourite genre of the students were Quiz/Puzzle games (26.7%, n=67), Sport games (25.1%, n=63), then Action/Adventure games (21.9%, n=55).

Most of the students do not play the games at all in their free time (31.1%, n=78), while 21.5% of them (n=54) said they play them few times a week and 21.5% (n=54) play them once every two weeks. Only 7.6% of those students (n=19) said that they play games every day.

When asked about the features they would like to see in an educational game; students stated that personal feedback is most important (30.9%, n=78), then a chat feature (15.7%, n=39) and social network support (13.7%, n=34).

Statements regarding the views on educational games and learning: Table 1: Responses to questions related to educational games and learning 5 point Likert scale where 1 is ‘Strongly disagree’ and 5 is ‘Strongly Agree’. Statement Games can be a constructive use of my time Games can be used to enhance my learning Games could decrease the time it takes for me to learn Games incorporated into my learning, would enhance my motivation to learn Feedback through grades/marks/scores enhances my learning I learn better when I am in control of when I am learning Social media sites and mobile device applications had a positive influence on my education

144

Mean response 3.55 3.85 3.49 3.78 4.31 4.13 3.23

Std. Deviation 1.058 .907 1.008 .902 .749 .880 1.093

Maciej Dudzinski et al. Sub analysis There was a statistical significance when comparing the results using Pearson chi square test for the following issues:

There were gender differences regarding motivation to study.(p‐value .004). Females are motivated more by grades than males, while males were more motivated by competition. Another point was that males are more likely to share their scores and performance indicators than females (p‐value .014).

There were different views on games usability amongst different age groups (p‐value .021). Over 30’s and 18‐20’s are least likely to agree that games are a constructive use of their time. What’s more, comparing the answers on educational games experience indicates that over 30’s are the least experienced group and 18‐20’s are the most experienced group (p‐value .042) with games. 18‐20’s mostly view them as boring and they are unsure about their positive effects on their education. The most e‐games friendly age group was 21‐24’s.

Pre‐intervention interview session Based on the data gathered a prototype of the game was created. A small group of students were invited to for a short playing session after which they were each interviewed and asked their opinions during a semi‐ structured interview, which was voice recorded. The main themes gathered from students about the prototype are as follows:

It is useful as it forces students to use the BNF to find the answer

It is interactive and social

It reinforces the knowledge, but it also helps to identify any gaps

It builds confidence and speed needed for professional practice

When learning is fun it results in higher attention and better learning outcomes

It simplifies and breaks down large topics

It encourages competition between students

It is a good preparation for exams

Students asked for more feedback

Minor technical glitches were noted

Knowledge quiz Prior to and after the game evaluation period, a knowledge quiz was taken by all Pharmacy students involved in the study. It was an open book MCQ test based on the BNF that was testing similar knowledge areas to those included in the game. Comparing the results from both quizzes indicated that there was no statistical difference in the scores obtained. Post‐intervention questionnaire A post‐intervention questionnaire was distributed to a sample of 250 Pharmacy students (year 3‐4) at Kingston University and 91 responses were gathered (response rate of 27.7%). The mean age was 25 years old and the group was formed of 35% male and 65% female students. The questionnaire was formed of 12 questions that asked students on their experience with the game and their opinions on educational games. General results

64.8% of students played the game (n=59)

59.3% of students said that they would use the game again to help them with future work (n=54)

145

Maciej Dudzinski et al.

60.4% of students stated that it would be beneficial to have a similar game concept used for other modules (n=55) Most students played the game 3‐5 times (23.1%, n=21) or twice (18.7%, n=17).

Perceptions: Table 2: Responses to the question related to effects of the game on students’ performance 5 point Likert scale where 1 is ‘Not at all’ and 5 is ‘A lot’

Statement How much did the game improve your pharmacy skills?

Minimum

Maximum

1.00

5.00

Mean Std. Deviation 3.0678

1.09646

Table 3: Responses to the questions related to perceptions after playing the game 5 point Likert scale where 1 is ‘Strongly disagree’ and 5 is ‘Strongly Agree’. Statement I really enjoyed playing the game The game was very stimulating The game motivated me to do well in my studies I learnt something new from playing the game The game was challenging The feedback the game provided was very helpful I found the game satisfying The goals/aims of the game were clear I would play the game again User interface was clear and well designed The game was boring /pointless

N MinimumMaximum Mean Std. Deviation 58 1.00 5.00 4.2586 .78495 57 1.00 5.00 4.0877 .93122 58 1.00 5.00 3.8103 .99924 58 1.00 5.00 4.1897 1.05060 57 1.00 5.00 4.1053 .93892 57 1.00 5.00 3.2982 1.25307 58 1.00 5.00 3.7586 .94238 57 1.00 5.00 3.7719 1.05251 58 1.00 5.00 4.2586 .82845 55 1.00 5.00 3.2545 1.30835 58 1.00 4.00 1.6034 .81520

Data Logs Each game session was recorded in the data logs and stored as a text file on the server. The data gathered includes sessions played between 9th and 23rd April 2013. Only logs of the file size greater than 400kb were taken into consideration, as they provide data on player activities over multiple rounds. During that time 222 sessions were played and 2872 questions were answered in total. The mean number of sessions taken by each player was 2 and the mean number of questions answered by each player was 26. Each log includes information on the date and time the session was played at, which questions were attempted by the student, what were their answers, how long it took them to answer those questions, what was their score and how did they bet on their answers. General results Analysing initial sample of 500 questions brought following results:

77% (n=384) of answers given by the students were correct, 19.2% (n=95) were incorrect and on 3.4% of cases the student ran out of time (n=17)

Students mostly played the game between 12am‐6am (35.9%, n=178), then between 6pm‐ 12am (28.2%, n=140). The least likely time for students to play is 6am to 12pm (16.1%, n=80).

Mostly students took between 60‐120 seconds to answer a question (28%, n=139), then 0‐15 seconds (19.6%, n=97), then 30‐60 seconds (19%, n=94).

55% (n=273) of students acquired the score within the minimal category of 0‐1000 points. 27% (n=134) of students acquired a score of 1000‐3000 points. 12.7% (n=63) of students acquired 3000‐6000 points and only 0.6% (n=3) of students acquired 9000‐12000 points.

146

Maciej Dudzinski et al.

The bets that students could place on their answers range from 0 to 500 points. 44.8% (n=273) of students placed bets in the range of 0‐50 points and 25.8% (n=128) of students placed bets in the range of 400‐500 points. The ranges of 50‐100, 100‐200 and 200‐400 points have percentages close to a 10%.

Sub analysis The following statements were identified with positive correlation (p‐value <.001)

As the time taken to answer increases students are more likely to give a wrong answer

Most incorrect answers are given when player chose to bet between 0‐50 points.

Most correct answers are given when player chose to bet between 400‐500 points.

Players who took between 120‐160 seconds to answer the question are most likely to bet between 0‐50 points (p‐value .001).

5. Discussion and conclusions The study was part of the MSc by Research programme at Kingston University. Its purpose was to design and evaluate an educational game that would help pharmacy students revise their curriculum. Before the evaluation students had mostly positive views regarding educational games. There were a lot of students who never played such games before, especially among older students. Students were enthusiastic about a new learning method and were open to try it out. The prototype session feedback indicated that the game was attractive to students and it could facilitate their learning. The evaluation period generated a much useful data that saw students using the game extensively, even after the evaluation was finished. Students were driven to get higher scores and were playing the game over multiple sessions in order to answer all questions. The post perceptions indicate that students enjoyed playing the game and found it interesting. They mostly agreed that it helped them with their studies and they would like to see similar concept used in other modules. Their responses were mostly positive and there is still room for improvement in the feedback given to them and clarity of the user interface. The results gathered from the knowledge quizzes indicate that there wasn’t any significant change in students’ performance after playing the game. This is mostly due to the fact that students were given only one week to play the game and it was voluntary. A longer evaluation would be required, but it was not possible within the time frame of the MSc programme and the term times at KU. Also a more consistent student body would be required, that could play the game over the specified amount of time. The data logs provided useful insight into students’ behaviour and answering patterns. It might be beneficial for future studies to identify players by their student ID in order to compare their game usage to their academic performance. Also it important to make sure that the data logs can be easily imported into the analysis software. If this process can be automated it would greatly speed up the time it takes to analyse the results. To conclude the study presented involved a significant cohort of pharmacy students and sought their opinions and preferences regarding use of educational games in their field of expertise. The initial response gathered indicates that students are willing to try new learning methods and they have mostly positive views on using technology to support their learning. The prototype was tested by students and their feedback indicates that further work is required in order to improve it in terms of usability and functionality, but they also acknowledge the educational value it possessed. Furthermore students’ reaction to the game was mostly positive and they used it moderately over the evaluation period. The post perceptions indicate that the students found the game interesting, managed to learn something with it and it would be beneficial to improve it and test next iterations of the game. The result gathered on academic performance didn’t show any significant difference. The evaluation duration might have an effect on the results, with students not having enough time to try out the game and benefit from it. Judging by the feedback from the students it is clear that games have a potential to motivate students and turn the competitive play into a positive force that can improve their skills, but it is challenging to effectively measure and isolate the factors that affect learning. A well designed methodological research is required to capture those effects in an academic setting.

147

Maciej Dudzinski et al. Further studies can help to identify effective usage for educational games. Multiplayer gaming is an important factor in that process. It would be advisable to create more prototypes that would record complex multiplayer interactions in various contexts. The presented study used Web 2.0 technologies for its turn based gameplay, but it would be beneficial to focus on real time multiplayer games that could engage students more and provide higher immersion. The further experiments could try to develop a prototype using next generation bi‐ directional communication technologies such as WebSockets. The use of WebSockets, or similar technologies, would allow for creation of the new type of educational games enabling students to interact more with their peers in a real time challenges. Another important point is to bring educational games to handheld devices. It would be interesting to extend the use of educational games to not only PC devices but to unite the gameplay across many platforms through for example the HTML 5 games. It would be possible for users to play together on any device and without the need to be at home. It is a challenge on its own due to compatibility and connection issues on mobiles, partial technology support and devices variety. Furthermore future studies should have longer evaluation periods and focus on measuring changes in students’ academic performance. The presented study allowed for anonymous use of the game, but it would be more effective to identify each student in order to find any correlations between the time spent on playing and the academic performance of the individual. The better integration of the study into the curriculum will result with clearer and more reliable output. New prototypes will help to find the right path to creating effective educational games for students.

References Akl Elie A., Pretorius Richard W., Erdley Scott W., and others(2010),”The effect of educational games on medical students’ learning outcomes: A systematic review”,Medical Teacher Vol.32 Appel Markus (2012), “Are heavy users of computer games and social media more computer literate?”, Computers & Education Vol.59 Cuenca‐Lopez Jose M., Martin‐Caceres Myriam J. (2010), “Virtual Games in Social Science education”, Computers & Education Vol.55 Chueng W.S., Hew K.F. (2009), “A review of research methodologies used in studies on mobile handheld devices in K‐12 and higher education settings”, Australasian Journal of Educational Technology Huang, W.‐H.D., Hood, D.W., Yoo, S.J. (2013), “Gender divide and acceptance of collaborative Web 2.0 applications for learning in higher education”, The Internet and Higher Education Vol.16 Hwang Gwo‐Jen,Wu Po‐Han (2012),“Advancements and trends in digital game‐based learning research: a review of publications in selected journals from 2001 to 2010”, British Journal of Educational Technology Vol.43 Hwang Gwo‐Jen, Sung Han‐Yu, Hung Chun‐Ming, Huang Iwen, Tsai Chin‐Chung (2012),”Development of a personalized educational computer game, based on students’ learning styles”,Educational Technology Research and Development Vol.60 Hwang Ming‐Yueh, Hong Jon‐Chao, Cheng Hao‐Yueh, Peng Yu‐Chi, Wu Nien‐Chen (2012), “Gender differences in cognitive load and competition anxiety affect 6th grade students’ attitude toward playing and intention to play at a sequential or synchronous game”, Computers & Education Vol.60 International Telecommunication Union (2012) “Measuring the Information Society ‐ Executive Summary”, available at: http://www.itu.int/ITU‐D/ict/publications/idi/material/2012/MIS2012‐ExecSum‐E.pdf Kapp Karl (2012), “Games,Gamification, And The Quest For Learner Engagement”, T+D Vol.66 Liu, E.Z.F (2011),”Avoiding internet addiction when integrating digital games into teaching”,Social Behaviour and Personality Vol.39 Melero J., Hernandez‐Leo D. (2012), “Considerations for the design of mini‐games integrating hints for puzzle solving ICT‐ related concepts”,12th IEEE International Conference on Advanced Learning Technologies Robles Gregorio, Gonzales‐Barahona Jesus M. (2012), “A synchronous on‐line competition software to improve and motivate learning”, IEEE Global Engineering Education Conference Ryu H., Parsons D. (2012),”Risky business or sharing the load? ‐ Social flow in collaborative mobile learning”, Computers & Education Vol.58 Wu W.‐H., Chiou W.‐B., Kao H.‐Y. , Alex Hu C.‐H., Huang S.‐H. (2012), “Re‐exploring game‐assisted learning research: The perspective of learning theoretical bases”, Computers & Education Vol.59 Young M.F., Slota S., Cutter A.B., Jalette G., Mullin G., Lai B., Simeoni Z., Tran M., Yukhymenko M. (2012), “Our princess is in another castle: A review of trends in serious gaming for education”, Review of Educational Research Vol.82

148

Cheating and Creativity in Pervasive Games in Learning Contexts Stine Ejsing‐Duun, Thorkild Hanghøj and Helle Skovbjerg Karoff Research Lab: IT, Learning & Design (ILD), Institute for Communication, Aalborg University, Copenhagen, Denmark sed@hum.aau.dk thorkild@hum.aau.dk karoff@hum.aau.dk Abstract: The frames that set the boundaries of play in pervasive games are ambiguous, thus players must negotiate what is part of the play when playing these games. This negotiation demands and develops creativity among players. The main contribution of this paper is to show how pervasive game designers and facilitators (e.g. game masters and/or teachers) of pervasive games can use the ambiguity and potential cheating in emergent play situations as a driver for promoting creative learning processes. More precisely, game facilitators need to respond to the on‐going negotiations of different situational frames so that players are productive in relation to the goals of the game and the learning objectives. The paper outlines what pervasive games are and presents a case involving a pervasive game on global coffee trade. Next, we develop a theoretical framework that allows us to analyse how both players and facilitators need to be creative during a game session in order to play and to facilitate the game and especially how to manage ambiguity. Finally, we discuss the results of our analysis and suggest perspectives for further studies. Keywords: creativity and learning, pervasive games, rules, framing, cheating, design and facilitation

1. Introduction In contrast to screen‐based digital games, where rules are mostly controlled by a computer, the rules of pervasive games are more reliant on players’ continual negotiation – both among players and between players and the game system. Thus, what is play and what is not play becomes ambiguous for the players (Ejsing‐Duun 2011). This paper explores this negotiation in relation to creativity and learning in pervasive games. More precisely, we explore how players create meaning by dealing with ambiguity when relating to rules as they play a pervasive game – and how facilitators must manage this complex situation within a formal learning context. The meaning‐making processes of play are two‐fold. On one hand, the players engage in social negotiation of what is part of the game and thereby what are meaningful actions between game participants (Bateson 2000). Players are creative when they extend the frame of the game – as defined by the rules – to involve what would appear to be a non‐play phenomenon and thus imbue this phenomenon with meaning within the frame (Ejsing‐Duun 2011). Players combine content, explore and sometimes even transform the conceptual space of the game and learning through play. On the other hand, game facilitators also play an important role in facilitating meaningful play. Salen and Zimmerman (2004) claim that meaningful play happens when the relationship between actions and outcome are discernible and integrated into the game’s context. Players should also be able to translate their intentions into in‐game behaviour (Sweetser and Wyeth 2005), which corresponds to pursuing a pleasurable arousal and the goal of the game that, again in a learning context, has to be aligned with the learning objectives. As we will show in the analysis, sometimes players must transform the very game system to be able to pursue specific learning and gaming goals.

2. Pervasive games To understand the concept of pervasive games, we need to revisit Huizinga’s concept of play. According to Huizinga (1950), play rules separate the play world from the ordinary world, and when players accept these rules, they enter a separate conceptual space. This space can be referred to as a magic circle (Salen and Zimmerman 2004), an area of temporary activity with its own rules that players enter of their own free will. In Huizinga’s definition of play, players must agree that certain actions at certain places at a certain time are addressed with a playful approach (1950). These actions are meaningful within the frame of the game. Montola et al. (2009) argue that pervasive games differ from traditional games, because the players in pervasive games systematically blur and break these traditional boundaries of the game, “[A] pervasive game is a game that has one or more salient features that expand the contractual magic circle of play socially, spatially, or temporally” (Montola 2005, p. 3, Montola et al. 2009, p. 12). In games that expand spatially, players never know where they will encounter game content. A game is only spatially expanded if the play

149

Stine Ejsing‐Duun, Thorkild Hanghøj and Helle Skovbjerg Karoff space is uncertain, and if the game is affected by the spatial context of the player. Pervasive games can also expand temporally by interweaving the game into everyday life. Another aspect of pervasive games is that the duration of the game is uncertain. In other words, the players never know when the game will end or even when it is on. Finally, social expansion is concerned with “playership”, i.e. who is and is not part of the game. Pervasive games expand in at least one of the three ways described and thus do not explicitly limit themselves as separate from their surroundings. Players are not necessarily aware of what is and is not part of a game, and pervasive games exploit this ambiguity (Montola 2005). Pervasive games do not need to involve technology, because expansions are possible without using technology (Montola 2005, Montola et al. 2009). Tough Road, the game analysed in this paper, however, uses technology to expand game space into everyday space. A pervasive game that mixes role playing elements, storytelling and educational content, Tough Road is partially facilitated by a game master and partially by a digital game engine. Coffee Road, the small Danish organisation that designed Tough Road uses new media and communication strategies, “to engage people in the fight against global poverty” (Coffe Road 2013). In order to achieve this aim, Tough Road has been designed as a technologically supported role‐playing game aimed at Danish upper secondary students and allows players to understand the dynamics of coffee trade from a global perspective. At the start of the game, which is designed to be played by three to five classes or roughly 75‐135 players, each player is provided with a character description that represents one of the following seven roles: farmer, trader, exporter, coffee company, café owner, banker and financial dealer. In the game, eight or more farmers have the option of creating cooperatives, which may offer them better prices when selling their “coffee bags”. Based on their roles, the players can interact with each other at their local school, which is divided into different areas corresponding to the different roles, e.g. farm villages, banks, the financial market etc. In order to simulate the complexity of global coffee trade, the game is supported by a digital game engine that keeps track of most of the transactions within the game. During the game, the game engine allows the game facilitators to trace the dynamic rise and fall of, for example, coffee prices, interest rates, company stocks and capital reserves. Several facilitators are necessary due to the complexity of the game and are typically comprised of the main game facilitator, a technical game facilitator and a handful of volunteers who are often interested in non‐governmental organisation issues involving global trade. Moreover, local teachers at the participating schools are required to introduce the topic of global trade prior to the game session and to help provide a detailed introduction to the various game roles. At the end of the game, which consists of ten rounds and lasts a whole day (six to seven hours), the winner is the student who made the largest relative gain in percentage in assets.

3. Methodological approach Ethnographically inspired observations of four game sessions conducted at three different Danish upper secondary schools were used to study Tough Road. In addition to observations, interviews were conducted with selected game participants and game facilitators. The aim of the observations was to describe relational patterns of interaction and framing during the course of the game sessions, for example by following selected items, players or facilitators on their routes through the game (Hanghøj, 2011). In this way, the observations focused less on the individual participants’ game experiences and more on the relationships and interpretive framings between the participants –and between the participants and the game system.

4. Theoretical framework This paper uses the notion of frame/framing to describe two distinct aspects of meaning‐making in relation to pervasive games used in formal learning contexts. The first meaning of frame is related to the player’s experience of rules and enactment of the game frame. The second meaning of frame focuses on how game players and facilitators negotiate the frames of a given fame in relationship to the learning situation. First, however we relate creativity to framing and rules, as creativity relates to a conceptual space defined by rules.

4.1 Game frames and creativity Creativity can happen through combinatorial, explorative and transformative processes (Boden 2009). None of these processes creates something out of nothing. They relate to forms or rules in three different ways: 1) combinatorial creativity exploits shared conceptual structures to create analogies or metaphors. Boden provides the example of how Harvey described the heart as a pump and explains that the process is guided by

150

Stine Ejsing‐Duun, Thorkild Hanghøj and Helle Skovbjerg Karoff associative rules (Boden 2009). In a game the rules become shared structures, which players can use when relating the game as a metaphor to, for instance, the everyday world; 2) explorative creativity relies on a culturally acceptable way of thinking, i.e., a recognised theory or an artistic genre. Restrained by a set of generative rules, this conceptual space is explored when being creative in an explorative fashion (Boden 2009). Playing pervasive games, players need to explore the conceptual aspect throughout the game since it is ambiguous; and 3) transformative creativity, defined by Boden, implies that this conceptual space is altered altogether Boden (2009). Transformative creativity means that the initial generative restrictions are altered. In pervasive games, players can change the frame of the game in a way that transforms play and the players’ relationship to the surrounding world. The creative processes can be directed towards generating fresh ideas or towards solving (unforeseen) problems through reflective inquiry (Dewey 1933). In this paper we are particularly interested in how players relate to the conceptual space in the game Tough Road in relation to learning about trade and trade, as well as how they solve problems during the game relating pervasive play to formal learning contexts. In this way, the meaning of creativity is related not only to the actual experience of playing the game, but also to the “validity criteria” by which particular forms of playful knowledge are seen as legitimate or illegitimate (Hanghøj 2011).

4.2 Game frames and the player experience Though the play is open, we play within boundaries that separate what is and is not in focus. We refer to these boundaries as frames that enhance the elements that should be interpreted in a special way and decrease the ones that do not have any particular significance (Bateson 2000). The boundaries of classical single‐player video games are often explicit as the computer program upholds the rules (Juul 2005). The time limits and the perimeter of the game world are also often finite in these types of video games. In other words, the rules are materialised and controlled from within the system. The boundaries of pervasive games, in contrast, are not controlled to the same degree as players do not always know which elements are part of the game. These boundaries are explored and negotiated between players, which implies that play is also managed from without the system. When we discuss the frame of a game in relation to meaningfulness, it concerns how we interpret events in relation to the game. To understand this negotiation as an aspect of play, Bateson’s idea of framing in relation to play is relevant in that he claims that if a situation is framed as play then the message being communicated is “this is play” (2000). Using meta‐communication in this way, an action suddenly does not mean what it would “normally” mean. The frame is neither physical nor logical, although it can be externalized (Bateson 2000), as is the case in games that have a clearly defined field, for instance, football. The frame directs attention to what is included in much the same way as a picture frame does. At the same time, it draws attention away from what is excluded. The frame is also a premise that tells the user about the frame that a certain interpretation or mode of thinking applies to what is within the frame (Bateson 2000). Instead of separating play from actions in daily life, pervasive players often connect these worlds when playing. By shifting between frames and also by re‐framing everyday elements through the frame of the game, players connect the pervasive game to their ordinary lives (Copier 2005). In this process pervasive players must shift focus from the everyday world to the conventions and rules of the game they have entered. Through play the players translate the game information onto the environment, making it meaningful in the context of ordinary life. Thus, the player is “interfacing” between the pervasive game and the everyday world (Nieuwdorp 2005). In sum, pervasive play is controlled and enacted both from within game systems through its rules, narrative and goals, but also from without the computer system as players perform the framing through play.

4.3 Game frames and contexts In addition to framing player experience from “within” and from “without”, the meaning of pervasive game frames is also related to the social contexts in which particular games are played, e.g. informal, semi‐formal, or formal learning contexts. In order to describe the relationship between game frames and social contexts, we draw upon Goffman’s micro‐sociological perspective on play and game frames, which further elaborates Bateson’s concept (Goffman 1961, 1974). According to Goffman, any social situation potentially involves multiple interpretive frames to be continually negotiated and re‐framed by social actors. In this way,

151

Stine Ejsing‐Duun, Thorkild Hanghøj and Helle Skovbjerg Karoff Goffman’s frame analysis is a way of asking “what’s going on” within particular situations such as game encounters where game participants negotiate meanings through the “world‐building activities” of games (1961: 25). Expanding upon Goffman’s frame analysis, Fine explores how table‐top role‐playing games basically operate through three “basic frames”: person, player, and persona (Fine 1983: 183). More specifically, “person” refers to the common‐sense life world of the game participant; “player” refers to the rules, structure and characters of a given game; and “persona” refers to the actual performances within a game world. In order to understand how role‐playing games – and pervasive games – are enacted within educational settings, however, it is necessary to add student as a fourth basic frame (Hanghøj 2011: 98‐99). By adding this fourth frame, it may further be argued that game participants and game facilitators continually shift back and forth between game goals and learning objectives when playing games in formal and semi‐formal learning contexts (Hanghøj 2013). The relationship between the different framings of actions in pervasive games in formal and semi‐formal learning contexts can be summarised as shown in the model below in figure 1: Learning objectives

Within

Without

Game goals

Figure 1: Relationship between framings of the actions in pervasive games in a learning context The two axes illustrate how actions in pervasive play in formal and semi‐formal learning contexts are framed. The vertical axis relates players’ actions to particular game goals and learning objectives. Correspondingly, the horizontal axis describes the framing in terms of “agency”. Actions are both framed by the game's inherent framework (from within) and framed from without the game system by the players. Based on the model presented above, we argue that player participation in pervasive games demands and develops creative competences since players need to negotiate conceptual space, i.e. play spaces limited by rules. Valuable learning processes are presumably supported by the continual use of creative combinations within the frames that set the conceptual space of the game as well as by the exploration and transformation of these frames, not to mention the shifting that takes place between different frames, experiences and goals in educational pervasive games. We will return to this hypothesis in the discussion.

5. Analysis Based upon data from the four Tough Road game sessions, this analysis focuses on how facilitators and players negotiate the frame of the game in relation to the goal of the game and the purpose of the learning situation. The aim of the analysis is to show how game facilitators often encounter dilemmas when trying to come up with flexible interpretations when students relatively often transgress the game rules of pervasive games. The analysis focuses on a particular game event that emerged in the fourth Tough Road game session and was referred to by the game participants as the failed cooperative. Designed to accommodate the particular needs of various upper secondary schools, each of the four game sessions involved solving numerous practical challenges locally at each school in order to run the game. Aspects that had to be taken into consideration included, for example, the physical set‐up of the different game areas, instructions for the participating facilitators and teachers and the negotiation of food and coffee prices between local school canteens and the real‐life economy of the cafés created as a part of the game. One of main problems in one particular game session was low student attendance (approximately 65 players) and meant there was an insufficient number of farmers in the game to form cooperatives within each of the different global regions of the game scenario. As the game progressed and several farmers died because they

152

Stine Ejsing‐Duun, Thorkild Hanghøj and Helle Skovbjerg Karoff had insufficient funds to pay their expenses (e.g. school fees, condoms, the acquisition of new fields, harvesting), it became clear to the farmers that their inability to form cooperatives due to a lack of players meant that they were cut off from getting better prices for their coffee, which would have been possible if they could have skipped the coffee traders and traded directly with the exporters. One of the farmers, “Maria”, was especially keen on forming an international cooperative. She eventually persuaded the main game facilitator to form an unregistered international cooperative with thirteen farmers from three different regions, while he tried to work out a technical solution with the technical game facilitator, who was not present at the school. The organisation of the inter‐regional cooperative progressed for a couple of game rounds and involved complicated workarounds, especially for Maria, who creatively transformed the game system by inventing methods of analogue transactions between farmers and banks across different regions. During this process, several of the students playing bankers decided that the formation of the cooperative involved too much risk as it would lead to direct competition between the different banks as well as their associated exporters and financial dealers. Eventually, Maria’s affiliated bank framed her actions as foul play and decided to terminate her account due to insufficient balance, which killed her in the game. A few seconds later, the farmer received a text message on her phone stating that she had died from AIDS as she was unable to pay for her medicine. Becoming quite frustrated upon learning about her untimely death, Maria asked the game facilitator to get her to get back into the game. In reply, the game facilitator insisted that since the cooperative had not been registered within the game system, it did not really exist and consequently had to be dissolved. Shortly after the incident, the game facilitator quickly wrote a text message to all the players in the game, which creatively summed up and re‐framed the event as a part of the overall game narrative, thus allowing players to associate the incident with the game and learning objectives: The cooperative that failed In spite of a foresighted farmer’s vision of [creating] a cooperative, it proved impossible to create a cooperative across national borders. It is a blow against unions across the world. The farmer died under mysterious circumstances”. In a subsequent interview conducted a few minutes after her death, the farmer reflected positively on her unexpected demise by saying that “sometimes things simply must go wrong”. Moreover, she found the game “very exciting” as it allowed her to collaborate with other farmers in order to raise the coffee price, which helped farmers who “were in a totally hopeless situation”. Later, a student who had the role of being a journalist in the game reported on the “failed cooperative” in the game blog. The event was also highlighted by the game facilitator, who prepared a speech that summed up the game session by ironically telling a nice story about “all those things that went wrong today”. In this way, the story about the failed cooperative was re‐framed into the overall narrative of this particular game session. In this way, the game facilitator managed to think and act creatively in two ways. First, he allowed a player (Maria) to explore and even transform the limits of the game. Next, he re‐framed and conceptualised the episode as a part of the game that also related to the learning objectives of the game, i.e. acknowledging and understanding how cooperatives work in a complex global economy driven by many interests with different levels of information and influence. The failed cooperative example shows how the facilitation of pervasive games in formal learning contexts is inextricably tied to dilemmas related to how players transgress the flexible rules of the game. On the one hand, Maria’s enthusiasm was clearly valuable to the game facilitator, both in order to let interesting narratives emerge from the game interaction and to create awareness on how forming cooperatives may empower farmers, which is one of the overall learning objectives of the game. On the other hand, the game facilitator had to deal with the technical constraints of the game engine, which prevented the formation of an international cooperative across different regions and banks. Moreover, his acceptance of Maria’s international cooperative as an “illegal” solution within the game generated a cascade of new problems within the game system. The system did not embrace Maria’s attempt to transform the rules of the conceptual space. This points to another learning goal of the game, which is to teach students about how the influence of certain actors on the global coffee market chain is limited. Even though one of the exporters benefited from being connected to the international cooperative, the two remaining exporters suffered financially from not being part of this collaboration. Maria was clearly aware of the imbalance created by her international cooperative. Shortly before the death of her character, she spoke to the game facilitator and when she learned that it was

153

Stine Ejsing‐Duun, Thorkild Hanghøj and Helle Skovbjerg Karoff impossible to overcome the technical hurdle of registering her cooperative, she explained how “very sorry” she felt about having created “such a mess”. At that point in the game, the transactions of the international cooperative had resulted in a severe financial imbalance, which caused a sense of growing frustration for both the farmer and her co‐players. Moreover, the game facilitator found himself in an impossible position, where he tried to skirt the technical hurdle of the game system and legitimise the illegal cooperative. Facing imminent chaos, Maria and the game facilitator were so‐to‐speak saved by the sudden death of Maria’s character, which allowed the game session to return to a more balanced progression. The transgression of rules surrounding the failed cooperative and the facilitator’s subsequent dilemmas are by no means a standalone example from the Tough Road game sessions. A week after the fourth game session, one of the participating social studies classes evaluated the game. Several students, especially those who valued fair competition, were upset that “it was far too easy to cheat in the game”, for example by transferring incorrect amounts of money or by gaining insider knowledge from other characters while smoking together outside the school premises. One student mentioned how “it gets back to you, if you cheat too much”. Maria, who had organised the international cooperative, praised the game, as did other students, for creating a “realistic” experience, which allowed her to “incorporate all those concepts and different things that I’ve learned in social studies in a different way”. The ambiguity surrounding the flexible rules illustrates an important premise about the facilitation of the Tough Road pervasive game. Namely, that no one – not even the game facilitator or the technical game facilitator – had a full overview of the unexpected events that emerged during the game sessions. One of the teachers who acted as co‐facilitator for the third game session even described the game as a monster since no one had a clear idea of where the game would end. This teacher valued the chaotic and unpredictable aspects of the game positively as it forced students into unknown territory. By contrast, another teacher emphasised how the overwhelming amount of information needed for playing the game, especially the banker roles and the financial dealers, made it too difficult for weaker students to grasp the complexity of game.

6. Discussion By using Boden’s categories, the analysis of the game episode suggests how the players dealt with ambiguity. They creatively combined their existing knowledge to be used “without” the pervasive game system (e.g. what they have been taught in class about global trade and what it meant to be a coffee farmer from Java etc.). The players imaginatively explored the conceptual game space through their actions by learning what was within the frame of the game, for instance, whether it was meaningful or acceptable for farmers to sell their coffee directly to exporters. Players even resourcefully transformed the play space by reinventing parts of the system as was the case with Maria’s failed cooperative. When players meet a complex game system such as Tough Road, which works as a conceptual space but has limits that remain open to interpretation, they need to work creatively as they interact with the rules. We argue that this demands and develops creative competencies. As the topic of the game requires that players understand the dynamics of global coffee trade, they must interact creatively with these dynamics and learn about them from within the game system. In this way, Tough Road allows players to learn about trade, cooperation and (eschewed) power. In order to fulfil the goals of the game, the facilitator must allow players to experiment with the contingent dynamics of the topic of the game and understand how these premises create a conceptual space. Consequently, players learn about the premises and conceptual space by exploring the rules and the frames of the game. The failed cooperative example shows how the game facilitator continually tried to legitimise new and relevant possibilities within the game, even though it entailed transforming the game rules. The important point here is that the transformation of the game rules was permitted as long as they adhered to the overall learning objectives. Thus, the Tough Road game sessions studied provide numerous examples of banal cheating such as when exporters tried to buy coffee directly from the farmers even though this was not allowed until the farmers had formed cooperatives. Upon discovering this, one of the game facilitators punished the exporters by ignoring their further requests for help with the game. The game facilitator also insisted on the rule that no trader or farmer should know what was actually happening on the stock market in the game – because this is also the case in the real world as well. By insisting on these rules, he helped sustain the conceptual space of the trade dynamics.

154

Stine Ejsing‐Duun, Thorkild Hanghøj and Helle Skovbjerg Karoff Seen in a broader perspective, these examples show that facilitators of pervasive games continually have to evaluate and respond to students’ creative re‐framing of game events as complex and unpredictable incidents emerge. Being able to embrace actions that allow players to creatively combine, explore and transform the conceptual space demands knowledge about the educational concepts being taught and an understanding of the game dynamics in relation to it. In addition the game rules need to actually relate to the dynamics of the conceptual space.

7. Conclusion The main contribution of the paper is to show how pervasive game designers and facilitators (game masters and/or teachers) of pervasive game may benefit from the ambiguity in play and potential cheating by using the ambiguity as a driver for creative learning processes. Pervasive games can provide players with tools to transform the frame of the game to relate it to their well‐known, everyday life world, which may promote creativity among players. However, game facilitators need to facilitate these negotiations so that they are productive in relation to both the goals of the game and the learning objectives. This leads us to the hypothesis that pervasive games are well‐suited for fostering creative learning processes, an avenue that should be the subject of further study.

References Bateson, G. (2000). "A Theory of Play and Fantasy." Steps to an Ecology of Mind, Chicago, University of Chicago Press. Boden, M. A. (2009). Computer models of creativity. AI Magazine, 30(3), 23. Coffe Road (2013) 'http://www.toughroad.dk/', [online], available: [accessed 16‐06‐13] Copier, M. (2005). "Connecting Worlds. Fantasy Role‐Playing Games, Ritual Acts and the Magic Circle." de Castell, S and Jenson, J. (eds.). Changing Views: Worlds in Play: Proceedings of the 2005 Digital Games Research Association Conference, Vancouver, University of Vancouver. Dewey, J. (1933 [1986]). How We Think. A restatement of the relation of reflective thinking to the educative process. J. A. Boydston (ed.), John Dewey. The Later Works (Vol. 8), Carbondale, Southern University Press. Ejsing‐Duun, S. (2011). Location‐Based Games: From Street to Screen. Danmarks Pædagogiske Universitetsskole, Institut for Didaktik, Center for Playware, Copenhagen, Aarhus University. Fine, G. A. (1983): Shared Fantasy. Role‐Playing Games as Social Worlds, Chicago, The University of Chicago Press. Goffman, E. (1961). “Fun in Games”. Encounters, London, The Penguin Press, pp. 15‐72. Goffman, E. (1974). Frame Analysis: An Essay on the Organization of Experience, New York, Harper & Row. Hanghøj, T. (2011). Playful Knowledge. An explorative study of educational gaming, Saarbrücken, LAMBERT Academic Publishing. Hanghøj, T. (2013). "Game‐Based Teaching: Practices, Roles, and Pedagogies." In: de Freitas, S. Ott, Popescu M. M. & Stanescu, I. (eds). New Pedagogical Approaches in Game Enhanced Learning: Curriculum Integration, Hershey, PA, IGI Global. Huizinga, J. (1950). Homo Ludens. A study of the play element in culture, New York, Roy Publishers. Juul, J. (2005). Half‐Real: Video Games between Real Rules and Fictional Worlds, MIT, The MIT Press. Montola, M. (2005). "Exploring the Edge of the Magic Circle: Defining Pervasive Games." DAC 2005 Conference, IT University of Copenhagen, DAC 2005 Conference. Montola, M., Stenros, J. and Waern, A. (2009). Pervasive Games: Theory and Design, Morgan Kaufmann Publishers Inc. Nieuwdorp, E. (2005). "The Pervasive Interface: Tracing the Magic Circle." DiGRA 2005 Conference: Changing Views – Worlds in Play, Vancouver, University of Vancouver. Salen, K. and Zimmerman, E. (2004). Rules of Play. Game Design Fundamentals, MIT, The MIT Press. Sweetser, P. and Wyeth, P. (2005). "GameFlow: a model for evaluating player enjoyment in games." Comput. Entertain. Vol 3, No 3, pp. 3‐3.

155

Supporting Teachers in the Process of Adoption of Game Based Learning Pedagogy Valérie Emin‐Martinez1 and Muriel Ney2 1 S2HEP, Institut Français de l’Education ‐ ENS Lyon, Lyon, France 2 Laboratoire d’Informatique de Grenoble, CNRS, Grenoble, France valerie.emin@ens‐lyon.fr muriel.ney@imag.fr Abstract: In an attempt to address the difficulty to integrate Game‐Based Learning (GBL) in the teaching practices, this paper proposes a model for the process of teachers’ adoption of games, based on a first research work which led to a structured question matrix designed to foster teacher reflection on key issues that arise during this process. We focus on formal education and consider not only digital (educational) games but also other game‐like activities such as role‐plays and simulations. In tackling the matter of adoption, this paper addresses a key issue: How does the adoption process unfold when teachers introduce games in their classes for the first time? To answer this question, Roger’s “Diffusion of Innovations” theory was used as the conceptual framework for analysing a case study. The case study took place in France with a group of six high school teachers who introduced three different games, in teams of two. We also provide different tools to support the adoption process: resources, activities, questionnaires, pedagogical scenario, patterns of activities and scenarios. Our efforts to support teachers’ adoption and use of GBL are not designed to offer a one‐size‐fits‐all solution. Rather, they are aimed at providing tools to foster reflection and facilitate the adoption process. It is hoped that this work will help overcome some teachers’ resistance to GBL, and this will be the subject of further verification. Keywords: game based learning in teaching practices, teacher adoption, serious games, technology enhanced learning, pedagogical scenarios

1. Introduction Although Game‐Based Learning (GBL) and digital learning games have been promoted and encouraged in recent years for formal learning, teachers still find it difficult to integrate this approach and tools in their current teaching practice (ProActive 2010). This study concerns teachers involved in integrating games into their class, not only digital games but other game‐like activities such as role‐plays, simulations, etc. This work was initiated as part of the activities of the European Team Game Enhanced Learning (GEL http://www.gel.itd.cnr.it/) funded by the European Network of Excellence STELLAR (De Freitas et al . 2012). This team consisted of six partner institutions also members of the Network of Excellence on serious games (GALA Games and Learning Alliance http://www.galanoe.eu/). Teachers involved in Game‐Based Teaching (GBT) have to choose a content adapted to the use of a game, to browse, test and select games, to design a pedagogical scenario, to facilitate the flow of the game, to ensure learning and assessment... These tasks represent a heavy burden for teachers who are new to game‐based learning and lead them to reflect on several issues (Hanghoj & Brund 2010, Kebritchi 2010): Why teaching with a game, what type of game, what skills will students develop, how to teach with a game, how to assess learning ... ? Answering these questions may help a teacher to create an educational potential around a game, because the learning does not come directly from a commercial game, which only provides the context for the learning experience. These issues are addressed in the literature from different perspectives: factors related to the intention to adopt games (Grove et al. 2012), the reasons and motivations for adopting games (Wastiau 09), different types of games (Garris 2002, De Freitas et al. 2006, Casares et al. 2010), different issues of pedagogy ‐ scenario design, assessment, etc. ‐ (Pivec 2009, Sandford et al. 2006 Wastiau 09). There are also comprehensive guides for teachers who want to introduce games, such as reports of national and european projects: Proactive (PROACTIVE 2010), Schoolnet (Felicia 2009, Wastiau 2009, Blamire 2010), and Futurelab (Ulicsak & Wright 2010). These reports provide guidelines for teachers to develop scenarios, choose an appropriate game, conduct a session including a game and example of games that have already been used and evaluated. In a previous work (Ney & al. 2012) we have realised a structured question matrix to identify a set of relevant questions for teachers to consider in preparation for adopting GBT approach. The first step in producing the question matrix was a literature search to ensure suitable scope and focus. Subsequently an empirical study

156

Valérie Emin‐Martinez and Muriel Ney was conducted with teachers to gauge the matrix’s usefulness for practitioners new to GBT and its soundness when set against the experience of those who had already used games in class. The resulting matrix (Ney & al. 2012) includes the following six sets of key issues that are further detailed in a number of questions: (A) teacher motivation and needs, (B) learners' characteristics (as a group or individually), (C) contents and game features to address teachers' needs (game motivational factors, etc), (D) practical needs (from the institution, colleagues, etc), (E) design of the pedagogical scenario and (F) assessment and capitalization issues. Therefore one can find in the literature many resources devoted to teachers. One may also study the process of integrating games itself, from initial awareness of game‐based learning and teaching to appropriation of such a pedagogy and finally to its regular use.

2. Research question and theoretical framework The prime purpose of this research is to model the process of game adoption for individual teachers. A theory widely used to analyse the adoption of an innovation in various fields is the Rogers’ theory of diffusion of innovation (Rogers 1962, 2003). An innovation is defined as an idea, practice or object perceived as new by an individual or a group. Diffusion is the process by which the innovation is communicated and adopted by the members of a social network. In our case, the innovation is the introduction in learning activities of game‐ based approaches including the use of digital games. Here we leave aside the sociological aspects of the diffusion of innovation in a group, and we focus on the relationship between the individual teacher and the innovation. Rogers’ theory has been applied to various applications in educational technology (Berger 2005, Martins 2004), including serious games. In the latter case, there is for instance the study of (Kebritchi 2010) that focused on the factors that facilitate or inhibit the adoption process. Rogers describes the “innovation‐decision” (or adoption) process in five stages, as shown in Figure 1 below.

Figure 1: A model of stages in the innovation‐decision process (Rogers 2003, p 163) During the knowledge stage the individual becomes aware of the innovation without having the goal of its adoption. Three types of knowledge are proposed in this theory: awareness‐knowledge (knowing about the existence of the innovation), how‐to‐knowledge (knowing how it works), principles‐knowledge (knowing the underlying principles). In the persuasion stage, the individual begins to focus on the possible adoption of the innovation and to actively seek information (see innovation attributes in Table 1). He/she forms an opinion and this gives an emotional dimension to this stage. The decision stage is when the individual engages in activities (analysis, debate, testing, etc.) to assess the advantages and disadvantages of the innovation; these lead to the final decision to adopt or reject it. Then, the implementation stage leads the individual (who may or may not be the one to have made the adoption decision) to introduce the innovation in daily practice. This implementation opens the way to reflection about the innovation’s positive and negative effects and evaluation of its usefulness in terms of the cost/benefit ratio. Finally, a confirmation stage takes place whereby individuals obtain information that reinforces their choice (adoption or rejection) and the sustainability of this choice. This stage involves both the individual and the group that will seek to confirm this choice. In addition to these five stages, Rogers' theory also describes this process in terms of five elements: the types of innovator (innovators, early adopters, early majority, late majority and laggards), the perceived attributes of the innovation (different ways in which innovation can be perceived ‐ see Table 1), the communication channels (how the innovation is transmitted from one individual to another), the social system (the group of individuals involved, how they are related, their roles, institutions, etc.) and temporal factors (the duration of each stage of the process and the rate of diffusion among members of a group).

157

Valérie Emin‐Martinez and Muriel Ney The “perceived attributes of an innovation are one important explanation of the rate of adoption of an innovation” (Rogers 2003, p 206), it explains “49 to 87 percent of the variance in rate of adoption” (Rogers 2003, p 206). Table 1 lists these attributes and their definition by Rogers. Table 1: Perceived attributes of innovations (Rogers 03) Relative advantage Compatibility Complexity Triability Observability

“is the degree to which an innovation is perceived as being better than the idea it supersedes”. “is the degree to which an innovation is perceived as consistent with the existing values, past experiences, and needs of potential adopters”. “is the degree to which an innovation is perceived as relatively difficult to understand and use”. “is the degree to which an innovation may be experimented with on a limited basis” “is the degree to which the results of an innovation are visible to others”.

3. Case study in France 3.1 Description, context and method The Scen@TICE team at the French Institute of Education comprises researchers and teachers of sciences and technology dedicated to the topic of innovative pedagogical scenarios using digital technologies. In 2011/2012 the team conducted field research on the use of serious games to teach sustainable development with a multidisciplinary group of teachers. This experiment involved six teachers from three schools in three different locations who worked in pairs with students from 14 to 16 years old. The subject matters were engineering, technology and biology. These teachers were compensated for their research work; they are regular users of ICT both for themselves and in the classroom, and are motivated by active pedagogies. Using Rogers’ terms, we can qualify these teachers as “early adopters” or even “innovators”. The Scen@TICE team worked in several ways: plenary (focus group) meetings were held in Lyon to compare experiences and scenarios produced in each institution and discipline; several meetings in each school helped implement a common scenario by pairs of teachers. Monitoring was done remotely: by exchanging emails, conference calls, shared documents on a web platform and scenario design with the online scenario editor ScenEdit (Emin 2010). At the end of the year, we carried out interviews using a double interview technique (Clot, 95). The interviewee was told to imagine that the interviewer was also a teacher and was intending to introduce games in class. The interviewed teacher was prompted to tell the “colleague” (played by one researcher) anything considered important for someone who is going to use games for the first time. Each of the six teachers was interviewed from 20 to 45 minutes. The answers were recorded, transcribed and analyzed. In order to also have the point of view of teachers involved in GBT, we have interviewed two teachers who are experienced users of games in class and have been using them for several years, including board games or card games and who have designed long‐lasting games. We call them “experienced teachers”. The six others were using a game in class for the first time. Therefore practices and beliefs of the teachers were variable.

3.2 Results First, we used the interviews to analyze teachers' perception of game‐based learning and teaching with the aim to identify how each of the five Rogers' attributes translates in the case of GBL. For that, we extracted verbal indicators of these attributes in teachers’ discourses (see Table 2). Furthermore, we suggest (Table 2, third column) factors that may help a teacher get a positive perception of GBL, and in turn a better likelihood of adoption. Table 2 aims to be a reflexive tool for anybody who wants to support teachers in adopting GBL. Moreover, by counting the questions that were mentioned spontaneously by almost all teachers (at least 7 out of 8), we obtain an “a posteriori” profile of our sample of teachers: the teachers share the motivation to use a GBL pedagogy to solve a specific educational problem (but different from one teacher to another) and to target specific learning goals (2 of the 11 motivations of our question matrix, Ney & al. 2012). They look at the curriculum coverage of the game and its scientific validity and use games that include competition and goals among all the motivational factors (Mariais & al. 2010). They use existing games and work with other teachers. They tackle issues about assessment. By contrast, factors that were never mentioned by any of our eight

158

Valérie Emin‐Martinez and Muriel Ney teachers were: ‐ use a game with the aim to foster creativity and imagination ‐ or with the aim to prepare students for the digital society of tomorrow, ‐ choose a game that uses chance and mystery as motivational factor ‐ or a game that proposes emotionally rich experiences, and ‐ create a new game using a model (authoring tool). Table 2: Perceived attributes of innovations and corresponding indicators in GBL Perceived Attributes of Innovations Relative advantage Compatibility

Teacher’s perception (verbal indicators)

External factors that may favour a positive perception from teachers

Motivation (teacher and learner), perceived benefit and cost compared to usual practice Audience issues, values, beliefs, needs, pedagogical approaches used in the past

Teacher motivation and needs and game features that answer to these needs. Game mechanisms and motivational factors (competition, collaboration, gain etc). Specificities of the learners. Recommendations, assistance, tutorial, work in groups of teachers

Complexity

Perceived complexity of the game rules, of the scenario

Triability

Teacher tries the game as a learner.

Observability

Perceived effect of the innovation in learner’s knowledge or behaviour

Evaluation version of the game, easy identification of domain knowledge and rules of the game Evaluation report of past experiences with this game

Then if we look at what the two experienced teachers have brought to our analysis, by contrast to the six novices, we find the following: they are the only ones to adapt existing games to their needs or to create new games and they evoke the time constraint. They work in larger teams that may include teachers from other schools (e.g. via forums), or some of their students. These teachers have moved over the years in the adoption process (Figure 1). The analysis of these interviews raised other points of discussion that could be investigated further. First, adoption of GBT may depend on the discipline. For example, the adoption of games for teachers in Biology is coming “naturally” through inquiry or observations of the natural world (real or virtual). In History and Geography, a special entry is the treatment of societal issues, also possible in Biology (e.g. on themes like ecology, energies, sexuality ...). In Technology, it is the methodological skills (e.g. problem solving) and debates on values (e.g. sustainable development) that seem to favour adoption of games. Another issue discussed in the interviews is the one of the acceptance of games: students themselves can reject it, but also colleagues, parents, and the institution. Another point is that not all teachers agree on the fact that the game will help students with learning difficulties: for some teachers this is the case, because games can highlight abilities not usually favoured at school, for other teachers these students will spend all their time playing without stepping back and learn. On the other hand, good students can also reject games because there are not anymore in a position to use their winning strategies. Teachers have also stressed the need to provide clear rules to students, but also to take the time to explain this change in their pedagogy. Finally, it seems to be necessary to test the game by putting oneself in the position of the player, which can be difficult for teachers who are not regular digital games players. In a second phase, we have used the stages of the theory of Rogers as a reading grid to analyze a posteriori the case as it unfolded over time. Table 3 lists the different tasks performed by teachers from the Scen@Tice team and links them to Rogers’ adoption process. Finally, we analyze the case both through the lens of Rogers’ attributes of innovation (Table 2) and stages (Table 3):

159

Valérie Emin‐Martinez and Muriel Ney Table 3: Table of tasks performed by teachers and stages of adoption of Rogers Tasks Design of scenarios using active pedagogies Exploratory research of serious games related to the class curriculum, list of games that can be exploited in the classroom Choice of 4 serious games to test and analyse using a co‐ designed grid Final choice of 3 games for classroom use (Ecoville, mission PlasTechnologie, Climate Challenge), feedback on the grids, individual testing of the 3 games and feedback Preparation of the scenario to implement an existing game, reflection on the implementation of the monitoring of the sequence, choice of pedagogical approach: problem based learning approach Collection of impressions about the question matrix on serious games design proposed by the researchers of the European project GEL Reflection on the success criteria of the sequence (related to learning objectives) Back to the scenario apriori, refining pedagogical phases Implementation of the game with classes, questionnaires to students Analyzing the results of experiments with students, analysis of students’ questionnaires Model the learning scenario and propose a learning sequence Interviews (double interview technique)

Period 2010‐2011 school year September

September‐ October November

November

Mode

Adoption stages Knowledge

Individual distance work with wiki Distance work in pairs Sharing experience and findings via skype meeting Focus group presential

Persuasion

Decision Decision

Implementation

November

Focus group presential

Implementation

December‐ January

Focus group presential then Distance work in pairs Distance work in pairs Focus group presential Distance work in pairs Individual interviews

Implementation

December‐ March April‐May June June ‐ July

Implementation Confirmation Confirmation Confirmation

relative advantage : this criterion could refer to the increased motivation expected of students and to the teacher’s motivation to do something new and "better" compared to previous practice (lab work, project‐ based pedagogy, inquiry‐based pedagogy). This emerged from the knowledge stage and the first focus group about the questionnaire.

compatibility: the game (and the GBL approach) must remain in line with the values of teachers and students, the curriculum and the quality of graphics they are used to; hence a pre‐selection of candidate serious games was performed during the persuasion stage (examples from the interviews: teacher rejection of a game involving killing humans or produced by an oil company, choosing a game because it questions the social pillar of sustainable development, or rejection by some students of a competitive game). This notion of values present in Rogers’ theory but absent in Kebritchi (2010) seems to be in our case a crucial attribute that may lead to adoption or rejection of a game, and maybe of GBL.

complexity: the game should be simple and intuitive for the learners, the teacher must have ready access to the game’s domain knowledge to assess its scope and depth. Complexity is also about: changing role (teacher becoming a player or a facilitator), different experience from one learner to another etc. Working in pairs, teachers were able to assess these aspects of complexity during the persuasion stage before deciding to use the game in the classroom.

trialability: teachers must be able to quickly assess the game, especially in terms of knowledge; the rules should be simple and fair (in the sense that they apply to everyone in the same way). Each teacher was able to assess a number of pre‐defined criteria in knowledge and persuasion stages.

observability: the benefits of using a game in classroom must be clear and visible in terms of student motivation and impact on learning. In the implementation stage, students’ answer to the questionnaire clearly showed the positive impact on their motivation. A student self‐assessment phase and a teacher‐led integration phase both helped to highlight student learning that was not obvious from the students' answers to the questionnaire. Observability could be assessed in the confirmation stage during the focus group, but also at the beginning of the process in the stages of knowledge and persuasion by studying

160

Valérie Emin‐Martinez and Muriel Ney various serious games portals and websites. This way one can find field results obtained with a particular game or with serious games in general.

4. Discussion In our experiment, we could show how the five stages of Rogers translate for adoption of game‐based teaching (GBT). This study is not about proving that these five stages occur during the innovation process of adopting GBT, but rather to describe what happens in each stage and how this can be used to accompany the process especially with a group of teachers. In the stage of knowledge, in our case teachers first became familiar with using active pedagogies based on technologies. It is a cognitive stage. There is a predisposition of individual teachers who feel a need either coming from daily practice (e.g. when teaching complex or domains inaccessible to observation, to motivate inattentive learners, Ney & al. 2012) or just because of GBL awareness (the innovation creates the need). In our case, it is the researchers that suggested to use a game in class. This does not mean that the teachers have decided to adopt games, but that they agreed to collaborate with researchers by the mean of GBL. In the persuasion stage, in our case teachers performed an exploratory search of serious games in their field of teaching and consistent with the curriculum. In this stage, teachers can look for information on GBL and games, anticipate mentally on how they would apply it in their class. It is an affective stage. The teacher forms a favourable or unfavourable attitude towards GBT. An attitude is usually consistent with behaviour and action and the choice to adopt the innovation will follow from a favourable attitude, but not always. However, a teacher that does not build a favourable attitude often wrecks the following implementation stage. Concerning our experienced teachers the persuasion stage took place since they are convinced of the value of games whatever the difficulties they still encounter in class. In the decision stage, in our case teachers tested thoroughly different games and filled a co‐designed analysis grid. Teachers tried to get familiar with some games by testing them on themselves or with a small group of students. The trial can be only a discussion with colleagues. Then follows the decision to use GBL and may be also a particular game. In the implementation stage, in our case teachers defined a pedagogical scenario that describes the integration of the game in a problem‐solving approach. They defined success criteria based on the learning goals and implemented this scenario in their class. Until now, the process was a mental exercise, now the innovation is put into use and this can lead to behaviour’s change in teachers. They face a certain degree of uncertainty inherent in GBL. Depending on individual trait they will live with it or try to remove it. This will in turn give more or less freedom to learners. We note that none of our teachers have mentioned chance and mystery as a motivational factor for using games or use games to favour creativity, imagination or emotionally rich experiences. These are very interesting features of GBL if one look at recent research project calls that seem to be under used by teachers today. At the implementation stage may occur re‐invention (a concept introduced by Rogers), the degree to which the innovation is changed or modify: minor or major reductions or even modifications of the game scenario or the game itself. It is common for teachers in the process of appropriation of an innovation or any kind of resources to re‐invent it. A teacher needs to experience it before it gives it to students. This customization allows adapting the innovation to the local school and changing conditions. Innovations that are more complex are more likely to be re‐invented (Rogers 2003). In the last stage of confirmation, in our case teachers analyzed their students' answers to questionnaires on motivation and learning, and they redesigned their scenario for the following year taking into account the challenges and improvements proposed in the focus group. For experienced teachers the confirmation phase had taken place before this study as they had been using games regularly for several years. One of them even said: “Since I've been using games, I've introduced humour and contextualized pupils' activities everywhere /.../ it has changed my practice.” The confirmation stage may be used to reduce a dissonant state, i.e. an uncomfortable state of mind generated by the innovation during the implementation stage. Rejection of GBT can come with replacement by another pedagogy or with disenchantment. The latter may be due to a perceived relative advantage compare to previous practice that is not adequate or to misuse of the game or scenario. Another reason of teachers rejecting games may be that they fear of loosing control and time and of the freedom GBL gives to learners.

161

Valérie Emin‐Martinez and Muriel Ney

5. Conclusion This study tries to provide a better understanding of the process of GBL teacher adoption, from the first knowledge on games for learning, to forming an attitude towards it; from the decision to adopt it and implement it in class to regular uses. We adopted a dynamic perspective and studied the process over a period of one school year with a group of six teachers. This led us to suggest a number of steps one could take to support teachers over time. This is just a first step towards modelling this adoption process. However, our study is limited since the teachers involved were all collaborating with the research team, and this could generate bias in the findings. The next step would be to follow a wider “independent” teacher population to obtain more balanced and reliable feedback.

Acknowledgements Thanks to those participating in and collaborating with the Game Enhanced Learning Theme Team (GEL) founded via Stellar Network of Excellence and a special thanks to participating teachers from France and Italy. Disclaimer ‐ This work was partially supported by the European Network of Excellence STELLAR under the Information and Communication Technologies (ICT) theme of the 7th Framework Program for R&D (FP7). This article does not represent the opinion of the European Community, and the European Community is not responsible for any use that might be made of its content.

References Berger, J. I. Perceived consequences of adopting the internet into adult literacy and basic education classrooms. Adult Basic Education, 15, 103–121, 2005. R. Blamire, “Digital Games for Learning: Conclusions and recommendations from the IMAGINE project”. European Schoolnet, novembre 2010. http://recursostic.educacion.es/blogs/europa/media/blogs/europa/informes/IMAGINE%20Conclusions%20and%20r ecommendations%202010‐3.pdf (last access july 2013) Clot, Y. « La compétence en cours d’activité », Education permanente, n° 123, 1995. De Freitas, S., Earp, J., Ott, M., Kiili, K., Ney, M., Popescu, M., Romero, M., Usart, M., Stanescu, I. Hot Issues in Game Enhanced Learning: The GEL Viewpoint. Procedia CS (PROCEDIA) 15:25‐31, 2012. De Grove, F., Bourgonjon, J., & Van Looy, J. « Digital games in the classroom? A contextual approach to teachers' adoption intention of digital games in formal education ». Computers in Human Behavior, 2012. Emin V., Pernin J.‐P. and Aguirre J.‐L. ScenEdit: An Intention‐Oriented Authoring Environnment to Design Learning Scenarios. In Wolpers M., Kirschner P., Scheffel M., Lindstaedt S., and Dimitrova V., (Eds) Sustaining TEL: From Innovation to Learning and Practice, vol. 6383 of Lecture Notes in Computer Science, pp. 626‐631. Springer Berlin / Heidelberg, 2010. P. Felicia, “Digital Games in Schools: A handbook for teachers”, European Schoolnet, mai 2009. (last access july 2013)http://www.academia.edu/193030/Digital_Games_in_Schools_A_handbook_for_teachers Garris, R. Ahlers, R. Driskell, J. E. « Games, motivation and learning: A research and practice model ». Simulation & Gaming, 33 (4), 2002, p. 441‐467. Hanghøj, T., & Brund, C. E. « Teacher Roles and Positionings in Relation to Educational Games ». ECGBL 2010 Proceedings. ed. / Bente Meyer. Reading, UK : Academic Publishing Limited, 2010, p. 115‐122. Kebritchi, M. Factors affecting teachers adoption of educational computer games: A case study. British Journal of Educational Technology, 41(2), 256‐270, 2010. Mariais, C., Michau, F. and Pernin, J‐P. « The Use of Game Principles in the Design of Learning Role‐Playing Game Scenarios », ECGBL 2010 Proceedings. ed. / Bente Meyer. Reading, UK : Academic Publishing Limited, October 2010, p. 462‐469. Martins, C. B. M. J., Steil, A. V. & Todesco, J. L. Factors influencing the adoption of the Internet as a teaching tool at foreign language schools. Computers & Education 42, 353‐374, 2004. Ney, M., Emin, V., Earp, J., Paving the Way to Game Based Learning: A Question Matrix for Teacher Reflection. Procedia CS (PROCEDIA) 15:17‐24, 2012. Pivec, P. Game‐based learning or game‐based teaching? British Educational Communications and Technology Agency (BECTA), corp creator, 2009 ProActive public deliverable “Production of creative game‐based learning scenarios: a handbook for teachers”, 2010. Rogers, E. M. Diffusion of innovations ‐ First Ed., Free Press, New York, 1962. Rogers, E.M. Diffusion of Innovations ‐ Fifth Ed., Free Press, New York, 2003. Ulicsak M. and Wright M. Games in Education : Serious Games. Futurelab, 2010. http://media.futurelab.org.uk/resources/documents/lit_reviews/Serious‐Games_Review.pdf (last access march 2013) Wastiau, P., Kearney, C., & Van den Berghe, W. How are digital games used in schools? European Schoolnet, 2009. http://games.eun.org/upload/gis‐synthesis_report_en.pdf (last access july 2013)

162

Cognitive Walkthrough for Learning Through Game Mechanics David Farrell and David Moffat Glasgow Caledonian University, Glasgow, UK david.farrell@gcu.ac.uk d.c.moffat@gcu.ac.uk Abstract: Whilst widely advocated, Games Based Learning (GBL) is still an unproven discipline. Results vary and there is no consensus for how best to teach a set of learning objectives using games. Designers may base their approach on reasonable pedagogical principles but the process of design is still driven largely by intuition and greater resembles craft than science. Humans are notoriously poor at unsupported methodical thinking and relying so much on intuition carries great risk in GBL design. Cognitive Walkthrough (CW) is a technique that improves our ability to predict how a user will understand an interaction. Whilst CW is long established in user‐interface design, it should be considered a general purpose technique for crafting experiences where a designer must predict the general thinking process of a user. Extending CW to GBL can help designers expose and question their implicit assumptions and can be used during design to lower risk or during evaluation to understand results. Extensions of CW should map to the GBL pedagogical approach chosen to provide the most cognitive support. We present an extension of Cognitive Walkthrough for Learning Through Game Mechanics and apply it to the previously evaluated e‐Bug Platform game to understand why one section achieved significant knowledge change and another did not. We found each section to assume several steps of logical understanding by users but those in the unsuccessful section were unreasonable assumptions. The new technique described in this paper explains hitherto puzzling results and identifies the strengths and weaknesses of game mechanics’ contributions to learning. Keywords: games based learning, cognitive walkthrough, game design, constructive alignment, game mechanics, Serious Games

1. Introduction Whilst there is much enthusiasm for using games in education, most claims have not been confirmed in studies (Foster and Mishra 2009). Conclusions about learning in games differ (Mitchell & Savill‐Smith 2004; Kirriemuir & McFarlane 2004) and many evaluations of Game Based Learning (GBL) are weak and unreliable (Wideman et al 2007). In order to improve GBL, it is necessary to build on sound pedagogical design principles (Freitas 2006), but game design is more of a craft than science and it is not clear how to do this. Some have attempted to integrate educational theories into game design for specific pedagogical approaches like Problem Based Learning (Kiili 2007), while others propose identifying reliable learning mechanic patterns that can then be easily integrated into games as required, without particular change to a game designer’s process (Plass et al 2011). For most GBL designers however, pedagogical principles act as guidelines rather than reliable processes and much is left to designer intuition. If a GBL game must help a player learn something new, the difference between the designer’s expertise and the players’ means that it is easy for a designer to make incorrect assumptions about what a player will and will not understand. Outside of extensive playtesting (expensive both in terms of time and cost), there are few techniques available to designers to bridge this “gulf of understanding” (Norman and Draper 1986), mitigate risk and increase the chances that the desired learning outcomes will be achieved. The lack of reliable, practical techniques to support GBL pedagogy design is paired with a similar problem when evaluating a completed design, or a finished game. It is not obvious how we can systematically analyse GBL to determine how closely it adheres to a given pedagogical approach. Assuming that pedagogy driven design was implemented in good faith across a whole game, it is further not clear how one can understand why results vary from section to section. An examination of the e‐Bug platform game serves as an example of the evaluation problem. The design team attempted to make decisions on sound pedagogical principles based on work by Koster (2005), Squire (2004), and Shaffer (2006) that suggested successful ways that games teach. However, after development, the game was evaluated for knowledge change. Of the 21 Learning Outcomes assessed, 3 achieved statistically significant knowledge change (Farrell et al 2011). The evaluation speculated on potential reasons for this but could draw no clear conclusions or suggest what changes should have been made to improve the results

163

David Farrell and David Moffat The Human Computer Interaction (HCI) technique Cognitive Walkthrough (CW) may be a suitable tool for addressing both of these concerns. This paper argues for wider adoption of CW in GBL and suggests that CW may be extended to any pedagogical approach to improve consistency both in designing GBL and in evaluating and understanding knowledge change results. As a proof of concept, we present one extension of CW for Learning Through Game Mechanics (CWLTGM) and use it to interpret the e‐Bug Platform Game results. The CWLTGM offers plausible reasons for the differences and supports the promise of the benefit of extending and applying CW to GBL.

2. The e‐Bug platform game e‐Bug was a European Commission, DG‐SANCO funded project that aimed to improve young people’s understanding of microbes, hygiene and antibiotics, with the ultimate aim of reducing antibiotic misuse. As part of e‐Bug, two games were designed that could be included in, or work independently of, curricula across 18 partner countries in Europe. One of those games was the e‐Bug Platform Game, designed for primary school children aged 9‐11. Because the game’s goal was to teach the player, it was important that it be based on good pedagogy.

2.1 Designing e‐Bug to be pedagogically sound It is not obvious how best to teach through games. Indeed, it is likely that there will never be a one‐size‐fits‐all solution, but rather, different pedagogical approaches will be adopted depending on the type of learning required. Each approach is motivated by a particular belief about the likely effect of interaction with the game. The game’s pedagogical approach was inspired by work by Koster (2005), Squire (2004), and Shaffer (2006) that explored how players construct knowledge by noticing how a game responds to their actions. These scholars highlight the role of game mechanics in constructivist learning. Game mechanics are “rule based systems / simulations that facilitate and encourage a user to explore and learn the properties of their possibility space through the use of feedback mechanisms” (Cook 2006). They are used to define the way the system behaves, and as a result, the things the player learns. However, even though Squire found that players of Civilization learned about history by interacting with game mechanics, the game wasn’t teaching the same content as school curricula and so is ultimately inappropriate as a teaching tool. Constructive Alignment (Biggs 1996) offers a way to take advantage of constructivist understandings of learning whilst being compatible with curricula. In a Constructively Aligned process, a teacher understands a learner’s existing knowledge, and the desired learning outcomes, and then attempts to deliberately align an experience to allow the learner to construct new knowledge as a result. Whilst unaware of Constructive Alignment at the time of designing e‐Bug, the designer’s intuition was based on the same concept. In this case, by crafting game mechanics to create a game experience that would allow the player the opportunity to achieve the learning objectives. For e‐Bug, the scientist domain experts and teachers identified Learning Outcomes (LOs) and the designer attempted to convert each LO into a game mechanic that would highlight its meaning in a game context. For example, as Figure 1 shows, to teach that “We use good microbes to make things like bread and yogurt”, a game mechanic was introduced that allowed the player to push a Lactobacillus bacterium into a glass of milk, which would then turn into a glass of yogurt.

Figure 1: Constructivist game mechanics supporting learning about microbes & yogurt

164

David Farrell and David Moffat

2.2 Evaluating e‐Bug To evaluate the effectiveness of e‐Bug, in‐game pre‐post tests were inserted on either side of each section of the game. Data was collected during school visits and via the e‐Bug website which allowed access to the game. The evaluation (Farrell et al 2011) used 1736 players (62 from schools and 1674 online). McNemar’s test was used to assess paired data, applied to individuals with a change in response from ‘wrong’ or ‘don’t know’ to correct, or vice versa. Of the 21 LOs targeted by the game, only three achieved statistically significant knowledge change: we use good microbes to make things like bread and yogurt (P<0.001, x2=14.46); if you cannot see a microbe it is not there (P=0.02, x2=5.60); soap can be used to wash away bad bugs (P=0.02, x2=5.28). The fundamental causes of variance between successful and unsuccessful parts of the game are not obvious. The same domain experts, teachers, and development team worked on all sections of the game and used the same constructivist, game mechanic oriented, constructively aligned process to create all sections. It is likely that some kind of “designer error” is responsible for the difference in effectiveness, but lacking GBL specific analysis tools, it is necessary to look to more established fields, such as HCI, for techniques that can be adapted to suit the domain‐specific needs of Serious Games.

3. Cognitive walkthrough for GBL Cognitive Walkthrough (CW) is an evaluation technique that helps an expert (who may also be a designer) to think like their target user through a structured process. This process encourages first thinking about the user and defining the system, and then takes a step‐by‐step approach to examining the design while the evaluator tries to imagine the user’s response. Because design is typically iterative, perhaps with several prototypes, CW can be applied during early design phases where problems in a design can be discovered prior to implementation. CW is also applicable in evaluating a final design to assess the suitability for a particular audience. Although adaptations of CW exist (e.g Antona et al 2007 for Universal Access assessment of interactive systems and Novick 1999 for operating procedures), it is still predominantly seen as a technique exclusively for use in HCI, typically for interface design. The value of CW however, is not derived from anything unique to interface design, but rather lies in exposing a designer’s hidden biases and implicit assumptions, and by helping experts think like users. The limitations on unstructured, analytical thinking in humans are well established and it seems intuitive that CW‐based procedures could be of great benefit in any area where a designer must predict how a user will think. GBL is a natural domain for such a technique since there exist so few tools to support effective GBL design and since so much is left to designer intuition. CW has been used in GBL to examine socio‐emotional learning (Dormann 2008) and to support reflection processes (Kiili 2008), but it has not yet been recognised as an approach that can help the wider GBL design community improve learning in their games, nor has it been applied in GBL evaluations to understand the meaning of successes and failures. Given the success of CW in traditional HCI and the similarity of the design process across both fields, it is likely that GBL design would benefit from the introduction CW or CW‐like processes. Of course, GBL is not one monolithic approach (Foster & Mishra 2009); different games are expected to be effective for a wide variety of reasons, so it is necessary to first understand the expected pedagogical process in a particular game, and then extend CW to best support that process. As such, we would expect there to be several GBL CWs, each designed to mirror the assumptions underpinning the pedagogical approach, and draw designer attention to the potential cognitive mistakes that she may make. As a proof of concept, the CW extension described here attempts to make explicit the implicit assumptions and intuitions that guided the e‐Bug design approach and is applicable to GBL where the player is expected to learn by interpreting her interaction with game mechanics. We then apply the extended CW to understand why two superficially similar sections of the e‐Bug Platform Game differed in effectiveness.

165

David Farrell and David Moffat

4. Cognitive walkthrough for learning through game mechanics A typical CW (Wharton & Rieman 1994) has two phases. In the Defining Input phase, the evaluator decides how to define their user. This phase also serves to document the exact process the user is expected to take to complete the task, whether or not the system has been built. The second phase is the Walkthrough of actions. The evaluator examines each required action from the perspective of the user, asking a set of predefined questions designed to help estimate the likelihood of the user successfully completing the task. When carrying out the walkthrough, the evaluator categorises each step as a success or a failure and provides detail on their reasoning. Results may be summarised in a table, perhaps with recommendations of appropriate action. In extending CW for Learning through Game Mechanics (CWLTGM), each of the standard steps is mapped to a GBL equivalent. Then, the final question in the Walkthrough is significantly expanded to become an iterative process that is intended help the evaluator predict whether or not it is reasonable to expect the player to learn. This final stage is where the majority of the benefit of CWLTGM lies.

4.1 Defining Input stage As with a traditional CW, the Defining Input stage of CWLGM is used to identify the target users and to describe the interactions required for success. The first three parts describe the user and the system. The final part describes the intended interactions that the player should learn from.

Figure 2: Mapping the defining input parts of CW to learning through game mechanics As illustrated in Figure 2, the key difference between CW and CWLTGM at this stage is that the focus is not on what the player is trying to achieve, but rather on what the designer wants the player to learn. The Walkthrough part of the CWLTGM uses the player and system description to ask whether the list of actions that defines the intended experience is likely to result in successful learning.

4.2 Walkthrough stage As in the previous stage, the first three parts of the Walkthrough stage are mainly contextualised for the GBL setting. These function very similarly to a traditional CW. They serve to ask whether or not the user will carry out the correct actions, but they don’t try to ascertain whether or not this results in learning. The key contribution and main benefit of CWLTGM lies in Step 4.

166

David Farrell and David Moffat

Figure 3: Mapping the walkthrough stage of CW to CWLTGM If the evaluator reaches this step with no ‘failures’, one can assume that the player is interacting successfully with the GBL as a game. In order to better estimate the success of learning, one must examine closely the list of actions identified in the Defining Inputs stage and decide whether it is likely that the user will understand the meaning these are intended to convey. For any GBL game, the designer must have assumed that the player would understand the meaning implied by her actions. But these assumptions may be misplaced. The purpose of the fourth step is to draw attention to these assumptions and allow the designer a chance to spot potential mistakes or areas where it is unlikely that the player will understand the intended meaning of an interaction. 4.2.1 Step 4 of the walkthrough stage Step 4 is broken down into a series of sub steps.

Figure 4: Step 4 of CWLTGM broken down into sub‐steps Step 4‐1: List every logical connection that must be made by the player in order to learn through playing this part of the game. The purpose of this step is to transform the ‘list of actions’ created during the Defining Input stage into a set of logical inferences required for the player to learn from these actions. For each action, the evaluator should ask questions like:

Which game entities must the player recognise as having subject domain meaning?

Which game entity interactions must the player recognise as having subject domain meaning?

Generally, the evaluator must try not to miss any ‘leaps’ of logic, but rather detail each small logical link that the player must make in order to complete the chain of inferences expected to lead to learning. Step 4‐2: For each of the logical links identified, consider again whether it disguises two or more actual steps of logic, and if so split into smaller steps.

167

David Farrell and David Moffat Naturally, the evaluator is prone to the same kinds of mistakes of assumption and expectation as a designer. However, by pausing and re‐examining each step of logical inference, we provide an opportunity for an evaluator to spot mistakes. Step 4‐3: Consider each of the logical links and ask whether it is reasonable to expect the player to make that connection. With close reference to the player description created earlier, the evaluator should ask questions like:

Will the player understand the visual metaphors?

Will the player read and understand required text?

Will the player’s attention be drawn to the correct elements?

Will the player understand that the meaning of game interactions has a subject domain application?

For each of the logical links, the evaluator should write a few sentences about the likelihood of the player making or missing this connection. Links can be categorised as low risk if it is likely the player will make the connection; moderate risk if the player should make the connection, but there is enough doubt to warrant further attention; high risk if it is unlikely that the player will make the connection.

4.3 Summarising findings As with a traditional CW, the evaluator summarises findings in a fashion relevant for the next stage of a project. If used during the design process, results of CWLTGM applied to a design can motivate actions prior to expensive development. High risk areas should be redesigned as they are fundamentally flawed ‐ the redesign should aim to support the player’s understanding of the broken chain of logic. Moderate risk areas are assumed to be successful, but by identifying uncertainty at this stage, focus groups and preliminary assessments can be used to confirm expectations. In a final evaluation phase, it may be too late to alter a game, but by examining successful and unsuccessful parts, we gain greater insight into our practice and share results with the community.

5. Applying CWLTGM to the e‐Bug platform game We applied the CWLTGM in an evaluation role to see if it could explain confusing results. Two sections of the game were chosen that appear to be very similar in their implementation, pedagogical approach, and complexity and yet differ in their effectiveness of improving knowledge in players. The CWLTGM resulted in a lengthy report, the key findings of which have been discussed in this section. For brevity many of the details have been summarised. It is worth noting however, that much of the value of CWLTGM lies in attention to detail absent in the summary. The first section chosen was aimed at teaching “soap can be used to wash bad bugs from hands.” This section was successful in changing knowledge (P=0.02, x2=5.28). In this ‘Soap Section’ of the game, the player can press a button to throw some soap. If the soap hits one a ‘bad bug’ it is washed away with a suitable animation.

Figure 5: The player throws soap at bad bugs, which are washed away

168

David Farrell and David Moffat The second chosen section was aimed at teaching two LOs: “Our bodies have natural defences that protect us from infection.”; “Most coughs and colds get better without medicine.” This “Natural Defences” section of the game was not successful in achieving knowledge change (P=0.21, x2=1.6 and P=0.74, x2=0.11 respectively). The difference in performance of these sections was surprising since they both employ very similar game mechanics. In this section, the player can press a button to throw a white blood cell. If they hit a virus, it is killed with a suitable animation. At design time, it appeared as though these two sections were almost identical, with only the “bullet” differing between them.

Figure 6: The player throws white blood cells at a virus which is killed The player was expected to understand the meaning of their actions, not just in terms of ‘shooting’ an enemy, but also in subject‐domain terms of ‘washing away’ or ‘killing’ microbes. For both sections, the first three steps of the CWLTGM were considered ‘successful’. Players could reasonably be expected to perform the actions (throwing items at bugs) required to finish the level. The analyses diverged in Step 4.

5.1 Walkthrough of learning: Soap section For the successful Soap Section, the game actions were converted into the following inferences:

The player must realise that the environment in which they play represents a human hand.

The player must understand that she isn’t “firing bullets” but rather throwing soap.

The player must realise that the game’s placement of bad bugs on skin represents the reality that we have harmful bugs on our skin.

The player must realise that the bad bugs are being washed away by the soap.

The player must realise that the bad bugs are bugs, and not aliens or other enemies.

The player must realise that this doesn’t just happen in the game; it represents reality in that soap is used in real life to remove microbes from the hand.

The player must understand holistically that washing her hands will remove harmful bugs.

Each of these steps of required logical inference was examined for ‘risk’ of the player not understanding the semantic meaning behind their actions. When condensing the findings for this paper, it became apparent that most of the logical links above depend on the results of one question. “Would the combination of the introductory text and the art chosen for the game element referred to in the logical step be sufficiently communicative to reasonably expect the player to make the logical inference?” The majority of the inferences in this section were considered low risk and only three were considered moderate risk. Link 5 requires that the player read and understand some text that may be misunderstood.

169

David Farrell and David Moffat Links 6 and 7 identify a possible “gulf of meaning” where the player may restrict her interpretation of the game’s meaning to its fiction.

5.2 Walkthrough of learning: Natural defences section At the end of Step 4‐2, the following logical inferences were identified:

Player must realise that the environment they are playing in represents "inside a human body".

Player must realise that the bad bugs are microbes (viruses) and not aliens etc.

Player must realise that the game's placement of the virus inside the body represents the real‐world concept of a human being infected with a virus.

Player must understand that she isn't just shooting bullets, or throwing soap, but rather is throwing white blood cells.

Player must realise that the bad bugs when hit by the WBC are being killed.

Player must realise that "white blood cells" are the body's natural defences.

Player must realise that this doesn't just happen in the game; it represents reality in the sense that white blood cells kill infections.

Player must realise that most “coughs and colds” are viruses.

Player must understand holistically that this means that their body can kill most infections like colds and flu’s by itself.

Only links 1, 2, and 5 were considered low risk. Links 7 and 9 were considered moderate risk due to the potential gulf of meaning mentioned previously. The other links were categorised as high risk. Whilst the specifics of each failure are important for this particular design, they generally failed for four reasons:

the player was considered unlikely to recognise the artwork as representing the desired domain concept from sight alone because the concept is new to the player (e.g. white blood cells);

the introductory text was unlikely to be understood (25 sentences over 5 screens, each shown for 5 seconds);

the language used in the game is not consistent with the test question asked (e.g. the game referred viruses, yet the quiz refers to “coughs and colds”);

there were too many new concepts being introduced to reasonably expect the player to understand the semantic mappings.

6. Discussion and further work The sections of the game analysed were designed by the same team of scientists, teachers and game developers, with the same process and with the same pedagogical underpinnings and were expected to be equally effective. It was a surprise to see such a degree of variance. The results of the walkthrough were genuinely surprising. It seems ridiculous in hindsight to have expected the Natural Defences section to succeed, but we did and could not understand, to our satisfaction, the reason for the difference. By definition, you cannot design a game to teach without some idea for how it is going to do so, but it is very difficult for people to find and critique their biases, implicit assumptions and logical fallacies. CW is a very simple tool but it offers a way to support one’s critical thinking and help the expert designer to identify these mistakes. We applied the CWLTGM in an evaluatory role against two sections of one game. Future work should assess its usefulness in understanding the results of other games that also teach through game mechanics. Similarly, whilst it is expected that applying a CW during the design phase should reduce risk, this should be investigated further. CW is a general approach and the CWLTGM presented here is only really applicable to games that use a constructively aligned, game mechanic based approach to teaching through play. Other extensions should be proposed and evaluated for supporting other types of learning in games.

170

David Farrell and David Moffat

7. Conclusion We argue that Cognitive Walkthrough (CW), which is currently widely regarded as an interface usability tool, should be adopted as a cognitive support tool for Games Based Learning (GBL) and other domains that rely on predicting how a user will think. Using CW during the design process can expose hidden flaws in reasoning of the design team. Using CW in an evaluation of a game can provide plausible reasons for knowledge change results that might help understand why an element of a design was or was not successful. Because different pedagogical approaches in GBL are based on different expectations of effects, CW is best considered a general approach that should be extended to fit a pedagogy’s underlying theoretical basis. We presented a proof of concept of this idea with an extension of Cognitive Walkthrough for Learning Through Game Mechanics (CWLTGM). We applied our CWLTGM to the e‐Bug game that has been evaluated for knowledge change to understand why two sections that appeared to be very similar, had differing results. The results suggested several plausible and unexpected reasons why one section failed to achieve significant knowledge change, whilst the other succeeded. Future work is necessary to establish the general validity of the technique and its usefulness during design.

References Antona, M., Mourouzis, A. and Stephanidis, C. (2007). Towards a Walkthrough Method for Universal Access Evaluation. Proceedings of the 4th international conference on Universal access in human computer interaction: coping with diversity, pp. 325–334. Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education, 32, pp 1‐18, [online], Available http://www.springerlink.com/index/l2q3820h2436l607.pdf [17 July 2013] Cook, D. (2006). What are game mechanics?, [online], Available http://www.lostgarden.com/2006/10/what‐are‐game‐ mechanics.html [17 June 2013]. Dormann, C. and Biddle, R. (2008). Understanding Game Design for Affective Learning. Fututre Play 2008, pp 41–48. Farrell, D., Kostkova, P., Weinberg, J., Lazareck, L., Weerasinghe, D., Lecky, D. M., and McNulty, C. a M. (2011). Computer games to teach hygiene: an evaluation of the e‐Bug junior game. The Journal of antimicrobial chemotherapy, 66 Suppl 5, v39–44. doi:10.1093/jac/dkr122 Foster, A. N., and Mishra, P. (2009). Games, Claims, Genres, and Learning. Handbook of Research on effective electronic gaming in education, pp 33–50, [online], http://punya.educ.msu.edu/publications/foster‐mishra‐08.pdf [17 July 2013]. Freitas, S. D. (2006). Learning in Immersive worlds A review of game‐based learning. Bristol, UK: Joint Information Systems Committee (JISC) Report. [online], Available http://www.jisc.ac.uk/media/documents/programmes/elearninginnovation/gamingreport_v3.pdf [17 June 2013] Kiili, K. (2007) “Foundation for Problem‐Based Gaming”, British Journal of Educational Technology – Special issue on Game‐ Based Learning, Vol 38, No. 3, pp 394‐404. Kiili, K. (2008). Reflection walkthrough method: designing knowledge construction in learning games, Proceedings of 2nd European Conference on Games Based Learning, Catalunya, Spain, pp. 237–241. Kirriemuir, J. (Ceangal), and McFarlane, A. (University of B. (2004), Literature Review in Games and Learning, Futurelab Report, Bristol. Koster, R. (2005). A Theory of Fun for Games Design. Paraglyph Press. Mitchell, A., and Savill‐Smith, C. (2004). The use of computer and video games for learning: A review of the literature. Development. Learning and Skills Development Agency, [online], Available http://dera.ioe.ac.uk/5270/1/041529.pdf [17 June 2013] Novick, D. Using the cognitive walkthrough for operating procedures. Interactions, 6(3):31‐37, 1999 Plass, J., Homer, B., Kinzer, C., Frye, J., and Perlin, K. (2011). Learning Mechanics and Assessment Mechanics for Games for Learning. Games for Learning Institute White Paper, [online], Available http://g4li.org/wp‐ content/uploads/2011/11/G4LI‐White‐Paper‐01‐2011‐Learning‐Assessment‐Mechanics.pdf [17 June 2013] Shaffer, D. W. (2006). How computer games help children learn, Media, Vol. 1, pp. 256, Palgrave Macmillan. doi:10.1057/9780230601994 Squire, K. (2004). Replaying history: Learning world history through playing Civilization III. Indiana University, Indianapolis, [online], Available http://website.education.wisc.edu/kdsquire/dissertation.html [17 June 2013] Wharton, C., Rieman, J., Lewis, C. and Polson, P. (1994), The Cognitive Walkthrough: A practitioner’s guide, In J. Nielsen & R. L. Mack (Eds.), Usability inspections methods, New York: Wiley, pp 105‐140. Wideman, H. H., Owston, R. D., Brown, C., Kushniruk, A., Ho, F., and Pitts, K. C. (2007). Unpacking the potential of educational gaming: A new tool for gaming research. Simulation & Gaming, 38(1), pp 10–30. doi:10.1177/1046878106297650 Norman, D. A., and Draper, S. W. (Eds.) (1986). User centered system design: New perspectives on human‐computer interaction. Hillsdale, NJ: Lawrence Erlbaum Associates

171

Global Math: Development of Online Platform for Mathematical Thinking Games Toru Fujimoto1, Keiichi Nishimura2, Kaoru Takahashi3, Masahiro Yachi4, Kiyoshi Takahashi4 and Yuhei Yamauchi3 1 Center for Research and Development of Higher Education, The University of Tokyo, Tokyo, Japan 2 Tokyo Gakugei University, Tokyo, Japan 3 Interfaculty Initiative in Information Studies, The University of Tokyo, Tokyo, Japan 4 Benesse Corporation, Tokyo, Japan tfujimt@he.u‐tokyo.ac.jp knishi@u‐gakugei.ac.jp kaorutkh@iii.u‐tokyo.ac.jp yachi@mail.benesse.co.jp kiyoshi@mail.benesse.co.jp yamauchi@iii.u‐tokyo.ac.jp Abstract: While some gaming portals provide learning‐based games, most of them either merely showcase games without offering any function for user feedback for the developers or do not provide open access to individual developers, even if a website has functions for data collection. Therefore, it is difficult for individual developers and small independent teams to obtain user feedback for making enhancements in their games in the prototyping phase. The purpose of this research is to develop and evaluate an open online platform system to host mathematical thinking games. Through a joint research project in collaboration with the University of Tokyo and Benesse Corporation, we have developed the ‘Global Math’ platform, which is an open online platform to host mathematical thinking games for Indie game developers and students interested in developing learning games. The platform features the ‘Global Math API’, which enables game developers to obtain play log data by simply registering and embedding certain JavaScript codes. The API offers an interface that stores play log data in the Global Math platform database. The platform offers data‐analytic functions to monitor how the games are played and received by audiences. As a formative assessment of the platform in terms of usability and effectiveness, four teams of undergraduate students who study game design participated in a game design project using the platform. The teams worked on the project for two months and uploaded four game prototypes successfully. The survey findings indicate that the students found that this project offered them an opportunity to think about different aspects of game design that they had not considered previously, and they found it appealing to develop mathematical learning games. It showed that developing mathematical games can be engaging for students as long as they are provided with the necessary resources. The survey also indicates that more instructional and technical support for developers is necessary to use the functions of the platform. Keywords: mathematical games, game‐based learning, game platform, embedded assessment, social media

1. Introduction Foreign attendees at international conferences have often asked us questions such as ‘Why do Japanese game developers and researchers not get involved in the serious games field?’ In contrast to other countries where the serious gaming movement has recently gained recognition from people outside the gaming industry, the emerging field of serious gaming has not grown significantly in Japan. In contrast to the success of the Japanese game industry, companies working in serious games are struggling to sustain themselves. The Ministry of Economy, Trade and Industry of Japan released a report entitled ‘Game Industry Strategy: A Vision for the Development and the Future of the Game Industry’ in 2006 (METI 2006). It proposed a vision for the future of Japan's game industry that would help its survival in the global competition and receive the widespread support of Japanese society. The use of games for purposes beyond entertainment, e.g., to address social issues such as education, social welfare and healthcare, has been considered as a strategy to realize this vision. In spite of some joint efforts conducted by the industry, academia and government to promote this business outside the entertainment market since the release of the report by linking the game industry and other fields beyond entertainment, such efforts have been sporadic and not as successful as expected. The grants funded

172

Toru Fujimoto et al. are not adequate to attract skilled game developers. The typical project budget for serious games in Japan tends to be less than $100,000 per title, which is far below the average development cost for commercial game titles. Thus, the size of the project is usually small, and such projects find it difficult to attract and sustain public attention. Moreover, the academic research base for serious games has not yet been established, and there are remarkably few researchers who concentrate mainly on this field. Therefore, it is difficult for young researchers to pursue an academic career in game studies. Due to the lack of a support system, even if independent game developers aspire to develop games in order to address social issues, they have limited opportunities to acquire the necessary support for the completion of their project. Moreover, if they release a product, no effective means exist by which they can attract audiences for most low‐budget games. To address these issues, we considered a platform system to support game developers that would make the development process more effective and would reach their audiences. While some learning games portal websites exist, most of them either merely showcase games without offering any functions to obtain user feedback for the developers or are not openly accessible by individual developers even if a website has functions for data collection. Therefore, it is difficult for individual developers and small independent teams to gain user feedback so as to improve their games in their prototyping phase. Our research intended to solve this problem by developing an open online platform system to host games for learning. Through a joint research project in collaboration with the University of Tokyo and Benesse Corporation, we developed the Global Math platform, which is an open online platform for Indie game developers and students interested in developing learning games to host mathematical thinking games. This paper reports the development and formative evaluation of the platform.

2. Design of the platform 2.1 Issues in mathematical education We designed the game platform system focusing on mathematical thinking skills. We were concerned that the low achievement level of students in mathematics would be a significant issue in mathematics education (Ministry of Education, Culture, Sports, Science and Technology 2009). Students with high mathematics anxiety tend to feel demotivated while learning mathematics. Learning activities in mathematics classes, such as taking examinations, increase overall anxiety (Richardson & Suinn 1972), which may contribute to mathematics avoidance and poor performance (Rounds & Hendel 1980). A combination of high anxiety and low interest in mathematics generally forms a tough barrier that prevents students from adequately learning mathematics. Researchers consider that digital games can be a powerful and alternative tool to build learning environments for mathematical‐problem solving and engage learners in mathematical thinking activities (Van Eck & Dempsey 2002, Devlin 2011, Ke & Grabowski 2007, Kebritchi & Hynes 2010). Yoshizawa pointed out that students tend to stumble and dislike mathematics because of the lack of clarity in the actual functioning of abstract mathematical concepts (Yoshizawa 2006). Because it is difficult for novice students to acquire an understanding of mathematical concepts without being able to connect concrete examples with the concepts, it may be necessary to provide them with learning opportunities that link actual examples with mathematical knowledge. By using digital games, learners can participate in playful activities that help them engage in challenging tasks supported by immediate and active feedback from the game system.

2.2 Issues in development of games for mathematics While many games have been developed and released for mathematical education, it has still not become a key approach for learning. As previously mentioned, it is virtually impossible for a small‐budget project to develop a game of excellent quality, evaluate its effectiveness and promote it so as to attract public recognition within the short project term. For this, a crucial issue was determined to be the lack of a common platform which offers the opportunity to reach a wider audience and obtain feedback based on a measurement standard for game‐based mathematical education. It is difficult for entertainment game developers to align appropriate learning goals with fun elements in the game because they are usually not experts in pedagogy. Therefore, we prepared not just development guidelines for technical matters but a design guideline on learning contents. By offering such a platform, we expected that games developed by small project teams would have a greater chance of providing learners with better learning experiences.

173

Toru Fujimoto et al.

2.3 Design features of the platform In this study, we considered the target users of the game platform to be undergraduate students and independent game developers who are interested in developing learning games. The platform system consisted of several elements: 1) a web‐content management system that accepts registration and configuration by game developers and showcases the game applications, 2) a play log database that stores the actions by learners during their game play, 3) the Global Math API, which enables the developers to obtain play log data by registering and embedding some JavaScript codes in the game and 4) a monitoring system to provide game developers with analytics information based on play log data (see Figure 1 for the system overview).

Figure 1: Overview of the Global Math platform system The platform currently supports Flash‐ and HTML5‐based gaming applications. Once registered, developers can register their own games on the administration page free of charge. Their applications need to include the Global Math API to collect play log data. The API offers an interface that stores play log data in the database. The platform offers data analytics functions to monitor how the games are played and received by audiences. The API consists of a set of codes for exchanging game data and user profile data between the platform and game application. The play log data is communicated in the JavaScript Object Notation (JSON format. When a person starts playing, the game application sends a request to the platform database to call a unique session ID for identifying the start and end of a gaming session. During a single session of gaming, the game tracks users’ actions on the basis of the method defined by the developer. Instead of providing fixed methods, the API offers flexibility to be able to define and register one’s original property on the basis of their evaluation needs and interests. For example, if a developer wants to count how many times players use a certain item, he/she can define it by using the request method. If the item is named Apple, the syntax for the same is ‘A3.request ({"item": "Apple", "location": {"stage": "2‐3", "position": 120}});’. The data collected during gaming is stored in the platform’s database by these data transaction methods. Game developers can refer to these data on the administration page. Figure 2 shows the overview of data collection structures and the items to be collected during the game play. We applied the framework of evidence‐centred assessment design (Mislevy et al. 1999) to coordinate target skills on mathematical thinking and actions during the game play and developed a user monitoring system to enable ‘stealth assessment’ (Shute 2011). We developed a tentative competency model which focused on mathematical thinking skills for 1) finding numerical and logical rules, 2) gaining insights into the complex process and 3) classifying things based on the player’s logical insights. The model was resolved into component elements based on skills, and it explained what types of tasks and actions in the game are relevant to these skills. For example, ‘finding numerical and logical rules’ may be relevant to the following tasks in the game: 1‐

174

Toru Fujimoto et al. 1) within given items, identifying uniqueness and difference to obtain clues to solve problems, 1‐2) dividing a problem into smaller units to make it solvable, 1‐3) determining the relationship between items and the pattern of changes on the basis of the given condition, 1‐4) ensuring that the relationships and rules are enacted and where these are in effect. These concepts were explained in the design guidelines offered to the game developers.

Figure 2: Overview of Play Log Data Collection The prototype of the platform and the design guideline documents were developed and evaluated. The remainder of this study examines the initial results of the formative evaluation and discusses how the platform supported game developers.

3. Evaluation 3.1 Methods For a formative assessment of the platform in terms of usability and effectiveness, we conducted a user test. Four teams consisting of undergraduate students (n = 13; 11 males and 2 females) who study game design at the Tokyo University of Technology participated in the game development project using the platform. They volunteered to join the project as an extracurricular opportunity. In the beginning, we provided them with a design guideline document to inform them about the implementation of the Global Math API into their game applications. We also provided them a document with instructions on how to align learning goals and game play. Starting in January 2013, the student teams started to work on the project, during which a proposal meeting, mid‐term presentation and final presentation were held consecutively. After the end of the two‐month project term, the teams uploaded four game prototypes on the platform in March 2013. Following the final project presentation, we conducted a survey with the students to assess their experiences during the development process. The survey questionnaire consisted of ten multiple‐choice questions and seven free‐answer questions on their experience of the project, including the usefulness of design guideline documents, their interest in the mathematical games and the user‐interface of the platform.

175

Toru Fujimoto et al.

3.2 Game examples The students successfully developed four mathematical thinking games on the basis of the provided development guidelines. Brief descriptions of the games are as below (see Figures 3 & 4 for the example screenshots of the games developed by the students).

Logi‐mon: It is a strategic action puzzle game which challenges players by using a logical puzzle and a limited number of available hands to defeat enemies. The players have to put the hands in the right order to go through battles within a certain time period. The game is named after the popular entertainment game and anime television series ‘Poke‐mon’.

The Kansu‐Makyu (The Functional Magic Pitch): It uses a baseball pitch as a metaphor to describe the curve of mathematical functions. Learners can learn the concept of mathematical functions by aiming baseballs at targets, which are done by adjusting coefficients in a function. To hit all targets on the field, the player has to estimate the correct coordinates and angles before throwing the ball. With the increase in the level, it includes quadratic and trigonometric functions.

Koi Yori Shouko (Evidence Superior than Love): This is a propositional logic quiz game targeted especially at girls. Players have to guess true/false answers in a young male student character’s statements by referring to logical clues.

Saki Yomi Tantei (A Look‐Ahead Detective): This is a minesweeper‐type of logical puzzle game in which players exactly locate an enemy’s hiding position on the map through logical clues that they receive.

Figure 3: Screenshot of Logi‐mon

3.3 Survey results The survey results (Figure 5) indicated that most students found the project engaging and wished to participate in such a project again, even though they thought this project was harder than other previously participated game development projects. The project seemed to offer them an opportunity to think about different aspects of game design that were not considered previously, and they found it appealing to develop mathematical learning games. The open platform environment enabled them to check other projects online,

176

Toru Fujimoto et al. which participants found encouraging. However, the results also indicated that the instructional and technical design guideline documents were not as helpful as anticipated. The document needs to be more detailed. Because this was a short‐term project, we found that it was also beneficial to provide students with instant support from developers so that they could update their games faster without consuming time for unnecessary confusion.

Figure 4: Screenshot of The Kansu‐Makyu The following are the comments of students on how this project offered them a positive experience: ‘It was truly a delightful experience for me as we learned a lot about programming and beyond. This experience would be helpful for our game production in the future’. ‘Through this project, I understood the difference between entertainment game development and educational ones. We could use of our knowledge such as user‐centred design and specification planning, which I learned through entertainment game design’. ‘I am so satisfied with the project as this was totally a unique experience for me. We appreciate that you made this kind of platform which enables us to release our games worldwide. This is extremely difficult for students like us. I got to understand mathematical knowledge more deeply as well.’

When I first heard about this project, it sounded interesting This project was harder than my previous projects

Somewhat disagree

This project made me think about the dfferent aspects of games

Neither

I want to participate in this type of game development project again

Somewhat agree

This project was a good opportunity to consider mathematical thinking skills

Strongly agree

It was encouraging to see how other teams progressed The technical information in the design guideline document was helpful.

(n = 13)

The instructional design guideline document was helpful. 0%

20%

40%

60%

80%

100%

Figure 5: Summary of the participant survey

177

Toru Fujimoto et al.

4. Conclusion Although this study did not provide conclusive results, the formative evaluation indicates that the game platform could work as a hub to connect game developers interested in develop mathematical games and engage them in a development project. We did not expect that the students would enjoy such a platform. They started studying game design because they wanted to develop entertainment games. Developing educational games, especially mathematical games, tends not to be such a popular area of focus for students. However, this project showed that developing mathematical games can be engaging for them as long as they are provided with the necessary resources. The study also indicated that the open platform can be a stage on which developers can compete and demonstrate their skills to a broader audience. In general, few opportunities exist to reach a broad audience to gain feedback for user testing. It is also difficult to compare different game prototypes because they are unlikely to share common measurement standards for objective evaluation. The Global Math platform offers measurement tools to enable such a comparison. There are some overall limitations in this study. First, because the data was collected from a limited number of participants in a formative evaluation, the results did not provide concrete evidence that showed the effectiveness of this platform. Second, the provided design guideline documents were not developed in detail. The Global Math API was not fully developed to analyse the details of play log data. It requires further developments to demonstrate its potential. Finally, substantial measurement of and further research on the platform is required on a large scale so that it is fully effective.

Acknowledgements This study was conducted as a research project of the Benesse Department for Educational Advanced Technology (BEAT) at the University of Tokyo. We especially thank Professors Yoshihiro Kishimoto and Koji Mikami at the Tokyo University of Technology for their support on the student project.

References Devlin, K. (2011) Mathematics Education for a New Era: Video Games as a Medium for Learning. AK Peters Ltd, Natick, MA. Ke, F. and Grabowski, B. (2007) Game Playing for Math Learning: Cooperative or Not? British Journal of Educational Technology, Vol 38, No 2, pp 249–259. Kebritchi, M. and Hynes, M. (2010) Games for Mathematics Education. In A. Hirumi (Ed.) Playing Games in School: Video games and Simulations for Primary and Secondary Education (pp 119–145). Washington DC: International Society for Technology in Education. Ministry of Economy, Trade and Industry. (2006) Game Industry Strategy: A Vision for the Development and the Future of the Game Industry. Ministry of Education, Culture, Sports, Science and Technology. (2009) White Paper on Education, Culture, Sports, Science and Technology. Retrieved from http://www.mext.go.jp/b_menu/hakusho/html/hpab200901/1305844.htm Mislevy, R.J., Steinberg, L.S. and Almond, R.G. (1999). Evidence‐Centered Assessment Design. Retrieved from: http://www.education.umd.edu/EDMS/ mislevy/papers/ECD_overview.html Richardson, F. and Suinn, R. N. (1972) The Mathematics Anxiety Rating Scale: Psychometric Data. Journal of Counseling Psychology, Vol 19, pp 551–554. Rounds, J. B and Hendel, D. D. (1980) Measurement and Dimensionality of Mathematics Anxiety. Journal of Counseling Psychology, Vol 27, No 2, pp 138–149. Shute, V. J. (2011) Stealth Assessment in Computer‐Based Games to Support Learning. In S. Tobias and J. D. Fletcher (Eds.), Computer Games and Instruction (pp 503–524). Charlotte, NC: Information Age Publishers Van Eck, R. and Dempsey, J. (2002). The Effect of Competition and Contextualized Advisement on the Transfer of Mathematics Skills in a Computer‐Based Instructional Simulation Game. Educational Technology Research and Development, 50(3), 23–41. Yoshizawa, M. (2006) Classification of Students' Stumbles while Learning Mathematics. Japan Journal of Mathematics Education. Vol 88, No 3, pp 24–28.

178

What Can Play Theory Tell us About Computer Games for Young Children? Georgy Gerkushenko1 and Svetlana Sokolova2 1 CAD, Volgograd State Technical University, Volgograd, Russian Federation 2 Preschool and primary education, Volgograd State Socio‐Pedagogical University, Volgograd, Russian Federation. mail@gerkushenko.ru svetlana@gerkushenko.ru Abstract: Play‐based learning is defined as a context for learning through which children organize and make sense of their social worlds, as they interact actively with people, objects and representations. Young children’s play allows them to explore, identify, negotiate, take risks and create meaningful ideas. Children who are constantly engaged in play experiences have much more developed memory skills, language development and self‐regulation than children who lack the play activity. The purpose of the paper is to find out what the most important factors are for teachers’ selection computer programs for kindergarten classroom activities. Whether the factors concern the theory of children’s play development? Do kindergarten teachers need the scaffolding program for their choice of computer games for using in the classroom? What should be the essence of the program? Preliminary study made by authors in Russian Federation shows the lack of teacher’s computer literacy. This situation leads the absence of computer games or incompetent using them in pedagogical work with children. By studying general characteristics of play we identify the main criteria which can be used for choosing appropriate game for classroom activities. For instance, choosing a game teacher should answer the question, if this computer game allows children to create their own scenarios, rules and characters of the play or if it enables children acting in an imaginary situation? The paper gives an overview of the computer games for preschool children used in Russian kindergartens; it also contains the scaffolding features on using computer games for children’s development. It summarized the problems and recommendations to scaffolding process for teachers who are interested in using computer games for effective children’s development. Keywordss: early childhood education, play‐based learning, kindergarten teachers’ training, computer games, scaffolding

1. Introduction One of challenges facing today’s kindergarten teachers is the constant pressure to teach more academic skills at a progressively younger age reducing the time for traditional Early Childhood activities. In contrast with this fact psychologists and education researchers stress play as preschool children leading type of activity, providing necessary skills and effective socialization (Vigotsky, 1977, Elkonin, 1978, Zaporozhets, 1978). Intellectual and social benefits of play as children’s activity have been documented widely (Lester & Russell, 2008, Vigotsky, 1977). Children engaged in play experiences are more likely to have well‐developed memory skills, language development, and are able to regulate their behavior, leading to enhanced school adjustment and academic learning (Bodrova & Leong, 2005). Following Bodrova and Leong (2003) nowadays young children spend less time at home playing with their peers and more time playing alone, in the classroom they tend to rely on realistic toys and props, and have a hard time using their imaginations to invent a substitute for a prop they do not have. According to long time play observation in kindergarten classrooms in Russia, China and France made by authors of this article children have frequent problems to try a new topic or plot, preferring instead to act out the familiar scenarios of family, school, or hospital. Unfortunately, play that exists in many of today’s kindergarten classrooms does not fit the definition of mature or well‐developed play. Even 5‐ and 6‐year‐old children who according to famous Russian psychologists Vygotsky and Elkonin should be at the top of their play performance often show immature play signs more typical for toddlers. Bodrova and Leong (2008) underline that important factors influencing such a serious situation with children’s development are following: increasing adult‐directed forms of children’s learning and recreation; proliferation of toys and games that limit children’s imagination, substitution of real play by “play impostors”. Analysis of the software for children shows the huge opportunities that computer games have for intellectual, emotional and social development, as well as for children’s learning (Verenikina, 2003). As we suppose the main goal for contemporary early childhood education practice is finding a balance between uncontrolled children’s playing computer games and adult‐directed activities for using computer programs to train or even

179

Georgy Gerkushenko and Svetlana Sokolova drill children’s academic skills. According to the theory of play and children’s development phenomena computer games and gaming platforms first of all should make an emphasis on make‐believe play and take into account the stages of play. Moreover, Susan Haugland (1992) underlines that adults play an essential role when computers are used successfully with young children. Meanwhile the survey showed the problem of negative attitude of significant amount of the Russian kindergartens teachers to computer games and their using in the classroom (Sokolova&Gerkushenko, 2002). The present paper is based on the idea of scaffolding children’s play by means of planed including of computer games into kindergarten classroom activities. As the benefits of computer games in teaching and learning are more or less evident for teachers and parents a variety of learning theories has been applied to examining the educational value of available software. However, in the literature, there is a lack of examination of kindergarten teachers’ competency in using computer games for children’s play development.

2. Motivation In 2011 International Center for the Childhood and education of Volgograd State Socio‐Pedagogical University started the project “Childhood without borders”. One of the objectives of the project was to support innovative practices in preschool education. The project had a number of subprojects which were oriented to different areas of children’s development. One of subproject realized cooperatively with Volgograd State Technical University was dedicated to using Information and Communication technologies in kindergarten. Main participants were teachers from municipal kindergartens. Totally 10 kindergartens were involved in the project. The essence of this subproject was in complex analysis of using computer games in kindergarten for improvement of children’s development. The area of problems we investigated was the competent scaffolding of children’s play activity as the necessary tool for their learning and development. The necessity of the research can be explained with the results achieved in 2002 when we define the kindergarten teachers’ attitude to computer games for children. We found that on average 90% of surveyed kindergarten teachers had negative attitude to computer games for young children. The interviews showed that teachers first of all were concentrated on possible harm that computer games can bring to children’s physical development. They were afraid that computer games can damage children’s eyesight (88%), cause their psychological dependence (49%), damage social skills like adapting, cooperative communication etc. (35%) (Sokolova&Gerkushenko, 2002). Such negative tendencies in teachers opinions were received as teachers supposed first of all the situations of uncontrolled children’s gaming, secondly the situations when children play computer games during their boring time, thirdly the examinees teachers mentioned on individual games where there is no possibility to cooperate with a peer. Most of the interviewed teachers (75%) never used computer games in their classroom supposing such a play activity as a redundant one and unfit for didactic goals. Ten years have passed and contemporary kindergarten teacher uses computer technologies much often than before. But still now computer games are not assessed by kindergarten teachers as learning tools. They prefer to use such games for physical diseases correction or for diagnostic reasons. Meanwhile Papert (1998) stresses that computers have an impact on children when the computer provides concrete experiences, children have free access and control the learning experience, children and teachers learn together, teachers encourage peer tutoring, and teachers use computers to teach powerful ideas. According to these ideas and the theory of play as the most powerful learning tool for preschoolers, the hypothesis of the present study we conclude in the statement that kindergarten teachers’ deliberate analysis of the play component of computer games could provide the enhancement of classroom activities quality by the developmentally appropriate choices of computer games.

3. Methods The research was carried out in Volgograd city over a two‐year period 2011‐2012 and involved 50 Russian kindergarten teachers who used computer software in education process. It aimed firstly to answer research questions:

What are the most important factors for selection computer programs by teachers during children’s classroom activities? Whether the factors concern the theory of children’s play development?

Do kindergarten teachers need the scaffolding program for their choice of computer games for using in the classroom? What should be the essence of the program?

180

Georgy Gerkushenko and Svetlana Sokolova The methods chosen to carry out the research concerned theoretical and empirical studies. Leading theoretical research we used methods of analysis, synthesis and classification. The empirical studies included teachers’ narrative interviews, the questionnaire analysis, and the experiment. For answering the research questions we used combination of open and closed type of questionnaires where teachers could choose the variant of answer or could write their own answer. The second goal was planned to achieve by organizing kindergarten teachers’ learning community for analysis the scaffolding methodology of children’s play development with support of computer game programs. The structure of research participants is shown in the Table 1. Table 1: The structure of kindergarten teachers Amount of teachers 4‐5 y.o. 5‐6 y.o. classroom classroom 10

Less than 30 y.o. 5

Teachers’ age 30‐40 y.o More than 40 y.o. 18

Experience of work Less 5‐10 More than 5 years than 10 years years 2 5 43

4. Theoretical studies 4.1 Play‐based learning in preschool age Play‐based learning is a context for learning through which children organize and make sense of their social world, relate actively with people, objects and representations. There is a long standing tradition in play research that focuses on play itself in its multiple forms, recognizing it as a distinct child‐initiated activity with its own unique contributions to child development. Jean Piaget (1962) and Lev Vygotsky (1978) were among the first who link play with children’s development. Roskok and Christie (2001) underline that play is not a singular construct but rather a continuum of playful behaviors that children engage in the context of Early Childhood classrooms, encompassing a set of behaviors that vary in terms of the degree of adult guidance and support. During the growing process new levels of play appear when children move from infancy to preschool age. There are several classification schemes for the play of young children. Piaget (1951) described three stages of play, in which children’s ability to think symbolically corresponds to the structure of the play. The first level is associated with the sensorimotor stage and is called functional or practice play and consists of repetitive motor movements with or without objects. Second level concerns symbolic, or pretend, make‐ believe play. The last stage of Piaget’s classification contains games with rules, which is based on children’s understanding and following rules in play activities. Parten (1932) described four categories of children’s play: nonsocial play, parallel play, associative play, and cooperative play. Those two last levels of play represent higher levels of interaction when children actually play together, doing similar things and coordinating their actions (Parten, 1933, in Dockett and Fleer, 1999). An essential characteristic of child’s play is pretending which is an action and interaction in an imaginary, “as if” situation, it usually contains some roles and rules and the symbolic use of objects (Leontiev 1981, Nikolopolou 1993).The investigation of the relationship between the quality of play and children’s educational outcomes discovers that mature (well developed) play is the most powerful tool for children’s learning and development. Bodrova and Leong (2003) define several quality characteristics of mature play such as:

Imaginary situations when children assign new meanings to the objects and people in a pretend situation. When children pretend, they focus on an object's abstract properties rather than its concrete attributes. They invent new uses for familiar toys and props when the play scenario calls for it. In doing so, they become aware of different symbolic systems that will serve them later when they start mastering letters and numbers.

Multiple roles which are not stereotypical or limited; the play easily includes supporting characters. For example, playing "hospital" does not mean that the only roles are those of doctors. Children can also pretend to be an ambulance driver or a phone dispatcher. When children assume different roles in play scenarios, they learn about real social interactions that they might not have experienced (not just following commands but also issuing them; not only asking for help but also being the one that helps). In addition, they learn about their own actions and emotions by using them "on demand." Understanding emotions and developing emotional self‐control are crucial for children's social and emotional development.

181

Georgy Gerkushenko and Svetlana Sokolova

Clearly defined rules. As children follow the rules in play, they learn to delay immediate fulfillment of their desires. Thus, mature play helps young children develop self‐regulation. To stay in the play, the child must follow the rules.

Flexible themes which are flexible enough to incorporate new roles and ideas previously associated with other themes. When children play at a more mature level, they negotiate their plans. By combining different themes, children learn to plan and solve problems.

Extensive use of language by children to plan their play scenario, to negotiate and act out their roles, to explain their "pretend" behaviors to other participants, and to regulate compliance with the rules. As the repertoire of roles grows, so do children's vocabulary, mastery of grammar and practical uses of language, and metalinguistic awareness.

Not limited length of play which can last for many days as children pick up the theme where they left off and continue playing. Creating, reviewing, and revising the plans are essential parts of the play. Staying with the same play theme for a long time allows children to elaborate on the imaginary situation, integrate new roles, and discover new uses for play props.

The theoretical analysis of the research papers of Lev Vygotskiy’s(1977), his student Daniil Elkonin (1978) and his follower Elena Bodrova (2010) on outcomes of children’s development through game‐based activity gave us opportunity to make a list of principal ways in which computer games could influence children’s psychological development.

Motives. Computer games can affect child’s motivation. Effective play scaffolding gives good opportunity to develop motives from the forms of affective immediate desires to a hierarchical system of children’s goals. Evidently it is more productive if the software gives possibility to children fix their planning results in graphic form (written or painted).

Decentering. Computer games can facilitate cognitive decentering. As a play role is the basement of such a decentering it is demonstrated in appearing of a role name and a role speech. This ability to take the role provides the possibility of new relationship form such as “I am” – “I am in role” where children can understand the difference between their actual position and the position of the objects, whose role they are playing.

Mental development. Computer games can advance the development of mental representations. Such a development takes place as the result of a child separating the meaning of objects from their physical form. In ordinary games it happens from using replicas to substitute for real objects, through using new objects which can perform the same function as the prototype, to such a substitution which takes place in the child’s speech with no objects present.

Self‐regulation. Computer games can foster the development of children’s deliberate behavior. It happens because of the necessity to follow the rules of a game. Later, this deliberateness extends to mental processes such as memory and attention.

4.2 Classification of educational computer games for preschool children Educational game‐based computer programs for preschool children are oriented for 3 to 8 years old users and according to developers are made with the ideas that play is the main activity for that age category. Classification of educational computer games is needed both for teachers and games developers. Teachers can find easily the necessary program if headings give answers for such questions as “The games for 3‐4 years old children”, “The games for speech development”, “Programs with animals images” etc. Moreover, for developers such classifying is helpful for their professional analysis of educational games market. Our analysis of educational computer programs existed in Russian computer market highlights following big groups of children’s computer games:

Developmental games. These programs can be described as “open” type ones, where the goal is not defined clearly and games become tools of children’s creativity and self‐expression. First of all these games are good for development of common cognitive abilities such as analysis, synthesis, critical thinking and others which are the basement of many kinds of human activities. Secondly, they can be a very strong tool for development children’s imagination and emotions. Such developmental games have a big potential for using them in education process of kindergartens being basement of lessons or other children’s extended activities (Perlmutter, 1985, Haugland, 1992).

182

Georgy Gerkushenko and Svetlana Sokolova

Learning games. These game programs are made especially for didactic goals and can be described as “closed” ones. Children are supposed to solve any learning task in a form of play. These are games for early mathematic learning, learning letters and sounds of language, writing through reading and reading through writing, for learning some ecological knowledge etc.

Games – experimentations. Goals and rules here are not defined very clear and are hidden in the game’s plot or in in the management tools. To succeed a child need to discover the goal and the mode of action by searching and solving problematic situations.

Games for entertainment. Such games do not have any goals by the first sight; they give opportunities to have some fun and to see the result of the game as a “micro cartoon”.

Computer games for diagnostic. In spite of the fact that all developmental and learning games could be defined as diagnostic games, there are special computer programs which can be identified as psycho diagnostic and validated methods. Those programs fix and memorize given parameters, then process and memorize the results. Further, the results could be shown on the display or be printed for interpretation by psychologist. Moreover variants of interpretation can be programmed and given by computer automatically. Results of diagnostic can be given as recommendations to kindergarten staff or parents. Also these types of programs can be computer methods of express diagnostic of different systems of child’s organism; they provide opportunities to define pathologies very fast. Empirical analysis shows that such programs can be used in kindergartens for: diagnostic children’s general cognitive skills; evaluation of development psychological functions: memory, attention etc.; diagnostic of creative abilities of children; identifying readiness of children to kindergarten life; identifying readiness of children to school life; express diagnostic of child’s fatigue during computer usage.

Computer games for therapy and correction. Kindergarten specialists use such games to correct or cure some physical or mental diseases. In inclusive kindergartens there are very useful game‐programs for blind and deaf children, autistic children and others.

5. The empirical study 5.1 Assessing kindergarten teachers’ potential in using game‐based software Before assessing the situation with using computer software in work with children we made a short survey on the teachers’ understanding of children’s play activity essence to contextualize our present research. The questionnaire included three dedicated questions on the importance of play in preschool age, essence of play, using of play in kindergarten classroom activities. 1). Answers to the first question “Do you think play is important for children’s development?” were as follows: 100% of teachers are sure that play is very important for children and it is very good way for learning. 2). The second question was open when teachers could answer using their own ideas. The investigation on the teachers’ thoughts of children’s play showed that: 5% of examinees think that play is children’s activity during their spare time; 10% of interviewed teachers suppose that play is every activity made by children including gardening, painting etc. 30% of teachers think that play essence is fun and pleasure; 55% of kindergarten teachers are sure that essence of play is in pretending. 80% from these teachers prefer to organize “as if” situation for children to achieve the curriculum goals and only 20% try to be a part of children’s imaginary situations give children opportunity to create their play freely. 3). Asking the questions “Do you use play organizing your classroom activities? What types of play do you organize?” we collected following data: 100% of teachers use play in their classroom. They told that almost all activities that they organize with children they do in the form of play.

183

Georgy Gerkushenko and Svetlana Sokolova 100% of interviewed teachers organize didactic play with children where learning goals are defined clearly, rules are necessary to accomplish. Didactic games used in kindergarten can be organized like board games, word games or games with objects. Thus, the findings can be summarized as follows: 1. Teachers mean different things by “play”. 2. The relationship of play to learning activity was articulated by all examined kindergarten teachers. 3. Even when teachers said about importance of play, or that play leads to learning, they were usually referring to an understanding of play as a highly scripted, teacher‐directed activity. The situation of multiple visions of teachers on the play essence on the one hand and the teachers’ confidence in importance of play activity for children’s learning on the other hand determine the character of the experimental work. First stage of our experiment was dedicated to defining whether teachers used computer software in education process and if they used it what kind of software it was. To collect the data we prepared a combined type questionnaire where teachers should mark whether they use computer for children’s education and then write the preferable software for classroom usage. The result is shown in the Table 2. Our statistics indicates that 40% of teachers do not use at all computer software for organizing children’s activities. As it is very big percent of research participants we organized further analysis of such a result reasons. We drafted open type questionnaire where teachers had to write the main reason of avoiding the computer games in organizing classroom activities (Table 3). Meanwhile 60 % of kindergarten teachers use different computer software during classroom activities. Teachers indicated PowerPoint presentation as the most useful in working with children. Much less teachers used computer games in their pedagogical work with children. Moreover no one of them used developmental games and games‐experimentations. In the interview they marked difficulties they faced during planning such game‐based activity, especially in the situation when game goals are not clear and there is a big possibility of unexpected results of the game. Table 2: Using software by kindergarten teachers Computer software Do not use computer software Use computer software:

Teachers (%) 40% 60%

1. 2. 3.

Power Point presentation Multimedia resources (audio, video) Computer games for diagnostic

92% 15% 12%

4. 5.

Computer games for therapy and correction Computer games for learning

18% 56%

Table 3: Reasons of avoiding computers by kindergarten teachers in their classroom activity Reasons Absence of computers in the classrooms Forbiddance of using computers in the classroom from the parents Forbiddance of using computers in the classroom from the kindergarten leaders Low level of computer literacy Absence of time Deficit of methodical information

Teachers (%) 63% 0,5% 0,5% 21% 4% 11%

Analyzing the teachers’ questionnaires we concluded that three causes of avoiding computer games and other software are the most popular in teachers’ answers. First of all a number of kindergarten classrooms are not equipped with computers, so teachers have no experience in using computer games in education process. Secondly some of the teachers (21%) confessed very low level of computer literacy. Teachers from 45 to 55 years old predominated in this group of respondents. They marked problems of using internet services (54%), anxiety during computer usage (2%), limited computer skills by using MS Word (34%) or even inability to use the computer (10%). Also big percent of teachers (11%) felt lack of methodical guidance of using different computer games in education of preschool children.

184

Georgy Gerkushenko and Svetlana Sokolova Meanwhile the teachers who marked computer games in the list of software were involved in further diagnostic process and had to define criteria of choosing digital games for children. We used method of narrative interview to receive data on teachers’ experience in using computer software in their work with preschoolers. Approximately 50 teachers were interviewed during 5 month. Such a qualitative method provided us opportunity to make a wide analysis of teachers’ priorities in organizing children’s education. There is an example of the interview with the teacher whose priority is speech therapy. “Due to curriculum limitation of time for using computer software in work with children till 5‐10 minutes a day I personally use computer games only as one stage of a lesson. Using special computer logopaedic software “Igry dlia Tigri (games for Little Tiger)” makes my lessons much interesting than before. These games are oriented to overcome children’s speech problems. Comparing traditional language correction methodic with this software I find computer programs more effective and dynamic because of its interactive and game‐based form of exercises. My computer literacy gives me possibility to make computer presentations for children’s speech correction goals. In my presentations I use graphic, text, sound and video tools noticing on the one hand the increasing of demands to educational presentations and on the other hand the increasing of services for presentations creators. The advantage of presentation is in ease of making. I can scan pictures from the books or find images in internet and then just put them like slides in MS PowerPoint. So the information that I prepare for children becomes colorful and interesting. In my opinion bright visibility is the main factor for choosing any computer technologies because animation and moments of surprise make correction process expressive and interesting for children”. Analysis of teachers’ interviews showed that all factors for choosing software indicated by teachers can be organized in a small list.

57% of teachers defined bright visibility of information as the main factor for choosing software;

23% examinees choose software guided by variety of learning tools;

10% are interested in cooperation skills development and are guided by the possibilities of software in initiating of children’s group work;

10% of teachers seek the compatibility of the software services with kindergarten curriculum aims.

The received results of the narrative interview showed the orientation of teachers on extrinsic side of organizing activity (90%) instead of intrinsic one based on play development. We could conclude rather low level of investigated teachers in using game‐based software. This situation showed the importance of special guiding work to involve teachers in competent analysis and future using of computer software for children’s game‐based learning.

5.2 Discussion on scaffolding of teachers’ choosing computer games for children’s play development Further empirical study we organized in accordance with Epstein (1993) ideas, who identified four critical components of teachers’ training: practical experience, workshops, models and mentors, and supervisory follow‐up. The essence of experiment as a research method is in changing of one or several components of the object environment. The object of our research was not a person but the process of using computer games in kindergarten classroom activity. We implemented Epstein’s model for the scaffolding program. As the first step, teachers explored software that could be developmentally appropriate for their classrooms. During this first stage of the experiment that continued 3 month teachers had to review 2 popular computer games for preschool children as potential for classroom usage. We did not provide any criteria for reviewing and gave teachers opportunity to use their practical experience in creating the criteria on the one hand and one the other hand during this time teachers accumulate the new experience in games evaluation. Thus, this stage allowed all teachers in spite of their previous experience to summarize and structure their own ideas and experience in the area of computer games for young children. The second stage of the experiment included discussion on the potential learning objectives of the activities that teachers could use to integrate particular software in their classrooms. Teachers participated in workshops that integrated the developmental theory and research regarding computer use with hands‐on experiences. During the seminars every teacher presented the experience on reviewing the games. The analysis of presentations showed the stable continuity of teachers’ orientation on extrinsic side of games

185

Georgy Gerkushenko and Svetlana Sokolova analysis (60%) instead of intrinsic one based on children’s play development (40%). The list of reviewing criteria consisted of following statements:

40% of respondents marked the interesting plot of a game as the criteria for game evaluation;

35% used as the criteria the colorful of a game picture;

60% of teachers put on the first place the learning potential of a game: the mathematic skills development, language learning etc.

25% were oriented on the possibilities of several children’s communication during evaluated games playing;

15% of teachers paid attention to possibilities of changing game components like plot, environment or game heroes according to children’s imagination.

On the third stage of experiment we added the guidance service for teachers‐participants of the research. We organized kindergarten teachers’ learning community on the base of Volgograd Socio‐Pedagogical University in the International Center for the Childhood and education to achieve the collective analysis of existing situation with game‐based kindergarten learning. The community included more than 50 teachers. Teachers were divided into several work groups for working on creation the data for scaffolding of children’s play development by using computer games. Each group should continue analyzing previously chosen developmental computer games for 5‐7 years old children. The groups were organized according to the working place of its participants. Each group presented one kindergarten and consisted of 5 teachers. Before started the group work we organized a workshop where discussed with teachers the play theories and in brainstorming way teachers tried to formulate and then discuss the common list of principal ways in which computer games could influence children’s psychological development. Aiming to help kindergarten teachers in games evaluation we made a table of helpful criteria for choosing developmentally appropriate games. Making the table we took in mind the main characteristics of play development: general characteristics of play, and theories of play (Table 4). The table had a mission to help teachers avoid the extrinsic approach to computer games choosing. Table 4: Criteria of choosing computer games for children Characteristics of play Play is a spontaneous, self‐initiated and self‐regulated activity. Play includes a dimension of pretend.

Play consolidates learning that has already taken place while allowing for the possibility of new learning in a relaxed atmosphere.

In play children achieve a mental representation of social roles and the rules of society.

Criteria for teachers This computer game allows children to freely engage in play. It provides a freedom of choice. This computer game allows children to create their own scenarios, rules and characters of the play. This computer game enables children acting in an imaginary situation? This computer game has the potential to develop new concepts and skills. What are the concepts and skills? This computer game allows for and nurtures the active participation of the child. This computer game engages the child in problem‐ solving and self‐discovery. This computer game involves and develops use of symbolic meaning. This computer game provides children with an opportunity to act out and explore the roles and rules of functioning in adult society. This computer game allows for group work and collaboration?

The teachers presented the results of collective research on monthly seminars and shared ideas in web activity on the internet page of the International Center for the Childhood and education. Each seminar topic concerned one play characteristic and corresponding criteria of computer games analysis. The groups were mixed and consisted of teachers who worked with computer software in their classrooms and who did not use any software before the experiment. The results of teachers’ researching activity are presented below.

186

Georgy Gerkushenko and Svetlana Sokolova 1. There is a difference between “real” games and computer games in visual separating the meaning of objects from their physical forms. For comparing play activity in real and virtual reality we used terms computer game and “real” game to underline the realistic or nonrealistic nature of actions. In computer games actions take place in imaginary reality but with real feelings of players. The oral speech loses the main role in creation and supporting of imaginary situation because every situation detail is seen on the screen. The potential of child’s cognitive development could be reduced also due to absence of symbolic substitutions necessity in most computer game programs. It occurs because technologies allow children to create any objects in virtual space of a game. These factors cause risks that play actions realized in computer games do not become generalized and minimized as they do in “real” games. To avoid these risks kindergarten teacher can organize special scaffolding of mental representations development by including speech actions (oral and written) on different computer game situations in classroom activities; also it can be helpful if teacher combines computer and real playing on one topic to create symbolic objects as substitutions of the real ones. 2. Analysis of computer software for children highlighted another difference of imaginary situations in “real” and computer games. The giant difference lies in nature of the situation. In computer games that we studied the situation was created by the game developers not by children. Children can play within the frameworks of created situation but cannot principally change it. If they play social situation with computer as a partner they should follow computer guided program of relationships. It can significantly narrow down the developmental potential of playing activity. Scaffolding program can include group playing of one situation when children share their ideas on the scenario development, plan actions in cooperative way, and the most important continue the computer game scenario in “real” play where they are absolutely free in their imagination. 3. The stage of preliminary orientation in computer games acts not on the semantic level but on the level of actions. Awareness of the action mode before its starting is the feature of child‐computer relationship. That is why knowing of the rule and actions modes should exist in child’s mind before computer playing and scaffolding program should include preliminary discussion of future play rules, kinds of actions and modes of manipulating. 4. The main problem of using computer games for education goals lies in the plane of taking the role. In most computer games the plot is defined externally. Therefore the roles are imposed with graphic images, actions modes even names. Sometimes the role prototype is not defined and should be created in playing process. In case of such “independent” existence of a role there are two variants of interrelation:

Identification with a role, transferring the part of “I am” on the computer game hero and further playing in the form of the hero management.

Partnership with a game hero, cooperative playing with the new friend.

5. Computer technologies give opportunity for trying both types of interrelations with the game hero. For the first type the games should show the game situation directly “from the player’s eyes”. For the cooperative playing with game personage there are games where situations provide the view to the personage “from the outside”. It is evident that the second type of games should be chosen by teachers for education and development goals because appearing in the game of other person will enable children to develop coordination of their cognitive perspectives with their learning partners and teachers. After cooperative analysis of the group research results teachers were involved in focus group interview. They discussed the new ideas they obtained during the experiment, assess the new experience and its future application in the classroom activity.

75 % of teachers noticed the new skills in organizing integration between educational areas with the computer games tools;

16% of respondents were surprised by discovering of learning possibilities of computer games – journeys;

25% noted the importance of the scaffolding program designed to support the process of choosing and using computer games in kindergarten classroom activity.

78% of teachers marked the increasing of their activity in implementing computer games in kindergarten education practice.

187

Georgy Gerkushenko and Svetlana Sokolova

68% of respondents noted the enhancement of classroom activities quality by guided implementing of computer games: increasing of children’s play plots, the enrichment of plays scenarios, the enhancement of children’s experimentation.

100% of teachers were satisfied with learning community working and underlined its impact into their professional development.

6. Conclusion This paper discussed the problem of preschool children’s play development by using opportunities of computer games. Study of the play‐based learning features in preschool age showed that play‐based learning is a context for learning through which children organize and make sense of their social world, relate actively with people, objects and representations. The main idea of the paper is in understanding of a play as not a singular construct but rather a continuum of playful behaviors that children engage in the context of Early Childhood classrooms, encompassing a set of behaviors that vary in terms of the degree of adult guidance and support. Special scaffolding actions made by teachers are necessary due to crucial importance of children’s development outcomes obtained through game‐based activity. Such outcomes include establishing of motives hierarchy, cognitive decentering, mental representations and others. To be a competent user of computer games developmental tools teachers need special scaffolding program which supports them on the way of realizing the developmentally appropriate education. The research included several stages. On the first stage aimed to clarify the factors influencing teachers’ choices of computer games the questionnaire allowed to define that all the teachers’ factors are not oriented directly to development of children’s play activity. This fact determined the orientation for further research work united more than fifty kindergarten teachers from Volgograd city. Next stages had experimental character and were associated with involving teachers to professional learning community for analyzing computer games for children. Teachers passed several steps from independent analysis of games to scaffold analysis oriented to the development of children’s play activity. The results showed that teachers were in need of special scaffolding program. The essence of this program was in sequence of organized teachers activities: practical experience, workshops, models and mentors, and supervisory follow‐up. The big role in this program played the professional learning communities of teachers provided the atmosphere of creation, exchange of experience and cooperation. After the experiment all teachers increased their computer literacy and acquired their own competent position on the question of using computer games with preschool children. Moreover, implementing the computer games to kindergarten classrooms in the framework of the experiment provided the improvement of the quality of children’s education. Thus we can conclude that deliberate analysis of the play component of computer games made by kindergarten teachers can really provide the enhancement of classroom activities quality by the developmentally appropriate choices of computer games.

References Barton, E., Wolery, M. (2008) “Teaching pretend play to children with disabilities: A review of the literature”, Topics in Early Childhood Special Education Vol 28, No.2, pp 109‐125. Bodrova, E., Leong, D. (2010) “Curriculum and play in early child development”. Encyclopedia on Early Childhood Development [online]. Montreal, Quebec: Centre of Excellence for Early Childhood Development, pp 1‐6. http://www.child‐encyclopedia.com/documents/Bodrova‐LeongANGxp.pdf Bodrova, E., Leong, D. (2003) “The Importance of Being Playful”, Educational leadership. April, Vol 60, No. 7 http://pdonline.ascd.org/pd_online/substitute/el200304_bodrova.html Dockett, S. and Fleer, M. (1999) Play and pedagogy in early childhood. Marrickville, NSW, Harcourt Brace. Elkonin, D. (1978) Psychologija igry [The psychology of play]. Moscow: Pedagogika. Epstein, A. S. (1993) Training for quality. Ypsilanti, MI: High/Scope Press. ED 370 674. Fleer, M. (2009) “Supporting scientific conceptual consciousness or learning in a roundabout way in play‐based contexts”, International Journal of Science Education, Vol 31, No. 8, pp 1069‐1089. Ginsburg, H. (2006) “Mathematical play and playful mathematics: A guide for early education”. In: Singer, D. Golinkoff, R., Hirsh‐Pasek, K. (Eds). Play = Learning: How play motivates and enhances children’s cognitive and social ‐ emotional growth, New York, NY: Oxford University Press. Haugland, S. W. (1992) The effect of computer software on preschool children's developmental gains. Journal of Computing in Childhood Education, 3(1), 15‐30. EJ 438 238. Leontiev, A.N. (1981) Problemy razvitiya myshlenia [Problems of the development of the mind]. Moscow, Progress Publishers. Nikolopolou, A. (1993) Play, cognitive development, and the social world: Piaget, Vygotsky, and beyond. Human Development. Vol 36, pp 1–23.

188

Georgy Gerkushenko and Svetlana Sokolova Novoselova, S.L. (2003) “V chem problema informatizacii doshkol’nogo obrazovania?” [Problems of preschool education informatization], Detskiy sad ot A do Ya [Kindergarten from A to Z], No.1, pp 6‐13. Parten, M. (1932) “Social participation among preschool children”, Journal of Abnormal and Social Psychology, Vol 27, pp 243–369. Papert, S. (1998) Technology in schools: to support the system or render it obsolete. Milken exchange on education technology [online]. http://www. mff.org/edtech/article.taf?_function=detail&content_uid1=106 . U.S. Congress. Washington, DC. Perlmutter, M. (1985) Social influence on preschool children’s computer activity. National Institute of Education, Washington, DC. Piaget, J. (1951) Play, dreams, and imitation in childhood. London: Routledge. Rogers, S., Evans, J. (2007) “Rethinking role play in the reception class”, Educational Research, Vol 49. No 2, pp 153–167. Roskos, K., Christie, J. (2001) “Examining the play–literacy interface: a critical review and future directions”, Journal of Early Childhood Literacy, Vol 1. No.1, pp 59–89. Saracho, O., Spodek, B. (2006) “Young children’s literacy‐related play”, Early Child Development and Care, Vol 176. No. 7, pp 707‐721. Sokolova, S. Gerkushenko, G. Dvoryankin, A. (2002) “Informacionnie y communicacionnie tehnologii v doshkol’nom obrazovanii [Information and communication technologies in Early Childhood education]”. Proceedings of the conference on Information technologies in education, technology and medicine: Volgograd. Vol 1. Volgorad State Technical university press. Uren, N, Stagnitti, K. (2009) “Pretend play, social competence and involvement in children aged 5‐7 years: The concurrent validity of the Child‐Initiated Pretend Play Assessment”, Australian Occupational Therapy Journal, Vol 56, No. 1, pp 33‐40. Verenikina I. (2003) Child’s Play: Computer Games, Theories of Play and Children’s Development. Australian Computer Society, Inc. Vygotsky, L. S. (1956) Izbrannye psychologicheskije Trudy [Psychological studies]. Moscow: RSFSR Academy of Pedagogical Sciences. Vygotsky, L. S. (1977) Play and its role in the mental development of the child. In M. Cole (Ed.), Soviet developmental psychology. White Plains, NY: M. E. Sharpe. Wood, E. (2009) Conceptualizing a pedagogy of play: International perspectives from theory, policy, and practice. In: Kuschner, D, From children to red hatters: Diverse images and issues of play. Lanham, MD: University Press of America. Zaporozhets, A. V. (1978) Printzip razvitiya v psichologii [Principles of development in psychology]. Moscow: Pedagogika.

189

Role Game Playing as a Platform for Creative and Collaborative Learning Lisa Gjedde Dept. for Learning and Philosophy, Faculty of Humanities, Aalborg University Copenhagen Denmark lg@learning.aau.dk Abstract: Game‐based learning may present a way of creating immersion and engagement for the learner through simulated experiences in a narrative environment, and may support the development of 21st. century skills of communication, collaboration, creativity and critical thinking. Role playing games have had a long history of usage in language learning and as a multidisciplinary activity in schools during theme weeks. Through the concept of serious digital games, which offers learning through digital simulations and immersion in virtual worlds, game‐based learning has been deployed increasingly in education where games have been used for specific subjects. The use of live role‐game playing in schools offers novel and innovative ways to work with the game genre, with the teachers contributing as game authors and game masters. A current project on live action role‐game playing looks at how role‐game playing can be used to present an entire curriculum within a narrative framework in order to enhance the learners’ motivation and zest for learning while developing 21st century learning skills. The exploration of how live action role‐game playing can function as an overarching framework for learning may offer fresh insights into game‐based learning in terms of multimodality, flexibility in the design of games and the role and interactivity of the learner and teacher. A unique residential school dedicated to teaching all subjects in grades 9‐10 through live role‐game play was studied for a year. The study employed qualitative and processual methodologies in order to capture the interactions between students' learning experiences and the role‐game based learning designs as well as the way they constitute a creative and collaborative learning environment. This paper presents the preliminary results of the project and discusses its implications for design and redesign of learning environments in the schools along with the roles of learners and teachers in the development of game‐based learning as a framework for creative, inclusive and collaborative learning. Keywords: role game play, game‐based learning ,creative learning through role game play, situated and contextual learning using role game play, role game play as a framework for the curriculum

1. Introduction Game‐based learning offers the learner the potential of creating immersion and engagement through simulated experiences in a narrative environment in support of the development of 21st. Century skills of communication, collaboration, creativity and critical thinking. By means of the concept of serious digital games ‐ which offer learning through digital simulations and immersion in virtual worlds (Dede 2009) – game‐based learning has been used increasingly in educational settings (Spires,Turner et al 2008). The use of Live Action Role‐game Playing (LARP) in schools, with teachers contributing as game authors and game masters, offers novel and innovative ways to work with the game genre. A current study of narrative game‐based learning at a residential school in Denmark, Østerskov Efterskole, examines how LARP, in combination with other game‐based approaches, can be deployed to present an entire curriculum nested within a narrative framework. Among the issues being explored here is how LARP may enhance the learners’ motivation and zest for learning and to develop 21st. century skills (Bellance 2010). The low‐tech analogue approach to games makes it possible to experiment with many different ways of integrating games into the curriculum. It offers accessible ways of testing the workings of games, and the embedding of the curriculum in a narrative framework as it is driven both by the dynamics of the unfolding participatory scenario and the dynamics of the game with its rule‐based environment. LARP as a genre also brings forward design considerations for narrative game‐based learning that may prove relevant to the development of designs for digital games and their pedagogical framing for their application in public schools. The research project is designed in three phases, exploring the pedagogical potential of educational LARPs (edu‐LARP) and how they can afford a creative, engaging and inclusive learning environment. Phase one involved observation of the implementation of a series of edu‐LARPS at Østerskov Efterskole that served as the narrative and game‐based frame for the curriculum of an entire school year. Phase two explored the implementation of two edu‐LARP learning designs framing the entire curriculum in a municipal school for a period of six weeks, involving grades six, eight and nine. In the third phase, the models of game‐based learning

190

Lisa Gjedde designs that evolved during the first two phases will be developed further and implemented by practicum teaching students in a number of municipal schools. This paper will focus on presenting some of the preliminary results from phase one of the research project. It will discuss the use of edu‐LARPs at Østerskov Efterskole in relation to concepts of creative learning, intrinsic motivation and inclusion, as well as the underlying designs that frame the learning experiences.

2. Background While much research has been conducted on implementing serious games in education, there is a dearth of research‐based knowledge on the use of narrative game‐based learning as a general platform for delivering the curriculum. By exploring the LARP approach, the research aims to provide a baseline for developing models and learning designs for the use of role game play in the public schools that is both inclusive and creative in its approach to learning. The point of departure for the project was the need to develop new forms of learning which both allow for the inclusion of pupils with special needs in the regular classroom, and offer adaptable challenges for the entire student spectrum ranging from the academically weak, a middle group and the academically strong. It calls for new forms that are motivating and engaging, as well as easily adaptable in a way that many digital resources are not, due to cost and development time. There is a need for the development of a learning environment that allows students to participate creatively and actively in the learning process, doing so according to their different abilities, backgrounds and preferred approaches. Many pupils find it difficult to gain sufficient competencies from their schooling and they risk leaving school without the necessary basic skills and with no desire to go further in education. The Danish Government's so‐ called flying squad research group, set up in early 2010, points to a lack of knowledge and research into what techniques and approaches work in the classroom and of how to produce effective educational design. Its general conclusion was that there is a need to strengthen the knowledge base for public school teaching Nordahl et al.(2010). Therefore, there is an identified need to develop knowledge about new learning methods and approaches that allow students to acquire academic skills while experiencing a meaningful learning context that provides the opportunity to participate creatively and productively. LARP is one method to explore these relationships and to work creatively. As a genre, it is familiar to the general public as part of the leisure facilities offered to children and youth in Denmark and many children have taken part in it through local Role‐Playing clubs and events hosted by national television. Also, there is a well‐establish Nordic scene for LARP, as a creative and artful mode of expression based on narrative scenarios (Montola & Stenros 2004). In 2006 a group of game enthusiasts, some with LARP backgrounds, founded the residential Østerskov Efterskole, which bases its entire pedagogical approach on the use of narrative game‐based learning, described in Hyltoft (2008). Denmark has a well‐established tradition for about 15% of the learners in grade 9 or 10 to go to a residential school, termed “Efterskole” or “Afterschool”. At Østerskov Efterskole, all subjects are embedded in and taught by means of a narrative game‐based environment, based on LARP. All the academic goals are met through a curriculum planned and implemented as narrative game‐based learning. Furthermore, the school includes, with very promising results, about 30 percent students with autism, ADHD or other diagnosed special educational needs, in the Game‐based teaching, together with students with high academic and social functioning. The school is now in its eighth year of the program with a constant number of 80‐90 pupils.

Figure 1: Learners and teachers in a Live Action Role Play at Østerskov Efterskole

191

Lisa Gjedde

3. Exploring creativity in live role game playing A point of departure for the research was the need for developing learning designs that empower the learners and enable them to practice 21st century learning skills. Of particular interest in this project is the ability to participate in creative collaboration and to provide a framework for inclusive learning. A hypothesis of the research is that role‐game playing, as a platform for learning, affords learners the opportunities for creative learning linked to intrinsic motivation. A further hypothesis is that this may be linked with the adaptive aspects of the LARP genre that furthers social creativity through a participatory culture, being able to be easily adapted by the teachers to the learners’ level and afford differentiation. The edu‐LARP approach to creative learning may afford meaningful learning contexts through the use of narrative and game‐ elements. Their potentials for differentiation can provide a more engaged learning experience for gifted and challenged learners alike. This present research highlights the area of LARP as a framework for learning. The schools two teams of teachers have conceived, planned and written the scenarios in creative processes. The learning environment they are providing is integrating ICT with new media as tools within the analogue game environment. One main focus for this project has been to explore the implications of role‐game playing for the learners’ experience and construction of learning, of their creativity and their appropriation of the potentials for creative learning. It is, therefore, a goal for the project to realize an understanding of the possibilities and models that can underpin the development of educational designs through the use of LARP, serving as a framework for learning. An example of an edu‐LARP produced by the teachers, and refined through many iterations ‐ is a scenario based on Jules Verne’s story Around the World in 80 days. In this LARP the learners plays ladies and gentlemen, and are organized in teams of different competing nationalities. On this journey the goal for the teams is to be the first to circumnavigate the world in less than 80 days in order to win the grand prize. Their journey takes them to many destinations where once they arrive in the capital of a particular country they need to solve tasks integrating the subject content of languages, geography, history and math. By solving the problems they may raise sufficient funds to buy tickets for further travels either by train, boat, plane or even a hot‐air balloon. A well designed holistic LARP, succeeds by integrating the curriculum into the game, while the learners work to deal creatively with the challenges and social interactions that are part of the game. The learners are challenged to draw on domain‐relevant skills, both with respect to the game’s integrated subject matter and to the Role Game Play itself. At the same time they are motivated intrinsically by the game‐play and the narrative. The design of the games opens up for students’ expression through creative writing. For example, a team might be tasked to help a group of merchants in Singapore by devising a plan for getting rich, drawing on knowledge gained about local customs, infrastructure, etc. Likewise, knowledge of natural science helps the team navigate a hot‐air balloon safely back to England. The use of these concepts learned within an emergent narrative in order to solve a problem, is a creative endeavor as is the participatory performance of it. Creative processes, interactions and expression in the LARP can happen at several levels:

At the level of embodiment (immersion): building a character, and expressing the character through costume, gestures and voice.

At the level of story : (epic) ‐ the emergent narrative development of the drama

At the level of game: (ludic) following the rules, receiving feed‐back through progression, obstructing competitors from other nationalities/teams

At the meta‐level – (mindful) becoming aware of the underlying game‐rules being empathic about the rules, roles and story unfoldment and even trying to tweek them.

192

Lisa Gjedde

Figure 2: Model of levels of creative expression in LARP

4. Designing for creative learning in narrative game‐based environments An important aspect of the game‐based learning environment is the sense of immersion it may offer the learners through participation in an emergent interactive story. This can be observed in the perspective of situated (Lave & Wenger 1991, Lave 1991, Wenger 1998) (Barab & Plucker 2002) and embodied learning (Johnson 2008). Immersion here is to be understood as the interaction between the learner and the environment, engaging all senses in an integrative mind/body experience, facilitating embodied and dynamically situated learning. The narrative immersion, supported by the game‐based learning environment, may therefore be conducive to embodied learning, bringing into play all the senses and supporting exploration of the possible universes that can lead to creative learning. According to the seminal creativity researcher Eduardo de Bono (1992), who has focused on strategies for creativity and cognition,: "Possibility is the key to creativity". Thus the interaction within a new possible narrative world of the learning games each week of the school year is a potential framing that may engender cognitive flexibility and creativity. The cognitive scientist and psychologist Maggie Boden, who has explored creativity in relation to the human mind, defines three modes of creativity: “a) combining ideas in novel ways b) exploring conceptual spaces and c) transforming conceptual spaces‐‐" in short, any disciplined way of thinking that' s familiar to (and valued by) a certain social group"( Boden 2004). The concept of learning through Live Action Role Play is an enlarged space compared to the conceptual space of the ordinary classroom and it opens up for the transformation of the conceptual space. In LARP there exist many basic elements that can be combined and recombined. The narrative scenarios draw on and present familiar stories that can then be tweaked by the single characters playing the roles in emergent interactive narratives embodying knowledge in different modalities (Kress, G.R. and Van Leeuwen, T. 2002) and at different levels. Amabile, one of the leading creativity theorists, points in her componential theory of creativity (1996, 2012) to the importance of the context in a creative environment, and suggests that: four components are necessary for any creative response: three components within the individual – domain‐relevant skills, creativity‐relevant processes, and intrinsic task motivation – and one component outside the individual – the social environment in which the individual is working.” Amabile states that creativity development is contingent on intrinsic motivation coupled with skills.

193

Lisa Gjedde In this study, we are looking both at creativity in relation to intrinsic motivation and skills and to creativity as it unfolds in a social context that is geared towards supporting and enhancing it. Vygotsky (2004) defines creative activity as: “when one combines, alters, and creates something new, no matter how small". We have been looking at the creative processes as they unfold, not necessarily producing creative artifacts, but rather as the intrinsically driven processes of exploration, coupling the learning of concepts with the creation of new meanings from multiple perspectives of the roles. This connection with the narrative framework allows for multiple perspectives of the learner who is provided with the opportunity of viewing it through the perspective of different roles. Sometimes the role is taken on only for the day, sometimes a role is developed over a longer period becoming part of the individual learner/player’s repertoire, which forms a creative construct and a way of embodying learning. The group dynamics of the weekly changing teams of learners can be an intrinsically motivating factor in itself as has been expressed by the learners in the interviews and the video‐diaries. However, the dynamics of a group containing some learners who are unwilling to be active collaborators, and in that particular situation act as “slackers”, can prove detrimental to the sense of engagement and creativity of the group as a whole and present a challenge for the committed learners. The model in Fig. 2 depicts the interplay between the different levels of the learning context and the emergent creativity anchored in the narrative game‐based learning environment.

Figure 3: Creativity through role‐game play model, drawing on Amabile’s componential creativity model (Amabile 1996,2012) The overarching concept of the game‐based narrative learning environment provides the context for the creative processes as they are expressed through interactions at the level of the Live Action Role Play and drawing on the relevant domain and curriculum: which can be related to the conceptual learning.

5. Methods The project has taken a mixed methods approach, employing surveys as well as qualitative and processual methodologies drawing on phenomenological methods (Gjedde & Ingemann 2008) in order to register and study the interactions between the students ' learning experiences and narrative game‐based learning designs. A qualitative approach was chosen to explore the learners’ subjective experiences and construction of meaning. Semi‐structured interviews were conducted with focus groups of learners in order to stimulate discussions as well as individual interviews and reflections. Teachers were asked to provide names of learners to be interviewed in order to include participants at different academic levels, ranging from challenged learners, to middle and high performers. Informants were selected for another round of interviews, based on the learners’ self‐reported level of involvement in the learning experiences in the weekly surveys. Video documentations of selected parts of the scenarios were made by the researchers, while learners and teachers contributed with video‐journals. Teams of two to three learners took turns producing video‐journals on their experiences with the scenario in terms of learning and collaboration. Select scenarios were video‐ documented by the teachers who were responsible for the particular scenario. Each week’s scenario was evaluated in a large group session with all the learners, which was video recorded. All the video data, evaluations and interviews have been content analysed and selected sections have been

194

Lisa Gjedde transcribed. Based on the analyses, models for learning have been developed, and will be further implemented in learning designs to be tested in the next phase of the project. Each of the different scenarios, which each lasted for a week, represented a unique learning design within the narrative framework of a live action role‐play. Some of these scenarios have been repeated and refined since the opening of the school in 2006. They have been through iterations with different teams of teachers each developing and redesigning them in a creative team‐based process. This collection of learning games ‐‐or edu‐ LARPs‐‐ can offer unique insights into the approbation of the concept as a platform for the entire curriculum.

6. Narrative learning, inclusion and game genres One of the key issues in integrating narrative game‐based learning into mainstream ordinary education is whether such a learning platform can fulfill the curricular goals, and enable the students to pass their final exams adequately. While there may be many positive implications in terms of developing social skills and creative expression, questions may arise about how the students actually perform measured in relation to traditional schooling, and how the school contributes to prepare them for youth educations. The exam results from the grade nine and grade ten learners graduating from Østerskov Efterskole, compared with the final exams countrywide, shows the school scored just as well as the average schools and, in some subjects, above the national average at the final examinations. That is in spite of the school having a much higher percentage of special needs learners than the average, about 30 percent. This indicates that this learning platform has potentials for inclusion, and that embedding the curriculum in a narrative game‐based learning environment, besides other potential benefits, provides educational outcomes that measured with standard procedures are at the same level or better than traditionally taught classes.. The learning environment constituting the LARP draws on different domains: a) the overarching narrative framework which anchors the curriculum with the pedagogical content b) the Live Role‐game Play and c) the way it integrates the creative processes. The first phase of the project was concluded with a final survey focusing on the learners’ preference in relation to the types of games and degree of narrative they had encountered in the scenarios that had framed their learning experiences throughout the year. As part of the survey conducted at the end of the year, the learners were asked to rate their preference for different game elements and genres. All the learners completed the survey (N 84). The learners were asked their individual preferences on a scale from 1 to 6, where 6 was the highest score, for the following genres and game elements:

Live Action Role Play

Table Role Play

Board/ figure/card Role Play

Narrative in the Role Play

Strategy Games

Competition and points

Narrative in the Role Play, Live Action Role Play and Strategy games were rated highly by most learners, with the genres Table Role Play and Board/ figure/card Role Play ranging lower. These results were triangulated with interviews with the students in which some of the informants chosen for case studies said that they were more motivated towards exploring the interactive story and developing the narrative, while others were more attracted to the excitement of strategy games and winning. The aspect of competition and points was on average the lowest rated element in the final survey, while the aspect relating to narrative was the highest rated. The learners who rated the competition aspect lowest were also those who rated the narrative highest in their personal preferences. The students were also asked to choose five of the scenarios that had meant most to them from all the scenarios they had played, and to explain their reasons for their choices. The preferred scenarios were the

195

Lisa Gjedde ones that had a strong narrative and provided interesting role‐game play, while many of the learners also pointed to scenarios that were based on a strategy game approach. These also had a narrative frame, which provided the learners with a sense of meaning to what they were learning.

7. Modalities of LARP’s and designs for learning From the observations of the scenarios, as well as interviews with the teachers and the learners, it was apparent that there were two main modalities of the games that were designed and played: ones that were primarily driven by a strong narrative with a focus on immersive role‐game play and developing the story, and ones there were primarily driven by the game‐system with a strong focus on strategy. However, games in education often have competitive aspects. The development of the emergent interactive story is strongly connected to the learners’ construction of meaning, and is a collaborative effort. Building and sustaining a narrative is the creation of a possible world, a joint imaginative venture. However, having a preference for narrative in the game did not conflict with also liking strategy games. Some learners prefer the narrative construction and the intrinsic quality of personal meaning making, and for them, it is very off‐putting to take part in a game in which the main driver is the gaining of points. For other learners the focus and drive is motivated by the experience of competition and winning more than by the collective creation of the epic story. The learners’ definition of a good learning game is one which converges the game‐play and the curriculum in a natural way, such that the content knowledge and concepts being learned advance the game and drive the narrative forward naturally. For example, taking part in solving a murder mystery calls for forensic knowledge which can be learned through natural science. To win a battle at sea calls for mathematical knowledge as well as imagination and creativity, which can then by applied etc. The variety of scenarios leads to a sense of novelty every week, a fresh beginning which can also be of importance to learners who otherwise through their schooling might have been restricted to limiting roles, not least the ones who have been within the realm of special needs education. Since the framework for demonstrating LARP as a vehicle for the curriculum is developed in a unique context of a residential school with especially interested teachers and students, it might however present conditions that are not directly transferable or applicable to the mainstream municipal schools. This raises issues of the potential limitations and bias of the study, in relation to its transfer to an ordinary school setting. The school which was studied is the only school of its kind in the world. It provided an opportunity for a naturalistic study of interactive narrative game‐based learning that could otherwise often only be studied in schools through planned interventions on a much smaller scale, and with a limited scope subject wise and time wise. A majority of the students in this school have a strong interest in various types of games and the teachers share this interest, taking an active part in recreational gaming with the students outside the lessons. Since it is a residential school these activities are happening in the spare time, building strong relations in between the students and between students and teachers. This can provide a bias in relation to the direct applicability of the methods and results from this school to the municipal schools, since the students and teachers there may not have the same interest in games, nor have the same sense of community. They may neither be especially inclined towards gaming nor find role‐playing a natural form of expression, and this may influence the degree of creative and collaborative learning experiences they may have or be able to provide if they were to try and implement the approach. However, there is a need for a st rethinking the entire school culture in order to provide 21 century learning and develop more inclusive and creative ways of learning. In its analogue and easily adaptable form, LARP may hold rewarding experiences for the learners and teachers alike who venture into the story‐based anchoring of the games and curriculum in the next phases of the project.

8. Conclusions and future research LARP has affordances in relation to anchoring the curriculum in a narrative interactive framework that draws on different game engines and genres. The students at Østerskov Efterskole are familiar with the genres and

196

Lisa Gjedde have different preferences, either they prefer the embodied learning through playing out the roles in a physical and emotional narrative environment or prefer gaming through using the artifacts of board games and a strategic game engine. The overarching concept of the game‐based narrative learning environment provides the context for the creative processes, which are expressed through interactions at the level of the LARP, and drawing on the relevant domains and curriculum. Furthermore, it has been identified that there are differences in the learners’ preferences of the game‐genres and elements being part of the narrative game‐based learning concept that has been studied. Most notably there are differences in relation to whether a learner is primarily drawn to narrative interaction and experience or a strategy or competition aspect of the game. These differences should be accommodated in the design of learning games in order to ensure that the learning environment supports the intrinsic motivation, which is vital for the creative learning potentials and for using LARP as a vehicle for emergent creativity and creative expression. In comparison to digital games, an analogue approach to the games provides for a large degree of adaptability and potential for differentiation, in relation to the interest of the students. Phase two and three in this project investigates these potentials through narrative game‐based interventions in municipal schools based on models of edu‐LARP. The outcomes of this research is to be reported.

Acknowledgements The research was supported by a grant from the Egmont Foundation, Denmark. The research project has been carried out in collaboration with the headmaster, the teachers and learners at Østerskov Efterskole, who are greatly thanked for their contribution of time and experience to the project.

References Amabile,T.M (1996) Creativity in Context (1996, p.113). Boulder, CO: Westview Press Amabile,T.M (2012) A Componential Theory of Creativity. Harvard School of Business, working papers. Barab, S. A. & Plucker, J. A. (2002). Smart people or smart contexts? Cognition, ability, and talent development in an age of situated approaches to knowing and learning. Educational Psychologist 37 (3), 165–182 Bellance, J, Brandt. R. (eds) (2010) 21st Century Skills: Rethinking How Students Learn .Leading Edge Boden, M. A. (2004). The Creative Mind: Myths and Mechanisms(London: Routledge). 2ndedn., revised/expanded De Bono, Eduardo (1992): Serious Creativity: Using the Power of Lateral Thinking to Create New Ideas.N.Y. Dede Chris et al (2009) Immersive Interfaces for Engagement and Learning in Science 323. Cambridge MA. Gjedde, L & Ingemann, B (2008). Researching Experiences. Cambridge Scholars Publishing. Cambridge. Gjedde, L: Inclusive curriculum design through narrative and imaginative interactive learning environments. In Engaging Imaginations and Developing Creativity in Education (2010) Cambridge Scholars Publishing Harding, T, (2007) Immersion revisited: role‐playing as interpretation and narrative in Lifelike, Eds.Jesper Donnis, Morten Gade & Line Thorup. Knudepunkt. Copenhagen. Huizinga, J. (1944/1980) Homo Ludens: A study of the play‐element in culture. London: Boston and Henley. Hyltoft, Malik (2008): The Role‐players’ School: Østerskov Efterskole.In Montola, M & Stenros, .J, Playground Worlds, Solmukohta Johnson, M. (2008) The Meaning of the Body. Chicago Kress, G.R. and Van Leeuwen,T. (2002). Multimodal Discourse: the modes and media of contemporary communication. London: Edward Arnold Lave, J. (1991) Situating learning in communities of practice. In L. B. Resnick, J. M. Levine, & S. D. Teasley (Eds.) Perspectives on socially shared cognition, (pp. 63–82). Washington, DC. American Psychological Association Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. New York, Cambridge University Press Montola, M. & Stenros, J.eds.( 2004) Beyond Role and Play : Tools, toys and theory for harnessing the imagination . Finland Nordahl, Thomas et al. ( 2010): Uligheder og variationer ‐ Danske elevers motivation,skolefaglige læringsudbytte og sociale kompetencer. Rapport til skolens rejsehold.Hamar/Aalborg Spires, H., Turner, K. & Lester, J. (2008). Twenty‐First Century Skills and Game‐Based Learning. In J. Luca & E. Weippl (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2008 (pp. 5438‐ 5443). Chesapeake, VA: AACE. Vygotsky, L.S. (2004). Imagination and creativity in childhood. Journal of Russian and East European Psychology, 42 (1), 7‐97. Wenger, E. (1998).Communities of practice: Learning, meaning, and identity, Cambridge, UK. Cambridge University Press.

197

Development and Evaluation of a Generic E‐CLIL Web2.0 Games Engine Thomas Hainey and Thomas Connolly University of the West of Scotland, UK thomas.hainey@uws.ac.uk thomas.connolly@uws.ac.uk Abstract: Games‐based learning is perceived by educationalists as a potentially highly motivational approach for learning and teaching at a supplementary level and is starting to be used more frequently at Primary Education (PE) and Secondary Education (SE) level. Content and Language Integrated Learning (CLIL) has also increased in popularity. CLIL is a bilingual educational approach where an additional language is used for the learning and teaching of both the language and the content and each are interwoven. This paper will discuss the development of a generic electronic Content and Language Integrated Learning (e‐CLIL) Web2.0 games engine developed as part of the EU Comenius e‐CLIL project by the University of the West of Scotland (UWS). The Web2.0 games engine directly answers the need for expanding, very quickly, the amount of content available (in any language) for teachers of CLIL. This paper will present an empirical evaluation of the piloting of the Web2.0 games engine. This paper will present both a student quantitative pilot evaluation consisting of 82 participants and a teacher quantitative pilot evaluation consisting of 18 participants of the e‐CLIL Web2.0 games engine. Keywords: empirical evidence, Web2.0 games engine, CLIL, e‐CLIL, electronic‐content language integrated learning, evaluation, student, teacher, quantitative, pilot

1. Introduction Games‐based learning has been discovered to be a potentially motivating supplementary teaching approach at Primary and Secondary Education Level in a variety of different subject areas including: learning of simple machines (Annetta et al., 2009), fire safety (Chuang and Chen, 2009), collaborative problem solving (Harris, 2008), nutrition (Hung et al., 2009), Maths (Ke, 2008), bullying prevention (Ruben‐Vaughan et al., 2011), natural science and ecology (Wrzesien and Raya, 2010) and reasoning (Wang et al., 2010). The number of academic papers producing and evaluating games in the academic literature seems to be increasing. Effective teaching in MFLs can make a significant contribution to young people’s ability to value diversity and challenge racism by providing opportunities for them to (QCA, 2007):

discover that many different languages are spoken throughout the world, and that many languages are spoken in a number of different countries and by people from different ethnic backgrounds;

recognise that understanding another language promotes a deeper appreciation of speakers of that language and of their culture;

learn that the ability to communicate with speakers of other languages can nurture mutual respect, tolerance and understanding;

appreciate that speakers of different languages may have beliefs, attitudes, behaviours and experiences that are of equal worth;

Crookall (2007) notes that language teachers make great use of simulation/gaming methodologies and there are many supporting textbooks and research papers that present various forms of role‐play, games, simulations, and other exercises (e.g. Garcia‐Carbonell, Rising, Montero and Watts, 2001; Gaudart, 1999; Halleck, 2007). While many of the simulations/games used are non‐computer based, during recent years the computer game has become an important development in popular culture. There are a number of examples of computer games used for the purposes of language learning in the literature, for example, EverQuest II (Rankin Gold and Gooch, 2006), the Tactical Language and Cultural Training System (Johnson and Wu, 2008) and Second Life (Rymaszewski et al., 2007). Connolly, Stansfield and Hainey (2011) evaluated a multi‐lingual Alternate Reality Game (ARG) which was piloted in 2009 with 328 14‐ 16 year old school students and 95 language teachers spread across 17 European countries. In general, student attitudes towards the ARG were very positive with evidence suggesting that the ARG managed to deliver the motivational experience expected by the students. The majority of students either agreed or strongly agreed that they would be willing to play the game over a prolonged period of time as part of a foreign language

198

Thomas Hainey and Thomas Connolly course. In addition, through using the ARG, students believed that they obtained skills relating to cooperation, collaboration and teamwork.

2. Previous work To understand what approaches had been taken to the use of games in primary schools and identify empirical evidence, a literature search was undertaken using a number of electronic databases: ACM, Science Direct, IEEE (Institute of Electrical and Electronics Engineers) Computer Society Digital Library (CSDL), Taylor and Francis, Proquest (including Applied Social Sciences Index and Abstracts, Proquest Computing and ProQuest Dissertations and Theses A and I), EBSCO (including CINAHL PsycINFO, SocINDEX, Library, Information Science and Technology Abstracts), Canbridge Journals, ERIC, Ingenta, Infotrac and Web of Knowledge. The following search terms derived from a previous search on the evaluation of computer games were used (Connolly, Stansfield and Hainey, 2007): ("computer games" OR "video games" OR "serious games" OR "simulation games" OR "games‐ based learning" OR "MMOG" OR "MMORPG" OR "M.U.D." OR "online games") AND (evaluation OR impacts OR outcomes OR effects OR learning OR education OR skills OR behaviour OR attitude OR engagement OR motivation OR affect) With a starting point of the year 2000, the search collated 18,298 entries in total. The returned papers were then restricted to the use of GBL within education and working with children. A total of 375 papers were returned. These were then narrowed down to the relevant papers that looked at work undertaken in the primary classroom and the methods used by the researchers, resulting in 112 papers. Out of these, 24 papers were used for languages and literacy and are summarised in Table 1. Table 1: Summary of the 24 papers on games associated with language learning Authors Akcaoglun (2011) Bottino, Ott and Vincenza (2009) Cobb and Horst (2011) Din and Calao (2001) Fontana and Beckerman (2004) Garzotto (2007) Lacasa, Martínez and Méndez (2008) Lacasa, Méndez and Martínez (2008) Lim (2008) Lu, Fan, Liu and Chuang (2010) Marques and de Souza (2012) Meyer (2009) Owston et al., (2009) Prassos, Sachtouris and Karakiza (2012) Robertson (2012) Robertson and Howells (2008) Rosas et al., (2003) Rowe, Naglieri and Conway (n.d.) Ruphina and Liu (2011) Saridaki et al., (2008) Segers and Verhoeven (2005) Suh, Kim, and Kim (2010) Vos, van der Meijden and Denessen (2011) Yang, Chen and Jeng (2010)

Research Method Quantitative Mixed

Age Range Mean = 12.58 years 8 – 10 years

Result Neutral Positive

Mixed Quantitative Quantitative

11 – 12 years 5 – 6 years 7 – 8 years

Positive Neutral Positive

Mixed Qualitative Mixed

7 – 8 years 8 – 9 years 8 – 9 years

Positive Positive Neutral

Mixed Mixed Quantitative Qualitative

10 – 11 years 8 – 9 years 7 – 9 years 10 – 11 years

Positive Positive Positive Positive

Quantitative Qualitative Mixed

9 – 10 years 8 – 12 years 11 – 12 years

Positive Positive Positive

Qualitative Mixed Quantitative

9 – 10 years 6 – 7 years 6 – 8 years

Positive Positive Neutral

Qualitative Mixed Quantitative

8 – 11 years 6 – 8 years Mean Age = 5.5 years

Positive Positive Positive

Quantitative Quantitative Mixed

11 – 13 years 10 – 12 years 7 – 8 years

Positive Positive Positive

3. e‐CLIL Web2.0 games engine description The e‐CLIL Web2.0games engine was developed as part of the E‐CLIL (e‐clil.uws.ac.uk) EU‐funded project to develop and build a resource centre for the use of Content Language Integrated Learning (CLIL). The project focused on language learning, learning strategies, multilingualism and multiculturalism. CLIL is dual‐focused instructional approach during which language is taught at the same time as content from a different school

199

Thomas Hainey and Thomas Connolly subject. It has already been established as a valuable approach to both teaching foreign languages and specific subjects. As part of this project, a games engine was developed that allows CLIL teachers to develop their own multi‐lingual content that can be delivered over the web. To make the content more engaging, the engine provides a scoring system based on how the students perform in their assessments and, as the students’ scores increase, they are given access to an expanding range of short web‐based games. The platform also maintains a range of high scores (overall score, per game, per subject) and students can compete against other students to gain the highest score. To assess learning, the engine currently supports the following question types: multiple‐choice, multiple responses, drag‐and‐drop, point‐and‐click, sorting and a range of maths‐ related games. Figure 1 shows an example of a multiple‐choice game and Figure 2 shows an example of a doodle game created by the games engine.

Figure 1: Example of Multiple‐choice game created by teacher using E‐CLIL engine

Figure 2: Example of Doodle God game created by teacher using E‐CLIL engine

4. Methodology Between April 2012 and July 2012, a number of CLIL teachers were invited to use the platform. The teachers who volunteered were using CLIL to teach English and German to French students and English to German students. Online forms were added to the platform to gather feedback from the students and the teachers. In addition, a page was added to the platform to allow students to rate each game on a score of 1 to 5.

5. e‐CLIL Web2.0 games engine evaluation results 5.1 Student feedback 82 participants evaluated the e‐CLIL Web2.0 Games Engine. 47 participants (57%) were male and 35 participants (43%) were female. The mean age of participants was 9.38 years (SD = 1.37) with a range of 6 to 12 years. A Mann‐Whitney U test indicated that there was no significant difference between male and female in relation to age (Z = ‐0.658, p < 0.511). Participants were asked to rate the games platform overall on a Likert scale ranging from 1 to 5, 1 being the highest and 5 being the lowest. The results were very positive where 78 participants (95%) gave a rating of one and the rest of the participants (4, 5%) gave a rating of 2. A Mann‐ Whitney U test indicated that there were no significant differences between male and female in terms of the ratings of the overall platform (Z = ‐1.332, p < 0.183). This means that the positive feedback on the overall platform was consistent regardless of gender. Participants were asked to rate the various game types

200

Thomas Hainey and Thomas Connolly available on the platform on a Likert scale from 1 to 5 (1 being the highest and 5 being the lowest). Table 2 shows the ratings for the game types by participants. Table 2: Ratings of game types available on the platform Game Type Multiple choice Sorting Image click Maths

Rank 1st 2nd 3rd 4th

Mean 1.01 1.10 1.18 1.66

SD 0.11 0.30 0.39 0.72

The results indicated that while all of the game types scored positively, the favourite type of game among participants was the multiple choice game while the least favourite was the maths games. Table 3 shows the ratings of the types of games split by gender. Table 3: Ratings of the game types available on the platform split by gender Gender Game Type Multiple choice Image click Sorting Maths

Rank 1st 2nd 3rd 4th

Male Mean 1.00 1.04 1.06 1.53

SD 0.00 0.20 0.25 0.62

Rank 1st 3rd 2nd 4th

Female Mean 1.03 1.37 1.14 1.83

SD 0.17 0.49 0.36 0.82

The results were consistent in the sense that both males and females rated the Multiple Choice game type as the highest and the Maths game type as the lowest. Mann‐Whitney U tests indicated that females rated the Image Click game type significantly higher than males (Z = ‐3.787, p < 0.000), however there were no significant differences between males and females in relation to the following game types: Maths (Z = ‐1.583, p < 0.113), Sorting (Z = ‐1.186, p < 0.236) and Multiple Choice (Z = ‐1.159, p < 0.247). Participants were asked to rate the following aspects of the games engine tool on a 5 point Likert scale (1 being the highest and 5 being the lowest): ease of use, graphics, mini‐games and high score system. The ratings for the aspects of the Web2.0 games engine were generally very positive. Table 4 shows the rankings of the aspects of the games engine tool. Table 4: Ratings of the aspects of the tool Game Type Ease of use High score system Mini‐games Graphics

Rank 1st 2nd 3rd 4th

Mean 1.01 1.02 1.07 1.27

SD 0.11 0.16 0.26 0.50

The aspects of the Web2.0 games engine tool that were rated most highly by participants were: ease of use and the high score system, while the aspects of the games engine with the lowest ratings were graphics and the mini‐games. Table 5 shows the ratings of the aspects of the games engine split by gender. Table 5: Ratings of aspects of the tool split by gender Gender Attribute Ease of use High score system Mini‐games Graphics

Rank 1st 1st 2nd 3rd

Male Mean 1.00 1.00 1.04 1.21

SD 0.00 0.00 0.20 0.41

Rank 1st 2nd 3rd 4th

Female Mean 1.03 1.06 1.11 1.34

SD 0.17 0.24 0.32 0.59

Mann‐Whitney U tests indicated that there were no significant difference between males and females in relation to the following aspects of the tool: ease of use (Z = ‐1.159, p < 0.247), graphics (Z = ‐0.879, p < 0.379), mini‐games (Z = ‐1.226, p < 0.220) and high score system (Z = ‐1.649, p < 0.099). All of the participants stated that they would continue to play the games provided by the Web2.0 games engine tool. In terms of age suitability, a Mann‐Whitney U test indicated that children under the age of 9 rated the overall platform significantly higher than children over the age of 9 (Z = ‐2.049, p < 0.041). A Mann‐Whitney U test

201

Thomas Hainey and Thomas Connolly also indicated that children under the age of nine rated the Math game type significantly higher than children above the age of nine (Z = ‐2.894, p < 0.004). Mann‐Whitney U tests indicated that children under the age of nine gave significantly higher ratings for the following aspects: graphics (Z = ‐3.550, p < 0.000), mini‐games (Z = ‐3.005, p < 0.003) and high score system (Z = ‐2.212, p < 0.027). Interestingly there was no significant difference in ease of use ratings (Z = ‐1.555, p < 0.120) in relation the whether participants were under or above the age of nine.

5.2 Game ratings The Web2.0 games engine tool had embedded functionality that allowed students to rate the games that they had played. The embedded functionality allowed the students to rate the games by assigning them a maximum total of 5 stars. 1,349 game ratings were collected over a period of a week. The overall ratings of the game types are shown in Table 6. Table 6: Ratings of game types Game Type Sorting Multiple choice Multiple sorting Image click Multiple maths Addition Division Multiplication Subtraction

Rank 1st 2nd 3rd 4th 5th 6th 7th 7th 8th

Mean 4.88 4.84 4.83 4.71 3.95 3.81 3.55 3.55 3.50

SD 0.32 0.42 0.38 0.56 0.78 0.92 0.63 0.63 0.60

In terms of playing and rating the games, Table 7 shows the frequencies, ranking and percentage of the types of games played and rated by participating students. The result indicates that the Multiple Choice game type was by far the most popular game type in terms of being played and rated. Table 7: Ranking frequencies and percentage of the types of games played Game Type Multiple choice Multiple maths Image click Sorting rating Multiple sorting Division Multiplication Addition Subtraction

Rank 1st 2nd 3rd 4th 5th 6th 7th 8th 9th

N 790 189 136 93 35 29 29 27 22

Percentage 58% 14% 10% 7% 3% 2% 2% 2% 2%

Participants were asked to rate the overall game types. The ratings are shown in Table 8. Table 8: Ranking of overall game type Game Type Sorting Multiple choice Image click Maths

Rank 1st 2nd 3rd 4th

Mean 4.87 4.84 4.71 3.83

SD 0.34 0.42 0.56 0.77

The teachers created a total of 32 games for the students to play in English and German. A total of 1,349 ratings were collected and analysed. Table 9 shows the mean ratings of some of the individual games. The results indicate that the Web2.0 games engine is capable of producing a diverse, infinite number of games suitable for all areas of the curriculum in any language. The games produced by teachers covered a diverse, wide area of the curriculum including: country mass, planets, dinosaurs, whales and sharks, recycling, animals, plants and general knowledge. Multiple choice was by far the most popular game type and the results indicate

202

Thomas Hainey and Thomas Connolly that teachers were able to produce games with varying levels of difficulty in the same subject areas in both English and German. The highest rating games that had the maximum mean rating were all games that were created in conjunction with the students that concerned topics that they had suggested such as recycling, country mass, general knowledge and the Doodle God game. The results for the ratings of all of the games were very positive. Table 9: Ratings of games Game Name

Language

Game Type

Mean Rating/SD

Number Bonds Addition (0‐10) Nummer Bonds Addition (0‐10) Number Bonds Division (0‐20) Nummer Bonds Abteilung (0‐20) Materials 1 Dinosaur Identification Whale and shark identification Doodle God Animal General Knowledge

English

Addition

3.40 (SD = 0.83)

N of ratings 15

German

Addition

4.33 (SD = 0.78)

English

Division

3.71 (SD = 0.76)

German

Division

3.50 (SD = 0.60)

English English English

Image Click Image Click Image Click

4.62 (SD = 0.56) 4.22 (SD = 0.73) 4.22 (SD = 0.73)

29 18 18

English English

Image Click Multiple Choice

5.00 (SD = 0.00) 4.28 (SD = 0.75)

71 18

English

Multiple Choice

4.12 (SD = 0.75)

English English English German German German English

Multiple Choice Multiple Choice Multiple Choice Multiple Choice Multiple Choice Multiple Choice Multiple Choice

4.13 (SD = 0.72) 5.00 (SD = 0.00) 5.00 (SD = 0.00) 4.99 (SD = 0.11) 4.95 (SD = 0.27) 4.93 (SD = 0.26) 4.87 (SD = 0.33)

16 32 32 79 79 81 81

Guess the animal family Guess the plant type General knowledge Recycling Sequenzen (Hart) Sequenzen (Leicht) Tiere Animals

5.3 Teacher feedback 18 teachers evaluated the e‐CLIL Web2.0 games engine. 11 participants (61%) were female and 5 participants (39%) were male. The teachers were asked to rate the general usefulness of the tool on a Likert scale from 1 to 5 (1 being the highest and 5 being the lowest). The results were generally positive and the participants gave a mean rating of 1.06 (SD = 0.24). Participants were asked to rate the different games that could be created by the tool. The teachers rated the Image Click and Multiple Sorting games as the highest and the Multiple Maths game as the lowest. The ratings of the game types are shown in Table 10. Table 10: Ratings of the types of games by teachers Game Type Image click Multiple sorting Multiple choice Sorting Multiple maths

Ranking 1st 1st 2nd 2nd 3rd

Mean 1.17 1.17 1.22 1.22 1.28

SD 0.38 0.51 0.55 0.55 0.57

Table 11 shows the ratings of the game types split by gender. Participants were asked to rate the tools attributes. The ease of use and the multilingual capability were rated as highest while the graphics was rated as lowest. Table 12 shows the ratings of the attributes of the games engine. Table 13 shows the ratings of the attributes of the games engine split by gender.

203

Thomas Hainey and Thomas Connolly Table 11: Ratings of game types split by gender Gender Game Type Multiple choice Multiple maths Image click Sorting Multiple sorting Maths

Rank 1st 2nd 2nd 2nd nd 2 3rd

Male Mean 1.00 1.14 1.14 1.14 1.14 2.29

SD 0.00 0.38 0.38 0.38 0.38 0.49

Rank 3rd 3rd 1st 2nd 1st 4th

Female Mean 1.36 1.36 1.18 1.27 1.18 2.36

SD 0.67 0.67 0.40 0.65 0.60 0.50

Table 12: Ratings of the attributes of the Web2.0 games engine by teachers Attribute Ease of use Multilingual capability Graphics

Rank 1st 2nd 3rd

Mean 1.00 1.06 1.39

SD 0.00 0.24 0.50

Table 13: Ratings of the attributes of the Web2.0 games engine by teachers split by gender Gender Attribute Ease of use Multilingual capability Graphics

Rank 1st 1st 2nd

Male Mean 1.00 1.00 1.14

SD 0.00 0.00 0.38

Rank 1st 2nd 3rd

Female Mean 1.00 1.09 1.55

SD 0.00 0.30 0.52

All participants stated that they would continue to use the tool for CLIL teaching. Participants were asked what other games they would like to see available on the platform and gave some of the following suggestions: fill in the gap questions, crossword questions and simple sudoko. The majority of the participants were happy with the tool. Participants suggested the following changes to the tool: more stock images, animations and animated graphics. Participants were asked to list five attributes that they liked about the tool. Some of the most popular answers were: ease of use, multilingual capability, applicable to various areas of the curriculum.

6. Discussion In terms of rating the Web2.0 games engine platform the results were very positive where 95% of participants gave a rating of one and the rest gave a rating of two. There were no significant differences between male and female in terms of the ratings of the overall platform suggesting that the positive feedback on the overall platform was consistent regardless of gender. This suggests that Web2.0 can be used as a platform for games regardless of the fact that the primary audience maybe male or female or a collection of both. Ratings of the games types were also positive with the Multiple Choice game being the most popular and the Math games being the least popular. This result was consistent for both males and females. Females rated the Image Click games significantly higher than males however there was no significant different in ratings of games types between the following types: Maths, Sorting and Multiple Choice. With regards to rating the aspects of the tool, the following aspects were rated most highly: ease of use and high score system while the following aspects were rated as the lowest: mini‐games and graphics. There were no significant differences between male and female with regards to tool aspects. Children under the age of 9 rated the overall platform significantly higher. They also rated the Maths game and the following aspects of the tool significantly higher: graphics, mini‐games and high score system. There was no significant difference in ease of use ratings in relation to whether participants were under or above the age of nine. The embedded functionality of the Web2.0 games engine allowed the students to rate the games by assigning them a maximum total of 5 stars. 1,349 game ratings were collected and the results show that the Multiple Choice game type was by far the most popular game type in terms of being played and rated. This is possibly due to the students being immediately familiar with the concept of the Multiple Choice game and also possibly because the Multiple Choice game can easily be adapted by teachers for a vast variety of curriculum areas in different languages.

204

Thomas Hainey and Thomas Connolly The teachers created a total of 32 games for the students to play in English and German. The results indicate that the games engine is capable of producing a diverse range of games suitable for all areas of the curriculum in any language. Some examples of games produced were for country mass, planets, dinosaurs, whales and sharks. The results for the ratings of all of the games were very positive. 18 teachers evaluated the e‐CLIL Web2.0 games engine. The results obtained in relation to general usefulness were generally positive. The teachers rated the Image Click and Multiple Sorting games as the highest and the Multiple Maths game as the lowest. The Image Click game was most likely the most popular to the teachers as it is most easily adaptable to any area of the curriculum. Teachers can select any area such as nature and wildlife, snakes, sharks, dinosaurs, trees, geographical landmarks, plants, planets or musical notation and suitably adapt this to their curriculum. The Maths games however are somewhat limiting in terms of content adaptability as it is such a focused subject. Overall, there were no significant differences between male and female in relation to game types. With regards to the attributes of the tool, the ease of use and the multilingual capability were rated as highest while the graphics were rated as lowest. This suggests that the graphics on the platform could be improved however a Web2.0 3D games engine would be incredibly difficult to implement and may not make a justifiable difference in terms of learning effectiveness, Overall there were no significant differences between male and female in relation to tool attributes. All participants stated that they would continue to use the tool for CLIL teaching and would like to see the following game types become available on the platform: fill in the gap questions, crossword questions and simple sudoko. The majority of the teachers were happy with the tool and the following changes were suggested: more stock images, animations and animated graphics. The teachers listed the most positive attributes of the tool as: ease of use, multilingual capability, applicable to various areas of the curriculum.

7. Conclusion This study has presented a systematic literature search and has presented 24 empirical papers found in this search associated with the use of computer games in language learning. An empirical pilot evaluation of an e‐ CLIL Web2.0 games engine for language learning was also presented from a teacher and student perspective. The results have proven to be generally positive from both a teacher and student perspective, however the evaluation has raised a number of interesting potential improvement areas such as: the ability for a teacher to monitor the progress of a particular student and for the Web2.0 games engine to test the particular language competency of a particular student and adjust the game difficulties accordingly. Future research will involve improvement and further empirical evaluation of the e‐CLIL Web2.0 games engine entailing making the games engine available to teachers and students to come up with their own games to support their curriculum. We currently have 400 teachers registered on the e‐CLIL Web2.0 games engine who are creating games. This data will be analysed and further empirical publications will be generated.

Acknowledgements This work has been co‐funded by the EU under the FP7, in the Games and Learning Alliance (GaLA) Network of Excellence, Grant Agreement nr. 258169. This work has been co‐funded by the EU Lifelong Learning Programme under contract 519057‐LLP‐1‐2011‐1‐ UK‐KA3‐KA3NW (Ed2.0Work – European Network for the integration of Web2.0 in education and work).

References Akcaoglu, M. (2011). Using games in classroom: All tutors and no stories make a virtual world a dull game. In M. Koehler & P. Mishra (Eds.), In Proceedings of Society for Information Technology & Teacher Education International Conference 2011 (pp. 64‐69). Chesapeake, VA: AACE. Annetta, L., Mangrum, J., Holmes, S., Collazo, K. and Cheng, M.‐T. (2009). Bridging Realty to Virtual Reality: Investigating gender effect and student engagement on learning through video game play in an elementary school classroom. International Journal of Science Education, 31, 1091‐1113. Bottino, R. M., Ott, M. and Beningo, V. (2009). Digital Mind Games: Experience‐Based Reflections on Design and Interface rd Features Supporting the Development of Reasoning Skills. In Proceedings of the 3 European Conference on Games‐ Based Learning (EC‐GBL), Graz, Austria. Chuang, T.‐Y. and Chen, W.‐F. (2009). Effect of Computer‐Based Video Games on Children: An Experimental Study. Educational Technology & Society, 12, 1‐10. Cobb, T., Horst, M. (2011). Does Word Coach Coach Words? CALICO Journal, 28(3), p‐p 639‐661.

205

Thomas Hainey and Thomas Connolly Connolly, T.M., Stansfield, M.H. and Hainey, T. (2007). An Application of Games‐based Learning within Software Engineering, British Journal of Educational Technology, Volume 38, Number 3, pp. 416‐428. Connolly, T. M., Stansfield, M. and Hainey, T. (2011). An alternate reality game for language learning; ARGuing for multilingual motivation. Computers and Education, 57(1), 1389 – 1415. Crookall, D. (2007). Second language acquisition and simulation. Simulation & Gaming, 38(1), 6–8. Din, F. S. and Calao, J. (2001). The Effects of Playing Educational Video Games on Kindergarten Achievement. Child Study Journal, 31(2), 95 –102. Fontana, L. and Beckerman, A. (2004). Childhood Violence Prevention Education Using Video Games. Information Technology in Childhood Education Annual, 49–62. Garcia‐Carbonell, A., Rising, B., Montero, B. and Watts, F. (2001). Simulation/gaming and the acquisition of communicative competence in another language. Simulation and Gaming, 32(4), 481–491. Garzotto, F. (2007). Investigating the Educational Effectiveness of Multiplayer Online Games for Children. In Proceedings of the IDC 2007: Games, June 6‐8, 2007, Aalborg, Denmark. Gaudart, H. (1999). Games as teaching tools for teaching English to speakers of other languages. Simulation and Gaming, 30(3), 283–291. Halleck, G. B. (2007). Second language acqusition and simulation. Simulation & Gaming, 38(1), 31–34. Harris, D. (2008). A comparative study of the effect of collaborative problem‐solving in a Massively Multiplayer Online Game (MMOG) on individual achievement. Doctor of Education, The University of San Francisco. Hung, C.‐M., Chiu, C.‐H., Chen, Y.‐T., SU, M.‐J. and Chen, H.‐S. (2009). Effectiveness of game‐based learning of a national health e‐learning network for nutrition education in elementary school. Proceedings of the 11th international conference on e‐Health networking, applications and services. Sydney, Australia: IEEE Press. Johnson,W. L. and Wu, S. (2008). Assessing aptitude for learning with a serious game for foreign language and culture. In: Proceedings of the ninth international conference on intelligent tutoring systems. Montreal. Ke, F. (2008). Computer games application within alternative classroom goal structures: cognitive, metacognitive, and affective evaluation. Educational Technology Research and Development, 56, 539‐556. Lacasa, P., Martinez, R. and Mendez, L. (2008). Developing New Literacies Using Commercial Videogames as Educational Tools. Linguistics and Education: An International Research Journal, 19(2), 85‐106. Lacase, P., Méndez, L. and Martinez, R. (2008). Bringing Commercial Games into the Classroom. Computers and Composition, 25(3), 341 – 356. Lim, C. P. (2008). Global citizenship education, school curriculum and games: Learning Mathematics, English and Science as a global citizen. Computers and Education, 51(3), 1073 – 1093. Lu, S. J., Fan, P. L., Liu, Y. C. and Chaung, Y.C. (2010). Design and Evaluation of Physical Interactive Games for Taiwanese Local Dialect in Elementary School Teaching, In Proceedings of 2nd International Conference on Education Technology and Computer (ICETC). Marques, L. B. and de Souza, D. G. (2012). Behavioural Evaluation of Preference for Games‐Based Learning Procedures. In th Proceedings of the 6 European Conference on Games‐Based Learning (EC‐GBL), Cork, Ireland. Mayer, B. (2009). Learning English through Serious Games – Reflections on Teacher and Learner Performance. In Proceedings of Learning by Playing. Game‐based Education System Design and Development Lecture Notes in Computer Science. 5670, p 293. Owston, R., Wideman, H., Ronda, N. S. and Brown, C. (2009). Computer Games Development as a Literacy Activity. Computers & Education, 53(3), 977‐989. Prassos, N., Sachtouris, S., Karakiza, T. (2012). Teaching Information and Communication Technology with Digital Games. In th Proceedings of the 5 European Conference on Games‐Based Learning (EC‐GBL), Athens, Greece. QCA, 2007. Qualifications and Curriculum Authority Respect for All: Modern foreign languages Available at: http://www.qca.org.uk/qca_7984.aspx Rymaszewski, M., Au, W. J., Wallace, M., Winters, C., Ondrejka, C. and Batsone‐Cunningham, B. (2007). Second life: The official guide. New Jersey: John Wiley & Sons. Rankin, Y., Gold, R. and Gooch B. (2006) Gaming as a language learning tool. In: Proceedings of the ACM SIGGRAPH educators program, 2006. Robertson, J. (2012). Making games in the classroom: Benefits and gender concerns. Computers & Education, 59(2). 385‐ 398. Robertson, J. and Howells, C. (2008). Computer game design: Opportunities for successful learning. Computers & Education, 50(1), 559 – 578. Rosas, R., Nussbaum, M., Cumsille, P., Marianov, V., Correa, M., Flores, P., Grau,V., Lagos, F., Lopez, X., Lopez, V., Rodriguez, P. and Salinas, M. (2003). Beyond Nintendo: design and assessment of educational video games for first and second grade students, Computers & Education, 40, 71–94. Rowe, E., Naglieri, J. A. and Conway, C. (n.d.). Evaluation of an Online Literacy Program: A Replication and Extension Study. Retrieved 10 May 2013 from http://www.skatekidsonline.com/parents_teachers/Naglieri_Evaluation_of_Online_Literacy_Program_UpDated.pdf Rubin‐Vaughan, A., Pepler, D., Brown, S., and Craig, W. (2011). Quest for the Golden Rule: An effective social skills promotion and bullying prevention program. Computers & Education, 56(1), 166–175.

206

Thomas Hainey and Thomas Connolly Ruphina, A. and Liu, M. (2011). Digital Games: Potential Integration of Mingoville Games in Learning English as a Foreign Language. In M. Koehler and P. Mishra (Eds.), Proceedings of Society for Information Technology and Teacher Education International Conference 2011 (pp. 2216‐2222). Chesapeake, VA: AACE. Saridaki, M., Chaniotakis, G., Manoli, V., Dionissios, M., Karafotia, M., Gouscos, D. and Meimaris, M. (2008). Applying Digital Game‐Based Learning Solutions to the Primary and Special Classroom: Results From Field Studies. In Proceedings of nd the 2 European Conference on Games‐Based Learning (EC‐GBL), Barcelona, Spain. Segers, E. and Verhoeven, L. (2005). Long‐Term Effects of Computer Training of Phonological Awareness in Kindergarten. Journal of Computer Assisted Learning, 21(1), 17‐27. Suh, S., Kim, S.W. and Kim, N. J. (2010). Effectiveness of MMORPG‐based instruction in elementary English education in Korea.Journal of Computer Assisted Learning, 26, 370–378. Vos, N., van der Miejden, H. and Denessen, E. (2011). Effects of constructing versus playing an educational game on student motivation and deep learning strategy use. Computers and Education, 56(1). 127‐137. Wrzesien, M. and Alcaniz Raya, M. (2010). Learning in serious virtual worlds: Evaluation of learning effectiveness and appeal to students in the E‐Junior project. Computers & Education, 55(1), 178–187. Wang, T. (2008). Web‐based quiz‐game‐like formative assessment: Development and evaluation. Computers & Education, 51(3), 1247–1263. Yang, C. J, Chen, C. H. and Jeng, M. C. (2010). Integrating video‐capture virtual reality technology into a physically interactive learning environment for English learning. Computers and Education, 55(3), 1346‐1356.

207

Designing Games to Disseminate Research Findings Claire Hamshire, Rachel Forsyth and Nicola Whitton Manchester Metropolitan University, Manchester, UK c.hamshire@mmu.ac.uk r.m.forsyth@mmu.ac.uk n.whitton@mmu.ac.uk Abstract: Sharing the findings of research projects to improve future practice is often an important objective of educational research. However disseminating the results to groups that will directly benefit can sometimes be problematic and there may also be complexities around presenting research in a context that gives real‐world relevance. The informal environment of game play is one method that can be utilised to promote targeted discussion and present research in a format that is both fun and engaging. This paper explores how two board games that had their beginnings in research projects were developed. One of these projects explored students’ perceptions of their higher education experiences (Staying the Course), and the other investigating staff experiences of course development (Supporting Responsive Curricula). Neither project was initially tasked with developing a game, but both project teams believed that games would help with sharing the findings of the projects widely. The underlying philosophy of both authors was to design an active learning environment in which players could learn via discussion activities and testing their understanding. By using the medium of a board game we aimed to provide an opportunity to examine problematic issues within the ‘magic circle’ of game play. This would provide an environment in which players could contribute to linked discussion and start thinking about different perspectives and how they could make improvements to existing situations. This paper describes the approaches used to design each game in relation to the differing contexts for game play: one of the games is intended for use by students and those advising them, whilst the other is for course development teams which may be composed of students, administrative, technical and administrative staff in universities. The identification of design elements to make the games effective is also discussed. Keywords: game design, higher education, dissemination, research

1. Introduction The two games presented in this paper were designed to disseminate the findings of research projects within a real world context. The first project explored healthcare students’ perceptions of their higher education experiences (Staying the Course), and the second investigated staff experiences of curriculum development (Supporting Responsive Curricula). Neither of the projects had an initial focus to develop a game, but in both cases, it was clear from the research findings that the intended audiences found it difficult to see the relevance of all of the outcomes to their own situations, or to feel comfortable in discussing potentially difficult issues openly. Both research teams wanted to ensure that the data gathered during the projects could be utilised to facilitate future developments and share good practice identified as a result of the research. The project teams believed that sharing the findings of the projects widely could help to change behaviours and bring about service improvements, but were unsure how to get most impact from the data. Therefore using the medium of a board game offered an opportunity to examine problematic issues within the ‘magic circle’ of game play for adults (Charlier, Ott et al. 2012).

2. Staying the course Staying the Course was a large scale mixed‐methods, regional study undertaken at nine Higher Education institutions in the North‐West of England. The purpose of the study was to explore healthcare students’ perceptions of their learning experiences and identify factors that contributed to early departure from courses. The study incorporated multiple strands within two phases. The first, qualitative, phase explored the experiences of current and discontinued students through narrative interviews. The second phase used an online survey, developed from a thematic analysis of the interview data, to further investigate the key issues identified and aid transferability of findings. The aim was to gain an in‐depth understanding of the factors that influenced students’ perceptions and contributed to attrition. Therefore the students’ perspectives were considered to be paramount to identify ‘tipping points’, where personal factors and experiences combined to lead to either a positive or negative encounter. The student sample for the study was drawn from a range of healthcare programmes, which were

208

Claire Hamshire, Rachel Forsyth and Nicola Whitton primarily nursing but also included a range of allied health professions. Over 1,100 students contributed to the project via interviews, focus groups and an online survey and the results were used to develop strategies to improve student retention in the North West of England. The results demonstrated that some students needed greater support during their initial transitions to university and therefore the concept of the ‘Staying the Course’ board game was developed with the underpinning philosophy of:

Facilitating social integration through game play and discussion.

Promoting personal integration by providing information on the institutional systems for pastoral care and learning support.

Setting reasonable student expectations by raising their awareness of academic systems and commonly reported concerns and problems within the first year.

3. Supporting responsive curricula Supporting Responsive Curricula (SRC) was a single‐institution study that used a case study approach to review current practice and identify key drivers for change. Its aim was to make the curriculum more responsive to the needs of students, employers and professional bodies and in undertaking this task, develop new approaches to the curriculum design and approval process to create lasting change at institutional level. One of the aims of the SRC project was that staff should be more involved in the articulation and management of institutional quality assurance processes; some staff seemed to see them as being burdensome, irrelevant to day to day course management and imposed externally. Findings from the project report highlighted that academic staff, in particular, found the processes bureaucratic and frustrating. Conversely, administrative staff thought that course teams did not always communicate their needs clearly. These positions made it more challenging to find ways in which to interest people in institutional processes and to get them to work together. The lack of ownership of the processes seemed to make it easier to critique them, but may have been a factor which militated against constructive suggestions for improvement.

4. Purpose of the games The underlying philosophy of both authors was to design an active learning environment in which players could learn via discussion activities and testing their understanding – effectively using a constructivist learning perspective. Following discussions with game designers, the possibility of using traditional board games to communicate the findings of the two projects and to encourage discussion began to take shape. Board games commonly have the simple objectives of entertainment or social facilitation and whilst these elements were also incorporated into both games, we were also guided by an awareness of what was appropriate within a given context and the logistics of game play within the ‘serious’ Higher Education environment. For Staying the Course, the team wanted to use the research data to directly support students’ transitions to higher education. As such the purpose of the design was to introduce students to common concerns and problems that they might encounter during their first year at university, and encourage them to think about possible approaches and solutions. The primary objective of the game was thus to facilitate students’ inductions to Higher Education by providing an opportunity for both collaborative learning and peer support. The game format was therefore used to provide a safe space in which students could interact and make mistakes, removing the pressure and adding a layer of fun to the learning experience. Guided by this philosophy, the game was developed in close partnership with both students and the Students’ Union staff to ensure that the game had real‐world relevance. With Supporting Responsive Curricula, the team wanted to focus on the actual process of getting curriculum material approved and accredited, to encourage staff and students to find out more about the accreditation process and also to discuss ways to make it more responsive to the needs of others. The aim was to use the game to highlight oddities, inconsistencies and important elements of the existing processes, whilst depersonalising them in a environment that made discussion more comfortable. When incidents occurred by chance in the game, like forgetting a key deadline, or missing out part of a consultation, or losing a laptop with

209

Claire Hamshire, Rachel Forsyth and Nicola Whitton all of the documentation on it, nobody was really responsible, which made it easier to have discussions about how difficulties could occur and how they could be avoided or at least mitigated against. Once these initial purposes had been set for each of the games, the next stage was to consider structure and format. As the aim of each of the games was to disseminate research findings a method of game play that would allow players to interact with a large range of information and material was required. The games had to be produced within a small budget and be acceptable to a diverse audience therefore the most efficient way to get players to engage was to construct the playing experience around thematic card sets. Each team subsequently developed a range of card sets that could be used within the format of a dice‐based, board game with game play directed by the students responses to the card questions. In both cases, several iterations were necessary in order to make the instructions on the cards concise, jargon‐free and credible.

5. Staying the course For Staying the Course the thematic analysis of the students’ narratives and comments on the survey about their experiences was used as a starting point and three broad themes used to scaffold the game design:

Academic issues and uncertainties;

Personal difficulties;

Placement problems and issues.

The most frequently occurring problems and concerns from these three themes were developed into quiz questions and dilemmas that became three sets of different coloured cards. A fourth set of cards – “Take a Chance” – was developed to incorporate unusual and unplanned circumstances that students had experienced and a fifth set of cards giving information on Student Services developed to raise students’ awareness of the campus‐based advice and support services. When players landed on a square of the corresponding colour after rolling the dice a card was taken from the top of the pile‐ the game board is pictured in figure 1 below.

Figure 1: Staying the course game board

6. Supporting responsive curricula A university quality assurance system requires academic staff, organised in a course team, to justify the structure, content and assessment strategy of their course to their peers. In the UK, a selection of colleagues who work internally and externally to the institution is selected to scrutinise the course design at various

210

Claire Hamshire, Rachel Forsyth and Nicola Whitton stages. Course teams are expected to relate the design of their course to issues of student experience and progression, disciplinary content and academic standards, as well as to institutional policies on the use of technology, assessment regulations and inclusive practice. The game was intended to encourage players to critique the processes and to discuss alternatives in a playful way that encourages freedom of thought and expression. The game was designed around a board and set of cards, which followed the process of accreditation through three stages which mirror the real process: strategic approval, course planning and final approval. At each stage of the game, players are faced with realistic dilemmas, gathered from interviews and discussions with stakeholders, and are expected to make decisions about them. A scoring system was devised that demonstrated the impact of certain elements of the process and the ways in which colleagues might use these processes. The goal of the game was to get round the board first to reach the final validation square. To progress to each new level the players had to earn quality stars, which would enable them to move up to the next ring, or stage in the process. As the players move round the board they have to make decisions that balance speed and quality. Quality stars are awarded for actions that focus on learning, teaching and student experience, while speed can be improved by focusing on box‐ticking actions ‐ the game board is pictured in figure 2 below.

Figure 2: Accreditation! game board

7. Designing the games Each game evolved through an iterative process of testing and evaluation in discussion with both students and staff to ensure that the games were seen as acceptable by their users and not frivolous or an unnecessary distraction. As such each game aimed to incorporate elements of effective games (Whitton 2009) to achieve its educational objectives including acceptability and real world relevance, completion goals, simple rules,

211

Claire Hamshire, Rachel Forsyth and Nicola Whitton competition/challenge, community/opportunities for interaction, creativity, narrative and problem‐solving and fun. This section outlines how each game was designed and discusses how we were guided by what was appropriate within a given context. One of the games was intended for use by students and those advising them, while the other was for course development teams which may be composed of students, administrative, technical and administrative staff in universities.

8. Acceptability and real‐world relevance In order for the games to be perceived as acceptable and appropriate by our target groups of players we believed that the purpose and the real‐world relevance of the games had to be clearly articulated. We wanted players to recognise that the games were an appropriate context in which they could explore problems and concerns and be able to transfer any knowledge gained into real‐world situations. In both cases we felt that it was therefore worth thinking about a working title at the beginning. The Staying the Course game retained the same name as the project, as the team believed that the name made it clear that the purpose of the game was to facilitate students’ journeys into higher education. The Supporting Responsive Curricula project called its game ‘Accreditation!’. Unfortunately, the title of ‘Frustration!’ was already trade‐marked for a board game. During testing, the title of ‘Monotony’ was also suggested; we were assured that this wasn’t a reflection on the game, but rather on the processes it described.

9. Completion goals For both these games the goal was to be the first to complete the game. Conventionally, games need to have a clearly stated goal and there are few choices about what this might be in traditional board games: to be the first to get to the end, to gain the most points, or to be the last person still playing. The enduring attraction of successful traditional games such as Snakes and Ladders, Draughts, Chess or Monopoly™ shows that goals should be simple, even if game play is complex. The choice of goal has an obvious impact on the game play as well as the learning outcomes: for instance, if the aim is to be the first person to the end, then potential acts of sabotage can be included to set back other players. However, if there is an underlying objective about co‐operative working, such a possibility would be counter‐productive – unless there is some benefit to one group teaming up to support each other and disadvantage players outside their team. The goals of game completion for each of these games also linked to the underlying philosophy – to keep going despite potentially encountering difficulties and challenges. As such, through playing ‘Staying the Course’ the students completed their first year at university whilst encountering common problems as well as personal successes and by completing ‘Accreditation!’ programme teams gained an insight into some of the complexities inherent in curriculum development.

10. Simple rules Rules set boundaries for the players about their behaviours and actions during the game. Players may not like the rules, or they may not play by them, but at least everyone knows where they stand. The advantage of designing a traditional board game is that most of the rules you might want to use have already been tried and tested, and are familiar to the players. This makes it easy for players to settle down to play without much introduction or support. In the case of both of these games, the rules were pretty simple:

You can play on your own or in teams

Follow the instructions on the cards you pick up

Move round the board in a clockwise direction unless instructed otherwise by a card.

In testing, the designers of Accreditation! found that players were almost too compliant to the rules, and could get stuck in an endless loop. A fourth rule was therefore added: ‘You can cheat and skip out of a zone if things

212

Claire Hamshire, Rachel Forsyth and Nicola Whitton seem to be getting tedious and/or repetitive.’ – this rule in itself caused much discussion about how that might work in real life, and what ‘cheating’ might be like if it were applied to course documentation.

11. Competition/challenge In order to introduce challenge into the games it was important that there was at least some degree of difficulty embedded. For each of these games this was achieved by designing card sets that included a range of challenging questions. Some cards included questions with very obvious answers or merely instructions; whereas others required that the players already had some background knowledge of the game content. This helped to get players moving at a differential pace around the board. There were also opportunities to help other players by using the ‘Take a Chance’ cards supportively, or to sabotage them if they were doing too well.

12. Community/opportunities for interaction In both of the games described here, the goal was to be the first to reach the last square on the board , but this turned out to be unimportant for most players. The process of playing, interacting and collaborating with the other players turned out to be the most important element of the experience, which entirely met our objectives of communicating research findings and encouraging creative discussion. One of the players of ‘Accreditation!’ declared that everything on the cards she picked up had actually happened to her during a similar process, leading another player from another institution to comment that ‘It’s not a game, it’s a biography!’. This led to very fruitful discussions about effects of institutional culture on approaches to course documentation and management.

13. Creativity Although creativity is often included in games in order to increase engagement, it did not form a part of the design considerations for these games, because of their focus on disseminating research outcomes. The designers were not aiming to encourage creativity specifically; the overall aim was to engage players with what others had reported about their experiences and use that to stimulate discussions that would hopefully later lead them to creative approaches to managing similar situations.

14. Narrative Incorporating a story within the game to increase player engagement and enable them to identify with the context was important to the design. Both games followed a narrative linked to their underlying purpose and which would be familiar to the players, either from direct experience, or in relation to their future plans or expectations. Staying the Course follows the course of an academic year and Accreditation! takes players through a cycle of course validation or review.

15. Problem solving Problem solving was addressed by the use of dilemmas for the players. The choice of solutions had a profound effect on the outcome for the player, in terms of how quickly they could move or whether they collected additional valuable points. Players soon begin to try to work out how the dilemmas are constructed, and what the likely outcomes of their decision will be on game play, rather than making ‘honest’ choices. This in turn led them to talk about how decisions are made and what factors are taken into account in real life, and what impact these have on overall performance or behaviours in the real situations considered in the games: in doing this, they were talking about the research outcomes and how these relate to them.

16. Fun The concept of fun was important to both authors. As previously stated, we were aware that the intended audiences found it difficult to see the relevance of the project outcomes to their own situations, or to discuss difficult issues. Therefore we incorporated humour into the scenarios used in the card sets in order to help to move the players outside their usual working or study environments. In addition both games also included a ‘Take a Chance’ set of cards to introduce an element of luck into the games and present real‐world problems. These Take a Chance’cards imposed realistic positive or negative events, which were not specific to the themes or the chronology of the game, such as losing your laptop on a train, illness, or learning something new and important from a chance meeting. In both games it is possible for any of the cards to send you backwards instead of forwards, which may have a major impact on relative positions in the game, and causing plenty of

213

Claire Hamshire, Rachel Forsyth and Nicola Whitton discussion, frustration and amusement. These chance cards were also probably the most entertaining part of the games to design and exaggeration was frequently considered acceptable.

17. Evaluation Initial evaluations of The ‘Staying the Course’ game, have been encouraging and both staff and student players have commented positively on their playing experiences. The majority of student comments have focused on their enjoyment of game‐play and on their perceptions of having gained greater knowledge and understanding of student support services after play. Observations from academic and support staff included positive feedback on both the game design and content and several constructive comments on how the game could be further developed. As such development of the Staying the Course game was been iterative and continuously informed by game‐ play sessions with both staff and students. Further versions are currently being developed for use with other student groups, and two additional card sets have been created: Finances and Futures. These card sets are coloured so that they are interchangeable with the Placement card set so the game can be adapted for the needs of a particular group. Student attrition from higher education is an international concern and there is evidence that students leave for a range of personal, social and academic issues (Hamshire, Willgoss et al. 2011). While there is no simple formula to ease students’ transitions into higher education and the retention of a diverse student body, evidence from this and other studies (Wilcox, Winn et al. 2005; Yorke and Longden 2007; Hamshire, Willgoss et al. 2011) indicate that interactions that promote social and academic integration should be encouraged. The findings from the initial evaluations indicate that ‘Staying the Course’ can be used to promote social, academic and personal integrations and this suggests that it could be used to support student transitions to higher education. The board game gives players an opportunity to progress through an academic year. There is also the potential to customise the game for any student group or age group by editing the cards and institutional logos on the board. The board game is currently being developed as a ‘giant’ game to be used at induction events and study days. Accreditation! is now used as a staff development tool and replaces what used to be a very boring lecture on accreditation. Feedback suggests that players retain understanding and knowledge of the different parts of the process and the importance of working as a multi‐disciplinary team to develop their course documentation. Institutions can customise the

18. Conclusion Moseley (2010) has argued that simple gaming activities, played face‐to‐face, can be used to create context in a host of learning and teaching situations. The two games described here did this, but also had the advantages of flattening hierarchies, providing unexpected connections between players and sparking off a whole host of discussions and follow ups. Being in a difficult situation as a playing piece gives anonymity to express opinions without them being linked directly to you. Because everyone in the game takes a turn, everyone has an equal voice, even if they would be too shy, or perhaps overawed by hierarchy, to offer an opinion in a more usual group discussion. This was particularly important for Accreditation!, where the designers wanted to create a discussion environment for a very mixed group of colleagues: students, administrative staff and academic staff. The use of games also provided a very neat way to disseminate research findings to audiences who might not seek them out in traditional academic papers, and to support people in making practical use of those findings to change practices. The traditional approaches to board game design provided familiar parameters to players, whilst introducing them to new ideas and evidence in a format which enabled discussion. It is definitely something which the researchers will consider for future projects where the findings are likely to be of use to a wider audience.

Acknowledgements The authors would like to acknowledge JISC for providing support for the Supporting Responsive Curricula project, NHS North West for providing support for the Staying the Course project and Peter Whitton for help with the board designs.

214

Claire Hamshire, Rachel Forsyth and Nicola Whitton

References Charlier, N., Ott, M., Remmele, B. and Whitton, N. (2012). Not Just for Children: Game‐Based Learning for Older Adults. in ECGBL 2012. Cork: Academic Conferences. Hamshire, C., Willgoss, T. G. and Wibberley, C. (2011). ‘The placement was probably the tipping point’–The narratives of recently discontinued students. Nurse Education in Practice. Moseley, A. (2010). Back two spaces, and roll again: the use of games‐based activities to quickly set authentic contexts. in 4th European Conference on Games Based Learning. Copenhagen: Academic Conferences Limited. Whitton, N. (2009). Learning with digital games: A practical guide to engage students in higher education: Routledge. Wilcox, P., Winn, S. and Fyvie‐Gauld, M. (2005). "It was nothing to do with the university, it was just the people": the role of social support in the first‐year experience of higher education. Studies in Higher Education 30/6: 707 ‐ 722. http://www.informaworld.com/10.1080/03075070500340036 Yorke, M. and Longden, B. (2007). The first‐year experience in higher education in the UK, Higher Education Academy. from http://escalate.ac.uk/downloads/3365.pdf

215

Facilitating Teacher Students’ Innovation Competence through Problem‐Based Game Design Processes Thorkild Hanghøj1 and Sia Hovmand Sørensen2 1 ResearchLab: IT, Learning and Design (ILD), Aalborg University, Denmark 2 National Centre of Competence Development, Aarhus University, Denmark thorkild@hum.aau.dk siah@dpu.dk Abstract: The aim of this paper is to describe how new teacher students develop innovation competence through problem‐ based game design processes by participating in an intro camp. The intro camp was held for 350 new teacher students at a Danish university college in 2011, which were asked to solve the real‐life problems of local schools by designing game solutions to be presented for and assessed by participating school directors and pupils. Based upon a pragmatist theoretical framework, we conceptualize the students’ development of innovation competence in relation to creative problem‐solving, game frames and the interplay of different knowledge domains “inside” and “outside” of teacher education. By taking a mixed methods approach, we combine qualitative and quantitative methodologies for studying our case. This involved observations and interviews with selected groups as well as a post‐camp survey with all the students. In the analysis, we focus on two analytical themes that relate to the teacher students’ problem‐based game design processes and their experience of becoming future innovative practitioners. The paper concludes by discussing future perspectives on the use of problem‐based game design for developing innovation competence – both within and beyond the context of teacher education. Keywords: problem‐based game design, teacher education, innovation competence, game frames

1. Introduction In order to meet the future challenges and opportunities related to the on‐going development of public schools and curricular demands within modern society, it may be argued that teachers need to develop innovation competence. This assumption corresponds with the aims of the Danish teacher education that are increasingly being articulated in terms of creativity and innovation, which teacher students will need in order to solve the contingent problems of everyday schooling. So far, the dominant discourse on entrepreneurship teaching in Denmark has focused on financial value creation in the private sector (e.g. Kirketerp 2010) and the studies on entrepreneurship teaching in educational contexts aimed at the public sector aimed is still a relatively unexplored field. Similarly, research on creativity within education has mostly focused on children’s creativity and less on creativity in relation to teachers and teacher students (Kaufman and Sternberg 2010). So in spite of the increased interest in the innovation and creativity among policy makers and practitioners, there exist very limited studies that have explored what these concepts mean in practice for teacher students. The aim of this paper is to empirically describe how new teacher students may be able to develop innovation competence through the course of a four day long intro camp held at a Danish teacher college. More precisely, the paper focuses on the students’ use of Problem‐Based Game Design (PBGD) as a particular pedagogical approach for designing game solutions that address and communicate real‐life problems defined by local schools. In summary, our primary research question is: How may new teacher students develop innovation competence through problem‐based game design by participating in an “intro camp”?

2. Case: “The Intro Camp” The empirical basis for our paper is based on a mixed methods study of an “intro camp” held for 350 new teacher students at a Danish university college in 2011. The intro camp lasted four days and divided the students into tracks of six‐eight working groups with five‐six members in each. All the groups in each track were given the same problem statement, defined by one of the local schools, which required them to develop an original and useful game (e.g. a board game, a physical exercise, a role‐playing scenario etc.) that could contribute to the schools’ on‐going work with the identified problem. During the camp, the teacher students were invited to visit the local schools and interact with relevant pupils, teachers, and school directors. The teacher students received feedforward from the pupils on their working ideas during the camp, and the winning group for each track would also receive more detailed feedforward based on the presentation of their game‐based solutions.

216

Thorkild Hanghøj and Sia Hovmand Sørensen

3. Theoretical framework Our theoretical framework is founded on a Dewey‐inspired pragmatist theory of science, which assumes a complex and context‐sensitive interplay between knowledge creation and development of practice (Biesta and Burbules 2003). As mentioned, the main aim of the paper is to describe how teacher students may develop innovation competence through problem‐based game design processes. Our conception of innovation competence is based upon research on competence development and innovative problem‐solving (Dewey 1916, 1933; Scheidel 1986; Darsø 2001), research on entrepreneurship teaching and self‐efficacy (Bager 2011; Fayolle and Gailly 2008; Kirketerp 2010; Bandura 1997), and research on how game scenarios may be used to “translate” knowledge practices across different domains (Hanghøj 2013). Drawing on this body of research, we conceptualize the development of innovation competence as a social, complex, dynamic and contextualized process, which requires meaningful reflection on first‐hand experiences to create understanding that can guide future action. Thus, innovation competence is defined as the ability to create and take part in collaboration through innovative problem solving in a specific domain with particular knowledge practices. Within the context of profession‐specific education, the development of enterprising behavior is a necessary component in the development of innovation competence as it is mandatory that e.g. teachers develop the ability to implement their innovative solutions (Kirketerp 2010). Moreover, we also assume that the development of enterprising behavior is supported by the development of self‐efficacy and by interaction with contexts where possible implementation could take place. Important research has been made on how pupils may develop game literacies when designing video games (Burn 2007; Peppler and Kafai 2010). Similarly, the design processes of professional game designers have also been explored through ethnographic studies (Salen 2007). However, our conception of Problem‐Based Game Design (PBGD) is primarily focused on how the game design processes are based upon “generative metaphors” (Schön 1978), which incorporate the way in which particular game formats frame and reframe different modes for interaction and learning (Goffman 1974; Hanghøj 2011). In this way, we assume that the game framing and reframing processes of PBGD may support participants in making meaningful translations across different knowledge domains that both exist inside and outside of formal educational contexts (Hanghøj 2013).

4. Methodological approaches As our research builds upon a pragmatic theory of science, we aim at creating a basis for development in practice and hence pragmatic validity is our criteria for the legitimation of our work (Onwuegbuzie and Johnsson 2004; Dewey 1916, 1933). Moreover, we see problem‐based game design and the development of innovation competence as complex, contextualized and continuous processes, which can only be separated into components for analytical purposes (Khalil 2003). Our methodological approach is based on a mixed methods design in the form of a concurrent partially mixed design, where dominant status is given to qualitative methods (Leech and Onwuegbuzie 2009). The quantitative and qualitative data has been collected within a short timespan due to the temporal delimitation of our field. The two types of data have not informed one another during the data collection and they have initially been analyzed separately. Subsequently the findings have been integrated into a joint analysis. The methods used for production of empirical data are observations (Spradley 1980) of one track and in particular one group during the camp, semi‐structured post‐camp interviews (Kvale and Brinkmann 2008) with two groups from the same track, and a survey on all participants at the camp. The track we observed were chosen by arrangement with the steering group of the camp, while the group one of us observed where selected randomly. The group members all gave consent to have their process documented as a part of our research project. We conducted two post‐camp interviews (appr. 90 minutes each): one interview with the observed group and one with another group from the track in order explore contrasting experiences of the camp. The short survey among all participants was conducted at the end of the camp. 74 % of the 350 participants (N = 258) answered the survey. The aim of the observations was to obtain an understanding of how the problem‐based game design processes unfolded during the camp. Similarly, the aim of the interviews was to describe the students’ understanding of their process, the camp as the context for this process, and the meaning they ascribed to different aspects of this process. Through the survey we examine the prevalence of choices and perceptions among the camp participants and the correlations between perceptions. Interviews were transcribed and analyzed both

217

Thorkild Hanghøj and Sia Hovmand Sørensen through an explorative search for the students’ ascribed meaning to their process and by a more structured coding (Kvale and Brinkmann 2008). We have conducted a descriptive analysis of the survey results and used Goodman and Kruskal’s gamma has as measurement for the correlations presented in this paper with the level of significance set to 0,05 (Agresti 2002).

5. Analysis We will now present two analytical themes that have emerged from analyzing our data (field notes, group interviews, and the survey). Each analytical theme is presented by integrating qualitative and quantitative findings through our mixed methods approach.

5.1 Problem‐based game design processes The first analytical theme focuses on two aspects of the students’ problem‐based game design processes and their post‐camp reflections, namely game framing and lacking conceptualization. Game framing The “framing” of the students’ problem‐based game design processes was to a large degree was influenced by their existing knowledge of different game designs. As mentioned, all the groups in each track had to design game solutions aimed at solving a specific problem at a specific school. On the morning of the second day of the camp, the track that we followed was introduced to this problem by the school director: “How can we introduce concepts related to democracy and citizenship so students at different stages are able to understand the use of the concepts in their everyday life?” Next, each of the five groups in the track visited a class in order to engage in dialogue with and learn about the pupils’ conception of the problem. After returning to the teacher college, the students worked together in groups in order to describe the “essence” of the problem in one sentence. Drawing upon their dialogue with a multi‐ethnic group of pupils, the group that we followed concluded that the most important aspect of the problem was: “Respect across cultural differences form the foundation for a healthy democracy and citizenship”. Based upon the original problem formulated by the school director and the students’ re‐statement of the problem, we expected the students to design a game solution that aimed at creating better understanding of cultural differences, citizenship and democracy in relation to an everyday perspective. However, when the group started developing ideas, something else happened. Two students in the group focused on how students should acquire concrete knowledge on democracy, e.g. by answering factual questions on politicians and political parties, and this idea played a formative role in their brainstorm process. From then on, the game design ideas of the group mainly centered on creating a board game like Trivial Pursuit that could test factual knowledge about “who’s who” in Danish politics. Consequently, the group’s original ideas concerning cultural differences became backgrounded. The group’s change of focus was not explicitly articulated by the group members. They simply stopped talking about cultural differences. Later on in the camp, the students were asked to brainstorm through “divergent thinking” (Scheidel 1986) and came up with wholly different game ideas, e.g. a role‐playing game where the students had to collect hearts by acting as “good citizens”. One of the students suggested that the TP format might be “too boring” for the pupils. He also pointed out how the game lacked connection with what “goes on all the time in the class”, by which he referred to everyday aspects of democracy. Still, the students returned to their original board game concept, which they eventually ended up presenting at the end of the camp. As this example suggests, it was quite clear that the students’ prior experience with game formats such as Trivial Pursuit (and similar quiz games) to a large degree framed their range of suggested design solutions (Goffman 1974). Similarly, the survey documents how 55 % of the students chose board games as their preferred format when designing game solutions, with the remaining choices being role‐playing games 23 %, debate games 18 %, and only 4 % designing “other” formats. During the group processes and group interviews, several students mentioned how they lacked time and had to work under high pressure. Moreover, the students also had a relatively unclear idea of the expectations that they had to meet ‐ i.e. what did their own teachers as well as the participating schools (teachers, directors, and pupils) expect them to deliver as design solutions. When evaluating their experience of the intro camp in a

218

Thorkild Hanghøj and Sia Hovmand Sørensen group interview, one of the students mentioned how she felt “too locked” in their attempt to come up with a specific solution and “afraid of taking chances in the same way as some of the other groups”. A second student in the group confirmed this: “We only wanted to do things the right way. I think that was it. But... the way things ended up we started doing it [making the game] a bit old‐fashioned”. As these quotes indicate, it seems quite likely that a combination of lacking time, unclear expectations and inexperience with collaborative problem‐solving made it difficult for the students to come up with game design solutions that differed from their prior game experiences. In spite of these challenges, the survey results indicate that the students’ experience with game‐based problem solving during the camp had a positive impact on a significant part of the students’ attitude towards “games as a way of learning”. When asked, 40 % of the students stated that their attitude was more positive than before the camp, 47 % stated that their attitude was unchanged, while only 1 % stated their attitude was more negative than before the camp. 12 % of the students state that they had not considered games as a way of learning before the camp. Similarly, the students were also quite positive, when asked if their experience of the camp would be useful for future innovative problem solving. 30 % of the students found that camp experiences was highly useful, and 58 % found that it would be useful to some extent while 12 % found it to only be useful to a lesser extent and 1 % found it not to be useful at all . Lacking conceptualization The students were not only challenged in relation to their lack of time and insufficient experience with design processes. Their game‐based problem‐solving design processes also suffered from limited knowledge of the complex concepts “democracy” and “citizenship”, which they had to address. This can be illustrated with an excerpt from the outset of the design processes, where the two students shortly discussed their own conceptions of their core concepts. Student 1: What is your conception of the concepts “democracy” and “citizenshipliness” [sic]? Student 2: Well, it is the way in which we can all live in harmony with each other. It doesn’t matter whether it is in the classroom or in a country The second students’ answer does not differentiate between the two concepts and indicates limited conceptual knowledge. Moreover, none of the other students in the group would comment or further discuss the concepts, which were central to the problem they had to address. In this way, it became difficult for the students to re‐frame their assigned problem through the generative metaphor of a game (Schön 1978). There were at least two reasons why the students did not discuss their concepts more thoroughly. First of all, the students were only two days into their new study and had received no instruction on the meaning of the complex concepts. Secondly, the students were new to each other and showed limited experience with the importance of group discussion and reflection upon such concepts as a way of conducting inquiries into particular problems. After the dialogue presented above, the group did not attempt any further problematization of their core concepts democracy and citizenship, neither in the remaining group work, nor in the final group interview. Seen from a Deweyan perspective, this lack of conceptualization shows how difficult, but also how important, it was for the students to be able to “reconstruct” local knowledge into workable design solutions by reflecting on their immediate experiences of meeting teachers and pupils at the participating schools (Dewey, 1933). This indicates how the ability to reflect upon different stages during problem‐solving process forms an important part of developing innovation competence. We have shown that issues concerning inquiry and the students’ balancing of internal and external expectations of the solution both had an impact on the groups’ collaboration. However, the survey indicates that the majority of the students to at least some extent still found these experiences useful to future collaboration: 23 % found that their experiences are highly useful in this regard and 67 % found them to be useful to some extent, while 9 % found them to be useful only to a lesser extent and 1 % found them not to be useful at all As a complementary perspective, it may be argued that the schools participating in the camp could have been more precise at stating concrete problems that could be meaningfully conceptualised and reconstructed as

219

Thorkild Hanghøj and Sia Hovmand Sørensen game design solutions. Thus, one of the students that made the quiz game for teaching democracy and citizenship complained that their assigned problem was too dry... too difficult. There was not enough space and movement in it. I like when problems gives the opportunity to be more creative and think more into the open and there is space for doing something different than what is implied from the start. As the example indicates, the problem statement concerning citizenship and democracy was quite abstract and intangible to the new teacher students. Similarly, the survey shows how some of the students also perceived this to be a challenge with the problem statements of other tracks: 8 % of all the students found that their problem statement was not at all suitable for their work and 30 % found it suitable only to a lesser extent, while 39 % found it to be suitable to some extent and 23 % found it to be suitable to a high extent

5.2 Becoming innovative practitioners When asked in the survey, 49 % of the students stated that their confidence in their ability to contribute to solve the challenges of the public school is greater than before the camp, while 50 % stated this confidence to be unchanged during the camp. Just 1 % stated this confidence to be lesser than before the camp. This result indicates that the camp had a positive impact on many of the students’ self‐efficacy. Thus, the second analytical theme describes how the students’ experienced and reflected on the camp in relation to becoming innovative practitioners. More precisely, the theme focuses on the students’ experience of self‐efficacy and how it was related both to their school visits and their creation of game designs. Self‐efficacy and school visits Several factors seem to have contributed to the development of the students’ self‐efficacy. We assume that most of the students had no professional experience with teaching and their images of public schooling mainly derived from their experience as pupils and from the media. The track we followed visited a school in an area, which has drawn the media’s attention due to its a status as “ghetto”, hence the students had a stronger image of what the school would be like and some were even nervous about going there. Most students had to revise their image after the visit. The following quote from one of the post‐camp group interviews illustrates the significance this revised image may have had for the students’ belief in their future possibilities: I was very overwhelmed at how well that school works and how much they do to make it work. I remember that Caroline said that she would like to work such a place when she had finished [studying]. Then I thought: Would you, really? It was not something that I could imagine myself [do]. I just wanted to work at a private school. But now... I don’t think that I am going to work at [area], but I can see what’s great about the challenge in it and in being able to do something right. As the quote shows, the visit changed this students’ perception of the school from being a place she was really nervous about visiting, and absolutely not a place she would ever want to work, into a place, where she could make a positive difference. Other students expressed how surprised they were at the schools’ high standard and its nice atmosphere. It was also important to the students’ experience that the pupils received them as professionals and with respect. Each group had less than an hour of interaction with a class. Still, this experience positively changed the fear of not being able to manage a class that some of the students had. Some of the students talked with relief and new self‐confidence about this experience both shortly after the visit and in the interview: it was cool to be in a class and observe how they somehow showed respect for you, the way that you could stand in front of them and talk with them and they kept themselves relatively quiet. These two examples shows how problem‐based game design processes may be organized in a way that can create meaningful “translations” between different domains (Hanghøj, 2013) – i.e. the professional domain of actual schools as possible future work places, the pedagogical domain of being a teacher student, and the everyday domain of the teacher students’ commonsensical conceptions of schooling and teaching. Self‐efficacy and game creation The camp’s contribution to the students’ increased sense of self‐efficacy was not only related to the students’ school visits, but also to the experience of creating useful games. Survey results show that there is moderate

220

Thorkild Hanghøj and Sia Hovmand Sørensen positive correlation between the students’ evaluation of their game’s ability to contribute a solution to the problem‐statement and their development of self‐efficacy . Of the students who had the least positive evaluation of their game only 17 % perceived a development in self‐efficacy. The same figure is 59 % for those students who made the most positive evaluation of their game. As mentioned in the first analytical theme, survey results indicate that the majority of students developed competence in regard to both collaboration and innovative problem solving. Survey results indicate that this development of competence also increased students’ perceived self‐efficacy. There is a strong correlation between the students perception of the usefulness of their experiences for future collaboration and the and there is a moderate correlation between perceived development of self‐efficacy the perceived usefulness of experiences in regard to future innovative problem solving and the perceived . As an example, one result shows that 21 % of the students, development of self‐efficacy who perceived their experiences during camp only to be useful to a lesser extent for future collaboration, perceived a development of self‐efficacy. The same figure is 72 % for the students, who perceived their experiences to be highly useful for future collaboration. Finally, the results from the survey indicate that the development of self‐efficacy may not only be important for the students’ innovation competence as a basis for enterprising behavior, but also had a moderate with a positive change in the students’ motivation for their new study. This correlation stresses the camp’s potential for contributing to the continual development of competence. As an example, one of the students mentioned how she experienced the intro camp positively, as it allowed her to “be peculiar and creative”, which marked a “good transition” from being a non‐student into being a student. She was backed by a co‐group member, who was similarly fond of the camp as it allowed him to focus less on school‐like results and focus “on the journey instead of the goal.” For these students, the camp served as a meaningful transition to their new study as the camp both differed from everyday forms of teaching and also gave the students a chance to meet the “real world” of public schools through the creation of a game solution.

6. Discussion The following discussion focuses on three topics: 1) the students’ development of innovation competence, 2) the use of Problem‐Based Game Design as pedagogical method, and 3) the validity of the findings.

6.1 Developing innovation competence The first issue to be discussed concerns whether the students that participated in the intro camp were able to develop innovation competence. Obviously, we cannot know for certain whether the students did develop innovation competence as we have no empirical data that allow us to describe the students’ behavioral change during larger time spans or their ability to transfer innovation competence across different contexts (Rychen and Salganik 2003, p. 55). However, our analysis offers several indicators that that the students were able to develop innovation competence. Thus, the survey shows how the intro camp resulted in an overall significant increase in students’ self‐perceived experience of self‐efficacy as well as their perception of gaining useful experiences with collaboration and innovative problem solving. Similarly, the qualitative findings of the analysis indicate how the students had to think creatively in order to re‐frame and reconstruct specific pedagogical problems into workable game design solutions. Due to their limited time frame, lacking conceptual knowledge, inexperience with collaborative game design processes, unclear expectations, and unfamiliarity among other group members, we observed how several of the students faced difficulties when trying to come up with “the right solution”. Still, it was clear from the group interviews how the collaborative design and problem‐solving processes had strengthened the students’ self‐efficacy, especially in relation to their school visits and their ownership of having created concrete game designs that were presented to local school staff and pupils. Based upon these indicators, we assume that the students were able to develop innovation competence during the course of the camp, which involves a strengthened outset for future enterprising behavior.

6.2 Problem‐based game design as pedagogical method The second issue to be discussed concerns the use of problem‐based game design (PBGD) as a pedagogical method. In this study, the teacher educators’ aim with using the PBGD approach for the intro camp was to introduce the new students to specific problems stated by local schools, while simultaneously requiring the

221

Thorkild Hanghøj and Sia Hovmand Sørensen new students to connect different domains (their everyday knowledge of games and pedagogy, their new identity as students, and their future as professional practitioners) and come up with “useful” game design solutions. Based upon the overall positive feedback from the participating students, teacher educators, and local schools, we argue that the pedagogical use of problem‐based game design has a potential for developing innovation competence, which reaches beyond the scope of an intro camp and also beyond the context of teacher education. At the same time, our observations and group interviews also made it clear how the facilitation of the PBGD method could be improved in several ways, which we will present as a series of recommendations Clarify goals, resources, and assessment criteria Our study indicates how the PBGD method is relatively demanding from a facilitator’s perspective in terms of setting up collaboration with external actors (e.g. schools), but especially demanding for participants, who must often conceptualize and design their game designs solutions within a limited time frame. In this way, the learning goals, available resources and assessment criteria for the game designs needs to be carefully clarified and communicated to the participants.

Select real‐life problems with relevant “gameness” Similarly, using PBGD as a pedagogical method requires that facilitators conduct a careful review of the “gameness” of the real‐life problems posed by external actors. Some problems are more relevant than others for being re‐framed as games – e.g. it can be quite difficult to design a safe and playable learning game required to specifically target problems related to a marginalized or unpopular group of pupils at a particular school. Moreover, it is important that students are given sufficient information that enables them to make relevant and meaningful games ‐ e.g. by asking schools not only to state problems, but also to describe typical scenarios and demands related to the stated problems. Ensure sufficient conceptualization of assigned problems Our study shows how the PBGD method may benefit from more facilitation of collaborative group processes and stronger scaffolding of the students’ conceptualization and re‐framing of problems into possible game design solutions. Thus, it is important that students are not “left on their own” and end up reproducing their commonsensical perceptions of complex concepts such as citizenship or democracy. Instead, they need guidance in order to re‐frame core concepts in relation to the problems they are asked to solve. This guidance will also support the development of innovation competence as conceptualization is a key component in innovation Support students’ choice and design of game format As indicated above, the use of the PBGD method could benefit from more guidance on how different game formats may support different types of learning processes. This process may challenge the participants’ prior game experiences and broaden their repertoire of possible game solutions (i.e. board games, roleplaying games, debate games, pervasive games, video games etc.). Ultimately, students should be able to argue why they choose a particular game format to facilitate learning processes related to their assigned problems.

Provide detailed feedback The intro camp described here clearly lacked a follow up that extended beyond the brief feedback given on game designs of the winning group from each track at the end of the camp. Thus, there was no focused attempt to evaluate the students’ game designs and let them reflect on their design processes, which would likely have helped them understand how their proposed game designs might (or might not) be able to solve their assigned real‐life problems ‐ i.e. how does a proposed quiz game reflect a particular conceptualization of citizenship education, and how does that game design match (or mismatch) the local problems with citizenship education identified by a particular school? In this way, more detailed feedback on the chosen game design could have benefited the students’ development of “game literacies” and a reflection on their design processes could have benefited their development of innovation competence.

222

Thorkild Hanghøj and Sia Hovmand Sørensen

6.3 Validity of the findings As a final point for discussion, the validity of our findings are limited by the fact that we only followed one group closely through the camp, and we only conducted two group interviews. There were relatively large differences between the camp experiences of the two groups. In this way, we cannot assume that the knowledge created on the basis of the group we observed is internally generalizable in a direct way (Onwuegbuzie and Johnsson 2006). Based upon our pragmatist theory of science, we believe that the knowledge created through our observations is recognizable and usable in relation to understanding other groups’ work processes. Still, the difference between the groups’ camp experiences suggest the need for further qualitative studies in order to provide more detailed maps of how participants enact, perceive and reflect upon problem‐based game design processes. Similarly, our study is only based on one intro camp, which means that we cannot infer that using problem‐based game design within the context of an intro camp will necessarily lead to the development of innovation competence. Finally, our empirical focus on the groups’ work processes instead on their actual game designs makes it impossible for us to determine to what degree the students’ development of innovation competence might be correlated to the quality of their actual game designs. This relationship between students’ problem‐based game design processes and actual game designs will be subject to further study.

7. Conclusion Following our analysis and discussion, we can conclude that new teacher students may indeed be able to develop innovation competence by participating in the problem‐based game design processes of an intro camp. In this way, we hope that our paper may contribute to a more empirically grounded discussion on the challenges and possibilities of developing innovation competence within teacher education. Moreover, the use of problem‐based game design as a pedagogical method is still a rather unexplored field of research. Thus, we hope the that our paper will contribute to further studies on how to facilitate Problem‐Based Game Design processes within teacher education ‐ and other educational contexts.

References Agresti, A. (2002). Categorical data analysis, New York, John Wiley and Sons Ltd. Bager, T. (2011). “The camp m del for entrepreneurship teaching”. International entrepreneurship and management journal, Vol 7, No. 2, pp. 279‐296. Bandura, A. (1997). Self‐efficacy. The exercise of control, New York, W.H. Freeman and Company. Biesta, G. and Burbules, N. (2003). Pragmatism and Educational Research, Lanham, Rowman & Littlefield. Burn, A. (2007). “Writing Computer Games: Game Literacy and New‐Old Narratives.” L1 – Educational Studies in Language and Literature, Vol 7, No. 4, pp. 45‐67. Darsø, L. (2001). Innovation in the making, København, Samfundslitteratur. Dewey, J. (1916 [1980]). Democracy and Education. An Introduction to the Philosophy of Education. J. A. Boydston (ed.). John Dewey. The Middle Works (Vol. 9). Carbondale, Southern Illinois University Press. Dewey, J. (1933 [1986]). How We Think. A restatement of the relation of reflective thinking to the educative process. J. A. Boydston (ed.). John Dewey. The Later Works (Vol. 8), Carbondale, Southern Illinois University Press Fayolle, A. and Gailly, B. (2008). “From craft to science. Teaching models and learning processes in entrepreneurship education.” Journal of European industrial training, Vol 32, No. 7, pp.569‐593. Goffman, E. (1974). Frame Analysis: An Essay on the Organization of Experience, New York, Harper & Row. Hanghøj, T. (2011). Playful Knowledge. An explorative study of educational gaming, Saarbrücken, LAMBERT Academic Publishing. Hanghøj, T. (2013). “Game‐Based Teaching: Practices, Roles, and Pedagogies”. In de Freitas, S. Ott, Popescu M. M. and Stanescu, I. (eds.). New Pedagogical Approaches in Game Enhanced Learning: Curriculum Integration, Hershey, IGI Global. Johnson, R. B. and Onwuegbuzie, A. J. (2004). “Mixed methods research: A research paradigm whose time has come”. Educational Researcher, Vol 33, No. 7, pp. 14‐26. Kaufman, J. C. and Sternberg, R. J. (2010). The Cambridge Handbook of Creativity, New York, Cambridge University Press. Khalil, E. (2003). “A transactional view of entrepreneurship: a Deweyan approach.” Journal of Economic Methodology, Vol 10, No. 2, pp. 161‐179. Kirketerp, A.L. (2010). Pædagogik og didaktik i entreprenørskabsundervisningen på de videregående uddannelser i et foretagsomhedsperspektiv, PhD Dissertation, University of Southern Denmark. Kvale, S. and Brinkmann, S. (2008). InterViews. Second Edition: Learning the Craft of Qualitative Research Interviewing. (2nd ed.), Thousand Oaks, Sage Publications. Leech, N. L. and Onwuegbuzie, A. J. (2009). A typology of mixed methods research designs. Quality and Quantity, Vol 43, No. 2, pp. 265‐275.

223

Thorkild Hanghøj and Sia Hovmand Sørensen Onwuegbuzie, A. J. and Johnsson, R. B. (2006). “The validity issue in mixed research.” Research in schools, Vol 13, No. 1, pp. 48‐63. Peppler, K. and Kafai, Y. (2010). “Gaming Fluencies: Pathways into Participatory Culture in a Community Design Studio”. International Journal of Learning and Media, Vol 1, No. 4, pp. 45‐58. Rychen, D. S. and Salganik, L. H. (2003). Key Competencies for a Successful Life and a Well‐Functioning Society, Göttingen, Hogrefe & Huber Publishers. Salen, K. (2007). “Gaming literacies: A game design study in action.” Journal of Educational Multimedia and Hypermedia, Vol 16, No. 3, pp. 301‐322. Scheidel, T.M. (1986). “Divergent and convergent thinking.” In Poole, M. S. and Hirokawa, R. Y. (eds.). Communication and group decision making, London, Sage Publications. Schön, D. (1978). “Generative Metaphor: A Perspective on Problem Setting in Social Policy”. In Ortony, A. (ed.), Metaphor and Thought, Cambridge, Cambridge University Press. Spradley, J. P. (1980). Participant observation, Belmont, Thomson Learning.

224

Deploying Serious Games for Management in Higher Education: Lessons Learned and Good Practices Jannicke Baalsrud Hauge1, Francesco Bellotti2, Rob Nadolski3, Michael Kickmeier‐Rust4, Riccardo Berta2 and Maira Carvalho2 1 Bremer Institut für Produktion und Logistik at the University of Bremen,Bremen, Germany 2 Dept of Electronic and Telecommunication Engineering, University of Genoa,Genoa, Italy 3 Centre for Learning Sciences and Technologies, Open University Netherlands, The Netherlands 4 Knowledge Management Institute, Graz University of Technology,Graz, Austria baa@biba.uni‐bremen.de franz@elios.unige.it rob.nadolski@ou.nl michael.kickmeier‐rust@tugraz.at berta@elios.unige.it Maira.Carvalho@elios.unige.it Abstract: The deployment rate of serious games (SGs) in higher education (HE) and their proper insertion in meaningful curricula isstill quite low. There is a lack of papers in literature describing deployment of SGs for HE in detail, critically showing educational benefits, and providing guidelines and best practices on their use. With the present work, we intend to make a first step in this direction, by reporting our experience in using state of the art managerial SGs in MSc Engineering/business courses in four different European universities. In order to describe and analyse the educational characteristics and effectiveness of each game, we propose to use two models that we have straightforwardly extracted from two major pedagogical paradigms: the Bloom’s revised cognitive learning goals taxonomy and the Kolb’s experiential learning cycle. Based on our experience in developing the SG‐based courses, we also propose a set of lessons and practices that we believe could be of interest to incentivize and better support deployment of SGs in HE courses. Keywords: case studies, assessment, selection methods for serious games

1. Introduction While there is a certain consensus about the educational potential of Serious Games (SGs) (Bellotti et al., 2010; Greitzer et al., 2007; De Gloria et al, 2012) in higher education (HE), the deployment rate of SG in HE and their proper insertion in meaningful curricula are still quite low. This is generally attributed to an undefinedteacher’s reluctance towards the use of games. However, there is also a lack of papers in literature describing deployment of SGs for HE in detail, critically showing their educational benefits and providing guidelines and practices on their use, in comparison with other educational tools/techniques. With the present paper, we intend to make a first step in the direction of better characterization of the effectiveness and the use of SGs in HE, by reporting our experience in using managerial SGs at different European universities, namely: Genoa (Italy), Bremen (Germany), Nottingham (UK) and Open University of The Netherlands. In particular we describe the deployment of three games, selected because of their quality and ability to cover the course’s managerial topics, that are being used in MSc courses in different engineering areas (civil, electronic and industrial). In order to describe and analyse the educational characteristics and effectiveness of each game, we propose the use of models that we have straightforwardly extracted from the major pedagogical paradigms. Several pedagogical theories and learning models have been employed to inspire SG design and to assess validity of SGs. Among the knowledge models we highlight the Nonaka SECI model (Nonaka et al., 2000; Nonaka, 1994) is mentioned as a theoretical basis for the use of SG‐based workshops, at least in the fields of business, management and manufacturing (Anghern and Maxwell, 2009), and Kirkpatrick’s “The Four Levels of Learning Evaluation”, which is a popular learning impact assessment model, involving the following levels: reaction, learning, behaviour, results (Kirkpatrick, 1998). A fifth level of evaluation has been added in new

225

Jannicke Baalsrud Hauge et al. versions of the model by (Phillips, 2007) and by (Watkins et al., 1998), considering also return on investment and impact on clients and society, respectively. In our work we have focused in particular on describing SGs through two models that we consider complementary, simple and particularly useful to analyse SGs: the Revised Bloom Taxonomy, which is the most popular cognitive approach to SG evaluation (Luccini et al., 2012) (Table 1); and the Kolb’s Experiential Learning model (Figure 1), which systemizes the work rooted on Piaget’s cognitive developmental genetic epistemology (Piaget, 1929), on Dewey’s philosophical pragmatism (Dewey, 1933), and on Lewin’s social psychology, putting the experience at the centre of the learning process. Table 1: Original and revised Bloom taxonomies

Figure 1: Kolb’s learning cycle

2. Case studies This section describes three case studies of serious games that the authors deployed in higher education contexts in four countries, namely The Netherlands, Italy, UK and Germany.

2.1 Estuarine systems: The Scheldt The Scheldt is a web‐based, role‐playing, single‐user game developed via the EMERGO methodology and toolkit (Nadolski et al. 2008). EMERGO‐games are developed in such a way that the user‐interface can be easily replaced without changing game‐structure or game‐content. Content resources are also separated from the game‐structure. This enables easy maintenance and supports sustainability. The learning objective is to analyze, understand and explain the problem of the soil‐water systems in the Scheldt (see Figure 2). This concerns a natural science approach towards the threats to our society, and complex spatial and temporal interactions between soil and water.

Figure 2a): Googlemaps ‐tool within the game b): Consulting resources while playing During gameplay, the student takes the role of a junior researcher‐trainee at a virtual company. He receives tasks and feedback from a senior researcher (embedded NPC) during the analysis of increasingly more complex problems and must propose/find workable solutions. He may use web‐based tools / GIS sites, multi‐various data and models to work towards his solution. This occurs by watching the phenomena and visually inspecting the area (e.g., video, satellite images, GIS sites) in order to solve the question on "Why is land reclamation or loss necessary from a scientific point of view?" Tasks and feedback are given via company‐mail, or via video. Feedback can consist of completed examples and discussion. Students need to compare their own solution these; such as such tasks don't have unequivocal

226

Jannicke Baalsrud Hauge et al. solutions. Then, based on examples, a consecutive task will be given. In other tasks, feedback is given in a very natural way (for example, reactions from NPC's when consulted during task execution). The Scheldt (0,7 ECTS) has been embedded in a distance learning course on soil and water (4.3 ECTS) since 2010 at the Open University Netherlands (OUNL). There has been no needed revision of the game. The Scheldt is meant for independent self‐study, so there are hardly any restrictions concerning the number of enrolled students. However, normally 30‐50 students are enrolled every year. The central theme for the game case is "a field study focused on research and exploration of the Scheldt estuary towards relevance for naturalness, accessibility and security." This case concerns a step‐wise approach towards the solution of the question "Why is land reclamation or loss necessary from a scientific point of view?" The case is highly realistic and centres on authentic tasks. Support for Bloom’s cognitive learning goals Analyzing the EMERGO game, we can see that it supports several learning goals, as reported in the following table. Table 2: Bloom’s cognitive learning goals covered by the Scheldt

Support for Kolb’s learning stages Although Kolb was not explicitly used in the design phase of the game, EMERGO targets thinking and doing, which leads to concrete outcomes and as such conforms to the Kolb cycle. Table 3: Kolb’s learning cycle the Scheldt

Support for soft skills As anticipated, the game supports several soft‐skills aspects such as problem solving, strategic thinking, meta‐ learning.

227

Jannicke Baalsrud Hauge et al.

2.2 Any business: A highly configurable online multiplayer business simulation GoVenture Any Business (http://goventureanybusiness.com) is an instructor‐customizable business simulation platform that can be used to simulate virtually any type of business, within any industry and any market. It is playable both individually and in teams. The game objective is to successfully manage a business while competing with other companies, managed by other players or by the computer. The Simulation Manager (usually an instructor) has a lot of freedom to configure the simulation, creating scenarios that can range from very simple to very complex/difficult, The Simulation Manager is also able to model specific events or situations to target specific learning goals. The gameplay consists of making business decisions, which means setting several parameters – price, product features, marketing, human resources, business ethics, among others – before the deadline of each period of the simulation. After the deadline, the simulation advances to the next period and the game presents the results of the previous decisions in terms of sales and profits. A performance score is provided as a weighted sum of different dimensions, and the instructor receives a detailed report with all the activities performed by the students. Every simulation is different (e.g., economic and market conditions), which makes performance not perfectly comparable, but allows for more engaging challenges. Teams compete against each other in the same settings, as in a strategy game, and computer‐driven competitors are also generated, creating a good model of the market.

a) b)

Figure 3: The strategy journal, where the player can set and review his strategy’s parameters (a), and the performance report (b) GoVenture Any Business is one of the serious games being used in the second edition of the course on “Entrepreneurship through Serious Games” (eSG) at the University of Genoa, Italy, for the Electronic Engineering M.Sc. degree. The course ‐ which is presently in progress ‐ aims to stimulate entrepreneurship in university students, especially future information technology engineers with little previous academic instruction in economics. The 3 ECTS course includes a series of lectures/workshops that introduce the theoretical foundations of entrepreneurship, discuss case studies and present the main features of the serious games that are used in the course. The games are played in teams as part of each week assignments, and in addition students are required to fill in questionnaires about the game and the concepts presented in the lectures. By the end of the course, students will have played a total of seven different simulations in GoVenture Any Business. The students’ actual performance in the games is considered for the final course grade. Support for Bloom’s cognitive learning goals Any Business shows a good capability for covering all the levels of the Bloom’s taxonomy (Table 4). Support for Kolb’s learning stages The authors do not know whether the game was designed on accordance with Kolb’s cycle. However, it is quite well supported, as shown in the following table

228

Jannicke Baalsrud Hauge et al. Table 4: Bloom’s cognitive learning goals covered by Any Business

Table 5: Kolb’s learning cycle Any Business

Support for soft skills The game supports mainly strategic thinking and decision making. In cases where the game is played in teams, it also supports interpersonal relations, as the decisions made must first be negotiated among all team members, who may also specialize and consider different aspects of company management (e.g., human resources, finance, etc.).

2.3 Seconds: A role playing game to improve decision making skills The game Seconds, developed at the University of Bremen, is used to train students in decision making on supply chains (SC) and in distributed production environment. It is a facilitated multi‐player, role‐based, online game. The game creates a safe learning environment in which the students can apply different approaches for improving the flexibility and efficiency of manufacturing and analyse the impact on the SC. It is configurable, and the level depends on the knowledge level of the player (pre‐configured). The goal is adaptable (depending on course setting), but is mostly used to produce a specific product in cooperation, while, taking all costs and expenditures into account. A simplified accounting system is implemented, i.e. the game delivers several performance indicators that are used for the analysis and calculation. The gaming scenario evolves as the players play the game. Depending on production volume and time, the player can gain experience and skills needed for producing higher quality. Target users are master students from industrial, production and system engineering and MSc logistics and operational management.

229

Jannicke Baalsrud Hauge et al.

Figure 3: GUI shows the input needed for producing robots, as well as all processes at the site in Bremen, dept. of production The game is used at the University of Bremen as part of a 3 ECTS lab course on “Decision making in distributed production environment”, which uses a blended learning concept. This part is comprised of 6 units‐ one for introduction to the basics of SCM and a tutorial on the gaming environment and five for playing. Methods for strategic decision making are successively introduced into the course. Each session lasts 5 hours. On average, the play time is 3‐3,5 hours for each session and at least 30‐45’ for debriefing and reflection. For two years, Seconds has also been used at the University of Nottingham. There, it is used as a supplement to a post graduate course on Supply Chain management. Therefore, there is no introduction to strategic decision making in supply chain or in the basics of SCM, since this is knowledge already known to the students, i.e. the students come solely to play the game. So far, the students had have played twice. The sessions have taken around 2,5 hours with an additional 30 minutes for debriefing. In this case, the students received predefined scenarios with all company processes already implemented; i.e. the degree of freedom for taking decision on production sites etc. was lower than in the German case Support for Bloom’s cognitive learning goals Actually, in the design phase of Seconds, Blooms taxonomy was hardly considered. Consequently only the higher levels are supported. This is typical for this type of game; it emphasizes on the two highest levels of evaluating and creating. Table 6: Bloom’s cognitive learning goals covered by seconds

230

Jannicke Baalsrud Hauge et al. Support for Kolb’s learning cycle The use of Seconds uses an extension of Kolb’s learning cycle: it uses the BIG (beyond the information given) defined by Perkins (1991, p. 20) BIG constructivism. Following the BIG approach, a facilitator directly introduces the concepts, provides examples to the students with concrete experience in activities that challenge them to apply, generalise and refine their initial understanding in multiple activities. This approach presents information to the learners but stresses the need to go beyond the information given. Table 7: Support of Kolb’s learning cycle due to the BIG approach

3. Lessons learned and good practices Quantitative results from the deployment of the games are not yet available from each game. However, the experience gained from the field allows us to make some considerations that we believe could be useful for educators. In general, in order to guarantee successful deployment it is important to carefully align gaming goals with course goals and course assessment (i.e., constructive alignment). Deploying a new game is a complex and time‐consuming activity that ideally requires the development of an ad‐hoc deployment plan, specifying goals (educational and in‐game) and context of use. Also student feedback should be carefully considered, in order to tune the game in terms of contents, difficulty levels, pace, etc. Fine‐tuning the parameters for the games and deciding how much playing will be employed in a course can be a challenge. It is difficult to organize a sequence of game matches/sessions that continuously engage the students, while representing a proper educational path usable during a whole course, or part of it. Also, the number and duration of the sessions is relevant to the manner of interaction between students, especially regarding collaboration. For example, in the game Seconds, it was observed that the students’ willingness to make compromises/trade‐offs and to make strategic collaborations is higher, on average, in the groups having five sessions than those having two (thus showing a collaboration learning effect). For competitive games, the teacher should thus support the weaker teams, in order to enhance the overall competitiveness, whereas for a collaborative game setting, it is more important that the game environment is able to support different competence levels within the same gaming scenario. In facilitated games, the facilitator’s role is essential. Documentation should be easily accessible online, in particular during the game and also the game developer support should be available, at least in the phases of course design and early deployment. While the term SG is appealing, in particular for students, state of the art SGs have generally limited entertainment features, especially if compared with the best selling videogames. Tools like AnyBusiness are frequently referred to as a business simulations, without specific serious games mechanics (i.e., able to join fun and instruction). However, inter‐team competition even through the simple mechanics of score and other performance indicatos (e.g., cashflow, profit and loss, etc.) are an excellent motivator.

231

Jannicke Baalsrud Hauge et al. SGs for complex scenarios should typically be used in blended learning settings, with briefing and debriefing sessions in order to complement and reflect on the experience, possibly also with questionnaires. Games designed to be facilitated should not be played in single mode or without facilitator, since intra‐team relationships are very useful and the overview of an expert is very important both for the contents and for the game procedures themselves. In addition to the time factor of each game session, another critical factor concerns the instructions given to the students before and during the game (both concerning the contents and the game itself). It is possible to allow the students to freely manage their roles. However – this is a common rule in education ‐ freedom should be limited for students having less knowledge about the target domain, in order to make the learning process more efficient and to help them overcome hurdles, concerning both the playability and the contents. Moreover, in‐game knowledge (typically procedural and intuitive) should be complemented with other type of information, typically verbal and objective. The facilitator or the teacher should pay attention at the students’ learning outcomes after (and possibly also during) the game, in order to detect misconceptions, that are likely to appear, according to our experience, given the students’ procedural and empirical approach. A crucial step when preparing a course exploiting SGs is the actual choice of the games. The first step involves the collection of requirements related to the course and the curriculum. Addressed items include: target group, credits, learning objectives, which skills and competences should be trained, connection to the overall curriculum, underlying technical infrastructure, course setting, embedding with other learning material, use of blended learning concepts or not, number and length of units, feedback and assessment needs, pre‐requisites (compare Nadolski, 2008). The candidate games’ features will need to be analysed in the light of the above mentioned requirements. Typical criteria for selection include various factors, such as: coverage of the needed educational topics; matching between the course’s learning objectives and the game’s features; costs (both in terms of software and of deployment and of maintenance); usability; quality of user assessment and provision of feedback (Bellotti et al., 2013); game adaptability; knowledge transferability; in house competencies and time availability in case development of a new serious game was considered; degree of freedom for players and teachers; support to collaboration; SG’s learning curve, difficulty level and long‐term playability; competences and effort needed on the teacher’s side; availability of additional educational material related to the game. A main challenge in the selection process is the difficulty in having a critical and complete overview of existing games. Depending on the weight of each one of the above criteria, existing games can be matched, and a “make or buy” decision may also be done. In the second case some of the collected requirements may need to be adapted. In the case studies presented in this paper, different needs led to different choices. In the first case (the Scheldt), the game had perfectly to comply with distance education and should partly serve as a replacement of fieldwork. Consequently, an ad‐hoc development was deemed as necessary. The second case study (Any Business), a comparative analysis of several state of the art games was carried out, based on the main criteria of entrepreneurship topic coverage and per‐license costs lower than €30. For the third case study (Seconds), a review was performed of the few available games on supply chain management and of several simulation environments as well. Since simulations were considered very good for mapping the real world processes, excessively difficult for the students, and the games not complex enough, in‐house development was decided, allowing also the implementation of collaborative features, mirroring the work carried out in the production network.

4. Conclusions and future work All the three studied games typically focus on the higher levels of the revised Bloom’s taxonomy (analysing, evaluating and creating), in different ways and to different extents. The Scheldt emphasises the analysing level, whereas Any Business the evaluating level. Seconds focuses on supporting students to learn to create new knowledge. These differences substantially reflect the targets of the games and the corresponding courses.

232

Jannicke Baalsrud Hauge et al. Our observations (though still qualitative) highlight that this has a strong relationship with the role of the teacher/instructor/facilitator. The higher level to be achieved according to Bloom, the more emphasis has to be put on supporting the abstraction process, which typically requires the teacher’s intervention. This process is only partly supported in the three games and related documentation, so it is very important that the teacher designs a proper pedagogical plan to make the game experience profitable from a real learning point of view. Also during the lectures, the presence of a teacher is very important, introducing and explaining topics and giving indications and discussing the experience with students. In Seconds, furthermore, the gameplay itself is accompanied by a teacher, while Any Business has been played at home by the competing teams and feedback has been provided by the teachers during the debriefing discussion. In The Scheldt, in‐game feedback is provided by an NPC player, which aims at nurturing the thinking process, even if with a lower quality than through a human teacher. This game is made for an online university course, and thus it is also only offered in single user mode. We do not agree that new education practices should turn the teacher from a “teaching machine” into a consultant nor a simple facilitator. We believe that SGs are powerful and complex tools that need the adult expertise and competences. Overall, the adult’s presence is necessary for the educational role of leading students to knowledge and understanding of/access to reality. A proper use of SGs, instead of limiting the teacher’s role, requires even better prepared teachers, and able to introduce students to aspects of reality by using a potentially powerful simulation tool. In the Any Business course, lectures involved the presence of three other researchers for supporting the official teacher by monitoring the teams’ behaviours during the de‐ briefing (probably one would have been enough, but we preferred two given the experimental case). Our analysis has shown that all the three games seem to correspond to the Kolb’s learning model, even though the model had not been explicitly taken into consideration at the design level. Whereas The Scheldt emphasises concrete experience and reflective observation, the other two games, at least in the multi‐player settings with debriefing, seem to be more focused on an abstract conceptualisation. Regarding the effectiveness of the games, it can be reported that the students who had used The Scheldt received high grades on the final exam that was conducted after completing the game (average score 8.9 (maximum 10)). For Any Business, the preliminary results after four game sessions of one week each show that the game supports strategic thinking and requires and stimulates a deep understanding of the simulation environment. It can also be seen that the different reports generated by the game help in analysing and taking decision. Moreover, competition is really compelling for the majority of the players. The market conditions are determined by the abilities of the competing teams. Seconds has been used for decision making for more than 6 years. The available results mainly show that the game helps the students in applying methods and constructing new knowledge, and support strategically thinking. Our experience in deploying the targeted games indicates some important reasons for the current low penetration rate of GBL in HE. The game that seems to be most easily to integrate is The Scheldt, for which only few adjustments were necessary. For the other courses that involved the presence of one or more teachers, it can be concluded that it is difficult to set up a course integrating the use of a SG, since the playing time, the length and plan of the sessions, the modalities for keeping the motivation etc. are difficult to estimate in advance. Thus, these courses often have to undergo an iterative design process, adapting the course set‐up depending on the evaluation of the learning outcomes. This requires a continuously monitoring and a proper working experience. Taken into account the cost of this work, the experimental design, and the low number of students in each class it is still a question if the resources used for implementing GBL are efficiently used. Despite the consensus on their potential (in particular due to the technological/graphic appeal, interactivity and the huge data processing/storing capabilities), the deployment rate of SG in HE is still quite low. We argue that this is due to the fact that games are more naturally suited to children than to adults. Moreover, educational effectiveness of games is easier to achieve with simpler content, while more complex and costly games are necessary in order to efficiently and effectively achieve the needed educational targets. Finally, integration in existing curricula is not straightforward and requires a careful pedagogical planning and a smart usage of games. So, we think that a lot of work is still to be done, in particular to understand how to use it and how to design its insertion in the course so that it is really effective for students.

233

Jannicke Baalsrud Hauge et al.

Acknowledgements This work was partially funded by the European Union, under the Framework Programme 7 (Information Society Technologies ‐ ICT), in the Games and Learning Alliance (GaLA) Network of Excellence, Grant Agreement nr. 258169. This work was supported in part by the Erasmus Mundus Joint Doctorate in Interactive and Cognitive Environments, which is funded by the EACEA Agency of the European Commission under EMJD ICE FPA n. 2010‐0012

References Anderson, L W., Krathwohl, D. R. Eds. (2001) A Taxonomy for Learning, Teaching and Assessment: A Revision of Bloom's Taxonomy of Educational Objectives. New York: Longman. Anghern, Albert A. and Maxwell, Katrina (2009) EagleRacing: Addressing Corporate Collaboration Challenges Through an Online Simulation Game, Innovate, Journal of Online Education, Vol. 5, Iss. 6. Baalsrud Hauge, J., Delhoum, S., Scholz‐Reiter, B., Thoben, Klaus‐Dieter (2007) The Evaluation of Learning the Task of Inventory Control with a Learning Lab. In: Learning with Games., Sofia Antipolis, France. pp. 1‐8. Bellotti F., Berta R. and De Gloria A. (2010) Designing Effective Serious Games: Opportunities and Challenges for Research, Special Issue: Creative Learning with Serious Games, Int.l Journal of Emerging Technologies in Learning (IJET), Vol. 5, pp. 22‐35 Bellotti F., R. Berta, De Gloria A., D'Ursi A., and Fiore V. (2012). A serious game model for cultural heritage. ACM J. Comput. Cult. Herit. 5, 4. Bellotti F., Kapralos B., Lee K., Moreno‐Ger P., and Berta R. (2013), “Assessment in and of Serious Games: An Overview”, Hindawi Advances in Human‐Computer Interaction Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R.(1956). Taxonomy of educational objectives: the classification of educational goals, Handbook I: Cognitive Domain New York, Longmans, Green. th De Gloria, A. Bellotti, F., Berta,R (2012) Building a Comprehensive R&D Community on Serious Games, 4 Int.l Conference on Games and Virtual Worlds for Serious Applications, VS‐Games 2012, Genova. Dewey, J. (1933) “How We Think”, New York: Heath. Greitzer F.L., Kuchar O.A., and Huston K. (2007), Cognitive Science Implications for Enhancing Training Effectiveness in a Serious Gaming Context, ACM J. Educational Resources in Computing, vol. 7, no. 3 Kirkpatrick, D. L. (1998) “Evaluating Training Programs: The Four Levels”. 2nd Edition, Berrett‐Koehler Publishers, Inc, San Francisco. Kolb, D. A. (1984) Experiential Learning, Englewood Cliffs, NJ.: Prentice Hall, pp. 20‐38, retrieved at http://academic.regis.edu/ed205/Kolb.pdf on 30 September 2012 Luccini, M, A. M. Luccini, M. Mortara, C. E. Catalano, M. Romero (2012) Thematic Application fields report, deliverable 3.2 of the Games and Learning (GaLA) Network of Excellence. M Mantakas, (2010) Using a management game within an entrepreneurship training project for the students of an informatics and telecommunications degree, Proceedings of the SGEED‐2010 Conference Serious Games, Education & Economic Development http://www.serious‐gaming.info/@api/deki/files/144/=12_Mantakas_business_game.pdf Nadolski, R. J., Hummel, H. G. K., Van den Brink, H. J, Hoefakker, R., Slootmaker, A., Kurvers, H., & Storm,J. (2008) EMERGO: methodology and toolkit for efficient development of serious games in higher education, Simulations & Gaming, 39(3), pp. 338‐352. Nonaka, I., Toyama, R. and Konno, N. (2000). “SECI, Ba, and leadership: a unified model of dynamic knowledge creation”. Long Range Planning, 33, 5‐ 34. Perkins, D. N.: Technology meets constructivism: Do they make a marriage?, in Phillips, J. J. (2007). “Measuring ROI – Fact, Fad, or Fantasy”. ASTD White Paper, retrieved at http://www.ddiworld.com/DDIWorld/media/articles/Best‐of‐TD‐Measuring‐and‐Evaluating‐Learning.pdf Piaget, J. (1929) “The Child's Conception of the World”. NY: Harcourt, Brace Jovanovich Riedel, J. and K. Pawar (2009) A Report On The Experiences Gained From Evaluating The Cosiga NPD Simulation Game, IPDMC09, International Product Development Management Conference, 7‐8, Twente, The Netherlands, CD‐Rom, EIASM, Brussels Watkins, R., Leigh, D., Foshay, R. and Kaufman, R. (1998). Kirkpatrick Plus: Evaluation and Continuous Improvement with a Community Focus, Educational Technology Research & Development, 46(4): 90‐96

234

Neuroeducational Research in the Design and use of Games‐Based Teaching Wayne Holmes1, 2, Paul Howard‐Jones3, Erico Tanimoto3, Carol Jones4, Skevi Demetriou3, Owen Morgan4, Philip Perkins5 and Neil Davies5 1 Department of Education, University of Oxford, UK 2 zondle, UK 3 Graduate School of Education, University of Bristol, UK 4 Chepstow School, Monmouthshire, UK 5 Duffryn Comprehensive School, Newport, UK wayne.holmes@education.ox.ac.uk paul.howard‐jones@bristol.ac.uk Abstract: Research has shown that a games‐based approach to learning can have many positive effects in the classroom, although less attention has been paid to the potential of applying a games‐based approach to teaching. Meanwhile, recent research into the brain’s reward system has provided fresh understanding about the educational potential of games and associated underlying cognitive and neural processes. However, the harnessing of neuroscientific understanding for educational benefit presents many challenges, not least because it potentially impacts on pedagogical theory as well as technological design, with outcomes in the classroom likely to depend on a successful interaction of both. The effective design and implementation of games‐based teaching might thus require a judicious interrelation of insights from diverse theoretical perspectives, such as neuroscientific, pedagogical and classroom praxis. Here we report on the design‐based research of a web app, known as zondle Team Play (zTP), that allows teachers to use a games‐based approach to teaching whole‐classes and which draws on learning theory, the practicalities of classrooms, and concepts from neuroscience. zTP was developed iteratively with teachers, in five cycles of design, intervention, analysis and reflection. Rather than just exploring ‘what works’ in terms of the technology, iterative prototyping helped us explore aspects of classroom praxis and affordances of the technological design that were contingent upon each other. Reflection revealed many potential benefits of a neuroeducational approach to the design of a teaching game, including the development of related pedagogy, identification of immediate and future neuroeducational research questions and the development of language and terms suitable for communicating across interdisciplinary boundaries. Keywords: teaching; games; neuroscience; motivation; rewards; pedagogy

1. Introduction With the rise in popularity of mobile games on technologies such as smart phones and tablet computers, the use of digital games and games‐based approaches to support learning has recently gained new prominence (Perrotta et al. 2013; Richards et al. 2013). Much of this interest focuses on the positive impact of games on student motivation and engagement (Obama 2011), for which various mechanisms have been proposed (Garris et al. 2002). However, discussions around games‐based learning now need to move beyond motivation and engagement to examine its mechanisms for and impact on learning, for which research grounded in neuroscience might prove useful (Howard‐Jones et al. 2011; Whitton 2007). Given that the potential of digital games‐based learning was first demonstrated more than thirty years ago (Malone 1981), and has been shown to have positive educational effects (Connolly et al. 2012), another important question is why it is still not being used extensively in most classrooms (Kenny & McDaniel 2011). The slow uptake might be because suitable games‐based learning is not often available. Alternatively, it might be because of the attitudes of teachers towards the use of games to support learning (Bourgonjon et al. 2013). It might also be because of the relative lack of individual access to appropriate technologies in many classrooms. The wide availability of interactive whiteboards (Hennessy 2011), on the other hand, raises the possibility that a digital game‐based approach to whole‐class teaching, which offers teachers more direct control, might be more widely taken up and might prove useful in schools (Grady et al. 2013; Jackson 2009). The evidence suggests that the development of effective games‐based teaching would require a judicious interrelation of insights from diverse theoretical perspectives: games‐based learning, pedagogical, classroom praxis, and neuroscientific (Howard‐Jones & Demetriou 2009). Accordingly, here we report on the design‐ based research of a web app, known as zondle Team Play (zTP), that enables teachers to teach whole‐classes

235

Wayne Holmes et al. using a games‐based approach. zTP was developed iteratively with teachers, in five cycles of design, intervention, analysis and reflection. The design involved a multi‐disciplinary team and drew on learning theory, the practicalities of classrooms, and concepts from neuroscience.

2. The neuroscience of learning games Fresh insight regarding the brain’s reward system provides a rudimentary basis for understanding the engagement provided by games. Our motivation to win points in a game generates signals in the brain’s reward system that are similar to those produced by our attraction to many other pleasures such as food (Koepp et al. 1998). This activity involves uptake of the neurotransmitter dopamine and is termed ‘dopaminergic activity’. A brief dopamine ‘spike’ will be generated simply by the awareness that a reward will certainly be provided (Figure 1a) or when a totally unexpected one is received (Figure 1b). However, with the awareness that an uncertain reward may be provided (i.e. when uncertainty exists about whether a reward will be received or not), there is a brief spike plus an additional ramping up of dopamine until the outcome is known (Figure 1c) (Fiorillo et al. 2003). Overall, uncertain rewards result in more dopamine being released, peaking when the likelihood of receiving a reward is 50%, providing an explanation for our attraction to games involving chance (Shizgal and Arvanitogiannis 2003).

Figure 1: Uptake of the neurotransmitter dopamine generated in response to the probability (P) of receiving a reward The attraction to uncertain reward appears to dissipate in schools where greater certainty of success is sought. The level of certainty preferred by learners in school has been measured at 88% (Clifford and Chou 1991) and this high figure is probably due to the implications of failure for self‐ and social‐esteem. This environmental diminishment of the ‘comfort zone’ for uncertainty can be expected to influence learning. Working at high levels of certainty may avoid the stronger motivational signals associated with dopaminergic activity. Emotional response during learning tasks has been found to increase when these tasks are integrated into a chance‐based game (Howard‐Jones and Demetriou 2009). Emotional response is known to support memory encoding (LaBar and Cabeza 2006), so we might also expect experiences involving less emotional response to also be less memorable. Combining learning with games of chance offers a way of increasing reward signals and the emotional content of learning, without threatening esteem. There are many examples in sport and in everyday life when success arises from a combination of ability and chance, and well‐matched competition (i.e. with around 50% likelihood of outcome, such as a football game) provides a highly engaging challenge. Children, especially boys, appear to prefer the inclusion of gameplay uncertainty in learning tasks (Howard‐Jones and Demetriou 2009). Importantly for education, a positive relationship between reward activity in the brain and memory formation has been demonstrated. In an educational learning game, dopaminergic activity due to gameplay rewards was estimated (based on the extent of expected gain) for each round. This signal predicted the success of memory recall more effectively than the size of the reward itself (Howard‐Jones et al. 2009). A previous classroom study has also shown that mediating rewards for learning with chance‐based events can affect the discourse around learning in positive ways (Howard‐Jones and Demetriou 2009). It tends to encourage open motivational talk and allows students to introduce a self‐serving bias that attributes failure to

236

Wayne Holmes et al. chance and success to ability. The following key points arise from the neuroscientific and neuroeducational research for educational practice with learning games:

Learning games can engage students through inclusion of chance‐based components that increase the uncertainty of rewards for learning.

The brain’s response to rewards can be very brief. That suggests a close intermingling of learning and gameplay elements is needed, if the engagement created by the gameplay is to support the learning.

Anticipation of an uncertain reward is likely to generate a more extended ‘window of enhanced attention’.

Avoiding a loss does not generate the same reward signals as a gain, suggesting a generally positive scoring system may support motivation.

3. Method The ‘translation’ of neuroscientific understanding to the classroom is clearly fraught with potential dangers, such as unscientific interpretation of neuroscientific concepts or departure from a grounded educational understanding (Geake 2008). To help minimise these dangers, the development of pedagogy or classroom interventions that draw on neuroscience require co‐construction of concepts by a team possessing expertise in both areas. Accordingly, the research team comprised academic researchers (who between them have experience in neuroscience research, psychology, education and games‐based learning) and teachers from two comprehensive schools in South Wales (hereafter referred to as School A and School B). The team used a design‐based research approach, which simultaneously pursues practical innovation and theory building by means of the iterative development of solutions in real world situations (Design‐Based Research Collective 2003), to investigate the design and use of a teachers’ tool grounded in neuroscience. The science of learning games outlined above was a touchstone for the first iteration of what eventually became the app known as zTP. After this, reflection on observations and outcomes in the classroom were the driving forces for developing both the further design of the app and good practice. Rather than a prescription for classroom practice, concepts about the brain provided a useful starting point for innovation and a helpful framework for stimulating reflection and understanding. The research comprised five cycles of design, intervention, analysis and reflection (the final cycle is ongoing). In each intervention, rather than comparing the effectiveness of the intervention with non‐gameplay approaches, we identified instances of apparent learning gain as critical examples that could inform discussions and reflection about pedagogy and subsequent cycles. In the first and fourth interventions, the quantifying of learning gains took the form of a written pre‐ and post‐test. In the second and third interventions, when we were focusing on a group with low literacy ability, we used a non‐written measure. The fifth iteration is currently the focus of an fMRI study. Two digital video cameras (facing class and teacher) recorded the interventions, the video recordings being used as a basis for subsequent discussion and group analysis. Informed consent was given by the parents of all participating students.

4. Results 4.1 Research cycle 1 4.1.1 Design Based on the science of learning games outlined above, the team developed a low‐fidelity game using Microsoft PowerPoint. The presentation comprised a repeating pattern of 1‐2 slides of content, on the topic of ‘reproduction’, followed by 1‐2 slides of multiple choice questions that assessed knowledge of this content in which each answer was labelled with one of four colours. A simple ‘student response system’ was also developed, comprising sets of four 15 cm square coloured cards (the same colours as those used to label the potential answers), hinged with tape, one set for each student. 4.1.2 Intervention The first intervention took place in School A, with 25 students in a Year 7 science class (13 males, 12 females: mean age 11 years 6 months). This initial intervention was exploratory in nature, and the assessed learning objectives focused on the acquisition of knowledge rather than understanding. For around ten minutes, the

237

Wayne Holmes et al. teacher taught an aspect of the topic ‘reproduction’ using the PowerPoint slides to structure and illustrate. They then revealed one of the multiple‐choice questions. To respond, the students had to choose an answer and note its colour on the slide, fold their squares so that that colour faced frontwards, and place the folded squares in a stand. The procedure discouraged cheating by making it awkward for the students to change their response once the answer had been announced. This pattern of teaching and gameplay was repeated until the end of the lesson. This approach to using multiple‐choice questions in whole‐class teaching was so far conventional. However, to enhance both motivation and learning, chance‐based uncertainty, as suggested by the neuroscience, was introduced to mediate the receipt of rewards. Each correct answer was rewarded with the option to receive a point, represented by a counter, or to take a chance and receive either zero or two points based on the spin of a ‘wheel of fortune’. This became known as the students choosing whether or not to ‘game their score’. In addition, over the lesson, the stakes for each question were gradually raised. As well as maintaining motivation, this made the final outcome even less predictable, since later rounds had more influence on scores than earlier ones. Other types of rounds, such as ‘bonus rounds’, were also used but because of space constraints these will not be considered here. 4.1.3 Analysis and reflection It was clear that the lesson generated intense levels of engagement, particularly amongst boys. However, this first session emphasised how engagement does not necessarily translate into learning. Mean scores out of a possible 14 marks for pre‐ and post‐tests were 4.6 (SD 2.4) and 5.8 (SD 3.0) (a Wilcoxon non‐parametric signed ranks test revealed this modest improvement was statistically significant: z = ‐2.82, p = 0.005, r = ‐0.40). Notable moments of engagement occurred when the correct answer was about to be announced and the wheel of fortune was turning, since this was when the students found out whether they would gain some points: in other words, the engagement of the students was mostly on the game rather than the learning content. In addition, the teacher tended to indicate the correct answer first, such that the potential ramping up of attention was not being fully exploited for learning. Accordingly, it was clear that, if the engagement fostered by the gameplay was to be of educational value, greater effort had to be made to ensure the learning content was closely associated with the gameplay. Although least successful as an intervention, a great deal was learned from this session. For example, this type of gameplay adds to the demands on the teacher, requiring them to divide attention between game hosting and teaching and this can require practice. The usual scaffolding strategies can easily be forgotten when presenting content (e.g. checking understanding through verbal exchanges or providing hints that focus minds on relevant content). If not harnessed correctly, the excitement of the game can distract students and teacher from the learning rather than help them engage with it. For example, students and sometimes the teacher used the colours in their dialogue rather than the learning concepts (“Why is the answer blue not red?” rather than “Why is the answer ‘gametes’ not ‘sperm’?”).

4.2 Research cycle 2 4.2.1 Design In the second iteration of the design, the content of the PowerPoint slides focused on the understanding of the grammatical concepts of noun, pronoun, verb and tense. In other respects, the game followed the pattern of the first intervention, with 3‐4 slides of content followed by 1‐2 slides of questions. The student response system and wheel of fortune were as before. Three outcomes from the first iteration were explained to the teacher, for them to apply:

students should be given support that furthers their understanding of the learning content when they are answering the questions;

attention should be drawn to incorrect answers, not their colours, before revealing the correct answer; and

238

Wayne Holmes et al.

potential problems should be discussed as the answers are being revealed (e.g. “If this was your answer, you may have forgotten that...”), so that those who answer incorrectly can receive additional instruction during this brief window of heightened engagement.

4.2.2 Intervention This second intervention took place in School B, with student participants belonging to a Year 9 group receiving additional support for literacy. This low‐literacy group, which consisted of 12 students (8 males and 4 females: mean age 13 years and 7 months) answered a pre‐test of five questions at the beginning of the game that allowed them to communicate responses using the coloured cards. This data was used to identify specific instances of apparent learning gain, when consistent correct responses were later offered on a type of question that students had answered incorrectly in this pre‐test. The teacher was an experienced teacher of literacy for the school. 4.2.3 Analysis and reflection The mean of pre‐test learning scores was 53%, and mean scores during the game (which used questions of similar type to the pre‐test) was 65%. Although statistical analysis was inappropriate (the sample size was small and the answering of questions was occasionally supported by the teacher), the data helped identify instances of apparent learning gain, with a clear example of four students who failed some pre‐test questions but answered correctly similar post‐test questions. The discourse that appeared to prompt this learning highlighted how the additional engagement provided by the game can be used as an opportunity to scaffold students’ learning. There were also several instances of the teacher checking understanding and praising it, and of students being supported as they were answering questions. Generally, however, the teacher was disappointed at not being able to apply consistently the principles given to them from the first iteration. Although an experienced teacher, the game format of the lesson made additional demands on her management and thinking processes and took up time (e.g. giving out counters and moving back and forth between whiteboard and wheel of fortune). Nonetheless, the students were highly engaged by the game format throughout. Chance‐based outcomes generated emotional teacher‐student empathy whatever the outcome, suggesting games can change the emotional content of teacher‐student exchanges. When outcomes arise through chance, the teacher can acknowledge failure as expressively and as strongly as success. This may make for a more authentic sharing of emotions than afforded by the conventional focus on the positive.

4.3 Research cycle 3 4.3.1 Design The third iteration focused on reducing the demands on the teacher by automating part of the game. A purpose‐built macro for PowerPoint was developed (Figure 2). While students still gave their responses using coloured cards, the macro allowed the teacher to record on‐screen the students’ responses to questions and their decisions whether or not to game their score. It also included an automatic flashing light version of the wheel of chance and automatically calculated the scores. 4.3.2 Intervention The third intervention involved the same participants as the second intervention, studying another set of grammatical concepts: adjectives, adverbs, capitals and full stops in sentences, commas and speech marks. This time, however, participating students were divided into pairs and competed as six teams, in order to give an opportunity for additional collaborative dialogue to support learning. 4.3.3 Analysis and reflection The mean of pre‐test learning scores across teams was 39%, and the mean of scores achieved during the game was 80%. Using the technology, which removed the need to manage counters, allowed around four times as many student‐teacher interactions of which a higher proportion were related to the learning content. Initially,

239

Wayne Holmes et al. students responded in brief to teacher questions. After around ten minutes, exchanges grew in duration and complexity, concepts and principles were discussed, and students offered unprompted examples and asked questions to verify understanding. The teacher was much more positive following this intervention. She felt she had been able to focus more on teaching, and was delighted with the level of engagement that the game had created. In addition, students showed signs of independent thinking about the principles, and the discussion became more spontaneous as students made unprompted contributions. Meanwhile, revealing the incorrect answers first and explaining why they were incorrect grabbed the attention of students very effectively. The intense engagement created by this strategy was very evident for one student who became increasingly animated as the teacher scrutinised each incorrect option in turn, building up the tension until the correct answer was finally revealed.

Figure 2: Screen shot of PowerPoint with additional interactivity for gameplay provided by a small macro programme

4.4 Research cycle 4 4.4.1 Design The fourth iteration used the PowerPoint macro and coloured cards developed previously. To keep the research domain‐neutral, a new topic area was chosen: the evaluation of plastic products. As described below, the sequence of the intervention was also slightly modified. 4.4.2 Intervention The sample was a mixed‐ability Year 10 Design and Technology group (in School A) comprising nine students (mean age 15 years 7 months; all males). As the teacher was confident that the students would benefit from greater opportunities for independent learning, the approach was slightly revised. General concepts were presented at the beginning of the lesson and discussed with the students. The group then used notes provided by the teacher to support them in making joint decisions in four ‘teams’ (comprising four pairs of students and one student supported by a classroom assistant). After twelve consecutive game rounds, notes were removed and they faced another twelve rounds. In this intervention the game was used almost exclusively to achieve formative assessment and learning. Incorrect responses to the questions were used to identify potential issues with understanding and these prompted additional explanations and teacher‐student discourse to scaffold learning. This change in strategy was a response to the needs of the particular learning context rather in response to new insights about teaching with games. However, it did draw attention to how a game‐based approach to teaching, like other types of teaching, is situated in contextual issues such as group dynamics, ability, level, and topic. The pre‐ and post‐test consisted of a different set of nine questions, each asking students to choose an appropriate type of plastic with which to manufacture a specified product. Four of the post‐test questions included products covered in the session and could be answered from memory, the other five were novel products requiring the students to recall and correctly apply the principles introduced.

240

Wayne Holmes et al. 4.4.3 Analysis and reflection Mean scores for pre‐ and post‐tests were 1.3 (SD 0.8) and 5.2 (SD 2.4) out of a possible nine. Students also achieved a mean score of 3.2 (SD 1.3) out of the five post‐test questions that involved novel problems. Despite the small sample size, a Wilcoxon signed ranks test confirmed the improvement as statistically significant (z = ‐ 2.49, p = 0.013, r = ‐0.89). By comparing outcomes as percentage scores, the pre‐ to post‐test difference remained significant when only the five novel problems in the post‐test were considered (z = ‐2.55, p = 0.011, r = ‐0.90). In addition, in the opinion of the teacher, good levels of understanding were achieved by the group. Students’ talk included a small number of queries to the teacher, publicly expressed gameplay talk (boasting, teasing and joking) and many furtive utterances as they quietly conferred with their partner. The audible conferring was chiefly about learning content and gameplay strategy. Often, during these exchanges, students maintained their visual attention on the teacher and question, as if trying to conceal their conversation from the rest of the class. When announcing answers, the teacher revealed incorrect answers first to exploit the window of attention created by anticipation. Both quiet conferring and public exclamations indicated close attendance to this information. There were several occasions when the teacher’s talk slipped into something resembling that of a game‐show host. This appeared aimed at generating more excitement, working up the emotions of the players, sometimes goading, sometimes a voice of caution or comfort. As observed in previous studies, those who were not in the lead towards the end of the lesson took all opportunities to game their scores, as their chances to win without doing so dwindled. Other strategies included teams avoiding giving away answers by hiding their response until the last minute, by not putting it up or covering it with their bag etc. Some went as far as beginning with an answer they knew was wrong and encouraging others to see it, before changing it at the last moment to their chosen response. This prompted the researchers to consider the potential benefits of an electronic response system instead of the coloured cards. This would allow responses to be covert until all students had committed themselves, so preventing plagiarism. It would also reduce the time taken for responses to be collected and liberate the teacher from some of the administrative tasks that teaching with the game involves, allowing them to focus their attention more on the teaching.

4.5 Research cycle 5 The fifth iteration of this game‐based approach to teaching involved the collaborative design of a web app, known as ‘zondle Team Play’ (zTP), in association with the developers of a games‐based learning platform (www.zondle.com). zTP (Figure 3) was itself designed iteratively: it was based on the low‐fidelity versions discussed above and scaffolded by a series of conversations between this paper’s lead authors and the developer (details of the app are available on the developers’ website and in Howard‐Jones and Fenton 2012).

Figure 3: The main screen from the app ‘zondle Team Play’, showing a question about the Tudors

241

Wayne Holmes et al. zTP, which is freely available on the developer’s website, was designed to be used on any interactive whiteboard (or with a computer and projector) that has Internet access. Teachers can import their PowerPoint slides into the system and can write appropriate multiple‐choice questions. Alternatively, they can use and, if they choose, amend or mash together any of the more than 12,000 zTP topics written by other teachers and currently on the system. The app provides a way to allocate answers to teams, automatically allocates and records points, and includes a wheel of chance that can be started by a child swiping the interactive whiteboard. Finally, students can interact directly with the app by using a mobile device or computer with Internet access, enabling students in different locations to compete in a single zTP session. Anecdotal evidence for the playability, practicality and effectiveness has so far been positive. However, zTP is currently being researched in an fMRI study which will be reported later.

5. Discussion The understanding and practice of teaching with games developed during the project in a number of key ways. For example, from a classroom praxis perspective, we learned that teaching with games adds considerably to the administrative burden on the teacher but this burden can be reduced by well‐designed technology. Meanwhile, emerging insights from neuroscience regarding the brain’s reward system suggested the following features for teaching games:

an embedding of many chance‐based components that increase the uncertainty of rewards received for learning;

a tight coupling of components of learning and gameplay;

generation of “windows of attention” (e.g. when anticipating uncertain rewards) that can be pedagogically exploited to support learning.

Neuroscience provides a useful source of insight, but our study also emphasised the role of the teacher in implementing successful whole‐class teaching games. Thoughtful pedagogy and mediation of classroom discourse is as essential for teaching with games as for any other teaching approach. Indeed, successful teaching with games appears characterised by several familiar aspects of good teaching, including the monitoring of understanding, use of appropriate praise, and the scaffolding of learning. However, teaching with games adds new dimensions to the role of the teacher, who must identify and generate windows of attention using the game, and exploit these as opportunities for learning. The teacher may play the role of game‐show host, as they stimulate emotional excitement around gameplay events and empathise with the students’ fortunes and misfortunes. This design‐based research project did not set out to evaluate the general educational value of the teaching game, nor does it make claims about the efficacy of this teaching game compared with any other types of teaching. Instead, the research was about the iterative design of a teaching game that draws on insights from neuroscientific research and about concepts for good practice using that game in a number of contexts, so as to inform the design of future teaching games. There are many questions requiring further research, such as how the game approach might work in the longer‐term and how suitable it may be for different contexts (such as ability, age groups, topics, and gender). However, in all interventions we observed very high levels of student engagement and definite learning, and each iteration of the research cycles generated new insights about how to focus the engagement on the learning. For these reasons, the authors of this report are optimistic that teaching with games can significantly contribute to learning in a variety of different contexts. This potential may be realised as pedagogy in this area develops, and as we learn more about the neurocognitive processes involved.

References Bourgonjon, J., Grove, F.D., Smet, C.D., Van Looy, J., Soetaert, R. & Valcke, M., 2013. Acceptance of Game‐Based Learning by Secondary School Teachers. Computers & Education, 67, pp.21–35. Clifford, M.M. and Chou, F.‐C. (1991). Effects of Payoff and Task Context on Academic Risk Taking. Journal of Educational Psychology, 83(4), pp.499–507. Connolly, T.M., Boyle, E.A., MacArthur, E., Hainey, T. & Boyle, J., 2012. A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59, pp.661–686. Design‐Based Research Collective, 2003. Design‐Based Research: An Emerging Paradigm for Educational Inquiry. Educational Researcher, 32(1), pp.5–8. Fiorillo, C.D., Tobler, P.N. and Schultz, W. (2003) Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons. Science, 299(5614), pp.1898–1902.

242

Wayne Holmes et al. Garris, R., Ahlers, R. and Driskell, J.E. (2002) Games, Motivation, and Learning: A Research and Practice Model. Simulation & Gaming, 33(4), pp.441–467. Geake, J. (2008) Neuromythologies in education. Educational Research, 50(2), pp.123–133. Gee, J.P. (2004) What Video Games Have to Teach Us About Learning and Literacy, New York: Palgrave Macmillan. Grady, S.E., Vest, K.M. & Todd, T.J., 2013. Student attitudes toward the use of games to promote learning in the large classroom setting. Currents in Pharmacy Teaching and Learning. Available at: http://linkinghub.elsevier.com/retrieve/pii/S1877129713000154 [Accessed April 13, 2013]. Hennessy, S., 2011. The role of digital artefacts on the interactive whiteboard in supporting classroom dialogue. Journal of Computer Assisted Learning, 27(6), pp.463–489. Howard‐Jones, P. and Demetriou, S. (2009) Uncertainty and engagement with learning games. Instructional Science, 37(6), pp.519–536. Howard‐Jones, P. and Fenton, K. (2012) Brains, Minds and Teaching with Immersive Games, Neuroeducational Services. Available at: http://www.lulu.com [Accessed April 21, 2013]. Howard‐Jones, P., Demetriou, S., Bogacz, R., Yoo, J.H. & Leonards, U., 2011. Toward a science of learning games. Mind, Brain, and Education, 5(1), pp.33–41. Howard‐Jones, P.A., Bogacz, R., Demetriou, S., Leonards, U. and Yoo, J. (2009) From gaming to learning: A reward‐based model of decision‐making predicts declarative memory performance in a learning game. British Psychological Society Annual Conference 2009. Brighton. Jackson, J. (2009). Game‐based teaching: what educators can learn from videogames. Teaching Education, 20(3), p.291. Kenny, R.F. & McDaniel, R., 2011. The role teachers’ expectations and value assessments of video games play in their adopting and integrating them into their classrooms. British Journal of Educational Technology, 42(2), pp.197–213. Koepp, M.J., Gunn, R.N., Lawrence, A.D., Cunningham, V.J., Dagher, A., Jones, T., Brooks, D.J., Bench, C.J. and Grasby, P.M. (1998) Evidence for striatal dopamine release during a video game. Nature, 393(6682), pp.266–268. LaBar, K.S. and Cabeza, R. (2006) Cognitive neuroscience of emotional memory. Nature Reviews Neuroscience, 7(1), pp.54– 64. Malone, T.W., 1981. Towards a Theory of Intrinsically Motivating Instruction. Cognitive Science, 4, pp.333–369. Obama, B., 2011. A Moral and Economic Imperative to Give Every Child the Chance to Succeed. Available at: http://www.whitehouse.gov/blog/2011/03/08/president‐obama‐talks‐education‐boston‐moral‐and‐economic‐ imperative‐give‐every‐child [Accessed May 20, 2013]. Perrotta, C., Featherstone, G., Aston, H. and Houghton, E. (2013) Game‐based learning: latest evidence and future directions, Slough: NFER. Richards, J., Stebbins, L. & Moellering, K., 2013. Games for a digital age: K‐12 market map and investment analysis, New York: The Joan Ganz Cooney Center at Sesame Workshop. Available at: http://www.joanganzcooneycenter.org/wp‐ content/uploads/2013/01/glpc_gamesforadigitalage1.pdf. Shizgal, P. and Arvanitogiannis, A. (2003). Gambling on dopamine. Science Signaling, 299(5614), p.1856. Whitton, N., 2007. Motivation and computer game based learning. In Proceedings of ASCILITE Singapore 2007. ASCILITE. Singapore, pp. 1063–1067.

243

Playing and Learning: An iPad Game Development Case Study Jennifer Jenson and Rachel Muehrer York University, Canada jjjenson@gmail.com

Abstract: While there seems to be a great deal of enthusiasm for the use of games in formal educational contexts, there is a notable and problematic lack of studies that make use of replicable study designs to empirically link games to learning (Young, et al., 2012). Where such studies exist, the multimodal literacies that games might cultivate are often misrepresented and/or obscured through conventional, text‐focused modes of evaluation. This study considers Compareware, an educational game (available in Flash and on the iPad) designed to build vocabulary and analytical skills in young learners. This paper documents the design, iterative development, user testing, and pilot study that included 146 grade 1 and 2 students playing the game. It also outlines our preliminary findings, which include high levels of student engagement, incidental learning, and improved demonstration of analytical skills in terms of identifying and understanding similarities and differences between two objects. Keywords: educational game design, play, learning

1. Introduction At the time of this writing (June 2013), there is still an enormous amount of hype being generated on the potential educational uses of iPads. They are devices that are being purchased at a rapid rate for classrooms around the world 1 , with very little empirical evidence to support either their educational uses or their educational potential generally (Peluso, 2012). For example, research to date on the use of iPads in the classroom can be characterized as both excessively hopeful (Henderson & Yeow, 2012) and primarily anecdotal (Dickens & Churches, 2011), as well as limited in scope (sometimes with just one or two participants c.f. Kagohara, Sigafoos, Achmadi, O’Reilly & Lancioni, 2012; McClanahan, 2012). While there are a few formal studies of iPads and their use in elementary education, those tend to be primarily exploratory (Preciado‐Babb, 2012; Dickens & Churches, 2011), with few decisive conclusions draw on whether or not they are either supporting educational ends and/or educating users in some way. This paper documents the design, development, user testing and pilot study of an educational game designed for the iPad IOS operating system, 2 Compareware . Compareware asks its players to choose examine two pictures that are set side by side and choose vocabulary that either indicates similarities or difference. For example, how is a picture of a tiger and the picture of a zebra different and how are they the same? It is targeted at ages 5‐8, both readers and non‐readers (there is voiceover support for those who cannot read), and is meant to scaffold and support players as they analyze two object for similarities and differences. Our intent in designing the game was to create an iPad game/”app” that could be used in an elementary classroom, that was first and foremost design for educational ends and that supported a fundamental attribute associated with higher order thinking skills in both adults and children, that is the ability to ascertain how two objects are similar (and by implication how they are different). This work grew out of a 3‐year long study of multimodal, multilingual education in an elementary school in the greater Toronto area. One small part of that study was to film children in their first language (Spanish, Italian, Tagalog, Mandarin, Cantonese, among 18 other languages spoken in the school) as participants interacted with a teacher or their parents and a series of objects. They were asked to describe the similarities and differences between two objects using a kind of ‘talk aloud’ protocol that included, often code‐switching on the part of participants and their parents as parents assisted them in answering how the objects were similar and/or different. What is salient to this study is the fact that we noted time and time again that participants often had true difficulty in articulating how something was similar – either because they could not describe how 1

A school district in San Diego, for example, purchased 26,000 iPads (http://www.tuaw.com/2012/06/26/san‐diego‐school‐district‐ purchases‐26‐000‐ipads/), another district in Michigan purchased them for all 1,800 high school students (http://www.padgadget.com/2011/09/21/one‐michigan‐school‐district‐gives‐every‐high‐school‐student‐an‐ipad/), a school district in B.C. spent $32,000 on iPads (http://blogs.vancouversun.com/2012/11/23/langley‐school‐district‐explains‐32000‐on‐ipads‐for‐trustees/) and a school district in Victoria, Australia is piloting an iPad program in K‐12 education (http://www.ipadsforeducation.vic.edu.au/ipad‐student‐ trial), among many others. 2 Compareware is playfully named after the popular Warioware franchise which is a series of quick minigames that are played in succession. Compareware is essentially a series of mini games that are played quickly and in succession.

244

Jennifer Jenson and Rachel Muehrer something was functionally similar (i.e. two very different pairs of shoes are similar because they “protect the feet” and/or “are used for walking” and/or “are used to run”) or because they had real difficulty relating the degree to which the two objects were similar (i.e. they are both for walking but one is for winter and the other summer). While degree of similarity seemed to be less difficult than articulate than the category or quality of the similarity (color, shape, size, form, function, etc.), it still was the case that we observed, especially among those aged 5‐8, some difficulties in mobilizing both the vocabulary and by implication the analytical skills necessary to articulate how two objects could be described as similar. So Compareware was an attempt to see if we could design a game to scaffold learners in artifact classification in terms of similarities of differences. In the next section, we briefly review literature to date on the use of games in education, especially focusing our attention on four recent reviews of literature in the area. We then overview research which has examined the categorization of artifacts with a view to showing how this game fits broadly within that research and as a way of contextualizing the pilot study design. Following those reviews, we briefly describe the game’s iterative design process, and then detail our pilot study methodology and some preliminary results.

2. Games and education: Less hype, more evidence Many have weighted in on the potential of games as potential sites of and for learning (Gee, 2003, 2005; Prensky, 2001; Squire, 2011) however, much of that early work was more polemical than empirical. In a recent review of the literature on digital games and education, which included examining studies of games that were built for educational purposes as well as commercial off the shelf games (COTS) mobilized for educational ends, Young, et al. (2012) quip: “After initial analyses, we determined that, to date, there is limited evidence to suggest how educational games can be used to solve the problems inherent in the structure of traditional K‐12 schooling and academia. Indeed, if you are looking for data to support that argument, then we are sorry, but your princess is in another castle” (p. 62). Their argument is that educational research needs better methodologies for studying games, including the use of software to track player behavior in game, as well as an accounting of individual play styles and characteristics. Tobias and Fletcher (2012) respond to the critique of the field of education and games offered in the Young et al. (2012) piece, arguing that they had not examined “transfer” in games – that is how a player might transfer a cognitive ability acquired in game to one outside the game (c.f. A. F. Anderson & Bavelier, 2011; Green & Bavelier, 2003). They also show that there is some difference between their own literature review earlier (Tobias, et al., 2011) and Young, et al. (2012), arguing that it is difficult to map the field when it is changing so rapidly, and positing the need for the development of a taxonomy for games that will allow for better clarity. What these meta‐reviews (and others (Fletcher & Tobias, 2006; Ke, 2009; Sitzmann, 2011) point to is what other researchers have been discussing for sometime with regards to game‐based learning (GBL): the fact that quite often it is not clear if games are effective learning tools. Some studies, for example, have found very little in terms of learning from playing games (Ke, 2008; Papastergiou, 2009; Tsai, Yu, & Hsiao, 2012), while others have show the opposite, that games can be effective sites for learning (Barab, et al., 2009; Fletcher & Tobias, 2012; Hsu & Wang, 2010). For the purpose of this paper, what is significant from the metareviews of games and education is that we acknowledge that this field is very much ‘emerging’ as indeed are its methods and questions. We therefore situate this work as an educational game, designed ‘in house,’ and, as we detail in the next section for a very particular purpose. This is in line with other GBL projects, that are designed, developed and tested, including, for example: the development of a road safety game (All, et al., 2013), a game for health (Jenson, Taylor & de Castell, 2011), a game for saving electricity (Tsai, Yu, & Hsiao, 2012), and a game to encourage empathy (Bachen, Hernández‐Ramos, & Raphael, 2012). We then play tested the game in multiple schools with students aged 6‐8, however, our questions did not focus on the ‘benefits’ of using iPads as a mode of delivery for an educational game. Instead, we situate our questions for this paper within GBL framework, asking what (if anything) do students learn from playing Compareware, and what might be some effective means of measuring that? Before turning to the design of the game and the methods used in the pilot study, we also situate this work within the literature on artifact categorization (similarities and differences).

3. Artifactual categorization: A brief overview The robust literature on artifact categorization in children and adults most typically divides that intellectual effort between a child’s apprehension of physical similarities (shape, size, color, etc.) and its function, arguing that the latter is a kind of ‘deeper’ understanding than the former (Bloom 1996; 2000). However, this research has been, for the most part, contradictory. For example, studies of children as young as 5 have shown that

245

Jennifer Jenson and Rachel Muehrer children attribute labels of physical similarities to objects at the expense of functional similarities (Graham, Williams, & Huber, 1999; Landau, Smith, & Jones, 1998; Merriman, Scott, & Marazita, 1993; Smith, Jones, & Landau, 1996), while other studies with children as young as 2 have shown the opposite (functional similarity is prioritized over physical similarity (Deak, Ray, & Pick, 2002; Diesendruck, Markson, & Bloom, 2003). However, most research does tend to show that preschool children are more likely to base their categorization on physical appearance rather than on function (Gentner & Rattermann, 1991; Woodward & Markman, 1998). Diesendruck, Hammer & Catz (2003) overview some of the methodological inconsistencies that might have produced these very different outcomes and argue that at least in their study “when functional and appearance information about artifacts are simultaneously available to children for the same length of time, through the same medium, and without adult direction, children weigh these two respects equally and highly” (Diesendruck, et al., 2003, p. 229). For our purposes, this is significant as the game does not need adult direction and it keeps players in the same medium. What is clear is that there are a number of confounding factors that have not yet necessarily been resolved in research to date on categorization. In order to foreground this as work in which the consensus is that “it depends” regarding whether young children are more likely to prioritize physical dimensions over an artifact’s function, it is the case that more studies have concluded that the physical can have more weight than function (Kelmer Nelson, Frankenfield, Morris & Blair, 2000). Because this study is carried out on an iPad and computer and because it involves playing a game rather than being directed to talk and/or write out similarities between objects, we are pointing to the extant literature in the field that this study is thematically related to, but was not designed to follow the experimental designs that are typical in this field of study. That is to say, the modality of presentation of the objects is wholly different (playing a game rather than paper and pencil activities) and therefore it falls outside the literature reviewed here.

4. Compareware: Design and process The title of the game plays on a title of a Nintendo DS game “WarioWare” in which the player creates their own minigames through a series of visual programming choices made possible through the game’s interface. Wario is a popular Mario character who has his own franchise of fast‐paced mini games. Compareware invites players to compare two objects of increasing difficulty and in later levels under time constraints. The game takes place in an environment that is graphically very bright and is divided into six thematic areas: school, home, ocean, grocery, town and outdoors. Players enter the game and are presented with two objects and ask “How are they the same?” in one instance and “How are they different?” in another. The images are randomly assigned and a set of six answers scrolls through the bottom of the screen, which the players must drag to the appropriate spot between the two images. The answers are in text and read to players if they so choose, supporting those who might not be as strong at reading. There are also multiple levels in the game, with progress being marked by advancing to the unlockable content as players win levels. They also receive instant feedback on whether or not they have chosen the correct answer, and are only penalized by the game restarting if they appear to be randomly dragging and dropping answers.

Figure 1: Title screen of the game Compareware

246

Jennifer Jenson and Rachel Muehrer

Figure 2: Home screen

Figure 3: Cornerstore vs. Diner, in Level 1 of "Town"

Figure 4: "Pie vs Bread" in Level 1 of "Grocery"

247

Jennifer Jenson and Rachel Muehrer

Figure 5: "Cruise ship vs cargo ship" in Level 2 of "Ocean" Answers are recorded on the iPad as the player progresses, and they are awarded a star system based on the number of correct answers in a given series. Those stars can then be used to decorate the night sky in the central, ‘main screen’ of the game. We also track the correct and incorrect answers in the game for each unique iPad user, and are able to track which set of images and which particular vocabulary is most often incorrectly chosen in each area of the game. The game was designed in 4 months, with rapid prototyping of 3 playable levels that were designed and play tested within the first 6 weeks of the project. Following that first round of play testing, voice over sound was added for all vocabulary present in the game, we removed time constraints in the early levels all together and we created a way for users to customize (turn on/off) both sound and time constraints. We also altered the graphical interfaces for the drag and drop vocabulary based on user feedback that it should stylistically “match” the associated area of the game – e.g. in the ocean section, the drag and drop phrase or words are in a fish (see Figure 5) while in the grocery section they are conveyed in a shopping basket (see Figure 4). Debugging of the game was extensive and time consuming and it took another 2 months post development in its first iteration, then another 6 weeks following the pilot study as a number of glitches (as expected) were evidenced when multiple users played the game. In addition, it became clear that some alteration of both of the questions and the answers was necessary, and that was commenced

5. Pilot study: Methods The purpose of this study was to document how students learned after completing tasks from the game Compareware in three different modalities: 1) in an iOS platform (iPad); 2) in Flash (on a PC in a computer lab); 3) on pen and paper. We used a mixed‐methods approach to evaluate Compareware in the classroom. The researchers collected qualitative data with audio‐video recordings and fieldnotes. In the quantitative portion of the study, students were given a questionnaire on their media and videogame habits as well as a pre‐test and post‐test that contained images from the game, used the same vocabulary and asked that they write about similarities and differences among objects. When possible we also used the games’ recording of accurate and inaccurate responses (this didn’t always work due to technical difficulties). The pilot study took place in 4 schools in 9 classrooms of 18–25 students aged 6‐11. In total 146 participated in the study of with kindergarten and grade 1 reading abilities. A quasi‐experimental design was followed for this study, with participants randomly assigned to three groups that completed four tasks in a different order, to determine whether. Students completed each of the following tasks in a 40 minute session: 1) experimental time on the iPad; 2) Compareware on the iPad; 3) a Compareware activity on pen and paper and 4) Compareware on Flash. In the first session each group took the pre‐quiz and were given one of the four activities. In the second, third and fourth sessions the students completed each of the other three tasks. On

248

Jennifer Jenson and Rachel Muehrer the final day students also completed the post‐quiz (identical to the pre‐quiz). In addition to the post‐quiz and qualitative data, metrics were collected in the game. The purpose of having students participate in activities in a different order was to examine game metrics and determine whether students became better at the game if they played in a different order. However, because the game metrics reset themselves on the iPad each time a new student logged her or himself in, there was no way to retrieve metrics from the iPad games. Similarly, in the school computer lab, each student had their own lab account and the game metrics were saved in these accounts. Because of quick time to change between users, there was not enough time to retrieve metrics from each computer before the next class arrived. Therefore, none of the game metrics were usable, and there was no way to assess how students were performing during the game.

6. Findings We hypothesized that students would have difficulty listing similarities between objects before they began playing Compareware. We also hypothesized that students would improve their ability to articulate both similarities and differences between objects after playing the game. Students had almost the same scores naming similarities (a mean of 3.8) and difficulties (mean of 3.7) on the pretest, although they were slightly more successful at naming similarities. 55% of the students increased their scores from the pretest to the post‐ test after participating in the activities. There was no significant difference in pre to post‐test scores among the three groups. Finally, the scores on the post‐test showed a good distribution of scores statistically, indicating that our instrument was the appropriate difficulty level for the participants. The Pen and Paper Activity proved to provide a detailed catalogue of which questions students answered correctly and incorrectly during their participation. Although this information could not be compared to game metrics from the flash and iPad game as intended, it painted a clear picture of how students might have misinterpreted questions. Because the worksheet that they were provided with was static, students could work at their most comfortable pace, and as a result often vocalized their thought processes. For example, one of the questions had a picture of a Polar Bear and a Black Bear. Students could indicate whether the characteristic “bear” was a similarity or a difference by circling their choice. One student reasoned that a Polar Bear and a Black Bear are the same because they are both bears. Another came to the opposite conclusion, circling bear as a difference because they are different kinds of bears. This process very quickly shed light on the way students might interpret questions, and we were able to flag any questions that might be confusing for this reason and worked to change them.

7. Redesign The pilot study was most useful as a tool to strengthen the study design and to streamline the game play so that students were encouraged to continue to play more challenging levels. After each question and the vocabulary were revised to minimize confusion, we realized that the gameplay needed more direction. Initially students had been given a choice of a variety of topics from a home menu, but they had no indication of where to start, how many levels were in each topic, or how many questions they had remaining. In order to give players a more clear idea of the structure of the game, we added a serious of progress bars and screens with detailed directions, and locked the hardest level so that players were required to successfully complete most of the game before they could move on to the most challenging of the questions. Finally, we found that students (especially on the iPad) were simply performing the motions of play by dragging and dropping answers over and over again, or by simply dragging and dropping answers randomly rather than attempting to correctly answer the question. We decided to interrupt dragging the same wrong answer three times with a screen that encourages players to try a new answer rather than continuing a mindless drag‐and‐drop.

8. Conclusions The pilot study was invaluable to strengthen the study design and to streamline the game play so that students were encouraged to continue to play more challenging levels. For example, after questions and in game vocabulary were revised to minimize confusion, we realized that students needed more direction in order to navigate through the game. Initially students had been given a choice of a variety of topics from a home menu, but there had been no indication of where to start, how many levels were in each topic, or how many questions they had remaining. In order to give players a clearer idea of the structure of the game, we also added a series of progress bars and screens with detailed directions, and locked the hardest level so that

249

Jennifer Jenson and Rachel Muehrer players were required to successfully complete most of the game before they could move on to the most challenging questions. The purpose of this paper has been to detail the making of an educational game and its implementation with a large play testing group (146 participants), identifying which features of that game were effective in advancing its educational purposes, and supporting research into what and how students learned through its playful activities, and which features needed be changed before a full study could be carried out. This work contributes to the scant research to date on the use of iPads in educational settings, as well as to research on games and education more generally. That the pilot we conducted was larger in scale than is typical was also extremely useful: it gave us large enough numbers for statistical power to validate the pre and post test which was used again in Phase 2 of the study. In conclusion, while the length of this paper does not permit us to adequately detail the real enthusiasm both our participants and their participating teachers exhibited throughout this project, we do want to underscore that playing games, as other studies have shown (c.f. Boyle, et al., 2012), is one very real way to foster student engagement. Compareware was not designed for hours and hours of play, but to be played in short segments well‐suited to the time constraints of schools, which very much appealed to and was understood by the young, st “21 century learners” who participated in the study.

References Anderson, A. F., & Bavelier, D. (2011). Action game play as a tool to enhance perception, attention, and cognition. In S. Tobias & J. D. Fletcher (Eds.), Computer games and instruction (pp. 307‐330). Charlotte, NC: Information Age. All, A., Nunez Castellar, E. P., & Looy, J. V. (2013). Development of an Educational Game for Road Safety. International journal of game‐based learning, 3, 1, 1‐17. Bachen, C. M., Hernández‐Ramos, P. F., & Raphael, C. (2012). Simulating REAL LIVES: Promoting global empathy and interest in learning through simulation games. Simulation & Gaming, 43(4), 437‐460. doi:10.1177/1046878111432108 Barab, S. A., Scott, B., Siyahhan, S., Goldstone, R., Ingram‐Goble, A., Zuiker, S. J., & Warren, S. (2009). Transformational play as a curricular scaffold: Using videogames to support science education. Journal of Science Education and Technology, 18, 305–320. Bloom, P. (1996) Intention, history, and artifact concepts. Cognition, 60, 1‐29. Bloom, P. (2000) How children learn the meanings of words. Cambridge, MA: MIT Press. Boyle, E.A., Connolly, T. M., Hainey, T. & Boyle, J. M. (2012). Engagement in digital entertainment games: A systematic review. Computers in Human Behavior, 28, 771‐780. Deák, G. O., Ray, S. D. & Pick, A. D. (2002). Matching and naming objects by shape or function: Age and context effects in preschool children. Developmental Psychology, 38, 503‐518. Dickens, H. & Churches, A. (2011). Apps for learning: 40 best iPad/iPod Touch/iPhone apps for high school classrooms. The 21st Century Fluency Series. Thousand Oaks, CA: Corwin Press. Diesendruck, G., Hammer, R. & Catz O. (2003). Mapping the similarity space of children and adults’ artifact categories. Cognitive Development, 18(2), 217‐231. Diesendruck, G., Markson, L., & Bloom, P. (2003). Children’s reliance on creator’s intent in extending names for artifacts. Psychological Science, 14, 168‐168. Fletcher, J. D., & Tobias, S. (2006). Using computer games and simulations for instruction: A research review. In The Proceedings of the Society for Applied Learning Technology Meeting (pp. 1‐14). Orlando, FL: New Learning Technologies. Gee, J. P. (2003). What videogames have to teach us about learning and literacy. New York: Palgrave Macmillan. Gee, J. P. (2005). Good video games and good learning. Phi Kappa Phi Forum, 85(2), 33‐37. Gentner, D., & Ratterman, M. J. (1991). Language and the career of similarity. In S. A. Gelman & J. P. Byrnes (Eds.), Perspectives on thought and language: Interrelations in development (pp. 225–277). London: Cambridge University Press. Graham, S. A., Williams, L. D., & Huber, J. F. (1999). Preschoolers’ and adults’ reliance on object shape and object function for lexical extension. Journal of Experimental Child Psychology, 74, 128–151. Green, C. S., & Bavelier, D. (2003). Action video game modifies visual selective attention. Nature, 423, 534‐537. Henderson, S. & Yeow, J. (2012, January). iPad in Education; A case study or iPad adoption and use in primary school. Paper th presented at the 45 Hawaii Conference on System Sciences, Honolulu HI. Hsu, H. & Wang, S. (2010). Using gaming literacies to cultivate new literacies. Simulation & Gaming, 41, 3, 400‐417. Jenson, J., Taylor, N. & de Castell, S. (2011). Epidemic: Learning games go viral. Journal of the Canadian Association for Curriculum Studies, 8(2), 28‐49. Kagohara, D. M., Sigafoos, J., Achmadi, D., O’Reilly, M. Lancioni, G. (2012). Teaching children with autism spectrum disorders to check the spelling of words. Research in Autism Spectrum Disorders, 6, 304‐310.

250

Jennifer Jenson and Rachel Muehrer Ke, F. (2008). A case study of computer gaming for math: Engaged learning for gameplay? Computers and Education, 51(4), 1609‐1620. Ke, F. (2009). A qualitative meta‐analysis of computer games as learning tools. In R. E. Ferdig (Ed.). Handbook of research on effective electronic gaming in education: Volume 1 (pp.1‐32). Hershey, PA: Information Science Reference. Kelmer Nelson, D.G., Frankenfield, A., Morris, C., & Blair, E. (2000). Young children’s use of functional information to categorize artifacts: Three factors that matter. Cognition, 77(2), 133‐168. Landau, B., Smith, L. B., & Jones, S. S. (1998). Object shape, object function, and object name. Journal of Memory and Language, 38, 1–27. McClanhan, B. (2012). A breakthrough for Josh: How use of an iPad facilitated reading improvement. Tech/Trends, 56 (3), 20‐28. Merriman, W. E., Scott, P., & Marazita, J. (1993). An appearance‐function shift in children’s object naming. Journal of Child Language, 20, 101–118. Papastergiou, M. (2009). Exploring the potential of computer and video games for health and physical education: A literature review. Computers & Education 53(3) 603‐622. Peluso, D. C. C. (2012). The fast‐paced iPad revolution: Can educators stay up to date and relevant about these ubiquitous devices? British Journal of Educational Technology, 43(4), E125‐E127. Preciado Babb, A. P. (2012). Incorporating the iPad2 in the mathematics classroom: Extending the mind into the Collective. International Journal of engineering Education 2(2), p. 23 ‐ 29. Prensky, M. (2001). Digital game‐based learning. New York: McGraw‐Hill. Sitzmann, T. (2011). A meta‐analytic examination of the instructional effectiveness of computer‐based simulation games. Personnel Psychology, 64, 489–528. Smith, L. B., Jones, S. S., & Landau, B. (1996). Naming in young children: A dumb attentional mechanism? Cognition, 60, 143–171. Squire, K. (2011). Video games and learning: Teaching and participatory culture in the digital age. New York: Teachers College Press. Tobias, S., Fletcher, J. D., Dai, D. Y., & Wind, A. P. (2011). Review of research on computer games. In S. Tobias & J. D. Fletcher (Eds.), Computer games and instruction (pp. 127–222). Charlotte, NC: Information Age. Tobias, S. & Fletcher, J. (2012). Reflections on “A review of trends in serious gaming.” Review of Educational Research 82(2), 233‐237. Tsai, F. H., Yu, K. C., & Hsiao, H. S. (2012). Exploring the factors influencing learning effectiveness in digital game‐based learning. Educational Technology & Society, 15 (3), 240–250. Woodward, A. M., & Markman, E. M. (1998). Early word learning. In W. Damon, D. Kuhn, & R. Siegler (Eds.), Cognition, perception, and language. Handbook of child psychology:Volume 2 (pp. 371±420). New York: Wiley. Young, M., Slota, S. Cutter, A., Jalette, G., Mullin, G., Lai, B., Simeoni, Z., Tran, M., & Yukhymenko, M. (2012). Our princess is in another castle: A review of trends in serious gaming for Education. Review of Educational Research, 82(1), 61‐89.

251

An Overview of Game Console Motion Sensor Technologies Exploited for Education Marina Kandroudi and Tharrenos Bratitsis Early Childhood Education Department, University of Western Macedonia, Florina, Greece kandroudimar@hotmail.com bratitsis@uowm.gr Abstract: This paper attempts to raise a discussion regarding video game consoles which integrate motion sensor technologies, by examining their exploitation within educational context. There are several motion sensing technologies, but only three of them stand out, based on their market share. These are: a) Nintendo Wii, b) Microsoft Kinect, and c) Sony PlayStation Move. The Nintendo Wii was created and developed by Nintendo Company. The main controller is the Wii remote, a handheld device which can be utilized as a gesture recognition and pointing tool. Kinect is a motion sensing input device, implemented by Microsoft for the Xbox 360 game console. The device provides a natural user interface that allows users to interact without any intermediary device. PlayStation Move is a motion‐sensing game controller platform by Sony Computer Entertainment, first released for the Play Station 3 game console. Based on a handheld motion controller wand, PlayStation Move uses a PlayStation Eye camera to track the wand's position, and inertial sensors in the wand to detect its motion. This paper will present an overview of the existing literature, while attempting to categorize the educational approaches which involve motion sensor technologies. This categorization will consist of two parts. The first one will concern the education of people with special needs, under which many research approaches can be found. The utilization of motion sensor technologies, incorporated by the three most common game consoles, in the education of people with special needs will be examined. The second one will refer to various educational approaches in regular education, under which not so many research approaches, but many teaching ideas can be found. The aim of the paper is to serve as a reference point for every individual/group, willing to explore the sensor‐based Games Based Learning (SBGBL) research area, by providing a complete and structured literature review. Keywords: Kinect, Wii, Playstation, games based learning, special needs education, categorization

1. Introduction Nowadays, technology has expanded rapidly, covering many aspects of everyday life, such as game playing. The latter has evolved to a whole new era, as motion sensing technologies have been integrated in game consoles, allowing more realistic and immersive gameplay. This paper will attempt to examine the exploitation of such technologies in console games for educational purposes. Motion sensors are devices which provide a natural user interface, allowing users to interact without any intermediary device. They can capture gestures and/or detect voice. Motion sensing is a technology which emerged widely in the market within the past 5‐7 years. Nevertheless, several research approaches can be found in the literature, exploiting this technology within educational settings. This paper aims at enumerating these approaches, while categorizing them using multiple perspectives. The latter vary from level of education, to types of education (namely common or special education), or even cognitive areas (e.g. mathematics, physics, etc.). This paper will present a literature review, while attempting to categorize the educational approaches which involve motion sensor technologies. The aim of the present study is to provide a complete and structured literature review which can be used as a reference point for every individual group willing to explore the sensor‐based GBL research area. Additionally, the paper will attempt to raise a discussion on how these technologies can be further exploited, taking into account that free tools (e.g. Scratch programming platform and Kodu programming language) are available for educators, enabling them to integrate motion sensors in educational activities. Some indicative examples will be provided, thus raising open research questions, building upon an existing classification attempt of commercial Microsoft Xbox Kinect games (Kandroudi & Bratitsis 2012). The paper is structured as follows; initially the most common sensing technologies for video game consoles are briefly presented. Then a theoretical background is established, followed by a review of the existing literature, before the concluding discussion

252

Marina Kandroudi and Tharrenos Bratitsis

2. Motion sensor technology Several sensing technologies can be found in the computer and console games market. Currently, three of them stand out, based on their market share and their innovative functionalities. The Nintendo Wii incorporates the Wii‐remote controller (Figure 1), a handheld device which can be utilized as a gesture recognition and pointing tool. Its main technology is an infrared led, combined with an infrared sensor, built‐in the console which allows the spatial detection of the controller. Thus, mimicking of actual game actions, such as swinging a racket, a sword or manipulating a fishing pole can be achieved by the user. Furthermore, the controller incorporates an accelometer for detecting rotation in all three axes, allowing more sophisticated and verisimilar gestures. Up to four Wii‐Remote controllers can be connected simultaneously using Bluetooth technology. The Wii‐Remote Plus also includes a speaker, a rumble feature and an expansion port. Several add‐ons have been designed for the Wii Remote controller by various manufacturers, allowing a more realistic integration of the end user to the game environment.

Figure 1: Wii remote sensor Kinect is a motion sensing input device for the Microsoft Xbox 360 console. It provides a natural user interface, allowing user interaction without any intermediary device. It is a horizontal bar connected to a motorized pivot base, designed to be positioned lengthwise above or below the video display (Figure 2). It features an RGB camera, an infrared depth sensor and a multi‐array microphone running proprietary software, which provides full‐body 3D motion capture, facial recognition and voice recognition capabilities. Kinect identifies individual players through face, body and voice recognition, allowing the most realistic immersion of a user in the game environment, as the user’s body is the actual gaming device.

Figure 2: XBOX 360 Kinect sensor PlayStation Move is a motion‐sensing game controller platform for the PlayStation 3 (PS3) game console. Built around a handheld motion controller wand (Figure 3), using the PlayStation Eye‐camera to track its position, and inertial sensors in the wand to detect its motion. The wand features an orb which can glow, using RGB LEDs. Based on the environment’s colors the system dynamically selects an orb color that can be distinguished from the rest of the scene. The colored light serves as a known size and shape active marker, the position of which can be tracked along the image plane by the camera in three dimensions, with high precision and accuracy.

253

Marina Kandroudi and Tharrenos Bratitsis

Figure 3: PlayStation Move motion controller

3. Theoretical background Playing can be a powerful learning approach, through which students could develop new skills and participate in new social environments, undertaking new social roles (Vygotsky, 1978). Gee (2003), argues that games allow players to be knowledge producers rather than in the school scenario where students too often consume but do not produce. Playing games requires deep thinking (Squire, 2008) and it is crucial for learning since students take part in active experiences and not simply receive information, passively (Whitton & Moseley, 2012). Researchers (Parsons et al. 2000; Charitos & Martakos 2000) support that through Virtual Environments users can practice skills safely, avoiding potentially dangerous real world consequences. For instance, users with special needs (e.g. children with autism spectrum disorders) could practice social skills, while reducing the impact of a possible social failure. Moreover, sensor based technology could enhance their abilities and help them to cultivate their skills by putting them in situations in which probably they couldn’t respond to in real life. Games’ virtual worlds are powerful as they make it possible to develop situated understanding, effective social practices, powerful identities, and shared values (Shaffer et al., 2004). Additionally, they can support the development of logical thinking and problem solving skills (Inkpen et al., 1995; Higgins, 2000). Computer and video games could be used as an innovative tool for constructive teaching approaches in classrooms. Constructivism states that people construct their own understanding and knowledge of the world, through realistic activities which allow them to gain experiences and reflection upon those experiences (Vygotsky, 1978). MacFarland et al. (2002), based on how learning is achieved distinguish games in which: a) learning is achieved as a result of tasks stimulated by the games’ content, b) knowledge is developed through the game content, and c) skills arise as a result of playing the game. It seems that the educational use of the games depends on the teacher’s creativity. Kandroudi & Bratitsis (2012) support that games could not teach anything on their own; the educator is responsible for finding creative ways of incorporating the simulated worlds of the games into exciting learning activities. Furthermore, MacFarland et al. (2002) indicate the crucial relationship between games and the associated learning, which teachers should take into consideration. Oblinger (2006) states that pupils should be involved, through technology, in the learning process to: a) be engaged with the theories, b) acquire knowledge through autonomous and discovery learning, c) cultivate thinking skills, d) learn how to learn (metacognition), e) interact and communicate, and f) operate as active knowledge producers.

4. Literature review In the previous section, the significance of games in the learning process and the arising interest in motion sensor technologies was established. This paper attempts to categorize the educational approaches which involve such technologies. A literature review was implemented during the first trimester of 2013, involving the ECGBL and the major ICT related international conferences of the past 6 years, resulting to about 100 papers. Also, keyword‐based search was conducted, using combinations of the following terms: XBOX Kinect, Nintendo Wii, Sony PlayStation, sensor based learning, games based learning, virtual worlds special needs

254

Marina Kandroudi and Tharrenos Bratitsis education, Kinect education. This search was applied in electronic databases and content specific websites (Google Scholar, Scopus, Elsevier, springer, schoolnet (http://www.eun.org/), kinecteducation (http://www.kinecteducation.com/), ERIC, IEEE xplore, Mendeley). Finally, the same keywords were used for surveying through the Google search engine. Repetitions of ideas and cross‐references were omitted, leading to subset of articles, from which the ones considered as more innovative and/or important were selected, to be presented further bellow.

4.1 Special needs education 4.1.1 Deaf education and sign language Aimaiti & Yan (2011) support that XBOX Kinect could be a powerful tool for computers to understand human body language, thus providing opportunities in sign language training or medical research. In accordance with this claim, Lang et al. (2012) exploited Kinect for supporting sign language learning. According to them, Kinect offers 3D data in a simple camera setup from which information regarding the users’ body parts can be distinguished and extracted. The latter, allows easier recognition of not just hands and head, but also other body parts, such as elbows, which can further facilitate the distinction of a variety of language signs. They also highlight the independency of lighting conditions, due to the use of infrared light by Kinect (Lang et al., 2012). Moreover, Pyfers (2012) argues that Kinect could be a useful asset for educating deaf people. She indicates that Kinect may finally enable sign recognition applications to reach real users in real life settings. Furthermore, Zafrulla et al. (2011) while investigating the potential of the Kinect camera for sign language recognition, they support that this sensor could be a viable option for sign verification. 4.1.2 Autism and Down Syndrome Hsu (2011) refers to a research conducted at Nottingham Trent University by Rachael Folds, a PhD researcher from the University's School of Education, who examined the effect of the Nintendo Wii and Xbox Kinect in helping college students with learning difficulties. The research population comprised of 16 to 24 years old students with disabilities, ranging from Down’s Syndrome to autism spectrum disorders. They were asked to play tennis using a Wii game and bowling using a Kinect game. The results indicate that with five weeks of computer game training, two groups of participants significantly improved their motor skills, comparing the pre‐ and post‐test scores, in realistic activities (Hsu, 2011). Wuang et al. (2011) indicated that the Wii console virtual environment improved the motor proficiency, the visual‐integrative abilities, and the sensory integrative functions in children with Down’s Syndrome, from 7 to 12 years old. Specifically, comparing control and study groups the post‐intervention results of MANOVA tests revealed a significant overall group effect (Wilks’ lambda=0.03, F[34,272]=42.31, p=0.000, partial η2=0.84). Rahman (2010), by examining the effectiveness of Nintendo Wii Fit balance games supports that as a virtual reality‐based therapy, it could improve the balance of children with Down Syndrome, from 10 to 13 years old. In particular, he compared the post‐Wii intervention mean values of balance for the control group and the study group by using the independent samples t‐test, revealing a high significant difference (p= 0.000). The Lakeside Center for Autism integrates Kinect’s full body play technology into its therapy sessions, for assisting children with autism to overcome various difficulties regarding physical and social development. For instance, they use commercial games, such as Kinect Adventures and Kinect Sports, to improve motor skills. Kee (2009) implemented a case study with the participation of autistic children, involving car racing video games for learning physics, on the PlayStation 3 console. He supports that racing video games provide a learning framework for building the sense of physics. In compliance to him, Li et al. (2012) support that a webcam motion sensor game has effective results in autistic students’ training. 4.1.3 Other disabilities Hammond et al. (2013) suggest that the Wii Fit game can be used to improve the motor skills of children with Developmental Co‐ordination Disorder. Moreover, they argue about exploiting Wii Fit within a therapeutic programme for children with movement difficulties. Shih et al. (2010) studied two children with multiple disabilities, while using the Nintendo Wii Balance Board. They support that the device helped them to adjust their abnormal standing posture. Butler & Willet (2010) discuss the benefits of the Wii Balance Board for patients’ rehabilitation, regarding gross motor co‐ordination, balance and strength. The Wii Balance Board is a

255

Marina Kandroudi and Tharrenos Bratitsis Bluetooth input device bundled with the Wii Fit It implements the same protocol as the Wii Remote for hosting communications, and exposes most of its functionality via an extension controller. Most of the basic activities involving the Wii Balance Board are carried out using the positions shown in Figure 4. There may be other types of activities depending on the type of game being played, usually provided by the game manual.

Figure 4: Wii balance board Table 1: Categorization of motion sensor utilization in the education of people with special needs Author Lang et al., 2012

Console XBOX Kinect

Aimaiti & Yan, 2011

XBOX Kinect

Butler & Willet, 2010 DePriest & Barilovits, 2011 Hammond et al., 2013

Wii XBOX Kinect Wii

Pyfers (2012) Rahman, 2010

PlayStation Move XBOX Kinect Wii

Shih et al., 2010

Wii

Urturi et al., 2012

XBOX Kinect

Wuang et al., 2011

Wii

Zafrulla et al., 2011

XBOX Kinect

Kee, 2009

Study/ Results Kinect for supporting sign language learning. Powerful tool for computers in order to understand human body language and thus providing opportunities in sign language or medical research. Benefits of Wii Balance Board in patients’ rehabilitation regarding gross motor co‐ordination, balance and strength. Valuable applications for physical therapy and home rehabilitation exercises. Effectiveness in motor skills of children with Developmental Co‐ordination disorder. Case study research with autistic children regarding racing car video games for learning physics. Kinect technology could be a useful asset for educating deaf people. Balance improvement for children with Down syndrome. Nintendo Wii Balance Board on two children with multiple disabilities helped them to adjust their abnormal standing posture. Exploitation of XBOX Kinect for disabled people, in wheel chairs indicated an increase in their motivation although the game concerned physical exercises. Wii virtual reality improved motor proficiency, visual‐integrative abilities, and sensory integrative functions in children with Down Syndrome. Kinect can be a viable option for sign verification.

Urturi et al. (2012) conducted a research study which exploited XBOX Kinect for training disabled people, permanently attached to wheel chairs. Their findings indicate an increase in users’ motivation although the game concerned physical exercises. Furthermore, the motion tracking abilities of Xbox Kinect seem to provide the opportunity of the implementation of valuable applications for physical therapy and home rehabilitation

256

Marina Kandroudi and Tharrenos Bratitsis exercises. Doctors and patients connected through Xbox Live carried out prescribed rehabilitation courses which were graded and assessed (DePriest & Barilovits, 2011). Patients who have suffered a stroke and other brain injury at the Royal Berkshire Hospital are promoting Kinect as an important part of their rehabilitation process. Canton et al. (2012) highlight the difficulties which people with motor disabilities face when using the computer mouse. They believe that gesture based devices could help children with learning or developmental disabilities to understand that using a hand gesture could be used instead, for pointing at digital artifacts. In this section, the utilization of motion sensor technologies, in the education of people with special needs was examined. Table 1 presents a classified overview of the corresponding approaches.

4.2 Motion sensor technology in regular education In this section, the exploitation of motion sensor technologies in regular education is examined. Microsoft wished to test the hypothesis that the language skills of primary school students can improve through exposure to English, as a foreign language, while playing games using the Xbox console with Kinect technology (Lisle, 2012). A pilot case study was implemented for one year in Lakeside Park Primary School, in rural KwaZulu‐Natal, South Africa. Two classes from each one of grades 1 to 3, with an average class size of 42 learners and 6 teachers participated the study. An overall marked improvement in vocabulary which in turn impacted on comprehension and overall literacy skills was observed. The researchers also documented an increase in general knowledge and an increased enthusiasm for learning (Schoolnet, 2012). DePriest & Barilovits (2011) outline two interesting learning opportunities via Xbox LIVE with Kinect, the Video Kinect and Avatar Kinect software. The important feature in these cases is that both can host synchronous or asynchronous conferences or classes and achieve high engagement, interaction and communication among users, for cultivating language proficiency. Lee et al. (2012) created an interactive, gesture controlled arithmetic Kinect math game, called Xdigit. It was created to help students with math learning difficulties, incorporating various levels of difficulties to increase arithmetic complexity. It has not yet been evaluated within a research approach. Angotti and Bayo (2012) used Kinect with a group of teachers who volunteered to test it, for supporting math lessons. The implementation of the courses was successful and the teachers built a positive attitude towards the use of such tools in their classroom. After the completion of the project, the teachers reflected upon the study and indicated that the use of video games into the classroom should reevaluate the following: 1) classroom norms 2) learning culture 3) curriculum design 4) implementation practices 5) post lesson reflection. Wu et al. (2012) developed several interactive motion controlled games, utilizing Kinect, addressed to 5‐8 year old students, to enhance learning. They indicate that the result of learning depends on the children's engagement with the game. Their study argues about the importance of integrating motion sensor technology into the classroom. Lien et al. (2012) studied the production of video portfolios in the classroom, allowing students to use their whole body, which was captured by Kinect and presented in a rich and context‐sensitive background. Students could review their own performance under their teachers’ guidance or alone to enhance their cognition or metacognition. The study results indicated that this approach could significantly enhance students’ metacognition on Reflective, Sensing and Sequential learning styles. Although several studies exist regarding the education of people with special needs, not many exist in regard to regular education. On the contrary, in the case of regular education, one can find many teaching ideas, mainly exploiting commercial games for the consoles under examination, but only a few research studies. As already mentioned, most of them are related to the Kinect sensor. Thus, for the rest of this section, an overview of the types of ideas, one can find across the internet. A logical classification of these ideas would follow the core constituents of almost every school curriculum, thus including the following categories: a) physical development, b) cognitive development, c) social development, and d) emotional development.

257

Marina Kandroudi and Tharrenos Bratitsis Table 2: Categorization of Kinect games’ utilization for educational purposes in various developmental axes (Kandroudi & Bratitsis, 2012) GAME Body and brain Connection Kinect sports Kinectimals The Fantastic Pets Kinect Adventures Disneyland Adventures Sesame street: Once upon a monster Kinect FunLabs

Physical development X X X X

Cognitive development X

Emotional development X X X X X

Social development X X X X X X

All three companies, namely Microsoft, Sony and Nintendo, have released games in the market, which they characterize as educational ones. For example, Sony has released the LittleBigPlanet 2 Teacher Kit, which is a puzzle‐platformer game, centered around user generated content. The Kit features levels of the game which are themed around the USA National Curriculum subjects including physics, maths, science, art and history and is designed to help engage students in these subjects. For this matter, Sony has created ConnectEd, a company branch which focuses on the education sector. Nintendo has also released several titles which can be exploited for educational purposes. All the sports related titles can be utilized for physical development lessons, while all puzzle and problem based titles can be utilized for cognitive development. Regarding the emotional development, both companies have released titles which involve caring for something living (e.g. virtual pets), whereas social development is facilitated both by face to face and online collaborative playing. The Greek Ministry of Education (2003), also states in the official Kindergarten’s curriculum that the core educational goal is to facilitate the physical, emotional, cognitive and social development of children. Based on this classification, Kandroudi & Bratitsis (2012) attempted to categorize the most popular Kinect games to help educators in exploiting them for teaching physical, cognitive, emotional and social skills. Table 2 summarizes the categorization of some of the most popular Kinect games, based on the four axes. This categorization complies with the discussion, initiated in the previous paragraph, regarding the other 2 technological platforms.

5. Conclusion This paper tried to present a literature review, while attempting to categorize the educational approaches which involve motion sensor technologies. This categorization consists of two parts. The first one concerns the education of people with special needs, under which many research approaches can be found. The second one refers to various educational approaches in regular education, under which not so many research approaches, but many teaching ideas can be found. As stated in the introduction of this paper, the intension of the authors was to categorize the existing approaches, based on the level of education they correspond to. The literature review reveals that almost all of the ideas and approaches regarding Games Based Learning are related to Primary (mainly) and Secondary Education. This was rather expected, considering that these ages are more interested in playing games in general. Moreover, much of the existing work in the Special Needs sector relates to rehabilitation or development of specific skills/dexterities. In this case, the ages span from minors to elderly people, but no classification related to level of education can be made, as the latter is rather relative to the situation and not the age of the students. The main conclusion of this paper is that motion sensing technologies have been developed rapidly over the past years and many game titles are available, which seem to be exploited for educational purposes. One can find quite an amount of ideas and teaching material, although not always tested in carefully designed research approaches. Nevertheless, the main characteristic of the digital age in which we live is that people create and share knowledge, information and ideas. Thus, many internet based communities are available, especially for educators willing to try innovative approaches. It is obvious that the corresponding research has still a long way to go. Another important issue, related to this discussion is the fact that some of the sensor technologies under examination are compatible with widespread educational tools. For example, the Kinect sensor can work with

258

Marina Kandroudi and Tharrenos Bratitsis the Scratch (http://www.scratch.mit.edu/) programming environment, which incorporates a huge internet community and an equally large number of case studies. With Scratch as a mediation platform, Kinect can be combined with educational robotics (e.g. Lego WeDo set), thus providing new perspectives for educational researchers. Overall, the motion sensing technology seems promising. Additional research is required to establish a concrete research basis, regarding the educational exploitation of these technologies.

References Aimaiti, N. and Yan, X. (2011). Gesture‐based Interaction and Implication for the Future, Umeå University. Angotti, R. and Bayo, I. (2012). Making Kinections: Using video game technology to teach math. CIRN Community Informatics Conference 2012. 6‐9 November 2012, Prato, Italy. Butler, D.P. and Willett, K. (2010). Wii‐habilitation: Is there a role in trauma? Injury, 41(9), 883‐885. Cantón, P., González, Á.L., Mariscal, G., & Ruiz, C. (2012). Applying new interaction paradigms to the education of children with special educational needs. Computers Helping People with Special Needs (pp. 65‐72). Springer Berlin Heidelberg. Charitos, D., Karantanos, G., Sereti, A. Triantafyllou, S., Koukouvinou, S. & Martakos, D. (2000) Employing virtual reality for aiding the organisation of autistic children behaviour in everyday tasks. In Sharkey, P., Cesarani, A., Pugnetti, L. & Rizzo, A. (eds.), 3rd International Conference on Disability, Virtual Reality and Associated Technologies (ICDVRAT) (pp 147‐152), Alghero, Sardinia, Italy. DePriest, D. Barilovits, K. (2011). LIVE: Xbox Kinect©s Virtual Realities to Learning Games. In Proceedings of TCC ‐ Teaching Colleges and Community Worldwide Online Conference 2011 (pp. 48‐54). Gee, J.P. (2003). What video games have to teach us about learning and literacy. New York: Palgrave Macmillan. Greek Ministry of Education, (2003). A cross thematic curriculum framework for compulsory education. Greek Ministry of Education Hammond, J., Jones, V., Hill, E.L., Green, D., & Male, I. (2013). An investigation of the impact of regular use of the Wii Fit to improve motor and psychosocial outcomes in children with movement difficulties: a pilot study. Child: care, health and development Higgins, S. (2000). The logical zoombinis. Teaching Thinking, 1(1), 12‐14 Hsu Hui‐mei Justina (2011). The Potential of Kinect in Education. International Journal of Information and Education Technology, 1(5), 365‐370. Inkpen, K.M., Booth, K.S., Gribble, S.D., and Klawe, M.M. (1995). Give and take: children collaborating on one computer. In J.M. Bowers and S.D. Benford (eds), CHI 95: Human Factors in Computing Systems (pp 258‐259), Denver, CO: ACM Conference Companion. Kandroudi, M., & Bratitsis, T. (2012). Exploring the Educational Perspectives of XBOX Kinect Based Video Games. 6th European Conference on Games Based Learning (pp. 219‐227). Academic Conferences Limited. Kee, N.K.N. (2009). Harnessing commercial off the shelf (COTS) video games in special education: A retrospective reflection of pedagogy and learning observed with three autistic children in game play. In Proceedings of “Learn to Game, Game to Learn" International Simulation and Gaming Association 40th Annual Conference (pp.1‐8). Singapore: The International Simulation and Gaming Association. Lang, S., Block, M., & Rojas, R. (2012). Sign language recognition using kinect. Artificial Intelligence and Soft Computing (pp. 394‐402). Springer Berlin Heidelberg. Lee, E., Liu, X. & Zhang, X. (2012). Xdigit: An Arithmetic Kinect Game to Enhance Math Learning Experiences. Fun and Games 2012 Conference Li, K.H., Lou, S.J., Tsai, H.Y., & Shih, R.C. (2012). The effects of applying Game‐Based Learning to Webcam Motion Sensor games for autistic students' sensory integration training. TOJET, 11(4), 451‐459. Lien, C.L., Huang, C.Y., Wang, C.Y. & Chen, G.D. (2012). "Using Kinect to Track Lea,rning Behavior of Students in the Classroom as Video Portfolio to Enhance Reflection Learning." The 20th International Conference on Computers in Education (ICCE 2012). November 26‐30, National Institute of Education, Singapore Lisle P. (2012). Using the Xbox Kinect in Foundation Phase English Language Acquisition. Schoolnet McFarlane, A., Sparrowhawk, A., & Heald, Y. (2002). Report on the educational use of games. Teachers evaluating educational multimedia. Parsons, S., Beardon, L., Neale, H.R., Reynard, G., Eastgate, R., Wilson, J.R. & Hopkins, E. (2000). Development of social skills amongst adults with Asperger’s Syndrome using virtual environments: the ‘AS Interactive project. 3rd ICDVRAT, Sardinia Italy, 163‐170. Pyfers, L. (2012). Sign 2.0: ICT for Sign Language Users: Information Sharing, Interoperability, User‐Centered Design and Collaboration. In K. Miesenberger, A. Karshmer, P. Penaz and W. Zagler (eds), ICCHP 2012, Part II, LNCS 7383 (pp. 188‐191). Springer Berlin Heidelberg. Rahman, S. (2010). Efficacy of virtual reality‐based therapy on balance in children with Down syndrome. World Applied Sciences Journal 10(3), 254‐261. Oblinger, D. (2006). Simulations, games, and learning. Educause Learning Initiative.

259

Marina Kandroudi and Tharrenos Bratitsis Schoolnet (2012). Schoolnet completes a study on the use of Xbox Kinect to improve literacy scores – interesting findings, [online], http://schoolnetsa.blogspot.gr/2012/03/schoolnet‐completes‐study‐on‐use‐of.html Shaffer, D.W., Squire, K, Halverson, R. & Gee, J.P. (2004). Video games and the future of learning, Phi Delta Kappan, Vol. 87(02), 104‐111 Shih, C.H., Shih, C.T., & Chu, C.L. (2010). Assisting people with multiple disabilities actively correct abnormal standing posture with a Nintendo Wii Balance Board through controlling environmental stimulation. Research in developmental disabilities, 31(4), 936‐942. Squire, K.D. (2008). Video games and education: Designing learning systems for an interactive age. Educational Technology 48(2), 17‐26. Urturi, Z.S., Zorilla, A.M., & Zapirain, B.G. (2012). JeWheels: kinect based serious game aimed at wheelchair users. In Ambient Assisted Living and Home Care (pp. 391‐398). Springer Berlin Heidelberg. Whitton, N. & Moseley, A. (2012). Using Games to Enhance Learning and Teaching: A Beginner's Guide, Routledge. Wu, E.H., Liao, C.W., Huang, S., Chang, C., Chen, Z., Yang, J., Li, D. (2012). "Kids Magic Learning: Kinect‐based Game Learning Development and Practice." Taiwan Academic Network Conference Wuang, Y.P., Chiang, C.S., Su, C.Y., & Wang, C.C. (2011). Effectiveness of virtual reality using Wii gaming technology in children with Down syndrome. Research in developmental disabilities, 32(1), 312‐321. Vygotsky, L. (1978). The Development of Higher Psychological Processes. Mind in Society: Harvard University Press, Cambridge. Zafrulla, Z., Brashear, H., Starner, T., Hamilton, H., & Presti, P. (2011). American sign language recognition with the kinect. In Proceedings of the 13th international conference on multimodal interfaces (pp. 279‐286). ACM.

260

Playing and Gaming – Studied in an Informal Learning Setting Helle Skovbjerg Karoff, Stine Ejsing‐Duun and Thorkild Hanghøj Aalborg University, The Department of Communication, Copenhagen, Denmark karoff@hum.aau.dk sed@hum.aau.dk thorkild@hum.aau.dk Abstract: The paper develops an approach of playing and gaming activities through the perspective of both activities as mood activities (Karoff 2013). The point of departure is that a game – whether it is ludic or paideiac ‐ is a tool with which we, through our practices, achieve different moods. This based on an empirical study of children´s everyday lives, where the differences emerge through actual practices (Schmidt 2011, Heidegger 1996), i.e. through the creation of meaning in the specific situations. The overall argument is that it is not that important whether it is a playing or a gaming activity – it is however crucial to be aware of how moods occur and what their optimal conditions are. Following Lave and Wenger (1991), participation in particular is essential. Learning this, the community of practice becomes crucial for learning the meanings of different moods as they emerge through because it is through the shared practices it becomes possible. This perspective has two dimensions: practices and moods. Practice is the concept of all the doing in the activities. Moods are the particular concept of sense and feeling of being, which is what we are drawn to when we are playing or gaming. Keywords: playing, gaming, practices, moods, children

1. Introduction Playing and gaming are difficult concepts to pin down. First and foremost, the words we have to describe these types of activities differ from language to language (Huizinga 1938). Practically, we usually operate with play as a part of the game or the game as a part of the play, as Salen & Zimmerman point out in their Rules of Play. Game Design Fundamentals (2004). In video game research there appears to be a tendency to place gaming and playing hierarchically into a scheme in a non‐productive way (Walther, 2003). Often the differences are discussed in relation to the structure of the activities through the inspiration of Callois (1961/2005), where Paideia denotes play – the unstructured and totally free activity, whereas Ludens refers to gaming as structured activities with clear goals from the very beginning, which does not bring us any closer to a clear perspective. The aim of this paper is to develop a different approach of playing and gaming activities through the perspective of both activities as mood activities (Karoff 2013). This is based on an empirical study of children´s everyday lives, where practices in non‐institutional settings – informal learning settings – deal with children´s creation of meaning in specific situations (Sørensen 2007). The consequence of defining play and games based on practices rather than on structures is that we are focusing on understanding the players’ productive role in creating the experience. This understanding can once again give us an understanding of play and game while also providing useful knowledge about how we can design tools that can be used to shape moods. This perspective has two dimensions: practices and moods. Practices is the concept of all the doing in the activities and moods is the particular concept of sense and feeling of being, which is what we are drawn to when we are playing or gaming. The point of departure is that games and play have structures, which are tools utilise in order to achieve different moods. The perspective will be analysed through the concept of “commonness”, i.e. shared practices (Schmidt, 2011) and the concept of mood as a way of being in the world (Heidegger 1996). Our thesis is that both playing and gaming are about creating a life (or a way of living) that is meaningful to you and the people you live with. The understanding of learning must be addressed with the point of departure following Lave & and Wenger (1991) that you learn the practices of play while sharing it, and you share it while you learn it. This learning process can only take place within a specific perspective (Heidegger 1996, Schmidt 2011, Bateson 1955), and it arises through relationships and to the extent that it has reference to the wholeness and in so far that the playing and gaming activity both exemplifies and underlines this perspective. This theoretical framework will be used in order to introduce and develop four concepts of practices; sliding, shifting, displaying and exceeding and four concepts of moods; devotion, intensity, tension and euphoria. The

261

Helle Skovbjerg Karoff, Stine Ejsing‐Duun and Thorkild Hanghøj main argument here is that the playing and gaming activity is performed in order to achieve a playful mood and this can only be achieved by using tools. Firstly, we want to show through Bateson´s (1955) concept of framing that the playing and gaming activity is only understandable within its own frame of reference. Secondly, the concept of practice will be defined as a “commonness practice" through Schmidt (2011). Commonness practice is something we share with others without having to discuss or articulate it. Learning them is mostly about taking part in defining what they could be. Four types of commonness practices will be developed in this section. Thirdly, we want to define play mood by introducing Heidegger’s (1996) understanding of mood and subsequently by introducing four additional types of moods. Finally, we conclude that commonness practices occur in order to achieve expressions of moods and gaming and playing are both activities that have the creation of moods as a goal. The conceptualisation of the commonness practices and moods is based on a variety of empirical examples taken from a larger empirical study. The empirical study was carried out in connection with the doctoral dissertation Play as practices of moods (Karoff 2010) and consists of ethnographic fieldwork with 17 children (aged 5‐14) and their families.

2. Playing and gaming as ways of being in the world In his concept of framing Bateson underlines that the production of meaning in the play activity has another reference than outside the playing activity as presented in the article “The Theory of Play and Fantasy” (1955). Bateson notices that human beings are capable of understanding the difference between the playing activity and the non‐playing activity, and he describes that phenomenon as a theoretical paradox. Framing is Bateson's term for the playing activity initiating "this is play" ‐ and thus separated from something that is not play. Bateson frames the playing activity as a different reality, i.e. a reality within a reality. He uses the example of two monkeys playing, biting each other in jest. When the monkeys are playing, a bite does not mean the same thing as when animals are fighting in the jungle. What is meaningful and what is not can, as a consequence of this perspective, only be understood fully within the framework of time and place for those who participate in the activity. When you are playing you create a universe of meaning through your actions or practices, where every action is understood in relation to the framework. Meaning in that sense can only occur within a specific perspective and it has no reference to anything beyond this. In that sense, by trying to understand and evaluate the meaning of playing activity outside of this you can only ever describe it as a theoretical paradox. An example of this was observed during the fieldwork (Karoff 2010) at a visit to one of the children, Elias, where Elias and Karoff play with LEGO‐blocks and a LEGO game on the computer. There are certain ways in which we hold the blocks, build with the blocks, drive with the LEGO‐car, and build with the LEGO‐blocks on the screen that creates a shared understanding of meaning in this situation. If we did something that was outside of this framework, e.g. threw the blocks out of the window or started fighting with the blocks, it would not make sense in this specific situation. Here, there is a relation between practices and creation of meaningfulness, and this is what we refer to as framing as defined by Bateson (1955). In the following section we will develop the idea of practices – commonness practice ‐ through this perspective.

3. Commonness practice as the doing in playing and gaming Schmidt defines practice as: “a doing and a making, done in a repetitive rhythm” (Schmidt 1999:37). In relation to playing and gaming, practices is the doing part of the activity, i.e. all types of behaviour such as physical and mental activities, the use of objects, of toys, ways of relating to feelings and motives for behaviour. Practice includes ways of being in a body, ways of thinking, ways of using toys, ways of feeling, ways to stay motivated etc. (Schmidt 2011, also following Reckwitz 2002). When Elias and Karoff are involved in LEGO building and gaming, we have certain ways of doing just as there are ways of playing war, ways of re‐enacting playful fighting, ways of climbing trees (Jessen & Karoff 2008). We share these ways of doing our practice, when we are participating together. Schmidt (2011) introduces the concept of commonness in order to underline that we share these doings socially and that they make sense to everyone without conflict when they are shared and without anyone taking any position on their advantages or disadvantages. Commonness practices from Schmidt’s perspective will be examined by looking at a specific play situation, where Elias and Karoff are playing with LEGO‐blocks on the floor and playing at the computer. This is a short extract from the field notes:

262

Helle Skovbjerg Karoff, Stine Ejsing‐Duun and Thorkild Hanghøj “Elias is very much into playing with his LEGO, when I visit him. First, we sit on the floor and start building LEGO cars. After finishing his car, Elias drives the car up and down the closets in his room while making car sounds. When he is about to hit the floor he screams “aaaahhhhhh”, just as kids do when they are bicycling down a hill. When he has been playing like this for a while, he goes to the computer and begins playing a LEGO game. He builds his car, and choses colour, type of car and the size of the decks. Having finished building the car he starts driving in the game, ignoring the rules and the predefined goals, but uses the car and engages in the process of building similar to what he did when playing with the physical LEGO car: driving his car in the game, he moves his body from one side to the other, making car sounds, and when he drives down a hill he says “aahhhhh””. In the play situation, Elias and Karoff are developing practises that make up their play experience. This consists of many different actions: finding the blocks, knowing all the sounds, building the car, driving the car around etc. If we want to be a part of this play situation together, we have to learn all these practices, and this learning process is going on through our participation (Lave & Wenger 1991). We both repeat the actions in certain ways because we have an idea of how this situation has to be performed. Through these actions we articulate what is meaningful to us while we are playing, i.e. we articulate what is within the frame of the game (Bateson, 1955). We can say that the actions constitute a commonness practice that forms the basis through which meaning can be expressed, and we can therefore not understand framing as something predefined, unchangeable. Instead, it is the order of the practices that defines the frame. When Elias and Karoff play, their commonness practice has a meaningful relation to one another and they are the conditions by which the play makes sense to them. We understand their finding the blocks, building the blocks, having the car, driving with the car and the Elias' sounds as a narrative that is unfolding. It is forming a possible circular trajectory or a rhythm for the practices and whenever Elias have found the blocks, and it opens up for the building blocks again and new possibilities for sharing commonness are shown. All the steps that Elias is engaging in are significant; relating to each other, and it continuously invites new commonness practices. By forming our framework through our practices, rules and goals from the game do not become a part of the tool, using it for practice. Elias and Karoff are therefore involved in a process of framing their understanding of what is meaningful to learn and what is not. They have not named their practices in the LEGO play in terms of “we do this and that”, but even though the basic premise underlying the commonness is non‐articulated, it doesn´t mean it does not exist, as shown above. It is through our commonness practices that our understanding of what is meaningful to learn and learnable shows itself and is negotiated, to some extent, even though it is not well defined. The commonness practice becomes meaningful, and it has different ways of showing itself and, as we saw in the situation with Elias, it is not well defined. But it would become much clearer if suddenly Elias or Karoff stopped agreeing on a commonness practice. If Karoff began to hit his car, Elias would probably say “No!”, and thus defining his practice as taking care of the car, not wanting it to be broken. In the moment of disagreement on a commonness practice, it would become clear what kind of commonness practice made sense to Elias in his play. Our practices are not clearly defined, but we nevertheless have a sense of what they are and what is not permitted. It means that our idea of what makes for great LEGO building‐driving‐cars‐play is to some extent present in our playing together, even though it is not possible to clearly define and characterise the commonness practice. The way we are able to identify the practice is through what it is not. That is to say that Elias and I do not agree in advance that we will practise this playing in a particular way. The best LEGO play is not a product that exists in a well‐defined form or framework, but instead something that is constantly in development through our doings together. This also means that the learning outcome cannot be defined from the beginning, because our sharing together is a part of defining what it could be, and this becoming emerge from the very situation of us sharing together. The changeable frame following Bateson and Schmidt, which shows its order through the commonness practices, actualises the discussion about playing and gaming in the sense that the actualisation of the commonness practices defines what is going on. What is going on can only be defined in the specific situations where the commonness practices, and thereby the framing, comes into life. In this understanding it is not the game or the play as a tool that in itself tells us something about the activities. We can have an idea about the future activity based on the tool, but it is only through practice and in the actual becoming of the tool that we can decide (also following Heidegger 1996).

263

Helle Skovbjerg Karoff, Stine Ejsing‐Duun and Thorkild Hanghøj With Elias the LEGO game with predefined rules and goals (Salen & Zimmerman 2004) are ignored as a game, but Elias uses some of the possibilities in the “game world” as a tool for his playing activities. In his production of meaning the structure of the order of the practices, which is creating the overall frame the rules and goals of the game, is not a part of his play. The argument is that Elias is using the LEGO blocks and the game world as a tool for his playing activities as practices, because of the goal of having the moods to continue. In the next section we will develop four types of play practices based on the empirical evidence gathered in the situations during a field study (Karoff 2010) and the argument will be further unfold.

4. Four types of commonness play practices In developing these four categories of practice the basic assumption is that commonness practice is about creating meaningfulness and production of meaning. The first commonness play practice is sliding. It has a strong repetitive rhythm and the children taking part in this regular beat are orientated towards the repetition, primarily in order to continue what is already going on and to make the change as minimal as possible. As shown with Elias and the LEGO building, there is only minimal variation over time in this kind of playing activity. The same actions recur time and time again. Elias continues the trajectory of practices over and over again, devoted to the repetition, changing it as little as possible – even when he is using the game‐world. The recurring rhythm is in focus and while playing the participants are involved in changing as little as possible as the actions are repeated again and again. The characteristics of sliding are flow and continuity, while conflict or discussions about practice do not generally dominate this practice. The second commonness practice is shifting. Like sliding it has a strong repetitive rhythm, but over time, shifting involves taking chances and creating surprises for other players. A wild trip on a roller coaster can constitute shifting practices; on the one hand there is a strong repetition of movement, but on the other there is a rapid change in speed and between different areas and different heights. It is precisely the shifting between fast and slow and high and low that characterise this type of commonness practice. The strong repetitive rhythm is there, often in the beginning of the play situation, but the speed or height are shifting surprisingly, and as a consequence change is unpredictable. Afterwards, there is a return to the strong repetitive and predictable rhythm. The rapid shifting between a strong predictable rhythm and short, fast change is characteristic for this type of commonness practice. The third commonness play practice is displaying. This practice is characterised by constant changes in the playing over time. Play situations that involve any kind of informal performing, showing off yourself and your skills through for example dancing or singing, taking photos of each other or dramatic (role) playing are all examples of displaying. The focus in the displaying practice is on putting yourself on a 'stage' and to let others players look at, learn from or critique the playing activity. When involved in the displaying commonness practice the player must be sensitive towards repetitive actions, in this case both the song and the dance sequence, but they must also put something of themselves into the practice. Making the practice your own and giving it your own 'style' plays an important role in commonness play practice as displaying. The last play in the commonness practice is exceeding and this is the polar opposite of sliding. In the exceeding commonness play practice, repetition is fleeting and the children expect the action to exceed expectation time and after time. The expectation is that the activity must be 'out of sync' with the rhythm, rather than being about 'finding the rhythm'. When Nuala and Freya (in Karoff 2010) play with their dolls, which are made of plastic, they bend them into all sorts of shapes. They are constantly fighting with each other and competing to see who can make the most bizarre forms. The heads are turned around in order for the dolls to look as if the head has been put on the corpus the wrong way round and the legs are bent towards the shoulders. Bizarre doll play, jack ash tricks and stories of faeces, dirty words and stories of frivolity are characteristic for the commonness practice as exceeding. When Karoff played a role play with Nelly, a dinosaur gets a pie in his head, and Peter is playing with his bird at his grandmother´s house, hoping that the bird will do a bird shit. The practice is oriented towards excessiveness, which means that the commonness practice must be in contrast to any notion of repetition. You only play with the idea of repetition in order to contrast it with change, often explosively and with a strong distance to the known repetition. As with Nuala and Freya their bending and torturing of the dolls is in stark contrast with the loving and caring practice of doll play.

264

Helle Skovbjerg Karoff, Stine Ejsing‐Duun and Thorkild Hanghøj We'll now turn to examining play moods. The idea is that through your doing, the commonness practice, you create ways of being; you create moods using tools, like dolls or games.

5. Mood as the way of being In the following we want to examine the idea of moods as the way of being using Heidegger´s (1996) idea of mood. According to Heidegger, the human being is always in some kind of mood. Even if our state of being appears to be completely moodless we are nevertheless in a mood. We will never merely observe the world, because things only make sense from the place in which they are seen in our lives and from the contexts in which they belong, in relation to what has previously made sense to us and how we feel they should be included in relation to an imminent future. In that sense the production of meaning is not just something we do from time to time, rather it constitutes the way we are as human beings. Heidegger captures that point in his conceptualisation of the human being as a Dasein – a being who is always already in place – da ‐ and doesn´t exists in distance to the world, but always in relation to the world. To be in a mood is the way that Dasein exists. According to Heidegger a mood is an indeterminate or undefined state that is revealed to you before any kind of meaning can be created. Mood tells you something about your relation to the world, or how you are tuned in to the world and to the people around you. But in a mood our understanding of a specific meaning is not fully articulated or completely un‐articulated. It is placed in an intermediate position where all is not said, but neither does it mean that nothing is said. Heidegger´s notion of mood is characterised as a way of being in the world that is not confined to a specific meaning, but open and ready for meaning to be articulated as something specific, even though the specification has not yet happened. Mood, according to Heidegger, is this non‐specific way of being where you are tuned in to the world openly so that meaning can be produced. The key point here is that we cannot understand mood only as an inner psychological state of mind (Csikszentmihalyi 1979) that comes from within, but rather as something that takes place in our engagement as human beings, as Dasein, in the world, in our doings. Heidegger captures that point very poetically when he says: “A mood assails us. It comes neither from the “outside” nor from the “inside”, but arises out of Being‐in‐the‐world, as a way of such Being” (Heidegger 1996:§29). Applied to our perspective what is important here is that mood comes before any meaning can be articulated as something specific. It is the state of being where you are distinctly open to new meaning production and where the possibilities exist for that to happen. It is not something that comes from within the players or from the outside, but instead it is happening through our engagement with the doings and in our relations towards the people we are with. In other words, mood is not only closely related to the commonness practices, as in the doing where meaning is produced and playing and gaming is happening; in commonness practices mood as the engagement in the world is expressed through that engagement. On the basis of the aforementioned theoretical ideas we can describe four (following Schmidt 2011) types of moods. These can be related to the four commonness practices already presented. The moods are categorised as ways of understanding how we are tuned into the world in the engagement in commonness practices. The point is that you ‐ through a commonness practice ‐ create a mood. Below, our four moods, combined with the commonness play practices:

6. Four types of play moods expressed through four commonness play practices The first play mood we will attribute is devotion ‐ characterised as a feeling of being in a flow, continuously being in the moment, which is accompanied by lightness. There is no hardness when being in this state, merely concentration and focus, and the body is often quiet or moving in slow motion. Often when children are playing with LEGO blocks, dolls or drawings in their own room this mood will be in focus. When Elias and Karoff are playing with the LEGO blocks the mood is devotion, where we are totally letting ourselves go into that state of being, letting go of “our doings” and seeing where this being leads us. Our openness towards new productions of meaning is first and foremost a wish for continuity and confirmation of what is already meaningful to us – or a confirmation of what we already know. The recurring rhythm is creating the continuity, and all we do is to follow the rhythm of devotion. The strong rhythm is the characteristic of the commonness practice sliding. The strong repetitive beat is in focus and there is a minimum of conflict; instead it is all about

265

Helle Skovbjerg Karoff, Stine Ejsing‐Duun and Thorkild Hanghøj continuing in the same vein as smoothly as possible. The play situation with Elias is a good example of devotion mood and sliding practice, and it also tells us, that players will do a lot for the way of being to continue. The second type of mood is intensity. A rush to the head or butterflies in the stomach can characterise this mood, where you have an intense experience and a feeling of your body being as ready and excited for more. This applies to extreme physical play situations, such as riding a rollercoaster, swinging on a swing or sliding down a slide, where you have an intense feeling of being in that moment. The openness towards production of meaning is characterised by expectation of fast change in rhythm and the unpredictable feeling that something is going to happen. In contrast to the mood devotion, in the play intensity change in meaningfulness is expected. The mood is related to the commonness practices shifting, which also has a strong repetitive beat, but over time, as shifting creates surprises for the players. The third type of mood is tension, which is characterised by being ready to show off or perform, and also by being aware of others´ awareness of you showing them selves. When the girls are dancing together they share the mood tension, and they constantly make themselves look good by showing off their style, knowing that they are not only looked at as presenting good expressions of style, but they are also looking for expressions of style themselves when they are taking part as an audience. There is an openness in the mood as an expectation of change of the production of meaning, not only a production of the expected, but the production of meaning to something unpredictable relating to one's own style. The mood is related to the commonness practice displaying and as we saw it this all about putting yourself forward on a stage and letting other players judging the play activity. The last type of play mood is called euphoria, and this is characterised by an intense expectation of silliness, where you are ready for both others´ and your own silliness. There is an expectation for you to come up with new silliness in order to stay euphoric. Children often laugh a lot: once they start, they find it difficult to – nor do they want to – stop and for people who are not involved the mood can seem manic. When children pull faces, have water fights, tease grown‐ups over and over again or cheat in games, the mood can be characterised as euphoric and the related commonness play practice is exceeding. When it comes to openness and production of meaning, the euphoria mood is the most open‐minded of the moods presented here, because the production of meaning is constantly seeking new ways of expression, and for the players it means a great extent of openness towards new ideas of overcoming the earlier meaningful practices in order for new production of meaning to come into life. In other words, the players must be tuned into change and to changing constantly, and the mood can be compared to a wild piece of rock music. The euphoria play mood is the complete opposite of devotion and its corresponding commonness practice ‐ sliding. Where devotion is characterised by quietness, flow and the very moment of fulfilling your expectation safely, the euphoria mood is characterised by the expectation of being surprised by what will follow.

7. Conclusion This article has drawn on analyses using the work of Bateson, Schmidt and Heidegger in order to set up mood as a way of describing the aim of the playing and gaming activity, the commonness practice. Moods are essential to these activities, and they are always in plural, depending on how players engage with the world and the people they are with. In this article four types of moods have been presented, which are related to four types of practices. When highlighting mood in it becomes possible to go beyond a dualistic understanding of the relationship between playing and gaming and instead see these types of activities as approaches which are interesting in the sliding between the types of tools and activities in order to keep achieving and continuing moods. Participating in playing activities is an important part of learning commonness practices and moods by sharing them in order to explore what they could emerge. If we want to be a part of the communities of practice, we have to learn the practices related to the specific situation through our participation. This participation in actions constitutes the communities of practice.

References Bateson, G. (1955) ”The Theory of Play and Phantasy” in Steps to an Ecology of Mind. Chicago: University of Chicago Press. Caillois, R. (2005/1961) Man, Game and Play, USA: University of Illinois Press. Csikszentmihalyi, M. (1989) Flow. The Psychology of Optimal Experience, New York: Haper Collins Publishers. Heidegger, M. (1996) Time and Being. Australia: Wiley‐Blackwell Publishing. Huizinga, J. (1993) Homo Ludens. A Study of the Play Element in Culture, Boston: The Beacon Press.

266

Helle Skovbjerg Karoff, Stine Ejsing‐Duun and Thorkild Hanghøj Jessen, C. & Karoff, H. (2008) ”Playware and new Play Culture” in Proceedings for BIN Conference: Æstetik og kultur, Island. Juul, J. (2005) Half‐Real: Video Games between Real Rules and Fictional Worlds. Cambridge, MA: The MIT Press. Karoff, H. & Johansen, S. (2009) ”Materiality, Practice and Body” in Proceedings for IDC2009, Italy. Karoff, H. (2010) Leg som stemningspraksis, Danish University of Education, Copenhagen. Karoff, H. (2013) ”Play Practices and Play Moods”, International Journal of Play (in press). Lave, J. & Wenger, E. (2001) Situated Learning: Legitimate Peripheral Participation, Cambridge: Cambridge University Press. Mouritsen, F. (1998) Child Culture ‐– Play Culture. Working Paper 2, The Department of Contemporary Cultural Studies. University of Southern Denmark. Reckwitz, A. (2002) ”Toward a Theory of Social Practices. A Development in Culturalist Theorizing” in European Journal of Social Theory, vol. 2:5: (245‐265). Salen, K. & Zimmerman, E. (2004), Rules of Play: Game Design Fundamentals, Cambridge: MIT Press. Schmidt, L.H. (2011) On Respect, Copenhagen: Danish School of Education, Copenhagen: University Press. Schmidt, L.H. (1999) Diagnosis I – Filosoferende eksperimenter, Copenhagen: University Press. Sørensen, B. (m.fl.) (2007) ”Children's Informal Learning in the Context of Knowledge Society” in Education and Information Technologies: Journal of the IFIP technical committee on Education, 12:1. Walther, B.K. (2003) ”Playing and Gaming – reflections and Classifications”. Game Studies, 3:1.

267

Picking the Right Interface for Engaging Physical Activity Into Game Based Learning Helle Skovbjerg Karoff1, Gunver Majgaard2, Lars Elbæk3 and Mona Have Sørensen3 1 Aalborg University, Department of Communication, Copenhagen, Denmark 2 University of Southern Denmark, The Maersk Mc‐Kinney Moller Institute, Odense, Denmark 3 University of Southern Denmark, Institute of Sports Science and Clinical Biomechanics, Odense, Denmark karoff@hum.aau.dk gum@mmmi.sdu.dk lelbaek@health.sdu.dk mhsorensen@health.sdu.dk Abstract: Following Qvortrup and Bateson, this paper aims to discuss and explore how technology, learning, and movement in synergy make several levels of learning possible. We will do that by introducing an application for painting developed by engineer students, as an illustrative example. In today's technology‐driven world, it is easy to forget that we are born movers. To a great extent we have engineered movement out of our lives. Both bodily and mental acuity increases with activity and declines with inactivity, and so the sedentary character of life in Western societies does not only affect our bodies. It affects our brains as well. However, the most recent results indicate a significant and positive effect of physical activity on children's learning potential. Most of these studies lack a connection to the learning processes in an educational setting because they are of an isolated, experimental nature set in a laboratory. An interesting challenge for future research, then, is investigating mind and body as a synergetic catalyst for learning through physical activity in a classroom setting. Results from this kind of research could have a great impact on the way we think of and organise our educational system. When using digital learning resources, children should be physically active as part of their learning process. With the paint application, we explore that field. Through bodily activity, children gain new perspectives on and insights into the learning materials. Supported theoretically by Bateson and Qvortrup children learn when they are experimenting, constructing, interacting, and physically active. Keywords: physical activity, learning, games, technology

1. Introduction This paper aims to explore how technology, learning, and movement in synergy enable learners to achieve all levels of learning following Qvortrup (2006) and Bateson (2000). We describe and analyze a learning resource in the form of a paint application developed by a group of engineer students. Interaction with the application interface is through full bodily movement. By introducing Luhmann´s theory of communication (2002) and Qvortrup's theory of learning (2006) (inspired by Bateson), our aim is to show how the paint application is an example of a tool for students to reach a number of different levels of learning. The combination of Luhmann´s and Qvortrup's theories is used as an analytical tool in uncovering the embodied learning potential of the application; revealing a deeper insight into the learning potential of full body interfaces. This insight applies to all situations of learning. What do you expect from this paper? What do you expect from the authors? When the academic reader lets his eyes wander over the headlines of this paper, he will probably expect the composition of the paper to be consistent with the written scientific tradition. But what if the reader achieves an even greater knowledge or an improved experience if the paper is communicated differently ‐ if the reader's body is used as a resource via technology? Expectations afford you an opportunity to prepare for your academic reading. Yet, expectations can also keep the reader rooted in rigid beliefs about reading where the body is given attention only in the event it is a nuisance. Traditionally, scientists assumed that the brain functioned independently from the rest of the body. A dual hegemony has led to a belief that body and senses must be eliminated from the quest for a purely intellectual and objective knowledge. The tendency to study brain and body separately continues. Knowledge is regarded as a separate phenomenon independent of bodily movement.

268

Helle Skovbjerg Karoff et al. Nevertheless, the most recent results indicate a significant and positive effect of physical activity on children's learning. (Davis CL, et al. 2011). These studies have investigated the relationship between increased amount of exercise in test subjects and learning. Children learn when they are experimenting, constructing, interacting and physically active. This is supported theoretically by Bateson (2000), Schön (1984), Wenger (1998) and Papert (1980). So by using digital game based learning resources, children should be physically active as part of the learning process. Through the interactive bodily activity they gain new perspectives on and insight into the learning materials. Various studies into the influence of physical activity and subsequent learning have sought to uncover the relationship between physical activity and children´s learning. (Andersen & Froberg 2006; Ericsson 2003; Hillman CH et al. 2009). These studies have showed unequivocally in what regard physical activity was enhanced, impaired, or had no effect on learning abilities. A growing interest into integrating the body in the study of cognition has arisen (E.g. Aziz‐Zadeh & Damasio 2008; Aziz‐Zadeh et al. 2006; Binkofski et al. 2004; Buccino et al. 2006; Gibbs 2007; Glenberg et al. 2008; Lutz & Thompson 2003; Pfeifer & Bongard 2007; Rizzolatti & Craighero 2004; Tettamanti et al. 2005). These studies have opened new perspectives on the body as a potential learning resource. However, most of these studies are isolated laboratory experiments, lacking connection to the learning processes present in an educational setting. A great need for research in classroom‐based physical activity characterized is needed, specifying meaningful connections between movements and learning objectives. Such research will connect cognitive phenomena with concrete bodily experience, which could optimize the comprehension and memory. A promising challenge for future research is investigating the connection between mind and body as a synergetic catalyst for learning in a classroom setting. Such research could have a great impact on the way we think of and organize our educational system. The point of departure in the paper is inspired by theories of embodiment and phenomenology (Dourish 2004, Apter 2001). We cannot exist as reflective individuals with identity and language without the body's perception of the world. Embodied identity and language is what Merleau‐Ponty refers to, when he talks about the importance of considering the body as something we are, rather than something we have. Merleau‐ Ponty's focus on cognition as embedded in the body connects his philosophy to modern cognitive science and the basic idea of the embodiment hypothesis that the body influences our perception of the world significantly. (Merleau‐Ponty 1945). The body allows interaction with the physical world influencing the perception as the individual organizes the world in relation to bodily and pre‐conceptual meaning. The embodiment‐hypothesis comprehends cognition and consciousness as phenomenons rooted in the body. We can only recognize the world through interacting with it, because thoughts and language cannot be treated separately from the body, senses, movement, nor social interaction or the need for communication. (Lakoff & Johnsson 1999). Our body is the instrument through which we learn and remember. We will describe and analyze the learning potential of a digital paint application. In using the artefact, the application aim at an embodied learning approach. As such, the interface is designed with the purpose of enabling full‐body interaction with the application. Describing the possibilities of different full‐body interactive interfaces is our initial starting point. Then we briefly describe the developed application. By applying essentials of Luhmann’s communication theory and Qvortrup’s understanding of learning, our aim is to explore the possible ways of learning through the application. We introduce three levels of learning: (1) Factual; (2) Situational and adaptive; and (3) Reflective or creative. Aftewards we explore how embodiment and the learning levels relate. We use the painting app as an illustrave example. We conclude that introducing an application with a full‐body interface allows the achievement of all levels of learning.

269

Helle Skovbjerg Karoff et al.

2. Technology as facilitator When using digital learning resources, children should be physically active as part of their learning process. And through interactive movement, they gain new insights into the learning materials. This paper aims to explore how technology can be used as a facilitator in combining knowledge of human movement and learning processes. Children learn when they are experimenting, constructing, interacting and through active participation. Basically we have three types of technological platforms, which support human movement and embodied learning: Handheld Interaction, Interactive Wearable’s and Interactive Surroundings. In actual applications, the platforms might be integrated. First, Handheld Interaction: We learn through touch, by using tangible devices such as smartphones, tablets and interactive cubes (Majgaard, 2009). It is customary to distinguish between screen‐based interaction on the one hand and the purely physical interaction ‐ as with interactive cubes ‐ on the other (Majgaard, 2012). Screen‐based media are chiefly PCs, tablets and smartphones. Screen‐based media's classic strength is their presentation of abstract, visual and auditory symbols through for example video clips, interactive simulations and graphics. Screen‐based units support intellectual learning processes through user interaction and contribution. Interactive cubes can provide a more tangible physical form of symbolic knowledge and they support more intuitive and embodied learning. However, there are indications that the two types of media are merging. Traditional screen‐based media are increasingly getting more physical interactive. And interactive blocks are being equipped with screens (Majgaard, 2012). Second, Interactive Wearables: Wearables, Body Area Networks, and Augmented Reality. Wearables count such things as google glasses, smart phones, GPS watches, heart rate monitors, interactive clothes, accelerometers etc. Body Area Networks are currently mostly being used as part of fitness training. For instance, you can monitor your pulse as a part of a spinning class. Or you can put touch sensors in your running shoes to measure your walking or running style. A very popular application is the Endomondo running app. You attach a smartphone on your arm and starts Endomondo. The program then tracks you by using GPS while being linked to Google Maps. During the exercise, you get precise feedback on speed and distance, supporting training ‐ as well as being an example of learning where the learner takes responsibility. Another growing field is design of interactive clothes, shoes, and accessories. Children can add interactive modules to their clothes such as programmable light diodes. Through using such interactive units, they acquire deeper knowledge of the technology; as well as being active learners and artistically creative (Melgar, 2012). Third, Interactive Surroundings: Sensor Networks and Gesture‐based interfaces. This technology can be integrated in the environment, as part of a room or furniture or some other object. Some of the technologies are not in direct physical contact with the body and requires a more controlled and limited environment. Examples are Microsoft Kinect and Camera tracking and hands‐free speech recognition. One of the most popular Kinect applications is Dance Central – combining game, play and human movement. In the following we will present an example, which illustrates Interactive Surroundings. Further, we will discuss it's learning potential in relation to Qvortrup's (2006), Bateson´s (2000) and Luhmann`s (2006) ideas of learning.

3. Illustrative example: The painting app The paint app prototype was developed by students at the engineering programme Learning and Experience Technology at University of Southern Denmark. The goals of the developed prototype were to enrich social interaction and motor skills among children with special needs and children without a handicap (Christensen et al, 2013). The prototype was a motion sensing input device for Windows PCs similar to Microsofts Kinect, based on the Asus Xtion Pro platform see figure 1(a). An infrared camera enabled users to control and interact with the

270

Helle Skovbjerg Karoff et al. computer without the need to touch a game controller, using only body and hand gestures (Melgar, 2012). The technology is an example of interactive surroundings – and as such it competes with Wii Remote and Eye Toy.

Figure 1: (a) Asus Xtion Pro. (b) Testing the painting prototype (the star is subsequently made more colourful) The graphical user interface of the prototype was divided into parts; see figure 1(b). The left side showed the user's painting and the right side showed a 3D image of the user. The setup simulated a colour pencil in the user's hand and enables switching between 10 different colours, deleting everything on the screen, and switching between five different pencils. The students tested the prototype against a group of four school children from the third grade. The feedback was useful. Besides some minor usability issues, the system was unexpectedly very playful to use. The children also had a lot of creative design ideas e.g. painting using feet instead of hands – and even sang and danced while they waited for their turn. If further developed, the application holds promising possibilities for full‐body interaction, participatory interaction, and creative interaction. What are children learning by using the painting application? In the next section, we will reflect upon that question.

4. The paint application as a learning resource Applying a synthesis of Luhmann's, Qvortrup´s (2006), and Bateson’s (1972) theories, affords us a better understanding of the synergy between movement, learning and technology. Central to Luhmann theory is the notion of a system. According to Luhmann, systems are emergent, interdependent, arranged and simultaneously bound to the surroundings in a non‐hierarchical manner. Autopoiesis is the expression of the system’s capability of automatically recreating itself through exchanging entity with the surrounding environment. A person is considered partly as a psychological system (a conscious or mental system) and partly as a number of other systems and subsystems of the different systems. These systems, the mental system viewed from the phenomenological perspective, the bodies movement system, and the neurological system are interrelated through the communicative processes of ‘structural coupling’ (Luhmann, 2006). In some interpretations of Luhmann's concept of system, a technological concept with hardware and software can be seen as a system that transports and converts (mediates and remediates) information. We understand the painting application as such a system that converts movement actions into visual information. The application reflects visually the children's movements. In that sense, we treat the learning process as an integration and development of systems. According to Luhmann, the integration and development is promoted by disruptions. Disruptions are the result of information bytes making a difference – communicating and thus resulting in learning. When the user is doing the virtual painting, disruptions caused by the palm movements are remediated into visual information that are grasped by the eye. This forces the user to learn how to paint virtually without the kinesthetic feedback that a normal paintbrush affords. Through this, information causing these disruptions is translated from one system’s coding to the other system's coding through structural coupling. The structural couplings indicated by physical movement become a visual line on the applications’ screen – a sign of change indicative of the occurrence of learning.

271

Helle Skovbjerg Karoff et al. According to Qvortrup (2006), the form of communication is divided in levels and thus types of learning processes – evident in his categories of knowledge and learning. Learning‐1 is the learning of facts and skills – in Qvortrup's vocabulary; this corresponds to acquire acquiring qualifications. In the present context, such qualifications amount to getting familiar with the app: What can you do? How do you do it? etc. In addition, the user at level‐1 learning learns the basic use of the line, by changing the line thickness or color. In the perspective of the theory of learning and communication, the learning process can fundamentally be described as a change in behavior. Two aspects of learning are evident: 1) Learning about something – in this context about using a virtual brush and 2) Learning to learn. While you learned the concrete subject matter, you also learned a way to learn. The learner is situated in the learning process. This is called learning‐2 learning. According to Qvortrup, in a broader interpretation this is also a reflective approach to the learners’ own actions. Through the right side of the interface, the user's interaction is mirrored back to her. The application offers a reflecting tool to the user on what normally is tacit and invisible for the child – it’s own movements. Within the communicative act of expressing herself through painting, lies an opportunity for learning a conscious awareness of her own body. Through the two sides of the interface, the application is the mediating unit between the child's artistic expression and her own body as a motor system. Displaying the child's own movements alongside the child's learned creation enables level 2‐learning. Through repetition, this learning form becomes a habit. Consciously changing learning habits to create new learning skills or knowledge is what Qvortrup calls learning‐3 learning. Learning‐3 has many dimensions. It constitutes the creation of new knowledge, of improvisation, of creatively expressing oneself artistically. Based on that approach to level 3‐learning in the interaction we have seen, the child learns how to use a virtual paintbrush as a cultural artifact, skillfully expressing herself in a language of painting. Also we have seen level 3‐learning when the children proposed redesigns of the application. In both forms of level 3‐learning the children create new knowledge. With at least the children contribute simultaneously with two forms of learning at level 3‐learning the critical question might be whether this multi‐dimensional learning agenda occurring in the child create an overload of possible structural couplings of the child as an creating, reflecting and skillful acting entity in the child's desire for communication with the environment. All at the same time, the child simultaneously learns to manage a virtual tool, a new form of cultural skill, and creative expression through a virtual painting medium – and at the same time have the ability to relate reflective to his own body and thus a narcissistic self‐reflection. Will this possible overload of information and structural couplings create a stress that affects the focused communicative act of expressing one self or will it open for opportunities for further parallel learning? Future research will need to emphasise these perspectives. In testing the interface, we also saw that from the very beginning, the children tried other ways of using the application by experimenting with using their feet. At the same time, they involved themselves as co‐ developers by making suggestions for improvements and new features. Future research needs to emphasise the possibility of children initiating learning processes inspired by their play and their communities of practices (Karoff, 2009, 2013).

5. How can embodiment enrich the learning process? What is the relation between levels of learning and embodiment? And how can embodiment enrich the learning process? By embodiment, we mean that intelligence requires a body (Pfeifer, 2007). The body is used for experiencing the world directly and for immediate interaction with the environment (Brooks, 1991). Pfeifer (2007) stresses that intelligence always requires a body and the intelligence is expressed in the interaction. The body is essential for perception and cognition e.g. learning new things. Pfeifer states that embodiment is an enabler for thinking and it is prerequisite for any kind of learning. The body isn’t something troublesome that simply is there to carry the brain around but it is necessary for cognition (Pfeifer, 2007). In the table below we connect the levels of learning and embodiment.

272

Helle Skovbjerg Karoff et al. Table 1: Levels of learning and embodiment Levels of learning and knowledge types

1 Factual Learning new thing by observations and instructions 2 Situational and adaptive Learning new things by experiencing specific situations. E.g. you learn about a new computer system while using it and in this process you become an expert.

3 Reflective or creative Being creative Reflections on which way best to solve problems or improve learning strategies

Embodiment in the Painting App The user is basically sensing by watching and listening. The children were learning while watching how other used the paint app Situational dynamics between system and user are based on the here and know interaction. The learner is experiencing the world directly and his actions are a part of the a dynamic with the world and the actions have immediate feedback on the learners own sensations (Brooks, 1991) The system senses in real time the users’ actions and then produces appropriate responses e.g. senses the immediate location of the child’s hand and draw accordingly. And visa verse the child senses the drawing and the location of her hand and decides in which direction to paint. The children used their hands and bodies while using the paint app. Digital‐physical interplay between children and system The child combines level 1 and 2 knowledge in the process of creating new approaches. The children were creative and used the system in ways it wasn’t indented. They spontaneously developed play situations which involved dancing and singing while painting and waiting in line.

The left column summarises the learning levels inspired by Qvortrup (2006). The column on the right connects each learning level types of embodiment. At level 1 the body is only required of absorbing information through ears and eyes. The learner is not required to interact with his environment in order to learn factual knowledge. Embodiment at level 2 is situated and takes place in the physical world. The learner deals with the “here” and “now” of the environment. The children use their body while interacting with each other and the system. The system senses the children physical activity and reacts immediately by drawing or changing color. The situational character of level 2 links this type of learning closely to the nature of embodiment. And embodied learning thereby has a huge potential of enhancing level 2 learning. At level 3 the children dynamically combine level 1 and 2 knowledge in the process of creating new approaches. In successful formal and informal learning situations all three levels of learning should take place.

6. Conclusion In this article, we introduced aspects of bodily movement, technology, and learning. We introduced three types of technology implementing learning through full body movement. The types of technology are Handheld Interaction, Interactive Wearable’s and Interactive Surroundings. In order to unfold our views on movement, technology, and learning, we introduced the illustrative example: The Painting App. Children used their hands for painting virtual drawings on the wall – a technology categorised as interactive surroundings. We introduced the learning perspective, inspired by Luhmann (2006), Bateson (2000) and Qvortrup (2006). We used Luhmann’s ideas of disruption for learning. We discussed our illustrative example in light of Qvortrup's learning levels. We found enriched learning potentials at all three learning levels. The next step in our research will be to develop and explore these technologies in a bigger study. We find the perspectives for combining full body movement, technology, and learning very promising. This future study will require cross‐disciplinary research. We would like to develop prototypes in all of all the three categories. Most research has been done

273

Helle Skovbjerg Karoff et al. in the area of handheld interaction. Interactive Wearable’s and Interactive Surroundings are upcoming technologies and could be the foundation for future interesting studies.

References Andersen, L. B., & Froberg, K. (red.) (2006). Sundhedsmæssige aspekter af fysisk aktivitet hos børn – et treårigt forsøg i to kommuner ved København Ballerup og Tårnby. København: Sundhedsstyrelsen Apter, M. (2001). Reversal Theory: The dynamics of Motivation, emotion and Personality, Oneworld Publication, NY. Aziz‐Zadeh, L., Wilson, S., Rizzolatti, G., Iacoboni, M. (2006). A comparison of premotor areas activated by action observation and action phrases. Current Biology, 16(18): 1818‐23 Aziz‐Zadeh L., & Damasio A. R. (2008). Embodied semantics for actions: Findings from functional brain imaging. Journal of Physiology – Paris, 102: 35–39. Bateson, Gregory, (2000): Steps to an Ecology of Mind: Collected Essays in Anthropology, Psychiatry, Evolution, and Epistemology. Forlaget Chicago Press. ISBN 0‐226‐03906‐4 Binkofski, F., Buccino, G., Riggio, L. (2004). The mirror neuron system and action recognition. Brain and Language, 89: 370‐ 376 Brooks R. A., (1991b): “New Approaches to Robotics”. ", I Science (253), September 1991, pp. 1227–1232. Buccino, G., Solodkin, A., & Small, S. (2006). Functions of the mirror neuron system: implications for neurorehabilitation. Cogn Behav Neurol, 19: 55‐63. Buechley, L., and Eisenberg, M. (2008). The LilyPad Arduino: Toward Wearable Engineering for Everyone. Wearable Computing Column in IEEE Pervasive. Christensen K., Andersen M., Monsen E., Safiri S., and Hansen J. (2013). Physical‐digital Interaction Design for Children. Conference, IT‐City Katrinebjerg, Aarhus University, Denmark. http://sider2013.au.dk/fileadmin/sider2013/0142‐ paper.pdf Davis et al. (2011) Exercise improves executive function and achievement and alters brain activation in overweight children: A randomized, controlled trial. Health Psychology, Vol 30(1) Dourish, P. (2004). Where The Action Is: The Foundations of Embodied Interaction. MIT Press Ericssons, I. (2003). Bunkefloprojektet: Kort sammenfatning af doktorafhandling ‐ Motorik, koncentrationsförmåga och skolprestationer. Malmö: Lärarutbildingen, Malmø Högskola Glenberg, M. A., Sato, M., Cattaneo, L., Riggio, L., Palumbo, D., og Buccino, G. (2008). Processing abstract language modulates motor system activity. The Quarterley Journal Of Experimental Psychology,. 61(6): 905‐919 Karoff, H.S (2013) “Play Practices and Play Moods” in International Journal of Play, (in press). Karoff, H.S. & Johansen, S.L (2009) ”Materiality, Practice, Body” in Proceedings, for IDC2009, Como, Italy. Lakoff, G., & Johnsson, M. (red.) (1999). Philosophy in the flesh: The embodied mind and its challenge to western thought. New York: Basic Books Luhmann, N. & Bednarz J. Jr. and Baecker, D. (1996). Social Systems ‐ (Writing Science) (First ed.). Stanford University Press. Luhmann, N. (2002), Das Erziehungssystem der Gesellschaft, Suhrkamp Verlag, Frankfurt an Main Lutz, A., Thompson E. (2003). Neurophenomenology: Integrating Subjective Experience and Brain Dynamics in the Neuroscience of Consciousness. Journal of Consciousness Studies, 10(9‐10): 31‐52 Majgaard, G., Nielsen, J. , Misfeldt, M. (2012). The Learning Potentials of Number Blocks. Towards Learning and Instruction in Web 3.0. Advances in Cognitive and Educational Psychology (s. 289‐302). Majgaard, G. (2009). The Playground in the Classroom – Fractions and Robot Technology. Cognition and Exploratory Learning in Digital Age proceedings (pp 10‐17). International Association for Development, IADIS. Melgar, E. R. og Diez, C. C.(2012). Arduino and Kinect Projects: Design, Build, Blow Their Minds. Apress Merleau‐Ponty, M. (1945). Phénoménologie de la perception. Paris: Gallimard Papert, S. (1993). Mindstorms Children, Computers, and Powerful Ideas, Basic Books. Pfeifer, R. and Bongard, J.(2007). How the body shapes the way we think. MIT Press Qvortrup, L. (2006). Knowledge education and learning ‐ e‐learning in the knowledge society, Frederiksberg: Samfundslitteratur Press. Rizzolatti, G., Craighero, L., (2004). The mirror‐neuron system. Annual Review of Neuroscience, 27:169‐92 Schön, D. (1983). The Reflective Practitioner ‐ how professionals think in action. Basic Book Tettamanti, M., Buccino, G., Saccuman, M. C., Gallese, V., Danna, M., Scifo, P., Fazio, F., Rizzolatti. G., Cappa, S. F., Perani, D. (2005). Listening to action‐related sentences activates fronto‐parietal motor circuits. Journal of Cognitive Neuroscience, 17(2): 273‐281 Wenger E. (1998). Communities of Practice: Learning, Meaning, and Identity. Cambridge University

274

Game Based Learning in Mathematics: Teachers' Support by a Flexible Tool Aikaterini Katmada, Apostolos Mavridis and Thrasyvoulos Tsiatsos Aristotle University of Thessaloniki, Department of Informatics, Thessaloniki, Greece akatmada@csd.auth.gr apmavrid@csd.auth.gr tsiatsos@csd.auth.gr Abstract: The inherent difficulty of the core subject of Mathematics makes it hard for students of all ages to fully and sufficiently grasp its concepts and engage with it. With such an important matter still being unresolved, in this context, we propose a Game‐Based Learning approach in order to assist the educational process. The main purpose of this paper is the presentation of the design and development of the configurable online two‐dimensional (2D) game “Volcanic Riddles”. This particular game was designed in cooperation with educators, in order to support the teaching of Mathematics in primary school and the first grades of secondary school. The game is configurable in the sense that the (non‐programmer) educator can easily alter several of its parameters, such as the content and total number of the game’s questions or images, via an administration website. Furthermore, the educator can choose which of the game's specific challenges wants to adapt according to the students' needs and level, and thus reuse the game in various educational contexts throughout the school year. Special attention was given so that the administration website is user‐friendly and does not require any programming or scripting knowledge from the user. Regarding the game, basic features of educational games were incorporated in order to enhance its educational value and effectiveness. More specifically, this paper presents the steps followed for the implementation of the game, its technical aspects, as well as the first impressions and evaluation results after it was piloted in the field using real pupils and teachers. The assessment was focused on its usability, effectiveness and th motivational appeal. In the pilot study, pupils of various nationalities aged 10‐12 (6 Grade), participated. The results are encouraging and suggest that the game can be used as an effective and motivational learning tool. Finally, some corresponding conclusions and suggestions for further improvement and research are being discussed concisely. Keywords: 2D game based learning, primary education, mathematics

1. Introduction In recent years, there has been an increasing interest in the potential use of computer games as learning and teaching tools. Research into computer games has demonstrated that they can contribute to the student’s social, emotional, and cognitive development (Squire 2003), while they offer advantages in terms of motivational effectiveness (Facer 2003). Additionally, they support (a) active, experiential and problem‐based learning, (b) self‐assessment by exploiting the immediate feedback and scoring mechanisms, (c) activation and use of prior learning in order to advance, and (d) transfer of existing learning to unique situations; all those attributes are associated with how people learn (Oblinger 2004). Given these benefits, a growing number of educators are using computer games in various subjects, and there is an ongoing discussion of how games can be successfully integrated into formal school settings. Accordingly, the purpose of this work is to support the learning of Mathematics, a subject that is often considered to be boring and complicated by elementary and middle school students. Moreover, students often face math learning difficulties, to which they may respond with high math anxiety and negative attitude towards Mathematics, and consequently withdraw any effort and adopt avoidance behavior (Ashcraft 2002). These math learning problems range from mild to severe and can have serious consequences to the student's self‐esteem or performance (Garnett 1998). However, by using educational computer games in the classroom, significant improvement has been noted in the students’ Mathematics understandings and achievement (Kebritchi et al. 2010). Taking into account all the above, a configurable 2D computer game has been constructed in an attempt to assist educators in the teaching of Mathematics. After thorough research of every available technology, we decided to implement this game from scratch, with the use of ActionScript and PHP. The aim of the particular game would be to help students practise and develop their math skills in a pleasurable environment, to change the negative attitude they may have towards the subject by making it more interesting, and to give teachers a new and flexible tool for the educational process.

275

Aikaterini Katmada, Apostolos Mavridis and Thrasyvoulos Tsiatsos In order to measure the effectiveness of the particular 2D educational game, this paper addresses the following research questions (RQ): RQ1: What are the impressions and the feelings of the students towards this game? Is it motivationally appealing? RQ2: Can the specific game be successfully integrated and used by educators in everyday scenarios in a formal school setting? The results are very promising, showing that 2D educational games can be integrated into the educational process, along with traditional educational approaches. The rest of the paper is organised as follows: In the following section, a closer look around the theoretical framework of Game‐Based Learning is being taken, along with certain examples of games for Mathematics as well as the research that has been conducted towards them. Afterwards, the components of the final framework, as well as the pilot study are being analysed. Finally, some concluding remarks and suggested steps to be followed are being described in brief.

2. Theoretical framework 2.1 Game‐based learning (GBL) and obstacles to using games in classroom The term Game‐Based Learning (GBL) refers to the educational approach which suggests the use of computer games that retain their entertaining qualities, but also contain educational aspects. These games, as well as the whole educational procedure, should have specific learning objectives and expected outcomes. This way, students are involved in game‐playing and enjoyable situations, while they learn in a learner‐centered, learner‐ guided and interactive environment. According to Prensky (2001), some of the basic features that such games should have, apart from their entertainment aspects, are the following: (a) rules, goals and objectives, (b) outcomes and feedback, (c) conflict/competition/challenge/opposition, (d) interaction, and (e) representation or story. GBL is already being used in various educational fields, with three different methods. According to Van Eck (2006), these three methods are the following: (a) educators use commercial games that have not been developed for education but have the ability to support learning, (b) educators use computer games that have been developed especially for this reason, and (c) educators guide their students towards creating their own educational games. Each approach has its own particular characteristics, as well as advantages and disadvantages. In general, GBL offers important benefits in terms of students’ motivation and engagement, familiarization with technology, and development of problem‐solving skills, regardless of which of the three methods is implemented in traditional education. However, GBL is still an evolving and radical idea, and there are many obstacles to integrating it into traditional education. According to Kirriemuir and McFarlane (2003), and Fisch (2005), special attention should be given to the context of the game so that it is accurate and appropriate, and the goals of the game align with the learning goals of the classroom. Moreover, the following impediments have to be considered (Kirriemuir and McFarlane 2003): Many contemporary games require either new and expensive computer hardware, or upgrades of the existing classroom technology. Also, teachers may have little time to familiarise themselves with the educational components of the game, and they may need support material and descriptions of scenarios that can be enacted through the game. It is also important to mention the limited time span of individual classes. This means that the students cannot play a bigger and more complex game because time cannot be wasted on learning complicated controls or watching introductory full motion video. Finally, there are versions of games that need to be tailored to the curriculum of the individual country.

2.2 Examples of educational computer games for mathematics Before the design and development of the game could begin, a detailed overview of other educational games and relevant research was necessary. As regards the use of computer games in Mathematics education, there are several studies providing empirical evidence of their use for the learning of Mathematics. Ahmad and Latih

276

Aikaterini Katmada, Apostolos Mavridis and Thrasyvoulos Tsiatsos (2010), for example, constructed an educational computer game in order to help primary school students understand fractions, one of the most complex concepts for students of younger ages, with positive results in terms of motivational effectiveness. Zavaleta et al. (2005) have reported on the use of an educational game for primary school Algebra and suggested that it contribute to the students’ better understanding of formal algebraic concepts. According to Ke and Grabowski (2007), computer games can also affect the students’ Mathematics performance favorably and promote positive attitudes, especially when they contain cooperative game playing. These findings are consistent with more recent empirical research, demonstrating that computer games change the students’ perceptions about Mathematics and diminish math phobia (Kebritchi et al. 2010). Concerning relevant research projects, a notable example would be E‐GEMS, the Electronic Games for Education in Math and Science project. E‐GEMS was a collaborative project centered at the university of British Columbia (UBC). Queen’s University, Electronic Arts, and several schools in British Columbia and Ontario participated in this project. It is no longer active, but it still managed to produce many educational games of Mathematics and Logic, as well as several important guidelines to be followed while designing educational computer games (Klawe and Phillips 1995). Several mini‐games can also be found on the Scratch website. Scratch is a programming language that makes it easy to create interactive stories, games and animations and is developed by the Lifelong Kindergarten research group at the MIT Media Lab. Scratch projects are available to download and edit for free, by opening them in Scratch and making changes to the scripts. Additionally, the UWS e‐CLIL Games Engine of the European Resource Centre for Content & Language Integrated Learning has been developed to provide teachers with a tool that will allow them to create simple games in one or more languages. Games created by teachers are available for other teachers to use and adapt for free. Students can then play these mini‐games on various subjects online. Naturally, many similar math games can be found online, on various educational websites. These games are usually quite simplistic, with no plot, and oriented to a specific unit of the subject, for example fractions or decimal numbers. Of course, there are also more complex online games that offer a complete supportive environment to the user, with different features for students and teachers or parents. Predominately, they are commercial products with subscription. Lastly, regarding Greek educational computer games, on the online educational portal of the Ministry of Education there is software designed to support the teaching of Mathematics in the Greek primary schools. Teachers can create their own activities and games, after they have downloaded and installed the software, as well as some specific plug‐ins. This, as well as the fact that it is specifically designed for the Windows operating system, makes the particular software difficult to be used by teachers with little or no computer experience.

3. Design and development 3.1 Development methodology and requirements analysis The production of the final framework has been divided into the following working phases: Analysis, Design, Development, Implementation, and Evaluation, according to the ADDIE model for Instructional Systems Design (ISD). In this model, the final product of one phase is the initial product of the next phase, after it has gone through formative evaluation in order to correct any mistakes in time. Before having proceeded to the game design and development, the necessary information for the learning needs that this specific software was going to cover, as well as any distinctive attributes of the environment that was going to be used in, had to be collected and recorded. Besides the research at the pertaining bibliography, a semi‐structured interview with two primary school teachers took place, and, additionally, the participation of students was observed during classes. The general objectives that were initially set for the final product are the following:

The educator should be able to easily change the questions and the images of the game, and their number should be variable, so that it can accommodate the level and the grade of the students.

277

Aikaterini Katmada, Apostolos Mavridis and Thrasyvoulos Tsiatsos

The educator should be able to change the help messages and the instructions of the game, so that it can support different thematic units.

The educator should be able to choose which specific challenges to change and which challenges the students should play in a school hour.

The game should have the aforementioned basic features of educational games, for example explicit rules and aims, a plot, different outcomes and feedback depending on the player’s actions, as well as elements of opposition and challenge.

Moreover, in order to eliminate the obstacles that can deter an educator from using the game, and after discussion with the two teachers, it has been decided that:

The game should not demand expensive computer hardware or resources, so that schools, especially public ones, can use it.

The game should be played online. This would only require Internet connection, which most schools have today.

The administration and configuration of the game should be via an easy‐to‐use administration website, without requiring any programming knowledge from the teacher.

Animations and video should not be added, as this would slow down the game and confuse the students.

The game should be in English in order to reach a wider audience. Long sentences and unknown words should be avoided, since the game will be addressed to younger ages.

The game should not require any complex computer usage from the players.

Since the implementation details above were established, we have proceeded with the design and development of the complete framework that consists of the configurable online 2D game and the administration website of the game.

3.2 Game description After careful study of the available game development software, it has been decided that the game “Volcanic Riddles” should be developed in Adobe Flash with the use of ActionScript. The graphics and the images of the game were created mainly in Adobe Illustrator and Adobe Photoshop. In order for the game to be configurable, without needing any changes in the Flash file, the changeable context is saved in external text files. Taking into account the initial functional requirements in combination with essential principles of educational game design (Prensky 2001; Fisch 2005), the following structural characteristics were integrated in our game:

A plot/story: To make the game more interesting, a story with three fictional characters was added. In the story, three friends try to escape from the island they reside on before the volcano of the island erupts. This is the main objective of the game. The players win when they have successfully completed the nine rooms (mini‐games) of the game. Each one of these nine rooms has its own minor goal, which is related to the story of the game.

Rules: The players are expected to fully interact with the game, while following a set of specific rules. For example, the players have in their disposal five lives (chances) for each one of the nine rooms of the game and one hundred seconds for each question. When they lose all of their lives or run out of time the game ends. There are two “game modes”: Challenge and Single game mode. If they play in Challenge game mode they have to start again from the beginning of the game (the first room). If they play in Single game mode they can play in another room they choose on the island’s map, or start all over again in the same room.

Objectives and goals: As it has already been mentioned, the main objective is to successfully complete the nine rooms of the game. To complete a room, the players must not lose all of their lives or run out of time.

Feedback: It was also important to consider the design of appropriate help, instructions and feedback. Instructions and help for each room were added, as well as some general instructions, in order to help the players navigate through the game and learn to use it. Immediate feedback is provided upon the players’ actions. For example, in case of a wrong answer, an appropriate and encouraging message with the correct answer appears.

278

Aikaterini Katmada, Apostolos Mavridis and Thrasyvoulos Tsiatsos

Challenge and opposition: The players have a limited number of lives for each room and a specific time period for each question. Every time they make a mistake, they lose a life. In order to gain points, the players have to answer to questions correctly, while each mistake subtracts points from the score. As the game progresses, the difficulty of the questions is increasing, except for room number 4 where the difficulty is increasing when the players answer correctly, and is decreasing when their answer is wrong. Correct answers to more difficult questions are rewarded with more points.

The game also contains an options’ page where the players can read the story, go through the help menu, see the credits and the high scores, adjust the game’s sound, and send us their feedback. Special attention was paid so that the screens are friendly, colourful and age‐appropriate (Figure 1).

Figure 1: Various screens of the game

3.3 Configuration of the game For the management and configuration of the game an administrative web environment was developed. The focus was on making it as user‐friendly and accessible as possible. As it has already been mentioned, the questions and the answers of the game are saved in external text files. These text files have been uploaded to the web server with the game and its images. Therefore, in order to alter the content of the game, all the user needs to do is change the content of these files. This is done with a PHP script that is called via the administration website without the user having to write a single line of code. In order to make the desired changes, the educator can simply edit the appropriate text fields in the administration website and upload the images that will appear in the game (Figure 2). The educators can login to the administrative environment of the game and choose which of the nine rooms of the game they want to alter. There is a preview image of each room so that they know exactly what to change each time and where that appears in the game (Figure 3). Briefly, the educator can make the following actions:

Change the number of the questions, by adding or deleting questions in every room.

Change the context of the questions and the answers in each room.

Change the context of the instructions of each room.

Change the context of the help messages and images of each room.

Change the images of rooms number 7, 8 and 9.

Add a more difficult question for each simple question (in room number 4).

4. Implementation and evaluation 4.1 Educational and supportive material for the game In order to evaluate the usability and the learning and motivational effectiveness of the software, a series of educational activities were organised in the private school we visited for the analysis of the initial requirements. In the pilot study that was conducted, 6th Grade pupils of various nationalities, who follow the

279

Aikaterini Katmada, Apostolos Mavridis and Thrasyvoulos Tsiatsos international K‐12 curriculum, participated. While writing this paper the educational activities with the 7th Grade were still in progress.

Figure 2: Architecture of the whole framework

Figure 3: The administration panel of the game Responsible for the support of the students during the educational activities were mainly the teachers. For this reason, prior meetings were established with them in order to carry out a detailed demonstration of the game and the administration website. Additionally, video tutorials and presentations in the English and the Greek language were created. In this support material, the configuration of the game with the use of the administration panel is explained methodically and there is also a presentation of the game.

280

Aikaterini Katmada, Apostolos Mavridis and Thrasyvoulos Tsiatsos An important issue that had to be taken into serious consideration was the selection of the educational material that would be used in our generic game, so that it would be sound and age‐appropriate. Briefly, the selected questions were from the following units of the Mathematics schoolbook: (1) review of numbers and basic operations, (2) computing distances, (3) using variables and solving one and two‐step equations, (4) operations with negative numbers, fractions, decimals and perfect squares, (5) computing with percentages, (6) ratios, proportions and sequences, (7) geometric figures and their properties, (8) Cartesian coordinates and graphs, and (9) angle measures and properties.

4.2 Description of the educational activities The educational activities that were concluded with twelve (12) pupils of the 6th Grade at the computer laboratory of the school will be described concisely. The group of 12 pupils consists of eight (8) girls and four (4) boys. The first activity was carried out at the beginning of the school year, and the exercises were based on the review of basic geometrical concepts, as well as operations with whole numbers. These questions were in fact repetition questions from the 5th Grade material. The second activity took place in the middle of the school year. This time, the game included questions on new concepts taught in 6th Grade, such as equations, percents, angle measurements, and the Cartesian coordinates. Each one of the game’s nine rooms had questions on a different thematic unit. In both cases, pupils had about 45’ minutes to play the game. They were encouraged to comment on the game and to mention anything that was difficult or confusing. In general, there were no difficulties as far as the usage of software is concerned or any technical problems with the game. The pupils were also encouraged to try the game at home, after the end of the school activities.

4.3 Results After the activities had taken place, in order to obtain quantitative data, pupils were asked to answer a questionnaire about the game’s usability and their opinion about it. Likert scale of 1‐5 was used, where 1 was assigned to “strongly disagree” and 5 to “strongly agree”. The questions were mainly focused on the evaluation of (a) the design and structure of the game’s content, as well as the user interface, (b) the capabilities and technical aspects of the game, and (c) the perceived usefulness and motivational appeal. Measures of central tendency and dispersion were computed, in order to summarize the data for the questionnaire’s variables and to understand the variability of scores for each one. The evaluation method also consists of personal observations from the educational activities, as well as a short interview with the teacher. Concerning the participants’ familiarization with educational games, two pupils strongly agreed and two pupils agreed that they often play similar games. However, three pupils were neutral, two pupils disagreed and three pupils strongly disagreed. The results of the statistical analysis are as follows; mean (M)=2.83, standard deviation (SD)=1.467, number of participants (N)=12. Based on the mean and the large standard deviation, it looks like their level of familiarization with such games varied quite a bit. Nevertheless, the pupils did not encounter any major difficulties in interacting with the game. As regards the initial research questions, the statistical conclusions are the following: RQ1: What are the impressions and the feelings of the students towards this game? Is it motivationally appealing? The pupils’ opinions regarding the game were mainly positive. Concerning the game’s usability, ten pupils strongly agreed that it was very easy to understand how the game is played, while two pupils were neutral (M=4.67, SD=0.778, N=12). Additionally, seven pupils strongly agreed and three pupils agreed that they could easily submit their answers, without that requiring any complex computer usage, whereas two pupils were neutral (M=4.42, SD=0.793, N=12). Besides the ease of use, the pupils’ answers about the game’s structure, navigation, and functions were also quite satisfactory. Furthermore, regarding the game’s screen design, nine pupils strongly agreed and two pupils agreed that its graphics and images are adequate, while one pupil was neutral (M=4.67, SD=0.651, N=12). Lastly, seven of the pupils strongly agreed and two of the pupils agreed that they considered the game to be a motivation to participate more in class and improve their math skills, whereas three pupils were neutral. The statistical results are the following: M=4.33, SD=0.888, N=12.

281

Aikaterini Katmada, Apostolos Mavridis and Thrasyvoulos Tsiatsos RQ2: Can the specific game be successfully integrated and used by educators in everyday scenarios in a formal school setting? Regarding the game, no major technical problems were encountered, and the activities were completed successfully as planned. The teacher considered the game, as well as the administrative environment, user‐ friendly and reliable and suggested further development. Additionally, ten pupils strongly agreed that they did not encounter any problems regarding the completion of an exercise because of a fault in the game itself, and two pupils were neutral (M=4.67, SD=0.778, N=12). Moreover, eight pupils strongly agreed and three pupils agreed that the game could help them learn and/or improve their skills, whereas only one pupil was neutral (M=4.58, SD= 0.669, N=12). Some of our observations from the activities are as follows: (a) the pupils seemed both physically and mentally involved during the activities, (b) they participated actively and constructively in the activities, and (c) they were interested in solving the exercises correctly, as well as helping each other. Finally, the pupils were free to comment and write down on the questionnaire suggestions for future improvements on the game. According to their submissions, the game is “very nice” and “great” and the overall activity was “fun” and “a nice practice”. Additionally, six pupils expressed their interest in the future of the game by suggesting updates and new features they would like to see, such as: (a) competitive or cooperative feature; the game would be more enjoyable if they could cooperate in teams or compete against each other, (b) the addition of more diverse challenges in the game’s rooms. From the activities described above, it became apparent that the game could actually be integrated and easily used by teachers in a classroom, as a supplementary educational tool.

5. Conclusions and future work In this paper we presented a configurable online 2D game that was created for the support of the teaching of Mathematics in primary and the first grades of secondary school, and tested in a real environment, in order to discover its benefits as well as its flaws. This game emphasized on pupils’ involvement in a playful and interactive environment guided by specific goals and rules. Even though this is the first edition of the game, we concluded, based on the results from the pilot study, that the pupils were positive and they considered it to be a friendly and beneficial learning tool. Concerning the administration website, it was well received as user‐ friendly and it can assist the learning objectives the educator wants to achieve. Overall, the whole framework proved to be reliable and stable. Our future work aims at improving the game by enriching it with new challenges and competitive or cooperative features, so that it can be more interesting and appealing to children. Furthermore, we consider it of utmost importance that new features should be added to the administration website for the educator, such as the ability to assign specific exercises to students and the appearance of analytical statistics for the students’ high scores, in order to evaluate the involvement and the progress of each student. At this point, it is important to note that, due to the relatively small number of participants and that of the educational activities, it is not possible to generalize the results of the pilot study regarding the motivational appeal of the game. However, they can be used as a foundation for further investigation. In conclusion, we hope that this game will become a valuable learning tool that can be used alongside with traditional educational approaches.

References Ahmad, W.F.B.W. and Latih, N.H.B.A. (2010) “Development of a Mathematics courseware: Fractions”, Proceedings of the Fifteenth Asian Technology Conference in Mathematics, Kuala Lumpur, Malaysia, 17‐21 December 2010. Ashcraft, M.H. (2002) “Math Anxiety: Personal, Educational, and Cognitive Consequences”, Current Directions in Psychological Science, Vol 11, No. 5, pp 181‐185. Facer, K. (2003) “Computer games and learning”, [online], http://admin.futurelab.org.uk/resources/documents/discussion_papers/Computer_Games_and_Learning_discpaper.pdf [Accessed: 22 April 2013]. Fisch, M.S. (2005) “Making Educational Computer Games ‘Educational’”, Proceedings of the 2005 conference on Interaction design and children (IDC 2005), pp 56‐61, Boulder, Colorado, 8‐10 June 2005. Garnett, K. (1998) “Math learning disabilities”, [online], LD Online www.ldonline.org/article/5896 [Accessed: 22 April 2013].

282

Aikaterini Katmada, Apostolos Mavridis and Thrasyvoulos Tsiatsos Ke, F. and Grabowski, B. (2007) “Gameplaying for maths learning: cooperative or not?”, British Journal of Educational Technology, Vol 38, No. 2, pp 249‐259. Kebritchi, M., Hirumi, A. and Bai, H. (2010) “The effects of modern mathematics computer games on mathematics achievement and class motivation”, Computers & Education, Vol 55, Issue 2, pp 427‐443. Kirriemuir, J. and McFarlane, A. (2003) “Use of computer and video games in the classroom”, Proceedings of the Level Up Digital Games Research Conference, Utrecht, The Netherlands, 4‐6 November 2003. Klawe, M. and Phillips, E. (1995) “A Classroom Study: Electronic Games Engage Children As Researchers”, Proceedings of the First International Conference on Computer Support for Collaborative Learning (CSCL '95), pp 209‐213, Bloomington, Indiana, 1995. Oblinger, D. (2004) “The Next Generation of Educational Engagement”, Journal of Interactive Media in Education, 2004 (8), Special Issue on the Educational Semantic Web, pp 1‐18. Prensky, M. (2001) Digital Game‐based Learning, McGraw‐Hill, New York. Scratch, Lifelong Kindergarten Group, MIT Media Lab, [online], http://scratch.mit.edu [Accessed: 22 April 2013]. Squire, K. (2003) “Video Games in Education”, International Journal of Intelligent Simulations and Gaming, Vol 2, Issue 1, pp 49‐62. UWS E‐CLIL Games Engine, E‐CLIL European project building CLIL resources for language learning, [online], http://e‐ clil.uws.ac.uk/index.php?option=com_content&view=article&id=26:extensions&catid=29:the‐cms&Itemid=40 [Accessed: 22 April 2013]. Van Eck, R. (2006) "Digital Game‐Based Learning: It's not just the digital natives who are restless...", EDUCAUSE Review, Vol 41, No. 2, pp 16‐30. Zavaleta, J., Costa, M., Gouvea, M.T. and Lima, C. (2005) “Computer games as a teaching strategy”, Proceedings of the Fifth IEEE International Conference on Advanced Learning Technologies (ICALT 2005), pp.257‐259, 5‐8 July 2005.

283

Learning Analytics with Games Based Learning Harri Ketamo1, 2 1 Satakunta University of Applied Sciences, Finland 2 Eedu Ltd, Finland harri.ketamo@eedu.fi Abstract: This paper focuses on learning analytics framework behind Math Elements mathematics game. The game, was introduced at ECGBL 2012. The novelty value of this design study is in development process of visualizing and data mining technologies behind the learning analytics. The analytic tools provide 1) easy access to follow progress and 2) real time analysis on the learning process. The analytics gives fast and easy to understand view into learning process, still supporting the story, the game play and motivation towards game play. In this paper we show the user centered development process, the improvements done according to user feedback and open the future research focuses. Keywords: educational data mining, learning analytics, games based learning, artificial intelligence

1. Introduction Games and interactive virtual environments can offer more than just entertainment, they can provide relevant and meaningful information for individual learner, his/her parents, teachers and even for whole educational system in an national level. This, however, requires careful planning in game design, data modeling and learning analytics. Experienced teachers are aware that when a pupil is asked to teach another pupil, both pupils learn. This fact has not been applied enough in educational games, mostly because of a lack of technology and game AI that enables players to teach conceptually challenging themes still remaining easy‐to‐use game play. In terms of constructive psychology of learning, people actively construct their own knowledge through interaction with the environment and through reorganization of their mental structures. The key elements in learning are accommodation and assimilation. Accommodation describes an event when a learner figures out something radically new, which leads to a change in his/her mental conceptual structure. Assimilation describes events when a learner strengthens his/her mental conceptual structure by means of new relations (Mayer 2004). Behavior modeling has a long research background: Neural and semantic networks, as well as genetic algorithms, are utilized to model a user's characteristics, profiles and pat‐terns of behavior in order to support or challenge the performance of individuals. Behavior recording have been studied and used in the game industry for a good time. In all recent studies the level of behavior is limited, more or less, to observed patterns (e.g. Brusilovsky 2001; Houllette 2003). In this study, user behavior, competence and learning were seen as Semantic (neural) network that produces self‐organizing and adaptive behavior/interaction. The behavior is evaluated in terms of the theory about existence of finite number of agents. The AI technology developed, emulates the human way to learn: According to cognitive psychology of learning, our thinking is based on conceptual representations of our experiences and relations between these concepts. Phenomena when the mental structure change is called learning. The data mining and analytics are based on this semantic modeling. When all the skills and knowledge is recorded as semantic network, all the mining can be done in terms of network analysis. The novelty value of this study is in approach: to build games based technologies that enable easy construction of intelligent and human like behaviors and so enables detailed analysis of learning achievements. Furthermore, we know that children are ready to do more work for their game characters than what they are ready to do for themselves. That's why we have developed games where game characters learn like humans do and children can take the role of a teacher. The framework and technology behind the games support detailed learning analytics and it provides real time analysis on learning process, difficulties in learning and challenges in curriculum. More details on Math Elements can be found on Eedu website (Eedu 2013) and in this paper we focus on learning analytics. One of our special focuses has been scientific proof of concept: We have shown educational outcomes as well as motivation towards teaching virtual pets: Under strict laboratory experiment settings, more than 60% of

284

Harri Ketamo players increases their skills remarkably during the gameplay. The outcome in natural learning environment with possibility to longer gameplay is even greater: In fact, we have shown that the best outcome is achieved when there are enough breaks and informal discussions between game play. When the player is responsible for character’s mental development, he/she records also his/her mental conceptual structure during the gameplay. The most important finding is that assessment done according to learning data collected during the game play correlates with assessment done with traditional paper tests: Taught conceptual structure is strongly related to paper tests score received after game play (0.4<r<0.7) with all tested content on mathematics and natural sciences. This is an important result in terms of reliability of the game as assessment/evaluation instrument. Because of this, we can produce detailed diagnostic information about learning (Ketamo 2009, Ketamo & Kiili 2010, Ketamo 2011).

2. Data modeling In economical game theory (Shoham & Layton‐Brown 2009) an agent behavior is widely studied in terms of Nash equilibrium. In this the agents are assumed to know the strategies of the other agents, and no agent has anything to gain by changing only its own strategy. A theory about existence of finite number of agents and their arbitrary relations based on other agent (Dukovska & Percikova 2011) describes a set of attributes or properties that are useful when evaluating the agent behavior: 1) every agent is an entity, 2) every agent exists even it does not have a physical characteristics, 3) every agent chose to be in a state of direct knowledge with other agent according to its free will and 4) every agent is different from others in what it is. In the Math Elements every character is an agent/entity that contains all the taught knowledge as semantic network. That semantic network can be used to produce character behaviors, including reasoning, without connection to the original game or other characters (where the knowledge is recorded). Because of every agent is taught by individual person, each agent is different is skills. Furthermore, as stated before, the knowledge and behavior recorded in agent's semantic network correlates with players real world knowledge. All this together, the agents / game characters are ideal entities to run different simulations, which can be competitions as well as complex data mining procedures. Teaching the character can be seen in terms of Inductive learning theories. The general idea behind Inductive learning theories is that we build our understanding by processing and connecting single concepts into large conceptual understanding piece by piece. Adding new concepts or modifying the existing conceptual structure is always based on previous learning and the context of learning. Because of that, both learning and mental conceptual structures are unique for everyone. Inductive learning has been applied in Math Elements in a way where player teaches the character piece by piece in different learning contexts. While playing, the conceptual structure will grow to thousands of relations and a single teaching phase only has a limited effect on the areas of the conceptual structure already taught. Understanding this phenomenon is valuable when trying to correct a wrongly taught part of the conceptual structure. Naturally, wrong teaching could be corrected by teaching the correct structure enough times. The game AI uses all the taught information to back its decisions, and therefore it takes time to override the wrong relations in the agent’s conceptual structure.

3. High‐level analytics = progress in learning Math Elements game include all mathematics in Finnish curriculum from kindergarten to 2nd grade, which means approximately 300 pages exercises in traditional books. In the game, there are 100 levels and one level represents approximately one school week, 2‐5 pages of exercises, in Finnish school. In the beginning, only first level is open. Rest of the levels will be opened after player has shown skills that forms enough good basis to proceed. Modeling and visualizing the progression in an understandable way for small children is different than visualizing progression for adults. That's why we used game‐like approaches: In all the games the progression is told either in terms of skill trees, appearance of new levels or by score. In following the different visualizations and ideas behind them are described.

285

Harri Ketamo In figure 1 on the left, the first analytics skill tree that shows the progression with colors. Green colors represents completed levels, yellow colors open levels and red represents locked levels. The visualization show also relations between different levels. However, in our pilot studies, users get confused on this visualization, so it was transferred into map‐design (figure 1 right).

Figure 1: Early stage progression summarizing views

Figure 2: Map analytics with dependencies and performance estimations. Users felt the map based progression view more easily approachable than the original skill tree. However, the map‐design looses a lot of information related to relations between different skills. In fact, the map‐design in figure 1 do not have relations between the levels, so the next map‐design (figure 2) consist of different paths that shows the progression and skills. Player can earn different prizes when completing the levels (weeks). Bronze prize represents satisfactory skills and golden prize stars represent good skills. Levels that are observed to be too difficult for learner are locked (figure 2 right) and learner have to earn the access to such levels.

Figure 3: Learning analytics visualizations in levels‐menu When the players, both adults and children, felt game‐like visualizations more easy to approach, also the starting menu was re‐designed to contain this analytics and progress information (figure 3). The open and locked levels are shown in the menu. Furthermore, the outcome (gold, silver, bronze) is shown in levels menu. This is obvious for gamer: also the performance in the game is always show on levels menu. When testing this

286

Harri Ketamo approach, the children didn't even notice that their skills ware analyzed. They just felt that their progression in the game was visualized in very clear way. The adults understood that levels menu is also learning analytics visualization, and they felt it very fast and simple way to get immediate understanding on children's skills and progress in the game. In general, the evaluation/assessment for a single level is more like indicative, but completing a whole grade (40 levels) reveals a good understanding on learner's skills. In fact completing a grade in Math Elements requires skills that would be required to pass the same grade in a Finnish school. In order to give more detailed view on skills and learning, we deigned in‐level analytics especially to show 1) difficulties in learning and 2) progress inside the level.

4. Detailed learning analytics = character development In‐level analytics tool is meant for parents and teachers to quickly observe what learner has taught for his/her pet. The early stage visualization (figure 4) places the first time taught concept in the middle of the left side of the visualization area. After the concept has been taught more, the analytics shows correctly taught concepts in the upper part of the skills ‐area and wrongly taught concepts in the lower part of the area. The quantity of the teaching is visualized in a way that what more the concepts has been taught, that further on the right side they are located in visualization. Quantity of teaching also mens that what more relations a concept do have, that more right it is located. Concepts that has not been taught do not appear in the skills ‐area. In general this is a radar‐view on learning achievements visualization. The visualization itself do provide a lot of information on learning, learning process and performance in game. However, the users felt this radar approach way too complex and scientific to use during the gameplay.

Figure 4: Early stage detailed analytics tool with radar ‐type of visualization. According to user feedback, the information was compressed and instead of radar‐view the learning process was visualized as bar diagram (figure 5). In the bar diagram, the concepts was ordered in a way that most taught concepts was placed in left side, while correct–incorrect axis was done with colors (green, red). This visualization was understood relatively clearly by adults, but especially the children wanted to have an indicator for game performance rather than “rather boring” diagrams. After group discussions with children and adults, also many adults felt that bar ‐type of visualization do not fit into game design in an optimal way, especially because of it do not clearly indicate the character's performance in game in one sight. To override this, the information shown in the visualization was compressed even further. In current visualization (figure 6) the pie chart shows correctly taught concepts in green and incorrect teaching in red. Before player has taught enough skills for his/her pet, there is a gray area showing the player that his/her pet's performance is not good enough to go further in the game. For example, in figure 6 left side, a player can

287

Harri Ketamo immediately see that he/she has not taught enough concepts for his/her pet. In figure 6 right side, the player can see that most of his/her teaching is incorrect.

Figure 5: Early stage detailed analytics tool with bars and colors

Figure 6: Learning analytics with pie chart. The information in this visualization is minimal, but players felt that this is optimal for the game play: too much information takes your focus away from the game, but too little information doesn't help to manage the game.

5. Future research Wrong answers or misconceptions are not the only relevant factor explaining learning outcome. From data mining point of view we could point out two interesting phenomena: 1) the finding about origin of the difficulties in learning fraction numbers, decimal numbers and percent numbers and 2) the relation between transfer in learning and transfer in avoiding learning. The biggest challenge in building learning analytics for game is in how to visualize all the information in a way that supports both the game play and learning. One solution was to divide the learning analytics into two parts: In‐Game Analytics and Parental Analytics. Parental Analytics can also be used as teacher analytics and it provides all the details in learning, while in‐game analytics provides most of the meaningful analytics in compressed form. The skill tree approaches will be revised and we believe we can develop skill trees that serves both game play and learning analytics. This, however, requires a lot of user centered research before completed. Another, even bigger, future research activity is related to building national and global learning analytics: When summarizing the individual game achievements, schools and national level policy makers can receive analysis about competences and skills in general level. They can apply this in order to develop their teaching instructions or formal curriculum. Our goal is not to rank countries, we’ll provide information for developing the practice: the analytics are that detailed that we can point out general bottlenecks of education. No matter

288

Harri Ketamo how good some country is in PISA, there is always something to improve: e.g. in Finland there is a clear bottleneck related to fraction numbers with odd nominator.

Figure 7: Global learning analytics Currently we have data from 106 countries with more than 50 000 players (figure 7), but that's not enough for global analysis. Future research consist of (really big) data collection and experimental studies in order to validate the framework in global context. After we do have enough players all over the world we can monitor in nearly real time how players are learning all over the world. Furthermore, we can get immediate feedback on cultural differences in learning and we can localize the game when cultural dependent challenges observed from the big data.

6. Conclusions Games and gamification are the new form of storytelling and social interaction for younger generation. Furthermore, learning has always been about storytelling and social interaction. Keeping in mind that children and young adults are ready to do more work for their game characters than what they are ready to do for themselves, we should be very interested on developing methods to take full advantage on this for educational purposes. According to our studies, users can relatively quickly and easily teach behavior to a game character. In terms of conceptual learning, the developed game AI emulates the way people learn: learning is about concepts and their relations. The semantic modeling makes it possible to model learning process and thus uncover the frequencies, dependencies and patterns behind conceptual learning. Based on knowledge and behavior taught for game character (agent) we can run both analytics and simulations for the character that highly represents the behavior and knowledge of the player. Because in this approach the characters are not depending on one context and game, additionally to the analytics we can produce human‐ like behaving game characters for Math Elements and other related platforms. Finally, designing adaptive learning technologies based on detailed learning analytics, we can build solutions and products that can reveal the blind spots of learning. This can be done, even with today’s technology, without teacher. In other words, we are solving the challenge between scale and quality in education: Qualified teacher can apply technologies in order to understand pupils and student’s individual learning needs in real time.

References Brusilovsky, P. (2001). Adaptive Hypermedia. User Modeling and User‐Adapted Interaction, vol 11, p. 87‐110. Dukovska, S.C. & Percinkova, B. (2011). A model that presents the states of consciousness of Self and Others. International Journal of Mathematical Models and Methods in Applied Sciences. Volume 5(3), pp. 602‐609. Eedu ltd.(2013). Website: www.eedu.fi. Houlette, R. (2003) Player Modeling for Adaptive Games. In Rabin, S. (ed.) AI Game Programming Wisdom II. Massachusetts: Charles River Media, Inc. Ketamo, H. (2011). Sharing Behaviors in Games and Social Media. International Journal of Applied Mathematics and Informatics, vol. 5(1), pp. 224‐232. Ketamo, H. (2009). Semantic networks ‐based teachable agents in an educational game. Transactions on Computers, vol 8(4), pp. 641‐650. Ketamo, H. & Kiili, K. (2010). Conceptual change takes time: Game based learning cannot be only supplementary amusement. Journal of Educational Multimedia and Hypermedia, vol. 19(4), pp. 399‐419. Mayer, R. (2004) Should there be a three‐strikes rule against pure discovery learning? American Psychologist, 59,14‐19. Shoham, Y. & Leyton‐Brown, K. (2009). Multiagent Systems: Algorithmic, Game‐Theoretic, and Logical Foundations. New York: Cambridge University Press.

289

Gamification and Intelligent Feedback Mechanisms for a Division Learning Tool Michael Kickmeier‐Rust and Dietrich Albert Knowledge Management Institute, Graz University of Technology, Austria michael.kickmeier‐rust@tugraz.at dietrich.albert@tugraz.at Abstract: Educational computer games are a highly popular but also a highly challenging field when it comes to an effective and efficient adoption in the classrooms. This holds true from the perspective of the necessary technical infrastructure, from the pedagogical embedment as well as the perspective of a meaningful and formative use of individual gaming results. Not least, there are increasingly critical questions about the effectiveness of using high quality computer games for (usually limited) subject matter. In the context of a European project we developed a rather light weight tool for learning and practicing divisions. The target age group of the tool is 6 to 8. To benefit from the motivational potential of games we used a “gamification” approach. Accordingly, we designed and developed a game‐like, attractive user interface and integrated elements of competition. The system is capable of providing students formative, competence‐based feedback in real‐time. Tailored to the age group this feedback is displayed in form of a smiley and a text block, the latest beta version also provides a text‐to‐speech output of the feedback. The theoretical foundation for the real‐time analysis is Competence‐ based Knowledge Space Theory on which basis competence states can be identified. Concretely, for the tool this means that the feedback says not only that an action was correct or incorrect but the feedback refers to the underlying skills. The tool thereby can distinguish which skills are available and which are lacking by associating the actions with a competency structure of the domain. We applied and evaluated the tool in Austrian classrooms and found some evidence for the motivational aspect of the gamification elements, in particular the scoring. We also found positive effects of an individualized and meaningful feedback about errors. Finally, there occurred certain gender difference, for example, girls were much less attracted by competition elements (e.g., by comparing high scores) then boys, however, more attentive towards feedback coming from the tool. Keywords: gamification, adaptivity, formative feedback, competence‐based knowledge space theory, evaluation

1. Gamification Educational computer games are a highly popular but also a highly challenging field when it comes to an effective and efficient adoption of modern computer games in the classrooms. This holds true from the perspective of the necessary technical infrastructure, from the pedagogical embedment as well as the perspective of a meaningful and formative use of individual gaming results. Not least, there are increasingly critical questions about the effectiveness of using high quality computer games for (usually limited) subject matter. One approach is to make use of the enormous quality and motivational potential of existing computer games (the so‐called commercial off‐the‐shelf games) or simulations for targeted educational purposes. Examples were reported by Kurt Squire (e.g., 2003) in the context of learning from simulation games such as Civilization or Age of Empires, Constance Steinkuehler in the context of massively multiplayer games (e.g., Steinkuehler & Johnson, 2009), Maja Pivec (2007), Sara de Freitas (e.g., de Freitas & Maharg, 2010), or David Shaffer (e.g., 2006). A recent trend is the concept of “gamification”, which refers to the idea of utilizing game characteristics and game features for non‐game applications in order to make them more fun, more engaging, and perhaps educationally more effective. Among the most regularly and successfully utilized gamification features is goal‐setting including progress paths and badges, awarding the player to identify goal completion. Ling et al. (2005) argues that the most motivating goals are those just out of comfortable reach and that this technique is most effective when users can see their progress toward the end goal. Furthermore, people often increase engagement and efforts when they believe that they are close to a specific goal (Fox & Hoffman, 2002). A related (or resulting) technique is providing badges – little but visible indicators of achievement, success, and ability – perhaps even status. Even if players never earn the badges, through viewing a set reachable and accomplishable challenges they come to understand valued activities within the system. Of course, it needs to be mentioned that tokenizing the achievements of players bears certain downsides, for example the avoidance of competition and fear of failure (cf. Montola et al., 2009). Other techniques are levelling (i.e., granting players access to new levels of the system – just novel interfaces, in its simplest case), graphical enhancements of the system, the use of (visually appealing) avatars, the implementation of additional challenges and quests, or the provision of mini games (such as board, card, or racing games) for diversion and recreation. Finally, an important aspect of gamification

290

Michael Kickmeier‐Rust and Dietrich Albert is sweepstakes, lotteries, and “real” giveaways. However, as Alfie Kohn in his book “Punished by Rewards” (1999) argued, this type of motivators may provoke more (perhaps too much) concentration on game‐related achievements while producing lesser quality. In his example he demonstrated that children draw more pictures, but in lesser quality, when paid for drawing pictures, more importantly, children did not like drawing pictures as much as before after they were stopped being paid (Kohn, 1999).Despite the risks, there is a clear trend towards the application of gamification (cf. Anderson & Rainie, 2012 for a critical review). Future work, however, must increasingly address the actual effects and benefits of gamification, in particular in school settings. In the context of a European project we developed a rather light weight tool for learning and practicing divisions. This tool incorporates a set of gamification features and this paper describes the preliminary results on motivation and achievement when applying this tool.

2. Intelligent, formative feedback The notion of gamification and the related techniques, provide a natural and convincing link to the ideas of formative feedback, which is considered a key driver of successful education. Especially new technologies enter the field, aiming at supporting teachers in gathering large scale data, aggregating and analysing them, and perhaps most importantly, to visualize and present the outcomes of analyses in a way that is most useful and beneficial for the learners. Certainly, such attempt is not new; teachers of all times have focused on supporting their students to the best possible extent and to bring them forward ‐ to identify knowledge/competence gaps to inform learners and to facilitate a deeper understanding. This trend is accompanied with the daily growing mass of data available about students. Information and communication technologies (ICT) enter the classrooms more and more, various online sources are linked together and are used for educational purposes and specific educational software and technical devices (e.g., tablets and smartphones) are used to enrich teaching and learning. Thus, it is not surprising that the communities of learning analytics (LA) and educational data mining (EDM) growing hand in hand with the stat‐of‐the‐art in formative feedback. Formative assessment enables teachers to break out of the conventional routines of lesson, exam, announcement of grades, new lesson, new exam, etc. Formative assessment, in essence, means identifying the current differences between current knowledge states and the educational target states of learners with the prime goal of promoting an effective competence development on an individual basis (Smit, 2009). “Formative” means identifying ways to utilizing the value of proper communication between learners and teachers, to strengthen an active role of learners, to optimize teaching/learning on an individual basis, and to acknowledge the psycho‐social value of assessment/appraisal (Black & Wiliam, 2006). According to Gijbels and Dochy (2006), key factors of formative assessment are appropriate, effective, and tailored feedback to learners, assigning responsibility for one’s own learning, discovering the need of learners to evaluate and appraise themselves appropriately, adjusting the teaching activities according to the insights of assessment, and finally, acknowledging the motivational aspect inherent to assessment and appraisal and the related impact on self‐esteem or perception. In principle LA techniques specifically target at supporting pedagogical activities by providing assistance to teachers in practically relevant questions (e.g., the quality of learning material or the engagement of students in specific exercises). In gamified systems, it is necessary to provide learners with direct and immediate feedback in an (a) non‐ distracting and (b) meaningful and formative way. In this paper we briefly describe an approach to provide autonomous formative feedback on the conceptual foundations of Competence‐based Knowledge Space Theory.

2.1 Competence‐centred learning analytics for formative feedback In the background of the Sonic Divider work a web services platform named ProNIFA; the name stands for probabilistic non‐invasive formative assessment and, in essence, establishes a set of server‐based algorithms as well as a handy user interface for educators to conduct the aggregation of educationally relevant data and the according analysis and visualization functions. Conceptually, these functions are based on Competence‐ based Knowledge Space Theory (CbKST), originally established by Jean‐Paul Doignon and Jean‐Claude Falmagne (1999, 2003), which is a well elaborated set‐theoretic framework for addressing the relations among problems (e.g., test items). It provides a basis for structuring a domain of knowledge and for representing the knowledge based on prerequisite relations. While the original idea considered on performance (the behaviour; for example, solving a test item) only, extensions of the approach introduced a separation of observable

291

Michael Kickmeier‐Rust and Dietrich Albert performance and latent, unobservable competencies which determine the performance (Albert & Lukas, 1999). CbKST assumes a finite set of more or less atomic competencies (in the sense of some well‐defined, small scale descriptions of some sort of aptitude, ability, knowledge, or skill) and a prerequisite relation between those competencies. A prerequisite relation states that competency a (e.g., to multiply two positive integers) is a prerequisite to acquire another competency b (e.g., to divide two positive integers). If a person has competency b, we can assume that the person also has competency a. To account for the fact that more than one set of competences can be a prerequisite for another competency (e.g., competency a or b are a prerequisite for acquiring competency c), prerequisite functions have been introduced, relying on and/or‐type relations. A person’s competence state is described by a subset of competencies. Due to the prerequisite relations between the competencies, not all subsets are admissible competence states. By utilizing interpretation and representation functions the latent competencies are mapped to a set of tasks (or test items) covering a given domain. By this means, mastering a task correctly is linked to a set of necessary competencies and, in addition, not mastering a task is linked to a set of lacking competencies. This assignment induces a performance structure, which is the collection of all possible performance states. Recent versions of the conceptual framework are based on a probabilistic mapping of competencies and performance indicators, accounting for making lucky guesses or careless errors. This means, mastering a task correctly provides the evidence for certain competencies and competence states with a certain probability. ProNIFA retrieves educationally relevant performance data and updates the probabilities of the competencies and competence states in a domain. When a task is mastered, all associated competencies are increased in their probability, vice versa, failing in a task decreases the probabilities of the associated competencies. A distinct feature in the context of formative assessment is the multi‐source approach. ProNIFA allows connecting the analysis features to a broad range of sources of evidence. This refers to direct interfaces (for example to Google Docs) and it refers to connecting, automatically or manually, to certain log files. Using this level of connectivity, multiple sources can be merged and can contribute to a holistic analysis of learners’ achievements and activity levels. As an example, ProNIFA enables a teacher to use the results of a Moodle test, exercises done in Google Spreadsheets, and the commitment displayed in a meeting in a virtual environment, to conduct a semi‐automated appraisal of students. The interpretation of the sources of evidence occurs depending on a‐priori specified and defined conditions, heurists, and rules which associate sets of available and lacking competencies to achievements exhibited in the sources of evidence. Very basically, the idea is to define certain conditions or states in a given environment (no matter if a Moodle test or a status of a problem solving process in a learning game). Examples for such conditions may be the direction, pace, and altitude a learner is flying with a space ship in an adventure game or a combination of correctly and incorrectly ticked multiple choice tasks in a regular online school test. The specification of such state can occur in multiple forms, ranging from simply listing test items and the correctness of the items to complex heuristics such as the degree to which an activity reduced the ‘distance’ to the solution in a problem solving process (technically this can be achieved by pseudo code scripting). The next step of this kind of planning/authoring is to assign a set of competencies that can be assumed being available and also lacking when a certain state occurs. This assumption can be weighted with the strength of the probability updates. In essence, this approach equals the conceptual framework of micro adaptivity as, for example, described by Kickmeier‐Rust & Albert (2011).

3. The Sonic Divider The division tool we introduce here is called Sonic Divider. The tool is design to rehearse the formal sequence of written division; the target age group of the tool is 6 to 8 years. Sonic Divider, in the first instance, is based on the domain of basic fractions of the nature 854/4; the divisor is always a single digit number and fractions do not have a remainder. Together with math teachers we analysed the domain established in total 35 skills involved in the domain and established a competence structure (in the CbKST sense); box 1 shows example skills. The division tasks are shown in Figure 1. To master a task, a student must drag ‘n drop the numbers displayed in the upper right corner of the window to its correct position in the division. Depending on the configuration settings made by the teacher, it may be mandatory to follow a fixed sequence of correctly processing written divisions. By the example show in the figure, first the question how often 2 is in 4 must be answered in the first box of the result, which is 2. Subsequently, the remainder in this case 0, must be added to the box in the computation area. Incorrect drops can be removed by using the eraser icon. Depending on the

292

Michael Kickmeier‐Rust and Dietrich Albert configuration settings, a task can be skipped, finished incompletely, finished incorrectly, or a task must be processed until the result is fully correct. Box 1: Types of modelled skills. Multiplication table (1x1) for the number space 100: separated by number rows 2 to 9; “Is in” rule for the number space 100: separated by number rows 1 to 9; Inverse task rule: adding up (“5 plus 3 makes 8”) : separated by number rows 1 to 9; Formal written division procedure; Sequence of written divisions; Division step 1: divider number is in number; Division step 2: multiplication and summing up; Division step 3: forming the new dividend; Mental arithmetic in the number space 100; Secure orientation in the number space 100; Digit space appropriate writing; Identification / estimation of the digit space oft the result in advance; Rough result estimation;

As mentioned, the design of the tool is kept very simple and with the idea of incorporating gamification elements. Concretely we implemented a scoring function; for each task certain points can be achieved that depend on the difficulty of the task. Depending on the general configuration (whether a task must be processed correctly or whether false result are possible to continue) several scoring options are available. In the simplest case the points are added to the student’s score when the task is correct. Another option is to deduct a certain amount of points from the maximum for a specific task for each incorrect action made during the division process. Also the time needed to process a task can be used to alter the points for a task. In addition, competition (within a class) is made possible by sharing and comparing high scores.

Figure 1: Screen shots of a division task and a smiley‐based feedback As mentioned, a special feature of Sonic Divider, however, is the feedback mechanism. The system is capable of providing students formative, competence‐based feedback in real‐time. Tailored to the age group this feedback is displayed in form of a smiley face (again a gamification feature) and a text block, the latest beta version also provides a text‐to‐speech output of the feedback. The theoretical foundations for the real‐time analysis are Competence‐based Knowledge Space Theory. As described above, on the basis of this theory competence states can be identified. Concretely, for the Sonic Divider this means that the feedback says not only that an action was correct or incorrect but the feedback refers to the underlying skills. The tool thereby can distinguish which skills are available and which are lacking by associating the actions with the skill structure of the domain. As an example, the tool can identify if a student has difficulties with the correct sequence of the written division process, if a student has problems with summing up remainders, or if a

293

Michael Kickmeier‐Rust and Dietrich Albert student has problems with a specific number row (meaning that a student might be able to perform calculations with even numbers very well but at the same time might have difficulties with the number 7). It is also important to emphasize the probabilistic character of the feedback mechanism. Feedback on problems or errors is not triggered immediately after the occurrence of an incorrect action. In a CbKST sense, with each action the probabilities of skills and skill states are increased or decreased. Feedback is triggered when a certain pre‐defined probability threshold is reached for a skill/skill state. As an example, take the knowledge about the correct sequence of written divisions; the initial probability of that skill might be 0.50. A student might have a problem with this sequence, the first action might be wrong and consequently the probability of this skill might be decreased to 0.40. The second action might be correct and the probability would go up to, say, 0.47 again. The third and fourth action might be incorrect again and the probability would go down to 0.27. Now a threshold, say, 0.30, is under run and now a tailored feedback would be triggered: the smiley would turn red and say something like ‘I noticed that you are not always using the correct sequence of written divisions [and given that the calculations themselves are correct] although your results are actually correct. Maybe you should have a look on the correct sequence of solving written divisions?!?’. Another positive aspect of the CbKST approach is that analysis and feedback is not limited to the current task, the probability updates are made over the entire domain. This means that in task 1 and error might occur for skill x and another in task 4 and another in task 5. Consequently the feedback would occur in task 5. The assignment of feedback rules can be freely made by a teacher. This means a teacher can specify the rules, the probabilities, thresholds, as well as the exact sentences and smiley colours.

Figure 2: Screen shots of the teacher tool As described so far, the Sonic Divider is a tool for practicing divisions and for providing students with feedback in an autonomous way. We also developed a feature that allows teacher to interpret and evaluate the activities in a formative sense – which is not trivial. Consider a class of 25 students and the set of 71 division tasks. It would take enormous efforts for a teacher to analyse the results with respect to the involved skills – for example to identify with which number row an individual student has difficulties. This, however, was the foundation of a meaningful formative feedback. Other highly important aspects and skills cannot be identified by such kind of work at all, for example the calculation sequence cannot be identified on the basis of the mere results. In order to support the teacher’s formative evaluation and subsequent meaningful support of students, Sonic Divider has a teacher analysis tool, shown in Figure 2. A teacher can open the log files or summary files generated by the system and display the results and meaningful summaries. As shown in the figure, one option is to display the general results, the time spend in total and the average time per task. In addition, the number and percentages for certain error types (e.g., aforementioned sequence errors or number row errors) can be shown, and finally also the probabilities of the skills and skill states. Besides the textual summary also graphs can be displayed for the error types. These types of summaries can be displayed on an individual level as well as summarized for several students.

4. Evaluating the Sonic Divider and its features In order to evaluate the principles and features of the tool we applied the tool in a primary school in Graz, Austria. The main purpose of this evaluation was not the tool itself but rather to look into the effects of gamification (and potential downsides) as well as CbKST‐based intelligent feedback mechanisms in educational

294

Michael Kickmeier‐Rust and Dietrich Albert settings. The tool is presently applied for rehearsing divisions in a 2nd grade class with 25 children. The study is rather qualitative in nature and also serves the ideas of co‐designing the look and feel of the tool and the implemented feedback styles. One part of this evaluation was a short survey after using the tool in class for the first time. In total each child could use the Sonic Divider for 30 minutes. The average time spend with the tool was 18.6 minutes (girls: 16.7, boys: 22.1 minutes). The survey consisted of six short questions that could be answered on a five‐step rating scale made of smileys ranging from 1 – “dislike it” to 5 – “like it very much”. The descriptive results are shown in the chart below (Figure 3).

1 … Using the Sonic Divider for practicing divisions in general 2 … Possibility to achieve scores for mastering divisions 3 … The look of the Sonic Divider, especially the smileys 4 … Getting feedback by the smileys colours 5 … Getting right/wrong feedback by the smileys 6 … Getting feedback where I did well and where I didn’t

Figure 3: Descriptive results of a survey after using the tool.

5. Conclusion The results indicate that the children much appreciated using the Sonic Divider for practicing divisions. Informal discussions with the children revealed that using the tool was more attractive and motivating than regular work on paper. This is remarkable to a certain extent because in fact the tool is not a game but incorporates very basic gamification elements such as scoring and the feedback by the smileys. One reason for these findings might also be the fact that children were not evaluated or monitored by a human teacher but got some performance feedback directly by the system. All in all, boys did rate the Sonic Divider and the feedback features slightly better than the girls. One distinct difference was the rating of the scoring feature. The possibility to obtain high scores was much more liked by the boys – which confirms a gender cliché to a certain extend. We also observed that boys immediately started comparing the scores among them without being told to do so or without even mentioning the possibility to do so. Future work will gather more systematically information about the gamification and feedback features and their effects. A focus will be on comparing learning performances and also on the effects on the attitude towards mathematics. In conclusion, we argue that such minor and cost effective form of gamifying learning tools can boost motivation and engagement.

Acknowledgements This work is supported by the European Community (EC) under the Information Society Technology priority of the 7th Framework Programme for R&D under contract no 258114 NEXT‐TELL. This document does not represent the opinion of the EC and the EC is not responsible for any use that might be made of its content.

References Albert, D., & Lukas, J. (Eds.). (1999). Knowledge spaces: Theories, empirical research, and applications. Mahwah, NJ: Lawrence Erlbaum Associates.

295

Michael Kickmeier‐Rust and Dietrich Albert Anderson, J., Q., & Rainie, L. (May 18, 2012). Gamification: Experts expect ‘game layers’ to expand in the future, with positive and negative results: Online: http://www.pewinternet.org/~/media/Files/ Reports/2012/PIP_Future_of_Internet_2012_Gamification.pdf Black, P., & Wiliam, D. (2006). Developing a theory of formative assessment. In J. Gardner (Ed.), Assessment and learning (pp. 81‐100). London: Sage. Doignon, J., & Falmagne, J. (1999). Knowledge Spaces. Berlin: Springer. de Freitas, S., & Maharg, P. (Eds) (2010). Digital games and learning. London & New York: Continuum Press. Falmagne, J., & Cosyn, E., Doignon, J., & Thiéry, N. (2003). The Assessment of Knowledge, in Theory and in Practice. Institute for Mathematical Behavioral Sciences, UC Irvine, USA. Available online at http://www.scribd.com/doc/3155044/Science‐Behind‐ALEKS. Fox, S., & Hoffman, M. (2002). Escalation behavior as a specific case of goal‐directed activity: A persistence paradigm. Basic and Applied Social Psychology, 24, 273–285. Gijbels, D., & Dochy, F. (2006). Students’ assessment preferences and approaches to learning: can formative assessment make a difference? Educational Studies, 32(4), 399‐409. Kickmeier‐Rust, M. D., & Albert, D. (2011). Micro adaptivity: Protecting immersion in didactically adaptive digital educational games. Journal of Computer Assisted Learning, 26, 95‐105. Kohn, A. (1999). Punished by Rewards. The Trouble with Gold Stars, Incentive Plans, A's, Praise, and Other Bribes. Boston: Houghton Mifflin. Ling, K., Beenen, G., Ludford, P., Wang, X., Chang, K., Li, X., Cosley, D., Frankowski, D., Terveen, L., & Rashid, A.M. (2005). Using social psychology to motivate contributions to online communities. Journal of Computer‐Mediated Communication 10, 1‐10. Montola, M., Nummenmaa, T., Lucero, A., Boberg, M., Korhonen, H. (2009). Applying Game Achievement Systems to Enhance User Experience in a Photo Sharing Service. Proceedings of the 13th International MindTrek Conference: Everyday Life in the Ubiquitous Era on ‐ MindTrek '09, 94. Pivec, M., & Kearney, P. (2007). Games for Learning and Learning from Games. Informatica, 31(4), 419‐423. Shaffer, D. W. (2006). How computer games help children learn. New York: Palgrave Macmillan. Smit, R. (2009). Formative Beurteilung und ihr Nutzen für die Entwicklung von Lernkompetenz [Formative assessment and ist value fort he development of learning competence]. Hohengehren: Schneider. Squire, K. (2003). Video games in education. International Journal of Intelligent Simulations and Gaming (2), 1. Steinkuehler, C., & Johnson, B. Z. (2009).Computational literacy in online games: The social life of a mod. The International Journal of Gaming and Computer Mediated Simulations, 1 (1), 53‐65.

296

Developing Games for Health Impact: Case Brains vs Zombies Kristian Kiili1, Manuel Ninaus2, Mikko Koskela1, M Tuomi1 and Antero Lindstedt3 1 Tampere University of Technology, Pori, Finland 2 University of Graz, Graz, Austria 3 Flow Factory Ltd., Pori, Finland kristian.kiili@tut.fi manuel.ninaus@uni‐graz.at mikko.koskela@tut.fi pauliina.tuomi@tut.fi antero.lindstedt@flowfactory.fi Abstract: The potential use of games for serious purposes is huge because a large and growing population is already engaged with playing entertainment games. However, only a tiny fraction of the overall playing time is devoted to games that are designed for learning or health impact. The challenge is to develop such games that have broad enough impact that transfers to a variety of tasks. Working memory is of central importance for acquiring knowledge and involved in a variety of complex cognitive tasks and thus the use of working memory training games can lead to a wide range of significant impacts in peoples’ life. The aim of this paper is to shed light on game design decisions that are founded on cognitive and neuropsychological theories, focusing especially on working memory training. We scrutinize the development of working memory game titled Brains vs Zombies that is designed to have transferrable impact on brain health and promote physical health in some level. The results of two small‐scale pilot studies in which the implementation of Brains vs Zombies tablet game were studied are reported. The paper focus especially on user interface solutions of the game comparing button and motion based solutions. The results indicated that the opinions about the user interface solutions varied, but children appreciated the motion based user interface more than older university students. Furthermore, the findings showed that meaningful game elements can be added to usually monotone brain training programs to engage users. Keywords: working memory, serious game, exergame, user experience, user interface, pilot study

1. Introduction In recent years Game studies has rapidly developed into an important interdisciplinary research field as well as a nascent academic discipline. The rapid growth of the game industry has aroused wide interest also in other domains besides entertainment. In fact the potential use of games in different settings is huge because a large and growing population is already engaged with playing entertainment games. During the last decade researchers have started to understand the meaning of cognitive, neurological, psychological, physical and pedagogical foundation of serious games design (e.g. Kiili & Perttula, 2013; Kiili et al., 2012; de Freitas & Liarokapis, 2011; Kickmeier‐Rust et al., 2011; Prins et al., 2011; Ketamo & Kiili, 2010;) and as a result the possibilities to develop high‐quality serious games has increased. Nevertheless only a tiny fraction of the overall playing time is devoted to games that are designed for learning or health impact. By definition games for impact need to produce such neurological, physiological or behavioural changes that are of value to society. Above all, the challenge is to develop games that have broad enough impact that transfers to a variety of tasks; especially desirable are observable benefits for untrained skills and tasks. Research on serious games as well as game industry have at some level attempted to address the biggest threats to our society: obesity, aging, social exclusion and memory diseases. There is an urgent need to develop usable and effective non‐medical treatments that can be used to prevent and diagnose memory diseases and promote brain health in different age groups. Serious games, especially working memory games can provide a potential vehicle to deliver such controlled brain training interventions. Obesity has also become a big problem in many countries recently. According to Gorgu, O’Hare and O’Grady (2009) the reasons for obesity include a high calorie diet and a serious lack of physical activities. It has been argued that video games are one of the main reasons for physical inactivity (Vanderwater, Shim & Caplovitz, 2004). The emerging exertion game genre has tried restrain obesity problem by encouraging players to perform physical movements during gameplay.

297

Kristian Kiili et al. In this paper we consider serious games that are designed to have an impact especially on brain health, obesity and fitness. The aim of this paper is to shed light on game design decisions that are founded on cognitive and neuropsychological theories, focusing especially on working memory training. First we consider the theoretical foundation of working memory games following with health impact of exergames. Secondly, based on this theoretical basis we scrutinize the development of a working memory game titled Brains vs Zombies that is designed to have transferrable impact on brain health and promote also fitness in some level. After that we present the results of two pilot studies in which the implementation of Brains vs Zombies game were studied.

2. Theoretical foundations for working memory games Working memory is a brain system that enables us to retain information over a brief period of time, thus a temporary storage for information. This limited storage allows us also to manipulate retained information in mind (Baddeley, 2003). Working memory is of central importance for acquiring knowledge (Pickering, 2006) and involved in a variety of complex cognitive tasks and abilities (Klingberg, 2010). Alloway and Alloway (2010) have demonstrated that working memory is even a better predictor for scholastic achievement than intelligence. Thus working memory is also a good predictor for school‐relevant tasks such as mathematical skills and reading comprehension (Buschkuehl, Jaeggi, & Jonides, 2012). Gathercole and colleagues (2006) showed that the severity of deficits in reading as well as mathematics in children with reading disabilities was closely associated with working memory. They suggest that an effective management of working memory load in structured learning activities may decreases learning problems that are associated with impairments of working memory. It is plausible that working memory plays an important part in everyday life and if trained properly working memory training can support and enhance this brain system, that is crucial for acquiring new knowledge and skills (Pickering, 2006). Given that working memory training can lead to a wide range of significant improvements, it is not surprising that research as well as industry is interested in improving working memory or working memory trainings, respectively. Claims have been made by several commercially available brain training programs that they are improving general cognitive function. However there is a lack of empirical support and recent studies implying that numerous commercially available brain training programs can’t prove their claims sufficiently (e.g. Owen et al., 2010). Nevertheless high quality research has been conducted to examine the effects of working memory training. Several empirical studies demonstrated positive effects on fluid intelligence, reasoning, cognitive control and reading comprehension etc. (e.g. Jaeggi et al., 2008). Whereas other studies are not able to demonstrate these effects (e.g. Redick et al., 2012). Working memory training remains a work in progress, especially because there is a great variability across working memory training studies in general and especially how the different research groups conducted the working memory training. Nevertheless the literature on working memory training also shows that especially core training of working memory is most promising. Core training studies typically involve tasks that are utilizing sequential processing and frequent memory updating and are designed to target domain‐general working memory mechanisms (Morrison & Chein, 2011). One very common and successful approach of core training studies is the “n‐back” or “dual n‐back” task. The “n‐back” task is a method to assess and to train working memory and requires a continuous monitoring of sequentially presented stimuli. The task of the participant is to give a signal whenever the current stimulus matches the stimulus occurring “n” positions back in the sequence (for further explanation of the n‐back task see description of Brains vs Zombies game). A modification of the conventional “n‐back” task is the “dual n‐ back” task (Jaeggi et al., 2003). In a “dual n‐back” task participants have to continuously monitor two independent stimuli sequences. The additional sequence increases the difficulty of the working memory task, but is also, based on the aforementioned literature, the most promising approach to yield general cognitive improvements with working memory training. As a consequence we have decided to use the “n‐back” and “dual n‐back” approach as a basic principle in developing our “Brains vs Zombies” working memory training game. Due to the difficulty of the task and the effort participants have to take during working memory training, motivation is a key factor for a successful training. Conventional working memory programs may quickly become boring or monotone, what in turn can lead to decreased training performance. Motivation is a significant factor for cognitive training. Game elements that can be added to conventional and rather

298

Kristian Kiili et al. monotone brain training can improve the outcome of the training. Prins and colleagues (2011) for example already demonstrated that a computerized working memory training with game elements enhances motivation and training efficacy in children with ADHD. In detail, ADHD children that used a game version of conventional working memory training showed greater motivation, better training performance and better working memory at post‐training than children using the conventional working memory training. Hence, it seems quite promising to combine evaluated psychological methods such as the “n‐back” task with games or game elements to increase the efficacy of working memory trainings.

3. Health impact of exergames According to Mueller et al. (2011) exergames are an emerging form of computer games that aim to leverage the advantages of sports and exercise in order to support physical, social and mental health benefits. An exertion game is controlled with an input mechanism that requires a player to intentionally invest physical exertion.

3.1 Physical impacts of exergames Many exergames can increase energy expenditure from sedentary or light levels to moderate levels (e.g. Papastergiou, 2009; Graf et al., 2009), but only few exergames result in vigorous levels of energy expenditure. For example, Graves et al. (2007) found that playing exergames with Wii expends significantly more energy than playing sedentary computer games. On the other hand Daley (2009) criticizes the previous studies and calls for more extensive and methodologically robust research. He argues that, although studies have produced some encouraging results regarding the energy expenditure of exergames, active gaming is no substitute for real sports. The controversial results may be result of different playing styles players motivations that affect greatly on energy expenditure. So far the effectiveness of exergames has been mainly assessed according to the energy expenditure level that is not solely an adequate measure for exergames. For example, in growing children the neuromuscular system is rapidly developing and the coordination of movements improves when they are exposed to different environments and various movement patterns. Muscles need activity and bones need impacts to become strong (Völgyi et al., 2010). For example, Wii games rarely cause needed impacts. All in all, whether the intensity and impacts are proper or not, players benefit from exergaming in some level; caloric expenditure, heart rate increment, and coordination skill developments (Bailey & McInnis, 2011; Staiano & Calvert, 2011).

3.2 Psychosocial impacts of exergames According to Staiano and Calfert (2011) exergame play may provide opportunities for social interaction that influence on friendship selection, self‐esteem, moods, and motivation. In general, Social aspect of gaming may reduce the risk of social isolation and loneliness (Mueller, Agamanolis, & Picard, 2003). The results have indicated that social interaction and meeting of other players are important motivations to play games (Lieberman, 2006). Furthermore, the results have indicated that exergaming can increase also self‐efficacy of overweight children (e.g. Staiano, Abraham & Calvert, 2012). The nature of exergames may partly explain this; when playing exergames players usually directs their attention toward a screen instead of peers, which may reduce body self‐consciousness during playing.

3.3 Cognitive and academic impacts of exergames Emerging research has shown that exergame interventions in schools can improve academic performance, reduce classroom absenteeism, tardiness, and negative classroom behaviors (Lieberman, et al 2011). Although only very limited evidence about the impact of exergaming in academic performance exists, great deal is known about the benefits of physical activity on cognitive functioning and academic performance. Overall, the results has indicated that increased physical activity has the potential to positively impact cognitive functioning, memory and academic achievement (e.g. Donnelly & Lambourne, 2011; Castelli, Hillman, Buck & Erwing, 2007).

4. Development of Brains vs Zombies game By simple definition Brains vs. Zombies is a working memory game targeted for tablets. The game is targeted for wide audience including children and adults. The used game mechanics would be beneficial also to elderly people but the zombie theme and audiovisual implementation is not appropriate for them. In the game zombies rise from graves one at a time and growl a word (Figure 1). In level one the player has to defend herself by remembering if the previous zombie raised from the same grave (n‐back condition) and if it made

299

Kristian Kiili et al. the same sound (dual n‐back condition) as the current zombie. Player earns points by answering correctly and suffers damage on wrong answer. The answer has to be given before the zombie manages to attack the player or otherwise it is interpreted as a wrong answer. Players’ health in the game is visually represented as a brain. Zombies bite pieces of players’ brain on wrong answer and the game ends when the brain is entirely eaten by zombies. The basic goal of Brains vs Zombies game is to keep the brain intact and keep on playing as long as possible to gain maximum amount of points.

Figure 1: The first version of Brains vs Zombies game (dual n‐back condition) The player can adjust game difficulty by selecting the number of positions zombies can appear at and number of steps back in the sequence the zombie has to be compared to. Also the game administrator can manage game’s difficulty via game system settings, which include game speed as well as zombies probability to rise from the same spot. Game system settings also dictate how many times the zombie can rise from the same spot consecutively.

4.1 The description of basic versions The first version of the game implemented dual n‐back training protocol. It means that player has to remember two separate things at once, which are zombies’ locations and spoken words. User answers on each round by pressing one of four buttons: ‘same location’, ‘same word’, ‘both are same’ or ‘both are different’. From the first user tests it became evident that remembering two stimuli (visual and aural) at once was very challenging for people with some degree of disability, especially for people with memory or hearing disabilities. Therefore a simpler version of the game was clearly needed. The second iteration of Brains vs Zombies dropped the aural element and concentrated only on visual aspects of remembering. In this version the player has to recall only the location of the zombie and answer by pressing one of two buttons indicating either ‘true‘ if the place was the same or ‘false’ if the place was different than condition determined by n‐back. By having to concentrate only on visual factor the game becomes much easier and as a result more playable for players with disabilities. Furthermore, the motion controls of exergame version are easier to implement in n‐back condition.

300

Kristian Kiili et al.

4.2 The description of exergame version The next step in enhancing the game was combining exercise with the cognitive gameplay, introducing Brains vs Zombies to the realm of exergames. In general, the combination of cognitive gameplay and exertion interfaces raises new game design challenges. However, in order to facilitate the development of cognitively challenging exergames Kiili and Perttula (2013) have proposed a design framework for educational exergames. The aim of the framework is to provide theoretical means to balance the amount of physical, cognitive, and sensomotoric workloads in order to optimize learning and health effects as well as describe ways to create more engaging exertion and learning experiences mediated by the technology. The framework distinguishes several focus points that provide means to consider game elements systematically and allow reflection of the design solutions. The framework was utilized in development of Brains vs Zombies exergame version (for more details about the framework see Kiili & Perttula, 2013). The first exergame version is a single player game for tablet devices but multiplayer version is already under development. The user controls the game with movements. The accelerometer sensor in the gaming device provides necessary motion information to detect player’s movements. The n‐back protocol was implemented instead of dual n‐back protocol to keep the gameplay simply enough. The player has to react to visual stimulus by either jumping (indicates ‘true’) or staying still (indicates ‘false’).

5. Pilot studies The Brains vs Zombies game is still under development. The aim of the conducted pilot studies was to investigate players’ experiences about the core game mechanics and possible game controls used in the game.

5.1 Pilot 1 We conducted a pilot study focusing on the basic version (button user interface) and exergame version (motion user interface) of Brains vs zombies game (n‐back condition) in spring 2013. The group consisted of six university students from which five were females. Participants’ age varied from 21 to 50. First the participants were introduced to the game one by one ‐ they were shown how the Brains vs Zombies is played (button and movement controls). Participants played the basic version first following the exergame version. One participant played each version approximately 5‐10 minutes depending how well the game proceeded. The playing behavior was observed and finally participants filled in an online questionnaire (likert scale 1‐7) about their playing experience. Most of the players reached levels 3 or 4, but the only male participant was actually able to reach level 6 during the session. This indicates that the n‐back game is much easier than dual n‐back game. In previous user tests most of the participants struggled already in level 2 in dual n‐back condition. Table 1 shows that the game worked well for the participants. They thought that the challenge was appropriate for them (M = 6,67). This was quite obvious because they could select the skill level themselves. In general, participants perceived the goal of the game well and the feedback that game provided about playing performance was appreciated. However, the observation data revealed that even though the game idea itself is simple, it was occasionally perceived a bit confusing. In the beginning participants reacted to stimulus although they did not mean to do that. It seems that it takes awhile to learn how to successfully control the game. Furthermore, it is clear that this kind of working memory training requires full concentration and cannot be played too loosely. Participants liked also the audio‐visual implementation (M = 5,75) and especially the cartoon like appearance of zombies was appreciated. Gender comparison is not reasonable because of small the sample size. Table 1: Mean scores of players’ experiences about Brains vs Zombies game (n= 6)

Table 2 compares participants’ experiences about the two different user interfaces tested. The button based interface was appreciated more. It was experienced more intuitive and easy to use (M = 6,00) than motion

301

Kristian Kiili et al. based user interface (M = 3,67). Participants felt that it was also easier to concentrate on game events when playing the game with buttons. The observation data was convergent with this finding. The observations revealed that reacting to wrong stimulus (zombie in other place) by staying still confused the participants. It seemed that it was hard to understand that without doing anything one actually answers to presented stimulus. However, when participants adopted the mechanism the playing performance increased. We assume that at least partially the high age of the participants and minor previous playing experience can explain this confusion. However, the participants gave almost identical score to both game versions in terms of desire to play again. Table 2: Mean scores of players’ experiences about button and motion user interfaces (n=6)

The verbal feedback collected during the sessions was useful to the future development of Brains vs Zombies game. For example, the buttons used in the basic version confused participants a bit. Participants did not fully understand how the Eye‐button related to right answer. This was obvious because we used same buttons that are used in dual n‐back condition and without other buttons (Ear and Ear & Eye) the used symbols do not make sense. Thus, we replaced these buttons with green correct sign and red incorrect sign before the second pilot.

5.2 Pilot 2 We conducted a focus group study in spring 2013 at one Finnish primary school. The participants (N = 14) were 8‐9 years old and both genders were equally represented. The aim of this pilot was to test how young children can handle the game, test new buttons of basic version and explore how children experience the button user interface and motion user interface (n‐back condition). First the participants were introduced to the game one by one ‐ they were shown how the Brains vs Zombies is played (button and movement controls). Half of the participants played first the basic version and rest the exergame version. One participant played each version approximately 5‐10 minutes depending how well the game proceeded. The playing behavior was observed and participants were informally interviewed during and after the playing session. In general, the children performed a bit worse than university students. Most of the participants (9/14) could handle the level 2 (2‐back) in both game versions. However, for four participants the challenge of level 1 was appropriate. These participants had difficulties to understand the meaning of n when it was bigger than 1. However, one of the players achieved the level three and two managed to play even in level 5. All the participants liked the game concept and would like to play the game again. Players learned to use both user interfaces very fast, although the button version was adopted more easily. In spite of that 9/14 participants preferred the exergame version. Approximately half of these (5) justified this by arguing that motion version was easier to play and they managed to get more points. Rest of this group such liked to control the game with movements instead of buttons. It is noteworthy that in some point participants started to use iPad as a sledgehammer instead of jumping with iPad. These participants thought that hammering the zombies down was more intuitive than jumping. The reason for this was that most of them had earlier played whack‐a‐mole kind of games. Only 3/14 preferred button‐based version. They thought that the user interface was clearer and it is nice to react to every stimulus. Although the sample sizes of the pilots were small these results indicate that children were more open to exergame version than university level students. However, further long‐term studies with bigger sample sizes are needed.

6. Conclusion This paper discussed the meaning and possibilities of serious games in working memory training. Motivation is a crucial factor in working memory training and conventional working memory programs may quickly become boring or monotone, what in turn can lead to decreased training performance. However, game elements can be added to usually monotone brain training programs to engage users and improve the outcome of the

302

Kristian Kiili et al. training. The aim of the present study was to investigate players’ first impressions about Brains vs Zombies working memory training game. The study focused on exploring the attractiveness of used game mechanics and the adoption of two different user interface implementations. In general, the participants liked the game a lot. The game concept was experienced motivating and participants would like to continue playing. The opinions about the button based and motion based user interfaces varied. In general, children appreciated the motion based user interface more than university students, but because of small size the results cannot be generalized. Children were more creative in selecting the movements that they used to control the motion based game. Unfortunately, children tended to use movements that do not contribute much to fitness, for example using the iPad as a sledgehammer. In contrast university students controlled the game by jumping. As a conclusion the overall game concept seems promising but further research and development is needed. In fact, there were several limitations to this study, which can be developed in future research. First, the sample size was very small and the findings represent only pilot work. Second, the different game versions need to be finalized and after that long‐term controlled studies with large sample sizes would serve to validate the benefits of the game and the meaning of motion based user interface. Third, the game needs to be tested in different contexts and in different user groups. For example, for elderly people we have to create a game that is themed differently and consider adaptation issues from challenge level and motion controlling perspectives.

References Alloway, T.P. and Alloway, R.G. (2010) “Investigating the predictive roles of working memory and IQ in academic attainment”, Journal of experimental child psychology, Vol 106, No. 1, pp 20–29. Baddeley, A. (2003) “Working memory: looking back and looking forward”, Nature reviews Neuroscience, Vol. 4, No. 10, pp 829–39. Bailey, B. and McInnis, K. (2011) “Energy cost of exergaming: a comparison of the energy cost of 6 forms of exergaming”, Arch Pediatr Adolesc Med, Vol 165, No. 7, pp 597‐602. Buschkuehl, M., Jaeggi, S. M. and Jonides, J. (2012) “Neuronal effects following working memory training”, Developmental Cognitive Neuroscience, Vol. 2, pp 167–179. Castelli, D.M., Hillman, C.H., Buck, S.M. and Erwin, H.E. (2007) “Physical fitness and academic achievement in third‐ and fifth‐grade students”, Journal of Sport & Exercise Psychology, Vol 29, pp 239–252. Daley, A.J. (2009) “Can Exergaming Contribute to Improving Physical Activity Levels and Health Outcomes in Children?” PEDIATRICS, Vol 124, No. 2, pp 763‐771. Donnelly, J. and Lambourne, K. (2011) “Classsroom‐based physical activity, cognition, and academic achievement”, Preventive Medicine, Vol 52, Supplement 1, pp 36‐42. de Freitas, S. and Liarokapis, F. (2011) Serious Games: A New Paradigm for Education? In M. Ma, A. Oikonomou and L.C. Jain (Eds.), Serious Games and Edutainment Applications (pp 9‐23) Spinger‐Verlag, London. Gathercole, S.E., Alloway, T.P., Willis, C., and Adams, A.‐M. (2006) “Working memory in children with reading disabilities”, Journal of experimental child psychology, Vol. 93, No. 3, pp 265–281. Gorgu, L., O’Hare, G.M.P. and O'Grady, M.J. (2009) Towards Mobile Collaborative Exergaming, In proceedings of 2nd International Conference on Advances in Human‐oriented and Personalized Mechanisms, Technologies, and Services, pp 61‐64. Graf, D.L., Pratt, L.V., Hester, C.N, and Short, K.R. (2009) “Playing Active Video Games Increases Energy Expenditure in Children” PEDIATRICS, Vol 124, No. 2, pp 534‐540. Graves, L., Stratton, G. Ridgers, N.D. and Cable, N.T. (2007) “Comparison of energy expenditure in adolescents when playing new generation and sedentary computer games: cross sectional study”, BMJ, Vol 335, pp 1282‐1284. Jaeggi, S.M., Buschkuehl, M., Jonides, J., and Perrig, W. J. (2008) “ Improving fluid intelligence with training on working memory”, Proceedings of the National Academy of Sciences of the United States of America, Vol 105, No. 19, pp 6829–6833. Jaeggi, S.M., Seewer, R., Nirkko, A.C., Eckstein, D., Schroth, G., Groner, R., and Gutbrod, K. (2003) “Does excessive memory load attenuate activation in the prefrontal cortex? Load‐dependent processing in single and dual tasks: functional magnetic resonance imaging study”, NeuroImage, Vol 19, No. 2, pp 210–225. Ketamo, H. & Kiili, K. (2010) “Conceptual change takes time: Game based learning cannot be only supplementary amusement. Journal of Educational Multimedia and Hypermedia”, Vol 19, No. 4, pp 399‐419. Kickmeier‐Rust, M.D., Mattheiss, E.E., Steiner, C.M. and Albert, D. (2011) “A Psycho‐Pedagogical Framework for Multi‐ Adaptive Educational Games”, IJGBL, Vol 1, No. 1, pp 45‐58. Kiili, K and Perttula, A. (2013) A design framework for educational exergames. In New Pedagogical Approaches in Game Enhanced Learning: Curriculum Integration, IGI Global, US, pp 136‐158. Kiili, K., de Freitas, S., Arnab, S. and Lainema, T. (2012) “The Design Principles for Flow Experience in Educational Games”, Procedia Computer Science, Vol 15, pp 78‐91.

303

Kristian Kiili et al. Klingberg, T. (2010) “Training and plasticity of working memory”, Trends in cognitive sciences, Vol 14, No. 7, pp 317–24. Lieberman, D., Chamberlin, B., Medina, E., Franklin, B., Sanner, B. and Vafiadis, D. (2011) “The power of play: innovations in getting active summit 2011: a science panel proceedings report from the American heart association,” Circulation, Vol 123, pp 2507‐2516. Lieberman, D. A. (2006) What can we learn from playing interactive games? In P. Vorderer and J. Bryant (Eds.), Playing video games: Motives, responses, and consequences Erlbaum, Mahwah, NJ, pp. 379–397. Morrison, A. B., and Chein, J. M. (2011) “Does working memory training work? The promise and challenges of enhancing cognition by training working memory”, Psychonomic bulletin & review, Vol 18, No. 1, pp 46–60. Mueller, F., Edge, D., Vetere, F., Gibbs, M. R., Agamanolis, S., Bongers, B., and Sheridan, J. G. (2011) Designing Sports: A Framework for Exertion Games. In proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, Canada. Mueller, F., Agamanolis, S. and R., Picard (2003) Exertion Interfaces: Sports over a Distance for Social Bonding and Fun,” In proceedings of the SIGCHI conference on Human factors in computing systems, ACM, Florida, USA. Owen, A.M., Hampshire, A., Grahn, J.A., Stenton, R., Dajani, S., Burns, A.S., Howard, R.J., et al. (2010) “Putting brain training to the test”, Nature, Vol 465, No. 7299, pp 775–778. Papastergiou, M. (2009) “Exploring the potential of computer and video games health and physical education: a literature review,” Computers & Education, Vol 53, pp 603‐622. Pickering, S. J. (2006) Working Memory and Education, Elsevier Academic Press. Prins, P. J. M., Dovis, S., Ponsioen, A., Ten Brink, E., & Van der Oord, S. (2011) “Does computerized working memory training with game elements enhance motivation and training efficacy in children with ADHD?”, Cyberpsychology, behavior and social networking, Vol 14, No. 3, pp 115–22. Redick, T. S., Shipstead, Z., Harrison, T. L., Hicks, K. L., Fried, D. E., Hambrick, D. Z., Kane, M. J., et al. (2012) “No Evidence of Intelligence Improvement After Working Memory Training: A Randomized, Placebo‐Controlled Study”, Journal of experimental psychology: General. Staiano, A. and Calvert, S. (2011) “Exergames for physical education courses: physical, social, and cognitive benefits,” in Child Development Perspectives, Vol 5, No. 2, pp 93‐98. Staiano, A.E., Abraham, A.A., and Calvert, S.L. (2012) “Adolescent Exergame Play for Weight Loss and Psychosocial Improvement: A Controlled Physical Activity Intervention”, Obesity Journal. Vanderwater, E.A., Shim, M.S. and Caplovitz, A.G. (2004) “Linking obesity and activity level with children’s television and video game use”, Adolescence, Vol 27, No. 1, pp 71‐85. Völgyi, E., Lyytikäinen, A., Tylavsky, F., Nicholson, P., Suominen, H., Alén, M. and Cheng, S. (2010) “Long‐term leisure‐time physical activity has a positive effect on bone mass gain in girls”, In J Bone Miner Res. Vol 25, No. 5, pp 1034‐1041.

304

Meleon ‐ a Casual Mobile Game Supporting Immersion and Reflection in Learning Luise Klein Hochschule Bremerhaven, Bremerhaven, Germany klui@gmx.de Abstract: Mobile applications are predestined for situational informal learning. However, for young learners, it is difficult to control, engage in and make sense of their learning experiences in unstructured environments. Whereas complex mobile games have been successfully implemented for situational learning in structured environments, casual mobile games have mostly been viewed as time wasting without opportunities for thorough learning. Can game design elements within a casual mobile application produce informal learning for teenagers in their everyday environment? How can a casual mobile game support immersing and reflective learning? The casual mobile game Meleon was developed and evaluated to approach these questions. The application’s concept is based on the proposed model of immersion and reflection in casual mobile game‐based learning. With ‘Meleon’, players practice computational thinking, by being inspired by their environment. The heart of the game are different mapping algorithms that use the device’s camera as input, and the colouring and movement of the game’s character Meleon as digital output. Two game modes show how the theoretic model can be translated into detailed design decisions so that immersion and reflection are supported on a micro, macro, and expanded game cycle. First the players become immersed when the game matches their interests. The appropriate level of direct feedback and visual appeal create sensory immersion and reflection on the interface elements. The hierarchical goal structure, balanced challenge and elaborate level feedback encourage challenge‐based immersion and in‐ game reflection that lead to declarative and strategic thinking skills. The integration of the game in a larger play environment, stimulates reflection after the game is completed even where there is no organised external debriefing. The first evaluation of Meleon affirms the high potential for casual mobile games to promote continuous engagement and learning on several levels and styles. The different modes offer initial game immersion for diverse people. A range of entry points, appealing aesthetics and themes, using the device as a tool to integrate the context, and short, rewarding yet challenging play sessions are key factors for thorough learning processes in casual mobile games. Keywords: experiential learning, immersion and reflection, mobile game‐based learning, casual games, informal learning

1. Introduction The educational value of mobile learning applications is widely acknowledged. It is commonly understood that mobility and connectivity features of mobile devices are especially interesting for situated informal learning, when there is no other formal framework to motivate, or guide learning processes. So far, most mobile learning applications are designed as productivity or reference tools ‐ they help users to efficiently obtain and communicate situational relevant information. Less attention has been paid to the playful interactions with the real and virtual world, children are attracted to in mobile devices (Jarkiewicz, Frankhammar, Fernaeus, 2008). Additionally to supporting experienced lifelong learners, mobile applications should also help to stimulate and structure informal learning for people who do not have the techniques, motivation or discipline yet to continuously learn from their everyday encounters. Game‐based learning proved to motivate and structure immersion and reflection of informal learning experiences. Most game‐based learning research refers to complex games (Prensky, 2005), and stresses the importance of externally guided debriefing (Gee 2008). Long involvement and full absorption in the virtual world, however, are no typical mobile interaction patterns. This paper analyses what potential there is for learning within unguided casual mobile games. Is it possible to elicit learning processes, immersion and reflection, with these games? Can we use mobile games to immerse learners and help them reflect upon their environment? First the model of immersion and reflection in casual mobile game‐based learning explains these experiential learning processes. Then the design of the application Meleon demonstrates how the model helps in finding concrete design solutions for two different game aesthetics. Finally, the evaluation of the game searches for evidence of immersion and reflection associated with the previously described elements ‐ can they elicit learning experiences in a casual mobile game?

305

Luise Klein

2. The model of immersion and reflection in casual mobile game‐based learning The model of immersion and reflection in casual mobile game‐based learning (figure1) integrates theories of mobile learning, experiential learning, game‐based learning and casual game design to visualise how the levels of learning processes and game design elements ideally work together. The theoretical background and resultant game design guidelines are summarised in the following.

Figure 1: Model of immersion and reflection in casual mobile game‐based learning

2.1 Theoretical background 2.1.1 Mobile learning Ryu and Parsons’(2009) framework for designing mobile learning spaces provides the structure of the model with four pillars:

mobile application environment: the main components of the app, adaptable by the game designer

mobile informal learning context: relevant environmental and situational features, cannot be predefined but only assumed

specific learning activities: in our model watching, thinking, doing, feeling – shown by the spiral of immersion and reflection processes

defined learning objectives: in the categories skill‐based, cognitive or affective learning

In casual situational learning the learning content and objectives vary between games and the situational context. Therefore this general model focuses on the learning activities and processes through which players deal with objectives. Thus, at the heart of the model is the experiential learning spiral. 2.1.2 Experiential learning Many theories of game‐based learning refer to experiential learning to explain the processes which keep players engaged in the game cycle, and help to make sense of their experiences (Garris, Ahlers, Driskell 2002, Kiilii 2005). Kolbs’ (2009) experiential learning theory explains how we build knowledge and understanding from our daily experiences in a repeated cycle of Reflective Observation, Abstract Conceptualization, Active Experimentation and Concrete Experience. The more we encounter a concept in slightly diverse contexts, the more abstract and deeper our understanding of it becomes. A successful learning environment encourages and leads the learner into all phases, so that abstractions can be built from concrete experiences, and

306

Luise Klein experimented within active engagement. To accentuate the most opposing parts of the cycle and bring the terms closer to game research, the model shows two essential processes: First, the immersion in the concrete experience, acting and feeling in the current situation. Secondly, reflection upon the experience from a detached point of view, that involves the observation and abstraction from it. The terms immersion and reflection can be compared to what constructivists call accommodation and assimilation, and what Ackerman (1996) coined ‘diving in’ and ‘stepping out’. 2.1.3 Levels of immersion and reflection Through qualitative research Brown and Cairns (2004) subdivided immersion of games into three levels: engagement, engrossment and total immersion. The involvement of a game increases by time when barriers or thresholds are removed. The game has to be attractive and accessible, but it also has to make the player invest in the game. The more the player is willing to give, the more they become emotionally attached and immersed in the experience. Three levels of depth also exist on the reflection side of the experience: Reflection in the micro, macro, and expanded game cycle. The model of recursive loops of game‐based learning (Kearney and Pivec 2007) analyses the cycles in more detail: In a micro game cycle (encompassing a set of simple user interactions) the player practices technical, motor skills, and cognitive abilities. A macro game cycle comprises several micro game cycles. It requires declarative, procedural and strategic knowledge of the player to do a sensible sequence of interactions in order to advance in the game. On a third level, debriefing in a social environment after playing promotes affective learning. Ideally the game itself offers access to that community, and encourages the player to relate to other situations outside the game world. Kearney and Pivec’s model emphasizes that reflection does not only occur through final debriefing, but also as reflection‐in‐action after each micro and macro cycle when the player abstracts from previous experiences within the game to build the next strategy. 2.1.4 Casual game design values in the expanded game experience cycle Despite the aim of the mobile game to guide players’ learning experience, it is undesirable to force them in a completely predefined structure of place, time and content. After all they should learn from their personal generic experiences. To suit the mobile environment the application should not only adapt to the situational context, the outside environment, but also integrate it into the game and learning processes. Furthermore, casual game design values (Kultima 2009) help to make the app suitable for flexible playing times, places, and a diverse range of users. This involves reducing thresholds and adding affordances to all phases of interaction with the application.

2.2 Immersion, reflection and learning in the micro, macro and expanded game cycle Experiential learning, immersion and reflection, in games takes place on these three nested levels, learning on the simple levels (micro cycle) helps immersion and reflection on the complex levels (expanded cycle). The model (figure 1) visualizes how the flow of immersion and reflection within the three interwoven cycles would ideally happen. 2.2.1 Immersion in the expanded game experience cycle In the expanded game experience cycle of casual games (Kultima 2009) the game experience starts when the user chooses to play, chooses a game, and opens it. Initial interest and immersion in the game’s story and style starts even before playing. Once the players start the game, they progress through many interactions with it, each a micro cycle, through several macro cycle levels of game play. 2.2.2 Immersion and reflection in the micro game cycle In the micro game cycle the user forms plans of action corresponding to the affordances of interface elements, acts, observes feedback and system state changes. Immersion on the micro level is often sensory – visual and sound affordances make the player experiment with game elements.

307

Luise Klein In micro cycle reflection game world changes are observed and evaluated whether the intended outcome is achieved for each interaction. Through the micro game cycle, players get to know the user interface with its functions and how to handle controls. These skills are required to further interact with the game, and thus also for learning cognitive and affective skills in the larger game cycles. 2.2.3 Immersion and reflection in the macro game cycle Within a macro game cycle players perceive the game elements’ interrelations. Challenges given by the game or set by players themselves immerse them in the macro game cycle – they experiment with those interrelations and make them work towards a goal. Between levels players reflect upon their success, how they utilised the game elements, adapt and build strategies for the next try. Through immersion and reflection in the macro game cycle, cognitive learning objectives can be achieved: The player constructs concepts of the game content and system; interface elements are interpreted in how they can help to pursue goals. Learned procedures are practiced, and strategies are devised for future challenges. 2.2.4 Reflection in the expanded game experience cycle In the expanded game cycle after playing, learners reflect on their general strategies, the story, and their relation to the game in the real world context. The after play phase can go on for even longer than the actual playing. Depending on how the game deals with interruptions and whether it integrates well into their everyday activities, players will find it appealing to come back to it. Affective learning happens through immersion and reflection in the expanded game experience cycle, especially in relation to the role players take in the game, its relation to their real life identity. Attitudes towards the game, and its content change. Being immersed in a character’s role within the game cycle, the sense of oneself, as a game player and playful learner emerges stronger afterwards. Ideally players reflect on themselves, their learning styles, the playful way of learning, and use these strategies in other situations as well.

3. Meleon ‐ immersive and reflective experiences with computational thinking Meleon is an educational casual mobile game for 10 to 15 year‐olds that provides a temporally, and locally independent, yet structured environment for practice of computational thinking. Hu (2011) describes computational thinking as a combination and concurrence of logical, algorithmic, scientific, mathematical, analytical, engineering‐oriented and creative thinking. He concludes that the thinking skills cannot be practiced as such, but in doing computation, seeking representations and models that are transformed in an algorithmic solution to a problem. Meleon’s objective is to lead the player along the cycles of immersion and reflection so that they do computational thinking and programming. Respectively players should understand the relations of input, output, and the transformation a programme achieves. They use programming to construct an envisioned output, by applying observation and experimentation in problem‐solving. Furthermore, they develop a positive attitude towards computation and see themselves as analytical thinkers and problem‐solvers.

3.1 Design process Understanding the immersive and reflective processes on the different game levels explained by the previously described model, help designers to make informed decisions specifically focusing on how to best support these processes. The model and game literature findings provide general requirements of the application (refer to Klein (2012) for a full list of requirements). The analysis of similar mobile applications and tools teaching programming provided more specific requirements for the learning of computational thinking. Many applications and tools exist that support novice

308

Luise Klein programmers, from simple tangible building blocks, to textual programme writing (Kelleher and Pausch 2005). Ambient programming as proposed by Elumeze and Eisenberg’s (2008) Buttonschemer is close to the informal casual learning experiences Meleon aims to create. The programme, a drawn barcode, is part of the environment that is scanned and executed by the small device. Meleon uses the colours of the players’ surroundings as programme input, captured by the device’s camera. Thus the game gets suitable input everywhere and still includes the unique environment within the game. Players experiment with their environment, find certain coloured objects to advance in the game, or purposefully draw their desired colour input. From the initial requirements, user‐centred design steps concretized the concept: personas, comic storyboards, paper prototypes, and focus groups (Ginsburg 2011). They revealed that two different aesthetics were appealing to youth. The first aesthetic allows lightly guided exploration, creativity and caring. The second aesthetic is based on challenge, and nurturing.

3.2 Two aesthetic modes – exploration and challenge Meleon lets players care for a visually impaired chameleon, by helping it adapt to its environment correctly, and lead it to its food sources. In the open‐ended exploration players are guided in finding out how the chameleon transforms the environment’s colours into coloured shapes onto its skin. The aim is to understand how the ambient code (colours from the captured camera) is computed to create an output of different coloured shapes. Players have to explore, by trying several inputs and observing the output while using problem‐solving strategies. The mode starts with a simple mapping of one detected colour (average of the centre of the image) used directly for the colour of the chameleon. The complexity of the input‐output‐mapping gradually increases to a combination of colour determined shapes and inverted colours. The exploration mode deals with the very nature of computation: Transferring an input to a different output based on rules defined by a program.

Figure 2: Meleon’s exploration aesthetic, mapping the first square per row to filling, the second to stroke colours In challenge mode players have to guide the chameleon along a coloured tree. To make the chameleon catch flies one has to control its movements over different coloured leaves ‐ without falling off, adapting to the leaves so that birds cannot see and catch Meleon. Step by step, programming concepts like commands, their parameters, functions, and conditions are introduced.

3.3 Design decisions for immersion and reflection within Meleon Meleon was designed with special attention to how game elements support immersion and reflection processes on the levels of micro, macro, and expanded game cycle interaction. They refer to casual and game‐ based learning literature (Kultima 2009, Garris, Ahlers and Driskell 2002, Gee 2008, Sweetser and Wyeth 2005, Prensky and Thiagarajan 2007), and user centred design findings.

309

Luise Klein

Figure 3: Meleon’s challenge aesthetic, level statistics for reflection

Figure 4: Meleon’s challenge aesthetic, taking the right colour and walking along the leaves, replay 3.3.1 Main game elements for immersion In the expanded game cycle accessing the game is quick through the devices’ application stores. The game’s feel, narration, and characters are positive and intelligible with the colourful chameleon that needs help navigating around finding food. Additional motivation for playing the game is learning programming as a secondary purpose that fits well into the users’ real life goals and identity. Thus the game offers two access points: For those who like caring for animals and creativity, and those who are interested in programming and thinking challenges. Furthermore, the game is flexible, able to be played in a variety of social environments, locations and times. Sensory immersion in the micro game cycle is achieved by visual appeal of the character and always changing colours in exploration mode, and movement and animation in challenge mode. The user interface is simple, and game objects have clear affordances – the live camera picture is directly shown behind the programme code cells it fills, and programme output changed according to each movement. They carry users forward from one interaction to the next. In the macro game cycle mainly challenge‐based immersion is addressed. A combination of different types of challenges and functionalities through the two game modes, and small surprises through integration of the ever changing outside colours as input increase experimentation.

310

Luise Klein Tutorials allow immediate doing with small hints pointing at the currently required action. The short story of lost Meleon presents clear short‐term goals, that are immediately visible in form of the flies to catch. The progress towards them is shown with each correct step closer. Also minimal input results in success and rewards already. In challenge mode some programme block colours are preset. In exploratio only moving the camera over some new colours achieves results. Meaningful feedback – the creative colouring output in exploration, and small statistics in challenge mode – help setting personal goals. Challenges integrate actions in the real world, moving around or drawing colour codes to find the correct camera input. It becomes part of the game. 3.3.2 Main elements for reflection In a micro cycle visible system changes and audio responses that refer to the previous action help reflection. In exploration the camera image is immediately interpreted and the output shown. In challenge mode the currently executed code blocks are highlighted and Meleons movements animate. The game puts no (time) pressure on the player, each action can be paused so that game world changes can be observed and evaluated before starting a new micro cycle action. Before each new macro cycle the game presents goals clearly, structured within the story. Success in challenge mode is rated – the highest achievement is obtained when the player caught all flies, used the minimal amount of programming commands, and solved it on the first try (figure 3). It makes players reflect where they could have done better. Several strategies need to be employed to be successful in the game – in exploration fixing the colour of a cell and observing the change the others reveals its meaning, in challenge mode imagination of the code’s meaning before acting is required. Intermittent states after each macro cycle in which there are no open short‐term goals, limit the desire to act, and the player can enter reflection. Moreover, a functionality to replay and observe the programme execution support reflection. Reflection in the expanded game cycle happens after play. Meleon can be interrupted or quit at any time ‐ the player is not punished for it. Some micro game cycles already bring a benefit for the player, and the game is still be interesting after many macro cycles with increasing complexity of mappings and tasks. Thus the player leaves the game with a positive feeling wanting to come back. Reflection on the overall game experience is enhanced through the integration into the personal environment of players. They can relate to game experiences and find them more meaningful. The game can be used in many diverse contexts, so players abstract from the concrete game experience, and come to a more general understanding.

4. Evaluation of immersion and reflection with Meleon The functional prototype of Meleon was evaluate to find if the game elements described above do give rise to immersion and reflection processes. The evaluation focused on evidence of immersion and reflection in the casual interaction with the prototype on all game cycle levels, and the possible differences between exploration and challenge modes.

4.1 Setting and method In their free time 15 volunteers, comprised of ten boys (age 10–17), and five girls (age 10–15) tested the application in a youth club. Play time was not limited or structured to allow casual informal play. On average a tester openly explored the app for 20 minutes. Several methods were used to investigate different aspects. Observations showed usage practices, general immersion and reflection activities. Screen capture logs were used to analyze usage pattern, game play strategies, and visualize learners’ experimentation and problem‐solving. Semi‐structured interviews disclosed

311

Luise Klein specific elements that support immersion and reflection on micro, macro, expanded game cycle levels. The interview transcript and observation notes were later analysed using a reduced version of Mayring’s (2000) qualitative content analysis. The analysis’ goal was to corroborate the already defined categories of immersion and reflection on micro, macro and expanded game cycle level and their related interface and gameplay elements through examples from deductive category application.

4.2 Results The game elements for immersion and reflection on micro, macro and expanded game cycles were validated through observation, screen captures, and interviews. The deductive category analysis of the collected data showed evidence of immersion and reflection activities for both modes, on all three game cycle levels, and revealed the game elements that gave rise to them. To give some examples: For the micro game cycle observations showed that in exploration mode children continuously moved the device around saying “Let’s try this colour of your shirt”, “Look what this does here…” The direct change of the game world with each actions carries the interaction forward. The interview revealed that the rating statistics after each macro game cycle made players think why they did not achieve the full rating after finishing the level. They then wanted to play it again, trying another strategy. Another insight is that pupils thought the game would suit somebody who likes animals. They did not think it is for computer whizzes only, but anyone of their friends. In the expanded game cycle, the context integration through the camera proved to be at least a novelty which makes players start and get immersed in the game, at best there are clues that through the integration and playing within a larger context, participants would also think about and be reminded of the game by events in this broader context. Interview data revealed that Meleon’s game elements’ design provide chances for immersion and reflection on all three levels. Requirements and design decisions based on the model of immersion and reflection in casual mobile game‐based learning and drawn from game‐based learning literature, as done for Meleon, provide a solid basis for mobile gaming through learning applications that support immersion and reflection. However, there are some limitations: The data suggests that not all of the game elements have to be used to design an immersive and reflective game. Some apply only to exploration mode, others only to challenge mode, still both parts independently provide chances for immersion and reflection. Results from a questionnaire comparing the two game aesthetics/modes show that both illicit immersive and reflective processes, with slight differences: Exploration aesthetics mostly initiated sensory and imaginative immersion, and challenge mode provides mostly challenge‐based immersion, still both are perceived equally immersive. This is different for reflection: Players assigned more reflective attributes to challenge mode than to exploration mode. Another aspect of evaluation aimed at the mobility and casualness of interaction with Meleon. In a rather casual open user test without predefined tasks and test time, different application adaptations occurred. Some participants tested the application sequentially, trying all exploration options first before going to the fly catching, where they mostly kept playing for a longer period of time. Others tried to make sense of the application as a whole and switched between modes up to six times in six minutes. Yet others found the challenge of catching flies most compelling right from the start, but could not make sense of the colour blocks yet. After they experimented in exploration mode for a while, they had the confidence to return to challenge mode, and were successful using a step by step problem‐solving approach. Most did interrupt game play to talk to friends, move to another seat. Few also walked around in the room finding new colours. These observation show that Meleon is flexible in time, place, length of play, and the combination of aesthetics offers specific micro and macro game cycle elements that are better suited for different people.

5. Conclusion The model of immersion and reflection in mobile game‐based learning integrates theories of experiential learning, mobile learning, game‐based learning and casual game design. The evaluation of Meleon and the design decisions made with special attention to immersive and reflective learning processes showed that

312

Luise Klein casual mobile games can be flexible enough to fit in the informal environment of teenagers, but still structure the learning experience and centre it around certain topics. With game elements for immersion as well as reflection games support different learning styles, and help learners to practice all of them. If designed with broader audiences in mind and supporting different aesthetics, mobile games give playful and casual access to topics that users would find complicated and daunting in educational applications or hardcore games. Casual games have many game elements in common with these complex games, just the aesthetics are mostly different making them acceptable for a wider audience, various situations, and more rewarding within short play times. The initial qualitative evaluation give reason to believe that the integration of the player’s context without limiting the function of the game to a certain context leads to meaningful activities even for players who are not initially interested in the learning content, and reflection about the game that exceeds game playing time. The topic programming as done by Meleon is in this respect one of the easier learning contents to design activities for. Through its abstractness, programming can be integrated into many different stories and aesthetics that use data from the environment as input. It integrates an abstract form of the learners’ context which is not depended on a certain time and place. In the future it will be interesting to see if the model of immersion and reflection in casual mobile game‐based learning can also be applied in the design of games with other contents. For Meleon it has proven a solid base of requirements and guiding design around learning processes, not necessarily learning outcomes.

References Ackermann, E. (1996) “Perspective‐Taking and Object Construction: Two Keys to Learning”. In Kafai, Y. B. and Resnick, M. (eds.) Constuctionism in Practice: Designing, Thinking, and Learning in a Digital World. Lawrence Erlbaum Associates, Mahwah. Brown, E. and Cairns, P. (2004) “A grounded investigation of game immersion”, CHI '04 extended abstracts on Human factors in computing systems, ACM, New York. Elumeze, N. and Eisenberg, M. (2008) “Buttonschemer: ambient program reader”, MobileHCI ’08, ACM, New York Garris, R., Ahlers, R. and Driskell, J. E. (2002) “Games, Motivation, and Learning: A Research and Practice Model”, Simulation and Gaming, Vol 33, No. 4, pp 441‐467. Gee, J. P. (2008) “Learning and Games”, The Ecology of Games: Connecting Youth, Games, and Learning, Salen. K. (ed.) The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning, The MIT Press, Cambridge. Ginsburg, S. (2011) “Designing the iPhone user experience: a user‐centered approach to sketching and prototyping iPhone apps”, Addison‐Wesley, Upper Saddle River. Hu, C. (2011) “Computational thinking: what it might mean and what we might do about it.” ITiCSE ’11, ACM, New York. Hunicke, R., Leblanc, M. and Zubek, R. (2004) MDA : A Formal Approach to Game Design and Game Research [online], http://www.aaai.org/Papers/Workshops/2004/WS‐04‐04/WS04‐04‐001.pdf, AAAI Press. Jarkiewicz, P., Frankhammar, M. and Fernaeus, Y. (2008) “In the hands of children: exploring the use of mobile phone functionality in casual play settings”, Proceedings of the 10th international conference on Human computer interaction with mobile devices and services, ACM, New York. Kearney, P. and Pivec, M. (2007) “Recursive Loops of Game‐Based Learning: a Conceptual model”, In C. Montgomerie and J. Seale (eds.) Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007, AACE, Vancouver. Kelleher, C. and Pausch, R. (2005). “Lowering the barriers to programming: A taxonomy of programming environments and languages for novice programmers”, ACM Comput. Surv., Vol 37, No. 2, pp 83–137. Kiili, K. (2005) “Digital game‐based learning: Towards an experiential gaming model.” The Internet and Higher Education, Vol 8, No. 1, pp 13‐24. Klein, L. (2012) “Immersion and Reflection in Informal Playful Learning with Mobile Technology”, [online], master’s thesis, Hochschule Bremerhaven, http://www.luiseklein.de/publ/mthesis.pdf Kolb, A. Y., and Kolb, D. A. (2009) “The Learning Way: Meta‐cognitive Aspects of Experiential Learning”, Simulation and Gaming, Vol 40, No. 3, pp 297‐327. Kultima, A. (2009) “Casual game design values”, Proceedings of the 13th International MindTrek Conference: Everyday Life in the Ubiquitous Era, ACM, New York. Mayring, P. (2000). “Qualitative content analysis”, [online] Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, http://www.qualitative‐research.net/index.php/fqs/article/view/1089 Prensky, M. and Thiagarajan, S. (2007) Digital Game‐Based Learning. Paragon House Publishers. Ryu, H. and Parsons, D. (2009) “Designing Learning Activities with Mobile Technologies”, In Ryu, H. and Parsons, D. (eds.) Innovative Mobile Learning: Techniques and Technologies, Information Science Reference, Hershey. Sweetser, P. and Wyeth, P. (2005) “Gameflow: a model for evaluating player enjoyment in games”, Comput. Entertain., Vol 3, No. 3.

313

The Literature Race ‐ NFC Based Mixed Reality Game Antti Koivisto, Harri Ketamo, Eero Hammais and Juho Salli Satakunta University of Applied Sciences, Pori, Finland antti.koivisto@samk.fi harri.ketamo@samk.fi eero.hammais@samk.fi juho.o.salli@samk.fi Abstract: In many countries people think TV and games takes hours away from reading and decreases the reading skills. We can not close our eyes and request games to be forbidden: games are part of our culture, the new form of storytelling and social interaction for younger generations. We should use that fact to build motivation around literacy and reading. This paper describes a Near Field Communication (NFC) based multiplayer mixed reality game “The Literature Race” that motivates the children to find information about books, apply that information in teams to solve the challenges in the game and finally get familiar with libraries. Keywords: NFC, games based learning, reading, library, children

1. Introduction In many countries, people are worried about kids reading habits. Furthermore, many people thinks TVs and games takes hours away from reading and decreases not only kids reading skills, but especially their imagination (e.g. Greenfield 2009). We can not claim that such phenomena does not exist, but games are the new form of storytelling and social interaction for younger generations. Literature, imagination and finally learning itself has always been about storytelling and social interaction. This paper describes a Near Field Communication (NFC) based multiplayer mixed reality game “The Literature Race” that motivates the children to find information about books, apply that information in teams to solve the challenges in the game and finally get familiar with libraries. The game is developed and designed by teachers and students at Satakunta University of Applied Sciences (FInland) in co‐operation with Luvia municipality library (Finland). The game is targeted for children aged 11‐15 with aims to get children familiar with literacy and libraries. However, there are several other hidden objective behind the game design: 1) it requires and teaches team work, 2) because kids are walking in the library, they get some physical exercise and 3) solving the tasks requires learning new knowledge e.g. in history, geography and biology. Applying NFC and RFID in game design has been studied e.g. in social mobile games with educational dimensions, with shared social experience and physical interaction between players (e.g. Nandwani, Coulton & Edwards, 2011; Garrido, Miraz, Ruiz & Gomez‐Nieto 2011). Furthermore, location based NFC games and location aware NFC UIs allow users to play games in mixed reality in that they can interact with both real and virtual objects within that location (e.g. Rashid, Coulton, Edwards & Bamford 2006; Broll, Graebsch, Holleis & Wagner 2010;Vajk, Coulton, Bamford & Edwards 2008; Sánchez, Cortés, Riekki & Oja 2011). In these environment NFC and RFID are used to extend mobile devices into game controllers. In our study we connect the RFID tagged book into game controller (mobile phone), but not only because of book as an object. The idea is to connect the textual content of the book into game in order to control the storytelling. In other words, in or approach we are connecting content into stories with RFID and NFC.

2. Description of the game The game is based on non‐linear interactions between movie clips shown in the game and user activities in real world. The format is chosen because there are many 2D, 3D and text based games for kids, but very few non‐ linear storytelling based games with human actors. The challenge in this kind of production is to build a manuscript flexible enough for non‐linear gameplay.

314

Antti Koivisto et al. Another motivator for building this game is the fact that there is not games for kids in libraries which uses NFC, RFID, videoclips and storytelling. This was done also to invoke our school students which types of videos could be added to the game and furthermore how the game could be created within a short period of a time. Our design principles were quite forward and the aim was to entertain and give information about the wide collection of books which the young students might not ever heard about. All the major information wanted to keep simple and easy to understand so this is also one reason to select videos to keep the player motivated. The game was implemented in Android platform, because when starting implementation only Android phones had NFC (Near Field Communication) chip. There aren’t many games available (that we know of) in the game industry that use NFC or RFID (Radio‐Frequency Identification) in their core game play. In this study we designed our user‐interaction completely based on NFC and sensors, because we think there is great underused potential in it. Also with this we can go beyond the traditional “touch the screen and something might happen” interfacing with mobile games. With NFC and sensors player can truly interact with the game by manner of “you must do something and then the game reacts to your actions”. The game is written with HTML5 and JavaScript that was packaged into Android software with PhoneGap (WebView application). PhoneGap is an open source framework for building mobile applications. We think that it is the fastest way to prototype this type of game. Also by using Phonegap we have the option in future to develop the same game in various platforms. Of course some minor modifications must be done but the basic elements can be transfer from other platform to an other.

3. The gameplay When player starts the game an intro video is started. In the video an agent gives instructions which explains the game objectives and goal (figure 1). The agent also gives a tip of the first book to be scanned. After the first book player must scan books so that the title of the book matches to the pervious books title. For example if the current books title would be ”Around the World in Eighty Days”, then the next book to be scanned could be ”World of Warcraft: Dark Riders” because both titles contains the word ”World”. After this the player can not use the same word again in order to make progress in the game. If this would be granted the player might not learn so much and would scan only familiar books. In the game we also prevented the player to scan the same books again. After any mistake there will be shown a short videoclip to explain that an error was made. Also the player will lose one life from a total of three lives.

Figure 1: The game starts with a video in which a literature agent gives the player a briefing So the basic idea in the game is to search books which titles matches to each others. In the game assistance agents will give feedback to the player how the game is going and what should be done next in order to finish the game (figure 2). When player has found a book that is believed to be the correct one, player places the book near the mobile device and the game reads the RFID‐tag placed at the back of the book (Figure 3). If the signal was good and tag was correctly read, the game extracts an ID string from the tag. The ID is then sent to game server. One ID corresponds to exactly one book in the database. A script at the server then queries the database for book title and sends the found book title back to players’ device. Game then checks if the title contains the specified word and if it does, the user will see a message (either video or text) that congratulates the player and gives

315

Antti Koivisto et al. new instructions. If, however, player did not find a correct book, the game gives tips on finding a better book (or ends the game with video, depending on the score).

Figure 2: The information is given by videos with human actors In the game players could complete tasks like “... You have to find five books that are named after former Finnish President.”. Because we wanted to get players introduced into the library itself, players are searching the information around the library with the tablets. Searching the answer requires teamwork. One can go and check things from indexes, another just wonders around, and finally asks librarians help (which is of course allowed).

Figure 3: Players have to find the correct book. Book´s RFID tag is read with a tablet The game re‐organizes itself according to the given answer. Finally, when the challenge id completed, the team gets rewarded by the agent that give introduction. The game takes estimated 30‐60 minutes to complete on average and during the game, kids walk 1‐2 kilometers. The challenge is completed when the player has successfully scanned 30 books. The challenge is failed when the player has made three mistakes. At the end screen player can see how other peoples have performed in the game. Also if the player manages to reach a place in the top‐ten‐table he/she can give a nickname and other players can admire his/hers achievement. The game score consists of three elements:

How fast the player can find a correct book

How silent the player was

How many books went correct

The game is now in pilot use and later 2013 we can report the measures and evaluations on it. Because game is played in a library, we decided to add monitoring for loud noises and too quick or sudden movement. If the player talks too loud or shouts, she/he loses some points. Points are also lost if running is detected. This is done in order to teach players how to behave in library.

316

Antti Koivisto et al.

Figure 4: Game process Movement detection is done with PhoneGap‐native feature that uses the devices’ accelerometer. At the time of developing the game there didn’t exists any plugin that had the functionality we wanted for sound monitoring. Therefore we developed a custom plugin that monitors microphone input and analyzes it. Because PhoneGap community is very active, there exists few very good plugins for reading NFC‐tags. However, since our books have RFID‐tags in them, we had to modify existing plugins because they didn’t (at the time of developing the game) support reading RFID with NFC‐chip. For reading RFID tags we had to look really close what data was stored inside the tag and how we can get it out with standard NFC reader. We did not wanted to modify the NFC readers in any way because the game are made available for the library customer also. The videos for the game were designed by the students of Satakunta University of Applied Sciences. (Figure 5) Before that, they had made the manuscripts with the help of the staff. Making these videos was part of their interactive media studies and served their learning well. The video production required teamwork in creating the manuscript and the storyboard. Also, it simulated an authentic situation of video shootings where the students could practice their skills for example as a director, producer, camera operator and actor. We noticed that the learning experience was excellent because the students were very motivated creating valuable and good‐quality videos for real use. While the importance of the technical skills in creating games is obvious, the desire for a great story is always there. We found out that video production for a storytelling based game is a natural and powerful way to teach the essence of the modern storytelling to students.

317

Antti Koivisto et al.

Figure 5: Video production serves as a vast learning experience for a media and communications student

4. Conclusions Location and context aware games allow users to interact not only with objects within that location but also with knowledge inside the object. In the Literature Race, the story and gameplay is based on mixture of physical exercise, social interaction in real world, knowledge in library databases and general understanding on literacy. The novelty value of the approach is in designing NFC and sensor based mixed reality multiplayer mobile game. From educational point of view, the main goal is to encourage children to visit libraries and find interesting readings. The game opens a new possibility to get familiar with libraries and books. Besides getting familiar with library as a building and books, the children will learn team work and taking responsibilities in team, st which are very relevant topics in terms of 21 century skills. Furthermore, walking around library as well as social interaction gives children necessary extra curricular activities. Future research consist of user experience studies with school groups in Luvia library in fall 2013. According to these studies, the story, gameplay and UI will be improved. At this point we are aware that there will be several improvements to be done before the game will reach publishable level. However, we assume, the gameplay itself will be motivating and engaging enough to get good outcome.

References Broll, G., Graebsch, R., Holleis, P. & Wagner, M. (2010).Touch to play: mobile gaming with dynamic, NFC‐based physical user interfaces. In Proceedings of the 12th international conference on Human computer interaction with mobile devices and services, MobileHCI '10, September 2010, Lisboa, Portugal, . pp. 459‐462. Garrido, P.C., Miraz, G.M., Ruiz, I.L. & Gomez‐Nieto, M.A. (2011). Use of NFC‐based Pervasive Games for Encouraging Learning and Student Motivation. Near Field Communication (NFC), 2011 3rd International Workshop on, IEEE 2011, pp. 32‐37. Greenfield, P.M. (2009). Technology and Informal Education: What Is Taught, What Is Learned. Science 2 January 2009, Vol323(5910), pp. 69‐71. Nandwani, A. Coulton, P. & Edwards, R. (2011). NFC Mobile Parlor Games Enabling Direct Player to Player Interaction. In proceedings of Near Field Communication (NFC), 2011 3rd International Workshop on Near Field Communication, 22 February 2011, Hagenberg, Austria, pp. 21‐25. Rashid, O. Coulton, P. Edwards, R. & Bamford, W. (2006). Utilising RFID for mixed reality mobile games. In proceedings of Consumer Electronics, 2006. ICCE '06. 2006 Digest of Technical Papers. International Conference on, IEEE 2006, pp. 459‐460. Sánchez, I., Cortés, M., Riekki, J. & Oja, M. (2011). NFC‐based interactive learning environments for children. In Proceedings of the 10th International Conference on Interaction Design and Children, IDC '11, June 20‐23, 2011, Ann Arbor, Michigan, USA, pp. 205‐208. Vajk, T., Coulton, P., Bamford, B. & Edwards, R. (2008). Using a Mobile Phone as a “Wii‐like” Controller for Playing Games on a Large Public Display. International Journal of Computer Games Technology Volume 2008, Article ID 539078, 6 pages.

318

Bringing Game Achievements and Community Achievements Together Johannes Konert, Nico Gerwien, Stefan Göbel and Ralf Steinmetz Technische Universität Darmstadt, Darmstadt, Germany johannes.konert@kom.tu‐darmstadt.de n.gerwien@gmail.com stefan.goebel@kom.tu‐darmstadt.de ralf.steinmetz@kom.tu‐darmstadt.de Abstract: When social media is used for game‐based learning one major issue is the rewarding of players for their efforts to provide user‐generated content to others (peer tutoring). This can be done by rewards and achievements gathered within the game or when content is created within a game‐related community platform. For serious games that foster the knowledge exchange among peer learners (players) the rewarding and tracking of both ‐ in‐game and in‐community assistance for help among learners ‐ is desired. Thus we propose an architecture and solution for an integrated achievements‐system which allows the combination and rewarding of player activities in games and related communities at the same time. The Achieve2Conquer platform provides game developers with a middleware architecture where achievements are created, visualized to users within a web‐frontend and updated by progress information from game instances and community platforms simultaneously. The architectural model of Achieve2Conquer allows a weight balancing of achievements from game and community, to prevent an overrating of one of them. In unbalanced achievement systems this may otherwise lead to an extensive use of community‐based achievements by eager players due to the fact that these achievements are usually available unlimited (e.g. like achievements for being the first person commenting a new post). Additionally we propose new achievement types for hybrid achievements and user‐generated or user‐awarded achievements to combine existing reward models of both worlds (games and social media applications). These allow the guidance of players, e.g. by first requesting achievements parts to be achieved within the game environment, then by conducting actions in the community and finally requesting a collaborative aspect. Additionally our new achievement type of reversible achievements allows to discourage undesired player behavior and still does not violate the expected characteristics of achievements. After a brief description of current models for reward systems, reputations systems and achievements for games and achievements in social media communities, the requirements for an achievement system supporting the combination of both, game and community, are defined. Afterwards we provide the Achieve2Conquer model with its achievement categorization, the necessary achievement components and the new achievement types as the core contributions of this publication. A prototypical implementation will then be presented with a middleware architecture connecting the existing serious game Woodment and a phpBB community bulletin board. Keywords: peer learning, achievement system, community achievements, serious games technology, social serious games

1. Introduction and motivation Social media platforms are a vital source of information for learners. Beside pure learning platforms like Open University 1 , social media platforms are used to exchange knowledge about learning content and computer game solutions(Li et al. 2011; Denny et al. 2008). While major game publishers provide their own game‐related community‐platforms (e.g. BattleNet 2 from Blizzard), for most computer games the discussions take place in stand‐alone community‐platforms related to the specific game. Especially for Serious Games it is desirable to foster a knowledge exchange among the players related to the quests and learning aspects of the game. At the same time such games normally do not provide in‐game community features due to limited resources in the game development process. Even though an in‐game community and exchange would be the most seamless solutions to emphasize exchange among players, it is a much more cost‐effective and suitable solution to provide a community‐platform for the players based on 3 4 existing social media solutions (e.g. like wiki systems of Wikimedia or bulletin boards from phpBB ). Players can then provide their hints and solutions for specific game quests (and related learning targets) to others. Then knowledge exchange among the learning peers happens based on the user‐generated content stored within such social media platforms(Konert et al. 2011). If this content is provided with respective meta data about the game quest, player, solution characteristics and game‐specific information then other players can 1

http://www.open.ac.uk/ http://battle.net/ 3 http://www.wikimedia.org/ 4 https://www.phpbb.com/ 2

319

Johannes Konert et al. find such content faster and more effective (Konert et al. 2012). The providers of answers to questions of other players are rewarded by the comments, (positive) ratings and feedback. Still, the incentives to provide such content are limited, even though one major motivation for creators of user‐generated content is fun/entertainment (Stoeckl et al. 2007). As computer game players are rewarded within the games directly (e.g. by an achievement system), it seems suitable to reward community‐based activities, too. Such a system can provide incentives for players of computer games to not only master the game and collect all game achievements, but as well try to get the achievements provided for community‐based activities. If all achievements are then integrated into one achievement system, players can be rewarded in a balanced and adjunctive way for progress on both sides: in game advancements and in‐community content creation (or consumption).

2. Related work To the best of our knowledge no model for an achievement system exists yet that focusses on the combination of game‐based and community‐based achievements in one system. Thus we discuss here existing reward systems, reputations and achievement systems for games and communities separately to derive the proposed model.

2.1 Reward systems To achieve a goal in a computer game users receive manifold feedback by the game: positively in case they solved a problem or make progress and negatively in case they did something wrong. The negative feedback may appear in form of energy decrease, game termination or setbacks. These special rewards, so called false rewards (Juul 2009), help players to learn how to master the game. In general the primary purpose of reward systems is to provide additional incentives and motivation for players to experience the full spectrum of a game. Reward systems can be viewed as player motivators or as compromises for easing disappointment. Wang & Sun (2011) identify eight forms of rewards in computer games:

Score systems

Avatars (e.g. experience systems, leveling)

Item granting systems (collecting and social status)

Resources (practical game currencies)

Achievement systems

Feedback (direct positive emotional intensifier)

Animations and pictures (milestones and game story enrichment)

Unlock mechanisms (for access to game content)

Even though achievement systems (R5) are by themselves a form of rewards it can serve as a whole rewarding system, because achievements contain rewards (like R1‐R8) issued to the player when receiving the achievement. Additionally, in modern computer games reward systems also provide a strong social meaning to players. Their player dossiers reflect the rewards and are mostly shown publicly to other game community members. As such, the interconnected players and their visible profiles serve as social capital (Ellison et al. 2007). Analogue to the social capital in the communities, gaming capital can be drawn from the knowledge about the game mechanics, quests and game industry (Consalvo 2007). Both, social and gaming capital, are made visible (in parts) by the collected rewards (and the visible completed achievements). From the focus on role playing games Hallford & Hallford (2001) classify rewards into four types that can be mapped to the eight reward forms listed above (mapping in parentheses):

Rewards of Glory (no direct impact to game play, e.g. R1, R5, R6, R7)

Rewards of Sustenance (social status and progress, e.g. R2, R3, R4, R5)

Rewards of Access (unlocking content, e.g. R8)

320

Johannes Konert et al.

Rewards of Facility (abilities and enhancements, e.g. R2, R3)

2.2 Reputation systems As achievement systems serve as well for the representation of social status as described above, the visibility to others relates to reputation systems, which allows access to “a summary of one’s relevant past actions with the context of a specific community presented in a manner that can help other community members make decisions with respect to whether and how to relate to that individual” (Dellarocas 2010, p.2). Consequently, a reputation system is only useful in a multiplayer context where a community of players is interconnected. An example of such a system is integrated with Yahoo Answers 5 ; a platform that addresses all of the four objectives users expect from a reputation system as proposed by Dellarocas: trust building (to differentiate good from bad), filtering (to identify good content and contributors), matching (to identify resources matching a users’ profile) and user lock‐in (to keep users attached to the platform maintaining their reputation profile). To address these objectives with an achievement system, Dellarocas propose four key questions to address as listed in Table 1. We answer these questions according to the targeted achievement system. Table 1: Key decisions for the design of reputation systems (Dellarocas 2010, pp.3–4) and the answers according to Achieve2Conquer Key decision What actions are relevant to include in one’s reputation profile? How to obtain information about relevant user actions? How to aggregate and display reputation information?

How to deal with manipulation and gaming?

Answer in relation to this publication’s achievement system The progress of all achievements and their time of completion should be included, as well as all available and already earned rewards Generally, there is the choice between internally generated (firsthand) information and feedback provided by others (secondhand). Achievement systems commonly obtain their information firsthand by monitoring the user actions (e.g. with trigger‐ events). Exceptions are user‐awarded achievements (see section 3.2.3). A variety of methods is possible: simple statistics (e.g. number of completed achievements), star ratings (e.g. rating how good an achievement was completed), numerical score (e.g. achievement points), numbered tiers (e.g. level‐based achievements), leaderboards (e.g. highscore lists, commonly not included in achievement systems, but possible) No incentives (achievements) for undesired user behavior should be given. As the achievement is usually tight to a trigger and is as well limited, manipulation is only fractionally possible. Still, a manipulation‐resistant achievement‐based reputation system is impossible to design if the underlying game is prone to manipulation.

2.3 Game‐based achievements Achievements have a long tradition for games; especially in sports where the best three participants of a discipline are rewarded (e.g. at Olympic Games). Achievement systems serve as a meta game if they are comprehensive and visualize achievements of several games at once, e.g. like the XBOX Live system (Jakobsson 2009). Then the achievements are designed independent of game and genre (Hamari & Eranti 2011). In contrast to the general motivations for playing as discussed at the beginning of chapter 2, a more specific list regarding the motivation related to achievements contains (1) social status, (2) completionism and (3) extended play time (Montola et al. 2009). Based on expert interviews Montola et al. found fourteen categories of achievements addressing these factors: A1 Tutorial (trying and learning features of the game) A2 Completion (finish a sequence of the game) A3 Collection (complete a collection of game items) A4 Virtuosity (play sequences perfectly) A5 Hard Mode (succeed on difficult levels) A6 Special Play Style (master parts of the game with even more restrictions like fast‐running) A7 Veteran (quantitative accumulation of game items like currency) A8 Loyalty (playing regularly) 5

see http://answers.yahoo.com/

321

Johannes Konert et al. A9 Curiosity (discovering unexpected secrets or master unlikely situations) A10 Luck (getting a rare item) A11 Mini‐Game (success in mini‐games) A12 Multi‐Player (outstanding performance in multi‐player scenarios) A13 Paragon (being rewarded for pioneer activities) A14 Fandom (attend out of game activities like purchasing merchandise articles) Orthogonally to the categorization games provide some achievements as hidden achievements which are not visible in the list of available achievements. They appear when completed. Thus the completion logic and requirements are unknown for players and such the achievements are usually seldom and provide Rewards of Glory (Montola et al. 2009).

2.4 Community‐based achievements The majority of research publications and studies relates to achievements in computer games and seldom focus on achievements in social media platforms and communities. Even though rewards and reputation systems exist widely to encourage users to not only passively consume, but actively contribute content to the social media communities, achievement systems could additionally help to foster content creation and break the rule that only 1% of users contribute content (Arthur 2006), while all others only consume (so called free riders). Especially for game‐based learning or for Social Serious Games that interconnect players of Serious Games for knowledge exchange, such achievements can support the creation of new tutorials, solutions and answers to questions (Konert et al. 2012). Such users contributing content without direct benefit for them are called zealots. Factors influencing the willingness of users to contribute content are

Cost / benefit ratio

Incentive system

Extrinsic / intrinsic motivations

Social capital

Social and personal cognition

(Chen & Hung 2010). An achievement system for game‐based and community‐based achievements can address most of the factors and thus influence the contribution positively. As achievements can be an additional incentive the cost / benefit ratio profits from provided achievements. As such it is by itself an incentive system and can positively increase the extrinsic motivation when the achievements offer rewards on completion. As publicly visible achievements serve as reputation systems and social capital at the same time the last two aspects are covered as well, if such a system is added to community systems. Different to the categories for game‐based achievements listed in Chapter 2.3, Montola et al. (2009) have identified seven categories for achievements in social media communities (categories overlapping with the game‐based achievement categories have been marked italic): C1 Tutorial (visit and read parts of the community‐stored content) C2 Participation (actively contribute to gatherings, votes, discussion threads) C3 Instructor (provide hints, solutions, guidelines) C4 Moderator (administer content and like/dislike elements) C5 Wiki Author (improve the knowledge base) C6 Veteran (participating a lot in activities) C7 Completion (focusing on completion of all achievements or a sequence)

2.5 Achievement definition As achievement systems can be defined game and genre independent, a general achievement structure definition is possible. The parts listed in Table 2 are derived and extended from Hamari & Eranti (2011).

322

Johannes Konert et al. Table 2: Achievement component definition, based on (Hamari & Eranti 2011)

Reward

Completion logic

Signifier

sub component

description The signifier is defined in the achievement system and consists of the visible parts. Name The name of the achievement Visual (optional) The visual representation that is commonly related to the name and the description. Description A textual description of the central parts of the unlocking logic of an achievement, or a vaguer hint as well as the description of the rewards. The completion logic is defined through mechanics in another system (e.g. game or community platform) Pre‐requirements Pre‐requirements are conditions for the game session that have to be met in order to allow the trigger or the other conditions. Trigger Trigger is either a user action or a system invoked event. Conditions Conditions include the requirements directed to the system state or to historical events in the system fulfilled in the past. Multiplier/Counter The amount of times the trigger has to go off while all pre‐requirements and conditions are met. Defines the reward(s) a user acquires after unlocking the achievement in‐game/ in‐community Rewards related to the game or community (e.g. R2, R3, R4, R6, R7 or R8). Achievement game Rewards related to the achievement system (e.g. R5). out‐game/ Rewards external to the achievement system, the game and the community out‐community (e.g. R1 or R3).

3. Model for game‐based and community‐based achievements Based on the related work the requirements for a combined achievement system, categories and the newly proposed achievement types are defined.

3.1 Requirements Beside the fact that a combined achievement system should support all features listed above for game‐based achievement systems and community‐based achievement systems separately, we can derive requirements resulting from the combination of both. The Achieve2Conquer model may support the combination of activities to enable the creation of achievements that encourage users to switch between game and community. Additionally the achievement system must address the balancing of achievement points between game and community that none dominates the other unintentionally. As in social media communities the creation of content is a crucial aspect, the creation and awarding of achievements from users for users (user‐ generated) should be supported, too. Finally achievements can be used to discourage undesired behavior without violating the achievement characteristics. This supports the improvement of the content quality (especially for achievements related to user‐generated content creation).

3.2 New types of achievements Based on the existing research results and models the following new achievement types are proposed. 3.2.1 Qualitative and quantitative achievements The authors of this publication propose a categorization of achievements into two types: qualitative and quantitative achievements. Most achievements in computer games are measured quantitatively and can be split up into achievement levels. All levels of a group of achievement share a multiplier/counter and the progress can be shown by the percentage of completion. These quantitative achievements are usually not rare, visible and related to the categories A2 (Completion), A3 (Collection), and A7 (Veteran). On the contrary, qualitative achievements have a binary status and cannot be expressed as a percentage of completion and are normally not split up into levels. They reward unusual game play behavior, may be seldom and represent rewards of glory or sustenance. They change the player’s perspective onto the game (e.g. A6 Special Play Style or A9 Curiosity).

323

Johannes Konert et al. 3.2.2 Cooperative achievements Achievements can be designed to be completed by a single user or a group of them. Achievements that require more than one participant are defined as cooperative achievements. They can be further split up by the time‐dependency of the actions into synchronous/asynchronous cooperative achievements. In community‐ based achievements the activities might be mostly asynchronous, e.g. participants get the achievement when one is creating an item and another participant successfully re‐edits this contribution. In games an example of a synchronous cooperative achievement is the cooperative solving of a multi‐user quest to unlock an A2 (Completion) achievement. 3.2.3 User‐generated and user‐awarded achievements If the achievement system allows the creation and addition of new achievements, users can be granted the right to define new achievements by themselves. Then they need to create and define the signifier, completion logic and rewards. This can be suitable and working if the completion logic and rewards are definable by users either via programming code directly or with selection from a catalogue of requirements, triggers and rewarding effects/actions. Even more attractive are user‐generated achievements in sandbox‐games where 6 users themselves create game play within the games (like in Minecraft . If the created achievements do not use pre‐requirements or conditions from the game, but are awarded by peer votes the achievements are called user‐awarded achievements. Still, the user‐awarded achievements must not be user‐generated but can be pre‐defined by game designers (e.g. like an achievement for the most famous player of the month awarded by peer votes). 3.2.4 Reversible achievements Reputation systems and their relation to achievements have been discussed in chapter 2.2. The most relevant objectives for game‐based and community‐based achievements are trust building and user lock‐in. As reputation systems normally reflect the desired and undesired behavior of a participant, achievement systems normally only reflect the positive progress. Instead of establishing a reputation system in parallel to the Achieve2Conquer model we propose so called reversible achievements which can reflect the occurrence of undesired behavior. They will be mostly quantitative achievements and reset their magnifier/counter in case an undesired behavior (or event) occurs. Thus the user has to re‐start with collecting the objects necessary for the achievement. Such achievements can even be designed as level‐based achievements and can e.g. reflect the ratio between positive and negative comments to user‐generated items. As it is unexpected and strongly discouraged to revoke once unlocked achievements, it is suggested to never revoke an achievement (or level) once it has been completed. Still, such achievements can reflect rating and reputation functionality. 3.2.5 Hybrid achievements As a natural consequence of combining game‐based and community‐based achievements into one system that visualizes them together, it is suitable to define achievements which require the user to complete actions in‐ game and in‐community. Such achievements are called hybrid achievements. They can provide a stronger binding between players who are more active in the game and players more active in the community. It is expected that they foster as well the content creation in the community systems by very experienced game‐ players as described earlier.

3.3 Combined categories of rewards As described, Wang & Sun (2011) identify eight forms of rewards in computer games. Furthermore they argue that the last two (R7, R8) are not applicable to communities. In contrast to this, we argue that even though they are less usual, these two categories can be applied to communities as well. Unlock mechanisms are already part of the reward system of Yahoo Answers where users gain more rights when successfully answering other users’ questions. Animations and pictures can as well be part of rewards gained within communities, e.g. to decorate a user’s profile or extend the list of possible items storable. Thus the categories for game‐based and community‐based rewards in an achievement system are identical.

See http://www.minecraft.net

324

Johannes Konert et al. Additionally we add the reward category of Reputation/Prestige even though some of the achievement types cover the aspect of prestige already (like hidden achievements). It is still a valuable reward category and plays an important role especially in community systems. R1 Score systems R2 Avatars (e.g. experience systems, leveling) R3 Item granting systems (collecting and social status) R4 Resources (practical game currencies) R5 Achievement systems R6 Feedback (direct positive emotional intensifier) R7 Animations and pictures (milestones and game story enrichment) R8 Unlock mechanisms (for access to game content) R9 Reputation/Prestige

3.4 Combined categorization of achievements Based on Montola et al. (2009) we define a complete list of achievement categories with a description for game‐based and community‐based implementations in Table 3. Table 3: Categorization of game‐based and community‐based achievements, based on (Montola et al. 2009) Category Tutorial (qualitative) Veteran (quantitative) Collection (quantitative) Completion (quantitative) Curiosity (qualitative) Cooperative (both) Virtuosity (qualitative) Fandom (neither) Loyalty (neither) Luck (neither) Mini‐Game (neither) Paragon (qualitative) Special Play Style (qualitative) Hard Mode (qualitative) Moderator (qualitative) Instructor (qualitative)

Game‐based description Trying and learning features of the game Accumulation of game items like currency Complete a collection of game items Finish a sequence of the game (or everything) Discovering unexpected secrets or master unlikely situations Outstanding performance in multi‐player scenarios. Play sequences perfectly Attend out of game activities, like purchasing merchandise articles Playing regularly Getting a rare item

Community‐based description Visit and read parts of the community‐stored content Participating a lot in community activities Take part in discussions, collect votes, collect answered questions (or best answers) Complete all community achievements or a specified collection. Contribute content about a game curiosity. (community feature: curiosity votes) Solve a crowd‐sourcing task, e.g. participate in a survey or tag community elements. Be a community role model; e.g. always get a high rating for content. Attend out of community fan activities; e.g. fanfests Contribute to the community regularly

Succeed in mini‐games

Perform an unlikely activity (e.g. the first post of the day) Succeed in mini‐games

Being rewarded for pioneer activities

Master parts of the game with even more restrictions; like fast‐running

Master activities in the community with more restrictions; like answering a question within 30 seconds (with good rating)

Succeed on high difficulty level

‐ Administer content and like/dislike elements

‐ Assist other players in the game (side‐kick or follow‐mode)

Provide hints, solutions, guidelines

3.5 Addressing the balancing When combining game‐based and community‐based achievements the ratio of achievement points partitioned among them has to be fixed. Otherwise inflation can cause the achievements of one side to be meritless. Thus, the design of the achievements needs to take into account limits for each achievement. For

325

Johannes Konert et al. example, a user can only gain once the achievement for being the first commentator on a new item. Such achievements can as well be split into levels as long as there is a defined maximum (100 first comments, 1000 first comments, ..). Consequently it can be guaranteed that the ratio between the achievable points and achievements of both parts (game and community achievements) is maintained. If the game or community allows an unlimited number of achievements (e.g. in sandbox‐games with user‐ generated achievements) it is suggested that both sides allow the creation of new achievements to let the player‐community balance the achievements. In such an environment the time needed for an achievement should be balanced that it takes the same time‐effort to achieve a specific amount of achievement(s) (or achievement points) in the game and in the community. Still, there needs to be a maximum time‐counting per achievement. For example, if the maximum is reached for a specific achievement, it is displayed as completed (100%) but the time spend is still updated (but no more achievement points or rewards are given).

4. Implementation 4.1 Middleware architecture The implementation of Achieve2Conquer has been made as a middleware with public APIs for the game‐ instances and the community‐instances to inform the central system about unlocked achievements. Due to the necessity to listen to system status changes in the game and/or the community, the completion logic of the achievements had to be implemented in the respective environment. An architectural diagram can be seen in Figure 1.

Figure 1: Architectural diagram and information flow of Achieve2Conquer

4.2 Achievement types We connected the Serious Game Woodment (Wendel et al. 2010) and a new instance of the bulletin board software phpBB with C# and PHP client implementations to the Archive2Conquer middleware and added in each platform three new achievements. For efficiency reasons the achievements are listed within Woodment as well as a website that is generated and delivered on a template base by Achieve2Conquer and displayed in an embedded browser directly in the game. Thus, the achievements are listed for community and game likewise as a website (see Figure 2).

326

Johannes Konert et al.

Figure 2: Web‐based visualization of achievements; here the community achievements

5. Conclusions The main contribution of this publication is the consistent categorization of achievements for the combination of game‐based and community‐based achievements based on a review of existing achievement classifications, reward systems and reputation systems. The requirements and features to be fulfilled by such a new system have been described and new achievement types were defined to bring community and game closer together. They fulfill the requirement of discouraging undesired user behavior (reversible achievements), the possibility to combine activities in both worlds (hybrids) and provide a solution for user‐generated achievements and allow the user‐based awarding of achievements based on votes. The classification into qualitative and quantitative achievements and the discussion of the aspect ratio allows the design of balanced achievement systems. Finally a prototypical implementation and connection to the Serious Game Woodment and an instance of the bulletin board software phpBB showed the ease of usage. The focus of this publication was on the theoretical aspects, but in future work a sophisticated user study is desired with all achievement types implemented to measure the user experience and impact the combined achievement types have on learning with computer games and the knowledge exchange in the connected community platform (e.g. facebook or phpBB).

References Arthur, C., 2006. What is the 1% rule? The Guardian. Available at: http://www.guardian.co.uk/technology/2006/jul/20/guardianweeklytechnologysection2 [Accessed May 2, 2013]. Chen, C. & Hung, S., 2010. Information & Management To give or to receive ? Factors influencing members ’ knowledge sharing and community promotion in professional virtual communities. Information & Management, 47(4), pp.226– 236. Available at: http://dx.doi.org/10.1016/j.im.2010.03.001. Consalvo, M., 2007. Cheating: Gaining Advantage in Videogames (Google eBook), Cambridge: MIT Press. Available at: http://books.google.com/books?hl=en&lr=&id=XbcNAUxUajAC&pgis=1 [Accessed May 2, 2013]. Dellarocas, C.N., 2010. Designing Reputation Systems for the Social Web. SSRN Electronic Journal, pp.1–9. Available at: http://www.ssrn.com/abstract=1624697 [Accessed May 2, 2013]. Denny, P., Luxton‐Reilly, A. & Hamer, J., 2008. The PeerWise system of student contributed assessment questions. , pp.69– 74. Available at: http://dl.acm.org/citation.cfm?id=1379249.1379255 [Accessed October 16, 2012]. Ellison, N.B., Steinfield, C. & Lampe, C., 2007. The Benefits of Facebook “Friends”: Social Capital and College Students’ Use of Online Social Network Sites. Journal of Computer‐Mediated Communication, 12(4). Hallford, N. & Hallford, J., 2001. Swords and Circuitry: A Designer’s Guide to Computer Role‐Playing Games S. Doell et al., eds., Stacy L. Hiquet. Available at: http://books.google.com/books?hl=en&lr=&id=GslPb621eXQC&pgis=1 [Accessed May 2, 2013]. Hamari, J. & Eranti, V., 2011. Framework for Designing and Evaluating Game Achievements. Proceedings of DiGRA 2011: Think Design Play, p.20. Available at: http://www.digra.org/dl/db/11307.59151.pdf [Accessed May 2, 2013]. Jakobsson, M., 2009. The Achievement Machine : Understanding the Xbox Live Metagame. Proceedings of DiGRA 2009: Breaking New Ground ‐ Innovation in Games, Play, Practice and Theory, pp.2–3. Juul, J., 2009. Fear of Failing? The Many Meanings of Difficulty in Video Games. In M. J. P. Wolf & B. Perron, eds. The Vdeo Game Theory Reader 2. New York: Routledge, pp. 237–252. Available at: https://www.sfu.ca/cmns/courses/2011/260/1‐Readings/Juul Fear of Failing Video Games.pdf [Accessed May 2, 2013].

327

Johannes Konert et al. Konert, J. et al., 2011. Supporting Peer Learning with Ad‐Hoc Communities. In C. S. Guido Rößling, Thomas L. Naps, ed. Proceedings of the 16th Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE 2011). Darmstadt, Germany: ACM SIGCSE, pp. 4503–4503. Konert, J., Göbel, S. & Steinmetz, R., 2012. Towards Social Serious Games. In T. Connolly et al., eds. Proceedings of the 6th European Conference on Games Based Learning (ECGBL). Cork, Ireland: Academic Bookshop. Li, C. et al., 2011. PeerSpace‐An Online Collaborative Learning Environment for Computer Science Students. In Advanced Learning Technologies (ICALT), 2011 11th IEEE International Conference on. IEEE, pp. 409–411. Available at: http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5992379 [Accessed December 11, 2011]. Montola, M. et al., 2009. Applying Game Achievement Systems to Enhance User Experience in a Photo Sharing Service. In Proceedings of the 13th International MindTrek Conference: Everyday Life in the Ubiquitous Era on ‐ MindTrek ’09. New York, New York, USA: ACM Press, p. 94. Available at: http://dl.acm.org/citation.cfm?id=1621841.1621859 [Accessed March 4, 2013]. Stoeckl, R., Rohrmeier, P. & Hess, T., 2007. Motivations to Produce User Generated Content: Differences Between Webloggers and Videobloggers. In Proceedings of the 20th Bled eConference eMergence. Bled, Slovenia, pp. 398–413. Wang, H. & Sun, C.‐T., 2011. Game Reward Systems: Gaming Experiences and Social Meanings. 5th DiGRA Conference: Think Design Play, pp.14–17. Available at: http://my.fit.edu/~pbernhar/Teaching/GameDesign/Wang.pdf [Accessed May 2, 2013]. Wendel, V. et al., 2010. Woodment: Web‐Based Collaborative Multiplayer Serious Game. Transactions on Edutainment IV, 6250, pp.68–78. Available at: http://link.springer.com/chapter/10.1007/978‐3‐642‐14484‐4_7 [Accessed May 2, 2013].

328

Modeling the Player, Learner and Personality: Independency of the Models of Bartle, Kolb and NEO‐FFI (Big5) and the Implications for Game Based Learning Johannes Konert, Stefan Göbel and Ralf Steinmetz Technische Universität Darmstadt, Darmstadt, Germany johannes.konert@kom.tu‐darmstadt.de stefan.goebel@kom.tu‐darmstadt.de ralf.steinmetz@kom.tu‐darmstadt.de Abstract: For adaptation and personalization of game play sophisticated player models and learner models are used in game‐based learning environments. Thus, the game flow can be optimized to increase efficiency and effectiveness of gaming and learning in parallel. In the field of gaming still the Bartle model is commonly used due to its simplicity and good mapping to game scenarios, for learning the Learning Style Inventory from Kolb or Index of Learning Styles by Felder and Silverman are well known. For personality traits the NEO‐FFI (Big5) model is widely accepted. When designing games it is always a challenge to assess one player’s profile characteristics properly in all three models (player/learner/personality). Still, it is valuable to collect information to refine the models continuously to adapt the game experience precisely to a player’s models. To reduce the effort and amount of dimensions and questionnaires a player might have to fill out, we proved the hypothesis that both, Learning Style Inventory and Bartle Player Types could be predicted by knowing the personality traits based on NEO‐FFI. Thus we investigated the statistical correlations among the models by collecting answers to the questionnaires of Bartle Test, Kolb LSI 3.1 and BFI‐K (short version of NEO‐FFI). The study was conducted in spring 2012 with six school classes of grade 9 (12‐14year old students) in two different secondary schools in Germany. 72 students participated in the study which was offered optionally after the use of a game‐based learning tool for peer learning. We present the results, statistics and correlations among the models as well as the interdependencies with the student’s level of proficiency and their social connectedness. In conclusion, the evaluation proved the independency of the models and the validity of the dimensions. Still, especially for all of the playing style preferences of Bartle’s model significant correlations with some of the analyzed other questionnaire items could be found. As no predictions of learning style preferences is possible on the basis of this studies data, the final recommendation for the development of game‐ based learning application concludes that separate modeling for the adaptation game flow (playing) and learn flow (learning) is still necessary. Keywords: player modeling, bartle test, learning style, personality, Big5

1. Introduction and motivation During the design phase of a computer game decisions have to be made, how the preferences of the user, his playing behavior, gained abilities and his personal characteristics are measured and represented in a model. Usually different types of measures are kept and updated in separate models for style of game play behavior (player model), skills and abilities achieved and proofed during game play (learner model) and more static characteristics of the player’s personality (personality model). To keep a player immersed into the computer game a continuing measurement and update of the model dimensions is necessary to refine the knowledge about players’ preferences and his changing (growing) set of game‐related skills. Thus, the adaptation can choose suitable alternatives of game content and/or learning content and balances the difficulty of challenges with the players’ abilities, well known as maintenance of a flow status (Chen 2007). Even though several theories and related, empirically validated, models exist to categorize player behavior into player types and learning behavior into learning styles, they all have a natural common aspect: they focus on decisions and behavior of the person to model. Concerning such modeling of a person, very sophisticated models exist in psychology that have been refined and empirically proved across manifold cultures and generations. The NEO‐FFI (also known as Big‐5 model) is a widely accepted model representing a person’s personality in five dimensions. Thus, we investigated how well player model characteristics and learning style characteristics of a person can be predicted by measuring mainly the personality traits. As the NEO‐FFI is widely accepted as one of the most precisely models for personality traits it might be possible to establish a standard on how player models are to be build, measured and how game adaptation

329

Johannes Konert, Stefan Göbel and Ralf Steinmetz may use such models then. Likewise the learning styles could be predicted based on the personality characteristics of a person. To find the dependencies and correlations among the models to predict player type and learning styles from personality traits of a person, we describe in the following sections the used models in more detail, how we setup the study and discuss the results.

2. Related work 2.1 Established models for player modeling, learner modeling and personality modeling In related work the underlying models are briefly described that are used for modeling and tracking player behavior. Considering the player modeling, Bartle (Bartle 1996) recommends a two‐dimensional playing style system: One axis between action‐orientation and interaction‐orientation and the other between player‐ orientation and world‐orientation. In these four areas of the coordinate system are lying the player styles Socializer (Interacting, Player‐oriented), Killer (Acting, Player‐oriented), Achiever (Acting, World‐oriented) and Explorer (Interacting, World‐Oriented). Bartle draws his classification from the analysis of a long and intense discussion of expert players of a specific MUD (Multi User Dungeon) game. Even though these playing style preferences have not been proven to be likewise suitable for other games or game genres, it is still very popular to be used for the mapping of alternate game content or game story variations and the assumed playing style preferences of a player. It can be assumed that this is due to the fact that Bartle first provided a simple model easy to implement and in the same time easy for mapping of alternate game content or game story variations to these playing style preferences: A Socializer (S) is interested in people and communication with them. The game is the environment to get to know people and establish a network of friendship. In difference, Killers (K) lurk for the competition and contest with other players. They like to conquer others and downsize their personae. This does not necessarily mean by having death match fights (as the name suggests), but finally Killers feel good if someone else in the game suffers from their actions. The other two playing style preferences are world‐oriented. The Achievers (A) mainly collect points and enrich their profile and/or character. Thus, they are eager to get all available achievements of a game, but are only interested in interaction or competition with others to reach these aims. The Explorers (E) focus on discovering the game world and game mechanics. Consequently they are eager to know the hidden places, the awkward way to solve a puzzle that is only possible due to a game bug and they know the game world map by heart. To be best of our knowledge no official Bartle test questionnaire has been published, but we corresponded with Erwin S. Andreasen who developed with Brandon Downey the questionnaire for Bartle’s playing style preferences and already collected data of more than 200.000 recipients (Andreasen n.d.; Bartle 2004, p.145). The provided questionnaire contains 6 or 7 questions per each of the 6 combinations of two different Bartle playing style preferences. Each of these items containing two possible answers to questions about the participant’s preferences. One of the possible choices relates to one of the two playing style preferences associated with the item. The other answering option relates to the second associated style. Thus, for all combinations of all playing style preferences the participant has to make choices which to prefer. In detail the following number of items for each combination is set: S/A: 7, S/E: 6, S/K: 7, E/A: 6, E/K: 7 and K/A: 6. The items sum up to percentages on how much each playing style is preferred by the participant. In total 200% are spread among the four styles, but each style has a maximum of 100% (Andreasen n.d.). In learning style preferences, several models like the revised version of the Index of Learning Styles (ILS) of Felder and Silverman (Felder & Silverman 1988) or the recent versions of the Learning Style Inventory (LSI) of Kolb and Kolb (2005) are widely used. While the ILS differentiates the four independent dimensions of learning style Sensing/Intuiting, Visual/Verbal, Active/Reflective and Sequential/Global, the LSI distinguishes among the two dimensions Concrete/Abstract and Active/Reflective Learning Style. In comparison, the ILS appears to be more detailed in the number of dimensions and differentiates among perception, provisioning, processing and understanding of the learning content, the LSI in contrast provides a very elaborated and empirically in many disciplines evaluated model of calculating the learning style preferences of participating learners. ILS contains 44 items with 11 questions per dimension allowing participants to choose among two alternate answers. Thus users must decide on each dimension their preferences. In LSI the 12 items allow four answers each that

330

Johannes Konert, Stefan Göbel and Ralf Steinmetz enable participants to choose always the learning style dimension that suits most for them. Thus the LSI appears to be more accurate in measuring the learning style preferences and is focused in the continuing paragraphs. A similar study could be conducted easily with the ILS as well by simply changing the questionnaire parts accordingly. The main model this article focusses on is the well‐established NEO‐FFI personality model consisting of the so called Big Five dimensions Openness to experiences, Conscientiousness, Extraversion, Agreeableness and th Neuroticism (De Fruyt et al. 2004). Over the years since the 1930 the model and questionnaires measuring the personality in these dimensions have been elaborated and been narrowed down to a short version of the Big Five Inventory (BFI‐K) questionnaire that only needs 21 items which measure for each dimension with four of the items the result and have once extra question for the dimension openness for experiences for reliability reasons (Rammstedt & John 2005).

2.2 Application of the established models in game‐based learning The described models are applicable for evaluation and selection of the next most suitable game content element (or game scene) in case several alternatives are available for the game engine in a specific game loop that needs to decide which element to load next. Therefore game elements are annotated by game designers with their suitability for the described model dimensions above. If not only one model, but several in parallel are used, weighting factors might be set (statically or dynamically) to decide in which ratio the diverse models’ suitability counts. Bartle’s playing styles are used to adapt the game quests and appearance to the specific kind of tasks the different styles stand for. Game‐based learning components of research projects like 80days (Kickmeier‐Rust et al. 2008) or Storytec (Mehm 2010) combine learning style annotation of game scenes using the competency‐based knowledge space theory (Albert & Lukas 1999) with the playing styles of Bartle by providing annotation possibilities for game authors in an authoring environment for both models and a runtime weighting possibility to influence the selection of game content to be selected next (Mehm et al. 2010). Commercially the usage of learning style or player style models is not widely published, but in several recent game releases or games to be released, the concept of providing the game content and the path through the game scenes (or quests) by adapting to a four playing style model is obvious. In example, the 1 massive multiplayer online role‐playing game WildStar (to be released in 2013) provides different player experiences of the game very closely related to Bartle’s playing styles. Likewise, the collaborative multi‐player game Woodment 2 uses Bartle’s model to adapt the occurrence of game events and content to the playing style preferences of the players whose preferences are tracked during game play and derived from decisions taken in game by the players (Wendel et al. 2010).

2.3 (In)dependency of the models for player type, learning style and personality traits As the NEO‐FFI model consists of five dimensions world‐wide accepted as the model for personality of an individual and provides evaluated questionnaires for nearly every language, we focus on the investigation of the predictability of the values in LSI and the Bartle model based on one individual person’s values in the NEO‐ FFI model. If such a correlation and predictability exists a game engine and learning engine of a computer game can adapt game content and learning content based on the NEO‐FFI model values of an individual only.

3. Study setup For the study we translated the questionnaires for the Bartle test and the Kolb LSI 3.1 to German with suitable vocabulary for secondary school class pupil aged 12 to 14. We conducted the study between 21st of March and 3rd of May 2012 in six secondary school classes (9th year) of two different schools in Germany. After the evaluation of an e‐learning diagnostic and learning environment in a math class scenario the paper‐based questionnaire was handed out to the pupils to fill them at home. All items of the three models were encoded and aggregated as described in the designated publications. The overall scores in the dimensions (4 dimensions for Bartle playing style preferences, 4 dimensions for Kolb’s LSI and 5 dimensions for NEO‐FFI) where then normalized to have consistent values in the (0,1) interval. Additionally the quality of the data was calculated to track the percentage of missing answers in the items underlying each of the aggregated values for the dimensions. Thus it was possible to leave out users from the data analysis that provided less than 75% of all answers needed to calculate the values for the models’ dimensions. 1

See NCSoft website of WildStar at http://www.wildstar‐online.com/ for details See project website and online demo at http://demos.storytec.de

331

Johannes Konert, Stefan Göbel and Ralf Steinmetz Within one week the class teachers collected the returned questionnaires and returned them for analysis. 74 of the overall 193 pupils returned their validly filled paper survey containing all items for the three models of Bartle’s playing style (39 items), Kolb’s LSI 3.1 (52 items) and the short version of BFI‐K (21 items; sum of all items was 112). In our scenario all 76 participating pupils filled the survey for all items by 87% (71.6% of the pupils filled 100% of the items). After filtering out participants with a lower value than 75% of provided answers for one of the items 72 valid datasets remained for the analysis (22 f/ 50 m). Additionally to the analysis of the dependency among the three models we used the items collected during the mentioned e‐learning environment testing with an electronic questionnaire. We analyzed the correlations of the mentioned three models’ dimensions with e.g. the pupils’ proficiency level (math course marks) and level of social connectivity with the classmates.

sig.

.028

corr. ‐.044

sig.

‐ .547* * .711 .000

corr.

.064

Bartle (Killer)

Bartle (Socializer)

Kolb LSI (Exper.) Kolb LSI (Reflecting) Kolb LSI (Thinking) Kolb LSI (Doing) BFI‐K (Extrav.) BFI‐K (Agreeabl.) BFI‐K (Consc.)

‐ .510* * sig. .000

‐ .547* * .000 1

‐ .619* * .593 .000

.065

.861

.136 ‐.128

.147

.130 ‐.122 .137

.217

.276

Kolb LSI (Thinking)

.366

.965

.255

.283

‐.061

‐.086

.024 ‐.202 ‐.202 .138 ‐.119

.717

.239

.608

.473

.841

.089

‐.096

‐.145

.004

.155

.191 .276* ‐.107 .007

.713

.420

.225

.973

.195

.108

.019

.372

.315* .518* .303* ‐.146 * * * .007 .000 .010 .220

.042

.054

.032

.725

.652

.315 * .420 .007

corr. ‐.049

‐.005 ‐.061 ‐.145 .518 .525 * * .018 .965 .608 .225 .000 .000

.328 ‐.197 * .005 .097

.136 ‐.086 .004 .303 .456 .328 * * * .255 .473 .973 .010 .000 .005

corr. ‐.121 ‐.128 .024

.155 ‐.146

‐.121

sig.

.195

.312

.953

.022 .855

.064

.004

.019 ‐.193

.593

.974

.876

.104

.060 ‐.192 ‐.062 .148 ‐.038

.618

‐.197

.060

‐.047 ‐.115 ‐.116 .097

.097

.618

.106

.603

.216

.749

.693

corr. ‐.068

.147 ‐.202 .191

.042 ‐.172

.064 ‐.192

‐.047

.037 ‐.229 ‐.015

sig.

.217

.725

.593

.106

.693

.755

.053

.904

.004 ‐.062

‐.115

.037

‐ .327* *

.149

.568

corr. ‐.218

.089

.108 *

.130 ‐.202 .276

.220

.318

‐ .287* .788 .015

.525 .456 ‐.121 ‐.172 ‐.115 .114 * * .000 .000 .312 .148 .337 .338

corr. .277

.247

.250

‐.044

.713

.089

.306

.841

.782

.043 ‐.141

.283

BFI‐K (Conscientiousness)

.568

.340

.366

.313

‐ .619* * .000

.680

.593

sig.

.018

.108 ‐.005

.108 ‐.141 ‐.096

sig.

.021

.064 ‐.114

corr. .203

.239

‐.068 ‐.218 .033

.088

‐.121

.533

sig.

.717

‐.049

.203 .277

.340

.075

corr. .075 ‐.114 .043 ‐.044 .533

BFI‐K (Openness)

BFI‐K (Neuroticism)

‐ .260*

BFI‐K (Agreeableness)

corr. Bartle (Explorer)

BFI‐K (Extraversion)

Bartle (Achiever)

Kolb LSI (Doing)

sig.

Bartle (Socializer)

‐ ‐.044 ‐ .260* .510* * .028 .711 .000

Kolb LSI (Reflecting)

Kolb LSI (Experiencing)

corr.

Bartle (Killer)

Bartle Explorer)

Bartle (Achiever)

Table 1: Correlations (Pearson) between the three models' dimensions (Bartle, Kolb LSI and BFI‐K). N=72, * significance level 0.01 ** significance level 0.05

.148

.054 ‐.115

332

.335

.330

.417

Bartle (Achiever)

Bartle Explorer)

Bartle (Killer)

Bartle (Socializer)

Kolb LSI (Experiencing)

Kolb LSI (Reflecting)

Kolb LSI (Thinking)

Kolb LSI (Doing)

BFI‐K (Extraversion)

BFI‐K (Agreeableness)

BFI‐K (Conscientiousness)

BFI‐K (Neuroticism)

BFI‐K (Openness)

Johannes Konert, Stefan Göbel and Ralf Steinmetz

sig.

.065

.276

.089

.019

.652

.337

.974

.603

.335

.755

.005

.211

corr. .033 ‐.122 .138 ‐.107

.032

.114

.019

.148 ‐.116

‐.229

sig.

.788

.338

.876

.216

BFI‐K (Neurot.)

BFI‐K (Openness)

.782

.306

.247

.372

.234*

.048 *

corr. .021

.137 ‐.119 .007

.022 ‐.193

‐.038

.097 ‐.015 .149 .234

sig.

.250

.855

.749

.417

.861

.318

‐ .287* .953 .015

.330

‐ .327* * .053 .005

.104

.904

.211

.048

4. Results As all items of the questionnaire have been scaled to the interval (0,1) the correlations are calculated by Pearson’s algorithm for two‐side effect. As shown in table 1 all correlations that are significant with an error probability < 0.01 are within the three models. In the results a Bartle Achiever is with a correlation of ‐0.51 not a Socializer simultaneously, likewise an Explorer correlates negative (‐0.547) with Killers and Killers seem not to be Socializers either (‐0.619). Within the Kolb LSI all four dimensions correlated significantly positive with each other (correlations between 0.3 and 0.53). The highest values are related to Kolb LSI style Thinking (Abstract Conceptualization) predicting Reflecting and Experiencing with correlations > 0.5. Finally within the BFI‐K personality dimensions Neuroticism and Conscientiousness correlate by ‐0.327 with each other. When focusing on the correlations between dimensions of two distinct of the three models we as well consider now the significant correlations that are significant within the <0.05 error level. Between Kolb LSI and Bartle (and vice versa) the only significant effect is reported for Kolb LSI Thinking and Bartle Achiever (0.277). Considering the correlations of BFI‐K dimensions and thus the predictability of dimension of the two other models based on BFI‐K values as stated in the motivation of this publication two significant correlations can be observed. First, between BFI‐K Conscientiousness and Bartle Socializer there is a positive correlation (0.276).Second BFI‐K Openness correlates negatively with Kolb LSI Experiencing style (‐0.287). As displayed in table 2 the pupils proficiency level (scaled to the interval (0,1) as well) correlates significantly positively within the error probability of 0.05 with Bartle Explorer (0.249) and BFI‐K Conscientiousness (0.413). The later even within significance level 0.01. A significantly negative correlation within the 0.05 error probability was found with Bartle Killer (‐0.242) and BFI‐K Extraversion (0.240). The pupils own agreement to the statement “To the most of my classmates I have a positive relationship” on a Likert scale from 0 to 3 (I totally disagree – I totally agree) is named as climate in the table 2 and correlates positively with Bartle Achiever (0.243) and with BFI‐K Neuroticism (0.278) within the significance level 0.05. Like the proficiency level the climate correlates as well within the significance level of 0.01 negatively with BFI‐K Extraversion.

5. Conclusions and implications for the design of games for learning All in all, the results do not fulfill the expectations of the study. Predicting the playing style preferences based on the BFI‐K profile of a gamer is only possible for the Socializer playing style. Moreover, the correlation of 0.276 does not even support a strong predictability based on the BFI‐K Conscientiousness value. Still, it sounds like a reasonable effect that pupils with higher values in Conscientiousness are as well more likely to focus in computer games on Socializing. For the learning style preferences the situation appears to be similar. Most of the learning style preferences cannot be predicted based on the BFI‐K values. Only for Kolb LSI Experiencing there is a significant effect that this value might be higher if a gamer has a low Openness for experiences (significant negative correlation of ‐.287). This is a surprising result as it might be a reasonable hypothesis that these two dimensions are potentially positive correlated. This could be corroborated by the not significant small positive correlation (0.137) to Bartle Explorer, but is at the same time contradicted by the negative

333

Johannes Konert, Stefan Göbel and Ralf Steinmetz correlation between Bartle Explorer and the Kolb LSI Experiencing. Thus it remains unclear why Openness for experiences and Experiencing correlate negatively. Table 2: Correlations (Pearson) of pupils' level of proficiency (math) and positive social connectedness (climate) with the three models' dimensions (Bartle, Kolb LSI and BFI‐K).* significance level 0.01 ** significance level 0.05 proficiency level (N = 70)

climate (N = 72)

corr.

sig.

corr.

sig.

Bartle (Achiever)

.016

.896

.243* .040

Bartle (Explorer)

.249*

.037

‐.029

.807

Bartle (Killer)

‐.242*

.043

‐.042

.728

Bartle (Socializer)

.073

.550

‐.140

.241

Kolb LSI (Experiencing)

.173

.152

‐.134

.262

Kolb LSI (Reflecting)

.166

.171

.014

.906

Kolb LSI (Thinking)

.217

.071

‐.116

.331

Kolb LSI (Doing)

.065

.590

‐.026

.826

BFI‐K (Extraversion)

‐.240*

BFI‐K (Agreeableness)

.023

.850

‐.008

.945

BFI‐K (Conscientiousness)

.413**

.000

‐.050

.674

BFI‐K (Neuroticism)

‐.128

.292

.278* .018

BFI‐K (Openness)

‐.139

.251

.196

.045 ‐.331** .005

.098

Beside the fact that the main expectations could not be fulfilled, it is worth to discuss the strong significant correlations within the models. Usually the model’s dimensions tend to be independent from each other to allow a maximum of diversity within the combination of values for the model dimensions. While the negative correlations of Bartle’s playing style preferences can be explained by the nature of these styles and as argued by Bartle (1996) himself in his publication, it remains unclear why all learning style preferences correlate among each other and why there are significant correlations within the BFI‐K dimensions. The latter are explicitly designed as independent dimensions and are evaluated in manifold studies. Thus it can be concluded that the sample group of pupils in the study at hand is not representative and has a bias concerning the personality dimensions of the pupils. This could be influenced by the schools we th selected, the personality state most pupils at 9 grade are in or as well the mood in which they were while answering the questionnaire. Still, none of these effects seems to be really a probable cause of the correlation effects in the normally independent personality dimensions of the five Neo‐FFI dimensions. Mainly the strong dependency of the Kolb LSI style preferences appears to be the most surprising effect of the within‐model correlations we found. Basically these values can be interpreted as such, that all pupils train and elaborate their skills within the four learning style dimensions in parallel and do not focus on one learning style. This can be interpreted as a direct result of a diversified teaching and instructional setup in the school classes and thus is a quality measure of the didactically well‐designed education the pupils received (i.e. the more positive correlations within the Kolb LSI model the better the educational design). This is supported by the fact that Kolb and Kolb (2005) themselves describe it as one application scenario for the LSI to find the learning style with the most deficits for each individual with the aim to improve the competencies in this learning style with the long‐term aim to have balanced values for all learning styles of each individual. This seems to be the case for the pupils in the study described here. Concerning table 2 the primary interest lies on the predictability of the values for the models of Bartle and Kolb LSI. None of the analyzed items correlates significantly with any of the Kolb LSI dimensions; even though several more items (not listed in the table) from the electronic questionnaire where as well analyzed for correlations (like computer expertise level and amount of time per week using computers). As a result, the

334

Johannes Konert, Stefan Göbel and Ralf Steinmetz significant correlations with the Bartle playing styles Achiever, Explorer and Killer remain (Climate correlates significant positively with Bartle Achiever .243; proficiency level correlates significant positively with Bartle Explorer .249 and significantly negative with Bartle Killer ‐.242). Even though it might be expectable that a pupil with a positive climate (social relationship) to his classmates may have then as well the playing style preference of Bartle’s Socializer, it can be argued that there is no significant correlation in this study results, because such pupils might be more focused on using the social interaction and connectivity for achieving personal goals (Achiever). Indeed not surprisingly there is a significant correlation among proficiency level and Bartle’s Explorer playing style preference as it suits highly skilled pupils to be as well eager to know all approaches to a quest or task and to know the most efficient (and rarely known) way to approach problems. Pupils with a high proficiency level significantly tend to dislike the Bartle’s Killer style of playing as they might not focus on causing negative impacts on their peers due to an already reached high level of competency (proficiency). As a consequence from the results displayed in table 2, predictions can be made based on the climate values and proficiency level of a pupil. The higher the two values, the more she may as well prefer the Bartle player styles Achiever and Explorer and won’t tend to the Killer style. In summary, the study revealed that the prediction of Bartle playing style preferences or Kolb LSI learning style preferences is not sophisticated possible on basis of the NEO‐FFI personality values conducted from the BFI‐K questionnaire. Precisely at least based on the data of the study described here such conclusions cannot be made. Still, the results showed some significant and positive correlations to predict Bartle’s playing style preference for Socializer based on BFI‐K Conscientiousness and Kolb’s LSI Experiencing based on the BFI‐K Openness value. Still, for the three other playing style preferences of Bartle’s model (Achiever, Explorer and Killer) a tendency can be seen that they are predictable based on pupils’ proficiency level and perceived social connectivity to their peers. A conclusion for the design of games and game‐based learning applications is the validity to model the dimensions for learning style preferences and player style preferences still separately within the game engines as there is no common strong correlation with the personality model of NEO‐FFI. The described study revealed some (small) significant correlations that primarily allow drawing some predictions for Bartle’s playing styles of a person when using additional item values. Researchers are encouraged to further investigate how the established models can be combined and used most effectively together to keep players in the state of flow in both models’ worlds: playing and learning.

References Albert, D. & Lukas, J., 1999. Knowledge Spaces: Theories, Empirical Research, and Applications D. Albert & J. Lukas, eds., Routledge. Andreasen, E., Bartle Test of Gamer Psychology. gamerDNA. Available at: http://www.gamerdna.com/quizzes/bartle‐test‐ of‐gamer‐psychology/ [Accessed April 27, 2013]. Bartle, R.A., 2004. Designing Virtual Worlds, New Riders. Bartle, R.A., 1996. Hearts, clubs, diamonds, spades: Players who suit MUDs. Journal of MUD research, 1(1), p.19. Available at: http://www.mud.co.uk/richard/hcds.htm [Accessed January 22, 2011]. Chen, J., 2007. Flow in Games (and Everything Else). Communications of the ACM, 50(4), pp.31–34. Felder, R.M. & Silverman, L.K., 1988. Learning and Teaching Styles. Engineering Education, 78(June), pp.674–681. De Fruyt, F. et al., 2004. The Five‐factor Personality Inventory as a measure of the Five‐factor Model: Belgian, American, and Hungarian comparisons with the NEO‐PI‐R. Assessment, 11(3), pp.207–15. Kickmeier‐Rust, M., Göbel, S. & Albert, D., 2008. 80Days : Melding Adaptive Educational Technology and Adaptive and Interactive Storytelling in Digital Educational Games. In B. F.‐M. H. K. Ralf Klamma Nalin Sharda & M. Spaniol, eds. Proceedings of the First International Workshop on Story‐Telling and Educational Games (STEG’08). CEUR Workshop Proceedings, p. 8. Kolb, A.Y. & Kolb, D.A., 2005. The Kolb Learning Style Inventory–Version 3.1 2005 Technical Specifications. Boston, MA: Hay Resource Direct, pp.1–72. Mehm, F., 2010. Authoring serious games. In Conference on the Foundations of Digital Games. pp. 271–273. Mehm, F. et al., 2010. Bat Cave: A Testing and Evaluation Platform for Digital Educational Games. In Proceedings of the 3rd European Conference on Games Based Learning. Copenhagen: Academic Conferences International, pp. 251–260. Rammstedt, B. & John, O.P., 2005. Kurzversion des Big Five Inventory (BFI‐K): Diagnostica, 51(4), pp.195–206. Wendel, V. et al., 2010. Woodment: Web‐Based Collaborative Multiplayer Serious Game. Transactions on Edutainment IV, 6250, pp.68–78.

335

Raising Awareness on Archaeology: A Multiplayer Game‐Based Approach With Mixed Reality Mathieu Loiseau1, 2, Élise Lavoué1, 3, Jean‐Charles Marty1, 4 and Sébastien George1, 2 Université de Lyon, CNRS, France 2 INSA‐Lyon, LIRIS, UMR5205, F‐69621, France 3 Université Jean Moulin Lyon 3, MAGELLAN, LIRIS, UMR5205, France 4 Université de Savoie, LIRIS, UMR5205, F‐69621, France 1

Mathieu.Loiseau@liris.cnrs.fr Elise.Lavoue@liris.cnrs.fr Jean‐Charles.Marty@liris.cnrs.fr Sebastien.George@liris.cnrs.fr Abstract: Our research deals with the development of a new type of game‐based learning environment: (M)MORPG based on mixed reality, applied in the archaeological domain. In this paper, we propose a learning scenario that enhances players’ motivation thanks to individual, collaborative and social activities and that offers a continuous experience between the virtual environment and real places (archaeological sites, museum). After describing the challenge to a rich multidisciplinary approach involving both computer scientists and archaeologists, we present two types of game: multiplayer online role‐playing games and mixed reality games. We build on the specificities of these games to make the design choices described in the paper. The proposed approach aims at raising awareness among people on the scientific approach in Archaeology, by providing them information in the virtual environment and encouraging them to go on real sites. We finally discuss the issues raised by this work, such as the tensions between the perceived individual, team and community utilities, as well as the choice of the entering point in the learning scenario (real or virtual) for the players’ involvement in the game. Keywords: game‐based learning, multiplayer game, mixed reality, learning scenario, archaeology

1. Introduction We believe that Game‐Based Learning (GBL) can significantly enhance learning, and does so in diverse application domains. In the last years, we have set up number of experiments using GBL environments. Immersing learners in such digital environments is a good way to enhance their learning, and especially their level of motivation. However, a well‐known limitation of GBL learning is the difficulty for learners to apply in real life the concepts they learn in the game. Indeed, the objects manipulated in the game are a modelization of real life objects. To apply in real life the concepts acquired in game, learners need to become aware, to a certain extent, of the abstraction underlying the modelization. An operation, which we believe can be eased by manipulating directly the real life objects. We address this issue in this paper in a specific GBL environment: (M)MORPGs, which rely on the collaboration and interactions amongst learners to increase their level of motivation. In 2012 computer scientists and archaeologists have launched a multidisciplinary project called JANUS, in order to design and experiment an educational MORPG. Additionnally, the pluridisciplinary approach underlying the project is in itself a challenge, as we will explain in section 2. Archaeology is a particularly appropriate domain to apply both this mixed (virtual/real) GBL approach and collaborative learning: it directly depends on physical artifacts as the seminal input to the various and complementary subsequent processes inherent to the several subdomains of Archaeology. The first result of our joint ideas is a learning scenario described in section 4 that explains how to integrate activities in a virtual world (the game and web sites) and activities on archaeological sites or in museums. The design of this scenario is based on two types of games that we study in section 3: multiplayer online role‐playing games and mixed reality games. In order to keep a strong motivation in the game, we have decided to organize a learning quest where students are grouped into guilds. We enhance the interrelationship of people inside their guilds, and the personal contributions of each student for the rest of the group (the community of learners). We also describe how we swap from activities in the virtual world to activities in the real world through a unique scenario. The idea here is to keep the same motivation and a continuous experience in the virtual game and at the museum

336

Mathieu Loiseau et al. or on the archaeological site. We finally conclude by drawing lessons about this work and by explaining generally the main research questions that have been raised and that we intend to address in the near future.

2. Designing new GBL activities As explained earlier, we would like to propose a GBL approach for raising awareness among students on the scientific approach in Archaeology. This is why, in 2012, computer scientists and archaeologists have launched a multidisciplinary project called JANUS, in order to design and experiment an educational MMORPG. Basically, this educational environment must allow individuals or groups of users to embark on a quest for knowledge through the discovery of archaeological items. However, a major goal of the project is also to create a community around archaeology. Peripheral aspects such as promoting the cultural heritage and make people discover it through city walks are also to be considered. These challenging objectives lead immediately to very complex questions: first of all, we believe that “learning by doing” is an interesting aspect to introduce in our learning sessions. This led us to consider how to take advantage of mixed (virtual/real) GBL features. In fact, even if general knowledge can be acquired in a virtual environment, more practical items must be studied on the archaeological site, in the museums or at the library. We can expect here that the motivation provided by a virtual game‐based environment can be improved with challenges in the real world. A side effect of having part of the activities on cultural sites is actually to promote the cultural heritage. Second, the learning process is not necessarily an individual process. Current trends on teaching tend to introduce collaborative learning, involving groups of learners to gather, share and use knowledge to solve new problems. Collaborative learning features are thus also central in our research work. Third, some activities must be designed in order to generate collective debates, to produce new information, vital to the achievements of new social behaviours. The project involves researchers in very different domains ranging from computer science to archaeology. There is another challenge to this rich multidisciplinary approach: the emergence of a shared vocabulary and the understanding of the research questions emphasized in each area. Our first step was thus to list the different project requirements. We subsequently held several meetings that were dedicated to exchanging knowledge and approaches to problems. As the domains were quite large to cover, we decided to design a learning scenario that explains how to integrate activities in a virtual world (the game and web sites) and activities on archaeological sites or in museums. To sum up, we were faced with the problem of designing a scenario where collaborative aspects are considered, where mixed (real/virtual) aspects are included, while respecting strong domain constraints (precise tasks to be taught). Furthermore, we wanted to integrate particular activities facilitating the emergence of an archaeological community. In order to understand well the domain constraints and because the requirements were somewhat vague, we decided to adopt an Agile software development process, following a Scrum approach (Kniberg, 2007). We have therefore divided our projects into sprints (3 weeks periods) to build our scenario. As recommended in the Scrum methodology, we have gathered all the ideas of functionalities to be included in the scenario in a file called the “backlog product”. At the beginning of each sprint, both the archaeologists and the computer scientists meet and discuss the new version of the scenario. Relevant changes can be performed as well as new ideas added in the backlog product. The team then decides what features have the highest priority to be included in the scenario for the next sprint. Since collaborative and community aspects are concerned, we must clearly rely on multiplayer games and also consider the work carried out in mixed reality games. We now present a state of the art for these domains and give more concrete examples on the scenario that is currently implemented in the project.

3. State of the art 3.1 Multiplayer games Game‐Based Learning (GBL) has the potential of improving training activities and initiatives thanks to its engagement, motivation, role‐playing, and repeatability (failed strategies can be modified and tried again). These games have proven to be useful to provide learners with pedagogical content in a ludic or/and realistic way (Flynn et al., 2011). However most LGs are played individually and learners evolve in the game without

337

Mathieu Loiseau et al. interaction with “real” learners. One way to enhance learning and engagement in the game is to allow collaboration by proposing a multiplayer environment. Currently, we observe the emergence and success of online multiplayer games in the world (Rosenbloom, 2004) and even in education (Purdy, 2007). Multiplayer Learning Games (MLG) usually immerse the players in a virtual 2D or 3D environment (Marty & Carron, 2011) and propose collaborative activities. This type of games can support development of a number of various skills: strategic thinking, planning, communication, collaboration, group decision‐making and negotiating skills (Squire & Jenkins, 2003). Players learn not only from the game, but also from each others (Purdy, 2007). Unfortunately, these games usually allow collaboration only among a limited number of students inside the virtual world. Furthermore, collaboration occurs according to a predefined learning scenario, often regulated by a teacher. Learners have thus restricted possibilities of interaction with other learners. We believe that it is one of the main reasons why students tend to consider Computer‐Based Learning Environments as unexciting. Massively Multiplayer Online Games (MMOG) are nowadays predominantly played by digital natives. Educational MMOG often works as tournaments and are based on competition between groups of students like in (Araya et al., 2001). The most popular type of MMOG, and the sub‐genre that pioneered the category, is the Massively Multiplayer Online Role‐Playing Game (MMORPG). An MMORPG is “an immersive 3D world where hundreds or thousands of players connect simultaneously from all over the world in order to meet each other in a simulated reality” (Bennassi et al., 2011). Many MMORPGs offer support for in‐game guilds or clans that are groups of players coming together to share knowledge, resources and manpower to reach common goals. For example, World of Warcraft is a MMORPG set in a fantasy world (like “The Lord of the Rings”). The aim of the game is to conduct a series of missions, so‐called quests, with progressive levels of difficulty. We can see interaction in a MMORPG as a condition for social learning (Wenger, 1998). Players exchange ideas, solve problems and create relationships, through technologies like chat or forum. (Paraskeva et al., 2010) have considered the functioning of online multiplayer educational games to highlight the importance of developing a strong sense of community among players to favor their motivation and engagement and the learning through a social experience. The sense of belonging in a community (game community or learning community) intrinsically motivates students to participate in the game and increase their performance. (Lavoué, 2012) defines Social Learning Games (SLG) as “games that enhance learning by offering educational contents according to a learning scenario and by supporting a community that offers condition for social learning”. The author highlights the different ways to enhance learning in SLG: the engagement in the game through the community, the mutual help in the community to make decisions in the game, the educational contents to initiate discussions in the community, the freedom in the community and the control in the game. MMORPG therefore foster learners’ engagement in the game and create dynamic learning opportunities due to the community. Role‐playing incites players to help each other to solve problems, by using their own different knowledge and capabilities. One of the major issues of our work is to combine three levels of communication and learning: individual, team (guild) and community. The players will be able to collaborate within their team and consequently to benefit from collaborative learning. They will also exchange information with other players and thus facilitate the emergence of a community interested in archaeology so that they will benefit from social learning. To design such a game, we will base our work on the four lessons given in (Zagal et al., 2006) for the design of collaborative games: (1) a collaborative game should introduce a tension between perceived individual utility and team utility, (2) individual players should be allowed to make decisions and take actions without the consent of the team, (3) players must be able to trace payoffs back to their decisions, (4) a collaborative game should bestow different abilities or responsibilities upon the players. Meanwhile, we have to facilitate the emergence of a community of players interested in archaeology. In such a context, we will induce collaboration amongst the members of a same team, and competition between the players from different teams. A game that is similar to our approach is described in (Wendel et al., 2010). The authors introduce methods and concepts of Woodment as a browser‐based Serious Multiplayer Game to teach and explore a customizable learning content in a game‐based and playful manner. The immersive environment sets the scene for a business simulation game in the field of logging: players can explore the island’s 3D world, manage the company, react to unexpected events, fight for the victory of the team, communicate with others via in‐game chat, level up and create learning content using the web interface. This game supports several levels of

338

Mathieu Loiseau et al. exchange (teams and global), but only by the way of a global chat and a team chat. In the course of the game, the players can take tests and answer quizzes individually or collectively. But there does not seem to be any connection between the educational content and the game itself, as the educational content can concern any topic. This can be considered a strength as it makes the game infinitely extendable — one question at a time — and repeatable, but it can also be perceived as a weakness as the conjunction of the business game and the quizzes can seem artificial to the learners. Here, we are developing a game under the form of a quest. It is scenarized to give the leaner a point of view on a precise field: archaeology. The form of the quest limits the repeatability of the game (in Woodment for instance the business game can be played many times) but integrates the learning activities in a consistent environment. Additionally, the quest mixes real and virtual activities, calling for a study of mixed reality games.

3.2 Mixed reality games Learning Games have sometimes some limitations when it is necessary to transfer the knowledge learned in other situations. It could be useful to create situations closer to reality, especially for the acquisition of professional skills (e.g. technical gestures or procedures performed in complex environments). To achieve this, a solution is to go beyond the conventional use of computers and to use Mixed Reality techniques. The Mixed Reality (MR) refers to a continuum that connects the physical and digital worlds and it includes schematically two components (Milgram & Kishino, 1994):

Augmented Reality (AR), where the real world is enriched by virtual information.

Augmented Virtuality (AV) where, in contrast, a virtual world is enriched with real objects (e.g. by using tangible interfaces to manipulate virtual objects).

Several devices can be used for implementing MR, including see‐through Head Mounted Displays (HMD), mobile devices such as digital tablets or smartphones, digital tabletops, or tangible interfaces that control or represent virtual information. Nilsen, Linton & Losser (2004) have shown that augmented reality kept both the benefits of real activities (especially communication between people) and the benefits of virtual environment, including the possibility of introducing virtual characters. Furthermore, adding information to real objects, information that is not perceptible naturally, is of great interest for educational situations. Some studies, such as Stedmon and Stone (2001), show that augmenting physical artifacts with associated information, facilitates the understanding of concepts. More recently, David et al. (2010) studied how physical artifacts (industrial machines or computers) could be augmented with digital data to promote just in time learning. Mixed Reality seems quite suitable for the learning of gestures, actions and operations, particularly in mobile and work situations. Mobile devices give the opportunity to develop an active pedagogy, favoring learning in authentical context and allowing the use of the natural environment as a source of information. Some knowledge needs students to learn through observation and is not always easy to teach during a traditional classroom or with a web‐ based learning environment. Mobile application proves to be useful in this case. For instance, Explore! (Ardito et al., 2009) supports young students learning ancient history during a visit to archaeological parks. The game allows students to explore 3D reconstructions of historical buildings, objects, and places, and also contextual sounds are played in order to recreate the historical atmosphere and enhance the overall user experience. While it is still too early to draw conclusions about the effectiveness of MR on learning (Anastassova & Burkhardt, 2008), learning outcomes seem to be rather short‐term. In order to overcome these short‐term limitations, more elaborate learning scenarios should be designed. By coupling Mixed Reality and Learning Games, it is possible to combine from one hand motivational and situated aspects of MR and on the other hand fun and scripted activity of LG. This point was the main goal of the SEGAREM project (George et al., 2013). During this project, an experimental study has been conducted to evaluate the impact of Mixed Reality interactions on learning. A Mixed Reality Learning Game (MRLG) was designed to teach Lean Management principles in an engineering school. Some actions are carried out with mixed reality, by using tangible interfaces over tabletops (e.g. a physical glue gun with an infrared light is used to stick virtual elements). The comparison between a learning session with the MRLG and an other without mixed reality interaction revealed a tendency for the MRLG situation to effectively have a positive impact on learning, particularly the understanding of theoretical concepts is favoured (George et al., 2013).

339

Mathieu Loiseau et al. The work presented in this paper is clearly situated within the trend of mixed reality games described in this section. The originality of the Janus project lies in the exploration of the possibilities of an educational MORPG, which combines both real and virtual worlds.

4. A learning scenario for a MORPG based on mixed reality Before entering the description of the scenario, we will describe the environment on which we build our game. We decided to reuse BrowserQuest 1 , an open‐source HTML5 retro‐gaming styled MORPG. It had two main features at its core that decided us to reuse it:

as a browser game both for PC and mobile devices, it can be used for MR applications;

it provides ingame communication mechanisms required for collaborative learning.

Being totally open, we have here a highly customizable tool, but it was not designed as a game engine, which should be taken into account by developers.

4.1 An exploration game Our game, like BrowserQuest, is an exploration game. The exploration will lead players to glean information and missions from NPC, to meet and talk with other players, including guild members. In Figure 1, we take into account the explorative nature of the game, one should therefore not perceive it as a linear timeline, but more as a spatial representation: the left being further away from the end of the quest than the right side. But the player is free to evolve in any direction (the dotted arrows display how new behaviours are unlocked) except in rare cases when an event triggers an activity (plain arrows). In such a setting, the progress of the player(s) is based on events that affect the narrative and the tasks at hand. In the rest of this section, we will explain how we handle these events in order to play our scenario, especially from the standpoint of its mixed‐reality and collaborative aspects.

Figure 1: A technical point of view on the first phases of the scenario

4.2 A mixed approach: Real and virtual experiences In the portion of scenario described above, we isolate three modalities respective to the role of the real world in the activity (virtual, mixed and real world experiences). Some actions take place exclusively in the virtual environment. This is the case of the introduction of the game, where the players learn about the storyline and their global role in the scenario. This is also the case of the exploration of BrowserQuest, which is of course solely virtual. All the same, being virtual does not mean it is disconnected from the real world: the underlying map contains representations of real places. These places act as landmarks of the real world. When visiting archaeological sites, the learners 1

https://github.com/browserquest/BrowserQuest (repository maintained http://www.littleworkshop.fr/browserquest.html (description by the developers)

340

the

community)

Mathieu Loiseau et al. can access this map from any place thanks to mobile devices, and so access information on that place and linked quests. Indeed aids provided for real world tasks come from the virtual environment. With this respect the learners need to be able to recognize in the virtual environment, the places they have visited in real life. Likewise, geolocalization is a means for the environment to provide contextual aid. For instance, the “taking pictures” activity is dedicated to allowing the learners to realize that various gallo‐ roman monuments exist in their city. They are therefore asked to photograph some of them. This allows explaining some of the standards of pictures used in archaeology archives. Learners are also asked to notice monuments that will later be at the center of other activities. To help players to find all the expected monuments, aid can be provided by NPCs (Non Playing Characters) depending on where the user is, giving more precise information if the user is close to the concerned location. Yet we do not consider this activity as mixed, as a player having studied well the task description and knowing very well the city, could finish it without any connection to the system, and with no other device than a camera. On the other hand, certain activities require the user to be in a specific location both in the virtual world (place on the map/url) and in the real world (artifact). In this case, the real artifact is contextualized by its representation in the system. For such an issue, geolocalization is inadequate in that it describes the players or their device, more than it describes the environment. In this case, we resorted to QR codes that both allow ensuring that the players are in contact with the considered artifact and that they will load the appropriate url to perform the task at hand. In the “observation task” in Figure 1, the players are asked to observe a monument and need to answer questions on site using a mobile device. The execution of such activities being highly circumstantial the QR code triggers the task and does not just unlock it.

4.3 A combination of individual, team and community experiences In order to keep a strong motivation in the game, we have decided to organize a learning quest where students are grouped into guilds and can also carry out social activities involving the whole community of learners. We describe in this part how the proposed scenario is meant to enhance collaboration into the teams and social interactions into the global community. The “observation task” (see Figure 1) is an example of a task we consider as individual: the players can play right after entering the game without waiting for others. Such a task is also meant to help them learn the 2 specialization associated to their role . This is the object of an individual progression, associated to an indicator, to be presented to the user as recommended in (Zagal et al., 2006). Individual activities also serve the group goal: players of different roles constitute guilds. The information gathered by players in individual tasks are meant to be reused for the progress of the team. All teams have the same goal that is to build a virtual information panel for an archaeological site. The information panel constitutes a virtual representation of the team's progress. It is therefore accessible to all players at any 3 moment for monitoring purpose. We use an etherpad to implement it. Each quest aims to provide information to the team to fill the notice. There is a competition between the teams, since at the end of the game the best virtual notice should be printed to be put in the real site. An example of such group progress is provided in the scenario. When all players in a guild have completed the “observation task”, a flag is lifted so that the NPC1 adapts its message to the task outcome. It reminds each player of the most important facts s/he has gathered in the tasks and suggests them to go on a mutualization task, i.e. to write them down in the etherpad, which allows both synchronous and asynchronous edition and also provides chat facilities.

So far we have implemented two roles: epigraphist and art historian, 1 other role is to be included in the game (architect). http://etherpad.org/

341

Mathieu Loiseau et al. Other tasks that take place later in the game require the guild members to collaborate in order to achieve completion. For example, in the “classification task” all players of a same guild have to be online at the same time: depending again on their role, the players are provided with different pieces of information. All are necessary to solve the puzzle. The circumstantial conditions required to perform this task make it necessary to trigger it at once when the players meet in the virtual environment (cf. NPC2). After each task, the last information gathered is reminded to the player by an NPC (cf. NPC1). The part played by the guild in the quests is both meant to support players’ involvement and the co‐ construction and appropriation of knowledge by learners. We therefore aim at enhancing collaboration within teams and social activities within the global community. Indeed, some tasks involve the players at community level. For example, the “taking picture” task (see Figure 1), linked to NPC3, can be carried out individually by players, but its outcome is meant to be shared with the community. The other teams' members can rate these photographs and comment them. We incite players to participate by making their social indicator evolve according to their level of participation in this quest (posting photos, rating and commenting them). The evaluation provided is in term used to compute individual and guild successes.

5. Discussion and perspectives The JANUS project is a feasibility study leading to a prototype. It examines the possibility to provide a social, mixed reality, serious game to raise awareness about archaeology as well as to promote the city's monuments. We described in this paper the design choices we have made in the implementation process and especially in the learning scenario. We performed the connection between the virtual and the real world, by representing real places in the virtual map, by contextualizing information in the game using geolocalization and by identifying artifacts using QR codes. To complete the quests the players evolve as individuals and members of teams and of the community. This social component adds a layer to the problem of collaboration as described in (Zagal et al., 2006). While recommendation (4) — players with different abilities — does not require adaptation to our context, the decisions we took provide a lead for an extension of recommendation (1) to a social context. With the team final goal (creating an information panel for a monument), by allowing players to choose among all team productions an actual panel, we introduce a tension between perceived individual and team utility (voting to favor one's own team) and the perceived community interest (sharing valuable resources with outside the community, here the people visiting the archaeological site). To this tension, one has to add the tension between the different social consequences for individuals of the behaviors used to favor individual or team utility. All the same, this does not deny the possibility to apply the recommendation at the level described in Zagal et al.'s study. We could extend our scoring mechanisms by computing the team and the individual indicators according to different criteria: the individual score favoring the quickness of responses, the team score penalizing errors harsher. Given a choice, the learners would be able to choose between taking the risk of an error (individual utility) or involving others' knowledge to eliminate as much uncertainty as possible through discussion (group utility). This leads directly to another recommendation: (3) — tracing payoff back to one's decision. In the case of a closed output activity (such as the classification task), providing feedback on a user action can easily be computed. But the regulation role of the community seems noteworthy. Indeed the community can be instrumental to providing feedback on actions performed in more open tasks in terms of outcome (a set of photos or writing a text) than a fully computational solution. Additionally the social actions of the users (sharing pictures for instance) can be traced to influence the progress of the game (e.g. after sharing five photos among its members a flag allows a guild to unlock the next phase of the game, cf. NPC3 in Figure 1). If successful in practice, this would further anchor the game in the social web paradigm by “harnessing collective intelligence” (Musser et al., 2007). In the long run, this feature could be taken further by performing real community tasks. After evaluating photos and notices, the various outcomes could serve as a base for a community written article about city monuments, for instance in a wiki. The mixed reality aspect of the game is not without social consequences either. By bringing players to actual real life artifacts, the environment relinquishes control over user interactions. Players are free to communicate in the real world or to perform tasks together with team members or other players. These untraced interactions should not be neglected in later analyses nor ignored in the game design. Let's follow on the example we gave for computing individual task scores: computing team work solely by counting chat interactions during the task would negate the

342

Mathieu Loiseau et al. possibility for players to interact in the real world. This would be problematic in that the task actually creates the possibility for players to be physically able to communicate. This makes the actual city locations part of the environment of the game in exactly the same way as BrowserQuest. The location of the “observation task” initiating our scenario is in effect an entry point to our game through the presence of a QR code that can be scanned by any passer‐by or bystander curious about players present in the premises. This calls for general thoughts on how to favor entering a mixed reality game such as this one and the consequences of design decisions in term of guild constitution and community engagement. In our future works, we plan to conduct several experiments with people of different backgrounds to study the implications of these decisions on their progress in the learning scenario and on the level of knowledge acquired in the archaeological domain at the completion of the game.

Acknowledgements This work was supported by the LABEX IMU (ANR‐10‐LABX‐0088) of Université de Lyon, within the program "Investissements d'Avenir" (ANR‐11‐IDEX‐0007) operated by the French National Research Agency (ANR).

References Araya, R., Jiménez, A., Bahamondez, M., Dartnell, P., Soto‐Andrade, J., Gonzalez, P. and Calfucura, P. (2011) Strategies used by students on a massively multiplayer online mathematics game. 10th International Conference on Advances in Web‐Based Learning (ICWL 2011). Hong Kong, China, 8‐10 December 2011, pp 1‐10. Ardito, C., Buono, P., Costabile, M. F., Lanzilotti, R. and Piccinno, A. (2009). Enabling Interactive Exploration of Cultural Heritage: An Experience of Designing Systems for Mobile Devices. Knowledge, Technology & Policy, Vol. 22, No. 1, pp 79‐86. Benassi, A., Orlandi, C., Cantamesse, M., Galimberti, C. and Giacoma, G. (2011) World of Warcraft in the Classroom: A Research Study on Social Interaction Empowerment in Secondary Schools. European Conference on Game Based Learning (ECGBL 2011). Athenes, Greece, 20‐21 October 2011, pp 35‐45. David, B.T., Yin, C. and Chalon R. (2009) Contextual Mobile Learning Strongly Related to Industrial Activities: Principles and Case Study, International Journal of Advanced Corporate Learning, Vol. 2, No. 3, pp 12‐20. Flynn, R., McKinnon, L., Bacon, E. and Webb, J. (2011) Maritime city: using games technology to train social workers ‐ some initial results. 10th international conference on Entertainment Computing (ICEC 2011). Vancouver, BC, Canada, 5‐8 October 2011, p. 415–418. George, S., Michel, C., Serna, A. and Bisognin L. (2013) Evaluating the impact of a Mixed Reality Learning Game on Learning, International Journal on Learning Technology (submitted).Musser, J., O'Reilly, T. and the O'Reilly Radar Team (2007). Web 2.0 Principles and Best Practices (Sebastopol, CA: O'Reilly). Kniberg, H. (2007) Scrum and XP from the Trenches — How we do Scrum (C4Media). Lavoué, E. (2011) Towards Social Learning Games. 11th International Conference on Web‐based Learning (ICWL 2012), Sinaia, Romania, 2‐4 September 20ht, pp 170‐1798. Marty, J.‐C. and Carron, T. (2011) Observation of Collaborative Activities in a Game‐Based Learning Platform. IEEE Transactions on Learning Technologies. Vol. 4, pp 98‐110.. Milgram, P. and Kishino, F. (1994). A Taxonomy of Mixed Reality Visual Displays. IEICE Transactions on Information and Systems, Vol. E77‐D, No. 12, pp 1321‐1329. Musser, J., O'Reilly T. and the O'Reilly Radar Team (2007) Web 2.0 Principles and Best Practices, Sebastopol, O'Reilly Media. Nilsen, T., Linton, S. and Looser, J. (2004) Motivations for augmented reality gaming. Proceedings of FUSE, New Zealand Game Developers Conference, pp 86–93. Paraskeva, F., Mysirlaki, S. and Papagianni, A. (2010). Multiplayer online games as educational tools: Facing new challenges in learning, Computers & Education, Vol. 54, No. 2. Purdy, J.A. (2007) Serious Games: Getting Serious About Digital Games in Learning, Corporate University Journal, Vol. 1, pp 3‐6. Rosenbloom, A. (2004) Interactive immersion in 3D graphics, Communications of the ACM, Vol. 47, pp 28–31. Squire, K. and Jenkins, H. (2003) Harnessing the power of games in education, Insight, Vol. 3, pp 5‐33. Stedmon, A. W. and Stone, R. J. (2001) Re‐viewing reality: human factors of synthetic training environments, International Journal of Human‐Computer Studies, Vol. 55, No. 4, pp 675‐698. Wendel, V., Babarinow, M., Hörl, T., Kolmogorov, S., Göbel, S. and Steinmetz, R. (2010). Woodment: web‐based collaborative multiplayer serious game, Transactions on edutainment IV, Zhigeng P., Cheok A.D., Müller, W., Zhang, X. and Wong, K. (Ed.), Berlin, Heidelberg, Springer‐Verlag, pp 68–78.. Wenger, E. (1998) Communities of practice: learning as a social system, The Systems Thinker, Vol. 9, No. 5. Zagal, J. P., Rick, J. and his, I. (2006) Collaborative games: lessons learned from board games, Simulation and Gaming Vol. 37, No. 1., pp 24–40.

343

Scientific Discovery Games for Authentic Science Education Rikke Magnussen1, Sidse Damgaard Hansen2, Tilo Planke2 and Jacob Friis Sherson3 1 ResearchLab: ICT and Design for Learning, Department of Communication, Aalborg University, Denmark 2 Department of Physics and Astronomy, Aarhus University, Denmark 3 AU Ideas Center for Community Driven Research, CODER, Aarhus University, Denmark rikkem@hum.aau.dk sdh06@phys.au.dk tilo@phys.au.dk sherson@phys.au.dk Abstract: This paper presents results from the design and testing of The Quantum Computer Game, a game that allows players to help solve actual scientific challenges in the effort to develop a quantum computer, which is a computer where individual bits can be both 0 and 1 simultaneously – potentially offering more computational power than all conventional computers combined. The main objective of scientific discovery games is to facilitate collaboration between researchers and gamers, but the focus of The Quantum Computer Game, in contrast, is multifaceted. The motivation for developing this type of game concept for science education stems from a critique that the way standardised skills are taught in today’s school system leads to students becoming experts at consuming rather than producing knowledge. The primary aim of developing a game‐based platform for student research collaboration is to investigate if and how this type of game concept can strengthen authentic experimental practice and the creation of new knowledge in science education as well as what elements play a central role in this. Researchers and game developers from the Department of Physics and Astronomy at Aarhus University and ResearchLab: ICT and Design for Learning at Aalborg University tested the game in three separate high school classes (Class 1, 2, and 3) and used video observations to record the students, aged 17‐20, playing the game. Qualitative interviews were conducted with the classes and their teachers after the game sessions and all students filled out surveys with qualitative and quantitative questions. The focus of the various tests was to understand the motivational aspects of students playing this type of game and how students felt about participating in authentic experiments as well as to detect whether the game could offer new types of educational approaches to highly complex subject areas such as quantum physics. The tests in the first two high schools showed that collaboration with researchers and contributing to research in quantum computing were highly motivating factors. In a survey with multiple possible answers conducted after the game session students were asked to state what the most interesting aspect of playing the game was. To this question 69% answered “To participate in real scientific research”, 69% answered “To solve physics problems” and 31% “To play games”. This is an interesting result as games in education often are viewed as a tool to motivate students to participate in educational activities. Here games become a tool to frame or facilitate processes where the motivation lies in the subject the game covers or in the research context outside the school context. Designing a game that facilitated professional research collaboration while simultaneously serving to introduce high school students to quantum physics at their level proved, however, to be a challenge. When asked whether they had learned about physics from playing the game using a five‐point scale ranging from 1 for “not at all” to 5 for “a lot”, 8% of the students in Class 2 answered 1; 46% answered 2; 23% answered 3; 23% wrote 4 and no one checked 5. The third round of testing in Class 3 incorporated a didactic design developed to integrate the game into a laboratory classroom setting that involved simulations, theoretical work and physical experiments to strengthen student expertise in these areas. When asked whether they had learned about physics, 14% answered 1 (“not at all”) and 7% answered 2, while 36%, 14% and 29% answered 3, 4 and 5 (“a lot”), respectively. The results presented in this paper show that scientific discovery games and the fact that they make participating in authentic scientific experiments possible is highly motivating for students. The findings also show, however, that the learning design in the class setting must be considered in order to improve the students’ experience of learning and that various design challenges remain to be developed even further. Keywords: scientific discovery games, science education, quantum computing

1. Introduction Teaching with the use of games and simulations in school science education was introduced in the 1970s and in the early 1980s the potential of games and simulations as a new teaching tool was discussed extensively (Ellington et al., 1981). In the early 1990s the first IT‐based games for the exploration of the natural sciences and technical subjects were developed (Egenfeldt‐Nielsen, 2005). After the turn of the millennium, there has been an increasing awareness about the possibilities new types of commercial computer games can offer science teaching, but also about developing new formats framing aspects of real‐life science learning environments that allow players to tackle complex problems in simulated professional contexts (e.g. Squire &

344

Rikke Magnussen et al. Klopfer, 2007; Magnussen, 2007). These games have proven to support new practices in science education, such as in student development of new professional inquiry tools adapted for schools, innovation in networks of new types of non‐school actors, e.g. fictional characters and authentic professional tools, and processes for the imaginative creation and representation of new knowledge (Magnussen, 2008). More recently, scientific discovery games have been developed where player contributions to real‐life research practice are an integral part of the game. The prime example of this type of games is the game Foldit where complex scientific problems are translated into puzzles and a game‐like mechanism is provided for non‐ expert players to help solve the problems presented (Cooper et al., 2010). The current paper presents the preliminary design considerations and initial test results of a scientific discovery game for an in‐school learning environment. We also discuss the potential and challenges of developing scientific discovery games for science education to enable students to work with unsolved scientific problems and the creation of new scientific knowledge.

2. Background: Gamified research collaboration in science education One of the main focus points in the development of science game formats over the past 10 year has been how the medium of games can introduce new approaches to authentic science education (Gee, 2003). Prime examples of this are profession simulation games that simulate some of the objectives and environments of a specific profession by using the technology, tools and/or methods of that profession. The motivation for developing these types of games stems from a critique of the teaching of standardised skills to children in today’s school system. It is argued that few schools teach students how to create knowledge; instead, students are taught that knowledge is static and complete, which means they become experts at consuming rather than producing knowledge (Sawyer, 2006). As a result, the medium of games has been used to create environments with simulations of complex real‐life situations, where students have to think like professionals and solve problems in innovative ways, just as professionals do (Shaffer & Gee, 2005). Interesting issues arise, however, in relation to this class of games that need to be addressed when discussing the integration of creation of new knowledge and authentic science practice in science education. Even though the games integrate professional values and tools, they remain simulations of professional practices. This aspect of the games brings up the matter of whether students learn to work as a scientific expert or whether they learn how to be a scientific expert. This may depend on various design elements of profession simulation games. First, the clients and experts students collaborate with in the games are fictional characters with fictional problems that need to be solved to play the game in school but that do not have relevance in the world outside school. Second, the fictional problems to be solved in these games often follow a linear path and have a clear starting and end point. This is clearly different from real‐life professional problem solving, where the processes are more multidimensional. Finally, even though these types of games have been shown to support student creation of new process tools, the solutions are often pre‐defined and already known by the teachers. This stands in contrast to the real‐life open‐ended tasks professionals face and that can be carried out in various ways, the chance of success or failure always an issue to be considered. Scientific discovery games address these issues that exist outside a formal learning setting. The main goal of this type of game is to create a platform that motivates players to contribute to solving scientific problems. One example, Foldit, mentioned earlier, is an online puzzle game where players participate in folding amino acid chains into new protein structures. Presented with a primary protein sequence or partially folded structure, players must find the lowest‐energy three‐dimensional structure, which can also be an unknown protein structure (Cooper et al., 2011). Players manipulate the protein structure by pulling, twisting and tugging the protein backbone and side chains into various configurations (Good & Su, 2011). Scientific discovery games contain specific design features that distinguish them from the majority of other games (Cooper et al., 2010; Good & Su, 20011). First, scientific discovery games are designed for non‐expert players to advance a scientific domain. As a result, the visual features and graphics must make it possible for beginners to experiment with highly complex solutions and scientific information. This requires that the game interface must be designed to introduce beginners to a highly complex field while simultaneously motivating them to play the game. Another distinctive feature of this class of games is that the puzzles do not have any pre‐defined solutions; even the game designers do not know the answers, of which there are potentially more than one. This also implies that the interactive design must make exploration and experimentation processes

345

Rikke Magnussen et al. possible while simultaneously respecting real scientific constraints. Consequently, the scoring mechanism must reward multiple player strategies while remaining true to the latest knowledge about the scientific phenomenon (Cooper et al., 2010; Good & Su, 20011). Various design features have to be reconsidered when designing scientific discovery games for an educational context. First of all we need to consider what new elements this class of games brings into science education. Scientific discovery games have the potential to introduce real‐life experiments and the processes behind the creation of new scientific knowledge into school science, but determining which aspects of an online game and the classroom are central for students to experience and engage in the open‐ended scientific inquiry process is necessary. Moreover, we also need to understand what the main motivating factor for playing this type of games in school is; is it, for instance, competing against other students, contributing to science or collaborating with scientists? Finally, other issues that need to be addressed are the implications of introducing this type of games for different types of students and the role of the teacher. How do students experience their learning in this type of games and will this class of games be reserved for the brightest students or will the less theoretical, more experimental approach employed open up complex subjects to other groups of students? The focus of this paper is to present the design and initial test results of the adaption of a scientific discovery game The Quantum Computer Game to a school environment in order to teach quantum physics to high school students in Denmark, and to discuss the potential and challenges of designing this type of game for school science education.

3. The quantum computer game The Quantum Computer Game represents a collaborative effort between researchers in physics, computer science and game‐based learning at the interdisciplinary Aarhus University Ideas Pilot Center for Community‐ Driven Research, established January 2012. The focus of the quantum game project is the research‐based production of a game‐based platform for player participation in quantum computing development and research.

3.1 The game Quantum computers are based on the principles of quantum mechanics and It has been proven that quantum computers will be able to perform certain important tasks much faster than all of the conventional computational power combined (Shor, 1994; Grover, 1996). The basic problem that players have to solve in the game is the optimisation of the transportation of atoms in a quantum computer (see figures 1 and 2) (Weitenberg, et al., 2011). We anticipate a community contribution on several levels. The computers were initially programmed to try out multitudes of transportation paths but failed to find the optimal ones. As a result the hope is that the graphical representation of the problem in the game will enable the human players to find better paths. This approach will be effective not only due to the sheer quantity of potential players but also because players can potentially apply the distinctly human skills of pattern recognition and intuition to perform a much more intelligent optimisation than computers can. Furthermore, an important aspect of in the Quantum Computer Game is extensive user participation in the initial design phase and in subsequent extensions.

Figure 1: Examples of tutorial games to introduce the quantum mechanical concepts and methods needed to understand the scientific challenge. (A) An illustration of the allowed quantum mechanical states and sliders to create mixtures of these. The basic lesson is that if the atom (represented as liquid like substance inside graph) is purely in one of the allowed states of the well, it will not move in time but in a mixture it will. (B) An atom is agitating in the well. The user then has to remove the kinetic energy by moving the well from side to side. The data on the position of the atom versus time is

346

Rikke Magnussen et al. listed to the right, allowing students to transfer the information to a plotting program to analyse the results of their experiment The game has two parts. The first part consists of tutorials that introduce players to quantum physics and that teach them how to operate the game (figure 1).

Figure 2: In this scientific part of the Quantum Computer Game players must move an atom (represented as liquid like substance inside graph) from the starting position to the target area (striped) by dragging the well. The scientific objective is to complete the transport without any agitation in the final position. (A) An introductory challenge where players must transport the atom without hitting the walls. (B) The real scientific challenge where players must keep the atom under the lighter curve lining the atom Players can continue playing games in the tutorial or they can move on to the advanced part (figures 2 and 3) to begin solving real research problems and to have their performance logged. The key scientific objective of The Quantum Computer Game is to develop algorithms with a small enough error probability to allow for quantum computations on a large scale without errors piling up. For each attempt that a player makes, a score is calculated based on the quality of the resulting quantum computer. The performance of every player is logged centrally and the overall highest score will always correspond to the state of the art of the research field and can thus be adjusted for each hour people play. The game also allows players to develop their own sub‐games, thus permitting them to contribute computational results and take part in continuously developing the game.

4. Test of the Quantum Computer Game The methodology used in developing the components of the Quantum Computer Game followed a design‐ based research process and involved various design cycles, interventions, analyses and redesign (Brown, 1992). The beta version of the game was completed in early 2012 and interventions were conducted in a number of high school classes and with online players. Results from two of the high school classes (Class 1 and 2) will be presented briefly below (Magnussen et. al, 2012). Based on results from the initial tests, the game was further developed and a new version was tested in April 2013 in a high school in west Denmark (high school 3). The test was not completed at the deadline of this paper, but preliminary results will be presented below.

4.1 Tests in Class 1 and 2 The beta version of the game was initially tested in two high school classes (Class 1 and 2) in February and October 2012, respectively (Magnussen et al., 2012). The results show both similarities and differences between the two settings. Class 1 consisted of 20 students 17‐19 years of age in their second year of high school, while Class 2 comprised 20 students 17‐ 20 years of age in their last year of high school. Due to the nature of Danish high schools, or gymnasier, which follow a three‐year curriculum, this meant that the students were in consecutive grades. Quantum physics had been introduced by the teachers in both classes. Quantum physicists from the development team introduced The Quantum Computer Game to the classes and conducted the testing over two class periods. Game play was recorded on video for later observation and semi‐structured interviews were conducted during the test with individual students and afterwards with the whole class. The classes were also asked to fill out a written survey after the test answering qualitative questions such as “What was the best worst/part of playing the Quantum game?” and “How does playing the Quantum Game differ from your other physics teaching?” and quantitative questions such as “Rate the following ‐ Have you learned physics by playing the game?” where students were asked to rate the statement on a scale from 1 (not at all) – 5 (a lot) Both classes tested an early beta version of the game (figure 3).

347

Rikke Magnussen et al.

Figure 3: An early beta version of The Quantum Computer Game tested in high school 1 and 2. Players had to move the atom from a well on the right to a well on the left while agitating the atom as little as possible As the individuals involved in the project represented developers, physicists and researchers in digital learning design, the initial testing included several points of focus, one of which was to understand what motivates players using this type of game. A central aspect of this class of games is the collaboration with researchers and the reason why determining whether or not this is a primary motivating factor is important. We wanted to ascertain whether the gaming elements that scientific discovery games share with other games, such as competing for high scores, also serve as a motivating factor for gamers in scientific discovery games. We conducted a number of similar observations in the two high school classes where the game was tested. Initially intensely interested and very motivated overall, the students in the two classes worked continuously with the game during the two class periods where we did observations. At the end of the second class period, some of the students had begun to lose interest. This was especially the case for the youngest group of students from school 1. The test was set up so the students had to download the game on their own laptops prior to the test session. The majority of the students used their own laptops, but a few shared laptops in groups of two or three. Students in Class 2 were asked to try the sub‐games on the Quantum Computer Game platform. In both classes, students discussed strategies and patterns for how to transport atoms in different environments as they played. They also discussed their scores and commented on results, high scores eliciting cheering and various remarks. Scores thus apparently seemed to be a central motivating factor for the students, but results from the qualitative interviews with the classes after the test and written survey show that other elements also played a role in motivating players. The answers provided during the interview and on the survey students filled out after playing the game varied in relationship to what participants thought was the best part about playing the game. The survey was completed by 7 out of 20 students. To the question “What is the most interesting part about playing the game?” 57% of students in Class 1 answered “To participate in real scientific research”, 14% answered “to solve physics problems, 14 % answered “To play games” and 14% answered “To participate in real scientific research, to solve physics problems, to play games”. In the qualitative interview with the whole class the class was asked what the greatest difference between playing the game and their normal teaching was and one student stated that, “In the normal teaching you only calculate the results, while in the game you get the feeling of directly doing the experiment”. When students and (after the interviews with the students) teachers were asked to expand on this comment, they explained that what happened in the game felt like an experiment compared to lab work done in class, which they saw as demonstrations and the theoretical premise for understanding the experiment. Class 2’s answers to questions to motivational factors differed slightly but overall showed the same results. In a survey with multiple possible answers conducted after the game session students in Class 2 were asked to state what the most interesting aspect of playing the game was. To this question 69% answered “To participate in real scientific research”, 69% answered “To solve physics problems” and 31% “To play games”. These results are interesting as games in education often are viewed as a tool to motivate students to participate in educational activities. Here games become a tool to frame or facilitate processes where the motivation lies in the subject the game covers or in the research context outside the school context. One student described what he felt was the most interesting aspect of playing the game as knowing, “that you have a real chance to help enable a quantum computer. It also irritated me when I didn’t get as many points in the

348

Rikke Magnussen et al. game as my friends did, which got me to play the games even more.” In response to the same question about what was most interesting, two other students explained, “The best thing about the game must be that it is one of the few games in this world that you can actually use for something” and “that they were relevant to physics, and that you have the chance to make a difference, even if it’s not a vital one, in the development of quantum computers (Cool!)”. The first answer indicates that the focus some students have on scores does not necessarily exclude a focus on research collaboration, whereas the two subsequent answers show that it is primarily, if not exclusively, the fact that they are contributing to research that motivates them to play the game. During the interview conducted immediately after the test in Class 2, 2 ‐ 3 students also said that they did not feel as if they had learned about physics. They explained that this was because moving the atoms was a simple task that did not expand their understanding of quantum physics. The survey results also showed that a larger part of the students had the esperience that the game did not teach them physics to a significant degree. When asked whether they had learned about physics from playing the game using a five‐point scale ranging from 1 for “not at all” to 5 for “a lot”, 8% of the students in Class 2 answered 1; 46% answered 2; 23% answered 3; 23% wrote 4 and no one checked 5.In the interview with the teacher after the test session, the teacher challenged the understanding that students had of “learning about physics”. He argued that their understanding of it was to practice to be able to complete assignments and added that the contact with the researchers and the game had given his students a deep understanding of quantum mechanics that he could not have given them. The teacher interpreted this as stemming from the strong focus the class had on the subject and from the fact that the students had to be prepared for their final exams. In summary, the results from these two classes showed that the main motivating factor proved to be research collaboration or solving physics problems and that 54% of students in Class 2 had the experience of learning none or little physics form playing the game. The game and setup in the class thus provided a strong experience of participating in an authentic experiment, but a less evident experience of learning physics form the participation. Teachers in both classes also commented on the different “tangible” approach the games had to a highly theoretical subject. In Denmark, high school physics is taught at a highly theoretical level, which may be the basis for the student comment that students “only calculate the results” in “normal” teaching, but that the game gave them the feeling of “directly doing the experiment”.

4.2 Test in Class 3: Experimenting with strengthening the student experience of learning Class 3 comprises a second‐year high school class consisting of 20 students 17‐20 years of age from a technical high school in west Denmark. The test in high school 3 has not yet been completed, which means that our initial observations are only preliminary, but worth including briefly to support the discussion of possible solutions to the design challenges involved in scientific discovery games. The feedback and findings from the tests conducted in high school 1 and 2 pointed the redesign of The Quantum Computer Game to a different focus. Part of the new focus for the further development of the game was to strengthen the student experience of participating in an authentic science experiment as this had proven to be a strong motivating factor for some students. Moreover, our hypothesis was that the authentic research collaboration aspect of the game could contribute with new didactical input to science education. One of the ways we strengthened the authentic aspects of the game was to make the researchers more visible on the game’s website scienceathome.org by including their pictures and by adjusting the graphics to match the atmosphere of the physics lab where data from the game were actually being used to develop a quantum computer. Another issue that the project group responded to with regard the second test was making improvements on what the teachers had commented on as having a tangible approach to a highly theoretically subject. We interviewed the teacher at high school 3 on this topic and other subjects before the test and she had observed that this “more intuitive” approach to a highly theoretical subject appealed to a group of students that was exceedingly interested in the subject, but felt that it was difficult. These teacher’s observations were in keeping with teacher comments from high school 1, where teachers commented that the game allowed for a more tangible approach to a theoretical subject. As a result, the test focused on investigating this issue further, but the final results have not yet been compiled. The final area of focus for implementing and testing the game in high school 3 was to strengthen student learning on the subject at both an experimental and theoretical level. Groups of students in Class 1 and Class 2

349

Rikke Magnussen et al. had explained that they did not feel that they had learned any physics while they were playing the game. Before testing the game the class teacher, in collaboration with the coder team, developed a didactical design focused on implementing theoretical approaches, experimental practice using physical tools and play with game simulations. The goal was to boost the authentic aspects as well as the theoretical and experimental practices by assigning students roles in the game as experts. This design was inspired by elements of simulation games on specific professions (Magnussen, 2007) described earlier in this paper and notions of collaborative learning (Dillenbourg, 1999). The game test session in Class 3 was conducted over four school periods and had been introduced in previous lessons by the teacher. The first session included an introduction to a simulation of a professional setup where the students introduced to a professional setup were they were a research team of physicists working on the development of a quantum computer by optimising the movement of your laser to transport atoms. Students were divided into three different teams, each one representing an area of expertise. One group comprised experimental physicists who was working on understanding the movement better by doing analogue experiments. The second group of experts was IT professionals specialised in simulations who did virtual experiments in the games. The last team worked theoretically and focused on understanding the mathematics behind the various elements of the movement. The teacher assigned students with skills suited to the expertise of the different teams. Students worked in groups with similar expertise for three class periods and were then mixed with students with different expertise with the goal of sharing results and to produce a poster or film with their conclusions. Overall students worked intensively with the game in the different teams. The simulation of the different professional approaches in the authentic framework of contributing to the scientific domain of these professions appeared to spur complex discussions concerning the results obtained from using physical experimental tools compared to the virtual experiments in the game. In the survey after the game students were asked whether they had learned about physics, 14% answered 1 (“not at all”) and 7% answered 2, while 36%, 14% and 29% answered 3, 4 and 5 (“a lot”), respectively. In summary results in this class showed that 79% of students answered 3 or above I Class 3 compared to Class 2 where only 46% answered 3 or above. In Class 2 no students answered 5 (have learned a lot physics) which differed to Class 3 where 29% answered 5. The data are preliminary and have not yet been fully analysed the above results indicate that the new design has strengthened the students’ experience of learning physics.

5. Discussion and conclusions The results presented in this paper show that scientific discovery games and the fact that they make participating in authentic scientific experiments possible is highly motivating for students. The findings also show, however, that the learning design in the class setting must be considered in order to improve the students’ experience of learning and that various design challenges remain to be developed even further. In order to successfully develop and introduce scientific discovery games into science education we need to focus on how the game operates and the didactical aspects that can strengthen importance elements in these games, such as authenticity and authentic experimentation. This paper described how a scientific discovery game can be didactically designed to fit a classroom setting by merging aspects from simulation science games about specific professions with the research collaboration approach. Other elements of importance we have detected are that the complexity of playing the game needs to correspond with the complexity of the scientific challenge. Another aspect that needs to be investigated further in future research is how this type of games can be applied for motivating weaker students in science education. In interviews with teachers from Class 1 and 3 the teachers stated that the intuitive or tangible approach of the game to quantum physics encouraged the weaker students to participate more actively. The student responses describe in this paper indicate that scientific discovery games must apply new approaches to integrating authentic knowledge creation and scientific practice into school science education for specific groups of students. Moreover, it needs to be further investigated how role playing and collaborative learning approaches can further strengthen the student learning experience and the outcome of that learning.

References Brown, A. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of Learning Sciences, 2(2), 141‐178. Cooper, S., F. Khatib, I. Makedon, H. Lu, J. Barbero, D. Baker, J. Fogarty, Popović and Foldit Players (2011) Analysis of social gameplay macros in the Foldit cookbook. Pp. 9‐14 in Proceedings of the Sixth International Conference on the

350

Rikke Magnussen et al. Foundations of Digital Games (FDG 2011), June 28‐July 1, 2011, Bordeaux, France. New York: ACM [online]. Available: http://grail.cs.washington.edu/projects/ protein‐game/fold it‐fdg11.pdf [Retrieved May 5, 2013]. Cooper, S., F. Khatib, I. Makedon, H. Lu, J. Barbero, D. Baker, J. Fogarty, Z. Popović and Foldit Players (2011). Analysis of social gameplay macros in the Foldit cookbook. In: Proceedings of Foundations of Digital Games, FDG 2011, Monterey, CA: SA. Cooper, S., Treuille, A., Barbero, J., Leaver‐Fay, A., Tuite, K., Khatib, F., Snyder, A. C., Beenen, M., Salesin, D., Baker, D., Popović, Z. and Foldit players (2010). The challenge of designing scientific discovery games. In Proceedings of the Fifth international Conference on the Foundations of Digital Games, FDG 2010. Dillenbourg, P. (1999). What do you mean by collaborative learning? in: P. Dillenbourg (Ed) Collaborative‐learning: Cognitive and Computational Approaches, p. 1‐19 Oxford: Elsevier. Egenfeldt‐Nielsen, S. (2005). Beyond Edutainment: Exploring the Educational Potential of Computer Games. IT University of Copenhagen. Ellington, H., F. Addinall & F. Percival (1981). Games and Simulations in Science Education. London: Kogan Page Ltd. Grover, L.K. (1996). A fast quantum mechanical algorithm for database search, Proc. 28th Annual Symposium on the Theory of Computing, NY, NY: ACM Press, 212‐218. Gee, J. P. (2003). What Video Games Have to Teach Us About Learning and Literacy. New York: Palgrave Macmillan. Good, B. M. & Su, A. I. (2011) Games with a scientific purpose. Genome Biol, 12, 135 Magnussen, R. (2007). Games as a platform for situated science practice. In: de Castell, S., & Jenson, J. (Eds.), Worlds in Play: International Perspectives on Digital Games Research (301–311). NY, NY: Peter Lang. Magnussen, R. (2008). Representational inquiry in science learning games. Doctoral dissertation, Copenhagen: Aarhus University. Magnussen, R., Hansen, S.D., Grønbæk, K., Mølmer, K., Sherson J.F. (2012). Game‐based research collaboration adapted to science education. Martin, C., Ochsner, A. & Squire, K. (ed.) In: Proceedings GLS 8.0 Games + Learning + Society Conference, Madison, Wisconsin. (431‐436). Sawyer, R.K. (2006). Educating for innovation. Thinking Skills and Creativity, 1(1), 41‐48. Shaffer, D.W., & Gee, J.P. (2005). Before every child is left behind: How epistemic games can solve the coming crisis in education (Tech. Rep. No. 2005‐7). Madison: University of Wisconsin‐Madison Center for Education Research. nd Shor, P.W. (1994). Algorithms for quantum computation: Discrete logarithms and factoring, Proc. 35 Annual Symposium on Foundations of Computer Science (Shafi Goldwasser, ed.), IEEE Computer Society Press, 124‐134. Squire, K., & Klopfer, E. (2007). Augmented reality simulations on handheld computers. Journal of the Learning Sciences, 16(3), 371‐413. Weitenberg, C., S. Kuhr, K. Mølmer, J. F. Sherson (2011) Quantum computation architecture using optical tweezers. Phys. Rev. A 84, 032322.

351

Creating Games in the Classroom – From Native Gamers to Reflective Designers Gunver Majgaard The Maersk Mc‐Kinney Moller Institute, University of Southern Denmark, Odense, Denmark gum@mmmi.sdu.dk Abstract: A group of first‐semester engineering students participated in a game design course. The overall goal was to learn about game design and programming while they were creating their own games. Additionally we wanted the students to transform some of their game experiences into active knowledge on designing games. It was the intension to give the students a more critical reflective view on video games and game design. The students in this study had all played various video games since they were 5‐6 years old, and were therefore regarded as native consumers in the game world. They grew up playing video and computer games as a natural part of their everyday lives. Some of them had played intensely, while others had played more sporadically. In order to make the transformation they developed their own digital prototypes. And they participated in reflective discussions on what games are: what makes them interesting and how they are constructed. The students used the tool GameMaker, which can be used without having any prior knowledge of programming. The tool gave an easy access to develop running game prototypes in 2D. The didactic approach was based on constructionistic and reflective learning philosophies. The constructionistic learning promotes a creative and innovative learning. But it doesn’t promote articulating and analysing competences. Besides the constructionistic learning process we wanted to promote our students analytic competences. We wanted the students to reflect on games in order to promote explicit knowledge. We believe the dialog based on the academic theory and their programming experiences reinforced the learning process. The constructionistic approach supported exploring and optimising ways of learning. The students used experimentation and exploration as part of the design process. As part of the exploration process they also optimised and balanced e.g. the gameplay. The constructionistic approach also supported creativity and innovative designs. The students turned their own ideas into interactive games. They used innovative design methods and used their creativity. They also developed an understanding of innovative design methods. Additionally this approach stimulated the double perspective ‐ playing and learning at the same time. The students played games while they were developing games. The reflection on games supported insights into others' gaming experiences. In the user test the students got other e.g. children’s perspective on the games they developed. The reflective approach also created thoughts on tomorrow’s teaching methods. The students evolved their own thoughts on how to use games in teaching and learning processes. In summary, we discussed the students' first voyages from natives in the game world to reflective designers. During the journey, they developed a reflective practice and an understanding of the profession they were entering. The article also shows a very dynamic and fruitful relationship between playing games and designing games. Keywords: learning, game‐based learning, game design, serious games, university pedagogy

1. Introduction A group of first‐semester students participated in a course where they designed and programmed games. The aim was to learn about game design and programming while they were creating their own games. They used the programming tool GameMaker, which can be used without any prior knowledge of programming (Habgood, 2006 and 2007). GameMaker is ideal for developing game prototypes in 2D. The idea was to provide a “low floor” (easy to get started) and a “high ceiling” (opportunities to create increasingly complex projects over time) (Resnick, 2009). Basically, we wanted the students to experiment, learn from errors and make interesting games. Papert (1993) used programming as a tool for teaching children mathematics. The children were constructing programs and new knowledge of mathematics while they were interacting with the programming tool. He thought of the programming tool as “an object to think with” and we wanted the same type of learning process in this course. Additionally, we wanted the students to transform some of their game playing experiences into active knowledge on game design. It was intended to give students a more critical and reflective approach to computer games and game design. The students in this study have all played various computer and console games since they were 5‐6 years old and were, therefore, regarded as native consumers in the game world. These young people grew up playing

352

Gunver Majgaard video games as a natural part of their everyday lives, where some have played intensely, while others had played more sporadicly. In order to make the transformation they had to distance themselves from the consumer’s role. They had to be able to reflect on what a game is, what makes them interesting, how they are constructed, and then develop interactive prototypes. The students were from the engineering programme “Learning and Experience Technology”. In addition to this course they also followed lectures on game design and theory of games where they for example read Fullerton (2007), Salen (2004), Csikszentmihalyi (2005), Juul (2005) and Sicart (2008). The research question in this article is: How to organise the didactics and how to benefit from the learners’ background as native consumers in the game world? The research method is inspired by Design‐based research. The intent is to produce new theories and practices based on digitally supported learning and teaching in naturalistic settings (Majgaard, 2011a; van den Akker, 2006; Barab, 2004). The method is interventionist: it involves some sort of design, it takes place in naturalistic contexts, and is iterative. In this study we designed a new practice for creating games in the classroom. The study was based on interventions in the classroom, teaching materials and student products. Finally, we did a qualitative email interview with 6 students (Kvale, 1997). These interviews reflected the students’ views on gaming and what they learned by playing and designing games. They were questioned about specific gaming experiences and how they affected the design. Organisation of the paper: First is a brief summary of the course and reflections on the didactics. We then present the underlying theory, which focuses on the dynamics between reflection and active participation. The student's active participation promotes reflection and acquisition of new knowledge. This underlying theory is especially based on Schön’s ideas on the practitioner's active participation and his reflection on practice (Schön, 2001; Argyris, 1978). In addition, focus is on the relationship between tacit and explicit knowledge (Scharmer, 2000 and 2007). The dynamics between tacit and explicit knowledge are brought into play in innovative design processes. This is followed by a discussion of the learning potentials based on the study. Finally, there is a summery and conclusion.

2. Background: Description of the course The course lasted for 15 weeks and the students had two one‐hour lectures every week. The course combined theory and practice. The primary aim of the course was to teach the students basic game programming. The course was intended for first‐semester students without programming skills. Furthermore, they should obtain knowledge of what makes games interesting and of game design methodology. And they would gain practical experience in iterative development processes. The programming platform was GameMaker. In the beginning the students imitated and copied already programmed games in order to become familiar with game programming concepts like sprites, objects, structures, events, actions, rooms, sound and motion (Habgood, 2006 and 2007). The students played GameMaker games, and afterwards they developed their own versions of the games. At first, their own versions were copies of existing example games but gradually they developed creative variants. After the first four weeks, the students got their first larger project task and it was to develop their own game idea. The requirements for the game were: at least one level; at least two objects moving; sound; collisions handling; title page and dialog box or a high‐score list. They had three weeks to develop the first version and every week we had a brief follow‐up in the classroom. After three weeks, the students presented their games in class. This led to a discussion of the strengths and weaknesses of the games. And it led to the next project task: the students were to formulate a prioritised list of requirements for the next version of the game prototypes. The aim was to make the students work iteratively, make goals and assess what they could reach within a deadline. Most of the students carried out three iterations.

353

Gunver Majgaard

3. Theory on learning while creating games: Constructionism and the reflection The learning philosophy was based on constructionism where the learner constructs knowledge while creating constructions in the real world (Papert, 1993). Constructionism was inspired by Piaget’s constructivism. Paperts learning tools supported both physical and virtual constructions. Papert focused on experimental and creative ways of learning mathematics he didn’t focus on game design. The idea of bringing games into the classroom are often based on the motivating nature of games, we hope to make the learning of academic matters more fun, if not easier (Kafai, 2006). According to Kafai few people have sought to turn the tables: making games for learning instead of playing games for learning. Kafai suggests rather than embedding “lessons” directly in games, the goal is to provide students with greater opportunities to construct their own games. And she suggests that constructionistic game design hold equal if not more potential for engaging learners’ enthusiasm. Resnick (2009) unfolded the idea of designing games and simulations and he suggested programming as a fundamental skill that everybody should be introduced to. He proposed to use the brick programming tool Scratch (Scratch, 2013). At our institute we have used both Scratch and App Inventor (App Inventor, 2013) for students without programming skills. Physical education students created games in Scratch as an interface for an interactive shoe sole. The App Inventor was used as a prototyping tool in a HCI course (Nielsen & Majgaard, 2013). This constructionistic learning promotes a creative and innovative learning. But it doesn’t promote articulating and analysing competences. Besides the constructionistic learning process we wanted to promote our students analytic competences. We wanted the students to reflect on and articulate their design process. We believed the dialog based on the academic theory and their programming experiences reinforced the learning process. This is theoretically supported by Schön (2001) and Bateson 2000). Knowledge evolves through active participation and reflection (Schön, 2001; Majgaard, 2011 and 2009). Active participation and the voyage towards a professional game‐designer perspective are the key terms in creating games in the classroom. The knowledge achieved by the students was expressed in actual designs of prototypes and reflections on these. Part of the knowledge expressed in action and in the design of prototypes will often be difficult to put into words and can be described as tacit knowledge (Schön, 2001; Agyris, 1978). The educational goals were to develop a new practice for the design of games. Knowledge‐in‐action is inherent in this practice and is difficult to make explicit in an adequate manner. It is for example difficult to explain how to use a hammer, and how to recognise a face in a large crowd. It is actions that we spontaneously know how to perform in the actual situation. The concept of knowledge‐in‐action alone is not sufficient in a learning process or in a field practice. This must be supported by the more retrospective forms of reflection ‐ reflection‐on‐games. Reflection‐on‐games helps the students to articulate conceptual knowledge on game programming and game design. In the retrospective reflection process their own experiences are connected to emerging conceptual knowledge. And conceptual knowledge is used in the professional communication amongst peers. The professional reflection divided into two parts: 1. Reflection‐in‐games, where multiple knowledge, experience and intuition merge during actions. Reflection‐ in‐games occurs in the context of game design, when a student solves programming problems here and now in the game, e.g. a programmed character disappears off the screen instead of being stopped by the game's virtual boundaries. This type of reflection is a here‐and‐now reflection, how to solve the here‐and‐now problems. This type of reflection might also occur during gameplay. 2. Reflection‐on‐games, is the subsequent reflection and evaluation on the process that has happened, and its potential consequences. It is precisely this type of reflection you want in the classroom as the evaluation of assignments and projects. For example, when the students analyse and present their prototypes. We want them to reflect upon what has happened in the design process, and how their experiences could be used in future designs. This type of reflection provides an overview of the design process. Furthermore, it offers an understanding of the design process and a holistic perspective. This type of reflection can be expressed in words and can be described as conceptual knowledge.

354

Gunver Majgaard In the classroom we want both reflection‐in‐games and reflection‐on‐games. It is in the interplay between these forms of reflections that the skilled game designer unfolds his potential. It is in this interplay that innovative processes evolve. It is also in this interplay that the students achieve a good learning depth. In addition, the students reflect on their own learning strategy and they adjust to a given situation and context. It is a concept developed by Bateson (2000), and adapted into today's educational context (Gleerup, 2005). This type of reflection is deliberately used in teaching when students are asked to articulate what they have learned by designing a game and how to improve the learning strategy next time. One would immediately think that reflection and playing games were opposites. When you play a game you are present and do not think about strategies for learning. However, there may easily in the play situation be reflection‐in‐action (Schön, 1983). For example, if you play a strategy game about World War II and continually are considering what alternatives provide for the best game performance. In a teaching situation we can analyse and evaluate a specific game, e.g. game mechanics, fun factors, the pros and cons of strategic choices, and ethical aspects. The process then changes from reflection‐in‐games to reflection‐on‐games. In this case the students were transformed from game consumers into reflective learners and future game designers. During the course the students analysed a game from their childhood. A group of students chose Super Mario, and they used the tools from game design theory to analyse rules, gameplay, and dramaturgy (Fullerton, 2004). The learning goal was to transform the students’ own user perspectives on the games into professional and reflective perspectives. The assignment was rooted in something they already knew, and they evaluated it from a new angle with professional tools. This provides for meaningful learning processes. And it exemplifies how learning processes can link a playful context and a professional game‐design context. And it is an example of how games can be a lever for the learning process. It is also, in this interplay, important to bring the students' game experiences into play in terms of game design. What makes a game interesting in the student's eyes, how can it be translated into new games and how does it fit with the theory?

4. Learning potentials based on the didactical approach Our didactical approach was to use constructivism and reflection as a lever for the students learning processes. Their views on professional games changed during the semester and they got a new kind of respect for the bigger game productions. The respect was based on their new knowledge on designing games. They were also able to point to game elements where the games were lagging and not working. This expresses an ability to evaluate and analyse, which requires distance and perspective. The students' reflections interchanged between reflection‐in‐action and reflection‐on‐action as part of the game design process. Reflection‐in‐action when they were playing and developing games. They reflected‐on‐action when they evaluated and analysed the design process. The students had different roles in the design process, e.g. as game testers, developers, and learners. Reflection‐in‐action and reflection‐on‐action are prerequisites for learning by creating games in the classroom. The interplay between construction and reflection promoted several learning perspectives. Below we highlight learning perspectives. First and foremost, the students changed from having a consuming approach towards games into having a participatory approach. They became creative, reflective and innovative contributors.

4.1 From digital native to becoming digital contributors and citizens The students were between 4 and 10 years old when they played their first video games and the majority were between 5 and 6 years old. It was games such as Pinball, Super Mario, Pixiline, Magnus, and the Gnat and Mummy Trolls. At present half of the students highlight the bigger games such as World of Warcraft, GTA (Grand Theft Auto) or FIFA (European Football video game). The other half highlighted casual games such as Plant versus Zombies or Tetris. Virtually everyone mentioned that they at some point had played Counter Strike and Super Mario. Super Mario was even the inspiration for some of the game prototypes. Today’s students are often regarded as digital citizens or digital natives since computers have been a part of their lives almost since the beginning. Resnick (2009) argues that everybody should be able to make their own interactive games, stories, animations, and simulations in order to fully participate and understand the digital community:

355

Gunver Majgaard “..everyone has an opportunity to become a fully fluent contributor to today’s digital society.” (Resnick 2009). He argues that everybody should be able to program at least a little. If you can make your own simple programs you get a better understanding of the digital world and it will be easier for you to influence the digital world. This means that the students are not fully digital citizens before they have started to make interactive digital contributions ‐ in this case digital games. The students might be digital natives and digital consumers since they had been using computers for playing activities most of their lives. But becoming a digital citizen requires a deeper understanding of the digital world and it requires that the students become digital contributors, see figure 1.

Consumer: Native gamer

Contributor: Game designer

Figure 1: Digital citizenship The figure visualises the students’ voyage from digital native game consumers to becoming contributing game designers. The voyage gave them a new and more reflected view on playing games. When they play games in the future for pleasure or part of their study they have new and deeper insight on games. Our two didactical tools constructionism and reflection created a distance from the consumer’s role that allowed the students to become creative. And they transformed their game knowledge into new games.

4.2 The constructionistic perspective and reflection‐in‐games The constructionistic approach supported reflection‐in‐games: Exploring and optimising ways of learning; creativity and innovative designs; and the double perspective ‐ playing and learning at the same. Exploring and optimising ways of learning. As part of the course the students played specific games developed in GameMaker. The students learned about games by exploring games. They learned about specific game mechanics by watching the code and experimenting with new code structures. The students played the games while they were imitating and developing new versions of specific game functionality. This was done especially by trial and error in a kind of "trial and error" learning. They used e.g. some of the predefined actions, and afterwards they evaluated the consequences by playing the game. Subsequently they could balance an action and thereby optimize the game strategies. This can be compared with Bateson's learning 1 and 2, which are fundamental learning processes (Bateson, 2000). This also exemplifies how students learn while they are interacting and they are, in fact, using GameMaker as an object to think with. Papert (1993) discovered back in the last century that programming tools were ideal as constructionistic learning tools because the learners got an interactive object to think with. This also means that the students can learn more by themselves without teacher support. Creativity and innovative designs: Where does the inspiration come from? ‐ Pre‐sensing, presence, and technological fascination. The students were inspired by their immediate situative environment, which required open‐minded openness and presence. This is comparable to Scharmer‘s (2000 & 2007) pre‐sensing, where you are present in the moment and not to be dictated by habits and conventions. In addition, they were inspired by the technological possibilities such as specific game mechanics, another game, something they read in the newspaper, learned from a movie or an experience from their school road. In the summer 2011 we had a cucumber crisis and a lot of people got ill by eating cucumbers and a student used this as a game idea. He developed a game about bacteria in cucumbers. Another student was inspired by a movie at that time called Cowboys and Aliens and developed a 2D game where a spaceship sucked up cows. Double perspective ‐ gaming and learning simultaneously: During classes the students played games and developed new games in dynamic alternation. They used the GameMaker tool as an object to think with as

356

Gunver Majgaard described by Papert (1993). Furthermore, they played their own prototypes in order to test them and this took place during the whole development process. In addition, they got an understanding of how the games could be balanced, and thereby made more interesting game plays.

4.3 The retrospective reflective approach: Reflection‐on‐games The reflection on games supported: Insight into others' gaming experiences and students’ thoughts on tomorrow’s teaching methods. Insight into others' gaming experiences: The students got an insight into their own and others' gaming experiences by testing the games on a target group. One of the tests took place in a 6th grade. Some of the students tested for usability problems other tested for engaging user experiences. But what really surprised most of the students was the users’ approach. They had other game strategies and other ideas on what was meaningful in the games. The students really learned how difficult it was to predict user behaviour. They also recognized that a designer does not necessarily get the same gaming experience by playing, because he/she knows his/her own games too well. Thoughts on tomorrow’s teaching methods: Some students saw games as the new learning revolution, which is visionary and ambitious. This shows that these students are developing a vision and mission for the new profession they are entering. They also highlighted some of the challenges related to play versus teaching, e.g. that games are based on voluntary participation and you play as long as it is interesting. While teaching and learning can be described as involuntary and compulsory activities. Games in the classroom bring voluntary and more compulsory activities together. This is a major challenge for using games in education.

5. Summary and conclusion In this article we described these engineering students' first voyage from native consumers in the game world to becoming reflective designers. In the learning process they needed to distance themselves from the consumer’s role in order to process the new knowledge on game design e.g. how to implement interesting game strategies. In addition they read a lot of theory on what games are and what makes them interesting. To transform this knowledge into new games was hard work. In this process they dynamically alternated between construction and reflection. The theory on games also gave them tools to analyse potentials and weaknesses of their own games. Their background as digital natives gave them insights and motivated them to create ideas in the design process. They became digital contributors and citizens of the game designers’ community. The didactics were organised in order to support constructionism, reflection‐in‐action and reflection‐on‐action. We wanted the students to explore and become aware of the iterative design process. Our teaching strategy was for the students to develop their creative and experimenting competences instead of us lecturing on programming commands and theory on methodology etc. This required a programming tool with a “low floor” and a “high ceiling”. It also required structured activities in the classroom. The structuring activities focused on making the iterative design phases visible. The activities also focused on retrospective reflections on how to balance their games, discussions on test results etc. The constructionistic approach supported: Exploring and optimising ways of learning; creativity and innovative designs; and the double perspective ‐ playing and learning at the same. The reflection on action supported: Insight into others' gaming experiences and students’ thoughts on tomorrow’s teaching methods. In summary, we discussed the students' first voyages from natives in the game world to reflective designers. During the journey, they developed a reflective practice and an understanding of the profession they were entering. The article also shows a very dynamic and fruitful relationship between playing games and designing games. Furthermore, they develop the professional's professional humility and an understanding of the mission, their profession is developing. In the following semesters the students have a theme about learning and design of digital systems for use in learning processes. Later on in their study the will explore the serious games from both a learning and gaming perspective

357

Gunver Majgaard

References App Inventor, (2003). http://appinventor.mit.edu/ last retrieved May 2nd, 2013 Argyris, C., Schön A. D., (1978). Organizational learning: A theory of action perspective. Reading, MA: Addison‐Wesley Barab, S., Squire K., (2004). “Design‐Based Research: Putting a Stake in the Ground.” The Journal of the Learning Sciences, 13(1), 1–14 Bateson, G., (2000). Steps to an Ecology of Mind: Collected Essays in Anthropology, Psychiatry, Evolution, and Epistemology. Chicago Press. ISBN 0‐226‐03906‐4 Csikszentmihalyi, M., (2005). Flow – Optimaloplevelsens psykologi. København: Munksgaard. Fullerton, T., (2008). Game Design Workshop. A playcentric approach to creating innovative games. Morgan Kaufmann Gleerup, J., (2005). ”Gyldighed, oprigtighed og ærlighed – om viden og læreprocesser.” Læring – en status. Klim Habgood, J. et al, (2006). The Game Maker 's Apprentice: Game Development for Beginners. Apress. Habgood, J., (2007). The effective integration of digital games and learning content. Thesis submitted to the University of Nottingham for the degree of Doctor of Philosophy July 2007 Kafai, Y. B., (2006). “Playing and Making Games for Learning Instructionist and Constructionist Perspectives for Game Studies”. Games and Culture Volume 1 Number 1. January 2006 36‐40. Sage Publications Kvale, S., (1997). Interview ‐ en introduktion til det kvalitative forskningsinterview. 1. udgave, Hans Reitzel. ISBN‐13 978‐87‐ 412‐2816‐7 Juul, J. (2005). Half‐real: video games between real rules and fictional worlds. Cambridge, MA: MIT Press Majgaard, G., (2011b). ”Læreprocesser og robotsystemer. Design af læreprocesser med robotter som medier og børn som med‐designere”. (Learning Processes and Robotic Systems – Design of Educational Tools and Learning Processes using Robotic Media and using Children As Co‐Designers) PhD‐thesis Majgaard, G., Misfeldt, M., Nielsen, J. (2011a). “How Design‐based Research, Action Research and Interaction Design Contributes to the Development of Designs for Learning.” Designs for Learning Nielsen J. & Majgaard G. (2013) Merging Design and Implementation in a First Semester HCI‐course for Engineering Students. IADIS Interfaces and Human Computer Interaction 2013 Papert, S,. (1993): Mindstorm – Children, Computers, and Powerful Ideas. Basic Books Resnick, M. et al (2009): “Growing up Programming: Democratizing the Creation of Dynamic” Interactive Media Salen, K., Zimmerman E., (2004). Rules of Play: Game Design Fundamentals. MIT Press 2004 Scharmer, C. O., (2000). ”Self‐transcendending knowledge: Sensing and Organizing Around Emerging Opportunities.” in: Journal of Knowledge Management ‐ Special Issue on Tacit Knowledge Exchange and Active Learning. http://www.ottoscharmer.com/docs/articles/2000_STK.pdf (retrieved on 020211) Scharmer, C. O., (2007). “Executive Summary: Theory U: Leading from the Future as it Emerges” (17 pages). http://www.ottoscharmer.com/publications/articles.php (retrieved on 020209) Schön, A. D., (2001). Den reflekterende praktiker. Hvordan professionelle tænker, når de arbejder. Klim Sicart, M., (2008). “Defining Game Mechanics.” The International Journal of Computer Game Research, volume 8 issue 2 December 2008, ISSN:1604‐7982 Scratch, (2003). http://scratch.mit.edu/ last retrieved May 2nd, 2013 van den Akker, J. (2006): Educational Design Research. Routledge

358

A Holistic Framework for the Development of an Educational Game Aiming to Teach Computer Programming Christos Malliarakis, Maya Satratzemi and Stelios Xinogalos University of Macedonia, Thessaloniki, Greece malliarakis@uom.gr maya@uom.gr stelios@uom.gr Abstract: Computer science is gradually changing, evolving and adapting according to the needs of each time period by incorporating the technological developments available. However, despite the occurring changes and the current progress in the domain, computer programming is still a vital chapter within computer science, and its teaching remains a difficult endeavour. On the other hand, students have changed the way with which they learn, interact with and search for knowledge. They spend significant amounts of their everyday lives from a very young age interacting with the computers by playing games. Thus, they are used to environments with impressive special effects and graphical interfaces where they have full control of the situation and interact with the environment’s elements. Therefore, today’s teachers are trying to connect computer programming learning with students’ everyday usage of the computer, which does not include simple textual editors for programming lines of code with no other interaction functionalities. Hence, teachers face the challenge of incorporating environments that are similar to students’ existing mentality and of creating tasks and assignments that can be executed within these environments and can provide students with the necessary programming knowledge and skills. A number of software solutions were developed towards facing the aforementioned difficulties. They can be classified into three main categories, namely educational programming environments, microworlds and educational games. Educational games used in computer programming courses are considered to present added value, due to their ability to motivate students towards actively participating in the learning process and to support high levels of interaction, group work and critical thinking. Thus, we have developed an educational game that aims to further enhance computer programming education by addressing occurring problems. This paper aims to introduce and elaborate on a holistic framework that has been constructed as a guide towards the development of this game. To this end, we collect documented difficulties identified in computer programming learning and teaching and study existing frameworks that have been proposed for the development of software solutions for computer programming courses and for the development of successful serious games that do not however focus on computer programming education. This information is thoroughly studied and refined and results in the proposed framework that could also be employed for the design and development of other future educational games focusing on computer programming education. Keywords: computer programming; educational programming environments; educational games; holistic framework; learning process

1. Introduction The continuous evolution of the computer science domain through the iterated emergence of new knowledge has brought forth the necessity that its education should be configured and adapted according to the global changes. This should be realized especially in the sub‐domains that face significant challenges that hinder their successful education. During the last years there have been many efforts in documenting the difficulties students and teachers face during computer programming learning and teaching correspondingly, so that they can be successfully addressed (Lahtinen et al. 2005; Ragonis & Ben‐Ari, 2005). At the same time, today’s generation of students is characterized as the “Nintendo Generation”, because they spend significant amounts of their everyday lives not only interacting with the computers but also by playing games (Guzdial & Soloway, 2002). These environments could thus aim to help students learn computer programming concepts more easily as well as adopt skills such as logical thinking and make the learning process more interesting. The relevant literature depicts three major categories of environments that have been constructed towards this goal, namely educational programming environments, microworlds (Brusilovsky et al, 1997) and educational games (Gunter et al, 2008). Out of the three, educational games are considered the most appropriate for training new generation students since they provide adequate motivation for learning through attractive graphics and scenarios and they promote interaction and collaboration towards a final goal by allowing students to engage in activities already familiar to them from computer games. We have developed an educational game that aims to address existing problems in computer programming education and to exploit advantages of such an environment. However, the design and development of an

359

Christos Malliarakis, Maya Satratzemi and Stelios Xinogalos educational game requires proper planning. This is facilitated by frameworks that guide designers and developers by demonstrating what concepts should be taken into consideration and/or what features should be supported. This paper aims to present a holistic framework that has been constructed to guide the design of our game. The framework takes into consideration related work and is augmented with concepts that intend to support features that will provide a highly motivational and interesting virtual world for computer programming courses. The second section presents the literature review carried out on frameworks proposed for the design of educational games. The following section discusses features that existing educational games for computer programming focus on and support. The next section elaborates on our framework by presenting the methodology followed, the results, and a brief overview of the concepts depicted. The paper concludes with a summary of the work done.

2. Related work This section presents interesting frameworks proposed for the design of educational games in general. We continue to distinguish features supported by the most commonly known educational games that focus on computer programming education, following a top‐down methodology approach.

2.1 Educational games frameworks 2.1.1 Four – dimensional framework The Four‐dimensional framework has been proposed by de Freitas & Jarvis (2006), as shown in .

Figure 1: Four – dimensional framework (de Freitas & Jarvis, 2006) The work done and portrayed within the framework shows that this model comprises of four basic principles, as follows:

Context. Each game is characterized by a specific context that will guide the scenarios as well as the ways students and teachers will interact with its features. During the context’s establishment, one must define characteristics such as required infrastructure, technical specifications, location of usage, type of game (e.g. role playing, multiplayer etc), activities to be performed etc.

Representation. This concept refers to all representations that are required to be properly portrayed within the game. For example, each player needs to be represented by avatars that will have specific characteristics based on the context of the game. Additionally, the virtual world should represent interesting scenery that will be integrated with all the features of the game in a harmonized and meaningful way.

360

Christos Malliarakis, Maya Satratzemi and Stelios Xinogalos Author name

Learner. This concept relates to all features corresponding to the learner within the game. Some of these include the ages of the students to be taught, their preferences, the availability and level of previous knowledge on the specific learning domain, our learning objectives regarding the learning outcomes etc. Additionally, it is considered important to try and promote learning through groups in educational games.

Pedagogy. The development of educational games should depend on the incorporation of learning strategies (e.g. problem based learning). The pedagogies used should promote learning processes that will be constructivist and cognitive so that they will allow the creation of new knowledge through active and collaborative participation. Additionally, learning processes have to be instructive, situative and associative, that is they are required to progress in a logical manner, be more social and provide assistance when needed.

2.1.2 Conceptual framework The second reviewed framework has been proposed by Yusoff et al. (2009) so as to provide a reference guide for the design of educational games, as shown in Figure 1.

Figure 1: Conceptual framework for educational games (Yusoff et al., 2009) A brief overview of the framework’s features is provided as follows:

Capability. This relates to the skills students should develop through their interaction with the game within the learning process. Such skills include analysis, recall, evaluation, proper attitude and logical thinking.

Instructional content. The game should be compliant with the educational material that students have to learn while interacting with it.

Intended learning outcomes. Learning outcomes represent goals students should be able to achieve once they successfully complete all tasks.

Game attributes. This concept includes all characteristics that aim to increase motivational and participatory learning. Such are scaffolding, interaction, learner control and sequence, incremental learning, rewards and authentic learning.

Learning activity. Each activity focuses on a specific set of tasks that need to be completed. It is important that all learning activities promote motivation so that students will remain interested and immersed in the game’s scenario.

Reflection. Students should be able to reflect on their experience within the game and be provided with an overview of their progress when requested.

Games genre. This concept describes the type of the game. Specific genres are accompanied by different features so it is important to define what type of an educational game this will be (e.g. strategy, role‐ playing etc).

Game mechanics. This concept relates to the technicalities (e.g. management of resources, 0environment layout etc) that should be taken into consideration during the development depending on the game’s genre, learning activities or instructional content.

Game achievement. The final concept refers to all the ways the game can represent a student’s achievement level and is also a significant metric of learning assessment (e.g. final scores, resources or “rewards” etc).

361

Christos Malliarakis, Maya Satratzemi and Stelios Xinogalos 2.1.3 The design, play, and experience framework The “Design, Play and Experience” framework aims to demonstrate and analyze the phases that correspond to each design layer. During the “Design” phase, the designer has to identify the learning objectives that will guide the activities’ design.

Figure 2: The design, play, and experience framework (Salen & Zimmerman, 2004) The “Play” phase represents the interaction of players with the game’s features. Thus, this phase is closely related to all characteristics of each individual player, such as knowledge background, skills etc. To this end, during the “Play” phase designers must take into consideration the target audience that will use the educational game and that will produce the different experiences during the “Experience” phase. According to this framework, four different layers guide the design of an educational game. A brief overview of these layers is provided as follows:

Learning. During the “Design” phase, the educational content that will be taught along with the pedagogical theory that will guide the learning process corresponds to the learning layer. In the “Play” phase, this refers to the actual teaching process, which is when students play the game. The “Experience” phase finally represents the learning that is accomplished through the teaching, and thus documents whether the set learning objectives have been achieved as learning outcomes.

Storytelling. Storytelling provides valuable information during the game’s design that will guide the virtual world development as well as all the scenarios to be supported within the game. The designer sets the stage by designing the different characters, the overall environment setting, the narrative and the layouts that will structure the world. Each player produces his individual storyline during the “Play” phase. Similarly, the final player’s story as it will be created after the execution of all tasks will represent the accomplished learning outcomes in the “Experience” phase.

Gameplay. This layer includes all information regarding the players’ allowed actions within the game. Initially, the designer has to define the specific mechanics of the game, such as the learning objectives, the challenges and allowed actions. Once these are integrated in the game, they are represented by the dynamics during game playing. The experience drawn from the play by each user is called an “affect” and represents all emotions players are left with after they are finished (e.g. satisfaction, disappointment etc).

User experience. The final layer of the reviewed framework includes the most visual part of the game, which has to be as entertaining and accessible as possible in order to increase motivation and participation. The design phase supports the planning of the user interface and aims to provide multiple and easy to use interactivity opportunities during game playing. Finally, this will lead to experiences that engage students in game playing during class.

2.2 Features in games teaching computer programming We continue to study existing games that focus on the computer programming learning domain and distinguish specific features that need to be supported by future educational games.

362

Christos Malliarakis, Maya Satratzemi and Stelios Xinogalos Author name 2.2.1 The SCRATCH environment The MIT Media Lab research group developed Scratch aiming to create an environment that would assist creative people to easily accomplish their goals. This aim is also depicted in Figure 3.

Figure 3: Ultimate aim of the MIT media lab (Resnick, 2007) The Scratch environment was based on the above design principles and introduces computer programming to students of ages 10 to 18 (Resnick, 2007). Towards this goal, students can create programs, interactive stories and games through drag & drop tiles instead of writing lines of code, thus avoiding making syntax errors. Interaction and scenarios play an important role in this game as they intend to stimulate the interest of young students by engaging them in a series of attractive assignments. The availability of multiple characters (sprites), backgrounds, sounds and images motivate students to create their own animations and games, play on their own and share their creations with their classmates, as well as reflect their creations to others. 2.2.2 Educational games for computer programming This subsection presents features supported by educational games constructed for teaching computer programming concepts. The work done is a combination of previous research (Malliarakis et al. 2012) and a more thorough overview of the educational games Catacombs (Barnes et al., 2008), Saving Sera (Barnes et al., 2008), EleMental (Chaffin et al., 2009), Prog & Play (Muratet et al., 2010), Robozzle (Li & Watson, 2011), Lightbot (Piteira & Haddad, 2011), Robocode (O'Kelly & Gibson, 2006), TALENT (Maragos & Grigoriadou, 2011), M.U.P.P.E.T.S. (Phelps et al., 2003), Wu’s Castle (Eagle & Barnes, 2009), Playlogo 3D (Paliokas et al, 2011) and Gidget (Lee & Ko, 2011). These educational games follow a "play‐learn‐improve‐win" pattern during learning, a pedagogical aim that is similar to the one employed in Scratch and depicted in Figure 4. According to the review carried out, the main features supported can be considered as requirements specification for future educational games’ design and development. A brief overview of the most commonly identified features is provided as follows:

Multiplayer / Role – playing

Interaction / Experimentation

Collaboration

Scaffolding

Drag & drop lines of code

Programming editor for writing lines of code

Multiple choice questions

Scenarios

Compiler that allows interaction with errors

Physical/ familiar metaphor

Visualization of concepts

Simplicity

363

Christos Malliarakis, Maya Satratzemi and Stelios Xinogalos Each game supports a different set of the features included in the list as well as of the features shown in the examined frameworks during their design. Thus, we have worked on developing a game that will include as many of these features and concepts as possible, while maintaining limited complexity of the environment.

3. Our framework This section provides information regarding the design process of our CMX educational game that focuses on the education of the computer programming domain. The CMX educational game is a Massive Multiplayer Online Role‐Playing Game (MMORPG) and aims to familiarize secondary school students that are novices to computer programming with concepts such as variables, if‐statements, loops etc, and engage them in algorithmic logic. It is different than existing systems, since the game supports different functionalities for the student (player), the teacher (tutor), the administrators and other educational agents, according to each user type’s role in the game. It is essential to initiate the design of the game by studying its characteristics and defining generic metrics that will need to be instantiated based on the different user type. Therefore, we initially define a user‐centric model for the design of educational games, as shown in Figure 4. Based on this model, the design phase takes into consideration each user type’s initial aspirations that motivate them to use the game, the specific targets they set to achieve through the game, the metrics that will determine the level in which the targets are achieved as well as the feedback provided to them by the system depending on the corresponding target. These inter‐connected concepts are different for each user type and therefore relate to different aspects of the game. For example, the game administrator’s aspirations can be the game’s and server’s problem‐free operation and the corresponding targets can be the backup of the data produced, the frequent monitoring for possible malfunctions etc. Performance metrics could include the server’s database capacity, the cache memory capacity etc, while feedback that will indicate whether the server is working properly can comprise of error messages produced by the game (e.g. in case new data cannot be saved). Similarly, a teacher’s aspirations can be the successful teaching of units on computer programming while the target can look like “at least three of the four game levels on loops will be achieved by all”. Thus, the set metrics can be each level’s and student’s score, log files from the game etc, while the feedback can be a report with the student’s progress at a given time/level. Furthermore, a student’s aspiration can be to enjoy his time within the game, to distinguish himself through accomplishing the assigned tasks and to learn. Based on these aspirations they can set their individual or group targets across the overall learning objectives. Their performance metrics can be the score/resources they have gathered, how many times they have had to retry before writing lines of code correctly etc. Finally, feedback can be explanatory messages from the agents, introductory narrative or video that explains the scenario, errors in the program they’re writing etc. This closely interconnected puzzle is shown in Figure 4 and provides a more abstract overview of the initial metrics taken into consideration during design phase, following a user‐centric approach. This model takes the form of a puzzle since all user types need to be addressed equally and represent a different point of view of the game. The next step that will lead to the proper construction of our framework is the identification of the methodology our game will be based on. More specifically, as shown in Figure 5, students will initially engage in actions, which will in turn generate their desire for improvement through attractive scenarios. These actions will help students gain new knowledge regarding the targeted educational content and through collaborative activities, students will be able to produce the set learning outcomes. In the end, the learning objectives will be achieved after the game experience is properly evaluated. Finally, both aforementioned steps are reinforced from the review presented in the related work section. More specifically, based on the review of the existing frameworks that guide the design of educational games in general as well as of the features supported by educational games focused on computer programming education, we move on to propose an extended framework that includes the most commonly identified concepts as well as a series of sub‐concepts that provide a more in‐depth visualization of the steps to be

364

Christos Malliarakis, Maya Satratzemi and Stelios Xinogalos Author name followed during the design and development process. Based on this framework, which is shown in Figure 6, we have designed and developed an MMORPG educational game that aims to teach computer programming.

Figure 4: CMX design strategy puzzle

Figure 5: Methodology steps of the CMX educational game

Figure 6: CMX design framework The CMX Design framework includes concepts that need to be represented within any educational game that aims to teach computer programming. It is abstract enough to be employed by future designers and developers and detailed enough to act as a solid guide without allowing many arbitraries. The most prominent concepts that define the game’s design are:

Infrastructure. The design initiates with the establishment of the infrastructure architecture, the technical requirements specification as well as the user interface and concepts visualization design. The infrastructure will have to support simplicity and ease of use.

Learning objectives. Designers will have to initially define learning objectives that the game will be required to successfully support. These objectives can include generic goals (e.g. more than 80% of the students will complete all game’s activities within the given deadline) or programming‐specific goals (e.g. the 90% of the students will drag & drop correctly the lines of code regarding if statements). Additionally, students can also define their own goals depending on what they desire to achieve through their interaction with the game (e.g. “I have understood if statements so I want to complete all tasks related to loops”).

365

Christos Malliarakis, Maya Satratzemi and Stelios Xinogalos

Pedagogy. The game’s layout will strongly depend on the learning strategy to be employed during class as well as on the way the course will be organized (i.e. number of units of learning, educational material to be taught etc). For example, it is possible that the teacher will want to create a level per unit of learning, or merge specific units of learning into one game that will correspond to a certain set of learning objectives. Thus, it is essential to determine these features at an early stage.

Learning outcomes. The determination of the learning outcomes is strongly inter‐connected with the set learning objectives. Target outcomes can include the comprehension of the taught material, i.e. computer‐programming concepts, such as variables declaration, if statements, loops etc, skills development, such as critical thinking, teamwork, leadership etc as well as interaction capabilities and engagement with innovative technologies and their features.

User. A user in an advanced educational game can represent different types (e.g. student, teacher, administrator, agent), where each type signifies another aspect of the game’s functionalities, as already explained in the CMX design strategy puzzle. Additionally, the specific characteristics of each user need to be determined (e.g. age, prior knowledge, preferences etc) as well the aspirations that will drive the user’s interaction with the system (e.g. win the game‐student, ensure problem free operation‐administrator, instruct and assist students‐teacher, guide and scaffold players‐agent). Finally, each user will need to be able to reflect on the game experience for feedback gathering and future refinement of the game’s functionalities.

Scenario. The game’s scenario should be thoroughly researched and planned out in order to produce an attractive and immersive virtual world (e.g. fighting arena, castle, forest etc) with interesting characters (e.g. wizards, robots, mentors, snowmen, prisoners etc) that are required to complete interim and final goals (e.g. save the princess, put the map’s pieces in their correct order, navigate through the tree etc). Designers have to also define what types of awards players will be granted with during the game, in order to increase their motivation to continue learning.

Activities. The design and development of individual activities is essential and will result to the interested and active participation of students. It is important that students will be able to interact with the world’s elements and collaborate with others towards the achievement of all or some of the goals. Additionally, the environment should provide scaffolding mechanisms throughout all activities that will assist students during challenging tasks. Finally, a number of different ways in which students can contribute their knowledge should be included, since not all students learn better using the same techniques. Thus, an educational game should incorporate a programming editor, along with the ability for students to drag & drop lines of code as well as answer multiple choice questions.

The following table lists all features included in our proposed framework and depicts which features are supported by the corresponding layers of each of the three frameworks studied during this research. Table 1: Comparison of features supported by frameworks for educational games’ design CMX design framework Learning objectives Generic goals Goals set by students Domain‐specific goals Pedagogy Learning strategy Course organization Learning content Units of learning Scenario Characters Awards Interim/final goals Virtual world Learning outcomes

Four – dimensional framework X (Learner) X (Learner) X (Pedagogy) X (Pedagogy) X (Representation) X (Representation) X (Context) X (Learner)

Skills Knowledge

Conceptual framework X (Instructional content) X (Game attributes) X (Game attributes) X (Game attributes) X (Intended learning outcomes) X (Capability)

366

The Design, Play, and Experience Framework X (Learning) X (Learning) X (Learning) X (Storytelling) X (Storytelling) X (Storytelling) X (Storytelling) X (Learning) X (Learning)

Christos Malliarakis, Maya Satratzemi and Stelios Xinogalos Author name CMX design framework Engagement Activities Scaffolding Interactivity Drag & drop Programming editor Collaboration Multiple choice questions Infrastructure User interface Compiler Game achievement

Four – dimensional framework X (Context) X (Representation) X (Learner)

Conceptual framework X (Game attributes) X (Learning activity) X (Game attributes) X (Reflection)

The Design, Play, and Experience Framework X (User experience) X (Gameplay) X (User experience)

X (Context) X (Context)

X (Game genre) X (Game mechanics) X (Game achievement)

X (User experience) X(User experience) X (Gameplay)

The concepts that were similar and included in the frameworks, even if they were not clearly represented by their models, were merged and included in our framework (e.g. the concepts within the Representation, Game attributes and Storytelling that referred to the game’s scenario are shown with the “Scenario” concept in the CMX framework). Overall, our proposal represents an augmented and computer programming‐specific design framework for educational games. This framework was employed during the design and consequently development of the MMORPG CMX game and was a valuable support, especially during the requirements and architecture specification processes.

4. Conclusions Educational games are gradually being integrated into the learning process, since they are considered to possess a variety of advantages, such as increased motivation for students due to the attractive virtual world and graphics as well as the interesting scenarios that most students are already familiar with from the commercial computer games. Additionally, the required active participation in the game develops highly well‐ educated students, since they have to comprehend the educational content in order to successfully pass the assigned activities. However, these advantages require a properly designed and developed educational game in order to be fully exploited. To this end, we followed a top‐down approach in constructing our own framework that guided the design and development of an educational MMORPG game focused on computer programming education. Initially, we studied frameworks that have been constructed and proposed by research studies towards the design of educational games. This review allowed for the identification of concepts more commonly incorporated in a domain‐independent framework for an educational game. As a second step, we reviewed educational games already developed for teaching computer programming concepts. Since these games do not follow a specified framework, we distinguished a refined list of features that should be supported by similar games. The combination of these two tasks resulted in the construction of the CMX Design framework, which was employed in the MMORPG CMX game design and development. This is a conceptual model that comprises of all characteristics to be taken into consideration during the design and development of an educational game for teaching computer programming. The CMX design framework can be used as a reference framework for the design and development of future educational games for computer programming, since its concepts can be successfully instantiated by any interested designer and developer.

References Barnes, T., Chaffin, A., Powell, E., Lipford, H. (2008). “Game2Learn: Improving the motivation of CS1 students”, Proceedings of the 3rd international conference on Game development in computer science education, p.1‐5, February 27‐March 03, 2008, Miami, Florida Brusilovsky, P., Calabrese, E., Hvorecky, J., Kouchnirenko, A., and Miller, P. (1997). “Mini‐languages: a way to learn programming principles.” International Journal of Education and Information Technologies, 2, 65–83. Chaffin, A. Doran, K., Hicks, D. and Barnes, T. (2009). “Experimental evaluation of teaching recursion in a video game”, In Proc. ACM SIGGRAPH Symposium on Video Games, Sandbox '09, p. 79‐86

367

Christos Malliarakis, Maya Satratzemi and Stelios Xinogalos de Freitas, S. and Jarvis, S. (2006). A Framework for Developing Serious Games to meet Learner Needs. Interservice/Industry Training, Simulation and Education Conference, 2006. Orlando, Florida. Eagle, M., and Barnes, Τ. (2009). “Experimental evaluation of an educational game for improved learning in introductory computing”, SIGCSE Bull. 41, 1 (March 2009), p. 321‐325. Gunter, G. A., Kenny, R. F., and Vick, E. H. (2008). Taking educational games seriously: using the RETAIN model to design endogenous fantasy into standalone educational games. Educational Technology Research and Development, 56(5/6), 511‐537. Guzdial, M. and Soloway, E. (2002). Log on Education: Teaching the Nintendo Generation how to Program, Communications of the ACM, 45(4). Lahtinen, E., Ala‐Mutka, K. and Jarvinen, H. (2005). A Study of Difficulties of Novice Programmers. In: Innovation and Technology, Computer Science Education, p. 14–18. Lee, M.J. and Ko, A.J. (2011). Personifying Programming Tool Feedback Improves Novice Programmers’ Learning, Conference on International Computing Education Research (ICER), August 8‐9, Providence, Rhode Island, USA, p. 109‐116. Li, F.W.B. and Watson, C. (2011). Game‐based concept visualization for learning programming, Proceedings of the third international ACM workshop on Multimedia technologies for distance learning, p. 37‐42, December 01‐01, 2011, Scottsdale, Arizona, USA. Malliarakis, C., Satratzemi, M. and Xinogalos, S. (2012). Towards the Constructive Incorporation of Serious Games Within Object Oriented Programming, Proceedings of the European Conference on Games Based Learning, p. 301. Maragos, K. and Grigoriadou, M. (2011). “Exploiting TALENT as a Tool for Teaching and Learning”, The International Journal of Learning, Volume 18, Issue 1, pp.431‐440. Muratet, M., Torguet, P., Viallet, F. and Jessel, J.P. (2010). "Experimental feedback on Prog&Play, a serious game for programming practice", Eurographics, p. 1‐8. O'Kelly, J. and Gibson, P. (2006). RoboCode & problem‐based learning: A non‐prescriptive approach to teaching programming. ACM SIGCSE Bulletin, Proceedings of the 11th Annual SIGCSE Conference Bonakdarian, 38, 3 (June 2006), p. 217‐221. Piteira, M. and Haddad, S. (2011). Innovate in Your Program Computer Class: An approach based on a serious game. OSDOC ‐ Open Source and Design of Communication Workshop. Paliokas, I., Arapidis, C. and Mpimpitsos, M. (2011). PlayLOGO 3D: A 3D interactive video game for early programming education. Third International Conference on Games and Virtual Worlds for Serious Applications, p. 24‐31. Phelps, A, Bierre, K, and Parks, D. (2003). MUPPETS: multi‐user programming pedagogy for enhancing traditional study, Proceeding of the 4th conference on Information technology education , October 2003, Lafayette, Indiana, USA, p. 100‐105. Ragonis, N. and Ben‐Ari, M. (2005). A long‐Term Investigation of the Comprehension by Novices, Computer Science Education, Vol. 15, No. 3, 203‐221. Resnick, M. (2007). All I Really Need to Know (About Creative Thinking) I Learned (By Studying How Children Learn) in Kindergarten. Proceedings of the SIGCHI Conference on Creativity and Cognition. Salen, K. and Zimmerman, E. (2004). Rules of Play: Game Design Fundamentals. Cambridge, Massachusetts: The MIT Press. Yusoff, A.; Crowder, R.; Gilbert, L.; Wills, G. (2009), "A Conceptual Framework for Serious Games," in The 9th IEEE International Conference on Advanced Learning Technologies (ICALT 2009) vol., no., pp.21‐23, 15‐17 July 2009 doi: 10.1109/ICALT.2009.19.

368

Examining Early Childhood Education Students’ Attitudes Toward Educational Computer Games in Kindergarten Dionissios Manessis National and Kapodistrian University of Athens, Athens, Greece manesis_d@yahoo.com Abstract: Pre‐service early childhood educators are in a position where they will be expected to help and support infant pupils to use computer and computer games. Unless teachers believe that the role of computer games with educational features is essential neither to their own nor to their students’ needs, they will be unable to introduce Games Based Learning (GBL) methods into their teaching. Therefore, it is important to gather information about which factors may influence Early Childhood Education (ECE) students’ attitudes toward using digital games in nursery school. The purpose of this study was to investigate ECE students’ attitudes toward educational computer games in Kindergarten. The data were collected from 200 freshmen and senior students attending a Bachelor in Education degree at the department of ECE, University of Athens, Greece. Questionnaires were given to the participants at the end of a 13‐week Information and Communication Technologies (ICT) course. The results of the study revealed that the majority of the ECE students had very positive attitudes toward using educational digital games in their future teaching and expressed great willingness to use GBL to benefit children in learning environments. They also appeared to have high levels of self‐efficacy in the ability of using computer games, which is linked to their behavioral intensions about integrating such innovative instructional methods into a kindergarten classroom. There are considerable parameters which affect pre‐service Early Childhood teachers’ attitudes towards digital games: year of study, frequency of computer usage per day, previous experience in playing computer games, experience in a pre‐school classroom, previous computer use in any environment and previous courses about the use/integration of ICT in early childhood classroom. The findings of the research suggested that attitudes were significantly affected by all the above variables. Given that computer games, when appropriately designed, can enhance young children’s learning and cognitive development and at the same time ECE teachers’ role is crucial, more research should be conducted in order to predict ECE students’ preparedness to successfully implement games based learning methods in their classroom, as future teachers. Keywords: early childhood education students, attitudes toward educational computer games, games based learning, kindergarten classroom, self‐efficacy in the ability of using computer games.

1. Introduction Early childhood educators know how important play is in children’s lives. Play is not only an enjoyable and spontaneous activity of young children but it also contributes significantly to children’s intellectual and psychological development (Verenikina, Harris & Lysaght, 2003). In addition, young children can learn and explore the world they live in through playing activities. Nowadays much traditional childhood play is being replaced by computer play. Although in previous decades there has been much controversy about whether computer games should be placed in the ECE settings (Armstrong & Casement, 2000; Cordes & Miller, 2000), digital games have been placed among modern societies of ICT and constitute an integral part of ICT’s age. This is the reason why only a limited number of previous studies mention exactly ECE (or kindergarten) and videogames, as it can be seen in the references (Din & Calao, 2001; Lieberman; Chesley Fisk; Biely, 2009; Tsai et. al., 2009). Literature on ECE and computer games as a part of computer technology has reported that digital games with educational features, when properly designed can provide fun, rich and interactive experiences that can promote young children’s learning, cognitive development, social interactions and healthy behaviors (Lieberman; Chesley Fisk; Biely, 2009). Computer learning games have been effective at increasing social, literacy, problem solving, mathematical and eye‐hand coordination skills of pre‐school children (Amory et. al., 1999; Divjac & Tomic, 2011; Koivisto et. al., 2011; Lonigan et. al., 2003; Papaloukas et. al., 2011; Yelland, 2002; Zevenbergen & Logan, 2008) and further foster their higher order thinking ability (Carbonaro et. al., 2010; Yien et.al.; 2011). Furthermore, the use of educational digital games may provide models of good learning practices, and by playing games infants will develop practical competencies and social practices (Manessis, 2011). Educators are the primary agents of educational innovation (Tsitouridou & Vryzas, 2003). The success of learning with computer technology will depend largely on the attitudes of teachers, from kindergarten to higher education, and their willingness to embrace the technology (Teo, 2006). Hence, the role of ECE students

369

Dionissios Manessis as future teachers is crucial. Unless they adopt the potential of the exploitation of computer games in the pre‐ school settings, they will be unwilling to introduce these particular games into their teaching. In order to successfully incorporate educational innovation such as GBL methods into an early childhood pre‐service curriculum course, it is important to identify the parameters that may have influence on early childhood pre‐ service teachers’ attitudes towards educational computer games. Attitude has been defined as “a learned predisposition or a tendency to respond positively or negatively to a specific object, situation, institution, concept, idea, or person” (Aiken, 1996). Attitudes toward digital games are considered in this paper as perceptions, beliefs and feelings ECE students have about the usefulness of these games in the nursery school or to themselves. Therefore, the profile of prospective teachers’ attitudes toward computer games influences their decisions to conduct GBL mediated teaching. In addition, these attitudes could be used to predict the successful or unsuccessful integration of computer games in the Kindergarten classroom. Hence, in‐ and pre‐service teachers’ views have an impact on their intensions and these, in turn, influence behavior (Dillon & Gayford, 1997; Gialamas, Nikolopoulou, 2010; Ma, Anderson & Streith, 2005). Another important issue is that the students, who are trained to become early childhood teachers, must be appropriately prepared to embody the use of ICTs’ into the kindergarten environment in such a way that they will respond to the developmental needs of the infants. All ECE departments should introduce ICT modules, specialized in GBL teaching methods into their programme of studies. Self‐efficacy refers to one’s belief of one’s ability to succeed in specific situations (Bandura, 1997). The term self‐efficacy in the ability of using digital games concerns the ECE students’ beliefs in their own capabilities with regard to the instructional use of computer and computer games with educational features in the classroom.

2. Participants The purpose of this paper was to investigate ECE students’ attitudes toward educational computer games in Kindergarten. These attitudes were examined using a sample consisted of 98 freshmen and 102 senior ECE students (total sample size 200), attending a Bachelor in Education degree at the department of ECE, University of Athens, in Greece. The participants who are trained to be kindergarten teachers were female (worldwide predominance of females in the population of early childhood teachers), who attended a 13‐week ICT course, which included some lectures about GBL applications in the kindergarten.

3. The instrument Data were collected by the use of a 5‐point Likert‐type scale questionnaire (1 = strongly agree, 2 = agree, 3 = I am not sure, 4 = disagree, 5 = strongly disagree), which was separated in two sections. The first section included participants’ demographics such as gender, year of study, having experience in an early childhood classroom and previous experience in playing computer games. It also included statements about having computer at home, frequency of computer usage per day, computer usage in any environment, and having attended courses about the use of ICT in ECE. The self‐efficacy in the ability of using computer games was measured by 3 items. The second part contained 20 statements in order to investigate early childhood teachers’ views and intensions about using digital games in the classroom. The questionnaire was administered at the end of the course. Before administering the questionnaire to the students, it was checked by three experts in ICT in education and a child psychologist, in order to examine its content validity and the appropriateness of specific items. The tests’ results were satisfactory in terms of both validity and appropriateness.

4. Results 4.1 Participants’ demographic characteristics All participants were female. 49% of them were freshmen and 51% were senior. All of them had a computer at home. Some of them (12,5%) were using it less than 1 hour per day, the majority of them were using it frequently (1‐3 hours per day) and only 3% of the total were using it more than 3 hours per day. It was also found that out of the 1st year students, 84% were using computer in any environment for 3 years or more, and

370

Dionissios Manessis out of the 4th year students, 95% were using computer in any environment for more than 3 years. In the total sample of 200 students, 100 (50%) had attended at least one course about the use of ICT in early childhood classroom and the other 50% had not. 83% of all students had previous experience in playing computer games and 17% of them had never played in the computer before. The senior students had experience in an early childhood classroom while the freshmen had not yet practiced in a kindergarten classroom.

4.2 Self‐efficacy in the ability of using computer games The students who participated in the research showed increased self‐efficacy in the ability of using computer games (Figure 1). Computer games self-efficacy 70% 61%

58%

60% 50% 40%

39%

Strongly agree

34%

Agree

30%

Disagree

20% 8%

10% 0%

0% I am good at using computers and computer games

I am quick to learn digital games needed for school

Figure 1: Computer/computer games self‐efficacy

4.3 Attitudes towards educational computer games in Kindergarten The ECE students of the sample had generally very positive attitudes toward the usefulness of educational computer games in Kindergarten, as illustrated in Table 1. Most of them agreed that digital games work as a useful education tool and expressed great willingness to use GBL to benefit children in learning environments, in the future. Table 1: Indicative statements of the questionnaire

I am Agree not sure

Disagree

Strongly disagree

Strongly agree (%)

371

Dionissios Manessis

Strongly agree

I am Agree not sure

Disagree

Strongly disagree

(%) I would not like to use educational digital games with children, unless it is required The integration and use of educational digital games in ECE are essential The narrative/thematic aspects of educational digital games support children with learning difficulties

4.3.1 The effect of year of study on attitudes towards educational computer games in Kindergarten Independent sample t‐tests were applied to determine whether “year of study” of the students have effect on their attitudes towards educational computer games in Kindergarten. The results are demonstrated in Table 2. The items of the attitude scale were assigned with numerical values ranging from 1 = “Strongly agree”, to 5 = “Strongly disagree”. Therefore, column “Mean” of the table refers to the mean attitude score, measured from 1 to 5. Table 2: The effect of year of study on attitudes towards educational computer games in Kindergarten

Digital games integration in ECE degrades the role of the teacher The use of educational digital games improves childrens' active learning I intend to use educational digital games with the children in the classroom The use of educational digital games may provide models of good learning practices The use of educational digital games by young children promotes their social isolation The use of educational digital games limits childrens' creativity I am not interested in using educational digital games with children I would not like to use educational digital games with children, unless it is required The integration and use of educational digital games in ECE are essential The narrative/thematic aspects of educational digital games support children with learning difficulties

Year of study

Mean

1st year

4,23

,89

4 year

102

4,57

,71

1st year

1,31

,46

4 year

102

1,23

,56

1st year

1,43

,56

4 year

102

1,16

,37

1st year

1,28

,51

4 year

102

1,25

,58

1st year

3,60

,70

4 year

102

4,24

,81

1st year

4,09

,58

4 year

102

4,52

,63

1st year

4,60

,65

th 4 year

102

4,77

,51

1st year

4,33

,69

4 year

102

4,57

,57

1st year

1,55

,75

th 4 year

102

1,20

,49

1st year

1,31

,46

102

1,13

,34

4 year

198

2,93**

198

1,11

166,64

4,07***

198

,79

198

5,90***

197,66

5,03***

182,57

2,09*

188,97

2,71**

165,99

3,99***

176,27

,002**

* p≤0,05 ** p≤0,01 *** p≤0,001 As shown in Table 2, there is a significant difference in attitude scores toward educational computer games in Kindergarten, between 1st year and 4th year students. The 4th year ECE students have significantly more positive attitudes toward educational digital games in the classroom than 1st year students.

372

Dionissios Manessis 4.3.2

The effect of frequency of computer usage per day on attitudes towards educational computer games in kindergarten

A one‐way ANOVA was performed in order to examine the effect of computer usage per day on attitudes towards educational computer games in Kindergarten. PostHoc analyses were conducted by Tukey’s HSD test. The summarized results of the analyses are presented in Table 3. Table 3: The effect of frequency of computer usage per day on attitudes towards educational computer games in Kindergarten

Frequency of computer usage per day < 1 hour ‐ A

2,88

,60

1‐3 hours ‐ B

169

4,61

,59

> 3 hours ‐ C

5,00

,00

< 1 hour ‐ A

2,04

,54

1‐3 hours ‐ B

169

1,16

,41

> 3 hours ‐ C

1,00

,00

I intend to use educational digital games with the children in the classroom

< 1 hour ‐ A

2,00

,50

1‐3 hours ‐ B

169

1,20

,40

> 3 hours ‐ C

1,00

,00

The use of educational digital games may provide models of good learning practices

< 1 hour ‐ A

1,92

,81

1‐3 hours ‐ B

169

1,18

,43

> 3 hours ‐ C

1,00

,00

The use of educational digital games by young children promotes their social isolation

< 1 hour ‐ A

2,72

,46

1‐3 hours ‐ B

169

4,07

,70

> 3 hours ‐ C

5,00

,00

The use of educational digital games limits childrens' creativity

< 1 hour ‐ A

3,60

,50

1‐3 hours ‐ B

169

4,39

,59

> 3 hours ‐ C

5,00

,00

< 1 hour ‐ A

3,84

,94

1‐3 hours ‐ B

169

4,80

,40

Digital games integration in ECE degrades the role of the teacher The use of educational digital games improves childrens' active learning

I am not interested in using educational digital games with children

Mean

> 3 hours ‐ C

5,00

,00

I would not like to use educational digital games with children, unless it is required

< 1 hour ‐ A

3,48

,51

1‐3 hours ‐ B

169

4,57

,53

> 3 hours ‐ C

5,00

,00

The integration and use of educational digital games in ECE are essential

< 1 hour ‐ A

2,64

,70

1‐3 hours ‐ B

169

1,20

,40

> 3 hours ‐ C

1,00

,00

< 1 hour ‐ A

1,88

,33

1‐3 hours ‐ B

169

1,12

,33

> 3 hours ‐ C

1,00

,00

The narrative/thematic aspects of educational digital games support children with learning difficulties

F (df)

Difference

99,18*** (2,20)

A‐B*** A‐C***

47,83*** (2,20)

A‐B*** A‐C***

44,24*** (2,20)

A‐B*** A‐C***

26,34*** (2,20)

A‐B*** A‐C***

52,51*** (2,20)

A‐B*** A‐C*** B‐C**

25,39*** (2,20)

A‐B*** A‐C*** B‐C*

42,90*** (2,20)

A‐B*** A‐C***

51,37*** (2,20)

A‐B*** A‐C***

119,03*** (2,20)

A‐B*** A‐C***

59,60*** (2,20)

A‐B*** A‐C***

* p≤0,05 ** p≤0,01 *** p≤0,001 As shown in Table 3, there is a significant difference in attitude scores toward educational computer games in Kindergarten, between the three groups of frequency of computer usage per day. The more often students use the computer per day, the more positive their attitudes towards the usefulness of digital games become. 4.3.3 The effects of experience in a pre‐school classroom and previous courses about the use/integration of ICT in ECE on attitudes towards educational computer games in Kindergarten Independent sample t‐tests were conducted to determine whether “experience in a pre‐school classroom” and “previous courses about the use/integration of ICT in ECE” have effect on ECE students’ attitudes towards

373

Dionissios Manessis educational computer games in Kindergarten. The results are demonstrated in Table 4. Here also, column “Mean” of the table refers to the mean attitude score, measured from 1 to 5 (1 = “Strongly agree”, 5 = “Strongly disagree). Table 4: The effects of “experience in a pre‐school classroom” and “previous courses about the use/integration of ICT in ECE” on attitudes towards educational computer games in Kindergarten

Classroom experience With experience

Without experience

Previous ICT courses Have attended

Mean (SD)

Have not attended Mean (SD)

Digital games integration in ECE degrades the role of the teacher

4,62*** (0,65)

4,21*** (0,91)

4,85*** (0,36)

3,96*** (0,91)

The use of educational digital games improves childrens' active learning

1,16** (0,44)

1,36** (0,56)

1,06*** (0,24)

1,47*** (0,63)

I intend to use educational digital games with the children in the classroom

1,13*** (0,33)

1,44*** (0,55)

1,03*** (0,17)

1,55*** (0,56)

The use of educational digital games may provide models of good learning practices

1,19 (0,47)

1,33 (0,60)

1,03*** (0,17)

1,50*** (0,67)

The use of educational digital games by young children promotes their social isolation

4,34*** (0,69)

3,55*** (0,75)

4,32*** (0,68)

3,53*** (0,76)

The use of educational digital games limits childrens' creativity

4,60*** (0,55)

4,05*** (0,60)

4,63*** (0,54)

3,99*** (0,56)

I am not interested in using educational digital games with children

4,84*** (0,37)

4,55*** (0,71)

4,94*** (0,24)

4,44*** (0,72)

I would not like to use educational digital games with children, unless it is required

4,65*** (0,48)

4,27*** (0,71)

4,82*** (0,39)

4,08*** (0,63)

The integration and use of educational digital games in ECE are essential

1,09*** (0,29)

1,62*** (0,78)

1,03*** (0,17)

1,71*** (0,77)

The narrative/thematic aspects of educational digital games support children with learning difficulties

1,09*** (0,29)

1,32*** (0,47)

1,00*** (0,00)

1,43*** (0,50)

* p≤0,05 ** p≤0,01 *** p≤0,001 As shown in Table 4, students with experience in a ECE classroom and with previous courses about the use/integration of ICT in early childhood classroom, have significantly more positive attitudes toward educational computer games in Kindergarten. 4.3.4 The effect of previous experience in playing digital games on attitudes towards educational computer games in kindergarten According to the results of independent sample t‐tests, there are significant differences between attitudes’ scores towards the usefulness of educational digital games in pre‐schools settings, for students who had previous experience in playing computer games and for those who did not (p<0,001). Players have significantly more positive attitudes than non‐players, as shown in Figure 2.

374

Dionissios Manessis The effect of previous experience in playing digital games on attitudes towards educational computer games in Kindergarten (1 = “Strongly agree”, 5 = “Strongly disagree) 2,5

2,38

1,94

1,91 1,65

1,5

Players

1,13

1,16

1,09

Non players

0,5

0 The integration and The use of The use of educational digital educational digital use of educational digital games in games improves games may provide ECE are essential models of good childrens' active learning practices learning

Digital games are a I intend to use useful way to educational digital enhance learning games with the children in the classroom

Bar represents the mean score of specific items of the students’ attitudes towards educational computer games in Kindergarten; 1 = strongly agree, 2 = agree, 3 = I am not sure (undecided), 4 = disagree, 5 = strongly disagree Figure 2: The effect of previous experience in playing digital games on attitudes towards educational computer games in kindergarten

5. Discussion The value of digital games as a vehicle for teaching concepts while inspiring students is now well accepted at almost all levels of education (Becker, 2001). The use of computer games in ECE, as an innovative teaching and learning tool, has growth prospects. Given the fact that early childhood teachers’ beliefs about digital games with educational features will influence their decision about using GBL methods in pre‐school settings, it is necessary to investigate ECE students’ attitudes toward the future use of computer games in the kindergarten classroom. The main aim of this research was to examine, among other things, the psychological impact of attitudes on students' perceived usefulness of digital games in kindergarten. The results showed that overall the participants were highly experienced with computers. All of them had a computer a home, which were using frequently. This finding is in agreement with other studies (Gialamas, Nikolopoulou, 2010; Jones, Copeland & Kalinowski, 2007; Yilmaz, Alici, 2011), confirming the world of the “digital natives” (Prensky, 2001), the new generation of students, who are much better acquainted with computer usage than their earlier counterparts. These young people having grown up with computers, digital media and the Internet have a natural aptitude and high skill levels when using new technologies (Jones et.al.; 2010; Myers, 1989). This implies that students are also familiar with playing quite often computer games from their early age, as shown in this research. Students perceive themselves as computer literate and competent computer games users, as depicted in Figure1. The high levels of students’ self‐efficacy in their ability of using computer and computer games, found in this study, are linked to their access to a computer at home and its frequent use. Strong links between university students’ access to and use of computer and computer games at home, were also shown in other studies (Nikolopoulou & Gialamas, 2009; Teo & Hwee Ling Koh, 2010). The majority of students of this research had positive attitudes towards educational computer games in Kindergarten. This finding is in agreement with other studies (Campbell & Scotellaro, 2009; Can & Cagiltay, 2006; Kutluca, 2011). Most of the pre‐service early childhood teachers agreed that this category of digital

375

Dionissios Manessis games work as a useful education tool which improves young childrens’ active learning. They also expressed intention to use educational computer games in their future teaching in the Kindergarten. Computer/computer games self‐efficacy has a direct significant effect on ECE students’ behavioural intensions to use digital games with children as future teachers. Hence, self‐efficacy in the ability of using educational computer games as an instructional tool could be used to predict ECE prospective teachers’ willingness and preparedness to successfully integrate GBL methods in the Kindergarten classroom. This important finding shows a remarkable consistency with the results of other studies, as much in the case of pre‐service teachers (Nikolopoulou & Gialamas, 2009; Teo & Hwee Ling Koh, 2010), as in the case of in‐service teachers (Wozney et. al., 2006). Research has reported significant differences between “year of study”, frequency of computer usage”, “experience in a pre‐school classroom”, “previous ICT courses”, “experience in playing computer games” and attitudes toward educational digital games in Kindergarten. Senior students with classroom experience, who frequently used computers, had attended ICT courses and were playing computer games, had a more positive attitude toward educational digital games than did those who did not frequently used computers, had no experience in a pre‐school classroom, as being freshmen, were not playing computer games and had not attended courses about the use/integration of ICT in ECE. These findings are in agreement with earlier studies (Campell & Scotellaro, 2009; Can & Cagiltay, 2006; Jones et. al., 2007; Yilmaz & Alici, 2011). A possible interpretation for these findings could be that senior students had previously attended the “Integration of ICT in preschool education” course, had practice in an Early Childhood institute and were familiar with computer games, as they were using computers quite often. Consequently, these students can better combine the use of GBL methods with both the developmental characteristics of infant pupils and the goals of ECE. Thus, they have more positive attitudes toward the usefulness of educational computer games. In that case, students may avail themselves of these innovative opportunities, thereby increasing the possibility of effective implementation of digital games in the kindergarten, as future teachers. A limitation of the study is that attitudes were examined as a one‐dimension factor. Future similar studies can contribute to the examination of cognitive (knowledge, beliefs and expectations), affective (emotional and motivational) and performance (behavior or actions) components of ECE students’ attitudes toward educational computer games in the Kindergarten, because positive attitudes are the key factors toward GBL methods integration in an Early Education classroom.

References Aiken, L. R. (1996) “Rating scales and checklists: Evaluating behavior, personality, and attitudes”. Canada: John Wiley & Sons. Armstrong, A. & Casement, C. (2000) “The child and the machine. Why computers may put our children's education at risk”. Beltville: Robins Lane Press. Bandura, A. (1997) “Self‐efficacy: The exercise of control”, New York: Freeman. Becker, K. (2001) “Teaching with games: The minesweeper and asteroid experience. Journal of Circuits, Systems and Computers, Vol 17, No.2, pp 23‐33. Campbell, A., & Scotellaro, G. (2009) “Learning with technology for pre‐service early childhood teachers”, Australian Journal of Early Childhood, Vol 34, No. 2, pp 11‐18. Can, G., & Cagiltay, K. (2006) “Turkish Prospective Teachers’ Perceptions Regarding the Use of Computer Games with Educational Features”, Educational Technology & Society, Vol 1, No. 1, pp 308‐321. Carbonaro, M., Szafron, D., Cutumisu, M., & Schaeffer, J. (2010) “Computer‐game construction: A genderneutral attractor to Computing Science”, Computers & Education, Vol 55, No.3, pp 1098‐1111. Amory, A., Naicker, K., Vincent, J., and Adams, C. (1999) “The use of computer games as an educational tool: identification of appropriate game types and game elements”, British Journal of Educational Technology, Vol,30, No. 4, pp. 311‐ 321. Cordes, C., & Miller, E. (Eds.) (2000) “Fool's gold: A critical look at computers in childhood”. College Park, MD: Alliance for Childhood. Dillon, P., & Gayford, C. (1997) “A psychometric approach to investigating the environmental beliefs, intentions and behaviours of pre‐service teachers”, Environmental Education Research, Vol 3, No.3, pp 283–297. Din, F.S., Calao, J. (2001) “The Effects of Playing Educational Video Games on Kindergarten Achievement”, Child Study Journal, Vol 3, No. 2, pp 95‐102.

376

Dionissios Manessis Divjak, B., & Tomic, D. (2011) “The Impact of Game‐Based Learning on the Achievement of Learning Goals and Motivation for Learning Mathematics ‐ Literature Review”, Journal of Information and Organizational Sciences, Vol. 35, No. 1, pp. 15‐30. Gialamas, V., Nikolopoulou, K. (2010) “In‐service and pre‐service early childhood teachers’ views and intentions about ICT use in early childhood settings: A comparative study” Computers and Education, Vol 55, No. 1, pp 333–341. Jones, G., Copeland, B., Kalinowski, K. (2007) “Pre‐Service Teacher’s Attitudes towards Computer Games”, Paper presented at the American Educational Research Association, Chicago. Jones, C., Ramanau, R., Cross, S., Healing, G. (2010) “Net generation or Digital Natives: Is there a distinct new generation entering university?”, Computers and Education, Vol 54, No. 3, pp 722–733. Koivisto, A., Kiili, K., and Perttula, A. (2011) “Designing Educational Exertion Games for Young Children”, ECGBL 2011 Proceedings, Athens, Greece, pp 322‐328. Kutluca, T. (2011) “A study on computer usage and attitudes toward computers of prospective preschool teacher”, International Journal on New Trends in Education and Their Implications, Vol 2, No. 1, pp 1‐14. Lieberman, D.A., Chesley Fisk, M., & Biely, E. (2009) “Digital Games for Young Children Ages Three to Six: From Research to Design”, Computers in the Schools, Vol 26, No. 4, pp 299‐313. Lonigan, C. J., Driscoll, K., Philips, B. M., Cantor, B. G., Anthony, J. L., & Goldstein, H. (2003) “A computer‐assisted instruction phonological sensitivity program for preschool children at‐risk for reading problems”, Journal of Early Intervention, Vol 25, No. 248. Ma, W., Anderson, R., & Streith, K. (2005) “Examining user acceptance of computer technology: An empirical study of student teachers”, Journal of Computer Assisted Learning, Vol 21, No. 6, pp 387–395. Manessis, D. (2011) “Early Childhood Post‐Educated Teachers’ Views and Intentions About Using Digital Games in the Classroom”, ECGBL 2011 Proceedings, Athens, Greece, pp 753‐758. Myers, J.P. (1989) “The new generation of computer literacy”, SIGCSE technical symposium on Computer science education, Vol 21, No. 1, pp 177‐181. Nikolopoulou, K. and Gialamas, V. (2009) “Investigating pre‐service early childhood teachers' views and intentions about integrating and using computers in early childhood settings: compilation of an Instrument”, Technology, Pedagogy and Education, Vol 18, No. 2, pp 201–219. Papaloukas, S., Patriarcheas, K., and Xenos, M. (2011) “Games’ Usability and Learning‐the Educational Videogame BeTheManager!”, ECGBL 2011 Proceedings, Athens, Greece, pp 449‐456. Prenksy, M. (2001) “Digital natives, digital immigrants”. On the Horizon, Vol.9, No. 5, pp 1–6. Teo, T. (2006) “Attitudes toward computers: A study of post‐secondary students in Singapore”, Interactive Learning Environments, Vol 14, No. 1, pp 17‐24. Teo, T. & Hwee Ling Koh, J. (2010) “Assessing the dimensionality of computer self‐efficacy among pre‐service teachers in Singapore: a structural equation modeling approach”, International Journal of Education and Development using Information and Communication Technology, Vol 6, No. 3, pp 7‐18. Tsai, C.M., Hong, J.C., & Ho, Y.J. (2009) “The Learning Effectiveness of Blended and Embodied Interactive Video Game on Kindergarten Students”, Proceedings of the 4th International Conference on E‐Learning and Games, pp 456‐463. Tsitouridou, M., & Vryzas, K. (2003) “Early childhood teachers’ attitudes towards computer and information technology: The case of Greece”, Information Technology in Childhood Education Annual, 1, pp 187–207. Verenikina, I., Harris, P. and Lysaght, P. (2003) “Child's Play: Computer Games, Theories of Play and Children's Development”, International Federation for Information Processing Working Group 3.5 Open Conference, Melbourne, Australia. Wozney, L., Venkatesh, V., & Abrami, P. (2006) “Implementing computer technologies: Teachers’ perceptions and practices”, Journal of Technology and Teacher Education, Vol 14, No. 1, pp 120–173. Yelland, N. (2002) “Playing with Ideas and Games in Early Mathematics”, Contemporary Issues in Early Childhood, Vol 3, No. 2, pp 197‐215. Yien, J.M., Hung, C.M., Hwang, G.J., Lin, Y.C. (2011) “A Game‐Based Learning Approach to Improving Students’ Learning Achievements in a Nutrition Course”, The Turkish Online Journal of Educational Technolog, Vol 10, No. 2. Yilmaz, N. & Alici, S. (2011) “Investigating Pre‐Service Early Childhood Teachers' Attitudes towards the Computer Based Education in Science Activities”, The Turkish Online Journal of Educational Technolog, Vol 10, No. 3, pp 161‐167. Zevenbergen, R. & Logan, H. (2008) “Computer use by preschool children: Rethinking practice as digital natives come to preschool”, Australian Journal of Early Childhood, Vol 33, No. 1, pp 37‐44.

377

Integrating Non‐Virtual Electronic Activities in Game‐Based Learning Environments Jean‐Charles Marty1, 3, Thibault Carron2, 3, Stéphane Talbot3, Gregory Houzet4 and Philippe Pernelle5 1 LIRIS, UMR5205, F‐69621, France 2 LIP6 Lab, UMR CNRS 7606, Université Pierre & Marie Curie, France 3 Université de Savoie, France 4 Imep‐Lahc Lab., Université de Savoie, Campus scientifique 73376 Le Bourget Du Lac, France 5 DISP Lab, Université de Lyon, France Jean‐Charles.Marty@liris.cnrs.fr Thibault.Carron@lip6.fr, Stéphane.Talbot@univ‐savoie.fr, Gregory.Houzet@univ‐savoie.fr, Philippe.Pernelle@univ‐lyon1.fr Abstract: Our past experiments with Game‐Based Learning multi‐players environments, have shown some weaknesses in specific learning activities. Learners seem to acquire a skill in the game, but they are not able to apply it easily in the real world. This is particularly the case for learning skills that require concrete manipulation with real objects. In fact, Game Based Learning Environments (GBLE) lack of means to learn know‐how aspects. Some learning processes involving real world objects are very difficult to reproduce in the GBLE and there is an essential technological issue in mixing virtual and real aspects in GBLE. In this article, we describe these problems through an example in the electronic domain. We explain how to consider activities taking place outside the numeric environment. We have set up an experiment, where students needed to design “electronic circuits” with concrete electronic elements before being allowed to continue a quest in a virtual world. A complete scenario aiming at learning this kind of knowledge thus swaps from activities in the virtual world to activities in the real world. New issues linked to this transition are explained. Keywords: know‐how activity evaluation; game‐based learning environment; online multiplayer game; user model, mixing numeric and face‐to‐face learning tasks

1. Introduction Due to their large educational features, Learning Games are currently spreading out. Both industrial and academic institutions want to use them in order to have motivating and flexible environments to train their employees / students. The complexity of Learning Games development is thus increasing significantly, since lots of new needs appear (Squire, 2003). New up‐to‐date functionalities are often wanted: collaborative aspects, observation features for awareness purpose, and links to tangible user interfaces, support for metacognition (Dimitracopoulou, 2005). The requirements for these learning environments imply building several components aiming at supporting specific activities (games of "snakes and ladders" type, puzzles, animated MCQs) (Djaouti, 2008) (Mariais, 2009). In light of this observation, there is an obvious need for realistic and reliable assessment about students’ skills, actions or behaviours especially for the teacher (Felicia, 2009). Indeed, for certain specific domains, the teacher needs to evaluate his/her pedagogical session according to several points and some are particularly difficult to assess via such learning environments. Some domains present for example the particularity to exhibit both theoretical knowledge and practical know‐how (gestures for manufacturing or medicine for example) (George and Serna, 2011). For such contexts, the current Learning Games are not efficient enough concerning this second point: a unique and full digitalisation of the objects is not sufficient to guarantee both a good learning and an assessment of the techniques (Schrier, 2006): it is mandatory to come back to the real world and thus develop a mixed‐reality learning game. In this article, we will focus on two points: a new way of know‐how assessment and the possible enhancement of learning games via communicating objects. We first give a short description of a general game‐based learning environment called "Learning Adventure" and point out the need for a new kind of assessments that

378

Jean‐Charles Marty et al. can’t be entirely supported via a digital/virtual learning session. We next focus on integrating both new communicating objects and learning stuff concerning the electronics domain in order to set up a mixed reality learning game. The main point of this article is to consider how to take into account the new issues due to the mixed reality. We also want to clarify how to use communicating objects to enhance the features of a Learning Game. Finally, in the last part, we will illustrate most of these points through a real experiment: We evaluate both theoretical content and practical know‐how in the electronic domain via such learning environments.

2. A games based learning environment: Learning adventure Short description of Learning Adventure In order to set up new learning sessions with our students, we have developed a Game Based Learning Management System called Learning Adventure (L.A.) and based on a role play approach (Baptista, 2008). Our approach is focused on game with intrinsic metaphor (Fabricatore et al., 2000). The latter can be defined as “a virtual environment and a gaming experience in which the contents that we want to teach can be naturally embedded with some contextual relevance in terms of the game‐playing [...]”. L.A. is a multiplayer 3D environment where the learning session takes place (see fig. 1). A particular map (environment with buildings, lakes, mountains and hills) is dedicated to a particular learning activity, for a particular subject. Each part of the map represents the place where a given (sub) activity can be performed. The map topology represents the overall scenario of the learning session, i.e. the sequencing between activities (Kinshuk, 2006). Fig. 2 shows an example of such a scenario (2013 experiment). There are as many regions as actual activities, and the regions are linked together through paths and Non‐Player Characters (NPC) guards, showing the attainability of an activity from other ones. Players (students or teachers) can move through the environment, performing a sequence of sub activities in order to acquire knowledge. Activities can be carried out in a personal or collaborative way. A list of cooperation abilities is given in (Dillenbourg, 1996): you can access knowledge through virtual objects available in the world, from documents, via help from teachers, or through work with other students. Each object can thus be relevant for the entertainment domain or the pedagogical one.

Figure 1: Screenshot of Learning Adventure into an industrial futuristic world

379

Jean‐Charles Marty et al.

Figure 2: Example of a learning scenario Design of a learning session There is no doubt about the high motivation of the students correlated to this way of learning. Immersion is an important aspect that one must take into account. However, this involves a significant software development cost when one wants to set up a new learning scenario, especially when one wants to keep a balance between the learning part (knowledge to be acquired) and the game part (motivation challenges). In figure 2, the central scenario is based on four main activities (Start, A1, A2, End) that constitute the main quest. It is simple and linear. For motivation and synchronisation purposes, five facultative sub‐quests are available (A.1.1, A.1.2, A2.1, A2.2, A2.3). Virtual objects representing learning contents (generally part of the course) are present in the game. A hidden contest also exists: some chests (entertainment virtual objects) containing the teachers’ caricatures are disseminated everywhere in the digital world. L.A. is a generic GBL environment: many topics can be tought. This year, we have set up an experiment linked to the electronics domain. According to the electronics teachers, it is important to be able to manipulate the electronic component and obtain a real result in order to acquire the know‐how and to help fixing the acquired knowledge (Garris et al., 2002). For such contexts, the current Learning Games are not efficient enough: from our point of view, a full digitalisation of these objects is unsatisfactory to guarantee both a good learning and an assessment of the techniques. As a matter of fact, in industrial domains, the learning processes are often based on the manipulation of some objects that are difficult to simulate into GBL environments: users sometimes need to learn by acting on real devices. Moreover, even if some games are collaborative, a collaborative activity is more effective in real context, particularly when a discussion about a specific object holds (e.g. observe/examine an object from every angle, show specificities or evaluate several tries). We were consequently faced with the problem of developing a new way for know‐how assessment that had to be achieved in the real world with real devices. As explained later, the purpose of the lesson is to study Electronic‐wiring diagrams through the use of Arduino boards. The students will thus construct electronic objects and use them in the real world. For that purpose, some parts of the scenario should thus take place in the real world. Virtual Objects (electronic wiring diagrams) discovered in the game must be transformed into real ones (Arduino boards) that are needed to achieve the quest (see fig.3). These objects are labelled as mixed reality objects. This approach allowed us to consider new needs. Consequently, new problems should also be addressed. In the next section, we explain how to tackle such a mixed reality approach.

3. From Virtuality to reality: The questions raised by specific learning assessment As explained in the previous section, adding real world activities in a learning game in order to build‐up new mixed‐reality learning games raises several challenging problems.

380

Jean‐Charles Marty et al.

Figure 3: Students collaborating with an Arduino board on a multi‐touch tabletop (left part) and an Arduino board (on the right) Keep a coherent story: First of all, motivation is directly connected to immersion in virtual worlds. In order to maintain the students’ motivation high, the return to reality must be well explained and should not be considered as a constraint. The scenario designers should be aware of writing smooth transitions both from virtual to reality and from reality to virtual. Story telling, games incentives must be continuous from virtual to real worlds. Insure a well‐defined access to the real devices: As opposed to virtual worlds where different groups of learners can access a particular resource simultaneously, only one group can access real devices (e.g. multi touch tables) at the same time. This can cause a serious problem connected to motivation of people. Indeed, if several groups need to access the real device nearly at the same time, only the first group will have the right to use it, while the other groups will have to wait. The scenario designer must be aware of this discouragement risk and design additional learning activities in order to propose them to different groups when such an event occurs. Complementary activities were designed for this purpose. Use rights acquired in the virtual world: In order to reinforce continuity between the virtual and the real world, the scenario designer would also have to ensure that the rewards acquired in the digital world could be used in the real world. For instance, it is often the case that a learner plays a role and has rewards proving that s/he did a good job connected to this role. In the real world, learners must only have the rights on the real devices corresponding to their roles. This has obviously the advantage of increasing collaboration for using the device, every member of the team having his/her own knowledge about the real collaborative task. Define a strict correspondence between real devices and their associated avatar in the virtual world: Again to straighten the links between virtual and real worlds, the scenario designer must establish a bijection between the real device and its representation in the virtual world. This also helps the users to understand easily how to use a new device, since one can provide them with explanations in the virtual world, before the actual device use. The results obtained through the real device can furthermore be integrated in the virtual world and should be accessible via the avatar associated to the real device. This short list of advices addressed to the scenario designers is issued from different experiments that we have set up in our university. Some of them are somewhat difficult to implement, since there is a need for mixed reality objects communicating with the GBL environment. Furthermore, one also needs to have means for detecting who is where in order to control the rights to perform actions on the real devices. For that purpose, as we will see now, we studied the features delivered by the use of communicating objects.

381

Jean‐Charles Marty et al. Table 1: Examples of mixed reality issues Domain Pedagogy

Issues Concrete assessment (know‐how, manipulation, gesture) with specific device. Maintain a deep immersion for the learner

Virtual World Virtual representation of the specific device (no evaluation) Coherent Story leading to the specific object

Collaboration (intra‐group)

Synchronize students in the same groups

Propose some facultative mini‐quests

Collaboration (extra‐group)

Synchronize groups of students (no delay for accessing to a group exclusive resource) Assessment of content

Change the order of the quests.

Keep a complete “game feeling”.

Beginning and end in the game.

Entertainment

Content

Scenarisation

Indicators thanks to traces

Real World Real Object/ Specific Device (effective work) Integration of clues and challenges that remind the playful aspects. Small groups with an active role for each participant. Regulate the access to the real world

Camera or results reintegrated in the game via digital input. Short –well identified times in real world.

Near field communication (NFC) and Radio‐Frequency Identification (RFID) are wireless communication technologies that connect two wireless devices. The devices have just to be placed close to each other for exchanging data. As explained before, when you set a mixed reality learning game, new problems are raised. For example, it is more difficult to identify where is the student. The mixed reality objects have to communicate with the learning platform for different purposes. This new technology is very versatile and allows many kinds of use that could be particularly suitable for learning environments. For example, let’s imagine that each learner has a communicating object (e.g. equipped/tagged with a NFC/RFID Tag). These devices are present both in the virtual world and outside. They can be used in different situations and will be helpful for the transition between V/R: Identify and/or authenticate the learners: The learners have their own tag and put it on the table to be automatically identified and log in the game with personalized parameters (avatar name, filled user model, etc.). In the real world, the presence of the tag can control the access to a specific tool. It is also possible to use recent smartphones that are equipped with such technology. Localise the learners: The learners that are not in the game anymore (mixed reality) are tracked outside the virtual world thanks to the objects they use. Memorise the pieces of information: Using external tools allows obtaining a specific result. The tags (or phones) are able to store such results (possibly a virtual object) and by just putting them on the table, the information can be sent back in the game. Communicate about the learners: Such tags or devices contain information. For instance, the tag gives information to the tabletop in order to automatically update the relevant information or configure the user interface according to the players’

382

Jean‐Charles Marty et al. preferences. The results of the activity in mixed reality may be stored in the tag and reused when back in the virtual world. Access control: A tag may be obtained and used in the real world to get access control to new objects. A similar use is already possible in the virtual world: “go and fetch all cards and I will give you the access to the collaborative planning tool”. Monitor the pedagogical session: A well‐known problem is to monitor the session, especially when collaboration is considered. All precedent imagined features bring some pieces of information about the on‐going session: where are the players? What tool do they use? For how long? It is then possible to continue to monitor the session even from outside the virtual world and for example, to identify the learners’ actions and update consequently their user model, as shown in (Carron and Marty, 2011).

4. An experiment combining L.A., mixed reality learning components and communicating objects technology We now illustrate how such interoperability via communicating object has been used to set up the training session described in the scenario of figure 2.

Pedagogical objectives and reuse/integration of specific electronic components

As mentioned earlier, the course was dedicated to the Electronics field. The learning content dealt with “Electronic wiring diagrams”. The aim of the session (role playing game) was to assess the knowledge and know‐how of the students about these latter. More precisely, they had to create a new very accurate controller: 6‐axis “wiimote‐like” controller thanks to an accelerometer. Knowledge assessment contained several items: Were the students able to identify a correct/erroneous electronic‐wiring diagram and to provide the teacher with an explanation? ; Do the students have the know‐how to create a new device by wiring an Arduino board and to use it to navigate in a labyrinth? We thus have set up the following knowledge quest using metaphors: “Several falsified documents have been retrieved. Identify them and thanks to the right information, create the innovative controller. When you succeed, you will be able to find a code and get access to critical information for identifying the falsifier. First, (a) you have to find the wiring diagrams, (b) identify the good one thanks to the different pieces of information contained in your lesson. After that, (c) you need to discover the needed electronic components. When you have them, (d) a similar quest begins with the programming software (computer and Arduino chip). At the end, (e) you are allowed to implement the 6‐axis controller in the real world and to use it for retrieving the code.” The narration and the questions are given by a NPC. The group explanation answers (step b) are communicated to the teacher for validation, through the “Spiral” pedagogical platform. Steps (b), (d) and (e) are collaborative in the real world: For step (b), the pictures of the different wiring diagrams are displayed on a multitouch tabletop, the learners discuss to identify and justify the wrong ones (see fig. 3). For step (d), wiring the Arduino board according to the correct identified diagram is done in a separate room containing the different electronic components. Finally, in step (e), the navigation into a labyrinth with the electronic Arduino board to get the code is also done via the tabletop (where the labyrinth is displayed).

Technical considerations:

The whole environment is coded in JAVA developed with the help of one engineer and students in training placement. The network part is based on the Red Dwarf1 project. The whole environment is built on a client‐ server architecture. The software integration of the multi‐touch tabletop and Arduino boards was developed with the help of a software engineer, a level designer and students on placement (internship). The multi‐touch tabletop (42 inches) was created from scratch in order to reduce the costs. It is based on a PQ Labs touch screen, a LG LCD Display and a small barebone computer. Correct actions are automatically awarded with the relevant skill level up for the learners (Felicia, 2009). 1

see http://www.reddwarfserver.org/

383

Jean‐Charles Marty et al.

Evaluation and Results

At the end of our experiment, the students were asked to fill in questionnaires to give feed back about their feelings concerning their work session. Twelve questions (Ranking and open‐ended) are used. The questionnaire evaluated aspects related to several parts of the learning game (pedagogical content, scenario‐ story, collaborative activities, and accessibility/impact of the mixed reality object). The final area in the questionnaire let the students propose some improvements concerning weak points of the game. Among 56 students, the main results were as follows:

96% of students consider that there is a good pedagogical content.

94% prefer to learn this way than in a “traditional” practical work.

85% qualify learning this way as enjoyable.

69% think that they have acquired additional knowledge during the session (21 % don’t know)

90% think that collaboration helps them to understand new concepts more easily.

65% are ready to continue the work at home.

63% would be interested by additional optional quests.

83% assert that switching to the real world does not break the immersion.

92% think that electronic‐wiring is a real know‐how (86% think that using the multi‐touch tabletop and the arduino are perfectly suited to acquire this know‐how).

We thus can consider that the first experiments that have been achieved in April 2013 with mixed reality objects are positive. The students were naturally very motivated by such practical work even though some optimizations concerning the number of quests were often suggested. Concerning the teacher, videotape analysis and further assessments have to be done in order to evaluate correctly the impact in terms of learning. But thanks to observation features (traces), a lot of data has already been collected and has still to be processed and analyzed, particularly to understand how the group solved the problem using the different members’ specific skills. Finally, the integration of some features of the pedagogical platform (Spiral) allow the teacher to collect the results of the groups directly from the GBL Environment. This evaluation process should also be improved in order to refine it (adaptation to the level of the group/student).

5. Conclusion In this article, we focused on the challenging problems raised by mixed reality learning games. In several specific domains, Game Based Learning Environments (GBLEs) lack of means to learn know‐how aspects. Some learning processes involving real world objects are very difficult to reproduce in a GBLE and there is an essential technological issue in mixing virtual and real aspects in GBLEs. We described how to consider activities taking place outside the numeric environment and focused on new issues linked to the transition between virtual and real worlds. To solve such problems, we propose to develop and use communicating objects in order to link the two worlds while keeping the advantages of GBLEs. We illustrated our proposal by the description of a concrete experiment that we have performed with students in our university. The case study concerns the electronic domain: teachers interested by learning game, wait for such new means to assess the know‐how of their students, thus coupling theoretical learning with practical assessment. As often with such environments, the students are very enthusiastic about such ways of teaching but from the teacher’s side, there are a lot of drawbacks. First of all, it is extremely time consuming to prepare the activity and several people must be involved to set up such experiments. However, and that is new, some parts of the software have been reused from our past experiments (map, configuration of NPC, sub quests). This may be a sign of maturity for this research work. The idea is to be able to build new experiments in a cheaper way. Indeed, we need to evaluate our approach in different contexts and with different people in order to really assess the effectiveness of this mixed approach of teaching.

384

Jean‐Charles Marty et al.

Acknowledgements We would like to thank the AIP Primeca and the Rhone‐Alps French Region for supporting this project. We would also like to thank Noura Benhajji, Alyzee Arnaud, Thibaut Fantin, Pierre Jacques, Ophélia Orenes, Mathilde Bulleté‐Herbaut, Gabriel Bertholon, Alain Perrier, Léa Schmidt and Guillaume Dumoulin for their help in developing the software and beta‐testing concerning the Arduino stuff and the multi‐touch tabletop integration.

References Baptista R. & Vaz de Carvalho C., (2008), Funchal 500 years: Learning Through Role Play Games, ECGBL’08, Barcelona, Spain. th Carron T., Marty J.‐Ch. : “Use of Profiling to optimise a collaborative session in a learning game”, Proc. of the 5 European Conference on Games Based Learning, Athens, Greece, pp. 88‐97 (2011) Dillenbourg P., Baker M., Blaye A., O'Malley C. (1996) "The evolution of research on collaborative learning" Learning in Humans and Machine: Towards an interdisciplinary learning science, pp. 189‐211. Dimitracopoulou, A., Bollen, L., Dimitriadis, Y., Harrer, A., Jermann, P., Kollias, V., Marcos, J., Martinez, A., Pedrou, A. (2005). State of the art of interaction analysis for Metacognitive support & diagnosis, in IAJEIRP. Djaouti D., Alvarez J., Jessel J‐P, Methel G., Molinier P. (2008). A Gameplay Definition through Videogame Classification, International Journal of Computer Game Technology, Hindawi Publishing Corporation. Fabricatore, C.: Learning and Videogames: an Unexploited Synergy. 2000 AECT National Convention ‐ a recap. Secaucus, NJ: Springer Science + Business Media, Long Beach, CA (2000). Felicia, P., (2009) Modelling Players’ Behaviours and Learning Strategies in Video Games, ECGBL’09, Graz, Austria. Garris, L., Ahlers R. and Driskell J. E., 2002. Games, Motivation, and Learning: A Research and Practice Mode, in Simulation & Gaming number 33; DOI: 10.1177/1046878102238607, pp 441‐467. George S. and Serna A. 2011. Introducing mobility in serious games: enhancing situated and collaborative learning. In Proceedings of the 14th international conference on Human‐computer interaction: users and applications ‐ Volume Part IV (HCII'11), Julie A. Jacko (Ed.), Vol. Part IV. Springer‐Verlag, Berlin, Heidelberg, 12‐20. Kinshuk S., Patel A., Oppermann R., Special issue: Current Research in Learning Design, Journal of Educational Technology & Society, V(9)‐1, 2006. Mariais, C., Michau F., Pernin J.‐P. (2009), Using Games Classifications to Support the Design of Learning Games, ECGBL’09, Graz, Austria. Marty J.‐C. and Carron T. (2011) Observation of collaborative activities in a game‐based learning platform. Transactions on Learning Technologies (TLT), 4(1) :98–110. Schrier, K. (2006). Using augmented reality games to teach 21st century skills. Conference proceedings, ACM Siggraph 2006 Educators Program, Boston, MA. Squire, K. (2003). Videogames in Education. International Journal of Intelligent Games & Simulations 2(1), 49‐62. Yee, N. (2007). Motivations of Play in Online Games. Journal of CyberPsychology and Behavior, 9, 772‐775.

385

From « Haute‐Couture » to « Ready‐to‐Wear »: Typology of Serious Games Implementation Strategies in Higher Education Hélène Michel Grenoble Ecole De Management, France Helene.michel@grenoble‐em.com Abstract: This article is an exploratory approach of the different strategies used in higher education institutions for implementing serious games. In the complex and often slow‐moving education sector, serious games are challenging not only the learners and teachers’ practices but also the organizations’ strategies. What performance criteria can be used in this specific context to evaluate the training? What types of strategies emerge? During the past decade, different types of serious games implementation have been experimented in higher education. Through a longitudinal analysis of five case studies, this article builds a typology of five strategies to implement serious games in higher education: Haute couture, Recycling, Ready‐to wear, Home‐made and Co‐branding. This study therefore helps managers deciding their own strategy according to their situation. Keywords: serious games, higher education, training performance criteria, implementation strategies, case studies, longitudinal approach

1. Introduction Serious games can be defined as “games in which education (in its various forms) is the primary goal, rather than entertainment” (Michael and Chen, 2006). These applications use the characteristics of video games to engage individual in a learning experience. They belong to the type of computer‐mediated environments of human learning, combining mediatized learning by machines, simulation, emotional reactions and professionalization. Serious Games as learning methods have been widely developed since the 2000s (Sawyer, 2002), Zyda (2005). Nevertheless, the optimism regarding the value of games as a means of education needs to be tempered. In 2006, researchers admitted that one of the elements that hinder the spread of games in the context of training is the lack of data making it possible to prove their effectiveness (De Freitas, 2006). There seems to have been little evolution since then, as, in their recommendations in Pivec and Pivec (2009) called for researchers to intensify the rate of data collection in particular through pilot experiences. In the complex and often slow‐moving education sector, serious games are challenging not only the learners and teachers’ practices but also the organizations’ strategies. What performance criteria can be used in this specific context to evaluate the training? What types of strategies emerge? How can higher education organizations use serious games in order to create a competitive advantage? During the past decade, different types of serious games implementation have been experimented in higher education. Through a longitudinal analysis of several case studies, this article offers an exploratory approach of these different strategies in order to build a typology. First the literature review describes how serious games have emerged as training method and identify their performance criteria. Then the methodology part details the research design and the case studies protocol. The results are presented for each case study and a general typology is detailed. Finally the conclusion discusses these results and defines the limitations and perspective of the study.

2. Literature review The literature review first describes how serious games have emerged as training method. Then, the different performance criteria to evaluate training processes and strategies are detailed.

2.1 The development of serious games as learning tools Serious Games have an historical and conceptual genealogy. The different evolutions reveal the way the concepts ‐ learning, simulation, game and professionalization – get developed and combined to elaborate the current serious game notion. We can divide this genealogy in five periods: emergence of machines as learning tools, introduction of the simulation notion, democratization through video games, professionalization of simulation games and finally the academic use in higher education. The idea of using a computer as a learning tool emerges with informatics. In 1924, psychologist Sydney Pressey already suggested with the « Drum Tutor

386

Hélène Michel » one of the first machine to learn through a dozen of quizzes. In the 1980s, with informatics democratization appeared the first software of computer‐aided learning. In this behaviorist approach computer transmits knowledge to the learner. Since the 1990s the Computer Assisted Language Learning and Teaching describes learning tools with socio‐constructivism background: Learner becomes in charge of his learning process by building his knowledge in and with action. Internet use has reinforced this approach trough e‐learning. In a managerial perspective, the main expected benefits are the cost reduction and the improvement of training quality. Simulation was introduced in 1946 with the MIT Whirlwind project, which enabled military airline pilots to train in a controlled situation. Learning was then achieved by trial and error in a systematic approach. Through offering a specific environment, reducing the risks and diffusing the necessary information, this process let the learners experiment the impact of their individual decision on a global situation. The learning process also evolves: the trial and error system offer the possibility to learners to experiment new scenarios without risks. It therefore encourages creativity. Novak et al. (2000) explain that simulation reinforces the flow Csikszentmihalyi (2000): a psychological optimal state that someone can reach when completely immerged in an action. When the challenge and the skills are perceived as high, the individual not only appreciates the moment but also increases his abilities on the long term. But this approach is efficient if the debriefing and corrections are reactive: Interactivity between learner and teacher, even virtual, is an imperative condition (Thorndike, 1932). The democratization of video games, for example in 1982 with “Flight Simulator”, made simulators available to a large audience. In the case of ELM (Elaboration Likelihood Model) (Petty and Cacciopo, 1984) persuasion model games can be seen as a fun approach, making it possible to increase motivation, as well as the individual’s perceived ability to deal with information in a cognitive manner. According to Huizinga (1955), play is free, is not “real” life, is distinct from “ordinary” life both as to locality and duration, creates order and finally is connected with no material interest. Games are therefore defined voluntary (Caillois, 1957) and therefore conflicts with the notion of “serious games”. However, even if playing can be seen as a futile activity, players develop a strong immersion and concentration. Vandeventer and White (2002) underline a high flow state during the game: Players are then more able to use complex information to go further in the process. A phase of professionalization in simulation games has been taking place since the 2000s. Games are again being used in professional training, but in a broader way and not only for gaining technical skills. Serious games can therefore be presented as technologies and video game platforms which have objectives other than simple entertainment Vorderer and Ritterfeld (2009). This virtual experience would aim at reengaging learners through a hyper‐real experience (Rheingold, 1993). The reintroduction of amusement has led to the appearance of the concept of edutainment (Prensky, 2001), (Gee, 2007). The commonly defended idea is that learner will be more interested in the subject thanks to the pleasure and the wealth of experience gained during the game. This increased interest and motivation leads to broader and more deep‐seated learning processes. Serious games could therefore reenchant learning (Ritzer, 1999). Finally, the applications of serious games in the field of education are very recent and remain rather limited. Thanks to the technologies development, the price of these tools decreased and made it more accessible to the academic field and especially in higher education. However; many researchers admitted that one of the elements that hinder the spread of games in the context of training is the lack of data making it possible to prove their effectiveness (De Freitas, 2006). There seems to have been little evolution since then, as, in their recommendations in 2009, Pivec and Pivec (2009) called for researchers to intensify the rate of data collection in particular through pilot experiences.

2.2 Measuring the performance of a training process To go further in this direction, this paper details the different models of training evaluation. We analyzed nine main models or frameworks for human resource training evaluation (DeSimone et al., 2002) (cf. Table 1). Each one of these models or frameworks focuses on different levels or categories. This article aims at analyzing an organizational strategy related to serious game. Therefore, it is necessary to use a model offering not only learning but also managerial criteria such as organizational benefits. We then decided to use the works of Kirkpatrick (1994) who proposed to assess the contribution of a learning method according to four levels: Level 1: satisfaction (did the learners appreciate the training?), Level 2 : the learning process (what did they learn?), Level 3 : individual skills (were the learners able to apply their new skills in the particular situations?), Level 4 :

387

Hélène Michel the organizational results (did the organization or the company improve its efficiency by training its employees?). We completed this model by the alternative framework of Phillips (1996) who proposes a fifth level, focusing on Return on Investment (Did the training investment pay off?) (Figure 1). Table 1: Human resource training evaluation models/frameworks (DeSimone et al., 2002) 1. Kirkpatrick (1994) 2. CIPP (Galvin, 1983) 3. CIRO (Warr et al. 1970) 4. Brinkerhoff (1987) 5. Systems approach (Bushnell, 1983) 6. Kraiger, Ford and Salas (1983)

7. Kaufman and Keller (1994) 8. Holton (1996)

9. Phillips (1996)

Four levels: Reaction, Learning, Job Behavior, and Results Four levels: Context, Input, Process, and Product Context, Input, Reaction, and Outcome Six stages: Goal Setting, Program Design, Program Implementation, Immediate Outcomes, Intermediate or Usage Outcomes, and Impacts and Worth Four sets of activities: Inputs, Process, Outputs, and Outcomes A classification scheme that specifies three categories of learning outcomes (cognitive, skill – based, affective) suggested by the literature and proposes evaluation measures appropriate for each category of outcomes Five levels: Enabling and Reaction, Acquisition, Application, Organizational Outputs, and Societal Outcomes Identifies five categories of variables and the relationships among them: Secondary Influences, Motivation Elements, Environmental Elements, Outcomes, Ability/Enabling Elements Five levels: Reaction and Planned Action, Learning, Applied Learning on the Job, Business results, Return on Investment

Figure 1: Kirkpatrick (1994) and Philips (1996) analytics

3. Methodology To analyze the different strategies developed by higher education organizations when implementing serious games, we have used the case study method. This method can be defined as an empirical inquiry that investigates a contemporary phenomenon within its real‐life context; when the boundaries between phenomenon and context are not clearly evident; and in which multiple sources of evidence are used (Yin, 1984, p. 23). This approach helps understanding a complex issue and extending experience to what is already known through literature. Through a detailed contextual analysis of a limited number of real‐life situations we expect to provide the basis of a typology of serious games implementation strategies. To do this, we have followed the classical six steps suggested by the method (Yin, 1984; Stake, 1995; Simons, 1980): Determine and define the research questions: to build our analysis framework, we have adapted the five criteria from Kirkpatrick (1994) and Philips (1996) analytics to the serious games context. To do this, we have used the literature and interview 10 experts: 5 serious games companies’ directors and 5 project managers in higher education. By analyzing the verbatim, we have identified the following adapted criteria:

Reaction: in a serious games context, this dimension is considered as the learners’ satisfaction through their immersion in the game and their motivation to participate to the process. Does the strategy enhance participants’ willingness to engage in the learning process? Does it reinforce the flow? Are the students’ appreciations of the lecture higher than usual?

Learning: in a serious games context, this dimension is described as the pedagogical pertinence: is the topic of the game in clear relevance with the learning objectives of the teacher? Is the level of the learners taken into consideration?

388

Hélène Michel

Behavior: in a serious games context, this dimension corresponds to the possibility of contextualization. What can be transferred from the game to the “real” world? Does it relies on the teacher only or is the game build in order to help the transfer from virtual to real situation?

Organization: in a serious games context, this dimension corresponds to the organizational benefits that an institution buying a game can expect. Does it rely on one person only (a teacher) or is there any collective support? Can the organization aim at positive effects on its internal communication, process quality or external image?

Return on Investment: in a serious games context, this dimension corresponds to the financial benefits. This can be analyzed in a short‐term and long‐term perspective. What are the direct costs of buying the game? What are the different possibilities concerning the copyrights and business models?

Select the cases and determine data gathering and analysis techniques: We have selected five French serious games companies, experimenting different implementation strategies with serious games in higher education 1 :

Case 1: Symetrix has developed a specific serious game on demand for a Business School

Case 2: Succubus has developed a game for the French government that is “recycled” and used by different teachers in their lecture in a Business School

Case 3: Daesign has developed a serious games store with generic simulations that are sold to a Business School

Case 4: Itycom offers an authoring tool that allows teachers in a Business School to develop their own serious game for their lecture.

Case 5: KTM Advance is working on co‐conception and co‐branding serious games with a Business School.

Prepare to collect the data: We have participated (attended the different meetings, lectures and debriefing) to all the five pilot projects during 3 years. We have built an interview guide for the 5 companies’ directors and the 5 project managers in higher education according to the 5 criteria we wanted to analyze. One year after the beginning the projects, we have made interviews to analyze the performance on each criteria. Collect data in the field: We have registered the interviews and gather all the data from the different meetings and analyzed the contracts between the different organizations. Evaluate and analyze the data: We have analyzed the content of all the verbatim. We have built a typology. Prepare the report: We have presented the typology to the different actors (serious games companies and project managers in higher education) for modification and validation.

4. Results The results describe the five cases and link each one to a specific strategy: Haute couture, Recycling Ready‐to‐ wear, Home‐made, Co‐branding. Every performance criteria related to the theoretical framework is detailed and evaluated using the signs 0=no impact; += small positive impact; ++=medium positive impact; +++=strong positive impact. The evaluation was made using the interviews of project managers in higher education.

4.1 Case 1: The haute couture strategy Context 2

Symetrix is a serious games company working mainly on demand with private customers. They developed solutions for long life learning is French major companies. They were not used to work with academic organizations but in 2012, for their first time they have developed a serious game on demand for a Business School. This school had a strong specialty on Innovation Management and wanted to have its own serious game related to this topic. They expected to develop their image towards their future learners, but also towards companies with whom they are developing different activities (consulting, long life learning, executive education). For the Business school, investing in an “Haute Couture” serious game strategy was a huge challenge.

As the Business School environment is very competitive, and as some of these projects are still in development, the business schools preferred not to be cited. http://www.symetrix.fr/

389

Hélène Michel Performance evaluation: Reaction (+++): The game was developed by experts. Therefore, the level of immersion, the game design and the game art were very high. The learners enjoyed using this training process and were really motivate and proud to participate to something unique. Learning (+++): The game was developed to reach a specific pedagogical need. The level of the learners and the context in completely integrated in the definition of the game’s content. Behavior (++): As the flow is high and the content is relevant, the learning‐doing gap can be reduced. Organization (+++): the game was a new and important challenge for the school’s team. The project manager has to build a core team. The internal communication was strong. The commitment to the project from the different internal stakeholders was high. The game had a positive impact on the image of the school related to the Innovation Management topic. Return on Investment (++): We cannot communicate the exact price of this game. However to develop this kind of serious game, the minimum cost is around 50.000 euros and the average price is around 150.000 euros. This represented a strong investment for the business school. But they gain a return on investment on two dimensions: using the copyrights that belong now to them, the school is selling the game to other programs. Then, this innovative process had a positive impact on their image and they have gained several private contracts on long life and executive education thanks to this process.

4.2 Case 2: The recycling strategy Context 3

Succubus is a serious games company working mainly on demand with private companies or public organizations. In 2010, they have developed a game for the French government to explain the entrepreneurship abilities and the different supports from the government to 4

the entrepreneurs . This game is available online, free and every citizen can experiment it. In a business school, a teacher decided to use it in the Entrepreneurship.

Performance evaluation: Reaction (+++): The game was developed by experts. Therefore, the level of immersion, the game design and the game art were very high. The learners enjoyed using this training process and but are conscious that this is not developed especially for them. Learning (+): The game was not developed on purpose for this type of seminar. So the teacher has to spend time and expertise to explain what is important, details the concepts. The learning process therefore relies totally on the way she coordinates and uses the game. Behavior (0): Even if the flow is high, the content is not exactly specific to the seminar. The contextualization relies only on the ability of the teacher to explain. Organization (0): This approach is an individual one, not an organizational. This kind of teacher, often considered as a “geek”, is lonely, often without organizational support. Even if this can be considered as innovative, the business school cannot increase a specific practice or even communicate on this as a global strategy.Return on Investment (0): The game is free. So the organization (or the teacher herself) didn’t have to negotiate any right or cost with the serious games company. But there isn’t any possibility to make financial profits using this approach .

4.3 Case 3: The ready‐to‐wear strategy Context 5

Daesign is a serious games company working mainly on demand with private companies. In 2011, it was also the first one which has developed a serious games store with generic simulations. These serious games are sold with lower price than an on demand game, through license or subscription. The average price for a one year utilization of a game in academic context is around 10.000 euros. This allows small companies or academic organizations to access these training processes. In the business school a professor was willing to experiment this training process in his seminar for sales management: With a large number of students in this class (more than 600 students split in groups of 40 students), it was almost impossible to train students face‐to‐face to specific soft skills. The business school subscribed a license to use a generic game.

http://www.succubus.fr/en/ www.macyberautoentreprise.pme.gouv.fr. 5 www.daesign.com 4

390

Hélène Michel Performance evaluation: Reaction (++): This type of games was developed by experts. Their first version was originally developed for private companies, on demand. The game design and game art is rich. Learning (+): The learning goal is clearly defined. The game was originally developed on a specific demand from private company. It therefore corresponds to professional needs. The learning content is “validated”. Behavior (+): As the game was originally developed on a specific demand from private company, it therefore corresponds to professional needs. The contextualization from virtual to real life is then possible. However the teacher has a specific role to play in order to develop a critical approach of the game. Organization (+): Buying a license involves medium costs and then implies that the organization wants to get involved in this kind of training process. The decision of the investment requires defining in which lecture, which program, for which students using this generic game. Return on Investment (+): the level of investment is low compared to « haute couture » strategy. But the main difference is the lack of copyrights from the business school on the game. If this choice seems interesting in a short term approach, in a middle and long term perspective, business schools seek new business models.

4.4 Case 4: The home‐made strategy Context 6

Since 2011, different authoring tools have been developed. Itycom is one of the first companies which have decided to offer a license to use its authoring tool: itystudio. These tools are user‐friendly in order to be used by teachers in order to create their own games, using their expertise to develop their content. In a business school, one teacher with a strong expertise on a very specific topic: mindfulness wanted to use a serious game to explain the concepts and let the learners experiment. As there were no serious games on store or to recycle on this topic and as the price for an “haute couture” approach was too high, the teacher decided to develop the game himself, using the authoring‐tool. The business school therefore subscribed a license for around 10.000 euros. And few months later the game was ready.

Performance evaluation: Reaction (+): the authoring tool helps to develop a scenario, choose avatars and context and to establish the scoring. The level of immersion is interesting but cannot compete with professional developments. Learning (+++): the content of the game is developed by the teacher. In higher education the professors are also developing research and can be considered as experts in their fields. The serious game becomes a new way to present intellectual contribution. The learning relevance is therefore high. Behavior (++): The game is built by the teacher and used later by the same teacher in his lecture. The contextualisation is considered as medium. Organization (+++): This strategy requires a strong support from the organization in order to invest in terms of finance, but also in terms of expertise and training. When the organization is willing to buy a license to develop its own serious game, it invest around 10.000 euros, but also 30 days from a teacher and time spent to train him how to use the authoring tool. This strategy is therefore interesting when the organization expected to promote in a different and innovative way its expertise. In the case of the professor teaching mindfulness, the serious game is a way to promote the development of a Chair. Return on Investment (++): Investing in an authoring tool license represents almost the same cost than investing in a generic store through a store. But the copyrights of the home‐made games belong to the business schools. So this strategy is interesting when foreseeing a middle or long term perspective. In this case, the game The Mindful Manager is now expected to become a generic game, in a store, to be sold as license to other academic organizations. 6

www.itycom.com

391

Hélène Michel

5. Case 5: The co‐branding strategy Context KTM Advance is one of the leaders of the serious games companies in France. They developed mainly on demand game for private companies. To differentiate themselves, they are willing to experiment the development of serious games collection on specific managerial and business topics. To do so, the challenge is to collaborate with academic institution which can provide specific and high quality content. Higher education organizations such as universities or accredited business schools represent therefore key stakeholders. On the other side, a business school with a strong expertise and image on the topic of innovation management is willing to gather its intellectual contributions through an innovative tool. As this is not just related to knowledge management but also to image promotion, they prefer to work with experts rather than using authoring tool to develop their serious games.

Performance evaluation: Reaction (+++): The game was developed by experts. Therefore, the level of immersion, the game design and the game art were very high. The learners enjoyed using this training process and were really motivate and proud to participate to something unique. Learning (+++): The content of the game is developed by the teacher. In higher education the professors are also developing research and can be considered as experts in their fields. The serious game becomes a new way to present intellectual contribution. The learning relevance is therefore high. Behavior (+++): As the flow is high and the content is relevant, the learning‐doing gap can be reduced. The games are co‐developed by teachers who are going to use them. So the contextualization is high. Organization (+++):, The partnership help both stakeholders improving their level of expertise. Both gain benefits from this learning curve. This innovative process is expected to have a high lever effect on the image of both organizations. Return on Investment (+++): Working in a consortium, through a partnership help reduce the cost of production, promotion and distribution of the game. Moreover, the two stakeholders manage a part of the diffusion process: the serious games company can use a store to promote the serious game while the business school can uses the academic networks. The degree of investment is lower than “Haute couture” strategy but the visibility is higher, as the expected return on investment.

6. Discussion and conclusion This article is an exploratory approach of the different strategies used in higher education institutions for implementing serious games. Through a longitudinal analysis of five case studies, this article builds a typology of five strategies to implement serious games in higher education: The haute couture strategy details how a business school ordered a specific serious game to reach its own specific needs. The recycling strategy relates how a university uses free‐to‐use serious games. The ready‐to‐wear strategy explains how a university is using a serious games store with generic games. The home‐made strategy describes how a business school experiments a serious games authoring tool. The co‐branding strategy details how a business school is developing cooperation with a serious games company in order to co‐develop and co‐brand a collection of serious games. We can summarize and compare the five strategies in a typology (Table 2). The results might help the higher education organization in their decision of serious games’ implementation. This study suffers some limitations: this exploratory approach relies on qualitative data and can be therefore submitted to interpretation. To reduce this effect the validation of the typology is made with the interviewees. To go further, a quantitative analysis to test the different strategies and their criteria performance should be made on a larger sample of higher education organizations. Table 2: The five Serious Games implementation strategies in higher education Performance Criteria Reaction Learning Behavior Organization Return on Investment

Haute couture +++ +++ ++ +++ ++

Recycling +++ + 0 0 0

392

Ready‐to wear ++ + + + +

Home made

Co‐branding

+ +++ ++ +++ ++

+++ +++ +++ +++ +++

Hélène Michel

References Brinkerhoff, R. O., (1987), Achieving results from training, Jossey‐Bass San Francisco. Bushnell, D. S. , (1990) , Input, process, output: A model for evaluating training, Training and Development Journal, 44(3), 41‐43 Caillois, R., (1957), Les jeux et les hommes, Gallimard Paris. Csikszentmihalyi, M. , (1990), Flow: The psychology of optimal experience, Harper and Row New York. De Freitas, S. , (2006), Learning in immersive worlds: A review of game‐based learning, Technical report, JISC e‐Learning Programme. DeSimone, R.L. , (2002) Werner, J.M, Harris, D.M, Human resource development, (3rd ed), Harcourt College Publishers Orlando. Galvin, J. C. , (1983), What can trainers learn from educators about evaluating management training? Training and Development Journal, 37(8), 52–57. Gee, J. , (2007), Good video games and good learning: collected essays on video games, learning, and literacy, Peter Lang Publishing Inc, New York. Holton, E.F. , (1996), The flawed four‐level evaluation model, Human Resource Development Quarterly, 7 (1), 5‐21. Huizinga, J. (1955), Homo Ludens, A study of the play element in culture, Beacon Press, Boston. Kaufman, R.; Keller, J. M. (1994), Levels of evaluation: Beyond Kirkpatrick, Human Resource Development Quarterly, 5 (4), 371‐380. Kearney, P. R. , (2006) , Immersive environments: What can we learn from commercial computer games?, In M. Pivec (Ed.), Affective and emotional aspects of human‐computer interaction: Emphasis on game‐based and innovative learning approaches, Amsterdam: IOS Press BV. Kirkpatrick, D.L., (1994) , Evaluating Training Programs: The Four Levels, Berrett‐Koehler, San Francisco CA. Kraiger, K. , Ford, K. , Salas, E., (1993), Application of Cognitive, Skill‐Based and Affective Theories of Learning Outcomes to the New Methods of Training Evaluation, Journal of Applied Psychology, 78 (2), 311‐328. Michael, D. , Chen, S. , (2006) Serious games: games that educate, train, and inform, Thomson Course Technology, Tampa, FL . Novak, T.P. , Hoffman, D., Yung, Y. , (2000), Measuring the consumer experience in online environments: a structural modeling approach, Marketing Science, 19, (1), 22‐42. Petty, R. E. , Cacciopo, J. T. , (1984), The effects of involvement on responses to argument quantity and quality: Central and peripheral routes to persuasion, Journal of Personality and Social Psychology, 46, (1), 69‐81. Pivec, M., Pivec, P., (2009) What do we know from research about the use of games in education, in European Schoolnet , « How are digital games used in schools? » 122‐165 Prensky, M., (2001), Digital game‐based learning, McGraw‐Hill New York. Phillips, J.J. , (1996), How Much Is the Training Worth? Training and Development, 50(4), 20‐24. Rheingold, H. , (1993), The virtual community: homesteading on the electronic frontier, Addison‐Wesley, New‐York. Ritzer, G. , (1999), Enchanting a disenchanted world: revolutionizing the means of consumption, Pine Forge Press, Thousand Oaks. Sawyer, B. , (2002), Serious games: improving public policy through game based learning and simulation, Foresight and Governance Project, Woodrow Wilson International Center for Scholars, Simons, H. (1980), Towards a science of the singular: Essays about case study in educational research and evaluation. Norwich, UK: University of East Anglia, Centre for Applied Research in Education. Stake, R. E. (1995). The art of case study research. Thousand Oaks, CA: Sage. Thorndike, E. , (1932) ,The fundamentals of learning, Teachers College, New York. Vandeventer, S. S. White, J. A. , (2002), Expert Behavior in Children’s Video Game Play, Simulation & Gaming, 33(1), 28‐ 48.7. Vorderer, P. , Ritterfeld, U., (2009), Digital games, The SAGE handbook of media processes and effects, Thousand Oaks, CA, 455‐467,. Warr, P. Bird M. , Rackham, N. , (1970), Evaluation of management training, Gower Press, London,. Yin, R. K. (1984). Case study research: Design and methods. Newbury Park, CA: Sage. Zyda, M. , (2005), From Visual Simulation to Virtual Reality to Games, Computer, 38(9), 25‐32.

393

Motivation and Manipulation: A Gamification Approach to Influencing Undergraduate Attitudes in Computing Nicholas Mitchell, Nicky Danino and Lesley May University of Central Lancashire, Preston, UK NPMitchell@uclan.ac.uk NDanino@uclan.ac.uk LMay@uclan.ac.uk Abstract: This paper describes how the introduction of competitive elements to an introductory undergraduate module in Computing at the University of Central Lancashire (UCLan) enabled the teaching team to motivate, engage, and influence the behaviour and expectations of new students. The Four Week Challenge (4WC) forms the first module that all students encounter on a number of different Computing courses. It is run in full‐time “burst mode” during the students’ first month at UCLan, with the start of regular teaching delayed until after this module has finished. It is designed to lead students through a challenging (yet highly scaffolded) project to show them where their course could take them. The students work in teams of six on a series of incremental challenges. The ultimate goal for teams on the module is to build a sophisticated mobile phone game, market it on‐line, and present it in an academic context. Moreover, the module itself is run as a game, with teams competing against each other not only to build the best game, but also to be the most effective team. Important in getting the students to embrace the idea of the module as a game was instilling a belief that the teams were fairly matched, and that each had an equal chance of success. To this end we devised a method of sorting students into balanced teams based on: a) their chosen course within Computing; b) their preferred team role; and c) their existing competence at computer programming. The challenges in each week follow themes within computing, giving the opportunity for different individuals in each team to come to the fore throughout the module according to their interest. Points were awarded to each team on a daily basis for various activities, and a running total displayed as a Leader Board in the foyer of the Computing building. This public and regularly updated display fuelled a strong sense of competition between the teams, motivating them to work harder and achieve better results. Since the 4WC is an assessed module (10 ECTS credits) both output and teamwork are graded each week. As well as contributing to the teams’ final grade for the module, these marks are also translated into points for the Leader Board. Besides academic learning outcomes, the 4WC has been designed to address issues of retention and engagement in Computing at UCLan. One aim is to foster a culture of peer‐support, where no student would feel isolated on the course. Specific activities where points are awarded only if all members of the team make a defined contribution encourage the stronger team‐members to support the less experienced. Again with the promise of points, students are encouraged to share knowledge and techniques. In presentations, students are supportive of other teams, asking questions and offering advice. By the end of the month are publishing their own tutorials and running their own help classes. An awards ceremony is held at the end of the module, with prizes for the overall winning team, and also in several other categories. Keywords: gamification, motivation, engagement, computing

1. Introduction The manner in which learning in higher education is understood has considerably altered since the last century. Teaching styles and approaches have progressed through several different phases since their dependence on the early behaviourist studies of Pressey (1922) and Skinner (1958). Recently, the constructivist paradigm has been applied to define the learning that takes place in the classroom (Cooper, 1993). One of the chief theorists in the constructivist approach, Bruner (1996), suggests that learning is an active and social process, in which students create fresh notions or ideas, founded on their existing knowledge. Bruner’s theory of constructivism lies in the cognitive domain. Students, via the use of inquiry and experience in learning, are thought of as designers and thinkers. Consequently, educators have to re‐evaluate both the approach of their delivery and their method of assessment, given that education has been experiencing a paradigm change away from teaching‐as‐instruction towards student‐centred learning (Jonassen, 1993; Ramsden, 1992). Honebein (1996) offers various pedagogic goals of the design of constructivist education environments, claiming that said environments should aim to encourage learners to take responsibility for their own learning. Purposely, learners need to be supported through the process of becoming self‐aware of the learning process, by carrying out relevant learning based on real‐life actions.

394

Nicholas Mitchell, Nicky Danino and Lesley May

1.1 The four week challenge This study is set in the context of the Computing first year at the University of Central Lancashire (UCLan). This paper describes and analyses the effectiveness of curriculum design change within the 1st year Computing undergraduate cohort, specifically, how the introduction of competitive elements enabled the teaching team to motivate, engage, and influence the behaviour and expectations of new students. Computing courses at UCLan have traditionally been delivered as six long thin modules over the academic year. The structure of delivery of the 1st year has been redesigned such that the Computing Skills module (now known as The Four Week Challenge) is delivered full‐time over the first four weeks of the semester, acting as an introduction to university life, and the course in general. The other five modules are delivered concurrently over the subsequent academic year, and each builds upon themes introduced in the first module. 1st Year Computing students at the University of Central Lancashire (UCLan) come from all manner of academic backgrounds. Many have studied either Computing or IT at school or college, whilst others have not undertaken any formal qualifications in the subject. Computing is run as a common 1st Year, with entry requirements of 280 UCAS tariff points at A2 or BTEC National Diploma MMM‐DMM AND 5 GCSEs at grade C or above including Maths and English. Students study the 1st year to gain a broad grounding in Computing, and when they progress to year 2 they choose a specialism. These specialisms range from Computer Games Development, Computer Network Technology, Information Systems, Forensic Computing, Multimedia Development, Software Engineering, and Applied Computing, the latter offering a more flexible programme of study, with modules chosen by the student from any area. Comments from students who leave computing courses consistently point to a lack of understanding of what their course is about until too late in the year, when they slowly disengage as they realise it is “not the course for them”. They also remark that they find programming boring and not relevant to their specific course choice from the second year on. Students want to start university and dive straight into the ‘fun’ stuff. The motivation for this study was to improve the student experience generally, whilst specifically targeting issues surrounding student engagement and retention in the 1st year, and to help students make the transition from school to university so that they are better prepared to enter the second year of their degree. Part of this involves a need to build a strong social network, and part of it involves the need to get a flavour of the course in its totality, as soon as possible. Throughout the Four Week Challenge, students work in teams of six on a series of incremental challenges. The ultimate goal for teams on the module is to build a sophisticated mobile phone game, market it online, and present it in an academic context. Moreover, the module itself is run as a game, with teams competing against each other, not only to build the best game, but also to be the most effective team.

2. Gamification in the four week challenge classroom Presky (2007) attempts to validate why humans engage in games. He suggests that the key structures of games can be classified into six key categories: rules, goals and objectives, outcomes and feedback, competition, interaction, and representation. Using Presky’s definition, a game varies from a simulation in that a game is fundamentally motivating and includes competition. Corcoran (2010) highlights that gamification includes “providing instantaneous feedback, egging on the competition and rewarding even tiny steps of progress. Gamification assumes that the player isn't especially motivated ‐ at least at the beginning ‐ and then provides barrels of incentives to ramp up that motivation.” The approach we employed within the 4WC directly draws on these two elements of competition and reward highlighted by Corcoran (2010), using them directly to encourage specific behaviours in our students.

2.1 Motivation and engagement While the prospect of marks counting towards passing a module can, and does, motivate students to undertake academic exercises, some of the explicit and implicit aims of this particular module could not be translated into Learning Outcomes for assessment in a traditional way. Rather, these particular aims

395

Nicholas Mitchell, Nicky Danino and Lesley May emphasise supporting the students in making the transition from school or college to university during their critical first few weeks. Essentially we want them to learn what it means to be a successful student, specifically in Computing at UCLan. These aims fall into two broad categories. The first aligns with our desire to instil the correct learning discipline among students: setting expectations about the level and intensity of work, and promoting an independent learning culture which is quite different from that which most of our intake have previously experienced. The second concerns the social adjustment to university life: encouraging the students to make friends and form a strong support network among their peers. Our previous experience is that computing students in particular can easily become isolated at university, and when difficulties (either academic or personal) arise it is most often those without a network of friends who tend to disengage from their studies. The strategy of running the module as a game, therefore, allows us to build into the day‐to‐day tasks, a system of rewards for students engaging in activities which support these behavioural and social outcomes. This is done alongside a traditional system of marks for activities which directly asses the students’ competencies against the academic learning outcomes.

2.2 Implementing gamification The module is overtly run as a team based competition, with students working in teams of six. The aim of the competition is to accrue the most points over the course of the module, and a trophy is awarded to the winning team at the end. To maintain momentum over the entire four week period, an electronic leader‐board is installed in the foyer of the computing building, and the teams’ scores updated on a daily basis. This provides rapid feedback to the teams, and by making the progress immediately visible, creates a highly competitive environment. The awarding of points is the direct reward that students receive for completing specific tasks designed to support the modules aims. The aforementioned tasks are set in the context of a substantial team project, the primary aim of which is to expose the students to a wide range of topics in computing, and to give them an appreciation of the different directions their degree could take them, depending on their choice of specialism from the second year on. Through a series of incremental challenges given out each week, the project leads the team through the design, development, on‐line marketing and deployment of a relatively sophisticated, interactive mobile app – itself an interesting and intrinsically motivating product. Every activity, from lectures and tutorials, down to assessment activities, is designed to directly support the completion of the project, so that students are more likely to remain engaged at all times. The students are required to write programs, build content‐rich websites and use social media effectively, understand how to connect their app to a database using a communication protocol over a network, design user interfaces, give presentations and write academically about their product. The teaching is structured so that each of the four weeks follows a similar pattern: at the beginning of the week lectures introduce a new topic, and structured worksheets guide students through the basics in timetabled labs, so that all students (regardless of their specialism) have a grounding each topic. The first points of the week are awarded for each student who completes the worksheet. It is in the team’s interest that all team members complete the worksheet so that the maximum points can be scored for the team, and the onus placed is on the stronger members of the team to help the struggling members. In this situation, both parties benefit, and the social bonds within the team are strengthened. No student is allowed to sit in the corner and do nothing. As the week progresses, the practical tasks become more open ended and diverse, and the team must decide how to divide work packages between themselves. As the weeks go on, some themes continue from previous weeks, and new ones are added, so different roles and responsibilities within the team (both technically, and organisationally) will come to the fore at different times. Rollings and Adams (2003) discuss choice (in a non‐ trivial sense) as a core mechanic of game design. It is argued that a game (as distinct from a competition) involves an element of choice that can affect the ultimate outcome. In this sense, the “game” of the four week challenge becomes a much more interesting experience as teams are allocating finite resources to meet an increasingly complex weekly challenge.

396

Nicholas Mitchell, Nicky Danino and Lesley May To support the team, a short mid‐week stand‐up meeting is held between two tutors and each team to discuss both their progress and strategy. Points are awarded for each team’s organisation and success, and feedback is given verbally. Teams where a subset of members are doing all the work and excluding weaker members would not gain points at this juncture, therefore efforts to maintain good communication and peer inclusion are further encouraged. If a team appears to be heading for disaster, the teaching team can intervene and give extra support and guidance as required. In order to discourage any underhanded tactics, and also to widen their social networks, students are encouraged to help each other outside of their own teams. Points are available during the week for teams whose members help others. This includes providing good answers to questions on social media and helping other teams technically in the labs. On the Friday of each week, teams present their achievement in front of other teams in one of several parallel “Show and tell” sessions. These sessions are both a celebration of success and a chance to share discoveries between teams. Points are awarded for successfully meeting the challenges for the week, for presentation, and also for asking interesting questions or making suggestions from the floor after other teams have presented. The final Friday expands this concept, and the teams present an academic poster at a mini‐ symposium during the morning. As well as being assessed by the teaching team, posters and presentations are peer‐assessed by other teams for points. In the afternoon, as a final challenge the teams engage in a game‐ within‐a‐game. The product they have built has developed over the weeks into an interactive location based treasure hunt app, which the teams use to find treasure around the city of Preston. The better their app, the faster they can find the treasure, and a final set of points is transferred to the leader‐board.

2.3 Separating assessment and reward When designing the Four Week Challenge, we felt that is was necessary to separate the competition from the formal assessment of the module, so that members of teams at the bottom of the leader‐board did not feel that they were precluded from achieving a good individual score for the module. Whilst most of the activities for which points are scored simply reward positive “good student‐like” behaviour, some points are given for tangible achievement and teamwork, both of which form part of the module learning outcomes. Consequently, these points are taken forward and a formula applied to form a team mark. An element of peer evaluation has been introduced within each team, whereby team members can place value on their own and each other’s contribution to the team’s achievement by distributing a fictional 1,000 UCLan Dollars between the team members as wages. This review is carried out individually in secret, and the results averaged across team members. The team mark is then adjusted up or down according to a formula to become an individual mark for each student. The components of the individual marks derived from the team stand‐up meetings are only awarded if the person was present at the meetings. In the week following the end of the Four Week Challenge, students must also complete an individual reflective report on one aspect of their experiences, and relating to their choice of subject specialism. This forms the second and final component of individual assessment.

3. Creating competitive teams Literature indicates that team size affects team performance. Both in scientific research (Tunzelmann et al., 2003) as well as in empirical work (Hoegl, 2005), a relation is established concerning team size and performance. An archetypal conclusion is that in the sciences around five to nine individuals is an ideal team size (Qurashi, 1993). For the 4WC, in order to create a competitive element that was both fair and balanced, students were strategically engineered into teams of six. Important in getting the students to embrace the idea of the module as a game was instilling a belief that the teams were fairly matched, and that each had an equal chance of success. To this end we devised a method of sorting students into balanced teams based on: a) their chosen course within Computing; b) their preferred team role; and c) their existing competence at computer programming. There were three options initially considered for how teams were created. Option one was to allow the students to pick their own teams. This was discounted as it went against the teaching team’s desire to help students make friends and form social groups. Option two was to select teams based on degree course. This

397

Nicholas Mitchell, Nicky Danino and Lesley May seemed attractive, as it would help the course groups to bond and form a strong identity. It was also a seemingly straightforward task that required little time and effort on behalf of the teaching team. On the other hand, it wouldn’t help students who were unsure about the course they had chosen (another of the reasons for doing the 4WC) and might lead to teams focussing on one aspect of the challenge to the exclusion of all else – reinforcing the ‘everything but X is irrelevant’ belief we were keen to dispel. Option three was to select teams that were inter‐disciplinary. This was ultimately the preferred option as it was thought that with a ‘balanced’ team, each member would be able to contribute something of their specialism at different points during the challenge – increasing the likelihood of success. The staff spent a long time discussing the team structure, and how each structure would affect not only student activity, but also teaching practice and staff motivation. This is an example of the use of the Johns’ (2000) model of reflection used in the design of the curriculum. Although, as with all forms of reflection, this approach is couched in constructivism (Moon, 2004).

3.1 Team roles Previous experience shows that students report team working as the thing they like most and hate most about their degree experience. Although they see the benefits (Tsay and Brady, 2010), lots of things can go wrong, causing the team to break down. Payne et. al. (2006) aimed to identify weak elements of student group work. However, as the teaching team didn’t know the students very well, they had little to go on other than the chosen computing specialism of each student. It was decided that it would be desirable to look at other things as well, so that the groups had a balance of interests that would see them though the 4WC activities, as well as a balance of personalities that would make it easier for the group to function as a team – some leadership, some technical expertise, and so forth. Gati et. al. (2010) argue that profiles are important in career decisions, so the teaching team decided to try and profile each student. It was decided that Belbin’s team roles could be used for inspiration on helping to sort students into their different group functions. According to Belbin (2004), each person can be characterised by nine role types. Belbin provided a clear insight into the internal group relationships and the clarification of the roles needed for a team to work efficiently. The resulting teams are called balanced teams. Rather than the time‐consuming orthodox Belbin analysis, something more “lightweight” was required, and we devised an approach based on the descriptions of each of Belbin’s team roles (excluding the Specialist role), that could be administered electronically as a survey immediately before the start of the 4WC. We compressed the full range of roles into three categories, such that each team had someone who could play a leadership role, someone capable of thinking problems through, and someone who could get on with things.

3.2 The programming problem Connolly et. al. (2009) describes a longitudinal research study that investigates the variance of anxiety amongst undergraduate computing students, with specific emphasis upon their learning programming during their first year in higher education. According to Connolly, low retention rates in computing courses present a worrying concern. For some computing students, learning programming is intimidating, and causes a lack of confidence and anxiety. From a constructivist point of view, the lecturer’s role is to ensure that ‘alignment’ happens, which includes creating an education setting that fosters the learning undertakings suitable to attaining the anticipated learning outcomes. Alignment is dependent on consideration being given to establishing clear learning outcomes, teaching methods, assessment procedures, an atmosphere encouraging to student/teacher communication and a sympathetic organisational environment (Biggs, 1996). The curriculum had to be designed so that programming was introduced in such a way that did not appear intimidating or cause students to immediately worry. The 4WC includes a gentle introduction to programming using AppInventor. A small number of students arrive with significant programming experience, and it was thought desirable to distribute these students as technical experts within the teams, to ensure each team had a chance of tackling the more awkward programming challenges. A question was added to the team role questionnaire, with a scale of responses to measure how comfortable and experienced the student was with programming. This was designed to replace Belbin’s role of the Specialist. From the response to this question, those who indicated an existing aptitude were marked as such.

398

Nicholas Mitchell, Nicky Danino and Lesley May The students were formed into teams the next morning, and immediately set to an ice‐breaking activity. In the afternoon, teamworking was discussed formally, though not in depth. Hartley (1997, p104) argues that we should not be teaching these theories to students, but instead we should be “enabling our students to develop their own critical enquiry into the nature and processes of project groups”.

4. Outcomes The Four Week Challenge in Computing at the University of Central Lancashire (UCLan) came about in response to concern over the number of undergraduate students leaving their course before completion during their first and second years. Our new curriculum aimed to address this retention problem by engaging students from the start of their course, and giving them realistic expectations about their course concerning both the academic content and working culture. Also, we provided a framework in which we believed we could foster a strong peer support network. Given the emphasis we have placed on the social transition to university, it is important to consider whether the gamification approach has had the desired impact in addressing these key issues. However, the module has only been running for 2 years, and consequently has not yet seen its first cohort of students complete their degrees. As such, any such assessment of success can only be preliminary and very cautiously undertaken. Although overall retention in the first year has not significantly changed, last date of attendance data reveals that those who leave the course do so much earlier in the year, suggesting that the 4WC helps students to better assess whether their chosen course is the right one for them. Furthermore, after the first year of the 4WC, average exam for the cohort were significantly higher. Most notably, the perennial large cluster of marks just above the pass grade was no longer present. We surveyed the students using on‐line questionnaires immediately before the start of the 4WC and again afterwards. When asked about their fears on coming to university, four issues were raised consistently: financial concerns, worries about making friends, fear of academic failure, and loss of immediate family support. After the 4WC, students reported that where these fears were still present, they were much less worried about them. Building a peer support network has been very important in preventing isolation due to the absence of friends and family when new to University. Our strategy of rewarding students for helping each other in various contexts appears to have worked. By the end of the challenge, students were creating their own tutorial worksheets and posting them on‐line. One team even ran “out of hours” drop‐in sessions for their fellow students. In a post‐4WC survey, 66% of respondents who said that they needed some help during the challenge also said that the sometimes or always received it from students outside their own team, while 84% sometimes or always received it from within their own team. Lasting friendships appear to have been made, with many students reporting at the end of the year that they are still in regular social contact with some or all of their teammates from the 4WC. When asked to place aspects of the 4WC in order, around a third of respondents said that making friends and getting used to university life as the most important objective of the 4WC at its outset, granting it greater importance than the grade they achieved in its assessment. Most students seemed pleased with the care we had put in to selecting their teams, and significantly, all respondents to the survey indicated that they thought their team had as good a chance as any of success at the start of the 4WC. 70% thought they were well balanced. During the second time the module was run, some teams decided to “play the system” in the first week by putting on a show of harmony at the weekly stand‐up meetings in order to score more points. This was a tactical error, because the teams involved failed to get the tutor advice and support they really needed, and as a result lost the points they had gained by not doing as well as they might have done in the show and tell at the end of the week. From the second week on, we introduced a variation of the peer assessment where the team had to distribute 1,000 UCLan Dollars as wages for the week amongst themselves immediately prior to the meeting. This time, however, the distribution was done together as a team, and every member had to sign it. Whilst such a strategy might be considered provocative, the vast majority of students saw it as a positive initiative that brought issues into the open during the team discussion with the tutors that could then be quickly resolved rather than left to fester. The element of competition was built into the 4WC in order to motivate and engage the students. Responses to the survey indicate that, from the student perspective at least, this was achieved to a high degree.

399

Nicholas Mitchell, Nicky Danino and Lesley May Responses to the survey indicated that the presence of the leader‐board fuelled a strong sense of competition, with 69% of students consulting it at least once a day. 77% of respondents agreed that its presence motivated them to work harder than they might otherwise have done. One student commented that seeing his team move up and down the leader‐board was like seeing his graded go up and down each day, and this made him increase the effort he put in. All respondents said they enjoyed the competitive element of the 4WC, and also that the module was fun. The Theory of Constructive Alignment (Biggs, 1996) is highlighted as offering an explanation of how the 4WC meets some of the challenges we face in engaging students in higher education. Biggs’ Theory of Constructive Alignment suggests that, if any actual learning is to happen, that student characteristics, aims and actions must be consistent with those of the teacher‐constructed learning environment.

References Belbin, M. (2004) Management Teams: Why They Succeed or Fail, 2nd ed. Oxford: Butterworth‐Heinemann. Biggs, J.B. (1996) Enhancing teaching through constructive alignment, Higher Education, Vol. 32, No. 3, pp 347‐364. Bruner, J. S. (1966) Toward a Theory of Instruction, Oxford: Oxford University Press. Connolly, C. Murphy, E., Moore, S. (2009) Programming Anxiety Amongst Computing Students—A Key in the Retention Debate? IEEE Transactions on Education, Vol. 52, No. 1, p52‐56. Cooper, P. A. (1993) Paradigm shifts in designed instruction: from behaviorism to cognitivism to constructivism, Educational Technology, Vol. 33, No.5, pp 12–19. Corcoran, E. (2010) The ‘gamification’ of education. [online] http://www.forbes.com/2010/10/28/education‐internet‐ scratch‐technology‐gamification_2.html [retrieved 25th April 2013]. Gati, I. et al (2010). From career decision‐making styles to career decision‐making profiles: A multidimensional approach, Journal of Vocational Behavior, Vol. 76, No. 2, pp 277‐291. Hoegl, M. (2005) Smaller teams‐better teamwork: how to keep project teams small. Bus. Horiz., Vol. 48, pp 209–214. Honebein, P. C. (1996) Seven goals for the design of constructivist learning environments. In Wilson, B. G. (Ed.) Constructivist Learning Environments: Case Studies in Instructional Design. Englewood Cliffs, NJ: Educational Technology Publications. Johns, C. (2000) Becoming a reflective practitioner, Oxford: Blackwell Science. Jonassen, D. H. (1993) Thinking technology: context is everything. Educational Technology, Vol. 31, No. 6, pp 35–37. Moon, J. (2004) A Handbook of Reflective and Experiential Learning. London, Routledge Falmer. Prensky, M (2001) Digital Game‐based Learning New York: McGraw‐Hill. Pressey, S., and Pressey, L. (1922) Introduction to the Use of Standard Tests. Yonkers‐on‐Hudson, NY: World Book. Qurashi, M. M. (1993) Dependence of publication‐rate on size of some university groups and departments in UK and Greece in comparison with NCI, Scientometrics, Vol. 27, pp 19–38. Ramsden, P. (1992) Learning to Teach in Higher Education. London: Routledge. Skinner, B. F. (1958) Teaching machines. Science, Vol. 128, No. 3330, pp 969‐977. Rollings, A., Adams, E. (2003) On Game Design, New Riders. Tsay, M., & Brady, M. (2010) A Case Study of Cooperative Learning and Communication Pedagogy: Does Working in Teams Make a Difference? Journal Of The Scholarship Of Teaching And Learning, Vol. 10, No. 2, pp 78‐89. Tunzelmann, M. R., Martin, B. Geuna, A. (2003) The effects of size on research performance: a SPRU review, Report Prepared for the Office of Science and Technology, Department of Trade and Industry. Brighton: SPRU‐Science and Technology Policy Research, The Freeman Centre, University of Sussex.

400