Senior University Teaching Qualification by University of Twente

SENIOR UNIVERSITY TEACHING QUALIFICATION

SENIOR UNIVERSITY TEACHING QUALIFICATION UNIVERSITY OF TWENTE PROJECTS OF 2017-2018

SENIOR UNIVERSITY TEACHING QUALIFICATION

THIS BOOKLET CONTAINS THE PRODUCTS OF THE FIRST GROUP OF TEACHERS WHAT HAVE BEEN WORKING ON THEIR SENIOR UNIVERSITY TEACHING QUALIFICATION. THE SENIOR UNIVERSITY TEACHING QUALIFICATION The SUTQ is based on the SoTL approach; the Scholarship of Teaching and Learning. Participants (senior teachers) do research and/or design innovative education within their own teaching practice. For about one year participants worked 160 hours on their individual educational (SUTQ) question. Within the SUTQ participants determined their own personal learning path (based on the Student - Driven Learning principle). In order to gather all the necessary information to answer the education question, participants received facilitations, support and feedback from the Centre of Expertise and Learning and Teaching (CELT) and Centre of Engineering Education (CEE).

SUTQ IMPROVES EDUCATION The University of Twente educates the professionals of tomorrow. To be able to provide a high standard of education, the UT emphasizes the importance of highly skilled teaching staff. We stimulate teaching excellence by implementing a wide range of activities. SUTQ is designed for experienced teachers who achieved their UTQ (or exemption) and have the ambition to improve the/their education. The UTQ and SUTQ are embedded strongly in our HR policy of life long learning. The UTQ and SUTQ programmes are coordinated by the Centre of Expertise in Learning and Teaching (CELT). www.utwente.nl/sutq

SENIOR UNIVERSITY TEACHING QUALIFICATION

A Hybrid Test for Mathe

A HYBRID TEST FOR MATHEMATICS Introduction

Examples of exerc

Nowadays electronic testing packages offer excellent opportunities for students for practicing mathematics. However, traditionally mathematics exams still consist of merely open INTRODUCTION exercises. Nowadays electronic testing packages offer

excellent opportunities for students for practicing QuestionHowever, traditionally mathematics mathematics. exams still consist of merely open exercises.

To what extent can traditional summative

SUTQ QUESTION matics tests be transformed into mathe

To what extent can traditional summative g validity and t harmin digital tests, withou mathematics tests be transformed into digital reliability? tests, without harming validity and reliability?

Solution SOLUTION Compose a hybrid test containing both open (pen ning both contaiexercises. testanswer) hybrid andComp paper)ose and aclosed (final

open (pen and paper) and closed (final answer) exercises.

EXAMPLES OF TRADITIONAL EXERCISES

Examples of traditional exercises:

HARRY AARTS Harry Aarts (1960) studied mathematics at the Radboud University in Nijmegen. After receiving his Masterâ&#x20AC;&#x2122;s degree, he did his PhD in game theory at the University of Twente. From 1994 onwards he has been a lecturer at the department of Applied Mathematics of this university, except for a four year period at the Hogeschool Larenstein in Arnhem. At the University of Twente he teaches Calculus, Linear Algebra, Discrete Mathematics , Analysis, Abstract Algebra and Graph Theory to students of several programs.

Hybrid Test for Mathematics; Harry Aarts lts: form: rry Aarts Resu in close ; dHa isestics exerc ma Maofthe for ples TestOFExam bridEXAMPLES Pass rate: 76.5 % EXERCISES IN CLOSED FORM (comparable to traditional tests) Hybrid Test for Mathematics; Harry Aarts Results: Examples of exercises in closed form: Pass rate: 76.5 % ybrid Test for Mathematics; Harry Aarts to traditional tests) arable (compRESULTS Results: Examples of exercises in closed form: SENIOR UNIVERSITY TEACHING QUALIFICATION

Pass rate: 76.5 % (comparable to Pass rate: 76.5 % traditional tests)

(comparable to traditional tests) Resu lts:

Examples of exercises in closed form:

Pass rate: 76.5 % (comparable to traditional tests)

Lessons learned: • No complex exercises in closed form.

ed:ts with sufficient representative ns learn Lesso e studen • Provid

LESSONS LEARNED • No complex exercises in closed form. • Provide students with sufficient representative practicing material. • Not all educational targets are suitable to be tested with closed exercises. • The test package itself might have unexpected technical restrictions as well.

practicing material. form. closedare ses in suitable to be exerci targets • No• compl tional Lessons learned: Not allexeduca entative repres nt sufficie with ts e studen • Provid with closed exercises. tested unexpected al. haveform. materi ing practic ge itself might packa test • •The No complex exercises in closed Lesso ns learn ed: to be le suitab are targets tional well. educa as tions • Not •all restric cal students with sufficient representative techni Provide exercises. with closed testedpracticing material. • No Follo complw ex up: exerci ses closed form. ected have might ge itself in packa • The•test Not all educational targets areunexp suitable to be • Provid e studen ts with sufficie nt repres entativ e well. as exercises. cal restric technitested withtions closed practic ing materi matics al. mathe other to tests ion hybrid Extens •allThe testtional package itself might have unexpected • Follo Notcourse up: weduca targets are suitable to be s. restrictions as well. testedtechnical with closed exercises. matics • Extens The test mathe otherhave getests itselftomight hybrid ionpacka unexp Follow up: ected techni cal restric tions as well. courses. FolloExtension w up: hybrid tests to other mathematics courses. Extension hybrid tests to other mathematics courses.

FOLLOW UP Extension hybrid tests to other mathematics courses.

MY EXPERIENCE WITH THE SUTQ WHAT MY SUTQ PROJECT WAS ABOUT My SUTQ project concerns a study to digital testing of Mathematics at an academic level. I designed a hybrid test (partially open, partially closed exercises) for one of the Calculus courses at the University of Twente.

WHAT I HAVE LEARNED In this study I learned what pitfalls come across when designing closed exercises for a mathematics test and how to avoid them.

WHAT WAS THE BIGGEST CHALLENGE The biggest challenge in this project was to convince my collegue-mathematicians that a valid and reliable test for a mathematics course, mainly consisting of closed exercises (only the final answer is graded), could be designed at all.

WHAT ARE YOU PROUD OF I am proud of the final result: the hybrid test was subjected to almost 500 students of various programs and their scores were analyzed. Regarding validity and reliability, the conclusion is that this test is comparable to the traditional pen and paper tests of this course.

SENIOR UNIVERSITY TEACHING QUALIFICATION

IS STUDENT CENTRED TEACHING ALWAYS FAIR? EFFECT OF CULTURE ON THE OPINION TOWARDS STUDENT CENTRED LEARNING BY INTERNATIONAL STUDENTS BACKGROUND

HYPOTHESIS

• Student centred teaching is becoming more popular as the initiative and ownership of the learning is placed with the student and not the teacher. • This should stimulate the intrinsic motivation of students to participate in the learning. • When students from a cultural background where own initiative is less valued are subjected to student centred learning this might have ad-verse effects to their learning.

Students that are classified as hierarchists would have a less favorable opinion to SCL that students that are classified otherwise.

RESULTS

THEORETICAL MODEL Hierarchist Fatalist

Individualist Egalitarian

DATA

Data collection on: # perception # opinion # readiness

}

~ 27 students ~ 14 nationalities ~ Questionnaires ~ Open question on student centred teaching ~ Focus group discussion with students

Egalitarian student profiles

QUESTIONNAIRE Perceptions of students (likert scale rating) towards activated learning, with respect to: • Initiative (who should take initiative?) • Benchmark (who should decide what to learn?) • Learning goals (how does activating teaching affect your performance?) • Impact (how does activating teaching affect your performance?) • Interaction (how should you interact with students and staff?) Hierarchist student profiles

SENIOR UNIVERSITY TEACHING QUALIFICATION

THOMAS GROEN CONCLUSIONS • Most students in sample seem hierarchists • Overall sample size too small for real quantitative analyses, mainly qualitative results. • Overall positive opinions regarding SCL suggesting to falsify the hypothesis • Students seem ready for SCL as well • Little change in opinion over time • Open questions and discussion revealed suggestions regarding better organizing SCL including: - more formative feedback - more formal meetings with teachers during SCL projects - better formulation of assignments - better balance between group and individual assignments - more control in group formation by teachers

Thomas Groen is Assistant Professor at the department of Natural resources of the Faculty of Geo-Information Science and Earth Observation (ITC) at the University of Twente. He is active in teaching for Natural Resources Msc specialisation of the Faculty’s Master programme, and also active in the distance based iGEON course which is offered in cooperation with Lund University. He specialized in tropical ecology at Wageningen university, and has a specific interest in ecosystem modelling, spatial modelling and remote sensing.

MY EXPERIENCE WITH THE SUTQ WHAT MY SUTQ PROJECT WAS ABOUT My research was about identifying the opinion of students from different cultural backgrounds towards a student centred learning approach. The hypothesis was that students with a more hierarchical background would have a less favourable opinion towards student centred learning than student from more individualistic background.

WHAT I HAVE LEARNED The study showed that most students that participated in the survey could be classified as having an hierarchical background. Nevertheless, their readiness and acceptance of student centred learning was quite high, seemingly falsifying the initial hypothesis. The group size, however, was small, so a follow up study is currently prepared to more rigorously test this finding.

WHAT WAS THE BIGGEST CHALLENGE This was the first time I conducted qualitative research in the form of open-ended questions in a questionnaire, as well as focus group interviews. This was unexplored territory for me and taught me a lot about surveying with these kinds of methods.

WHAT ARE YOU PROUD OF I believe that the very honest and useful feedback that I acquired through the various questionnaires that the students filled in have helped me to become a better teacher, because they forced me to critically reflect on my own way of implementing student centered learning. I’m proud that I took on this challenge, and that there will probably be a follow up.

SENIOR UNIVERSITY TEACHING QUALIFICATION

TOWARDS PRACTICAL AND EFFECTIVE VIDEO INTERVISION ROUTINES CONTEXT

PARTICIPANTS

In teacher training, intervision is an important component. The goal of intervision is to achieve professional development by discussing concrete cases from teaching practice. Traditionally, one of the students brings in a case orally, that is discussed by means of a certain intervision method. This rarely produces a complete picture. Use of video offers a less subjective perception. The goal of this study is taking a first step towards practical and effective video intervision routines for preservice teachers.

Traditional intervision - 6 economics students, 1 teacher trainer - 8 physics students, 1 teacher trainer Video intervision - 6 computer science students, 1 teacher trainer - 8 chemistry students, 3 teacher trainers

MODEL OF PROFESSIONAL GROWTH This study was setup by linking video club and video feedback coaching research to the interconnected model of professional growth of Clarke and Hollingsworth (see Figure 1).

RESEARCH QUESTIONS This study sought to get insight in the course of current, traditional intervision sessions and new video intervision sessions as well as the experiences of the participants with the latter. It focused on: - Input of the case owner - Group process - Output for the case owner - Output for the peers - Outcomes for the case owner

METHOD Analysis of video recordings of 5 traditional and 3 video intervision sessions. - Description of the sessions - Timing of the sessions on: input case, asking clarifying questions, discussion without case owner, discussion with case owner, and conclusion case owner - Teacher trainer questionnaire - Student questionnaire

JAN VAN DER MEIJ Jan van der Meij is Program Director of UT Teacher Education, ELAN, Department of Teacher Professional Development. He has a vocational secondary education teacher qualification in electronics and a masterâ&#x20AC;&#x2122;s degree plus PhD in educational technology. In the UT teacher training program, Jan teaches the general pedagogical courses. Janâ&#x20AC;&#x2122;s research interest lies in teacher and learner use of ICT in the classroom. Currently he is investigating video coaching of pre-service and starting teachers.

SENIOR UNIVERSITY TEACHING QUALIFICATION

Student brings in case Intervision group

External Domain

External Source of Information or Stimulus

Personal Domain

Domain of Practice Classroom Experience

Knowledge Beliefs And Attitude

Professional Experimentation

Enactment Reflection

Student tries strategy

Domain of Consequence

Student knows strategy Experience influences PD Preliminary conclusion influences PD Intervision influences PD New conclusion influences PD

Salient Outcomes

Student draws preliminary conclusion Student draws new conclusion

Figure 1. Possible professional development (PD) through intervision according to the interconnected model of professional growth.

RESULTS The teacher trainers used several intervision methods in the traditional and video intervision sessions. Video supported Input case owner Group process Output case owner Output peers Outcomes case owner

MY EXPERIENCE WITH THE SUTQ

students ✓ ✓✗ ✓✗ ✓✗ ✓✗

teacher trainers ✓ ✓ ✓? ✓? ✓?

discuss the pros, cons, and implications of an educational innovation with the stakeholders.

WHAT MY SUTQ PROJECT WAS ABOUT My SUTQ research focused on the use of video during peer group review (intervision) of our pre-service teachers. The study sought to get insight in the course of current, traditional intervision sessions and new video intervision sessions as well as the experiences of the participants with the latter.

WHAT I HAVE LEARNED I have learned that it is very important to invest time to

WHAT WAS THE BIGGEST CHALLENGE For my SUTQ project, the biggest challenge was to involve my colleagues in such a way that they were excited to participate.

WHAT I AM PROUD OF I am proud of the way I introduced and implemented the use of the video recording and sharing platform IRIS Connect in our teacher training program and our partner schools.

SENIOR UNIVERSITY TEACHING QUALIFICATION

MARK VAN DER MEIJDE Prof Dr Mark van der Meijde is a geophysicist who works for the faculty ITC and is strongly involved in the MSc programmes ‘Applied Earth Sciences’ and ‘Spatial Engineering’, but also teaches in the bachelor HTHT module ‘Smart Ways To make Smart Cities Smarter’. He is active in curriculum development and is teaching himself topics on ‘earthquakes and engineering’ and ‘exploration and environmental geophysics’.

MY EXPERIENCE WITH THE SUTQ WHAT MY SUTQ PROJECT WAS ABOUT The University of Twente has started a new initial 2-year MSc program. The program will be based on a project-based learning approach. It will be formed around 3 projects (one quartile each) and each project has an increased level of wickedness. Wicked problems deal with issues that cannot be resolved merely by gathering additional data, defining issues more clearly, or breaking them down into small problems. A wicked problem has innumerable causes, is tough to describe, and doesn’t have a right answer. The prime educational question for this SUTQ project is to define how wicked problems, problems with innumerable causes that are tough to describe and don’t have a right answer, can be used in project based education, with a particular focus on how to: - define objectives that intrinsically can’t be defined - give structure to an unstructured project - assess the quality when there is no true answer or preferred approach - provide guidelines to staff that need to supervise such a wicked problem based module.

WHAT I HAVE LEARNED Gaining knowledge and experience on a whole MSc course based on project-based education (PBE) was a prime goal. The further complication of the inclusion of wicked problems in PBE is a strongly complicating factor that required a deep understanding of wicked problems, their main role and importance in PBE, and how to make it efficient for learning and teaching. Deepening my understanding on these points helped me to define solutions that can make wicked problems suitable for learning in PBE. My main focus was on providing a challenging learning environment with objective learning/ testing/assessment procedures that provide clarity for students and supervising staff on what is required on group and individual student level. It required a different view on content related assessment since the outcome of each project is inherently undefined and

therefore no strict assessment criteria are available on the correctness of execution of the content component of projects. This required to dive deep into PBE and wicked problems, in relation to rules for assessment, to derive the optimal solution for assessing the learning in wicked PBE assessments. In relation to the assessment of student performance, and guaranteeing the knowledge gain (the learning) throughout the course, the role of the teacher is very important. The teacher should be working in such a teaching environment that supervision could be provided, but meanwhile not reducing the wickedness of the problem. This is a precarious balance that I have learned to address. Overall, the main personal learning experience was on how we can develop a whole new MSc program while in the meantime adopting new educational approaches based on wicked problems in PBE. The focus was, therefore, strongly oriented towards the definition of wicked problems in PBE and the corresponding assessment and teaching approach.

WHAT WAS THE BIGGEST CHALLENGE The biggest challenge was to find the right meaning of all the abbreviations in the educational field. The P in the abbreviation PBE can mean, for example, process or project. But also there are often different definitions for the same term or the same definitions for different terms. If you then try to find all the detailed differences and commonalities it was often a challenge to get this coherently done (and I’m even wondering if I got it all right in the end…).

WHAT ARE YOU PROUD OF The SUTQ project was meant as a development for a single course. During the development it already became clear that the framework for the very complex project I was studying was also a very suitable framework for the earlier projects. It was in the end implemented in the whole MSc which was a very nice added value to the original SUTQ plan.

SENIOR UNIVERSITY TEACHING QUALIFICATION

TAMING THE WICKED EDUCATION WICKED PROBLEMS HAVE ONE ESSENTIAL PROBLEM: THEY ARE BASICALLY NOT SUITABLE FOR EDUCATION… Provide instructions in terms of process to follow – scientific goal not main purpose

Process-based approach is solution for problems to describe science content

STAKEHOLDERS CANNOT AGREE

ONLY BETTER OR WORSE SOLUTIONS

End-product defined on learning process rather than learning content

STRONG ETHICAL, MORAL, POLITICAL AND PROFESSIONAL DIMENSIONS

ITERATION

REQUIRES COMPLEX JUDGEMENT

WICKED

PRO NO CLEAR STOPPING RULES

NO RIGHT OR WRONG SOLUTIONS

Assessment on process of analysis in mid-term, final product and oral exam

ISSUES: NO FIXED CONTENT, NO CLEAR END GOALS, NO RIGHT SOLUTION À NO FIXED GUIDELINES, NO FIXED ASSESSMENT RULES

BLEMS

EVE TS RY TR IAL COUN

NO OBJECTIVE MEASURES OF SUCCESS

mid-term is to reduce problem to subset for which solution will be sought

ALTERNATIVES MUST BE DISCOVERED

Final solution is assessed on group product, oral exam based on group and individual process

source figure: http://socialdoers.com/neet-an-increasing-wicked-social-problem/

Do you think classmates Frequency of copy-paste SENIOR UNIVERSITY TEACHING QUALIFICATION copy-paste without without referencing referencing

Ic Ju

Students ~94% Teachers 100%

Students ~35% Teachers ~9%

WHY STUDENTS PLAGIARISE? Discouraging factors Do you think classmates copy-paste without referencing

Self esteem

Moral reasoning Students ~35% Teachers ~9%

~46% would not feel guilty when plagiarising

Students ~94% Fairness to others Teachers 100%

Encouraging facto

I copy 1 or 2 sentences Just to get inspired

Personal

Frequency of copy-paste without referencing

Guilt and/or fear

Self esteem Moral reasoning Fairness to others Desire to learn Guilt and/or fear

Personal

Negative

ENCOURAGING FACTORS

~ 2 3%: benefit might outweigh risk of getting caught

Effort Danger Professors knowledge Chance of getting caught Effort Danger Specific Assignments Knowledge needed in future

Teachers “loose attitud

Broad task or assignme

Do all teachers followfollow the same Do all teachers the same rules rules

+ Masters score better than bachelors DOI: 10.1177/1469787404043815 + Integration in project PD MacLead, , Ph.D Thesis, Univ. of Calgary, 2014. no DL+McCabe, 1993 Write essay on notorious case MA Wideman, 2008 - Detection: too little too late? Thanks to all colleagues and students who gave input! - Many things not reported - Exam board sometimes too slow Standing on the shoulders of giants Szabo & Underwood, 2004.

ways you can write things”

Temptation Positive

Pressure (grade,time,task) Lenient penalties Temptation Teachers “loose attitude” Broad task or assignment

Specific Assignments Knowledge needed in future

Lenient penalties

Situational

Negative

Teachers ~13%

Pressure (grade,time,ta ~ 7 6% : “there are only so many

Chance of getting caught

Situational

might outweigh risk of getting caught

Lack of Students competences ~46%

Negative attitudes Lack of Awareness Lack of competences Second language skills Knowledge viewed as irrelevant

Professors knowledge

~ 4 6% would not feel guilty ~23%: benefit when plagiarising

Lack of Awareness

Second language skil Knowledge viewed as irrelevant

Desire to learn DISCOURAGING FACTORS

Negative attitudes

maybe yes

~ 4 0%: short deadlines give right to plagiarise

Standing on the s

Szabo & Underwoo

DOI: 10.1177/1469787404

PD MacLead, , Ph.D Calgary, 2014. DL McCabe, 1993

MA Wideman, 2008

Thanks to all collea who gave input!

SENIOR UNIVERSITY TEACHING QUALIFICATION

CORA SALM Cora Salm obtained a master degree in Applied Physics and a Ph.D degree in Electrical Engineering in the field of semiconductor device physics and fabrication. Her present research field is the reliability physics of semiconductor devices and systems. She holds a 1st degree high school physics teaching qualification and completed the DUIT course (predecessor of the University Teaching Qualification) with a redesign of the master course Reliability Engineering. Presently she is the Bachelor Coordinator of Electrical Engineering (EE). She teaches in the bachelor programs EE, Creative Technology, Industrial Design and the master program EE. She is also a tutor in the bachelors Applied Physics and Advanced Technology.

MY EXPERIENCE WITH THE SUTQ WHAT MY SUTQ PROJECT WAS ABOUT My research focussed on plagiarism and tried to identify the reasons students commit plagiarism with the aim of improving our teaching on this topic. This should lead to graduates that have the knowledge, the skills and the motivation to write proper scientific documents. SUMMARY + Masters score better than bachelors + Integration in project + Write essay on notorious case - Detection: too little too late? - Many things not reported - Exam board sometimes too slow - Teachers do not have same interpretation

WHAT I HAVE LEARNED There are a large amount of factors (positive and negative) determining the attitude towards committing plagiarism and fraud. This holds for both students and teachers The most interesting factor is the willingness of students to cheat/plagiarize when they feel the course is not properly taught (e.g. unrealistic deadlines or unfair teachers). The most surprising conclusion to me was the percentage of teachers that admit to also bend the rules themselves and the differences in interpretation and application of the governing rules at the university both within (a) and between educational program(s).

WHAT WAS THE BIGGEST CHALLENGE Being an engineer by training and heart I had never ventured in the field of behavioural sciences. Making and analysing surveys even when there are questionnaires from literature you can use as inspirations is not as simple as one thinks. Randomising the order of totally agree to totally disagree from one of the references turned out to be an outdated principle. Fortunately the target group students know me and also gave feedback on how to improve the survey.

WHAT ARE YOU PROUD OF In combining knowledge obtained from literature with data obtained from two surveys I gained a better picture of the factors involved which helps me to prioritise actions to be taken. I am proud of the fact that discussions with fellow teachers have resulted in increased awareness and willingness to at least think about and hopefully implement changes in the programs. â&#x20AC;&#x192;

SENIOR UNIVERSITY TEACHING QUALIFICATION

TO PLAY OR NOT TO PLAY: ON THE MOTIVATIONAL ASPECTS OF SERIOUS GAMES INTRODUCTION

APPROACH

The use of games as educational tools has gained increased interest over the last decennia. Referred to as serious games their primary purpose is to educate and train the player which differentiate them from pure entertainment games. Games are often assumed to possess an inherent motivational power through which individuals become immersed and absorbed in a game and experience the game play as enjoyable. This engaging potential of games is brought forward to argue that games are also suitable in the educational context. However, previous research indicates that the motivational appeal of games as demonstrated for entertaining computer games does not play out in the educational context. The aim of this research is to investigate the role of different motivational forms in serious games and the influence of the game environment on students to get involved and stay involved in game playing.

Based on self-determination theory and a mixed-method research design the role of intrinsic motivation (IM), identified regulation (IR), external regulation (ER) and amotivation (AM) in seven serious games (Table 1) is determined and the influence of game attractiveness, game learning and game operativeness on these motivational forms is revealed. The data collection is conducted in two consecutive years of a postgraduate course in which the games are used. It combines observations, panel discussions, and questionnaire surveys for all games in both years.

MY EXPERIENCE WITH THE SUTQ

Figure 1 Game 4 impression

Figure 2 Game 6 impression

of motivation and should not only focus on the game content but on operational and learning issues as well

WHAT MY SUTQ PROJECT WAS ABOUT I investigated the role of different motivational forms in serious gaming and the influence of the game environment on studentsâ&#x20AC;&#x2122; motivation to get involved and stay involved in game playing.

WHAT I HAVE LEARNED The research put my initial overconfidence in the motivational power and, thus, learning effectiveness of serious games more into perspective. It revealed that different motivational forms (intrinsic and extrinsic) can co-exist when students play serious games and that the motivational effects of games will depend on the circumstances under which the games are applied. A main learning outcome is that design and use of games for education purposes should address multiple forms

WHAT WAS THE BIGGEST CHALLENGE The integration of research design and course structure was challenging. On the one hand, the research design required a certain number of games to be played and the replication of the game play in a consecutive year to ensure valid and reliable research results. On the other hand, the games had to fit in the overall course structure to allow for the intended learning of the students.

WHAT ARE YOU PROUD OF It was an achievement for me that I was able to conduct research about my own education and could generate some interesting findings written up in a scientific publication. It was also nice to see that most students had fun while playing the games.

Figure 1 Game 4 impression

Figure 2 Game 6 impression

Approach SENIOR UNIVERSITY TEACHING QUALIFICATION Based on self-determination theory and a mixed-method research design the role of intrinsic motivation (IM), identified regulation (IR), external regulation (ER) and amotivation (AM) in seven serious games (Table 1) is determined and the influence of game attractiveness, game learning and game operativeness on these motivational forms is revealed. The data collection is conducted in two consecutive years of a postgraduate course in which the games are used. It combines observations, panel discussions, and questionnaire surveys for all games in both years. Table 1 Overview of games investigated No

Name

Type

Subject

Learning goal

GasSolution

Computer-based Single player

RiskSwitch

Computer-based Single player

Building a gas network to deliver gas in a safe, reliable and sustainable manner Increasing the reliability of railway switches

RAMSes

Computersupported Single/Multiple player

Developing a competitive bridge design for a DBFM tender

HighwayStakes

Computer-based Single player

Improving the intervention strategy for a highway link

AMImplementation

Computersupported Single/multiple players

Improving a road section by taking strategic, tactical and operational decisions

RoadRoles

Board game Multiple players

Preparing tenders for the maintenance of a road network

BridgeGame

Computer-based Single/multiple players

Monitoring and maintaining a bridge to reduce performance risks

Understanding the complexity and trade-offs of infrastructure management decisions Understanding the consequences of decisions on reliability, cost, maintainability and availability of infrastructure Understanding the consequences of design decisions on the costs and risks over the life-cycle of infrastructure assets Understanding the consequences of decisions for different stakeholders involved in infrastructure intervention projects Understanding the relationship between decisions on strategic, tactical and operational level of an asset management organization Understanding the relationship between procuring road maintenance and the condition of a road network Understanding the relationship between infrastructure objectives, infrastructure performance, and infrastructure interventions

Table 1 Overview of games investigated

CONCLUSION • Different motivational forms can co-exist when students play serious games. • The use of serious games, either computerbased or not, does not automatically lead to intrinsically motivated students in educational context. • Game attractiveness is a driver for intrinsic motivation but not sufficient to explain the existence of other motivational forms. • Game learning can particularly explain the emergence of different forms of extrinsic motivation.

• Game operativeness is a basic condition for serious games to unfold their challenge and engagement potential which in turn will frame the learning experience of students. • Design and use of games for education purposes should address multiple forms of motivation and should not only focus on the game content but on operational and learning issues as well.

ANDREAS HARTMANN Andreas Hartmann is Associate Professor of Infrastructure Management at the Faculty of Engineering Technology. He studied Civil Engineering and obtained a PhD in Construction Management. Andreas is teaching in the Master’s programmes Civil Engineering and Management and Construction Management and Engineering. His research focuses on the interplay of the social, technical and environmental aspects in providing and managing modern infrastructure and aims at developing new methods and models for optimized decision making throughout the lifecycle of infrastructure systems.

.141*

Attractiveness R2=.303

.436**

.626**

-.150 ns Game Learning R2=.390

-.272**

-.410*

**p < .05; *p < .1; ns – not significant

Figure 3 Motivation model estimation

Conclusion • Different motivational forms can c serious games. • The use of serious games, either automatically lead to intrinsically context. • Game attractiveness is a driver fo sufficient to explain the existence • Game learning can particularly ex forms of extrinsic motivation. • Game operativeness is a basic co unfold their challenge and engag frame the learning experience of • Design and use of games for edu multiple forms of motivation and s content but on operational and le

SENIOR UNIVERSITY TEACHING QUALIFICATION

CARINE DOGGEN I am a clinical epidemiologist, and head of the department Health Technology and Services Research. We focus our research on the effects of technological innovations on clinical outcome, quality of life, quality, safety and costs of healthcare. I am teaching Clinical Research Methods to students Health Sciences, Technical Medicine and Biomedical Engineering. Teaching how to translate problems from clinical practice to a specific scientific research question and using an evidence-based research approach to answer this question is challenging.

MY EXPERIENCE WITH THE SUTQ WHAT MY SUTQ PROJECT WAS ABOUT The aim of my research was to assess the effects of flipped classroom in teaching clinical research methods to 2nd year students Technical Medicine on learning effectiveness.

WHAT I HAVE LEARNED I have learned that a flipping the classroom approach does not notably increase learning effectiveness of all students Technical Medicine. It is not easily accepted and student do not like to change. For some students the traditional approach of lectures and tutorials might have led to better exam results. It would be a challenge to found out which students prosper by what type of (mixed) approach.

WHAT WAS THE BIGGEST CHALLENGE The biggest challenge was to design new teaching materials which bridge pre-class work to in-class work in time before the start of the course, and to assess the effects of the flipping the classroom approach during the 10-week course.

WHAT ARE YOU PROUD OF I am very proud of what we (my colleague who was involved in the overall process and myself) have achieved and that we did not let ourselves influence by the negative comments we sometimes received from students. What I am even more proud of is that the second time we taught this course using this new approach, students were really engaged with the materials and evaluated this part of the course as good to very good.

SENIOR UNIVERSITY TEACHING QUALIFICATION

Effect the flipped classroom in learning Effect ofof the flipped classroom in learning EFFECT OF THE FLIPPED clinical research methods clinical research methods CLASSROOM IN LEARNING CLINICAL RESEARCH METHODS Dr. C.J.M. Doggen, HealthHealth Technology and Services Research, Faculty BMS, University of Twente of Twente Dr. C.J.M. Doggen, Technology and Services Research, Faculty BMS, University

Rationale Rationale

Effect Effect of of the the flipped flipped classroom classroom in in learning learning Examination results Examinationmethods results clinical clinical research research methods

Dr. C.J.M. Doggen, Health Technology Services Research, Faculty BMS, University Twente Table 1.and Presence of all 118 students duringof all 6 tutorials (T) - Student-driven learning is encouraged and Research, of 1. Presence of allBMS, 118University students during all 6 tutorials (T) Dr. C.J.M. C.J.M. Doggen, Doggen, Health Health Technology TechnologyTable and Services Services Research, Faculty Faculty BMS, University of Twente Twente - Student-driven learning is encouragedDr. (customized, on-demand learning, self-paced) T1 T3 T4 T5 T2 T7 (customized, on-demand learning, self-paced) T2 T7 T1 T3 T4 T5 Examination - Online educational materials available Rationale Examination results Study Researchresults Frequency association experimental observational Rationale Examination results Online educational materials available Rationale Research Frequency association experimental observation Study - Flipped classroom: basic knowledge moved to design question measures measures design design Table 1. Presence of all 118 students during all 6 tutorials (T) -- Student-driven learning is encouraged - Flipped classroom: knowledge moved to Table of all during all question measures design design design Student-driven learning is encouraged RATIONALE EXAMINATION pre-class, application andbasic syntheses in-class Table 1. 1. Presence Presence ofRESULTS all 118 118 students students duringmeasures all 6 6 tutorials tutorials (T) (T) -(customized, Student-driven learning is encouraged on-demand learning, self-paced) T1 T3 T4 T5 T2 T7 (customized, on-demand learning, self-paced) pre-class, application and syntheses in-class T1 T3 T4 T5 T2 T7 (customized, on-demand learning, self-paced) - - Integrating online materials order to actively Student-driven isinencouraged (customized, T1 95 (80.5) T2 T3 T4 T5(44.9) T747 (39.8) Present, N(%) 79 (66.9) (66.1) 68Study (57.1) 53 -- learning Online educational materials available Research Frequency78association experimental observational educational available Study association experimental - on-demand Integrating online materials in materials order actively -- Online Online educational materials available association78 (66.1) experimental Research Frequency Study observational Present, N(%) Research 79 (66.9)Frequency 95 (80.5) 68design (57.1) observational 53 (44.9) 47 (39.8) engage studentslearning, and enhance learning isknowledge ato challenge classroom: basic moved to self-paced) design question measures measures design -- Flipped Flipped classroom: basic knowledge moved to design question measures measures design design Flipped classroom: basic knowledge moved to question measures measures design design design engage students and enhance learning isclinical a challenge pre-class, application and syntheses in-class - - Effects on learning effectiveness in course pre-class, application and syntheses in-class Online educational materials available application and syntheses in-class --pre-class, Integrating online materials in order to actively - Effects on learning effectiveness in course clinical N(%) 79 (66.9) 95 (80.5) 78 (57.1) 53 (44.9) 47 (39.8) research methods yet unknown Integrating online materials in order actively Present, N(%) 79 95 (66.1) 68 (57.1) 53 47 Table 2. Present, Grades by presence/absence of (66.1) the 11868 -engage Integrating online materials in order to to actively - Flipped classroom: basic knowledge moved to pre-class, Present, N(%) 79 (66.9) (66.9) 95 (80.5) (80.5) 78 78 (66.1) 68students (57.1) 53 (44.9) (44.9) 47 (39.8) (39.8) students and enhance learning is a challenge engage students and enhance learning is a challenge research methods yet unknown engage students and enhance learning is a challenge Table 2.Presence Gradesorby presence/absence of the 118 students -- Effects on learning effectiveness in course clinical Cohort <5.5 >=5.5 Totaal application and syntheses in-class Effects on learning effectiveness in course clinical -research Effectsmethods on learning in course clinical Table 1. absence Presence of all 118 students during allpassed) 6 tutorials (T) yet effectiveness unknown (% failed) (% aantal Table 2. Grades by presence/absence of the 118<5.5 students research methods unknown Totaal Cohort Presence or >=5.5 - Integrating online materials in order to actively engage Table research methods yet yet unknown Objective Table 2. 2. Grades Grades by by presence/absence presence/absence of of the the 118 118 students students students studenten Cohort Presence or <5.5 >=5.5 Totaal absence aantal (% failed) (% passed) students and enhance learning is a challenge Cohort Presence or Totaal <5.5 >=5.5 Objective or <5.5 >=5.5 Totaal (% failed) failed) (%56.8% passed) absence aantal 2015-2016 Cohort >=5Presence 44aantal 43.2% to- assess the effects of flipped classroom absence (% (% passed) students studenten Objective (% failed) (% passed) absence aantal Effects on learning effectiveness in course clinical research Objective students 74 studenten 71.6% Objective <5 students 28.4%* into teaching Research Methods 2015-2016 >=5 studenten 43.2% 56.8% students studenten 44 assessClinical the ofthe flipped classroom 2015-2016 >=5 44 43.2% 56.8% methods yeteffects unknown to assess effects of flipped classroom 2015-2016 >=5 44 43.2% 56.8% 118 61.0% 39.0% nd to assess the effects of flipped classroom 2015-2016 >=5<5 44 43.2% 71.6% 56.8% year students Technical Medicine, toin2 teaching to assess the effects of flipped classroom 74 28.4%* <5 74 71.6% 28.4%* Clinical Research Methods in teaching Clinical Research Methods <5 74 71.6% 28.4%* Being present leads to in teaching Clinical Methods <5 74 71.6% 28.4%* in teaching Clinical Research Research Methods ontolearning effectiveness (engagement, 118 11861.0% 61.0% 61.0% 39.0% nd year 39.0% nd students Technical Medicine, to 2 118 39.0% nd 2 year students Technical Medicine, year students Technical Medicine, to 2 higher % passing exam. 118 61.0% 39.0% nd Being present leads to topresent 2014-2015 102 48.0% 52.0%* year students Technical Medicine, to 2learning OBJECTIVE Being present leads Being le and grades) on effectiveness (engagement, Being present leads to on learning effectiveness (engagement, on learning effectiveness (engagement, higher % passing exam. No large differences on learning effectiveness (engagement, 2014-2015 102 48.0% 52.0%* higher % passing exam. 2014-2015 -102 48.0% 52.0%* to assess the effects of flipped classroom and grades) grades) higher % passing exam. higher % passin 2014-2015 102 48.0% 52.0%* and 2014-2015 102 48.0% 52.0%* No large large differences over the years. and grades) and grades) No differences 2013-2014 116 60.3% 39.7% Noover large differences in teaching Clinical Research Methods No large differe the years. 2013-2014 116 60.3% 39.7% over the years. * chi-square2013-2014 p-value <0.01 -116 60.3% 39.7% over the years. 2013-2014 116 60.3% 39.7% ** chi-square Methods to 2nd year students Technical Medicine, over the yea chi-square p-value p-value <0.01 <0.012013-2014 116 60.3% 39.7% * chi-square p-value <0.01 Methods Methods Methods(engagement, * chi-square p-valueby<0.01 on learning effectiveness Table 2. Grades presence/absence of the 118 students Methods Table 3. Table Grades by presence frequency ofofthe Cohort of 118 2nd Cohort yr TM of students 2015-2016 3. Grades by presence frequency the118 118 students students 118 2nd TM students 2015-2016 nd yr Table and grades) Cohort of 118 yr 2015-2016 Table 3. 3. Grades Grades by by presence presence frequency frequency of of the the 118 118 students students Cohort ofCourse 118 2 2ndEpidemiology yr TM TM students students 2015-2016 10 week Feb 2017 10 week Course Epidemiology Feb 2017 Presence N Grade Min-Max 10 week Course Epidemiology Feb 2017 Presence N Grade Min-Max Being more oftenoften Presence N Grade Min-Max 10 week Course Epidemiology Feb 2017 Being more Being more often nd Presence N Grade Min-Max Two teachers Being more often Table 3. Grades by presence frequency of the 118 students Cohort of 118 2Two yrteachers TM students 2015-2016 Mean (SD) Two teachers present leads to to Mean (SD) Mean (SD) Two teachers present leads to Mean (SD) present leads METHODS present leads to Not at all 8 5.0 (1.0) 3.5 –– 6.8 10 week Course Epidemiology Feb 2017 higher grades. grades. Not at all 8 5.0 (1.0) 3.5 6.8 Not at all 8 5.0 (1.0) 3.5 – 6.8 higher Presence N Grade Min-Max Not at all 8 5.0 (1.0) 3.5 – 6.8 Being more oft Learning effectiveness measured by: higher grades. higher grades. 1 11 4.7 (0.9) 3.7 –– 6.4 Learning effectiveness measured by: 10 week Cohort of 118 2nd yr measured TM students 2015-2016 1 11 4.7 (0.9) 3.7 6.4 Learning effectiveness measured by: Two teachers Learning effectiveness by: 1 11 4.7 (0.9) 3.7 – 6.4 1 11 16 4.7 (0.9) 6.4 Mean (SD) 3.7 -- Teachers impression 2 4.9 (1.0) 3.2 – 6.8 present leads 2 16 4.9 3.2 -- Teachers Teachers impression Epidemiology Feb impression 2017 Two teachers 2 16 4.9 (1.0) (1.0) 3.2 –– 6.8 6.8 - Course Teachers impression results comparing grades 2 Not at 3.2 6.8 3.5 – 6.8 3 all 16 22 8 4.9 (1.0) 5.0 (0.9) (0.9) 3.2 – – 7.3 7.3 5.0 (1.0) -- Examination Examination results comparing grades 3 22 5.0 3.2 – higher grade Examination results comparing grades 3 22 5.0 (0.9) 3.2 – 7.3 a) students present absent 5.3 (1.3) 3.2 - Learning Examination results comparing grades 3 22 17 5.0 (0.9) –– 7.8 7.3 3.7 – 6.4 effectiveness measured by: vs a) students present vs 17 5.3 3.2 1444 4.7 (0.9) 3.2 a) students present vs absent absent 17 11 5.3 (1.3) (1.3) 3.2 –– 7.8 7.8 5 21 5.7 (1.3) 3.5 – 7.8 b) previous cohorts 5 21 5.7 (1.3) 3.5 – 7.8 b) previous cohorts a) students present vs absent 4 17 21 16 5.3 (1.3) 7.8 3.2 – 6.8 Learning effectiveness measured by: - Teachers impression 5.7 (1.3) 3.5 – 7.8 b) previous cohorts 256 4.9 (1.0) 3.2 23 5.7 (1.1) 3.5 –– 7.8 -- Evaluation reports comparing cohort 6 5.7 3.5 Evaluation reports comparing cohort 5 * ANOVA 210.06 23 3.5 7.8 3.2 – 7.3 b) previous cohorts 6 P-value 23 22 5.7 (1.3) 5.7 (1.1) (1.1) 3.5 – – 7.8 7.8 -2015-2016 Evaluation reports comparing cohort - - Teachers impression Examination results comparing grades 3 5.0 (0.9) vs previous cohorts on engagement, * ANOVA P-value 0.06 do not not like this this kind kind of of 2015-2016 vs cohorts on engagement, ‘‘‘III do 6 * ANOVA P-value 230.06 5.7 (1.1) 3.5 – 7.8 2015-2016 vs previous previous cohorts on of engagement, - - Evaluation comparing cohort do not like like this kind of motivation and comments on type teaching teaching, II really prefer a)reports students present vs absent Examination results comparing grade Table 43. Grades by presence frequency 17 5.3 (1.3)of the 118 students 3.2 – 7.8 motivation and on teaching, motivation and comments comments on type type of of teaching teaching teaching, I really really prefer prefer * ANOVA P-value 0.06 lectures’. 2015-2016 vsb)previous cohorts on engagement, lectures’. ‘ I do not like this kind of 5 21 5.7 (1.3) 3.5 – 7.8 previous cohorts lectures’. a) students present vs absent motivation and comments on type of teaching Evaluation reports teaching, I really prefer Evaluation reports Flipped classroom: 6 23 5.7 (1.1) 3.5 – 7.8 Evaluation reports comparing cohort Evaluation reports b) previous cohorts EVALUATION REPORTS Flipped classroom: lectures’. Flipped classroom: Students: own efforts and interest * ANOVA P-value 0.06 Students: own efforts and interest 2015-2016 previous cohorts on engagement, Previous years: Students: own efforts and interest - Evaluationvsreports comparing cohort 2015-2016 vs Previous years: Previous years: Results good, did did not not like like no no lectures lectures were were‘I do not like this k Evaluation reports Previous years: Results good, Results classroom: Students: appreciated variety good, did not liketutorials, no lectures wereteaching, I really p motivation comments on type ofmotivation teaching and Students: appreciated variety previous and cohorts on engagement, Students: appreciated varietyFlipped lectures and lectures given, motivation and attendance Students: appreciated variety ‘‘Are we really not getting given, motivation and attendance we really not lectures and tutorials, lectures given, motivation and attendance Students: own efforts and interest lectures’. ‘Are Are we reallyfrom not getting getting lectures and tutorials, lectures low, tutorials (quiz) useful, book any lectures you?’ Teachers impression of lectures and tutorials, lectures low, tutorials (quiz) useful, book comments onTeachers type ofimpression teaching of were clear, tutorials were useful, took time, book not read as any Previous years: were clear, tutorials were useful, low, tutorials (quiz) useful, book any lectures lectures from from you?’ you?’ Teachers impression of were clear, tutorials were useful, Results good, did not like no lectures were not nice, teachers did not give students: clear,book tutorials wereas useful, not teachers did not give Evaluation reports students: took time, not read a not nice, nice, teachers did not used give to awere standard, teachers enthusiastic, topic uninteresting. Students: appreciated variety students: Flipped classroom: took time, book not read as a clear answers. Teachers: given, motivation and attendance took time, book not read as a clear answers. Teachers: used to ‘Are we really not getting teachers enthusiastic, clear answers. Teachers: used to lecturesstandard, and tutorials, lectures standard, teachers enthusiastic, more active students, expected RESULTS Students: efforts -- were very surprised, standard, teachers enthusiastic, more active students, expected low, tutorials (quiz) own useful, bookand interest any lectures from you?’ were very surprised, Teachers impression of topic uninteresting. more active students, expected -- did were very surprised, topic uninteresting. Previous years: more self-reflection and an were clear, tutorials were useful, not want to believe this, topic uninteresting. more self-reflection and an Results Flipped classroom: Teachers impression of students: good, did not like no lectures were did not want to believe this, not nice, teachers did not give more self-reflection and an students: did not want to believe this, academical attitude. Students: appreciated --- were somewhat confused, took time, book not read as a variety academical attitude. somewhat confused, academical attitude. given, motivation and attendance Students: own efforts and interest good, did not like no clear answers. Teachers: used to -- were were somewhat confused, ‘ Are we really not getting doing pre-class work was necessary lectures and enthusiastic, tutorials, lecturesmore active students, expected standard, teachers -- realized realized doing pre-class work was necessary realized doing pre-class work was necessary low, tutorials (quiz) useful, book any lectures from you?’ lectures were given, motivation and attendance low, tutorials were very surprised, - -were very surprised, Discussion Teachers impression of to make in-class meeting useful, Discussion to in-class meeting useful, topic uninteresting. were clear, tutorials were useful, Discussion to make make in-class meetingmeetings useful, were really more self-reflection and an who came to the in-class not nice, teachers did not give - -did notnot want to believe this,to (quiz) useful, book approach not nice,does teachers didincrease not give clear did want believe Flipping the classroom not notably learning students: --to who came the in-class meetings were Flipping the classroom approach not increase learning who came and tothis, the in-classwith meetings were really really took time, not read as does a academical Flipping thebook classroom approach does not notably notably increase learning motivated engaged the material. attitude. effectiveness of all students Technical Medicine. It is not easily accepted motivated and clear answers. Teachers: used to - -were somewhat confused, effectiveness of all students Technical Medicine. It is not easily accepted answers. Teachers: used to more active students, expected were somewhat confused, motivated and engaged engaged with with the the material. material. effectiveness of all students Technical Medicine. It is not easily accepted standard, teachers enthusiastic, and student do not like the change. For some students the traditional and student do not like the change. For some students the traditional - -realized doing pre-class work was necessary more active students, expected - were very surprised, and student do not like the change. For some students theexam traditional more self-reflection and an might academical attitude. realized doingOver pre-class work was necessary make time less students showed up (tableto1) and more approach of lectures and tutorials have led to better results. Over time less students showed up topic uninteresting. approach of lectures and tutorials might have led to better exam results. Over timeneeded less students showed up (table (table 1) 1) and and more moreDiscussion make in-class meeting useful, approach of lectures and tutorials might have led to better exam results. more self-reflection and an time was for in-class work. -todid not want to believe this, It would be a challenge to found out which students prosper by what in-class meeting time useful, was for work. It would be to out students prosper what was needed needed for in-class in-class It would be a a challenge challenge to found founddoes out which which students prosper by by what - who came to thetime in-class meetings were work. really academical attitude. type of (mixed) approach. Flipping the classroom approach not notably increase learning were somewhat confused, type of (mixed) approach. - who came to the in-class meetings were really motivated DISCUSSION type of (mixed) approach. and engaged withwork the material. effectiveness of all students Technical Medicine. It is not easily accepted -motivated realized doing pre-class was necessary and engaged with the material. Flipping the classroom approach does not notably increase and student do not like the change. For some students the traditional Carine Doggen, Associate Prof. Discussion to make in-class meeting useful, Carine Doggen, Associate Prof. learning effectiveness of all students Technical Medicine. Carine Doggen, Associate Prof. Over time less students showed upEmail: (table 1) and more c.j.m.doggen@utwente.nl approach of lectures and tutorials might have led to better exam results. Email: c.j.m.doggen@utwente.nl - whotime came tostudents the in-class meetings were Email: c.j.m.doggen@utwente.nl Health Technology and Services Research Flipping the classroom approach doesdo notnot notably increase learning Health Technology and Research Over less showed up (table 1)really and more timeIt would Itbe is not easily accepted student like the time was needed for in-class work. Health Technology and Services Services Research a challenge to found and out which students prosper bychange. what motivated and engaged with the material. effectiveness of all students Technical Medicine. It is not easily accepted was needed for in-class work. some students the traditional approach of lectures and type of For (mixed) approach. and student do not like the change. For some students the traditional tutorials might have led to better exam results. It would be Over time less students showed up (table 1) and more approach of lectures might have led tobybetter a challenge to foundand out tutorials which students prosper what exam results time was needed Carine for in-class Doggen, work. Associate Prof. Ittype would be a challenge to found out which students prosper by what of (mixed) approach. Email: c.j.m.doggen@utwente.nl

Health Technology and Services Research

type of (mixed) approach.

SENIOR UNIVERSITY TEACHING QUALIFICATION