The Content of the Book

ematics, or science. Using achievement gains in reading and mathematics may be justified because of their reliability, their proven correlation with progress in other subject domains, and for the profound importance of reading and mathematics to learning in many other fields of knowledge. Yet such measures still represent only a small share of all the esteemed benefits expected from education. A broad understanding of teaching quality, or of teaching effectiveness, cannot be delineated only through its contribution to learning of reading and mathematics, but must also consider to what extent the teaching promotes good judgment in ethical dilemmas, democratic ideals, ability to connect historical events and structures, capacity for analyzing scientific arguments, long-term motivation for learning and intellectual growth etc. Even if studies should prove that reading comprehension correlates well with many of these loftier educational goals, that in itself does not prove that they are attributable to the same teaching practices. Therefore, the idea of teaching quality must be maintained as a multi-layered concept that requires consideration from a number of different perspectives.

Moreover, correlating prevalent features of teacher’s instruction with measures of their students’ learning is certainly relevant, and pinpoints quality in terms of effectiveness and contribution to learning, but it only represents one of the possible ways of conceptualizing quality. In addition to effective strategies for extending curriculum content, teachers must also make wise decisions about what to teach and when, to consider students’ well-being and motivation for learning, to support disadvantaged students in order to improve equity in education and so on. These factors all pertain to the problem of how to measure quality in teaching. Research on teaching quality also needs to investigate, for instance, the extent to which students’ own perception of teaching quality relates to achievement-based identification of effective instructional features (see e.g., Fauth et al., 2014). Similarly, it will be necessary, especially when moving between national educational contexts, to examine the extent to which different measures of achievement, or of student perceptions of quality, correlate with assumed contextual indicators of teaching effectiveness (see e.g., Grossman et al., 2014).

THE CONTENT OF THE BOOK

Thus, while there is broad agreement that teaching quality matters and that teachers’ instructional repertoires in classrooms are key requisites for students’ learning, measuring instructional quality has proven to be challenging. Scholars around the world strive to agree on the ‘what’, ‘how’, and even ‘why’ when measuring teaching quality. Against this backdrop, the Nordic Centre of Excellence: Quality in Nordic

Teaching, QUINT, organized a conference with the theme “Analysing Teaching Quality: Perspectives, Principles and Pitfalls”. The conference brought together scholars from around the world, and made available different perspectives and approaches to teaching quality. In this book, we have invited some of the key contributors from the conference to provide insights into the matter of instructional quality. The book consists of eight chapters that together address both methodological, theoretical and substantial aspects when measuring teaching quality. While all chapters touch upon issues of theory, methodology and empirical findings, they are organized according to one these three main themes depending on the overall focus of the chapter.

Theoretical contributions

The first chapter, written by Courtney Bell & Robert Mislevy, underscores how teaching is critical to achieving the goal of thriving societies, and that teaching assessments play a role in helping researchers, policy makers, practitioners, and teachers themselves understand and improve teaching. In addition to the issues that readily come to mind when we think of assessment – e.g., the degree to which an instrument captures the complexity of teaching, the meaning of scores, rater reliability, and the consequences associated with assessments – stakeholders frequently do not agree on the goals of assessing teaching. Bell and Mislevy describe four metaphors for understanding assessments of teaching: assessment as a cultural practice, a feedback loop, an evidentiary argument, and a measurement. Their very informative analysis helps us understand why researchers, policy makers, and practitioners often have incompatible views of how to gather and use teaching assessment data.

In chapter 2, Nikolaj Elf discusses how to study quality in teaching. The author’s starting point is the hypothesis that any claim on or conceptualization of quality teaching implies a conceptualization of teaching. Exploring this hypothesis from a practice theory perspective, the chapter analyzes how theoretical and methodological premises are established and applied in three empirical projects from QUINT. A key point made by Elf, after comparing analyses of the three projects, is that one should be wary of applying a one-dimensional approach to quality teaching. Rather, one should think multidimensionally and plurally of quality teaching both in research and practice. To grasp this plurality, Elf suggest the term ‘the surplus of quality in teaching’, inspired by Paul Ricoeur, and underlines the importance of combining different frameworks and perspectives when attempting to capture different aspects of teaching quality.

Methodological contributions

Measuring instruction using observation instruments across divergent teaching contexts, especially across national boundaries, is a complex intellectual challenge. In chapter 3, Mark White argues that on the one hand, instruction is a very fuzzy construct that needs to be operationalized in concrete terms, which in itself is a value-laden process. On the other hand, White argues, instruction varies widely across days, students, content being taught, schools, and various other facets of the teaching context. In his chapter, he presents a validity framework for comparing instructional quality across contexts using standardized observation systems. The framework explicitly breaks down the steps in operationalizing teaching quality and sampling instruction. In doing so, White highlights various levels to which observation scores can be generalized and the processes affecting generalization. The explicitness of the framework helps to structure potential validity arguments that are necessary to support study conclusions.

In chapter 4, Armin Jentsch, Hanna Heinrichs, Lena Wilms, Gabrielle Kaiser, Johannes König and Sigrid Blömeke provide another methodological contribution. Drawing on an influential framework of instructional quality, they have developed an observational protocol with which both generic and subject-specific characteristics were assessed in different teacher samples (N = 76) by trained observers. As an approach to validation, the authors have combined generalizability and measurement invariance analyses to investigate the psychometric quality of the observational protocol. An important contribution from this study is that that psychometric properties can differ not only between observational instruments, but sometimes even between measures of a single observational instrument. The authors argue that this finding could be relevant for the development of future observational instruments, both in educational research and instructional practice.

Instructional quality has been identified as one of the most important predictors of student outcomes. Yet, how to conceptualize and measure the concept, what aspects of it are important, in what subject domains, in which countries, for what cohort, and for what type of outcome (cognitive and affective) remains unclear. In chapter 5, Bas Senden, Trude Nilsen and Sigrid Blömeke seek to disentangle these challenges by reviewing previous research and showcasing findings from studies made across countries, cohorts, subject domains, and outcomes. The chapter provides relevant perspectives for anyone interested in the concept of teaching quality, and a key take-away from this work are the findings that the role of instructional quality for student learning might vary across contexts, hinting towards the importance of differential effectiveness for instructional quality.

Empirical contributions

In chapter 6, Kirsti Klette, Astrid Roe and Marte Blikstad-Balas combine video data from a large number of language arts and mathematics lessons (grade 8), scored with the Protocol for Language Arts Teaching Observations (PLATO), with student achievement gains on the national tests in reading and mathematics. The aim of their study is to investigate correlations between PLATO scores and achievement gains for all students and for subgroups of students. They examine whether certain PLATO elements are more closely related to student achievement gains than others and discuss what possible relations between PLATO scores and student achievement gains tell us about how various instructional practices might affect different students’ learning. Their study highlights that: (a) instructional facets might support students differentially; (b) high-achieving students might profit more from the indicators of instructional quality as measured though PLATO and (c) instructional quality may encompass different features in different subjects.

In chapter 7, Eva Weingartner reports on a video study investigating the cognitive activation potential of tasks used in the subject ‘Economy & Society’ at commercial vocational schools in Switzerland. Weingartner analyzes both the objective and the realized cognitive activation potential of the tasks – i.e., both the construction of the task and its implementation by the teacher. The results show that the objective cognitive activation potential was generally on a low to medium level and rarely changed through implementation in class. The insights of this project are useful for future teacher training and may contribute to raise teachers’ awareness of the way in which tasks and materials activate students’ thinking.

In chapter 8, Jennifer Maria Luoto and Alexander Jonas Viktor Selling address the possible tension between the need to standardize observations and always look for the same observable features of instruction to enable reliable comparisons across multiple classrooms, and the need to consider the context of each lesson to make sense of what observation scores really tell us about instruction. In the chapter, they use the perspective of the teachers’ set learning goals as context-sensitive lenses and analyze their instruction in relation to these goals. In addition, Luoto and Selling analyze the same instruction with a standardized observation system, the PLATO-manual, and discuss the results from these two approaches. These perspectives – the teachers’ own goals and the standardized measure – may be seen as contrasting, but the authors argue that by using them as complementary lenses, we may come closer to finding ways of measuring instruction in rigorous and contextsensitive ways.