Book review
The Global Education Race Taking the Measure of PISA and International Testing by Sam Sellar, Greg Thompson and David Rutkowski Brush Education Inc (2017) Reviewed by Wayne Jones This book focuses on what is currently the most highly influential educational assessment tool for making comparisons between educational systems on a global scale. What its authors do not do, however, is to offer an unequivocal answer to the question as to whether or not the Programme for International Student Assessment (PISA) is a testing panacea or pandemic since it neither unduly vilifies nor heaps undeserved praise on the Organisation for Economic Co-operation and Development’s (OECD) highprofile test. Instead, Sellar, Thompson and Rutkowski aim to provide key stakeholders such as school leaders, teachers and parents – who may have little knowledge of PISA and only indirectly experience its effects via changes in educational policy – with important information about the test. By enhancing their assessment literacy in this way, it is the authors’ hope that these stakeholders will be better informed and better placed to engage with the test and the public debates it engenders. To achieve this aim, the book endeavours to educate readers on many aspects of PISA such as the test’s design and administration, and the analysis of the actual results. It also provides us with a better understanding of the role and motives of the OECD in PISA’s development and growing global prominence. It is noticeable and commendable that there is a marked absence of testing jargon throughout the book, even when the discussion turns to the more technical aspects of PISA; concepts related to areas such as sampling theory, achievement estimation and statistical analyses are clearly explained, thus enhancing rather than obfuscating readers’ understanding of the strengths and limitations of the test. From the outset, the authors are at pains to stress that they are neither anti-testing in general nor against large-scale international student assessments such as PISA, since they fully accept the need for and value of Global Learning Metrics as a means of providing comparisons between education systems. The basic idea behind PISA is sound: providing data to enable governments, school leaders and citizens to evaluate their education systems, assess students’ preparedness for future employment and act as a catalyst for educational reform. The authors also fully recognise the impressive amount of work that culminates in an examination which Winter
Summer |
| 2018
adheres to standards of high quality, as well as the validity, reliability and depth of analysis of the information contained within the technical reports. These qualities notwithstanding, the test does have its limitations, particularly when it comes to making the all-important comparisons between results of participating countries. In terms of comparative data, it is the global rankings based on participating nations’ test scores that steal the limelight in the media frenzy surrounding the release of test results and form the basis of discussions about the effectiveness of educational systems and reform agendas. The lively debate about these league tables highlights serious caveats that need to be carefully considered when analysing the rankings, and can severely limit the interpretations that can be made from the rankings as a result of measurement errors and the multitude of variables that could potentially affect students’ test performance. Moreover, the fact that information within the global rankings represents a mere 1% of the PISA test reports underlines both the disproportionate attention given to the rankings and the narrowness of the discussion surrounding the results. While the OECD has to take some responsibility for encouraging the high-profile media publicity which reduces complex concepts and findings to simple messages and sound bites, it is argued that participating countries also have a responsibility for rigorously assessing the use and consequences of the test results within their own countries. The writers caution governments, for example, against a tendency to isolate a single cause related to the results that can be addressed through policy solutions without taking into account many other factors within complex educational environments. Furthermore, as key stakeholders such as
65
