EDITORIAL:
WHY MAKING SENSE OF MOCK RESULTS ISN’T EASY “Correlation is not causation”. I’ve been thinking of using this as a mantra. Far too often, even those of us who are aware of its truth can fall prey to this logical trap. After all, how many times have you tried out a new teaching technique, with students seemingly succeeding as a result, only to find out later on that they only really made progress because of your newfound enthusiasm, rather than the clever new strategy you used? The key question is, how do we separate the truth from the noise? We need to ask the same question about mock exam results. As we move into “mock season”, we should be especially mindful of the correlation/ causation problem. The whole point of Autumn mocks is to find out what students do or don’t know, what they can or can’t do and to work out what I need to do next, in response to their performance. Making Valid Inferences is the name of this game and it’s a lot harder than it seems. Just imagine this: a student answers a “describe” question with a one-line response, when they should be writing a detailed paragraph or two. What should we make of that as their teacher? • Have they misunderstood the amount of detail required? • Do they have gaps in knowledge? • Is their understanding accurate, but shallow? • Did they merely guess the answer correctly without actually knowing it? • Is their response just a regurgitation from a revision sheet? Without an accurate answer to these questions (and more besides), our next move may not have any impact. So what should we do? Well, let’s look at the responses from the whole class: • Do many other students struggle similarly on that question? HWRKMAGAZINE.CO.UK
• Are other students having issues with that same exam “skill”, e.g. do they evaluate instead of describe? • Did they all run out of time?
back to when you taught that topic: how strong was your subject-knowledge? And are you the best judge of that?
Or is it actually more complex than that? Sometimes there’s a mixed response from different students across the class. Do students from one group perform better or worse than others? A change in seating arrangements might help. But then again, it might not. Maybe it was the weather that day. Did a wasp fly into the room during your explanation? Were your students in a bit of a rush after being late from PE? Was there a funny smell from the farmer’s field next door that students kept getting distracted by? Did you (without a hint of irony) forget to set a recall homework task on the topic where they underperformed?
To tackle post-mock issues then should be a collaborative effort, not siloed off for a Head of Department or a Key Stage Coordinator to deal with alone. As a departmental team, it is worth discussing not just “how well did they answer question 8”? but also “how well did we teach the students to be able to answer question 8?” By posing the question in this way, we are much less likely to make assumptions about the student’s answer and much more likely to find the true reason for their response. We should discuss and model our own inclass explanations, how we scaffold and how we assess as we teach, checking for misconceptions and encouraging detail and nuance in students’ responses.
A Question Level Analysis can be helpful, but it won’t always provide the answers that we as teachers need. A good QLA can still only give you a limited amount of information. The information you actually need often comes from your memory of teaching that topic at the time.
Having these discussions also stops us from letting ourselves off the hook. Much of a student’s attainment is down to things that occur beyond the walls of our classrooms and this is why holding teachers solely accountable for exam results is highly problematic. But we are responsible for how we teach and this impacts student responses in exams in arguably the most significant way. If we have taught it well, the students will typically perform well in assessments.
What was it that helped or hindered your teaching? This might be a resource issue, a timetabling one, a staffing conundrum, or something on a whole school level, largely beyond the control of the class teacher or Head of Department.
There are schools across the country, whose cohorts are classed as “disadvantaged” in various ways, but who also routinely outperform other schools whose students “have it easier” (at least on paper). This comes down to the teaching.
It might even be that your own knowledge just wasn’t strong enough on that topic. That’s an uncomfortable thought, isn’t it? Well, it shouldn’t be. And we can address it without stigma, shame or professional embarrassment. In fact, I’d argue that if we are teaching a challenging curriculum, then from time to time we should fully expect it and actually embrace it in our practice, both individually and as a department. Think
As Dylan Wiliam puts it, “Every teacher needs to improve, not because they are not good enough, but because they can be even better.” So, scrap “correlation is not causation”. My mantra should simply be “Keep improving my teaching”. Everything else is just noise.
In other words, did the problem occur during the teaching, rather than during the exam?
Andy McHugh
Editor | HWRK Magazine N OV E M B E R 2 0 2 1 // H W R K M AG A Z I N E // 0 5
