COGnITIVE SCIEnCE AS A FRAMEwORk FOR ORGAnISInG GREAT TEACHInG

PEDAGOGY

THE CASE FOR ADAPTIVE QUIZZING IN ASSESSMENT

As teachers, we know all too well how valuable good questioning in the classroom is, when teaching new information to students. We adapt our questions well to draw the right information out from students during the “delivery” part of teaching. But how well do we adapt our questions when assessing our students after the delivery? Kristian Still argues the case for adaptive quizzing in his latest piece for HWRK Magazine.

By Kristian Still

At the beginning of the summer, I stumbled upon the work of Dr Svenja Heitmann. Three papers in fact. In this series, Heitmann et al. (2018, 2021, 2022) investigated two mechanisms to ‘optimise’ practice quizzing when compared to notetaking, starting o in the laboratory and moving to the lecture hall with a fi eld experiment with 155 undergraduate pre-service teachers at Bielefeld University.

Dr Heitmann and her colleagues focused their attention on the adaptive (personalisation) mechanisms. Two adaption models were investigated in the laboratory: performance-based and cognitive demand-based.

In the performance-based approach, performance on quiz questions was used as an indicator of retrieval success. In the cognitive demand-based approach, “perceived cognitive demand” when answering the quiz questions was used to indicate retrieval-e ort (Heitmann et al., 2022), in other words, how hard the questions were to answer.

Both models led to performance benefi ts. Personalising the adaptive quizzing, using perceived cognitive demand-based adaptations “substantially increased the quizzing e ect” (Heitmann et al., 2018: 10). This mechanism was then applied in the fi eld study.

The fi eld study results led Heitmann et al. (2021: 603) to conclude that the benefi ts of practice quizzing “in authentic learning contexts are even greater when the quiz questions are adapted to learners’ state of knowledge”.

In addition to improved test performance on familiar questions, their research also provided further evidence for knowledge transfer, suggesting that practice quizzing is a suitable tool to foster meaningful learning.

As for learner achievement motivations (much like a character trait), quizzing benefi ts were moderated by ‘hope of success’ scores but not ‘fear of failure’ scores.

Their conclusion is clear in the title of the 2018 paper – “Testing Is More Desirable When It Is Adaptive and Still Desirable When Compared to Note-Taking.”

Hardly groundbreaking. But give me a moment -- the benefi ts of testing led learners to achieve a higher test performance and, interestingly, “lower perceived cognitive demand during testing.”

“Without question, the most e cient schedule [for spaced learning] is an adaptive one, accounting for the learner’s rates of forgetting and prior knowledge”. (Latimier et al., 2021: 980).

WHAT DOES THAT ACTUALLY MEAN AND WHY MIGHT IT INTEREST TEACHERS?

The inference is, that in addition to knowing more, quizzing (retrieval practice) frees up cognitive capacity or thinking space. As Dr Heitmann commented:

‘The post-test performance was better because their mental resources weren’t as exhausted in the learning phase.’ Dr. Svenja Heitmann

The students’ ‘mental resources were less exhausted due to the adaptation of the test questions to their level of knowledge’. As a result the students ‘profi ted from the freed-up capacity for the execution of benefi cial learning processes.’ In simple terms, students had more resources left for processes that are benefi cial for learning.

So, what can teachers take from Dr Heitmann’s research?

First, the benefi ts of quizzing over note-taking. Hopefully, you knew that already.

Second, the more a student knows going into an exam, the more cognitive capacity they will have to attend to the mechanics of that exam. Importantly, that is not exclusive to adaptive testing. Quizzing of any sort, ahead of exams, helps free up cognitive capacity, which could be allocated to attending to the mechanics of the exam. That can only be a good thing right?

Third, perceived cognitive demandbased ratings, how di cult pupils fi nd questions rather than how many marks they were awarded, might be a more useful measure for personalisation and whether a pupil should relearn or revise a topic area. If nothing else, it is a simple indicator to collect and consider alongside question outcomes.

WHAT MIGHT THIS LOOK LIKE IN PRACTICE?

End-of-term assessments provide excellent opportunities to explore the ‘perceived cognitive demand’ of the questions you set. First, very high ratings for perceived cognitive load would indicate that learners have not yet acquired the knowledge necessary to master a question. They might benefi t more from quiz questions of lower complexity. Similarly, very low cognitive load ratings would indicate that learners have already

PEDAGOGY

acquired the knowledge necessary to master a question and might benefi t more from quiz questions of higher complexity.

However, Wisely, Heitmann et al., (2022) advise caution with any metacognitive judgement, due to the common biases and heuristics of learners’ self-assessment. A simple rating scale next to the question would su ce in this case.

I would also add my position, that any assessment with metacognitive judgements (be that confi dence or perceived cognitive load), with feedback, promotes metacognitive accuracy. And metacognitive accuracy brings with it a crucial academic advantage.

Now, with these two pieces of data (the rating and the question outcome), there is plenty to discuss with your pupils. First the perceived cognitive load ratings and second the di erence between the rating and the performance.

DO WE HAVE TO USE SOFTWARE FOR ADAPTIVE QUIZZING?

There are plenty of digital platforms on the market using phrases like AI, or adaptive, personalisation. However it does not have to be so. As Dr Heitmann implored:

“Adaptive quizzing could just as easily be done with di erent folders containing di erently di cult questions… and then the students use some kind of rating scale (maybe even smiley faces for the young students) to then choose the folder their next question would be coming from. There is still a whole lot of paper pencil schooling going on out there - and adaptive quizzing is available there too.”

I asked Dr Heitmann her professional thoughts on the benefi ts of needing to make the perceived cognitive demand. Are the performance gains part-memorial and partmetacognitive? “It’s not all about refl ecting, I think that’s more of a nice side e ect (to strengthen metacognition as you wrote). The adaptation is more focused on providing students with fi tting questions when you do not have the resources to sit down with every single student to adapt the di culty yourself according to your personal assessment that’s based on your interaction with that student... because no teacher teaching in regular schools has those resources!”

WHY HAS ADAPTIVE QUIZZING STAYED IN THE SHADOWS?

Dr Heitmann argues, “What’s been missing is informed teachers in classrooms, teaching with adaptive quizzing and teachers with a broader audience who can make adaptive quizzing better known. Teachers need to know that it’s a good idea to adapt questions.”

Our thanks to Dr Svenja Heitmann.

References:

Heitmann, S., Grund, A., Berthold, K., Fries, S., & Roelle, J. (2018). Testing is more desirable when it is adaptive and still desirable when compared to note-taking. Frontiers in Psychology, 9. https://doi.org/10.3389/fpsyg.2018.02596. Article 2596. Heitmann, S., Obergassel, N., Fries, S., Grund, A., Berthold, K. and Roelle, J. (2021) Adaptive practice quizzing in a university lecture: a pre-registered fi eld experiment. Journal of Applied Research in Memory and Cognition, 10(4), 603–620. https://doi.org/10.1037/h0101865 Heitmann, S., Grund, A., Fries, S., Berthold, K., & Roelle, J. (2022). The quizzing e ect depends on hope of success and can be optimized by cognitive load-based adaptation. Learning and Instruction, 77, 101526. https://doi.org/10.1016/j.learninstruc.2021.101526 Latimier, A., Peyre, H. and Ramus, F. (2021) A meta-analytic review of the benefi t of spacing out retrieval practice episodes on retention. Educational Psychology Review, 33, 959–987. https://doi.org/10.1007/s10648-020-09572-8

“Memory is the residue of thought”

KNOWLEDGE ORGANISERS: RESEARCH & IMPLEMENTATION

This is the fi rst of two articles around the subject of knowledge organisers in the classroom. In this fi rst article, I discuss the research behind memory and retaining information, giving rise to the benefi ts of knowledge organisers as a form of retrieval practice before discussing how they have been implemented in my school.

By Adam Woodward

PEDAGOGY

In recent years, knowledge organisers have been the subject of many conversations around the subject of retrieval practice and its implementation both in primary and secondary schools. The term was fi rst coined by Joe Kirby in 2015 as “the most awesome tool in the arsenal of the curriculum designer... These organise all the most vital, useful and powerful knowledge on a singular page.” This focus was primarily based around their inclusion for secondary students as a form of independent study and revision for GCSE and A-Level students. but the conversation since then has also been as to whether they could be implemented in the primary setting.

Kirby also goes on to explain how, when a new teacher starts in a school, one of the fi rst questions they have is ‘what do I teach?’ At a glance, knowledge organisers answer that. Everything pupils (and teachers) need to know is set out clearly in advance – like any well thought out curriculum should be.

Based on this, teachers, including myself, have seen a great benefi t to both teachers and pupils alike in primary and I have spent the past two years (since the fi rst lockdown) creating my own, based on the thoughts of Kirby, Jon Hutchinson and Kate Jones as to how they can be most e ective in the primary classroom.

The introduction of these in my setting has been gradual, focusing on History and Geography initially and focuses the attention of key dates, key events, important individuals and the links to prior learning. It is then my job to create these for my class teachers. However, to understand the benefi ts of knowledge organisers as a form of retrieval practice in the primary classroom, it is important to think about the research behind how we retain information.

Daniel Willingham, in Why Don’t Students Like School? said that “Memory is the residue of thought” and this quote has been used worldwide when discussing cognitive science and memory. So what makes something stick in your memory, and what is likely to slip away?

Willingham states that one important element to consider is the environment where learning is taking place and the attention paid by the student as teaching is taking place and it was here that the memory model was born and has since been replicated by Oliver Caviglioli as seen here:

This understanding of memory and the environment where teaching is taking place provides us with an understanding of a need to make sure that teaching is focused on what needs to be learned. An example of such is the teaching about Stonehenge by building it out of custard creams. Reader, I can see your eyes rolling into the back of your heads as the mere thought of this idea. Yes, it sounds lovely and the children will have a great time taking part in such a fun activity. However, what they will remember is that they used custard creams to build Stonehenge. What they don’t remember is that Stonehenge was built in the Neolithic and Bronze Age periods; that the top stones on each arch are called lintel stones and that the standing stones are called sarsen stones, and this is where knowledge organisers can play a key part.

Bjork (2012) states that “using your memory, shapes your memory”, and Ebbinghaus’ Forgetting Curve of 1885 shows a key link to the benefi ts of retrieval practice when looking to use and shape said memory. Agarwal (2021) defi ned retrieval practice as “…the act of recalling previously learned information— improving long-term learning and memory.” Retrieval practice – such as the use of knowledge or-ganisers as one example of this - enables taught information to become embedded, and therefore learned information, over time.

Our working memory is very limited, so it is naïve to think that everything that we ‘teach’ our stu-dents will stick. This is made even more di cult for those children who have a more limited working memory than others. Through rehearsal and retrieval, more information can be stored in the long-term memory. However, this information is not immune to being forgotten and by revisiting this knowledge through retrieval practice, it is easier to bring this knowledge back from the longterm memory to the working memory.

Kate Jones (2019) states that “as educators, our role isn’t to simply transfer information to students’ long-term memory, we also need to support them so that they can retrieve that information when required.”

I am not the fi rst teacher to have thought about the use of knowledge organisers in the primary classroom. One pioneer of this was Jon Hutchinson. He explained how a well-planned curriculum (and the use of knowledge organisers within this curriculum) meant that teachers thought carefully about what they wanted children to know at the end of a topic, as opposed to what activities they wanted to fi ll lessons with.

I really enjoyed the example that he gave at the end of a Stone Age unit covered with Year 3. His children took part in many ‘fun’ activities including building neolithic roundhouses out of card and straw; learning a stone age song and creating stone age jewellery using beads and string to name a few. Hutchinson explained how the children loved it.