ASSESSMENT FOR LEARNING

Diagnose your students’ learning needs by asking the right questions

STORY BY Oliver Quinlan

Mention assessment in schools and most people think of tests. Memories of creaky exam halls and regulation stationery may come first, but there are several developments in assessment that can change how we think about discovering what students have learned.

Back in the late nineties, Black and Wiliam challenged educators’ views on assessment with their seminal work Inside the Black Box, which popularised the idea of ‘assessment for learning’. They suggested an approach which brought assessment into the learning activities themselves. When this is executed well, students are given feedback as they learn and have the opportunity to act on this feedback immediately. It’s about the teacher not just delivering and then assessing later, but regularly checking for understanding and adapting their teaching. It’s also about the learners getting regular insight into how they are learning and, crucially, having an opportunity to act on the feedback they get and ensure that they are making progress. The assessment is designed to serve the students’ learning, and not to certify that they have achieved a set standard.

Assessment revolution

This work led to many developments in schools, with the government in the UK taking up ‘assessment for learning’, teachers being trained to regularly assess students’ learning within lessons, and students being provided with feedback to act on as they learn. More recently this approach has been supported by John Hattie’s meta-analysis of influences on achievement in schools, which puts feedback at the very top in terms of the size of the measured effect (helloworld.cc/hattie).

Truly effective formative assessment is not just about finding out whether students have ‘got it’ yet. It’s about understanding how they are thinking about a topic, what misconceptions or naive understandings they have, and how your teaching and activities can be adjusted to address them. Given the abstract nature of computing, the potential for misconceptions is very high.

Diagnostic questions

Edtech company Diagnostic Questions is seeking to address this with its online assessment platform. Its assessments look familiar at first: multiple-choice questions with four answers. Multiple-choice questions have been given a bad reputation by some educators, but their quality comes down to how the questions themselves are put together. If a question has a correct answer and three laughably implausible answers, it won’t be a useful tool. However, if each answer represents a different level of understanding, or a common misconception, then the answer the student gives is useful, even if it is the wrong one. Imagine being told after an assessment not just who got the right answers, but why those who got it wrong did so, and potentially the misconception you need to address for each group of students.

In 2017, Diagnostic Questions started working with Computing at School, the Centre for Evaluation and Monitoring, and later, Cambridge Assessment to bring this approach to computer science. Project Quantum is a project exploring the potential for crowdsourcing diagnostic questions for the computing curriculum, and using them for formative assessment. They have been encouraging teachers to add questions to the platform, and use the questions already available with their students to help them better understand their learning. This project brings the potential of a relatively

CREATIVITY AND PROBLEM-SOLVING ARE KEY TO COMPUTING, BUT DIFFICULT TO ASSESS USING TRADITIONAL APPROACHES

new approach to assessment to computer science teachers, and a chance to better understand how students make sense of difficult topics.

Comparative judgement

Creativity, problem-solving, and original approaches are key to computing, yet these things are very difficult to assess using traditional approaches. It’s very common in education to use criteria to assess how well students are performing. We might set them a programming problem and then tick off whether they have used a loop or an ‘if’ statement, showing that they have understood those things. However, real-life programming is often more about the elegance of the design of a solution. What if the most accomplished student doesn’t use the things you have on your checklist? This is a particular problem when using wide briefs for tasks, or even openended projects.

Comparative judgement is a field relatively new to education practice, which offers huge potential for solving this problem. It’s based on wellestablished research showing that humans are relatively poor at making

objective judgements about individual objects, but very good at making comparisons. Play a musical note to most people and ask them what it is and they will struggle. Play them two notes and ask them which is higher and they are likely to be successful. Repeat this several times, with a clever algorithm to keep track, present them with the right combinations, and you can come up with a scale. These rankings have been shown to be very reliable — even more so if you involve several people as judges.

This method has been shown to work well even for judging things for which we don’t have clearly defined criteria, such as looking at workings in maths and asking, ‘Who is the better mathematician?’ It can also be reliable even when the judges are peers at a similar level of proficiency. This opens up some exciting new ground for the assessment of skills that involves approaching problems in an open-ended way, and assessing complex skills without resorting to trying to predefine what successful students must do. Assessment organisation No More Marking is exploring this approach for English and maths in partnership with schools.

Assessment is all about students getting better at something, but it seems there are also some promising avenues for educators to get better at assessment.

FURTHER INFORMATION

n Inside the Black Box:

helloworld.cc/blackbox

n Diagnostic Questions:

diagnosticquestions.com

n Project Quantum:

helloworld.cc/quantum

n No More Marking:

nomoremarking.com