4.2.5 Judgement

effectively beyond the end of the course and be able to make judgments about their own learning outcomes” (Boud & Soler, 2016, p. 2).

Other studies, for example of apprenticeship, have shown that being immersed in a learning environment also profoundly structures the learners’ “social knowledge, worldviews and moral principles that denote membership and status in a trade” (Marchand, 2008, p. 246), apart from facilitating technical skills. In the rota commander course, learners/officer cadets are acculturated as members of the officer corps in the Singapore Civil Defence Force through elaborate ceremonies like the commissioning parade. Designing assessment holistically means that assessment outcomes, criteria and tasks, and all the other components in the teaching system, have to incorporate learners as participants in assessment (rather than as receivers of instructions, results and feedback).

Sociocultural theories of learning (e.g. Vygotsky, 1978) and anthropological research on learning (e.g. Marchand, 2008; Ross, 1999) support a more holistic view of learning as an “embodied phenomenon” that not only rejects dichotomies of mind–body, knowledge–skill and theory–practice but focuses instead on “whole-person” learning, where learning is regarded as an ongoing process of participation in relevant activities, and engagement in meaningful undertakings, rather than as a “thing”, “product” or acquisition of certain “products”. Therefore, holism is a perspective that focuses on the “integratedness” of learning. We are aware of the importance for “individual components” or tasks to be taught and assessed. Task-specific practice is fundamental in vocations like cooking, and what we are suggesting here is that these tasks can contribute to the overall wholeness of what is essentially a good cook, for example.

4.2.5 Judgement

Judgement is an important aspect of assessment for and as learning. It is a fundamental feature of the outcome of sustainable assessment where learners develop the ability to judge the quality of their work, understand standards and identify their learning needs (Boud & Soler, 2016). Outcomes of sustainable assessment refer to the ability of learners to make informed judgements of their own learning (sustainable assessment relies on assessors and/or learners to exercise judgement). Judgement is also an essential part of the learning and assessment process because the development and use of judgement are fundamental in enabling learners to understand their own work, and also for assessors making assessments:

Human judgment is needed to collect and collate information, especially if – in a programme of assessment – information from various types of assessment needs to be combined. When human judgment is central in the assessment process, it may be clear that the quality and expertise of the person who is making the judgment is decisive for the quality of the assessment … in assessments involving human observation and evaluation the quality of the user is central. (Schuwirth & van der Vleuten, 2011, p. 481)

Our findings suggest that standards of “objectivity” and “reliability” are often privileged in assessment, which has led to a reliance on psychometric assessment/“measurement” methods and breaking things down into smaller components, rather than an emphasis on the connectivity of things and their integratedness or wholeness. One may argue that the exercise of professional judgement through the use of psychometric measurements gives an impression of objectivity and reliability. However, this approach can result in tensions between learning and measurement as the situation is more complex than that: course designers, instructors and assessors recognise the importance of professional judgement, but they also operate in an institutional setting which demands (public) accountability. For example, in the rota commander course, the use of a checklist as a reporting format for assessment may limit capturing holistic performance, but judgement of holistic performance was taking place. Here,

54

assessors have some flexibility in determining how to conduct the assessment, and this is largely determined by the nature of the exercise, whether it is a high-rise fire, oil-tank fire, hazardous material incident etc., which then shapes the assessment in terms of the emphasis on particular aspects of the job/role, and the different domain knowledge. It requires assessors to exercise professional judgement but presents a challenge to those designing assessment where consistency of standards, judgement and objectivity are sought, especially in and for a summative assessment situation. This tension between institutional demands and professional judgement of performance that is holistic is managed, in the case of the rota commander course, by course designers, coordinators and instructors being cognisant of how professional competencies such as responsiveness, alertness, awareness and decisiveness involve combinations of tasks or activities, and modes of understanding, interpreting and communicating, which can be developed only with time and experience as learners/officer cadets grow into their role as rota commanders upon graduation from the course.

In terms of learners making informed judgements about their own learning, our findings show that in a number of case studies, learners are generally dependent on their instructors and assessors for information about their learning: in the F&B menu-change training, cooks rely on the chefs to tell them if they have got the taste right (rather than determine for themselves), whether they have met the requirements and/or what they need to do in order to get it right. However, in the rota commander course, the learners/officer cadets were able to identify, assess and self-regulate their own developmental needs such as physical fitness (a fundamental competency). For example, they conducted running sessions in bunker gear after/outside of official training time, at night, in order to acclimatise their bodies to the gear and develop their fitness level. But for more complex tasks/activities such as “command and control”, the officer cadets depended heavily on the assessor’s judgement about their performance (as observed in the final exercises), and they responded primarily to cues from the assessor. In the certified IP associate course, learners were able to approach complex network problems and identify the issues during the course because they could judge where and how they are going in their learning through the use of a heuristic8 .

In situations where direct guidance is less forthcoming from assessors and instructors, in a more openended and less controlled learning environment such as in the workplace learning programme, learners were able to reflect on their own performances and experiences using summative assessment tools like the “enterprise report”:

Enterprise report is really, really comprehensive and it captures so much information ... Really, really taking a lot of time, so if you can actually fill in all these sections in different part of the report then those are the actual evidences and while we are filling out the enterprise report, the report actually forces us to go through the structured learning process. Make us think, makes us reflect on what we are doing so this is in a way is good for us but then it’s quite a lot of work to do it but then it’s something that will make us keep on track and make us reflect on what we are doing and see how we are doing and things like that. (Harriet, learner)

While learners were given support in writing the report, they were left to figure out how to “use” and “make sense” of the report for the development of their own learning, and to make meanings out of the writing process individually. At the same time, they developed the report together with other learners

8 Here, the “heuristic” refers to a decision flow chart that explicates particular ways of thinking analytically. It seeks to help learners approach network problems and identify issues. It is understood by learners and instructors as a guide to decision making and pattern identification.

55