MEDICAL EDUCATION
Situational Judgment Testing Improving Medical School Selection Processes BY MOJCA REMSKAR, MD, PHD, AND DIMPLE PATEL, MS
S
ituational Judgment Testing (SJT) is a tool generally recognized as having been invented by the psychologist Alfred J. Craddall around 1942, as a way to predict appropriate action by employees in the workplace. Administered through a series of questions, SJT presents several potential solutions to specific workplace scenarios.
Why is there a need for consideration of a tool like SJT? Let’s admit it—the institutional student selection processes in medicine favor specific groups of people over others. As argued in a recent article in Annals of Internal Medicine, medical schools, professional organizations, academic departments and private practices are racialized organizations, which to a large extent, continue to use structures and processes that promote certain groups. Traditionally, we have relied heavily on the use of academic measures, such as cognitive tests, as a leading factor in our decision-making. It is welldocumented that using these measures disadvantages certain populations of applicants. In evidence-based medicine, it seems counterintuitive and hard for us to admit, but studies show that “good looks’’ and being thin improve your chance of being selected into a residency program. Being Black, Asian or obese is a disadvantage, according to various studies. Practice of individual,
one-on-one interviews with faculty, no matter how unbiased and openminded faculty are, leads to acceptance of “the best fit” candidates, who often reflect the leadership structure/image, which in medicine still predominantly consists of white, heterosexual, cisgender males.
Changing a paradigm In recent years, we have seen a significant emphasis being given to non-cognitive domains related to interpersonal communication and professionalism, with the intent to counter the above-mentioned trend within medical schools, professional medical institutions and hospitals. To support the need for non-cognitive competencies to be included in the selection process, there is evidence that competencies, such as teamwork and accountability, correlate with improved patient outcomes, patient satisfaction and adherence to treatment plans. The AAMC and the Accreditation Council for Graduate Medical Education (ACGME) both emphasize professionalism and communication skills as essential components of a well-rounded physician by including them into their assessment frameworks. However, at present time, we do not routinely use reproducible, validated, easy-to-use tools to measure the professionalism and communication domains in our interview processes. Instead, we rely on one-on-one interviews as the most important way to assess applicants’ interpersonal and communication skills, maturity, interest in the field, dependability and honesty. This process is not standardized and lacks inter-rater reliability, and therefore, its outcomes are biased by personal preferences of interviewers. The SJT can measure non-cognitive skills, such as those related to professionalism and attributes of an individual that mimic actual real-life events related to specific professions. Its ability to objectively measure interpersonally oriented skills is what makes it attractive for the selection processes.
Compassionate, Comprehensive, & Personalized care for adult and pediatric patients with neurological conditions, including:
Head Injury/Concussion Epilepsy/Seizures Headache/Migraine Neck/Back Pain Sleep Disorders Movement Disorders Parkinson’s Disease Tremors Alzheimer’s Disease Dementia Muscle Weakness Carpal Tunnel Syndrome
Sciatica Neuromuscular Disease Muscular Dystrophy Dizziness Numbness Stroke Multiple Sclerosis ALS And other neurological disorders
612.879.1500
NoranClinic.com Blaine | Edina | Lake Elmo/Woodbury | Lakeville | Minneapolis | Plymouth
22
JULY 2021 MINNESOTA PHYSICIAN
Development of the SJT starts with analysis of professional critical events that present specific constructs or domains relevant to the profession. Experts and non-experts are asked to develop a list of best, worst and in-between responses or approaches to address different situations and incidents. In the final step, a scoring key is developed based on the ability of specific responses to differentiate between experts and novices and the correlation with job performance in specific domains. The administration of SJT is relatively simple. It comes in a series of online scenarios for which the candidate is asked to rank the responses from the best to the worst. It can be administered as a written test, a video or a cartoon. As mentioned previously, AAMC has developed its own SJT which measures eight pre-professional competencies relevant to medical school applicants – service orientation, social skills, cultural competencies, teamwork, ethical responsibility to self and others, reliability and dependability, resilience and adaptability and capacity for improvement. Understanding the importance of the inclusion of the professionalism domains into candidate selection, the SJT presents around 30 scenarios with multiple responses required to each one. Test takers select a response most closely related to how they would handle that situation. The entire test takes about 75 minutes to complete. Since SJT requests individuals to
