MEDICAL EDUCATION

Situational Judgment Testing

Improving Medical School Selection Processes

BY MOJCA REMSKAR, MD, PHD, AND DIMPLE PATEL, MS

Situational Judgment Testing (SJT) is a tool generally recognized as having been invented by the psychologist Alfred J. Craddall around 1942, as a way to predict appropriate action by employees in the workplace. Administered through a series of questions, SJT presents several potential solutions to specific workplace scenarios.

Why is there a need for consideration of a tool like SJT? Let’s admit it—the institutional student selection processes in medicine favor specific groups of people over others. As argued in a recent article in Annals of Internal Medicine, medical schools, professional organizations, academic departments and private practices are racialized organizations, which to a large extent, continue to use structures and processes that promote certain groups. Traditionally, we have relied heavily on the use of academic measures, such as cognitive tests, as a leading factor in our decision-making. It is welldocumented that using these measures disadvantages certain populations of applicants. In evidence-based medicine, it seems counterintuitive and hard for us to admit, but studies show that “good looks’’ and being thin improve your chance of being selected into a residency program. Being Black, Asian or obese is a disadvantage, according to various studies. Practice of individual, one-on-one interviews with faculty, no matter how unbiased and openminded faculty are, leads to acceptance of “the best fit” candidates, who often reflect the leadership structure/image, which in medicine still predominantly consists of white, heterosexual, cisgender males.

Compassionate, Comprehensive, & Personalized

care for adult and pediatric patients with neurological conditions, including:

 Head Injury/Concussion  Epilepsy/Seizures  Headache/Migraine  Neck/Back Pain  Sleep Disorders  Movement Disorders  Parkinson’s Disease  Tremors  Alzheimer’s Disease  Dementia  Muscle Weakness  Carpal Tunnel Syndrome  Sciatica  Neuromuscular Disease  Muscular Dystrophy  Dizziness  Numbness  Stroke  Multiple Sclerosis  ALS  And other neurological disorders

612.879.1500

NoranClinic.com

Blaine | Edina | Lake Elmo/Woodbury | Lakeville | Minneapolis | Plymouth Changing a paradigm

In recent years, we have seen a significant emphasis being given to non-cognitive domains related to interpersonal communication and professionalism, with the intent to counter the above-mentioned trend within medical schools, professional medical institutions and hospitals. To support the need for non-cognitive competencies to be included in the selection process, there is evidence that competencies, such as teamwork and accountability, correlate with improved patient outcomes, patient satisfaction and adherence to treatment plans. The AAMC and the Accreditation Council for Graduate Medical Education (ACGME) both emphasize professionalism and communication skills as essential components of a well-rounded physician by including them into their assessment frameworks. However, at present time, we do not routinely use reproducible, validated, easy-to-use tools to measure the professionalism and communication domains in our interview processes. Instead, we rely on one-on-one interviews as the most important way to assess applicants’ interpersonal and communication skills, maturity, interest in the field, dependability and honesty. This process is not standardized and lacks inter-rater reliability, and therefore, its outcomes are biased by personal preferences of interviewers.

The SJT can measure non-cognitive skills, such as those related to professionalism and attributes of an individual that mimic actual real-life events related to specific professions. Its ability to objectively measure interpersonally oriented skills is what makes it attractive for the selection processes.

Development of the SJT starts with analysis of professional critical events that present specific constructs or domains relevant to the profession. Experts and non-experts are asked to develop a list of best, worst and in-between responses or approaches to address different situations and incidents. In the final step, a scoring key is developed based on the ability of specific responses to differentiate between experts and novices and the correlation with job performance in specific domains. The administration of SJT is relatively simple. It comes in a series of online scenarios for which the candidate is asked to rank the responses from the best to the worst. It can be administered as a written test, a video or a cartoon.

As mentioned previously, AAMC has developed its own SJT which measures eight pre-professional competencies relevant to medical school applicants – service orientation, social skills, cultural competencies, teamwork, ethical responsibility to self and others, reliability and dependability, resilience and adaptability and capacity for improvement. Understanding the importance of the inclusion of the professionalism domains into candidate selection, the SJT presents around 30 scenarios with multiple responses required to each one. Test takers select a response most closely related to how they would handle that situation. The entire test takes about 75 minutes to complete. Since SJT requests individuals to

respond to the questions to the best of their ability, there is a possibility that the individual will respond with the option they feel will satisfy the exam, and not necessarily respond as they would act in the situation. This has been studied and the impact of fakability has been determined to be relatively small.

Creating a holistic review

The University of Minnesota Medical School is one medical school with two campuses. The mission of the Medical School states that it seeks to enroll and educate skilled, compassionate and socially responsible physicians. Furthermore, the school is looking to educate individuals who appreciate working in diverse communities with an interdisciplinary focus and within inter-professional teams. The aforementioned attributes and others are being screened for in the current admissions process, but using a valid and reliable assessment tool that can accurately identify these non-cognitive attributes in applicants is a necessary tool for holistic review.

There are both opportunities and challenges to including a non-cognitive assessment. The challenges include the added cost to taking an examination on top of an already expensive endeavor. There are no fees in this initial year for taking the test, though the AAMC plans to add them moving forward. Exactly how they will be charged is undetermined at this point. It already costs on average $1,400 for an applicant to apply to medical school. Furthermore, applicants would need to spend additional time preparing on top of the many hours of academic coursework and co-curricular activities. Adding another requirement could cause additional stress and anxiety to an already demanding process. It will also require committee members to take additional time to learn a new tool, and some might feel unwilling to embrace new and unproven methodologies.

The opportunities, however, outweigh the challenges in the long run. A non-cognitive assessment directly compliments holistic review and fits with the mission of many medical schools. Implementing an assessment like the SJT has the potential to make medical education more accessible to a wider audience. There is already plenty of data showing that lower MCAT scores keep under-represented in medicine (UIM) populations out of medical education in greater numbers. The holistic review process takes a great deal of time and involves many people. A non-cognitive assessment could help make the review process more efficient for the institution and potentially free up time for an already highly committed group of people.

Finally, and maybe most importantly, could a non-cognitive assessment like the SJT be as highly valued as the MCAT? It would be worth figuring out, so that when we are speaking about applicant access and medical student failure and success, our first question is something other than “What is the candidate’s MCAT score?”

Improving resident selection

In parallel to the efforts by the U of M Medical School, U of M Graduate Medical Education Office, under the leadership of Drs. John Andrews and Michael Cullen, has also developed SJT geared towards resident selection. The model includes the following dimensions of professionalism – conscientiousness, aspiring to excellence, integrity, accountability, teamwork, patient centered care and stress tolerance. The resident-specific

Situational Judgment Testing to page 244

Moving online creating a healthier Minnesota

Juniper offers evidence-based programs that help prevent escalation of disease, reduce hospital admissions, lower health care costs and improve independence. Our nationally recognized locally delivered programs concentrate on fall prevention, managing and preventing diabetes, managing other chronic conditions and pain.

In March all of our community-based courses were canceled due to COVID-19. With a network of 160 statewide partner programs, and over 850 certified program leaders we can help your patients overcome the challenges of self-isolation and take charge of their health.

We are pleased to offer these programs in an online setting that can now be accessed in the safety of your patients homes. The live, online courses are facilitated by trained and certified leaders in HIPAA-secure, easy to use video conferencing settings.

Refer your patients to Juniper directly or start a program through your clinic or in your community.

For more information please visit: www.yourjuniper.org or call 1-888-215-2174

3Situational Judgment Testing from page 23

SJT has been shown to predict overall ACGME milestone performance and multisource professionalism assessment performance.

In the resident selection process, any given department generally receives about 1,000 applications each year. The screening process frequently includes scores on a cognitive test as the primary filter due to the simplicity of using a number. This is combined with review of application packets, which include the applicant’s medical school transcripts, AOA membership, letters of recommendation, personal statement and description of their research activities, work experiences and volunteer activities. Based on these, the final decision is made to interview about one-tenth of the applicants. In the actual interview process, we have used United States Medical Licensing Exam scores and assessment of applicants by faculty as the most important components of applicant selection. As discussed earlier, this is a flawed process, which selects a specific population of applicants and more often than not can work against minority applicants.

In the recent few years, the U of M anesthesiology residency program has worked on standardization of the interview process to minimize subjectivity. Several factors have led our residency program to look for opportunities to incorporate new measures of applicant non-cognitive domains into our process. Most importantly, our residency program is making a structured effort to increase the ratio of UIM applicants in our program. As part of this transition, we have started using standardized questions in our interview process in 201617. And in 2018, we have added SJT as part of the applicant interview process.

Unanswered questions

A lot remains to be investigated to inform the best use of a tool like SJT. We have yet to define what an appropriate weight of all the different new measures is in the decision-making process about applicants. Longitudinal studies of how inclusion of situational judgment tests change our successful applicant pools are not well described. We have yet to answer questions about how new measures correlate with medical student, resident and attending work performance.

However, at the U of M Medical School, we believe a standardized holistic process that relies less on traditional academic performance measures and emphasizes interpersonal and professionalism dimensions of individual applicants can diversify our medical population to better reflect population statistics.

We believe medicine as a whole has to find ways to diversify the physician population to better represent the U.S. population. A tool like SJT is a move in the right direction toward a more balanced physician pool in the future.

Mojca Remskar, MD, PhD, MACM, is a professor and the Executive Vice Chair in the Department of Anesthesiology at the University of Minnesota Medical School.

Dimple Patel, MS, is the Associate Dean of Admissions at the University of Minnesota Medical School.

3Understanding Developmental Trauma from page 21

disrupt connectedness and the ability to co-regulate. Ultimately, it is our quest for safety that determines our physiological state. At birth for mammals, the bidirectional neural communication between the face and the heart forms the core of the social engagement system with the primary caregiver. This involves variables such as prosody, gaze, facial expressivity, mood and affect, posture during social engagement, emotional state regulation, and sound hypersensitivities. These assist in creating a co-regulation between the child and parent in which a caregiver can transmit safety to the child and allow the child to then relax into the comfort and influence of that parent, even after having experienced trauma and abuse. I call this “borrowing serenity.”

Developmental trauma is pervasive. Fully two-thirds of children in the United States report at least one traumatic event by age 16. In 2013, the national average of child abuse and neglect victims was 679,000 children, or 9.1 victims per 1,000 children. Developmental trauma crosses socioeconomic status, race, gender and all other social boundaries. Its impact is lifelong.

Those who have grown up with Developmental Trauma are at higher risk for significant health problems as adults. According to the American Society for

the Positive Care of Children, these problems include alcoholism, depression, drug abuse, eating disorders, obesity, high-risk sexual behaviors, smoking, suicide and certain chronic diseases. A number of research studies have found a correlation between childhood abuse and diagnosed mental health disorders in adulthood. The occurrence of emotional, sexual and physical child abuse is found to be a leading risk factor for the development of depression. Sexual child abuse A child cannot remove itself from a and family violence were found to be significant risk home in order to protect itself. factors for anxiety disorders. Strong correlations have also been found between family violence or physical neglect and later substance abuse. In summary, much of the care that physicians provide to adults today can be attributed to the care that patient received as an infant.

Treatment options

Treatment for developmental trauma involves education, awareness and safety. All caregivers and treatment team members need education to understand the unique features of developmental trauma so that behaviors and effect can be correctly interpreted. What is seen as anti-social in a typical environment actually makes sense in the context for which it was created. There is a reason for this behavior and the feelings supporting it, and rarely does it have to do with wanting to be difficult or oppositional. It is about coping and surviving, and the professional’s task is to make a correct interpretation of the context that created and shaped these behaviors. Writing a prescription is insufficient treatment. While it might dull the effect, it does little to assist in facilitating co-regulation between A Place To Be Your Best. a parent and child - the mechanism that most promotes neural healing. Psychotherapists and caregivers should be educated in modalities and Dr. Julie Benson, interventions specific to the unique needs of these children, such as Hope MN Academy Family Physician of the Year for Healing through ATTACh.org, Dyadic Developmental Psychotherapy, Trust-Based Relational Intervention, Occupational Therapy Sensory Interventions and others. It is imperative that all professionals who work with children understand the deleterious impact of developmental trauma. For too many children, their responses to perceived danger have created outcomes that have made their conditions worse because of a lack of understanding on the part of a treating professional. It is our task to assess the full context of the behaviors, as well as the detailed history of the children and parents with whom we work. Only then are we best able to assist them in their complete healing and a promising future.

POSITIONS AVAILABLE:

OB GYN & FAMILY MEDICINE – Full-scope practice available (ER, OB, C-Section, Hospitalist, Clinic)

• Independent/growing system • Located in the heart of lakes country, Staples, MN • Critical access hospital with 5 primary clinics and a senior living facility • 15 family medicine physicians and 16 advanced practice clinicians • Competitive salary, benefits, and sign-on bonus available

Norm Thibault, PhD, LMFT, is the current President of the Association for Training on Trauma and Attachment in Children and is the Founding Owner and CEO of Three Points Center, a Residential Treatment Center, with campuses in Utah and North Carolina for treating adopted adolescents and their families.

Contact Michael Paul at 218.894.8633, or michaelpaul@lakewoodhealthsystem.com

3The Importance of Medical Associations from page 9

Physicians 30+ year legacy of providing confidential support for physicians experiencing substance use disorders by launching a new initiative aimed at expanding our wellness offerings called the Physicians Wellness Collaborative (PWC) in 2020. PWC provides eight confidential, free counseling sessions for physicians, residents, medical students and their immediate family members.

We’ve seen an incredible response to our work over the past year and a half. We have recently extended our free counseling services for our Advance Practice Providers colleagues. We have also partnered with the Metro Minnesota Council on Graduate Medical Education on two innovative new programs: setting up proactive wellness appointments with licensed mental health professionals for residents as they enter into their program and a pilot program using a mobile app, PWC PeerConnect, which is confidential space for residents to connect with recent residency graduates and practicing physicians who are passionate about supporting residents as Peer Support Mentors.

We are also dedicated to continuing to address systemic barriers to physician wellness, including partnering with other organizations to change medical licensure language which prevents many Minnesota physicians from accessing mental health services.

How are you addressing institutional or systemic racism in health care?

The depth and urgency of the need to combat systemic racism has been reinforced over the past year, and we have seen strong leadership around health care system change from local physician and medical associations. TCMS will continue to focus on how systemic racism creates conditions outside the clinic that lead to ill health.

Public health advocacy around menthol and other flavored tobacco – a key driver in racial disparities in tobacco use and ultimately tobaccorelated death and disease—will continue to be an emphasis of our organization’s work. Additionally, we are committed to continuing our work to address the harm of sugary drinks which, like tobacco, are disproportionately and aggressively marketed toward Black and Latino children. We are actively working to extend our model of physician education and advocacy to support communities working on other pressing health issues like universal school meals, a living wage and new models of public safety.

What would you like physicians to know about the Twin Cities Medical Society?

Twin Cities Medical Society has provided me with opportunities to advocate on issues that are close to my heart, to mentor the next generation of physician-advocates, and to connect with an incredible community of physician activists. I warmly invite you to attend one of the engagement opportunities we will be hosting this Fall, and join us in working to create real change for the health of our community!

Sarah Traxler, MD, serves as Chief Medical Officer of Planned Parenthood North Central States and is President of Twin Cities Medical Society. She is a board-certified obstetrician gynecologist. Dr. Traxler has special interests in reproductive health policy and healthcare for the underserved.

MEDICAL ILLUSTRATION AND GRAPHIC DESIGN

PERFECT CIRCLE CREATIVE is a full service Minneapolis-based design studio specializing in the needs of the healthcare delivery system. Whether it be for academia, marketing, patient education or corporate communication, we'll work with you to create technically accurate, deeply detailed and aesthetically pleasing images that clearly convey information and messaging to your target audience.

PERFECTCIRCLECREATIVE.COM

custom illustration • graphic design • web design • online marketing • promotional merchandise phone: 612.986.2371 email: pete@perfectcirclecreative.com

Helping physicians communicate with physicians for over 30 years.

MINNESOTA

AUGUST 2018

PHYSICIAN

THE INDEPENDENT MEDICAL BUSINESS JOURNAL Volume XXXII, No. 05

Physician/employer direct contracting

Exploring new potential

BY MICK HANNAFIN

With the continuing escalation of health care costs, large and midsized selfinsured employers are once again looking for an edge to manage their medical plan costs and their bottom line. They understand that they are ultimately funding health care as they pay for their population’s claims.

Many of these employers have employed the same overarching set of strategies: shop for a new carrier that is willing to lower the administrative costs or underprice the risk,

Physician/employer direct contracting to page 124 CAR T-cell therapy

Modifying cells to fight cancer

BY VERONIKA BACHANOVA, MD, PHD

University of Minnesota Health is now among the few selected centers in the nation to offer two new immunotherapy drugs for the treatment of diffuse large B-cell lymphoma. Both drugs—Yescarta and Kymriah—are part of an emerging class of treatments, called CAR T-cell therapies, that harness the power of a patient’s own immune system to eliminate cancer cells.

CAR T-cell therapy involves drawing blood from patients and separating out the T cells. Using a disarmed virus, the patient’s own T cells are genetically engineered to produce chimeric antigen receptors, or CARs, that allow them to recognize and attach to a specific protein, or antigen, on tumor cells. This process takes place in a laboratory and takes about 14 days. After receiving the modification, the engineered CAR-T cells are infused into the patient, where they recognize and attack cancer cells. Kymriah received initial FDA approval in 2017 for the treatment of pediatric acute lymphoblastic leukemia.

CAR T-cell therapy to page 144

Advertising in Minnesota Physician is, by far, the most cost-effective method of getting your message in front of the over 17,000 doctors licensed to practice in Minnesota. Among the many ways we can help your practice: • Share new diagnostic and therapeutic advances • Develop and enhance referral networks • Recruit a new physician associate Advertise! IN MINNESOTA PHYSICIAN

www.mppub.com (612) 728-8600

Unique Practice Opportunity

Join an established independent internal medicine practice

Be your own boss in a collaborative business model with a healthcare philosophy that puts patients first and allows physicians to have complete control of their practice. The specialties we are looking for are:

Internal Medicine, Family Practice, Preventive Medicine, Cardiology, Dermatology, Allergist, or any other office-based specialty. Preferred Credentials are MD, DO, PA, and NP.

• Beautiful newly remodeled space in a convenient location • Competitive Wages and a great Professional Support Staff

Contact Mitchell for more information | mitch@brandtmgmt.com 6565 France Ave S Ste 350 Edina

PHYSICIAN

Minneapolis VA Health Care System

Metro based opportunities include:

• Chief of General Internal Medicine • Chief of Cardiology • Cardiologist • Internal Medicine/Family Practice • Gastroenterologist • Psychiatrist • Tele-ICU (Las Vegas, NV) • Nephrologist Practice Opportunities throughout Greater Minnesota:

Our nation faces an unprecedented number of individuals who having served their country now receive health care benefits through the VA system. We offer an opportunity for you to serve those who have served their country providing community based health care in modern facilities with access to world-leading research and research opportunities. We provide outstanding benefits with less stress and burnout than many large system policies create. We allow you to do what you do, best – care for patients.

Ely VA Clinic

Current opportunities include:

Internal Medicine/Family Practice

Hibbing VA Clinic

Current opportunities include:

Internal Medicine/Family Practice

US citizenship or proper work authorization required. Candidates should be BE/BC. Must have a valid medical license anywhere in US. Background check required. EEO Employer. Possible Education Loan Repayment • Competitive Salary • Excellent Benefits • Professional Liability Insurance with Tail Coverage

For more information on current opportunities, contact: Yolanda Young: Yolanda.Young2@va.gov • 612-467-4964

One Veterans Drive, Minneapolis, MN 55417 • www.minneapolis.va.gov

3Private Equity in Health Care from page 11

Conclusion

reviews are mixed. In the best situations, the physicians have no further business or regulatory responsibility, and see patients as they walk through the door. The physician relies upon the management company to assure everything is profitable and that the physician will always have a job. The equity fund has the incentive to increase every physician’s compensation so the physician can ignore the business side of the medical environment.

But this type of transaction and practice structure is not without risk. If the arrangement is not properly structured, some physicians feel oppressed and part of a corporate machine focused upon maximizing profits to the detriment of patient care. There may also be reduced ability to spend the time with a patient that the patient needs. Finally, joining a larger corporate structure may reduce physician autonomy and entrepreneurial spirit. The practice becomes a job instead of a profession. Most significantly, if the management company fails (as has happened frequently in the past), the physician’s practice may be forced to close requiring the physician to find a new practice setting. Physicians may also be subject to “clawbacks” of the original payments if practice liabilities are higher than originally expected. These may include regulatory non-compliance, billing errors and issues, or other unanticipated liabilities.

An equity fund/physician practice transaction can lead to many benefits for a practicing physician. Enhanced management expertise, savings through economies of scale and access to enhanced reimbursement rates can result in more financial resources for the practice of medicine and enhanced compensation for physicians. But the arrangements must be properly structured to protect the physician owners’ financial and practice interests. In considering such a transaction, it is important for physicians to work with legal experts with This type of transaction and practice structure is not without risk. industry knowledge. Legal issues related to health care regulatory compliance, tax, ERISA, and securities law all must be fully considered when entering into such a deal. Randal Schultz, JD is a health care attorney at Lathrop GPM who counsels health care organizations of all types across the country for the development of business/financial structures, entity formation, program/product creation and regulatory compliance. Ben Peltier, JD, is a health care attorney at Lathrop GPM who focuses his practice on a wide range of transactional matters including contracting, mergers and acquisitions, regulatory guidance, governance and compliance.

Three patients. Who is at risk for diabetes?

1 in 3 adults are at risk!

When there are no signs or symptoms, you may not know until it’s too late. Act now. Screen your patients for type 2 diabetes. It’s easy. It’s covered. It will reduce their risk.

• Refer your at-risk patients to a proven lifestyle change program and help cut their risk of developing type 2 diabetes in half. • For patients who already have diabetes, send them to a quality diabetes self-management program to improve control and reduce complications. Find groups in Minnesota at

www.health.mn.gov/diabetes/programs

Minnesota Department of Health

DIABETES PROGRAM

David Schultz, MD Chief Executive Officer Nura Pain Clinics Peter Schultz, MD, MPH Medical Director Nura Pain Clinics

R. Scott Stayner, MD, PhD Medical Director Nura Surgery Centers A powerful resource for your patients in chronic pain.

Some physicians turn to Nura for help pinpointing the cause of pain. Some look to us for specific treatments or medication management, while others rely on Nura for total pain management of complex cases. In every case, we are here for the long term, with the resources, stability and commitment to make a genuine difference in the lives of patients. Our multi-disciplinary approach includes: • Medication Management • Physical Therapy • Warm Saltwater Pool • Behavioral Health • Interventional Procedures • Neurostimulation • Targeted Spinal Drug Delivery

If you have a patient struggling with chronic pain, please call our Provider Hotline at 763-537-1000. If the situation is urgent, we will do our very best to see your patient the same or next day.

Edina & Coon Rapids | nuraclinics.com | 763-537-1000

©2021 Nura PA. All rights reserved.