CDEM CORD Special Education Issue

ACOEP stands with all emergency physicians and providers on the front line. We thank you for your tireless work and effort.

ACOEP stands with all emergency physicians and providers on the front line. We thank you for your tireless work and effort.

Western Journal of Emergency Medicine: CDEM/CORD 2024 Special Education Issue

A Note from the Editors:

We are excited to publish the 9th issue of the Western Journal of Emergency Medicine (WestJEM) Education Issue and first year of a rolling decision process. Ov er 10 years ago a unique relationship was formed between WestJEM, the Council of Residency Director for Emergency Medicine and the Clerkship Directors of Emergency Medicine. The idea was to promote and di sseminate educational scholarship which has been accomplished over the past decade. Senior and junior r esearchers have an opportunity to publish in the education issue because of the diverse nature of our submis sion categories, ranging from original research to brief educational advances. A successful issue requires the courage of the authors to submit their work for peer review and we do our best to provide detailed feedback reg ardless of the final decision. Publication of the issue requires the commitment and hard work of the publication staff, leadership of the organizations, editors, and peer reviewers. We want to thank them all for their efforts and professionalism. The topics of this year’s education issue likely reflect the focus of educators as we ente red a post-covid reality. Many of the topics were related to innovative curriculums and focused on the benefi ts derived. There were also several articles that were dedicated to the administrative aspects of residency and fellowships and how that has changed after COVID. We have already started to receive and review submissions for next year’s education issue. The editorial staff review every submission on a rolling basis and, once accepted, the articles are available on PubMed. There are also no processing fees when accepted to the Education Issue. This is a great opportunity to submit your educational scholarship, thereby enhancing your professional development and disseminating your work to others. We are excited that this experiment has flourished, and we look forward to seeing your work in our 10th anniversary issue.

Jeffrey Love, MD

Georgetown University School of Medicine

Co-Editor of Annual Special Issue on Education Research and Practice

Douglas Ander, MD

Emory University

Co-Editor of Annual Special Issue on Education Research and Practice

The Western Journal of Emergency Medicine: Integrating Emergency Care with Population Health would like to thank The Clerkship Directors in Emergency Medicine (CDEM) and the Council of Residency Directors in Emergency Medicine (CORD) for helping to make this collaborative special issue possible.

Western Journal of Emergency Medicine:

Integrating Emergency Care with Population Health

Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded

JOURNAL FOCUS

Emergency medicine is a specialty which closely reflects societal challenges and consequences of public policy decisions. The emergency department specifically deals with social injustice, health and economic disparities, violence, substance abuse, and disaster preparedness and response. This journal focuses on how emergency care affects the health of the community and population, and conversely, how these societal challenges affect the composition of the patient population who seek care in the emergency department. The development of better systems to provide emergency care, including technology solutions, is critical to enhancing population health.

Table of Contents

BRIEF RESEARCH REPORT

S1 Virtual Interviews and the Pediatric Emergency Medicine Match Geography: A National Survey

A Baghdassarian, JA Bailey, D Caglar, M Eckerle, A Fang, K McVety, T Ngo, JA Rose, CG Roskind, MM Tavarez, FT Benedict, J Nagler, ML Langhan

S6 Perception of Quiet Students in Emergency Medicine: An Exploration of Narratives in the Standardized Letter of Evaluation

JK Quinn, J Mongelluzzo, A Nip, J Graterol, EH Chen

S10 Changes in Residency Applicant Cancellation Patterns with Virtual Interviews: A Single-site Analysis

M Bouldin, C Eastin, R Freeze-Ramsey, A Young, M von Dohlen, L Evans, T Eastin, S Greenberger

S15 Foundations of Emergency Medicine: Impact of a Standardized, Open-access, Core Content Curriculum on In-Training Exam Scores

J Jordan, N Wheaton, ND Hartman, D Loke, N Shekem, A Osborne, PL Weygandt, K Grabow Moore

S19 Integrating Hospice and Palliative Medicine Education Within the American Board of Emergency Medicine Model

R Goett, J Lyou, LR Willoughby, DW Markwalter, DL Gorgas, LT Southerland

S27 Staffing Patterns of Non-ACGME Fellowships with 4-Year Residency Programs: A National Survey

DA Haidar, LR Hopson, RV Tucker, J Koehler, N Theyyunni, N Klekowski, CM Fung

BRIEF EDUCATIONAL ADVANCES

S33 Nudge Theory: Effectiveness in Increasing Emergency Department Faculty Completion of Residency Assessments

A Gurley, C Jenkins, T Nguyen, A Woodall, J An

S36 The Effect of a Simulation-based Intervention on Emergency Medicine Resident Management of Early Pregnancy Loss

SD Bellew, E Lowing, L Holcomb

ORIGINAL RESEARCH

S41 Integration of Geriatric Education Within the American Board of Emergency Medicine Model

LT Southerland, LR Willoughby, J Lyou, RR Goett, DW Markwalter, DL Gorgas

S51 Emergency Medicine Resident Needs Assessment and Preferences for a High-value Care Curriculum

BH Lane, SK Mand, S Wright, S Santen, B Punches

Policies for peer review, author instructions, conflicts of interest and human and animal subjects protections can be found online at www.westjem.com.

2023 Gold Standard Reviewers

The WestJEM Special Issue in Educational Research & Practice couldn’t exist without our many reviewers. To all, we wish to express our sincerest appreciation for their contributions to this year’s success. Each year a number of reviewers stand out for their (1) detailed reviews, (2) grasp of the tenets of education scholarship and (3) efforts to provide feedback that mentors authors on how to improve. This year’s “Gold Standard” includes:

• Ignacio Calles/Jeffrey Riddel*

• April Choi, Kirlos Haroun, Linda Regan*

• Nathan Dreyfus/Jessie Werner*

• Eric Flounders, Samual Clarke*

• Rowan Kelner/Allie Beaulieu/Jeffrey Druck/Christine Raps*

• Andrew Kendle, Marcus Wooten, Simiao Li-Sauerwine*

• Matt Magda/Kevin Scott*

• Dan Mayer

• Joe-Ann Moser/Ben Schnapp*

• Daniela Ortiz/Tyson Pillow*

• Elspeth Pearce

• Jessica Pelletier/Albert Kim*

• Thaddeus Schmitt

• Nicole Schnabel/Laura Hopson*

• Samantha Stringer/Albert Kim*

• Olivia Urbanowicz, Edmond Irankunda, Sally Santen*

• Ivan Zvonar/Jon Ilgen*

*Mentored Peer Reviews from Emergency Medicine Education Fellowship Programs

CDEM/CORD Guest Consulting Editors

We would also like to recognize our guest consulting editors who assisted with pre-screening submissions during our initial peer-review stages. Thank you for all of your efforts and contributions.

CDEM

• Christine Stehman

• Andrew Ketterer

• Eric Shappell

• Sharon Bord

CORD

• Jenna Fredette

• Jaime Jordan

• Anne Messman

• Kendra Parekh

• William Soares III

• Paul “Logan” Weygandt Consulting Statistician/ Psychometrician

• David Way

Western Journal of Emergency Medicine:

Integrating Emergency Care with Population Health

Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded

Jeffrey N. Love, MD, Guest Editor

The George Washington University- Washington, District of Columbia

Danielle Hart, MD, Associate Guest Editor Hennepin County Medical Center- Minneapolis, Minnesota

Chris Merritt, MD, MPH, Associate Guest Editor

Alpert Medical School of Brown University-Providence, Rhode Island

Wendy Macias-Konstantopoulos, MD, MPH, Associate Editor Massachusetts General Hospital- Boston, Massachusetts

Danya Khoujah, MBBS, Associate Editor University of Maryland School of Medicine- Baltimore, Maryland

Patrick Joseph Maher, MD, MS, Associate Editor Ichan School of Medicine at Mount Sinai- New York, New York

Yanina Purim-Shem-Tov, MD, MS, Associate Editor Rush University Medical Center-Chicago, Illinois

Gayle Galletta, MD, Associate Editor University of Massachusetts Medical SchoolWorcester, Massachusetts

Resident Editors

AAEM/RSA

John J. Campo, MD Harbor-University of California, Los Angeles Medical Center

Tehreem Rehman, MD

Advocate Christ Medical Center

ACOEP

Justina Truong, DO Kingman Regional Medical Center

Section Editors

Behavioral Emergencies

Erin Dehon, PhD University of Mississippi Medical Center

Leslie Zun, MD, MBA Chicago Medical School

Marc L. Martel, MD Hennepin County Medical Center

Cardiac Care

Fred A. Severyn, MD University of Colorado School of Medicine

Michael C. Kurz, MD University of Alabama at Birmingham

Sam S. Torbati, MD

Cedars-Sinai Medical Center

Clinical Practice

Cortlyn W. Brown, MD Carolinas Medical Center

Casey Clements, MD, PhD Mayo Clinic

Patrick Meloy, MD Emory University

Nicholas Pettit, DO, PhD Indiana University

Eric Snoey, MD Alameda County Medical Center

David Thompson, MD University of California, San Francisco

Kenneth S. Whitlow, DO Kaweah Delta Medical Center

Critical Care

Christopher “Kit” Tainter, MD University of California, San Diego

Gabriel Wardi, MD University of California, San Diego

Joseph Shiber, MD University of Florida-College of Medicine

Matt Prekker MD, MPH Hennepin County Medical Center

Dan Mayer, MD, Associate Editor Retired from Albany Medical College- Niskayuna, New York

Julianna Jung, MD, Associate Guest Editor Johns Hopkins Hospital, Baltimore, Maryland

Douglas Franzen, MD, Associate Guest Editor Harborview Medical Center, Seattle, Washington

Gentry Wilkerson, MD, Associate Editor University of Maryland

Michael Gottlieb, MD, Associate Editor Rush Medical Center-Chicago, Illinois

Sara Krzyzaniak, MD Associate Guest Editor Stanford Universtiy-Palo Alto, California

Gavin Budhram, MD, Associate Editor Tufts University- Medford, Massachusetts

Susan R. Wilcox, MD, Associate Editor Massachusetts General Hospital- Boston, Massachusetts

Donna Mendez, MD, EdD, Associate Editor University of Texas-Houston/McGovern Medical School- Houston Texas

David Page, MD University of Alabama

Erik Melnychuk, MD Geisinger Health

Quincy Tran, MD, PhD University of Maryland

Disaster Medicine

Christopher Kang, MD Madigan Army Medical Center

Education

Danya Khoujah, MBBS University of Maryland School of Medicine

Jeffrey Druck, MD University of Colorado

John Burkhardt, MD, MA University of Michigan Medical School

Michael Epter, DO Maricopa Medical Center

ED Administration, Quality, Safety

David C. Lee, MD Northshore University Hospital

Gary Johnson, MD

Upstate Medical University

Brian J. Yun, MD, MBA, MPH

Harvard Medical School

Laura Walker, MD Mayo Clinic

León D. Sánchez, MD, MPH Beth Israel Deaconess Medical Center

William Fernandez, MD, MPH University of Texas Health-San Antonio

Emergency Medical Services

Daniel Joseph, MD Yale University

Joshua B. Gaither, MD University of Arizona, Tuscon

Julian Mapp

University of Texas, San Antonio

Shira A. Schlesinger, MD, MPH Harbor-UCLA Medical Center

Geriatrics

Cameron Gettel, MD Yale School of Medicine

Stephen Meldon, MD Cleveland Clinic

Luna Ragsdale, MD, MPH

Duke University

Health Equity

Emily C. Manchanda, MD, MPH

Boston University School of Medicine

Official Journal of the California Chapter of the American College

Douglas S. Ander, MD, Guest Editor Emory University School of Medicine-Atlanta, Georgia

Edward Ullman, MD, Associate Guest Editor Harvard University-Cambridge, Massachusetts

Abra Fant, MD, Associate Guest Editor Northwestern University Feinberg School of MedicineChicago, Illinois

Matthew Tews, DO, MS, Associate Guest Editor Indiana University School of Medicine, Augusta, Georgia

Rick A. McPheeters, DO, Associate Editor Kern Medical- Bakersfield, California

Elizabeth Burner, MD, MPH, Associate Editor University of Southern California- Los Angeles, California

Shahram Lotfipour, MD, MPH, Managing Associate Editor University of California, Irvine School of Medicine- Irvine, California

Mark I. Langdorf, MD, MHPE, Editor-in-Chief University of California, Irvine School of Medicine- Irvine, California

Mandy J. Hill, DrPH, MPH

UT Health McGovern Medical School

Infectious Disease

Elissa Schechter-Perkins, MD, MPH

Boston University School of Medicine

Ioannis Koutroulis, MD, MBA, PhD

Drexel University College of Medicine

Kevin Lunney, MD, MHS, PhD

University of Maryland School of Medicine

Robert Derlet, MD

Founding Editor, California Journal of Emergency Medicine

University of California, Davis

Stephen Liang, MD, MPHS

Washington University School of Medicine

Injury Prevention

Mark Faul, PhD, MA

Centers for Disease Control and Prevention

Wirachin Hoonpongsimanont, MD, MSBATS

Eisenhower Medical Center

International Medicine

Heather A.. Brown, MD, MPH

Prisma Health Richland

Taylor Burkholder, MD, MPH

Keck School of Medicine of USC

Christopher Greene, MD, MPH University of Alabama

Chris Mills, MD, MPH

Santa Clara Valley Medical Center

Shada Rouhani, MD

Brigham and Women’s Hospital

Legal Medicine

Melanie S. Heniff, MD, JD

Indiana University School of Medicine

Greg P. Moore, MD, JD

Madigan Army Medical Center

Statistics and Methodology

Shu B. Chan MD, MS

Resurrection Medical Center

Stormy M. Morales Monks, PhD, MPH

Texas Tech Health Science University

Soheil Saadat, MD, MPH, PhD University of California, Irvine

James A. Meltzer, MD, MS

Albert Einstein College of Medicine

Musculoskeletal

Juan F. Acosta DO, MS

Pacific Northwest University

Neurosciences

Antonio Siniscalchi, MD

Annunziata Hospital, Cosenza, Italy

Rick Lucarelli, MD

Medical City Dallas Hospital

William D. Whetstone, MD

University of California, San Francisco

Pediatric Emergency Medicine

Paul Walsh, MD, MSc

University of California, Davis

Muhammad Waseem, MD

Lincoln Medical & Mental Health Center

Deena Berkowitz, MD, MPH

Children’s National Hospital

Cristina M. Zeretzke-Bien, MD

University of Florida

Public Health

Jeremy Hess, MD, MPH

University of Washington Medical Center

Jacob Manteuffel, MD

Henry Ford Hospital

John Ashurst, DO

Lehigh Valley Health Network

Tony Zitek, MD

Kendall Regional Medical Center

Trevor Mills, MD, MPH

Northern California VA Health Care

Erik S. Anderson, MD

Alameda Health System-Highland Hospital

Technology in Emergency Medicine

Nikhil Goyal, MD

Henry Ford Hospital

Phillips Perera, MD

Stanford University Medical Center

Trauma

Pierre Borczuk, MD

Massachusetts General Hospital/Havard Medical School

Toxicology

Brandon Wills, DO, MS

Virginia Commonwealth University

Jeffrey R. Suchard, MD

University of California, Irvine

Ultrasound

J. Matthew Fields, MD

Thomas Jefferson University

Shane Summers, MD

Brooke Army Medical Center

Robert R. Ehrman

Wayne State University

Ryan C. Gibbons, MD Temple Health

Available in MEDLINE, PubMed, PubMed Central, CINAHL, SCOPUS, Google Scholar, eScholarship, Melvyl, DOAJ, EBSCO, EMBASE, Medscape, HINARI, and MDLinx Emergency Med. Members of OASPA. Editorial and Publishing Office: WestJEM/Depatment of Emergency Medicine, UC Irvine Health, 333 City Blvd, West, Rt 128-01, Orange, CA 92868, USA. Office: 1-714-456-6389; Email: Editor@westjem.org.

Volume 25, Issue 4.1: May 2024

Western Journal of Emergency Medicine:

Integrating Emergency Care with Population Health

Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded

Editorial Board

Amin A. Kazzi, MD, MAAEM

The American University of Beirut, Beirut, Lebanon

Brent King, MD, MMM University of Texas, Houston

Christopher E. San Miguel, MD Ohio State University Wexner Medical Center

Daniel J. Dire, MD University of Texas Health Sciences Center San Antonio

Douglas Ander, MD Emory University

Edward Michelson, MD Texas Tech University

Edward Panacek, MD, MPH University of South Alabama

Francesco Della Corte, MD Azienda Ospedaliera Universitaria “Maggiore della Carità,” Novara, Italy

Jaqueline Le, MD Desert Regional Medical Center

Jeffrey Love, MD The George Washington University School of Medicine and Health Sciences

Katsuhiro Kanemaru, MD University of Miyazaki Hospital, Miyazaki, Japan

Leslie Zun, MD, MBA Chicago Medical School

Linda S. Murphy, MLIS University of California, Irvine School of Medicine Librarian

Pablo Aguilera Fuenzalida, MD Pontificia Universidad Catolica de Chile, Región Metropolitana, Chile

Peter A. Bell, DO, MBA Baptist Health Sciences University

Peter Sokolove, MD University of California, San Francisco

Rachel A. Lindor, MD, JD Mayo Clinic

Robert Suter, DO, MHA UT Southwestern Medical Center

Robert W. Derlet, MD University of California, Davis

Scott Rudkin, MD, MBA University of California, Irvine

Scott Zeller, MD University of California, Riverside

Advisory Board

Elena Lopez-Gusman, JD

California ACEP American College of Emergency Physicians

Jennifer Kanapicki Comer, MD FAAEM

California Chapter Division of AAEM Stanford University School of Medicine

DeAnna McNett

American College of Osteopathic Emergency Physicians

Kimberly Ang, MBA UC Irvine Health School of Medicine

Randall J. Young, MD, MMM, FACEP California ACEP American College of Emergency Physicians Kaiser Permanente

Mark I. Langdorf, MD, MHPE, MAAEM, FACEP

UC Irvine Health School of Medicine

Robert Suter, DO, MHA American College of Osteopathic Emergency Physicians UT Southwestern Medical Center

Shahram Lotfipour, MD, MPH FAAEM, FACEP UC Irvine Health School of Medicine

Jorge Fernandez, MD, FACEP UC San Diego Health School of Medicine

Steven H. Lim Hoon Chin, MD Changi General Hospital, Simei, Singapore

Wirachin Hoonpongsimanont, MD, MSBATS

Siriraj Hospital, Mahidol University, Bangkok, Thailand

Editorial Staff

Isabelle Nepomuceno, BS Executive Editorial Director

Emily Kane, BS WestJEM Editorial Director

Stephanie Burmeister, MLIS WestJEM Staff Liaison

Cassandra Saucedo, MS Executive Publishing Director

Nicole Valenzi, BA WestJEM Publishing Director

June Casey, BA Copy Editor

Official Journal of the California Chapter of the American College of Emergency Physicians, the America College of Osteopathic Emergency Physicians, and the California Chapter of the American Academy of Emergency Medicine

Available in MEDLINE, PubMed, PubMed Central, Europe PubMed Central, PubMed Central Canada, CINAHL, SCOPUS, Google Scholar, eScholarship, Melvyl, DOAJ, EBSCO, EMBASE, Medscape, HINARI, and MDLinx Emergency Med. Members of OASPA. Editorial and Publishing Office: WestJEM/Depatment of Emergency Medicine, UC Irvine Health, 333 City Blvd, West, Rt 128-01, Orange, CA 92866, USA Office: 1-714-456-6389; Email: Editor@westjem.org

Western Journal of Emergency Medicine:

Integrating Emergency Care with Population Health

Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded

This open access publication would not be possible without the generous and continual financial support of our society sponsors, department and chapter subscribers.

Professional Society Sponsors

American College of Osteopathic Emergency Physicians California ACEP

Academic Department of Emergency Medicine Subscriber

Alameda Health System-Highland Hospital Oakland, CA

Ascension Resurrection Chicago, IL

Arnot Ogden Medical Center Elmira, NY

Atrium Health Wake Forest Baptist Winston-Salem, NC

Baylor College of Medicine Houston, TX

Baystate Medical Center Springfield, MA

Beth Israel Deaconess Medical Center Boston, MA

Brigham and Women’s Hospital Boston, MA

Brown University-Rhode Island Hospital Providence, RI

Carolinas Medical Center Charlotte, NC

Cedars-Sinai Medical Center Los Angeles, CA

Cleveland Clinic Cleveland, OH

Desert Regional Medical Center Palm Springs, CA

Eisenhower Health Rancho Mirage, CA

State Chapter Subscriber

Arizona Chapter Division of the American Academy of Emergency Medicine

California Chapter Division of the American Academy of Emergency Medicine

Florida Chapter Division of the American Academy of Emergency Medicine

International Society Partners

Emory University Atlanta, GA

Franciscan Health Carmel, IN

Geisinger Medical Center Danville, PA

Healthpartners Institute/ Regions Hospital Minneapolis, MN

Hennepin Healthcare Minneapolis, MN

Henry Ford Hospital Detroit, MI

Henry Ford Wyandotte Hospital Wyandotte, MI

Howard County Department of Fire and Rescue Marriotsville, MD

Icahn School of Medicine at Mt Sinai New York, NY

Indiana University School of Medicine Indianapolis, IN

INTEGRIS Health Oklahoma City, OK

Kaweah Delta Health Care District Visalia, CA

Kent Hospital Warwick, RI

Kern Medical Bakersfield, CA

Emergency Medicine Association of Turkey Lebanese Academy of Emergency Medicine Mediterranean Academy of Emergency Medicine

California Chapter Division of AmericanAcademy of Emergency Medicine

Loma Linda University Medical Center

Loma Linda, CA

Louisiana State University Shreveport Shereveport, LA

Massachusetts General Hospital/ Brigham and Women’s Hospital/ Harvard Medical Boston, MA

Mayo Clinic in Florida Jacksonville, FL

Mayo Clinic College of Medicine in Rochester Rochester, MN

Mayo Clinic in Arizona Phoeniz, AZ

Medical College of Wisconsin Affiliated Hospital Milwaukee, WI

Mount Sinai Medical Center Miami Beach Miami Beach, FL

Mount Sinai Morningside New York, NY

New York University Langone Health New York, NY

North Shore University Hospital Manhasset, NY

NYC Health and Hospitals/ Jacobi New York, NY

Ochsner Medical Center New Orleans, LA

Great Lakes Chapter Division of the AmericanAcademyofEmergencyMedicine

Tennessee Chapter Division of the AmericanAcademyofEmergencyMedicine

Norwegian Society for Emergency Medicine Sociedad Argentina de Emergencias

Ohio State University Wexner Medical Center Columbus, OH

Oregon Health and Science University Portland, OR

Penn State Milton S. Hershey Medical Center Hershey, PA

Poliklinika Drinkovic Zagreb, Croatia

Prisma Health/ University of South Carolina SOM Greenville Greenville, SC

Rush University Medical Center Chicago, IL

Rutgers Robert Wood Johnson Medical School New Brunswick, NJ

St. Luke’s University Health Network Bethlehem, PA

Southern Illinois University School of Medicine Springfield, IL

Stony Brook University Hospital Stony Brook, NY

SUNY Upstate Medical University Syracuse, NY

Temple University Philadelphia, PA

Texas Tech University Health Sciences Center

El Paso, TX

Uniformed Services Chapter Division of the American Academy of Emergency Medicine Virginia Chapter Division of the American Academy of Emergency Medicine

forEmergencyMedicine

To become a WestJEM departmental sponsor, waive article processing fee, receive print and copies for all faculty and electronic for faculty/residents, and free CME and faculty/fellow position advertisement space, please go to http://westjem.com/subscribe or contact:

Stephanie Burmeister

WestJEM Staff Liaison

Phone: 1-800-884-2236

Email: sales@westjem.org

Sociedad Chileno Medicina Urgencia ThaiAssociation

Western Journal of Emergency Medicine:

Integrating Emergency Care with Population Health

Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded

This open access publication would not be possible without the generous and continual financial support of our society sponsors, department and chapter subscribers.

Professional Society Sponsors

American College of Osteopathic Emergency Physicians

California ACEP

Academic Department of Emergency Medicine Subscriber

The University of Texas Medical Branch Galveston, TX

UT Health Houston McGovern Medical School Houston, TX

Touro University College of Osteopathic Medicin Vallejo, CA

Trinity Health Muskegon Hospital Muskegon, MI

UMass Memorial Health Worcester, MA

University at Buffalo Program Buffalo, NY

University of Alabama, Birmingham Birmingham, AL

University of Arizona College of Medicine-Tucson Little Rock, AR

University of Arkansas for Medical Sciences Galveston, TX

University of California, Davis Medical Center Sacramento, CA

University of California San Francisco General Hospital San Francisco, CA

University of California San Fracnsico Fresno Fresno, CA

University of Chicago Chicago, IL

University of Cincinnati Medical Center/ College of Medicine Cincinnati, OH

University of Colorado Denver Denver, CO

University of Florida, Jacksonville Jacksonville, FL

University of Illinois at Chicago Chicago, IL

University of Iowa Hospitals and Clinics Iowa City, IA

University of Kansas Health System Kansas City, IA

University of Louisville Louisville, KY

University of Maryland School of Medicine Baltimore, MD

University of Miami Jackson Health System Miami, FL

University of Michigan Ann Arbor, MI

University of North Dakota School of Medicine and Health Sciences Grand Forks, ND

University of Southern Alabama Mobile, AL

State Chapter Subscriber

Arizona Chapter Division of the American Academy of Emergency Medicine

California Chapter Division of the American Academy of Emergency Medicine

Florida Chapter Division of the American Academy of Emergency Medicine

International Society Partners

Emergency Medicine Association of Turkey Lebanese Academy of Emergency Medicine

Mediterranean Academy of Emergency Medicine

California Chapter Division of American Academy of Emergency Medicine

University of Southern California Los Angeles, CA

University of Vermont Medical Cneter Burlington, VA

University of Virginia Health Charlottesville, VA

University of Washington - Harborview Medical Center Seattle, WA

University of Wisconsin Hospitals and Clinics Madison, WI

UT Southwestern Medical Center Dallas, TX

Franciscan Health Olympia Fields Phoenix, AZ

WellSpan York Hospital York, PA

West Virginia University Morgantown, WV

Wright State University Boonshoft School of Medicine Fairborn, OH

Yale School of Medicine New Haven, CT

Great Lakes Chapter Division of the American Academy of Emergency Medicine

Tennessee Chapter Division of the

American Academy of Emergency Medicine Uniformed Services Chapter Division of the

Academy of Emergency Medicine Virginia Chapter Division of the American Academy of Emergency Medicine

Norwegian Society for Emergency Medicine Sociedad Argentina de Emergencias Sociedad Chileno Medicina Urgencia Thai Association for Emergency Medicine

To become a WestJEM departmental sponsor, waive article processing fee, receive print and copies for all faculty and electronic for faculty/residents, and free CME and faculty/fellow position advertisement space, please go to http://westjem.com/subscribe or contact:

Stephanie Burmeister

WestJEM Staff Liaison

Phone: 1-800-884-2236

Email: sales@westjem.org

EDUCATION SPECIAL ISSUE:BRIEF RESEARCH REPORT

VirtualInterviewsandthePediatricEmergencyMedicineMatch

Geography:ANationalSurvey

AlineBaghdassarian,MD,MPH*†

JessicaA.Bailey,MD‡

DeryaCaglar,MD§∥

MichelleEckerle,MD,MPH¶#

AndreaFang,MD**

KatherineMcVety,MD††‡‡ ThuyNgo,DO,MEd§§

JerriA.Rose,MD∥∥¶¶

CindyGanisRoskind,MD##

MelissaM.Tavarez,MD,MS***

FrancesTurcotteBenedict,MD,MPH†††‡‡‡ JoshuaNagler,MD,MHPEd§§§∥∥∥

Authorscontinuedatendofpaper

*InovaL.J.MurphyChildren'sHospital,DepartmentofPediatrics, FallsChurch,Virginia

† UniversityofVirginia,SchoolofEducation,Charlottesville,Virginia ‡ OregonHealth&ScienceUniversity,DepartmentofPediatricsand EmergencyMedicine,Portland,Oregon

§ UniversityofWashington,DepartmentofPediatrics,Seattle,Washington

∥ SeattleChildren’sHospital,DepartmentofPediatrics, Seattle,Washington

¶ UniversityofCincinnatiCollegeofMedicine,DepartmentofPediatrics, Cincinnati,Ohio

# CincinnatiChildren’sHospital,DepartmentofPediatrics,Cincinnati,Ohio

Affi liationscontinuedatendofpaper

SectionEditors:KendraParekh,MD,andChrisMerritt,MD

Submissionhistory:SubmittedNovember15,2023;RevisionreceivedFebruary10,2024;AcceptedFebruary21,2024

ElectronicallypublishedMarch14,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18581

Introduction: Virtualinterviews(VI)arenowapermanentpartofpediatricemergencymedicine(PEM) recruitment,especiallygiventhecostandequityadvantages.Yetinabilitytovisitprogramsinpersoncan impactdecision-making,leadingapplicantstoapplytomoreprograms.Moreover,thecostadvantagesof VImayencourageapplicantstoapplytoprogramsfartherawaythantheymightotherwisehavebeen willingorabletotravel.Thiscouldcreateunnecessarystrainonprograms.Weconductedthisstudyto determinewhetherPEMfellowshipapplicantswouldapplytoalargernumberofprogramsandindifferent geographicpatternswithVI(2020and2021)ascomparedtoin-personinterviews(2018and2019).

Methods: WeconductedananonymousnationalsurveyofallPEMfellowscomparingtwocohorts: currentfellowswhointerviewedinperson(appliedin2018/2019)andfellowswhounderwentVIsin2020/ 2021(currentfellowsandthoserecentlymatchedin2021).ThestudytookplaceinMarch–April2022. Questionsfocusedongeographicconsiderationsduringinterviewsandthematch.Weuseddescriptive statistics,chi-squareand t-testsforanalysis.

Results: Overallresponseratewas42%(231/550);32%(n = 74)interviewedinpersonand68% (n = 157)virtually.Fellowsappliedtoamedianof4/6geographicregions(interquartilerange2,5).Most appliedforfellowshipbothinthesameregionasresidency(216,93%)andoutside(192,83%).Onlythe PacificregionsawastatisticallysignificantincreaseinapplicantsduringVI(59.9%vs43.2%, P = 0.02). Therewasnostatisticaldifferenceinthenumberofprogramsappliedtoduringin-personvsVI (meandifference(95%confidenceinterval0.72, 2.8 – 4.2).Amajoritymatchedintheirpreferred statebothduringVI(60.4%)andin-personinterviews(65.7%).Thedifferencewasnotstatistically significant(P = 0.45).

Conclusion: WhilemorePEMfellowshipapplicantsappliedoutsidethegeographicareawheretheir residencywasandtothePacificregion,therewasnooverallincreaseinthenumberofprogramsor geographicareasPEMapplicantsappliedtoduringVIascomparedtoin-personinterviewseasons.As thiswasthe firsttwoyearsofVI,ongoingdatacollectionwillfurtheridentifytrendsandtheimpactofVI. [WestJEmergMed.2024;25(4.1)1–5.]

INTRODUCTION

Since2020,virtualinterviews(VI)havebeenpreferredfor traineerecruitment.1 Withthebenefitsoflowercostand greaterequity,itislikelytoremainapermanentpartof recruitment,despiteageneralpreferenceforface-to-face interviews.2–5 TheVIprocessandassociatedperceptions havebeendescribedintheliterature.2,3,6–9 Theinabilityto visitaprograminpersoncanimpactdecision-makingduring ranking,4,10–14 andanincreasednumberofapplications couldcreateunduestrainonprograms.15–17

Geographiclocation,senseof “fit,” andprogram leadershipweredescribedasmajorcontributorsto applicants’ rankpreference.18 Anationalcohortofpediatric emergencymedicineprogramdirectors(PEMPD),inajoint statement,raisedconcernthatVIcouldleadapplicantsto applytomoreprogramsandtoprogramsfartherawaythan theymaybewillingorabletotravel.10 Weconductedthis studytodeterminewhetherPEMfellowshipapplicants wouldapplytoalargernumberofprogramsandindifferent geographicpatternswithVI(2020and2021)ascomparedto in-personinterviews(2018and2019).

METHODS

DesignandParticipants

Thiswasananonymous,self-administered,crosssectional,web-basedsurveyofPEMfellowsinthe UnitedStates.Participationwasvoluntary,andnoincentive wasprovidedforcompletion.Thestudywasexempted bytheinstitutionalreviewboardatYaleUniversity, withinformedconsentimpliedbycompletionofthesurvey byparticipants.

SurveyDevelopment

Thesurveyquestionnairewasdevelopedthroughiterative feedbackandamodifiedDelphiprocesstodetermine itemimportance.ThirteenPEMPDswithexpertisein performanceandevaluationparticipatedinmultiplerounds ofrevisionsandediting.Pilottestingwasconductedwithtwo pediatrichospitalmedicinefellowswhohadappliedtothe matchduringVIsandtwopediatricchiefresidentswhowere alsointerviewingforfellowshipsusingVI,atthelead institution.Revisionsweremadebasedonpilotfeedback (surveyprovidedin SupplementaryAppendix1).Thesurvey includedmultiple-choicequestionsaboutlocationof residency,statesappliedtoandinterviewedforfellowship, preferredlocationforfellowship,statesvisitedinpersonfor thepurposeofthematch,andstatematchedin.Italsoasked fellowstoindicatestatesofresidenceofimmediatefamily (parents,siblings,orpartners)andaboutcompellingreasons (otherthanfamily)thatmayhaveledfellowstofavorastate orregion(freetext).Geographicregionsweredefinedas Northeast,Southeast,Midwest,Southwest,Rocky Mountain,andPacificregions.19

Whatdowealreadyknowaboutthisissue?

Virtualinterviewsareapermanentpartof recruitment.Theyoffercostandequity advantageswhileposingchallengestoboth applicantsandprograms.

Whatwastheresearchquestion?

DidPEMfellowshipapplicantsapplytoa largernumberofprogramsandindifferent geographicpatternswithVIascomparedto in-personinterviews?

Whatwasthemajor findingofthestudy?

VIdidnothaveasigni fi cantimpactonthe numberofprogramsorgeographicareas applicantsappliedto.

SurveyDistribution

ThesurveywasreviewedandapprovedbytheAmerican AcademyofPediatrics(AAP)SectiononEmergency Medicine(SOEM)PDsurveysubcommitteepriorto distributiononQualtrics(Qualtrics,Provo,UT)toallPEM PDs,viatheAAPSoEMPDCommitteelistserv.ThePDs forwardedthesurveylinktotheircurrentandincoming fellows(thoserecentlymatchedtostartinJuly2022).Each PDcompletedaseparatequestionnaireindicatingthetotal numberofcurrentandrecentlymatchedfellowstowhom theyforwardedthesurvey.

Analysis

Participantsweredividedintotwogroups:VI(2020or 2021)andinperson(2018or2019).Weperformed descriptivestatisticsincludingfrequencies,percentages, meanswithstandarddeviations,andmedianswith interquartilerange(IQR).Chi-squaretestscompared categoricalvariablesandt-tests,continuousvariableswith 95%confidenceintervals(CI).Weconsideredatwo-tailed alphaof <0.05tobestatisticallysignificant.Weconducted analysesinIBMSPSSStatisticsversion28(IBM Corporation,Armonk,NY).

RESULTS

ThePDsreportedthattheyforwardedthesurveyto406 currentfellowsand144incomingfellows.Theresponserate forcurrentfellowswas35%(143/406)andforincoming fellows,61%(88/144).Overall,theresponseratewas42% (231/550).Ofthetotalrespondents,62%(143/231)were currentfellowsand38%(88/231)incoming.Twofellows (1%)didnotcompleteresidencyintheUS,and12(5%) appliedtoPEMfellowshipmorethanonce.

AllincomingfellowshadundergoneVI,whereas 48%ofthecurrentfellowshadundergoneVI(69/143). Overall,32%ofrespondents(74/231)interviewedinperson and68%(157/213)virtually.Therewasnostatistical differenceinthenumberofprogramsappliedtoduringinpersonvsVI(meandifference(95%CI):.72[ 2.8,4.2]) (Appendix2Table).

Datadescribingthegeographictrainingandlocation preferenceofparticipants arepresentedinthetablein appendix2.Fellowsappliedtoamedianoffourofthe sixgeographicregions(IQR2,5).Mostparticipantsapplied forfellowshipinthesamegeographicregionastheirresidency (216,93%)andoutsidetheirresidencyregionaswell (192,83%).OnlythePacificregionsawastatistically significantincreaseinapplicantsduringVI(59.9%vs43.2%, P = 0.02)(Table1).

Lessthanhalfofrespondentshadimmediatefamily memberslivinginthesamestateasresidency(N = 111,48%), fellowship(N = 90,39%),ortheirpreferredmatch state(N = 95,41%).Compellingreasonstoapplyto anareaincludedfamiliaritywithlocation(N = 128,55%); similarlocationtoresidency(N = 65,28%);anda desiretotraininanewarea(N = 53,23%).Partner’ s employmentwasanimportantfactorfor89(38%),salary

andcostoflivingfor76(33%),andschoolforchildren for20(9%).

DISCUSSION

OurresultsshowthatVImayallowsomecandidatesto exploreandconsiderregionstheymaynothaveotherwise duetologisticalor financialconstraints,withoutincreasing thenumberofprograms,regionsorstatestheyapplyto. Theseresultsareconsistentwiththe2021NRMPsurvey where52%reportednoimpactoftheVIonthenumberof programsappliedto.5 Residencyprogramshavereportedan increaseinmatchedinternalcandidatesduringVI.11,12,20,21 InPEM,apre-pandemicstudyofPDsshowedthat29%of fellowscompletedresidencyatthesameinstitution.22 While wedidnothavedataattheinstitutionallevel,therewasno significantincreaseinfellowsmatchingwithinthestateof theirresidencyprogramwithVI.ThissuggeststhatVIwere notasignificantdetrimenttoapplicantsrankingprograms andgeographicareas,despitetheabsenceofopportunitiesto meetinpersonandvisitprograms.Thisalsoallowsprograms tohaveaccesstoalargerandpotentiallymorediversepool ofcandidates.9

Proximitytofamilywasnotasignificantconsiderationfor mostapplicantsandwasnotimpactedbyVI.Residency

Table1. Influenceofvirtualinterviewsonapplicantbehaviorandoutcomes. In-person interviews(N = 74)

(N = 157)

ficance (P valueor95%CI)

Appliedtoregionforfellowship,N(%)

Northeast59(79.7)123(78.3)0.81

Southeast41(55.4)102(65)0.16

Midwest50(67.6)111(70.7)0.63

Southwest38(51.4)86(54.8)0.63

RockyMountains31(41.9)73(46.5)0.51

Pacific32(43.2)94(59.9)0.02

Appliedtosamegeographicregionas residency,N(%) 71(98.6)145(94.8).278

Appliedoutsidegeographicregionas residency,N(%) 56(77.8)136(88.9)0.03

Numberofregionsappliedto,mean(SD)3.4(1.8)3.8(1.8)Meandifference(95%CI):.36( .15,.89)

Numberofstatesappliedto,mean(SD)9(7.3)9.7(6.8)Meandifference(95%CI):.73( 1.2,2.7)

Numberofprogramsappliedto,mean(SD)13.3(12.8)14(12.5)Meandifference(95%CI):.72( 2.8,4.2)

Numberofprogramsinterviewedat, mean(SD) 7.2(4.7)6.9(5.2)Meandifference(95%CI): 3.1( 1.7,1.1)

Matchedinpreferredstate,N(%)46(65.7)84(60.4)0.46

Matchedinsamestateasresidency,N(%)31(42%)59(38%)0.58

Preferredtomatchinstatewithimmediate familypresent,N(%) 36(52.9)59(46.8)0.42

Wenttovisitstate/program,N(%)9(14)23(17)0.61

CI,confidenceinterval.

applicantsreportedgeography,qualityoflife,casevariety, curriculum,institutionalreputation,expertiseinareasof interest,andprogramsizeaskeyfactors.23 Applicantsto PEMhighlightedfamiliaritywiththeregionorwantingto exploreanewareaasfactorsforexploringprogramsin differentregions.

LIMITATIONS

Limitationsofthisstudyincludethesmallerresponserate ofthecurrentfellowsascomparedtotheincomingfellows. Thislowresponseratelimitedthesamplesizeofthein-person cohort,impactingthestatisticalsignificanceofourresults. Thisdifferentialresponsefromtheincomingfellowsmay havebeenduetodesirabilitybiaswherethiscohortof applicantsmayhavetendedtostatethattheymatchedin theirpreferredstate.Tominimizethis,wedesignedourstudy tobefullyanonymousandself-administered,andthe questionswerewordedtoretainobjectivityoftheanswers. Respondentsmayalsohaveexperiencedrecallbiasregarding thestatesandprogramstowhichtheyapplied.Thisbias couldpotentiallyhavecontributedtothelowerresponserate amongthecurrentfellowswhohadinterviewedin2018/2019, 3–4yearspriortothesurveydate,comparedtothemore recentapplicantswhohadamorerecentrecollectionofthe questionsaskedinthesurvey.

Anotherlimitationisthatwedidn’texplicitlyaskthetotal numberoffellowsineachclasscohort;however,sincethe PEMfellowshipclasssizeintheUSdoesn’tvarysignificantly fromyeartoyear(byvirtueoftheapprovedfellowship positionsavailable),thedenominatorisexpectedtobe relativelyconstant.

Thisstudywasnotdesignedtolookattheratesof applicationstoindividualprogramsnorassessthepostmatchopinionsofprogramsandfellowsregardingthe resultsofthematch.Thisinformationwouldprovidea deeperinsightintotheimpactoftherecruitmentprocess; however,itisalsopronetobiasasfellowsonlyexperience trainingatasingleinstitution.Wealsodidnottakeinto considerationtheconcentrationofPEMprogramsby regionortheavailablefellowshipslotsperprogramor region.However,theobjectiveofthisstudywastolookat thedifferencesbeforeandduringVIs,andtherewasnota signi fi cantchangeinavailablefellowshipslotsorprograms duringtheseyears.Asthenumberofpediatricfellowship applicantsrises,furtherinvestigationintotheimpactofVIs isnecessarytogainadeeperunderstandingofits implicationsandtooptimizethisprocessbothfor applicantsandprograms. 24

CONCLUSION

WhilemorePEMfellowshipapplicantsappliedoutside thegeographicareawheretheirresidencywasandtothe Pacificregion,therewasnooverallincreaseinthenumberof

programsorgeographicareasthatPEMapplicantsapplied toduringVIduringthe firsttwoyearsofitsinstitution,as comparedtoin-personinterviewseasons.Ongoing monitoringoftheinterviewandmatchseasonswillhelp identifyfuturetrendsandimpactofVIs.

AUTHORSANDAFFILIATIONSCONTINUED

MelissaL.Langhan,MD,MHS

** StanfordUniversitySchoolofMedicine,Departmentof PediatricEmergencyMedicine,PaloAlto,California

††Children’sHospitalofMichigan,DepartmentofPediatrics, Detroit,Michigan

‡‡CentralMichiganUniversity,SchoolofMedicine, DepartmentofPediatrics,Detroit,Michigan

§§JohnsHopkinsUniversity,SchoolofMedicine,Department ofPediatrics,Baltimore,Maryland

∥∥

RainbowBabies&Children’sHospital,Departmentof Pediatrics,Cleveland,Ohio

¶¶CaseWesternReserveUniversity,SchoolofMedicine, DepartmentofPediatrics,Cleveland,Ohio

##ColumbiaUniversityIrvingMedicalCenter,Pediatricsin EmergencyMedicine,NewYork,NewYork

*** UniversityofPittsburgh,SchoolofMedicine,Departmentof Pediatrics,Pittsburgh,Pennsylvania

†††UniversityofMissouriofKansasCitySchoolofMedicine, DepartmentofPediatrics,KansasCity,Missouri

‡‡‡UniversityofKansasMedicalCenter,KansasCity, Missouri

§§§BostonChildren’sHospital,DepartmentofPediatricsand EmergencyMedicine,Boston,Massachusetts

∥∥∥HarvardMedicalSchool,DepartmentofPediatricsand EmergencyMedicine,Boston,Massachusetts

¶¶¶YaleUniversitySchoolofMedicine,Departmentof PediatricsandEmergencyMedicine,NewHaven, Connecticut

AddressforCorrespondence:AlineBaghdassarian,MD,MPH, InovaFairfaxL.JMurphyChildren’sHospital,3300GallowsRd., FallsChurch,VA22042-3300.Email: aline.baghdassarian@ inova.org

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Baghdassarianetal.Thisisanopenaccess articledistributedinaccordancewiththetermsoftheCreative CommonsAttribution(CCBY4.0)License.See: http:// creativecommons.org/licenses/by/4.0/

REFERENCES

1.AssociationofPediatricProgramDirectors,CouncilofPediatric Subspecialties,AssociationofMedicalSchoolPediatricDepartment Chairsetal.APPD/CoPS/AMSPDC/NextGenPediatricianslettertoour pediatricscommunityaboutthefellowshiprecruitmentprocess.2021. Availableat: https://downloads.aap.org/AAP/PDF/2021_05_19_-_ Pediatric_Fellowship_Recruitment_Recommendations_Final.pdf AccessedJuly15,2022.

2.VanDerLaanL,GeorgeR,NesiamaJA,etal.Virtualinterviewingfor pediatricemergencymedicinefellowship-anationalsurvey. Pediatr EmergCare. 2022;38(4):e1207–12.

3.GuptaS,GrierArthurL,ChandlerN,etal.Isthechanginglandscapeof fellowshiprecruitmentduringCOVID-19heretostay? JPediatrSurg. 2022;57(10):445–50.

4.DasAJ,AugustinRC,CorbelliJA,etal.Residencyandfellowship programleaders’ perceptionsofvirtualrecruitmentandinterviewing. JGradMedEduc. 2022;14(6):710–3.

5.NationalResidentMatchingProgram.2021Applicantandprogram directorsurvey findings:impactofthevirtualexperienceonthetransition toresidency.2021.Availableat: https://www.nrmp.org/wp-content/ uploads/2021/08/Research-Brief-Virtual-Experience-2021-FINAL.pdf AccessedJuly15,2022.

6.WolffMandBurrowsH.Planningforvirtualinterviews:residency recruitmentduringapandemic. AcadPediatr. 2021;21(1):24–31.

7.BernsteinSA,GuA,ChretienKC,etal.Graduatemedicaleducation virtualinterviewsandrecruitmentintheeraofCOVID-19. JGradMedEduc. 2020;12(5):557–60.

8.McCainC,KempB,BaierMB,etal.Aframeworkforthevirtualmedical interviewprocess:considerationsfortheapplicantandtheinterviewer. OchsnerJournal. 2022;22(1):61–70.

9.PetersenTL,KingJC,FussellJJ,etal.Benefitsandlimitationsofvirtual recruitment:perspectivesfromsubspecialitydirectors. Pediatrics. 2022;150(4):e2022056735.

10.AllisterL,BaghdassarianA,CaglarD,etal.Pediatricemergency medicinefellowshipdirectors’ 2021collectivestatementonvirtual interviewsandsecondlooks. PediatrEmergCare. 2021;37(11):585–7.

11.EderleA,ShahriariS,WhisonantC,etal.TheimpactofCOVID-19on thedermatologymatch:anincreaseinthenumberofstudentsmatching athomeprograms. DermatolOnlineJ. 2021;27(9).

12.FaletskyA,ZitkovskyH,GuoL.TheimpactofCOVID-19on plasticsurgeryhomeprogrammatchrates. AnnPlastSurg. 2022;88(1):4–6.

13.MulcahyCF,TerhaarSJ,BoulosS,etal.Didmoreotolaryngology residencyapplicantsmatchattheirhomeinstitutionsin2021? InvestigatingtheimpactoftheCOVID-19pandemic. AnnOtolRhinol Laryngol. 2022;131(12):1375–80.

14.QuinnA,MannE,RaikinJ,etal.PD24-06Theeffectofthe COVID-19pandemiconurologymatchbylocation. JUrol. 2021;206(Suppl3):e427–8.

15.InclanPM,WoiczikMR,CummingsJ,etal.Virtualpediatricorthopaedic fellowshipinterviewsduringthepandemic:Whatdidtheapplicantsand programsthink? JPediatrOrthop. 2022;42(7):e806–10.

16.LewkowitzAK,RamseyPS,BurrellD,etal.Effectofvirtualinterviewing onapplicantapproachtoandperspectiveofthematernal-fetal medicinesubspecialtyfellowshipmatch. AmJObstetGynecolMFM. 2021;3(3):100326.

17.ReamMAandThompson-StoneR.Virtualresidencyinterview experience:thechildneurologyresidencyprogramperspective. PediatrNeurol. 2022;126:3–8.

18.DiGiustoM,LupaMC,CorridoreM,etal.TheimpactoftheCOVID-19 pandemiconthe2020pediatricanesthesiologyfellowship applicationcycle:asurveyofapplicants. PaediatrAnaesth. 2021;31(9):968–76.

19.STUDYGUIDEUSII.2cGeographyThemes.Availableat: https://www.solpass.org/7ss/standards/StudyUSII.2c.htm AccessedFebruary4,2024.

20.WhisonantCT,ShahriariSR,HarrisonJ,etal.Evaluatingtheintegrated plasticsurgeryresidencymatchduringthenovelcoronaviruspandemic. Cureus. 2021;13(8):e16988.

21.CotnerCE,MercadanteSF,SheaJA.Assessingtheimpactofthe COVID-19pandemicongeographicresidencyplacementrelativeto medicalschoollocation. JGradMedEduc. 2022;14(1)108–11.

22.BradleyT,ClingenpeelJM,PoirierM.Internalapplicantstopediatric emergencymedicinefellowshipsandcurrentuseofthenationalresident matchingprogrammatchasurveyoffellowshipdirectors. PediatrEmerg Care. 2015;31(7):487–92.

23.LoveJN,HowellJM,HegartyCB,etal.Factorsthatinfluence medicalstudentselectionofanemergencymedicineresidency program:Implicationsfortrainingprograms. AcadEmergMed. 2012;19(4):455–60.

24.TheAmericanBoardofPediatrics.Yearlygrowthinpediatricfellowsby subspecialtybydemographicsandprogramcharacteristics.Available at: https://www.abp.org/content/yearly-growth-pediatric-fellowssubspecialty-demographics-and-program-characteristics AccessedJanuary26,2023.

EDUCATION SPECIAL ISSUE:BRIEF RESEARCH REPORT

PerceptionofQuietStudentsinEmergencyMedicine:An ExplorationofNarrativesintheStandardizedLetterofEvaluation

JohnK.Quinn,MD

JillianMongelluzzo,MD,MAEd

AlyssaNip,MD

JosephGraterol,MD

EstherH.Chen,MD

UniversityofCalifornia,SanFrancisco,DepartmentofEmergencyMedicine, SanFrancisco,California

SectionEditor:JeffreyLove,MD,MHPE,andDanielleHart,MD,MACM

Submissionhistory:SubmittedJune15,2022;RevisionreceivedMarch16,2023;AcceptedMarch17,2023

ElectronicallypublishedJuly12,2023

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.57756

Introduction: TheStandardizedLetterofEvaluation(SLOE)isdesignedtoassistemergencymedicine (EM)residencyprogramsindifferentiatingapplicantsandinselectingthosetointerview.TheSLOE narrativecomponentsummarizesthestudent’sclinicalskillsaswellastheirnon-cognitiveattributes.The purposeofthisqualitativeinvestigationwastoexplorehowstudentsdescribedintheSLOEasquietare perceivedbyfacultyandtobetterunderstandhowthismayimpacttheirresidencycandidacy.

Methods: ThisretrospectivecohortstudyincludedallSLOEssubmittedtooneEMresidencyprogram duringoneapplicationcycle.WeanalyzedsentencesintheSLOEnarrativedescribingstudentsas “quiet,”“shy,” and/or “reserved.” Usinggroundedtheory,thematiccontentanalysiswithaconstructivist approach,weidentified fivemutuallyexclusivethemesthatbestcharacterizedtheusageofthese targetwords.

Results: Weidentified fivethemes:1)quiettraitsportrayedasimplied-negativeattributes(62.4%); 2)quietstudentsportrayedasovershadowedbymoreextravertedpeers(10.3%);3)quietstudents portrayedasunfitforfast-pacedclinicalsettings(3.4%);4) “quiet” portrayedasapositiveattribute (10.3%);and5) “quiet” commentsdeemeddifficulttoassessduetolackofcontext(15.6%).

Conclusion: Wefoundthatquietpersonalitytraitswereoftenportrayedasnegativeattributes.Further, commentsoftenlackedclinicalcontext,leavingthemvulnerabletomisunderstandingorbias.More researchisneededtodeterminehowquietstudentsperformcomparedtotheirnon-quietpeersandto determinewhatchangestoinstructionalpracticesmaysupportthequietstudentandhelpcreateamore inclusivelearningenvironment.[WestJEmergMed.2024;25(4.1)6–9.]

INTRODUCTION

Theemergencymedicine(EM)StandardizedLetterof Evaluation(SLOE)isahigh-stakesassessmentdesignedto assistresidencyprogramsindifferentiatingapplicantsandis consideredimportantinthedecisiontointerview.1,2 The narrativecomponentsummarizesthestudent’sknowledge, clinicalskills,andnon-cognitiveattributesshowntobe predictorsofperformance.3–5 However,thenarrativemaybe difficulttointerpretduetotheuseofoverlygenerallanguage andhiddencode,bothcommoninwrittenassessment.6–9

Further,commentsaboutpersonalityoftenlackclinical context,whichreducestheirusefulnessandmakesthem vulnerabletomisinterpretationorbias.6–8 WebecameinterestedinSLOEnarrativesreferencing quietstudentsduringapplicantreviewwhenweobservedless enthusiasmforstudentsdescribedasquiet,evenforthose withstrongobjectiveapplicationdata.Whilenon-cognitive attributesareimportantcomponentsofholisticassessment, personalitytraitsshouldnotnecessarilyhinderastrong application.3–5 Nostudiesshowthatquietindividualsare

unsuitedforEMorarelesssuccessfulinEMcareers. However,inaninternalmedicinesetting, “quiet” was interpretedbyattendingsasa “red flag ” inclerkshipwritten evaluations, 9,10 andstudentsdescribedasquietintheir SLOEscoredloweronbothglobalassessmentand anticipatedranklist.11 Wefoundnootherresearch examininghowquietindividualsperformorhowthey wereperceivedinEM.Thepurposeofthisqualitative investigationwastoexplorehowquietstudentsaredescribed intheSLOEnarrativeandhowthislanguagemay impactcandidacy.

METHODS

StudyDesignandPopulation

Weconductedasubgroupanalysisofaretrospective cohortstudyofallcoreEMrotationSLOEssubmitted throughtheElectronicResidencyApplicationService (ERAS)tooneEMresidencyprogramduringthe2016–2017 applicationcycle.WeexcludedSLOEsfromnon-Liaison CommitteeonMedicalEducationaccreditedschoolsand applicantswhograduatedfrommedicalschoolbeforeor duringtheapplicationcycle.Thestudywasapprovedbythe institutionalreviewboardandtheAssociationofAmerican MedicalColleges.

StudyProtocolandDataAnalysis

AuthorJMdownloadedSLOEsfromERASinto REDCap(ResearchElectronicDataCapturetoolshostedat UCSanFrancisco).andde-identifiedthempriortoanalysis. AnalysiswasperformedbyJKQ,EHC,andJM,allwith traininginmedicaleducationresearchmethodologyand educationleadershipexperience(chiefresident,associate residencydirector,andassistantresidencydirector).JKQ andEHCbrainstormedwordstypicallyusedtodescribe quietindividualsandchosethetarget-descriptorsquiet,shy, andreserved(collectivelytermed “quiet”)becausepassive, introverted,andtimidwereuncommon(3,2,and1, respectively)andalwaysco-occurredwithtarget-descriptors. Weanalyzedonlythesentencecontainingthetargetdescriptorswithoutexploringtheentirenarrative.We analyzeddatausinggroundedtheorythematiccontent analysiswithaconstructivistapproach.12 Therewasnopreexistingtheoryaboutthedatathatweaimedtoproveor disprove;instead,thegoalwastoexploreSLOEnarrativecommentsandconstructmeaningfromthemtoprovide perspectiveonhowquietstudentsareperceived.

Withoutapresetideaofhowdatawouldbesorted,JKQ andEHCindependentlybegantheinitialcodingbyreading eachcommentandconsideringhowitwasusedtodescribe thestudent.Asusagepatternsemergedtheywerecodedas like-comments.JMreadasubsetofthedata.Toestablish thatthedatasetwassufficientforthepurposeofthe investigation,wecodedthe firsthalfofthedatasetandthen determinedthatnonewpatternsemergedinthesecondhalf.

Weprogressedtoexplainingourcodingschemes,comparing them,andlookingforsimilaritiesanddifferences.Through aniterativeprocessofconstantcomparisonwecombined, deleted,andrefinedcodes,mergingthemintooverarching themes.Weusedaspreadsheettovisuallyorganizecodesand finalthemes.

RESULTS

Wereviewed1,582SLOEsfrom696applicants.Ofthese, 117SLOEsreferencedquietapplicantsandwereanalyzed. Theadjective “quiet” occurredin102SLOEs. “Reserved” occurredin28SLOEsandco-occurred14timeswith “quiet.” “Shy” occurredin11SLOEsandco-occurred fivetimes with “quiet.”

Initialcodingrevealedusagerelatedtointerpersonal skills,initiative,disposition,patientinteractions,leadership, medicalknowledge,responsetofeedback,workhabits,and fitnessforEM.Furtheranalysisrevealedthatmanytarget sentencesdidnot fitintothesecategories,lackedclinical context,andweredifficulttointerpret.Weeventually reachedaconsensusonaframeworkof fivemutually exclusiveoverarchingthemesthatincludedallcomments, bestrepresentedthescopeofusagepatterns,andwouldbe mostmeaningfulinaddressingourstudypurpose(Table1).

Theme1comments,62.4%describequiettraits asimplied-negativeattributes.Commentsarelabeled “implied” becausequietisnotexplicitlycallednegativebut istypicallycoupledwithacontrastingpositivetraitthat appearstobeanefforttomitigatethenegativityofthequiet comment(eg, “Quietbuthardworking”).Thestructureofthe sentencemakesitclearthatquietisnegative,butitisnot evidentinwhatwayortowhatdegreeitisnegative.Asmaller numberofcommentslinkedthequiettraitwith anotherseeminglynegativeattribute(eg, “Quietandtimidat times”).Theimpliednegativityofthesecomments coupledwiththelackofcontextmayadverselyaffectthe applicant’scandidacy.

Theme2comments(10.3%)describequietstudentsas beingovershadowedbymoreextravertedpeersandthus moredifficulttoassess.Thesecommentsalsodidnotexplain howperformancewasimpactedbythequiettrait,onlythat the studentwasnotabletodemonstratevalueasacandidate orperformattheleveloftheirpeers,whichpresumably hindersapplicantcandidacy.

Theme3comments3.4%)questionthe fitnessofquiet studentsforfast-pacedclinicalsettings.However,these commentsdidnotdetailhow,ortowhatdegree,thestudent’ s quietnessspecificallyaffectedperformance,makingthem vulnerabletomisinterpretation.Thesecommentswould likelyalsohindercandidacy,astheabilitytoperformwellin allclinicalsettingsispresumablyseenasnecessaryina successfulEMresident.

Theme4comments(10.3%) “quiet” isportrayedas apositiveattributeandtendstodescribeleadershipstyle,

Table1. Thematicanalysisof117sentencescontainingthewords

ThemeSubthemesExamples

Theme1)Impliednegative (n = 73)

Theme2)Quietstudentsmay beovershadowedbyothers (n = 12)

1A) Quietnatureismitigatedby associatingwithapositiveinterpersonal skill.

1B)Quietnatureismitigatedby associatingwithapositiveattribute unrelatedtoquietpersonality.

2A)Quietstudentsovershadowedby moreextravertedstudents.

2B)Quietstudents’ clinicalskillsdif ficult toassessduetotheirquietpersonality.

“Quietbutwasalwaysabletocommunicateeffectively.” “Somewhatreservedbutcanbeassertivewhen necessary.”

“Quietbuthardworking.”“Canbereservedattimesbut isincrediblyintelligent.”

“Quietdemeanorandpresenceof flashierstudents preventedahigherranking.”“Overshadowed,quieter thanpeers,disappearedintobackgroundmostofthe month.”

“Truncatedpresentationsandquietdemeanormakeit dif ficulttoevaluatetruepotential.”“SoquietIcouldnot judgelevelofengagement.”

Theme3)Quietstudentsmay belesssuitedforcertainclinical settings(n = 4)

3A)Quietstudentsperceivedastoo passive,slow,orunassertiveforabusy clinicalsetting.

3B)Quietstudentsperceivedasless adaptabletothedemandsofabusy clinicalsetting.

Theme4)Positivetrait(n = 12)

Theme5)Equivocal(n = 16)

ED, emergencydepartment; EM,emergencymedicine.

patientinteractions,orabilitytoperformunderpressure, ratherthandescribingstudentpersonality.Thisadditional contextmayhavecontributedtotheoverallperceptionof “quiet” asapositiveattribute.Theme5comments(15.6%) wereconsideredequivocalinthattheinvestigatorseitherdid notagreeonthepositivityornegativityoftheir interpretation,orthecommentslackedsufficientcontextto interprettheintendedmeaning(eg, “Studentwas initiallyquiet”).

DISCUSSION

Wefoundthatquiettraitswereusuallyportrayedas negativeattributesand,therefore,hadthepotentialto adverselyaffectthecandidacyofaconsiderablenumberof applicants.Theanalysisalsorevealedthatacrossthemes thequiettraitwasrarelydescribedintermsofclinical competency.Thisisconcerningbecauseanegativecomment thatlackscontextrequiresthereadertorelyoninferences orassumptionsthatmayresultinunfairlyjudgingthe applicant.Providingexamplesthatdescribeobserved behaviorandclinicalskill,ratherthanreferencing personality,willimprovethequalityandfairnessof theassessment.6,7

“Quiet,passivenaturemaynotbesuitedforhighpaced inner-cityED.”“Quietandunassumingpersonality, somenotedthistobeaconcern,particularlyinabusy countyED,othersdidn’t.”

“Calm,quiet,reserveddemeanor-somestaffquestion adaptabilitytochaoticED.”

“Soothingdemeanorandquietconfidencewillsuitquite wellthroughouttheircareer.”“Quietdemeanor,kind bedsidemannerwhichisanassetwithpatients.”

“Alittlequiet,wedonotthinkthiswillhinderabilitytobe averycapableEMresident.”“Quiet”

Our findingsthatquietstudentsaredescribedasbeing overshadowedbymoreextravertedpeers,moredifficultto assess,andless fitforfast-pacedclinicalsettingssuggestthe possibilitythatcurrentinstructionalpracticesfavormore outgoingstudents.Inaclinicalsettingwherebeingassertive isviewedfavorably,quietstudentsmaybejudgedunfairlyas beinglessknowledgeableorprepared.3,13 Changesto instructionalpracticesthatmaybetterservequietstudents includethefollowing:providingadditionalstudent observations6;usingstandardizedassessment-tools14,15; expandingassessmentcriteriatoincludestrengthsofthe introvert13;providingfacultydevelopmenttoimprove qualityofwrittenassessment7;usinggroup-writtenSLOEs thatmayreducebias1,2;andprovidingstudentmentorship.3

LIMITATIONS

ThisstudywaslimitedtoSLOEsfromapplicantstoa singleinstitutionduringoneapplicationcycle.Weanalyzed onlythesentencecontainingthetarget-descriptors;reading theentirenarrativemayhaveprovidedadditionalcontext. Target-descriptorsmaybedefineddifferentlybydifferent evaluatorsandreadersandmayormaynotbeused interchangeably.Further,readersmayinterpretthe

positivityornegativityoftheusagedifferentlythanthe investigators.Thetarget-descriptorsmaynotreflectstudent personalitybutratherhowtheywereperceivedbytheir evaluatorintheclinicalsetting.Applicantsdidnotreceivea personalityinventorynordidtheyself-reporttheir personalitytype.Wedidnotidentifythegenderofapplicant ortheSLOEwriter,whichpreventedusfromdetermining whetherour findingswereaffectedbygender.Nordidwe identifythepositionorexperienceofthewriter,orwhether individualorgroupprocesswasused.Wedidnotattemptto associatequietvsnon-quietstatuswithaninvitation tointerview.

CONCLUSION

Wefoundthatquietpersonalitytraitswereoftenportrayed asnegativeattributesintheStandardizedLetterof Evaluation.Additionally,clinicalcontextwasrarely provided,leavingcommentsopentovariableinterpretation andpossiblemisunderstandingofstudentcompetency. These findingsaddtoourunderstandingaboutquietstudents inEM,butmoreresearchisneededtodeterminehow quiet-labeledstudentsperformcomparedtotheirnon-quiet peersandtodeterminewhatchangestoinstructionalpractices maysupportthequietstudentandhelpcreateamoreinclusive learningenvironmentwhereallstudentscanthrive.

AddressforCorrespondence:JohnK.Quinn,MD,Universityof California:SanFrancisco,DepartmentofEmergencyMedicine,505 ParnassusAve,SanFrancisco,CA94143.Email: john.quinn@ucsf. edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Quinnetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.LoveJN,DotyCI,SmithJL,etal.Theemergencymedicinegroup StandardizedLetterofEvaluationasaworkplace-basedassessment: ThevalidityIsinthedetail. WestJEmergMed.2020;21(3):600–9.

2.NegaardM,AssimacopoulosE,HarlandK,etal.Emergencymedicine residencyselectioncriteria:anupdateandcomparison. AEMEduc Train.2018;2(2):146–53.

3.KhanMA,MalviyaM,EnglishK,etal.Medicalstudentpersonalitytraits andclinicalgradesintheinternalmedicineclerkship. MedSciEduc 2021;31(2):637–45.

4.PinesJM,AlfarajS,BatraS,etal.FactorsImportanttotop clinicalperformanceinemergencymedicineresidency:results ofanideationsurveyandDelphipanel. AEMEducTrain 2018;2(4):269–76.

5.SobowaleK,HamSA,CurlinFA,etal.Personalitytraitsareassociated withacademicachievementinmedicalschool:anationally representativestudy. AcadPsychiatry.2018;42(3):338–45.

6.JacksonJL,KayC,JacksonWC,etal.Thequalityofwrittenfeedbackby attendingsofinternalmedicineresidents. JGenInternMed 2015;30(7):973–8.

7.LedfordR,BurgerA,LaRochelleJ,etal.Exploringperspectivesfrom internalmedicineclerkshipdirectorsintheUSAoneffectivenarrative evaluation:resultsfromtheCDIMnationalsurvey. MedSciEduc 2020;30(1):155–61.

8.LyePS,BiernatKA,BraggDS,etal.Apleasuretoworkwith–ananalysis ofwrittencommentsonstudentevaluations. AmbulPediatr 2001;1(3):128–31.

9.GinsburgS,KoganJR,GingerichA,etal.Takenoutofcontext:hazards intheinterpretationofwrittenassessmentcomments. AcadMed 2020;95(7):1082–8.

10.GinsburgS,McIlroyJ,OulanovaO,etal.Towardauthenticclinical evaluation:pitfallsinthepursuitofcompetency. AcadMed 2010;85(5):780–6.

11.QuinnJK,MongelluzzoJ,AddoN,etal.TheStandardizedLetterof Evaluation:howweperceivethequietstudent. WestJEmergMed 2023;24(2):259–63.

12.CoatesWC,JordanJ,ClarkeSO.Apracticalguideforconducting qualitativeresearchinmedicaleducation:Part2-Codingandthematic analysis. AEMEducTrain.2021;5(4):e10645.

13.DavidsonB,GilliesRA,PelletierAL.Introversionandmedicalstudent education:challengesforbothstudentsandeducators. TeachLearn Med.2015;27(1):99–104.

14.DavisKR,BankenJA.Personalitytypeandclinicalevaluationsinan obstetrics/gynecologymedicalstudentclerkship. AmJObstetGynecol 2005;193(5):1807–10.

15.SchellRM,DilorenzoAN,LiHF,etal.Anesthesiologyresident personalitytypecorrelateswithfacultyassessmentofresident performance. JClinAnesth.2012;24(7):566–72.

EDUCATION SPECIAL ISSUE:BRIEF RESEARCH REPORT

ChangesinResidencyApplicantCancellationPatternswith VirtualInterviews:ASingle-siteAnalysis

MeryllBouldin,MD* CarlyEastin,MD*†

RachaelFreeze-Ramsey,MD*

AmandaYoung,MD*

MeredithvonDohlen,MD*

LaurenEvans,MD*

TravisEastin,MD,MS*

SarahGreenberger,MD*

*UniversityofArkansasforMedicalSciences,DepartmentofEmergencyMedicine, LittleRock,Arkansas

† UniversityofArkansasforMedicalSciences,DepartmentofPediatrics, SectionofEmergencyMedicine,Toxicology,andPharmacology, LittleRock,Arkansas

SectionEditors:AbraFant,MD,andWilliamEdwardSoares,MD

Submissionhistory:SubmittedOctober1,2023;RevisionreceivedFebruary7,2024;AcceptedFebruary21,2024

ElectronicallypublishedMarch14,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.18487

Background: ResidencyprogramstransitionedtoprimarilyvirtualinterviewsduetotheCOVID-19 pandemic.Thisshiftraisedquestionsregardingexpectationsandpatternsofapplicantcancellation timeliness.Thepurposeofthisstudywastoexaminechangesinapplicantcancellationsafter transitioningtovirtualinterviews.

Methods: Thiswasaretrospectivecohortstudyofinterviewdatafromathree-yearemergencymedicine residencyatatertiary-careacademicmedicalcenter.UsingarchiveddatafromInterviewBroker,we examinedschedulingpatternsbetweenonein-person(2019–2020)andtwovirtualinterviewcohorts (2020–2021and2021–2022).Ouroutcomesweretheoverallcancellationratesrelativetointerviewslots aswellastheproportionofcancellationsthatoccurredwithin7or14daysoftheinterviewdate.

Results: Therewere453interviewslotsand568applicantsinvited.Overall,applicantscanceled17.1% ofscheduledinterviews.Comparedwithin-personinterviews,applicantscanceledsignificantlyfewer virtualinterviews(inperson:40/128(31.3%),virtualyear1:22/178(12.4%),virtualyear2:15/143 (10.5%), P = 0.001).Conversely,applicantscanceledsignificantlymorevirtualinterviewswithinboththe 14-daythreshold(inperson:8/40(20%),virtualyear1:12/22(55.5%),virtualyear2:12/15(80%), P < 0.001)andthe7-daythreshold(inperson:0/40(0%),virtualyear1:3/22(13.6%),virtualyear2:4/15 (26.7%), P = 0.004).

Conclusion: Whilelimited,atoursite,changingtoavirtualinterviewformatcorrelatedwithfewer cancellationsoverall.Theproportionofcancellationswithin14dayswasmuchhigherduring virtualinterviewseasons,withmostcancellationsoccurringduringthattimeframe.Additional studiesareneededtodeterminetheeffectsofcancellationpatternsonemergencymedicinerecruitment. [WestJEmergMed.2024;25(4.1)10–14.]

INTRODUCTION

Historically,residencyapplicantstraveledtoUSprograms forin-personinterviews.In2020,theCOVID-19pandemicled theCoalitionforPhysician Accountability(COPA)to recommendthatresidencyprogramsconductonlyvirtual interviews.1 Proponentsofvirtualinterviewscitedcostand safetyaspotentialupsides,andapplicantshavereportedoverall satisfactionwithvirtualinterviewsandmoreadvantagesthan barriers.2–4 However,programshaveexpressedcontinued doubtsaboutsomeaspectsofvirtualrecruitment.2

Evenbeforethepandemic,therewerenoestablishedrules acrossspecialtiesregardinganacceptabletimeframefor interviewcancellations.Foremergencymedicine,the EmergencyMedicineResidentAssociation(EMRA) recommendedatleasttwoweeks’ noticeintheir2019 “EMRAandCORDStudentAdvisingGuide.”5 In2020,the firstyearofvirtualinterviews,emailcommunicationonthe CouncilofResidencyDirectorsinEmergencyMedicine (CORD)listservsuggestedthatprogramdirectors’ acceptablecancellationthresholdsrangedfrom48hoursto 10dayspriortotheinterviewdate.6 Ultimately,CORD statedthatsevendayswasrecommendedforapplicantsina 2020blogpostaboutinterviewingduringthepandemic, whileotherpublicationsstillrecommendedtwoweeks.7,8 Currently,the2023CORDApplicationProcess ImprovementCommitteeandthe2022–2023National ResidentMatchingProgram(NRMP)agreementhave advisedapplicantstocancelnolaterthan1–2weeksbefore theirinterviewdates.9,10

Virtualinterviewsmaybeheretostay,asevidencedby recentCOPAandAssociationofAmericanMedicalColleges (AAMC)statements,aswellasthe2023-24CORD guidelines.11–13 Understandingpatternsofvirtualinterview cancellationbehaviormayhelpprogramdirectors, applicants,andtheiradvisorsprepareforasuccessfulMatch. Tocharacterizetheeffectsofvirtualrecruitmentoninterview cancellations,wecomparedin-personinterviewcancellation patternstothoseofvirtualrecruitmentcyclesatour academicemergencymedicine(EM)residency.

METHODS

Thiswasaretrospectivecohortstudyatathree-yearEM residencysponsoredbyatertiary-care,academicmedical centerinanurbansettinginthesouth-centralUnitedStates. Thisresidencyisanestablishedprogram(foundedin1984) withaclasssizeof10residentsperyear,whichincreasedto12 residentsforthe2022Match.TheUniversityofArkansasfor MedicalSciencesInstitutionalReviewBoard(IRB) approvedthisstudyinexemptstatus.

Ourprogrambeganusingtheonlineinterviewscheduling softwareInterviewBroker(TheTenthNerve,LLC,Lewes, DE;www.interviewbroker.com)inFall2019toinvite applicantstointerview.InFall2020,interviewstransitioned frominpersontovirtualandadditionalslotswereadded, withCORDcontinuingtorecommendvirtualinterviewsfor EMresidenciesinsubsequentcycles.Similartoin-person interviews,applicantsforvirtualinterviewsareinvitedina 1:1applicanttoslotratioandgiven48hourstorespond beforeanotherapplicantisinvited.

UsingarchiveddatafromInterviewBroker,weexamined schedulingpatternsbetweenthein-personinterviewcohort (2019–2020season)andtwovirtualinterviewcohorts (VirtualYear1:2020–2021andVirtualYear2:2021–2022). Unfortunately,cancellationdatapriortotheinitiationof

InterviewBrokeratoursitewasnotavailable.Asingle investigatorabstracteddatafromInterviewBrokerin aggregateformbyacademicyearusingoverallcountsof relevantvariables,includingnumberofinterviewslots,days, invitations,interviewsscheduled/unscheduled(ie,no applicantresponsereceived)/declined,cancellations,andthe timingofthosecancellationsrelativetotheinterviewdate. Wedefinedaninterviewcancellationasaninterviewthatwas scheduled,canceled,andneverrescheduled;interviewsthat wererescheduledwereconsideredcompleted.Demographic variableswerenotavailableasInterviewBrokeronlyrecords thestudent’snameandAAMCID;accessingadditional informationwouldhaverequiredqueryingtheElectronic ResidencyApplicationService,whichwasnotcoveredinour exemptIRBagreement.

Ouroutcomesweretheoverallproportionofinterview cancellationsrelativetointerviewslots,aswellasthe proportionofinterviewcancellationsthatoccurredwithin 14daysoftheinterviewdateandwithinsevendaysofthe interviewdate.Descriptivestatisticswereperformed.We performedcomparisonsusingchi-squaredortheFisherexact testassomeobservationswereuncommon.Allcomparisons weretwo-sidedwith ɑ = 0.05.Analyseswereperformedusing SPSSStatisticsforMacintoshVersion28.0(IBM Corporation,Armonk,NY).

RESULTS

Overthreeyears,therewere453interviewslotsand568 applicantsinvited.Mostoftheinterviewslotswerevirtual (71.7%).Overall,theprogramsentout1.25interview applicationsperslotandapplicantscanceled17.1%of scheduledinterviews(Table1).Wefoundasignificant decreaseintheproportionofoverallcancellationsrelativeto filledinterviewslots,with40/128(31.3%),22/178(12.4%), and15/143(10.5%)cancellationsforin-person,virtualyear 1,andvirtualyear2,respectively(P < 0.001).When analyzedfurtherandadjustingformultiplecomparisons,the decreasewassignificantwhencomparinginpersonvs.either virtualyear,butnotwhencomparingthetwovirtualyears. Whilefewerinterviewswerecanceled,theproportionof virtualinterviewcancellationsthatoccurredwithin14days oftheinterviewdatewassignificantlyhigher(inperson:8/40 (20%),virtualyear1:12/22(55.5%),virtualyear2:12/15 (80%), P < 0.001).Similarly,morevirtualinterviewswere canceledwithinsevendaysoftheinterviewdate(inperson:0/ 40(0%),virtualyear1:3/22(13.6%),virtualyear2:4/15 (26.7%), P = 0.004),althoughthesenumberswerelow overall.Inboththe14and7daycancellationanalyses,these dataindicatedayear-over-yearincrease,meaninginboth14 and7daycomparisonswesawasignificantincreasein cancellationsbetweeninpersonandvirtualyear1,andagain sawasignificantincreasebetweenvirtualyear1andvirtual year2.See Figures1 and 2 forgraphicalbreakdownofthe

Table1. Breakdownofin-personandvirtualinterviewcohorts;totalcountsprovidedunlessotherwisespecified.

InterviewsandCancellations

InterviewgroupInpersonVirtualyear1Virtualyear2

Numberofinterviewdays151616

Numberofinterviewslots128180145

Numberofapplicantsinvited195206167

Numberofinvitationsperinterviewslot1.521.141.15

Totalinterviewslots filled128178143

Numberofunscheduledinvitations(ie,noapplicantresponsereceived)1413

Numberwhodeclinedwithoutscheduling1356

Overallcancellations(%ofscheduled)40(31.3%)22(12.4%)15(10.5%)

Numberwhocanceled < 7days(%ofcanceled)0(0%)3(13.6%)4(26.7%)

Numberwhocanceled7–14days(%ofcanceled)8(20%)9(40.9%)8(53.3%)

Numberwhocanceled >14days(%ofcanceled)32(80%)10(45.5%)3(20.0%)

Overalldeclined,unscheduled,orcanceled(%oftotalinvited)67(34.4%)28(13.6%)24(14.3%)

overalldistributionofinvitedapplicantsandinterview cancellationrates.

DISCUSSION

Comparedwithin-personinterviews,applicantstoour programwerelesslikelytocanceltheirvirtualinterview.Of thosewhodidcancel,severalvirtualapplicantscanceled withinsevendays,andmostcancellationsoccurredwithin 14daysoftheinterviewdate.Forin-personinterviews, applicantsweretraditionallyinstructedtocancelassoonas possibleandatleasttwoweekspriortotheinterviewdate.5 Asdiscussedpreviously,recommendationsforEMvirtual interviewcancellationshaverangedfrom48hourstotwo weeks,withtheNRMPcurrentlyrecommendingatleast1–2 weeksinadvance.10 Ourresultssuggestthatshort-notice

cancellations(ie,lessthantwoweeks)bystudentsmaybe morecommoninthevirtualera.

Wearenotawareofliteratureregardingthespecifictiming ofvirtualinterviewcancellations,butour findingoffewer overallcancellationsisconsistentwithLewkowitzetal’ s findingsthatmaternal-fetalmedicinefellowshipvirtual interviewshadalowerrateofcancellationscomparedwith in-personinterviews(39.1%vs72.3%).14 Thiscouldstemfrom thereducedtimeandcostrequiredtointerviewvirtually.15,16

Unfortunately,fewerinterviewcancellationsoverallcould contributetointerviewhoardingandaninequitable distributionofinterviews.TheAAMCandsomespecialties haveexpressedconcernsabouthigherqualityapplicants receivinginvitationsforandschedulingexcessivelyhigh numbersofinterviewsandleavinglowertierstudentswith

Figure1. Overalldistributionofinvitedapplicants.

feweroptions.15,17 WhilethishasnotbeenstudiedinEM specifically,theEmergencyMedicineConsensusStatement onthe2020–2021ResidencyApplicationprocesssuggested aninterviewlimitof17interviewsandencouragedapplicants nottointerviewattheirless-preferredprogramsloweron theirlistto “maketheseslotsavailabletootherstudents,” indicatingapotentialconcernfortheeffectsofhoardingsuch as “peersnotmatchingand/orprogramsnot filling.”18

Short-noticeinterviewcancellationsposeafewother challengesforresidencyprograms.Previously, fillinganinpersoninterviewslotrequired findingareplacementwho couldstillarrangetraveltotheinterviewlocation,whichisno longerrelevantforvirtualinterviews.Nonetheless,the NRMPrequiresthatprogramsprovidenolessthan48hours forapplicantstorespondtointerviewinvitations.10 If applicantsarecancelingonlyafewdaysbeforeaninterview, fillingtheopenspotmaybeachallengesinceprograms cannotinvitemorethanoneapplicantatatimeperspot. Short-noticecancellationscanalsobeproblematicas interviewersmayhavetoreviewcandidates’ applications wellinadvanceoftheinterviewdate.Withshort-notice cancellations,thiscouldmeanlosttimeforinterviewerswho hadalreadyreviewedthoseapplicationsorinadequatetime toreviewthereplacements.

Conversely,programdirectorswanttoavoidinterviewing applicantswhoarenotinterestedintheirprogram,anda cancellation evenonshortnotice providesan opportunitytointerviewanapplicantwithgreaterinterestin theprogram.Inourcase,wehadonlyfouropeninterview spotsoverthe firsttwovirtualyears(twounfilledperyear), indicatingthatwe filledmostcanceledspots.Therefore, whilenoofficialopinionexists,programdirectorsmaynot mindshort-noticecancellationaslongastheinterview scheduleisfull.Infact,theymaypreferfortheapplicantnot tofeelpressuredtointerviewataprograminwhichtheyare uninterestedonlybecausetheyareconcernedabout canceling,withshortnoticebeingviewedasunprofessional. Asvirtualinterviewsappeartobeheretostay,understanding cancellationpatternswillbeimportantforprograms, especiallyinbalancingthetimingcancellationswithnew

invitationssoprogramscanideallymaintainafull interviewschedule.

LIMITATIONS

Thisstudywaslimitedtoonespecialtyatasingle institution,thereforethegeneralizabiltyofthese fi ndingsto otherinstitutionsorspecialtiesisunclear,especiallygiven thesmallsamplesizeandlimitedpre-postperiod.The changinglandscapeofEMresidencyrecruitmentmayalso affectthegeneralizabilityofthese fi ndings.Unfortunately, wehadonlyoneyearofin-personinterviewdataaswedid notkeeptheserecordspriortotheuseofInterviewBroker, whichcouldhaveintroducedbias.Wealsohadanincrease inresidentcomplementduringvirtualyear2,whichmay haveconfoundedtheresults.Unfortunately,wewere unabletoincludedemographicdata,whichmighthave helpedtoidentifyadditionalcancellationpatterns.Lastly, examiningtrendsinthosewhorescheduleinterviewswas notperformedinthisstudyandmaybeofvalueinfuture investigations,assomedownsidesdiscussedwithshortnoticecancellations(eg, fi llingemptyslots;havingtimeto reviewapplications)wouldstilloccurinapplicantswhoare reschedulingwithshortnotice.

CONCLUSION

Comparedwithin-personinterviewcycles,applicantsto ourresidencyprogramweresignificantlylesslikelytocancel virtualinterviews.However,themajorityofvirtual cancellationsthatdidoccurwerewithin14daysofthe interviewdateandnearlyone-fifthoccurredinunderseven days.Additionalstudies,ideallymultisitethatinclude applicantdemographicdata,areneededtodeterminehow cancellationpatternsaffectEMrecruitmentandmatch outcomesinthevirtualera.

AddressforCorrespondence:MeryllBouldin,MD,Universityof ArkansasforMedicalSciences,DepartmentofEmergency Medicine,4301WMarkhamSt.,Slot584,LittleRock,AR72205. Email: mebouldin@uams.edu

Figure2. Interviewcancellationsbylengthoftimefrominterview. d,day.

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Bouldinetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.TheCoalitionforPhysicianAccountability’sWorkGrouponMedical StudentsintheClassof2021MovingAcrossInstitutionsforPost GraduateTraining.Finalreportandrecommendationsformedical educationinstitutionsofLCME-accredited,U.S.osteopathic,andnonU.S.medicalschoolapplicants.2020.Availableat: https://www.aamc. org/system/files/2020-05/covid19_Final_Recommendations_ 05112020.pdf.AccessedMay22,2022.

2.DavisMG,HaasMRC,GottliebM,etal.Zoominginversus flyingout: virtualresidencyinterviewsintheeraofCOVID-19. AEMEducTrain. 2020;4(4):443–6.

3.DomingoA,RdesinskiRE,StensonA,etal.Virtualresidencyinterviews: applicantperceptionsregardingvirtualintervieweffectiveness, advantages,andbarriers. JGradMedEduc. 2022;14(2):224–8.

4.Li-SauerwineS,WeygandtPL,SmylieL,etal.Themorethingschange themoretheystaythesame:factorsinfluencingemergency medicineresidencyselectioninthevirtualera. AEMEducTrain. 2023;7(6):e10921.

5.EmergencyMedicineResidentsAssociationJarouZ,HillmanEA, KellogA,etal.(Eds.). EMRAandCORDStudentAdvisingGuide: AnEvidence-basedApproachtoMatchinginEM. 2019.Availableat: https://www.emra.org/books/msadvisingguide/msag AccessedJanuary19,2024.

6.EmergencyMedicineProgramDirectors.Re:interviewcancellations. CORDListservprivateemailchain.LastaccessedJanuary19,2024.

7.GorgensS.TheEMinterviewseason:pandemicedition.2020. Availableat: https://cordemblog.com/2020/09/03/the-em-interviewseason-pandemic-edition/.AccessedDecember20,2023.

8.HopsonLR,EdensMA,GoodrichM,etal.Calmingtroubledwaters:a narrativereviewofchallengesandpotentialsolutionsintheresidency interviewofferprocess. WestJEmergMed. 2020;22(1):1–6.

9.CouncilofResidencyDirectorsinEmergencyMedicine.Application ProcessImprovementCommittee(Apic):BestPracticesforthe 2023–2024ResidencyApplicationandInterviewSeason.2023.

Availableat: https://www.cordem.org/siteassets/files/match/ apic-2023-2024-application-interview-season-best-practices.pdf AccessedJuly26,2023.

10.NationalResidentMatchingProgram.MatchParticipationAgreement forPrograms.2023.Availableat: https://www.nrmp.org/wp-content/ uploads/2022/09/2023-MPA-Main-Match-Program-FINAL-3.pdf AccessedJune28,2023.

11.CouncilofResidencyDirectors.2023CORDstatementonresidency interviews.2023.Availableat: https://www.cordem.org/siteassets/files/ board/adv.–position-statements/2023-cord-statement-on-residencyinterviews.pdf.AccessedJuly26,2023.

12.AAMC.InterviewsinGME:Wheredowegofromhere?2023. Availableat: https://www.aamc.org/about-us/mission-areas/ medical-education/interviews-gme-where-do-we-go-here AccessedJuly23,2023.

13.TheCoalitionforPhysicianAccountability’sWorkGrouponMedical StudentsintheClassof2022MovingAcrossInstitutionsforInterviews forPostgraduateTraining.Recommendationson2021–22residency seasoninterviewingformedicaleducationinstitutionsconsidering applicantsfromLCME-accredited,U.S.osteopathic,andnon-U.S. medicalschools.Availableat: https://physicianaccountability.org/ wp-content/uploads/2021/08/Virtual-Rec_COVID-Only_Final.pdf AccessedJune22,2022.

14.LewkowitzAK,RamseyPS,BurrellD,etal.Effectofvirtualinterviewing onapplicantapproachtoandperspectiveofthematernal-fetal medicinesubspecialtyfellowshipmatch. AmJObstetGynecolMFM. 2021;3(3):100326.

15.BoydCJ,AnanthasekarS,VernonR,etal.Interviewhoarding: disparitiesintheintegratedplasticsurgeryapplicationcycleinthe COVID-19pandemic. AnnPlastSurg. 2021;87(1):1–2.

16.GordonAM,ConwayCA,ShethBK,etal.Howdidcoronavirus-19 impacttheexpensesformedicalstudentsapplyingtoanorthopaedic surgeryresidencyin2020to2021? ClinOrthopRelatRes. 2022;480(3):443–51.

17.WhelanA.Openletteronresidencyinterviews.2020.Availableat: https://www.aamc.org/media/50291/download?utm_source=sfmc& utm_medium=Email&utm_campaign=ERAS&utm_content=Interviews AccessedJanuary19,2024.

18.FarcyD,JungJ,AintablianH,etal.Consensusstatementonthe 2020–2021residencyapplicationprocessforUSmedicalstudents planningcareersinemergencymedicineinthemainresidency match.2020. https://www.emra.org/be-involved/be-an-advocate/ working-for-you/residency-application-process AccessedJanuary22,2024.

EDUCATION SPECIAL ISSUE:BRIEF RESEARCH

FoundationsofEmergencyMedicine:Impactofa Standardized,Open-access,CoreContentCurriculum onIn-TrainingExamScores

JaimeJordan,MD,MAEd*

NatashaWheaton,MD*

NicholasD.Hartman,MD,MPH† DanaLoke,MD,MS‡ NathanielShekem,MPAS§ AnwarOsborne,MD,MPM∥ P.LoganWeygandt,MD,MPH¶ KristenGrabowMoore,MD,MEd∥

SectionEditor:DanielleHart,MD

*UniversityofCaliforniaLosAngeles,DavidGeffenSchoolofMedicine, DepartmentofEmergencyMedicine,LosAngeles,California † WakeForestUniversitySchoolofMedicine,DepartmentofEmergency Medicine,Winston-Salem,NorthCarolina

‡ NorthwesternUniversityFeinbergSchoolofMedicine,Departmentof EmergencyMedicine,Chicago,Illinois

§ UniversityofIowa,DepartmentofEmergencyMedicine,IowaCity,Iowa

∥ EmoryUniversity,DepartmentofEmergencyMedicine,Atlanta,Georgia

¶ JohnsHopkinsUniversitySchoolofMedicine,Baltimore,Maryland

Submissionhistory:SubmittedJune28,2023;RevisionreceivedDecember9,2023;AcceptedJanuary12,2023

ElectronicallypublishedMarch25,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.18387

Introduction: Learnersfrequentlybenefitfrommodalitiessuchassmall-group,case-basedteaching andinteractivedidacticexperiencesratherthanpassivelearningmethods.Thesecontemporary techniquesarefeaturesofFoundationsofEmergencyMedicine(FoEM)curricula,andparticularlythe FoundationsI(F1)course,whichtargets first-yearresident(PGY-1)learners.TheAmericanBoardof EmergencyMedicineadministersthein-trainingexam(ITE)thatprovidesanannualassessmentofEMspecificmedicalknowledge.WesoughttoassesstheeffectofF1implementationonITEscores.

Methods: WeretrospectivelyanalyzeddatafrominternsatfourEMresidencyprogramsaccreditedbythe AccreditationCouncilforGraduateMedicalEducation.Wecollecteddatain2021.Participatingsiteswere geographicallydiverseandincludedthree-andfour-yeartrainingformats.Wecollecteddatafrominterns twoyearsbefore(controlgroup)andtwoyearsafter(interventiongroup)implementationofF1ateachsite. YearofF1implementationrangedfrom2015–2018atparticipatingsites.Weabstracteddatausinga standardformincludingprogram,ITErawscore,yearofITEadministration,USMedicalLicensingExam Step1score,Step2ClinicalKnowledge(CK)score,andgender.Weperformedunivariableand multivariablelinearregressiontoexploredifferencesbetweeninterventionandcontrolgroups.

Results: Wecollecteddatafor180PGY-1s.Step1andStep2CKscoresweresignificantpredictorsof ITEinunivariableanalyses(bothwith P < 0.001).AfteraccountingforStep1andStep2CKscores,we didnot findF1implementationtobeasignificantpredictorofITEscore, P = 0.83.

Conclusion: ImplementationofF1curriculadidnotshowsignificantchangesinperformanceontheITE aftercontrollingforimportantvariables.[WestJEmergMed.2024;25(4.1)15–18.]

INTRODUCTION

Residencyprogramsprovideeducationandtrainingto developcompetentphysicians.Boardcertificationin emergencymedicine(EM)requirescompletionofan

AccreditationCouncilforGraduateMedicalEducation (ACGME)-accreditedtrainingprogramandapassingscoreon theQualifyingExamination(QE)andOralCertification Examination(OCE)administeredbytheAmericanBoardof

EmergencyMedicine(ABEM).1,2 TheABEMIn-training Examination(ITE)isanimportanttoolusedbytraining programstoassessmedicalknowledgeandprepareresidents fortheQE.1,3 TheITEisdesignedtoreflectthecontentofthe ModelofClinicalPracticeofEmergencyMedicine(EM Model)andhaspredictivevalueinestimatingthelikelihoodof individualresidentspassingtheQE.3 Priorliteraturesuggests thatclinicalexposurealoneleavessignificantgapsin fundamentalknowledgedefinedbytheEMModel.4 Residency didacticcurriculaprovideanopportunitytosupplementcore knowledge;however,thebestmethodsforproviding instructionoutsideoftheclinicalsettingandpreparingtrainees forsuccessfulperformanceontheITEareunknown.

FoundationsofEmergencyMedicine(FoEM)isa national,free,open-access,onlineEMcurriculumthathas beenwidelyadoptedintheUnitedStates.5,6 FoEMbecame availablein2015;registrationforuseofFoEMcoursesfor the2022–2023academicyearincluded237registered educationalprograms,serving6,326residentphysicians.5,6 FoEMoffersstandardized,level-specific,corecontentfor EMresidentsusinglearner-centriceducationalstrategies thathavebeenshowntobenefitlearningsuchassmall-group discussion,peerlearning,andindividualizedguidance.5–11

FoundationsI(F1)isa flippedclassroom,case-basedcourse targetingpostgraduateyear(PGY)-1residentsthatincludes a30-unit,systems-basedcurriculumoffundamentalcontent intheEMModel.5,6,12 Priorliteraturedemonstratespositive effectsofthe flippedclassroommodelonlearning outcomes.13–15 TheF1curriculumincludescuratedselfstudyresourcescalled “LearningPathways” forlearnersto reviewpriortodidacticmeetings,inwhichresidentswork throughmultipleF1caseswithaknowledgeablefacilitator providinginformationinanoral-boardsstyleformat.6 The F1summarizesessentiallearningpointsandsharesthem withlearnersto fillknowledgegapsandallowforspaced repetition.6 AlthoughtheF1curriculumisnotspecifically designedforITEreview,third-partypairedassessmentsfor eachunithavebeenavailableforusesince2017.6

LimitedoutcomedataofFoEMF1establishedquality anddemonstratedhighsatisfactionamongfacultyleaders andresidentlearners.5,6 However,therehasnotbeenan assessmentofobjectivemeasuressuchasmedicalknowledge andITEperformanceThisinformationcanprovideamore comprehensiveassessmentofthevalueofimplementingsuch aprogram.Inthisstudy,wesoughttoevaluatetheeffectof F1courseimplementationonITEperformanceinthe PGY-1year.Wehypothesizedthatimplementationofthe structuredF1curriculumwouldleadtoimproved performanceontheITE.

METHODS

WeperformedaretrospectivecohortstudyofITEdata collectedfromPGY-1residentsatfourACGME-accredited EMresidencyprogramsintheUnitedStatesbeforeandafter

implementationoftheFoEMF1curriculum.Weselected participatingsitesthatweregeographicallydiverseand included3-and4-yeartrainingformats.Wecollecteddatain December2021.AllPGY-1residentsatparticipatingsites duringthestudyperiodwereeligibletoparticipate.We excludedPGY-1residentswhoweremissingdata.

Wedeterminedthattodetecta5%differenceinITEscore with80%powerandanalphaof0.05,wewouldneedto enroll81participantsineachgroup(controland intervention)foratotalof162participants.Ourcontrol groupconsistedofdatafromPGY-1residentsforthetwo yearspriortoimplementationateachsite.Ourintervention groupconsistedofdatafromPGY-1residentsforthetwo yearsafterimplementationateachsite.YearofF1 implementationrangedfrom2015–2018atparticipating sites.Theleadauthorfromeachsiteabstracteddatausinga standardformthatincludedprogram,ITErawtotalscore, yearofITEadministration,UnitedStatesMedicalLicensing Examination(USMLE)Step1score,USMLEStep2 ClinicalKnowledge(CK)score,andresidentgender.Priorto dataabstraction,theauthorgroupreadeachitemonthe formaloudandtrialedabstractingasmallportionof representativedatatoensureclarityofmeaningand consistencyinprocess.

Wecalculateddescriptivestatisticsfordemographicdata andITEperformance.Weperformedregressionanalysesto exploredifferencesbetweentheinterventionandcontrol groups.We firstperformedunivariablelinearregression analysesforvariablesincludingimplementationofF1, residencyprogram,yearofITEadministration,USMLE Step1score,USMLEStep2score,andresidentgenderwith ITErawscoreasouroutcomeofinterest.Weincluded variableswitha P -value < 0.1intheunivariableregressionin amultivariablelinearregressionwiththesameoutcome variable.Weconsideredvariableswitha P -valueof < 0.05in themultivariablemodelasstatisticallysignificant.We performedallanalysesinSPSSv27.0(IBMCorporation, Armonk,NY).

ThisstudywasapprovedbytheInstitutionalReview BoardoftheDavidGeffenSchoolofMedicineatUCLA.

RESULTS

Weabstracteddatafromatotalof224interns.We excluded44internswhoweremissingdata.Weanalyzed datafrom180interns(88pre-implementationand92postimplementation)whohadcompletedata.Thedemographics ofparticipantswithcompletedataareshownin Table1.The meanITErawscoreforinternsinthecontrolgroupwas 72.15 ± 6.72.ThemeanITEscoreforinternsinthe interventiongroupwas72.74 ± 7.93.Intheunivariable regressionanalyses,onlyUSMLEStep1andUSMLEStep2 CKscoresyielded P -valuesof < 0.1(Table2).Becauseour hypothesiscenteredontheimpactofimplementationofthe F1curriculumonITEscores,weforcedthisvariableasthe

Table1. Demographicdataofparticipatinginterns.

Controlgroup n(%)totaln = 88

Interventiongroup n(%)totaln = 92

Gender Male3231

Female5660

Non-binary01

MeanUSMLE

Step1score(SD)

MeanUSMLE

232(14.26)232(15.59)

Step2score(SD) 244(17.02)246(14.54)

USMLE,UnitedStatesMedicalLicensingExamination.

Table2. Resultsofunivariableregressionanalysisof recordedvariables. Variable

ImplementationofFoundationsF1curriculum0.59

Residencyprogram0.22

YearofITEadministration0.14

USMLEStep1score <0.001

USMLEStep2CKscore <0.001

Residentgender0.24

USMLE,UnitedStatesMedicalLicensingExamination; ITE, in-trainingexam; CK,clinicalknowledge.

lastvariableafterblockentryofvariablesofUSMLEStep1 scoreandUSMLEStep2CKscoreinthemultivariable regressionanalysis,despiteithavinga P -valueof0.59inthe univariableanalysis.AftercontrollingforStep1scoreand Step2CKscore,F1implementationwasnotasignificant predictorofITEscore,Rsquarechange = 0, P = 0.83.The datasatisfiedallassumptions.

DISCUSSION

OurstudydemonstratesthatbothStep1andStep2CK weresignificantpredictorsofITEscore.Thisisconsistent withpriorliteratureinmultiplespecialtiesdemonstrating associationsbetweenUSMLEscoresandITE performance.16–19 Wefoundthatourinterventiongrouphad aslightlyhigherrawITEscoreshowever,aftercontrolling forUSMLEscores,thisincreasewasnotstatistically significant,despitebeingadequatelypowered.Thiswas somewhatsurprisinggiventhatF1providesaconsistent structureandcomprehensivecoverageofcontentintheEM modelandalsoincorporatesteachingmethodsthathave beenshowntoenhancelearning.2,6–11 However,ourresults alignwithpreviousstudies,whichhavedemonstratedthat changesincurriculumwerenotassociatedwithsignificant differencesinITEperformance.20,21 Specifically,converting

anhourofsynchronousdidacticconferencetoasynchronous learning,andconvertingconferencelecturestosmallgroup, “flipped-classroom” stylelearninghavepreviouslybeen foundtohavenosignificanteffectonITEscores.20,21

ItisimportanttonotethattheobjectiveofF1isto improveEMcoreknowledgeandapplicationintheclinical environmentandisnotspecificallytargetedtowardsITEtest preparationorperformance.Additionally,performanceon theITEmaynotcomprehensivelyrepresentlearner knowledgeofEM.Thismaybeonereasonthatwedidnot findsignificantchangesinITEperformance.Additionally, variableimplementationandusageofF1atdiffering programscouldinfluencepotentialgains.Althoughthe FoEMcoursesarestandardized,participatingprograms mustaddresstheirownuniqueneedsandbarriers;thismay resultinvariabilityincourseimplementation,including variableuseof flipped-classroomstyleasynchronous resourcesandpairedassessments.Itisalsoimportanttonote thattheITEisadministeredinFebruaryofeachyear;thus, participatingPGY-1residentsinthisstudywereonly exposedtoapproximatelysevenmonthsoftheyear-longF1 curriculumpriortotheITE.

Itispossiblethatadditionalimprovementsmaybeseenwith additionaltimespentinthecurriculum.Thenonsignificant improvementseeninthisstudymaybeaugmentedwith implementationofFoundationsII(F2),whichisdesignedfor PGY-2residents,andFoundationsIII(F3),whichisdesigned forPGY-3andPGY-4residents.Theseoutcomesmeritfurther investigation.Whileourstudydidnot findasignificant increaseinITEscorescomparedtostandardcurricula,itwas notworsethanstandardpracticeandhasadditionalbenefitsof afree,standardized,pre-packaged,high-quality,adaptable formatwithuseracceptability.6

Overall,theresultsofthisstudyprovideimportant insightsforboththenumerousprogramsalreadyusing FoEMandthoseEMresidenciesconsideringincorporating itintotheirtrainingprograms.6 Inadditiontoprior feasibilityanduseracceptabilitydata,thisstudyprovidesan evaluationofobjectiveoutcomes,namelyknowledge,the firstlevelinMiller’spyramidofclinicalcompetence.6,22 Therearestillmanyunansweredquestions.Further investigationintotheeffectoftheF1curriculumonABEM QEandOCEperformanceshouldbepursued.Additionally, asFoEMisdesignedtosupportknowledgeapplicationinthe clinicalspace,futureworkcouldevaluatetheimpactof FoEMonotherdomainsofresidentperformance.

LIMITATIONS

Thisstudyhaslimitations.Theremaybeconfoundersnot accountedforinouranalysisthatcouldhaveinfluenced results.WedidnotcollectdataonspecificITEpreparation curriculaatparticipatingsites,individualusageofexternal ITEpreparationmaterialsoutsideoftrainingprogram curricula,timespentusingF1curriculum,useofpaired

assessments,totalnumberofF1unitscompletedby participatingresidents,ortimespentstudyingforITEin general.However,tothebestofourknowledge,therewere noothermajorchangestothesite’sdidacticcurriculumor methodsofpreparingtraineesfortheITEduringthestudy period.AlthoughtheF1courseincludesstandardized content,participatingprogramsmustaddresstheirown uniqueneedsandvariablesthatimpacttheconsistencyof courseadministration.Theremaybedifferencesinthe personnelwhodeliverthecontent,attendancerequirements, etc,whicharenotaccountedforinourstudy.Theresultsseen inthisstudymaynottransfertoothersiteswhereadherence toimplementationguidelinesismoreorlessconsistent.

CONCLUSION

OurstudysuggeststhattheFoEMF1curriculumisnot associatedwithsignificantchangesinperformanceonthe ITEinEMtrainingprogramsaftercontrollingforimportant variables.Theseresultsmayinformtheuseand implementationofFoEMcoursesinEMtrainingprograms.

AddressforCorrespondence:JaimeJordan,MD,MAEd,University ofCaliforniaLosAngeles,DavidGeffenSchoolofMedicine, DepartmentofEmergencyMedicine,924WestwoodBlvd.,Suite 300,LosAngeles,CA90024.Email: jaimejordanmd@gmail.com

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Jordanetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.ACGME.MilestonesbySpecialty:EmergencyMedicine.Accreditation CouncilforGraduateMedicalEducation.2021.Availableat: https://www.acgme.org/globalassets/pdfs/milestones/ emergencymedicinemilestones.pdf.AccessedNovember,2022.

2.AmericanBoardofEmergencyMedicine.BecomeCertified. Availableat: https://www.abem.org/public/become-certified AccessedJune23,2023.

3.AmericanBoardofEmergencyMedicine.In-TrainingExam.Available at: https://www.abem.org/public/for-program-directors/in-trainingexamination.AccessedJune23,2023.

4.BischofJJ,EmersonG,MitzmanJ,etal.Doestheemergencymedicine in-trainingexaminationaccuratelyreflectresidents’ clinical experiences? AEMEducTrain. 2019;3(4):317–22.

5.FoundationsofEmergencyMedicine.Availableat: https:// foundationsem.com/.AccessedJune23,2023.

6.MooreKG,KettererA,WheatonN,etal.Development,implementation, andevaluationofanopenaccess,level-specific,corecontent curriculumforemergencymedicineresidents. JGradMedEduc. 2021;13(5):699–710.

7.TwengeJM.Generationalchangesandtheirimpactintheclassroom: teachingGenerationMe. MedEduc. 2009;43(5):398–405.

8.Moreno-WaltonL,BrunettP,AkhtarS,etal.Teachingacrossthe generationgap:aconsensusfromtheCouncilofEmergencyMedicine ResidencyDirectors2009AcademicAssembly. AcadEmergMed. 2009;16(Suppl2):S19–24.

9.HartDandJoingS.TheMillennialGenerationand “thelecture.” Acad EmergMed. 2011;18(11):1186–7.

10.RobertsDH,NewmanLR,SchwartzsteinRM.Twelvetipsforfacilitating Millennials’ learning. MedTeach. 2012;34(4):274–8.

11.CooperAZandRichardsJB.Lecturesforadultlearners:breakingold habitsingraduatemedicaleducation. AmJMed. 2017;130(3):376–81.

12.BeesonMS,AnkelF,BhatR,etal.The2019modeloftheClinical PracticeofEmergencyMedicine. JEmergMed. 2020;59(1):96–120.

13.KingAM,GottliebM,MitzmanJ,etal.Flippingtheclassroomingraduate medicaleducation:asystematicreview. JGradMedEduc. 2019;11(1):18–29.

14.ChenF,LuiAM,MartinelliSM.Asystematicreviewoftheeffectiveness of flippedclassroomsinmedicaleducation. MedEduc. 2017;51(6):585–97.

15.ChenKS,MonrouxeL,LuYH,etal.Academicoutcomesof flipped classroomlearning:ameta-analysis. MedEduc. 2018;52(9):910–24.

16.FeningK,HorstAV,ZirwasM.CorrelationofUSMLEStep1scoreswith performanceondermatologyin-trainingexaminations. JAmAcad Dermatol. 2011;64(1):102–6.

17.NelsonMandCalandrellaC.DoesUSMLEStep1&2scorespredict successonITEandABEMQualifyingExam:areviewofanemergency medicineresidencyprogramfromitsinception. AnnEmergMed. 2017;70(4):58–9.

18.PatzkowskiMS,HauserJM,LiuM,etal.Medicalschoolclinical knowledgeexamscores,notdemographicorotherfactors,associated withresidencyin-trainingexamperformance. MilMed. 2023;188(1-2):e388–91.

19.PerezJAJrandGreerS.CorrelationofUnitedStatesMedicalLicensing examinationandinternalmedicinein-trainingexaminationperformance. AdvHealthSciEducTheoryPract. 2009;14(5):753–8.

20.KingAM,MayerC,BarrieM,etal.Replacinglectureswithsmallgroups: theimpactof flippingtheresidencyconferenceday. WestJEmergMed. 2018;19(1):11–7.

21.WrayA,BennettK,Boysen-OsbornM,etal.Efficacyofan asynchronouselectroniccurriculuminemergencymedicineeducation intheUnitedStates. JEducEvalHealthProf. 2017;14:29.

22.MillerGE.Theassessmentofclinicalskills/competence/performance. AcadMed. 1990;65(9Suppl):S63–7.

EDUCATION SPECIAL ISSUE:BRIEF RESEARCH REPORT

IntegratingHospiceandPalliativeMedicineEducationWithinthe AmericanBoardofEmergencyMedicineModel

RebeccaGoett,MD*

JasonLyou,MD†

LaurenR.Willoughby,MD†

DanielW.Markwalter,MD‡§

DianeL.Gorgas,MD†

LaurenT.Southerland,MD,MPH†

*RutgersNewJerseyMedicalSchool,DepartmentofEmergencyMedicine, Newark,NewJersey

† TheOhioStateUniversityWexnerMedicalCenter,DepartmentofEmergency Medicine,Columbus,Ohio

‡ UniversityofNorthCarolinaSchoolofMedicine,DepartmentofEmergency Medicine,ChapelHill,NorthCarolina

§ UniversityofNorthCarolinaSchoolofMedicine,UNCPalliativeCareProgram, ChapelHill,NorthCarolina

SectionEditor:ChrisMerritt,MD

Submissionhistory:SubmittedAugust29,2023;RevisionreceivedNovember20,2023;AcceptedJanuary12,2023

ElectronicallypublishedMarch25,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.18448

Background: Hospiceandpalliativemedicine(HPM)isaboard-certi fiedsubspecialtywithinemergency medicine(EM),butpriorstudieshaveshownthatEMresidentsdonotreceivesufficienttraininginHPM. ExpertsinHPM-EMcreatedaconsensuslistofcompetenciesforHPMtraininginEMresidency.We evaluatedhowtheHPMcompetenciesintegratewithintheAmericanBoardofEmergencyMedicine Milestones,whichincludetheModeloftheClinicalPracticeofEmergencyMedicine(EMModel)andthe knowledge,skills,andabilities(KSA)list.

Methods: ThreeemergencyphysiciansindependentlymappedtheHPM-EMcompetenciesontothe 2019EMModelitemsandthe2021KSAs.Discrepancieswereresolvedbyafourthindependent reviewer,andthe finalmappingwasreviewedbyallteammembers.

Results: TheEMModelincluded78%(18/23)oftheHPMcompetenciesasadirectmatch,andwe identifiedrecommendedareasforincorporatingtheother five.TheKSAsincluded43%(10/23).Most HPMcompetenciesincludedintheKSAsmappedontoatleastonelevelB(minimalnecessaryfor competency)KSA.ThreeHPMcompetencieswerenotclearlyincludedintheEMModelorintheKSAs (treatingend-of-lifesymptoms,caringfortheimminentlydying,andcaringforpatientsunder hospicecare).

Conclusion: ThemajorityofHPM-EMcompetenciesareincludedinthecurrentEMModelandKSAs andcorrespondtoknowledgeneededtobecompetentinEM.ProgramsrelyingontheEMMilestonesto plantheircurriculumsmaymisstraininginsymptommanagementandcareforpatientsattheendoflife orwhoareonhospice.[WestJEmergMed.2024;25(4.1)19–26.]

INTRODUCTION

Athirdofadultswhodiewillreceiveemergency departmentcareinthemonthpriortotheirdeath.1 Emergencyphysiciansneedtrainingtoprovidethehighquality,goal-concordantcarethatthesepatientsdeserve. Hospiceandpalliativemedicine(HPM)isasubspecialtyof emergencymedicine(EM)thataddsanadditionalfocuson

symptommanagement,goal-concordantcare,andqualityof life,especiallyforpatientswithchronicdiseaseorlifethreateningconditions,orwhoareattheendoflife.2 Prior researchhasshownthatcurrentEMresidencytraininglacks instructioninHPM.3–7 Toaddressthis,theAmerican CollegeofEmergencyPhysiciansPalliativeMedicineSection publishedalistof23criticaldevelopmentalmilestonesin

HPMtrainingforEMresidents.8 However,itisunclearhow besttointegratetheserecommendationsintoanEM residencycurriculum.

ManyEMresidencycurriculumsarebasedonthe knowledgeneededtopasstheEMboardcertificationexams. ThisknowledgeiscodifiedintheAmericanBoardof EmergencyMedicine(ABEM)ModeloftheClinical PracticeofEmergencyMedicine(EMModel)andalistof knowledge,skills,andabilities(KSA).9,10 TheEMmodel alongwiththeKSAsarethefoundationaldocumentsusedto createtheEMMilestones,acompendiumubiquitously employedinbothEMtrainingandassessment.Ourgoalin thisstudywastodeterminewheretheHPMcompetencies fit orcould fitwithintheEMModelandKSAs.Thismapping couldhelpguidecurriculumdesignortheincorporationof theHPMcompetenciesintotestingcontent.

METHODS

Thisstudywasnothumansubjectsresearchandwas deemedexemptfrominstitutionalreviewboardreview.We comparedthe2019EMModelandthe2021KSAstothe HPMcompetencies.TheHPMcompetencieswereassigned numerals.TheEMModelitemswereannotatedbytheir numberandcategory.ThenotationsfortheKSAcategories andcodeswereuseddirectlyfromthe2021document.We dividedtheKSAsintooverarchingcategories(eg,diagnosis, pharmacotherapy,reassessment)whichwethenfurther dividedintosetsofcompetencieswithinthatcategory.10 Eachcompetencywasgivenahierarchyintraining correspondingtoanalphabeticlevel(withAthemost advancedlevelofcompetencyandEtheleast).LevelAis reservedforadvancedknowledgeorskills.LevelBisthe minimalcompetencylevel,definedastheminimumskilllevel everyEMresidentshouldattaintograduate.LevelsC,D, andEareskillsinthedevelopmentofreachinglevelB.

Asthistypeofanalysishasnotbeendonebefore,weuseda sequentialapproachwithinitialindependentreviewers,a mediatorstep,andthen finalconsensusgroupdiscussion. Theconsensusgroupresultswerethenreviewedbytwo independentexternalexperts.Inthe firstphaseofconsensus mapping,tworesidents(EMpostgraduateyear(PGY)-2and EM/internalmedicine(PGY-4)andanEMattending independentlymappedpalliativecarecompetenciesusinga MicrosoftExcelspreadsheet(MicrosoftCorporation, Armonk,NY).Thethreeinitialconceptmappershad independentdatasheetsandwereblindedtoeachother’ s determinations.Acompetencycouldmapontomorethan oneareaoftheEMModel.First,keywordsfromeachHPM competencyweresearchedforintheEMModel.Ifno matcheswerefound,theEMModelwasreviewedlinebyline todeterminewhethertherewereconceptualmatches.Ifthere wasnodirectmatch,buttheHPMcompetencycouldbe incorporatedunderatopic,thiswaslistedasapotentialarea forincorporation.

Anytopicthatdidnothaveatleast2/3agreementonthe initialindependentreviewwasreviewedbyafourth emergencyphysicianwithexpertiseinEMresidenteducation andEMModeldevelopment.Shewasblindedtotheinitial reviewer’snamesbutdidhavetheirresults.Thefullgroup metandreviewedallthemappinguntilconsensuswas reached.Theconsensustableswerethenreviewed independentlybytwoadditionalexternalHPMboardcertifiedEMattendingsinvolvedinresidenteducationattwo differentEMresidencyprograms.Thesameprocesswasused formappingtheKSAs.

RESULTS

IncorporationintotheEmergencyMedicineModel

Fifty-oneof963EMModelitemsweretaggedinthe independent firstroundofmapping,with98.7%consensus (951/963)betweentheinitialthreeindependentreviewerson whetheranitemwasorwasnottaggedasamatch.The final reviewbytheindependentHPM-boardedEMattendingsdid notresultinadjustmentstoanyoftheexistingmappingbut didaddtothepotentialareasof fitfortheHPM competenciesthatdidnotdirectlymatchontotheEM Model. Table1 liststhecompetenciesincludedinthe2019 EMModel(18/23,78%).Manycompetencies fitinto EM Modelcategory20:OtherCoreCompetencies section, whichincludescommunicationskills,transitionsofcare, culturalcompetency,andhealthcarecoordination. Discrepancydiscussionscenteredaroundmanagementvs diagnosis.Thecompetency HPM2:Treatingdistressing symptoms(eg,nausea/vomiting,dyspnea) wasfeltto fitby keywordmatchunderEMModelcategory 1.0Signs, SymptomsandPresentations .However,thatcategorydoes notmentiontreatmentofsymptomsdirectly.Similarly, HPM18:ComplicationsofCancer couldmaptomany itemsintheEMmodel,butagainreferstopalliative managementofcancercomplicationsratherthandiagnosis.

PotentialAreasofFitintheEmergencyMedicineModel FiveHPMcompetenciesdidnot fitintotheEMModel. The firsttwo, HPM7:Treatingcommonend-of-life symptoms and HPM8:Carefortheimminentlydying (expectingdeathwithinhourstodaysorrecentlydeceased patientandtheirfamilymembers) ,couldbetaughtunder EMModelitem 20.4.4.2.2:Systems-basedPractice: Withdrawalofsupport. ThisEMModelitemcouldbe clarifiedtoensurethatitincludessymptomcontrolandendof-lifecare.Thenext, HPM11:Caringforpatientsunder hospicecare, couldbetaughtwhenteaching 20.4.4.2.3: Systems-basedPractice:HospiceReferral. However,the hospice-referralEMModelitembettermappedonto HPM 17 ,whichincludesassessingforandinitiatinghospice referrals.Theteamfeltthatidentifyingandreferringpatients tohospicewasaseparateskillsetthancaringforpatientson hospice.ThelasttwoHPMcompetencieswithoutaclear

Table1. ThehospiceandpalliativeemergencymedicineresidencyeducationcompetenciesmappedontotheAmericanBoardof EmergencyMedicineEMModel.

Hospiceand palliative competencyDescriptionEMmodelitem

1Paincontrol:a.chronicpain,b.malignant andnon-malignantpain.

19.3.1Anesthesiaandacutepain management-regionalanesthesia

19.3.2Anesthesiaandacutepain management-proceduralsedation

19.3.3Anesthesiaandacutepain management-analgesia

2Treatingdistressingsymptoms (eg,nausea/vomiting,dyspnea)

3Difficultcommunication:a.deliveryofbad news(eg,prognosisanddeathtelling) b.conflictresolution(eg,betweenfamily members

4Goalsofcarediscussions:a.assisting familieswithdecisionmaking.b.assisting patientswithdecisionmaking

1.3.32Nausea/vomiting 1.3.42Shortnessofbreath

*unclearwhethertheseEMmodelelementsreferto treatingthesesymptomsordevelopingadifferential diagnosisforthesesymptoms,butbothshould betaught.

20.1.2.2Interpersonalandcommunication skills-conflictmanagement

20.1.2.4Interpersonalandcommunicationskillsdeliveringbadnews/deathnotifications

20.4.4.1Healthcarecoordination-advancedirectives

5Caregiversupport20.3.4.6Well-beingandresilience-careforthecaregiver

6Non-initiationorstoppingofnonbeneficial interventions

19.2Resuscitation-cardiopulmonaryresuscitation

20.1.1.3Interpersonalskills-patientandfamily experienceofcare

20.4.4.2.2Healthcarecoordination-withdrawalofsupport

9Bereavementandgrieving14.2.4Mooddisordersandthought disorders-griefreaction

10Family-witnessedresuscitation19.2Resuscitation-cardiopulmonaryresuscitation

12Copingandself-care20.3.4.1Well-beingandresilience-fatigue andimpairment

20.3.4.1.1Well-beingandresilience-sleephygiene

20.3.4.3Well-beingandresilience-work/lifebalance

13End-of-lifemanagementinthemass casualtyincident/event

16Screeningforpalliativecareneeds:

a.identifyingpatientswhomaybenefit fromHPMspecialistreferral,b.identifying theimminentlydyingpatient(expected deathwithinhours-days).

17Rapidpalliativecareassessment:

a.aligningdiagnosticsandtherapeuticsto patientgoals,b.functional,psychosocial, andspiritualassessment,c.assessingfor andinitiatinghospicereferrals,d.toolkits tohelpidentifypatientneedsfor appropriatereferrals/resources, e.caregiverburden.

20.4.2.2.1Patienttriageandclassification

20.4.4.2.1Healthcarecoordination-patientidentification forpalliativecare

20.4.4.2.3Healthcarecoordination-hospicereferral

20.3.4.6Well-beingandresilience-careforthecaregiver

20.4.4.2.3Healthcarecoordination-hospicereferral

20.4.4.3.1Healthcarecoordination-activitiesofdaily living/functionalassessment (Continuedonnextpage)

Table1. Continued.

Hospiceand palliative competencyDescriptionEMmodelitem

18Complicationsofcancer:a.disease complications(eg,spinalcord compression,hypercalcemia), b.treatmentcomplications (eg,pancreatitis,tumorlysis,neutropenia, acuterenalfailure).

2.9.2.3Largebowel-radiationcolitis

2.9.2.5Largebowel-neutropenicenterocolitis/typhlitis

3.6.1Diseasesofthepericardium-pericardial tamponade

8.7Oncologicemergencies

8.7.1Oncologicemergencies-febrileneutropenia

8.7.2Oncologicemergencies-hypercalcemia ofmalignancy

8.7.3Oncologicemergencies-hyperviscositysyndrome

8.7.4Oncologicemergencies-malignant pericardialeffusion

8.7.5Oncologicemergencies-spinalcordcompression

8.7.6Oncologicemergencies-superiorvena cavasyndrome

8.7.7Oncologicemergencies-tumorhemorrhage

8.7.8Oncologicemergencies-tumorlysissyndrome

11.1.4.2Bonyabnormalities-tumor-relatedfractures

16.2.3Disordersofthepleura,mediastinum,andchest wall-pleuraleffusion

16.6.2Pulmonaryembolism/infarct-venous thromboembolism

16.6.2.1Pulmonaryembolism/infarct-massiveand submassiveembolism

19Ethical,spiritual,andculturalissues aroundend-of-lifeanddeath

20Advancedirectives:a.physicianorderfor life-sustainingtreatment(POLST), b.medicalorderforlife-sustaining treatment(MOLST),c. fivewishes.

21ethicalandlegalissues:a.decisionmakingcapacity,b.futility.

22Multidisciplinaryteamandsupport systems.(understandingteamrolesand systemresources):a.spiritualchair, b.socialchair,c.hospicecareeligibility, d.continuingcare,e.importanceoflocal andcommunitysupportsystems.

23Transitionsacrosscaresettings,eg, inpatientvshomehospice,palliative careunit

20.1.2.5Interpersonalandcommunication skills-culturalcompetency

20.4.4.1Healthcarecoordination-advancedirectives

20.3.2.4Professionalism-medicalethics

20.4.5.4Regulatory/legal-consent,capacityandrefusal ofcare-consent,capacityandrefusalofcare

20.1.1.1Interpersonalskills-inter-departmentaland medicalstaffrelations

20.1.1.2Interpersonalskills-intra-departmentalrelations, teamwork,andcollaborationskills

20.4.2.4.1EDadministration-alliedhealthprofessionals

20.4.4.2.1Healthcarecoordination-patientidentification forpalliativecare

20.4.4.2.3Healthcarecoordination-hospicereferral

associationwiththeEMModelwere HPM14:Trajectories ofdying:a.Terminalillness,b.OrganFailure,c.Frailty,d. SuddenDeath, and HPM15:Prognostication. Whilethese competenciesnecessitatehavingsoundunderstandingofthe naturalhistoryofdiseaseaswellasphysicalexaminationand clinicalworkupcomponentsinformingprognosis,theseare alsoskillsforexplainingthelikelihoodofdeathand communicatingwithpatientsandfamilies.Theteam consensuswasthatthesecouldbetaughtwithintheEM Modelitems 20.1.2.4InterpersonalandCommunication

Table2. Thepalliativeemergencymedicinecompetenciesincorporatewiththe2021AmericanBoardofEmergencyMedicineknowledge, skills,andabilities.

Hospiceandpalliative medicinecompetencyKSAcodeDescriptionLevel

3Difficultcommunicationa.deliveryofbad news(eg,prognosisanddeathtelling) b.conflictresolution (eg,betweenfamilymembers)

4Goalsofcarediscussions:a.assisting familieswithdecisionmaking.b.assisting patientswithdecisionmaking.

CS17Use flexiblecommunicationstrategiesto negotiateeffectivelywithstaff,consultants, patients,families,andotherstoprovideoptimal patientcare,recognizingandresolving interpersonalconflicts

CS3Elicitpatients’ reasonsforseekinghealthcareand theirexpectationsfromtheEDvisit

CS7Considertheexpectationsofthosewhoprovide orreceivecareintheEDandusecommunication methodsthatminimizethepotentialforstress, conflict,andmiscommunication

CS15Solicitpatientparticipationinmedicaldecisionmakingbydiscussing,risks,benefits,and alternativestocareprovided

ES15Elicitthepatient’sgoalsofcarepriortoinitiating emergencystabilization,includingevaluatingthe validityofadvanceddirectives

13End-of-lifemanagementinthemass casualtyincident/event

14Trajectoriesofdying:a.terminalillness, b.organfailure,c.frailty, d.suddendeath.

DM11Participateinamasscasualtydrilloreventinan EDinvolvingmultiplepatients,prioritizingcare, containingpotentialexposures,andappropriately assigningresources

ES6Recognizeinatimelyfashionwhenfurther clinicalinterventionisfutile

PE6Educatepatientsonthenaturalcourseoftheir diseaseandimpactofpossibletreatmentin relationtoprognosis

15PrognosticationES6Recognizeinatimelyfashionwhenfurther clinicalinterventionisfutile

ES15Elicitthepatient’sgoalsofcarepriortoinitiating emergencystabilization,includingevaluatingthe validityofadvanceddirectives

PE6Educatepatientsonthenaturalcourseoftheir diseaseandimpactofpossibletreatmentin relationtoprognosis

TC11Determine,summarize,andcommunicatethe diagnosisordiagnosticuncertainty,anticipated course,prognosis,dispositionplan,medications, futurediagnostic/therapeuticinterventions,signs andsymptomsforwhichtoseekfurthercareand follow-uptopatientorsurrogate

17Rapidpalliativecareassessment: a.aligningdiagnosticsandtherapeuticsto patientgoals,b.functional,psychosocial, andspiritualassessment,c.assessingfor andinitiatinghospicereferrals,d.toolkits tohelpidentifypatientneedsfor appropriatereferrals/resources, e.caregiverburden.

CS7Considertheexpectationsofthosewhoprovide orreceivecareintheEDandusecommunication methodsthatminimizethepotentialforstress, conflict,andmiscommunication

(Continuedonnextpage)

Table2. Continued.

Hospiceandpalliative medicinecompetencyKSAcodeDescriptionLevel

20Advancedirectives:a.physicianorderfor life-sustainingtreatment(POLST), b.medicalorderforlife-sustaining treatment(MOLST),c. fivewishes.

21Ethicalandlegalissues:a.decisionmakingcapacity,b.futility.

CS6Elicitinformationfrompatients,families,andother healthcaremembersusingverbal,nonverbal, written,andtechnologicalskills

ES15Elicitthepatient’sgoalsofcarepriortoinitiating emergencystabilization,includingevaluatingthe validityofadvanceddirectives

CS15Solicitpatientparticipationinmedicaldecisionmakingbydiscussing,risks,benefits,and alternativestocareprovided

ES6Recognizeinatimelyfashionwhenfurther clinicalinterventionisfutile

LI12Balancepatientautonomywithpatientprotection andadvocacywhenaddressingconsentand refusalofcareinaccordancewithlegaland ethicalstandards

TI9Obtaininformedconsentfromthepatientor appropriatesurrogatewhenindicated

22Multidisciplinaryteamandsupport systems.(understandingteamrolesand systemresources):a.spiritualchair, b.socialchair,c.hospicecareeligibility, d.continuingcare,e.importanceoflocal andcommunitysupportsystems.

23Transitionsacrosscaresettings, eg,inpatientvshomehospice, palliativecareunit

CS5Communicateinformationtopatientsandfamilies usingverbal,nonverbal,written,and technologicalskills,andconfirmunderstanding

CS10Communicatepertinentinformationtohealthcare colleaguesineffectiveandsafetransitions ofcare

TC11Determine,summarize,andcommunicatethe diagnosisordiagnosticuncertainty,anticipated course,prognosis,dispositionplan,medications, futurediagnostic/therapeuticinterventions,signs andsymptomsforwhichtoseekfurthercareand follow-uptopatientorsurrogate

TC15Ensuretransitionsofcareareaccuratelyand efficientlycommunicatedbetweencliniciansusing bestpractices

Skills:Deliveringbadnews/DeathNoti fi cations and 20.1.1.3InterpersonalandC ommunicationSkills:Patient andfamilyexperienceofcare.

IncorporationintotheKnowledge,SkillsandAbilities

Thirtyitemsof214weretaggedinthe firstroundwith87% consensus(187/214)betweentheinitialthreeindependent reviewersonwhetheranitemwasorwasnottaggedasa match.Tenofthe23HPMcompetencies(43%)mappedonto 16differentKSAs(Table2).Ofthe16matcheswithinthe KSAs,nonewereadvancedskills(levelA).AllbutHPM13

mappedontoatleastonelevelBskill.Atableshowingallthe HPMcompetenciesandtheirincorporationwithinthe EMModelandKSAstogetherisincludedas SupplementalDataA.

PotentialAreasofFitintotheKnowledge,Skills andAbilities

ThreeadditionalKSAswereidentifiedashavingareasof potential fitorincorporation. HPM5:Caregiversupport and HPM12:Copingandself-care couldbetaughtwhile discussing CS2:Establishrapportwithanddemonstrate

empathytowardpatientsandtheirfamilies .Finally, HPM 16:Screeningforpalliativecareneeds couldbetaughtwith TC18:Correctlydeterminetheappropriatedisposition .

DISCUSSION

ThisstudyshowedfairtogoodinclusionofHPM competencieswithinthepublishedEMKSAsandEM Model,demonstratingthattheHPMcompetenciesare representedintheMilestones.However,keytopicareaswere identifiedthatcouldimprovethefocusofEMtrainingin HPM.DemonstratingtheoverlapoftheHPMandEM contentmayhelpEMeducatorsensurethatHPMtrainingis incorporatedintotheircurriculums.Lackoftrainingon thesetopicsisaconsistent findinginnationaland internationalstudies,andeducatorsneedbetterwaysto incorporateHPM-EMtrainingintoresidency curriculums.3–7,11–13 ImprovedteachingoftheHPM-EM competencieshasthepotentialtodecreasethecaregaps seeninEDsymptommanagementandend-of-lifecare, includinglackofgoalsofcareconversationsforcritically illpatients.14,15

AlimitationoftheHPMcompetenciesisthattheyhave notbeenexternallyassessedorinvestigatedandarebasedon expertconsensus.Noneoftheinitialfourreviewerswere involvedinthedevelopmentoftheHPMcompetenciesand theyfoundthemtoalmostallmapontotheEMModelor identifiedplacesintheEMModelthatcouldbeexpandedto includethemmoreexplicitly.Additionally,theHPM competenciesthatmappedontoKSAsallmetatleastone KSAontheminimalcompetencylevel.These findingsimply thattheHPMcompetenciesareskillsthatareat residentlevel.

ThedescriptionsintheHPMcompetenciescanadddepth tothecorrespondingEMMilestonesforcurriculum developmentandsummativeevaluation.Forexample,most residenciesprovidetrainingorsimulationsofmasscasualty care.Thestudygroupenvisionedwaysinwhichend-of-life managementcouldbeaddedintothattraining(HPM13). Likewise,alectureonpost-cardiacarrestcarecould incorporatetrainingonthenon-initiationorcompassionate discontinuationofinterventionssuchasmechanical ventilation(HPM6).Summativecompetencyassessmentsat endoftrainingtogainboardcertificationcouldalso incorporatemoreHPMcompetency-basedquestions.

MuchoftheoverlapbetweentheHPMcompetenciesand theEMModelandKSAswasin Interpersonaland CommunicationSkills (EMModel)andthe CS –Communication&InterpersonalSkills (KSAs). Communicationskills,althoughchallengingtoteach,are criticalinpatient-centeredcareandwilllikelyhavean increasedemphasisasartificialintelligenceandmachine learningbecomemoreuniversallyintegratedintoclinical care.Currentmodelsforcommunicationinstructionrely heavilyonrolemodeling.16 Residentshavesuggestedthat

formaltrainingincommunicationshouldfocusongeneral communicationskillsandshouldprovidesyntaxtousein futurediscussions.17 Developingcommunicationskills requiresdeliberatepracticeoftechniques,includingNURSE statements(naming,understanding,respecting,supporting, andexploring)andAsk-Tell-Ask.17,18 Additionally, educatorsmustbecomefamiliarwithmethodsfor real-timeteachingofcommunication,suchas “CouldI addsomething?”19

Trajectoriesofdying(HPM7)andprognostication (HPM8)aretwoskillsusedtocounselpatients/familieswith seriousillnessorattheendoflifethatdidnot fitclearlywithin theEMModel.Thesearedifficultskills,andpriorstudies haveidentifiedsomediscordancebetweenwhatfamilies/ caregiversunderstandaboutaperson’sdeathandthe underlyingcausesofdeathidentifiedbythephysician-led team.20 Thus,thisskillshouldbehonedthroughouttraining. ItisourexperiencethatEMresidentsrarelyreceiveexplicit educationonprognostication,andsowerecommendits incorporationintocurriculums.Ourresultsfurthersuggest thattrainingontreatingend-of-lifesymptoms,careforthe imminentlydying,andcaringforpatientsunderhospicecare couldbeoverlookedbycurrentresidentcurriculumswith strictadherencetotheEMModel.

LIMITATIONS

Alimitationofthisprojectisthateventhoughaconsensus processwasusedwithexpertsinresidencyeducationand HPM,othereducationexpertsmayinterpretthedomains andcompetenciesdifferently.Forexample,the EMModel item20.3.4.6Well-beingandResilience-Careforthe caregiver wasmatchedtoHPM5and17aboutpatient caregivers.However,thiscouldalsobeinterpretedas residentself-careasitisunderthewell-beingsection.Finally, whiletrainedHPMemergencyphysiciansreviewedallthe mapping,theinitialmappingdidincluderesidentinput.This couldbeconsideredanadvantage,astheyareexperiencing lecturesweekly,orareapotentialsourceofbias,astheyhave nothadafullEMcurriculumyet.

CONCLUSION

WeidentifiedareasofoverlapwheretheHPM-EM subspecialtycompetenciescanbeemphasizedorintegrated intoEMModel-basedresidencycurriculums.This knowledgecanbeusedforcurriculumplanningand incorporatingHPMintodefinitionsforcompetencyinEM. Thesecouldalsobereflectedin finalsummativeevaluations forcertification.

AddressforCorrespondence:LaurenT.Southerland,MD,TheOhio StateUniversityWexnerMedicalCenter,DepartmentofEmergency Medicine,725PriorHall,376W10thAve.,Columbus,OH43210. Email: Lauren.Southerland@osumc.edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. LaurenT.SoutherlandhasNIHgrantfundingnotpertainingtothis study.DianeL.GorgasisaboardmemberontheAmericanBoardof EmergencyMedicine.LaurenT.SoutherlandandDanielW. Markwalterhavecontributedtosomeofthefreeeducational websitesmentionedinthediscussion.Therearenootherconflictsof interestorsourcesoffundingtodeclare.

Copyright:©2024Goettetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.ElmerJ,MikatiN,ArnoldRM,etal.Deathandend-of-lifecarein emergencydepartmentsintheUS. JAMANetwOpen. 2022;5(11):e2240399.

2.GeorgeN,BowmanJ,AaronsonE,etal.Past,present,andfutureof palliativecareinemergencymedicineintheUSA. AcuteMedSurg. 2020;7(1):e497.

3.WoodsEJ,GinsburgAD,BellolioF,etal.Palliativecareinthe emergencydepartment:asurveyassessmentofpatientandprovider perspectives. PalliatMed. 2020;34(9):1279–85.

4.MeoN,HwangU,MorrisonRS.Residentperceptionsofpalliative caretrainingintheemergencydepartment. JPalliatMed. 2011;14(5):548–55.

5.KrausCK,GreenbergMR,RayDE,etal.Palliativecareeducationin emergencymedicineresidencytraining:asurveyofprogramdirectors, associateprogramdirectors,andassistantprogramdirectors. JPain SymptomManage. 2016;51(5):898–906.

6.BaylisJ,HarrisDR,ChenC,etal.Palliativeandend-of-lifecare educationinCanadianemergencymedicineresidencyprograms: anationalcross-sectionalsurvey. CJEM. 2019;21(2):219–25.

7.AdeyemiOJ,SimanN,GoldfeldKS,etal.Emergencyproviders’ knowledgeandattitudestowardhospiceandpalliativecare:acrosssectionalanalysisacross35emergencydepartmentsintheUnited States. JPalliatMed. 2023;26(9):1252–60.

8.ShoenbergerJ,LambaS,GoettR,etal.Developmentofhospiceand palliativemedicineknowledgeandskillsforemergencymedicine

residents:usingtheAccreditationCouncilforGraduateMedical EducationMilestoneframework. AEMEducTrain. 2018;2(2):130–45.

9.AmericanBoardofEmergencyMedicine.The2019ModeloftheClinical PracticeofEmergencyMedicine.Availableat: https://www.abem.org/ public/resources/em-model.AccessedNovember3,2020.

10.AmericanBoardofEmergencyMedicine.2021Knowledge,Skills,& Abilities.Availableat: https://www.abem.org/public/resources/ emergency-medicine-milestones-ksas.AccessedNovember17,2022.

11.SandersS,CheungWJ,BakewellF,etal.Howemergencymedicine residentshaveconversationsaboutlife-sustainingtreatmentsincritical illness:aqualitativestudyusinginductivethematicanalysis. AnnEmerg Med. 2023;82(5):583–93.

12.ZengH,EugeneP,SupinoM.Wouldyoubesurprisedifthispatient diedinthenext12months?Usingthesurprisequestiontoincrease palliativecareconsultsfromtheemergencydepartment. JPalliatCare. 2020;35(4):221–5.

13.BeneschTD,MooreJE,BreyreAM,etal.Primarypalliativecare educationinemergencymedicineresidency:amixed-methodsanalysis ofayearlong,multimodalintervention. AEMEducTrain. 2022;6(6):e10823.

14.WalkerLE,StanichJA,BellolioF.Aqualitativeassessmentofaguide forgoalsofcareconversationsintheED. AmJEmergMed. 2023:75:185–7.

15.YilmazS,GrudzenCR,DurhamDD,etal.Palliativecareneedsand clinicaloutcomesofpatientswithadvancedcancerintheemergency department. JPalliatMed. 2022;25(7):1115–21.

16.OldeBekkinkM,FarrellSE,TakayesuJK.Interprofessional communicationintheemergencydepartment:residents’ perceptions andimplicationsformedicaleducation. IntJMedEduc. 2018;9:262–70.

17.RisingKL,PapanagnouD,McCarthyD,etal.Emergencymedicine residentperceptionsabouttheneedforincreasedtrainingin communicatingdiagnosticuncertainty. Cureus. 2018;10(1):e2088.

18.TalkVital.QuickGuideResources.Availableat: www.vitaltalk.org/ resources/.AccessedAugust14,2023.

19.BackAL,ArnoldRM,TulskyJA,etal. “CouldIaddsomething?”: teachingcommunicationbyinterveninginrealtimeduringaclinical encounter. AcadMed. 2010;85(6):1048–51.

20.MoonF,KissaneDW,McDermottF.Discordancebetweenthe perceptionsofcliniciansandfamiliesaboutend-of-lifetrajectories inhospitalizeddementiapatients. PalliatSupportCare. 2021;19(3):304–11.

EDUCATION SPECIAL ISSUE:BRIEF RESEARCH REPORT

DavidA.Haidar,MD*

LauraR.Hopson,MD*

RyanV.Tucker,MD†

RobD.Huang,MD*

JessicaKoehler,MD*

NikTheyyunni,MD*

NicoleKlekowski,MD*

ChristopherM.Fung,MD,MS*

SectionEditor:BenjaminSchnapp,MD

*UniversityofMichigan,DepartmentofEmergencyMedicine,AnnArbor,Michigan † UniversityofColorado,DepartmentofEmergencyMedicine,Aurora,Colorado

Submissionhistory:SubmittedSeptember13,2023;RevisionreceivedNovember10,2023;AcceptedDecember7,2023

ElectronicallypublishedFebruary28,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.18454

Introduction: Emergencymedicine(EM)isoneoffewspecialtieswithvariabletraininglengths.Hiringa three-yeargraduatetocontinuefellowshiptraininginadepartmentthatsupportsafour-yearresidency programcanleadtoconflictsaroundresidentsupervision.Wesoughttounderstandhiringandclinical supervision,orstaffing,patternsofnon-AccreditationCouncilforGraduateMedicalEducation(ACGME) fellowshipshostedatinstitutionssupportingfour-yearresidencyprograms.

Methods: Weperformedaweb-based,cross-sectionalsurveyofnon-ACGMEfellowshipdirectors(FD) hostedatinstitutionssupportingfour-yearEMresidencyprograms.Wecalculateddescriptivestatistics. Ourprimaryoutcomewastheproportionofprogramswithfour-yearEMresidenciesthathirenonACGMEfellowsgraduatingfromthree-yearEMresidencies.

Results: Of119eligibleFDs,88(74%)completedthesurvey.SeventyFDs(80%)indicatedthattheyhire graduatesofthree-yearresidencies.Fifty-six(80%)indicatedthatthree-yeargraduatessupervise residents.MostFDs(74%)indicatednoadditionalrequirementsexisttosuperviseresidentsoutsideof beinghiredasfaculty.TheFDsciteddepartmentpolicy,concernsaboutqualityandlengthoftraining, andresidentcomplaintsasreasonsfornothiringthree-yeargraduates.Amajority(10/18,56%)noted thatnothiringfellowsfromthree-yearprogramsnegativelyimpactsrecruitmentandgivesthemaccessto asmallerapplicantpool.

Conclusion: Mostnon-ACGMEfellowshipsatinstitutionswithfour-yearEMprogramsrecruitthree-year graduatesandallowthemtosuperviseresidents.Thissurveyprovidesprogramsinformationonhow comparablefellowshipsrecruitandstafftheirdepartments,whichmayinformpoliciesthat fittheneedsof theirlearners,thefellowship,andthedepartment.[WestJEmergMed.2024;25(4.1)27–32.]

INTRODUCTION

Emergencymedicine(EM)isoneoffewspecialtiesinthe UnitedStateswithvariabletraininglengths.1,2 Most residenciesimplementathree-yearmodel,whileonly20% implementafour-yearmodel.3–5 Thereislittledatato supporteithertraininglength.3–7 Somearguethatfour-year

graduateshavemoretimetogainconfidence,develop proceduralskills,developacademicinterests,andgain experiencesupervisinglearners.Advocatesofthree-year programsarguethatanextrayearasfacultywould providethesesameexperiences.1,8,9 Thesepersonal biasesmayimpactrecruitmentandhiringof

three-yeargraduatesatinstitutionssupportingfour-year residencyprograms.6,7

Whenaninstitutionhostingafour-yearresidencyhiresa three-yeargraduateintofellowshiptraining,thiscanleadto conflictsaroundclinicalsupervision,orstaffing,ofresidents relatedtoperceptionsofseniorityandqualityoftraining.1,9 Therearecurrentlynobestpracticesorguidelinestoinform programsonhowtoaddressthissituation.Thesituationis furthercomplicatedasnon-AccreditationCouncilfor MedicalEducation(ACGME)fellowshipsfrequentlylack uniformrulesthatgovernrecruitment,program requirements,andclinicalresponsibilities.10 Nostudies currentlyevaluatetheprevalenceoftheseissuesorexamine variabilityinrecruitment,hiring,andclinicalresponsibilities oftraineesatnon-ACGMEfellowships.Inthisstudy,we soughttounderstandthehiringandstaffingpatternsofnonACGMEfellowshipshostedatinstitutionswithfour-year EMresidencyprograms.

METHODS

StudyDesignandParticipants

Thiswasacross-sectionalsurveyoffellowshipdirectors (FD)ofnon-ACGMEfellowshipshostedatinstitutions supportingafour-yearEMresidencyprogram.We conductedthesurveybetweenJanuary–April2023.This studywasdeemedexemptbyourinstitutionalreviewboard (HUM00221519).InNovember2022,wegeneratedalistof 54four-yearEMresidencyprogramsfromtheEmergency MedicineResidents’ Association(EMRA)Matchrosterand ElectronicResidencyApplicationServicedirectory.11–13 We identifiednon-ACGMEfellowshipsofferedusingeach program ’swebpage,theSocietyforAcademicEmergency MedicineFellowshipDirectory,andtheSocietyforClinical UltrasoundFellowshipsdirectory.14,15

SurveyDevelopmentandDistribution

WedevelopedthesurveybasedonPanacek’sgeneral surveyprinciples,literaturereview,andexpertopinionto providecontentvalidityevidence.6,16–18 Allauthorshave experiencedevelopingsurveystudies,andthegroup (includingfourcurrentorformerFDs)iterativelypilotedand revisedthesurveyforoptimalphrasing,surveylength, functionality,andappropriatemixofsuggestedand open-endedresponses,whichprovidedcontentand responseprocessvalidityevidence.18 WeusedQualtrics (QualtricsXM,Provo,UT),aweb-basedsurveyplatform,to distributethesurveyviaemailwithapersonalizedlinkfor eachFDtocollectandanalyzethedata.Wesentweekly reminderstoFDs’ institutionalemails,withanoptionto declineparticipation,foreightweeks.Wethensent personalizedweeklyreminderemailsforanadditional fourweeks.Wecollectedindividualresponsestothe surveyanonymously.

OutcomesandDataAnalysis

WeaskedFDstoreporttheirfellowshiptype,yearsin currentrole,anddemographicdatasuchasnumberof clinicalsites,programenvironment(academic,county, community,etc),andgeographiclocation.Ourprimary outcomewastheproportionofprogramsaffiliatedwithfouryearEMresidenciesthathirenon-ACGMEfellows graduatingfromthree-yearEMresidencies.Wealsoasked clarifyingquestionstobetterunderstandtheirstaffingmodel, andrecruiting,hiring,andclinicaloversightpolicies.The surveyincludedspaceforcommentssothattheFDscould providecontexttotheiranswers,butwedidnotanalyzethese forthemes.Thefullsurveyisavailablein AppendixA1.We analyzedthedatausingExcel365(MicrosoftCorporation, Redmond,WA)togeneratedescriptivestatisticsand analysis.Weassessedtheassociationbetweencategorical variablesusingtheFisherexacttest.Wedidnotcalculatean apriorisample-sizeestimateasweattemptedtocapturea 100%responserate.

RESULTS

Of54four-yearEMresidenciesintheUS,32institutions offeredatleastonenon-ACGMEfellowshipwithatotalof 128fellowshipsidentified(median3.5;range1–10).We received88responsesafterexcludingnineopt-outsandone blankresponse(88/119)foraresponserateof73.9%. ProgramandFDcharacteristicsarelistedinthe Table. Freetextresponsesareincludedin AppendixA2

Ofthe88responses,70FDs(80%)reportedhiring graduatesofthree-yearEMprogramsfortheirrespective fellowships.Fifty-sixFDs(80%)whoacceptthree-year graduatesindicatedthattheirfellowscansuperviseEM residents.Wefoundvariationinwhofellowscouldsupervise. Themostcommonpolicy(40%)wasthatfellowscan superviseEMpostgraduate-year(PGY)-3residentsand below.MostFDs(74%)indicatedthattheyhadno additionalrequirementstosuperviseresidentsoutsideof beinghiredonasfaculty.Fullsurveyresultsappearin the Figure

Programswithmultipleclinicalsitesaremorelikelytohire three-yeargraduates.Tenof23programs(57%)withone clinicalstaffingsitehiredthree-yeargraduatescomparedto 88%(57/65)ofsiteswithtwoormoreclinicalsites (P < 0.001).TheFDsreportedtheimplementationofvarious strategiestomitigatepotentialconflicts.Oneprogram hostsajointfellowshipcurriculumfortheirfellows,which incorporatesinstructiononbedsideteaching,giving feedback,andteachingvariousskills.OtherFDsreported thattheirprogramspreventedtheirfellowsfromstaffingin highacuityareasordelayworkingwithresidents.

Twenty-sevenFDs(50%)citeddepartmentpolicyasthe reasonfortheirhiringandstaffingpolicies.Selected commentsfromotherFDsincludedconcernsaboutquality andlengthoftrainingandresidentcomplaints.Others

Table. Demographicdetailsofthefellowshipsrepresentedinoursurveyoffellowshipdirectorsofnon-ACGMEfellowshipprograms. DemographicsNumberofresponses(%)

Fellowshiptype

Central(IL,IN,IA,KS,MI,MN,MO,NE,OH,WI)13(15%)

Northeast(CT,DC,DE,MA,MD,ME,NH,NJ,NY,PA,RI,VT)45(51%)

Southern(AL,AR,FL,GA,KY,LA,MS,NC,OK,PR,SC,TN,TX,VA,WV)0(0%)

Western(AZ,CA,CO,NM,NV,OR,UT,WA)30(34%)

Categoryofnon-ACGMEfellow’sprimaryclinicalsite*

Numberofclinicalsitesnon-ACGMEfellowsclinicallystaff

*Respondentscouldselectmorethanonetypeofclinicalsite.

reportedtheirclinicalenvironmentwasnotconduciveto separatingfellowsfromresidents.SevenFDsreported wantingtoavoidPGY-4fellowsstaffingPGY-4residents. OneFDindicatedthat “becausewearea4-yearprogram,we

wanttoacknowledgetoourresidentsthat4yearsiswhatwe thinkisrequiredforgraduation.” Amongtheprogramsnothiringfellowsfromthree-year programs,56%(10/18)ofFDsnotedthatthispolicy

Figure. Flowdiagramdetailingthesurveyresponsehierarchyoffellowshipdirectorsofnon-AccreditationCouncilforGraduateMedical Educationfellowshipsregardingclinicalsupervisionpatternsfor3-yearemergencymedicine(EM)residencygraduatesatinstitutionswitha 4-yearEMprogram.

EM,emergencymedicine; PGY,postgraduateyear; FPPE,focusedprofessionalpracticeevaluation.

negativelyimpactedtheirfellowshiprecruitmentandgave themaccesstoasmallerpoolofapplicants.

DISCUSSION

Toourknowledgethisstudyisthe firsttodescribestaffing patternsofnon-ACGMEfellowshipshostedatinstitutions withfour-yearEMresidencies.MostoftheFDswesurveyed hirethree-yeargraduatesasfellows,andmostprograms permitthree-yeargraduatestostaffresidentswithno additionalrequirementsbeyondbeinghired.Wealso identifiedpotentialnegativeimpactsonfellowshipsasthey restricttheirapplicantpool.OneFDindicatedthattheir fellowshipwasmostlygoingunfilledduetotheirrecruitment policy.Anotherindicatedthatthe financialsacrificeofa four-vsthree-yearresidencymayunintentionallyfavor recruitmentofthosewithout financialneedorburden, especiallysincethedebtloadofEMapplicantsisreportedly higherthanforothermedicalspecialties.1

Someprogramsoffertheirfellowsalternativeclinical sites – suchasVeteransAffairshospitals,freestandingEDs, orurgentcares.Bystaffingmultiplelocations,non-ACGME fellowscanworkwithoutaresidentpresence.This flexibility allowsprogramstohirethree-yeargraduatesandpermits fellowstointerfacewithresidentsacademicallywithout havingtosupervisethemclinically.Thisallowsforatraining

environmentconducivetotheneedsofalllearners’ growth anddevelopment.

TheFDscitedclinicalconcernsanddepartmentpolicyas themainreasonsfortheirstaffingandhiringpolicies.There isalackofobjectivedatathatfour-yeargraduates outperformthree-yeargraduatesclinicallyoronthe qualifyingwrittenboardexam,suggestingthatthismaybe rootedinbias.1,6,7 Intheabsenceofrobustdatatosupport theclinicalcapabilitiesoftraineesfromeitherthree-orfouryearprograms,theprinciplesofcompetency-basedmedical education(CBME)mayoffersolutions.19 Theprinciplesof CBMErequiredemonstrationofcompetencyanddecouple attainmentofcompetencyfromtime-in-training.19 Theuse ofCBMEtodeterminereadinessforunsupervisedpractice throughaprocessknownas “promotioninplace” hasbeen pilotedbysomeresidencyprogramsandmaybeauseful modeltoreplicateindeterminingfellowreadinessfor staffing,regardlessofPGYstatus.19,20 Ifweremovethefocus fromtime-boundedtrainingandfocusondemonstratedskill acquisition,programsmaydesignprocessestoonboard three-yeargraduatesbyfocusingondevelopingandassessing appropriateskillsforsupervisionoftrainees.

Futurestudiescouldexplorewhosetsdepartmental policiesregardingfellowstaffing,evaluatefellowand residentperceptionsofstaffingpolicies,andcompare

careeroutcomesoffellowsworkinginvarious staffingenvironments.

LIMITATIONS

Wemaynothavecapturedallnon-ACGMEfellowships atfour-yearinstitutions.Wedidnotidentifyfellowship directoriesbesidesultrasound,whichmayhaveledto samplingbias.Weattemptedtomitigatethisbysearching specificprogramwebsitesforlistedfellowships.TheFDs whodidnotparticipateinourstudymayrepresentaunique populationwithdifferenthiringandstaffingpatterns.Wedid notidentifynon-ACGMEfellowshipshostedatfour-year EMprogramsinthesouthernUS,nordidwereceive responsesfromprimarilycommunityEMprograms,which couldalsohavebiasedourresults.Wedidnotsurvey ACGME-accreditedfellowships,asfellowsvaryintheway they “maintaintheirprimaryBoardskills.”21 SomeACGME fellowships(eg,criticalcare,emergencymedicalservices)do notrequireminimumclinicalhoursintheemergency department,whichleadstoaqualitativelydifferent experiencefromnon-ACGMEfellowships,wherefellowsare appointedasclinicalfaculty.2,21,22

CONCLUSION

Ourresultsindicatethatmostnon-ACGMEfellowships hostedatinstitutionswithfour-yearEMprogramsrecruit graduatesofthree-yearprogramsandallowthemto superviseresidents.Thissurveydataprovidesprogram informationonhowcomparablefellowshipprogramsrecruit andstafftheirdepartments,whichmayinformpoliciesthat fittheneedsoftheirlearners.

AddressforCorrespondence:DavidA.Haidar,MD,Universityof Michigan,DepartmentofEmergencyMedicine,1500E.Medical CenterDr.,B1-380,AnnArbor,MI48109.Email: dahaidar@med. umich.edu

Con

flictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Haidaretal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.RossTM,WolfeRE,MuranoT,etal.Three-vs.four-yearemergency medicinetrainingprograms. JEmergMed. 2019;57(5):e161–5.

2.AccreditationCouncilforGraduateMedicalEducation(ACGME). ACGMEprogramrequirementsforgraduatemedicaleducationin

emergencymedicine.2023.Availableat: https://www.acgme.org/ specialties/emergency-medicine/program-requirements-and-faqsand-applications/.AccessedJanuary6,2023.

3.LotfipourS,LuuR,HaydenSR,etal.Becominganemergencymedicine resident:apracticalguideformedicalstudents. JEmergMed. 2008;35(3):339–44.

4.NelsonLS,CalderonY,AnkelFK,etal.AmericanBoardofEmergency Medicinereportonresidencyandfellowshiptraininginformation (2021–2022). AnnEmergMed. 2022;80(1):74–83.e8.

5.AccreditationCouncilforGraduateMedicalEducation(ACGME).Listof programsbyspecialty.Availableat: https://apps.acgme.org/ads/Public/ Reports/Report/1.AccessedApril29,2023.

6.HopsonL,ReganL,GisondiMA,etal.Programdirectoropiniononthe ideallengthofresidencytraininginemergencymedicine. AcadEmerg Med. 2016;23(7):823–7.

7.NikollaDA,ZocchiMS,PinesJM,etal.Four-andthree-yearemergency medicineresidencygraduatesperformsimilarlyintheir firstyearof practicecomparedtoexperiencedphysicians. AmJEmergMed. 2023;69:100–7.

8.HaydenSandPanacekE.Proceduralcompetencyinemergency medicine:thecurrentrangeofresidentexperience. AcadEmergMed. 1999;6(7):728–35.

9.WeichenthalL.Advantagesofafour-yearresidency. CalJEmergMed. 2004;5(1):18–9.

10.AccreditationCouncilforGraduateMedicalEducation(ACGME). Commonprogramrequirements(fellowship).2022.Availableat: https:// www.acgme.org/globalassets/pfassets/programrequirements/ cprfellowship_2022v3.pdf.AccessedJanuary6,2023.

11.EmergencyMedicineResidents’ Association(EMRA).Matchlist.2023. Availableat: https://webapps.acep.org/utils/spa/match#/search/list AccessedJanuary6,2023.

12.AssociationofAmericanMedicalColleges(AAMC).Electronic residencyapplicationservice(ERAS)directory.2023.Availableat: https://systems.aamc.org/eras/erasstats/par/display.cfm? NAV_ROW=PAR&SPEC_CD=110.AccessedJanuary6,2023.

13.AssociationofAmericanMedicalColleges(AAMC).Residencyexplorer tool.Availableat: https://www.residencyexplorer.org/Home/Dashboard AccessedJanuary6,2023.

14.SocietyforAcademicEmergencyMedicine(SAEM).Fellowship directory.Availableat: https://member.saem.org/SAEMIMIS/ SAEM_Directories/Fellowship_Directory/SAEM_Directories/P/ FellowshipList.aspx.AccessedJanuary6,2023.

15.SocietyofClinicalUltrasoundFellowships(SCUF).Programlist. Availableat: https://www.eusfellowships.com/programs AccessedJanuary6,2023.

16.PanacekEA.Survey-basedresearch:generalprinciples. AirMedJ. 2008;27(1):14–6.

17.AlerhandS,Situ-LacasseE,RamdinC,etal.Nationalsurveyofpoint-ofcareultrasoundscholarlytracksinemergencymedicineresidency programs. WestJEmergMed. 2021;22(5):1095–101.

18.HillJ,OgleK,GottliebM,etal.Educator’sblueprint:ahow-toguidefor collectingvalidityevidenceinsurvey-basedresearch. AEMEducTrain. 2022;6(6):e10835.

19.RyanMS,LomisKD,DeiorioNM,etal.Competency-basedmedical educationinanorm-referencedworld:arootcauseanalysisof challengestothecompetency-basedparadigminmedicalschool. AcadMed. 2023;98(11):1251–60.

20.GoldhamerMEJ,Martinez-LageM,Black-SchafferWS,etal. Reimaginingtheclinicalcompetencycommitteetoenhanceeducation andprepareforcompetency-basedtime-variableadvancement. JGenInternMed. 2022;37(9):2280–90.

21.AccreditationCouncilforGraduateMedicalEducation(ACGME). ACGMEprogramrequirementsforgraduatemedicaleducation inemergencymedicalservices.2021.Availableat: AccessedApril29,2023.

22.AccreditationCouncilforGraduateMedicalEducation(ACGME). ACGMEprogramrequirementsforgraduatemedicaleducationincritical caremedicine.2022.Availableat: https://www.acgme.org/specialties/ internal-medicine/program-requirements-and-faqs-and-applications/ AccessedApril29,2023.

EDUCATION SPECIAL ISSUE:BRIEF EDUCATIONAL ADVANCES

NudgeTheory:EffectivenessinIncreasingEmergencyDepartment FacultyCompletionofResidencyAssessments

AmeliaGurley,MD

ColinJenkins,MD

ThienNguyen,MD

AllisonWoodall,MD

JasonAn,MD

RiversideCommunityHospital,DepartmentofEmergencyMedicine,Riverside,California

SectionEditor:JeffreyLove,MD,MHPE

Submissionhistory:SubmittedJune14,2022;RevisionreceivedFebruary8,2023;AcceptedFebruary15,2023 ElectronicallypublishedDecember6,2023

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.57721

BACKGROUND

Assessmentsareacorecomponentofresidencytrainingto assessdevelopmentinthegeneralcompetenciesexpectedof allphysicians.1 Manymethodsareemployedtoevaluate performance,fromcheckliststocomputer-based questionnaires,asnosinglebestpracticeexists.2 Common tomost,however,arebarrierstothecompletionof assessments.3 Forexample,residentsandfacultyoftencitea perceivedlackoftimetoperformassessments,whichmay leadtosuboptimalcomplianceincompletingassessments.3 Somemethodsofassessment,suchasprovidingnarrative feedbacktoresidentsbyfaculty,maybeseenastoo burdensome.3 Theemergencydepartmentrepresentsan especiallychallengingenvironmenttoovercomethese barriersgiventhehighcognitivedemandplacedonfaculty andresidentsbydefault.

Onepossiblestrategytoenhancefacultycompliancein completingassessmentsistoimplementbehavioralnudging intosocialandphysicalenvironments.Borrowedfrom behavioraleconomics,nudgetheoryinvolvesuseof evidence-based “nudges” thatincorporatepositive reinforcementandindirectsuggestionstoinfluencedecisions andbehavior.4 Nudgescanincludeuseofthefollowing: priming(environmentalcuestosubconsciouslydrive behavior);defaultoptions(desirableoptionsarepreselected asthedefaultchoiceandtherebyeasiestforindividualsto take);norm-basednudges(comparingindividualbehavior topeerpractice);commitment(makingapublicpromise tocompleteatask);andsalience(drawingattentiontoa particularoptionthroughcolorsoracompellingstory), amongothers.4 Forinstance,inthesurgicalintensivecare unit,handhygienecompliancewasenhancedwhen individualswereprimedwithacitrus-likefragrancethatwas dispensedintotheenvironment.5 Inanotherexample, medicalstudentassessmentswerecompletedmoreoften whenfacultywerepromptedwithelectronicformsattheend ofshifts,ratherthanrelyingonthemtocompletepaperforms

attheirowndiscretion.6 Inthisstudy,weevaluatedthe effectivenessoftwoprimingnudgesandonenorm-based nudgeinincreasingcomplianceoffacultyincompleting assessmentsofemergencymedicineresidents.

OBJECTIVES

Ourprimaryobjectiveinthestudywastoassessthe effectivenessofnudgeinterventionsinincreasingthenumber ofresidentperformanceassessmentscompletedbyattending physicians.Thiswasassessedbycomparingthenumber ofassessmentscompletedduringtheyearpriorto implementationofthenudgeinterventionswiththeyears followingtheirimplementation.Oursecondaryobjectivewas toidentifywhichparticularmethodwasemployedwiththe greatestfrequency.

CURRICULARDESIGN

Thisprojectqualifiedasaresearchstudyconductedin establishedorcommonlyacceptededucationalsettings.The ResearchOversightCommitteeapprovedtheInstitutional ReviewBoardExemptReviewFormrequestforexemption. ThestudytookplaceatRiversideCommunityHospital,a tertiary-carereferralacademic/communitymedicalcenterin Riverside,California.TheresidencyprogramatRiverside CommunityHospitalisathree-yearemergencymedicine residencyaccreditedbytheAccreditationCouncilfor GraduateMedicalEducation.Eachclasshas13residentsper yearforatotalof39residents.Wehadapproximately28–30 facultyduringthestudy,and28facultyreceivedprior trainingoncompletingend-of-shiftassessments.

Wecollectedpre-interventiondatafromJuly1,2019–June 30,2020withanemaillinksenttofacultyatthebeginningof theacademicyear.Theyweresentperiodicemailreminders tocompletethesurvey.TheinterventionstartedonJuly1, 2020.Thepost-interventiondatawascollectedfromJuly1, 2020–May11,2021.

Table. Numberofassessmentscompletedovertimechartedagainsttimelineofinterventions.

Timeframe

7/1/19–6/30/20 (pre-intervention)

7/1/20–6/20/21 (post-intervention)

7/1/21–6/1/22 (post-intervention)

Numberofresponses3,6634,2434,534

Evaluationspermonth305354453

Threeprimarynudgeswereusedastheinterventionto increasethenumberofend-of-shiftassessments.Weselected thenudgesbasedonpreviousstudies,whichshowedpeople changebehaviorbasedonsocialcomparison.7 Peoplealso tendtochoosethemostvisibleoption.8 The firstnudgewas tocreateahomepageonthefacultyphonewithadirectlink totheend-of-shiftassessmentsurvey.Thesecondnudgewas aquickresponse(QR)codepostedatthefacultywork stationsthroughoutthedepartment:inthemainED;inthe rapidcare(loweracuity)zone;andinthefacultybreakroom. Thethirdnudgewasbasedonasocialproofheuristic.Atthe endofeachblockanemailwassenttoallfacultywiththe totalnumberofassessmentscompletedfortheblock,with comparisonstootherfacultymembers’ completionrateand alinktothesurvey.

Attheendofthestudyperiod,allfacultyreceivedasurvey askingwhichnudgewasusedthemostoften.Facultywere askedtorankeachintervention,fromusedmostoften (weightedscoreof3)toleastoften(weightedscoreof1). Thesurveylinkintheemailreminderwascreatedin Surveymonkey.com(Momentive,SanMateo,CA).

WecreatedtheQRcode flyeroncanva.com (SurryHills,Australia).

Webelievethattheinterventionsinthisstudycanbe replicatedatmanyotherinstitutions.TheQRcodeshouldbe postedinhighlyvisiblelocationsnearthefacultyworkspace intheED.Wediscoveredthatmanyfacultymembers requireddetailedinstructionsonhowtocreateahomepage ontheirmobiledevices.However,thefacultyreportedthat oncethehomepagewassetup,itwastheeasiestwayto completetheassessments.Theend-of-the-blocksummaryof thetotalnumberofassessmentscompletedbyfacultymaybe anadministrativeburdentosomeinstitutions.

IMPACT/EFFECTIVENESS

Asshownin Table1,therewasa15.8%increaseinthe numberofassessmentscompletedintheyearafterthese interventionswereimplemented,withthenumberof completedassessmentsincreasingfrom3,663(305 assessmentspermonth)inthepre-interventionyearto4,243 (354assessmentspermonth)inthe firstpost-intervention year.Thisincreasewassustainedinthefollowingyear,with 4,534assessments(453assessmentspermonth)completedto date.Thistrendsuggeststhatour “nudge” interventionsmay havebeeneffectiveinproducingalong-termchangein facultybehaviorpatterns.

Whensurveyingthe28facultytodeterminewhichnudge wasmosteffective,therewasan85.7%(24)responserate. Oftherespondents,19(79%)indicatedthattheirmost frequentlyusednudgewasthesurveylinksavedontotheir phone,andthattheycompletedover75%oftheirassessments thisway.Thirteenrespondents(54%)reportedthatthenudge basedonsocialheuristics thelinkattheendofthemonthly emails wasthesecondmostfrequentlyused.Onlyone respondentusedtheQRcode flyersmostfrequently,and20 (83%)statedtheyneverusedtheQRcodeatall.

Fromourexperimentaldesign,welearnedthatnudgesused onlinecouldbeeffectiveinincreasingcompletionratesof assessments.Asurprisinglimitationwasthegroupingofdata intocertaintimeframes,whichcouldbedelineatedinfuture iterationstodeterminetheimpactthattimeofyearhason responserates.Wecouldalsocompareefficaciesofdifferent interventions,suchascomparingabaselinerateofusing home-screensurveylinksonlytothisbaselineplusanadded intervention,toassesstheimportanceofeachadded variableandhelpdeterminewhichinterventionstruly providebenefit.

Thisassessmentofourinterventions’ impactislimitedby severalfactors.Asthenumberandmakeupoffaculty changedduringtheintervention,itwasnotpossibleto determinewhetherastatisticallysignificantnumberof facultychangedtheirpracticeasaresultofthisintervention. Theincreaseintheassessmentcompletionratemayalsobe duenotonlytoourinterventionsbutalsotooutsidefactors suchaschanginghospitalpolicies,numberoffaculty,the impactoftheCOVID-19pandemic,overalldepartmental shiftsinattitude,ortheHawthorneeffect,anyofwhichmay haveplayedaroleininfluencingbehavior.Itisalsodifficult todistinguishwhichofthevariousinterventionsactually impactedattendingbehavior,asallwereimplemented simultaneously,andsurveyreplieswereanonymousandmay besubjecttorecallbias.Forexample,itispossiblethatthe presenceofQRcodesatworkstationswasresponsibleforthe largeincreaseinphonehome-screenassessmentcompletion.

AddressforCorrespondence:AllisonWoodall,MD,Riverside CommunityHospital,DepartmentofEmergencyMedicine, 4510BrocktonAvenue,Suite223,RiversideCA92501. Email: Allison.woodall@vituity.com

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived

aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Gurleyetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.GibbsT,BrigdenD,HellenbergD.Assessmentandevaluationin medicaleducation. SAfrFamPract.2006;48(1):5–7.

2.PerkinsSQ,DabajaA,AtiemoH.Bestapproachestoevaluationand feedbackinpost-graduatemedicaleducation. CurrUrolRep 2020;21(10):36.

3.MalikMU,DiazVossVarelaDA,StewartCM,etal.Barriersto implementingtheACGMEoutcomeproject:asystematicreviewof programdirectorsurveys. JGradMedEduc.2012;4(4):425–33.

4.YoongSL,HallA,StaceyF,etal.Nudgestrategiestoimprove healthcareproviders’ implementationofevidence-basedguidelines, policiesandpractices:asystematicreviewoftrialsincludedwithin Cochranesystematicreviews. ImplementSci.2020;15(1):50.

5.KingD,VlaevI,Everett-ThomasR,etal. “Priming” hand hygienecomplianceinclinicalenvironments. HealthPsychol 2016;35(1):96–101.

6.TewsMC,TreatRW,NanesM.IncreasingcompletionrateofanM4 emergencymedicinestudentend-of-shiftevaluationusingamobile electronicplatformandreal-timecompletion. WestJEmergMed 2016;17(4).

7.CheungTTL,KroeseFM,FennisBM,etal.TheHungerGames:using hungertopromotehealthychoicesinself-controlconflicts. Appetite 2017;116:401–9.

8.KroeseFM,MarchioriDR,deRidderDT.Nudginghealthyfoodchoices: a fieldexperimentatthetrainstation. JPublicHealth(Oxf) 2016;38(2):e133–e137.

EDUCATION SPECIAL ISSUE:BRIEF EDUCATIONAL ADVANCES

TheEffectofaSimulation-basedInterventiononEmergency MedicineResidentManagementofEarlyPregnancyLoss

ShawnaD.Bellew,MD,MPH* EricaLowing,MD† LeahHolcomb,MPH,MS‡

*PrismaHealth-Upstate,UniversityofSouthCarolinaSchoolofMedicineGreenville, DepartmentofEmergencyMedicine,Greenville,SouthCarolina

†

PrismaHealth-Upstate,UniversityofSouthCarolinaSchoolofMedicineGreenville, DepartmentofObstetricsandGynecology,Greenville,SouthCarolina ‡ ClemsonUniversity,DepartmentofPublicHealthSciences,Clemson,SouthCarolina

SectionEditor:AbraFant,MD

Submissionhistory:SubmittedNovember22,2023;RevisionreceivedJanuary3,2024;AcceptedJanuary17,2024

ElectronicallypublishedMarch25,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18596

Background: Theevaluationofpatientswith first-trimestervaginalbleedingandconcernforearly pregnancyloss(EPL)frequentlyoccursintheemergencydepartment(ED),accountingforapproximately 1.6%ofallEDvisits.1 Unfortunately,thesepatientsconsistentlyreportnegativeexperienceswithED care.2–8 Inadditiontoenvironmentalconcerns,suchaslongwaittimes,patientsoftendescribenegative interactionswithstaff,includingaperceivedlackofempathy,theuseofinsensitivelanguage,and inadequatecounseling.2,3 ThesepatientsandtheirpartnersoftenviewEPLasatraumaticlossoflifeand commonlyexperienceprolongedgriefreactions,includinganxietyanddepression.9–11 Poorsatisfaction withcarehasbeenassociatedwithworsementalhealthoutcomes.12 Thesecomplaintsrepresentan importantopportunityforimprovementinemergencymedicine(EM)training.13

WhilenopublishedliteraturetodatedescribestheperformanceofEMresidentsinmanaging patientspresentingwithEPL,studiessuggestthatevenobstetricsandgynecology(OB/GYN)residents findtheseinteractionschallenging.14,15 Simulation-anddidactic-basedtraininghasbeenshowntobe beneficialinimprovingOB/GYNresidentEPLcounselingandhasbeenassociatedwithimprovedpatient outcomes.16 Toourknowledge,thishasyettobereplicatedinEMresidencytraining.

Objectives: Weaimedtodevelopandevaluateasimulation-basededucationalinterventiontoimprove EMresidentmanagementofpatientspresentingwithEPL.[WestJEmergMed.2024;25(4.1)36–40.]

CURRICULARDESIGN

Theeducationalinterventionconsistedofthreephases (Figure1)andwasdesignedtooptimizelearningbasedon Kolb’slearningcycle.17,18 Residentswerepresentedwitha challengingscenario(concreteexperience)andthen promptedtoreflectonareasforimprovement(reflective observation).Theythencompletedanasynchronousmodule followedbyaninteractivegroupdiscussion(abstract conceptualization).Thelearningcyclecontinuedthrough activeexperimentationviaarepeatedopportunitytodothe simulation,followedbydebriefing.Thisformofrepetitive simulationhasbeenshowntobemoreeffectivewhen comparedwithnon-repeatedsimulation.19,20

WeimplementedtheinterventioninMay2023and conductedapre/poststudyofitsimmediateimpact,which

wasdeemedexemptbyourinstitutionalreviewboard.The interventiontookplaceatthesimulationcenterofthe affiliatedmedicalschool,duringthetwo-hourperiod typicallyallottedformonthlyresidentsimulation-based education.Postgraduateyear(PGY)1–3EMresidentswere recruitedbasedonaconveniencesampleincludingall residentsattendingsimulationthatday.Theresidents werenotinformedofthetopicoftheintervention priortothedayofthestudy,whichistypicalofour simulationcurriculum.

Sixstandardizedpatients(SP)werehiredtoportray patientsexperiencingEPL.Sixvolunteerfacultyemergency physicians(twomen,fourwomen)observedandevaluated thesimulationsandprovidedinstructionanddebriefing.One facultyOB/GYNphysicianandonefacultyemergency

Figure1. Sequenceofaneducationalinterventionforearlypregnancylosscounseling. EPL,earlypregnancyloss.

physician(bothwomen)co-facilitatedtheguidedgroup discussion-basededucation.

First,residentsparticipatedina10-minutesimulated patientencounterinwhichtheywereinstructedtocarefora SPwhoportrayedapatientwhowaseightweekspregnant andpresentedwithvaginalbleeding.Priortoevaluatingthe patient,eachresidentwasprovidedwithultrasoundresults indicatingthepregnancywasnonviable(presumably obtainedintriage).

Followingtheencounter,residentsindividuallydebriefed withanEMfacultyobserver.Residentsthenhad30minutes tocompleteanasynchronousonlineeducationalmodulethat includedcontentabouttheassessmentofearlypregnancy bleeding;diagnosingandmanagingectopicpregnancy; preventingalloimmunization;andEPLcounseling. Particularattentionwaspaidtooptimizingcaretoaddressa patient’sphysical,emotional,andcognitiveneeds,a frameworkrecommendedbyEmondetal.21 Themodulewas deliveredviaaninteractiveeducationalplatform,Rise360 Articulate(Articulate,NewYork,NY).22 Aftercompleting themodule,residentsparticipatedina15-minuteguided groupdialogwithEMandOB/GYNfaculty,discussingbest practicesandmodelingpracticalcommunicationskills. Facilitatorsgaveexamplesofhowtheywouldaddress patientsinvariousscenariostocommunicateclearlywhile alsousingsensitivelanguage.

Followingthisdiscussion,residentsrepeatedthesame 10-minutesimulatedpatientencounterfollowedby individualdebriefingwithEMfaculty.Theinterventionwas designedtoaccommodateupto24residentswiththe resourcesdescribed.

IMPACTANDEFFECTIVENESS

Tostudytheimmediateimpactoftheintervention, residentperformancewasevaluatedusingfourmeasures: 1)completionofcriticalactionsduringthesimulationviaan 11-itemchecklist;2)self-reportedconfidence;3)a10-item multiple-choicetestoffoundationalEPLknowledge;and4) SPperceptionsofresidentempathyduringthesimulationvia

themodifiedJeffersonScaleofEmpathy(JSE).23,24 Allfour evaluativemeasuresweredeliveredimmediatelyfollowing theinitialsimulatedencounter(Phase1)andafterthe final simulationencounter(Phase3).Inadditiontothese measures,residentswereinvitedtoparticipateinabrieffocus groupinterview,conductedbyanon-facultyfacilitator (woman),aftertheinterventiontodiscusstheirimpressions oftheintervention.

FacultyinEMandOB/GYNdevelopedthetaskchecklist toincludecriticalactionsandevidence-basedbestpractices intreatingpatientsexperiencingEPL.Thislistwasadapted fromachecklistemployedinasimilarstudyandmodifiedto reflectEDcare.25 Residentswereaskedtoratetheir perceivedconfidencelevelfromleast(1)tomost(10) confidentregardingthefollowing:knowledgeaboutthe evaluationandmanagementofpatientswith first-trimester bleeding;abilitytocommunicateinasensitiveandempathic mannerwithpatientswithEPL;andabilitytocounsela patientexperiencingEPLregardingwhattoexpectafter discharge.Theyalsocompleteda10-questionmultiplechoicetest,whichEMandOB/GYNfacultydevelopedto assessbasicobjectiveknowledge.Aftereachsimulated encounter,SPscompletedthemodifiedJSE,avalidatedtool forSPevaluationofclinicianempathyandcommunication. ThemodifiedJSEincludes fivequestionsonaseven-point Likertscalerangingfromstronglydisagree(1)tostrongly agree(7).23,24 Anoutlineofthesimulatedcase,themodule, andtheassessmenttoolsareincludedinthe supplemental material accompanyingtheonlinearticle.

Ofthe16residentswhoparticipated,75%identifiedas men,andtherewasrelativelyequalrepresentationofPGY-1 (31.3%),PGY-2(37.5%),andPGY-3(31.3%)residents. Residentsimprovedfrompre-topost-interventionacrossall fourevaluativemeasures(Table1).Beforetheintervention, fewresidentsprovidedinformationaboutwhattoexpect afterdischarge,includingthepotentialpainlevel,the likelihoodofpassingtissue,returnprecautions,andlongtermemotionalramifications.Aftertheintervention, residentsweresignificantlymorelikelytousesensitive

Table1. Residentassessmentoutcomespre-topost-intervention.

PrePostSignedrank

MeasureMaximumscoreMean(SD)Mean(SD) SP-value

Performancechecklist114.94(1.80)9.50(1.51)67.0 <.001

Self-confidence3020.06(3.38)24.69(3.50)68.0 <.001

Knowledge105.84(1.29)8.00(1.41)45.5 <.001

Empathy3521.25(6.04)28.06(5.47)65.5 <.001

Table1b. Residentchecklistperformance.

PrePost

Checklistitemn(%)n(%) P-value

1.Deliversbadnewsusingsimplelanguageandwithavoidanceof non-preferredterms(fetus,embryo)

10(62.5)16(100)0.03

2.Allowssilenceforthepatienttoabsorbthenews14(87.5)14(87.5)1.00

3.Acknowledgespatient’semotions15(93.8)15(93.8)1.00

4.Dispelsguilt15(93.8)16(100)1.00

5.Counselspatientabouttheamountofexpectedbleeding2(12.5)11(68.8)0.004

6.Counselspatientonexpectedpain1(6.3)10(62.5)0.004

7.Counselspatientonthepossibilityofpassingtissue2(12.5)12(75.0)0.006

8.Counselspatientonreturnforseverebleeding3(18.8)14(87.5)0.003

9.Counselspatientonreturnforfever2(12.5)15(93.8)0.001

10.NormalizesemotionalramificationsofEPL5(31.3)13(81.3)0.008

11.Discussesfollow-upplan10(62.5)16(100)0.030 EPL,earlypregnancyloss.

languageandtoincludeinformationaboutexpected outcomesandreturnprecautions(Table1b).

Theseresultsindicatethatfocusedtrainingresultedin immediateimprovementsinresidentperformance, particularlyregardingcounselingandcommunication. Giventhepositiveresultsofsimilarinterventionsundertaken inotherlearnerpopulations,thisimmediateimpactlikely indicatesimprovedabilitytocareforpatientsinclinical practice.Verhaegheetalpublishedtheimpactofathreehourin-situsimulationtrainingforOB/GYNresidents, whichresultedinlong-termimprovementsinpsychologic outcomesaswellasreducedneedforreturnvisits.16 As comparedtothesepreviousinterventions,ourcurriculum enhancedefficiencybyemployinganonlinetrainingmodule, whichcoveredadditionalfoundationalknowledgeofearly pregnancybleedingcare(includingectopicpregnancyand threatenedEPL).Thisefficiencyisparticularlyimportantin EMgiventhebreadthofrequiredknowledge.

Whiletheeightresidentswhoparticipatedinthefocus groupinterviewgenerallyreportedpositivefeedback,two residentsdidnotethattheywereconfusedbytheorderofthe simulationsuchthattheyhadadiagnosispriortoany interactionwiththepatient.Inthefuture,thismaybe

amelioratedbyprovidingtheresidentswithmorecontextto thecaseorsimplyrevisingthescenariosothattheultrasound reportisreceivedafteraninitialevaluationandrequestfor imaging.Additionally,thetimeallottedforthe asynchronousmodulewas30minutes,butmostresidents completeditinabout20minutes,indicatingthepossibility ofadditionalcontentorexpansionofanotheraspectof theintervention.

LIMITATIONSANDCONCLUSION

Thisstudydescribesresidentperformanceinasimulated patientencounter,andwecannotconcludethatthisreflects actualclinicalcare.Thisstudyonlyassessedtheimpactofthe trainingonlearning(Kirkpatricklevel2)anddidnotattempt toevaluatetheresidents’ ongoingclinicalbehaviororits effectonpatients.26 Thestudywasconductedduringone sessionand,therefore,wecannotinferinformationabout retentionoflearning.Futureworkshouldassesstheeffectof interventionssuchasthisonclinicianbehaviorandresultant patientoutcomes.Facultyevaluatorswerenotblinded duringthesimulatedpatientencounters,whichcouldhave introducedbiasintotheevaluationprovidedviathe checklist.Thisconcernissomewhataddressedbythebinary

natureofthechecklist,inwhicheitherataskwasperformed oritwasnot.Oftheassessmenttools,onlythemodifiedJSE hasbeenexternallyvalidated.CreatingandvalidatingEMspecificmeasurementtoolsforEPLcarewouldensuremore robustdatagoingforward.

“Participantsdisproportionatelyidentifiedasmen(75%), ascomparedtothenationalaverageinemergencyresidencies of62%.27 Giventhesmallpopulationfromwhichthestudy samplewasderived,wedidnotaskparticipantswhetherthey werecis-ortransgendertoavoidlossofanonymity. Similarly,wedidnotaskparticipantsaboutpersonal experienceswithEPL.Futureworkcouldexplorethe relationshipofthesecharacteristicsandexperienceswith clinicalperformance.Despitetheselimitations,theresultsof thisstudyindicateaneedforEPL-specificeducationinEM residencyandthatabrief,simulation-basedinterventionwas effectiveinproducingimmediateimprovements. Consideringtheresultsofsimilarstudiesconductedinother populations,aninterventionsuchasthismayresultin improvedclinicalcareandlong-termpatientoutcomesinthis common,butdevastating,presentation.

5.HoAL,HernandezA,RobbJM,etal.Spontaneousmiscarriage managementexperience:asystematicreview. Cureus. 2022;14(4):e24269.

6.MeluchAL.Waitingtobeseen:provider-patientcommunicationinthe emergencyroomaboutmiscarriage. HealthCommun. 2021;37(11):1452–4.

7.MillerCA,RoeAH,McAllisterA,etal.Patientexperienceswith miscarriagemanagementintheemergencyandambulatorysettings. ObstetGynecol. 2019;134(6):1285–92.

8.PunchesBE,JohnsonKD,AcquavitaSP,etal.Patientperspectivesof pregnancylossintheemergencydepartment. IntEmergNurs. 2019;43:61–6.

9.PrettymanRJ,CordleCJ,CookGD.Athree-monthfollow-upof psychologicalmorbidityafterearlymiscarriage. BrJMedPsychol. 1993;66(Pt4):363–72.

10.ZaccardiR,AbbottJ,Koziol-McLainJ.Lossandgriefreactionsafter spontaneousmiscarriageintheemergencydepartment. AnnEmerg Med. 1993;22(5):799–804.

11.BellhouseC,Temple-SmithM,WatsonS,etal. “Thelosswas traumatic somehealthcareprovidersaddedtothat”:Women’s experiencesofmiscarriage. WomenBirth. 2019;32(2):137–46.

AddressforCorrespondence:ShawnaD.Bellew,MD,MPH,Prisma Health-Upstate,UniversityofSouthCarolinaSchoolofMedicine Greenville,DepartmentofEmergencyMedicine,701GroveRoad, Greenville,SC29605.Email: Shawna.bellew@prismahealth.org

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Bellewetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.WittelsKA,PelletierAJ,BrownDF,etal.UnitedStatesemergency departmentvisitsforvaginalbleedingduringearlypregnancy, 1993–2003. AmJObstetGynecol. 2008;198(5):523.e1–6.

2.Larivière-BastienD,deMontignyF,VerdonC.Women’sexperiencesof miscarriageintheemergencydepartment. JEmergNurs. 2019;45(6):670–6.

3.MacWilliamsK,HughesJ,AstonM,etal.Understandingtheexperience ofmiscarriageintheemergencydepartment. JEmergNurs. 2016;42(6):504–12.

4.DaintyKN,SeatonMB,McLeodS,etal.Reframinghowearlypregnancy lossisviewedintheemergencydepartment. QualHealthRes. 2021;31(6):1119–28.

12.deMontignyF,VerdonC,MeunierS,etal.Women’spersistent depressiveandperinatalgriefsymptomsfollowingamiscarriage:the roleofchildlessnessandsatisfactionwithhealthcareservices. Arch WomensMentHealth. 2017;20(5):655–62.

13.EvansCS.Earlypregnancylossintheemergencydepartment:lessons learnedasaspouse,newfather,andemergencymedicineresident. Ann EmergMed. 2021;77(2):233–6.

14.BrannMandButeJJ.Communicatingtopromoteinformeddecisionsin thecontextofearlypregnancyloss. PatientEducCouns. 2017;100(12):2269–74.

15.ButeJJandBrannM.Tensionsandcontradictionsininterns’ communicationaboutunexpectedpregnancyloss. HealthCommun. 2020;35(5):529–37.

16.VerhaegheC,GicquelM,BouetPE,etal.Positiveimpactofsimulation trainingofresidentsonthepatients’ psychologicalexperience followingpregnancyloss. JGynecolObstetHumReprod. 2020;49(3):101650.

17.KolbDA. ExperientialLearning:ExperienceastheSourceofLearning andDevelopment.2nd ed.Hoboken,NJ:PearsonEducation,2015.

18.StockerM,BurmesterM,AllenM.Optimisationofsimulatedteam trainingthroughtheapplicationoflearningtheories:adebatefora conceptualframework. BMCMedEduc. 2014;14:69.

19.AuerbachM,KesslerD,FoltinJC.Repetitivepediatricsimulation resuscitationtraining. PediatrEmergCare. 2011;27(1):29–31.

20.NgC,PrimianiN,Orchanian-CheffA.Rapidcycledeliberatepracticein healthcaresimulation:ascopingreview. MedSciEduc. 2021;31(6):2105–20.

21.EmondT,deMontignyF,GuillaumieL.Exploringtheneedsof parentswhoexperiencemiscarriageintheemergencydepartment: Aqualitativestudywithparentsandnurses. JClinNurs. 2019;28(9-10):1952–65.

22.FirsttrimesterbleedingandEarlyPregnancyLoss.Availableat: https:// rise.articulate.com/share/CfDnfG-CI6UEdqU1t-FEUDtPnu_UU5mQ AccessedMay9,2023.

23.HojatM. EmpathyinHealthProfessionsEducationandPatientCare 1st ed.NewYorkCity,NY:SpringerInternationalPublishing,2016.

24.MalloryL,FloyedR,DoughtyC,etal.ValidationofamodifiedJefferson ScaleofEmpathyforobserverstoassesstrainees. AcadPediatr. 2021;21(1):165–9.

25.MarkoEK,Buery-JoynerSD,SheridanMJ,etal.Structuredteaching ofearlypregnancylosscounseling. ObstetGynecol. 2015;126(Suppl4):1s–6s.

26.JohnstonS,CoyerFM,NashR.Kirkpatrick’sevaluationofsimulation anddebriefinginhealthcareeducation:asystematicreview. JNursEduc. 2018;57(7):393–8.

27.DeFazioCR,CloudSD,VerniCM,etal.Womeninemergencymedicine residencyprograms:ananalysisofdatafromAccreditationCouncilfor GraduateMedicalEducation-approvedresidencyprograms. AEMEduc Train. 2017;1(3):175–8.

28.LeeL,MaW,DaviesS,etal.Towardoptimalemotionalcareduringthe experienceofmiscarriage:anintegrativereviewoftheperspectivesof women,partners,andhealthcareproviders. JMidwiferyWomens Health. 2023;68(1):52–61.

EDUCATION SPECIAL ISSUE:ORIGINAL RESEARCH

IntegrationofGeriatricEducationWithintheAmerican BoardofEmergencyMedicineModel

LaurenT.Southerland,MD,MPH*

LaurenR.Willoughby,MD*

JasonLyou,MD*

RebeccaR.Goett,MD†

DanielW.Markwalter,MD‡§

DianeL.Gorgas,MD*

*TheOhioStateUniversityWexnerMedicalCenter,DepartmentofEmergency Medicine,Columbus,Ohio

† RutgersNewJerseyMedicalSchool,DepartmentofEmergencyMedicine, Newark,NewJersey

‡ UniversityofNorthCarolinaSchoolofMedicine,DepartmentofEmergency Medicine,ChapelHill,NorthCarolina

§ UniversityofNorthCarolinaSchoolofMedicine,UNCPalliativeCareProgram, ChapelHill,NorthCarolina

SectionEditors:ChrisMerritt,MD,andJeffreyLove,MD

Submissionhistory:SubmittedMay1,2023;RevisionreceivedAugust30,2023;AcceptedNovember3,2023

ElectronicallypublishedDecember22,2023

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.60842

Background: Emergencymedicine(EM)residenttrainingisguidedbytheAmericanBoardof EmergencyMedicineModeloftheClinicalPracticeofEmergencyMedicine(EMModel)andtheEM Milestonesasdevelopedbasedontheknowledge,skills,andabilities(KSA)list.Theseareconsensus documentsdevelopedbyacollaborativeworkinggroupofsevennationalEMorganizations.External expertsingeriatricEMalsodevelopedcompetencyrecommendationsforEMresidencyeducationin geriatrics,butthesearenotbeingtaughtinmanyresidencyprograms.Ourobjectivewastoevaluatehow thegeriatricEMcompetenciesintegrate/overlapwiththeEMModelandKSAstohelpresidency programsincludethemintheireducationalcurricula.

Methods: Trainedemergencyphysiciansindependentlymappedthegeriatricresidentcompetencies ontothe2019EMModelitemsandthe2021KSAsusingExcelspreadsheets.Discrepancieswere resolvedbyanindependentreviewerwithexperiencewiththeEMModeldevelopmentandresident education,andthe finalmappingwasreviewedbyallteammembers.

Results: TheEMModelincluded77%(20/26)ofthegeriatriccompetencies.TheKSAsincludedmostof thegeriatriccompetencies(81%,21/26).Allbutoneofthegeriatriccompetenciesmappedontoeitherthe EMModelortheKSAs.WithintheKSAs,mostofthegeriatriccompetenciesmappedontonecessary levelskills(rankedB,C,D,orE)withonly five(8%)alsomappingontoadvancedskills(rankedA).

Conclusion: AllbutoneofthegeriatricEMcompetenciesmappedtothecurrentEMModelandKSAs. ThegeriatriccompetenciescorrespondtoknowledgeatalllevelsoftrainingwithintheKSAs,from beginnertoexpertinEM.EducatorsinEMcanusethismappingtointegratethegeriatriccompetencies withintheircurriculums.[WestJEmergMed.2024;25(4.1)41–50.]

INTRODUCTION

Emergencymedicine(EM)residentshave3–4yearsof trainingtolearnanextensivearrayofskills.Thisincludesthe skillsneededtocareforolderpatients,whomakeup16–20% oftheirpatients.1,2 TheAmericanBoardofEmergency Medicine(ABEM)codifiestheskillsneededforcompetency

inEMintheModeloftheClinicalPracticeofEmergency Medicine(EMModel)andthe2021knowledge,skills,and abilities(KSA).3,4 TheEMModellistsclinicalpresentations anddiseasetypesandtheKSAsarealistofskillsandabilities integraltoEMpractice.Manyresidencyprogramsbasetheir curriculumsonthesedocuments.However,itisunclearhow

besttointegrategeriatricteachingwithinthese complexcurricula.

In2010Hoganetalpublishedeightdomainswith26 competenciesofgeriatriceducationderivedfromanexpert consensuspanelthatareconsideredessentiallearningduring EMresidencyforthecareofolderadultsintheemergency department(ED).5 Thesecompetenciesarealsousedfor categorizinggeriatriccontinuingeducationforgeriatricED accreditationandhavebeenpivotaltothedevelopmentof geriatricEMasasubspecialty.6,7 Despitethisguidance, geriatricconceptsarestillonlyminimallyintegrated intoresidenteducation.8 Withoutdedicatedtraining, residentknowledgeofgeriatriccompetenciesispoor.9–11 Butthereiscurrentlynoguidanceonhowtointegrate thegeriatriccompetencieswithinanEM residencycurriculum.

OurcurriculumisbasedontheEMModelandKSAs.Our goalwastodeterminewhetherthegeriatriccompetenciescan becoveredbyanEMModel-basedcurriculum.

METHODS

Thisprojectisnothumansubjectsresearchanddidnot requireinstitutionalboardreview.Thestudywasa descriptivecomparisonofthe2019EMModelandthe2021 KSAstothe2010geriatriccompetenciesusingaconsensusbasedprocess.TheKSAsincludebothadescriptionanda level.Theyaredividedintooverarchingcategories (eg,diagnosis,pharmacotherapy,reassessment)whichare thendividedintosteps.4 Eachstepisgivenahierarchyin training(withAthehighestandEthelowest).LevelAis foradvancedknowledgeorskills.LevelBistheminimal competencylevelforpassingEMresidency.LevelsC,D, andEareskillstepstoreachlevelB.

Inthe firstphaseofconsensusmapping,tworesidents (asecond-yearEMresidentandafourth-yearEM/internal medicineresident)andageriatricfellowship-trainedEM attendingindependentlymappedgeriatriccompetencies usingExcel(MicrosoftCorporation,RedmondWA).They wereinstructedto firstusethesearchbuttontolookforexact languageandthengoitembyitemthroughtheEMModel andtheKSAstomapsimilarlanguageorconcepts.For example,theconceptofdeliriumcouldbedescribedas alteredmentalstatusorencephalopathy.Aclearassociation wasdefinedbytheteamas1)akeywordmatchor 2)consensusthatitwaslikelythatanemergencyphysician lecturing/teachingontheEMModelcontentitemwould,in normalteachingpractices,teachthegeriatriccompetency.If thiswasnotthecase,butthegeriatriccompetencycouldbe incorporatedunderthistopicbysomeone intentionally teachingthecompetencies,thiswaslistedasasuggestedarea forincorporation.Reviewerswereinstructedtobegenerous withmappingduringthis firstround.

Ifallthreeor2/3agreed,thiswasconsideredinitial consensus.Anyremainingdiscrepancieswerethen

independentlyreviewedbyanotheremergencyphysician withexpertiseinresidenteducation(formerEMprogram directorandcurrentABEMexecutivecommitteemember). Thefullgroupmetandreviewedthe finaldiscrepanciesuntil consensuswasreached.Theconsensustableswerethen reviewedindependentlybytwomoreemergencyphysicians atexternalresidencyprogramsforcontentvalidity.Asimilar processwasusedformappingKSAs.Reviewerswereblinded totheKSAlevel(A-Edesignation).

RESULTS

Incorporationintothe2019EMModel

TheEMModelhas963items.Onthe firstround,126 items(13%ofcontent)wereidentifiedaspotentialmatches, includingallof 17.1DrugandChemicalClasses .Round1 consensuswas96.2%(927items). Table1 liststhe20geriatric competencies(77%)includedinthe2019EMModel.Key wordmatchesincludedcompetency#6: “Demonstrate abilitytorecognizepatte rnsof(physical/sexual, psychological,neglect/abandonment)thatareconsistent withelderabuse[,] ” whichmapsto “ModelContent 14.6.1.3PatternsofViolence/ Abuse/Neglect:Intrapersonal Violence:Elder . ” Otherswerematchedbyconcept,suchas competency#11: “Assessandcorrect(ifappropriate) causativefactorsinagitatedelderssuchasuntreatedpain, hypoxia,hypoglycemia,useofirritatingtethers(de fi nedas monitorleads,bloodpressurecuff,pulseoximetry, intravenousaccess,andFoleycatheter),environmental factors(light,temperature),anddisorientation[,] ” which couldbeincorporatedintoteachingon 12.14Nervous SystemDisorders:Delirium . Initialdisagreementsincludedwhethersignsand symptomsweremeanttobeusedtoformulateadifferential diagnosisforthatsymptomortodescribemanagementofthe symptoms.TherewasalsoaquestionastowhetherG11, whichdiscusses “irritatingtethers” asacauseofdelirium, shouldbemappedtoallproceduressuchas 19.4.1.4. Nasogastrictube .Thegroupdecidedthatthiswouldbe betterencompassedundertheEMModelitemfordelirium. Table2 liststhesixgeriatriccompetencieswithoutaclear fit withintheEMModelandsuggestionsfromtheteamon wheretoincludethem.

Incorporationintothe2021Knowledge,Skills,andAbilities

Theinitialindependentmappingresultedinconsensuson 84%oftheitems(179/214).Ofthegeriatriccompetencies, 216(81%)mappedontoKSAs(Table3).Themostcommon categorieswereCommunication&InterpersonalSkills (CS0),Pharmacotherapy(PT0),andTransitionsofCare (TC0).Ofthe fivecompetenciesthatdidnotmapdirectly ontotheKSAs,allhadmappingitemsintheEMModel exceptone.Theonecompetencythatdidnotmapdirectlyto anyEMModelorKSAwasEffectsofComorbidConditions (G24): “Assessanddocumentthepresenceofcomorbid

Table1. ThegeriatricteachingcompetenciesmappedontotheEmergencyMedicineModelofCare.

Geriatric competencyDescriptionEMmodelitem

G1Generateadifferentialdiagnosisrecognizingthatsignsandsymptoms suchaspainandfevermaybeabsentorlessprominentinelderswith acutecoronarysyndromes,acuteabdomens,orinfectiousprocesses.

G2Generateanage-specificdifferentialdiagnosisforelderpatients presentingtotheEDwithgeneralweakness,dizziness,falls,oraltered mentalstatus.

G3Documentconsiderationofadversereactionstomedications,including drug-druganddrug-diseaseinteractions,aspartoftheinitialdifferential diagnosis.

G4Inpatientswhohavefallen,evaluateforprecipitatingcausesoffallssuch asmedications,alcoholuse/abuse,gaitorbalanceinstability,medical illness,and/ordeteriorationofmedicalconditions.

G5Assessforgaitinstabilityinallambulatoryfallers;ifpresent,ensure appropriatedispositionandfollow-upincludingattempttoreachprimary carephysician.

G6Demonstrateabilitytorecognizepatternsoftrauma(physical/sexual, psychological,neglect/abandonment)thatareconsistentwithelder abuse.Managetheabusedpatientinaccordancewiththerulesofthe stateandinstitution.

G7Instituteappropriateearlymonitoringandtestingwiththeunderstanding thateldersmaypresentwithmutedsignsandsymptoms(eg,absentpain andneurologicchanges)andareatriskforoccultshock.

G8Assesswhetheranelderisabletogiveanaccuratehistory,participatein determiningtheplanofcare,andunderstanddischargeinstructions.

1.1Abnormalvitalsigns

1.2Pain

1.3.1General-alteredmentalstatus

1.3.4General-ataxia

1.3.19General-fatigue/malaise

1.3.28General-lightheadedness/dizziness

1.3.53General-weakness

18.3.2Multisystemtrauma-falls

1.3.55General-toxidromes

17.1Drugandchemicalclasses: entire section

1.3.4General-ataxia

1.3.53General-weakness

18.3.2Multisystemtrauma-falls

18.3.2Multisystemtrauma-falls

14.6.1.3Patternsofviolence/abuse/ neglect-elder

1.3.41General-shock

12.8.1Otherconditionsofthebraindementia

14.5.2Organicpsychoses-dementia

20.4.5.4Regulatory/legal-consent,capacity andrefusalofcare

G9Assessanddocumentcurrentmentalstatusandanychangefrom baselineineveryelder,withspecialattentiontodeterminingwhether deliriumexistsorhasbeensuperimposedondementia.

G10Emergentlyevaluateandformulateanage-specificdifferentialdiagnosis forelderswithnewcognitiveorbehavioralimpairment,includingselfneglect;initiateadiagnosticworkuptodeterminetheetiology;andinitiate treatment.

G11Assessandcorrect(ifappropriate)causativefactorsinagitatedelders suchasuntreatedpain,hypoxia,hypoglycemia,useofirritatingtethers (definedasmonitorleads,bloodpressurecuff,pulseoximetry, intravenousaccess,andFoleycatheter),environmentalfactors(light, temperature),anddisorientation.

1.3.1General-alteredmentalstatus

12.8.1Otherconditionsofthebraindementia

12.14.1Delirium-exciteddelirium syndrome

14.5.2Organicpsychoses-dementia

1.3.18General-failuretothrive

12.14.1Delirium-exciteddelirium syndrome (Continuedonnextpage)

Table1. Continued.

Geriatric competencyDescriptionEMmodelitem

G12Recommendtherapybasedontheactualbenefittoriskratio,including butnotlimitedtoacutemyocardialinfarction,stroke,andsepsis,sothat agealonedoesnotexcludeeldersfromanytherapy.

G14Prescribeappropriatedrugsanddosagesconsideringthecurrent medication,acuteandchronicdiagnoses,functionalstatus,and knowledgeofage-relatedphysiologicchanges(renalfunction,central nervoussystemsensitivity).

G15Searchforinteractionsanddocumentreasonsforusewhenprescribing drugsthatpresenthighriskeitheraloneorindrug-drugordrug-disease interactions(eg,benzodiazepines,digoxin,insulin,NSAIDs,opioids,and warfarin).

G16Explainallnewlyprescribeddrugstoeldersandcaregiversatdischarge, assuringthattheyunderstandhowandwhythedrugshouldbetaken,the possiblesideeffects,andhowandwhenthedrugshouldbestopped.

G19Withrecognitionofuniquevulnerabilitiesinelders,assessanddocument suitabilityfordischargeconsideringtheEDdiagnosis,includingcognitive function,theabilityinambulatorypatientstoambulatesafely,availability ofappropriatenutrition/socialsupport,andtheavailabilityofaccessto appropriatefollow-uptherapies.

G20Selectanddocumenttherationaleforthemostappropriateavailable disposition(home,extendedcarefacility,hospital)withtheleastriskof themanycomplicationscommonlyoccurringineldersduringinpatient hospitalizations.

G21Rapidlyestablishanddocumentanelder’sgoalsofcareforthosewitha seriousorlife-threateningconditionandmanageaccordingly.

G22AssessandprovideEDmanagementforpainandkeynon-pain symptomsbasedonthepatient’sgoalsofcare.

G23Knowhowtoaccesshospicecareandhowtomanageeldersinhospice carewhileintheED.

NSAID,non-steroidanti-in flammatorydrug; ED,emergencydepartment.

conditions(eg,pressureulcers, cognitivestatus,fallsinthe pastyear,abilitytowalkandtransfer,renalfunction,and socialsupport)andincludetheminyourmedicaldecisionmakingandplanofcare ” Incorporatingthepotential consequencesofcomorbidconditionsisincludedinKSA

PR2: “Performtheindicatedprocedureonanuncooperative patient,patientattheextremeso fage(pediatric,geriatric), multipleco-morbidities,poorlyde fi nedanatomy, hemodynamicallyunstable,highriskforpainorprocedural complications,sedationrequi red,oremergentindicationto performprocedure,andrecognizetheoutcomeand/or

12.11.1.1Stroke-intracerebralhemorrhagic stroke

12.11.1.2Stroke-subarachnoid hemorrhagicstroke

12.11.2.1Stroke-embolicischemicstroke

12.11.2.2Stroke-thromboticischemic stroke

20.4.4.1Healthcarecoordination-advance directives

17.1Drugandchemicalclasses: entire section

17.1Drugandchemicalclasses: entire section

20.1.1.3Interpersonalskills-patientand familyeducation

20.3.2.6Ethicalprinciples-careof vulnerablepopulations

20.4.4.3.1Healthcarecoordinationactivitiesofdailyliving/functional assessment

20.4.4.2.3Healthcarecoordination-hospice referral

20.4.4.1Healthcarecoordination-advance directives

20.4.4.2.1Healthcarecoordination-patient identificationforpalliativecare

19.3.3Anesthesiaandacutepain management-analgesia

20.4.4.2.3Healthcarecoordination-hospice referral

complicationsresultingfromtheprocedure ” (KSALevelB). WhilethegeriatricscompetencyaddressesmedicaldecisionmakingandtheKSAaddressdifficultprocedures,thereis someoverlapinthetrainingrequired.

Ofthe63matcheswithintheKSA, five(8%)mappedonto advancedlevelAskills(DX7,Identifyobscure,occult,or rarepatientconditions ;and TI6,Developprotocolstoavoid potentialcomplicationsofinterventions ).Abouthalf(31, 49%)mappedontorequiredcompetencyskills(LevelB),and theremaining27(43%)weredevelopingskills(LevelC,Dor E,27,43%)(Table3).

Table2. Suggestionsforteachingthegeriatriccompetenciesthatdonot fitclearlywithintheEmergencyMedicineModel.

Geriatric competencyDescription

G13Identifyandimplementmeasuresthatprotecteldersfrom developingiatrogeniccomplicationscommontotheED includinginvasivebladdercatheterization,spinal immobilization,andcentrallineplacement.

G17Documenthistoryobtainedfromskillednursingor extendedcarefacilitiesoftheacuteeventsnecessitating EDtransferincludinggoalsofvisit,medicalhistory, medications,allergies,cognitiveandfunctionalstatus, advancecareplan,andresponsiblePCP.

G18Provideskillednursingorextendedcarefacilitiesand/or PCPwithEDvisitsummaryandplanofcare,including follow-upwhenappropriate.

G24Assessanddocumentthepresenceofcomorbid conditions(eg,pressureulcers,cognitivestatus,fallsin thepastyear,abilitytowalkandtransfer,renalfunction, andsocialsupport)andincludetheminyourmedical decision-makingandplanofcare.

G25Developplansofcarethatanticipateandmonitorfor predictablecomplicationsinthepatient’scondition (eg,gastrointestinalbleedcausingischemia).

Suggestionsforteachinggeriatriccompetencies withoutaclearassociationwithEMModelitems

Couldbediscussedunder ProcedureDomain or PracticebasedLearningandImprovement:Patientsafetyand Medicalerrors

Notransitionsofcare,nursingfacility,ordispositionareas. Couldbetaughtunder InterpersonalandCommunication Skills:Intra-departmentalrelations,teamwork,and collaborationskills

Notransitionsofcare,nursingfacility,ordispositionareas. Couldbetaughtunder InterpersonalandCommunication Skills:Intra-departmentalrelations,teamwork,and collaborationskills

Whileindividualelementslistedareinthemodel (eg,ulcerativelesions:decubitus),theconceptof comorbidityinolderadultsisdistinctfromdisease-oriented items.

Couldbediscussedunder Practice-basedLearningand Improvement:PatientsafetyandMedicalErrors

G26Communicatewithpatientswithhearing/sightimpairmentCouldbediscussedunder Interpersonaland CommunicationSkills:CulturalCompetency.

ED,emergencydepartment; PCP,primarycarephysician.

DISCUSSION

ThegeriatriccompetenciesforEMresidencytraining integratewellwithintheEMModelandKSAs,with onlyonecompetencynothavingadirectmatch. Demonstratingthisoverlapbetweenthesuggested subspecialtycurriculumandtheEMmodelcanhelpEM educatorsensurethatthegeriatriccompetenciesare incorporatedintotheircurricula.Thismappingcouldalso guidethedevelopmentofboardexamquestions,lectures, orsimulationcases.

TheEMModelisverybrief,whichcanmakedirecting educationdifficult.Forinstance,trainingontheEMModel item 18.3Multi-systemTrauma:Falls isexpoundeduponin geriatriccompetency#4: “Inpatientswhohavefallen, evaluateforprecipitatingcausesoffallssuchas medications,alcoholuse/abuse,gaitorbalanceinstability, medicalillness,and/ordeteri orationofmedicalconditions . ” Oranotherexample,KSA DX1 “ Synthesizechief complaint,history,physicalexamination,andavailable medicalinformationtodevelopadifferentialdiagnosis ” can includeadiscussionofgeriatriccompetency#3 “Document considerationofadversereacti onstomedications,including drug-druganddrug-diseaseinteractions,aspartofthe initialdifferentialdiagnosis. ” Theybothdescribetheinitial generationofadifferentialdiagnosis,butthegeriatric

competencyaddspharmacologyinteractionsandadverse reactionstobeconsideredinthedifferential.

Asecond findingofthisstudywasthatthegeriatric competenciesalignwithelementsrequiredforminimalKSA competency.Thisimpliesthatdifferentaspectsofgeriatric carecan(andweargue,should)betaughtthroughouta resident’straining.Italsosuggeststhatthegeriatric competencieswerewelldevelopedfortheresidencylevelof trainingandshouldnotbeconsidered “tooadvanced” or “subspecialtytraining.” Whilepriorresearchhasevaluated separategeriatric-specificcurricula,9–11 ourworkshowsthat geriatriccompetenciescanbeintegratedthroughouta curriculumbasedontheEMModelandKSAs.Asof2021, therewereonly25geriatricfellowship-trainedemergency physicians,whichisnotenoughforeveryresidency program. 12 Programswithoutfacultywhohavenointerestor trainingingeriatricscouldalsouseexternaltraining resourcessuchastheonlinelearningmodulesat https://geri-em.com/andattheGeriatricEmergency DepartmentCollaborative(https://gedcollaborative.com/ online-learning/).

LIMITATIONS

Onelimitationofthisprojectwastheconsensusdefinitions used.Wewereunableto findanyexistingmethodstohelpus

Table3. Thegeriatriccompetenciesweremappedontothe2021ABEMknowledge,skills,andabilitieslist.

Geriatric competencyDescription KSA codeDescriptionLevel

G1Generateadifferentialdiagnosisrecognizingthat signsandsymptomssuchaspainandfevermay beabsentorlessprominentinelderswithacute coronarysyndromes,acuteabdomens,or infectiousprocesses.

G2Generateanage-specificdifferentialdiagnosisfor elderpatientspresentingtotheEDwithgeneral weakness,dizziness,falls,oralteredmentalstatus.

DX1Synthesizechiefcomplaint,history,physical examination,andavailablemedicalinformationto developadifferentialdiagnosis

DX7Identifyobscure,occult,orrarepatientconditionsA

DX8Constructalistofpotentialdiagnosesbasedonthe chiefcomplaint D

DX1Synthesizechiefcomplaint,history,physical examination,andavailablemedicalinformationto developadifferentialdiagnosis

DX7Identifyobscure,occult,orrarepatientconditionsA

DX8Constructalistofpotentialdiagnosesbasedonthe chiefcomplaint D

G3Documentconsiderationofadversereactionsto medications,includingdrug-druganddrug-disease interactions,aspartoftheinitialdifferential diagnosis.

G6Demonstrateabilitytorecognizepatternsoftrauma (physical/sexual,psychological,neglect/ abandonment)thatareconsistentwithelderabuse. Managetheabusedpatientinaccordancewiththe rulesofthestateandinstitution.

DX1Synthesizechiefcomplaint,history,physical examination,andavailablemedicalinformationto developadifferentialdiagnosis

PT5Recognize,monitor,andtreatadverseeffectsof pharmacotherapy

LI8Adheretoprocessesandprocedurestoensurethat appropriateagenciesarenotifiedinsituationsthat couldposeathreattoindividualorpublichealth (eg,violenceandcommunicabledisease)in accordancewithlocallegalstandards

LI10Adheretolegalandethicalstandardstoassess andtreatpatientspresentingtotheED

LI11Advocateforpatientsvulnerabletoviolenceor abuseinaccordancewithlegalandethical standards

LI13Identifypatientsvulnerabletoabuseorand/or neglect

G7Instituteappropriateearlymonitoringandtesting withtheunderstandingthateldersmaypresent withmutedsignsandsymptoms(eg,absentpain andneurologicchanges)andareatriskforoccult shock.

G8Assesswhetheranelderisabletogivean accuratehistory,participateindeterminingtheplan ofcare,andunderstanddischargeinstructions.

DX7Identifyobscure,occult,orrarepatientconditionsA

DS1PrioritizeessentialtestingD

DS2Determinenecessityandurgencyofdiagnostic studies E

CS5Communicateinformationtopatientsandfamilies usingverbal,nonverbal,written,andtechnological skills,andconfirmunderstanding

CS15Solicitpatientparticipationinmedicaldecisionmakingbydiscussing,risks,benefits,and alternativestocareprovided

HP2Prioritizeessentialcomponentsofahistoryand physicalexaminationgivenlimited(eg,altered mentalstatus)ordynamic(eg,acutecoronary syndrome)situations

TC13Ensurepatienthasresourcesandtoolstocomply withdischargeplan,whichmayincludemodifying theplanorinvolvingadditionalresources(ie,PCP, socialwork, financialaid)tooptimizecompliance B (Continuedonnextpage)

Continued.

G9Assessanddocumentcurrentmentalstatusand anychangefrombaselineineveryelder,with specialattentiontodeterminingwhetherdelirium existsorhasbeensuperimposedondementia.

G10Emergentlyevaluateandformulateanage-specific differentialdiagnosisforelderswithnewcognitive orbehavioralimpairment,includingself-neglect; initiateadiagnosticworkuptodeterminethe etiology;andinitiatetreatment.

G12Recommendtherapybasedontheactualbenefitto riskratio,includingbutnotlimitedtoacute myocardialinfarction,stroke,andsepsis,sothat agealonedoesnotexcludeeldersfromany therapy.

TC17Explainclearlyandensurepatientunderstandingof diagnosis,dischargeinstructions,andthe importanceoffollow-upandcompliancewith treatments.

HP6Identifyrelevanthistoricalandphysical findingsto guidediagnosisandmanagementofapatient’s presentingcomplaintinthecontextoftheirbaseline condition

DX1Synthesizechiefcomplaint,history,physical examination,andavailablemedicalinformationto developadifferentialdiagnosis

HP2Prioritizeessentialcomponentsofahistoryand physicalexaminationgivenlimited(eg,altered mentalstatus)ordynamic(eg,acutecoronary syndrome)situations

CS14Communicaterisks,benefits,andalternativesto diagnosticandtherapeuticprocedures/interventions topatientsand/orappropriatesurrogates,and obtainconsentwhenindicated

DS4Reviewrisks,benefits,contraindications,and alternativestoadiagnosticstudyorprocedure

TI8Assessindications,risks,benefits,andalternatives forthetherapeuticintervention.

G13Identifyandimplementmeasuresthatprotect eldersfromdevelopingiatrogeniccomplications commontotheEDincludinginvasivebladder catheterization,spinalimmobilization,andcentral lineplacement.

DS4Reviewrisks,benefits,contraindications,and alternativestoadiagnosticstudyorprocedure

PR2Performtheindicatedprocedureonan uncooperativepatient,patientattheextremesof age(pediatric,geriatric),multiplecomorbidities, poorlydefinedanatomy,hemodynamically unstable,highriskforpainorprocedural complications,sedationrequired,oremergent indicationtoperformprocedure,andrecognizethe outcomeand/orcomplicationsresultingfromthe procedure

PR7Recognizetheindications,contraindications, alternatives,andpotentialcomplicationsfora procedure

TI8Assessindications,risks,benefits,andalternatives forthetherapeuticintervention.

G14Prescribeappropriatedrugsanddosages consideringthecurrentmedication,acuteand chronicdiagnoses,functionalstatus,and knowledgeofage-relatedphysiologicchanges (renalfunction,centralnervoussystemsensitivity).

PT2Identifyrelativeandabsolutecontraindicationsto specificpharmacotherapy

PT5Recognize,monitor,andtreatadverseeffectsof pharmacotherapy

PT6Selectandprescribeappropriatepharmaceutical agentsbasedonintendede ectandpatient allergies

(Continuedonnextpage)

Table3. Continued. Geriatric

G15Searchforinteractionsanddocumentreasonsfor usewhenprescribingdrugsthatpresenthighrisk eitheraloneorindrug-drugordrug-disease interactions(eg,benzodiazepines,digoxin,insulin, NSAIDs,opioids,andwarfarin).

PT9Select,prescribe,andbeawareofadverseeffects ofappropriatepharmaceuticalagentsbasedupon relevantconsiderationssuchasintendedeffect, financialconsiderations,possibleadverseeffects, patientpreferences,institutionalpolicies,and clinicalguidelines.

PT2Identifyrelativeandabsolutecontraindicationsto specificpharmacotherapy

PT5Recognize,monitor,andtreatadverseeffectsof pharmacotherapy

PT9Select,prescribe,andbeawareofadverseeffects ofappropriatepharmaceuticalagentsbasedupon relevantconsiderationssuchasintendedeffect, financialconsiderations,possibleadverseeffects, patientpreferences,institutionalpolicies,and clinicalguidelines.

PT10Conductfocusedmedicationreviewandidentify agentsincludingnutraceuticalsandcomplementary medicinesthatmaybecausinganadverseeffect

TI6Developprotocolstoavoidpotentialcomplications ofinterventions

TI8Assessindications,risks,benefits,andalternatives forthetherapeuticintervention.

G16Explainallnewlyprescribeddrugstoeldersand caregiversatdischarge,assuringthatthey understandhowandwhythedrugshouldbetaken, thepossiblesideeffects,andhowandwhenthe drugshouldbestopped.

G17Documenthistoryobtainedfromskillednursingor extendedcarefacilitiesoftheacuteevents necessitatingEDtransferincludinggoalsofvisit, medicalhistory,medications,allergies,cognitive andfunctionalstatus,advancecareplan,and responsiblePCP.

G18Provideskillednursingorextendedcarefacilities and/orPCPwithEDvisitsummaryandplanof care,includingfollow-upwhenappropriate.

CS5Communicateinformationtopatientsandfamilies usingverbal,nonverbal,written,andtechnological skills,andconfirmunderstanding

TC17Explainclearlyandensurepatientunderstandingof diagnosis,dischargeinstructions,andthe importanceoffollow-upandcompliancewith treatments.

CS6Elicitinformationfrompatients,families,andother healthcaremembersusingverbal,nonverbal, written,andtechnologicalskills

CS10Communicatepertinentinformationtohealthcare colleaguesineffectiveandsafetransitionsofcare

CS10Communicatepertinentinformationtohealthcare colleaguesineffectiveandsafetransitionsofcare

TC14Identifypatientswhowillrequiretransfertoa facilitythatprovidesahigherlevelofcareand coordinatethistransitionofcarebyensuring communicationwiththereceivingprovider, completionoftransferdocumentation,educationof thepatientorsurrogatethereasonsfortransfer, consentfortransfer,andarrangementof appropriatetransportation.

TC16Useappropriatetoolsfortransitionsofcare, dischargeinstructions,prescriptions,follow-up instructions,andanypendingdiagnosticstudiesto promoteeffectivecareanddecreaseerror

(Continuedonnextpage)

Table3. Continued.

Geriatric competencyDescription

G19Withrecognitionofuniquevulnerabilitiesinelders, assessanddocumentsuitabilityfordischarge consideringtheEDdiagnosis,includingcognitive function,theabilityinambulatorypatientsto ambulatesafely,availabilityofappropriatenutrition/ socialsupport,andtheavailabilityofaccessto appropriatefollow-uptherapies.

G20Selectanddocumenttherationaleforthemost appropriateavailabledisposition(home,extended carefacility,hospital)withtheleastriskofthemany complicationscommonlyoccurringineldersduring inpatienthospitalizations.

OB9Reassess,manage,andprognosticatethecourse ofpatientsinEDobservationstatustodetermine appropriatedisposition.

TC13Ensurepatienthasresourcesandtoolstocomply withdischargeplan,whichmayincludemodifying theplanorinvolvingadditionalresources(ie,PCP, socialwork, financialaid)tooptimizecompliance

TC18CorrectlydeterminetheappropriatedispositionC

CS10Communicatepertinentinformationtohealthcare colleaguesineffectiveandsafetransitionsofcare

OB1IdentifypatientsappropriateformanagementinED observationstatus

OB9Reassess,manage,andprognosticatethecourse ofpatientsinEDobservationstatustodetermine appropriatedisposition.

TC12Assignadmittedpatientstoanappropriatelevelof care

TC14Identifypatientswhowillrequiretransfertoa facilitythatprovidesahigherlevelofcareand coordinatethistransitionofcarebyensuring communicationwiththereceivingclinician, completionoftransferdocumentation,educationof thepatientorsurrogatethereasonsfortransfer, consentfortransfer,andarrangementof appropriatetransportation.

TC18CorrectlydeterminetheappropriatedispositionC

G21Rapidlyestablishanddocumentanelder’sgoalsof careforthosewithaseriousorlife-threatening conditionandmanageaccordingly.

G22AssessandprovideEDmanagementforpainand keynon-painsymptomsbasedonthepatient’s goalsofcare.

G25Developplansofcarethatanticipateandmonitor forpredictablecomplicationsinthepatient’s condition(eg,gastrointestinalbleedcausing ischemia).

G26Communicatewithpatientswithhearing/sight impairment

CS3Elicitpatients’ reasonsforseekinghealthcareand theirexpectationsfromtheEDvisit

ES15Elicitthepatient’sgoalsofcarepriortoinitiating emergencystabilization,includingevaluatingthe validityofadvanceddirectives

DS4Reviewrisks,benefits,contraindications,and alternativestoadiagnosticstudyorprocedure

TI6Developprotocolstoavoidpotentialcomplications ofinterventions

CS5Communicateinformationtopatientsandfamilies usingverbal,nonverbal,written,andtechnological skills,andconfirmunderstanding

CS7Considertheexpectationsofthosewhoprovideor receivecareintheEDandusecommunication methodsthatminimizethepotentialforstress, conflict,andmiscommunication

CS18Demonstrateinterpersonalandcommunication skillsincludingadjustmentofinteractionsto accountforfactorssuchasculture,gender,age, language,disability,thatresultintheeffective exchangeofinformationandcollaborationwith patients,families,andallotherstakeholders.

,knowledge,skills,abilities; ED,emergencydepartment; NSAID,non-steroidalanti-inflammatorydrug; PCP,primarycarephysician.

definecurricularoverlap.Whilewewerestrengthenedby havingrepresentationfrommultipleEMresidencyprograms, othereducationexpertsmayhaveadifferentinterpretationof thedomainsandcompetenciesandhowtheyaretypically taught.Additionally,thereviewerswerenotallattendingsand notallgeriatric-fellowshiptrained.Despitethis, first-round consensuswasveryhigh(84-96%),whichsuggestsshared knowledgeamongthegroup.TheEMresidentsinvolvedin thisprojecthavesincestartedfellowshipsinmedicaleducation andpalliativemedicine,demonstratingtheirpassionand additionalunderstandingintheseareas.

CONCLUSION

ThegeriatriccompetenciesareincludedwithintheEM Modelandknowledge,skills,abilitieslist.Thecompetencies providemoredetailforeducationorboardquestions. Weidentifiedareasofoverlapwherethesesubspecialty competenciescanbeemphasizedinEMresidencycurriculums.

ACKNOWLEDGMENTS

TheauthorsthankDr.MelissaBarton,ABEMDirector ofMedicalAffairs,forheradviceindesigningthisstudy. WealsothankallthemembersoftheSocietyofAcademic EmergencyMedicine’sAcademyofGeriatric EmergencyMedicine.

AddressforCorrespondence:LaurenT.Southerland,MD,MPH,The OhioStateUniversityWexnerMedicalCenter,Departmentof EmergencyMedicine,725PriorHall,376W10thAve,Columbus, OH43210.Email: Lauren.Southerland@osumc.edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. DianeL.GorgasisaboardmemberoftheAmericanBoardof EmergencyMedicine.LaurenT.Southerlandhascontributedto someofthefreeeducationalwebsitesmentionedinthediscussion.

Copyright:©2024Southerlandetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.KizziahMA,MillerKN,BischofJJ,etal.Emergencymedicineresident clinicalexperiencevs.in-trainingexaminationcontent:anational databasestudy. AEMEducTrain. 2022;6(2):e10729.

2.AshmanJJ,SchappertSM,SantoL.Emergencydepartmentvisits amongadultsaged60andover:UnitedStates,2014–2017. NCHSData Brief,no367.Hyattsville,MD:NationalCenterforHealthStatistics. 2020.

3.AmericanBoardofEmergencyMedicine.The2019ModeloftheClinical PracticeofEmergencyMedicine.Availableat: https://www.abem.org/ public/resources/em-model.AccessedSeptember9,2020.

4.AmericanBoardofEmergencyMedicine.2022Knowledge,Skills,& Abilities.Availableat: https://www.abem.org/public/resources/ emergency-medicine-milestones-ksas.AccessedAugust9,2022.

5.HoganTM,LosmanED,CarpenterCR,etal.Developmentofgeriatric competenciesforemergencymedicineresidentsusinganexpert consensusprocess. AcadEmergMed. 2010;17(3):316–24.

6.ACEPGeriatricEmergencyDepartmentAccreditationCriteria.Version July11,2023.Availableat: https://www.acep.org/siteassets/sites/geda/ documnets/ged-criteria.pdf.AccessedFebruary21,2023.

7.MagidsonPDandCarpenterCR.Trendsingeriatricemergency medicine. EmergMedClinNorthAm.2021;39(2):243–55.

8.RingerT,DoughertyM,McQuownC,etal.Whitepaper–geriatric emergencymedicineeducation:currentstate,challenges,and recommendationstoenhancetheemergencycareofolderadults. AEMEducTrain. 2018;2(Suppl1):S5–16.

9.HoganTM,HansotiB,ChanSB.Assessingknowledgebaseongeriatric competenciesforemergencymedicineresidents. WestJEmergMed 2014;15(4):409–13.

10.HesselinkG,SirÖ,ÖztürkE,etal.Effectsofageriatriceducation programforemergencyphysicians:amixed-methodsstudy. Health EducRes. 2020;35(3):216–27.

11.BieseKJ,RobertsE,LaMantiaM,etal.Effectofageriatriccurriculumon emergencymedicineresidentattitudes,knowledge,anddecisionmaking. AcadEmergMed.2011;18Suppl2:S92–6.

12.ThatphetP,RosenT,KayarianF,etal.Impactofgeriatricemergency fellowshiptrainingonthecareersofemergencyphysicians. Cureus. 2021;13(9):e17903.

EDUCATION SPECIAL ISSUE:ORIGINAL RESEARCH

EmergencyMedicineResidentNeedsAssessmentandPreferences foraHigh-valueCareCurriculum

BennettH.Lane,MD*† °

SimanjitK.Mand,MD* ° StewartWright,MD,MEd* SallySanten,MD,PhD* BrittanyPunches,PhD‡

*UniversityofCincinnatiCollegeofMedicine,DepartmentofEmergencyMedicine, Cincinnati,Ohio

† UniversityofCincinnatiHealthAirCare&MobileCare,Cincinnati,Ohio ‡ OhioStateUniversity,CollegesofNursingandMedicine,DepartmentofEmergency Medicine,Columbus,Ohio

° BennettH.LaneandSimanjitK.Mandcontributedequallytothiswork

SectionEditors: SaraKrzyzaniak,MD,andJeffreyLove,MD

Submissionhistory:SubmittedDecember13,2022;RevisionreceivedOctober3,2023;AcceptedNovember3,2023

ElectronicallypublishedDecember8,2023

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.59622

Introduction: Considerationofthecostofcareandvalueinhealthcareisnowarecognizedelementof physiciantraining.Despitetheurgencytoeducatetraineesinhigh-valuecare(HVC),educational curriculaandevaluationofthesetrainingpathsremainlimited,especiallywithrespecttoemergency medicine(EM)residents.Weaimedtocompleteaneedsassessmentandevaluatecurricular preferencesforinstructiononHVCamongEMresidents.

Methods: Thiswasaqualitative,exploratorystudyusingcontentanalysisoftwofocusgroupsincluding atotalofeightEMresidentsfromasingleMidwesternEMresidencytrainingprogram.Participantsalso completedasurveyquestionnaire.

Results: Thereweretwothemes.Withintheoverallthemeofresidentexperiencewithandperceptionof HVC,wefound fivesub-themes:1)understandingofHVCfocusesondiagnosisanddecision-making; 2)concernaboutpatientcosts,includingtheeffectsonpatients’ livesandtheirabilitytoengagewith recommendedoutpatientcare;3)conflictbetweeninternalbeliefsandexternalexpectations,including patients’ perceptionsofvalue;4)approachtoHVCchangeswithincreasingclinicalexperience;and 5)slow-moving,politicaldiscussionaroundHVC.Withintheoverallthemeofdesirededucationand curriculardesign,weidentifiedfoursub-themes:1)limitedprioreducationonHVCandhealth economics;2)motivationtoreceivetrainingonHVCandhealtheconomics;3)desirefordiscussionbasedformatforHVCcurriculum;and4)curriculumtargetedtoleveloftraining.Respondentsindicated greatestacceptabilityofinteractive,discussion-basedformats.

Discussion: WeconductedatargetedneedsassessmentforHVCamongEMresidents.Weidentified broadinterestinthetopicandlimitedself-reportedbaselineknowledge.Curricularcontentmaybenefit fromincorporatingresidentconcernsaboutpatientcostsandconflictbetweenexternalexpectationsand internalbeliefsaboutHVC.Curriculardesignmaybenefitfromafocusoninteractive,discussion-based modalitiesandtailoringtothelearner’sleveloftraining.[WestJEmergMed.2024;25(4.1)51–58.]

INTRODUCTION

Arecentshifttofocuson “value” inhealthcare,often definedashealthoutcomesachievedperdollarspent,has emergedinresponsetopersistentlyrisingcostsoverdecades.1 Recenteventshavehighlightedthecostofemergencycarein thenationalspotlight,includingfederallegislationon surprisebilling,insurerdenialsofclaimsforemergency department(ED)visitswithouta finalemergentdiagnosis, andregulationsonpaymentsforairambulance transports.2–5 Consistentwiththesedevelopments,current

AccreditationCouncilforGraduateMedicalEducation (ACGME)guidelinesstatethat “residentsmustdemonstrate competencein ::: incorporatingconsiderationsofvalue, equity,costawareness,deliveryandpayment,andriskbenefitanalysisinpatientand/orpopulation-basedcare asappropriate.”6

DespitethecurrentACGMEguidelinesandincreasing demandsforhigh-valuecare(HVC),theappropriate educationalcontentandinstructionalmethodshavenotbeen clearlyestablished.Moriatesandcolleaguesdelineated21 HVCcompetencieswithbeginning,proficient,andexpert levelsthroughaniterativeprocessledbyamultidisciplinary committee.7 Whilerigorousandexpert-led,thisapproachdid notincludearesident-focusedneedsassessment,and subsequentneedsevaluationshavebeenlimitedtosurveysof internalmedicineorpediatricsresidentsatasinglesite.8–10 Similarly,evaluationofproposedinternalmedicineor pediatricsresidentcurriculahavebeenlimitedtosingle-site pre-/post-surveys,withonestudyalsoincludingpostimplementationfocusgroups.10–13

Withinemergencymedicine(EM),HVCandhealth economicseducationalresourcesarelimited,asa2010 systematicreviewofcost-effectivenesscurriculaidentified onlyasingleEMcurriculumfocusedontheOttawaankle rules.Sincethatreview,twoadditionalcontributionsthatwe areawareofinclude1)theEmergencyMedicineResidents’ Association Residents ’ AdvocacyHandbook addressing policy-relatedtopicsinatextbook-likeformatand2)acostconsciouscarecurriculumdevelopedbyLinandLaskowski atasinglesiteinNewYork(personalcommunication,L. Laskowski).14,15 Thereisapaucityofformal,residentfocusedneedsassessmentsacrossspecialties,particularly inEM.Ourobjectivewastoperformatargetedneeds assessmenttoassessEMresidents’ needsandinterests inHVCandpreferencesforinstructionalmodality.

METHODS

StudyDesign

Aspartofacurriculumdevelopmentprocess,we performedaproblemidentificationandtargetedneeds assessmentforEMresidents,correspondingtoKern’ssixstepapproachtocurriculardevelopment.16 Toachieveour objective,weconductedaqualitative,exploratorystudy usingconventionalcontentanalysis.Thismethodallowedus tocriticallyexaminetheparticipantresponsestoidentify commoncategoriesandelucidatethemes.Oursecondary objectivetodeterminepreferencesforinstructionalmodality includedacollectionofrespondents’ self-assessmentsusinga surveyquestionnaire.Weobtainedinstitutionalreview board(IRB)approvalforallstudyprocedures.

SettingandParticipantSelection

ThesettingwasasingleMidwesternUnitedStatesEM residencyprogramwith56totalresidents.Twophysician

PopulationHealthResearchCapsule

Whatdowealreadyknowaboutthisissue?

Residenteducationguidelinesnow incorporatethetopicofvalueinhealthcare, butfewresident-focusedneedsassessments forthisconceptareavailable.

Whatwastheresearchquestion?

Forhigh-valuecare,whatareemergency medicineresidents ’ needs,interests,and preferencesforinstructionalmodality?

Whatwasthemajor findingofthestudy?

Residentsself-reportlowknowledgebutare interestedineducationonhigh-valuecare. Theypreferdiscussion-basedmodalities.

Howdoesthisimprovepopulationhealth?

Addressingcostofcarethroughgraduate medicaleducationmayhelpaddress accessibilityandaffordabilityofcare.

authorswereresidentsatthetimeofthedatacollection phaseoftheproject(BHL,SKM).Recruitmentofa conveniencesampleofeightEMresidentswasperformedvia emailbyoneoftheauthors(SKM)totheremaining54 residents.Nineresidentsresponded.(Oneresidentcouldnot participateduetoschedulingconstraints.)Noparticipant terminatedtheirparticipationduringthefocusgroup.

DataCollectionProcedures

Weobtaineddocumentationofinformedconsentpriorto studyprocedures.Asemi-structuredinterviewguidefor focusgroupswasprimarilyauthoredbyasingleauthor (BHL)andreviewedsequentiallybyadditionalauthorsfor revisionofcontentandphrasing(SKM,BP).Theinterview guideisincludedas Appendix1.Focusgroupswereco-ledby twophysicianauthorswhowereresidentsatthetime(BHL, SKM)followingtheinterviewguide.Bothfocusgroupswere audiorecordedandsubsequentlytranscribed.No fieldnotes weremade,norweretranscriptsreturnedtoparticipantsfor comment.ThefocusgroupsoccurredduringSeptember2020 inamedicalschoolconferenceroomwithnootherperson presentasidefromfocusgroupleadersandparticipants. Afterthefocusgroupdiscussionwascomplete,participants independentlycompletedasurveyquestionnaireusingLikert scaleandrankorderquestionsonpaper(Appendix2).Each focusgroupincludedfourparticipantswithatleastone intern(postgraduateyear[PGY]1)ineachgroup.Intotal,

the firstgroupincludedonePGY-1,onePGY-2,andtwo PGY-3residents;thesecondfocusgroupincludedtwoPGY1andtwoPGY-4residents.Eachfocusgrouplastedbetween 75–85minutes.Norepeatinterviewswerecompleted. Participantsreceiveda$15giftcardforcompensation, consistentwithIRBguidelines.

DataAnalysis

Thetranscriptswerereviewedandconventionalcontent analysiswithline-by-linecodingwascompletedbytwo independentcoders(BHL,SKM).Usinganopencoding technique,importantstatementswereidentified(generally termed “the firstcut”).17 Codesweredevelopedinvivoand didnotreferencepreviousliterature.(Theyaredepictedina codingtreein Appendix3.)Significantredundancyincodes wasidentified,whichwasfelttobeconsistentwiththematic saturation.18 Theanalysisteamcametogetherwithathird reviewer(BP)tocategorize,refine,andclusterimportant statements,andsubsequentthemesanddomainsemerged. Weusedtheconsolidatedcriteriaforreportingqualitative research(COREQ)asreportingguidelines(Appendix5).19

DescriptivestatisticswereperformedinMicrosoftExcel forquestionnairedata,andweusedWord(Microsoft, Redmond,WA)fortranscriptsandcodingdocumentation. Theuseofindependentcodersandateamofthree tocategorizeanddevelopthemesenhancedcredibility, andinvestigatortriangulationaidedconfirmabilityof theresults.18

ReflexivityStatement

Reflexivityoftheresearchteamincludedrecognitionthat thefocusgroupleadersandcoderswereknowntothe participantsandidentifiedtheirrespectivespecificinterestsin HVC/healtheconomics(BHL)andmedicaleducation (SKM)totheparticipantsaspartoftheintroduction.The focusgroupleadersidentifiedasmale(BHL)andfemale (SKM).BHLandSKMwereresidentsatthetimeofthe study.BPprovidedtrainingtoBHLandSKMregarding techniquesinsemi-structured,focusgroupfacilitation;BHL hadlimitedpriorexperiencewithfocusgroupfacilitation.A methodologicallimitationisthatthesameresidents comprisedthefocusgroupsandcompletedsurvey questionnaires;surveyquestionnaireresultsmayhavebeen influencedbytheprecedingfocusgroupdiscussion,although allquestionnaireswerecompletedindependentlybyall participantswithoutadditionaldiscussion.

RESULTS

Atotalofeightresidentsparticipated.Withrespectto theimportanceofeducationaboutHVCtopics,residents endorsedtherelevanceofHVCtopicstotheresident physician(7/8,[88%])andtheimportanceofaHVC curriculum(8/8,[100%])(Appendix4,Figure2).We identifiedtwooverarchingthemes:1)experiencewithand

perceptionofHVC;and2)desirededucationandcurricular design.Foreachoverarchingtheme,componentsub-themes summarizedclustersofresidentcommentsforwhichwe includerepresentativecommentsand(ifidentified) participantrecommendations.

OverarchingTheme1:ExperiencewithandPerceptionof High-valueCare

Sub-theme1:Understandingofhigh-valuecarefocuseson diagnosisanddecision-making.

ResidentsmostfrequentlyassociatedHVCwiththe activitiesthatfacilitatediagnosisanddecision-makinginthe ED.Forexample,whenaskedwhethertheyhadageneral definitionfororhadheardofthephrase “high-valuecare,” oneresidenthighlightedusingtheEDevaluationto “appropriately fi gureoutwhatisgoingonwiththispatient anddecidewheretosendthem ” (Resident#1,PGY-1).In thisunderstanding,residentsbelievecareactivitiesarehigh valueiftheyallowthecliniciantomakeadiagnosisor disposition.Lesscommonly,otherresidentsmentioned aspectsofHVCsuchasresourceuse,stewardship(citinga specificexampleofacost-savingsinitiativerelatedtotheuse ofcombatgauze[Resident#7,PGY-4]),andtheconceptof cost-benefitanalysis: “clinicaldecisionrulesthat ::: reduce unnecessaryheadCTs,notonlyfromaradiation perspective,butalsofromacost-savingsperspective ” (Resident#8,PGY-4).

Sub-theme2:Concernaboutpatientcosts.

Inthefocusgroupdiscussion,residentsvoiceduncertainty duetovaryingpatientinsurancereimbursementofcare providedintheEDandconcernssurroundinghighpatient costs,inlargepartduetoaself-identifiedlackofknowledge. Becauseofthisknowledgegap,residentsfeltinadequately preparedtohaveconversationssurroundingcostand insurancecoveragewithpatients.Oneoftheparticipants recalledapatientencounterinwhichtheresidentfelt uninformedtoaddressthepatient’sreactionafterthe residentdisclosedthepresenceofanewmassconcerning forcancer:

Howmuchisthisgoingtocostme?HowamIgoingtopay forthis? ’ [and]Ididn ’ tknowtheanswer. It ’ dbenice ifIactuallyhadsomedata ::: likeyou ’ reuninsured,it ’ s ok,becauseit ’ sgoingtobelikethisforthe fi nancialplan, ifyou ’ reinsured,thisiswhathappens.Ihaveno clue. ” (Resident#5,PGY-3)

Otherresidentsstatedthattheywereunawareofthecosts ofcommonlyordereddiagnosticsandtherapeuticsinthe ED.Theydescribedbeingconcernedandunawareofthe financialandsocialramificationsofcareactivitieson patients’ livesoutsideofthehospital,andtheyparticularly worriedabouttheimpactonpatients’ abilitytoengagewith

recommendedoutpatientcare: “It ’ showmuchthepatient getschargedthatwouldactuallymatterfromasocial determinantsofhealthperspective ” (Resident#2,PGY-1).

Residentsparticularlycitedfeelingchallengedby shareddecision-makingdiscussionswhenpatientshad financialconcerns.

Sub-theme3:Conflictbetweeninternalbeliefsandexternal expectations.

Residentsnotedthattheremaybeaconflictbetweena physician’spersonalbeliefsandtheexternalexpectationsand pressurestheyface.Someexternalexpectations,suchas thosefromsystems-level “hurdles” placedintheelectronic healthrecord-orderinginterface,areexplicitlyidentifiable forresidents: “Itrytoorder[intravenousacetaminophen] allthetime.ITtakesyouthrough,youhavetogothroughall thesequestionsbecausethey ’ retryingtokeepmefrom ordering[it]. Iknowthey ’ retryingtokeepmefrom orderingit,butI ’ mgoingtokeeponorderingit ” (Resident #8,PGY-4).Otherexternalexpectationsareperceivedtobe implicitwithinthemedicalcommunity: “ Eventhoughwe talkaboutinanacademicsetting,orinaboardroom,it ’ sOK tohaveamissfromastatisticalperspective,Ithink culturallythat ’ snotacceptable. It ’ sjustnotplayingout intherealworld,inmyopinion,acceptingthatthereisa missrate ” (Resident#4,PGY-2).

Residentsparticularlyhighlightedthatpatientsarea sourceofexternalexpectationsandrecognizedthat patientsmayviewcost,quality,andvalueofcare differentlyfromtheemergencyphysician.Thisdifference inperceptionmayleadtoadisconnectinexpectations: “ Valuecanreallybeintheeyesofthebeholder ::: makes methinkaboutwhatIthinkmightbethebestthingforthe patientmaynotbeatallthesameaswhatthepatientvalues ” (Resident#6,PGY-3).Moreover,thecon fl ictbetween internalbeliefsandexternalexpectationscanovershadow attemptstoprioritizeHVC.Acontextcitedforthiscon fl ict wereEDvisitsofpatientswhocommonlyfrequenttheED. Forthesepatients,thelackofcommunityresourcesfor patientscanbefrustratingandrenderalearnerfeeling helplessorunabletoprovideholisticpatientcare.Forthese patients,traineesnotedfeelingadisconnectbetweenthe caretheyfeltexpectedtoprovideandthecaretheydesired toprovide.

Subtheme4:Approachtohigh-valuecarechangeswith increasingclinicalexperience.

Residentssharedanecdotesthatdemonstratehowthe definitionofandapproachtooptimizeHVCchanges withincreasingclinicalexperience.Onejuniorresident highlighted “wantingtoknow” asmotivationforordering testing: “ I ’ mascuriousas[thepatients]are,tobehonest;so Iwanttoknowthatthispatientisperhapsapresentationof

[aspeci fi cdiagnosis] ” (Resident#4,PGY-2). Similarly,as onenon-internresidentreflected:

“Andhonestly,that’ssomethingthatcomeswithtime – like ifyoutoldmeasaninternIcouldorderamillion-dollartest andgettheanswerthatIneed,Iwould100%doitbecause it’seasy,I’llberight,andIcanhelpthepatient.Butasyou practicemedicineyourealize ::: ifyouhaveamilliondollartesttoanswerifit’sGERD ::: it’snotgoingto changeyourmanagement AsI’mprogressingthrough residencyIgetmoreandmorecurious,andI’ mmore willingtoacceptinformationabout[HVC]” (Resident #5,PGY-3)

Sub-theme5:Slow-moving,politicaldiscussionaroundhighvaluecareinmedicine.

Ingeneral,residentsdescribethemselvesaslooselyaware ofthepolitical,academic, financial,andclinicalimplications ofnationaldiscussionsonHVCtopicsforfutureemergency physicians.Forexample, “Howyoudeterminevalue?I rememberbackwhenObamawasstillaroundandinof fi ce,I rememberthatwasabigdiscussion,youknow whatisreal valueandwhodeterminesthat?That ’ ssortofablackbox ” (Resident#8,PGY-4). Anotherresidentreflected,

“ Thereisalwayschatteroutthereinthe politicaland insuranceworld.AndI ’ mnotsureIknowwherelikethe landmarkpolicyor ::: guidingfoundationisforthat conversation.So,certainly,outsidethereisafeeling thatthereisalwaysthischatterhappening ” (Resident #4,PGY-2).

Whenaskedaboutproposedphysicianreimbursement modelscurrentlyundergoingfederalregulatoryreview,most residentsdidnotknowwhatthosefuturepoliciesentailed.In addition,manyresidentsreportednotbeingwellversedin currentreimbursementmodels,althoughnon-intern residentsreportedmoreinterestincurrent reimbursementinformation.

OverarchingTheme2:DesiredEducationandCurricular Design

Sub-theme1:Limitedmedicaleducationonhealth economicsandhigh-valuecare.

Whenaskedabouttheirpriortraininginhealtheconomics andHVCtopics,allresidentsnotedminimaltonoprior exposureduringtheirmedicaltraining.Inthesurvey questionnaire,allparticipants(8/8[100%])eitherstrongly disagreedordisagreedwiththestatement “Ifeelconfident thatIknowthecostofthecarethatIprovidetopatientsin theemergencydepartment” (Appendix4).Muchoftheprior exposuredescribedbyresidentswascomprisedofbriefand infrequentdidactic-baseddiscussionsthatweredescribed

asleadingtolimitedinformationretentionandlimited applicationtoclinicalpractice.

Beyondthis,theyvoicedthebeliefthattherewerefew opportunitiesforknowledgeacquisitionduetolackof availableresources,particularlywithrespecttopricesand costsofhealthcareactivities.Residentswerenotfamiliar withhospital-basedornationallybasedresourcesthatcould assistwithday-to-dayclinicalhealthcarequestionssuchas patientcost: “Ithinkhospitalsaremandatedtohavesome sortoflist,masterlist,ofhowmuchthingscosts,butit ’ salso superhardto fi nd ::: IhavenoideawhereIwould fi ndthat information ” (Resident#7,PGY-4).

Sub-theme2:Motivatedtoreceivetrainingonhigh-value careandhealtheconomics.

TheEMresidentsidentifiedthemselvesasfrontline healthcareworkers.Intheirrole,theyinteractdirectlywith thecommunityandpatientswithdiversebackgrounds, particularlyindividualsfacing financialbarrierstoaccessing care.Becauseofthisuniquepositioninthemedical field, residentsbelievethat financialandinsurancepressures mayunderliepatients’ utilizationoftheEDandthat cliniciansshouldthereforeunderstandthesefactors.One residentreflected

“Ithinkwhenyou look athealthcareasagestalt, peopleareseeingprimarycar e[clinicians]lessandless andrelyingontheEDmoreforprimarycare.And assumingthatthattrendcontinues Ithinkasan emergencyphysicianitisimportanttoknowthose things[healthcareeconomicstopics]becauseofthat reason,justtheutilizationoftheEDingeneral ” (Resident#1,PGY-1).

Oneresidentalsonotedthatthelackofhealtheconomics knowledgecanputemergencyphysiciansatadisadvantage ininfluencingandleadingsystems-basedpractice:

“ Ithinknotunderstanding[HVCandhealtheconomics topics]takesawayalotofourpowertobealeaderand makesusmorepawnscarryingoutsomeoneelse ’ svision ofhowmedicineshouldbepracticed ” (Resident#6, PGY-3).

Residentsrecognizedtheimportanceofandneedfor furthertrainingonHVCtopicstounderstandtheimpact thattheirdecision-makinghasonpatientsandthe healthcaresystem.

Sub-theme3:Desirefordiscussion-basedformatforhighvaluecarecurriculum.

ResidentswereaskedwhattheoptimalformatforHVC curriculumwouldbeforresidency-levellearners,andthe majoritywereinsupportofadiscussion-basedformat.

“ Iliketheideaofthecase-based,small-groupdiscussion. Especiallywhenyouhaveattendingsthere,andyouhave variedlearnerlevels,andIkindoflikethatbecauseyou getvariedsortsofinputsandthat ’ sinteresting.AndIjust feellikethissortofstuff,thesesortsoftopics,arebest,for me,exploredverbally ” (Resident#8,PGY-4).

Oneresidentnotedthatbecausethisisnotcommon knowledgeamongemergencyclinicians,involvingacontent expertwouldbecriticaltoasuccessfulcurriculum:

“Anotherpartofincorporatingthis,iswhoisthecontent expert ::: .[HVCcareis]atopicthat ::: atypical academic[emergency]physicianwould[not]knowabout. Italmostneedstobeacollaboration [someone]with healtheconomicinterestandknowledgeandsomeonewith aneducationbackground,too,to figureouthowto incorporatethis” (Resident#7,PGY-4).

Consistentwiththisqualitativetheme,thehighest percentageofresidentsrankedmodalitieswiththe opportunityforinteractivesmall-groupdiscussionhighly, whetherasonlineappsorinperson,onthesurvey questionnaire(Figure 1).

Sub-theme4:Curriculumtargetedtotheirleveloftraining. Whilemostoftheresidentsrecognizedtheneedfora formalHVCcurriculumduringmedicaltraining,therewas variationinwhentheythoughtthiscurriculumshouldbe introducedattheresidency-traininglevel.ThePGY-1and PGY-2residentsvoiceddesiretofocusonclinicalknowledge acquisitioninlieuofhealtheconomicstopics:

“Asanintern,I’dratherbemoretowardstheclinicalaspect ofthingsrightnow ::: Idon’tthinkI’vedevelopedthatskill enoughtowanttosacrificeoneofthosejournalclubsfor healtheconomics.Ithinkasalaterresident,I’dbeon board ::: ” (Resident#1,PGY-1).

“ MyinitialthoughtwasthatIwouldwantsomething clinicallyrelevantbecaus eIfeellike[Iam]earlierin trainingandjusttryingtobuildthatfoundation ” (Resident#4,PGY-2).

Anon-internresidentnoted “ IfeellikeasI ’ mprogressing throughresidency,IgetmoreandmorecuriousandI ’ m morewillingtoacceptinformationaboutthatstuff[HVC] ” (Resident#5,PGY-3).Inreplytoaninternindicatingthe topicof “[relativevalueunits] andphysician-associated income ::: wouldn ’ tappealorapplytomerightnowwhenI wouldjustforgetit ” (Resident#3,PGY1),Resident#7 (aPGY-4)reflectedthatnon-internresidentswouldbe interestedduetopersonaldecision-making: “ Iwouldsaythe PGY-3sbecausesomeofthePGYswouldstartsigning contractsinthesummer ”

Percentageofrespondentsrankingeachinstructionalmodalityamongtoptwochoicesandmeanrankingwithineightmodality options(n = 8).

Aninterestingperspectiveraisedbyoneofthenon-intern residentswasthepotentialtonegativelyinfluencejunior learners’ practicepatternsiftopicsofHVCwereintroduced tooearlyinmedicaltraining:

“ Iwonderfromaneducationmissionside,couldyou in fl uenceearlytrainees ’ ::: practicepatternsbecauseof knowledgeofthis.AndIdon ’ twantthattohappen youneedtoseewhereyoufallinthatspectrumto developyourpracticepattern.AndIwonderifyou fi nd outthatatestcoststhisamountofmoney,maybeyou won ’ tgettofullyexplorethatspectrumanddevelop yourownpointonthatspectrum ” (Resident#7,PGY-4).

Lastly,acoupleofresidentsvoicedconcernaboutthe integrationofanovelcurriculuminanEMtrainingprogram giventhatEM’sscopeofpracticealreadyaddressesmany adjacentdisciplines:

“We’reallkindofinagreementthatabaselinelevelof understandingyoushouldhave butasfarasabout dataandliterature ::: I’dkindofreservethatforpeople thathaveaninterestinit,similartohowwedowithother things,likesportsmedicine” (Resident#3,PGY-1).

“ Youhavesomanythingstolearn.Notonlyclinically, butalsoournon-clinicalcurriculum ::: ispretty impressive,soit ’ stough[to]addawholeother curriculum ” (Resident#8,PGY-4).

DISCUSSION

ResidentsrecognizedtheimportanceoflearningHVC principlesforapplicationinbothpatientcareandtoinform systems-basedpractice;however,theyfeltinadequately trainedonthetopic.Ourneedsassessmentidentifiedtwo

mainthemestoinformEM-specificcurriculaaddressing HVCtopics:residentexperiencewithandperceptionof HVC,anddesirededucationandcurriculardesign.

Consistentwithstudiesinotherdisciplinesandsettings, theresidentsreportedlimitedconfidenceintheirknowledge ofbasicHVCprinciples,andthe financialimpactsofcostof careforindividualpatientsandthehealthcaresystemasa whole.9,10,13 Sub-themes1(understandingofHVCfocuses ondiagnosisanddecision-making)and2(residentconcerns aboutpatientcosts)inthisstudywereconsistentwiththemes fromfocusgroupscompletedwithgeneralpediatrics residentsattwocentersof “howaninterventionchanges management” and “thinkingaboutthecostasaharm.”10

Residentsstatedthatearlyonintheirtraining,HVC knowledgegapsarerelatedtopatientcosts,patientinsurance reimbursement,cost-benefitanalysis,andresource stewardship.Later,self-identifiedknowledgegapsemerging asnon-internlearnerswereprimarilyrelatedtophysician reimbursement.Areviewoftheliterature,includingprior workwithinpediatricsandinternalmedicine,suggestedno priorevidenceofresidentknowledgeorinterestvaryingby experiencelevel;ifvalidatedinadditionalsettings,such variationswithlearnerexperiencewouldprovidevaluable guidanceinthedesignofeducationalcurricula.

Theresidentparticipantsstatedtheirlackofformal traininginandbasicknowledgeofHVCwasabarrierto providinghigh-valueemergencycare.Theyalsoreported limitedawarenessofnationalhealthpolicyyetwereless interestedinadetailedunderstandingofthesetopics.This findingsuggeststhataspecializedelectivemaybebetter suitedtoeducationregardinghealthpolicytopicsthatdonot directlytieintoday-to-dayemergencycare,asintheexample describedbyGreysenandcolleagues.21 Finally,the participantsalsoindicatedtheneedformoreeducationon system-widereimbursementandHVCpolicies.Tomeetthis

Figure1.

need,priornational-levelsurveydatafrominternalmedicine residentsandprogramdirectorssuggeststhatinstitutional supportforbothHVCfacultydevelopmentandprovisionof physiciancost-of-careperformancedataareassociatedwith anincreaseinresidentreportsofeducationonHVC.22

UnanticipatedaspectsofHVCthatwereviewedaslearner obstaclesincludeddynamicconflictsbetweeninternallearner beliefsandexternalexpectationsandthevariabilityinvalue perceptionbetweenpatientsandclinicians.Theseissuesmay complicateresidents’ perceptionofandimplementationof HVCintheclinicalsetting;addressingtheseissueswithin HVCeducationiscriticaltoavoidunintentionalcreationof anxiety,orevenmoraldistress,inthetrainingenvironment. Inanintern-targetedcurriculumininternalmedicine,Hom andcolleaguesalsodiscussedresident-perceivedbarriers surroundingintra-team,interdisciplinary,andpatientand familydynamicsandhowtheycomplicateunderstanding andimplementationofHVCprinciplesatanearlylearner stage.14 Thus,futurecurriculawillneedtofocusbothon foundationalknowledgedisseminationandtechniqueson howtoapproachtheabovebarriers.

Anadditionalunexpectedbarrierraisedbyresidentsinthe focusgroupwastheconcernthattheexistingEMtraining curriculumdoesnothavethecapacitytoincorporateHVC; and,therefore,HVCtrainingmaynot fitasacoreelement. Whilenotaddressedinthesefocusgroups,afuturedirection forworkinthisareashouldincludeevaluationofhow residentswouldweighHVCtrainingcomparedtoother curricularelementsandwhethertherewouldbeopportunity tomakepotential “tradeoffs.”

Intermsofcurriculardesignandformat,themesemerged tooptimizenotonlyknowledgeacquisitionand understanding,butalsotimingduringtheresidencytraining program.Theresidentparticipantswereinsupportofan expert-led,discussion-basedcurriculumtolearnthe principlesofHVC,consistentwiththeexperiencesofHom andcolleagues.13 These findingsalsocoincidewiththoseof Stammenetalintheirsystematicreview,concludingthat reflectivepracticethroughfeedbackandgroupdiscussions incentivizephysicianstothinkcriticallyaboutmedical decisions.19TheresidentsalsosuggestedthatHVCtopics shouldbetargetedmoretowardnon-internresidentswho havemasteredproficiencyinbasicclinicalknowledgeand skillsandwouldbeabletoapplythesenewprincipleswith morepurposethantheirjuniorcounterparts,althoughsome earlierknowledgebasetosupplementformativeexperiential growththroughoutresidencymaybebeneficial.Theydid voiceconcernthattheintroductionofHVCtooearlyin residencycouldjeopardizeearlylearners’ practice patterndevelopment.

LIMITATIONS

Thereareseverallimitationstoconsiderwithregardtoour study.First,thisstudyreflectsasampleofresidentsfroma

single-center,largeacademichospitalandmaynotbe applicabletoallacademic-andcommunity-basedtraining programs.Becauseitwasasingle-centerstudy,wecouldnot distinguishhowthree-yearprogramsorfour-yearprograms withdifferentapproachestoresidentprogressionor “seniority” woulddifferfromthe findingsidentifiedhere. Second,onlyasmallsubsetofprogramresidentsparticipated ineitherfocusgroup,leadingtothepossibilityofselection biaswithregardtotheparticipantswhovolunteeredto discusstheirthoughtsonHVC.Theseresidentsmayhave hadaparticularinterestinmedicaleducationorHVCthat maynotbeapplicabletoallEMresidentsacrossthecountry. Thesmallsubsetofparticipatingresidentsalsolikelylimited thenumberofavailableperspectivestobecollectedand informthematicsaturation.

Third,thestudyincludedamixofjuniorandsenior residents.Whilethestudyallowedforarichspectrumof experiencetoinformprevious exposuretoHVCprinciples, itmaynothavebeenasimpactfulasevaluatingthe perspectivesofthemostexperiencedresidentsina programwhohadnearlycompletedtheentireprogram curriculumandcouldidentifyareasfornuanced improvement.Fourth,whiletheuseoffocusgroups (ratherthanone-on-oneinterviews)allowedemergent discussionbetweenparticipa nts,thepresenceofpeersmay haveledsomeparticipantsto avoidmakingstatementsdue tofearofbeingperceivedascontroversial.Fifth,dueto transitionsinroles,membercheckingcouldnotbe performed.Whileourstudyaddsacriticallynecessary needsassessmenttothecurrentbodyofliterature,further andmorerigorousstudiesthatincludealargernumberof residencyprogramsandpartic ipatingresidentsareneeded toverifythese fi ndingstoaccuratelyinformfuture EMcurricula.

CONCLUSION

Ourtargetedneedsassessmentindicatesthatresidents currentlyfacegapsinknowledgeofhigh-valuecare topicspertainingtothemedicalcarethattheyprovideand maybenefitfromadditionaltrainingduringresidency. Residentsinterviewedinthisstudyidentifiedseveral perceivedbarrierstounderstandingHVC,butthey consistentlyexpressedinterestinaformalcurriculumto addressthesechallenges.Wefoundapreferencefor interactive,small-groupdiscussion-basedformats withcontentadjustedbylevelofclinicaltraining.

AddressforCorrespondence:BennettH.Lane,MD,MS,University ofCincinnatiCollegeofMedicine,DepartmentofEmergency Medicine,231AlbertSabinWayML0769,Cincinnati,OH452670769.Email: lanebt@ucmail.uc.edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsourcesand financialormanagementrelationshipsthatcouldbeperceivedas potentialsourcesofbias.Thisworkwassupportedbyinternal departmentalfundingfromadesignatedResidentResearchFund.No authorreportsanyrelatedconflictsofinterest.BennettH.Lanereports receivingresearchgrantsupportfromtheSocietyforAcademic EmergencyMedicineFoundationforinvestigator-initiatedworkon interhospitaltransfersbyair.BennettH.Lanealsoreportsanequity interestandconsultingfeesfromTriAxiaHealthforunrelatedwork.

Copyright:©2024Laneetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.PorterME.Whatisvalueinhealthcare? NEnglJMed 2010;363(26):2477–81.

2.KliffSandSanger-KatzM.Forsurprisemedicalbills,it’sthebeginningof theend. TheNewYorkTimes.PublishedJuly1,2021.Availableat: https://www.nytimes.com/2021/07/01/upshot/surprise-medicalbills-biden.html.AccessedJuly19,2021.

3.ChouSC,GondiS,BakerO,etal.Analysisofacommercial insurancepolicytodenycoverageforemergencydepartment visitswithnonemergentdiagnoses. JAMANetwOpen 2018;1(6):e183731.

4.KellyM.Sky-highairambulanceprices. AnnEmergMed 2020;76(5):A17–A20.

5.MoleBeth.BiggesthealthinsurerplanstodenyERbillsifitdoubtsyou hadanemergency. ArsTechnica.PublishedJune10,2021.Availableat: https://arstechnica.com/science/2021/06/biggest-health-insurerplans-to-deny-er-bills-if-it-doubts-you-had-an-emergency/ AccessedJuly19,2021.

6.AccreditationCouncilforGraduateMedicalEducation.ACGME CommonProgramRequirements(Residency).PublishedonlineJuly1, 2023.Availableat: https://www.acgme.org/globalassets/pfassets/ programrequirements/cprresidency_2023.pdf AccessedOctober2,2023.

7.MoriatesC,DohanD,SpetzJ,etal.Definingcompetenciesfor educationinhealthcarevalue:recommendationsfromtheUniversityof California,SanFranciscoCenterforHealthcareValueTrainingInitiative. AcadMed.2015;90(4):421–4.

8.KohlwesRJandChouCL.Acurriculuminmedicaleconomicsfor residents. AcadMed.2002;77(5):465–6.

9.DineCJ,MillerJ,FuldA,etal.Educatingphysicians-in-trainingabout resourceutilizationandtheirownoutcomesofcareintheinpatient setting. JGradMedEduc.2010;2(2):175–80.

10.DewanM,HerrmannLE,TchouMJ,etal.Developmentandevaluation ofhigh-valuepediatrics:ahigh-valuecarepediatricresidentcurriculum. HospPediatr.2018;8(12):785–92.

11.SmithCD.Teachinghigh-value,cost-consciouscaretoresidents:the allianceforAcademicInternalMedicine–AmericanCollegeof PhysiciansCurriculum. AnnInternMed.2012;157(4):284–6.

12.MoriatesC,SoniK,LaiA,etal.Thevalueintheevidence: teachingresidentsto “choosewisely.” JAMAInternMed 2013;173(4):308–10.

13.HomJ,KumarA,EvansKH,etal.Ahighvaluecarecurriculumfor interns:adescriptionofcurriculardesign,implementationand housestafffeedback. PostgradMedJ.2017;93(1106):725–9.

14.VarkeyP,MuradMH,BraunC,etal.Areviewofcost-effectiveness, cost-containmentandeconomicscurriculaingraduatemedical education:teachingcost-effectiveness. JEvalClinPract 2010;16(6):1055–62.

15.SchlicherN,HaddockA,SolnickR,etal. EMRAEmergencyMedicine AdvocacyHandbook.5thed.EmergencyMedicineResidents’ Association;2019.

16.ThomasPA,KernDE,HughesMT,etal. CurriculumDevelopmentfor MedicalEducation:ASix-StepApproach, 3rd ed.Baltimore,MD:Johns HopkinsUniversityPress;2016.

17.PattonMQ.Qualitativeresearch.In: EncyclopediaofStatisticsin BehavioralScience;Hoboken,NJ;JohnWiley&Sons,2005.

18.LincolnYSandGubaEG. NaturalisticInquiry.ThousandOaks,CA; SagePublications,Inc.1985.

19.TongA,SainsburyP,CraigJ.Consolidatedcriteriaforreporting qualitativeresearch(COREQ):a32-itemchecklistforinterviewsand focusgroups. IntJQualHealthCare.2007;19(6):349–57.

20.StammenLA,StalmeijerRE,PaternotteE,etal.Trainingphysiciansto providehigh-value,cost-consciouscare:asystematicreview. JAMA 2015;314(22):2384–400.

21.GreysenSR,WassermannT,PayneP,etal.Teachinghealthpolicyto residents three-yearexperiencewithamulti-specialtycurriculum. JGenInternMed.2009;24(12):1322–6.

22.RyskinaKL,SmithCD,AroraVM,etal.Relationshipbetween institutionalinvestmentinhigh-valuecare(HVC)performance improvementandinternalmedicineresidents’ perceptionsofHVC training: AcadMed.2018;93(10):1517–23.

BENEFITS

- Western Journal of Emergency Medicine Subscription - CAL/AAEM News Service email updates - Free and discounted registration to CAL/AAEM events

- And more!

CAL/AAEM NEWS SERVICE

- Healthcare industry news

- Public policy

- Government issues

- Legal cases and court decisions