Western Journal of Emergency Medicine:
Integrating Emergency Care with Population Health Indexed in MEDLINE
Behavioral Health
144
Characteristics for Low, High and Very High Emergency Department Use for Mental Health Diagnoses from Health Records and Structured Interviews
Marie-Josée Fleury, Zhirong Cao, Guy Grenier
Cardiology
155 Bridging the Gap: Evaluation of an Electrocardiogram Curriculum for Advanced Practice Clinicians
Steven Lindsey, Tim P. Moran, Meredith A. Stauch, Alexis L. Lynch, Kristen Grabow Moore
160 Stage B Heart Failure Is Ubiquitous in Emergency Patients with Asymptomatic Hypertension
Kimberly Souffront, Bret P. Nelson, Megan Lukas, Hans Reyes Garay, Lauren Gordon, Thalia Matos, Isabella Hanesworth, Rebecca Mantel, Claire Shubeck, Cassidy Bernstein, George T. Loo, Lynne D. Richardson
166
Performance of Intra-arrest Echocardiography: A Systematic Review
Yi-Ju Ho, Chih-Wei Sung, Yi-Chu Chen, Wan-Ching Lien, Wei-Tien Chang, Chien-Hua Huang
Education
175 Staffing Patterns of Non-ACGME Fellowships with 4-Year Residency Programs: A National Survey
David A. Haidar, Laura R. Hopson, Ryan V. Tucker, Rob D. Huang, Jessica Koehler, Nik Theyyunni, Nicole Klekowski, Christopher M. Fung
181 Changes in Residency Applicant Cancellation Patterns with Virtual Interviews: A Single-site Analysis
Meryll Bouldin, Carly Eastin, Rachael Freeze-Ramsey, Amanda Young, Meredith von Dohlen, Lauren Evans, Travis Eastin, Sarah Greenberger
Volume 25, Number 2, March 2024 Open Access at WestJEM.com ISSN 1936-900X A Peer-Reviewed, International Professional Journal Western Journal of Emergency Medicine VOLUME 25, NUMBER 2, March 2024 PAGES 144-302
continued on page iii West
Contents
Western Journal of Emergency Medicine:
Andrew W. Phillips, MD, Associate Editor DHR Health-Edinburg, Texas
Edward Michelson, MD, Associate Editor Texas Tech University- El Paso, Texas
Dan Mayer, MD, Associate Editor
Retired from Albany Medical College- Niskayuna, New York
Wendy Macias-Konstantopoulos, MD, MPH, Associate Editor Massachusetts General Hospital- Boston, Massachusetts
Gayle Galletta, MD, Associate Editor
University of Massachusetts Medical SchoolWorcester, Massachusetts
Yanina Purim-Shem-Tov, MD, MS, Associate Editor Rush University Medical Center-Chicago, Illinois
Resident Editors
AAEM/RSA
John J. Campo, MD
Harbor-University of California, Los Angeles Medical Center
ACOEP
Justina Truong, DO Kingman Regional Medical Center
Section Editors
Behavioral Emergencies
Leslie Zun, MD, MBA Chicago Medical School
Marc L. Martel, MD Hennepin County Medical Center
Cardiac Care
Fred A. Severyn, MD University of Colorado School of Medicine
Sam S. Torbati, MD
Cedars-Sinai Medical Center
Clinical Practice
Cortlyn W. Brown, MD Carolinas Medical Center
Casey Clements, MD, PhD Mayo Clinic
Patrick Meloy, MD Emory University
Nicholas Pettit, DO, PhD Indiana University
David Thompson, MD University of California, San Francisco
Kenneth S. Whitlow, DO Kaweah Delta Medical Center
Critical Care
Christopher “Kit” Tainter, MD University of California, San Diego
Gabriel Wardi, MD
University of California, San Diego
Joseph Shiber, MD University of Florida-College of Medicine
Matt Prekker MD, MPH Hennepin County Medical Center
David Page, MD University of Alabama
Erik Melnychuk, MD Geisinger Health
Quincy Tran, MD, PhD University of Maryland
Disaster Medicine
John Broach, MD, MPH, MBA, FACEP University of Massachusetts Medical School UMass Memorial Medical Center
Mark I. Langdorf, MD, MHPE, Editor-in-Chief University of California, Irvine School of MedicineIrvine, California
Shahram Lotfipour, MD, MPH, Managing Editor
University of California, Irvine School of MedicineIrvine, California
Michael Gottlieb, MD, Associate Editor Rush Medical Center-Chicago, Illinois
Niels K. Rathlev, MD, Associate Editor Tufts University School of Medicine-Boston, Massachusetts
Rick A. McPheeters, DO, Associate Editor Kern Medical- Bakersfield, California
Gentry Wilkerson, MD, Associate Editor University of Maryland
Christopher Kang, MD Madigan Army Medical Center
Education
Danya Khoujah, MBBS
University of Maryland School of Medicine
Jeffrey Druck, MD University of Colorado
John Burkhardt, MD, MA
University of Michigan Medical School
Michael Epter, DO Maricopa Medical Center
ED Administration, Quality, Safety
Tehreem Rehman, MD, MPH, MBA Mount Sinai Hospital
David C. Lee, MD Northshore University Hospital
Gary Johnson, MD Upstate Medical University
Brian J. Yun, MD, MBA, MPH Harvard Medical School
Laura Walker, MD Mayo Clinic
León D. Sánchez, MD, MPH
Beth Israel Deaconess Medical Center
William Fernandez, MD, MPH University of Texas Health-San Antonio
Robert Derlet, MD
Founding Editor, California Journal of Emergency Medicine
University of California, Davis
Emergency Medical Services
Daniel Joseph, MD Yale University
Joshua B. Gaither, MD
University of Arizona, Tuscon
Julian Mapp
University of Texas, San Antonio
Shira A. Schlesinger, MD, MPH Harbor-UCLA Medical Center
Geriatrics
Cameron Gettel, MD Yale School of Medicine
Stephen Meldon, MD Cleveland Clinic
Luna Ragsdale, MD, MPH Duke University
Health Equity
Emily C. Manchanda, MD, MPH Boston University School of Medicine
Faith Quenzer
Temecula Valley Hospital San Ysidro Health Center
Shadi Lahham, MD, MS, Deputy Editor
Kaiser Permanente- Irvine, California
Susan R. Wilcox, MD, Associate Editor
Massachusetts General Hospital- Boston, Massachusetts
Elizabeth Burner, MD, MPH, Associate Editor
University of Southern California- Los Angeles, California
Patrick Joseph Maher, MD, MS, Associate Editor Ichan School of Medicine at Mount Sinai- New York, New York
Donna Mendez, MD, EdD, Associate Editor
University of Texas-Houston/McGovern Medical School- Houston Texas
Danya Khoujah, MBBS, Associate Editor
University of Maryland School of Medicine- Baltimore, Maryland
Mandy J. Hill, DrPH, MPH
UT Health McGovern Medical School
Payal Modi, MD MScPH
University of Massachusetts Medical
Infectious Disease
Elissa Schechter-Perkins, MD, MPH Boston University School of Medicine
Ioannis Koutroulis, MD, MBA, PhD
George Washington University School of Medicine and Health Sciences
Kevin Lunney, MD, MHS, PhD University of Maryland School of Medicine
Stephen Liang, MD, MPHS Washington University School of Medicine
Victor Cisneros, MD, MPH Eisenhower Medical Center
Injury Prevention
Mark Faul, PhD, MA
Centers for Disease Control and Prevention
Wirachin Hoonpongsimanont, MD, MSBATS
Eisenhower Medical Center
International Medicine
Heather A.. Brown, MD, MPH Prisma Health Richland
Taylor Burkholder, MD, MPH
Keck School of Medicine of USC
Christopher Greene, MD, MPH University of Alabama
Chris Mills, MD, MPH
Santa Clara Valley Medical Center
Shada Rouhani, MD Brigham and Women’s Hospital
Legal Medicine
Melanie S. Heniff, MD, JD Indiana University School of Medicine
Greg P. Moore, MD, JD
Madigan Army Medical Center
Statistics and Methodology
Shu B. Chan MD, MS Resurrection Medical Center
Stormy M. Morales Monks, PhD, MPH Texas Tech Health Science University
Soheil Saadat, MD, MPH, PhD University of California, Irvine
James A. Meltzer, MD, MS
Albert Einstein College of Medicine
Musculoskeletal
Juan F. Acosta DO, MS Pacific Northwest University
Rick Lucarelli, MD Medical City Dallas Hospital
William D. Whetstone, MD
University of California, San Francisco
Neurosciences
Antonio Siniscalchi, MD
Annunziata Hospital, Cosenza, Italy
Pediatric Emergency Medicine
Paul Walsh, MD, MSc
University of California, Davis
Muhammad Waseem, MD Lincoln Medical & Mental Health Center
Cristina M. Zeretzke-Bien, MD University of Florida
Public Health
Jacob Manteuffel, MD
Henry Ford Hospital
John Ashurst, DO Lehigh Valley Health Network
Tony Zitek, MD
Kendall Regional Medical Center
Trevor Mills, MD, MPH
Northern California VA Health Care
Erik S. Anderson, MD
Alameda Health System-Highland Hospital
Technology in Emergency Medicine
Nikhil Goyal, MD
Henry Ford Hospital
Phillips Perera, MD
Stanford University Medical Center
Trauma
Pierre Borczuk, MD
Massachusetts General Hospital/Havard Medical School
Toxicology
Brandon Wills, DO, MS
Virginia Commonwealth University
Jeffrey R. Suchard, MD
University of California, Irvine
Ultrasound
J. Matthew Fields, MD Thomas Jefferson University
Shane Summers, MD Brooke Army Medical Center
Robert R. Ehrman
Wayne State University
Ryan C. Gibbons, MD Temple Health
Volume 25, No. 2: March 2024 i Western Journal of Emergency Medicine Available in MEDLINE, PubMed, PubMed Central, CINAHL, SCOPUS, Google Scholar, eScholarship, Melvyl, DOAJ, EBSCO, EMBASE, Medscape, HINARI, and MDLinx Emergency Med. Members of OASPA. Editorial and Publishing Office: WestJEM/Depatment of Emergency Medicine, UC Irvine Health, 3800 W. Chapman Ave. Suite 3200, Orange, CA 92868, USA Office: 1-714-456-6389; Email: Editor@westjem.org
Integrating Emergency Care with Population Health Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded
Official Journal of the California Chapter of the American College of Emergency Physicians, the America College of Osteopathic Emergency Physicians, and the California Chapter of the American Academy of Emergency Medicine
Western Journal of Emergency Medicine:
Amin A. Kazzi, MD, MAAEM
The American University of Beirut, Beirut, Lebanon
Brent King, MD, MMM University of Texas, Houston
Christopher E. San Miguel, MD Ohio State University Wexner Medical Center
Daniel J. Dire, MD University of Texas Health Sciences Center San Antonio
Douglas Ander, MD Emory University
Edward Michelson, MD Texas Tech University
Edward Panacek, MD, MPH University of South Alabama
Francesco Della Corte, MD Azienda Ospedaliera Universitaria “Maggiore della Carità,” Novara, Italy
Gayle Galleta, MD
Editorial Board
Sørlandet Sykehus HF, Akershus Universitetssykehus, Lorenskog, Norway
Hjalti Björnsson, MD
Icelandic Society of Emergency Medicine
Jaqueline Le, MD Desert Regional Medical Center
Jeffrey Love, MD
The George Washington University School of Medicine and Health Sciences
Katsuhiro Kanemaru, MD University of Miyazaki Hospital, Miyazaki, Japan
Kenneth V. Iserson, MD, MBA University of Arizona, Tucson
Leslie Zun, MD, MBA Chicago Medical School
Linda S. Murphy, MLIS
University of California, Irvine School of Medicine Librarian
Niels K. Rathlev, MD
Tufts University School of Medicine
Pablo Aguilera Fuenzalida, MD
Pontificia Universidad Catolica de Chile, Región Metropolitana, Chile
Peter A. Bell, DO, MBA Baptist Health Sciences University
Peter Sokolove, MD University of California, San Francisco
Rachel A. Lindor, MD, JD Mayo Clinic
Robert Suter, DO, MHA UT Southwestern Medical Center
Robert W. Derlet, MD University of California, Davis
Rosidah Ibrahim, MD Hospital Serdang, Selangor, Malaysia
Scott Rudkin, MD, MBA University of California, Irvine
Scott Zeller, MD
University of California, Riverside
Steven H. Lim, MD
Changi General Hospital, Simei, Singapore
Terry Mulligan, DO, MPH, FIFEM ACEP Ambassador to the Netherlands Society of Emergency Physicians
Wirachin Hoonpongsimanont, MD, MSBATS
Siriraj Hospital, Mahidol University, Bangkok, Thailand
Elena Lopez-Gusman, JD
California ACEP
American College of Emergency Physicians
Jennifer Kanapicki Comer, MD FAAEM
California Chapter Division of AAEM Stanford University School of Medicine
DeAnna McNett, CAE
American College of Osteopathic Emergency Physicians
Kimberly Ang, MBA
UC Irvine Health School of Medicine
Randall J. Young, MD, MMM, FACEP
California ACEP
American College of Emergency Physicians
Kaiser Permanente
Mark I. Langdorf, MD, MHPE, MAAEM, FACEP
UC Irvine Health School of Medicine
Robert Suter, DO, MHA
American College of Osteopathic Emergency Physicians
UT Southwestern Medical Center
Shahram Lotfipour, MD, MPH FAAEM, FACEP
UC Irvine Health School of Medicine
Jorge Fernandez, MD, FACEP
UC San Diego Health School of Medicine
Isabelle Nepomuceno, BS Executive Editorial Director
Visha Bajaria, BS WestJEM Editorial Director
Emily Kane, MA WestJEM Editorial Director
Stephanie Burmeister, MLIS WestJEM Staff Liaison
Cassandra Saucedo, MS Executive Publishing Director
Nicole Valenzi, BA WestJEM Publishing Director
June Casey, BA Copy Editor
Western Journal of Emergency Medicine ii Volume 25, No. 2: March 2024 Available in MEDLINE, PubMed, PubMed Central, Europe PubMed Central, PubMed Central Canada, CINAHL, SCOPUS, Google Scholar, eScholarship, Melvyl, DOAJ, EBSCO, EMBASE, Medscape, HINARI, and MDLinx Emergency Med. Members of OASPA. Editorial and Publishing Office: WestJEM/Depatment of Emergency Medicine, UC Irvine Health, 3800 W. Chapman Ave. Suite 3200, Orange, CA 92868, USA Office: 1-714-456-6389; Email: Editor@westjem.org Official Journal of the California Chapter of the American College of Emergency Physicians, the America College of Osteopathic Emergency Physicians, and the California Chapter of the American Academy of Emergency Medicine
Advisory
Editorial Staff
Board
Integrating Emergency Care with Population Health Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded
Western Journal of Emergency Medicine:
Integrating Emergency Care with Population Health
Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded
JOURNAL FOCUS
Emergency medicine is a specialty which closely reflects societal challenges and consequences of public policy decisions. The emergency department specifically deals with social injustice, health and economic disparities, violence, substance abuse, and disaster preparedness and response. This journal focuses on how emergency care affects the health of the community and population, and conversely, how these societal challenges affect the composition of the patient population who seek care in the emergency department. The development of better systems to provide emergency care, including technology solutions, is critical to enhancing population health.
Table of Contents
186
191
197
Virtual Interviews and the Pediatric Emergency Medicine Match Geography: A National Survey Aline Baghdassarian, Jessica A. Bailey, Derya Caglar, Michelle Eckerle, Andrea Fang, Katherine McVety, Thuy Ngo, Jerri A. Rose, Cindy Ganis Roskind, Melissa M. Tavarez, Frances Turcotte Benedict, Joshua Nagler, Melissa L. Langhan
Analysis of Anonymous Student Narratives About Experiences with Emergency Medicine Residency Programs
Molly Estes, Jacob Garcia, Ronnie Ren, Mark Olaf, Shannon Moffett, Michael Galuska, Xiao Chi Zhang
Rapid Cycle Deliberate Practice Training for Simulated Cardiopulmonary Resuscitation in Resident Education
Jaron D. Raper, Charles A. Khoury, Anderson Marshall, Robert Smola, Zachary Pacheco, Jason Morris, Guihua Zhai, Stephanie Berger, Ryan Kraemer, Andrew D. Bloom
205
209
213
221
Simulation Improves Emergency Medicine Residents’ Clinical Performance of Aorta Point-of-Care Ultrasound
Brandon M. Wubben, Cory Wittrock
Foundations of Emergency Medicine: Impact of a Standardized, Open-access, Core Content Curriculum on In-Training Exam Scores
Jaime Jordan, Natasha Wheaton, Nicholas D. Hartman, Dana Loke, Nathaniel Shekem, Anwar Osborne, P. Logan Weygandt, Kristen Grabow Moore
Integrating Hospice and Palliative Medicine Education Within the American Board of Emergency Medicine Model
Rebecca Goett, Jason Lyou, Lauren R. Willoughby, Daniel W. Markwalter, Diane L. Gorgas, Lauren T. Southerland
The Effect of a Simulation-based Intervention on Emergency Medicine Resident Management of Early Pregnancy Loss
Shawna D. Bellew, Erica Lowing, Leah Holcomb
Emergency Department Operations
226 Root Cause Analysis of Delayed Emergency Department Computed Tomography Scans Arjun Dhanik, Bryan A. Stenson, Robin B. Levenson, Peter S. Antkowiak, Leon D. Sanchez, David T. Chiu
Geriatrics
230 Usability of the 4Ms Worksheet in the Emergency Department for Older Patients: A Qualitative Study Mackenzie A. McKnight, Melissa K. Sheber, Daniel J. Liebzeit, Aaron T. Seaman, Erica K. Husser, Harleah G. Buck, Heather S. Reisinger, Sangil Lee
Volume 25, No. 2: March 2024 iii Western Journal of Emergency Medicine
for peer review, author instructions, conflicts of interest and human and animal subjects protections can be found
at www.westjem.com.
Policies
online
Western Journal of Emergency Medicine:
Integrating Emergency Care with Population Health
Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded
Table of Contents continued
Pediatrics
237 National Characteristics of Emergency Care for Children with Neurologic Complex Chronic Conditions
Kaileen Jafari, Kristen Carlin, Derya Caglar, Eileen J. Klein, Tamara D. Simon
246 Pediatric Outcomes of Emergency Medical Services Non-Transport Before and During the COVID-19 Pandemic
Lori Pandya, Brandon Morshedi, Brian Miller, Halim Hennes, Mohamed Badawy
Research Methodology
254 Development and Validation of a Scoring Rubric for Editorial Evaluation of Peer-review Quality: A Pilot Study
Jeffrey N. Love, Anne M. Messman, Jonathan S. Ilgen, Chris Merritt, Wendy C. Coates, Douglas S. Ander, David P. Way
Ultrasound
264 Novel Scoring Scale for Quality Assessment of Lung Ultrasound in the Emergency Department
Jessica R. Balderston, Taylor Brittan, Bruce J. Kimura, Chen Wang, Jordan Tozer
268 Diagnostic Accuracy of a Handheld Ultrasound vs a Cart-based Model: A Randomized Clinical Trial
Ryan C. Gibbons, Daniel J. Jaeger, Matthew Berger, Mark Magee, Claire Shaffer, Thomas G. Costantino
275 Space Ultrasound: A Proposal for Competency-based Ultrasound Training for In-flight Space Medicine Chanel Fischetti, Emily Frisch, Michael Loesche, Andrew Goldsmith, Ben Mormann, Joseph S. Savage, Roger Dias, Nicole Duggan
282 Ultrasound Performed by Emergency Physicians for Deep Vein Thrombosis: A Systematic Review
Daniel Hercz, Oren J. Mechanic, Marcia Varella, Francisco Fajardo, Robert L. Levine
Women’s Health
291 User Experience of Access to Sexual Assault Nurse Examiner and Emergency Contraception in Emergency Departments in the United States: A National Survey
Colleen Cowdery, Diana Halloran, Rebecca Henderson, MA Kathleen M. Allen, Kelly O’Shea, Kristen Woodward, Susan Rifai, Scott A. Cohen, Muhammad Abdul Baker Chowdhury, Cristina Zeretzke-Bien, Lauren A. Walter, Marie-Carmelle Elie-Turenne
Letters to the Editor
301 Factors Associated with Overutilization of Computed Tomography Cervical Spine Imaging
Tessy La Torre Torres, Jonathan McGhee
302 Reply to “Factors Associated with Overutilization of Computed Tomography Cervical Spine Imaging” Karl Chamberlin
Western Journal of Emergency Medicine iv Volume 25, No. 2: March 2024
Western Journal of Emergency Medicine:
Integrating Emergency Care with Population Health
This open access publication would not be possible without the generous and continual financial support of our society sponsors, department and chapter subscribers.
Professional Society Sponsors
American College of Osteopathic Emergency Physicians
California American College of Emergency Physicians
Academic Department of Emergency Medicine Subscriber
Albany Medical College Albany, NY
Allegheny Health Network Pittsburgh, PA
American University of Beirut Beirut, Lebanon
AMITA Health Resurrection Medical Center Chicago, IL
Arrowhead Regional Medical Center Colton, CA
Baylor College of Medicine Houston, TX
Baystate Medical Center Springfield, MA
Bellevue Hospital Center New York, NY
Beth Israel Deaconess Medical Center Boston, MA
Boston Medical Center Boston, MA
Brigham and Women’s Hospital Boston, MA
Brown University Providence, RI
Carl R. Darnall Army Medical Center Fort Hood, TX
Cleveland Clinic Cleveland, OH
Columbia University Vagelos New York, NY
State Chapter Subscriber
Arizona
Florida
International
Lebanese
California Chapter Division of American Academy of Emergency Medicine
Conemaugh Memorial Medical Center Johnstown, PA
Crozer-Chester Medical Center Upland, PA
Desert Regional Medical Center Palm Springs, CA
Detroit Medical Center/ Wayne State University Detroit, MI
Eastern Virginia Medical School Norfolk, VA
Einstein Healthcare Network Philadelphia, PA
Eisenhower Medical Center Rancho Mirage, CA
Emory University Atlanta, GA
Franciscan Health Carmel, IN
Geisinger Medical Center Danville, PA
Grand State Medical Center Allendale, MI
Healthpartners Institute/ Regions Hospital Minneapolis, MN
Hennepin County Medical Center Minneapolis, MN
Henry Ford Medical Center Detroit, MI
Henry Ford Wyandotte Hospital Wyandotte, MI
INTEGRIS Health Oklahoma City, OK
Kaiser Permenante Medical Center San Diego, CA
Kaweah Delta Health Care District Visalia, CA
Kennedy University Hospitals Turnersville, NJ
Kent Hospital Warwick, RI
Kern Medical Bakersfield, CA
Lakeland HealthCare St. Joseph, MI
Lehigh Valley Hospital and Health Network Allentown, PA
Loma Linda University Medical Center Loma Linda, CA
Louisiana State University Health Sciences Center New Orleans, LA
Louisiana State University Shreveport Shereveport, LA
Madigan Army Medical Center Tacoma, WA
Maimonides Medical Center Brooklyn, NY
Maine Medical Center Portland, ME
Massachusetts General Hospital/Brigham and Women’s Hospital/ Harvard Medical Boston, MA
Mayo Clinic Jacksonville, FL
Mayo Clinic College of Medicine Rochester, MN
Mercy Health - Hackley Campus Muskegon, MI
Merit Health Wesley Hattiesburg, MS
Midwestern University Glendale, AZ
Mount Sinai School of Medicine New York, NY
New York University Langone Health New York, NY
North Shore University Hospital Manhasset, NY
Northwestern Medical Group Chicago, IL
NYC Health and Hospitals/ Jacobi New York, NY
Ohio State University Medical Center Columbus, OH
Ohio Valley Medical Center Wheeling, WV
Oregon Health and Science University Portland, OR
Penn State Milton S. Hershey Medical Center Hershey, PA
Mediterranean Academy of Emergency Medicine
To become a WestJEM departmental sponsor, waive article processing fee, receive electronic copies for all faculty and residents, and free CME and faculty/fellow position advertisement space, please go to http://westjem.com/subscribe or contact:
Stephanie Burmeister
WestJEM Staff Liaison
Phone: 1-800-884-2236
Email: sales@westjem.org
Volume 25, No. 2: March 2024 v Western Journal of Emergency Medicine
Society Partners
Chapter Division of the American Academy of Emergency Medicine
Chapter Division of the American Academy of Emergency Medicine
California
Chapter Division of the American Academy of Emergency Medicine
Lakes
of the American Academy of Emergency Medicine
Chapter Division of the American Academy of Emergency Medicine
Great
Chapter Division
Tennessee
Emergency Medicine Association of Turkey
Academy of Emergency Medicine
Norwegian Society for Emergency Medicine Sociedad Argentina de Emergencias
Sociedad Chileno Medicina Urgencia Thai Association for Emergency Medicine
Uniformed Services Chapter Division of the American Academy of Emergency Medicine
Virginia Chapter Division of the American Academy of Emergency Medicine
Indexed in MEDLINE, PubMed,
Index Expanded
and Clarivate Web of Science, Science Citation
Western Journal of Emergency Medicine:
This open access publication would not be possible without the generous and continual financial support of our society sponsors, department and chapter subscribers.
Professional Society Sponsors
American College of Osteopathic Emergency Physicians
California American College of Emergency Physicians
Academic Department of Emergency Medicine Subscriber
Prisma Health/ University of South Carolina SOM Greenville Greenville, SC
Regions Hospital Emergency Medicine Residency Program St. Paul, MN
Rhode Island Hospital Providence, RI
Robert Wood Johnson University Hospital New Brunswick, NJ
Rush University Medical Center Chicago, IL
St. Luke’s University Health Network Bethlehem, PA
Spectrum Health Lakeland St. Joseph, MI
Stanford Stanford, CA
SUNY Upstate Medical University Syracuse, NY
Temple University Philadelphia, PA
Texas Tech University Health Sciences Center El Paso, TX
The MetroHealth System/ Case Western Reserve University Cleveland, OH
UMass Chan Medical School Worcester, MA
University at Buffalo Program Buffalo, NY
State Chapter Subscriber
University of Alabama Medical Center Northport, AL
University of Alabama, Birmingham Birmingham, AL
University of Arizona College of Medicine-Tucson Tucson, AZ
University of California, Davis Medical Center Sacramento, CA
University of California, Irvine Orange, CA
University of California, Los Angeles Los Angeles, CA
University of California, San Diego La Jolla, CA
University of California, San Francisco San Francisco, CA
UCSF Fresno Center Fresno, CA
University of Chicago Chicago, IL
University of Cincinnati Medical Center/ College of Medicine Cincinnati, OH
University of Colorado Denver Denver, CO
University of Florida Gainesville, FL
University of Florida, Jacksonville Jacksonville, FL
California Chapter Division of American Academy of Emergency Medicine
University of Illinois at Chicago Chicago, IL
University of Iowa Iowa City, IA
University of Louisville Louisville, KY
University of Maryland Baltimore, MD
University of Massachusetts Amherst, MA
University of Michigan Ann Arbor, MI
University of Missouri, Columbia Columbia, MO
University of North Dakota School of Medicine and Health Sciences Grand Forks, ND
University of Nebraska Medical Center Omaha, NE
University of Nevada, Las Vegas Las Vegas, NV
University of Southern Alabama Mobile, AL
University of Southern California Los Angeles, CA
University of Tennessee, Memphis Memphis, TN
University of Texas, Houston Houston, TX
University of Washington Seattle, WA
University of WashingtonHarborview Medical Center Seattle, WA
University of Wisconsin Hospitals and Clinics Madison, WI
UT Southwestern Dallas, TX
Valleywise Health Medical Center Phoenix, AZ
Virginia Commonwealth University Medical Center Richmond, VA
Wake Forest University Winston-Salem, NC
Wake Technical Community College Raleigh, NC
Wayne State Detroit, MI
Wright State University Dayton, OH
Yale School of Medicine New Haven, CT
International
Lebanese
Mediterranean
To become a WestJEM departmental sponsor, waive article processing fee, receive electronic copies for all faculty and residents, and free CME and faculty/fellow position advertisement space, please go to http://westjem.com/subscribe or contact:
Stephanie Burmeister
WestJEM Staff Liaison
Phone: 1-800-884-2236
Email: sales@westjem.org
Western Journal of Emergency Medicine vi Volume 25, No. 2: March 2024
Society Partners
Chapter Division of the American Academy of Emergency Medicine California Chapter Division of the American Academy of Emergency Medicine
Chapter Division of the American Academy of Emergency Medicine
Lakes Chapter Division of the American Academy of Emergency Medicine
Chapter Division of the American Academy of Emergency Medicine
Arizona
Florida
Great
Tennessee
Emergency Medicine Association of Turkey
Academy of Emergency Medicine
Academy of Emergency Medicine Norwegian Society for Emergency Medicine Sociedad Argentina de Emergencias
Sociedad Chileno Medicina Urgencia Thai Association for Emergency Medicine
Medicine
Uniformed Services Chapter Division of the American Academy of Emergency Medicine Virginia Chapter Division of the American Academy of Emergency
Integrating Emergency
Population
Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded
Care with
Health
Save the Date Spring Seminar 2024 | April 27 - May 1 Signia Orlando Bonnet Creek • Orlando, Florida #ACOEP24
ORIGINAL RESEARCH
CharacteristicsforLow,HighandVeryHighEmergency DepartmentUseforMentalHealthDiagnosesfromHealth RecordsandStructuredInterviews
Marie-JoséeFleury,PhD*†
ZhirongCao,MSc†
GuyGrenier,PhD†
*McGillUniversity,DepartmentofPsychiatry,Montreal,Canada
† DouglasMentalHealthUniversityResearchCentre,Montreal,Canada
SectionEditors:BradBobrin,MD,andYaninaPurim-Shem-Tov,MD,MS
Submissionhistory:SubmittedMay24,2023;RevisionreceivedNovember17,2023;AcceptedNovember22,2023
ElectronicallypublishedFebruary9,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18327
Introduction: Patientswithmentalhealthdiagnoses(MHD)areamongthemostfrequentemergency department(ED)users,suggestingtheimportanceofidentifyingadditionalfactorsassociatedwiththeir EDusefrequency.Inthisstudyweassessedvariouspatientsociodemographicandclinical characteristics,andserviceuseassociatedwithlowEDusers(1–3visits/year),comparedtohigh(4–7) andveryhigh(8+)EDuserswithMHD.
Methods: OurstudywasconductedinfourlargeQuebec(Canada)EDnetworks.Atotalof299patients withMHDwererandomlyrecruitedfromtheseEDin2021–2022.Structuredinterviewscomplemented datafromnetworkhealthrecords,providingextensivedataonparticipantprofilesandtheirqualityof care.WeusedmultivariablemultinomiallogisticregressiontocomparelowEDusetohighandveryhigh EDuse.
Results: Overa12-monthperiod,39%ofpatientswerelowEDusers,37%high,and24%veryhighED users.ComparedwithlowEDusers,thoseatgreaterprobabilityforhighorveryhighEDuseexhibited moreviolent/disturbedbehaviorsorsocialproblems,chronicphysicalillnesses,andbarrierstounmet needs.Patientspreviouslyhospitalized1–2timeshadlowerriskofhighorveryhighEDusethanthose notpreviouslyhospitalized.ComparedwithlowEDusers,highandveryhighEDusersshowedhigher prevalenceofpersonalitydisordersandsuicidalbehaviors,respectively.Womenhadgreaterprobability ofhighEDusethanmen.PatientslivinginrentalhousinghadgreaterprobabilityofbeingveryhighED usersthanthoselivinginprivatehousing.Usingatleast5+ primarycareservicesandbeingrecurrentED userstwoyearspriortothelastyearofEDusehadincreasedprobabilityofveryhighEDuse.
Conclusion: FrequencyofEDusewasassociatedwithcomplexissuesandhigherperceived barrierstounmetneedsamongpatients.VeryhighEDusershadmoresevererecurrentconditions,such asisolationandsuicidalbehaviors,despiteusingmoreprimarycareservices.Results suggestedsubstantialreductionofbarrierstocareandimprovementonbothaccessandcontinuity ofcareforthesevulnerablepatients,integratingcrisisresolutionandsupportedhousing services.Limitedhospitalizationsmaysometimesbeindicated,protectingagainstED use.[WestJEmergMed.2024;25(2)144–154.]
Keywords: emergencydepartment;frequencyofemergencydepartmentvisits;lowserviceusers; highserviceusers;veryhighserviceusers;mentalhealthdiagnoses;probabilityfactors; associatedvariables.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 144
INTRODUCTION
Emergencydepartment(ED)crowdingisamajor impedimenttotheefficacyofhealthcaresystems,1 causedin partbyaminorityofpatientswhousetheEDfrequently.2
Accordingtoa2019systematicreview,theestimated prevalenceofhighEDuserswas4-16%,yetthesepatients accountedfor14–47%ofallEDvisits,averaging6.9ED visitsperyear.3 HighEDusers,commonlydefinedashaving 4+ EDvisitsina12-monthperiod,4,5 aremorelikelythan otherpatientstobehospitalizedfrequently6 andhave2.2 timesgreaterprobabilityofdeaththanotherEDusers accordingtoa2015systematicreview.7 Mentalhealth diagnoses(MHD),includingsubstance-relateddisorders (SRD),areveryprevalentamonghighEDusers.1,4,8 Another 2013reviewreportedthatbetween0.3–18%ofpatientswith MHDwerefrequentEDusers.8 A2019Canadianstudy showedthatQuebecpatientswithMHDhadusedtheED roughlytwiceasoftenaspatientswithoutMHD,and17%of thesepatientswerehighEDusersin2015-16.9 AstheEDis notanappropriatesettingfortreatingrecurrentpatientswith MHD,theidentificationofhighEDusersandtheir characteristicsiskeytoimprovingcareamongthese vulnerablepatientsandforreducingcrowdingand healthcarecostsintheED,giventhatEDuseisoneofthe costliestcomponentsofhealthcare.10
Severalstudieshaveassessedpatientcharacteristics associatedwithhighEDuseamongpatientswithMHD, mostcomparinghighEDusersvsotherEDusers.11–17 The sociodemographiccharacteristicsdistinguishinghighED usersfromotherEDusersincludedbeingmale,15 younger, 14 single,16 havingpublichealthinsurance,11,12 andlivingin moresociallyormateriallydeprived15,18 ormetropolitan15 areas.Personalitydisorders,11,13,15,16 seriousMHD15,17 or SRD,5,17 andhavingchronicphysicalillnesses12 werethe mainclinicalcharacteristicsassociatedwithhighEDuse. HighEDusersalsodifferedfromotherEDusersintermsof higheroveralluseofmentalhealthservices.15,19,20 Toour knowledge,fewstudieshavecomparedsubgroupsoflow, high,andveryhighEDusersamongpatients.1,21 Those studieshavefocusedonMHDtoexplainthefrequencyof EDuse,includingpatientswithmultipleconditionsand withSRD,asthemainfactorleadingtoincreaseduse. VeryhighEDusersalsoreportedmorerecurrentEDusein previousyears.22 Yet,howthefrequencyofEDusewas categorizeddifferedgreatlyamongthesestudies: “veryhighEDuse” couldbeanywherebetween 8+1 and18+ visits/year.21
Abetterunderstandingofpatientcharacteristics associatedwithlow,high,andveryhighEDusersmayhelp tailorinterventionsandprogramstoEDprofilesandreduce EDuse,particularlyforhighandveryhighusers.Wefound nopreviousresearchcomparinglowEDuserstohighand veryhighusersamongpatientswithMHDorSRD.Also, moststudieswerebasedsolelyonsingle-sitehospitalhealth
PopulationHealthResearchCapsule
Whatdowealreadyknowaboutthisissue?
Emergencydepartment(ED)crowdingisa majorimpedimenttotheef fi cacyof healthcaresystems,causedinpartby aminorityofpatientswhousethe EDfrequently.
Whatwastheresearchquestion?
Wesoughttoassesspatients ’ characteristics andserviceusepatternsassociatedwithlow, highandveryhighEDusers.
Whatwasthemajor findingofthestudy?
Violent/disturbedbehaviorsorsocial problemsincreased5.55timestheprobability ofveryhighEDuse.
Howdoesthisimprovepopulationhealth?
Areductionofbarrierstocareandbetter accessandcontinuityofoutpatientcare shouldbeprovidedforthemost vulnerablepatients.
records.Ourstudyisoriginalinthatitintegratespatient structuredinterviewswithhealthrecordsfromfourlarge mentalhealthnetworksthatincludehospitalsand community-basedservices.VeryfewstudiesonEDuse integrateoveralloutpatientserviceuse,fromprimaryto specializedcare,andassesshowtheseservicesrelateto patientEDusefrequency.22 Moreover,fewstudieshave testedassociationsbetweenEDusefrequencyandquality ofoutpatientcareormotivationalbehaviors,suchas satisfactionwithcare,unmetneedsorperceivedstigma thatmaytriggerEDuse.
Basedontheliterature,wehypothesizedthatveryhigh EDusers,followedbyhighEDusers,wouldbemorelikely thanlowEDuserstohavecomplexhealthandsocialissues andunmetneeds,andtouseoutpatientcaremorefrequently. Weassessedvariouspatientsociodemographicandclinical characteristics,andserviceusepatternsassociatedwithlow EDuserswithMHD(1–3visits/year),comparedwithhigh EDusers(4–7visits)andveryhighEDusers(8+ visits)in fourlargeEDnetworksinQuebec(Canada).
METHODS
DescriptionoftheQuebecMentalHealthSystem
InCanada,allresidentsarecoveredbyauniversalhealth insurancemanagedattheprovinciallevel.23 Mentalhealth
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 145
Fleuryetal.
PatientCharacteristicsAssociatedwithEDUseforMHD
services,includingmedication,aremainlypublic,except servicessuchaspsychologicalservices,whichareusually paidbytheuserorcoveredbysomeemployers.Quebec publichealthcareservicesaremainlymanagedthrough22 largenetworks,integratinghospitals,long-termand addictionfacilities,andcommunityhealthcarecenters.24 In thesenetworks,specializedmentalhealthcareisprovidedin psychiatricdepartmentsofgeneralhospitalsorinpsychiatric hospitals,orinspecializedaddictiontreatmentcenters.25 HospitalEDstaffincludespecializedorgeneralemergency physicians,psychiatrists,andpsychosocialclinicians mostlynursesandsomesocialworkersandaddiction specialists.Primarymentalhealthcareisofferedinmedical clinicsstaffedbygeneralpractitioners,incommunity healthcarecentersmainlyprovidingpsychosocialservices, andbypsychologistsmostlyworkinginprivatepractice. Community-basedorganizations,thevoluntarysector, integratecrisisandsuicidepreventioncenters,detoxcenters, andpeersupportgroups.
StudySettingsandDataCollection
ThestudywasconductedinfourEDnetworksserving abouttwomillionpeople roughlyone-fourthofQuebec’ s population.StudyparticipantshadtobeEDusers,18+ years old,abletocompleteastructuredinterview,knowFrenchor English,andhadtogranttheresearchteamaccesstotheir healthrecords.Studyparticipantswererecruitedrandomly byEDstaffbasedonahealthrecordlistof1,751EDusers whohadMHD,includingSRD,andhadusedtheEDatleast oncewithinthefourEDnetworksinthe12monthspreceding recruitment.Ofthe first563eligiblepatientsreached,450 (80%)agreedtobereferredtotheresearchteamfor considerationasstudyparticipants.Theywerethen contactedbytheresearchcoordinatorandaskedtotakepart inastructuredtelephoneinterview,donebytrained interviewerscloselymonitoredbytheresearchteam.
TheseinterviewswereadministeredbetweenMarch1, 2021–May13,2022.Averagecompletiontimewas45 minutes.Healthrecordsforthe12monthspriortointerviews werecollectedtocomplementinterviewdata,exceptfor previousEDuse,whichwasmeasuredwithinthetwoyears priortothelastyearofEDuse.Healthrecordsdata concernedEDuse(Banquededonnéescommunesdes urgences[BDCU]database),psychiatricoutpatientservices used,hospitalization(MED-ÉCHOdatabase),and psychosocialservicesfromcommunityhealthcarecenters(ICLSCdatabase).PatientdiagnoseswereincludedinBDCU andMED-ÉCHO,andframedbytheInternational ClassificationofDiseases,Canada,10th Rev(Appendix).All healthrecordsincludedinformationonpatientserviceuse (eg,type,frequency)butexclusivelywithintheEDnetwork. Validatedbyasteeringcommitteeintegratingclinicians, structuredinterviewdataconsideredserviceuseoutsideED networksandservicesnotincludedinhealthrecords
(eg,medicalclinics,psychologists).Thesemergeddata allowedforabroaddatasetonpatientserviceuseandother patientcharacteristicspriortorecruitment.Participationin thestudywasvoluntary.Patientswhoprovidedconsent receivedamodest financialcompensation.Themultisite protocolwasapprovedbytheethicsreviewboardofthe DouglasMentalHealthUniversityInstitute.
StudyVariables
ThedependentvariablewasEDusefrequencyformental healthreasonsamongpatientswithMHD,measured12 monthspriortointerviews.Patientswerecategorizedaslow EDusers(1–3visits/year),highEDusers(4–7visits/year)or veryhighEDusers(8+ visits/year).Thestandarddefinition ofhighEDuseis4+ times/year,11,12,26 whileveryhighuse wasdefinedas8+ times/yearbasedonprevious1,27 studies andonaminimaldistributionofveryhighEDvisitsinthe studysample.Independentvariablesweresociodemographic characteristics,clinicalcharacteristics,andserviceuse patterns,againbasedonpreviousresearch.21,28
Sociodemographiccharacteristicsincludedthefollowing: sex;agegroup;educationlevel;civilstatus;employment status(eg,worker,unemployed);householdincome($Can); typeofhousing(eg,supervised);numberofsignificantsocial supportnetwork;andstigma.Allexcept “ agegroup ” were determinedbyinterviewdata.BasedontheCanadian CommunityHealthSurvey(CCHS),socialsupportwas measuredwiththefollowingquestion: “Doyouhaveoneor morepeoplearoundyouonwhomyoucanrelyforhelpwith problems?Ifyes,howmanypeople?” Alsobasedonthe CCHS,ona5-pointscale,withresponsesrangingfrom “totallydisagree” to “totallyagree” (greatest stigmatization),stigmawasmeasuredwiththefollowing affirmation: “Mostpeopleinmycommunitytreataperson withaMHDorSRDinthesamemannerastheywouldtreat anyotherperson.”
Clinicalcharacteristicsincludedthefollowing:MHD; SRD;suicidalbehaviors(suicideideationorattempt); violent/disturbedbehaviorsorsocialproblems;chronic physicalillnesses(eg,heartdiseases,diabetes);co-occurring MHD-SRD;andhightriagepriorityamongEDusers.All thesevariableswerebasedonhealthrecords,exceptSRD, whichwasbasedonbothhealthrecordsandthestructured interviews.TheMHDincludedseriousMHD(schizophrenia spectrumandotherpsychoticdisorders,andbipolar disorders),personalitydisorders,andcommonMHD (anxiety,depressiveandadjustmentdisorders;attention deficit/hyperactivitydisorder).TheSRDintegratedalcoholanddrug-relateddisorders(use,induced,intoxicationand withdrawal),measuredusinghealthrecordsalongwiththe AlcoholUseDisordersIdentificationTest29 andtheDrug AbuseScreeningTest-20.30 Thesewereincludedinthe structuredinterviews,asSRDareoftenunderdiagnosedin healthrecords.31 Weidentifiedchronicphysicalillnessesand
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 146 PatientCharacteristicsAssociatedwithEDUseforMHD Fleuryetal.
theirseverity(0to2+)basedonanadaptedversion integratingboththeCharlsonandElixhausercomorbidity indexes.32 TheEDtriageprioritywasbasedontheCanadian TriageAcuityScale,33 consistingof fiveprioritylevelsor illnessseverity,withlevels4–5consideredtreatablein outpatientcare.33 Inthisstudy,hightriagepriorityEDuse (1–3)wasconsideredaproxyforfunctionaldisability,based onmeanofnumberofEDvisitsperpatient,with1–3triage prioritydividedbytotalofEDvisitsperpatient(1–5).
Patientserviceuseincludedthefollowing:knowledgeof mentalhealthoraddictionservices;havingafamilydoctoror otherregularcareclinician;frequencyofprimarycare, community-based,andspecializedoutpatientservicesused; overallsatisfactionwithoutpatientservicesused;numberof barriersrelatedtounmetneeds;frequencyofhospitalization, andfrequencyofpreviousEDuse.Patientserviceuseinthe EDnetworks,mostlymentalhealthspecializedcareand someprimarycareservices(communityhealthcarecenters), wasbasedonhealthrecords,andservicesoutsidetheED networkswerereportedinthestructuredinterviews mostly primarycare,community-based,orspecializedaddiction services.Serviceusemeasuredwithbothtypesofdata integratedonlythehighestfrequencyofserviceuse patientsreported.Asaproxyofcontinuityofcare,patients wereaskediftheywerefollowedregularlybyafamilydoctor orotherclinicians.Basedonapreviousstudy,34 the benchmarkforfrequentserviceuse,orminimalintensityof optimalcare,was5+ follow-upappointments/year.Primary careincludedservicesreceivedfromfamilydoctors,general practitionersinwalk-inclinics,psychologistsinprivate practice,andpsychosocialcliniciansincommunity healthcarecenters.
Community-basedorganizationsintegratedcrisisand suicidepreventioncenters,etc.Specializedoutpatientcare includedpsychiatricservices(eg,treatmentfrompsychiatrist teams,assertivecommunitytreatment,andintensivecase managementprograms),andservicesfromaddiction treatmentcenters.Patientswereaskedtoindicateona 5-pointscaletheiryearlysatisfactionwitheachoutpatient servicereceived.Wecalculatedthemeansatisfactionscore, withhigherscoresindicatinggreatersatisfaction.Unmet needsweremeasuredthroughthefollowingCCHSquestion:
“Couldyouexplainthereasonswhyservicesoutsideofthe EDdidnotrespondtoyourneeds?” includingmultiple choiceofbarrierstocare(eg, “Iprefertomanagebymyself;” “Thehelpisnotreadilyavailable”).Thenumberofbarriers wascountedas0,1–2,or3+.FrequencyofpreviousEDuse included4–7(highEDusers)and8+ EDvisits(veryhighED users),measuredforthetwo-yearperiodprecedingthe 12-monthinterviewperiod.
Analyses
Missingvalues(<1%)wereimputedbymeanfor continuousvariablesandmodeforcategoricalvariables.35
Descriptiveanalysesincludedpercentagesforcategorical variablesandmeanvaluesforcontinuousvariables.Weused bivariatemultinomiallogisticregressiontoexaminethe associationsbetweeneachindependentvariableandthe dependentvariable,frequencyofEDuse.Theintraclass correlationcoefficient(ICC)forthestudywassmall(<0.01), indicatinglowsharedvarianceamongpatientsfromtheED networks;multilevelanalysiswasnotrequired.Basedon criterionproceduresforforwardmodelselection, independentvariablesidentifiedassignificantinthebivariate analyses(Alpha:0.20)36 wereenteredsequentiallyintothe multivariablemultinomiallogisticregressionmodelfor frequencyofEDuse,withlowEDuse(1–3visits/year)asthe referencegroup.WeusedtheAkaikeInformationCriterion (AIC)37 tocomparetherelativegoodnessof fitamong differentmodelsbeforeselectingthe finalmultivariatemodel withthesmallestAICthatbest fitthedata.Wealsoused varianceinflationfactor(VIF)tomeasuretheamountof multicollinearityinregressionanalysisandfoundsmaller than4,indicatingthatmulticollinearitywasnotaconcern.38 Relativeriskratios(RRR)and95%confidenceintervals(CI) werecalculatedinthe finalmodel.Weperformedstatistical analysesusingStata17(StataCorpLLC,College Station,TX).
RESULTS
Ofthe450EDusersreferred,50couldnotbereachedand 300agreedtoparticipateinthestudy(75%responserate). Onepatientwaswithdrawn.Ofthe299patientsinthe final sample,amajority(55%)werewomen;39%were30–49years old,82%single,and57%unemployedorretired;47%hada householdincomeoflessthanCAN$20,000;57%hadpostsecondaryeducation,58%livedinrentalhousing,and50% perceivedhighstigma(Table1).Overhalf(57%)had commonMHD,44%seriousMHD,42%personality disorders,59%SRD,and45%chronicphysicalillnesses;38% hadco-occurringMHD-SRD,54%suicidalbehaviors,and 17%violent/disturbedbehaviorsorsocialproblems.Interms ofEDuse,39%werelowEDusers(1–3visits/year),37%high EDusers(4–7visits/year),and24%veryhighEDusers(8+ visits/year)(Table2).Nearlyhalf(46%)hadpoortofair knowledgeofmentalhealthoraddictionservices;88%hada familydoctor(74%)orotherregularcareclinician(58%).In thepreviousyear,58%hadused5+ primarycareservices, 26%5+ servicesfromcommunity-basedorganizations,and 65%5+ specializedoutpatientcare.Overallsatisfactionwith outpatientservicesaveraged4.02/5;37%ofparticipantshad unmetneeds,with15%identifying3+ barriers.Amajority (56%)werehospitalized,35%ofthose1–2times, and39%hadbeenveryhighEDusersovertheprevious two-yearperiod.
Wecomparedvariablesassociatedwithhighorveryhigh EDuserswithvariablesamonglowEDusers(Table 3). Womenhad1.30timesmoreprobabilityofbeinghighED
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 147 Fleuryetal. PatientCharacteristicsAssociatedwithEDUseforMHD
Table1.
1Patientstructuredinterviews. 2Banquededonnéescommunesdesurgences (BDCU,EDdatabase). 3Thesamplewastoosmalltoseparate unemployedfromretired. 4Supervisedhousingincludedgrouphomes,residentialcare,supportedapartments,etc. 5Maintenanceet exploitationdesdonnéespourl’étudedelaclientèlehospitalière (MED-ÉCHO,hospitalizationdatabase). 6Patientsmayhavemorethan oneMHD. 7AlcoholUseDisordersIdenti ficationTest(AUDIT). 8DrugAbuseScreeningTest-20(DAST-20).Detailsofdiagnosticcodes arepresentedinthe Appendix ED,emergencydepartment.
Group LowED users(1–3 visits/year) HighED users(4–7 visits/year) Veryhigh EDusers (8+ visits/ year)Total Bivariate analysis 11739.1310936.457324.41299100 n%n%n%n% Size(N)meanSDmeanSDmeanSDmeanSD P-value Sociodemographiccharacteristics(measuredintheprevious12months) Women1 5345.36963.34358.916555.18 <0.20 Age2 18–29years3025.643633.032635.629230.77 <0.20 30–49years4841.034137.612838.3611739.13 50+ years3933.333229.361926.039030.1 Educationlevel1 Highschoolorless4841.035045.873243.8413043.48 ≥0.2 Post-secondaryeducation6958.975954.134156.1616956.52 Civilstatus1 Single(includingseparated, divorced,orwidowed) 9278.638981.656589.0424682.27 <0.20 Incouple2521.372018.35810.965317.73 Employmentstatus1 Workerorstudent5849.574137.613142.4713043.48 ≥0.20 Unemployedorretired3 5950.436862.384257.5316956.52 Householdincome (Can$/year)1 0–$19,9995446.155247.713547.9514147.16 <0.20 $20,000–$39,9993025.643834.862128.768929.77 $40,000+ 3328.211917.431723.296923.07 Typeofhousing1 Private2823.932522.9479.596020.07 <0.20 Rental6353.856357.84764.3817357.86 Supervised4 2622.222119.271926.036622.07 Numberofsignificantsocialsupportnetwork(mean/SD)1 3.523.193.615.083.635.403.584.51 ≥0.20 Stigma1 High5647.865651.383750.6814949.83 ≥0.20 Medium2319.661917.431216.445418.06 Low3832.483431.192432.889632.11 Clinicalcharacteristics(measuredintheprevious12months) Seriousmentalhealthdiagnoses(MHD)2,5,6 5547.014137.613750.6813344.48 <0.20 Personalitydisorders2,5,6 3126.505247.714460.2712742.47 <0.20 CommonMHD2,5,6 6152.146458.724460.2716956.52 ≥0.20 Substance-relateddisorders1,2,5,7,8 6252.996559.634865.7517558.53 <0.20 Suicidalbehaviors(suicideideationorattempt)2,5 4437.616357.805473.9716153.85 <0.20 Violent/disturbedbehaviorsorsocialproblems2 97.692119.272027.405016.72 <0.20 Chronicphysicalillnesses2,5 3832.484844.045068.4913645.48 <0.20 Severityofchronicphysical illnesses2,5 09379.497266.063041.119565.22 <0.20 11512.821816.512736.996020.07 2+ 97.691917.431621.924414.72 Co-occurringMHD-SRD1,2,5,7,8 3529.914339.453547.9511337.79 <0.20 Percentageofhighpriorityin EDtriage2 0–33%1916.242018.35912.334816.05 ≥0.20 34%–66%2420.512926.612230.147525.08 67%–100%7463.256055.054257.5317658.86
Sociodemographicandclinicalcharacteristicsofpatientsusingtheemergencydepartment(N = 299).
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 148 PatientCharacteristicsAssociatedwithEDUseforMHD Fleuryetal.
Serviceuse(measuredintheprevious12months,orotherasspecified)
Group
LowED users(1–3 visits/year)
HighED users(4–7 visits/year)
Veryhigh EDusers (8+ visits/
1Seenote 1belowTable 1 2Seenote 2belowTable 1 3Systèmed’informationpermettantlagestiondel’informationcliniqueetadministrative dansledomainedelasantéetdesservicessociaux (I-CLSC,communityhealthcarecenterdatabase). 4Psychiatricoutpatientservicesused database. 5BasedontheCCHS,barrierstocareexplainingunmetneedswerea)Ipreferredtomanagebymyself;b)Ihaven’tgottenaround toityet(eg,toobusy);c)Ididn’thaveenoughconfidenceinthehealthcaresystemorsocialservices;d)Iwasafraidaboutwhatotherswould thinkofme;e)Ipreferredtoaskmyfamilyorfriendsforhelp;f)Iamdissatisfiedwiththequalityofservices;g)Idon’tknowhoworwheretoget thiskindofhelp;h)Myjobinterferedwithpossibletreatment(eg,hoursofwork);i)Thehelpisnotreadilyavailable;j)Icouldnotaffordtopay; myinsurancedidn’tcoverthecost;andk)Servicesarenotofferedinmylanguage. 6Seenote 5belowTable 1 ED,emergencydepartment.
usersthanmen.Patientslivinginrentalhousinghad2.09 timesmoreprobabilityofbeingveryhighEDusersthan thoselivinginprivatehousing.Patientsexhibitingviolent/ disturbedbehaviorsorsocialproblems,orchronicphysical illnesses,respectively,showed2.87and1.02timesincreasein probabilityofhighEDuse,anda5.55and4.95timesgreater probabilityofveryhighEDuse.Patientswithpersonality disordershad1.06timesgreaterprobabilityofhighEDuse, andthosewithsuicidalbehaviors,a1.29increased
probabilityofveryhighEDuse.Patientswith3+ barriers relatedtounmetneedshad1.64and2.27timesgreater probabilityofbeinghighorveryhighEDusers, respectively.Patientswith5+ primarycareservicesandhigh recurrentEDusehad2.5and1.53timesgreaterprobability ofbeingveryhighEDusers.Patientshospitalized 1–2timeshadareducedprobabilityof54%forhigh and79%forveryhighEDuse,comparedwiththose nothospitalized.
Table2. Serviceuseofpatientsusingtheemergencydepartment(N=299).
Bivariate analysis 11739.1310936.457324.41299100 n%n%n%n% P-value Size(N)meanSDmeanSDmeanSDmeanSD Verygoodtoexcellentknowledgeofmentalhealth oraddictionservices1 5950.436357.803953.4216153.85 ≥0.2 Havingafamilydoctororotherregularcareclinician1–3 10287.189688.076690.4126488.29 <0.20 Frequencyofprimarycare serviceuse1 02521.372220.1856.855217.39 <0.20 1–42924.793229.361419.187525.08 5+ 6353.855550.465473.9717257.53 Frequencyofserviceuseof community-basedorganizations1,3 06858.125146.792939.7314849.50 <0.20 1–42420.513330.281621.927324.41 5+ 2521.372522.942838.367826.09 Frequencyofspecializedoutpatient careuse1,4 01916.242018.351216.445117.06 <0.20 1–42823.931816.51912.335518.39 5+ 7059.837165.145271.2319364.55 Overallsatisfactionwithoutpatientservicesused(mean/SD)1 4.180.703.980.773.830.814.020.76 <0.20 Numberofbarriersrelatedto unmetneeds1,5 08169.236660.554156.1618862.88 <0.20 1–22420.512422.021723.296521.74 3+ 1210.261917.431520.554615.38 Frequencyofhospitalizations1,6 05446.154743.123041.113143.81 <0.20 1–25042.743733.941824.6610535.12 3+ 1311.112522.942534.256321.07 FrequencyofpreviousEDuse (measuredwithinthe2yearspriorto the12-monthperiodinwhich interviewswereconducted)1,2 0–34538.463733.941419.189632.11 <0.20 4–7(highEDusers)4437.613128.441115.078628.76 8+ (veryhighEDusers)2823.934137.614865.7511739.13
year)Total
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 149 Fleuryetal. PatientCharacteristicsAssociatedwithEDUseforMHD
HighEDusers (4–7visits/year)
VeryhighEDusers (8+ visits/year)
RRR* P-value95%CI*RRR* P-value95%CI* Sociodemographiccharacteristics(measuredintheprevious12months)
Womenvsmen2.300.0071.254.231.480.3070.703.16
Typeofhousing1
Rentalvsprivate1.430.3260.702.943.090.0361.088.85
Supervisedvsprivate0.810.6310.341.942.180.2000.667.18
Clinicalcharacteristics(measuredintheprevious12months)
Personalitydisorders2.040.0391.044.012.260.0550.985.18
Suicidalbehaviors(suicideideationorattempt)1.810.0630.973.382.290.0461.015.16
Violent/disturbedbehaviorsorsocialproblems3.870.0051.529.856.550.0012.2619.00
Chronicphysicalillnesses2.020.0431.024.005.950.0002.5014.13
Serviceuse(measuredintheprevious12months,orotherasspecified)
Frequencyofprimarycareserviceuse
1
FrequencyofpreviousEDuse(measuredwithinthe2yearspriortothe12-monthperiod inwhichinterviewswereconducted)
ED,emergencydepartment;*RRR,relativeriskratio; CI,confidenceinterval. 1Seenote 4belowTable 1 2Seenote 5belowTable 2
DISCUSSION
Inthisstudyweaimedtoidentifysociodemographicand clinicalcharacteristics,aswellasserviceuse,amongpatients withMHD,comparinglow(1–3visits/year)tohigh(4–7 visits)andveryhighEDuse(8+ visits)formentalhealth reasons.Mostpatientshadhigh(37%)orveryhigh(24%)ED use,whichmaybeexplainedbythesubstantialsocialand healthissuestheyfaced.Theirlevelsofsocialandmaterial deprivationwerehigh,aswastheirperceivedstigma.Nearly halfhadseriousMHD,personalitydisordersorchronic physicalillnesses,whilemostexperiencedSRDandsuicidal behaviors.About40%reportedunmetneedsorpoorto fairknowledgeofservices,whichmayexplaintheirhigh overallEDuse.Asfoundinotherstudies,13,28 mosthighED userswerealsohighusersofoutpatientcareandwere frequentlyhospitalized.
Findingspartlyconfirmedthehypothesesthatveryhigh EDusers,followedbyhighEDusers,weremorelikelythan
lowEDuserstohavecomplexhealthandsocialissues,unmet needs,andtomakemorefrequentuseofoutpatientcare.The result showingthatdisturbed/violentbehaviorsorsocial problemswerethepatientcharacteristicsmoststrongly associatedwithbothveryhighandhighEDuse underlined thespecialneedsofthesepatients,whoforsomewerelikely involuntaryEDusers.Policearefrequentlycalledintodeal withpeoplepresentingviolentorerraticbehaviorsandto transportthemtoED.39
Interventionplans40 integratingbehavioraltreatment41 andhelpincrisisresolution42,43 maybebetterdeployedfor thesehighandveryhighEDusers.Studieshaveshownthat fewoverallinterventionsarebeingdeployedintheEDfor highusers.44,45 Previousstudieshavealsoshownthat patientswithchronicphysicalillnessesmademoreED visits.21,26 Thosewithco-occurringissueshadpoorerhealth overall,higherriskofmedicationinteractions46 andmore distress,47 explainingtheirfrequentEDuse.Improving
Table3. Estimationsofmultivariablemultinomiallogisticregressionmodelonemergencydepartment(ED)visits(referencegroup: lowEDusers,1–3visits/year).
–4vs.00.970.9410.412.311.260.7370.334.75 5+ vs.00.830.6410.381.803.510.0361.0911.35 Numberofbarriersrelatedtounmetneeds2 1–2vs.01.050.8920.512.151.130.7880.462.76 3+ vs.02.640.0321.096.423.270.0281.149.44 Frequencyofhospitalizations 1–2vs.00.460.0370.220.960.210.0020.080.56 3+ vs.01.470.4100.593.691.150.7970.393.45
4–7(highEDusers)vs.0–30.700.3080.351.400.560.7880.462.76 8+ (veryhighEDusers)vs.0–30.930.8550.441.972.530.0281.149.44
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 150 PatientCharacteristicsAssociatedwithEDUseforMHD Fleuryetal.
collaborativecare48 betweenpsychiatristsandprimarycare servicesforbettertreatmentofpatientswithco-occurring issuesmayalsoreducetheirEDuse.
Higherperceivedbarriersforunmetneedswerealso stronglyassociatedwithmoreEDuse.Barriersmaybe structural(eg,lackofaccesstoservices)ormotivational (eg,duetodistrustordissatisfactionwithservices).49 AUS studyonbarrierstocareamongfrequentEDusersfoundthat mostofthemperceivedtheEDastheonlyplacewheretheir healthproblemswouldbetreated.50 Theseresultshighlight theimportanceofacknowledgingbarrierstooutpatientcare anddevelopingmorepersonalizedpatientcarebasedon recovery-orientatedserviceswithpatient-centred interventions,51,52 oralternative “rapid” specialized responsesforpatientswithMHDincrisis.53,54 Evenifvery highEDusersreceivedprimarycaremorefrequently,it doesn’tmeanthoseserviceswereadequateorsufficientto reduceorpreventunmetneeds.
Our findingthatbeinghospitalized1-2times,butnot3+ times/year,wasprotectiveagainsthighorveryhighEDuse comparedwithnotbeinghospitalized,wasanoriginalresult. Mosthospitalizedpatientsarereferredbyemergency physicians,55 whichmightsuggestthattheserepeated hospitalizedpatientshaveveryserioushealthconditionsand thattheirinpatientcareepisodesmaybeunavoidable.Lack ofabilitytorefer(eg,timeofday)orpossibilitytorefer (eg,longwaitinglists)tooutpatientcare,lackofmental healthsupportintheED(eg,briefinterventionteams)56,57 or ofcomfortintreatingpatientswithmorecomplexMHD profilesinoutpatientcaremightalsoexplainfrequentpatient hospitalizations.Hospitalizationmaysometimesbethemost appropriatesolutionformaximizingpatientrecovery.58 For patientswith1–2hospitalizations/year,closefollow-up care,59,60 whichisincreasinglyrecommendedfollowing discharge,mayhavecontributedtoreducingtheirEDuse. Diversifiedstrategiessuchasassertivecommunitytreatment programs, 61 hometreatmentteams,62 short-staycrisis units,63 andcrisisinterventionteams64 arealsoincreasingly beingpromotedtohelpreduceacutecareuse.Althoughsuch interventionsremaininsufficientlydeployedinQuebec,the province’snewMentalHealthActionPlan(2022–2026) promisestoincreasetheiruse.25
ComparedtolowEDusers,veryhighEDusershada higherprobabilityofhavingsuicidalbehaviors,whilehigh usersshowedhigherprobabilityofhavingpersonality disorders.Previousstudieshavefoundassociationsforboth theseissueswithgreaterEDuse.13,16,28 Consideringthat healthcaresystemstendtorespondpoorlytocrisis situations,55 especiallythosethatoccuroutsideregular businesshours,thefactthatthesestudyparticipantswere veryhighEDuserswasnotsurprising.Greateravailabilityof sustainedpsychosocialprogramsinprimarycareandmore specializedcrisisandsuicidalpreventionservices65 mayhelp preventEDvisitsforsuicidalbehaviors.66 Dialectical
behaviortherapymayalsobepromotedmoreextensivelyto reducesymptomsofpersonalitydisorders,borderline personalitydisorderinparticular,asreportedinasystematic review.67 Ingeneral,theEDshouldnotreplaceoutpatient careforvulnerablepatients,astheircapacitytotreatsuch patientswasidentifiedaslimited.68,69
WomenhadagreaterprobabilityofhighEDusethan men,andpatientslivinginrentalhousingshowedagreater probabilityofveryhighEDusethanthoseinprivate housing.Womenreportedlyusemorehealthservicesthan men, 70 whichforhighEDusecontradictedpreviousstudies thatfoundmoremenwerehighEDusers.15,26 Becausehigh andveryhighEDusersweredifferentiatedinourstudy,it mayaccountforthisdivergentresult,withnodifference foundbetweenwomenandmeninveryhighEDusers.The compositionofourstudysamplecouldalsoexplainthis finding,asamajorityofparticipantsrecruitedrandomlyby EDstaffwerewomen.Concerningpatientsresidinginrental housing,theymayexperiencegreaterdeprivation,including inadequatehousingsupport,comparedwiththoselivingin privateorsupervisedhousing,whichmayaccountfortheir veryhighEDuse.Sometypeofsupportivehousingwithcase management71 mayhelpthesepatientsavoidfrequentED use.Difficultytoaccessoutpatientcarebecauseoflong waitinglistsortransportationissuesmightalsoexplainvery highEDuseamongthesepatients.
Using5+ primarycareservices/yearandrecurrenthigh EDusewereonlyassociatedwithveryhighEDusers comparedtolowEDusers,butnothighEDusers.Asfor highEDusers,studieshaveidentifiedthemashighservice usersingeneral,72 andasbeing “recurrent” EDusersover severalconsecutiveyears.6,28 Ourstudyaddedtothis literaturebyspecifyingthatonlypatientswhomadeatleast fiveprimarycareappointmentsinthepreviousyearandeight EDvisitsintheprevioustwoyearshadagreaterprobability ofbeingveryhighEDusers(8+ EDvisits/year).Thegreater useofprimarycareservicesamongveryhighEDusersmay beexplainedbytheirhigherratesofchronicphysicalillnesses andthegreaterseverityoftheseconditions,comparedwith ratesforlowandhighEDusers.Perhapsprimarycarewas notadequateorcontinuousenoughtopreventEDuse 22,73 or topreventorreduceunmetneeds.Generalpractitionershave beenshowntolacktrainingorsufficientteamcapacityto adequatelyfollowuponvulnerablepatientswithMHD.74,75 Collaborativecaremaybemorepromotedbetweenprimary andpsychiatriccareandteamworktoreduceEDuseand bettertreatthesepatients.76,77
LIMITATIONS
Thisstudyhadcertainlimitationsthatshouldbenoted. First,thereisnoconsensualdefinitionforlow,high,andvery highEDuse.Differentdefinitionsthanthosechosenhere couldhaveledtodifferent findings.Second,thestudyresults weredifficulttocomparewiththeliteratureasmoststudies
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 151 Fleuryetal. PatientCharacteristicsAssociatedwithEDUseforMHD
havecomparedhighEDusewithotherEDuse.Third, structuredinterviewsmaybebiasedduetothepatients’ abilitytorecall,andthehealthrecordsthatwereused reflectedserviceuseonlywithintheparticipatingnetworks. Finally,thediversityofhealthcaresystemsmaylimitthe generalizationofthestudy findings,especiallyincountries thatdon’thavepublichealthcarecoveragefor deprivedpopulations.
CONCLUSION
Thisstudywasinnovativeinthewayitcomparedlow, high,andveryhighEDusersamongpatientswithMHDin Canada,andbyusingbothpatientstructuredinterviewsand healthrecords.The findingsconfirmedthathigherEDuse wasassociatedwithcomplexpatienthealthissuesandhigher perceivedbarrierstounmetneeds.Patientswithveryhigh overallEDusehadthemostsevereconditions,including greaterhousingvulnerabilityandisolation,andmore suicidalbehaviors.Theyalsousedmoreprimarycare services,possiblybecauseoftheirseverechronicphysical healthconditions.
RecurrentEDuseovertheyearsalsodistinguishedvery highEDusersfromlowusers.Bycontrast,theriskofhigh andveryhighEDusewasreducedinpatientswith1–2 hospitalizations/year,whichunderlinesthepotentialbenefits andpertinenceofhospitalizationforsomepatients.Overall, barrierstocareshouldbereducedandbetteraccessand continuityofoutpatientcareprovidedforthemost vulnerablepatients,integratingcrisisresolutionand supportedhousingservices.Thismayreducethenumberof patientswithMHDintheED,decreasingwaittimesand improvingcareintheED.
2.MinassianA,VilkeGM,WilsonMP.Frequentemergencydepartment visitsaremoreprevalentinpsychiatric,alcoholabuse,anddual diagnosisconditionsthaninchronicviralillnessessuchashepatitisand humanimmunodeficiencyvirus. JEmergMed.2013;45(4):520–5.
3.GiannouchosTV,KumHC,FosterMJ,etal.Characteristicsand predictorsofadultfrequentemergencydepartmentusersintheUnited States:asystematicliteraturereview. JEvalClinPract 2019;25(3):420–33.
4.KriegC,HudonC,ChouinardMC,etal.Individualpredictorsoffrequent emergencydepartmentuse:ascopingreview. BMCHealthServRes 2016;16(1):594.
5.LaCalleEandRabinE.Frequentusersofemergencydepartments:the myths,thedata,andthepolicyimplications. Review.AnnEmergMed 2010;56(1):42–8.
6.BillingsJandRavenMC.Dispellinganurbanlegend:frequent emergencydepartmentusershavesubstantialburdenofdisease. HealthAff(Millwood).2013;32(12):2099–108.
7.MoeJ,KirklandS,OspinaMB,etal.Mortality,admissionratesand outpatientuseamongfrequentusersofemergencydepartments: asystematicreview. EmergMedJ.2016;33(3):230–6.
8.VandykAD,HarrisonMB,VanDenKerkhofEG,etal.Frequent emergencydepartmentusebyindividualsseekingmentalhealthcare: asystematicsearchandreview. ArchPsychiatrNurs.2013;27(4):171–8.
9.FleuryMJ,FortinM,RochetteL,etal.Assessingqualityindicators relatedtomentalhealthemergencyroomutilization. BMCEmergMed 2019;19(1):8.
10.OndlerC,HegdeGG,CarlsonJN.Resourceutilizationandhealthcare chargesassociatedwiththemostfrequentEDusers. AmJEmergMed 2014;32(10):1215–9.
11.ChangG,WeissAP,OravEJ,etal.Predictorsoffrequentemergency departmentuseamongpatientswithpsychiatricillness. GenHosp Psychiatry.2014;36(6):716–20.
AddressforCorrespondence:Marie-JoséeFleury,PhD,Douglas MentalHealthUniversityInstituteResearchCentre,6875,LaSalle Blvd.,Verdun,Canada.Email: flemar@douglas.mcgill.ca
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.ThisstudywasfundedbytheCanadian InstitutesofHealthResearch(CIHR,grantnumber:8400997).
Copyright:©2024Fleuryetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.LiuSW,NagurneyJT,ChangY,etal.FrequentEDusers:aremostvisits formentalhealth,alcohol,anddrug-relatedcomplaints? AmJEmerg Med.2013;31(10):1512–5.
12.BuhumaidR,RileyJ,SattarianM,etal.Characteristicsoffrequentusers oftheemergencydepartmentwithpsychiatricconditions. JHealthCare PoorUnderserved.2015;26(3):941–50.
13.Richard-LepourielH,WeberK,BaertschiM,etal.Predictorsofrecurrent useofpsychiatricemergencyservices. PsychiatrServ 2015;66(5):521–6.
14.SirotichF,DurbinA,DurbinJ.Examiningtheneedprofilesofpatients withmultipleemergencydepartmentvisitsformentalhealthreasons: across-sectionalstudy. SocPsychiatryPsychiatrEpidemiol 2016;51(5):777–86.
15.FleuryMJ,RochetteL,GrenierG,etal.Factorsassociatedwith emergencydepartmentuseformentalhealthreasonsamonglow, moderateandhighusers. GenHospPsychiatry.2019;60:111–9.
16.SlankamenacK,HeidelbergerR,KellerDI.Predictionofrecurrent emergencydepartmentvisitsinpatientswithmentaldisorders. FrontPsychiatry.2020;11:48.
17.VuF,DaeppenJB,HugliO,etal.Screeningofmentalhealthand substanceusersinfrequentusersofageneralSwissemergency department. BMCEmergMed.2015;15:27.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 152 PatientCharacteristicsAssociatedwithEDUseforMHD Fleuryetal.
18.GaulinM,SimardM,CandasB,etal.Combinedimpactsof multimorbidityandmentaldisordersonfrequentemergencydepartment visits:aretrospectivecohortstudyinQuebec,Canada. CMAJ 2019;191(26):E724–32.
19.MitchellMS,LeonCLK,ByrneTH,etal.Costofhealthcareutilization amonghomelessfrequentemergencydepartmentusers. PsycholServ 2017;14(2):193–202.
20.KorczakV,ShanthoshJ,JanS,etal.Costsandeffectsof interventionstargetingfrequentpresenterstotheemergency department:asystematicandnarrativereview. BMCEmergMed 2019;19(1):83.
21.DoupeMB,PalatnickW,DayS,etal.Frequentusersofemergency departments:developingstandarddefinitionsanddefiningprominent riskfactors. AnnEmergMed.2012;60(1):24–32.
22.GentilL,GrenierG,VasiliadisHM,etal.Predictorsofrecurrenthigh emergencydepartmentuseamongpatientswithmentaldisorders.
IntJEnvironResPublicHealth.2021;18(9):4559.
23.MinistèredelaSantéetdesServicesSociaux(MSSS).Lesystèmede santéetdeServicesSociauxauQuébec,enbref.Quebec:Government ofQuebec.2017.Availableat: https://publications.msss.gouv.qc.ca
AccessedJanuary27,2022.
24.MinistèredelaSantéetdesServicessociaux(MSSS).Projetdeloino 10(2015,chapitre1).Loimodifiantl’organisationetlagouvernancedu réseaudelasantéetdesservicessociauxnotammentparl’abolitiondes agencesrégionales.Quebec:GovernmentofQuebec.2015.
Availableat: https://www2.publicationsduquebec.gouv.qc.ca
AccessedJanuary21,2022.
25.Ministèredelasantéetdesservicessociaux(MSSS).LePland’action interministérielensantémentale2022-2026-S’unirpourunmieux-être collectifPublicationsduministèredelaSantéetdesServicessociaux. Quebec:GovernmentofQuebec.2022.Availableat: https:// publications.msss.gouv.qc.ca/msss/document-003301 AccessedJanuary27,2022.
26.BrennanJJ,ChanTC,HsiaRY,etal.Emergencydepartmentutilization amongfrequentuserswithpsychiatricvisits. AcadEmergMed 2014;21(9):1015–22.
27.MoeJ,BaileyAL,OlandR,etal.Defining,quantifying,and characterizingadultfrequentusersofasuburbanCanadianemergency department. CJEM.2013;15(4):214–26.
28.ArmoonB,CaoZ,GrenierG,etal.Profilesofhighemergency departmentuserswithmentaldisorders. AmJEmergMed 2022;54:131–41.
29.BohnMJ,BaborTF,KranzlerHR.Thealcoholusedisorders identificationtest(AUDIT):validationofascreeninginstrumentforusein medicalsettings. JStudAlcohol.1995;56(4):423–32.
30.SkinnerHA.Thedrugabusescreeningtest. AddictBehav 1982;7(4):363–71.
31.HuynhC,KiselyS,RochetteL,etal.Usingadministrativehealthdatato estimateprevalenceandmortalityratesofalcoholandothersubstancerelateddisordersforsurveillancepurposes. DrugAlcoholRev 2021;40(4):662–72.
32.SimardM,SiroisC,CandasB.Validationofthecombinedcomorbidity indexofCharlsonandElixhausertopredict30-daymortalityacross ICD-9andICD-10. MedCare.2018;56(5):441–7.
33.CanadianAssociationofEmergencyPhysicians.Canadiantriageacuity scale.2012.Availableat: https://ctas-phctas.ca AccessedDecember18,2021.
34.HarrisMG,HobbsMJ,BurgessPM,etal.Frequencyandqualityof mentalhealthtreatmentforaffectiveandanxietydisordersamong Australianadults. MedJAust.2015;202(4):185–9.
35.DziuraJD,PostLA,ZhaoQ,etal.Strategiesfordealingwithmissing datainclinicaltrials:fromdesigntoanalysis. YaleJBiolMed 2013;86(3):343–58.
36.MickeyJandGeenlandS.Astudyoftheimpactofcounfounderselectioncriteriaoneffectestimation. AmJEpidemiol 1989;129(1):125–37.
37.AkaikeH.Informationtheoryandanextensionofthemaximum likelihoodprinciple.In:PetrovBNandCsakiF(Eds.), Proceedingsofthe 2ndInternationalSymposiumonInformationTheory (267–81). Hungary,Budapest:AkademiaiKiado,1973.
38.HairJ,BlackWC,BabinBJ,etal.(2001). MultivariateDataAnalysis (7thed.).NewJersey,UpperSaddleRiver:PearsonEducation.
39.ShortTBR.Thenatureofpoliceinvolvementinmentalhealthtransfers. PolicePractRes.2014;15(4):336–48.
40.AbelloA,Jr.,BriegerB,DearK,etal.Careplanprogramreduces thenumberofvisitsforchallengingpsychiatricpatientsintheED. AmJEmergMed.2012;30(7):1061–7.
41.FrazierSNandVelaJ.Dialecticalbehaviortherapyforthetreatmentof angerandaggressivebehavior:areview. AggressViolentBehav 2014;19(2):156–63.
42.WheelerC,Lloyd-EvansB,ChurchardA,etal.Implementationofthe crisisresolutionteammodelinadultmentalhealthsettings:asystematic review. BMCPsychiatry.2015;15:74.
43.VakkalankaJP,NeuhausRA,HarlandKK,etal.Mobilecrisisoutreach andemergencydepartmentutilization:apropensityscore-matched analysis. WestJEmergMed.2021;22(5):1086–94.
44.GabetM,ArmoonB,MengX,etal.Effectivenessofemergency departmentbasedinterventionsforfrequentuserswithmentalhealth issues:asystematicreview. AmJEmergMed.2023;74:1–8.
45.GabetMandFleuryM-J.Innovationsorganisationnellesauxurgences pouraméliorerlaqualitédessoinsdispensésauxpatientssouffrantde troublesmentaux:perspectivesinternationales. JGestÉconSanté 2022;40(2-3):100–15.
46.KangHJ,KimSY,BaeKY,etal.Comorbidityofdepressionwith physicaldisorders:researchandclinicalimplications. ChonnamMedJ 2015;51(1):8–18.
47.QinP,HawtonK,MortensenPB,etal.Combinedeffectsofphysical illnessandcomorbidpsychiatricdisorderonriskofsuicideinanational populationstudy. BrJPsychiatry.2014;204(6):430–5.
48.IvbijaroGO,EnumY,KhanAA,etal.Collaborativecare:modelsfor treatmentofpatientswithcomplexmedical-psychiatricconditions. CurrPsychiatryRep.2014;16(11):506.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 153 Fleuryetal. PatientCharacteristicsAssociatedwithEDUseforMHD
49.MojtabaiR,OlfsonM,SampsonNA,etal.Barrierstomentalhealth treatment:resultsfromtheNationalComorbiditySurveyReplication. PsycholMed.2011;41(8):1751–61.
50.BirminghamLE,CochranT,FreyJA,etal.Emergencydepartmentuse andbarrierstowellness:asurveyofemergencydepartmentfrequent users. BMCEmergMed.2017;17(1):16.
51.ChesterP,EhrlichC,WarburtonL,etal.Whatistheworkofrecovery orientedpractice?Asystematicliteraturereview. IntJMentHealthNurs 2016;25(4):270–85.
52.CareyTA.Beyondpatient-centeredcare:enhancingthepatient experienceinmentalhealthservicesthroughpatient-perspectivecare. PatientExpJ.2016;3(2):46–9.
53.HubbelingDandBertramR.CrisisresolutionteamsintheUKand elsewhere. JMentHealth.2012;21(3):285–95.
54.BoucheryEE,BarnaM,BabalolaE,etal.Theeffectivenessofapeerstaffedcrisisrespiteprogramasanalternativetohospitalization. PsychiatrServ.2018;69(10):1069–74.
55.JohnsonS,Dalton-LockeC,BakerJ,etal.Acutepsychiatriccare: approachestoincreasingtherangeofservicesandimprovingaccess andqualityofcare. WorldPsychiatry.2022;21(2):220–36.
56.GabetM,GrenierG,CaoZ,etal.Implementationofthreeinnovative interventionsinapsychiatricemergencydepartmentaimedatimproving serviceuse:amixed-methodstudy. BMCHealthServRes 2020;20(1):854.
57.TurnerSBandStantonMP.Psychiatriccasemanagementinthe emergencydepartment. ProfCaseManag.2015;20(5):217–27; quiz228-9.
58.JunWHandYunSH.Mentalhealthrecoveryamonghospitalized patientswithmentaldisorder:associationswithangerexpressionmode andmeaninginlife. ArchPsychiatrNurs.2020;34(3):134–40.
59.GentilL,GrenierG,FleuryMJ.Factorsrelatedto30-dayreadmission followinghospitalizationforanymedicalreasonamongpatientswith mentaldisorders[Facteursliesalarehospitalisationa30jourssuivant unehospitalisationpouruneraisonmedicalechezdespatientssouffrant detroublesmentaux]. CanJPsychiatry.2021;66(1):43–55.
60.KurdyakP,VigodSN,NewmanA,etal.Impactofphysicianfollow-up careonpsychiatricreadmissionratesinapopulation-basedsampleof patientswithschizophrenia. PsychiatrServ.2018;69(1):61–8.
61.AddingtonD,AndersonE,KellyM,etal.Canadianpracticeguidelines forcomprehensivecommunitytreatmentforschizophreniaand schizophreniaspectrumdisorders. CanJPsychiatry 2017;62(9):662–72.
62.BoisvertA,BouffardA-P,PaquetK.Letraitementintensifbrefà domicile. SantéMentale.2016;204:68–72.
63.AndersonK,GoldsmithLP,LomaniJ,etal.Short-staycrisisunitsfor mentalhealthpatientsoncrisiscarepathways:systematicreviewand meta-analysis. BJPsychOpen.2022;8(4):e144.
64.MurphySM,IrvingCB,AdamsCE,etal.Crisisinterventionfor peoplewithseverementalillnesses. CochraneDatabaseSystRev 2015;2015(12):CD001087.
65.MisharaBL,CoteLP,DargisL.Systematicreviewofresearchand interventionswithfrequentcallerstosuicidepreventionhelplinesand crisiscenters. Crisis.2023;44(2):154–67.
66.GentilL,GrenierG,FleuryMJ.Determinantsofsuicidalideationand suicideattemptamongformerandcurrentlyhomelessindividuals. SocPsychiatryPsychiatrEpidemiol.2020;56(5):747–57.
67.BloomJM,WoodwardEN,SusmarasT,etal.Useofdialecticalbehavior therapyininpatienttreatmentofborderlinepersonalitydisorder: asystematicreview. PsychiatrServ.2012;63(9):881–8.
68.DeLeoK,MaconickL,McCabeR,etal.Experiencesofcrisiscare amongserviceuserswithcomplexemotionalneedsoradiagnosisof 'personalitydisorder',andotherstakeholders:systematicreviewand meta-synthesisofthequalitativeliterature. BJPsychOpen 2022;8(2):e53.
69.StanleyB,BrownGK,BrennerLA,etal.Comparisonofthesafety planninginterventionwithfollow-upvsusualcareofsuicidalpatients treatedintheemergencydepartment. JAMAPsychiatry 2018;75(9):894–900.
70.PinkhasovRM,ShteynshlyugerA,HakimianP,etal.Aremen shortchangedonhealth?Perspectiveonlifeexpectancy,morbidity,and mortalityinmenandwomenintheUnitedStates. IntJClinPract 2010;64(4):465–74.
71.McPhersonP,KrotofilJ,KillaspyH.Mentalhealthsupported accommodationservices:asystematicreviewofmentalhealthand psychosocialoutcomes. BMCPsychiatry.2018;18(1):128.
72.PinesJM,AsplinBR,KajiAH,etal.Frequentusersofemergency departmentservices:gapsinknowledgeandaproposedresearch agenda. AcadEmergMed. 2011;18(6):e64–9.
73.BarkerLC,SunderjiN,KurdyakP,etal.Urgentoutpatientcarefollowing mentalhealthedvisits:apopulation-basedstudy. PsychiatrServ. 2020;71(6):616–9.
74.FleuryMJ,BamvitaJM,FarandL,etal.GPgroupprofilesand involvementinmentalhealthcare.Researchsupport,non-U.S.Gov’t. JEvalClinPract.2012;18(2):396–403.
75.LoebDF,BaylissEA,BinswangerIA,etal.Primarycarephysician perceptionsoncaringforcomplexpatientswithmedicalandmental illness. JGenInternMed.2012;27(8):945–52.
76.AupontO,DoerflerL,ConnorDF,etal.Acollaborativecaremodelto improveaccesstopediatricmentalhealthservices. AdmPolicyMent Health.2013;40(4):264–73.
77.RaneyLE.Integratingprimarycareandbehavioralhealth:therole ofthepsychiatristinthecollaborativecaremodel. AmJPsychiatry 2015;172(8):721–8.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 154 PatientCharacteristicsAssociatedwithEDUseforMHD Fleuryetal.
BridgingtheGap:EvaluationofanElectrocardiogramCurriculum forAdvancedPracticeClinicians
StevenLindsey,MD*
TimP.Moran,PhD*
MeredithA.Stauch,MSN,APRN,FNP-BC,ENP-C†
AlexisL.Lynch,MSN,APRN,ENP-C,FNP-BC,ANP-BC*
KristenGrabowMoore,MD,MED*
SectionEditor:NielsRathlev,MD
*EmoryUniversity,DepartmentofEmergency Medicine,Atlanta,Georgia
† EmoryUniversitySchoolofMedicine, Atlanta,Georgia
Submissionhistory:SubmittedApril21,2023;RevisionreceivedNovember17,2023;AcceptedNovember22,2023
ElectronicallypublishedFebruary9,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18085
Background: Trainingprogramsforadvancedpracticeproviders(APP)oftenhavesignificantvariability intheircurriculum,includingelectrocardiogram(ECG)education.DespitelimitationsinformalECG training,APPsintheemergencydepartment(ED)maybethe firstpractitionertointerpretanECG. FoundationsofEmergencyMedicine(FoEM)offersfree,open-accesscurriculathatarewidelyused forresidenteducation.WesoughttoimproveAPPECGinterpretationskillsbyimplementingthe FoEMECGIcourse.
Methods: Thiswasasingle-site,pre-andpost-interventionstudyof23APPsatourhigh-acuity,urban ED.Inthefallof2020,APPlearnersenrolledinaFoEMECGIcourseledbyfacultyandseniorresident instructors.Thecourseconsistedofsixvirtual,small-group,active-learningECGworkshops. Participantscompleteda15-questionmultiple-choicetestbefore,immediatelyafter,andsixmonths post-interventiontoquantifyknowledgeacquisition.Additionally,apre-andpost-intervention knowledge,attitudes,andpracticessurveywasadministeredonECGinterpretationskillsandto evaluatethecourse.WeevaluatedchangeinECGknowledgescoresusingaWilcoxonsigned-ranktest. Changesinself-assessedknowledgewereevaluatedusinganordinallogisticmixed-effectsregression.
Results: Atotalof23APPsenrolledinthecourse.KnowledgeassessmentsshowedAPPsimproved fromthepre-test(median9/15,interquartilerange[IQR]7–11)tothepost-test(median12/15,IQR 10–13; P = 0.001).Testscoresdidnotsignificantlychangefromthepost-testtothedelayedpost-test (median12/15,IQR12–13; P = 0.30).Respondents’ subjectiveratingoftheirskilldidnotsignificantly change(P = 0.06).Respondentsreportednochangeintheirlikelihoodofapproachinganattending whenuncertainofthecorrectinterpretationofanECG(P = 0.16).Overall,91%weresatisfiedwiththe courseand96%agreedthatthecoursedifficultywasappropriate.
Conclusion: TheFoEMECGcourseprovidedastandardizedcurriculumthatimprovedAPPknowledge forinterpretingECGs.Despitethis,thecoursedidnotalterAPPs’ willingnesstoapproachphysiciansfor guidancewithinterpretationofabnormalECGs.These findingsmayinformexpansionofthisconceptfor otherprogramswhodesireformalizedAPPECGeducation.[WestJEmergMed.2024;25(2)155–159.]
INTRODUCTION
Advancedpracticeproviders(APP),comprisingphysician assistants(PAs)andnursepractitioners(NPs),havea substantialpresenceinemergencydepartments(ED)inthe
UnitedStates.Emergencydepartmentsemployed77%of APPsin2006,increasingfrom28%in1997.1 Thereareover 13,000PAsandover10,000NPscurrentlypracticinginthe acutecaresetting.2,3 ItisestimatedthatAPPssee21%ofall
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 155 BRIEF RESEARCH REPORT
EDvisitsandtheproportionofhighacuityservices independentlybilledbyAPPsisincreasing.4,5 Somemodels ofEDcare,suchaspractitioner-in-triage,oftenemploy APPsasthe firstpointofcontactforpatientsandaretasked withorderinginitialdiagnosticssuchas electrocardiograms(ECG).6,7
DespitethevolumeandacuityofpatientstreatedbyAPPs intheED,arelativelysmallproportionofAPPshave receivedformalizedpostgraduatetraininginemergency medicine(EM),with10%ofPAsand21%ofNPshaving completedsuchtraining.2,8 BoththeAmericanAcademyof EmergencyNursePractitioners(AAENP)9 andtheSociety ofEmergencyMedicinePhysicianAssistants(SEMPA)10 identifyECGinterpretationasarequisiteskillforAPPs practicinginEM.However,noconsistentapproachis appliednationwidetoaddressthislackof EM-specifictraining.11,12
FoundationsofEmergencyMedicine(FoEM)isafree, open-accesscurriculumthatiswidelyusedandvalidatedin EMresidenteducation. 13, 14 FoEMoffersstandardized, level-speci fi c,corecontentthatprimarilytargetsresident physiciansinEM.TheFoEMECGIcourseiscomposedof sixunitsthatreviewfundamentalconceptsinECG interpretationusinga fl ippedclassroomapproach ( Appendix1). 15 Implementationguidelines,unit summaries,challengeECGs,andinterpretationguidesare allfoundontheFoEMwebsite,whichmaybeaccessedby programleadersafterfreeregistration. 16 Wesoughtto addressagapintrainingandimproveAPPECG interpretationskillsbyimplementingtheFoEM ECGIcourse.
METHODS
StudyPopulationandDesign
WeincludedAPPsinthissingle-sitestudyiftheycurrently practicedatalarge,urban,countyhospitalandwereenrolled intheFoEMECGIcourseduringOctober2020-June2021. Whileenrollmentinthecoursewasrequiredtostaffhigher acuityEDzones,participationinthestudywasvoluntary. Participantsreviewedunitsummariesandpracticedselect ECGspriortoeachworkshop.Duringtheworkshop,APPs weredividedintosmallgroupstocollaborativelyreviewfour challengeECGswithinterpretationanddiscussionprompts. Uponcompletionofsmall-groupdiscussion,facultyor seniorresidentinstructorsfacilitatedinteractivesessionswith alllearners,reviewingcoreconceptsandECG challengeanswers.
Studyparticipantscompletedaknowledge,attitudes,and practices(KAP)surveyatthebeginningandcompletionof thecourse(Appendix2).Additionally,weobtainedobjective knowledgeacquisitionthrougha15-questionmultiplechoiceassessmentadministeredinOctober2020(preintervention),December2020(immediatepostintervention),andJune2021(delayedpost-intervention).
PopulationHealthResearchCapsule
Whatdowealreadyknowaboutthisissue?
Advancedpracticeproviders(APP)are responsibleforseeingasigni fi cantnumberof patientsinmanyEDsettings,yettheir in-trainingandpost-trainingcurricula arevariable.
Whatwastheresearchquestion?
WouldtheimplementationoftheFoundations ofEmergencyMedicine(FoEM)ECGI courseimproveelectrocardiogram interpretationskillsofAPPs?
Whatwasthemajor findingofthestudy?
Knowledgeassessmentsimprovedfromthe pre-test(median9/15,IQR7 – 11)totheposttest(median12/15,IQR10 – 13;P = 0.001).
Howdoesthisimprovepopulationhealth?
UseoftheFoEMECGIcurriculumforAPP learnersledtoanimprovementinECG knowledge,whilemaintainingattending physicianguidanceinthesetting ofuncertainty.
ThisstudywasdeemedexemptbytheInstitutionalReview BoardofEmoryUniversity.
StatisticalAnalysis
Wedescribedcategoricalvariablesusingfrequencies andpercentages.Continuousandscalevariableswere describedusingmediansandinterquartileranges(IQR). WeevaluatedthechangeinECGknowledgescores betweenthepre-test,post-test,anddelayedpost-testsessions usingtheFriedmanrepeated-measuresrank-order ANOVA.Ordinalself-assessmentvariableswereevaluated usingamixed-effectsordinallogisticregression.Weused mixedeffectstoaccountformultipleresponsesfrom individualstudyparticipants.Oddsratios(OR)and 95%confidenceintervals(CI)arepresentedfromthe regressions.Two-tailed P -values ≤0.05wereconsidered significant.WeconductedstatisticalanalysesusingRversion 4(RCoreTeam,FoundationforStatisticalComputing, Vienna,Austria).
RESULTS
Atotalof23APPsenrolled,withthemajorityidentifying asfemale(74%)withamedianageof37(IQR33–40)years
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 156 ECGCurriculumforAPPEducation Lindseyetal.
(Table1).Learnerswereprimarilyfamilynursepractitioner (FNP)(48%),followedbyphysicianassistrantPAs(26%) andFNP-emergencynursepractitionerENPs(22%).They reportedamedianof fiveyearsofpostgraduateexperiencein EM(IQR3–6),andasmallproportionreportedcompleting formalizedpostgraduatetraininginEM(13%).
Self-assessedconfidenceofECGinterpretationwas higherinthepost-testassessmentcomparedtothepre-test assessment;however,thedifferencewasnotsignificant (oddsratio[OR]2.94(95%confidenceintervalCI0.94–9.1), P = 0.06)(Figure1A).Incontrast,theobjective knowledgeassessments(Figure1B-D)indicatethatECG interpretationimproved(P < 0.001).Post-hoctestsindicated thatpost-testscores(median12/15,IQR10–13)were significantlygreaterthanpre-testscores(median9/15,IQR 7–11; P < 0.001).Delayedpost-testscores(median12/15, IQR12–13)didnotdifferfrompost-testscores(P = 0.30) indicatingthattheimprovedunderstandingwaslargely maintainedovertime.
OntheKAPsurvey,APPsreportedimprovedconfidence indetectinganST-segmentelevationmyocardialinfarction (STEMI)onECG(P = 0.01)(Appendix3).Nooverall changewasnotedinconfidencetointerpretalife-threatening arrhythmia(P = 0.27).Participantswerenomoreorless likelytoapproachanattendingphysicianforhelpregarding anuncertainECGbeforeandaftertheECGcurriculum (P = 0.16).Withrespecttoparticipants’ viewofthecourse, 21(91%)reportedbeingsatisfiedorhighlysatisfied;9(39%) satisfied;and12(52%)highlysatisfied.Onlyoneparticipant (4%)wasneutralandone(4%)wasunsatisfied.Twenty-two
participants(96%)believedthatthecoursewastaughtatthe correctlevelofdifficulty:13(57%)stronglyagreed;and9 (39%)agreed.Onlyoneparticipant(4%)wasneutral.No respondentdisagreed.
DISCUSSION
TheFoEMECGIcurriculumwasadministeredtoAPP learnersandevaluatedusingapre-andpost-intervention, self-reportedKAPsurveyinconjunctionwithanobjective measureofknowledgeacquisition.Therewasan improvementinobjectiveknowledgeandretention,atrend towardimprovedconfidenceinECGinterpretation,anda significantimprovementinSTEMIidentification.Despite theseimprovements,therewasnochangeintheAPPs’ likelihoodofreachingouttophysiciansforassistancewith ECGinterpretation.
CharacteristicValueN = 23
Age,median(IQR)*37(33–40)
Gender,n(%)
Female17(74%)
Male6(26%)
Certi
*IQR,interquartilerange;**AGNP,adultgerontologynurse practitioner; αFNP,familynursepractitioner; βFNP-ENP,familynurse practitioner-emergencynursepractitioner; ¥PA,physicianassistant.
Auniqueadvantageofourstudyisitsdemonstrationof measurableimprovementinclinicallyrelevantECG interpretation.Whilethisisencouragingwithrespecttothe easeandefficiencyofthecourse,whatismoreimpressiveis theretentionofknowledgeovertime.TheAPPscommonly workintriageandloweracuityareasandareoftenthe first practitionerstoevaluatepatientsintheED.6,7 Todetect manylife-threateningillnesses,promptECGacquisitionand interpretationisessential.Thus,APPsshouldbeableto interpretECGswhenaphysicianisnotimmediately available,asmaybethecaseinsomepractices.17 Despitethe importanceofECGinterpretation,APPsoften findthis clinicalskillchallenging,withonestudydemonstrating50% proficiencyofECGinterpretationamonggraduatingPAs,a metriccommensuratewiththebaselinecompetency demonstratedinourcohort.18 Inthisstudy,wewereableto increaseandmaintaincompetencyat80%,underscoringthe valueaddedbytheFoEMECGIcourse.Despitethis increasedknowledge,APPswerejustaslikelytoreachoutto aphysicianforECGinterpretationguidance,showingthat thecoursedidnotdecreaserelianceonphysicianknowledge andjudgment.
OurstudydemonstratedthatAPPs,despiteshowing increasedknowledgefollowingthecourse,onlygained confidenceinidentifyingSTEMIsonECG,butdidnot improveconfidenceinotherdomainsofECGinterpretation followingtheFoEMECGIcourse.Thisrepresentsa mismatchinperceivedandactualECGinterpretation competence.Thismayreflectlimitedindividualtimespenton eachmoduleand/ormodulesfocusedonmorespecificECG pathologies.Furtherstudiesmayevaluatewhether confidencemaybeimprovedwithcontinuedtrainingand exposuretomorediverseECG findings.
Whileimplementingthecourseandshowingitwas effectivefromaknowledge,attitudes,andbehaviors standpointisofparamountimportance,aneducational programmustalsobewell-receivedbythelearner.Ourstudy demonstratedveryhighlevelsofsatisfactionwiththecourse
Table1. Demographiccharacteristicsofadvancedpractice providersenrolledintheFoundationsofEmergencyMedicineECGI course,October2020–June2021.
α 11(48%) FNP-ENPβ 5(22%) PA¥ 6(26%)
6)
trainingprogram,n(%) 3(13%)
fication,n(%) AGNP**1(4%) FNP
Postgraduateexperience,median(IQR)5(3–
Completedemergencymedicinepostgraduate
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 157 Lindseyetal. ECGCurriculumforAPPEducation
Figure1. Advancedpracticeproviderperceivedandobjectiveknowledgeacquisitionbeforeandafterfoundationsofemergencymedicine (FoEM)ElectrocardiogramGram(ECG)Icourse.A)Self-assessmentofrespondents’ understandingofECGinterpretationasafunctionof time.B)Boxplotdepictingknowledgetestscoresasafunctionoftime.C)Waterfallplotdepictingthechangeinknowledgetestscoresfrom pre-testtopost-testforeachindividualrespondent.D)Waterfallplotdepictingthechangeinknowledgetestscoresfrompost-testtodelayed post-testforeachindividualrespondent.
amongourAPPs,alongwithAPPsreportingthatthe conceptstaughtwereappropriatefortheirleveloftraining. Thistranslatesintomoreengagementandknowledge acquisitionandretentioninthecurriculum,in fittingwithpriorstudieslookingatAPPsand case-basededucation.19
LIMITATIONS
Ourstudylimitationsincludedasmallsamplesizewitha relativelyhomogeneousstudypopulation(eg,primarilyNPs, allpracticingatasinglecountyhospital).Additionally,our studycohortdidnotreflecttheroughly50/50distributionof NPsandPAspracticinginacutecaresettings,withour grouponlyhaving26%PArepresentation.Finally, ourstudydidnotincludeacontrolgroupwhichdidnot receivetrainingtherebyallowingforthepossibilityoftest/ retesteffects.
CONCLUSION
FormalizedpostgraduateECGinterpretationtrainingfor APPsinEMisatbestinconsistent,yetbothSEMPAand AAENPlistECGinterpretationasanecessaryskillfor practicinginEM.9,10 Inresponsetothis,weimplementedthe FoEMECGIcourseandfoundthatitwaseasyto implement,ledtoimprovedECGknowledgeandconfidence inECGinterpretation,andwaswellreceivedbytheAPP group.Theseresultsmayinformtheuseofthisfree, structuredECGcurriculumatbothacademicand community-basedprogramsthatsupportcontinuing educationforAPPs.Futurestudiesshouldinvestigatethe impactofincreasedsamplesizes,morevariablepractice locationsanddepartmentaldesigns,andahigherproportion ofPAs,allofwhichwouldservetomakethedatamore reflectiveoftheAPPpopulationasawholeand,therefore, moregeneralizable.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 158 ECGCurriculumforAPPEducation Lindseyetal.
AddressforCorrespondence:StevenLindsey,MD,FACEP, FACEM,EmoryUniversity,DepartmentofEmergencyMedicine, 531AsburyCircle,AnnexBuilding,SuiteN340,Atlanta,GA30322. Email: steven.m.lindsey@emory.edu
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.
Copyright:©2024Lindseyetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.MenchineMD,WiechmannW,RudkinS.Trendsinmidlevelprovider utilizationinemergencydepartmentsfrom1997to2006. AcadEmerg Med.Oct2009;16(10):963–9.
2.StatisticalProfileofCertifiedPAsAnnualReport.2021:National CommissiononCertificationofPhysicianAssistantswebsite.2022. Availableat: https://www.nccpa.net/wp-content/uploads/2022/08/ 2021StatProfileofCertifiedPAs-A-3.2.pdf.AccessedApril12,2023.
3.NPFactSheet.AmericanAcademyofNursePractitionerswebsite. 2022.Availableat: https://www.aanp.org/about/all-about-nps/ np-fact-sheet.AccessedApril12,2023.
4.WuFandDarracqMA.Comparingphysicianassistantandnurse practitionerpracticeinU.S.emergencydepartments,2010-2017. West JEmergMed.2021;22(5):1150–5.
5.GettelCJ,SchuurJD,MullenJB,etal.Risinghigh-acuityemergency careservicesindependentlybilledbyadvancedpracticeproviders,2013 to2019. AcadEmergMed.2023;30(2):89–98.
6.MoretzJ.Emergencydepartmentcaremodels.BerkeleyResearch Groupwebsite.2018.Availableat: https://www.thinkbrg.com/insights/ publications/emergency-department-care-models AccessedApril11,2023.
7.WylieK,CrillyJ,TolooGS,etal.Reviewarticle:emergencydepartment modelsofcareinthecontextofcarequalityandcost:asystematic review. EmergMedAustralas.2015;27(2):95–101.
8.TheStateoftheNursePractitionerProfession.AmericanAcademyof NursePractitionerswebsite.2021.Availableat: https://www.aanp.org AccessedApril12,2023.
9.PracticeStandardsfortheEmergencyNursePractitionerSpecialty. AmericanAcademyofEmergencyNursePractitionerswebsite.2018. Availableat: https://www.aaenp-natl.org/assets/docs/ practice_standards_for_the_emergency_nurse_practitioner.pdf AccessedApril11,2023.
10.EmergencyMedicinePhysicianAssistantPracticeGuidelines.Society ofEmergencyMedicinePhysicianAssistants.2021.Availableat: https:// www.sempa.org/about-sempa/guidelines-and-statements/ empa-practice-guidelines.AccessedApril12,2023.
11.ChekijianSA,EliaTR,MontiJE,etal.Integrationofadvancedpractice providersinacademicemergencydepartments:bestpracticesand considerations. AEMEducTrain.2018;2(Suppl1):S48–55.
12.KrausCK,CarlisleTE,CarneyDM.Emergencymedicinephysician assistant(EMPA)postgraduatetrainingprograms:program characteristicsandtrainingcurricula. WestJEmergMed 2018;19(5):803–7.
13.BurnsWP,HartmanND,WeygandtPL,etal.Criticalelectrocardiogram curriculum:settingthestandardfor flipped-classroomEKGinstruction. WestJEmergMed.2019;21(1):52–7.
14.GrabowMooreK,KettererA,WheatonN,etal.Development, implementation,andevaluationofanopenaccess,level-specific,core contentcurriculumforemergencymedicineresidents. JGradMed Educ.2021;13(5):699–710.
15.EKGcourses.FoundationsofEmergencyMedicinewebsite.Available at: https://foundationsem.com/ekg-courses.AccessedMarch29,2023.
16.FoundationsofEmergencyMedicine.Availableat: https:// foundationsem.com.AccessedNovember6,2022.
17.KeoughVA,TellD,AndreoniC,etal.Uniqueeducationalneedsof emergencynursepractitioners. AdvEmergNursJ.2016;38(4):300–7.
18.LundbergEL,Stagnaro-GreenA,ZhangY,etal.Assessmentof competencyinelectrocardiograminterpretationofgraduatingphysician assistantstudents. JPhysicianAssistEduc.2013;24(4):15–8.
19.BreunigMJ,HerberAJ,JenkinsSM,etal.Characteristicsofeffective continuingmedicaleducationforphysicianassistantsandnurse practitionersinhospitalmedicine. JPhysicianAssistEduc 2020;31(1):2–7.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 159 Lindseyetal. ECGCurriculumforAPPEducation
ORIGINAL RESEARCH
StageBHeartFailureIsUbiquitousinEmergencyPatientswith AsymptomaticHypertension
KimberlySouffront,PhD,RN,FNP-BC,FAAN*†‡
BretP.Nelson,MD,RDMS,FACEP*
MeganLukas,RDMS,MBA*
HansReyesGaray,MD,MPH§
LaurenGordon,MPH,CHES*‡∥
ThaliaMatos,BA*
IsabellaHanesworth,BS*
RebeccaMantel,BA*
ClaireShubeck,BA*
CassidyBernstein,BA†
GeorgeT.Loo,DrPH,MPH,MPA,CHPCP*∥
LynneD.Richardson,MD,FACEP*‡∥¶
SectionEditor:FredSeveryn,MD
*IcahnSchoolofMedicineatMountSinai,Departmentof EmergencyMedicine,NewYork
† CenterforNursingResearchandInnovation,MountSinai HealthSystem,NewYork
‡ IchanSchoolofMedicineatMountSinai,InstituteforHealth EquityResearch,NewYork
§ MountSinaiMorningside,DepartmentofCardiology,NewYork
∥ IcahnSchoolofMedicineatMountSinai,Departmentof PopulationHealthScienceandPolicy,NewYork
¶ IcahnSchoolofMedicineatMountSinai,Departmentof ArtificialIntelligenceandHumanHealth,NewYork
Submissionhistory:SubmittedMarch23,2023;RevisionreceivedNovember21,2023;AcceptedDecember19,2023
ElectronicallypublishedFebruary9,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.17990
Introduction: Hypertensionistheleadingriskfactorformorbidityandmortalitythroughouttheworldand ispervasiveinUnitedStatesemergencydepartments(ED).Thisstudydocumentsthepointprevalence ofsubclinicalheartdiseaseinemergencypatientswithasymptomatichypertension.
Method: ThiswasaprospectiveobservationalstudyofEDpatientswithasymptomatichypertension conductedattwourbanacademicEDsthatbelongtoaneight-hospitalhealthcareorganizationin NewYork.Adult(≥18yearsofage)English-orSpanish-speakingpatientswhohadaninitialblood pressure(BP) ≥160/100millimetersofmercury(mmHg)andsecondBP ≥140/90mmHg,andpending discharge,wereinvitedtoparticipateinthestudy.Weexcludedpatientswithcongestiveheartfailure, renalinsufficiency,andatrial fibrillation,orwhowerepregnant,aprisoner,cognitivelyunableto provideinformedconsent,orexperiencingsymptomsofhypertension.Weassessedechocardiographic evidenceofsubclinicalheartdisease(leftventricularhypertrophy,anddiastolicand systolicdysfunction).
Results: Atotalof53patientswereincludedinthestudy;amajoritywereyoung(mean49.5yearsold, [SD14–52]),self-identifiedasBlackorOther(n = 39;73.5%),andfemale(n = 30;56.6%).Meaninitial bloodpressurewas172/100mmHg,and24patients(45.3%)self-reportedahistoryofhypertension. Fiftypatientscompletedanechocardiogram.All(100%)hadevidenceofsubclinicalheartdisease, with41(77.4%)displayingleftventricularhypertrophyand31(58.5%)diastolicdysfunction.There wasasignificantrelationshipbetweendiastolicdysfunctionandfemalegender[x2 (1,n = 53) = 3.98; P = 0.046];Blackorotherrace[x2 (3,n = 53) = 9.138; P = 0.03]andHispanicorotherethnicity [x2 (2,n = 53) = 8.03; P = 0.02].Lessthanonethirdofpatientsdemonstratedsystolic dysfunctiononechocardiogram,andthiswasmorelikelytooccurinpatientswithdiabetesmellitus [x2 (1,n = 51) = 4.84; P = 0.02].
Conclusion: ThereisahighprobabilitythatBlack,Hispanic,andfemalepatientswithasymptomatic hypertensionareonthecontinuumfordevelopingovertheartfailure.Emergencycliniciansshould provideindividualizedcarethatconsiderstheiruniquehealthneeds,culturalbackgrounds,andsocial determinantsofhealth.[WestJEmergMed.2024;25(2)160–165.]
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 160
INTRODUCTION
Hypertensionistheleadingriskfactorformorbidityand mortalitythroughouttheworldandispervasiveinUnited Statesemergencydepartments(ED).1,2 Itisacommon misconceptionthathighbloodpressurefoundduringtheED visitisrelatedtopainoranxiety;however,studiesshowthat elevatedbloodpressureintheEDreliablyidentifies hypertensioninanoverwhelmingmajorityofpatients3 andis anindependentriskfactorforhavingamajorcardiovascular eventafterdischarge.4
TheclinicalpolicyrecommendedbytheAmericanCollege ofEmergencyPhysicians(ACEP)forpatientswhohave persistentasymptomatichypertensionintheED without signsandsymptomsofacutetargetorganinjuryincludes promptreferraltoprimarycare.5 However,thisclinical policyisoutdatedandinconsistentlyadheredto.6
Furthermore,ofthepatientswhoarereferred,lessthanhalf adheretorecommendationsregardingfollow-up,regardless oftheirinsurancestatusoraccesstoroutinemedicalcare.7
Automatingrecommendationsforfollow-upusingthe electronichealthrecordisonewaytoensureadherencetothe ACEPpolicy;8 however,thisisnotusefulormeaningfulto patientswhenitisnotfollowedbyanyclinician-to-patient communicationabouttheimportanceoffollow-up.Besides, thispolicydoesnotconsidertheemergingevidencethatan overwhelmingmajorityofemergencypatients,particularly Blacks,withasymptomatichypertensionshowevidenceof subclinicalheartdisease,9,10 withupperlimitsreachingmore than90.7%inonesentinelstudybyLevyetal.9
Thisstudydocumentsthepointprevalenceofsubclinical heartdiseaseinthe first50EDpatientsenrolledinan ongoingstudy.Thiswasour firststeptoexploreuniqueways forimprovinghealthcaredeliveryforthishigh-riskpatient populationinourED.
METHODS
StudyDesign,Sample,andSetting
Thiswasaprospectiveobservationalstudyofemergency patientswithasymptomatichypertension.Datacollection beganafterreceivinginstitutionalreviewboardapproval (#18-00197).Thestudywasconductedattwourban academicEDsthatprovidecaretoadiversepatient population,servingthelocalcommunitiesofHarlem, NewYorkCity.Harlemhasaconcentratedburdenof hypertensionandisamongthepoorestneighborhoodsin NewYorkCity.Morethantwo-thirdsoftheresidentsarea racialorethnicminority.11
Datacollectionbeganin2018atoneEDsiteandstopped twicedueto1)lackoffundingtocontinueparticipant recruitment,and2)apauseinresearchactivitiesatthe recruitinginstitutioninresponsetotheCOVID-19 pandemic.Inlate2021,anadditionalsitewasaddedtoboost patientrecruitment.Annually,thereareabout100,000 patientvisitsineachED,andthereisastate-of-the-art
electronicinformationsystemtofacilitatedatacollection acrosssites.Theestimatedprevalenceofpatientswhovisit theEDateachsitewhohaveuncontrolledasymptomatic hypertensionis48%–50%.12
Adult(≥18yearsofage)English-orSpanish-speaking patients,whohadaninitialbloodpressure(BP) ≥160/ 100millimetersofmercury(mmHg)andsecond BP ≥140/90mmHgandwerependingdischarge,were invitedtoparticipateinthestudy.Weexcludedpatientswith congestiveheartfailure,renalinsufficiency,andatrial fibrillation,orwhowerepregnant,aprisoner,cognitively unabletoprovideinformedconsent,orexperiencing symptomsofhypertension,suchaschestpain,paresthesia,or shortnessofbreath.
ProtocolChange
Initiallywealsoexcludedpatientsiftheyhadtakenblood pressuremedicationwithinthepriorthreemonths;however, beginningatstudyparticipantnumber44,wemodifiedour protocoltoincludeallpatientsregardlessofwhetherthey hadtakenmedicationwithinthepreviousthreemonths.We revisedthestudyprotocoltobemoreinclusive,specificallyto avoidexcludingpatientswhomighthaveuntreated conditionsandtoensurethatallindividuals,includingthose potentiallyinneedofintervention,couldbeconsideredinour research.Thischangeinprotocolallowedforamore comprehensiveassessmentofthestudypopulationandalso assistedwithourrecruitment.
DataCollectionProcedure
Approximatelythreedays/weekfrom8 AM to4 PM,a researchcoordinatorenrolledandobtainedinformed consentelectronicallyfrompatientswhometinclusion criteria.Studydatawerecollectedandmanagedusing REDCap(ResearchElectronicDataCapture)13,14 electronicdatacapturetoolshostedatourinstitution.1,2 REDCapisasecure,web-basedsoftwareplatformdesigned tosupportdatacaptureforresearchstudies,providing1)an intuitiveinterfaceforvalidateddatacapture;2)audittrails fortrackingdatamanipulationandexportprocedures; 3)automatedexportproceduresforseamlessdatadownloads tocommonstatisticalpackages;and4)proceduresfordata integrationandinteroperabilitywithexternalsources.13,14
Eachmorningofpatientrecruitment,theresearch coordinatorusedMicrosoftTeams(MicrosoftCorp, Redmond,WA)toinformthesonographerthatrecruitment hadbegun.15 BeforeapproachinganypatientintheED,the researchcoordinatorusedtheEpicchatfeature(Epic SystemsCorp,Verona,WI)toinformtheEDattending physicianthathis/herpatientmetinclusionforthestudyand wouldbeapproached.15 Ifanelectrocardiogram(ECG)was notconductedaspartofusualcare,theECGwasconducted bytheresearchcoordinator.Thebedsideechocardiogram wasconductedbyonememberofthestudyultrasoundteam,
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 161 Souffrontetal. StageBHeartFailureinPatientswithAsymptomaticHypertension
whichincludedoneEDsonographer,fouremergency physicians,andonecardiologist,eachtrainedonthestudy protocol.Eachechocardiogramtookapproximately 10minutestocompleteatbedside.
Measures
DemographicMeasures
Studymeasureswereobtainedbyelectronicdata extractionandincludedthepatient’ s firstandsecondsystolic anddiastolicBPlevelsincearrivaltotheED,age,gender, race/ethnicity,pastmedicalhistory,smokinghistory,and accesstoprimarycare.Alldataweredoubleenteredintothe REDCapdatabasebytwomembersoftheresearchteamto assureaccuracyofchartreviewanddataentry.Ourprotocol wassetsuchthatanydifferenceswerereconciledbythe principalinvestigator(KS);however,nodifferences wereidentified.
Echocardiogram
Threeendpointswereobtainedandmodeledafterthe AmericanCollegeofCardiology,AmericanHeart Association(ACC/AHA),HeartFailureSocietyofAmerica, andLevyetal.9,20:
1. Leftventricularhypertrophy(LVH) .Definedas presentifleftventricularseptalthicknesswas ≥ 1.1 centimeter(cm)orabsentifleftventricularseptal thicknesswas <1.1cm.(Leftventricularfreewall thicknesswasmeasuredintheparasternallongview atenddiastole.)
2. Systolicdysfunction. Ejectionfractionnotedas normal(≥55%)orabnormal(<55%).
3. Diastolicfunction. Evaluatedbyestimatingleftatrial pressureusingtheE/e’ ratio.Diastolicdysfunctionwas presentifE/e’(septal)was ≥15,ifE/e’(lateral)was ≥12, ifseptale’ was <8cm/sec,oriflaterale’ was <10cm/sec.3 Gradeofdiastolicdysfunctionwasnot determinedforthisinitialanalysis.
Electrocardiogram
ThestudydiagnosedLVHwithECG findingsthat satisfiedtheCornellvoltagecriteria(whenthesumoftheR waveinleadaVLandtheSwaveinleadV3is >20mmin womenand23mminmen).DataobtainedfromtheECG includedheartrate,PRinterval,andtheQRS intervalnumber.
DataAnalysisPlan
WeexporteddatafromREDCaptoSASanalytic softwareversion9.4(SASInc,Cary,NC)fordataanalysis. Demographicvariablesarepresentedaspercentages(%)or means(M )andstandarddeviations.Evidenceofsubclinical heartdiseasewasdichotomizedasabnormal/normaland
presentedaspercentages.Weconductedbivariateanalysesto testforsignificantrelationshipsbetweenindependent variables(age,gender,race/ethnicity,pastmedicalhistory, bodymassindex,smokinghistory,andaccesstoprimary care)andsubclinicalheartdisease.
RESULTS
CharacteristicsofSample
Atotalof53patientswereexamined.(Twopatientsdid notcompleteanechocardiogramandonepatientdidnot completeanechocardiogramandECGduetotime constraints.)Themajorityofthesamplewereyoung (mean49.5yearsold,[SD14–52]),self-identifiedasBlackor Other(n = 39;73.5%),andfemale(n = 30;56.6%). 31wereHispanicrace(58.5%)and30female(56.6%).Mean initialBPwas172/100mmHg,and24patients(45.3%) self-reportedahistoryofhypertension;36(68%)hadtaken theirantihypertensivemedicationwithinthepriorthree months.Meanbodymassindexwas31.9,whichcorrelatesto ObesityClassIormoderatelyobese.17 Fifteenpatients (28.3%)werecurrentsmokers,and12(28.3%)]hadahistory ofdiabetesmellitus.Thirty-one(58.5%)reportedtheyhada primarycarephysician(Table1).
Echocardiogram
Ofthe50patientswhocompletedanentire echocardiogram,100%hadevidenceofsubclinicalheart disease;41(77.36%)showedevidenceofLVHand31 (58.49%)diastolicdysfunction.Weperformedachi-square testofindependencetoassesstherelationshipbetween subclinicalheartdiseaseandourindependentvariables. Therewasasignificantrelationshipbetweendiastolic dysfunctionandfemalegender[x2 (1,n = 53) = 3.98; P = 0.046];Blackorotherrace[x2 (3,n = 53) = 9.138; P = 0.03] andHispanic[x2 (2,n = 53) = 8.03; P = 0.02].Lessthanonethirdofpatientsdemonstratedsystolicdysfunctionon echocardiogram,andthiswasmorelikelytooccurinpatients withdiabetesmellitus[x2 (1,n = 51) = 4.84; P = 0.02].There werenootherdifferencesbetweenourindependentvariables andourmainoutcome,subclinicalheartdisease.
OneparticipantwasmissinganECG.Of49participants, fivedemonstratedevidenceofLVH.
DISCUSSION
Wefoundthatsubclinicalheartdiseaseisubiquitousin EDpatientswithasymptomatichypertension.Our findings areconsistentwiththoseofLevyetal(2012),whowerethe firsttodocumentthealarmingprevalenceofstructuralheart changesinEDpatientswithasymptomatichypertension, particularlyamongBlackswhosoughtcareintheEDofan innercity.9 Ourstudyaddstotheexistingliteraturethat BlackandHispanicpatientswhohaveasymptomatic hypertensiondiagnosedintheEDhaveahighprobabilityof havingmyocardialtargetorgandamage,thatisconsistent
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 162 StageBHeartFailureinPatientswithAsymptomaticHypertension Souffrontetal.
MeasureVariableStatisticN = 53(%)
Independentvariables – demographiccharacteristicsAge,mean(SD)49.47(14.5)
Gender,n(%)
Female30(56.6)
Male23(43.4)
Race,n(%)
White5(9.4)
Blackorother48(90.6)
Ethnicity,n(%)
Non-Hispanic27(49.1)
Hispanicorother31(58.5)
Bloodpressure,meanmmHg
1st systolicbloodpressure172.13(15.1)
1st diastolicbloodpressure100.74(15.0)
2nd systolicbloodpressure162.27(17.5)
2nd diastolicbloodpressure95.37(16.5)
Hypertensionhistory,yes24(45.3%)
Tookbloodpressuremedicinewithin3months36(68.0%)
Smoker,yes15(28.3%)
Diabetesmellitushistory,yes12(22.6%)
Bodymassindex,mean(SD)31.93(8.9)
Primarycarephysician,yes31(58.5)
Dependentvariable – subclinicalheartdiseaseSubclinicalheartdisease,yes50(100)
Diastolicdysfunction,yes31(58.5)
Leftventricularhypertrophy,yes41(77.4)
Systolicdysfunction,yes10(19.6)
withadiagnosisofStageBheartfailureaccordingto theACC/AHA.18
Congestiveheartfailureisaprogressivedisorder(Stages A,B,C,andD)thatoftenbeginswithleftventricularsystolic dysfunctionandresultsinsymptomsfrom fluidoverloadand poorend-organperfusion.18 StageBheartfailure encompassespatientswhoareasymptomaticandhave evidenceofstructuralheartabnormalities,suchasLVH, systolicordiastolicdysfunction.18 Asymptomaticleft ventricularsystolicdysfunctionhasanestimatedprevalence of3–6%inthecommunity,19 whichissignificantlylessthan ourstudy findingsof19%.Prendergastetal10 (2015)found thepointprevalenceofsubclinicalheartdiseaseinemergency patientswithasymptomatichypertensiontobenearly40%, alsosignificantlylessthanour findingsof100%.Toour knowledge,nootherstudyexiststhatdocumentsthepoint prevalenceofsubclinicalheartdiseaseinEDpatientswith asymptomatichypertension.BothLevyetal(2012)and Prendergastetal(2015)conductedtheirstudiesininner-city
Detroitandinner-cityChicago,respectively settings similartoours.9,10
Inrandomizedtrials,individualswithasymptomaticleft ventricularsystolicdysfunctionhavehighratesofincident heartfailureanddeath,increasedcardiovascular mortality19–21;all-causemortality20;andnonfatal cardiovascularevents,suchasmyocardialinfarctionand stroke.20,22 Whiletherelativecontributionofeachcondition (systolicanddiastolicdysfunction,orLVH)toovert congestiveheartfailureisunknown,fromthelensof emergencymedicineandpopulationhealth,identifying patientswithasymptomatichypertensionisofutmost importance,regardless.
OurstudydemonstratesthatEDpatientswith asymptomatichypertensionareonthecontinuumtoovert heartfailure,whichhasseriousconsequences.Aconsiderable numberofmissedopportunitiesoccur,whichcontributeto delaysindiagnosis,suboptimaltreatment,increased morbidityandmortalityand,aboveall,anexacerbationof
Table1. Demographiccharacteristicsandsubclinicalheartdisease.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 163
Souffrontetal. StageBHeartFailureinPatientswithAsymptomaticHypertension
disparitiesrelatedtohypertensionamongBlacks.Studies suggestreferralratesare5%.12 Despitethemanyknowledge, attitudinal,andorganizationalbarriersthatexist,6,23,24 theidentificationandmanagementofasymptomatic hypertensioniscriticaltopreventtheprogression ofthediseaseanditscomplications.
LIMITATONS
Theresultsofourstudymustbeinterpretedconsideringits limitations,whichincludeitssmallsamplesizeandstudy design.Weanalyzedonlythe first50patientsofanongoing study,whichlimitstheabilitytogeneralizetolarger populations.Additionally,ourprospectiveobservational studydesignlimitstheabilitytodrawconclusionsabout theprogressionofcongestiveheartfailureorother cardiovasculareventsovertime.Lastly,wedidnotevaluate subclinicalheartdiseaseinnormotensivepatients. Nonetheless,thisstudyhasstrengthsinthatitdocumentsthe significantburdenofStageBheartfailureintheemergency populationwithasymptomatichypertension.
IMPLICATIONS
Emergencycliniciansshouldrecognizetheprevalenceof asymptomatichypertensioninBlackpatientsandprovide individualizedcarethatconsiderstheiruniquehealthneeds, culturalbackgrounds,andsocialdeterminantsofhealth. Thisincludesprovidingadequatefollow-upcareandsupport topreventhypertension-relatedcomplicationsandimprove overallhealthoutcomes.
CONCLUSION
Inthisstudywedocumentedthepointprevalenceof subclinicalheartdiseaseinthe first50EDpatientswhowere enrolledinanongoingstudy,asour firststeptoexplore uniquewaysforimprovinghealthcaredeliveryforthishighriskpatientpopulationinouremergencydepartment.While subclinicalheartdiseasewasubiquitous,weconcludethat female,Black,andHispanicpatientswhohave asymptomatichypertensiondiagnosedintheEDhaveahigh probabilityofalreadybeingonthecontinuumfordeveloping overtheartfailure.
ACKNOWLEDGMENTS
Weexpressoursincereappreciationtotheancillarystaff, staffnurses,andmedicalstaffoftheMountSinaiEmergency Departmentfortheirinvaluablecontributionstothisstudy. Ourgratitudeextendstothepatientswhoseparticipation madethisresearchpossible.SpecialthankstotheSouffront Collaboratorymembers,includingnursingleadership AshleyCacerasMSN,RN,LucioBarettoDNP,AGACNPBC,APRN-BC,RN,CCRN,NE-BC,CPXP,CLSSBB, MICN,andOliviaSgambelloneBSN,RN,CENwhose guidanceandsupportenrichedthecollaborativenatureof thisstudy.
AddressforCorrespondence:KimberlySouffrontPhD,IcahnSchool ofMedicineatMountSinai,DepartmentofEmergencyMedicine, 555West57th St.,5th Floor,Room5-33,NewYork,NY10019.
Email: Kimberly.souffront@mountsinai.org
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Thisprojectwasfundedbythe DepartmentofEmergencyMedicineattheIcahnSchoolofMedicine atMountSinai,theCenterforNursingResearchandInnovationat theMountSinaiHealthSystem,andtheGordonandBettyMoore FoundationthroughtheBettyIreneMooreFellowshipforNurse LeadersandInnovators(GBMF9048).Therearenootherconflictsof interestorsourcesoffundingtodeclare.
Copyright:©2024Souffrontetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.CenterforDiseaseControl.Factsabouthypertension.2023. Availableat: https://www.cdc.gov/bloodpressure/facts.htm AccessedFebruary1,2023.
2.McNaughtonCD,SelfWH,ZhuY,etal.IncidenceofhypertensionrelatedemergencydepartmentvisitsintheUnitedStates,2006to2012. AmJCardiol. 2015;116(11):1717–23.
3.GoldbergEM,WilsonT,SaucierC,etal.AchievingtheBpTRUth: emergencydepartmenthypertensionscreeningandtheCentersfor Medicare&MedicaidServicesqualitymeasure. JAmSocHypertens. 2017;11(5):290–4.
4.OrasP,HäbelH,SkoglundPH,etal.Elevatedbloodpressureinthe emergencydepartment:ariskfactorforincidentcardiovasculardisease. Hypertension. 2020;75(1):229–36.
5.WolfSJ,LoB,ShihRD,etal.AmericanCollegeofEmergency PhysiciansClinicalPoliciesCommittee.Clinicalpolicy:criticalissuesin theevaluationandmanagementofadultpatientsintheemergency departmentwithasymptomaticelevatedbloodpressure. AnnEmerg Med. 2013;62(1):59–68.
6.SouffrontK,ChyunD,KovnerC.Barrierstoreferralforelevatedblood pressureintheemergencydepartmentanddifferencesbetween providertype. JClinHypertens(Greenwich). 2015;17(3):207–14.
7.CornellSD,ValerioMA,KrauseT,etal.Lowadherencetopost emergencydepartmentfollow-upamonghypertensivepatientswith medicalinsurance. JEmergMed. 2020;58(2):348–55.
8.SouffrontK,RiveraL,LooG,etal.Integratingaclinicaldecisionsupport remindertoimprovebloodpressurereassessmentforpatientswith uncontrolledhypertension. ClinCardiolCardiovascInterv. 2019;2:01–3.
9.LevyP,YeH,ComptonS,etal.Subclinicalhypertensiveheartdisease inblackpatientswithelevatedbloodpressureinaninner-city emergencydepartment. AnnEmergMed. 2012;60(4):467–74.e1.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 164 StageBHeartFailureinPatientswithAsymptomaticHypertension Souffrontetal.
10.PrendergastHM,CollaJ,PatelN,etal.Correlationbetweensubclinical heartdiseaseandcardiovascularriskprofilesinanurbanemergency departmentpopulationwithelevatedbloodpressures:apilotstudy. JEmergMed. 2015;48(6):756–61.
11.FurmanCenterforRealEstateandUrbanPolicy.Harlemstatistics. Availableat: https://furmancenter.org/neighborhoods/view/ central-harlem.AccessedFebruary1,2023.
12.SouffrontK,GestalC,MelkusGD,etal.Recognitionofasymptomatic hypertensioninanurbanemergencydepartmentwherearewenow? AdvEmergNursJ. 2016;38(4):320–6.
13.HarrisPA,TaylorR,ThielkeR,etal.Researchelectronicdatacapture (REDCap)–ametadata-drivenmethodologyandworkflowprocessfor providingtranslationalresearchinformaticssupport. JBiomedInform. 2009;42(2):377–81.
14.HarrisPA,TaylorR,MinorBL,etal.TheREDCapconsortium:building aninternationalcommunityofsoftwareplatformpartners. JBiomed Inform. 2019;95:103208.
15.EPICResearch.Availableat: https://www.epicresearch.org/ AccessedFebruary1,2023.
16.CenterforDiseaseControl.DefiningAdultOverweight&Obesity.2022. Availableat: https://www.cdc.gov/obesity/basics/adult-defining.html AccessedFebruary1,2023.
17.GoldbergLRandJessupM.StageBheartfailure:managementof asymptomaticleftventricularsystolicdysfunction. Circulation. 2006;113(24):2851–60.
18.WangTJ,LevyD,BenjaminEJ,etal.Theepidemiologyof “asymptomatic” leftventricularsystolicdysfunction:implicationsfor screening. AnnInternMed. 2003;138(11):907–16.
19.SaraJD,ToyaT,TaherR,etal.Asymptomaticleftventriclesystolic dysfunction. EurCardiol. 2020;15:e13.
20.AmmarKA,JacobsenSJ,MahoneyDW,etal.Prevalenceand prognosticsignificanceofheartfailurestages:applicationofthe AmericanCollegeofCardiology/AmericanHeartAssociationheart failurestagingcriteriainthecommunity. Circulation. 2007;115(12):1563–70.
21.Echouffo-TcheuguiJB,ErqouS,ButlerJ,etal.Assessingtheriskof progressionfromasymptomaticleftventriculardysfunctiontoovertheart failure:asystematicoverviewandmeta-analysis. JACCHeartFail. 2016;4(4):237–48.
22.SouffrontK,ShastryS,BennettC,etal.Emergencynursingpolicyand hypertensionawareness:anintegrativereviewoftheliterature. CurrHypertensRep. 2019;21(7):55.
23.SouffrontK,ChyunD,KovnerC,etal.Barriersandfacilitators toreferralforasymptomatichypertensionintheemergencydepartment: amultidisciplinarysurveyofEDproviders. JHypertensManag. 2016;2(1).
24.BrodyA,TwinerM,KumarA,etal.Surveyofemergencyphysician approachestomanagementofasymptomatichypertension. JClin Hypertens(Greenwich).2017;19(3):265–9.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 165
Souffrontetal. StageBHeartFailureinPatientswithAsymptomaticHypertension
REVIEW
PerformanceofIntra-arrestEchocardiography: ASystematicReview
Yi-JuHo,MD*
Chih-WeiSung,MD*†
Yi-ChuChen,PhD‡
Wan-ChingLien,MD,PhD*§
Wei-TienChang,MD,PhD*§
Chien-HuaHuang,MD,PhD*§
*NationalTaiwanUniversityHospital,DepartmentofEmergencyMedicine, Taipei,Taiwan
† NationalTaiwanUniversityHsin-ChuHospital,DepartmentofEmergency Medicine,Hsinchu,Taiwan
‡ NationalTaiwanUniversity,InstituteofEpidemiologyandPreventiveMedicine, CollegeofPublicHealth,Taipei,Taiwan
§ NationalTaiwanUniversity,CollegeofMedicine,DepartmentofEmergency Medicine,Taipei,Taiwan
SectionEditor:JosephShiber,MD
Submissionhistory:SubmittedAugust21,2023;RevisionreceivedDecember8,2023;AcceptedDecember21,2023
ElectronicallypublishedFebruary9,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18440
Introduction: Intra-arresttransthoracicechocardiography(TTE)andtransesophageal echocardiography(TEE)havebeenintroducedinadultpatientswithcardiacarrest(CA).Whetherthe diagnosticperformanceofTTEorTEEissuperiorduringresuscitationisunclear.Weconducteda systematicreviewfollowingPRISMAguidelines.
Methods: WesearcheddatabasesfromPubMed,Embase,andGoogleScholarandevaluatedarticles withintra-arrestTTEandTEEinadultpatientswithnon-traumaticCA.Twoauthorsindependently screenedandselectedarticlesforinclusion;theythendual-extractedstudycharacteristicsandtarget conditions(pericardialeffusion,aorticdissection,pulmonaryembolism,myocardialinfarction, hypovolemia,leftventriculardysfunction,andsonographiccardiacactivity).Weperformedquality assessmentusingtheQualityAssessmentofDiagnosticAccuracyStudiesVersion2criteria.
Results: Atotalof27studieswereincluded:14studieswith2,145patientsassessedTTE;and16with 556patientsassessedTEE.Ahighriskofbiasorapplicabilityconcernsinatleastonedomainwas presentin20studies(74%).BothTTEandTEEfoundpositive findingsinnearlyone-halfofthepatients. TheetiologyofCAwasidentifiedin13%(271/2,145),andinterventionwasperformedin38%(102/271) ofpatientsintheTTEgroup.InpatientswhoreceivedTEE,theetiologywasidentifiedin43%(239/556), andinterventionwasperformedin28%(68/239).IntheTEEgroup,ahigherincidenceregardingthe etiologyofCAwasobserved,particularlyforthosewithaorticdissection.However,theoutcomeofthose withaorticdissectionintheTEEgroupwaspoor.
Conclusion: WhileTEEcouldidentifymorecausesofCAthanTTE,sonographiccardiac activitywasreportedmuchmoreintheTTEgroup.TheimpactofTTEandTEEonthereturnof spontaneouscirculationandfurthersurvivalwasstillinconclusiveinthecurrentdataset.[WestJEmerg Med.2024;25(2)166–174.]
Keywords: cardiacarrest;resuscitation;transthoracicechocardiography; transesophagealechocardiography.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 166
INTRODUCTION
Ultrasound(US)isconsideredavaluablediagnostictool whenthereisaclinicalsuspicionforaspecificreversible causeinpatientswithcardiacarrest(CA).1 TheuseofUS duringresuscitationhasbecomemorecommonbecauseofits non-invasiveandreadilyaccessiblecharacteristics.2,3
Transthoracicechocardiography(TTE)hasbeen introducedinresuscitativescenariosinrecentdecades.2,4,5 However,previousstudieshaveshownthatTTElengthensa singlepauseformorethan17seconds,6,7 possiblydelaying chestcompressions.Also,devicessuchasmechanicalchest compressionsystemsordefibrillationpadswouldinterfere withimageacquisition.Bycontrast,transesophageal echocardiography(TEE)couldovercomesuchlimitations, allowingreal-timevisualizationoftheheartwithout interruptingchestcompressions.8 However,thedisadvantages ofTEEincludehighcost,theneedforadvancedoperator skills,andthepotentialforiatrogenictraumaduetoits invasivenature.WhetherthediagnosticperformanceofTTE orTEEissuperiorduringresuscitationisunclear.We conductedasystematicreviewofintra-arrestTTEandTEE ontargetconditionsincludingpericardialeffusion,cardiac tamponade,aorticdissection,pulmonaryembolism(PE), myocardialinfarction(MI),hypovolemia,leftventricular (LV)dysfunction,andsonographiccardiacactivity.
METHODS
Weperformedasystematicreviewfollowingthelatest statementofthePreferredReportingItemsforSystematic ReviewsandMeta-Analysis(PRISMA2022).Thisreview protocolwassubmittedtotheInternationalProspective RegisterofSystematicReviews(PROSPERO)onMarch17, 2022(registrationnumber:CRD42022310670).
DataSourcesandSearchStrategy
Twoindependentinvestigatorssearchedliterature publisheduptoApril30,2023,inPubMed,Embase,and GoogleScholarwithoutlanguageorstudy-typerestriction. Eligibletrialswereidentifiedwiththefollowingkeywords: “echocardiography,CA,resuscitationorrescue.”
PopulationHealthResearchCapsule
Whatdowealreadyknowaboutthisissue?
Transthoracicechocardiography(TTE) delayschestcompressions;transesophageal echocardiography(TEE)offers real-timevisualizationwithout interruptingcompressions.
Whatwastheresearchquestion?
IsthediagnosticperformanceofTTE superiortoTEE?
Whatwasthemajor findingofthestudy?
Theetiologyofcardiacarrest(CA)was identi fi edin13%ofpatientsthroughTTEand in43%ofpatientsthroughTEE.
Howdoesthisimprovepopulationhealth?
WhileTEEcouldidentifymorecausesofCA sonographiccardiacactivity,itwasreported muchmoreintheTTEgroup.Theimpact ofTTEandTEEonfurthersurvival areinconclusive.
StudySelection
Twoauthors(YHandWL)independentlyexamined referencesusingtitlesandabstracts.Fulltextsofrelevant studieswereretrieved.Thestudyselectioncriteriawere framedusingthePICOST(Population,Intervention, Comparator,Outcome,StudyDesign,Timeframe)format asdescribedin Table1
DataExtractionandQualityAssessment
Thetwoauthors(YHandWL)extracteddatafrom eligiblestudiesincludingauthors,publicationyear,study design,casenumbers,gender,age,applicationofTTEor
InclusioncriteriaExclusioncriteria
PopulationAdultnon-traumaticpatientswithout-of-hospitalandin-hospital cardiacarrest
Traumaticarrestandpediatricpopulation
InterventionIntra-arrestTTE/TEEPost-arrestTTE/TEE
ComparativeStandardresuscitationaccordingtoALSguidelines
OutcomeIdenti ficationofthetargetconditions
StudytypeObservationalstudies(prospectiveandretrospective)and interventionalstudies(randomizedandnon-randomized)
TimeNolimitationsonthepublicationperiod
Casereportsandcaseseries,animalstudies, reviewarticles,guidelines,andeditorials
TEE,transesophagealechocardiography; TTE,transthoracicechocardiography; ALS,AdvancedLifeSupport.
Table1. Eligibilitycriteriaforstudyselection.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 167
Hoetal. PerformanceofIntra-arrestEchocardiography
TEE,sonographers,sonographicdiagnoses,reference standard,andrateofreturnofspontaneouscirculation (ROSC)orsurvival.Qualityassessmentwasperformedusing theQualityAssessmentofDiagnosticAccuracyStudies
Version2(QUADAS-2)criteria,whichisanadequatetool fordiagnostictestaccuracyinsystematicreviews.9
Thetwoauthorsindependentlyevaluatedeachincluded study;anyinconsistencyordisagreementwasresolvedupon detaileddiscussion.
OutcomeMeasures
Theprimaryoutcomewastheincidenceproportionsof targetconditionsincludingpericardialeffusion,cardiac tamponade,aorticdissection,PE,MI,hypovolemia, LVdysfunction,andsonographiccardiacactivityby intra-arrestTTE/TEE.
DataSynthesis,StatisticalAnalysis,and SensitivityAnalysis
Duetomarkedheterogeneityamongtheincludedstudies, conductingarobustmeta-analysiswasnotfeasible.Thus,we usedanarrativesynthesistopresentthestudyresults.We calculatedthepooledincidenceproportionsoftarget conditionsofTTEandTEEandpresentthemas proportionand95%confidenceintervals(CI)using ComprehensiveMeta-AnalysisV4.0software(BiostatInc, Englewood,NJ).
RESULTS
LiteratureSearchandStudySelectionResults
Ourliteraturesearchidentifiedatotalof333resultsfrom PubMed,Embase,andGoogleScholar.Afterduplicates wereremoved,wescreened308titlesforinclusionwith34 excludedfornotmeetingthestudycriteria.Wealsoexcluded 31meta-analysesorsystematicreviewsand61casereports, caseseries,andanimalstudies.Ofthe182remainingstudies thatunderwentathoroughfull-textretrievalandreview,we selected27for finalreview(Figure).2,3,10–34
SummaryofStudies
ElevenstudiesassessedTTE,2,3,10–20 13assessed TEE,10,13,21–33 andthreeincludedboth(Supplementary Table1).10,13,34 Ofthetotal2,701patientsincluded,2,145 receivedTTE,and556receivedTEE.Onepatientwas excludedduetoatraumaticruptureofthethoracicaorta.21 Ninestudiesincludedpatientswithout-of-hospitalCA (OHCA),sevenincludedpatientswithin-hospitalCA,and another11assessedamix.Echocardiographywasperformed byemergencyphysiciansin17studies.
RiskofBiasandConcernsofApplicability
Ahighriskofbiasorapplicabilityconcernsinatleastone domainwaspresentin20studies(74%)(Table2).Theriskof biaswasunclearin10studies(38%)duetoalackof
informationregardingpatientselection.Theriskwashighin 15studies(58%)duetoaconveniencesample;onlyonestudy ratedlowriskenrolledaconsecutivesample.14 Intwostudies inwhichTTE,TEE,orbloodsamplinganalysiswas performedatthediscretionofphysicians19 theriskofbias wasratedhighbecauseofconcernfortheindextest.Therisk ofbiaswasunclearin23studies(88%)relatedtoalackof standardizedreferenceandinformationregardingthe flow andtiming.Lowrisksofbiasrelatedtoreferencestandards and flowandtimingwereratedintheotherfourstudiesin whichallimageswerereviewed(ie,uniformconfirmatory testing)andinter-raterreliabilitywasassessed.26,31,32,34
Theriskofbiasforapplicabilitywashighforpatient selectioninninestudies(35%)becauseoftheenrollmentof patientswithhemodynamicinstabilityorpulselesselectrical activity.2,11–14,22,26–28 Forapplicabilitytotheindextest,four studieswereevaluatedwithahighriskduetolargevariations indiagnosticassessment.10,13,19,34 Fourstudiesinwhich referencestandardswereprovidedwereratedwithalowrisk ofapplicabilityinreferencestandards.26,31,32,34 However,the remainingwereunclear.
PerformanceofTTEandTEEAmongthe TargetConditions
TransthoracicechocardiographyandTEEfoundpositive findingsin51%(1,101/2,145)and47%(264/556)ofpatients, respectively.Themostcommon findingwasthepresenceof sonographiccardiacactivityin855patients(830intheTTE groupand25intheTEEgroup).TheetiologyofCAwas identifiedin13%(271/2,145)ofpatientswithTTEand43% (239/556)ofpatientswithTEE(Table3, Supplementary Table2).Ahighincidenceproportionwasobservedinthe targetconditionintheTEEgroup,particularlyinthosewith aorticdissection.However,theoutcomeofpatientswith aorticdissectionwaspoor.
Thesummaryofdetailedsonographic findingsislistedin SupplementaryTable3.
InterventionAccordingtoSonographicFindings
Excludingpatientswithsonographiccardiacactivity,the effectonmanagementwasobservedin38%(102/271)of patientsreceivingTTEand28%(68/239)ofthosereceiving TEE.Themostcommoninterventionwaspericardial effusiondrainage,whichwasperformedin51%(58/113)of patientsintheTTEgroup,and51%(18/35)inthe TEEgroup.Surgerywasperformedononepatient(1/10, 10%)withsuspectedaorticdissectionintheTTEgroup17 and two(2/54,4%)intheTEEgroup.2 Thrombolysis/ embolectomywasperformedinsevenpatients(7/34,21%) and28patients(28/70,40%)withsuspectedPEinthe TTE3,16 andTEE10,24,25,27,30,31 groups,respectively. Coronaryangiographyorbypasswasperformedonone patient(1/4,25%)withsuspectedMIbyTTE17 and11 patients(11/28,39%)byTEE.21,24,25 Administrationof fluid
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 168 PerformanceofIntra-arrestEchocardiography Hoetal.
wasreportedin13patients(13/26,50%)withhypovolemia byTTE,2,12 andfourpatients(4/6,67%)byTEE.13,24,25 Inotropictherapywasadministeredin26%(22/84)of patientswithLVdysfunctionbyTTE2 and11%(5/46) byTEE.25,27
DISCUSSION
Intra-arrestTTEwasperformedin2,145patientsin14 studies,andTEEwasusedin556patientsin16studies.Both TTEandTEEfoundpositive findingsinnearlyone-halfof thepatients.TheetiologyofCAwasidentifiedin13%of patientswithTTEand43%ofpatientswithTEE.Prompt therapywasadministeredin38%ofpatientswithTTEpositive findingsand28%ofthosewithTEE-positive findings.IntheTEEgroup,ahigherincidenceproportion wasobservedinidentifyingtheetiologyofCA,particularly forthosewithaorticdissection.However,ahighdegreeof heterogeneityinreferencestandardsandsmall-samplesize
precludedfurthermeta-analysisforthediagnostic performanceofintra-arrestTTEandTEE.
Themajorweaknessoftheincludedstudiesisthatthe referencestandardsareinconsistent.Applyingauniform standardtoalltargetconditionsisnoteasy,andimagereview maybeaneffectivesolution.26,31,32,34 Theautopsycanbe regardedasthegoldstandardandwasperformedinthree studies,21,22,25 butthereferencestandardsarediversein patientswithROSC.Usingspecificmanagementasreference standardstojudgetargetconditionsisnotfeasible.For example,patientswithsuspectedPEdonotreceive pericardiocentesis.Also,evenfailureinpericardiocentesis doesnotindicatetheabsenceofpericardialeffusion.By contrast,pericardialeffusiondrainageisrarelyperformedin asmallamountofeffusion.True-positivecaseswereoften reported,butverificationbiasexisted.Informationwas limitedinthetrue-negative,false-positive,orfalse-negative cases.Therefore,thissystematicreviewpresentedthe
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 169
Figure. ThePreferredReportingItemsforSystematicReviewsandMeta-Analyses(PRISMA)diagram.
Hoetal. PerformanceofIntra-arrestEchocardiography
Table2. Thequalityassessmentofdiagnosticaccuracystudies(QUADAS)-2riskofbiasassessmentoftheincludedstudies.
RiskofbiasConcernsregardingapplicability Study Patient selection Index test Reference standard Flowand timing Patient selection Index test Reference standard
Transthoracicechocardiography
Varrialeetal(1997)HLUULHU
Kürkciyanetal(2000)LHLLLHL
Tayaletal(2003)ULUUHLU Breitkreutzetal(2010)HLUUHLU Chardolietal(2012)HLUUHLU Shillcuttetal(2012)HLUUHHU
Flatoetal(2015)LLUUHLU Gasparietal(2016)HLUULLU Zenginetal(2016)ULUULLU Chuaetal(2017)ULUULLU Lienetal(2018)HLUULLU Balderstonetal(2021)HLUULLU Heikkilä1etal(2023)UHUULHU Lienetal(2023)HLUULLU
Transesophagealechocardiography
Varrialeetal(1997)HLUULHU vanderWouwetal (1997) ULUULLU
Comessetal(2000)ULUUHLU Kürkciyanetal(2000)LHLLLHL Miyakeetal(2004)HLUUHLU Linetal(2006)ULUULLU Mentsoudisetal(2006)ULUULLU Shillcuttetal(2012)HLUUHHU Hilberathetal(2014)HLLLHLL Burrageetal(2015)ULUUHLU Arntfieldetal(2016)ULUUULU
Teranetal(2019)HLUULLU Jungetal(2020)ULUULLU Kimetal(2020)HLLLLLL Jungetal(2022)HLLLLLL Poppe(2023)HLUULLU H,highriskofbias; L,lowriskofbias; U,unclearriskofbias.
incidenceoftargetconditionsandcouldnotfurtherexplore whetherTTEorTEEwasbetterduringresuscitation.
Oneofthemostimportantindicatorsofthelikelihoodof ROSCorsurvivalisthepresenceofsonographiccardiac activity.35 Sonographiccardiacactivitywasmuchmore frequentlydetectedbyTTEthanbyTEE(830vs25)inour review.Interestingly,regionalwallmotionabnormality suggestiveofMIandLVdysfunctionwasdetectedinthe
presenceofsonographiccardiacactivity.Evenaddingthe numbersoftheseconditions,thetotalnumberwasstillhigher inpatientswithTTE.Whetheritoccurredduetotakinga longertimetosetupTEE(possiblyresultinginresuscitation timebias36)wasuncertain.Furtherstudiesareneededto determinewhetherTEEcanbettercharacterizeintra-arrest myocardialmovementorcardiacactivitydetected byTEEisunder-reported.TheimpactofTTEandTEEon
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 170
PerformanceofIntra-arrestEchocardiography Hoetal.
Findings
TransthoracicechocardiographyN; incidenceproportion[95%CI]
TransesophagealechocardiographyN; incidenceproportion[95%CI]
Pericardialeffusion113;0.068[0.046;0.100]35;0.117[0.056;0.226]
Cardiactamponade30;0.059[0.041;0.083]25;0.095[0.036;0.228]
Aorticdissection10;0.023[0.008;0.064]54;0.119[0.074;0.186]
Pulmonaryembolism34;0.053[0.021;0.126]70;0.220[0.116;0.378]
Myocardialinfarction4;0.022[0.002;0.192]28;0.291[0.131;0.528]
Hypovolemia26;0.044[0.013;0.142]6;0.147[0.067;0.291]
LVdysfunction84;0.181[0.086;0.343]46;0.535[0.170;0.866]
Sonographiccardiacactivity830;0.488[0.374;0.604]25;0.243[0.138;0.390]
LV,leftventricular; CI,confidenceinterval.
ROSCandfurthersurvivalwasstillinconclusiveinthe currentdataset.
Themostcommoninterventionwaspericardialeffusion drainageduringresuscitation.Theoutcomesvaried dependingontheetiologyofeffusions.Onepatientwith cardiactamponadeandLVfreewallrupturesecondaryto transmuralMIbyTTEdiedafteranexploratory thoracotomy.14 Returnofspontaneouscirculationwasnot achievedinonepatientwithtamponadeandaorticdissection byTTE.17 Onepatienthadcardiactamponadethatwasnot visibleonTTEowingtopooracousticwindowsbutwas evidentonTEEandsurvivedaftersurgicaltreatment.21 One patientwithtamponadeandrightventricularrupture survivedtodischargeafterreceivinganemergentwall repair.25 Earlyterminationofresuscitationwasconducted insixpatientswithtamponadeduetomyocardialrupture andonepatientwithtamponadeduetoaorticdissection byTEE.21,31
TheincidenceofaorticdissectionwashigherbyTEE, whichwasrelatedtodirectvisualizationoftheaorticroot anddescendingaortabythelong-axisandshort-axisviews. However,thedatashouldbecautiouslyinterpretedbecause Jungetalreported19patients 30 andKimetalreported10 patients,31 whichcouldskewtheresults.Moreover,the outcomeswerepoorinthatonlyonepatientwithTTE20 and twowithTEE23 hadROSC.
Pulmonaryembolismisthemostreported findingbyTEE, whichillustratesthethrombidirectly13,22,24,25,27,30 or obstructiontocolor flowinthepulmonaryartery.10 Transthoracicechocardiographyusestheindirectsignof rightventriculardilatation,indicativeofPE.2,3,14–16 However,false-positiveandfalse-negativecaseswere reported,andnotallthepatientsreceivedthrombolysisor thrombolectomy.VandeWouwetalreportedonehada TEEdiagnosisofPEbutnoemboluswasfoundatautopsy.21 Comessetalreportedbilateralperipheralpulmonaryemboli
atautopsybutnotseenbyTEE.22 Jungetalreportedone patientwithinitialnegativeTEE findingshadthrombiinthe mainpulmonaryarteryatthe finalreview,andonepatient withsalinebubblesinthepulmonaryarterywas misinterpretedasPE.30
Gasparietalreportedoneofthe15patientswith suspectedPEreceivingthrombolysissurvivedhospital discharge.3 Chuaetalreportedoneoffourpatients suggestiveofmassivePEbyTTEreceivingthrombolysis survivedtodischarge.16 AlthoughtheROSCratesofPEby TTEorTEEwerestilllower,theywerebetterthanthoseof aorticdissection.
Thesonographic findingsuggestiveofMIonintra-arrest TTEandTEEisregionalwallmotionabnormalityinthe presenceofsonographiccardiacactivity.VandeWouwetal reportedtwopatientshadMIatautopsythatcouldnotbe demonstratedwithTEEowingtolackofspontaneous rhythm.21 Lienetalreportedextensiveanteriorwallakinesia oftheleftventriclethatwasidentifiedinonepatientwith pulselesselectricalactivity.17
AlowLVend-diastolicvolumeisacharacteristic finding indicativeofhypovolemiabyTTE2 andTEE.13,24 Fluid resuscitationwasreportedintwostudiesofTTE,2,12 and threestudiesofTEE.13,24,25 LactatedRingersolution infusioninOHCAincreasedthelikelihoodofprehospital ROSC37;however,theinformationregardingthedetailsof fluidwaslackingintheincludedstudies.
ReducedLVfunctionpresentedasacommon findingby intra-arrestTTEandTEE.Insteadofprecisemeasurement, reducedLVfunctionisestimatedbyvisualassessment (eyeballing)viaECHOofanejectionfractionlessthan 45–55%.13,14,18 Also,theventricularfunctioncouldbe assessedonlyduringintervalsofspontaneouscardiac contraction.22 InotropictherapywasadministeredinonefourthofpatientswithLVdysfunctionbyTTE2 andonetenthofthosebyTEE.25,27
Table3. Pooledresultsoftarget findingsontransthoracicechocardiographyandtransesophagealechocardiographyinpatientswith cardiacarrest.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 171
Hoetal. PerformanceofIntra-arrestEchocardiography
Intra-arrestTTEisaconvenientimagingmodality,butit ischallengedbythetechnicaldifficultyinobtainingadequate cardiacwindowsduringthepausesandpotentialdelaysin chestcompressions.7 Transesophagealechocardiography hasbeenrecognizedasanalternativewithoutinterferingwith ongoingresuscitationefforts.Itprovidesreal-timefeedback onthelocationofchestcompressionsandthequalityof cardiopulmonaryresuscitation(CPR).38 However,TEEhas disadvantagessuchashighcost,highlevelofoperatorskill trainingrequired,andpotentiallyiatrogenictrauma includingoropharyngealesophagealandgastriclacerations, andperforation.39 Threestudiesreportednocomplications ordelaysinresuscitationprocedures.22,32,33 Thetransgastric viewwasexcludedfromtheTEEprotocoltoavoidpotential complicationsortolesseninterferencewiththechest compressionprocedure.30,31
LIMITATIONS
Therewerelimitationsinthisreview.First,theselected studieswerehighlyheterogeneous,small-sizedsamples,with alackofreferencegroupsandstandardizedconfirmation tests.Mostofthepatientswerecollectedfromaconvenience samplefromasingleinstitution.Beyondtheselectionbias, resuscitationfacilitiesandinterventionsforspecificdiseases maydifferdependingontheinstitution’scapability,limiting thegeneralizability.Futurestudiesassessingthediagnostic accuracyofUSinpatientswithCAshouldavoid methodological flaws;arandomizedcontrolledtrial comparingTTEwithTEEwouldbeasolution.Second,the detailedtrainingbackgroundofsonographerswasunclearin someoftheincludedstudies.10,13,19,21,23–25,34 The2022 guidelinessuggestUScanbeperformedbyexperienced personnelwithoutinterruptingCPR.1 Third,thetimingof theintroductionofTTEandTEEwasnotclearinthestudies; soresuscitationtimebiascouldnotbeestimated.39 Lienetal usedUSwithin10minutesofAdvancedLifeSupport(ALS), andGasparietalintroducedUSafter fiveminutesof ALS.3,17 Ontheotherhand,JungetalandKimetal introducedTEEafter10minutesofALS.30,31 Lienetal reportedthatTTEwasperformedinpatientswithlonger resuscitationtime.20
Lastly,theimpactofTTEandTEEonROSCandfurther survivalwasnotthoroughlydiscussed.TheetiologyofCA wasidentifiedin13%ofpatientswithTTEand43%of patientswithTEE.Amongthem,approximately20–25%of patientswitheachtargetconditionachievedROSC exceptthosewithaorticdissection(10%).However,the resuscitationdataassociatedwithROSCsuchaswitnessed arrest,earlyALS,andearlydefibrillationwerenot presented.Also,patientswithearlyROSCbeforeUSwas attemptedwereexcludedfromsomestudies.3,30 Future researchwouldfocusonevaluatingthevaluesofTTE andTEEonROSC,hospitalsurvival,orlong-term neurologicaloutcomes.
CONCLUSION
Transesophagealechocardiographycouldidentifymore causesofcardiacarrestthantransthoracic echocardiography.However,sonographiccardiacactivity, indicativeofbetterratesofreturnofspontaneouscirculation, wasreportedmuchmoreintheTTEgroup.Theimpactof TTEandTEEonROSCandfurthersurvivalwas inconclusiveinthecurrentdata.Ahighdegreeof heterogeneityinpatientselectionandalackof referencestandardsprecludedfurthermeta-analysis forthediagnosticperformanceofintra-arrest
TTEandTEE.
AddressforCorrespondence:Wan-ChingLien,MD,PhD,National TaiwanUniversityHospitalandNationalTaiwanUniversityCollege ofMedicine,DepartmentofEmergencyMedicine,No.7,ChungShanSouthRd.,Taipei100,Taiwan.
Email: wanchinglien@ntu.edu.tw
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Thisstudyreceivedgrantfundingfrom theNationalScienceandTechnologyCouncil,Taiwan(NSTC1122410-H-002-171).Therearenootherconflictsofinterestorsources offundingtodeclare.
Copyright:©2024Hoetal.Thisisanopenaccessarticledistributed inaccordancewiththetermsoftheCreativeCommonsAttribution (CCBY4.0)License.See: http://creativecommons.org/licenses/by/ 4.0/
REFERENCES
1.WyckoffMH,GreifR,MorleyPT,etal.2022InternationalConsensuson CardiopulmonaryResuscitationandEmergencyCardiovascularCare SciencewithTreatmentRecommendations:summaryfromtheBasic LifeSupport;AdvancedLifeSupport;PediatricLifeSupport;Neonatal LifeSupport;Education,Implementation,andTeams;andFirstAidTask Forces. Circulation.2022;146(25):e483–557.
2.BreitkreutzR,PriceS,SteigerHV,etal.Focusedechocardiographic evaluationinlifesupportandperi-resuscitationofemergencypatients:a prospectivetrial. Resuscitation.2010;81(11):1527–33.
3.GaspariR,WeekesA,AdhikariS,etal.Emergencydepartmentpoint-ofcareultrasoundinout-of-hospitalandin-EDcardiacarrest. Resuscitation.2016;109:33–9.
4.HernandezC,ShulerK,HannanH,etal.C.A.U.S.E.:cardiacarrest ultrasoundexam–abetterapproachtomanagingpatientsin primarynon-arrhythmogeniccardiacarrest. Resuscitation 2008;76(2):198–206.
5.ClattenburgEJ,WroePC,GardnerK,etal.Implementationofthe cardiacarrestsonographicassessment(CASA)protocolforpatients withcardiacarrestisassociatedwithshorterCPRpulsechecks. Resuscitation.2018;131:69–73.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 172 PerformanceofIntra-arrestEchocardiography Hoetal.
6.HuisIn ’tVeldMAAllisonMG,BostickDS,etal.Ultrasounduseduring cardiopulmonaryresuscitationisassociatedwithdelaysinchest compressions. Resuscitation.2017;119:95–8.
7.ClattenburgEJ,WroeP,BrownS,etal.Point-of-careultrasoundusein patientswithcardiacarrestisassociatedprolongedcardiopulmonary resuscitationpauses:aprospectivecohortstudy. Resuscitation Jan2018;122:65–8.
8.FairJ3rd,MallinMP,AdlerA,etal.Transesophagealechocardiography duringcardiopulmonaryresuscitationisassociatedwithshorter compressionpausescomparedwithtransthoracicechocardiography. AnnEmergMed.2019;73(6):610–6.
9.WhitingPF,RutjesAW,WestwoodME,etal.QUADAS-2:arevisedtool forthequalityassessmentofdiagnosticaccuracystudies. AnnIntern Med.2011;155(8):529–36.
10.VarrialePandMaldonadoJM.Echocardiographicobservations duringinhospitalcardiopulmonaryresuscitation. CritCareMed 1997;25(10):1717–20.
11.TayalVSandKlineJA.Emergencyechocardiographytodetect pericardialeffusioninpatientsinPEAandnear-PEAstates. Resuscitation.2003;59(3):315–8.
12.ChardoliM,HeidariF,RabieeH,etal.Echocardiographyintegrated ACLSprotocolversusconventionalcardiopulmonaryresuscitationin patientswithpulselesselectricalactivitycardiacarrest. ChinJ
Traumatol.2012;15(5):284–7.
13.ShillcuttSK,MarkinNW,MontzingoCR,etal.Useofrapid “rescue” perioperativeechocardiographytoimproveoutcomesafter hemodynamicinstabilityinnoncardiacsurgicalpatients. JCardiothorac VascAnesth.2012;26(3):362–70.
14.FlatoUA,PaivaEF,CarballoMT,etal.Echocardiographyfor prognosticationduringtheresuscitationofintensivecareunit patientswithnon-shockablerhythmcardiacarrest. Resuscitation 2015;92:1–6.
15.ZenginS,YavuzE,AlB,etal.Benefitsofcardiacsonographyperformed byanon-expertsonographerinpatientswithnon-traumatic cardiopulmonaryarrest. Resuscitation.2016;102:105–9.
16.ChuaMT,ChanGW,KuanWS.Reversiblecausesincardiovascular collapseattheemergencydepartmentusingultrasonography (REVIVE-US). AnnAcadMedSingap.2017;46(8):310–6.
17.LienWC,HsuSH,ChongKM,etal.US-CABprotocolfor ultrasonographicevaluationduringcardiopulmonaryresuscitation: validationandpotentialimpact. Resuscitation.2018;127:125–31.
18.BalderstonJR,YouAX,EvansDP,etal.Feasibilityoffocusedcardiac ultrasoundduringcardiacarrestintheemergencydepartment. CardiovascUltrasound.2021;19(1):19.
19.HeikkiläE,JousiM,NurmiJ.Differentialdiagnosisandcause-specific treatmentduringout-of-hospitalcardiacarrest:aretrospective descriptivestudy. ScandJTraumaResuscEmergMed 2023;31(1):19.
20.LienWC,ChongKM,ChangCH,etal.Impactofultrasonographyon chestcompressionfractionandsurvivalinpatientswithout-of-hospital cardiacarrest. WestJEmergMed.2023;24(2):322–30.
21.vanderWouwPA,KosterRW,DelemarreBJ,etal.Diagnosticaccuracy oftransesophagealechocardiographyduringcardiopulmonary resuscitation. JAmCollCardiol.1997;30(3):780–3.
22.ComessKA,DeRookFA,RussellML,etal.Theincidenceofpulmonary embolisminunexplainedsuddencardiacarrestwithpulselesselectrical activity. AmJMed.2000;109(5):351–6.
23.MiyakeM,IzumiC,TakahashiS,etal.Efficacyoftransesophageal echocardiographyinpatientswithcardiacarrestorshock. JCardiol Nov2004;44(5):189–94.
24.LinT,ChenY,LuC,etal.Useoftransoesophagealechocardiography duringcardiacarrestinpatientsundergoingelectivenon-cardiac surgery. BrJAnaesth.2006;96(2):167–70.
25.MemtsoudisSG,RosenbergerP,LofflerM,etal.Theusefulnessof transesophagealechocardiographyduringintraoperativecardiacarrest innoncardiacsurgery. AnesthAnalg.2006;102(6):1653–7.
26.HilberathJN,BurragePS,ShernanSK,etal.Rescuetransoesophageal echocardiographyforrefractoryhaemodynamicinstabilityduring transvenousleadextraction. EurHeartJCardiovascImaging 2014;15(8):926–32.
27.BurragePS,ShernanSK,TsenLC,etal.Emergenttransesophageal echocardiographyinhemodynamicallyunstableobstetricpatients. IntJObstetAnesth.2015;24(2):131–6.
28.ArntfieldR,PaceJ,HewakM,etal.Focusedtransesophageal echocardiographybyemergencyphysiciansisfeasibleandclinically influential:observationalresultsfromanovelultrasoundprogram. JEmergMed.2016;50(2):286–94.
29.TeranF,DeanAJ,CentenoC,etal.Evaluationofout-of-hospitalcardiac arrestusingtransesophagealechocardiographyintheemergency department. Resuscitation.2019;137:140–7.
30.JungWJ,ChaKC,KimYW,etal.Intra-arresttransoesophageal echocardiographic findingsandresuscitationoutcomes. Resuscitation 2020;154:31–7.
31.KimYW,JungWJ,ChaKC,etal.Diagnosisofaorticdissectionby transesophagealechocardiographyduringcardiopulmonary resuscitation. AmJEmergMed.2021;39:92–5.
32.JungWJ,ChaK,RohYI,etal.Right-to-leftshuntsoccurduring cardiopulmonaryresuscitation:echocardiographicobservations. CritCareMed.2022;50(10):1486–93.
33.PoppeM,MagnetIAM,ClodiC,etal.Resuscitativetransoesophageal echocardiographyperformedbyemergencyphysiciansinthe emergencydepartment:insightsfroma1-yearperiod. EurHeartJAcute CardiovascCare.2023;12(2):124–8.
34.KürkciyanI,MeronG,SterzF,etal.Pulmonaryembolismasacause ofcardiacarrest:presentationandoutcome. ArchInternMed. 2000;160(10):1529–35.
35.BlythL,AtkinsonP,GaddK,LangE.Bedsidefocused echocardiographyaspredictorofsurvivalincardiacarrestpatients:a systematicreview. AcadEmergMed.2012;19(10):1119–26.
36.AndersenLW,GrossestreuerAV,DonninoMW. “Resuscitationtime bias” auniquechallengeforobservationalcardiacarrestresearch. Resuscitation.2018;125:79–82.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 173
Hoetal. PerformanceofIntra-arrestEchocardiography
37.HagiharaA,HasegawaM,AbeT,etal.Prehospitallactated Ringer’ssolutiontreatmentandsurvivalinout-of-hospitalcardiacarrest: aprospectivecohortanalysis. PLoSMed.2013;10(2):e1001394.
38.TeranF,PratsMI,NelsonBP,etal.Focusedtransesophageal echocardiographyduringcardiacarrestresuscitation:
JACCreviewtopicoftheweek. JAmCollCardiol 2020;76(6):745–54.
39.HilberathJN,OakesDA,ShernanSK,etal.Safetyof transesophagealechocardiography. JAmSocEchocardiogr 2010;23(11):1115–27;quiz1220-1.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 174
PerformanceofIntra-arrestEchocardiography Hoetal.
EDUCATION SPECIAL ISSUE:BRIEF RESEARCH REPORT
StaffingPatternsofNon-ACGMEFellowshipswith4-Year ResidencyPrograms:ANationalSurvey
DavidA.Haidar,MD*
LauraR.Hopson,MD*
RyanV.Tucker,MD†
RobD.Huang,MD*
JessicaKoehler,MD*
NikTheyyunni,MD*
NicoleKlekowski,MD*
ChristopherM.Fung,MD,MS*
SectionEditor:BenjaminSchnapp,MD
*UniversityofMichigan,DepartmentofEmergencyMedicine,AnnArbor,Michigan † UniversityofColorado,DepartmentofEmergencyMedicine,Aurora,Colorado
Submissionhistory:SubmittedSeptember13,2023;RevisionreceivedNovember10,2023;AcceptedDecember7,2023
ElectronicallypublishedFebruary28,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18454
Introduction: Emergencymedicine(EM)isoneoffewspecialtieswithvariabletraininglengths.Hiringa three-yeargraduatetocontinuefellowshiptraininginadepartmentthatsupportsafour-yearresidency programcanleadtoconflictsaroundresidentsupervision.Wesoughttounderstandhiringandclinical supervision,orstaffing,patternsofnon-AccreditationCouncilforGraduateMedicalEducation(ACGME) fellowshipshostedatinstitutionssupportingfour-yearresidencyprograms.
Methods: Weperformedaweb-based,cross-sectionalsurveyofnon-ACGMEfellowshipdirectors(FD) hostedatinstitutionssupportingfour-yearEMresidencyprograms.Wecalculateddescriptivestatistics. Ourprimaryoutcomewastheproportionofprogramswithfour-yearEMresidenciesthathirenonACGMEfellowsgraduatingfromthree-yearEMresidencies.
Results: Of119eligibleFDs,88(74%)completedthesurvey.SeventyFDs(80%)indicatedthattheyhire graduatesofthree-yearresidencies.Fifty-six(80%)indicatedthatthree-yeargraduatessupervise residents.MostFDs(74%)indicatednoadditionalrequirementsexisttosuperviseresidentsoutsideof beinghiredasfaculty.TheFDsciteddepartmentpolicy,concernsaboutqualityandlengthoftraining, andresidentcomplaintsasreasonsfornothiringthree-yeargraduates.Amajority(10/18,56%)noted thatnothiringfellowsfromthree-yearprogramsnegativelyimpactsrecruitmentandgivesthemaccessto asmallerapplicantpool.
Conclusion: Mostnon-ACGMEfellowshipsatinstitutionswithfour-yearEMprogramsrecruitthree-year graduatesandallowthemtosuperviseresidents.Thissurveyprovidesprogramsinformationonhow comparablefellowshipsrecruitandstafftheirdepartments,whichmayinformpoliciesthat fittheneedsof theirlearners,thefellowship,andthedepartment.[WestJEmergMed.2024;25(2)175–180.]
INTRODUCTION
Emergencymedicine(EM)isoneoffewspecialtiesinthe UnitedStateswithvariabletraininglengths.1,2 Most residenciesimplementathree-yearmodel,whileonly20% implementafour-yearmodel.3–5 Thereislittledatato supporteithertraininglength.3–7 Somearguethatfour-year
graduateshavemoretimetogainconfidence,develop proceduralskills,developacademicinterests,andgain experiencesupervisinglearners.Advocatesofthree-year programsarguethatanextrayearasfacultywould providethesesameexperiences.1,8,9 Thesepersonal biasesmayimpactrecruitmentandhiringof
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 175
three-yeargraduatesatinstitutionssupportingfour-year residencyprograms.6,7
Whenaninstitutionhostingafour-yearresidencyhiresa three-yeargraduateintofellowshiptraining,thiscanleadto conflictsaroundclinicalsupervision,orstaffing,ofresidents relatedtoperceptionsofseniorityandqualityoftraining.1,9 Therearecurrentlynobestpracticesorguidelinestoinform programsonhowtoaddressthissituation.Thesituationis furthercomplicatedasnon-AccreditationCouncilfor MedicalEducation(ACGME)fellowshipsfrequentlylack uniformrulesthatgovernrecruitment,program requirements,andclinicalresponsibilities.10 Nostudies currentlyevaluatetheprevalenceoftheseissuesorexamine variabilityinrecruitment,hiring,andclinicalresponsibilities oftraineesatnon-ACGMEfellowships.Inthisstudy,we soughttounderstandthehiringandstaffingpatternsofnonACGMEfellowshipshostedatinstitutionswithfour-year EMresidencyprograms.
METHODS
StudyDesignandParticipants
Thiswasacross-sectionalsurveyoffellowshipdirectors (FD)ofnon-ACGMEfellowshipshostedatinstitutions supportingafour-yearEMresidencyprogram.We conductedthesurveybetweenJanuary–April2023.This studywasdeemedexemptbyourinstitutionalreviewboard (HUM00221519).InNovember2022,wegeneratedalistof 54four-yearEMresidencyprogramsfromtheEmergency MedicineResidents’ Association(EMRA)Matchrosterand ElectronicResidencyApplicationServicedirectory.11–13 We identifiednon-ACGMEfellowshipsofferedusingeach program ’swebpage,theSocietyforAcademicEmergency MedicineFellowshipDirectory,andtheSocietyforClinical UltrasoundFellowshipsdirectory.14,15
SurveyDevelopmentandDistribution
WedevelopedthesurveybasedonPanacek’sgeneral surveyprinciples,literaturereview,andexpertopinionto providecontentvalidityevidence.6,16–18 Allauthorshave experiencedevelopingsurveystudies,andthegroup (includingfourcurrentorformerFDs)iterativelypilotedand revisedthesurveyforoptimalphrasing,surveylength, functionality,andappropriatemixofsuggestedand open-endedresponses,whichprovidedcontentand responseprocessvalidityevidence.18 WeusedQualtrics (QualtricsXM,Provo,UT),aweb-basedsurveyplatform,to distributethesurveyviaemailwithapersonalizedlinkfor eachFDtocollectandanalyzethedata.Wesentweekly reminderstoFDs’ institutionalemails,withanoptionto declineparticipation,foreightweeks.Wethensent personalizedweeklyreminderemailsforanadditional fourweeks.Wecollectedindividualresponsestothe surveyanonymously.
OutcomesandDataAnalysis
WeaskedFDstoreporttheirfellowshiptype,yearsin currentrole,anddemographicdatasuchasnumberof clinicalsites,programenvironment(academic,county, community,etc),andgeographiclocation.Ourprimary outcomewastheproportionofprogramsaffiliatedwithfouryearEMresidenciesthathirenon-ACGMEfellows graduatingfromthree-yearEMresidencies.Wealsoasked clarifyingquestionstobetterunderstandtheirstaffingmodel, andrecruiting,hiring,andclinicaloversightpolicies.The surveyincludedspaceforcommentssothattheFDscould providecontexttotheiranswers,butwedidnotanalyzethese forthemes.Thefullsurveyisavailablein AppendixA1.We analyzedthedatausingExcel365(MicrosoftCorporation, Redmond,WA)togeneratedescriptivestatisticsand analysis.Weassessedtheassociationbetweencategorical variablesusingtheFisherexacttest.Wedidnotcalculatean apriorisample-sizeestimateasweattemptedtocapturea 100%responserate.
RESULTS
Of54four-yearEMresidenciesintheUS,32institutions offeredatleastonenon-ACGMEfellowshipwithatotalof 128fellowshipsidentified(median3.5;range1–10).We received88responsesafterexcludingnineopt-outsandone blankresponse(88/119)foraresponserateof73.9%. ProgramandFDcharacteristicsarelistedinthe Table. Freetextresponsesareincludedin AppendixA2
Ofthe88responses,70FDs(80%)reportedhiring graduatesofthree-yearEMprogramsfortheirrespective fellowships.Fifty-sixFDs(80%)whoacceptthree-year graduatesindicatedthattheirfellowscansuperviseEM residents.Wefoundvariationinwhofellowscouldsupervise. Themostcommonpolicy(40%)wasthatfellowscan superviseEMpostgraduate-year(PGY)-3residentsand below.MostFDs(74%)indicatedthattheyhadno additionalrequirementstosuperviseresidentsoutsideof beinghiredonasfaculty.Fullsurveyresultsappearin the Figure
Programswithmultipleclinicalsitesaremorelikelytohire three-yeargraduates.Tenof23programs(57%)withone clinicalstaffingsitehiredthree-yeargraduatescomparedto 88%(57/65)ofsiteswithtwoormoreclinicalsites (P < 0.001).TheFDsreportedtheimplementationofvarious strategiestomitigatepotentialconflicts.Oneprogram hostsajointfellowshipcurriculumfortheirfellows,which incorporatesinstructiononbedsideteaching,giving feedback,andteachingvariousskills.OtherFDsreported thattheirprogramspreventedtheirfellowsfromstaffingin highacuityareasordelayworkingwithresidents.
Twenty-sevenFDs(50%)citeddepartmentpolicyasthe reasonfortheirhiringandstaffingpolicies.Selected commentsfromotherFDsincludedconcernsaboutquality andlengthoftrainingandresidentcomplaints.Others
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 176 EMFellowshipsStaffingPatternsSurvey Haidaretal.
Central(IL,IN,IA,KS,MI,MN,MO,NE,OH,WI)13(15%)
Northeast(CT,DC,DE,MA,MD,ME,NH,NJ,NY,PA,RI,VT)45(51%)
Southern(AL,AR,FL,GA,KY,LA,MS,NC,OK,PR,SC,TN,TX,VA,WV)0(0%)
Western(AZ,CA,CO,NM,NV,OR,UT,WA)30(34%)
’
*Respondentscouldselectmorethanonetypeofclinicalsite.
reportedtheirclinicalenvironmentwasnotconduciveto separatingfellowsfromresidents.SevenFDsreported wantingtoavoidPGY-4fellowsstaffingPGY-4residents. OneFDindicatedthat “becausewearea4-yearprogram,we
wanttoacknowledgetoourresidentsthat4yearsiswhatwe thinkisrequiredforgraduation.”
Amongtheprogramsnothiringfellowsfromthree-year programs,56%(10/18)ofFDsnotedthatthispolicy
DemographicsNumberofresponses(%)
Admin/operations 14(15%) Cardiologyandresuscitation 1(1%) Climateandhealthpolicy 1(1%) Digitalhealth 1(1%) Disastermedicine 3(3%) Globalhealth/internationalmedicine 7(7%) Healthhumanities 1(1%) Healthpolicy 1(1%) Medicaleducation 18(19%) Neurologicemergencies 1(1%) Pediatricultrasound 1(1%) Physicianwellness 1(1%) Research 9(9%) Simulation 5(5%) Socialmedicine 3(3%) Ultrasound 22(23%) Wildernessmedicine 3(3%) Programregion
Table. Demographicdetailsofthefellowshipsrepresentedinoursurveyoffellowshipdirectorsofnon-ACGMEfellowshipprograms.
Fellowshiptype
Categoryofprimaryresidencysite* Academic(universitybased) 81(82%) Community 0(0%) County 15(15%) Other 3(3%)
sprimaryclinicalsite* Academic(universitybased) 74(46%) Community 46(29%) County 23(14%) Other 17(11%) Numberofclinicalsitesnon-ACGMEfellowsclinicallystaff 1 22(25%) 2 33(38%) 3 26(30%) 4 6(7%)
Categoryofnon-ACGMEfellow
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 177 Haidaretal. EMFellowshipsStaffingPatternsSurvey
EM,emergencymedicine; PGY,postgraduateyear; FPPE,focusedprofessionalpracticeevaluation.
negativelyimpactedtheirfellowshiprecruitmentandgave themaccesstoasmallerpoolofapplicants.
DISCUSSION
Toourknowledgethisstudyisthe firsttodescribestaffing patternsofnon-ACGMEfellowshipshostedatinstitutions withfour-yearEMresidencies.MostoftheFDswesurveyed hirethree-yeargraduatesasfellows,andmostprograms permitthree-yeargraduatestostaffresidentswithno additionalrequirementsbeyondbeinghired.Wealso identifiedpotentialnegativeimpactsonfellowshipsasthey restricttheirapplicantpool.OneFDindicatedthattheir fellowshipwasmostlygoingunfilledduetotheirrecruitment policy.Anotherindicatedthatthe financialsacrificeofa four-vsthree-yearresidencymayunintentionallyfavor recruitmentofthosewithout financialneedorburden, especiallysincethedebtloadofEMapplicantsisreportedly higherthanforothermedicalspecialties.1
Someprogramsoffertheirfellowsalternativeclinical sites – suchasVeteransAffairshospitals,freestandingEDs, orurgentcares.Bystaffingmultiplelocations,non-ACGME fellowscanworkwithoutaresidentpresence.This flexibility allowsprogramstohirethree-yeargraduatesandpermits fellowstointerfacewithresidentsacademicallywithout havingtosupervisethemclinically.Thisallowsforatraining
environmentconducivetotheneedsofalllearners’ growth anddevelopment.
TheFDscitedclinicalconcernsanddepartmentpolicyas themainreasonsfortheirstaffingandhiringpolicies.There isalackofobjectivedatathatfour-yeargraduates outperformthree-yeargraduatesclinicallyoronthe qualifyingwrittenboardexam,suggestingthatthismaybe rootedinbias.1,6,7 Intheabsenceofrobustdatatosupport theclinicalcapabilitiesoftraineesfromeitherthree-orfouryearprograms,theprinciplesofcompetency-basedmedical education(CBME)mayoffersolutions.19 Theprinciplesof CBMErequiredemonstrationofcompetencyanddecouple attainmentofcompetencyfromtime-in-training.19 Theuse ofCBMEtodeterminereadinessforunsupervisedpractice throughaprocessknownas “promotioninplace” hasbeen pilotedbysomeresidencyprogramsandmaybeauseful modeltoreplicateindeterminingfellowreadinessfor staffing,regardlessofPGYstatus.19,20 Ifweremovethefocus fromtime-boundedtrainingandfocusondemonstratedskill acquisition,programsmaydesignprocessestoonboard three-yeargraduatesbyfocusingondevelopingandassessing appropriateskillsforsupervisionoftrainees.
Futurestudiescouldexplorewhosetsdepartmental policiesregardingfellowstaffing,evaluatefellowand residentperceptionsofstaffingpolicies,andcompare
Figure. Flowdiagramdetailingthesurveyresponsehierarchyoffellowshipdirectorsofnon-AccreditationCouncilforGraduateMedical Educationfellowshipsregardingclinicalsupervisionpatternsfor3-yearemergencymedicine(EM)residencygraduatesatinstitutionswitha 4-yearEMprogram.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 178 EMFellowshipsStaffingPatternsSurvey Haidaretal.
careeroutcomesoffellowsworkinginvarious staffingenvironments.
LIMITATIONS
Wemaynothavecapturedallnon-ACGMEfellowships atfour-yearinstitutions.Wedidnotidentifyfellowship directoriesbesidesultrasound,whichmayhaveledto samplingbias.Weattemptedtomitigatethisbysearching specificprogramwebsitesforlistedfellowships.TheFDs whodidnotparticipateinourstudymayrepresentaunique populationwithdifferenthiringandstaffingpatterns.Wedid notidentifynon-ACGMEfellowshipshostedatfour-year EMprogramsinthesouthernUS,nordidwereceive responsesfromprimarilycommunityEMprograms,which couldalsohavebiasedourresults.Wedidnotsurvey ACGME-accreditedfellowships,asfellowsvaryintheway they “maintaintheirprimaryBoardskills.”21 SomeACGME fellowships(eg,criticalcare,emergencymedicalservices)do notrequireminimumclinicalhoursintheemergency department,whichleadstoaqualitativelydifferent experiencefromnon-ACGMEfellowships,wherefellowsare appointedasclinicalfaculty.2,21,22
CONCLUSION
Ourresultsindicatethatmostnon-ACGMEfellowships hostedatinstitutionswithfour-yearEMprogramsrecruit graduatesofthree-yearprogramsandallowthemto superviseresidents.Thissurveydataprovidesprogram informationonhowcomparablefellowshipprogramsrecruit andstafftheirdepartments,whichmayinformpoliciesthat fittheneedsoftheirlearners.
AddressforCorrespondence:DavidA.Haidar,MD,Universityof Michigan,DepartmentofEmergencyMedicine,1500E.Medical CenterDr.,B1-380,AnnArbor,MI48109.Email: dahaidar@med. umich.edu
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.
Copyright:©2024Haidaretal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.RossTM,WolfeRE,MuranoT,etal.Three-vs.four-yearemergency medicinetrainingprograms. JEmergMed. 2019;57(5):e161–5.
2.AccreditationCouncilforGraduateMedicalEducation(ACGME). ACGMEprogramrequirementsforgraduatemedicaleducationin
emergencymedicine.2023.Availableat: https://www.acgme.org/ specialties/emergency-medicine/program-requirements-and-faqsand-applications/.AccessedJanuary6,2023.
3.LotfipourS,LuuR,HaydenSR,etal.Becominganemergencymedicine resident:apracticalguideformedicalstudents. JEmergMed. 2008;35(3):339–44.
4.NelsonLS,CalderonY,AnkelFK,etal.AmericanBoardofEmergency Medicinereportonresidencyandfellowshiptraininginformation (2021–2022). AnnEmergMed. 2022;80(1):74–83.e8.
5.AccreditationCouncilforGraduateMedicalEducation(ACGME).Listof programsbyspecialty.Availableat: https://apps.acgme.org/ads/Public/ Reports/Report/1.AccessedApril29,2023.
6.HopsonL,ReganL,GisondiMA,etal.Programdirectoropiniononthe ideallengthofresidencytraininginemergencymedicine. AcadEmerg Med. 2016;23(7):823–7.
7.NikollaDA,ZocchiMS,PinesJM,etal.Four-andthree-yearemergency medicineresidencygraduatesperformsimilarlyintheir firstyearof practicecomparedtoexperiencedphysicians. AmJEmergMed. 2023;69:100–7.
8.HaydenSandPanacekE.Proceduralcompetencyinemergency medicine:thecurrentrangeofresidentexperience. AcadEmergMed. 1999;6(7):728–35.
9.WeichenthalL.Advantagesofafour-yearresidency. CalJEmergMed. 2004;5(1):18–9.
10.AccreditationCouncilforGraduateMedicalEducation(ACGME). Commonprogramrequirements(fellowship).2022.Availableat: https:// www.acgme.org/globalassets/pfassets/programrequirements/ cprfellowship_2022v3.pdf.AccessedJanuary6,2023.
11.EmergencyMedicineResidents’ Association(EMRA).Matchlist.2023. Availableat: https://webapps.acep.org/utils/spa/match#/search/list AccessedJanuary6,2023.
12.AssociationofAmericanMedicalColleges(AAMC).Electronic residencyapplicationservice(ERAS)directory.2023.Availableat: https://systems.aamc.org/eras/erasstats/par/display.cfm? NAV_ROW=PAR&SPEC_CD=110.AccessedJanuary6,2023.
13.AssociationofAmericanMedicalColleges(AAMC).Residencyexplorer tool.Availableat: https://www.residencyexplorer.org/Home/Dashboard AccessedJanuary6,2023.
14.SocietyforAcademicEmergencyMedicine(SAEM).Fellowship directory.Availableat: https://member.saem.org/SAEMIMIS/ SAEM_Directories/Fellowship_Directory/SAEM_Directories/P/ FellowshipList.aspx.AccessedJanuary6,2023.
15.SocietyofClinicalUltrasoundFellowships(SCUF).Programlist. Availableat: https://www.eusfellowships.com/programs AccessedJanuary6,2023.
16.PanacekEA.Survey-basedresearch:generalprinciples. AirMedJ. 2008;27(1):14–6.
17.AlerhandS,Situ-LacasseE,RamdinC,etal.Nationalsurveyofpoint-ofcareultrasoundscholarlytracksinemergencymedicineresidency programs. WestJEmergMed. 2021;22(5):1095–101.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 179
fingPatternsSurvey
Haidaretal. EMFellowshipsStaf
18.HillJ,OgleK,GottliebM,etal.Educator’sblueprint:ahow-toguidefor collectingvalidityevidenceinsurvey-basedresearch. AEMEducTrain. 2022;6(6):e10835.
19.RyanMS,LomisKD,DeiorioNM,etal.Competency-basedmedical educationinanorm-referencedworld:arootcauseanalysisof challengestothecompetency-basedparadigminmedicalschool.
AcadMed. 2023;98(11):1251–60.
20.GoldhamerMEJ,Martinez-LageM,Black-SchafferWS,etal. Reimaginingtheclinicalcompetencycommitteetoenhanceeducation andprepareforcompetency-basedtime-variableadvancement.
JGenInternMed. 2022;37(9):2280–90.
21.AccreditationCouncilforGraduateMedicalEducation(ACGME). ACGMEprogramrequirementsforgraduatemedicaleducation inemergencymedicalservices.2021.Availableat: AccessedApril29,2023.
22.AccreditationCouncilforGraduateMedicalEducation(ACGME). ACGMEprogramrequirementsforgraduatemedicaleducationincritical caremedicine.2022.Availableat: https://www.acgme.org/specialties/ internal-medicine/program-requirements-and-faqs-and-applications/ AccessedApril29,2023.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 180 EMFellowshipsStaffingPatternsSurvey Haidaretal.
EDUCATION SPECIAL ISSUE:BRIEF RESEARCH REPORT
ChangesinResidencyApplicantCancellationPatternswith VirtualInterviews:ASingle-siteAnalysis
MeryllBouldin,MD*
CarlyEastin,MD*†
RachaelFreeze-Ramsey,MD*
AmandaYoung,MD*
MeredithvonDohlen,MD*
LaurenEvans,MD*
TravisEastin,MD,MS*
SarahGreenberger,MD*
*UniversityofArkansasforMedicalSciences,DepartmentofEmergencyMedicine, LittleRock,Arkansas
† UniversityofArkansasforMedicalSciences,DepartmentofPediatrics, SectionofEmergencyMedicine,Toxicology,andPharmacology, LittleRock,Arkansas
SectionEditors:AbraFant,MD,andWilliamEdwardSoares,MD
Submissionhistory:SubmittedOctober1,2023;RevisionreceivedFebruary7,2024;AcceptedFebruary21,2024
ElectronicallypublishedMarch14,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18487
Background: ResidencyprogramstransitionedtoprimarilyvirtualinterviewsduetotheCOVID-19 pandemic.Thisshiftraisedquestionsregardingexpectationsandpatternsofapplicantcancellation timeliness.Thepurposeofthisstudywastoexaminechangesinapplicantcancellationsafter transitioningtovirtualinterviews.
Methods: Thiswasaretrospectivecohortstudyofinterviewdatafromathree-yearemergencymedicine residencyatatertiary-careacademicmedicalcenter.UsingarchiveddatafromInterviewBroker,we examinedschedulingpatternsbetweenonein-person(2019–2020)andtwovirtualinterviewcohorts (2020–2021and2021–2022).Ouroutcomesweretheoverallcancellationratesrelativetointerviewslots aswellastheproportionofcancellationsthatoccurredwithin7or14daysoftheinterviewdate.
Results: Therewere453interviewslotsand568applicantsinvited.Overall,applicantscanceled17.1% ofscheduledinterviews.Comparedwithin-personinterviews,applicantscanceledsignificantlyfewer virtualinterviews(inperson:40/128(31.3%),virtualyear1:22/178(12.4%),virtualyear2:15/143 (10.5%), P = 0.001).Conversely,applicantscanceledsignificantlymorevirtualinterviewswithinboththe 14-daythreshold(inperson:8/40(20%),virtualyear1:12/22(55.5%),virtualyear2:12/15(80%), P < 0.001)andthe7-daythreshold(inperson:0/40(0%),virtualyear1:3/22(13.6%),virtualyear2:4/15 (26.7%), P = 0.004).
Conclusion: Whilelimited,atoursite,changingtoavirtualinterviewformatcorrelatedwithfewer cancellationsoverall.Theproportionofcancellationswithin14dayswasmuchhigherduring virtualinterviewseasons,withmostcancellationsoccurringduringthattimeframe.Additional studiesareneededtodeterminetheeffectsofcancellationpatternsonemergencymedicinerecruitment. [WestJEmergMed.2024;25(2)181–185.]
INTRODUCTION
Historically,residencyapplicantstraveledtoUSprograms forin-personinterviews.In2020,theCOVID-19pandemicled theCoalitionforPhysician Accountability(COPA)to recommendthatresidencyprogramsconductonlyvirtual
interviews.1 Proponentsofvirtualinterviewscitedcostand safetyaspotentialupsides,andapplicantshavereportedoverall satisfactionwithvirtualinterviewsandmoreadvantagesthan barriers.2–4 However,programshaveexpressedcontinued doubtsaboutsomeaspectsofvirtualrecruitment.2
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 181
Evenbeforethepandemic,therewerenoestablishedrules acrossspecialtiesregardinganacceptabletimeframefor interviewcancellations.Foremergencymedicine,the EmergencyMedicineResidentAssociation(EMRA) recommendedatleasttwoweeks’ noticeintheir2019 “EMRAandCORDStudentAdvisingGuide.”5 In2020,the firstyearofvirtualinterviews,emailcommunicationonthe CouncilofResidencyDirectorsinEmergencyMedicine (CORD)listservsuggestedthatprogramdirectors’ acceptablecancellationthresholdsrangedfrom48hoursto 10dayspriortotheinterviewdate.6 Ultimately,CORD statedthatsevendayswasrecommendedforapplicantsina 2020blogpostaboutinterviewingduringthepandemic, whileotherpublicationsstillrecommendedtwoweeks.7,8 Currently,the2023CORDApplicationProcess ImprovementCommitteeandthe2022–2023National ResidentMatchingProgram(NRMP)agreementhave advisedapplicantstocancelnolaterthan1–2weeksbefore theirinterviewdates.9,10
Virtualinterviewsmaybeheretostay,asevidencedby recentCOPAandAssociationofAmericanMedicalColleges (AAMC)statements,aswellasthe2023-24CORD guidelines.11–13 Understandingpatternsofvirtualinterview cancellationbehaviormayhelpprogramdirectors, applicants,andtheiradvisorsprepareforasuccessfulMatch. Tocharacterizetheeffectsofvirtualrecruitmentoninterview cancellations,wecomparedin-personinterviewcancellation patternstothoseofvirtualrecruitmentcyclesatour academicemergencymedicine(EM)residency.
METHODS
Thiswasaretrospectivecohortstudyatathree-yearEM residencysponsoredbyatertiary-care,academicmedical centerinanurbansettinginthesouth-centralUnitedStates. Thisresidencyisanestablishedprogram(foundedin1984) withaclasssizeof10residentsperyear,whichincreasedto12 residentsforthe2022Match.TheUniversityofArkansasfor MedicalSciencesInstitutionalReviewBoard(IRB) approvedthisstudyinexemptstatus.
Ourprogrambeganusingtheonlineinterviewscheduling softwareInterviewBroker(TheTenthNerve,LLC,Lewes, DE;www.interviewbroker.com)inFall2019toinvite applicantstointerview.InFall2020,interviewstransitioned frominpersontovirtualandadditionalslotswereadded, withCORDcontinuingtorecommendvirtualinterviewsfor EMresidenciesinsubsequentcycles.Similartoin-person interviews,applicantsforvirtualinterviewsareinvitedina 1:1applicanttoslotratioandgiven48hourstorespond beforeanotherapplicantisinvited.
UsingarchiveddatafromInterviewBroker,weexamined schedulingpatternsbetweenthein-personinterviewcohort (2019–2020season)andtwovirtualinterviewcohorts (VirtualYear1:2020–2021andVirtualYear2:2021–2022). Unfortunately,cancellationdatapriortotheinitiationof
InterviewBrokeratoursitewasnotavailable.Asingle investigatorabstracteddatafromInterviewBrokerin aggregateformbyacademicyearusingoverallcountsof relevantvariables,includingnumberofinterviewslots,days, invitations,interviewsscheduled/unscheduled(ie,no applicantresponsereceived)/declined,cancellations,andthe timingofthosecancellationsrelativetotheinterviewdate. Wedefinedaninterviewcancellationasaninterviewthatwas scheduled,canceled,andneverrescheduled;interviewsthat wererescheduledwereconsideredcompleted.Demographic variableswerenotavailableasInterviewBrokeronlyrecords thestudent’snameandAAMCID;accessingadditional informationwouldhaverequiredqueryingtheElectronic ResidencyApplicationService,whichwasnotcoveredinour exemptIRBagreement.
Ouroutcomesweretheoverallproportionofinterview cancellationsrelativetointerviewslots,aswellasthe proportionofinterviewcancellationsthatoccurredwithin 14daysoftheinterviewdateandwithinsevendaysofthe interviewdate.Descriptivestatisticswereperformed.We performedcomparisonsusingchi-squaredortheFisherexact testassomeobservationswereuncommon.Allcomparisons weretwo-sidedwith ɑ = 0.05.Analyseswereperformedusing SPSSStatisticsforMacintoshVersion28.0(IBM Corporation,Armonk,NY).
RESULTS
Overthreeyears,therewere453interviewslotsand568 applicantsinvited.Mostoftheinterviewslotswerevirtual (71.7%).Overall,theprogramsentout1.25interview applicationsperslotandapplicantscanceled17.1%of scheduledinterviews(Table1).Wefoundasignificant decreaseintheproportionofoverallcancellationsrelativeto filledinterviewslots,with40/128(31.3%),22/178(12.4%), and15/143(10.5%)cancellationsforin-person,virtualyear 1,andvirtualyear2,respectively(P < 0.001).When analyzedfurtherandadjustingformultiplecomparisons,the decreasewassignificantwhencomparinginpersonvs.either virtualyear,butnotwhencomparingthetwovirtualyears.
Whilefewerinterviewswerecanceled,theproportionof virtualinterviewcancellationsthatoccurredwithin14days oftheinterviewdatewassignificantlyhigher(inperson:8/40 (20%),virtualyear1:12/22(55.5%),virtualyear2:12/15 (80%), P < 0.001).Similarly,morevirtualinterviewswere canceledwithinsevendaysoftheinterviewdate(inperson:0/ 40(0%),virtualyear1:3/22(13.6%),virtualyear2:4/15 (26.7%), P = 0.004),althoughthesenumberswerelow overall.Inboththe14and7daycancellationanalyses,these dataindicatedayear-over-yearincrease,meaninginboth14 and7daycomparisonswesawasignificantincreasein cancellationsbetweeninpersonandvirtualyear1,andagain sawasignificantincreasebetweenvirtualyear1andvirtual year2.See Figures1 and 2 forgraphicalbreakdownofthe
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 182 ResidencyApplicantCancellationPatterns Bouldinetal.
InterviewsandCancellations
InterviewgroupInpersonVirtualyear1Virtualyear2
Numberofinterviewdays151616
Numberofinterviewslots128180145
Numberofapplicantsinvited195206167
Numberofinvitationsperinterviewslot1.521.141.15
Totalinterviewslots filled128178143
Numberofunscheduledinvitations(ie,noapplicantresponsereceived)1413
Numberwhodeclinedwithoutscheduling1356
Overallcancellations(%ofscheduled)40(31.3%)22(12.4%)15(10.5%)
Numberwhocanceled < 7days(%ofcanceled)0(0%)3(13.6%)4(26.7%)
Numberwhocanceled7–14days(%ofcanceled)8(20%)9(40.9%)8(53.3%)
Numberwhocanceled >14days(%ofcanceled)32(80%)10(45.5%)3(20.0%)
Overalldeclined,unscheduled,orcanceled(%oftotalinvited)67(34.4%)28(13.6%)24(14.3%)
overalldistributionofinvitedapplicantsandinterview cancellationrates.
DISCUSSION
Comparedwithin-personinterviews,applicantstoour programwerelesslikelytocanceltheirvirtualinterview.Of thosewhodidcancel,severalvirtualapplicantscanceled withinsevendays,andmostcancellationsoccurredwithin 14daysoftheinterviewdate.Forin-personinterviews, applicantsweretraditionallyinstructedtocancelassoonas possibleandatleasttwoweekspriortotheinterviewdate.5 Asdiscussedpreviously,recommendationsforEMvirtual interviewcancellationshaverangedfrom48hourstotwo weeks,withtheNRMPcurrentlyrecommendingatleast1–2 weeksinadvance.10 Ourresultssuggestthatshort-notice
cancellations(ie,lessthantwoweeks)bystudentsmaybe morecommoninthevirtualera.
Wearenotawareofliteratureregardingthespecifictiming ofvirtualinterviewcancellations,butour findingoffewer overallcancellationsisconsistentwithLewkowitzetal’ s findingsthatmaternal-fetalmedicinefellowshipvirtual interviewshadalowerrateofcancellationscomparedwith in-personinterviews(39.1%vs72.3%).14 Thiscouldstemfrom thereducedtimeandcostrequiredtointerviewvirtually.15,16
Unfortunately,fewerinterviewcancellationsoverallcould contributetointerviewhoardingandaninequitable distributionofinterviews.TheAAMCandsomespecialties haveexpressedconcernsabouthigherqualityapplicants receivinginvitationsforandschedulingexcessivelyhigh numbersofinterviewsandleavinglowertierstudentswith
Table1. Breakdownofin-personandvirtualinterviewcohorts;totalcountsprovidedunlessotherwisespecified.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 183
Figure1. Overalldistributionofinvitedapplicants.
Bouldinetal. ResidencyApplicantCancellationPatterns
feweroptions.15,17 WhilethishasnotbeenstudiedinEM specifically,theEmergencyMedicineConsensusStatement onthe2020–2021ResidencyApplicationprocesssuggested aninterviewlimitof17interviewsandencouragedapplicants nottointerviewattheirless-preferredprogramsloweron theirlistto “maketheseslotsavailabletootherstudents,” indicatingapotentialconcernfortheeffectsofhoardingsuch as “peersnotmatchingand/orprogramsnot filling.”18
Short-noticeinterviewcancellationsposeafewother challengesforresidencyprograms.Previously, fillinganinpersoninterviewslotrequired findingareplacementwho couldstillarrangetraveltotheinterviewlocation,whichisno longerrelevantforvirtualinterviews.Nonetheless,the NRMPrequiresthatprogramsprovidenolessthan48hours forapplicantstorespondtointerviewinvitations.10 If applicantsarecancelingonlyafewdaysbeforeaninterview, fillingtheopenspotmaybeachallengesinceprograms cannotinvitemorethanoneapplicantatatimeperspot. Short-noticecancellationscanalsobeproblematicas interviewersmayhavetoreviewcandidates’ applications wellinadvanceoftheinterviewdate.Withshort-notice cancellations,thiscouldmeanlosttimeforinterviewerswho hadalreadyreviewedthoseapplicationsorinadequatetime toreviewthereplacements.
Conversely,programdirectorswanttoavoidinterviewing applicantswhoarenotinterestedintheirprogram,anda cancellation evenonshortnotice providesan opportunitytointerviewanapplicantwithgreaterinterestin theprogram.Inourcase,wehadonlyfouropeninterview spotsoverthe firsttwovirtualyears(twounfilledperyear), indicatingthatwe filledmostcanceledspots.Therefore, whilenoofficialopinionexists,programdirectorsmaynot mindshort-noticecancellationaslongastheinterview scheduleisfull.Infact,theymaypreferfortheapplicantnot tofeelpressuredtointerviewataprograminwhichtheyare uninterestedonlybecausetheyareconcernedabout canceling,withshortnoticebeingviewedasunprofessional. Asvirtualinterviewsappeartobeheretostay,understanding cancellationpatternswillbeimportantforprograms, especiallyinbalancingthetimingcancellationswithnew
invitationssoprogramscanideallymaintainafull interviewschedule.
LIMITATIONS
Thisstudywaslimitedtoonespecialtyatasingle institution,thereforethegeneralizabiltyofthese fi ndingsto otherinstitutionsorspecialtiesisunclear,especiallygiven thesmallsamplesizeandlimitedpre-postperiod.The changinglandscapeofEMresidencyrecruitmentmayalso affectthegeneralizabilityofthese fi ndings.Unfortunately, wehadonlyoneyearofin-personinterviewdataaswedid notkeeptheserecordspriortotheuseofInterviewBroker, whichcouldhaveintroducedbias.Wealsohadanincrease inresidentcomplementduringvirtualyear2,whichmay haveconfoundedtheresults.Unfortunately,wewere unabletoincludedemographicdata,whichmighthave helpedtoidentifyadditionalcancellationpatterns.Lastly, examiningtrendsinthosewhorescheduleinterviewswas notperformedinthisstudyandmaybeofvalueinfuture investigations,assomedownsidesdiscussedwithshortnoticecancellations(eg, fi llingemptyslots;havingtimeto reviewapplications)wouldstilloccurinapplicantswhoare reschedulingwithshortnotice.
CONCLUSION
Comparedwithin-personinterviewcycles,applicantsto ourresidencyprogramweresignificantlylesslikelytocancel virtualinterviews.However,themajorityofvirtual cancellationsthatdidoccurwerewithin14daysofthe interviewdateandnearlyone-fifthoccurredinunderseven days.Additionalstudies,ideallymultisitethatinclude applicantdemographicdata,areneededtodeterminehow cancellationpatternsaffectEMrecruitmentandmatch outcomesinthevirtualera.
AddressforCorrespondence:MeryllBouldin,MD,Universityof ArkansasforMedicalSciences,DepartmentofEmergency Medicine,4301WMarkhamSt.,Slot584,LittleRock,AR72205. Email: mebouldin@uams.edu
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 184
Bouldinetal.
Figure2. Interviewcancellationsbylengthoftimefrominterview. d,day.
ResidencyApplicantCancellationPatterns
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.
Copyright:©2024Bouldinetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.TheCoalitionforPhysicianAccountability’sWorkGrouponMedical StudentsintheClassof2021MovingAcrossInstitutionsforPost GraduateTraining.Finalreportandrecommendationsformedical educationinstitutionsofLCME-accredited,U.S.osteopathic,andnonU.S.medicalschoolapplicants.2020.Availableat: https://www.aamc. org/system/files/2020-05/covid19_Final_Recommendations_ 05112020.pdf.AccessedMay22,2022.
2.DavisMG,HaasMRC,GottliebM,etal.Zoominginversus flyingout: virtualresidencyinterviewsintheeraofCOVID-19. AEMEducTrain. 2020;4(4):443–6.
3.DomingoA,RdesinskiRE,StensonA,etal.Virtualresidencyinterviews: applicantperceptionsregardingvirtualintervieweffectiveness, advantages,andbarriers. JGradMedEduc. 2022;14(2):224–8.
4.Li-SauerwineS,WeygandtPL,SmylieL,etal.Themorethingschange themoretheystaythesame:factorsinfluencingemergency medicineresidencyselectioninthevirtualera. AEMEducTrain. 2023;7(6):e10921.
5.EmergencyMedicineResidentsAssociationJarouZ,HillmanEA, KellogA,etal.(Eds.). EMRAandCORDStudentAdvisingGuide: AnEvidence-basedApproachtoMatchinginEM. 2019.Availableat: https://www.emra.org/books/msadvisingguide/msag AccessedJanuary19,2024.
6.EmergencyMedicineProgramDirectors.Re:interviewcancellations. CORDListservprivateemailchain.LastaccessedJanuary19,2024.
7.GorgensS.TheEMinterviewseason:pandemicedition.2020. Availableat: https://cordemblog.com/2020/09/03/the-em-interviewseason-pandemic-edition/.AccessedDecember20,2023.
8.HopsonLR,EdensMA,GoodrichM,etal.Calmingtroubledwaters:a narrativereviewofchallengesandpotentialsolutionsintheresidency interviewofferprocess. WestJEmergMed. 2020;22(1):1–6.
9.CouncilofResidencyDirectorsinEmergencyMedicine.Application ProcessImprovementCommittee(Apic):BestPracticesforthe 2023–2024ResidencyApplicationandInterviewSeason.2023.
Availableat: https://www.cordem.org/siteassets/files/match/ apic-2023-2024-application-interview-season-best-practices.pdf
AccessedJuly26,2023.
10.NationalResidentMatchingProgram.MatchParticipationAgreement forPrograms.2023.Availableat: https://www.nrmp.org/wp-content/ uploads/2022/09/2023-MPA-Main-Match-Program-FINAL-3.pdf
AccessedJune28,2023.
11.CouncilofResidencyDirectors.2023CORDstatementonresidency interviews.2023.Availableat: https://www.cordem.org/siteassets/files/ board/adv.–position-statements/2023-cord-statement-on-residencyinterviews.pdf.AccessedJuly26,2023.
12.AAMC.InterviewsinGME:Wheredowegofromhere?2023. Availableat: https://www.aamc.org/about-us/mission-areas/ medical-education/interviews-gme-where-do-we-go-here
AccessedJuly23,2023.
13.TheCoalitionforPhysicianAccountability’sWorkGrouponMedical StudentsintheClassof2022MovingAcrossInstitutionsforInterviews forPostgraduateTraining.Recommendationson2021–22residency seasoninterviewingformedicaleducationinstitutionsconsidering applicantsfromLCME-accredited,U.S.osteopathic,andnon-U.S. medicalschools.Availableat: https://physicianaccountability.org/ wp-content/uploads/2021/08/Virtual-Rec_COVID-Only_Final.pdf AccessedJune22,2022.
14.LewkowitzAK,RamseyPS,BurrellD,etal.Effectofvirtualinterviewing onapplicantapproachtoandperspectiveofthematernal-fetal medicinesubspecialtyfellowshipmatch. AmJObstetGynecolMFM. 2021;3(3):100326.
15.BoydCJ,AnanthasekarS,VernonR,etal.Interviewhoarding: disparitiesintheintegratedplasticsurgeryapplicationcycleinthe COVID-19pandemic. AnnPlastSurg. 2021;87(1):1–2.
16.GordonAM,ConwayCA,ShethBK,etal.Howdidcoronavirus-19 impacttheexpensesformedicalstudentsapplyingtoanorthopaedic surgeryresidencyin2020to2021? ClinOrthopRelatRes. 2022;480(3):443–51.
17.WhelanA.Openletteronresidencyinterviews.2020.Availableat: https://www.aamc.org/media/50291/download?utm_source=sfmc& utm_medium=Email&utm_campaign=ERAS&utm_content=Interviews AccessedJanuary19,2024.
18.FarcyD,JungJ,AintablianH,etal.Consensusstatementonthe 2020–2021residencyapplicationprocessforUSmedicalstudents planningcareersinemergencymedicineinthemainresidency match.2020. https://www.emra.org/be-involved/be-an-advocate/ working-for-you/residency-application-process AccessedJanuary22,2024.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 185 Bouldinetal. ResidencyApplicantCancellationPatterns
Geography:ANationalSurvey
AlineBaghdassarian,MD,MPH*†
JessicaA.Bailey,MD‡
DeryaCaglar,MD§∥
MichelleEckerle,MD,MPH¶#
AndreaFang,MD**
KatherineMcVety,MD††‡‡
ThuyNgo,DO,MEd§§
JerriA.Rose,MD∥∥¶¶
CindyGanisRoskind,MD##
MelissaM.Tavarez,MD,MS***
FrancesTurcotteBenedict,MD,MPH†††‡‡‡
JoshuaNagler,MD,MHPEd§§§∥∥∥
Authorscontinuedatendofpaper
*InovaL.J.MurphyChildren'sHospital,DepartmentofPediatrics, FallsChurch,Virginia
† UniversityofVirginia,SchoolofEducation,Charlottesville,Virginia ‡ OregonHealth&ScienceUniversity,DepartmentofPediatricsand EmergencyMedicine,Portland,Oregon
§ UniversityofWashington,DepartmentofPediatrics,Seattle,Washington
∥ SeattleChildren’sHospital,DepartmentofPediatrics, Seattle,Washington
¶ UniversityofCincinnatiCollegeofMedicine,DepartmentofPediatrics, Cincinnati,Ohio
# CincinnatiChildren’sHospital,DepartmentofPediatrics,Cincinnati,Ohio Affi liationscontinuedatendofpaper
SectionEditors:KendraParekh,MD,andChrisMerritt,MD
Submissionhistory:SubmittedNovember15,2023;RevisionreceivedFebruary10,2024;AcceptedFebruary21,2024
ElectronicallypublishedMarch14,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18581
Introduction: Virtualinterviews(VI)arenowapermanentpartofpediatricemergencymedicine(PEM) recruitment,especiallygiventhecostandequityadvantages.Yetinabilitytovisitprogramsinpersoncan impactdecision-making,leadingapplicantstoapplytomoreprograms.Moreover,thecostadvantagesof VImayencourageapplicantstoapplytoprogramsfartherawaythantheymightotherwisehavebeen willingorabletotravel.Thiscouldcreateunnecessarystrainonprograms.Weconductedthisstudyto determinewhetherPEMfellowshipapplicantswouldapplytoalargernumberofprogramsandindifferent geographicpatternswithVI(2020and2021)ascomparedtoin-personinterviews(2018and2019).
Methods: WeconductedananonymousnationalsurveyofallPEMfellowscomparingtwocohorts: currentfellowswhointerviewedinperson(appliedin2018/2019)andfellowswhounderwentVIsin2020/ 2021(currentfellowsandthoserecentlymatchedin2021).ThestudytookplaceinMarch–April2022. Questionsfocusedongeographicconsiderationsduringinterviewsandthematch.Weuseddescriptive statistics,chi-squareand t-testsforanalysis.
Results: Overallresponseratewas42%(231/550);32%(n = 74)interviewedinpersonand68% (n = 157)virtually.Fellowsappliedtoamedianof4/6geographicregions(interquartilerange2,5).Most appliedforfellowshipbothinthesameregionasresidency(216,93%)andoutside(192,83%).Onlythe PacificregionsawastatisticallysignificantincreaseinapplicantsduringVI(59.9%vs43.2%, P = 0.02). Therewasnostatisticaldifferenceinthenumberofprogramsappliedtoduringin-personvsVI (meandifference(95%confidenceinterval0.72, 2.8 – 4.2).Amajoritymatchedintheirpreferred statebothduringVI(60.4%)andin-personinterviews(65.7%).Thedifferencewasnotstatistically significant(P = 0.45).
Conclusion: WhilemorePEMfellowshipapplicantsappliedoutsidethegeographicareawheretheir residencywasandtothePacificregion,therewasnooverallincreaseinthenumberofprogramsor geographicareasPEMapplicantsappliedtoduringVIascomparedtoin-personinterviewseasons.As thiswasthe firsttwoyearsofVI,ongoingdatacollectionwillfurtheridentifytrendsandtheimpactofVI. [WestJEmergMed.2024;25(2)186–190.]
VirtualInterviewsandthePediatricEmergencyMedicineMatch
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 186 EDUCATIONAL SPECIAL ISSUE:BRIEF RESEARCH REPORT
INTRODUCTION
Since2020,virtualinterviews(VI)havebeenpreferredfor traineerecruitment.1 Withthebenefitsoflowercostand greaterequity,itislikelytoremainapermanentpartof recruitment,despiteageneralpreferenceforface-to-face interviews.2–5 TheVIprocessandassociatedperceptions havebeendescribedintheliterature.2,3,6–9 Theinabilityto visitaprograminpersoncanimpactdecision-makingduring ranking,4,10–14 andanincreasednumberofapplications couldcreateunduestrainonprograms.15–17
Geographiclocation,senseof “fit,” andprogram leadershipweredescribedasmajorcontributorsto applicants’ rankpreference.18 Anationalcohortofpediatric emergencymedicineprogramdirectors(PEMPD),inajoint statement,raisedconcernthatVIcouldleadapplicantsto applytomoreprogramsandtoprogramsfartherawaythan theymaybewillingorabletotravel.10 Weconductedthis studytodeterminewhetherPEMfellowshipapplicants wouldapplytoalargernumberofprogramsandindifferent geographicpatternswithVI(2020and2021)ascomparedto in-personinterviews(2018and2019).
METHODS
DesignandParticipants
Thiswasananonymous,self-administered,crosssectional,web-basedsurveyofPEMfellowsinthe UnitedStates.Participationwasvoluntary,andnoincentive wasprovidedforcompletion.Thestudywasexempted bytheinstitutionalreviewboardatYaleUniversity, withinformedconsentimpliedbycompletionofthesurvey byparticipants.
SurveyDevelopment
Thesurveyquestionnairewasdevelopedthroughiterative feedbackandamodifiedDelphiprocesstodetermine itemimportance.ThirteenPEMPDswithexpertisein performanceandevaluationparticipatedinmultiplerounds ofrevisionsandediting.Pilottestingwasconductedwithtwo pediatrichospitalmedicinefellowswhohadappliedtothe matchduringVIsandtwopediatricchiefresidentswhowere alsointerviewingforfellowshipsusingVI,atthelead institution.Revisionsweremadebasedonpilotfeedback (surveyprovidedin SupplementaryAppendix1).Thesurvey includedmultiple-choicequestionsaboutlocationof residency,statesappliedtoandinterviewedforfellowship, preferredlocationforfellowship,statesvisitedinpersonfor thepurposeofthematch,andstatematchedin.Italsoasked fellowstoindicatestatesofresidenceofimmediatefamily (parents,siblings,orpartners)andaboutcompellingreasons (otherthanfamily)thatmayhaveledfellowstofavorastate orregion(freetext).Geographicregionsweredefinedas Northeast,Southeast,Midwest,Southwest,Rocky Mountain,andPacificregions.19
Whatdowealreadyknowaboutthisissue?
Virtualinterviewsareapermanentpartof recruitment.Theyoffercostandequity advantageswhileposingchallengestoboth applicantsandprograms.
Whatwastheresearchquestion?
DidPEMfellowshipapplicantsapplytoa largernumberofprogramsandindifferent geographicpatternswithVIascomparedto in-personinterviews?
Whatwasthemajor findingofthestudy?
VIdidnothaveasigni fi cantimpactonthe numberofprogramsorgeographicareas applicantsappliedto.
SurveyDistribution
ThesurveywasreviewedandapprovedbytheAmerican AcademyofPediatrics(AAP)SectiononEmergency Medicine(SOEM)PDsurveysubcommitteepriorto distributiononQualtrics(Qualtrics,Provo,UT)toallPEM PDs,viatheAAPSoEMPDCommitteelistserv.ThePDs forwardedthesurveylinktotheircurrentandincoming fellows(thoserecentlymatchedtostartinJuly2022).Each PDcompletedaseparatequestionnaireindicatingthetotal numberofcurrentandrecentlymatchedfellowstowhom theyforwardedthesurvey.
Analysis
Participantsweredividedintotwogroups:VI(2020or 2021)andinperson(2018or2019).Weperformed descriptivestatisticsincludingfrequencies,percentages, meanswithstandarddeviations,andmedianswith interquartilerange(IQR).Chi-squaretestscompared categoricalvariablesandt-tests,continuousvariableswith 95%confidenceintervals(CI).Weconsideredatwo-tailed alphaof <0.05tobestatisticallysignificant.Weconducted analysesinIBMSPSSStatisticsversion28(IBM Corporation,Armonk,NY).
RESULTS
ThePDsreportedthattheyforwardedthesurveyto406 currentfellowsand144incomingfellows.Theresponserate forcurrentfellowswas35%(143/406)andforincoming fellows,61%(88/144).Overall,theresponseratewas42% (231/550).Ofthetotalrespondents,62%(143/231)were currentfellowsand38%(88/231)incoming.Twofellows (1%)didnotcompleteresidencyintheUS,and12(5%) appliedtoPEMfellowshipmorethanonce.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 187
Baghdassarianetal. VIandthePEMMatchGeography
AllincomingfellowshadundergoneVI,whereas 48%ofthecurrentfellowshadundergoneVI(69/143). Overall,32%ofrespondents(74/231)interviewedinperson and68%(157/213)virtually.Therewasnostatistical differenceinthenumberofprogramsappliedtoduringinpersonvsVI(meandifference(95%CI):.72[ 2.8,4.2]) (Appendix2Table).
Datadescribingthegeographictrainingandlocation preferenceofparticipants arepresentedinthetablein appendix2.Fellowsappliedtoamedianoffourofthe sixgeographicregions(IQR2,5).Mostparticipantsapplied forfellowshipinthesamegeographicregionastheirresidency (216,93%)andoutsidetheirresidencyregionaswell (192,83%).OnlythePacificregionsawastatistically significantincreaseinapplicantsduringVI(59.9%vs43.2%, P = 0.02)(Table1).
Lessthanhalfofrespondentshadimmediatefamily memberslivinginthesamestateasresidency(N = 111,48%), fellowship(N = 90,39%),ortheirpreferredmatch state(N = 95,41%).Compellingreasonstoapplyto anareaincludedfamiliaritywithlocation(N = 128,55%); similarlocationtoresidency(N = 65,28%);anda desiretotraininanewarea(N = 53,23%).Partner’ s employmentwasanimportantfactorfor89(38%),salary
andcostoflivingfor76(33%),andschoolforchildren for20(9%).
DISCUSSION
OurresultsshowthatVImayallowsomecandidatesto exploreandconsiderregionstheymaynothaveotherwise duetologisticalor financialconstraints,withoutincreasing thenumberofprograms,regionsorstatestheyapplyto. Theseresultsareconsistentwiththe2021NRMPsurvey where52%reportednoimpactoftheVIonthenumberof programsappliedto.5 Residencyprogramshavereportedan increaseinmatchedinternalcandidatesduringVI.11,12,20,21 InPEM,apre-pandemicstudyofPDsshowedthat29%of fellowscompletedresidencyatthesameinstitution.22 While wedidnothavedataattheinstitutionallevel,therewasno significantincreaseinfellowsmatchingwithinthestateof theirresidencyprogramwithVI.ThissuggeststhatVIwere notasignificantdetrimenttoapplicantsrankingprograms andgeographicareas,despitetheabsenceofopportunitiesto meetinpersonandvisitprograms.Thisalsoallowsprograms tohaveaccesstoalargerandpotentiallymorediversepool ofcandidates.9
Proximitytofamilywasnotasignificantconsiderationfor mostapplicantsandwasnotimpactedbyVI.Residency
In-person interviews(N = 74) Virtualinterviews (N = 157)
Statisticalsignificance (P valueor95%CI)
Appliedtoregionforfellowship,N(%)
Northeast59(79.7)123(78.3)0.81
Southeast41(55.4)102(65)0.16
Midwest50(67.6)111(70.7)0.63
Southwest38(51.4)86(54.8)0.63
RockyMountains31(41.9)73(46.5)0.51
Pacific32(43.2)94(59.9)0.02
Appliedtosamegeographicregionas residency,N(%)
Appliedoutsidegeographicregionas residency,N(%)
71(98.6)145(94.8).278
56(77.8)136(88.9)0.03
Numberofregionsappliedto,mean(SD)3.4(1.8)3.8(1.8)Meandifference(95%CI):.36( .15,.89)
Numberofstatesappliedto,mean(SD)9(7.3)9.7(6.8)Meandifference(95%CI):.73( 1.2,2.7)
Numberofprogramsappliedto,mean(SD)13.3(12.8)14(12.5)Meandifference(95%CI):.72( 2.8,4.2)
Numberofprogramsinterviewedat, mean(SD)
Matchedinpreferredstate,N(%)46(65.7)84(60.4)0.46
Matchedinsamestateasresidency,N(%)31(42%)59(38%)0.58
Preferredtomatchinstatewithimmediate familypresent,N(%)
Wenttovisitstate/program,N(%)9(14)23(17)0.61 CI,confidenceinterval.
3.1( 1.7,1.1)
Table1. Influenceofvirtualinterviewsonapplicantbehaviorandoutcomes.
7.2(4.7)6.9(5.2)Meandifference(95%CI):
36(52.9)59(46.8)0.42
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 188
VIandthePEMMatchGeography Baghdassarianetal.
applicantsreportedgeography,qualityoflife,casevariety, curriculum,institutionalreputation,expertiseinareasof interest,andprogramsizeaskeyfactors.23 Applicantsto PEMhighlightedfamiliaritywiththeregionorwantingto exploreanewareaasfactorsforexploringprogramsin differentregions.
LIMITATIONS
Limitationsofthisstudyincludethesmallerresponserate ofthecurrentfellowsascomparedtotheincomingfellows. Thislowresponseratelimitedthesamplesizeofthein-person cohort,impactingthestatisticalsignificanceofourresults. Thisdifferentialresponsefromtheincomingfellowsmay havebeenduetodesirabilitybiaswherethiscohortof applicantsmayhavetendedtostatethattheymatchedin theirpreferredstate.Tominimizethis,wedesignedourstudy tobefullyanonymousandself-administered,andthe questionswerewordedtoretainobjectivityoftheanswers. Respondentsmayalsohaveexperiencedrecallbiasregarding thestatesandprogramstowhichtheyapplied.Thisbias couldpotentiallyhavecontributedtothelowerresponserate amongthecurrentfellowswhohadinterviewedin2018/2019, 3–4yearspriortothesurveydate,comparedtothemore recentapplicantswhohadamorerecentrecollectionofthe questionsaskedinthesurvey.
Anotherlimitationisthatwedidn’texplicitlyaskthetotal numberoffellowsineachclasscohort;however,sincethe PEMfellowshipclasssizeintheUSdoesn’tvarysignificantly fromyeartoyear(byvirtueoftheapprovedfellowship positionsavailable),thedenominatorisexpectedtobe relativelyconstant.
Thisstudywasnotdesignedtolookattheratesof applicationstoindividualprogramsnorassessthepostmatchopinionsofprogramsandfellowsregardingthe resultsofthematch.Thisinformationwouldprovidea deeperinsightintotheimpactoftherecruitmentprocess; however,itisalsopronetobiasasfellowsonlyexperience trainingatasingleinstitution.Wealsodidnottakeinto considerationtheconcentrationofPEMprogramsby regionortheavailablefellowshipslotsperprogramor region.However,theobjectiveofthisstudywastolookat thedifferencesbeforeandduringVIs,andtherewasnota signi fi cantchangeinavailablefellowshipslotsorprograms duringtheseyears.Asthenumberofpediatricfellowship applicantsrises,furtherinvestigationintotheimpactofVIs isnecessarytogainadeeperunderstandingofits implicationsandtooptimizethisprocessbothfor applicantsandprograms. 24
CONCLUSION
WhilemorePEMfellowshipapplicantsappliedoutside thegeographicareawheretheirresidencywasandtothe Pacificregion,therewasnooverallincreaseinthenumberof
programsorgeographicareasthatPEMapplicantsapplied toduringVIduringthe firsttwoyearsofitsinstitution,as comparedtoin-personinterviewseasons.Ongoing monitoringoftheinterviewandmatchseasonswillhelp identifyfuturetrendsandimpactofVIs.
AUTHORSANDAFFILIATIONSCONTINUED
MelissaL.Langhan,MD,MHS¶¶¶
** StanfordUniversitySchoolofMedicine,Departmentof PediatricEmergencyMedicine,PaloAlto,California
††Children’sHospitalofMichigan,DepartmentofPediatrics, Detroit,Michigan
‡‡CentralMichiganUniversity,SchoolofMedicine, DepartmentofPediatrics,Detroit,Michigan
§§JohnsHopkinsUniversity,SchoolofMedicine,Department ofPediatrics,Baltimore,Maryland
∥∥RainbowBabies&Children’sHospital,Departmentof Pediatrics,Cleveland,Ohio
¶¶CaseWesternReserveUniversity,SchoolofMedicine, DepartmentofPediatrics,Cleveland,Ohio
##ColumbiaUniversityIrvingMedicalCenter,Pediatricsin EmergencyMedicine,NewYork,NewYork
*** UniversityofPittsburgh,SchoolofMedicine,Departmentof Pediatrics,Pittsburgh,Pennsylvania
†††UniversityofMissouriofKansasCitySchoolofMedicine, DepartmentofPediatrics,KansasCity,Missouri
‡‡‡UniversityofKansasMedicalCenter,KansasCity, Missouri
§§§BostonChildren’sHospital,DepartmentofPediatricsand EmergencyMedicine,Boston,Massachusetts
∥∥∥HarvardMedicalSchool,DepartmentofPediatricsand EmergencyMedicine,Boston,Massachusetts
¶¶¶YaleUniversitySchoolofMedicine,Departmentof PediatricsandEmergencyMedicine,NewHaven, Connecticut
AddressforCorrespondence:AlineBaghdassarian,MD,MPH, InovaFairfaxL.JMurphyChildren’sHospital,3300GallowsRd., FallsChurch,VA22042-3300.Email: aline.baghdassarian@ inova.org
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.
Copyright:©2024Baghdassarianetal.Thisisanopenaccess articledistributedinaccordancewiththetermsoftheCreative CommonsAttribution(CCBY4.0)License.See: http:// creativecommons.org/licenses/by/4.0/
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 189 Baghdassarianetal. VIandthePEMMatchGeography
REFERENCES
1.AssociationofPediatricProgramDirectors,CouncilofPediatric Subspecialties,AssociationofMedicalSchoolPediatricDepartment Chairsetal.APPD/CoPS/AMSPDC/NextGenPediatricianslettertoour pediatricscommunityaboutthefellowshiprecruitmentprocess.2021. Availableat: https://downloads.aap.org/AAP/PDF/2021_05_19_-_ Pediatric_Fellowship_Recruitment_Recommendations_Final.pdf AccessedJuly15,2022.
2.VanDerLaanL,GeorgeR,NesiamaJA,etal.Virtualinterviewingfor pediatricemergencymedicinefellowship-anationalsurvey. Pediatr EmergCare. 2022;38(4):e1207–12.
3.GuptaS,GrierArthurL,ChandlerN,etal.Isthechanginglandscapeof fellowshiprecruitmentduringCOVID-19heretostay? JPediatrSurg. 2022;57(10):445–50.
4.DasAJ,AugustinRC,CorbelliJA,etal.Residencyandfellowship programleaders’ perceptionsofvirtualrecruitmentandinterviewing. JGradMedEduc. 2022;14(6):710–3.
5.NationalResidentMatchingProgram.2021Applicantandprogram directorsurvey findings:impactofthevirtualexperienceonthetransition toresidency.2021.Availableat: https://www.nrmp.org/wp-content/ uploads/2021/08/Research-Brief-Virtual-Experience-2021-FINAL.pdf AccessedJuly15,2022.
6.WolffMandBurrowsH.Planningforvirtualinterviews:residency recruitmentduringapandemic. AcadPediatr. 2021;21(1):24–31.
7.BernsteinSA,GuA,ChretienKC,etal.Graduatemedicaleducation virtualinterviewsandrecruitmentintheeraofCOVID-19. JGradMedEduc. 2020;12(5):557–60.
8.McCainC,KempB,BaierMB,etal.Aframeworkforthevirtualmedical interviewprocess:considerationsfortheapplicantandtheinterviewer. OchsnerJournal. 2022;22(1):61–70.
9.PetersenTL,KingJC,FussellJJ,etal.Benefitsandlimitationsofvirtual recruitment:perspectivesfromsubspecialitydirectors. Pediatrics. 2022;150(4):e2022056735.
10.AllisterL,BaghdassarianA,CaglarD,etal.Pediatricemergency medicinefellowshipdirectors’ 2021collectivestatementonvirtual interviewsandsecondlooks. PediatrEmergCare. 2021;37(11):585–7.
11.EderleA,ShahriariS,WhisonantC,etal.TheimpactofCOVID-19on thedermatologymatch:anincreaseinthenumberofstudentsmatching athomeprograms. DermatolOnlineJ. 2021;27(9).
12.FaletskyA,ZitkovskyH,GuoL.TheimpactofCOVID-19on plasticsurgeryhomeprogrammatchrates. AnnPlastSurg. 2022;88(1):4–6.
13.MulcahyCF,TerhaarSJ,BoulosS,etal.Didmoreotolaryngology residencyapplicantsmatchattheirhomeinstitutionsin2021? InvestigatingtheimpactoftheCOVID-19pandemic. AnnOtolRhinol Laryngol. 2022;131(12):1375–80.
14.QuinnA,MannE,RaikinJ,etal.PD24-06Theeffectofthe COVID-19pandemiconurologymatchbylocation. JUrol. 2021;206(Suppl3):e427–8.
15.InclanPM,WoiczikMR,CummingsJ,etal.Virtualpediatricorthopaedic fellowshipinterviewsduringthepandemic:Whatdidtheapplicantsand programsthink? JPediatrOrthop. 2022;42(7):e806–10.
16.LewkowitzAK,RamseyPS,BurrellD,etal.Effectofvirtualinterviewing onapplicantapproachtoandperspectiveofthematernal-fetal medicinesubspecialtyfellowshipmatch. AmJObstetGynecolMFM. 2021;3(3):100326.
17.ReamMAandThompson-StoneR.Virtualresidencyinterview experience:thechildneurologyresidencyprogramperspective. PediatrNeurol. 2022;126:3–8.
18.DiGiustoM,LupaMC,CorridoreM,etal.TheimpactoftheCOVID-19 pandemiconthe2020pediatricanesthesiologyfellowship applicationcycle:asurveyofapplicants. PaediatrAnaesth. 2021;31(9):968–76.
19.STUDYGUIDEUSII.2cGeographyThemes.Availableat: https://www.solpass.org/7ss/standards/StudyUSII.2c.htm AccessedFebruary4,2024.
20.WhisonantCT,ShahriariSR,HarrisonJ,etal.Evaluatingtheintegrated plasticsurgeryresidencymatchduringthenovelcoronaviruspandemic. Cureus. 2021;13(8):e16988.
21.CotnerCE,MercadanteSF,SheaJA.Assessingtheimpactofthe COVID-19pandemicongeographicresidencyplacementrelativeto medicalschoollocation. JGradMedEduc. 2022;14(1)108–11.
22.BradleyT,ClingenpeelJM,PoirierM.Internalapplicantstopediatric emergencymedicinefellowshipsandcurrentuseofthenationalresident matchingprogrammatchasurveyoffellowshipdirectors. PediatrEmerg Care. 2015;31(7):487–92.
23.LoveJN,HowellJM,HegartyCB,etal.Factorsthatinfluence medicalstudentselectionofanemergencymedicineresidency program:Implicationsfortrainingprograms. AcadEmergMed. 2012;19(4):455–60.
24.TheAmericanBoardofPediatrics.Yearlygrowthinpediatricfellowsby subspecialtybydemographicsandprogramcharacteristics.Available at: https://www.abp.org/content/yearly-growth-pediatric-fellowssubspecialty-demographics-and-program-characteristics AccessedJanuary26,2023.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 190 VIandthePEMMatchGeography Baghdassarianetal.
ORIGINAL RESEARCH
AnalysisofAnonymousStudentNarrativesAboutExperiences withEmergencyMedicineResidencyPrograms
MollyEstes,MD*
JacobGarcia,MD†
RonnieRen,MD,MSEd‡§
MarkOlaf,DO∥
ShannonMoffett,MD¶
MichaelGaluska,MD#
XiaoChiZhang,MD,MS**
*LomaLindaUniversity,DepartmentofEmergencyMedicine,LomaLinda,California
† MountSinaiMorningside/West,DepartmentofEmergencyMedicine,NewYork,NewYork
‡ UniversityofMassachusetts,DepartmentofEmergencyMedicine,Boston,Massachusetts
§ UniversityofFloridaHealthShandsHospital,DepartmentofEmergencyMedicine, Gainesville,Florida
∥ GeisingerCommonwealthSchoolofMedicine,DepartmentofEmergencyMedicine, Scranton,Pennsylvania
¶ RutgersNewJerseyMedicalSchool,DepartmentofEmergencyMedicine, Newark,NewJersey
# ConemaughMemorialMedicalCenter,DepartmentofEmergencyMedicine, Johnstown,Pennsylvania
**ThomasJeffersonUniversity,DepartmentofEmergencyMedicine, Philadelphia,Pennsylvania
SectionEditors:JeffreyDruck,MD,andMichaelGottlieb,MD
Submissionhistory:SubmittedMarch15,2023;RevisionreceivedOctober18,2023;AcceptedNovember21,2023
ElectronicallypublishedFebruary5,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.17973
Background: Academicemergencymedicine(EM)communitieshaveviewedanonymousonline communities(AOC)suchasRedditorspecialty-specific “applicantspreadsheets” aspooradvisingresources. Despitethis,robustEMAOCsexist,withlargeuserbasesandheavyreadership.Insightsaboutapplicants’ authenticexperiencescanbecriticalforapplicantsandprogramleadershipdecision-making.Todate,there arenoEMstudiestoqualitativelyassessEMAOCnarrativesduringtheapplicationcycle.Ourgoalwasto performaqualitativeanalysisofstudents’ EMprogramexperiencesthroughapubliclyavailableAOC.
Methods: Thiswasaqualitativeanalysisofapubliclyavailable,time-stamped,user-lockedAOC dataset: “Official2020–2021EmergencyMedicineApplicantSpreadsheet.” Weextractedandthendeidentifiedalldatafromselectedsub-sheetsentitled “VirtualInterviewImpressions” and “Rotation Impressions.” Fourinvestigatorsusedconstantcomparativemethodtoanalyzethedatainductively,and theysubsequentlymettogeneratecommonthemesdiscussedbystudents.Preliminarythematic analysiswasconductedonarandomsampleof 37/183(20%)independentnarrativestocreatetheinitial codebook.Thiswasusedandupdatediterativelytoanalyzetheentirenarrativesetconsistingof841 discretestatements.Finally,twouniquecodeswerecreatedtodistinguishwhethertheidentifiedsubthemes,orprogramattributes,werelikely “modifiable” or “non-modifiable.”
Results: Weidentifiedsixmajorthemes:livingandworkingconditions;interpersonalrelationships;learning experiences,postgraduatereadiness,andonline/virtualsupplements.Commonsub-themesincludedpatient population(13%);residentpersonality(7%);programleadershippersonality(7%);relationshipwithfaculty/ leadership(6%);geography(4%);practicesetting(4%);programreputation(4%),andpostgraduateyear-3 experiences(4%).Modifiablesub-themesoutnumberednon-modifiablesub-themes,60.7%to39.3%.
Conclusion: Inthisanalysisofselectedmedicalstudents’ narrativesinanAOC,themajorityofidentified themesrepresentedtopicsthatmayserveasexternalfeedbackforEMresidencyprogramsandtheir clerkships.SelectiveuseofAOCsmaysetaprecedentforfutureprogramassessmentsbyapplicants andinformprogramleadershipofimportantprogrammaticelementsintheeyesofapplicants.It elucidatesimportantthemesintheirinteractionsorlearningexperienceswithprogramsandcreates opportunitiesforlearner-centricprogramimprovement.[WestJEmergMed.2024;25(2)191–196.]
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 191
INTRODUCTION
Theacademiccommunityhastraditionallyviewed anonymousonlinecommunities(AOC)aspoorresourcesfor advising,recommendingthatstudentsbewaryofthemor avoidthemaltogether.1 Commonthemesaddressedagainst theseforumsincludelackofcommenterprofessionalism,2–4 informationinaccuracy,5 andbreachofethicsviamalicious postingoffalsified,incomplete,orprivilegedinformation.2,3 PriorstudieshavealsoshownthatanonymousAOC commentersmaynotnecessarilyreflecttheentireapplicant population.6 Withinemergencymedicine(EM),students reportinformationfromAOCssuchasRedditandStudent DoctorNet(SDN)asthe “leasttrustworthy” comparedto otheradvisingresources.7
Despitethis,mostspecialties,includingEM,haverobust AOCsformedicalstudents,boastinglargeuserbasesand robustdiscussionthreadswithheavyreadership.8–11 Within theseanonymousforums,studentsdiscussdiversetopics abouttheapplicationprocess,specialty-specificquestions, andstudentexperiencesapplyingto,rotatingat,or interviewingwithspecificprograms.12,13 Forexample, studiesidentify “program-specificinformation” asa commonthemeinotolaryngology-andradiology-applicant AOCs;however,their findingswerelimitedincharacterizing specifictopicsdiscussed.12,13 Thereisalsoaconsolidated, annuallyrenewed,anduser-generatedGoogleSpreadsheet circulatingwithinEMforumswithastatedgoalto “provideacentrallocationforapplicantstoresearch differentresidencyprograms,viewinformationabout otherapplicantsandwheretheyareapplying, andshareinformationaboutaway,interviews,and generaladvice.”14
ForEM,thediscussionofauthentic,program-specific experiences,suchasthestudent’sinterviewdayexperience andinteractionwithresidents,havehistoricallybeenranked asthetoptwofactorsinimpactingtheirrankorder,making thisinformationhighlyvaluabletobothapplicantsand programstakeholders.15 Ourprimarygoalwasto characterizewhatprospectiveEMapplicantsdiscussinan AOCforumregardingtheirexperienceswithspecific programs.Oursecondarygoalwastoidentifypotentially usefulinformationforprogramimprovement anddevelopment.
METHODS
Thiswasaqualitative,retrospectivereviewofapublicly availableAOCforEMrotationsin2020–2021.Itwas submittedforinstitutionalreviewboardreviewthrough ThomasJeffersonUniversityanddeterminedtonotmeetthe definitionofhumansubjectsresearch.Weanalyzedextracted datafromanonline,time-stamped,anduser-lockedGoogle Sheetentitled “Official2020–2021EmergencyMedicine ApplicantSpreadsheet,” whoselinkcanbefoundwithin popularAOCssuchasReddit,SDN,andDiscord.14 “The
Spreadsheet” allowsanonymousindividualstopost requestedinformationregardingspecificEMprograms.The spreadsheetcontainsmultiplesub-pages,or “sheets,” to addressdifferenttypesofinformationanapplicantmight seek.Thisincludessheetslistingprogram-specificfactssuch as “ProgramBenefits” and “ProgramInformation”;sheets describingstudentexperienceswithaprogramlike “RotationExperience,”“VirtualIV(Interview) Impressions,” or “NameandShame”;andsheetsaddressing miscellaneousapplicationtopicssuchas “Rejection/Wait List” or “DroppedInterviews” tohelpapplicants coordinatelogistics.
Withpermissionfromthepageadministrator,confirmed tobeacurrentEMresident,wecreatedareplicaofthe spreadsheetonSeptember12,2021,forthepurposeofthis study.Uponreviewofallavailablesheetswithinthe spreadsheet,thesub-pagesentitled “VirtualIV” and “RotationImpressions” werepurposefullysampledvia groupconsensusforanalysisastheywerefelttomostlikely includestudents’ directimpressionsofprograms.Incontrast, weexcludedsub-pagessuchas “NameandShame” from qualitativeanalysisduetohighlikelihoodofcontaining causticandcontroversialnarratives.Asthepurposeofthis studywastoinvestigate “what” isbeingsaid,not “who” is discussingthemorto “whom” itisaddressed,one investigatortransferredallcommentsfromtheselectedpages intoasingledatasetwhileremovingpotentiallyidentifying userorprograminformation.
Weperformedqualitativeanalysisprimarilyusingthe constantcomparativemethod,16 whereexcerptsofrawdata areorganizedintogroupsaccordingtoattributesandthose groupsarefurtherstructuredtoformulateanewtheory.The selectedsub-pagesyielded183individualnarratives discussingstudents’ impressionofthede-identified programs.Arandomnumbergeneratorwasusedtoselect 20%ofindividualnarrativesasaconveniencesamplefor investigatorstoinductivelycreateaworkingcodebook,de novo.Allduplicateswereidentifiedandremoved,untilthe excerptswereallunique.Thedatasetwasindependently analyzedbytheinvestigators[ME,JG,RR,XCZ]toidentify thematiccontentwithineachnarrativeforinductivecoding. Individuallyidentifiedthemeswerecomparedamong investigatorstogeneratecommonthemes.Thesethemes wereorganizedintomajor “themes” and “sub-themes” to createtheinitialcodebook.Thiswasusedbyasingle investigator(JG)tocodetheentiredataset.Additionalsubthemesidentifiedduringthisprocesswereupdatedintothe codebookunderexistingmajorthemes.Uponcompletion, allthemesandtheirassociatedexcerptswerereviewedbythe remaininginvestigatorstoensurecodingconsensus.The entiredatasetwasrevieweduntilnoadditionalthemeswere identified(see Appendix1).
Uponreviewingthe finalcodebook,wecreatedtwo additionalthematiccategories:1)potentially “modifiable”
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 192 StudentNarrativesRegardingEMResidency Estesetal.
programattributesand2)lesslikelyor “non-modifiable” programattributes.Drawinguponourcollective experiences,wedefined “modifiable” subthemesasattributes mostlikelyunderthedirectcontroloftheeducation leadershipand “non-modifiable” subthemesasattributes thatareeithertrulynon-modifiableorwouldrequire significantinputfromoutsidestakeholderstochange.This distinctionwasmadewiththeunderstandingthatdifferent programshavedifferentabilitiestomodifycertainattributes.
RESULTS
From183comments,841discretestatementswere identifiedandcoded.Weidentifiedsixthemes:working conditions;interpersonalrelationships;learningexperience; livingconditions;postgraduatereadiness;andonline/virtual
supplement,asshownin Figure and Table1.Thetoptwo encodedthemes workingconditionsandinterpersonal relationships comprised572(68%)ofthetotalcoded statements(324[38.5%]and248[29.5%],respectively).
Sub-themesidentifiedwithineachtheme(see Appendix1 forafulllistingofsub-themeswiththeircorresponding numberofcodedstatements)werethensubdividedto representmodifiableandnon-modifiableclerkship/program aspects(Tables2 and 3).Modifiablesub-themes outnumberednon-modifiablesub-themes(60.7%vs39.3%). Thesub-themeshousedwithinthethemeofinterpersonal relationshipsrepresentedthelargestsinglecategoryof modifiableattributeswith248(29.5%)statements. Commentsonworkingconditionsandlearningexperience werethesecondandthirdlargestcategories,with109(13%) and118(14%)comments,respectively.Themajorityofnonmodifiablesub-themeswerefoundwithinthethemeof workingconditionswith215(25.6%)individualcomments, whichrepresented65%ofallnon-modifiablecomments.The secondlargestnon-modifiablesub-themewaswithinthe themeoflivingconditions,includingcommentsonthelocal geography,costofliving,ornearbyamenities.
DISCUSSION
Anonymousonlinecommunitieshavebeenhistorically viewedbyclerkshipandresidencyprogramleadershipas unreliableforumsforstudentdiscussionthatfoster confabulationoffactsandoperatesonrumorsandhearsay,a communicationtoolofthedisgruntled,andnotaresourceto betakenseriously.1–3,5–7 Thisisthe firststudytodescribe,in detail,thenarrativecontentofstudentsdiscussingtheir programimpressionsonanAOC.Our findingssuggestthat manyofthediscusseditemsarecommonconsiderationsfora studentseekingto findtheidealnextstageoftraining.
ThemeRepresentativecomment
Workingconditions
“ academicinstitutionmeansalotofconsultssometimes,some12-hourshifts(butmixof8sand12s), 50%ofshiftsasaninternareovernight nodebriefingprocessaftercodes/traumas ”
Interpersonal relationships “Everyprogrammentionsfamilyfeelbuthavingrotatedhereitwastrulytightknit.Facultyandresidenthang outsoftenincludingbeachtrips.”
Learningexperience “Mostattendingstoleratestudents,andtherestarereallyproactiveaboutteachingandgettingthestudents involved Whenitdoessettleabit,residentsareenthusiasticaboutyoureducationforthemostpart.You’ll gettodoalmostanyprocedureyouwantbecausetheresidentshavealreadydonethemathousandtimes before.”
Livingconditions “Arearightoutsideof[location]canbeaprooracon.Probably[would]havetodealwithalotoftrafficand highcostofliving.”
Postgraduate readiness “Reallyoldprogram,soalumnialloverthecountrytohelpwithjobplacement(lastclass1/2community, 1/4fellowship,1/4academic).With4differenthospitals,varietyoftrainingisverygoodandwillbeprepared foranytypeofjobcomingout.”
Online/virtual supplement “BestEDtour,literallytookaGoProfromtheEDdepartmenttotheactualEDsoyoucouldactually seetheED.”
Figure. Thematiccategoriesofcodedstatements,includingthe numberofindividualstatementsandpercentageoftotalstatements.
Table1. Identi fiedthemeswithrepresentativecommentscodedtoeach.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 193
Estesetal. StudentNarrativesRegardingEMResidency
ThemesSub-themes#Totalcodedcomments
WorkingconditionsPerks(fundingfortravel/activities,food,lounge,parking,etc)37109(13.0%)
DEI(includesLGBTQ+)27
Relationshipwithotherspecialties23
Wellness20
Scutwork2
InterpersonalrelationshipsResidents76248(29.5%)
Otherleadership/facultypersonality76
PDpersonality56
Responsivenesstoupwardfeedback16
Opportunityforupwardfeedback14
Generic8
Objectiveexperience2
LearningexperienceProcedures25118(14.0%)
Didactics/conference20
On-shiftteaching17
Autonomy16
POCUS12
Pediatrictraining12
EMS/prehospitaltraining5
Scholarlytracks5
Research5
Personalpatientload1
Online/virtualsupplementVirtualinterviewday2835(4.2%)
Virtualtour4
Virtualrotation2
Website1
MentorsinEMhavehistoricallyencouragedprospective EMapplicantstoinquireaboutinterpersonalinteractions andresidentworkingconditionswithinaspecificprogram. Ouranalysisrevealsthatstudentsarealsoseekingmore informationandcommentingonmanyofthesamefactors wehavebeenadvisingthemtoseekout.17
Moreover,analysisofthesub-themesrevealsauniquetrend towardpotentiallymodifiableprogramattributesthat,if addressed,couldbemutuallybeneficialforprogramsand applicants.Topicssuchasperceivedresidentwellness, diversity,equity,andinclusion,opportunitiesforupward feedback,andeffectivenessofon-shiftteachingareallunder thecontrolofaprogramtopotentiallyimprove.Manyofthese topicsareofrisingimportancetostudents.18,19 Theavailability ofthisinformationraisesaveryinterestingquestionfor programsandrecruitment:ifprogramswereawareofthese discussedtopicsandthestudentcommentsrelativetoeach
topic,wouldaprogrambelikelytochangeinternalelement(s) tomakeitselfmoreappealingtostudents?20
InlightoftherecentNationalResidentMatching Program(NRMP)resultsfrom2022andthecontinued downtrendofapplicationsin2023,21,22 manyEMprograms mustcontendwithasmallerapplicantpool,whichreduces thelikelihoodof fillingprograms,andoverallprogram competitivenessforapplicantrecruitment.Whilewecannot predictfuturetrends,ourspecialtyhasfaceddeclining studentapplicationsfortwoyearsinarowwitharising numberofresidencyprogramsandpositionsoverthelast severalyears.Astraditionalmatchingpatternsbeginto falter,residencyleadershipshouldconsideraddressing criticalelementsfromAOCs,insteadofignoringthemas traditiondictates.
Aninteresting finalobservationfromourstudyisthe relativelyscarcenumberofcommentsfromstudentson
Table2. Modifiablemajorthemesandsub-themesdeterminedbyauthorconsensusbyaclerkshiporresidencyprogram;percentagesareof thetotalnumberofcomments,N = 841.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 194 StudentNarrativesRegardingEMResidency Estesetal.
DEI,diversity,equity,inclusion; PD,programdirector; POCUS,point-of-careultrasound; EMS,emergencymedicalservices.
ThemesSub-themes#Totalcodedcomments
WorkingconditionsPatientpopulation(underserved,volume,trauma,pathologyetc)66215(25.6%)
Practicesetting(community,academic,county,Lvl1,HCA,etc)66
Programreputation/prestige/age21
Workhours17
Ancillaryhealthcarestaff15
EHR12
Salary9
Metrics6
Moonlighting3
LivingconditionsGeography5370(8.3%)
Amenities11
Costofliving6
PostgraduatereadinessFellowships1746(5.5%)
Jobs13
PGY4experience(lengthoftraining)12
PGY3experience(lengthoftraining)4
HCA,HospitalCorporationofAmerica; EHR,electronichealthrecord; PGY,postgraduateyear.
virtualoronlinecomponentsofaprogram.Ourdataset reflectedthe firstapplicationcycleduringtheCOVID-19 pandemicwithradicalparadigmshiftsinaway-rotation restrictionsandstudentsexploringvirtualinterview processesforthe firsttime.Despitetheseunprecedented large-scalechanges,only4.2%ofthetotalcommentsfocused onthe “virtual” aspectofprogramrecruitment.Thisisin starkcontrasttothesignificantamountoftimespentby institutionsandnationalorganizationsonvirtualrotations, virtualtours,ongoingvirtualinterviews,virtualresidency fairs,andvirtualhangoutsforstudentstosocializewith residents.Thedataremainsunclearbasedonthis informationfromasingleyeartoexplainthislackof commentary.Itmayperhapsbeduetolackofstudent participationinvirtualrotations,giventhiswastheir first yearbeingavailableasarotationoption,orperhapsvirtual rotationswerejustsimplynotseenasappealing,thus demandinglessdiscussiontimeonAOCs.Furtheranalysisof subsequentyearswouldbeneededtofullyanalyzethe effectivenessofvirtualoptionsforstudentapplicants.
LIMITATIONS
Potentiallimitationsrestlargelyondata fidelity.2,3,5 Prior studieshavealsoshownthatAOCcommentersmaynot necessarilyreflecttheentireapplicantpopulation.6 Thereis limited-to-nodemographicinformationprovidedonthe analyzedAOC.Additionally,the183narrativesanalyzed fromthespreadsheetarerelativelylowcomparedtothe
numberofapplicantsrankingEMastheirpreferredspecialty orthe273EMprogramsinexistenceatthetimeof2021 NRMPMatch.21 Thismayhaveputouranalysisatriskof notreachingthematicsaturation.Nevertheless,basedonour collectiveexperiencesasEMresidencyapplicantsandasEM applicationadvisers,wedidnot findanyidentifiedsubthemesparticularlysurprisingorcontroversial.Although onlyasingleAOCwasanalyzedinthisstudy,webelieveitto befairlyrepresentativeofcommonlyrecurringstudent opinionsandobservations.Forthepurposeofthisstudy,we specificallyselectedtwosub-pageswiththehighestdensityof meaningfulcommentaryforanalysis;thereisthepotential thatcommentsfromotherpagesmayrevealfurtherthemes orsub-themes.
CONCLUSION
Ourqualitativeanalysisofasingleanonymousonline communityrevealedsixmajorthemesdiscussedamong studentswithregardtoEMresidencyprograms: workingconditions;interpersonalrelationships;learning experience;livingconditions;postgraduatereadiness;and online/virtualsupplement.Mostofthesub-themestothese categoriesrepresentedaspectsofclerkshipsandresidency programsthatarepotentiallymodifiablebythe programleadership.These findingssuggestthatAOC narrativescoverseveraltopicsthatmayserveasuseful externalfeedbackforEMresidencyprogramsorclerkships. Iterativereviewofprogram-speci ficAOCnarratives
Table3. Non-modifiablemajorthemesandsub-themesdeterminedbyauthorconsensusbyaclerkshiporresidencyprogram;percentages areofthetotalnumberofcomments,N = 841.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 195
Estesetal. StudentNarrativesRegardingEMResidency
couldserveasadditionaldataindeterminingwhethera program ’sinternalimprovementeffortsarenoticed bystudents.Additionalstudiesmayhelpcharacterize thelevelofinterestbykeyprogramstakeholdersto considerandmakechangesbasedonfeedbackfrom AOCsources.
AddressforCorrespondence:MollyEstes,MD,LomaLinda University,DepartmentofEmergencyMedicine,11234AndersonSt, A-890,LomaLinda,CA92354.Email: mollykestes@gmail.com
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy.This researchwasfundedinpartbyanEMRABetheChangeGrant awardedin2021.
Copyright:©2024Estesetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.JainSHandMaxsonER.Risksofonlineforumsforpremedicaland medicalstudents. AcadMed. 2011;86(2):152.
2.GusehJS2nd,BrendelRW,BrendelDH.Medicalprofessionalisminthe ageofonlinesocialnetworking. JMedEthics. 2009;35(9):584–6.
3.ThompsonLA,DawsonK,FerdigR,etal.Theintersectionofonline socialnetworkingwithmedicalprofessionalism. JGenInternMed. 2008;23(7):954–7.
4.YapKandTiangYL.Recommendationsforhealthcareeducatorson e-professionalismandstudentbehavioronsocialnetworkingsites. Medicolegalandbioethics. 2014;4:25–36.
5.FacultyofMedicineMDUndergraduateProgram,TheUniversityof BritishColumbia.NAQMythsDebunked-Part1.2014.Availableat: https://mdprogram.med.ubc.ca/2014/12/15/ naq-myths-debunked-part-1/.AccessedJune5,2021.
6.SuraK,WilsonLD,GrillsIS.Comparisonofself-reported dataonStudentDoctorNetworktoobjectivedataofthe NationalResidentMatchingProgram. JAmCollRadiol. 2017;14(12):1594–7.
7.CarleD,ChristensenR,JarouZ,etal.Studentuseandperceived reliabilityofemergencymedicineadvisingsources. WestJEmergMed. 2018;19(4.1):S21–2.
8.CuratoM,HusainA,ShahK,etal.Theirresponsibleuseofsocialmedia amongmedicalstudents. AEMEducTrain. 2019;4(2):168–70.
9.DornerSC,SwansonMB,ZuckerSM.Theresponsibleuseofsocial mediaamongmedicalstudentsandresidents. AEMEducTrain. 2019;4(2):171–3.
10.FabrisC.Anonymousfeedback, fine.Insults?Notontheseplatforms. 2015.Availableat: https://www.chronicle.com/article/ anonymous-feedback-fine-insults-not-on-these-platforms/ AccessedJune5,2021.
11.LeeTandRosenC.Socialmediaandothersourcesofinformationused duringtheemergencymedicineresidencyapplicationprocess. WestJ EmergMed.2017;18(5.1):S27.
12.KozinED,SethiRK,LehmannA,etal.Analysisofanonlinematch discussionboard:improvingtheotolaryngology-headand necksurgerymatch. OtolaryngolHeadNeckSurg. 2015;152(3):458–64.
13.YiPH,NovinS,VanderT,etal.Howdoescurrentgenerationofmedical studentsviewtheradiologyMatch?AnanalysisoftheAuntMinnieand StudentDoctorNetworkonlineforums. AcadRadiol. 2018;25(6):699–707.
14.Official2020-2021EmergencyMedicineApplicantSpreadsheet.2021. Availableat: https://docs.google.com/spreadsheets/d/ 1knDL01qZfYB6HATiyL32gBPmuhsiW-n52a2baISVR8M/ edit#gid=286889120.AccessedJune1,2021.
15.LoveJN,HowellJM,HegartyCB,etal.Factorsthatinfluencemedical studentselectionofanemergencymedicineresidencyprogram: implicationsfortrainingprograms. AcadEmergMed. 2012;19(4):455–60.
16.Hewitt-TaylorJ.Useofconstantcomparativeanalysisinqualitative research. NursStand. 2001;15(42):39–42.
17.TrainorA,JurvisA,MessinaM.Chapter4:FindingYourFit:Learningthe LandscapeofEM.InJarouZ(Ed.) EMRAandCORDStudent AdvisingGuide. Irving,Texas:EmergencyMedicineResidency Association;2019.
18.HammoudMM,StandifordTC,CarmodyJB.The2020-2021residency applicationcycle:lessonslearnedandlingeringproblems. JAMA. 2021;325(22):2249–50.
19.KatsufrakisPJ,UhlerTA,JonesLD.Theresidencyapplicationprocess: pursuingimprovedoutcomesthroughbetterunderstandingofthe issues. AcadMed. 2016;91(11):1483–7.
20.BuckleyRJJ,BrownC,StringerS,etal.Onblast:aframeworkfor monitoringandrespondingtoonlinecommentsaboutyourgraduate medicaleducationprogram. JGradMedEduc. 2020;12(6):651–4.
21.TheMatch.MatchData&ReportArchives,NationalResidentMatching Program.Availableat: https://www.nrmp.org/match-data-analytics/ archives/.AccessedAugust3,2021.
22.CouncilofResidencyDirectorsinEmergencyMedicine.2022.Cord BODStatementonApplicantNumbersforEMResidencyPositions. Availableat: https://www.cordem.org/globalassets/files/ position-statements/cord-bod-response-on-applicant-numbers-for-emresidency-positions.pdf.AccessedOctober14,2022.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 196 StudentNarrativesRegardingEMResidency Estesetal.
RapidCycleDeliberatePracticeTrainingforSimulated CardiopulmonaryResuscitationinResidentEducation
JaronD.Raper,MD*
CharlesA.Khoury,MD*
AndersonMarshall,MD†
RobertSmola,MD†
ZacharyPacheco,MD*
JasonMorris,MD†
GuihuaZhai,PhD‡
StephanieBerger,MD§
RyanKraemer,MD†
AndrewD.Bloom,MD*
*UniversityofAlabamaatBirmingham,DepartmentofEmergencyMedicine, Birmingham,Alabama
† UniversityofAlabamaatBirmingham,DepartmentofInternalMedicine, Birmingham,Alabama
‡ UniversityofAlabamaatBirmingham,CenterforClinicalandTranslationalScience, Birmingham,Alabama
§ UniversityofAlabamaatBirmingham,DepartmentofPediatrics,Birmingham,Alabama
SectionEditors:DavidThompson,MD,andDanMayer,MD
Submissionhistory:SubmittedMarch3,2023;RevisionreceivedNovember21,2023;AcceptedNovember27,2023
ElectronicallypublishedFebruary9,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.17923
Background: Simulation-basedmedicaleducationhasbeenusedinmedicaltrainingfordecades. Rapidcycledeliberatepractice(RCDP)isanovelsimulationstrategythatusesiterativepracticeand feedbacktoachieveskillmastery.Todate,therehasbeenminimalevaluationofRCDPvsstandard immersivesimulation(IS)fortheteachingofcardiopulmonaryresuscitationtograduatemedical education(GME)learners.OurprimaryobjectivewastocomparethetimetoperformanceofAdvanced CardiacLifeSupport(ACLS)actionsbetweentraineeswhocompletedRCDPvsIS.
Methods: Thisstudywasaprospective,randomized,controlledcurriculumevaluation.Atotalof55 postgraduateyear-1internalmedicineandemergencymedicineresidentsparticipatedinthestudy. ResidentswererandomizedtoinstructionbyRCDP(28)orIS(27).Stressandabilitywereself-assessed beforeandaftertrainingusingananonymoussurveythatincorporated five-pointLikert-typequestions. WemeasuredandcomparedtimestoinitiatecriticalACLSactionsbetweenthetwogroupsduringa subsequentIS.
Results: PriorlearnerexperiencebetweenRCDPandISgroupswassimilar.Timestocompletionofthe firstpulsecheck,chestcompressioninitiation,backboardplacement,padplacement,initialrhythm analysis, firstdefibrillation,epinephrineadministration,andantiarrhythmicadministrationweresimilar betweenRCDPandISgroups.However,RCDPgroupstooklesstimetocompletethepulsecheck betweencompressioncycles(6.2vs14.2seconds, P = 0.01).Followingtraining,learnersintheRCDP andISgroupsscoredtheirabilitytoleadandtheirlevelsofanticipatedstresssimilarly(3.43vs3.30, (P = 0.77),2.43vs.2.41, P = 0.98,respectively).However,RCDPgroupsratedtheirabilitytoparticipate inresuscitationmorehighly(4.50vs3.96, P = 0.01).TheRCDPgroupsalsoreportedtheirrealizedstress ofparticipatingintheeventaslowerthanthatoftheISgroups(2.36vs2.85, P = 0.01).
Conclusion: Rapidcycledeliberatepracticelearnersdemonstratedashorterpulsecheckduration, reportedlowerstresslevelsassociatedwiththeirexperience,andratedtheirabilitytoparticipateinACLS caremorehighlythantheirIS-trainedpeers.OurresultssupportfurtherinvestigationofRCDPinother simulationsettings.[WestJEmergMed.2024;25(2)197–204.]
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 197
ORIGINAL RESEARCH
INTRODUCTION
Despiteadvancesinresuscitationscienceandtraining, cardiacarrestremainsthethirdleadingcauseofdeathinthe UnitedStates.1 MillionsofcliniciansreceiveBasicLife Support(BLS)andAdvancedCardiacLifeSupport(ACLS) training,yetpatients’ survivalratesvaryconsiderably.2,3,4 Immediaterecognitionofcardiacarrest,highquality cardiopulmonaryresuscitation(CPR),andtimely defibrillationarethemainstaysofcare.5,6 Effectiveeducation iscrucialtoexecutetheseprinciples,improveteam performance,andenhanceoutcomes.7
Simulation-basedmedicaleducation(SBME)iswell establishedinmedicaltrainingandgraduatemedical education(GME).InSBME,learnersgainexperiencefroma realisticclinicalscenariowithoutthepossibilityofcausing harmtoapatient.8 Learnersaregiventhefreedomtodevelop skillsthroughpracticeandgainvaluablefeedbackvia debriefing.Asaresult,SBMEhasbeenassociatedwith improvedskilldevelopmentandpatientoutcomes.9,10,11
Rapidcycledeliberatepractice(RCDP)isaninnovative simulationstrategythatusesiterativepracticeandfeedback toachieveskillmastery.DevelopedfromAndersEricsson’ s workondeliberativepractice,RCDPallowsforadvanced learningthroughrepetitionandskillrefinement.12 Itwas originallydescribedbyHuntin2014andimplementedin pediatricresuscitationtraining.13 InRCDP,learnersbegina simulatedscenario,butincontrasttotheclassicalpostsimulationdebrief,thecaseisfrequentlypausedbythe instructor.Eachbreakservesasanopportunityforcorrective instruction,coaching,feedback,andsubsequent supervisedrepitition.13
Overthelastdecadetherehasbeenanincreasedfocuson RCDPtraininginresuscitation,withmoststudiesfocusedon pediatrictrainees.13,14 Whencomparedtothestandard immersivesimulation(IS)approach,RCDPhas demonstratedshortertimetoinitialchestcompressionand defibrillationinpediatricmedicinetrainees,improvedchest compressionfractioninadultmedicaltrainees,andbetter skillretention.13,14,15,16 Evenmorerecently,wehaveseen RCDPimplementedintoproceduraltrainingwhereithas alsodemonstratedpositivelearneroutcomes.Groupstrained inRCDPdemonstratedbetterpreparednessforintubation andpost-procedurecareinpediatricairwaymanagement.17 Similarly,RCDP-basedtraininghasbeensuggestedforthe donninganddoffingofpersonalprotectiveequipment,and ourobstetriccolleagueshaveprovenitsutilityforforcepsbaseddeliveries.18,19
InstructionbasedinRCDPhasstrongevidencetosupport itsuseinareasofmedicaleducationthatarealgorithmicin nature,and/orrequireahighdegreeofproceduralskill.The AmericanHeartAssociation(AHA)recognizedthisas recentlyas2020,recommendingthatdeliberatepracticebe incorporatedintoBLSandACLStraining,simultaneously
PopulationHealthResearchCapsule
Whatdowealreadyknowaboutthisissue?
Rapidcycledeliberatepractice(RCDP)isa simulationstrategythatusesiterative practiceandcoachingtoachieveskillmastery andiseffectiveinproceduralinstruction.
Whatwastheresearchquestion?
IsRCDPorimmersivesimulation(IS)more effectiveintrainingresidentstoperform cardiopulmonaryresuscitation(CPR)?
Whatwasthemajor findingofthestudy?
RCDPshortenspulsechecks,andlearners reportedlessstressandgreatercon fi dence performingCPR.
Howdoesthisimprovepopulationhealth?
ResuscitationinstructionbasedinRCDP showspromiseasatooltoenhanceresidents ’ masteryoflifesavingCPRskills.
identifyingitasaneducationalstrategywarrantingfurther research.7 Despitethiscalltoaction,therehasbeenapaucity ofliteratureevaluatingRCDPinACLStrainingforthecare ofadultpatients,regardlessoflearnertype.20 Wesoughtto addressthisknowledgegapthroughtheevaluationofRCDP forACLSasitisappliedtopostgraduateyear(PGY)-1 residentsinGME.Wedidthisthroughacomparisonoftime tocompletionofcriticalACLSactionsbetweenRCDPand ISgroups(ourprimaryobjective).Asasecondaryobjective, wecomparedresidentperceptionsbetweenRCDP-and IS-trainedgroups.
METHODS
StudyDesign
InJuly2022,weconductedaprospective,randomized, controlledstudyapprovedbytheinstitutionalreviewboard.
SettingandParticipants
Thestudywasconductedinanaccreditedsimulation centerthatispartofalargeacademicteachinghospitaland involved43internalmedicine(IM)and12emergency medicine(EM)PGY-1residentswhohadobtainedACLS certificationinthetwoweeksprecedingthisstudy.Noother coachingorinstructionregardingthecareofapulseless patientwasprovidedpriortostudyimplementation.All55 residentsparticipatedvoluntarilyinthestudy.Faculty facilitatorsofallsimulationsessionswereIMandEMfaculty whowereboardcertifiedintheirrespective fields.Each
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 198 RapidCycleDeliberatePracticetoTeachResuscitation Raperetal.
facilitatorunderwentformalIS-andRCDP-facilitator trainingpriortoinvolvementinthestudy.Facilitatorswere notblindedtothestudyobjectives.
Protocol
Intheweekprior,residentswereprovidedwitha descriptionofthestudyandanelectroniccopyofthe informedconsentdocumenttoallowforadetailedand privatereview.Eachofthe55participantsthenprovided writteninformedconsentonthedateoftheirscheduled simulationevent.Ourstudyused fiveteamsforeach instructionalintervention.Eachteamwascomprisedof five orsixmemberswhowererandomlyassignedtoeitherRCDP orIS,foratotalof55participants(28inRCDPgroups,27in ISgroups).Whiletherewasa fixedandlimitednumberof availableparticipants(IMandEMinterns),weperformeda post-hocpoweranalysistoestablishabasisforfuturework. Withan α = 0.05,thisstudyhad29%powertodetectalarge effectsize(d = 1)forprimaryoutcomesand71%powerto detectamediumeffectsize(w = 0.3)forsecondaryoutcomes (G*Power3.1.9.7).Weusedanonlinerandomization generator(https://www.randomizer.org/)todivide participantsinto10teams,with fiveteamsforeach instructionalmethod.
Duetoschedulingdifferences,IMandEMparticipants wereseparatedandcompletedtheirrespectiveexperienceson differentdays.TheIMfacultyfacilitatedallIMresident sessions.Tominimizeconfoundingrelatedtothe effectivenessoftheindividualfacilitator,thetwoIMfaculty facilitatorsledboththeRCDPandtheISsessionsfortheIM residents.The12EMparticipantscompletedtheirexperience thefollowingweekintwoteamsofsix,oneofwhichwas assignedtoRCDPandtheothertoIS.TheEMfaculty facilitatedbothEMresidentsessions.Allfacultyfacilitators weretrainedinimplementationofRCDPandIS.This trainingwasprovidedbycertifiedhealthcaresimulation instructorsinourinternationallyaccreditedinstitutional simulationcenter.Nofacultyfacilitatorswereinvolvedinthe extractionofperformancedata.
Thesametwoembeddedsimulationparticipants(ESP) functionedasnursesforallsessions.TheESPsinallsessions wereregisterednursesandcertifiedhealthcaresimulation educatorsemployedbyourinstitutionalsimulationcenter. TheESPswereinstructedtoassistonlywithcaretaskswhen directlyaskedforspecifictaskassistance(eg,locatingcare items)butdidnottriggerinitiationofindividualtask completionorprovideguidanceontaskperformance.
Followinginformedconsent,learnerswereaskedto completeapre-simulationsurveytoestablishbaseline learnercharacteristics.Thesurveyqueriedeachparticipant’ s priorlevelofexperienceaswellasself-perceivedabilityto leadandparticipateinthecareofapulselesspatient.The surveysalsoassessedthelearner’santicipatedandprior experiencedstressassociatedwithcodeleadershipand
participation.Eachmeasurewasassessedusinga five-point Likert-typescale.
ImmersiveSimulationProtocol
AllISteamswereprovidedwiththesamescriptedprebrief,whichdescribedthebasictenetsofsimulationand informedparticipantsthattheywouldbecaringfora pulselesspatient.Teamswerenotinstructedregardingthe assignmentofclinicalrolesbutwereallowedtoself-assignas theydeemedappropriate.TheISteamswerethenactivated byanESPfunctioningintheroleofanursewhobroughtthe participantstothecareareaandaskedparticipantsto evaluateanunresponsivepatient.
Onceoutsidethepatient’sroom,participantsassumed careforthepatientwithoutfurthercoachingorintervention. TheISparticipantswerepermittedtonavigatethepatient’ s casewithoutinterruption,whilephysicianfacilitators observedtheiractionsfromasimulationcontrolroomwith audioandvisualsurveillanceofthesimulationarea.TheIS learnerswereallowedtonavigatetheircasewithout interruptionuntilthefourthpulsecheckoruntil30minutes hadelapsed,atwhichtimefacilitatorsinitiatedreturnof spontaneouscirculationandthecasewasterminated.Given thenatureoftheISeducationalsessions,learnersdidnot havetheopportunitytorotateroles.Learnersthenreturned tothebriefingroom,andphysicianfacilitatorsdebriefed basedonobservedperformanceaccordingtoastandardized debriefingguideanduntiltotalcasetimereached45minutes. Theguideemphasizedcoachingregardingresuscitationand time-sensitiveinterventionsthatmatchedtheprimary outcomemeasures(eg,timetoidentificationofpulselessness, timetoinitiationofchestcompressions,etc).
RCDPSimulationProtocol
AllRCDPgroupsweregivenastandardpre-briefthat describedthebasictenetsofsimulation.Groupswerethen providedwithanintroductiontothesimulationmodality assignedtothem.Teamswerenotinstructedregardingthe assignmentofclinicalrolesbutwereallowedtoself-assignas theydeemedappropriate.TheRCDPteamswereactivated byanESPwhobroughttheparticipantstothecareareaand askedthemtoevaluateanunresponsivepatientwhile physicianfacilitatorsobservedatthebedside.TheRCDP groupsrotatedroles,allowingthemtheopportunitytodirect theresuscitationandreceivefeedback.
Inadditiontotheirstandardizedtraining,allfacilitators wereprovidedwithanRCDPcoachingguide,whichwas focusedonthesameresuscitationandtime-sensitive interventionsastheimmersivecasedebriefingguide. Facilitatorsprovidedreal-timecoachingandfeedbackbased ontheRCDPcoachingguide.Caseswerethenrestarted, rewound,orresumedaccordingtofacilitatordiscretion. Totallearnersimulationanddebriefingtimewas45minutes foreachRCDPcase.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 199 Raperetal. RapidCycleDeliberatePracticetoTeachResuscitation
ProtocolOverlap
Uponcompletionofdebriefingandclosureoftheir respectivecases,allparticipantsreturnedtothebriefing space.MaintainingseparationofinitialRCDPvsISgroups, asubsequentISsessionwascompletedbyallparticipants duringwhichaudioandvisualrecordingswereobtained.
DataabstractionoftimestocompletionofcriticalACLS actionswasobtainedfromthissession.Participantswere activatedasecondtimebytheESPtocareforanadditional, unresponsivepatient.Learnerswereallowedtoroleassign andcompletethecasewithoutinterventionfromtheESP orfacilitator.
PrimaryOutcomeMeasures
Timetocompletionofcriticalactionswasusedasa surrogateforproficiencyintheperformanceofanACLSbasedresuscitation.Thesecriticalactionsweredefinedby researchteamconsensusafterreviewingACLSprotocols. Timezerowasdeterminedbaseduponlearnerentryintothe carearea,andtimestocompletionofresuscitativetimebasedinterventionswereextractedthroughvideoreviewby theprimaryinvestigator.Tomitigatebiasfromfaculty workingwiththeirownresidents,dataabstractionfrom videorecordingswasperformedbytheprimaryinvestigator, whowasnotinvolvedinsimulationsessionfacilitation.The primaryinvestigatorwasblindedtoRCDPvsISgroup assignmentatthetimeofdataabstraction.Timesfromroom entryto firstpulsecheck, firstchestcompression,backboard placement,defibrillatorpadattachment,initialrhythm analysis,initialdefibrillation,initialepinephrine administration,andantiarrhythmicadministrationwere recorded.Thedurationofpausebetweencompressioncycles wasalsoobtainedforeachsession.
SecondaryOutcomeMeasures
Learnerswerequeriedusingpre-andpost-experience surveys,whichweredistributedinpaperformatimmediately beforeandafterthesimulationsessions.Wedevelopedthe surveysbasedonKirkpatrick’stheoryofeducational trainingandevaluation,focusingprimarilyonlevel1and2 analyses.21 Allsurveyitemsuseda1–5Likert-typescaleto quantifyallqualitativequestions,andsurveyresponserates forallsurveyswere100%.Priortotheeducational intervention,learnerswereaskedtoratetheirself-perceived abilitytoparticipateinandabilitytoleadacode(1notatall capable,to5extremelycapable).Theywerealsoaskedto ratetheiranticipatedstressassociatedwithparticipationand leadershipofacode(1notatallstressful,to5extremely stressful).Finally,theywerequeriedregardingthenumberof simulatedcodestheyhadparticipatedinorled,aswellasthe numberofactualcodestheyhadparticipatedinorled.
Followingtheeducationintervention,learnerswereasked toagainratetheirself-perceivedabilitytoparticipateinand leadacode.Theywerealsoaskedtoratetheoverall
effectivenessoftheirexperience(1notatalleffective, 5extremelyeffective).Finally,learnerswereaskedtoratethe stressleveltheyperceivedtobeassociatedwithparticipating andleadingtheirsimulatedexperience(1notatallstressful, to5extremelystressful).
StatisticalAnalysis
First,wecomparedpriorsimulatedandgenuineCPR experiencesasleaderandasparticipantforRCDPandIS groups,usingtheCochran-Mantel-Haenszeltest,giventhe ordinalnatureoftheLikert-typescale.Wedefinedsimulated experiencesasthoseinvolvingCPRtrainingthatdidnot involvethecareofapatient.Genuineexperienceswere definedasthoseinvolvingtheCPR-basedresuscitationofa codingpatient.Wethencomparedthetime-based differencesbetweenRCDPandISgroupsusingaStudent t -testoraWilcoxontestwhentherewassubstantialdeviation fromnormality.Oursamplesizeforallprimaryoutcome measureswas10teams.Wecomparedmeantimedifferences betweenthetwogroupsfor firstpulsecheck, firstchest compression,pauseduration,backboardplacement, defibrillatorpadplacement, firstrhythmanalysis, first defibrillation, firstepinephrineadministration,and amiodaroneadministration.
Oursamplesizeforallsecondaryoutcomemeasureswas 55individuals.Wealsocomparedpre-andpost-training surveydatabetweenthetwogroupsusingtheCochranMantel-HaenszeltestgiventheordinalnatureoftheLikerttypescale.Thelearner’sexperienceascodeleaderand participantandoveralleffectivenessofexperiencewerealso includedinthepost-trainingsurvey.Abilitytolead,abilityto participate,anticipatedstressleading,andanticipatedstress participatingwereincludedinbothsurveys.Finally,we comparedstressleadingandstressparticipatinginpre-and post-trainingforbothgroupsusingageneralizedStuartMaxwelltesttoevaluatetheimprovementaftertraining.22 Weusedanalphalevelof0.05forallstatisticaltests.A Benjamini-Hochbergfalsediscoveryrateadjustmentwas appliedformultiplecomparisons.Allprogramswerewritten inSAS9.4.(SASInstituteInc,Cary,NC).
RESULTS
PriorLearnerExperience
Priorlearnerexperiencewassimilarbetweenthegroups anddidnotappeartobeasignificantconfounder(Table1). Thenumbersofexperiencesarereportedasmedians withminimumandmaximumvaluesduetolackof normaldistribution.
PrimaryOutcome:Time-basedDifferences
Althoughthereweretrendstowardshortermeantimesto completionofcriticalactionsforRCDPvsISgroups,we observedonlyonecategorywithastatisticallysignificant
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 200 RapidCycleDeliberatePracticetoTeachResuscitation Raperetal.
GroupMedianMinMax P-value
SimulationIS10170.34
leaderRCDP206
GenuineIS00400.81
leaderRCDP003
SimulationIS30500.46
participantRCDP3.5015
GenuineIS20750.67
participantRCDP2025
IS,immersivesimulation; RCDP,rapidcycledeliberatepractice. *Genuinereferstoexperiencesinactualpatientcarescenarios.
difference:CPRmeanpausedurationinsecondswas6.20vs 14.20seconds(P = 0.01)inRCDPvsISgroups(Table2).
SecondaryOutcome:AbilityandStress
Forstresslevelsandself-reportedability,learners providedratingsona five-pointLikert-typescale.Wepresent themeanvaluesin Tables3 and 4.Priortotraining,RCDP andISlearnersratedtheiranticipatedstressofleadingand participatinginCPRsimilarly(4.36vs4.00(P = 0.44);3.18 vs3.00(P = 0.08),respectively).TheRCDPandISlearners alsoratedtheirpre-trainingabilitytoleadaswellas participateintheeventsimilarly(2.50vs2.37(P = 0.75);3.61 vs3.52(P = 0.59)(Table3).Therewasnosignificant differenceintheanticipatedstresslevelsoffutureevents followingtraining,whetherconsideringtheroleofleader (P = 0.93)orparticipant(P = 0.98)(Table4).Similarly,there wasnosignificantdifferenceinexperiencedstressasaleader betweenRCDPandISlearners(P = 0.93)andtheoverall effectivenessoftheexperiencewasratedsimilarlybetween groups(P = 0.09).However,RCDPlearnersreportedlower levelsofexperiencedstressasaparticipant(P = 0.01)
Table3. Pre-simulationmeanLikert-typeratings.
GroupMedianMinMax P-value
AbilitytoleadIS2140.75 RCDP313
AbilitytoparticipateIS3350.59 RCDP425
Stressanticipatedas leader IS4350.44 RCDP435
Stressanticipatedas participant IS4240.08 RCDP325
IS,immersivesimulation; RCDP,rapidcycledeliberatepractice.
(Tables3, 4).Whenwecomparedpre-andpost-training responsesregardinganticipatedstress,theanticipatedstress offutureresuscitationexperiencesdroppedsignificantlyfor bothleaderandparticipantcategoriesfollowingtraining, regardlessofinstructionalmethod.
DISCUSSION
LearnersreceivingRCDPinstructionshoweda significantlyshortenedpauseduration,reducedstress,and improvedself-perceivedCPRskillscomparedtoIS.The RCDPinstructionalsoshortenedvarioustime-basedACLS metrics,althoughstatisticalsignificancewasnotreacheddue tothesmallsamplesize.Areducedpausedurationcarries notableclinicalsignificance.Pausedurationisanimportant metricofhigh-qualityCPRandisassociatedwithimproved patientoutcomes.23,24 Reducedpausedurationhasa significantimpactonterminatingarrhythmiasandincreasing returnofspontaneouscirculation,whileincreasedpause durationisassociatedwithadecreaseinsurvival.26
AlthoughthereisapaucityofliteraturecomparingRCDP toISinthecareofanadultpatient,whatlittledatathatdoes existdemonstratesimprovementsinchestcompression fractioninRCDPvsISgroups.20 Manyofthesepriorstudies
±2.95)0.27
Backboardplacement40.40(±31.33)193.40(±183.36)0.25
Padplacement66.40(±12.56)74.80(±20.75)0.46
Firstrhythmanalysis73.60(±13.50)111.20(±37.63)0.25
Firstdefibrillation93.00(±17.46)150.60(±63.49)0.25
Firstepinephrine131.60(±28.75)158.20(±55.21)0.41
Pauseduration6.20(±2.07)14.20(±6.53)0.01
Antiarryhthmic376.60(±94.25)438.80(±99.19)0.41
IS,immersivesimulation; RCDP,rapidcycledeliberatepractice.Timeisinseconds.
Table1. Cardiopulmonaryresuscitationexperienceprior tosimulation.
RCDPmeantime(±SD)ISmeantime(±SD) P-value Firstpulsecheck4.00(±1.00)5.60(±1.52)0.25 Firstchestcompression12.40(±3.13)15.20(
Table2. Rapidcycledeliberatepracticevsimmersivesimulationtimeinseconds.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 201
Raperetal. RapidCycleDeliberatePracticetoTeachResuscitation
GroupMedianMinMax P-value
AbilitytoleadIS3350.77
RCDP334 AbilitytoparticipateIS4350.01
RCDP4.545 Stressanticipated
RCDP324 Stressanticipated
Stressexperienced
Stressexperienced
RCDP214
RCDP325
RCDP213 OveralleffectivenessIS4350.09
RCDP545
IS,immersivesimulation; RCDP
weredoneinpediatrics,buttheresultsshouldhaveclinically similarinterpretationsasthosecompletedinadults.13–16 HuntetalconductedtheonlypriorstudyexaminingtimebasedmetricsasasurrogateforproficiencyandfoundRCDP tobesuperiorforinstructionofBLSinterventionsinjunior medicalstudents.27 Ourresultsaddtothisworkthroughthe furtherexaminationoftime-basedmetricsandlearner perceptions.Althoughlimited,theseresultslendfurther credibilitytotheargumentthatRCDPmaybesuperiortoIS forACLStraining.
WhileRCDP-trainedlearnersinourstudyexhibited trendstowardotherfavorableACLSmetrics,therewereno otherstatisticallysignificantdifferences.Priorworkhas demonstratedimprovementintimetodefibrillation,initial chestcompression,andbackboardplacementwithRCDP traininginpediatricresuscitations.13,15,20 Ourworkdoesnot independentlysupportthese findings;however,ourtrends areinlinewithexistingliterature.
Timeto firstdefibrillationsuggestedfavorabilityinthe RCDPgroup(93vs150seconds[sec]),althoughdifferences didnotreachstatisticalsignificance.Thisdistinctionis important,however,astheRCDPgroupwasableperform thisactionwithintheAHA’ s “GetwiththeGuidelines” recommendationof firstdefibrillationinlessthantwo minutes.Similarly,timeto firstepinephrineadministrationin RCDPvsIS(131vs158sec),suggestsreducedtimeinthe RCDPgroupwithoutreachingstatisticalsignificance.Both groupsperformedwithinthe five-minutemetricoutlinefrom “GetwiththeGuidelines” recommendations.Asboth groupsperformedwellwiththisaction,obtainingstatistical significancemayprovedifficult.Itisunclearwhyother
metricssuchaspadplacementoradministration antiarrhythmicshowednosignificantchangebetween groups.Theseactionsaredependentonavarietyoffactorsin ateamfocusedonCPR,andasLemkeetalsuggest,theymay bedifficulttomeasureeffectively.15
Aspreviouslynoted,ourstudywasunderpowered,which playedaroleintheabsenceofstatisticallysignificant differencesformanyofouroutcomemeasures.TheLikertscalemeasureswerebetterpowered,astheyrepresented55 individualsurveyresponsesasopposedtothe10totalteams dividedintwoforeachinstructionalmethod.For comparison,Huntetalstudiedtheperformanceof81 individualpediatricresidentswhoparticipatedinthepostinterventionassessmentandfoundthatRCDPimproved learnerconfidence,buttherewasnocontrolgroupfor comparisonorpowercalculation.13 DeCastroetalused five teamsfortheirRCDPgroupandfourteamsfortheircontrol group,withan80%powertodetecta20%differenceinthe primaryoutcome.Theauthorsfoundahigherchest compressionfractionandshortertimestorhythm identification/defibrillationintheRCDPgroup.However, duetodatalosstheywereunabletoachievetheplanned power. 20 Lemkeetalstudiedthegreatestnumberoflearners, with102participantsin21teamsfortheircontrolcohort,and 108participantsin20teamsfortheirRCDPcohortand foundthatRCDPgroupsdemonstratedshortertimesto defibrillation.Whilenoformalpowercalculationwas performed,Lemke’sworkappearstobethebestpowered thusfar.15 Futureworkshouldincludemorerobustpowering withlargersamplesizes,whichwilllikelyrequireinterinstitutionalcollaboration.
Anotherfactorcontributingtoourinabilitytodetect significantdifferencesinmanytime-basedmetricsmaybethe learnerlevelstudied.Byitsverynature,RCDPservesasa methodtodevelopperfectpractice.Providingthelearner withreal-timefeedbackandcoachingbuildsmicro-skill developmentandmastery,asopposedtoproficiencyalone. ThisstudyfocusedonPGY-1residentsfortworeasons.First, inanefforttoavoidconfoundingbyvariationsintraining, westudiedPGY-1levellearnersintheir firstmonthof residency.Second,weexcludedadvancedlearnersdueto concernsthattheirinvolvementwouldconfoundthe studyofthejuniorlearnerthroughadvancingthe performanceoftheentiregroup.Conversely,priorworkthat founddifferencesinsimilarcategoriesevaluatedlearners fromPGYlevels1–3orstudiedlargerlearnergroups.14,15,27 Therefore,trueskillmasterymaybemoreattainable throughtheinclusionofmoreadvancedlearners andmaycontributetomorestatisticallysignificant results.13,15 Conversely,theinclusionofmoreadvanced learnersmayinfluencetheentiregroup,leadingtoa moreuniformperformance.Thismaylimitor reduceobservabledifferencesbetween instructionalmethods.
Table4. Post-simulationmeanLikert-typeratings.
asleader IS3250.93
asparticipant IS2130.98
asleader IS3350.93
asparticipant IS3240.01
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 202 RapidCycleDeliberatePracticetoTeachResuscitation Raperetal.
,rapidcycledeliberatepractice.
HuntetalalsonotesadoseresponsewithRCDP(ie, increasingexperienceandrepetitionfostersimproved performanceandskillmastery).13 Westudiedthelearners’ firstperformance,butwedidnotconductadditional simulatedexperiencesbeyondthis.Furtherrepetitionmay haveexpandeddifferencesinRCDPandISgroups.
Acommongoalofsimulationinmedicaleducationisto reducethestressandanxietyexperiencedbythelearner,and thisisespeciallytrueforhigh-stakesscenariossuchasthe careofapulselesspatient.However,thesimulation experiencecanbeindependentlystressfulforlearners,and priorworkhassuggestedthatRCDP-basedinstructionmay provideanoverallpreferredexperience.Thisiswell illustratedbytheworkofChanceyetal,whoselearners expressedapreferenceforthefrequentinterruptionsand improvedsenseofemotionalsecurityassociatedwithRCDP instruction.25 Chancey’slearnersalsoreportedincreased confidenceintheirownresuscitationskills.Ourresults supportthese findings,demonstratinganincreased confidenceinabilitytoparticipateintheRCDPgroups. Similarly,ourlearnersreportedlowerstresslevels experiencedduringtheirRCDP-basedsimulation.
LIMITATIONS
Duetothestudy’snature,blindingparticipantsand facilitatorswasnotpossible.Skillretentionwasnotassessed, andthesmallsamplesizelimitsgeneralizability. Additionally,whileallfacilitatorshadundergone standardizedtraininginbothinstructionalmethods, individualfacilitatorsmayhavebeenmoreeffectiveatone strategyvstheother.Allparticipatingresidentscompleteda standardACLScourseinthetwoweeksprecedingthestudy. Also,mostoftheresidentshadsignificantexperienceaspart ofresuscitationteams(Table1).Asaresult,theremayhave beenlessofadifferenceinperformancebetweenthetwo groups.OurstudyfoundRCDPwaswellreceivedbyour learners,butthedataislimitedbylearnerevaluationat KirkpatricklevelsIandII.Whilewebelievelearner perceptionsininstructionareimportantforengagement, futureinvestigationsshouldfocusonobjective impactsandclinicalperformancewith patient-orientedoutcomes.
Surveyswerenotbasedonanypriorsurveyinstrumentbut werecreated,reviewed,andeditedbytheresearchteam.The surveyswerenovelinstruments,andwedidnotobtain validityevidencepriortotheiruse.Recallbiaswas minimizedthroughtheimplementationofsurveys immediatelyfollowinginstructionandperformanceofthe learners.Wewereunabletoeliminatetheeffectsofsocial desirabilitybiasforourlearnersandsuspectthatlearners wouldtendtoreportimprovedperformanceregardlessof instructionalmethod.However,thepotentialforthisbias existedinbothRCDPandISgroups.Samplingandnonresponsebiaswerenotfactorssecondarytoour100%
responserate,butduetothenatureofour five-pointLikerttypequestionscale,thepotentialforneutralbiasexists.
Duetothefrequentinterruptionsassociatedwiththe RCDPmethod,RCDPparticipantswereabletorotate througheachroleontheresuscitationteam.However,IS groupsdidnothaveanopportunitytochangerolesasapart oftheirtraining,andthisintroducesaconfounderin comparingthelearnerexperienceaswellasproficiency betweentheseinstructionalmethods.
Finally,thisstudyfocusedontimetocompletionof criticalactionsbutdidnotassessthequalityofthoseactions, includingfactorssuchaschestcompressionfraction(CCF). However,CCFhasbeenpreviouslystudiedandfoundtobe superioringroupsundergoingRCDP-basedinstructionas comparedtostandardIS.13,20,27
CONCLUSION
Rapidcycledeliberatepracticewasfavoredbylearnersfor ACLS-basedCPRinstruction,improvingself-perceived skillsandreducingpauseduration.ThissuggestsRCDPisa validstrategytoteachresidentsACLS-basedCPRand supportsfurtherinvestigationofRCDPinothersettings.
AddressforCorrespondence:JaronD.Raper,MD,Universityof AlabamaatBirmingham,DepartmentofEmergencyMedicine,214 GeneralServicesBuilding,52119thStreetSouth,Birmingham,AL 35233.Email: jdraper@uabmc.edu
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.
Copyright:©2024Raperetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.TaniguchiD,BaernsteinA,NicholG.Cardiacarrest:apublichealth perspective. EmergMedClinNorthAm. 2012;30(1):1–12.
2.CouteRA,NathansonBH,MaderTJ,etal.Trendanalysisofdisabilityadjustedlifeyearsfollowingadultout-of-hospitalcardiacarrestinthe UnitedStates:astudyfromtheCARESSurveillanceGroup. Resuscitation. 2021;163:124–29.
3.PerkinsGDandCookeMW.Variabilityincardiacarrestsurvival:the NHSAmbulanceServiceQualityIndicators. EmergMedJ. 2012;29(1):3–5.
4.ChengA,NadkarniVM,ManciniMB,etal.AmericanHeartAssociation EducationScienceInvestigators;onbehalfoftheAmericanHeart AssociationEducationScienceandProgramsCommittee,Councilon Cardiopulmonary,CriticalCare,PerioperativeandResuscitation;
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 203 Raperetal. RapidCycleDeliberatePracticetoTeachResuscitation
CouncilonCardiovascularandStrokeNursing;andCouncilonQuality ofCareandOutcomesResearch.Resuscitationeducationscience: educationalstrategiestoimproveoutcomesfromcardiacarrest:a scientificstatementfromtheAmericanHeartAssociation. Circulation. 2018;138(6):e82–122.
5.CommitteeontheTreatmentofCardiacArrest:CurrentStatusand FutureDirections;BoardonHealthSciencesPolicy;Instituteof Medicine.(2015).In:GrahamR,McCoyMA,SchultzAM(Eds.), StrategiestoImproveCardiacArrestSurvival:ATimetoAct (firstpage ofchapter-lastpage).Washington,DC:NationalAcademiesPress(US).
6.CumminsRO,ThiesW,ParaskosJ,etal.Encouragingearly defibrillation:theAmericanHeartAssociationandautomatedexternal defibrillators. AnnEmergMed. 1990;19(11):1245–8.
7.ChengA,MagidDJ,AuerbachM,etal.Part6:resuscitationeducation science:2020AmericanHeartAssociationguidelinesfor cardiopulmonaryresuscitationandemergencycardiovascularcare. Circulation. 2020;142(16_suppl_2):S551–79.
8.EdmondsonA.Psychologicalsafetyandlearningbehaviorinwork teams. AdmSciQ. 1999;44(2):350–83.
9.CookDA,BrydgesR,HamstraSJ,etal.Comparativeeffectivenessof technology-enhancedsimulationversusotherinstructionalmethods.A systematicreviewandmeta-analysis. SimulHealthc. 2012;7(5):308–20.
10.McGaghieWC,IssenbergSB,PetrusaER,etal.Effectofpracticeon standardizedlearningoutcomesinsimulation-basedmedicaleducation. MedEduc. 2006;40(8):792–7.
11.WayneDB,ButterJ,SiddallVJ,etal.Simulation-basedtrainingof internalmedicineresidentsinadvancedcardiaclifesupportprotocols:a randomizedtrial. TeachLearnMed. 2005;17(3):210–6.
12.EricssonKA.Deliberatepracticeandtheacquisitionandmaintenanceof expertperformanceinmedicineandrelateddomains. AcadMed. 2004;79(10Suppl):S70–81.
13.HuntEA,Duval-ArnouldJM,Nelson-McMillanKL,etal.Pediatric residentresuscitationskillsimproveafter “rapidcycledeliberate practice” training. Resuscitation. 2014;85(7):945–51.
14.LemkeDS,FielderEK,HsuDC,etal.Improvedteamperformance duringpediatricresuscitationsafterrapidcycledeliberatepractice comparedwithtraditionaldebriefing:apilotstudy. PediatrEmergCare. 2019;35(7):480–6.
15.LemkeDS,YoungAL,WonSK,etal.Rapid-cycledeliberatepractice improvestimetodefibrillationandreducesworkload:arandomized
controlledtrialofsimulation-basededucation. AEMEducTrain. 2021;5(4):e10702.
16.WonSK,DoughtyCB,YoungAL,etal.Rapidcycledeliberatepractice improvesretentionofpediatricresuscitationskillscomparedwith postsimulationdebriefing. SimulHealthc. 2022;17(1):e20–7.
17.GrossTI,AbrahamDG,KumarA,etal.Rapidcycledeliberatepractice (RCDP)asamethodtoimproveairwaymanagementskills – a randomizedcontrolledsimulationstudy. Cureus. 2019;11(9):e5546.
18.OliveiraHC,SouzaLC,LeiteTC,etal.Personalprotectiveequipmentin thecoronaviruspandemic:trainingwithrapidcycledeliberatepractice. RevBrasEnferm. 2020;73Suppl2(Suppl2):e20200303.
19.PascoalE,MuellerV,MoraisM.Rapidcycledeliberatepracticeversus traditionalsimulationtoimproveresidentperformanceinforceps deliveries. JObstetGynaecolCan. 2022;44(8):934–7.
20.deCastroLT,CoriolanoAM,BurckartK,etal.Rapid-cycledeliberate practiceversusafter-eventdebriefingclinicalsimulationin cardiopulmonaryresuscitation:aclusterrandomizedtrial. AdvSimul (Lond). 2022;7(1):43.
21.KirkpatrickJDandKirkpatrickWK.Associationfortalentdevelopment. In: Kirkpatrick’sFourLevelsofTrainingEvaluation(73-90).Alexandria, VA:ATDPress;2016.
22.StuartA.Atestforhomogeneityofthemarginaldistributionsina two-wayclassification. Biometrika. 1955;42(3–4):412–6.
23.CrowleyCP,SalciccioliJD,KimEY.TheassociationbetweenACLS guidelinedeviationsandoutcomesfromin-hospitalcardiacarrest. Resuscitation. 2020;153:65–70.
24.BhanjiF,DonoghueAJ,WolffMS,etal.Part14:Education:2015 AmericanHeartAssociationEducation:2015American HeartAssociationguidelinesupdateforcardiopulmonary resuscitationandemergencycardiovascularcare. Circulation. 2015;132(suppl2):S561–73.
25.ChanceyR,SampayoE,LemkeD,etal.Learners’ experiencesduring rapidcycledeliberatepracticesimulations:aqualitativeanalysis. Simul Healthc. 2019;14(1):18–28.
26.CheskesS,SchmickerRH,ChristensonJ,etal.Perishockpause:an independentpredictorofsurvivalfromout-of-hospitalshockablecardiac arrest. Circulation. 2011;124(1):58–66.
27.HuntEA,Duval-ArnouldJM,ChimeNO,etal.Integrationofin-hospital cardiacarrestcontextualcurriculumintoabasiclifesupportcourse:a randomized,controlledsimulationstudy. Resuscitation. 2017;114:127–32.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 204 RapidCycleDeliberatePracticetoTeachResuscitation Raperetal.
SimulationImprovesEmergencyMedicineResidents’ Clinical PerformanceofAortaPoint-of-CareUltrasound
BrandonM.Wubben,MD CoryWittrock,MD
SectionEditor:ShaneSummers,MD
UniversityofIowaHospitalsandClinics,DepartmentofEmergencyMedicine, IowaCity,Iowa
Submissionhistory:SubmittedAugust30,2023;RevisionreceivedDecember4,2023;AcceptedDecember5,2023 ElectronicallypublishedFebruary9,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18449
Purpose: Usingpoint-of-careultrasound(POCUS)todiagnoseabdominalaorticaneurysm(AAA)isan essentialskillinemergencymedicine(EM).Whilesimulation-basedPOCUSeducationiscommonly used,thetranslationtoperformanceintheemergencydepartment(ED)isunknown.Weinvestigated whetheraddingcase-basedsimulationtoanEMresidencycurriculumwasassociatedwithchangesin thequantityandqualityofaortaPOCUSperformedbyresidentsintheED.
Methods: Acase-basedsimulationwasintroducedtoresidentdidacticsatouracademic,LevelItrauma center.Acaseofundifferentiatedabdominalpainwaspresented,whichrequiredexaminationofan ultrasoundphantomtodiagnoseanAAA,withahands-ondidactic.Wecomparedthequantity,quality, anddescriptiveanalysesofaortaPOCUSperformedintheEDduringthefourmonthsbeforeandafter thesimulation.
Results: Forparticipatingresidents(17/32),therewasan86%increaseintotalstudiesandan80% increaseinclinicalstudies.Onanopportunity-adjusted,per-residentbasis,therewasnosignificant differenceinmediantotalscansper100shifts(4.4[interquartilerange(IQR)0–15.8vs8.3[IQR]
3.3–23.6, P = 0.21)oraveragetotalqualityscores(3.2 ± 0.6vs3.2 ± 0.5, P = 0.92).Thetotalnumberof limitedorinadequatestudiesdecreased(43%vs19%, P = 0.02),andtheproportionofscanssubmitted byinternsincreased(7%vs54%, P =< .001).
Conclusion: Aftersimulationtraining,aortaPOCUSwasperformedmorefrequently,andEDinterns contributedahigherproportionofscans.Whiletherewasnoimprovementinquantityorqualityscoreson aper-residentbasis,thereweresignificantlyfewerincompleteorlimitedscans.[WestJEmergMed. 2024;25(2)205–208.]
Keywords: point-of-care;ultrasonography;simulation;abdominalaorticaneurysm;POCUS; emergencymedicine.
INTRODUCTION
Point-of-careultrasound(POCUS)oftheaortato diagnoseabdominalaorticaneurysm(AAA)isacore emergencyultrasoundapplicationandanessential componentofemergencymedicine(EM)residency education.1 Emergencydepartment(ED)POCUShas previouslybeenshowntohaveexcellentperformance characteristicsfortheevaluationofAAA.2 Simulationisa commonlyusededucationaltoolforresidentultrasound
education,butascopingreviewfoundthemajorityofprior studiesexaminedchangesinconfidence,knowledge,and skillsratherthanobjectiveclinicalperformance.3 However, simulationinadditiontoclinicaltraininghasbeenshownto beeffectiveattranslatingtoclinicalperformanceinother specialties,suchasforobstetricsandgynecologyresidents learningtransvaginalultrasound.4
Itiscriticalthatemergencyphysicianslearntoquickly identifyAAAatthebedside,asthisisatime-sensitiveand
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 205
ORIGINAL RESEARCH
potentiallydeadlydiagnosisrequiringagoaltimefrom presentationtoemergencysurgeryof <90minutes.5 We aimedtoinvestigatewhethertheadditionofcase-based ultrasoundsimulationtotheexistingEMresidency curriculumwasassociatedwithanincreaseinthequantity andqualityofaortaPOCUSperformedonEDpatients.
MATERIALSANDMETHODS
Design,Setting,andIntervention
Atouracademic,LevelItraumacenterwithathree-year EMresidencyprogramandadvancedpracticeprovider trainingprogram,EMresidentshaveexistingsimulationbasedlearningbuiltintotheirmonthlyacademicconference. However,theuseofPOCUSisnottypicallyincorporated intosimulationatourinstitution.InSpring2023,we introducedanewsimulatedcaserequiringtheuseofPOCUS fordiagnosisofAAAinthesettingofundifferentiated abdominalpain.Duringsimulation-basedlearning,residents aredividedintogroupsof5–8residentsofvaried postgraduateyear(PGY)andcyclebetweenthesimulation sessionandothereducationalactivities.Inadditiontoverbal promptsregardingcasehistory,physicalexam findings,and patientresponsestointerventions,residentswereaskedto useacart-basedultrasoundsystem(SonositePX,Fujifilm, Bothell,WA)toexamineanultrasoundphantomabdomen (41903–000,KyotoKagaku,Japan)withmultiple findings includinganinfrarenalAAAwithintermuralthrombus,free abdominal fluid,andnormalbowelsandrenalsystem.The caseconcludedwithahands-ondidacticledbyaPOCUS fellowship-trainedemergencyphysicianandincludedtimeat theendofthesimulationforhands-onscanningbyresidents.
UltrasoundStudyReviewandOutcomes
AllultrasoundstudiesperformedintheEDaresubmitted throughaqualityassuranceworkflowforreviewbyateamof POCUSfellowship-trainedfaculty.Allresidentswho participatereceivecreditforperformingthePOCUS.The submittingresidentwhoperformedthePOCUScompletesa worksheetdescribingthe findings,interpretation,andstudy limitations.Studiesaremarkedcompleteifresidentsindicate thataviewwasobtainedofthesuprarenalaorta,infrarenal aorta,andiliacbifurcationandincompleteifoneormoreof theseviewswasnotobtained.Imagesandworksheetsare thensentforreviewandsignaturetothefacultycaringforthe patientwiththeresident.Facultycaneitherplacethestudyin aneducationalarchive(iftheydidnotusethestudyfor medicaldecision-making)oraclinicalarchive(iftheyused thestudyformedicaldecision-makingandwishforthe imagestotransfertothepatient’shealthrecord).
Studiesarereviewedforqualityandassignedaquality scorefrom1(worst)to5(best)aswellasnotationofanyfalse positivesorfalsenegatives,withEMultrasoundfaculty servingasthegoldstandard.Qualityscores1and2are consideredinsufficientfordiagnosis,withscores
PopulationHealthResearchCapsule
Whatdowealreadyknowaboutthisissue? Simulationhasincreasinglybeenusedto prepareEMresidentsforlesscommon conditions,suchasdiagnosingabdominal aorticaneurysmusingPOCUS.
Whatwastheresearchquestion?
Doescase-basedPOCUSsimulationaffect thequantityorqualityofaortaPOCUSthat residentsperformintheED?
Whatwasthemajor findingofthestudy?
AortaPOCUSincreased86%,andthe numberoflimitedorinadequatestudies decreased(43%vs19%,P = 0.02).
Howdoesthisimprovepopulationhealth?
AortaPOCUSsimulationtrainingmayhelp physicianswholessfrequentlyencounter aorticaneurysmtoidentifythis time-sensitivecondition.
of3–5consideredadequate.Weexaminedcumulative measuresofsensitivityandspecificitybeforeand afterthesimulation.
Analysis
WecomparedthemediannumberofaortaPOCUS studiesthatEMresidentsperformedintheEDinthefour monthspriortothesimulationsessiontothemediannumber ofexamsperformedinthefourmonthsfollowingthe simulationsession.Thesequantitieswerereportedasscans performedper100shiftsperresidentandcomparedusing Wilcoxonsigned-ranktest.Wecomparedmeanquality scoresoverthefourmonthspriortotheinterventiontothe qualityscoresoverthefourmonthsaftertheinterventionas describedabovewiththequantityofexamsusingapaired t -test.Proportionsoflimitedstudiesandtrainingyear distributionwerecomparedwithPearsonchi-square.The significancelevelofalltestswassetto0.05withBonferroni correctionappliedwhereappropriate.Analysiswas performedinSPSSforMacintosh,v28.0(SPSSInc, Chicago,IL).Thisstudyreceivedinstitutionalreviewboard approvalforwaiverofsignedinformedconsent.
RESULTS
Overhalfofresidents17/32(53%)participatedinthe simulationsessionandhadatleastoneclinicalshiftbefore
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 206 EMResidents’ ClinicalPerformanceofAortaPOCUS Wubbenetal.
ClinicalEducationalTotal
APP,advancedpracticeproviderresident; PGY,postgraduateyear.
andafterthesimulationsession.Thedistributionofthe participatingresidentsasprimarystudyauthorsis demonstratedinthe Table,withasignificantincreaseinthe proportionofaortaPOCUSsubmittedbyinterns.Overall, therewasan86%increaseintotalstudiesandan80% increaseinclinicalstudiesafterthesession.However,when comparingonaper-residentbasiswhileadjustingforclinical opportunities,therewasnosignificantdifferencein mediantotalscanningfrequencyper100shifts
(4.4[interquartilerange(IQR) = 0–15.8]vs8.3[IQR = 3.3–23.6], P = 0.21).Therewasalsonosignificantchangein averagetotalqualityscoresonaperresidentbasis (3.2 ± 0.6vs3.2 ± 0.5, P = 0.92).
Therewerenofalsenegativeorfalsepositivesusing facultyreviewofimagesasthegoldstandard.Therewereno differencesintheproportionofstudieswithagreementvs disagreementwiththeresidentinterpretation(100%vs96%, P = 0.29).Therewasadecreaseinthetotalnumberoflimited orinadequatestudies(12/28(43%)vs10/52(19%), P = .02 [X 2]).Therewasnosignificantchangeintheproportionof clinicalstudiessubmittedas “limited” or “inadequate” (2/5 (40%)vs4/9(44%), P = 0.87[X 2]),butthenumberof educationalstudiessubmittedas “limited” or “inadequate” improved(10/23(44%)vs6/43(14%), P =<.001).
DISCUSSION
Overall,thetotalnumberofaortaPOCUSstudies performedintheEDafterthesimulationincreased,albeit withoutademonstrablechangeinquantityorqualityrating onaper-residentbasis.However,therewereanumberof positive findings,whichsupporttheinclusionofultrasound simulationinresidencytraining,includingasignificant increaseintheproportionofstudiescontributedbyinterns andasignificantdecreaseintheproportionofstudiesthat wereincompleteorlimited.
Itseemsunlikelythatthesignificantincreaseinintern POCUSstudieswasduetocontentmasterybasedon compoundingclinicalexperiencealone.AortaPOCUSisone oftheapplicationsrequiringthemostexperiencetogain proficiency,andwithpreviouslydemonstratedplateau
pointsininterpretationandacquisitionat66studiesand84 studies,respectively,whichwerenotapproachedbyanyone inourstudy.6 ThesamestudyfoundthataortaPOCUS qualityactuallydecreasesinitiallywithincreasingnumberof scansbeforeiteventuallyimprovesabovebaseline,which maybecontributingtotheabsenceofimprovementin medianqualityscoresseeninourstudy.6
MuchofthepublishedresearchregardingPOCUS simulationreportsoutcomesrelatedtotheassessmentof learnerexperienceandskillperformanceoutsidetheclinical context.7,8 Whiletheseoutcomesareimportant,thereisa desiretoassessmoretranslationaloutcomesresultingfrom simulationinterventions.9,10 Therearefewtranslational studiesavailablefordirectcomparisontothecurrentstudy. Oursimulationexperiencewasdeliveredasasinglesession, whichislesstime-intensivethanapriorstudyofEMinterns thatfoundpositiveclinicaleffectsofsimulation-based masterylearningonperformanceoffocusedassessmentwith sonographyintrauma.11 OutsideofEM,arandomizedtrial ofultrasoundsimulationforobstetricsandgynecology residentsthatwasalsomorelongitudinalthanthecurrent studyalsofoundpositiveclinicaleffectsofearlysimulation training,inadditiontoclinicalpracticein first-year residents.4 Furtherstudyisneededtodeterminewhetherthe case-basedsimulationapproachinourstudywouldbemore successfulifthetrainingweremoretimeintensive,more longitudinal,andmosttargetedtowardjuniorlearners.
LIMITATIONS
Our findingsaresubjecttothelimitationsofabefore-andafterstudy,includingthepossibilitythatotherfactorsmay havecontributedtotheobservedchangesotherthanthe simulation;however,wearenotawareofanyothertargeted efforttoeducateourresidentsaboutaortaPOCUSduring thestudyperiod,andbelieveitislikelymostchangeswere associatedwiththesimulation.Second,thiswasastudyof residentswhosecategorizationofstudyintentwassubjectto theirsigningfaculties’ preferences.Therefore,weincluded botheducationalandclinicalarchivestudiestoprovideafair portfolioofeachresident’swork,althoughsomeeducational
Table. Numberofaortapoint-of-careultrasoundstudiessubmittedbeforeandaftersimulationtraining,stratifiedbytraininglevelofthe primarystudyauthorbyarchive.(*):P =< .05withBonferronicorrection.
PrePost P (X2)PrePost P (X2)PrePost P (X2) Level.34 <.001 <.001 APP1(20%)1(11%)0(0%)1(2%)1(4%)2(4%) PGY-10(0%)3(33%)2(9%)25(58%)*2(7%)28(54%)* PGY-23(60%)2(22%)12(52%)5(12%)*15(54%)7(14%)* PGY-31(20%)3(33%)9(39%)12(28%)10(36%)15(29%)
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 207
Wubbenetal. EMResidents’ ClinicalPerformanceofAortaPOCUS
studiesmayhavenotbeenintendedforpatientcare.Third, EMresidentsdonotgettochoosewhichpatientstheytake careofintheED(becauseemergenciesareunplannedand unpredictable);sosomeresidentslikelyhadslightlymore exposuretopatientswithindicationsforaortaPOCUSthan others.Inaddition,whilewewereabletoadjustforclinical opportunitiesbasedontimespentintheED,residentsoften seemorepatientsperhourastheyadvancethroughtraining, andwewereunabletoaccountfortotalpatientsseenduring thestudyperiod.Evenconsideringtheselimitations,we believethedatapresentedprovidesanaccuratereal-world assessmentofscanningfrequencyandqualityonEDpatients byEDresidents.
CONCLUSION
Inthefourmonthsfollowingacase-basedsimulationto diagnoseabdominalaorticaneurysmusingpoint-of-care ultrasound,theproportionofaortaPOCUSstudies performedintheEDbyinternsincreasedsignificantly,and theproportionofstudiesthatwereincompleteorlimited significantlydecreased.Whiletherewasnooverallincrease inthemediannumberofscansormeanqualityscoreswhen adjustedforclinicalopportunitiesonaper-residentbasis, amongresidentsasawholetherewasan86%increasein submittedaortaPOCUSstudies.
ACKNOWLEDGMENTS
TheauthorswouldliketothanktheCarverCollegeof MedicinefortheirEducationalInnovationandScholarship Granttosupportthepurchaseoftheultrasoundphantom usedinthisstudy,JohnKeechforhisassistanceingrant writing,PriyankaVakkalankaforherassistanceingrant writingandmanuscriptreview,andDanielMillerand NathanielShekemfortheirassistanceimplementing thesimulation.
AddressforCorrespondence:BrandonM.Wubben,MD,University ofIowaHospitalsandClinics,DepartmentofEmergencyMedicine, 200HawkinsDrive,IowaCity,IA52242.Email: brandon-wubben@ uiowa.edu
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy.This workwassupportedbyanEducationalInnovationandScholarship
GrantprovidedbytheCarverCollegeofMedicineattheUniversity ofIowa.
Copyright:©2024Wubbenetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.AmericanCollegeofEmergencyPhysicians.Policystatement: UltrasoundGuidelines:Emergency,Point-of-Care,andClinical UltrasoundGuidelinesinMedicine.2023.Availableat: https://www. acep.org/siteassets/new-pdfs/policy-statements/ultrasoundguidelines–emergency-point-of-care-and-clinical-ultrasoundguidelines-in-medicine.pdf.AccessedJuly1,2023.
2.RubanoE,MehtaN,CaputoW,etal.Systematicreview:emergency departmentbedsideultrasonographyfordiagnosingsuspected abdominalaorticaneurysm. AcadEmergMed.2013;20(2):128–38.
3.PoSawLL,WubbenBM,BertucciN,etal.Teachingemergency ultrasoundtoemergencymedicineresidents:ascopingreviewof structuredtrainingmethods. JAmCollEmergPhysiciansOpen 2021;2(3):e12439.
4.TolsgaardMG,RingstedC,DreislerE,etal.Sustainedeffectof simulation-basedultrasoundtrainingonclinicalperformance:a randomizedtrial. UltrasoundObstetGynecol.2015;46(3):312–8.
5.ChaikofEL,DalmanRL,EskandariMK,etal.TheSocietyforVascular Surgerypracticeguidelinesonthecareofpatientswithanabdominal aorticaneurysm. JVascSurg.2018;67(1):2–77.e2.
6.BleharDJ,BartonB,GaspariRJ.Learningcurvesinemergency ultrasoundeducation. AcadEmergMed.2015;22(5):574–82.
7.SinghJ,MaternLH,BittnerEA,etal.Characteristicsofsimulationbasedpoint-of-careultrasoundeducation:asystematicreviewof MedEdPORTALcurricula. Cureus.2022;14(2):e22249.
8.LewissRE,HoffmannB,BeaulieuY,etal.Point-of-careultrasound education:theincreasingroleofsimulationandmultimediaresources. JUltrasoundMed.2014;33(1):27–32.
9.McGaghieWC,DraycottTJ,DunnWF,etal.Evaluatingtheimpactof simulationontranslationalpatientoutcomes. SimulHealthc 2011;6(Suppl):S42–7.
10.LienWC,LinP,ChangCH,etal.Theeffectofe-learningon point-of-careultrasoundeducationinnovices. MedEducOnline 2023;28(1):2152522.
11.SmithS,LoboV,AndersonKL,etal.Arandomizedcontrolledtrialof simulation-basedmasterylearningtoteachtheextendedfocused assessmentwithsonographyintrauma. AEMEducTrain 2021;5(3):e10606.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 208 EMResidents’ ClinicalPerformanceofAortaPOCUS Wubbenetal.
EDUCATION SPECIAL ISSUE:BRIEF RESEARCH REPORT
FoundationsofEmergencyMedicine:Impactofa Standardized,Open-access,CoreContentCurriculum onIn-TrainingExamScores
JaimeJordan,MD,MAEd*
NatashaWheaton,MD*
NicholasD.Hartman,MD,MPH†
DanaLoke,MD,MS‡
NathanielShekem,MPAS§
AnwarOsborne,MD,MPM∥
P.LoganWeygandt,MD,MPH¶
KristenGrabowMoore,MD,MEd∥
SectionEditor:DanielleHart,MD
*UniversityofCaliforniaLosAngeles,DavidGeffenSchoolofMedicine, DepartmentofEmergencyMedicine,LosAngeles,California
† WakeForestUniversitySchoolofMedicine,DepartmentofEmergency Medicine,Winston-Salem,NorthCarolina
‡ NorthwesternUniversityFeinbergSchoolofMedicine,Departmentof EmergencyMedicine,Chicago,Illinois
§ UniversityofIowa,DepartmentofEmergencyMedicine,IowaCity,Iowa
∥ EmoryUniversity,DepartmentofEmergencyMedicine,Atlanta,Georgia
¶ JohnsHopkinsUniversitySchoolofMedicine,Baltimore,Maryland
Submissionhistory:SubmittedJune28,2023;RevisionreceivedDecember9,2023;AcceptedJanuary12,2023
ElectronicallypublishedMarch25,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18387
Introduction: Learnersfrequentlybenefitfrommodalitiessuchassmall-group,case-basedteaching andinteractivedidacticexperiencesratherthanpassivelearningmethods.Thesecontemporary techniquesarefeaturesofFoundationsofEmergencyMedicine(FoEM)curricula,andparticularlythe FoundationsI(F1)course,whichtargets first-yearresident(PGY-1)learners.TheAmericanBoardof EmergencyMedicineadministersthein-trainingexam(ITE)thatprovidesanannualassessmentofEMspecificmedicalknowledge.WesoughttoassesstheeffectofF1implementationonITEscores.
Methods: WeretrospectivelyanalyzeddatafrominternsatfourEMresidencyprogramsaccreditedbythe AccreditationCouncilforGraduateMedicalEducation.Wecollecteddatain2021.Participatingsiteswere geographicallydiverseandincludedthree-andfour-yeartrainingformats.Wecollecteddatafrominterns twoyearsbefore(controlgroup)andtwoyearsafter(interventiongroup)implementationofF1ateachsite. YearofF1implementationrangedfrom2015–2018atparticipatingsites.Weabstracteddatausinga standardformincludingprogram,ITErawscore,yearofITEadministration,USMedicalLicensingExam Step1score,Step2ClinicalKnowledge(CK)score,andgender.Weperformedunivariableand multivariablelinearregressiontoexploredifferencesbetweeninterventionandcontrolgroups.
Results: Wecollecteddatafor180PGY-1s.Step1andStep2CKscoresweresignificantpredictorsof ITEinunivariableanalyses(bothwith P < 0.001).AfteraccountingforStep1andStep2CKscores,we didnot findF1implementationtobeasignificantpredictorofITEscore, P = 0.83.
Conclusion: ImplementationofF1curriculadidnotshowsignificantchangesinperformanceontheITE aftercontrollingforimportantvariables.[WestJEmergMed.2024;25(2)209–212.]
INTRODUCTION
Residencyprogramsprovideeducationandtrainingto developcompetentphysicians.Boardcertificationin emergencymedicine(EM)requirescompletionofan
AccreditationCouncilforGraduateMedicalEducation (ACGME)-accreditedtrainingprogramandapassingscoreon theQualifyingExamination(QE)andOralCertification Examination(OCE)administeredbytheAmericanBoardof
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 209
EmergencyMedicine(ABEM).1,2 TheABEMIn-training Examination(ITE)isanimportanttoolusedbytraining programstoassessmedicalknowledgeandprepareresidents fortheQE.1,3 TheITEisdesignedtoreflectthecontentofthe ModelofClinicalPracticeofEmergencyMedicine(EM Model)andhaspredictivevalueinestimatingthelikelihoodof individualresidentspassingtheQE.3 Priorliteraturesuggests thatclinicalexposurealoneleavessignificantgapsin fundamentalknowledgedefinedbytheEMModel.4 Residency didacticcurriculaprovideanopportunitytosupplementcore knowledge;however,thebestmethodsforproviding instructionoutsideoftheclinicalsettingandpreparingtrainees forsuccessfulperformanceontheITEareunknown.
FoundationsofEmergencyMedicine(FoEM)isa national,free,open-access,onlineEMcurriculumthathas beenwidelyadoptedintheUnitedStates.5,6 FoEMbecame availablein2015;registrationforuseofFoEMcoursesfor the2022–2023academicyearincluded237registered educationalprograms,serving6,326residentphysicians.5,6 FoEMoffersstandardized,level-specific,corecontentfor EMresidentsusinglearner-centriceducationalstrategies thathavebeenshowntobenefitlearningsuchassmall-group discussion,peerlearning,andindividualizedguidance.5–11
FoundationsI(F1)isa flippedclassroom,case-basedcourse targetingpostgraduateyear(PGY)-1residentsthatincludes a30-unit,systems-basedcurriculumoffundamentalcontent intheEMModel.5,6,12 Priorliteraturedemonstratespositive effectsofthe flippedclassroommodelonlearning outcomes.13–15
TheF1curriculumincludescuratedselfstudyresourcescalled “LearningPathways” forlearnersto reviewpriortodidacticmeetings,inwhichresidentswork throughmultipleF1caseswithaknowledgeablefacilitator providinginformationinanoral-boardsstyleformat.6 The F1summarizesessentiallearningpointsandsharesthem withlearnersto fillknowledgegapsandallowforspaced repetition.6 AlthoughtheF1curriculumisnotspecifically designedforITEreview,third-partypairedassessmentsfor eachunithavebeenavailableforusesince2017.6
LimitedoutcomedataofFoEMF1establishedquality anddemonstratedhighsatisfactionamongfacultyleaders andresidentlearners.5,6 However,therehasnotbeenan assessmentofobjectivemeasuressuchasmedicalknowledge andITEperformanceThisinformationcanprovideamore comprehensiveassessmentofthevalueofimplementingsuch aprogram.Inthisstudy,wesoughttoevaluatetheeffectof F1courseimplementationonITEperformanceinthe PGY-1year.Wehypothesizedthatimplementationofthe structuredF1curriculumwouldleadtoimproved performanceontheITE.
METHODS
WeperformedaretrospectivecohortstudyofITEdata collectedfromPGY-1residentsatfourACGME-accredited EMresidencyprogramsintheUnitedStatesbeforeandafter
implementationoftheFoEMF1curriculum.Weselected participatingsitesthatweregeographicallydiverseand included3-and4-yeartrainingformats.Wecollecteddatain December2021.AllPGY-1residentsatparticipatingsites duringthestudyperiodwereeligibletoparticipate.We excludedPGY-1residentswhoweremissingdata.
Wedeterminedthattodetecta5%differenceinITEscore with80%powerandanalphaof0.05,wewouldneedto enroll81participantsineachgroup(controland intervention)foratotalof162participants.Ourcontrol groupconsistedofdatafromPGY-1residentsforthetwo yearspriortoimplementationateachsite.Ourintervention groupconsistedofdatafromPGY-1residentsforthetwo yearsafterimplementationateachsite.YearofF1 implementationrangedfrom2015–2018atparticipating sites.Theleadauthorfromeachsiteabstracteddatausinga standardformthatincludedprogram,ITErawtotalscore, yearofITEadministration,UnitedStatesMedicalLicensing Examination(USMLE)Step1score,USMLEStep2 ClinicalKnowledge(CK)score,andresidentgender.Priorto dataabstraction,theauthorgroupreadeachitemonthe formaloudandtrialedabstractingasmallportionof representativedatatoensureclarityofmeaningand consistencyinprocess.
Wecalculateddescriptivestatisticsfordemographicdata andITEperformance.Weperformedregressionanalysesto exploredifferencesbetweentheinterventionandcontrol groups.We firstperformedunivariablelinearregression analysesforvariablesincludingimplementationofF1, residencyprogram,yearofITEadministration,USMLE Step1score,USMLEStep2score,andresidentgenderwith ITErawscoreasouroutcomeofinterest.Weincluded variableswitha P -value < 0.1intheunivariableregressionin amultivariablelinearregressionwiththesameoutcome variable.Weconsideredvariableswitha P -valueof < 0.05in themultivariablemodelasstatisticallysignificant.We performedallanalysesinSPSSv27.0(IBMCorporation, Armonk,NY).
ThisstudywasapprovedbytheInstitutionalReview BoardoftheDavidGeffenSchoolofMedicineatUCLA.
RESULTS
Weabstracteddatafromatotalof224interns.We excluded44internswhoweremissingdata.Weanalyzed datafrom180interns(88pre-implementationand92postimplementation)whohadcompletedata.Thedemographics ofparticipantswithcompletedataareshownin Table1.The meanITErawscoreforinternsinthecontrolgroupwas 72.15 ± 6.72.ThemeanITEscoreforinternsinthe interventiongroupwas72.74 ± 7.93.Intheunivariable regressionanalyses,onlyUSMLEStep1andUSMLEStep2 CKscoresyielded P -valuesof < 0.1(Table2).Becauseour hypothesiscenteredontheimpactofimplementationofthe F1curriculumonITEscores,weforcedthisvariableasthe
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 210 ImpactofStandardized,Open-access,CoreContentCurriculumonITEScores Jordanetal.
Gender
Controlgroup n(%)totaln = 88
Interventiongroup n(%)totaln = 92
Male3231
Female5660
Non-binary01
MeanUSMLE
MeanUSMLE
232(14.26)232(15.59)
USMLE,UnitedStatesMedicalLicensingExamination.
Residencyprogram0.22 YearofITEadministration0.14
USMLEStep1score
Residentgender0.24
USMLE,UnitedStatesMedicalLicensingExamination; ITE, in-trainingexam; CK,clinicalknowledge.
lastvariableafterblockentryofvariablesofUSMLEStep1 scoreandUSMLEStep2CKscoreinthemultivariable regressionanalysis,despiteithavinga P -valueof0.59inthe univariableanalysis.AftercontrollingforStep1scoreand Step2CKscore,F1implementationwasnotasignificant predictorofITEscore,Rsquarechange = 0, P = 0.83.The datasatisfiedallassumptions.
DISCUSSION
OurstudydemonstratesthatbothStep1andStep2CK weresignificantpredictorsofITEscore.Thisisconsistent withpriorliteratureinmultiplespecialtiesdemonstrating associationsbetweenUSMLEscoresandITE performance.16–19 Wefoundthatourinterventiongrouphad aslightlyhigherrawITEscoreshowever,aftercontrolling forUSMLEscores,thisincreasewasnotstatistically significant,despitebeingadequatelypowered.Thiswas somewhatsurprisinggiventhatF1providesaconsistent structureandcomprehensivecoverageofcontentintheEM modelandalsoincorporatesteachingmethodsthathave beenshowntoenhancelearning.2,6–11 However,ourresults alignwithpreviousstudies,whichhavedemonstratedthat changesincurriculumwerenotassociatedwithsignificant differencesinITEperformance.20,21 Specifically,converting
anhourofsynchronousdidacticconferencetoasynchronous learning,andconvertingconferencelecturestosmallgroup, “flipped-classroom” stylelearninghavepreviouslybeen foundtohavenosignificanteffectonITEscores.20,21
ItisimportanttonotethattheobjectiveofF1isto improveEMcoreknowledgeandapplicationintheclinical environmentandisnotspecificallytargetedtowardsITEtest preparationorperformance.Additionally,performanceon theITEmaynotcomprehensivelyrepresentlearner knowledgeofEM.Thismaybeonereasonthatwedidnot findsignificantchangesinITEperformance.Additionally, variableimplementationandusageofF1atdiffering programscouldinfluencepotentialgains.Althoughthe FoEMcoursesarestandardized,participatingprograms mustaddresstheirownuniqueneedsandbarriers;thismay resultinvariabilityincourseimplementation,including variableuseof flipped-classroomstyleasynchronous resourcesandpairedassessments.Itisalsoimportanttonote thattheITEisadministeredinFebruaryofeachyear;thus, participatingPGY-1residentsinthisstudywereonly exposedtoapproximatelysevenmonthsoftheyear-longF1 curriculumpriortotheITE.
Itispossiblethatadditionalimprovementsmaybeseenwith additionaltimespentinthecurriculum.Thenonsignificant improvementseeninthisstudymaybeaugmentedwith implementationofFoundationsII(F2),whichisdesignedfor PGY-2residents,andFoundationsIII(F3),whichisdesigned forPGY-3andPGY-4residents.Theseoutcomesmeritfurther investigation.Whileourstudydidnot findasignificant increaseinITEscorescomparedtostandardcurricula,itwas notworsethanstandardpracticeandhasadditionalbenefitsof afree,standardized,pre-packaged,high-quality,adaptable formatwithuseracceptability.6
Overall,theresultsofthisstudyprovideimportant insightsforboththenumerousprogramsalreadyusing FoEMandthoseEMresidenciesconsideringincorporating itintotheirtrainingprograms.6 Inadditiontoprior feasibilityanduseracceptabilitydata,thisstudyprovidesan evaluationofobjectiveoutcomes,namelyknowledge,the firstlevelinMiller’spyramidofclinicalcompetence.6,22 Therearestillmanyunansweredquestions.Further investigationintotheeffectoftheF1curriculumonABEM QEandOCEperformanceshouldbepursued.Additionally, asFoEMisdesignedtosupportknowledgeapplicationinthe clinicalspace,futureworkcouldevaluatetheimpactof FoEMonotherdomainsofresidentperformance.
LIMITATIONS
Thisstudyhaslimitations.Theremaybeconfoundersnot accountedforinouranalysisthatcouldhaveinfluenced results.WedidnotcollectdataonspecificITEpreparation curriculaatparticipatingsites,individualusageofexternal ITEpreparationmaterialsoutsideoftrainingprogram curricula,timespentusingF1curriculum,useofpaired
Table1. Demographicdataofparticipatinginterns.
Step1score(SD)
Step2score(SD)
244(17.02)246(14.54)
Variable P-value
Table2. Resultsofunivariableregressionanalysisof recordedvariables.
ImplementationofFoundationsF1curriculum0.59
<0.001
USMLEStep2CKscore <0.001
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 211
Jordanetal. ImpactofStandardized,Open-access,CoreContentCurriculumonITEScores
assessments,totalnumberofF1unitscompletedby participatingresidents,ortimespentstudyingforITEin general.However,tothebestofourknowledge,therewere noothermajorchangestothesite’sdidacticcurriculumor methodsofpreparingtraineesfortheITEduringthestudy period.AlthoughtheF1courseincludesstandardized content,participatingprogramsmustaddresstheirown uniqueneedsandvariablesthatimpacttheconsistencyof courseadministration.Theremaybedifferencesinthe personnelwhodeliverthecontent,attendancerequirements, etc,whicharenotaccountedforinourstudy.Theresultsseen inthisstudymaynottransfertoothersiteswhereadherence toimplementationguidelinesismoreorlessconsistent.
CONCLUSION
OurstudysuggeststhattheFoEMF1curriculumisnot associatedwithsignificantchangesinperformanceonthe ITEinEMtrainingprogramsaftercontrollingforimportant variables.Theseresultsmayinformtheuseand implementationofFoEMcoursesinEMtrainingprograms.
AddressforCorrespondence:JaimeJordan,MD,MAEd,University ofCaliforniaLosAngeles,DavidGeffenSchoolofMedicine, DepartmentofEmergencyMedicine,924WestwoodBlvd.,Suite 300,LosAngeles,CA90024.Email: jaimejordanmd@gmail.com
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.
Copyright:©2024Jordanetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.ACGME.MilestonesbySpecialty:EmergencyMedicine.Accreditation CouncilforGraduateMedicalEducation.2021.Availableat: https://www.acgme.org/globalassets/pdfs/milestones/ emergencymedicinemilestones.pdf.AccessedNovember,2022.
2.AmericanBoardofEmergencyMedicine.BecomeCertified. Availableat: https://www.abem.org/public/become-certified AccessedJune23,2023.
3.AmericanBoardofEmergencyMedicine.In-TrainingExam.Available at: https://www.abem.org/public/for-program-directors/in-trainingexamination.AccessedJune23,2023.
4.BischofJJ,EmersonG,MitzmanJ,etal.Doestheemergencymedicine in-trainingexaminationaccuratelyreflectresidents’ clinical experiences? AEMEducTrain. 2019;3(4):317–22.
5.FoundationsofEmergencyMedicine.Availableat: https:// foundationsem.com/.AccessedJune23,2023.
6.MooreKG,KettererA,WheatonN,etal.Development,implementation, andevaluationofanopenaccess,level-specific,corecontent curriculumforemergencymedicineresidents. JGradMedEduc. 2021;13(5):699–710.
7.TwengeJM.Generationalchangesandtheirimpactintheclassroom: teachingGenerationMe. MedEduc. 2009;43(5):398–405.
8.Moreno-WaltonL,BrunettP,AkhtarS,etal.Teachingacrossthe generationgap:aconsensusfromtheCouncilofEmergencyMedicine ResidencyDirectors2009AcademicAssembly. AcadEmergMed. 2009;16(Suppl2):S19–24.
9.HartDandJoingS.TheMillennialGenerationand “thelecture.” Acad EmergMed. 2011;18(11):1186–7.
10.RobertsDH,NewmanLR,SchwartzsteinRM.Twelvetipsforfacilitating Millennials’ learning. MedTeach. 2012;34(4):274–8.
11.CooperAZandRichardsJB.Lecturesforadultlearners:breakingold habitsingraduatemedicaleducation. AmJMed. 2017;130(3):376–81.
12.BeesonMS,AnkelF,BhatR,etal.The2019modeloftheClinical PracticeofEmergencyMedicine. JEmergMed. 2020;59(1):96–120.
13.KingAM,GottliebM,MitzmanJ,etal.Flippingtheclassroomingraduate medicaleducation:asystematicreview. JGradMedEduc. 2019;11(1):18–29.
14.ChenF,LuiAM,MartinelliSM.Asystematicreviewoftheeffectiveness of flippedclassroomsinmedicaleducation. MedEduc. 2017;51(6):585–97.
15.ChenKS,MonrouxeL,LuYH,etal.Academicoutcomesof flipped classroomlearning:ameta-analysis. MedEduc. 2018;52(9):910–24.
16.FeningK,HorstAV,ZirwasM.CorrelationofUSMLEStep1scoreswith performanceondermatologyin-trainingexaminations. JAmAcad Dermatol. 2011;64(1):102–6.
17.NelsonMandCalandrellaC.DoesUSMLEStep1&2scorespredict successonITEandABEMQualifyingExam:areviewofanemergency medicineresidencyprogramfromitsinception. AnnEmergMed. 2017;70(4):58–9.
18.PatzkowskiMS,HauserJM,LiuM,etal.Medicalschoolclinical knowledgeexamscores,notdemographicorotherfactors,associated withresidencyin-trainingexamperformance. MilMed. 2023;188(1-2):e388–91.
19.PerezJAJrandGreerS.CorrelationofUnitedStatesMedicalLicensing examinationandinternalmedicinein-trainingexaminationperformance. AdvHealthSciEducTheoryPract. 2009;14(5):753–8.
20.KingAM,MayerC,BarrieM,etal.Replacinglectureswithsmallgroups: theimpactof flippingtheresidencyconferenceday. WestJEmergMed. 2018;19(1):11–7.
21.WrayA,BennettK,Boysen-OsbornM,etal.Efficacyofan asynchronouselectroniccurriculuminemergencymedicineeducation intheUnitedStates. JEducEvalHealthProf. 2017;14:29.
22.MillerGE.Theassessmentofclinicalskills/competence/performance. AcadMed. 1990;65(9Suppl):S63–7.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 212 ImpactofStandardized,Open-access,CoreContentCurriculumonITEScores Jordanetal.
EDUCATION SPECIAL ISSUE:BRIEF RESEARCH REPORT
IntegratingHospiceandPalliativeMedicineEducationWithinthe AmericanBoardofEmergencyMedicineModel
RebeccaGoett,MD*
JasonLyou,MD†
LaurenR.Willoughby,MD†
DanielW.Markwalter,MD‡§
DianeL.Gorgas,MD†
LaurenT.Southerland,MD,MPH†
*RutgersNewJerseyMedicalSchool,DepartmentofEmergencyMedicine, Newark,NewJersey
† TheOhioStateUniversityWexnerMedicalCenter,DepartmentofEmergency Medicine,Columbus,Ohio
‡ UniversityofNorthCarolinaSchoolofMedicine,DepartmentofEmergency Medicine,ChapelHill,NorthCarolina
§ UniversityofNorthCarolinaSchoolofMedicine,UNCPalliativeCareProgram, ChapelHill,NorthCarolina
SectionEditor:ChrisMerritt,MD
Submissionhistory:SubmittedAugust29,2023;RevisionreceivedNovember20,2023;AcceptedJanuary12,2023
ElectronicallypublishedMarch25,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18448
Background: Hospiceandpalliativemedicine(HPM)isaboard-certi fiedsubspecialtywithinemergency medicine(EM),butpriorstudieshaveshownthatEMresidentsdonotreceivesufficienttraininginHPM. ExpertsinHPM-EMcreatedaconsensuslistofcompetenciesforHPMtraininginEMresidency.We evaluatedhowtheHPMcompetenciesintegratewithintheAmericanBoardofEmergencyMedicine Milestones,whichincludetheModeloftheClinicalPracticeofEmergencyMedicine(EMModel)andthe knowledge,skills,andabilities(KSA)list.
Methods: ThreeemergencyphysiciansindependentlymappedtheHPM-EMcompetenciesontothe 2019EMModelitemsandthe2021KSAs.Discrepancieswereresolvedbyafourthindependent reviewer,andthe finalmappingwasreviewedbyallteammembers.
Results: TheEMModelincluded78%(18/23)oftheHPMcompetenciesasadirectmatch,andwe identifiedrecommendedareasforincorporatingtheother five.TheKSAsincluded43%(10/23).Most HPMcompetenciesincludedintheKSAsmappedontoatleastonelevelB(minimalnecessaryfor competency)KSA.ThreeHPMcompetencieswerenotclearlyincludedintheEMModelorintheKSAs (treatingend-of-lifesymptoms,caringfortheimminentlydying,andcaringforpatientsunder hospicecare).
Conclusion: ThemajorityofHPM-EMcompetenciesareincludedinthecurrentEMModelandKSAs andcorrespondtoknowledgeneededtobecompetentinEM.ProgramsrelyingontheEMMilestonesto plantheircurriculumsmaymisstraininginsymptommanagementandcareforpatientsattheendoflife orwhoareonhospice.[WestJEmergMed.2024;25(2)213–220.]
INTRODUCTION
Athirdofadultswhodiewillreceiveemergency departmentcareinthemonthpriortotheirdeath.1 Emergencyphysiciansneedtrainingtoprovidethehighquality,goal-concordantcarethatthesepatientsdeserve. Hospiceandpalliativemedicine(HPM)isasubspecialtyof emergencymedicine(EM)thataddsanadditionalfocuson
symptommanagement,goal-concordantcare,andqualityof life,especiallyforpatientswithchronicdiseaseorlifethreateningconditions,orwhoareattheendoflife.2 Prior researchhasshownthatcurrentEMresidencytraininglacks instructioninHPM.3–7 Toaddressthis,theAmerican CollegeofEmergencyPhysiciansPalliativeMedicineSection publishedalistof23criticaldevelopmentalmilestonesin
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 213
HPMtrainingforEMresidents.8 However,itisunclearhow besttointegratetheserecommendationsintoanEM residencycurriculum.
ManyEMresidencycurriculumsarebasedonthe knowledgeneededtopasstheEMboardcertificationexams. ThisknowledgeiscodifiedintheAmericanBoardof EmergencyMedicine(ABEM)ModeloftheClinical PracticeofEmergencyMedicine(EMModel)andalistof knowledge,skills,andabilities(KSA).9,10 TheEMmodel alongwiththeKSAsarethefoundationaldocumentsusedto createtheEMMilestones,acompendiumubiquitously employedinbothEMtrainingandassessment.Ourgoalin thisstudywastodeterminewheretheHPMcompetencies fit orcould fitwithintheEMModelandKSAs.Thismapping couldhelpguidecurriculumdesignortheincorporationof theHPMcompetenciesintotestingcontent.
METHODS
Thisstudywasnothumansubjectsresearchandwas deemedexemptfrominstitutionalreviewboardreview.We comparedthe2019EMModelandthe2021KSAstothe HPMcompetencies.TheHPMcompetencieswereassigned numerals.TheEMModelitemswereannotatedbytheir numberandcategory.ThenotationsfortheKSAcategories andcodeswereuseddirectlyfromthe2021document.We dividedtheKSAsintooverarchingcategories(eg,diagnosis, pharmacotherapy,reassessment)whichwethenfurther dividedintosetsofcompetencieswithinthatcategory.10 Eachcompetencywasgivenahierarchyintraining correspondingtoanalphabeticlevel(withAthemost advancedlevelofcompetencyandEtheleast).LevelAis reservedforadvancedknowledgeorskills.LevelBisthe minimalcompetencylevel,definedastheminimumskilllevel everyEMresidentshouldattaintograduate.LevelsC,D, andEareskillsinthedevelopmentofreachinglevelB.
Asthistypeofanalysishasnotbeendonebefore,weuseda sequentialapproachwithinitialindependentreviewers,a mediatorstep,andthen finalconsensusgroupdiscussion. Theconsensusgroupresultswerethenreviewedbytwo independentexternalexperts.Inthe firstphaseofconsensus mapping,tworesidents(EMpostgraduateyear(PGY)-2and EM/internalmedicine(PGY-4)andanEMattending independentlymappedpalliativecarecompetenciesusinga MicrosoftExcelspreadsheet(MicrosoftCorporation, Armonk,NY).Thethreeinitialconceptmappershad independentdatasheetsandwereblindedtoeachother’ s determinations.Acompetencycouldmapontomorethan oneareaoftheEMModel.First,keywordsfromeachHPM competencyweresearchedforintheEMModel.Ifno matcheswerefound,theEMModelwasreviewedlinebyline todeterminewhethertherewereconceptualmatches.Ifthere wasnodirectmatch,buttheHPMcompetencycouldbe incorporatedunderatopic,thiswaslistedasapotentialarea forincorporation.
Anytopicthatdidnothaveatleast2/3agreementonthe initialindependentreviewwasreviewedbyafourth emergencyphysicianwithexpertiseinEMresidenteducation andEMModeldevelopment.Shewasblindedtotheinitial reviewer’snamesbutdidhavetheirresults.Thefullgroup metandreviewedallthemappinguntilconsensuswas reached.Theconsensustableswerethenreviewed independentlybytwoadditionalexternalHPMboardcertifiedEMattendingsinvolvedinresidenteducationattwo differentEMresidencyprograms.Thesameprocesswasused formappingtheKSAs.
RESULTS
IncorporationintotheEmergencyMedicineModel
Fifty-oneof963EMModelitemsweretaggedinthe independent firstroundofmapping,with98.7%consensus (951/963)betweentheinitialthreeindependentreviewerson whetheranitemwasorwasnottaggedasamatch.The final reviewbytheindependentHPM-boardedEMattendingsdid notresultinadjustmentstoanyoftheexistingmappingbut didaddtothepotentialareasof fitfortheHPM competenciesthatdidnotdirectlymatchontotheEM Model. Table1 liststhecompetenciesincludedinthe2019 EMModel(18/23,78%).Manycompetencies fitinto EM Modelcategory20:OtherCoreCompetencies section, whichincludescommunicationskills,transitionsofcare, culturalcompetency,andhealthcarecoordination.
Discrepancydiscussionscenteredaroundmanagementvs diagnosis.Thecompetency HPM2:Treatingdistressing symptoms(eg,nausea/vomiting,dyspnea) wasfeltto fitby keywordmatchunderEMModelcategory 1.0Signs, SymptomsandPresentations .However,thatcategorydoes notmentiontreatmentofsymptomsdirectly.Similarly, HPM18:ComplicationsofCancer couldmaptomany itemsintheEMmodel,butagainreferstopalliative managementofcancercomplicationsratherthandiagnosis.
PotentialAreasofFitintheEmergencyMedicineModel
FiveHPMcompetenciesdidnot fitintotheEMModel. The firsttwo, HPM7:Treatingcommonend-of-life symptoms and HPM8:Carefortheimminentlydying (expectingdeathwithinhourstodaysorrecentlydeceased patientandtheirfamilymembers) ,couldbetaughtunder EMModelitem 20.4.4.2.2:Systems-basedPractice: Withdrawalofsupport. ThisEMModelitemcouldbe clarifiedtoensurethatitincludessymptomcontrolandendof-lifecare.Thenext, HPM11:Caringforpatientsunder hospicecare, couldbetaughtwhenteaching 20.4.4.2.3: Systems-basedPractice:HospiceReferral. However,the hospice-referralEMModelitembettermappedonto HPM 17 ,whichincludesassessingforandinitiatinghospice referrals.Theteamfeltthatidentifyingandreferringpatients tohospicewasaseparateskillsetthancaringforpatientson hospice.ThelasttwoHPMcompetencieswithoutaclear
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 214 IntegratingHPMEducationWithintheABEMModel Goettetal.
Hospiceand palliative competencyDescriptionEMmodelitem
1Paincontrol:a.chronicpain,b.malignant andnon-malignantpain.
19.3.1Anesthesiaandacutepain management-regionalanesthesia
19.3.2Anesthesiaandacutepain management-proceduralsedation
19.3.3Anesthesiaandacutepain management-analgesia
2Treatingdistressingsymptoms (eg,nausea/vomiting,dyspnea)
3Difficultcommunication:a.deliveryofbad news(eg,prognosisanddeathtelling) b.conflictresolution(eg,betweenfamily members
4Goalsofcarediscussions:a.assisting familieswithdecisionmaking.b.assisting patientswithdecisionmaking
1.3.32Nausea/vomiting 1.3.42Shortnessofbreath
*unclearwhethertheseEMmodelelementsreferto treatingthesesymptomsordevelopingadifferential diagnosisforthesesymptoms,butbothshould betaught.
20.1.2.2Interpersonalandcommunication skills-conflictmanagement
20.1.2.4Interpersonalandcommunicationskillsdeliveringbadnews/deathnotifications
20.4.4.1Healthcarecoordination-advancedirectives
5Caregiversupport20.3.4.6Well-beingandresilience-careforthecaregiver
6Non-initiationorstoppingofnonbeneficial interventions
19.2Resuscitation-cardiopulmonaryresuscitation
20.1.1.3Interpersonalskills-patientandfamily experienceofcare
20.4.4.2.2Healthcarecoordination-withdrawalofsupport
9Bereavementandgrieving14.2.4Mooddisordersandthought disorders-griefreaction
10Family-witnessedresuscitation19.2Resuscitation-cardiopulmonaryresuscitation
12Copingandself-care20.3.4.1Well-beingandresilience-fatigue andimpairment
20.3.4.1.1Well-beingandresilience-sleephygiene
20.3.4.3Well-beingandresilience-work/lifebalance
13End-of-lifemanagementinthemass casualtyincident/event
16Screeningforpalliativecareneeds:
a.identifyingpatientswhomaybenefit fromHPMspecialistreferral,b.identifying theimminentlydyingpatient(expected deathwithinhours-days).
17Rapidpalliativecareassessment:
a.aligningdiagnosticsandtherapeuticsto patientgoals,b.functional,psychosocial, andspiritualassessment,c.assessingfor andinitiatinghospicereferrals,d.toolkits tohelpidentifypatientneedsfor appropriatereferrals/resources, e.caregiverburden.
20.4.2.2.1Patienttriageandclassification
20.4.4.2.1Healthcarecoordination-patientidentification forpalliativecare
20.4.4.2.3Healthcarecoordination-hospicereferral
20.3.4.6Well-beingandresilience-careforthecaregiver 20.4.4.2.3Healthcarecoordination-hospicereferral
20.4.4.3.1Healthcarecoordination-activitiesofdaily living/functionalassessment
Table1. ThehospiceandpalliativeemergencymedicineresidencyeducationcompetenciesmappedontotheAmericanBoardof EmergencyMedicineEMModel.
Continuedonnextpage
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 215 Goettetal. IntegratingHPMEducationWithintheABEMModel
(
)
Table1. Continued.
Hospiceand palliative competencyDescriptionEMmodelitem
18Complicationsofcancer:a.disease complications(eg,spinalcord compression,hypercalcemia), b.treatmentcomplications (eg,pancreatitis,tumorlysis,neutropenia, acuterenalfailure).
2.9.2.3Largebowel-radiationcolitis
2.9.2.5Largebowel-neutropenicenterocolitis/typhlitis
3.6.1Diseasesofthepericardium-pericardial tamponade
8.7Oncologicemergencies
8.7.1Oncologicemergencies-febrileneutropenia
8.7.2Oncologicemergencies-hypercalcemia ofmalignancy
8.7.3Oncologicemergencies-hyperviscositysyndrome
8.7.4Oncologicemergencies-malignant pericardialeffusion
8.7.5Oncologicemergencies-spinalcordcompression
8.7.6Oncologicemergencies-superiorvena cavasyndrome
8.7.7Oncologicemergencies-tumorhemorrhage
8.7.8Oncologicemergencies-tumorlysissyndrome
11.1.4.2Bonyabnormalities-tumor-relatedfractures
16.2.3Disordersofthepleura,mediastinum,andchest wall-pleuraleffusion
16.6.2Pulmonaryembolism/infarct-venous thromboembolism
16.6.2.1Pulmonaryembolism/infarct-massiveand submassiveembolism
19Ethical,spiritual,andculturalissues aroundend-of-lifeanddeath
20Advancedirectives:a.physicianorderfor life-sustainingtreatment(POLST), b.medicalorderforlife-sustaining treatment(MOLST),c. fivewishes.
21ethicalandlegalissues:a.decisionmakingcapacity,b.futility.
22Multidisciplinaryteamandsupport systems.(understandingteamrolesand systemresources):a.spiritualchair, b.socialchair,c.hospicecareeligibility, d.continuingcare,e.importanceoflocal andcommunitysupportsystems.
23Transitionsacrosscaresettings,eg, inpatientvshomehospice,palliative careunit
associationwiththeEMModelwere HPM14:Trajectories ofdying:a.Terminalillness,b.OrganFailure,c.Frailty,d. SuddenDeath, and HPM15:Prognostication. Whilethese competenciesnecessitatehavingsoundunderstandingofthe naturalhistoryofdiseaseaswellasphysicalexaminationand
20.1.2.5Interpersonalandcommunication skills-culturalcompetency
20.4.4.1Healthcarecoordination-advancedirectives
20.3.2.4Professionalism-medicalethics
20.4.5.4Regulatory/legal-consent,capacityandrefusal ofcare-consent,capacityandrefusalofcare
20.1.1.1Interpersonalskills-inter-departmentaland medicalstaffrelations
20.1.1.2Interpersonalskills-intra-departmentalrelations, teamwork,andcollaborationskills
20.4.2.4.1EDadministration-alliedhealthprofessionals
20.4.4.2.1Healthcarecoordination-patientidentification forpalliativecare
20.4.4.2.3Healthcarecoordination-hospicereferral
clinicalworkupcomponentsinformingprognosis,theseare alsoskillsforexplainingthelikelihoodofdeathand communicatingwithpatientsandfamilies.Theteam consensuswasthatthesecouldbetaughtwithintheEM Modelitems 20.1.2.4InterpersonalandCommunication
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 216 IntegratingHPMEducationWithintheABEMModel Goettetal.
Table2. Thepalliativeemergencymedicinecompetenciesincorporatewiththe2021AmericanBoardofEmergencyMedicineknowledge, skills,andabilities.
Hospiceandpalliative medicinecompetencyKSAcodeDescriptionLevel
3Difficultcommunicationa.deliveryofbad news(eg,prognosisanddeathtelling)
b.conflictresolution (eg,betweenfamilymembers)
4Goalsofcarediscussions:a.assisting familieswithdecisionmaking.b.assisting patientswithdecisionmaking.
CS17Use flexiblecommunicationstrategiesto negotiateeffectivelywithstaff,consultants, patients,families,andotherstoprovideoptimal patientcare,recognizingandresolving interpersonalconflicts
CS3Elicitpatients’ reasonsforseekinghealthcareand theirexpectationsfromtheEDvisit
CS7Considertheexpectationsofthosewhoprovide orreceivecareintheEDandusecommunication methodsthatminimizethepotentialforstress, conflict,andmiscommunication
CS15Solicitpatientparticipationinmedicaldecisionmakingbydiscussing,risks,benefits,and alternativestocareprovided
ES15Elicitthepatient’sgoalsofcarepriortoinitiating emergencystabilization,includingevaluatingthe validityofadvanceddirectives
B
13End-of-lifemanagementinthemass casualtyincident/event
14Trajectoriesofdying:a.terminalillness, b.organfailure,c.frailty, d.suddendeath.
D
B
C
B
DM11Participateinamasscasualtydrilloreventinan EDinvolvingmultiplepatients,prioritizingcare, containingpotentialexposures,andappropriately assigningresources C
17Rapidpalliativecareassessment: a.aligningdiagnosticsandtherapeuticsto patientgoals,b.functional,psychosocial, andspiritualassessment,c.assessingfor andinitiatinghospicereferrals,d.toolkits tohelpidentifypatientneedsfor appropriatereferrals/resources, e.caregiverburden.
ES15Elicitthepatient’sgoalsofcarepriortoinitiating emergencystabilization,includingevaluatingthe validityofadvanceddirectives
TC11Determine,summarize,andcommunicatethe diagnosisordiagnosticuncertainty,anticipated course,prognosis,dispositionplan,medications, futurediagnostic/therapeuticinterventions,signs andsymptomsforwhichtoseekfurthercareand follow-uptopatientorsurrogate
CS7Considertheexpectationsofthosewhoprovide orreceivecareintheEDandusecommunication methodsthatminimizethepotentialforstress, conflict,andmiscommunication
ES6Recognizeinatimelyfashionwhenfurther clinicalinterventionisfutile B PE6Educatepatientsonthenaturalcourseoftheir diseaseandimpactofpossibletreatmentin relationtoprognosis B
clinicalinterventionisfutile B
15PrognosticationES6Recognizeinatimelyfashionwhenfurther
B PE6Educatepatientsonthenaturalcourseoftheir
B
diseaseandimpactofpossibletreatmentin relationtoprognosis
B
Continuedonnextpage) Volume25,No.2:March2024WesternJournal of EmergencyMedicine 217 Goettetal. IntegratingHPMEducationWithintheABEMModel
B (
Table2. Continued.
Hospiceandpalliative medicinecompetencyKSAcodeDescriptionLevel
20Advancedirectives:a.physicianorderfor life-sustainingtreatment(POLST), b.medicalorderforlife-sustaining treatment(MOLST),c. fivewishes.
21Ethicalandlegalissues:a.decisionmakingcapacity,b.futility.
22Multidisciplinaryteamandsupport systems.(understandingteamrolesand systemresources):a.spiritualchair, b.socialchair,c.hospicecareeligibility, d.continuingcare,e.importanceoflocal andcommunitysupportsystems.
23Transitionsacrosscaresettings, eg,inpatientvshomehospice, palliativecareunit
CS6Elicitinformationfrompatients,families,andother healthcaremembersusingverbal,nonverbal, written,andtechnologicalskills
ES15Elicitthepatient’sgoalsofcarepriortoinitiating emergencystabilization,includingevaluatingthe validityofadvanceddirectives
CS15Solicitpatientparticipationinmedicaldecisionmakingbydiscussing,risks,benefits,and alternativestocareprovided
ES6Recognizeinatimelyfashionwhenfurther clinicalinterventionisfutile
D
B
C
B
LI12Balancepatientautonomywithpatientprotection andadvocacywhenaddressingconsentand refusalofcareinaccordancewithlegaland ethicalstandards B TI9Obtaininformedconsentfromthepatientor
TM1OrganizepatientcareteamsB
CS5Communicateinformationtopatientsandfamilies usingverbal,nonverbal,written,and technologicalskills,andconfirmunderstanding B
CS10Communicatepertinentinformationtohealthcare colleaguesineffectiveandsafetransitions ofcare C
TC11Determine,summarize,andcommunicatethe diagnosisordiagnosticuncertainty,anticipated course,prognosis,dispositionplan,medications, futurediagnostic/therapeuticinterventions,signs andsymptomsforwhichtoseekfurthercareand follow-uptopatientorsurrogate B
TC15Ensuretransitionsofcareareaccuratelyand efficientlycommunicatedbetweencliniciansusing bestpractices
Skills:Deliveringbadnews/DeathNoti fi cations and 20.1.1.3InterpersonalandC ommunicationSkills:Patient andfamilyexperienceofcare.
IncorporationintotheKnowledge,SkillsandAbilities
Thirtyitemsof214weretaggedinthe firstroundwith87% consensus(187/214)betweentheinitialthreeindependent reviewersonwhetheranitemwasorwasnottaggedasa match.Tenofthe23HPMcompetencies(43%)mappedonto 16differentKSAs(Table2).Ofthe16matcheswithinthe KSAs,nonewereadvancedskills(levelA).AllbutHPM13
mappedontoatleastonelevelBskill.Atableshowingallthe HPMcompetenciesandtheirincorporationwithinthe EMModelandKSAstogetherisincludedas SupplementalDataA.
PotentialAreasofFitintotheKnowledge,Skills andAbilities
ThreeadditionalKSAswereidentifiedashavingareasof potential fitorincorporation. HPM5:Caregiversupport and HPM12:Copingandself-care couldbetaughtwhile discussing CS2:Establishrapportwithanddemonstrate
appropriatesurrogatewhenindicated B
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 218 IntegratingHPMEducationWithintheABEMModel Goettetal.
B
empathytowardpatientsandtheirfamilies .Finally, HPM 16:Screeningforpalliativecareneeds couldbetaughtwith TC18:Correctlydeterminetheappropriatedisposition .
DISCUSSION
ThisstudyshowedfairtogoodinclusionofHPM competencieswithinthepublishedEMKSAsandEM Model,demonstratingthattheHPMcompetenciesare representedintheMilestones.However,keytopicareaswere identifiedthatcouldimprovethefocusofEMtrainingin HPM.DemonstratingtheoverlapoftheHPMandEM contentmayhelpEMeducatorsensurethatHPMtrainingis incorporatedintotheircurriculums.Lackoftrainingon thesetopicsisaconsistent findinginnationaland internationalstudies,andeducatorsneedbetterwaysto incorporateHPM-EMtrainingintoresidency curriculums.3–7,11–13 ImprovedteachingoftheHPM-EM competencieshasthepotentialtodecreasethecaregaps seeninEDsymptommanagementandend-of-lifecare, includinglackofgoalsofcareconversationsforcritically illpatients.14,15
AlimitationoftheHPMcompetenciesisthattheyhave notbeenexternallyassessedorinvestigatedandarebasedon expertconsensus.Noneoftheinitialfourreviewerswere involvedinthedevelopmentoftheHPMcompetenciesand theyfoundthemtoalmostallmapontotheEMModelor identifiedplacesintheEMModelthatcouldbeexpandedto includethemmoreexplicitly.Additionally,theHPM competenciesthatmappedontoKSAsallmetatleastone KSAontheminimalcompetencylevel.These findingsimply thattheHPMcompetenciesareskillsthatareat residentlevel.
ThedescriptionsintheHPMcompetenciescanadddepth tothecorrespondingEMMilestonesforcurriculum developmentandsummativeevaluation.Forexample,most residenciesprovidetrainingorsimulationsofmasscasualty care.Thestudygroupenvisionedwaysinwhichend-of-life managementcouldbeaddedintothattraining(HPM13).
Likewise,alectureonpost-cardiacarrestcarecould incorporatetrainingonthenon-initiationorcompassionate discontinuationofinterventionssuchasmechanical ventilation(HPM6).Summativecompetencyassessmentsat endoftrainingtogainboardcertificationcouldalso incorporatemoreHPMcompetency-basedquestions.
MuchoftheoverlapbetweentheHPMcompetenciesand theEMModelandKSAswasin Interpersonaland CommunicationSkills (EMModel)andthe CS –Communication&InterpersonalSkills (KSAs). Communicationskills,althoughchallengingtoteach,are criticalinpatient-centeredcareandwilllikelyhavean increasedemphasisasartificialintelligenceandmachine learningbecomemoreuniversallyintegratedintoclinical care.Currentmodelsforcommunicationinstructionrely heavilyonrolemodeling.16 Residentshavesuggestedthat
formaltrainingincommunicationshouldfocusongeneral communicationskillsandshouldprovidesyntaxtousein futurediscussions.17 Developingcommunicationskills requiresdeliberatepracticeoftechniques,includingNURSE statements(naming,understanding,respecting,supporting, andexploring)andAsk-Tell-Ask.17,18 Additionally, educatorsmustbecomefamiliarwithmethodsfor real-timeteachingofcommunication,suchas “CouldI addsomething?”19
Trajectoriesofdying(HPM7)andprognostication (HPM8)aretwoskillsusedtocounselpatients/familieswith seriousillnessorattheendoflifethatdidnot fitclearlywithin theEMModel.Thesearedifficultskills,andpriorstudies haveidentifiedsomediscordancebetweenwhatfamilies/ caregiversunderstandaboutaperson’sdeathandthe underlyingcausesofdeathidentifiedbythephysician-led team.20 Thus,thisskillshouldbehonedthroughouttraining. ItisourexperiencethatEMresidentsrarelyreceiveexplicit educationonprognostication,andsowerecommendits incorporationintocurriculums.Ourresultsfurthersuggest thattrainingontreatingend-of-lifesymptoms,careforthe imminentlydying,andcaringforpatientsunderhospicecare couldbeoverlookedbycurrentresidentcurriculumswith strictadherencetotheEMModel.
LIMITATIONS
Alimitationofthisprojectisthateventhoughaconsensus processwasusedwithexpertsinresidencyeducationand HPM,othereducationexpertsmayinterpretthedomains andcompetenciesdifferently.Forexample,the EMModel item20.3.4.6Well-beingandResilience-Careforthe caregiver wasmatchedtoHPM5and17aboutpatient caregivers.However,thiscouldalsobeinterpretedas residentself-careasitisunderthewell-beingsection.Finally, whiletrainedHPMemergencyphysiciansreviewedallthe mapping,theinitialmappingdidincluderesidentinput.This couldbeconsideredanadvantage,astheyareexperiencing lecturesweekly,orareapotentialsourceofbias,astheyhave nothadafullEMcurriculumyet.
CONCLUSION
WeidentifiedareasofoverlapwheretheHPM-EM subspecialtycompetenciescanbeemphasizedorintegrated intoEMModel-basedresidencycurriculums.This knowledgecanbeusedforcurriculumplanningand incorporatingHPMintodefinitionsforcompetencyinEM. Thesecouldalsobereflectedin finalsummativeevaluations forcertification.
AddressforCorrespondence:LaurenT.Southerland,MD,TheOhio StateUniversityWexnerMedicalCenter,DepartmentofEmergency Medicine,725PriorHall,376W10thAve.,Columbus,OH43210. Email: Lauren.Southerland@osumc.edu
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 219 Goettetal. IntegratingHPMEducationWithintheABEMModel
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy.
LaurenT.SoutherlandhasNIHgrantfundingnotpertainingtothis study.DianeL.GorgasisaboardmemberontheAmericanBoardof EmergencyMedicine.LaurenT.SoutherlandandDanielW. Markwalterhavecontributedtosomeofthefreeeducational websitesmentionedinthediscussion.Therearenootherconflictsof interestorsourcesoffundingtodeclare.
Copyright:©2024Goettetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.ElmerJ,MikatiN,ArnoldRM,etal.Deathandend-of-lifecarein emergencydepartmentsintheUS. JAMANetwOpen. 2022;5(11):e2240399.
2.GeorgeN,BowmanJ,AaronsonE,etal.Past,present,andfutureof palliativecareinemergencymedicineintheUSA. AcuteMedSurg. 2020;7(1):e497.
3.WoodsEJ,GinsburgAD,BellolioF,etal.Palliativecareinthe emergencydepartment:asurveyassessmentofpatientandprovider perspectives. PalliatMed. 2020;34(9):1279–85.
4.MeoN,HwangU,MorrisonRS.Residentperceptionsofpalliative caretrainingintheemergencydepartment. JPalliatMed. 2011;14(5):548–55.
5.KrausCK,GreenbergMR,RayDE,etal.Palliativecareeducationin emergencymedicineresidencytraining:asurveyofprogramdirectors, associateprogramdirectors,andassistantprogramdirectors. JPain SymptomManage. 2016;51(5):898–906.
6.BaylisJ,HarrisDR,ChenC,etal.Palliativeandend-of-lifecare educationinCanadianemergencymedicineresidencyprograms: anationalcross-sectionalsurvey. CJEM. 2019;21(2):219–25.
7.AdeyemiOJ,SimanN,GoldfeldKS,etal.Emergencyproviders’ knowledgeandattitudestowardhospiceandpalliativecare:acrosssectionalanalysisacross35emergencydepartmentsintheUnited States. JPalliatMed. 2023;26(9):1252–60.
8.ShoenbergerJ,LambaS,GoettR,etal.Developmentofhospiceand palliativemedicineknowledgeandskillsforemergencymedicine
residents:usingtheAccreditationCouncilforGraduateMedical EducationMilestoneframework. AEMEducTrain. 2018;2(2):130–45.
9.AmericanBoardofEmergencyMedicine.The2019ModeloftheClinical PracticeofEmergencyMedicine.Availableat: https://www.abem.org/ public/resources/em-model.AccessedNovember3,2020.
10.AmericanBoardofEmergencyMedicine.2021Knowledge,Skills,& Abilities.Availableat: https://www.abem.org/public/resources/ emergency-medicine-milestones-ksas.AccessedNovember17,2022.
11.SandersS,CheungWJ,BakewellF,etal.Howemergencymedicine residentshaveconversationsaboutlife-sustainingtreatmentsincritical illness:aqualitativestudyusinginductivethematicanalysis. AnnEmerg Med. 2023;82(5):583–93.
12.ZengH,EugeneP,SupinoM.Wouldyoubesurprisedifthispatient diedinthenext12months?Usingthesurprisequestiontoincrease palliativecareconsultsfromtheemergencydepartment. JPalliatCare. 2020;35(4):221–5.
13.BeneschTD,MooreJE,BreyreAM,etal.Primarypalliativecare educationinemergencymedicineresidency:amixed-methodsanalysis ofayearlong,multimodalintervention. AEMEducTrain. 2022;6(6):e10823.
14.WalkerLE,StanichJA,BellolioF.Aqualitativeassessmentofaguide forgoalsofcareconversationsintheED. AmJEmergMed. 2023:75:185–7.
15.YilmazS,GrudzenCR,DurhamDD,etal.Palliativecareneedsand clinicaloutcomesofpatientswithadvancedcancerintheemergency department. JPalliatMed. 2022;25(7):1115–21.
16.OldeBekkinkM,FarrellSE,TakayesuJK.Interprofessional communicationintheemergencydepartment:residents’ perceptions andimplicationsformedicaleducation. IntJMedEduc. 2018;9:262–70.
17.RisingKL,PapanagnouD,McCarthyD,etal.Emergencymedicine residentperceptionsabouttheneedforincreasedtrainingin communicatingdiagnosticuncertainty. Cureus. 2018;10(1):e2088.
18.TalkVital.QuickGuideResources.Availableat: www.vitaltalk.org/ resources/.AccessedAugust14,2023.
19.BackAL,ArnoldRM,TulskyJA,etal. “CouldIaddsomething?”: teachingcommunicationbyinterveninginrealtimeduringaclinical encounter. AcadMed. 2010;85(6):1048–51.
20.MoonF,KissaneDW,McDermottF.Discordancebetweenthe perceptionsofcliniciansandfamiliesaboutend-of-lifetrajectories inhospitalizeddementiapatients. PalliatSupportCare. 2021;19(3):304–11.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 220 IntegratingHPMEducationWithintheABEMModel Goettetal.
EDUCATION SPECIAL ISSUE:BRIEF EDUCATIONAL ADVANCES
TheEffectofaSimulation-basedInterventiononEmergency MedicineResidentManagementofEarlyPregnancyLoss
ShawnaD.Bellew,MD,MPH*
EricaLowing,MD†
LeahHolcomb,MPH,MS‡
*PrismaHealth-Upstate,UniversityofSouthCarolinaSchoolofMedicineGreenville, DepartmentofEmergencyMedicine,Greenville,SouthCarolina
† PrismaHealth-Upstate,UniversityofSouthCarolinaSchoolofMedicineGreenville, DepartmentofObstetricsandGynecology,Greenville,SouthCarolina
‡ ClemsonUniversity,DepartmentofPublicHealthSciences,Clemson,SouthCarolina
SectionEditor:AbraFant,MD
Submissionhistory:SubmittedNovember22,2023;RevisionreceivedJanuary3,2024;AcceptedJanuary17,2024
ElectronicallypublishedMarch25,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18596
Background: Theevaluationofpatientswith first-trimestervaginalbleedingandconcernforearly pregnancyloss(EPL)frequentlyoccursintheemergencydepartment(ED),accountingforapproximately 1.6%ofallEDvisits.1 Unfortunately,thesepatientsconsistentlyreportnegativeexperienceswithED care.2–8 Inadditiontoenvironmentalconcerns,suchaslongwaittimes,patientsoftendescribenegative interactionswithstaff,includingaperceivedlackofempathy,theuseofinsensitivelanguage,and inadequatecounseling.2,3 ThesepatientsandtheirpartnersoftenviewEPLasatraumaticlossoflifeand commonlyexperienceprolongedgriefreactions,includinganxietyanddepression.9–11 Poorsatisfaction withcarehasbeenassociatedwithworsementalhealthoutcomes.12 Thesecomplaintsrepresentan importantopportunityforimprovementinemergencymedicine(EM)training.13
WhilenopublishedliteraturetodatedescribestheperformanceofEMresidentsinmanaging patientspresentingwithEPL,studiessuggestthatevenobstetricsandgynecology(OB/GYN)residents findtheseinteractionschallenging.14,15 Simulation-anddidactic-basedtraininghasbeenshowntobe beneficialinimprovingOB/GYNresidentEPLcounselingandhasbeenassociatedwithimprovedpatient outcomes.16 Toourknowledge,thishasyettobereplicatedinEMresidencytraining.
Objectives: Weaimedtodevelopandevaluateasimulation-basededucationalinterventiontoimprove EMresidentmanagementofpatientspresentingwithEPL.[WestJEmergMed.2024;25(2)221–225.]
CURRICULARDESIGN
Theeducationalinterventionconsistedofthreephases (Figure1)andwasdesignedtooptimizelearningbasedon Kolb’slearningcycle.17,18 Residentswerepresentedwitha challengingscenario(concreteexperience)andthen promptedtoreflectonareasforimprovement(reflective observation).Theythencompletedanasynchronousmodule followedbyaninteractivegroupdiscussion(abstract conceptualization).Thelearningcyclecontinuedthrough activeexperimentationviaarepeatedopportunitytodothe simulation,followedbydebriefing.Thisformofrepetitive simulationhasbeenshowntobemoreeffectivewhen comparedwithnon-repeatedsimulation.19,20
WeimplementedtheinterventioninMay2023and conductedapre/poststudyofitsimmediateimpact,which
wasdeemedexemptbyourinstitutionalreviewboard.The interventiontookplaceatthesimulationcenterofthe affiliatedmedicalschool,duringthetwo-hourperiod typicallyallottedformonthlyresidentsimulation-based education.Postgraduateyear(PGY)1–3EMresidentswere recruitedbasedonaconveniencesampleincludingall residentsattendingsimulationthatday.Theresidents werenotinformedofthetopicoftheintervention priortothedayofthestudy,whichistypicalofour simulationcurriculum.
Sixstandardizedpatients(SP)werehiredtoportray patientsexperiencingEPL.Sixvolunteerfacultyemergency physicians(twomen,fourwomen)observedandevaluated thesimulationsandprovidedinstructionanddebriefing.One facultyOB/GYNphysicianandonefacultyemergency
Volume25,No.2:March2024 WesternJournal of EmergencyMedicine 221
physician(bothwomen)co-facilitatedtheguidedgroup discussion-basededucation.
First,residentsparticipatedina10-minutesimulated patientencounterinwhichtheywereinstructedtocarefora SPwhoportrayedapatientwhowaseightweekspregnant andpresentedwithvaginalbleeding.Priortoevaluatingthe patient,eachresidentwasprovidedwithultrasoundresults indicatingthepregnancywasnonviable(presumably obtainedintriage).
Followingtheencounter,residentsindividuallydebriefed withanEMfacultyobserver.Residentsthenhad30minutes tocompleteanasynchronousonlineeducationalmodulethat includedcontentabouttheassessmentofearlypregnancy bleeding;diagnosingandmanagingectopicpregnancy; preventingalloimmunization;andEPLcounseling.
Particularattentionwaspaidtooptimizingcaretoaddressa patient’sphysical,emotional,andcognitiveneeds,a frameworkrecommendedbyEmondetal.21 Themodulewas deliveredviaaninteractiveeducationalplatform,Rise360 Articulate(Articulate,NewYork,NY).22 Aftercompleting themodule,residentsparticipatedina15-minuteguided groupdialogwithEMandOB/GYNfaculty,discussingbest practicesandmodelingpracticalcommunicationskills. Facilitatorsgaveexamplesofhowtheywouldaddress patientsinvariousscenariostocommunicateclearlywhile alsousingsensitivelanguage.
Followingthisdiscussion,residentsrepeatedthesame 10-minutesimulatedpatientencounterfollowedby individualdebriefingwithEMfaculty.Theinterventionwas designedtoaccommodateupto24residentswiththe resourcesdescribed.
IMPACTANDEFFECTIVENESS
Tostudytheimmediateimpactoftheintervention, residentperformancewasevaluatedusingfourmeasures: 1)completionofcriticalactionsduringthesimulationviaan 11-itemchecklist;2)self-reportedconfidence;3)a10-item multiple-choicetestoffoundationalEPLknowledge;and4) SPperceptionsofresidentempathyduringthesimulationvia
themodifiedJeffersonScaleofEmpathy(JSE).23,24 Allfour evaluativemeasuresweredeliveredimmediatelyfollowing theinitialsimulatedencounter(Phase1)andafterthe final simulationencounter(Phase3).Inadditiontothese measures,residentswereinvitedtoparticipateinabrieffocus groupinterview,conductedbyanon-facultyfacilitator (woman),aftertheinterventiontodiscusstheirimpressions oftheintervention.
FacultyinEMandOB/GYNdevelopedthetaskchecklist toincludecriticalactionsandevidence-basedbestpractices intreatingpatientsexperiencingEPL.Thislistwasadapted fromachecklistemployedinasimilarstudyandmodifiedto reflectEDcare.25 Residentswereaskedtoratetheir perceivedconfidencelevelfromleast(1)tomost(10) confidentregardingthefollowing:knowledgeaboutthe evaluationandmanagementofpatientswith first-trimester bleeding;abilitytocommunicateinasensitiveandempathic mannerwithpatientswithEPL;andabilitytocounsela patientexperiencingEPLregardingwhattoexpectafter discharge.Theyalsocompleteda10-questionmultiplechoicetest,whichEMandOB/GYNfacultydevelopedto assessbasicobjectiveknowledge.Aftereachsimulated encounter,SPscompletedthemodifiedJSE,avalidatedtool forSPevaluationofclinicianempathyandcommunication. ThemodifiedJSEincludes fivequestionsonaseven-point Likertscalerangingfromstronglydisagree(1)tostrongly agree(7).23,24 Anoutlineofthesimulatedcase,themodule, andtheassessmenttoolsareincludedinthe supplemental material accompanyingtheonlinearticle.
Ofthe16residentswhoparticipated,75%identifiedas men,andtherewasrelativelyequalrepresentationofPGY-1 (31.3%),PGY-2(37.5%),andPGY-3(31.3%)residents. Residentsimprovedfrompre-topost-interventionacrossall fourevaluativemeasures(Table1).Beforetheintervention, fewresidentsprovidedinformationaboutwhattoexpect afterdischarge,includingthepotentialpainlevel,the likelihoodofpassingtissue,returnprecautions,andlongtermemotionalramifications.Aftertheintervention, residentsweresignificantlymorelikelytousesensitive
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 222 Simulation-basedInterventiononEMResidentManagementofEPL Bellewetal.
Figure1. Sequenceofaneducationalinterventionforearlypregnancylosscounseling. EPL,earlypregnancyloss.
Bellewetal. Simulation-basedInterventiononEMResidentManagementofEPL
5.Counselspatientabouttheamountofexpectedbleeding2(12.5)11(68.8)0.004
6.Counselspatientonexpectedpain1(6.3)10(62.5)0.004
7.Counselspatientonthepossibilityofpassingtissue2(12.5)12(75.0)0.006
8.Counselspatientonreturnforseverebleeding3(18.8)14(87.5)0.003
9.Counselspatientonreturnforfever2(12.5)15(93.8)0.001
10.NormalizesemotionalramificationsofEPL5(31.3)13(81.3)0.008
11.Discussesfollow-upplan10(62.5)16(100)0.030 EPL,earlypregnancyloss.
languageandtoincludeinformationaboutexpected outcomesandreturnprecautions(Table1b).
Theseresultsindicatethatfocusedtrainingresultedin immediateimprovementsinresidentperformance, particularlyregardingcounselingandcommunication. Giventhepositiveresultsofsimilarinterventionsundertaken inotherlearnerpopulations,thisimmediateimpactlikely indicatesimprovedabilitytocareforpatientsinclinical practice.Verhaegheetalpublishedtheimpactofathreehourin-situsimulationtrainingforOB/GYNresidents, whichresultedinlong-termimprovementsinpsychologic outcomesaswellasreducedneedforreturnvisits.16 As comparedtothesepreviousinterventions,ourcurriculum enhancedefficiencybyemployinganonlinetrainingmodule, whichcoveredadditionalfoundationalknowledgeofearly pregnancybleedingcare(includingectopicpregnancyand threatenedEPL).Thisefficiencyisparticularlyimportantin EMgiventhebreadthofrequiredknowledge.
Whiletheeightresidentswhoparticipatedinthefocus groupinterviewgenerallyreportedpositivefeedback,two residentsdidnotethattheywereconfusedbytheorderofthe simulationsuchthattheyhadadiagnosispriortoany interactionwiththepatient.Inthefuture,thismaybe
amelioratedbyprovidingtheresidentswithmorecontextto thecaseorsimplyrevisingthescenariosothattheultrasound reportisreceivedafteraninitialevaluationandrequestfor imaging.Additionally,thetimeallottedforthe asynchronousmodulewas30minutes,butmostresidents completeditinabout20minutes,indicatingthepossibility ofadditionalcontentorexpansionofanotheraspectof theintervention.
LIMITATIONSANDCONCLUSION
Thisstudydescribesresidentperformanceinasimulated patientencounter,andwecannotconcludethatthisreflects actualclinicalcare.Thisstudyonlyassessedtheimpactofthe trainingonlearning(Kirkpatricklevel2)anddidnotattempt toevaluatetheresidents’ ongoingclinicalbehaviororits effectonpatients.26 Thestudywasconductedduringone sessionand,therefore,wecannotinferinformationabout retentionoflearning.Futureworkshouldassesstheeffectof interventionssuchasthisonclinicianbehaviorandresultant patientoutcomes.Facultyevaluatorswerenotblinded duringthesimulatedpatientencounters,whichcouldhave introducedbiasintotheevaluationprovidedviathe checklist.Thisconcernissomewhataddressedbythebinary
MeasureMaximumscoreMean(SD)Mean(SD) SP-value Performancechecklist114.94(1.80)9.50(1.51)67.0 <.001 Self-confidence3020.06(3.38)24.69(3.50)68.0 <.001 Knowledge105.84(1.29)8.00(1.41)45.5 <.001 Empathy3521.25(6.04)28.06(5.47)65.5 <.001
Table1. Residentassessmentoutcomespre-topost-intervention.
PrePostSignedrank
PrePost Checklistitemn(%)n(%) P-value 1.Deliversbadnewsusingsimplelanguageandwithavoidanceof non-preferredterms(fetus,embryo) 10(62.5)16(100)0.03 2.Allowssilenceforthepatienttoabsorbthenews14(87.5)14(87.5)1.00 3.Acknowledgespatient’semotions15(93.8)15(93.8)1.00 4.Dispelsguilt15(93.8)16(100)1.00
Table1b. Residentchecklistperformance.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 223
natureofthechecklist,inwhicheitherataskwasperformed oritwasnot.Oftheassessmenttools,onlythemodifiedJSE hasbeenexternallyvalidated.CreatingandvalidatingEMspecificmeasurementtoolsforEPLcarewouldensuremore robustdatagoingforward.
“Participantsdisproportionatelyidentifiedasmen(75%), ascomparedtothenationalaverageinemergencyresidencies of62%.27 Giventhesmallpopulationfromwhichthestudy samplewasderived,wedidnotaskparticipantswhetherthey werecis-ortransgendertoavoidlossofanonymity. Similarly,wedidnotaskparticipantsaboutpersonal experienceswithEPL.Futureworkcouldexplorethe relationshipofthesecharacteristicsandexperienceswith clinicalperformance.Despitetheselimitations,theresultsof thisstudyindicateaneedforEPL-specificeducationinEM residencyandthatabrief,simulation-basedinterventionwas effectiveinproducingimmediateimprovements.
Consideringtheresultsofsimilarstudiesconductedinother populations,aninterventionsuchasthismayresultin improvedclinicalcareandlong-termpatientoutcomesinthis common,butdevastating,presentation.
5.HoAL,HernandezA,RobbJM,etal.Spontaneousmiscarriage managementexperience:asystematicreview. Cureus. 2022;14(4):e24269.
6.MeluchAL.Waitingtobeseen:provider-patientcommunicationinthe emergencyroomaboutmiscarriage. HealthCommun. 2021;37(11):1452–4.
7.MillerCA,RoeAH,McAllisterA,etal.Patientexperienceswith miscarriagemanagementintheemergencyandambulatorysettings. ObstetGynecol. 2019;134(6):1285–92.
8.PunchesBE,JohnsonKD,AcquavitaSP,etal.Patientperspectivesof pregnancylossintheemergencydepartment. IntEmergNurs. 2019;43:61–6.
9.PrettymanRJ,CordleCJ,CookGD.Athree-monthfollow-upof psychologicalmorbidityafterearlymiscarriage. BrJMedPsychol. 1993;66(Pt4):363–72.
10.ZaccardiR,AbbottJ,Koziol-McLainJ.Lossandgriefreactionsafter spontaneousmiscarriageintheemergencydepartment. AnnEmerg Med. 1993;22(5):799–804.
11.BellhouseC,Temple-SmithM,WatsonS,etal. “Thelosswas traumatic somehealthcareprovidersaddedtothat”:Women’s experiencesofmiscarriage. WomenBirth. 2019;32(2):137–46.
AddressforCorrespondence:ShawnaD.Bellew,MD,MPH,Prisma Health-Upstate,UniversityofSouthCarolinaSchoolofMedicine Greenville,DepartmentofEmergencyMedicine,701GroveRoad, Greenville,SC29605.Email: Shawna.bellew@prismahealth.org
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.
Copyright:©2024Bellewetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.WittelsKA,PelletierAJ,BrownDF,etal.UnitedStatesemergency departmentvisitsforvaginalbleedingduringearlypregnancy, 1993–2003. AmJObstetGynecol. 2008;198(5):523.e1–6.
2.Larivière-BastienD,deMontignyF,VerdonC.Women’sexperiencesof miscarriageintheemergencydepartment. JEmergNurs.
2019;45(6):670–6.
3.MacWilliamsK,HughesJ,AstonM,etal.Understandingtheexperience ofmiscarriageintheemergencydepartment. JEmergNurs. 2016;42(6):504–12.
4.DaintyKN,SeatonMB,McLeodS,etal.Reframinghowearlypregnancy lossisviewedintheemergencydepartment. QualHealthRes. 2021;31(6):1119–28.
12.deMontignyF,VerdonC,MeunierS,etal.Women’spersistent depressiveandperinatalgriefsymptomsfollowingamiscarriage:the roleofchildlessnessandsatisfactionwithhealthcareservices. Arch WomensMentHealth. 2017;20(5):655–62.
13.EvansCS.Earlypregnancylossintheemergencydepartment:lessons learnedasaspouse,newfather,andemergencymedicineresident. Ann EmergMed. 2021;77(2):233–6.
14.BrannMandButeJJ.Communicatingtopromoteinformeddecisionsin thecontextofearlypregnancyloss. PatientEducCouns. 2017;100(12):2269–74.
15.ButeJJandBrannM.Tensionsandcontradictionsininterns’ communicationaboutunexpectedpregnancyloss. HealthCommun. 2020;35(5):529–37.
16.VerhaegheC,GicquelM,BouetPE,etal.Positiveimpactofsimulation trainingofresidentsonthepatients’ psychologicalexperience followingpregnancyloss. JGynecolObstetHumReprod. 2020;49(3):101650.
17.KolbDA. ExperientialLearning:ExperienceastheSourceofLearning andDevelopment.2nd ed.Hoboken,NJ:PearsonEducation,2015.
18.StockerM,BurmesterM,AllenM.Optimisationofsimulatedteam trainingthroughtheapplicationoflearningtheories:adebatefora conceptualframework. BMCMedEduc. 2014;14:69.
19.AuerbachM,KesslerD,FoltinJC.Repetitivepediatricsimulation resuscitationtraining. PediatrEmergCare. 2011;27(1):29–31.
20.NgC,PrimianiN,Orchanian-CheffA.Rapidcycledeliberatepracticein healthcaresimulation:ascopingreview. MedSciEduc. 2021;31(6):2105–20.
21.EmondT,deMontignyF,GuillaumieL.Exploringtheneedsof parentswhoexperiencemiscarriageintheemergencydepartment: Aqualitativestudywithparentsandnurses. JClinNurs. 2019;28(9-10):1952–65.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 224 Simulation-basedInterventiononEMResidentManagementofEPL Bellewetal.
Bellewetal. Simulation-basedInterventiononEMResidentManagementofEPL
22.FirsttrimesterbleedingandEarlyPregnancyLoss.Availableat: https:// rise.articulate.com/share/CfDnfG-CI6UEdqU1t-FEUDtPnu_UU5mQ AccessedMay9,2023.
23.HojatM. EmpathyinHealthProfessionsEducationandPatientCare 1st ed.NewYorkCity,NY:SpringerInternationalPublishing,2016.
24.MalloryL,FloyedR,DoughtyC,etal.ValidationofamodifiedJefferson ScaleofEmpathyforobserverstoassesstrainees. AcadPediatr. 2021;21(1):165–9.
25.MarkoEK,Buery-JoynerSD,SheridanMJ,etal.Structuredteaching ofearlypregnancylosscounseling. ObstetGynecol. 2015;126(Suppl4):1s–6s.
26.JohnstonS,CoyerFM,NashR.Kirkpatrick’sevaluationofsimulation anddebriefinginhealthcareeducation:asystematicreview. JNursEduc. 2018;57(7):393–8.
27.DeFazioCR,CloudSD,VerniCM,etal.Womeninemergencymedicine residencyprograms:ananalysisofdatafromAccreditationCouncilfor GraduateMedicalEducation-approvedresidencyprograms. AEMEduc Train. 2017;1(3):175–8.
28.LeeL,MaW,DaviesS,etal.Towardoptimalemotionalcareduringthe experienceofmiscarriage:anintegrativereviewoftheperspectivesof women,partners,andhealthcareproviders. JMidwiferyWomens Health. 2023;68(1):52–61.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 225
RootCauseAnalysisofDelayedEmergencyDepartment ComputedTomographyScans
ArjunDhanik,MD*†∘
BryanA.Stenson,MD*†∘
RobinB.Levenson,MD*†
PeterS.Antkowiak,MD,MPH*†
LeonD.Sanchez,MD,MPH†‡
DavidT.Chiu,MD,MPH*†
SectionEditor:MarkLangdorf,MD,MPHE
*BethIsraelDeaconessMedicalCenter,DepartmentofEmergencyMedicine, Boston,Massachusetts
† HarvardMedicalSchool,HarvardUniversity,Boston,Massachusetts
‡ BrighamandWomen’sFaulknerHospital,DepartmentofEmergencyMedicine, Boston,Massachusetts
∘ Co- firstauthorswhoequallycontributedtothiswork
Submissionhistory:SubmittedJanuary23,2023;RevisionreceivedOctober25,2023;AcceptedDecember5,2023
ElectronicallypublishedFebruary9,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.17831
Introduction: Asolutionforemergencydepartment(ED)congestionremainselusive.Asrelianceon imaginggrows,computedtomography(CT)turnaroundtimehasbeenidentifiedasamajorbottleneck.In thisstudywesoughttoidentifyfactorsassociatedwithsignificantlydelayedCTintheED.
Methods: WeperformedaretrospectiveanalysisofallCTimagingcompletedatanurban,tertiarycare EDfromMay1–July31,2021.Duringthatperiod,5,685CTswereperformedon4,344patients,witha mediantimefromCTordertocompletionof108minutes(Quartile1[Q1]:57minutes,Quartile3[Q3]:182 minutes,interquartilerange[IQR]:125minutes).Outliersweredefinedasstudiesthattooklongerthan 369minutestocomplete(Q3 + 1.5 × IQR).Wesystematicallyreviewedoutlierchartstodetermine factorsassociatedwithdelayandidentified fivefactors:behaviorallynon-compliantormedically unstablepatients;intravenous(IV)lineissues;contrastallergies;glomerular filtrationrate(GFR) concerns;anddelaysrelatedtoimagingprotocol(eg,needforIVcontrast,requestfororaland/orrectal contrast).Wecalculatedconfidenceintervals(CI)usingthemodifiedWaldmethod.Inter-raterreliability wasassessedwithakappaanalysis.
Results: Weidentifiedatotalof182outliers(4.2%oftotalpatients).Fifteen(8.2%)caseswereexcluded forCTtime-stampinconsistencies.Ofthe167outliersanalyzed,38delays(22.8%,95%confidence interval[CI]17.0–29.7)wereduetobehaviorallynon-compliantormedicallyunstablepatients;30 (18.0%,95%CI12.8–24.5)wereduetoIVissues;24(14.4%,95%CI9.8–20.6)wereduetocontrast allergies;21(12.6%,95%CI8.3–18.5)wereduetoGFRconcerns;and20(12.0%,95%CI7.8–17.9) wererelatedtoimagingstudyprotocols.Thecauseofthedelaywasunknownin55cases(32.9%,95% CI26.3–40.4).
Conclusion: Ourreviewidentifiedbothmodifiableandnon-modifiablefactorsassociatedwith significantlydelayedCTintheED.Patientfactorssuchasbehavior,allergies,andmedicalacuitycannot becontrolled.However,institutionalpoliciesregardingdifficultIVaccess,contrastadministrationinlow GFRsettings,andstudyprotocolsmaybemodified,capturingupto42.6%ofoutliers.[WestJEmerg Med.2024;25(2)226–229.]
INTRODUCTION
Asolutionforemergencydepartment(ED)congestion remainselusive.Asrelianceonimaginggrows,computed
tomography(CT)turnaroundtime fromCTorderto completion hasbeenidentifiedasamajorbottleneck.1,2 Onestudyshowedthatpatientswhohadradiological
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 226 BRIEF RESEARCH REPORT
diagnosticswere4.4timesmorelikelytostayoverfourhours intheEDthanthosewhodidnothavethesetests.2
NumerousstudieshaveidentifiedstrategiestodecreaseCT turnaroundtimes.Whiteetalmappedthecomplexprocessof EDradiologytransportandappliedsystemsengineering principlestoimproveefficiencywithoutincreasingresource use. 3 Perotteetalassembledamultidisciplinarystakeholder teamtoidentifybarriersandimplementsolutionstoreduce CTturnaroundtimefrom5.8to4.6hoursdespitea 13.8%increaseinthenumberofscans.1 Variousstudieshave demonstratedtheefficacyofapplyingLeanand SixSigmaprinciples.4,5 Finally,queuingtheoryhasbeen usedtomodelEDdelayswithvaryinglevelsof resourceutilization.6,7
Therehasnotyetbeenadedicatedoutlieranalysisof delayedCTscansintheED.Inthisstudywesoughtto identifyfactorsassociatedwithsignificantlydelayedCT. ThisisconsequentialgiventhatpatientswithEDstayslonger thansixhoursdirectlycontributetocrowding.8
METHODS
WeperformedaretrospectiveanalysisofallCTs completedatanurban,tertiarycareEDinBoston, Massachusetts,fromMay1–July31,2021.Duringthat period,5,685scanswereperformedon4,344patients,witha mediantimefromCTordertocompletionof108minutes (Quartile1[Q1]:57minutes,Quartile3[Q3]:182minutes, interquartilerange[IQR]:125minutes).Outlierswere definedasstudiesthattooklongerthan369minutesto complete(Q3 + 1.5 × IQR).WedefinedCTcompletion timeasthepointatwhichtheCTtechnologistmarksthe studyascompleted,therebyremovingtheconfounderof radiologistreadtime.
Wesystematicallyreviewedoutlierchartsand communicationsbetweenmembersofthecareteamto determinefactorsassociatedwithdelayandidentified five factors:behaviorallynon-compliantormedicallyunstable patients;intravenous(IV)lineissues(eg,IVinfiltration, difficultIVaccess,inadequateIVsize);contrastallergies; glomerular filtrationrate(GFR)concerns;anddelaysrelated toimagingprotocol(eg,need/requestforcontrast administration,includingIV,oral,and/orrectal).
Confidenceintervals(CI)werecalculatedusingthemodified Waldmethod.Weperformedakappaanalysistoassessfor inter-raterreliability.Thiswasdoneoneachcategory individuallyassomeoutliercaseshadmultiplecontributing factors.Thisstudydesignwasapprovedbyourinstitutional reviewboardwithadeterminationofexemption.We observed10ofthe12methodsofhealthrecordreviewas outlinedbyWorsteretal,withtheexceptionsofabstractor performancemonitoringandabstractorblinding tohypothesis.9
RESULTS
Weidentified182outliers(4.2%oftotalpatients)and excluded15cases(8.2%)forCTtime-stampinconsistencies. Ofthe167outliersanalyzed,38delays(22.8%,95%CI 17.0–29.7)wereduetobehaviorallynon-compliantor medicallyunstablepatients;30(18.0%,95%CI12.8–24.5) wereduetoIVissue;,24(14.4%,95%CI9.8–20.6)weredue tocontrastallergies;21(12.6%,95%CI8.3–18.5)weredueto GFRconcerns;and20(12.0%,95%CI7.8–17.9)wererelated toimagingstudyprotocol.Thecauseofthedelaywas unknownin55cases(32.9%,95%CI26.3–40.4).The distributionofCTtypesforoutliercasesisillustrated in Table1
Kappavaluesrangedfrom0.69–0.98forallthecategories (Table2).Intravenousissueshadthelowestdegreeof agreement,whiledelaysduetoallergyprotocolshadthe highestdegreeofagreement.
DISCUSSION
Ourreviewidentifiedbothmodifiableandnon-modifiable factorsassociatedwithsignificantlydelayedCTintheED. Patientfactorssuchasbehavior,allergies,andmedicalacuity cannotbecontrolled.However,institutionalprotocols regardingdifficultIVaccess,contrastadministrationinlow GFRsettings,andstudyprotocolsmaybemodified.Oneof thesemodifiablefactorsisIVaccess:earlyinvolvementofan
Table1. Distributionofoutliersinemergencydepartment computedtomography.
ComputedtomographytypeNumber(%total)
Torso(anychest/abdomen/pelvisimaging)124(62.0%)
Non-contrasthead37(18.4%)
Spine15(7.5%)
Angiogramheadandneck13(6.5%)
Face,orbits,softtissueneck7(3.5%)
Extremity5(2.5%)
Kappaanalysisoffactorsassociatedwithsignificantly delayedcomputedtomography.
Factorsassociatedwith delay
Kappa(95%confidence interval)
Intravenouslineissues0.69(0.55–0.83)
Contrastallergy0.98(0.93–1.00)
Renalfunctionconcerns0.86(0.74–0.98)
Behaviorallyormedically unstablepatient 0.85(0.75–0.94)
Imagingprotocol0.83(0.70–0.96)
Unknown0.86(0.78–0.95)
Table2.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 227 Dhaniketal. RootCauseAnalysisofDelayedEDCTs
IVteamorutilizationofultrasoundforIVplacementmay expediteimaging.Ourdatasuggeststhat18.0%ofoutliers canbemoreefficientlyimagedbyimprovingIVplacement strategies.Studieshaveshownthatnearly9%ofEDpatients havedifficultIVaccess,definedinonepaperasrequiring ≥3 attemptsoranultrasound-guidedline.Thesepatients experiencestatisticallysignificantdelaysinestablishingIV access,obtaininglabresults,andreceivinganalgesia,aswell asexperiencinglongerEDlengthofstay.10 Therefore,the benefitsofexpeditiousIVplacementextendsbeyond enhancedCTthroughput.
Thesecondmodifiablefactorpertainstocontrast administrationinlowGFRsettings.Thereisgrowing evidencethattheriskofacutekidneyinjuryresultingfrom contrastadministrationinpatientswithreducedGFRmay havebeenoverestimated.11 Thisshifthasbeenattributedto thefactthatmuchoftheexistingliteraturewasnot sufficientlycontrolledtodistinguishbetweencontrastinducedandcontrast-associatednephropathy.11 Institutions mayconsiderrevisingpolicies,suchasforgoingmandatory pre-hydrationorradiologistconversationsandamending exitingGFRcutoffs,toexpediteimaging.
JudiciousprotocolingofCTmayaddressaproportionof outliers.Onestudyfoundthatpatientswhohadan abdominal/pelvicCTwithonlyIVcontrasthadan approximatelytwo-hourshorterEDlengthofstaywhen comparedtopatientswhoreceivedaCTwithoralandIV contrast.12 Thisdifferencewasevenmorepronouncedwhen comparingpatientswhounderwentCTwithoralcontrast withthosewhowereimagedwithnocontrast:patientswho receivednocontrasthadanapproximatelyfour-hourshorter lengthofstay.13 Finally,eliminationoftheroutineuseoforal contrastinabdominal/pelvicCThasbeenshowntoshorten EDlengthofstaywithoutaffectingdiagnosticaccuracy.14 ConsidereduseofcontrastmayimproveCTthroughput.
WemodifiedEDworkflowtoimproveCTthroughput andaddresssomeoftheoutliersidentifiedinthisstudy.We revisedinstitutionalpoliciesregardingcontrast administrationinlowGFRpatientsandstreamlined communicationbetweentheEDandradiologyteams.
Previously,CTinapatientwithaGFRof45–60milliliters perminute(mL/min)triggeredaconversationbetween radiologyandtheEDcareteamregardingoralhydration. Underthenewguidelines,patientswithaGFR ≥45mL/min mayproceeddirectlytoCTwithIVcontrast.ForGFR 30–45mL/min,radiologywillcalltheEDteamanddiscuss themeritsofadministeringIVcontrast.IftheEDteamelects toproceedwithIVcontrast,thevolume,timing(pre-orpostCT),androuteof fluidhydrationareallatthediscretionof theED.Computedtomographythroughputistherefore maximizedaspatientsmaybehydrated after receivingCT. ForcaseswithaGFR ≤30mL/min,radiologywilldiscussthe meritsofIVcontrastwiththeEDteam.Ifcontrastistobe administered,onehourofIVpre-hydrationisrecommended
priortoimagingifthereisnocontraindication. CommunicationbetweentheEDandradiologyteamshas beenstreamlinedwiththeintroductionofautomated messagesthatindicatewhenpre-hydrationhasbeeninitiated andcompleted.
Analysisofthecommunicationbetweentheradiologyand EDteamsrevealedthattherewereoftenmultiplecalls regardingapatient’shydrationstatus.Weintendtorepeata similaranalysiswiththeaboveinterventionstoassess forachangeinthenumberofdelayedCTstudiesdueto GFRconcerns.Werecommendthatinstitutions performtheirownanalysisofoutlierstounderstand opportunitiesforimprovementandtoexpediteoverall EDthroughput.
LIMITATIONS
Limitationsofourstudyincludethefactthatitwas conductedatasingle,urban,academic,tertiary-careED. Thispopulationmaynotbeindicativeofthatseenbyother EDs.Furthermore,residentsinourEDtakeownershipof difficultIVplacementaspartoftheirtraining.Thus,difficult IVplacementmaynotbeassociatedwithdelayedCTinother EDsthathavededicatedIVaccessteams.TheGFRcutoffs forcontrastadministrationinourEDareadmittedly stringent.Otherinstitutionswithlessstringentcutoffsmay notseeasmanysignificantlydelayedCTstudiesdueto GFRconcerns.
Forthepurposesofthisanalysis,patientfactorssuchas behavior,allergies,andmedicalacuitywereconsiderednonmodifiable.Futurestudiesmayconsiderreviewingprotocols forallergyprophylaxisorbehavioralde-escalation.Finally, weexcludedatotalof41.1%ofoutliers:8.2%duetoCTtimestampinconsistenciesand32.9%becausethecauseofthe delaycouldnotbeidentifieddespitethoroughreviewof outliercharts.
CONCLUSION
Weidentifiedmodifiableandnon-modifiablefactors associatedwithsignificantlydelayedCTintheED. InterventionssuchaspromptIVteaminvolvementor utilizationofultrasoundforIVplacement,revisionof institutionalpoliciesregardingcontrastadministrationin lowGFRsettings,andCTprotocolconsiderationmay addressupto42.6%ofoutliers.Theseinterventionsmay improveCTturnaroundtimesandEDthroughput. Futureresearchwillextendthisanalysisbymeasuringthe effectofrevisedinstitutionalpoliciesregarding contrastadministration.
AddressforCorrespondence:BryanA.Stenson,MD,BethIsrael DeaconessMedicalCenter,RosenbergBuilding-2,One DeaconnessRoad,Boston,MA02215.Email: bstenson@bidmc. harvard.edu
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 228 RootCauseAnalysisofDelayedEDCTs Dhaniketal.
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.
Copyright:©2024Dhaniketal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.PerotteR,LewinGO,TambeU,etal.Improvingemergencydepartment flow:reducingturnaroundtimeforemergentCTscans. AMIAAnnu SympProc. 2018;2018:897–906.
2.RyanA,HunterK,CunninghamK,etal.STEPS:leanthinking,theoryof constraintsandidentifyingbottlenecksinanemergencydepartment. Ir MedJ. 2013;106(4):105–7.
3.WhiteBA,YunBJ,LevMH,etal.Applyingsystemsengineeringreduces radiologytransportcycletimesintheemergencydepartment. WestJ EmergMed. 2017;18(3):410–8.
4.RachhP,PendleyAM,DuongPT,etal.DecreasingCTacquisitiontime intheemergencydepartmentthroughleanmanagementprinciples. Radiographics. 2021;41(3):E81–9.
5.KleinDandKhanV.Utilizingsixsigmaleanstrategiestoexpedite emergencydepartmentCTscanthroughputinatertiarycarefacility. JAmCollRadiol. 2017;14(1):78–81.
6.JosephJW.Queuingtheoryandmodelingemergencydepartment resourceutilization. EmergMedClinNorthAm. 2020;38(3):563–72.
7.StensonBA,JosephJW,AntkowiakPS,etal.Estimatingmaximum patientthroughputatanEDscanner:isthreepatientsperhourthemagic number? AmJEmergMed. 2021;40:159–61.
8.HennemanPL,NathansonBH,LiH,etal.Emergencydepartment patientswhostaymorethan6hourscontributetocrowding. JEmerg Med. 2010;39(1):105–12.
9.WorsterA,BledsoeRD,CleveP,etal.Reassessingthemethodsof medicalrecordreviewstudiesinemergencymedicineresearch. Ann EmergMed. 2005;45(4):448–51.
10.DavisEM,FeinsmithS,AmickAE,etal.Difficultintravenousaccessin theemergencydepartment:performanceandimpactofultrasoundguidedIVinsertionperformedbynurses. AmJEmergMed. 2021;46:539–44.
11.DavenportMS,PerazellaMA,YeeJ,etal.Useofintravenousiodinated contrastmediainpatientswithkidneydisease:consensusstatements fromtheAmericanCollegeofRadiologyandtheNationalKidney Foundation. Radiology. 2020;294(3):660–8.
12.HopkinsCL,MadsenT,FoyZ,etal.Doeslimitingoralcontrastdecrease emergencydepartmentlengthofstay?. WestJEmergMed. 2012;13(5):383–7.
13.HuynhLN,CoughlinBF,WolfeJ,etal.Patientencountertimeintervals intheevaluationofemergencydepartmentpatientsrequiring abdominopelvicCT:oralcontrastversusnocontrast. EmergRadiol. 2004;10(6):310–3.
14.LevensonRB,CamachoMA,HornE,etal.Eliminating routineoralcontrastuseforCTintheemergencydepartment: impactonpatientthroughputanddiagnosis. EmergRadiol. 2012;19(6):513–7.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 229 Dhaniketal. RootCauseAnalysisofDelayedEDCTs
forOlderPatients:AQualitativeStudy
MackenzieA.McKnight,MD*
MelissaK.Sheber,MS*
DanielJ.Liebzeit,PhD†
AaronT.Seaman,PhD‡
EricaK.Husser,PhD§ HarleahG.Buck,PhD†
HeatherS.Reisinger,PhD‡ SangilLee,MD,MS‡∥
*UniversityofIowa,RoyJ.andLucilleA.CarverCollegeofMedicine,IowaCity,Iowa
† UniversityofIowa,CollegeofNursing,IowaCity,Iowa
‡ UniversityofIowa,RoyJ.andLucilleA.CarverCollegeofMedicine,Departmentof InternalMedicine,IowaCity,Iowa
§ PennStateUniversity,RossandCarolNeseCollegeofNursing, UniversityPark,Pennsylvania
∥ DepartmentofEpidemiology,UniversityofIowaCollegeofPublicHealth, DepartmentofEmergencyMedicine,IowaCity,Iowa
SectionEditors:StephenMeldon,MD,andMarkLangdorf,MD,MHPE
Submissionhistory:SubmittedApril21,2023;RevisionreceivedOctober30,2023;AcceptedNovember21,2023
ElectronicallypublishedJanuary25,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18088
Introduction: Olderadultsoftenhavemultiplecomorbidities;therefore,theyareathighriskforadverse eventsafterdischarge.The4Msframework whatmatters,medications,mentation,mobility hasbeen usedinacuteandambulatorycaresettingstoidentifyriskfactorsforadverseeventsinolderadults, althoughithasnotbeenusedintheemergencydepartment(ED).Weaimedtodeterminewhether1)use ofthe4Msworksheetwouldhelpemergencycliniciansunderstandolderadultpatients’ goalsofcareand 2)useoftheworksheetwasfeasibleintheED.
Methods: Weconductedaqualitative,descriptivestudyamongpatientsaged ≥60yearsand emergencycliniciansfromJanuary–June2022.Patientswereaskedto fillouta4Msworksheet; followingthis,semi-structuredinterviewswereconductedwithpatientsandcliniciansseparately. Weanalyseddatatocreatecodes,whichweredividedintocategoriesandsub-categories.
Results: Atotalof20olderpatientsand19emergencyclinicianswereinterviewed.Weidentifiedtwo categoriesbasedonouraims:understandingpatientgoalsofcare(sub-categories:clinician/patient concordance;understandingunderlyinggoalsofcare;underlyinggoalsofcarediscrepancy)anduseof 4MsWorksheet(sub-categories:worksheettodiscussiondiscrepancy;challengesusingworksheet; challengecompletingworksheetbeforedischarge).Ratesofconcordancebetweenpatientandclinician onmainconcern/goalofcareandunderlyinggoalsofcarewere82.4%and15.4%,respectively.
Conclusion: Wefoundthatmostpatientsandemergencycliniciansagreedonthemaingoalofcare, althoughcliniciansoftenfailedtoelicitpatients’ underlyinggoal(s)ofcare.Additionally,manypatients preferredtohavetheinterviewer fillouttheworksheetforthem.Therewasoftendiscrepancybetweenwhat waswrittenandwhatwasdiscussedwiththeinterviewer.Moreresearchisneededtodeterminethebest waytointegratethe4Msframeworkwithinemergencycare.[WestJEmergMed.2024;25(2)230–236.]
INTRODUCTION
Olderadultswillaccountforover20%oftheUS populationinthenextdecade,andtheyaremorelikelyto havemultiplecomorbidities,takemoremedications,anduse morehealthcareresourcesthanindividualsinyoungerage
groups. 1 Thevisitratetoemergencydepartments(ED)inthe USin2019totaled43visitsper100forindividuals 65–74yearsofageand66visitsper100forindividuals ≥75years.Thevisitrateforthose ≥75washigherthanall otheragegroups(rangingfrom19–25visitsper100
Usabilityofthe4MsWorksheetintheEmergencyDepartment
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 230 ORIGINAL RESEARCH
individuals)exceptforthoseunderoneyearofage.Several studieshaveshownthatthereisalackofrecognitionofrisk factors(eg,polypharmacy,fallrisk,delirium)foradverse outcomesamongolderadultsintheED.2–4 Several assessmenttoolshavebeendevelopedintheEDtoevaluate riskfactors(eg,identificationofseniorsatrisk5),butthereis notooltoeffectivelycommunicatetheneedsof olderadults.6–8
The4Msframeworkofage-friendlyhealthsystemswas createdbytheJohnA.HartfordFoundationincollaboration withtheInstituteforHealthcareImprovement,American HospitalAssociation,andCatholicHealthAssociation.It incorporatesfourkeyelements: whatmatters,medication, mentation ,and mobility 9 Thesefourelementswere developedwithcurrentevidence-basedpracticeswiththe intentionofguidingclinicianconversationswitholderadults. The4Msframeworkconsidersthecommonriskfactorsfor adverseoutcomesinolderadultpatients(eg,riskfor delirium,potentiallyinappropriatemedications,and challengeswithmobility).
Whiletherehasbeenafocusonimplementingthe4Ms frameworkinacuteandambulatorycaresettings,therehave beenfewstudiesonitspotentialapplicationintheED.10 The EDisoneclinicalsettinginwhichitcouldbeimportantto discussthe4Mssothatallpotentialriskfactorsareidentified andcareistailoredtotheneedsofolderpatients.Thereis oftenatimeconstraintfortheemergencycliniciantoengage inalengthyconversationaboutwhatmatterstothepatient.
(“Whatmatters” entailsdiscussingthespecificdetailsthat mattertoeachpatientonadeeplevel,includingtheirgoals andpreferencesforcare,whichcanguidethecareteamand aligncaretowhattrulymatterstothepatient.)Useofthe 4Msframeworkcouldpotentiallypreventadverseoutcomes forolderadultpatientsreceivingcareintheEDby recognizingriskfactorssuchaspolypharmacy,fallrisk, anddelirium.
The4MsworksheetwasdevelopedbytheteamatAgeFriendlyCare,PA,aGeriatricWorkforceEnhancement ProgramatthePennStateRossandCaroleNeseCollegeof Nursing.Theworksheetisapatient-facingtoolthatallows individualstoidentifywhatmatterstothemandwhat questionstheymayhaveaboutpotentialproblemswith mobility,mentation,andmedications.Thetoolwas developedtohelpfacilitateconversationsbetweenpatients andclinicians,butitsusehasnotyetbeenevaluatedinthe ED(Appendix1).Weaimedtoevaluatethepotential usabilityofthe4MsworksheetintheEDtofacilitate conversationsaboutwhatmatters,medications,mention, andmobilitybetweenolderpatientsandemergency clinicians,andtoassesswhetherthe4Msworksheetmay supportemergencyclinicians’ understandingofpatients’ goalsofcare,includingbarriersandfacilitatorstousing theworksheet.
PopulationHealthResearchCapsule
Whatdowealreadyknowaboutthisissue?
Olderadultswithmultiplecomorbiditiesface highriskspost-discharge.The4Ms framework whatmatters,medication, mentation,andmobility isusedin varioussettings.
Whatwastheresearchquestion?
Cantheuseofthe4MsframeworkintheED settinghelpcliniciansunderstandolderadult patients ’ goalsofcare,andisitfeasible?
Whatwasthemajor findingofthestudy?
Ratesofconcordancebetweenpatientand clinicianonthemainconcern/goalofcareand underlyinggoalsofcarewere82.4%and 15.4%,respectively.
Howdoesthisimprovepopulationhealth?
Integrationofthe4Msframeworkin emergencycarecouldenhanceunderstanding andalignmentwitholderadults ’ underlying goalsofcare.
METHODS
StudyDesign
Weconductedthestudyusingaqualitative,descriptive approachinvolvingsemi-structuredinterviewswitholder adultpatients(aged ≥60years)andtheiremergencyclinician fromJanuary–June2022.Thisstudyispartofalargerstudy thatexaminedpatientgoalsreportedthroughthe4Ms worksheetusingthequalitativemethod.11–13 Inthepresent analysiswefocusedonusabilityoftheworksheetbypatients, aswellasemergencyclinicians’ understandingofpatients’ goalsofcare.11 Thelocalinstitutionalreviewboardapproved thisstudyanddeterminedittobeexempt.Weadheredtothe ConsolidatedChecklistforReportingQualitativeResearch (COREQ)guidelines.(Appendix2).
StudySetting
ThisstudytookplaceinasingleacademicEDwithLevelI traumaaccreditationandanannualcensusofapproximately 60,000patients.ClinicianspracticinginthisspecificED includeattendingphysicians,residents,andadvanced practiceproviders(APP).TheEDhasathree-year emergencymedicineresidencyprogramandan18-month fellowshipforAPPs;anyEMresidentorAPPwhomthe patientssawwouldbeenrolledinofoneofthesetraining
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 231 McKnightetal. Usabilityofthe4MsWorksheetintheED
programs.Theinstitutioniscurrentlyapartofthe age-friendlyhealthsystemsmovement.
4MsWorksheet
The4MsworksheetwascreatedbyAge-FriendlyCare, PAatthePennStateRossandCarolNeseCollegeof Nursing,whichisaCenterofGeriatricNursingExcellence. Theworksheetexplainseachcategoryofthe4Ms framework whatmatters,medications,mentation,and mobility aprogramhousedintheage-friendlyhealthcare systemandprovidesspaceforthepatienttowriteashort answerresponseabouteach.
ParticipantsandSampling
Participantswererecruitedforaninterviewif1)theywere aged ≥60years;2)theywerecurrentlyreceivingcareinthe ED;3)theirchiefcomplaintwasnotrelatedtoalteredmental status;4)theywereabletoreadandunderstandthe4Ms worksheetwritteninEnglish,and5)theyhadalreadybeen evaluatedbyaclinicianintheED.Patientswhomet eligibilitycriteriawereapproachedbyamemberofthe researchteamtoprovidestudyinformation.Consentwas obtainedfromallparticipantsbeforebeginning theinterview.
DataCollection
Twomedicalstudents,MMandMS whoweretrained bytheprimaryinvestigatorSLandbyDLwhohasextensive experiencewithqualitativeresearch conductedsemistructuredinterviewswithpatientsandclinicians.The researchteam(SL,MS)developedaninterviewguide (Appendix3).Interviewresponseswerecapturedas handwrittennotesinsteadofbydigitalrecordingduetocost. Theresearchteamcollectedpatientage,gender,andtypeof clinicianinterviewed(staffphysician,residentphysician,or APP).Interviewersalsotook fieldnotes,whichcontainedthe reasonforthevisitandcontextualfactors(symptomrelief, diagnostictest,disposition,non-verbalaspectsofthe interview).TheinterviewerenteredalldataintoREDCap,a secureelectronicdatacapturetoolshostedatTheUniversity ofIowaHospitalsandClinics.14 Nocompensationwas providedforinterviews.
Patientinterviewstookplaceintheroomwherethe patient’sEDevaluationtookplace.Theemergencyclinician wasnotpresentforthepatientinterview.First,thepatient completedthe4Msworksheeteitherindependentlyor verballytoamemberoftheresearchteam,followedby discussion.(Patientswerenotaskedwhethertheyhad completeda4Msworksheetinpriorhealthcareencounters.) DiscussionincludedpatientgoalsforthecurrentEDvisit, questionsabouteachcategorywithinthe4Msframework, andhowthepatientfelttheirvisithadgoneoverall.Atthe endoftheinterview,theinterviewerofferedthepatientto keepthe4Msworksheetforuseasareferenceinfuture
healthcareencounters.Patientinterviewslasted 30–60minutes(includingthetimespentcompletingthe4Ms worksheet).Afterthepatientinterviewwascomplete,the patient’sclinicianwasinterviewedabouttheirperceptionof thepatient’sgoalsofcareandhowthosegoalswereelicited duringpatientassessment.Clinicianinterviewsoccurred withoutthepatientpresentandlasted1–5minutes.
Analysis
InterviewdatastoredinREDCapwasanalyzedby researchteammembersSL,MM,MS,andDLusingcontent analysis.15 Eachinterviewwasinitiallycodedbytwo members(MS,MM),whoreceivedabrieftrainingon qualitativecontentanalysis.Twofacultyinvestigators(SL andDL)reviewedthesecodesandmadefurther recommendationbeforeconsensuswasreached.Theentire researchteammettodiscusscoding;anydiscrepancieswere resolvedthroughgroupdiscussion.Identifiedcodeswere enteredintoacodebook(Appendix4),whichwas maintainedandupdatedthroughoutdataanalysis.Codes weregroupedintocategoriesandsub-categoriestodescribe thedata.Datacollectionandanalysisfollowedaniterative processandoccurredsimultaneously,whichallowedfor revisionstotheinterviewguidetoaddressgapsinthedata. Interviewscontinueduntiltheresearchteamjointly determinedthatnonewinformationrelevanttotheresearch aimswasemerging.
Rigor
Datawasanalyzedbyaresearchteamwithavarietyof backgroundstoreduceindividualbiasandimprove credibilityoftheresults.16,17
RESULTS
Weapproached21patientstoconductsemi-structured interviews;onedeclinedduetounknownreasons.Intotalwe interviewed20patientsand19cliniciansduringthesixmonthperiod.Itshouldbenotedthatthereasonforthesmall samplesizeinasix-monthperiodwasduetointerviewers MMandMSalsohavingmedicalschoolduties.Allbutone patienthadacliniciantointerview;theoriginalclinicianfor thatonepatientwasnolongerintheED,andthenew clinicianwasunabletoanswerthequestions,asthepatient wasbeingdischarged.Nineattendingphysicians,eight residents,andoneAPPparticipatedinthestudy(Table1). Theinterviewprocessforpatienttookabout30–60minutes (median45minutes),includingtimeto filloutthe4Ms worksheet,and1–5minutes(median3minutes)tointerview emergencyclinicians.
Patientandclinicianinterviewsresultedinthreethemeson thetopicofunderstandingpatientgoalsofcare:clinician/ patientconcordance;understandingunderlyinggoalsof care;andunderlyinggoalsofcarediscrepancy(summarized in Table2).The4Msworksheetwasusedtofacilitate
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 232 Usabilityofthe4MsWorksheetintheED McKnightetal.
conversationsaboutwhatmatters,mentation,mobility,and medications,whichsupportunderstandingofpatients’ goals ofcare,includingbarrierstoandfacilitatorsofits use(Appendix4).
UnderstandingPatientGoalsofCare Clinician/patientconcordance
Inmanycases,patientsandcliniciansarrivedat concordantperceptionsofthegoalsofcare.Thiswas indicatedwhenpatientandclinicianagreedonthemain concernandgoalsofcareforthevisit,suchasinthe followingexamples:
Ininterview2,boththepatientandclinicianagreedthat themainconcernwasaddressingthepatient’sfeverand coordinatingcaretoaddressrecurrentfeversrelatedto chemotherapy;ininterview3,boththepatientandclinician agreedthatthegoalofcarewasevaluationafterfalland beingabletocontinuelivingindependently;ininterview4, boththepatientandclinicianagreedthatthegoalofcarewas addressingsymptomsofconstipationandabdominalpain;in interview9,boththepatientandclinicianagreedthatthe
mainconcernthatbroughtthepatienttotheEDwaschest pain;ininterview10,boththepatientandclinicianagreed thatthegoalofcarewasaddressingsymptomsoffatigue;and ininterview11,boththepatientandclinicianagreedthatthe goalofcarewasrulingoutseriouscardiacpathology.Inthis lastcase,boththepatientandclinicianidentifiedafearthat thepatient’schestpainmayhavepointedtoseriouscardiac pathologygiventhepatient’shistory.Theclinicianelicited theforebodingfeelingthatthepatientwashaving.
Ininterview14,boththepatientandclinicianunderstood thatthepatient’schestpainwaswhatmatteredmost.In interview15,boththepatientandclinicianwantedtoaddress abdominalpain.Ininterview16,boththepatientand clinicianagreedthatthemaingoalofcarewaspain management.Ininterview17,thepatientfeltthattheywere treatedwellandwhenaskedwhethertheclinicianhad addressedtheirconcerns,answered, “Yeah,everyonewas nice.” However,forthispatient,therewasnoclinician perspectivetocompareto.Ininterview20,thepatient reportedthatshewaskeptuptodate(onhercare)andfelt thattheEDcliniciandid “just fine ” inaddressinghergoalsof care,questions,andconcerns.
Overall,14of19patientsandcliniciansagreedonthe mainconcernorgoalsofcareforthevisit(Table3).Further, wegroupedtheseintosymptomevaluation(Interviews2,4, 11);symptommanagement(Interview16);symptom evaluation/management(Interview3);genericagreement (Interviews9,10,14);andmiscellaneous(interviews17,20).
UnderstandingUnderlyingGoalsofCare
Insomecases,thepatientandclinicianagreedon underlyinggoalsofcareforthepatient.Anunderlyinggoal ofcareisdefinedasaspectsofwhatmatterstothepatientin theirdailylifeorhealththataffecttheirmainconcernand goalofcare.Examplesareasfollows:Ininterview2,boththe patientandcliniciansuggestedthatarelatedgoalofcarewas tocoordinatewiththeteammanagingthepatient’scancerto developacareplangoingforwardthatwouldaddresstheir recurrentfeversandchemotherapyissues;andininterview3, boththepatientandclinicianagreedthatanunderlyinggoal ofcarewastobeabletocontinuelivingindependently. Overall,2of19patientsandcliniciansagreedonunderlying goalsofcare(Table3).
UnderlyingGoalsofCareDiscrepancy
Clinician/patient
Understandingunderlying
Underlyinggoalsofcare
Despiteagreeingonmainconcernsandgoalsofcare, patientsandcliniciansoftendidnotagreeonunderlying goalsofcare.Thiswasthecasewheninterpretationofwhat mattersforthepatientandclinicianwasdiscrepant,asshown intheseexamples:Ininterview5,theclinicianmentionedthat themaingoalofcarewaspainrelief,andthatthepatient wouldratherbeathomeand “soilhimself” thanbeatthe [deidentified]hospital.Thepatient’smaingoalofcarewasto workonphysicaltherapy,gainstrength,andgetoffsome
Variable N Percent(%) Patientage(years)
–70840
–80945
–90210
+ 15 Patientgender Male945 Female1155 Clinicianrole
Resident840
Unknown15
Table1. Patientandemergencycliniciandemographicdata.
60
70
80
90
Attendingphysician945
Advancedpracticeprovider15
Categories
goalsofcare Utilizationof4Msworksheet
Table2. Categoriesandsub-categoriesforunderstandingpatient goalsofcareandutilizationofthe4Msworksheet.
Understandingpatient
concordance
discrepancy
Worksheettodiscussion
goalsofcare Challengesusingtheworksheet
discrepancy
worksheetbeforedischarge Volume25,No.2:March2024WesternJournal of EmergencyMedicine 233 McKnightetal. Usabilityofthe4MsWorksheetintheED
Challengecompletingthe
Attendingn = 10,(%) response = yes
Resident(MD/DOorAPP) n = 8,(%)response = yes
APPn = 1,% response = yes Concordance (%)
medications.Thus,weidentifiedthemesofmedication concernsandmaintainingindependencefromthisinterview. Ininterview4,theclinicianidentifiedimprovingsymptoms asthemaingoalofcare,butthepatientidentifiedthemain goalasimprovingindependence.Thus,weidentifiedtheme ofmedicationconcernsfromthisinterview.
Duringthe19clinicianinterviews,17wereabletoidentify themainconcernthatbroughtthepatienttotheED.Ofthese 17clinicians,14(82.35%)mentionedamainconcernorgoal thatmatchedwiththepatient’sgoal.Thirteenclinicians mentionedanunderlyinggoalofcareforthepatient.Ofthese 13,onlytwoclinicians(15.38%)mentionedanunderlying goalofcarethatmatchedwiththepatient’sunderlyinggoals (Table3).Theresponsesonpatientperspective,main concern,andunderlyinggoalsofcareweresimilarbetween attendingandresidentphysicians.
Utilizationofthe4MsWorksheet
Implementationofthe4Msworksheetrevealedmultiple potentialbarrierstoandfacilitatorsofitsuse,including worksheettodiscussiondiscrepancy,challengesusingthe worksheet,andchallengecompletingtheworksheetbefore discharge(Table2).
WorksheettoDiscussionDiscrepancy
Insomecases,patients’ answerstopromptsonthe4Ms worksheetdidnotmatchinformationobtainedthrough discussionwiththeresearchteam,asshowninthese examples:Ininterview4,thepatientanswered “ no ” to medicationconcerns,butdiscussedmanyissuesrelatedto medications;ininterview3,thepatientdiscussedinformation relevanttodischarge/dispositionandrevealedopportunityto learnaboutmedicationsthatwasnotcapturedinthe worksheetanswers;andininterview20,thepatient wrote “ no ” tomedicationconcerns,buthadconcernsabout twooftheirmedications.Theseexamplescouldindicate thattherewasnotenoughspaceontheworksheetto providetheinformation,orthatthepatientdidnotcareto fill outtheworksheetindetail.Apossibilityalsoexists thatthepatientwasremindedofmoredetails throughdiscussionthattheydidnotthinkaboutbefore.
ChallengesUsingtheWorksheet
Manypatientspreferredaverbaldiscussionaboutwhat matteredtothemintheircareasopposedto fillingoutthe worksheet.Nineof20participantsdidnotfeelcomfortable with fillingoutthe4Msworksheet,andinterviewersoffered to fillitinforthem.Someappearedtobeuneasycompleting theworksheet,asshownintheseexamples:Patient2began fidgetingwiththeworksheetandexpresseddiscomfortwith fillingitout,statingthatuneasewithworksheetsextended backtobeinginschoolasachild;patient5couldnotreadthe questions,andtheinterviewerreadthemtothepatientand also filledouttheworksheetwiththeiranswers;patient11did notwantto fillouttheworksheetalonebutwashappyto allowtheinterviewertodoso;patient19hadParkinsonism and,therefore,wasunabletowriteontheworksheet;the interviewer filledouttheworksheetforthispatient.
ChallengeCompletingtheWorksheetBeforeDischarge
Therewerealsologisticalchallengeswithcompletingthe 4Msworksheet,includingnothavingenoughspaceonthe worksheettoadequatelyanswerthequestions,asthereare onlysomanylinesavailabletowriteundereachelement. Limitedtimewasanotherchallenge,asshowninthis example:Patient17’sinterviewwasperformedjustbefore theyweredischarged,soitfeltrushedasthepatientwas gettingreadytoleave.Theinterviewerandpatientwerealso interruptedtwiceduringtheinterview.
Overall,theuseofthe4Msworksheetrequiredadditional personneltohelpinterpretquestionsand filloutquestion items.Anydowntimewasusedto finishthissentence,which providedopportunitytocompletetheworksheet.
DISCUSSION
Wefoundthatemergencyclinicianshaveagood understandingofproblem-orientedgoalsofcarebutnot underlyinggoals.Further,thepotentialusability of4Msworksheettofacilitatetheconversationbetween patientsandemergencycliniciansfaceschallenges.A successfulimplementationofthe4Msframeworkin theEDiskeyinintegratingemergencycareintothe age-friendlyhealthsystem.Themesweidentifiedhighlight
Table3. Clinicianunderstandingofpatientgoalsofcare.
Patientperspectiveonwhether clinicianunderstoodgoalsofcare 4(40)5(62.5)NANA Wasthemainconcernaddressedby clinician? 7(70)6(75)1(100)82.35 Wastheunderlyinggoalofcare addressedbyclinician? 2(20)0(0)0(0)15.38
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 234 Usabilityofthe4MsWorksheetintheED McKnightetal.
APP,advancedpracticeprovider.
theuniqueaspectof4Msandtheworksheettofacilitate suchcare.
Intermsofunderstandingpatientgoalsofcare,we identifiedclinician/patientconcordance,understanding underlyinggoalsofcare,andunderlyinggoalsofcare discrepancy.Inmostinterviews,patientsandclinicians agreedonaproblem-orientedgoalrelatedtothepatient’ s reasonforpresentingtotheED.However,wefoundthat mostemergencycliniciansdidnotevokethepatient’ s underlyinggoalsforthevisit.Theliteratureonusingthe4Ms frameworktoelicitgoalsofcareamongolderadultsinthe EDislimited.Onestudyfoundthatwhendiscussing “what matters” tothepatient,emergencycliniciansandpatients agreedthatdischarginghomeorreduction/resolvingof symptomswasahighpriority,butemergencycliniciansoften didnotidentifythepatients’ desiretoreturntoprior functionalability.18 Ourstudy findingsaresimilarinthat manypatientsinterviewedhadunderlyinggoals,butthese goalswerenotdescribedbytheclinician.
Intermsofbarriersinusingthe4Msworksheet,we identifiedworksheet-to-discussiondiscrepancy,challenges usingtheworksheet,andchallengescompletingthe worksheetbeforedischarge.Nearlyhalfofthepatientsdid notfeelcomfortable fillingouttheworksheetandrequired theinterviewertoassistthemindoingso.Thiswasduetoa varietyofreasons.Ininstancesinwhichthepatientsdid fill outtheworksheet,therewasoftenadiscrepancybetween whatwaswrittenontheworksheetandwhatwasdiscussed duringtheinterview.Inoneinstancetheusageofthe worksheetanddiscussionfeltrushedduetothepatient’ s impendingdischarge.Inthesecases,patientsmaynothave fullydescribedtheirgoals,andinterviewersmaynothave askedmorein-depthquestionsabouttheirgoals.10
Thereislimitedliteratureontheuseofthe4Msworksheet inthecontextoftheED.Onepapersuggestedusingateam approachtoevaluatingthe4MsinelderlypatientsintheED (eg,pharmacistsshoulddiscussmedications,andsocial workersshoulddiscussmobility).Anotherstudyused transitionalcarenursesintheEDtoevaluateelderlypatients forcognitionandmobilityandfoundthatusingsuchcare nursesdecreasedadmissionsandreadmissionstothe hospital.19,20 Ourstudyisuniqueinthatweusedaworksheet basedonthe4Msframeworkandevaluateditsfeasibilityfor useintheED.Giventheamountoftimethatthediscussions take,wesuggestusingateamapproachorhaveadedicated persontohave4Msdiscussionswithpatients.
StrengthsandLimitations
Strengthsofourstudyincludethein-depthdiscussionwith patients,whichallowedustounderstandtheirgoalsand whatmatterstothem.Anotherstrengthisthatwehad multiplemembersoftheresearchteamcodingthesame interviews;thisallowedustoaddmoreobjectiveanddiverse pointsofview.However,therewerealsoseverallimitations
toourstudy.Thesamplesizewaslimited,andwewere unabletorecruitasufficientnumberofAPPs.Sincewe recruitedthesesubjectsduringactiveclinicalcare,thetime thatphysicians/APPshadforthisinterviewwasabout1-5 minutes,whichmayhavecausedbias.Patientswereenrolled fromasinglecenter,limitingtransferabilitytootherhealth systemsandgeographicregions.Theinterviewswerenot recorded;soverbatimquoteswerenotpossibleinallcases, whichmayhavecausedrecallbias.Also,wedidnothave accesstodemographicvariables,againaffecting transferabilityof findings.Therewasnoquantitative measurementofdiscrepancyinthecodingresults.
FutureImplications
Implicationsofuseofthe4Msworksheetforclinicians includeincreasedtimespentwithpatientsandgreaterpatient satisfaction,butalsoincludesincreasedprobabilityoffalling behindinpatientcare.Implicationsofuseofthe4Ms worksheetforpatientsincludeincreasedsafetyandneeds beingmet,potentialavoidanceofhospitaladmission,and improvedpatientoutcomes.Itwouldbeinterestingtosee whetherearlyuseofthe4MsworksheetintheEDcourse withsubsequentavailabilityforEDcliniciansallowsgreater concordanceinthegoalsofcare.Theuseofthe4Ms frameworkforemergencycareisnotfullydeveloped,andthe useoftheworksheetcanfacilitatesituation-specificcare(eg, dischargeplanning).
CONCLUSION
Wefoundthatusingthe4Msframeworkasaguideforthe careofolderadultpatientsintheEDcanhelpelicit underlyinggoalsofcare.Wewereabletoanswerwhether patients’ goalsofcarewerecongruentwithwhatthe emergencyclinicianbelievedthepatients’ goalswererelated tothepresentingproblemandwiththepatients’ underlying goals.Ourstudyalsofoundthattheuseofthe4Msworksheet intheEDneedsmoreresearchonhowtobestincorporateit intothecareofolderadultpatients,asmanyolderadults mayneedadditionalassistanceto fillitout.Wesuggestthat the4Msworksheetcanbeusedwitholderpatientswho presenttotheEDtoguideconversationswithclinicians.This studyispreliminary,andrequiresavalidationstudyto furthertesttheworksheet’sutilityandacceptabilityin theED.
AddressforCorrespondence:SangilLee,MD,MS,Universityof Iowa,RoyJ.andLucilleA.CarverCollegeofMedicine,Department ofInternalMedicine,200HawkinsDR,IowaCity,IA52242.Email: sangil-lee@uiowa.edu
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 235 McKnightetal. Usabilityofthe4MsWorksheetintheED
and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.
Copyright:©2024McKnightetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.AminzadehFandDalzielWB.Olderadultsintheemergency department:asystematicreviewofpatternsofuse,adverseoutcomes, andeffectivenessofinterventions. AnnEmergMed. 2002;39(3):238–47.
2.LaMantiaMA,MessinaFC,HobgoodCD,etal.Screeningfordeliriumin theemergencydepartment:asystematicreview. AnnEmergMed. 2014;63(5):551–60.e2.
3.CarpenterCR,AvidanMS,WildesT,etal.Predictinggeriatricfalls followinganepisodeofemergencydepartmentcare:asystematic review. AcadEmergMed. 2014;21(10):1069–82.
4.HusteyFM,WallisN,MillerJ.InappropriateprescribinginanolderED population. AmJEmergMed. 2007;25(7):804–7.
5.AsomaningNandLoftusC.Identificationofseniorsatrisk(ISAR) screeningtoolintheemergencydepartment:implementationusingthe plan-do-study-actmodelandvalidationresults. JEmergNurs. 2014;40(4):357–64.e1.
6.HanJH,ZimmermanEE,CutlerN,etal.Deliriuminolderemergency departmentpatients:recognition,riskfactors,andpsychomotor subtypes. AcadEmergMed. 2009;16(3):193–200.
7.HusteyFM,MeldonSW,SmithMD,etal.Theeffectofmentalstatus screeningonthecareofelderlyemergencydepartmentpatients. Ann EmergMed. 2003;41(5):678–84.
8.BatenV,BuschH-J,BuscheC,etal.Validationofthebriefconfusion assessmentmethodforscreeningdeliriuminelderlymedicalpatientsin
aGermanemergencydepartment. AcadEmergMed. 2018;25(11):1251–62.
9.CacchionePZ.Age-friendlyhealthsystems:the4Msframework. Clin NursRes. 2020;29(3):139–40.
10.BambachKandSoutherlandLT.Applyinggeriatricprinciplesto transitionsofcareintheemergencydepartment. EmergMedClinNorth Am. 2021;39(2):429–42.
11.SheberM,McKnightM,LiebzeitD,etal.335 “4Ms” conversationinthe emergencydepartment:aqualitativestudy. AnnEmergMed. 2022;80(4):S143.
12.SandelowskiM.Whateverhappenedtoqualitativedescription? Res NursHealth. 2000;23(4):334–40.
13.HsiehH-FandShannonSE.Threeapproachestoqualitativecontent analysis. QualHealthRes. 2005;15(9):1277–88.
14.GarciaKKSandAbrahãoAA.ResearchdevelopmentusingREDCap software. HealthcInformRes. 2021;27(4):341–9.
15.GaleNK,HeathG,CameronE,etal.Usingtheframeworkmethodfor theanalysisofqualitativedatainmulti-disciplinaryhealthresearch. BMCMedResMethodol. 2013;13:117.
16.SandelowskiM.Rigororrigormortis:theproblemofrigorinqualitative researchrevisited. ANSAdvNursSci. 1993;16(2):1–8.
17.SandelowskiM.Theproblemofrigorinqualitativeresearch. ANSAdv NursSci. 1986;8(3):27–37.
18.GettelCJ,VenkateshAK,DowdH,etal.Aqualitativestudyof “what matters” toolderadultsintheemergencydepartment. WestJEmerg Med. 2022;23(4):579–88.
19.HwangU,DresdenSM,RosenbergMS,etal.Geriatricemergency departmentinnovations:transitionalcarenursesandhospitaluse. JAm GeriatrSoc. 2018;66(3):459–66.
20.DresdenSM,HwangU,GarridoMM,etal.Geriatricemergency departmentinnovations:theimpactoftransitionalcarenurses on30-dayreadmissionsforolderadults. AcadEmergMed. 2020;27(1):43–53.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 236 Usabilityofthe4MsWorksheetintheED McKnightetal.
NationalCharacteristicsofEmergencyCareforChildrenwith NeurologicComplexChronicConditions
KaileenJafari,MD*†
KristenCarlin,MPH†
DeryaCaglar,MD*†
EileenJ.Klein,MD,MPH*†
TamaraD.Simon,MD,MSPH‡
SectionEditor:RonnieWaldrop,MD
*UniversityofWashington,DepartmentofPediatrics,Seattle,Washington † SeattleChildren’sResearchInstitute,CenterforClinicalandTranslational Research,Seattle,Washington
‡ UniversityofSouthernCalifornia,KeckSchoolofMedicine,Departmentof Pediatrics,LosAngeles,California
Submissionhistory:SubmittedJanuary25,2023;RevisionreceivedNovember16,2023;AcceptedNovember21,2023
ElectronicallypublishedFebruary9,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.17834
Introduction: Mostpediatricemergencycareoccursingeneralemergencydepartments(GED),where lesspediatricexperienceandlowerpediatricemergencyreadinessmaycompromisecare.Medically vulnerablepediatricpatients,suchasthosewithchronic,severe,neurologicconditions,arelikelytobe disproportionatelyaffectedbysuboptimalcareinGEDs;however,littleisknownaboutcharacteristicsof theircareineitherthegeneralorpediatricemergencysetting.Inthisstudyourobjectivewastocompare thefrequency,characteristics,andoutcomesofEDvisitsmadebychildrenwithchronicneurologic diseasesbetweengeneralandpediatricEDs(PED).
Methods: Weconductedaretrospectiveanalysisofthe2011–2014NationwideEmergencyDepartment Sample(NEDS)forEDvisitsmadebypatients0–21yearswithneurologiccomplexchronicconditions (neuroCCC).Wecomparedpatient,hospital,andEDvisitscharacteristicsbetweenGEDsandPEDs usingdescriptivestatistics.Weassessedoutcomesofadmission,transfer,criticalprocedure performance,andmortalityusingmultivariablelogisticregression.
Results: Therewere387,813neuroCCCEDvisits(0.3%of0–21-year-oldEDvisits)inoursample.Care occurredpredominantlyinGEDs,andvisitswereassociatedwithahighseverityofillness(30.1% highestseverityclassificationscore).ComparedtoGEDvisits,PEDneuroCCCvisitswerecomprisedof individualswhowereyounger,morelikelytohavecomorbidconditions(32.9%vs21%, P < 0.001),and technologyassistance(65.4%vs.45.9%)butunderwentfewerproceduresandhadlowerEDcharges ($2,200vs$1,520, P < 0.001).VisitstoPEDshadloweradjustedoddsofcriticalprocedures(adjusted oddsratio[aOR]0.74,95%confidenceinterval[CI]0.62–0.87),transfers(aOR0.14,95%CI0.04–0.56), andmortality(aOR0.38,95%CI0.19–0.75)comparedtoGEDs.
Conclusion: CareforchildrenwithneuroCCCsinapediatricEDisassociatedwithlessresource utilizationandlowerratesoftransferandmortality.IdentifyingfeaturesofPEDcareforneuroCCCscould leadtolowercostsandmortalityforthispopulation.[WestJEmergMed.2024;25(2)237–245.]
INTRODUCTION
Approximately30millionemergencydepartment(ED) visitsaremadebychildrenintheUnitedStatesannually.1 Previousstudieshaveshownthatmostpediatricpatients receiveemergencycareingeneralEDs(GED),whereless pediatricexperienceandlowerpediatricemergency
readinessmaycompromisecare.2,3 CareinGEDsmaybeless likelytofollowevidence-basedguidelinesforcommon pediatricconditionssuchasheadtrauma,croup,and asthma.Inaddition,GEDsmayhavehigherratesof potentiallyavoidabletransfers(giventhelikelihoodof transferwhenthepatientdidnotpresentinitially
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 237
ORIGINAL RESEARCH
toaspecialtyhospital)andhighermortalityin criticalillness.4–6
Childrenandyoungadultswithchronic,severeneurologic conditionsareamedicallyvulnerablepopulationwithpoor functionalstatusandhighratesofcomorbiddiseaseandneed fortechnologyassistance(ie,gastrostomytubeorventricular peritonealshunts).Thesecharacteristicsmakethis populationatriskoffrequentEDutilization,uniquemedical presentations,andhighacuityofillness.7–11 Priorstudies havefoundchildrenwithneurologicdiseaseaccountforup to13%ofinpatientpediatricadmissions,andonethirdof inpatientpediatrichealthcarecosts.7,8 AlthoughGEDs providemostemergencycareforchildren,emergency medicine(EM)traineesandGEDclinicianshaveless experienceandconfidenceinprovidingcaretopediatric patientscomparedtotreatingadults.12,13 Despitegrowing evidencethatchildrenwithchronicneurologicconditionsare atriskoffrequentEDusage,thereislimiteddatacomparing thedistribution,characteristics,andoutcomesofemergency careforthisspecificpopulationbetweenGEDsandpediatric EDs(PED).
Inthisstudyweaimedtodescribethenationalestimatesof childrenandyoungadultpatientswithneurologiccomplex chronicconditions(neuroCCC)andcomparethe characteristicsandoutcomesofEDvisitsinGEDsand PEDs.WehypothesizedthatPEDvisitswouldbeassociated withlowerratesofresourceutilization,transfer, andmortality.
METHODS
StudyDesignandDataSource
Weconductedacross-sectionalanalysistoidentifythe frequencyandcharacteristicsofEDvisitsamongpatients aged0–21yearswithneuroCCCsbetween2011–2014inthe NationwideEmergencyDepartmentSample(NEDS).The NEDSispartoftheHealthcareCostandUtilizationProject databasethatissponsoredbytheAgencyforHealthcare ResearchandQuality.Itisthelargestall-payer,nationally representativeEDdatabaseintheUnitedStatesandcontains a20%stratifiedsampleofhospital-basedEDs.Thesampling strategydeliberatelyencompassesbetween945–955hospitalbasedEDsin24–34statesandapproximately 120–137millionweightedvisitstotheEDannually.14 Hospital-basedEDsarestratifiedbyUScensusregion, traumadesignation,urban-rurallocation,hospital ownership,andteachingstatus.Additionally,asnopatient identifiersareavailable,individualpatientscannotbe trackedlongitudinally,andencountersthatoriginated inonefacilityandweretransferredwillhaveaseparate encounterinthereceivingfacility.Theunitofanalysisisthe EDvisit.ThestudywassubmittedtotheSeattleChildren’ s InstitutionalReviewBoard(IRB)andwasdetermined tobeIRBexempt.
PopulationHealthResearchCapsule
Whatdowealreadyknowaboutthisissue?
Childrenwithchronicneurologicdiseasesare atriskforsevereillnessandpooroutcomesin theemergencydepartment.
Whatwastheresearchquestion?
Doesemergencycaredifferbetweengeneral (GED)andpediatricEDs(PED)for childrenwithchronicneurologicdisease?
Whatwasthemajor findingofthestudy?
ChronicneurologicalpediatricvisitsinPEDs hadloweroddsofmortalityintheED(aOR 0.4,95%CI0.2 – 0.8)comparedtoGEDs (0.04%vs0.13%).
Howdoesthisimprovepopulationhealth?
IdentifyingfeaturesofpediatricEDcarefor childrenwithchronicneurologicconditions couldimprovemortalityinthis high-riskpopulation.
WeincludedallEDvisitsmadebypatients0–21yearsof agewithanInternationalClassificationofDiseases,9th Rev (ICD-9)diagnosisconsistentwithaneuroCCC.15 The classificationofCCCs,originallydevelopedbyFeudnterand colleagues,isanorgan-systembasedclassificationof complexdiseasesofchildhood.15 WeexcludedEDvisitsthat weremissingage,primarydiagnosis.ortheprimary diagnosiswasinvalid.
OurprimarypredictorvariablewasthecategoryofED, eitherGEDorPED.TheNEDSdoesnotspecifically designatePEDs;thus,consistentwithsimilarpublished reports,wedesignatedPEDsasthosewhere ≥75%ofvisits werechildren <18yearsofage.AllotherEDswere categorizedasGEDs.3,5,16–18 Ofnote,whilewedefinedPED baseduponproportionofvisitsmadebychildren <18years, weincludedencountersinthisstudyuptoage21,aspatients withchronicmedicalconditionsoftencontinuetoseekcare inPEDsintoyoungadulthoodastheytransitiontoadult care.2,3 Patient-levelpredictorsincludedthefollowing: 1)demographics,insurancepayer,medianincomequartile, andtheurban-ruralclassificationofthepatient’sareaof residence;2)neuroCCCdiagnosticcategory;3)numberof non-neuroCCCs);4)presenceoftechnologyassistanceand CCCICD-9codes.15 Weidentifiedspecifictechnologies, includingventricularshunts,feedingtubes(gastrostomy, gastro-jejunostomy,andjejunostomytubes)and tracheostomies,specificallyusingthecorresponding
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 238 EmergencyCareforChildrenwithChronicNeuroComplaints Jafarietal.
technologyassistancecodes.15 Hospital-levelpredictors includedteachingstatusoftheED,traumacenter designation,andhospitalregion.
Ourprimaryoutcomesofinterestwereresource utilization,severityofillnesspresentation,disposition (admissionandtransfer),andmortality.Resourceutilization wasassessedthroughEDcharges,frequencyofprocedures performed,anddiagnosticimaging.AstheNEDSprovides onlyfacilitycharges,andcost-to-chargeratioswerenot availablefortheyearsselected,wereporttotalchargesforthe EDandinpatientstay.Thisapproachisconsistentwithprior publishedstudies.19,20 Weusedtotalnumberofcurrent proceduralterminology(CPT)codes,ratherthanICD-9 proceduralcodes,toassessprocedurefrequencyasa significantlyhigherproportionofEDvisitshadCPTcodes available.Diagnosticimaging(includingradiograph, ultrasound,computedtomography(CT)ormagnetic resonanceimaging(MRI)andcross-sectional imaging(CTorMRIonly)wasreportedbasedontheCPT andICD-9proceduralcodesassociatedwiththe EDvisit.
Weassessedtheoutcomeofseverityofclinical presentationusingseverityclassificationscores(SCS)and criticalprocedureperformance.2,21 TheSCSisaPediatric EmergencyCareAppliedResearchNetworkconsensusderiveddiagnosticsystemthatreliesonthemostsevereICD9diagnosticcodesattachedtoeachrecordtoassigneachED visitaseverityscore.Theseverityscorerangesfrom1 (minimalresourcesused)to5(maximalresourcesused).21
CriticalproceduresweredefinedasthepresenceofanICD-9 codeforendotrachealintubation,centrallineplacement,and chesttubeplacementaspreviouslydescribedinthe literature.22,23 Mortalitywascategorizedas(1)EDmortality and(2)visitmortality(deathatanypointduringED visitorhospitalization).
StatisticalAnalysis
Weincorporatedsamplingweightstoconsiderthe significantsurveydesignandsamplingproceduresofthe NEDS.Descriptivestatistics,includingfrequencies, proportionsandsumsasappropriate,wereusedto summarizepatientandhospitalcharacteristics.Wemade comparisonsusingchi-squareorANOVAtestforcategorical variables,and t -testsforcontinuousvariables.Multivariable logisticregressionwasperformedfor fivedifferentED outcomevariables(admission,transfer,transferor admission,mortality,andcriticalprocedureperformance). Predictorvariablesincludedinlogisticregressionwere patient-levelvariables(demographics,numberofCCCs, technologyassistance,SCSscore),andhospital-level predictors(traumacenterdesignation,geographic location,PEDvsGED).Resultswerereportedas adjustedoddsratios(aOR)and95%confidence intervals(CI).
RESULTS
Oftheestimated141millionweightedEDvisitsmadeby patientsaged0–21yearsinthe2011-2014NEDS,387,987 (0.3%)hadaneuroCCCdiagnosis.MostneuroCCCED visitsoccurredinGEDs(74.9%),andtheremainderoccurred inPEDs(25.1%).NeuroCCCsvisitsrepresented proportionatelymoreofall0–21-year-oldPEDvisits comparedtoGEDvisits(1.0%vs0.2%).Thepatient-level characteristicsofneuroCCCEDvisitsareshownin Table1. Youngerpatients(ages ≤9years)represented proportionatelymoreofPEDthanGEDvisits(63vs48%, P < 0.01).TherewasapredominanceofmalesinbothGED andPEDvisits(55.8%vs55.9%).Theprimarypayerformost GEDandPEDvisitswasMedicaid(56.1%vs60.6%),and incomequartile(notshown)wasnotsignificantlydifferent betweenGEDsandPEDs.
Therewasahighrateofcomorbidchronicconditions overallinchildrenwithneuroCCCs,withoneinfourED visitsassociatedwithatleastonenon-neurologicCCC (93,075,24%, Table1).Themedicalcomplexityofneuro CCCvisitswashigherinPEDscomparedtoGEDs,with 32.9%ofpatientsinPEDswithatleastoneadditionalCCC comparedto21%inGEDs.Technologyassistancewasmore frequentinPEDthanGEDencounters(65.5%vs49.5%)and wascomprisedmostlyofventricularshunts(161,868, 41.7%),feedingtubes(56,568,14.6%)and,lesscommonly, tracheostomies(17,653,4.6%). SupplementalTable1
demonstratesthefrequencyofsubcategoriesofneuroCCCs andthemostcommoncategoriesofnon-neuroCCCs.
Hospitalcharacteristicsarealsoshownin Table1.Over 80%ofPEDvisitswereinmetropolitanlocations,teaching facilities,andLevelI/IItraumacenters,consistentwith underlyingdifferencesbetweenthesetwocategoriesof EDs.2,18 Regionally,PEDvisitswerepredominantlyfrom theWest,whileGEDvisitswerepredominantlyfromthe South.TheNortheastregionaccountedforthelowest proportionofvisits,18.9%ofneuroCCCEDvisitsoverall andonly1.4%ofPEDvisits(datanotshown).
CharacteristicsofemergencyvisitcareforneuroCCCED visitsaredemonstratedin Table1.Severeillness presentationswerecommon;30.1%ofvisitshadaSCS5 indicatingcriticalillness.ThePEDshadfewerSCS5 presentationsthanGEDs,(26.4%vs31.3%vs. P = 0.002), andmoreSCS4presentationsthanGEDs(55.8%vs.47.5%, P =< 0.001).PEDsvisitshadfeweroverallprocedures performed(0proceduresperformedin34.9%PEDvs13.4% inGEDs(P = 0.048)andlessimaging(45.7%vs24.4%,not significant).Endotrachealintubationwasthemost frequentlyperformedcriticalprocedureandoccurredless frequentlyinPEDscomparedtoGEDs(5.7%vs8.3%, P = 0.003).
MedianEDchargesweresignificantlylowerinPEDs comparedtoGEDs(P < 0.001).VisitstoPEDS,hadhigher proportionofadmissions(55.1%vs42.8%, P < 0.001)and
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 239 Jafarietal. EmergencyCareforChildrenwithChronicNeuroComplaints
N(%)GeneralED(n = 290,641)PediatricED(n = 97,346)AllEDvisits(n = 387,987)Pvalue
Patientcharacteristics
Ageinyears
0–9139,249(47.9%)61535(63.2%)200,706(51.7%) <0.001
10–1779,886(27.5%)29,138(29.9%)109,024(28.1%)
18–2171,505(24.6%)6,712(6.9%)78,257(20.2%)
Urbanicity
Largemetro142,792(49.1%)72,560(74.5%)215,353(55.5%)0.18
Medium/smallmetro100,482(34.5%)15,978(16.4%)116,460(30.0%)
Non-metro/unknown47299(16.3%)8808(9.0%)56107(14.5%)
Primarypayer
Medicaid162,965(56.1%)58,950(60.6%)221,916(57.2%)0.50
Privateinsurance98780(34.0%)30,832(31.7%)129,612(33.4%)
Medicare/other28,807(9.9%)7393(7.6%)11,868(9.4%)
Complexity
>1additionalCCC69,038(21.0%)32,037(32.9%)93,044(24.0%) <0.001
Technologyassistance143,788(49.5%)63,707(65.45%)207,495(53.5%) <0.001
Hospitalcharacteristics
Teachinghospital195,506(67.3%)97,035(99.7%)292,542(75.4%) <0.001
Traumacenter(I/II)165,244(56.9%)90,905(93.4%)256,149(66.2%) <0.001
Largemetrolocation150,888(51.9%)80,818(83.0%)231,744(59.7%)0.015
Visitcharacteristics
Disposition
Admission124,350(42.8%)53,659(55.1%)178,008(45.9%) <0.001
Transfer27,392(9.4%)1,242(1.3%)28,633(7.4%) <0.001
DeathintheED380(0.13%)47(0.04%)427(0.11%)0.003
Criticalprocedures1
Endotrachealtube20,059(8.3%)4,220(5.7%)24,278(7.7%)0.003
Centralvenousline11,210(4.6%)3,873(5.3%)15,115(4.8%)1,763(0.6%)0.32
Chesttube1,511(0.6%)255(0.3%)77,574(20.0%)0.004
Severityclassificationscore
<360,545(20.8%)17,030(17.5%)192,480(49.6%)0.007
4138,084(47.5%)54,357(55.8%)116,823(30.1%) <0.001
591,060(31.3%)25,762(26.4%)$2,031(1170–3743)0.002 EDcharges,median(IQR)$2,200(1237–3943)$1,520(873–2783)$2,031(1170–3743) <0.001
1Criticalproceduresincludedbothcurrentproceduralterminology(CPT)andInternationalClassificationofDiseases,9thRev(ICD-9) procedurecodes.VisitswithCPT/ICD-9procedurecodeslisted,Totaln = 367,108;generalEDn = 241,401;pediatricEDn = 73,947. ED,emergencydepartment; CCC,complexchronicconditions; IQR,interquartilerange.
lowerproportionoftransfers(1.3%vs9.4%, P < 0.001).In thecombinedoutcomeofadmissionortransfer,therewere nosignificantdifferencesbetweenPEDsandGEDs(52.2% vs56.4%, P = 0.09).DeathintheEDwasaninfrequent outcome,representingonly0.11%ofvisits.However,ED
mortalitywaslowerforPEDvisitscomparedtoGEDvisits (0.02%vs0.11%, P = 0.003).Visitmortality(deathatany pointduringEDorinpatientstay)wassimilarly lowerforPEDvisitscomparedtoGEDs (1.27%vs2.39%, P = 0.003).
Table1. Selectedpatient,hospital,andvisitcharacteristicsinvolvingvisitstogeneralandpediatricemergencydepartmentsfor neuromuscularcomplexchronicconditions.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 240 EmergencyCareforChildrenwithChronicNeuroComplaints Jafarietal.
Jafarietal. EmergencyCareforChildrenwithChronicNeuroComplaints
Table2 summarizesresultsofourlogisticregression modelstoexploretherelationshipbetweenthecategoryof EDandvisitdispositionandcriticalprocedureperformance. ThePEDvisitshadsignificantlyloweradjustedoddsof transfercomparedtogeneralEDs(aOR0.14,95%CI 0.04–0.56).Conversely,PEDshadasignificantlyhigher adjustedoddsofadmission(aOR1.52,95%CI1.19–1.96). Additionalpredictorsinadmissionandtransfermodels includedpresenceofnon-neurologicCCCs,increased severityofillness,rural/fringemetropolitanresidences,and inthosewhoseinsurancewasself-pay.Forthecombined outcomeofadmissionortransfer,therewasnosignificant differencebetweenPEDsandGEDs(aOR1.07,95% CI0.86–1.33).
Intheadjustedmodelsofcriticalprocedures(Table3), PEDvisitshadloweroddsofcriticalprocedures(aOR0.74, CI0.62–0.87)comparedtoGEDvisits.Overall,increased severityofillnesswasassociatedwithadramatically increasedoddsofcriticalprocedures(aOR11.9,95%CI 10.3–13.6).Non-neurologicCCCswerealsoassociatedwith increasedofcriticalproceduresperformance(aOR1.51,95% CI1.42–1.59).TheEDvisitsinwhichapatienthada tracheostomyhadlowerodds(aOR0.48,95%CI0.38–0.59) ofcriticalprocedureascomparedtothosewithouta tracheostomy;otherformsoftechnologyassistancewerenot significantlydifferent.
ThelogisticmodelsforEDmortalityandvisitmortality arealsoshownin Table3.Severityofillnessscoreswerenot
includedintheadjustedmodelsofmortality,astherewas collinearitywiththisvariableandtheoutcome.Theadjusted oddsofEDmortalitywassignificantlylowerforPEDvisits (aOR0.37,CI0.19–0.73)comparedtoGEDvisits.Patients withventriculorperitonealshuntshadaloweradjustedodds ofmortalitycomparedtothosewithout.Similarly,inthe modelofoverallvisitmortality,PEDvisitshadalowerodds ofvisitmortalitycomparedtoGEDs(aOR0.62, P < 0.001). Thepresenceofnon-neurologicCCCswaspredictiveof increasedoddsofmortality,whileallformsoftechnology assistancehadloweradjustedoddsofvisitmortality.
DISCUSSION
InthisnationalsampleofEDvisits,weestimate387,000 annualEDvisitsweremadebypatientsaged0–21yearswith neurologiccomplexitybetween2011–2014.NeuroCCC patientshadhighratesofmedicalcomplexityandtechnology dependence,andoftenpresentedwithsevereillnesstothe ED.Mostoftheemergencycareforthispopulationoccurred inGEDs,wherevisitshadhigherratesofdiagnostictesting, criticalprocedures,andED-associatedcharges.After adjustmentfordifferencesindemographics,comorbidities, andseverityofillnesspresentation,GEDshadhigherratesof transfer.However,therewerenosignificantdifferences betweenGEDsandPEDsinacombinedmodelofadmission ortransfer.Adjustedoddsofcriticalprocedureperformance, EDmortality,andoverallvisitmortalitywerehigherin GEDscomparedtoPEDs.
–1.00)
Femalegender0.93(0.89–0.97)1.06(1.00–1.13)0.95(0.92–0.99)
Insurancepayer
PrivateinsuranceReferentReferentReferent
Medicaid0.85(0.78–0.93)1.10(0.99–1.23)0.88(0.82–0.95)
Medicare0.88(0.71–1.10)0.84(0.57–1.24)0.86(0.69–1.07)
Urbanicity
CentralmetroReferentReferentReferent
Smallmetro0.74(0.58–0.96)2.33(1.75–3.09)0.96(0.78–1.18)
Non-metro0.79(0.54–0.89)2.98(2.28–3.9)1.01(0.84-1.21)
Additionalnon-neurologicCCCs2.56(2.31–2.86)0.43(0.38–0.49)2.25(2.05–2.47)
Feedingtube1.30(1.17–1.45)0.82(0.69–0.98)1.23(1.1–1.37)
Ventricularshunt1.01(0.89–1.14)0.54(0.46–0.63)0.84(0.76–0.93)
Tracheostomy0.59(0.48–0.71)2.8(2.11–3.70)0.71(0.6–0.85)
Severityclassificationscore2.52(2.40–2.64)1.9(1.75–2.06)3.0(2.86–3.31)
PediatricED1.52(1.19–1.96)0.14(0.04–0.56)1.07(0.86–1.33)
OR,oddsratio; CI,confidenceinterval; CCCs,complexchronicconditions; ED,emergencydepartment.
Admissionadjusted OR(95%CI) Transferadjusted OR(95%CI) Admissionortransfer adjustedOR(95%CI) Age1.01(1.01–1.02)0.94(0.93–0.95)0.99(0.98
Table2. Logisticmodelsforoutcomeofadmission,transfer,andthecombinedoutcomeofadmissionortransferforemergencydepartment visitsforneurologiccomplexchronicconditions.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 241
–
Femalegender0.81(0.76–
Insurancepayer
PrivateinsuranceReferentReferentReferent Medicaid1.03(0.96–1.11)1.04(0.65–1.67)0.66(0.58–0.75)
–1.15)1.09(0.15–8.24)0.19(0.09–0.36)
Urbanicity
CentralmetroReferentReferentReferent
Smallmetro0.81(0.70–0.94)1.12(0.54–2.31)0.94(0.74–1.12)
Non-metro0.65(0.56–0.75)2.0(0.89–4.54)0.90(0.72–1.12)
Non-neurologicCCCs1.51(1.42–1.59)0.86(0.55–1.33)2.73(2.54–2.94)
Feedingtube0.92(0.83–1.02)1.34(0.49–3.67)0.23(0.18–0.30)
Ventricularshunt0.91(0.81–1.02)0.34(0.18–0.64)0.33(0.28–0.40)
Tracheostomy0.48(0.38–0.59)2.65(0.77–9.17)0.22(0.15–0.31)
Severityclassificationscore11.81(10.28–13.56)N/AN/A
PediatricED0.74(0.62–0.87)0.38(0.19–0.75)0.62(0.46–0.83)
OR,oddsratio; CI,confidenceinterval; CCCs,complexchronicconditions; ED,emergencydepartment.
Priorresearchhasdemonstratedthatchildrenwith complexchronicillnesseshaveahighriskofcriticalillness andpooroutcomesduringemergencies.17,18,24,25 Ourstudy addstotheliteraturebydemonstratingthatamonga nationalsampleofneuroCCCEDvisits,anestimated30% presentedwithcriticalseverityofillness(SCS5)and7.7% requiredendotrachealintubation.Incontrast,inanational sampleofall-comerPEDvisitsintheNEDSinwhichonly 5%ofpatientshad ≥1complexchroniccondition,only0.5% ofvisitshadacriticalseverityscoreof5andonly0.15% requiredintubation.2 Thecomparativelymuchhigher severitymeasuresfoundinourstudypopulationfurther supportsthehigh-acuityEDneedsamongchildrenwith neurologiccomplexityascomparedtoageneral pediatricpopulation.
Ourworkdemonstratesthatmostemergencycarefor childrenwithneuroCCCsoccursinGEDsratherthan specializedpediatriccenters,congruentwithprior characterizationsofemergencycareforchildrenwithCCCs.2
Priorresearchusingnationallyrepresentativedatasetshas shownthatPEDsmayperformbetteracrossseveralqualityof-caremetrics,includinglessdiagnostictestinginasthma, fewerantibioticsforviralinfections,andlowermortalityin criticalillnessessuchascardiacarrestandsepsis.4,5,25 Our studyexpandsuponthisexistingliterature,bycharacterizing thedisparitiesincharacteristicsofemergencycare inGEDsforchildrenwithchronicneurologicdiseases.
These findingssuggesttheremaybesomespecific benefitstoPEDcareforcertainhigh-risk,medically fragilepopulations,suchasthosewith neurologiccomplexity.
SomeofthedifferencesweobservedbetweenGEDsand PEDsmaybeduetounmeasuredinfluencesofapre-transfer evaluationandstabilization.Asthisdatasethasnopatient identifiersanddoesnotallowforlongitudinalassessmentof patientcare,wewereunabletoidentifywhichEDvisitswere self-referredvstransferredfromanotherED.However, amongthe28,633transferredencountersinthisstudy,27,392 (95.7%)originatedinaGED,anditislikelymanyofthese encountersweretransferredtoaPED.Oncethesepatients reachedthereceivingfacility,theylikelyhadreduced requirementsforadditionaldiagnostictestingorcritical interventions,whichcouldaccountforthecomparatively lowerEDcostsandprocedurefrequencyweobservedinPED encounters.Additionally,otherfactorsrelatedtothetransfer processmayhaveinfluencedprocedureratesinGEDs.For instance,referringemergencyphysiciansmighthavechosen tointubatepatientswithahigherriskofrespiratory decompensationbeforethetransfer,potentiallycontributing totherelativelyhigherintubationfrequencyseeninGEDs. Togainadeeperunderstandingoftheoriginsoftheobserved variationsinEDcostsandoutcomesbetweenGEDsand PEDs,futurestudiesincorporatinglongitudinalpatientdata areneeded.
Criticalprocedureperformance adjustedOR(95%CI) EDmortalityadjusted OR(95%CI) Visitmortalityadjusted OR(95%CI) Age1.0(0.994
1.005)0.97(0.94
1.0)1.03(1.02
Table3. Logisticmodelsfortheoutcomesofcriticalprocedureperformance,deathintheemergencydepartmentordeathatanypointinthe visit(visitmortality).
–
–1.04)
1.0)0.76(0.69
0.85)0.63(0.39–
–0.84)
Medicare0.85(0.62
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 242 EmergencyCareforChildrenwithChronicNeuroComplaints Jafarietal.
ItisworthnotingthatEDvisitsmadebychildrenwith neuroCCCshadhighratesoftechnologyassistanceoverall, andinourlogisticmodelspatientswithtracheostomyand ventricularshuntshadloweroddsofvisitmortality.
Althoughtechnologyassistanceinlargepopulationstudies ofall-comerpediatricpatientshasbeenidentifiedasarisk factorforsevereillnessandmortality,inchildrenwith neurologicdiseasesthereisevidencethattechnology assistancemaybeprotective.26,27 Ina2019Canadianstudy ofchildrenwithmedicalcomplexity,technologyassistance wasassociatedwithloweroddsofvisitmortalityinchildren withneurologicimpairmentandthosewithmultipleCCCs.8 Similarly,a2015analysisfromHongKongfoundthatin childrenwithsevereneurologicdiseases,tracheostomywas associatedwithloweroddsofmortality.28 Additionally, inherentdifferencesinthetypeofneurologiccomplexity betweenpatientswithandwithouttheseformsoftechnology assistancemayhelpexplaintheobserveddifferences inmortality.
Wehypothesizethehigherratesoftransferandlowerrates ofadmissioninGEDsarelikelysecondarytolimited inpatientpediatriccapabilitiesatthesecenters,thus necessitatingtransfer.6,7,29 Thishypothesisissupportedby our findingthatthecombinedoutcomeofadmission andtransferinourlogisticmodelsshowednodifferences betweenGEDsandPEDs.Thereisincreasingevidence thatpediatricinpatientcareisincreasinglylimitedin communityhospitals,resultinginincreasedregionalization ofhospitalpediatriccare.30 Thisislikelytobeparticularly trueforchildrenwithneurologiccomplexity,whomay requirespecialistconsultationonlyavailablein pediatriccenters.
These findingshaveimportantimplicationsforthe deliveryofpediatricemergencycaretomedicallyvulnerable patientsintheUnitedStates.Despitetheincreasing regionalizationofinpatientpediatriccare,emergencycare forchildrenislikelytocontinuetooccurpredominantlyin GEDsgiventhegeographiclimitationsinaccessto specializedpediatricemergencycentersformanypatients. Thus,ensuringadequateeducationandpreparationfor emergencyconditionsincomplexpediatricpatientsin communityandruralEDsiscritical.Experiencecaringfor criticallyill,medicallycomplexpediatricpatientsislacking formanyEMtraineesandrepresentsatargetforongoing educationalefforts.12,13
Simulationinterventions,suchasthosedeliveredbythe IMPACTSnetwork.areanotherpossibleinterventionto helpimprovethecareofthiscomplexpopulationbynonpediatriccliniciansincommunityEDsettings. 31 Pediatric emergencytelemedicinemaybeanotherpotentialstrategy toimprovethequalityofcarereceivedbycomplex pediatricpatientsinGEDs.Improvementsinthis technology,wideravailabilityoftelemedicineclinicians, andincreasingacceptanceofthisformatofcare
mayultimatelyaddressdisparitiesinaccesstocareby makingspecializedpediat ricemergencyphysicians moreavailable. 32 – 34
LIMITATIONS
Thisstudyhasseveralimportantlimitations.First,we useddatafrom2011–2014,whichmayimpacthow translatablethese findingsaretothepresent.Increasing regionalizationofcareinthelast10yearsmayhaveimpacted overalldistributionofpediatricneuroCCCEDcarebetween GEDsandPEDsandpotentiallyanincreasedfrequencyof transfers.Increasedeffortstowardpediatricreadinessin GEDsduringthistimeframecouldalsohaveimproved criticalillnessoutcomesinsomeGEDs.Additionally,this workreliesonlargeamountsofadministrativedata,whichis susceptibletoerrorsindataprocessingandvariabilityin coding.WeusedICD–9codestoidentifythepopulationof neuroCCCvisits,andtheICD–9codesascribedtoan encounteronlypertaintothecurrentlyrecordedED diagnosesandmaynotrepresentallpre-existingconditions. Thus,thisworklikelyunderestimatesthetruefrequencyof neuroCCCEDvisits,particularlyforloweracuitytreatand-releasevisits.2
Additionally,usingtheproportionofpediatricpatients seenwithinanEDtodeterminePEDdesignationhasitsown limitations.Specifically,ifaPEDandGEDare financially linked(commoninacademicinstitutionsthatsharethesame campus)thevisitsfromthesetwoinstitutionswilloftenbe groupedasasinglehospitalintheNEDS.Thisresultsin somePEDsbeinggroupedtogetherwithGEDs,usingour categorizationsystem.Giventhecollinearityoftheoutcome ofmortalitywithSCS,wedidnotincludethisinour modeling,andthusdifferencesinmortalitybetweenGEDs andPEDsmayinpartbeduetounmeasureddifferencesin severityofillness.Lastly,mortalitywasoverallaninfrequent outcome,andthusanybroadinterpretationsofthis finding shouldbetakenwithcaution.
CONCLUSION
Thisstudyisthe firstinourknowledgetodescribethe nationalstateofemergencycareforchildrenwithneurologic complexchronicconditionsinbothpediatricandgeneral EDs.Our findingsdemonstratethatmostemergencycarefor childrenwithneuroCCCsoccursinGEDs,andthatGEDs hadhigherratesofproceduresandcharges,transfers,and mortalityascomparedtoPEDs.Asthesepatientsarelikely tocontinuetopredominantlyreceiveemergencycarein GEDsettings,interventionstoensureappropriatetraining andpreparationofgeneralemergencyphysiciansforchildren withneurologiccomplexityisneeded.Additionally,further researcheffortstoexploretheimpactofpediatricemergency telemedicinesupportonimprovedqualityofcarefor medicallycomplexpatientsisneeded.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 243 Jafarietal. EmergencyCareforChildrenwithChronicNeuroComplaints
AddressforCorrespondence:KaileenJafari,MD,Universityof Washington,DepartmentofPediatrics,4800SandPointWayNE, M/SMB.7.520,Seattle,WA98105.Email: kaileen.jafari@ seattlechildrens.org
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.
Copyright:©2024Jafarietal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.MooreBJ,StocksC,OwensPL.Trendsinemergencydepartmentvisits, 2006–2014.HCUPStatisticalBrief#227.2017.Availableat:~https:// hcup-us.ahrq.gov/reports/statbriefs/sb227-EmergencyDepartment-Visit-Trends.jsp#:~:text=Overall%2C%20the%20number %20of%20ED,U.S.%20population%20grew%206.9%20percent December1,2020.
2.MurtaghKurowskiE,ByczkowskiT,Grupp-PhelanJM.Comparisonof emergencycaredeliveredtochildrenandyoungadultswithcomplex chronicconditionsbetweenpediatricandgeneralemergency departments. AcadEmergMed. 2014;21(7):778–84.
3.MichelsonKA,LyonsTW,HudginsJD,etal.Useofanationaldatabase toassesspediatricemergencycareacrossUnitedStatesemergency departments. AcadEmergMed. 2018;25(12):1355–64.
4.MichelsonKA,HudginsJD,MonuteauxMC,etal.Cardiacarrestsurvival inpediatricandgeneralemergencydepartments. Pediatrics. 2018;141(2):e20172741.
5.CushingAM,BucholzE,MichelsonKA.Trendsinregionalizationof emergencycareforcommonpediatricconditions. Pediatrics. 2020;145(4):e20192989.
6.BerryJG,PoduriA,BonkowskyJL,etal.Trendsinresourceutilization bychildrenwithneurologicalimpairmentintheUnitedStatesinpatient healthcaresystem:arepeatcross-sectionalstudy. PLoSMed. 2012;9(1):e1001158.
7.CohenE,BerryJG,CamachoX,etal.Patternsandcostsofhealthcare useofchildrenwithmedicalcomplexity. Pediatrics. 2012;130(6):e1463–70.
8.MassinMM,MontesantiJ,GérardP,etal.Childrenwithchronic conditionsinapaediatricemergencydepartment. ActaPaediatr. 2006;95(2):208–13.
9.O’MahonyL,O’MahonyDS,SimonTD,etal.Medicalcomplexityand pediatricemergencydepartmentandinpatientutilization. Pediatrics. 2013;131(2):e559–65.
10.FeudtnerC,LevinJE,SrivastavaR,etal.Howwellcanhospital readmissionbepredictedinacohortofhospitalizedchildren?A retrospective,multicenterstudy. Pediatrics. 2009;123(1):286–93.
11.JainPN,ChoiJ,KatyalC.Pediatriccareinthenonpediatric emergencydepartment:providerperspectives. HospPediatr. 2019;9(3):216–9.
12.LanghanM,KeshavarzR,RichardsonLD.Howcomfortableare emergencyphysicianswithpediatricpatients? JEmergMed. 2004;26(4):465–9.
13.HealthcareCostandUtilizationProject(HCUP).NEDSOverview. Availableat: https://www.hcup-us.ahrq.gov/nedsoverview.jsp AccessedDecember2,2020.
14.FeudtnerC,FeinsteinJA,ZhongW,etal.Pediatriccomplexchronic conditionsclassificationsystemversion2:updatedforICD–10and complexmedicaltechnologydependenceandtransplantation. BMC Pediatr. 2014;14:199.
15.HansenM,FleischmanR,MecklerG,etal.Theassociationbetween hospitaltypeandmortalityamongcriticallyillchildreninUSEDs. Resuscitation. 2013;84(4):488–91.
16.WhitfillT,AuerbachM,ScherzerDJ,etal.Emergencycareforchildren intheUnitedStates:epidemiologyandtrendsovertime. JEmergMed. 2018;55(3):423–34.
17.BourgeoisFTandShannonMW.Emergencycareforchildrenin pediatricandgeneralemergencydepartments. PediatrEmergCare. 2007;23(2):94–102.
18.ChandwaniHS,StrasselsSA,RascatiKL,etal.Estimatesofcharges associatedwithemergencydepartmentandhospitalinpatientcarefor opioidabuse-relatedevents. JPainPalliatCarePharmacother.
2013;27(3):206–13.
19.ChannaR,ZafarSN,CannerJK,etal.Epidemiologyofeyerelatedemergencydepartmentvisits. JAMAOphthalmol.
2016;134(3):312–9.
20.AlessandriniEA,AlpernER,ChamberlainJM,etal.Developinga diagnosis-basedseverityclassificationsystemforusein emergencymedicalservicesforchildren. AcadEmergMed. 2012;19(1):70–8.
21.SotoF,MurphyA,HeatonH.Criticalproceduresinpediatric emergencymedicine. EmergMedClinNorthAm. 2013;31(1):335–76.
22.MittigaMR,GeisGL,KerreyBT,etal.Thespectrumandfrequencyof criticalproceduresperformedinapediatricemergencydepartment: implicationsofaprovider-levelview. AnnEmergMed. 2013;61(3):263–70.
23.AmericanAcademyofPediatrics,CommitteeonPediatricEmergency MedicineandCouncilonClinicalInformationTechnology,American CollegeofEmergencyPhysicians,PediatricEmergencyMedicine Committee.Policystatement–emergencyinformationformsand emergencypreparednessforchildrenwithspecialhealthcareneeds. Pediatrics. 2010;125(4):829–37.
24.BurkeRTandAlversonB.Impactofchildrenwithmedicallycomplex conditions. Pediatrics. 2010;126(4):789–90.
25.AmesSG,DavisBS,MarinJR,etal.Emergencydepartmentpediatric readinessandmortalityincriticallyillchildren. Pediatrics. 2019;144(3):e20190568.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 244 EmergencyCareforChildrenwithChronicNeuroComplaints Jafarietal.
26.HeneghanJA,ReederRW,DeanJM,etal.Characteristics andoutcomesofcriticalillnessinchildrenwithfeedingand respiratorytechnologydependence. PediatrCritCareMed. 2019;20(5):417–25.
27.RussCMandAgusM.Triageofintermediate-carepatientsinpediatric hospitals. HospPediatr. 2015;5(10):542–7.
28.LamKFandLamP.Tracheostomyinchildrenwithsevereneurological impairment:asinglecentrereview. JPaediatrRespirolCritCare. 2015;11(4):4–11.
29.FrançaULandMcManusML.Trendsinregionalizationofhospitalcare forcommonpediatricconditions. Pediatrics. 2018;141(1):e20171940.
30.MichelsonKA,HudginsJD,LyonsTW,etal.Trendsincapabilityof hospitalstoprovidedefinitiveacutecareforchildren:2008to2016. Pediatrics. 2020;145(1):e20192203.
31.AbulebdaK,WhitfillT,MontgomeryEE,etal.Improvingpediatric readinessingeneralemergencydepartments:aprospective interventionalstudy. JPediatr. 2021;230:230–7.e1.
32.GattuR,TeshomeG,LichensteinR.Telemedicineapplicationsforthe pediatricemergencymedicine:areviewofthecurrentliterature. Pediatr EmergCare. 2016;32(2):123–30.
33.McConnochieKM,WoodNE,HerendeenNE,etal.Acuteillnesscare patternschangewithuseoftelemedicine. Pediatrics. 2009;123(6):e989–95.
34.DayalP,HojmanNM,KisseeJL,etal.Impactoftelemedicineonseverity ofillnessandoutcomesamongchildrentransferredfromreferring emergencydepartmentstoachildren'shospitalPICU. PediatrCritCare Med. 2016;17(6):516–21.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 245 Jafarietal. EmergencyCareforChildrenwithChronicNeuroComplaints
PediatricOutcomesofEmergencyMedicalServices
Non-TransportBeforeandDuringtheCOVID-19Pandemic
LoriPandya,MD*
BrandonMorshedi,MD,DPT,NREMT-P,FP-C†
BrianMiller,MD‡
HalimHennes,MD*
MohamedBadawy,MD*
*UniversityofTexasSouthwesternMedicalCenter,Department ofPediatrics,DivisionofEmergencyMedicine,Dallas,Texas
† UniversityofArkansasforMedicalSciences,Deprtmentof EmergencyMedicine,LittleRock,Arkansas
‡ UniversityofTexasSouthwesternMedicalCenter,Department ofEmergencyMedicine,Dallas,Texas
SectionEditors:JabeenFayyaz,MD,MCPS,MHPE,PhD,IHP,andYaninaPurim-Shem-Tov,MD,MS
Submissionhistory:SubmittedJuly24,2023;RevisionreceivedNovember22,2023;AcceptedNovember27,2023
ElectronicallypublishedFebruary9,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18408
Introduction: Pediatricpatientsaccountfor6–10%ofemergencymedicalservices(EMS)activationsin theUnitedStates.Approximately30%ofthesechildrenarenottransportedtoanemergencydepartment (ED).Adultdataintheliteraturereportshigherhospitalizationandcomplicationsfollowingnon-transport. Fewstudiesdiscussepidemiologyandcharacteristicsofpediatricnon-transport;however,dataon outcomeislimited.Ourprimaryaimwastodetermineoutcomesofnon-transportedchildrenwithinour urbanEMSsystembeforeandduringtheCOVID-19pandemic.Oursecondaryobjectivewastoexplore reasonsfornon-transport.
Methods: Thiswasaprospective,descriptivepilotstudy.WecomparedEMSdataforSeptember2019 (pre-COVID-19)toSeptember2020(pandemic).Includedwerechildrenaged0–17yearswhoactivated EMSanddidnotreceivetransporttotheprimaryhospitalfortheEMScapturearea.Wedefined outcomesasrepeatEMSactivation,EDvisits,andhospitaladmissions,allwithin72hours.Datawas obtainedviaelectroniccapture.Weuseddescriptivestatisticstoanalyzeourdata,chisquarefor categoricaldata,stepwiselogisticregression,andunivariatelogisticregressiontotestforassociationof covariateswithnon-transport.
Results: Therewere1,089pediatricEMSactivationsinSeptember2019and780inSeptember2020. Non-transportoccurredin633(58%)inSeptember2019and412(53%)inSeptember2020.Emergency medicalserviceswasreactivatedwithin72hoursinthefollowing:9/633(1.4%)in2019;and5/412(1.2%) in2020(P = 0.77).VisitstotheEDoccurredin57/633(9%)in2019and42/412(10%)in2020(P = 0.53). Hospitaladmissionsoccurredin10/633(1.5%)in2019and4/412(0.97%)in2020(P = 0.19).One non-transportedpatientwasadmittedtotheintensivecareunitinSeptember2020(<1%)andsurvived. Hispanicethnicity,age >12years,andfeverwereassociatedwithrepeatEMSactivation.Themost commonreasonfornon-transportinbothstudyperiodswasthattheparentfeltanambulance wasnotnecessary(47%).
Conclusion: Inoursystem,non-transportofpediatricpatientsoccurredin >50%ofEMSactivationswith nosignificantadverseoutcome.Age >12years,fever,andHispanicethnicityweremorecommonin repeatedEMSactivations.Themostcommonreasonfornon-transportwasparentsfeelingitwasnot necessary.FuturestudiesareneededtodevelopreliableEMSguidelinesforpediatricnon-transport. [WestJEmergMed.2024;25(2)246–253.]
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 246
ORIGINAL RESEARCH
INTRODUCTION
Pediatricpatientshistoricallyaccountforupto10%of emergencymedicalservices(EMS)activationsintheUnited States,1 withmorerecentliteraturesuggesting6%.2 A nationalEMSdatareviewnotedthat30%ofpediatric patientsarenottransportedtoamedicalfacilityforfurther evaluationandcare.2–6 Thereasonsfornon-transportare broad,includingfactorssuchasparentalrefusalandtypeof complaint(ie,musculoskeletaltrauma,respiratoryillness). Whiledataexistsregardingratesofandfactorsrelatedto pediatricEMSnon-transport,outcomesarelimitedtoafew studies.Intheadultliterature,non-transportwasassociated witha16%hospitalizationrate7 andinsomecasesseriousor fataloutcomes.8 Onepediatricstudynotednon-transported patients <3yearsofagewere1.3timesmorelikelytohavea subsequentemergencydepartment(ED)visit,9 whileanother reporteda10%hospitalizationrateafterpediatricnontransportforparentalrefusal.10 DuringtheCOVID-19 pandemic,datasuggeststhatEMScallvolumesandnontransportrateschanged,withadeclineinoverallEMS responsevolumesandanincreaseintherateofnontransports.11 Littleisknownaboutwhetherthisimpacted outcomesforchildrenwhowerenon-transported.
Ourprimaryobjectivewastodeterminepediatric outcomesofnon-transportwithinourlargeEMSsystem beforeandduringtheCOVID-19pandemic.Outcomeswere definedasrepeatEMSactivation,in-personEDvisits,and/or hospitaladmissions,allwithin72hoursofinitialEMS activation.Wealsoaimedtodescribedemographicfactors associatedwithsubsequentlyneedingmedicalattentionafter EMSnon-transport.Asecondaryobjectivewastoidentify reasonsfornon-transportwithinoursystembothpre-and duringtheCOVID-19pandemic.Wechosetocomparepreandduringthepandemictodeterminewhethertherewasa changeinutilizationorinEMSclinicians’/parents’ behavior duringapandemictobetterprepareoursystemsfor thefuture.
MATERIALSANDMETHODS
StudySetting
Thiswasaprospective,descriptivepilotstudyatalarge, urban, fire-basedEMSsystemintheCityofDallas,Texas, with59stationsand~1,800EMSrespondersservingatotal populationof1.3million,withapproximately25%ofthat population <18yearsofage.Thestudywasapprovedbythe institutionalreviewboard.
InclusionofPatients
Weincludedchildrenaged0–17yearswithEMS activationwhodidnotreceiveEMStransportduringthe studyperiod.Weselectedtwoone-monthtimeperiods, September2019(pre-COVID-19)andSeptember2020 (COVID-19pandemic).Duringthestudyperiod,allnontransportsofpediatricpatientswere,perprotocol,required
PopulationHealthResearchCapsule
Whatdowealreadyknowaboutthisissue?
Upto30%ofpediatricEMSactivationsare nottransportedtoanED.Adultdatareports adverseoutcomesfollowingnon-transport; pediatricdataislimited.
Whatwastheresearchquestion?
WeaimedtodetermineoutcomesofnontransportedchildrenwithinourEMSsystem beforeandduringCOVID-19.
Whatwasthemajor findingofthestudy?
Therewasnodifferenceinoutcomespre/ duringCOVID-19:EMSreactivation (1.3%ofallpatients)(P = 0.77);EDvisits (P = 0.53);andadmission(P = 0.19).
Howdoesthisimprovepopulationhealth?
Futurestudiesareneededtodevelopreliable guidelinesforpediatricnon-transport,which coulddecreaseburdenonthemedicalsystem especiallyduringpandemics.
tohaveonlinemedicalcontrol(OLMC)consultationand/or audiorecording.Audio-recordedrefusalwasobtainedvia handheldtabletusingastandardizedscript.AnyEMSinitiatednon-transportswerenotallowedinthesystem,and allnon-transportswereinitiatedbytheparentorguardian.
DataAcquisition
WeobtainedandcomparedEMSdatathrough comprehensivemanualreviewoftheprehospitalelectronic healthrecord(EHR)fromadailyautomatedreportofthe twoperiods.TheEMSrecordswereelectronicallymatched usingnameanddateofbirth(DOB)forrepeatEMS activationwithin72hours.Atourpediatrichospitalhealth system,whichistheprimarytertiarycarechildren’shospital fortheEMScapturearea,wequeriedtheEHRforEDvisits andhospitaladmissionswithin72hoursofEMSactivation usingthesamenameandDOB.Ifconcernsaroseforaname spellingerror,weusedDOBandaddresstoconfirman identitymatch.Demographicdata,chief/dispatch complaint,EMSvitals,non-transportvolume,andnontransportreasonweremanuallyabstractedfromourEMS electronicpatientcaredatabase/automatedreport(byeither theprincipalinvestigatorPIorasingle,trainedresearch assistant[RA]).Race/ethnicitywasEMSidentifiedusinga
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 247
Pandyaetal. PediatricOutcomesofEMSNon-TransportBeforeandDuringCOVID
drop-downmenuintheelectronicpatientcarerecord;the categoriesareperNEMSIS(NationalEMSInformation System).BecausepriorversionsofNEMSIScombinedrace andethnicitytherearenotseparate fields.In-personED visitsandhospitaladmissions(includinginpatient observationandintensivecareunit[ICU]admission)within 72hoursofEMSactivationand finaldisposition(discharge vsdeath)weremanuallyabstractedfromthehospitalhealth systemEHR(byeitherthePIorasingletrainedRA).This includedhospitalpresentationsafterrefusalthatcameby repeatEMSactivationandothermeans(eg,privatevehicle).
Outcomes
WedefinedprimaryoutcomesasrepeatEMSactivation, in-personEDvisits,and/orhospitaladmissions,allwithin 72hoursofinitialEMSactivation.Weusedthe72-hour follow-upwindowbasedonotherpublishedpapersinthis area. 12–16 PediatricEMSprotocolsdidnotchangebetween thesetwostudyperiods.Thepopulationwasstratifiedbyage group(similartopreviouslypublishedstudies2,17,18)and chiefcomplainttodeterminewhethertherewasahigher proportionofnon-transportbasedonageandthemost commonnon-transportdiagnosis.WeclassifiedEMSchief complaint/diagnosisintothefollowingcategories:fever; gastrointestinal;respiratory;trauma;neurological;pain; mentalhealth;andother.Reasonforparentalrefusalof transportwasdescribed(EMSdocumented).
Analysis
Weanalyzedcategoricaldatausingthechi-squaredtest. TheFisherexacttestwasusedforsmallersamplesizes(ie, hospitalandICUadmissiondata).Weusedthe t -testand Wilcoxonrank-sumtestforEMSvitalsigns.InitialEMS vitalsignsoftemperature,heartrate,respiratoryrateand
oxygensaturationwereabstractedforeachsubjectand definedasabnormalbasedonnormalage-relatedranges withinthePediatricAdvancedLifeSupportguidelines.19 We madecorrectionformultipletestingandusedonlycaseswith completedatainthe finalanalysis.Covariatesforanalysis wereaprioribasedonpreviousliterature.Toidentify covariates’ associationwiththeoutcomeweperformeda stepwiselogisticregression.However,theanalysisidentified onlyonecovariate,andweusedaunivariatelogistic regressiontotestforassociationofthatcovariatewithinnontransportoutcomes.Wedidnotcalculateapredetermined samplesize,asthiswasapilotstudy.Resultsarepresentedas oddsratioswith95%confidenceintervals(CI),taking P -valuesof <0.05assignificant.Statisticalanalyseswere performedusingSASforWindowsrelease9.4 (SASInstitute,Inc.,Cary,NC).
RESULTS
AnnualpediatricEMSvolumeswere12,663(2019)and 10,429(2020).Therewere1,089pediatricEMSactivationsin September2019vs780activationsinSeptember2020 (Figure).Non-transportoccurredin633(58%)activationsin September2019vs412(53%)inSeptember2020(Table1). PerourEMSprotocol,weobtainedOLMCand/oraudio recordingin84%ofnon-transports.Demographicsarelisted in Table2
Table1. Volumeofnon-transportedpediatricpatientsbefore(2019) andduring(2020)theCOVID-19pandemic.
Sept2019Sept2020 EMSactivations1089780 Non-transport(%)633(58)412(53)
Figure. STROBEdiagramillustratingpatientinclusion. EMS,emergencymedicalservices.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 248 PediatricOutcomesofEMSNon-TransportBeforeandDuringCOVID Pandyaetal.
Pandyaetal. PediatricOutcomesofEMSNon-TransportBeforeandDuringCOVID
2019(N = 633)2020(N = 412)All(N = 1,045) P-value
PatientAge(mos)113(38.0–182.0)111.5(33.4–181.5)
PatientAge(yrs) 0.9
0–2121(19.2%)76(18.6%)197(19.0%)
2–5107(17.0%)63(15.4%)170(16.4%)
5–12158(25.1%)105(25.7%)263(25.3%)
>=12244(38.7%)164(40.2%)408(39.3%)
Gender 0.7
Female312(49.5%)208(50.6%)520(50.0%)
Male318(50.5%)203(49.4%)521(50.0%)
Race 0.7
BlackorAfricanAmerican337(53.9%)207(50.9%)544(52.7%)
White63(10.1%)38(9.3%)101(9.8%)
HispanicorLatino205(32.8%)149(36.9%)354(34.3%)
Other20(3.2%)13(3.2%)33(3.2%)
Patientagenotedasmeanwithinterquartilerange.
PrimaryOutcomes
DuringSeptember2019EMSwasreactivatedwithin72 hoursin9/633(1.4%)activations,EDvisitsoccurredwithin 72hoursin57/633cases(9%),andhospitaladmissions occurredin10/633(1.5%).DuringSeptember2020,EMS wasreactivatedwithin72hoursin5/412(1.2%)activations, EDvisitsoccurredwithin72hoursin42/412cases(10%),and hospitaladmissionsoccurredin4/412(1%).OnenontransportedpatientwassubsequentlyadmittedtotheICUin September2020(<1%)andsurvivedtodischarge.Therewere nostatisticaldifferencesinoutcomesofnon-transportpreandduringthepandemic(Table3).
SecondaryOutcomes
Wefurtheranalyzednon-transportoutcomesin September2020todeterminewhethertherewasahigher proportionofnon-transportrelatedtospecificvariables (gender,race/ethnicity,age,EMSdiagnosis,andvitalsigns). Ourpercentageofmissingvariablesrangedfrom1–15%; however,intheanalysisweusedonlycaseswithcomplete data.InthosechildrenwhohadrepeatEMSactivation
within72hours,Hispanicethnicity,age >12years,andfever onEMSvitalswerestatisticallysignificantfactorsforrepeat activations.Therewasnodifferenceingender,EMS diagnosis,heartrate,respiratoryrate,oroxygensaturation (Table4).ForthosechildrenwithanEMSreactivation resultingintransporttotheED,adiagnosisoftrauma, Hispanicethnicity,age >12years,andfeverweresignificant (Table5).OfthosechildrenwithanEDvisitwithin72hours ofEMSnon-transport,malegenderwastheonlysignificant variable.Therewasnodifferenceinrace/ethnicity,age, diagnosis,orvitalsigns,includingtemperature(Table6).
Race/Ethnicity
Inourlarge,urbancountyin2020,Blackresidentsmade up22.8%ofthetotalpopulation20 andaccountedfor approximately53%ofallpediatricEMSactivationsduring ourstudymonth.Ofallnon-transportedchildreninour studymonth50%wereBlack.Hispanic/Latinoaccountedfor 41%ofthetotalpopulationand34%ofallpediatric activations;36%ofpediatricnon-transportswereidentified asHispanic/Latino.Inoururbancounty,27%ofthetotal
Outcomesofnon-transport(Within72hrs)2019(N = 633)2020(N = 412)All(N = 1,045) P-value RepeatEMSactivation9(1.4%)5(1.2%)14(1.3%)0.8 TransporttoEDonrepeatactivation9(1.4%)4(1.0%)13(1.2%)0.5 EDvisit57(9.0%)42(10.2%)99(9.5%)0.5 Inpatienthospitaladmission10(1.6%)4(1%)14(1.3%)0.2 ICUduringhospitaladmission0(0.0)1(0.2)1(0.1)0.4
Table2. Demographicsofnon-transportedpediatricpatientsbefore(2019)andduring(2020)theCOVID-19pandemic.
Table3. Outcomesofnon-transportedpediatricpatientsbefore(2019)andduring(2020)theCOVID-19pandemic.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 249
populationidentifiedasWhiteandmadeup10%percentof totalpediatricactivations.Ofthosenon-transportedchildren 9%wereWhite.
ReasonsforNon-transport
Inthepre-pandemicperiod(September2019),thereason fornon-transportwas filedfor354(55%)ofactivationsas follows:parentfeltambulancenotnecessary(47.7%);chief complaintresolved(24.9%);transportbyprivatevehicle (20%);andother(3.1%).InSeptember2020,thereasonfor non-transportwasdocumentedin207(49%)cases,withthe mostcommonreasonbeingparentfeltambulancewasnot necessary(58%);followedbytransportbyprivatevehicle (22.2%);chiefcomplaintresolved(15.5%);andother(4.8%).
DISCUSSION
Wefoundourratesofpediatricnon-transport(both pre-andduringpandemic)tobehigherthanthepreviously reported16.3%–30.1%.2–6 Despitethehigherrateof non-transport,ourpediatricoutcomeswerefavorable.The EMSreactivationandhospitaladmissionsoccurredinless
than1.5%ofthosechildrennottransportedtoahealthcare facility.Duringourselectedmonthinthepandemic,onlyone patient(<1%)requiredICUcareandsurvivedtohospital discharge.VisitstotheEDwithin72hoursoccurredin approximately10%ofchildrennottransported;furtherstudy isneededtoevaluatethissubsetofpatients.
ArecentpublishedstudyfromtheUnitedKingdom showedasimilarrateofpediatricEMSreactivation(2%) afterambulancenon-transport.SubsequentEDvisitswere higherthaninour findings(upto24%),andhospital admissionswerealsohigher(ashighas6%comparedtoour 1.5%).Asinourstudy,nodeathsoccurredinpediatricnontransport.16 Anotherstudyshowedapproximately14%ED visitsafternon-transport, <1%hospitaladmission,andagain nodeaths.21 AScandinavianstudyreported17.4%ofnontransportedchildrenvisitedtheED,althoughthiswaswithin 96hourscomparedtoour72-hourtimeframe.Twopatients wereadmittedtotheICU(comparedtooneinourstudy), andagainnodeathsoccurred.22
Allprimaryoutcomeswerenotsignificantlydifferent whencomparedtopre-pandemicdata.Ofnote,weusedthe
VariableCategoryOddsRatio95%C.I.forOddsRatio P-value GenderFemale Male1.310(0.468,3.668)0.60 Race/EthnicityBlack/AfricanAmerican White0.194(0.011,3.334)0.25 Hispanic0.167(0.031,0.909)0.03 Other0.588(0.033,10.526)0.71 Patientage(yrs)0–2 2–50.225(0.038,1.320)0.09 5–120.341(0.094,1.234)0.10 >=120.219(0.061,0.790)0.02 EMSchiefcomplaint/diagnosisFever Gastrointestinal0.527(0.073,3.799)0.52 MentalHealth2.414(0.293,19.912)0.41 Neurological0.288(0.041,2.043)0.21 Other0.206(0.052,0.817)0.02 Pain0.128(0.006,2.610)0.18 Respiratory0.225(0.043,1.187)0.07 Trauma0.108(0.016,0.758)0.02 EMSvitalsigns Temp2.645(1.007,6.943)0.04 HR1.007(0.979,1.036)0.60 RR1.023(0.991,1.057)0.16 Sat0.966(0.920,1.015)0.17
,temperature; HR,heartrate; RR,respiratoryrate; Sat,oxygensaturation. WesternJournal of EmergencyMedicineVolume25,No.2:March2024 250 PediatricOutcomesofEMSNon-TransportBeforeandDuringCOVID Pandyaetal.
Table4. Non-transportedoutcomeduringtheCOVID-19pandemic-repeatEMSactivation.
Temp
Temp
72-hourfollow-upwindowbasedonotherpublishedpapers inthisarea,12–16 whileacknowledgingthebalancebetweena longerwindowcatchingmorecasesbutincreasingtherisk thatthosearenotrelatedtotheindexvisit.
Themajorityofnon-transportedchildrenwereBlack (50%);however,thiswasexpectedbasedonour demographics(themajorityofallpediatricEMSactivations duringourstudymonthwereBlack).Similarly,Hispanic/ Latinochildrenaccountedfor34%ofpediatricEMS activationsand36%ofnon-transports.This findingdiffers frompriorstudiesthatshowalowerrateofnon-transportfor Black3 andHispanic6 children.Ourstudyissimilartoa recent,largenationalstudybyWardetal,whichshowedno associationofrace/ethnicitywithnon-transport.2
Althoughwefoundnoassociationwithrace/ethnicityfor non-transport,Hispanicchildreninourstudyweremore likelytohaverepeatEMSactivationswithin72hours.Age >12yearsoldanddocumentedfeverwerealsoassociated withrepeatEMSactivations.Thisageassociationwith repeatactivationsmaybeduetoaloweroverallrateof non-transportinyoungerkids,bothinourstudyandothers6
andthepostulatedlackofEMSresponders’ comfortlevel assessingyoungchildren.14 Wealsofoundthatchief complaint/diagnosiswasnotsignificantlyrelatedtoEMS non-transportduringthepandemic,althoughchildrenwith traumawerenotsurprisinglytransportedmoreoftentothe EDifEMSwasreactivatedwithin72hours.Interestingly, EMSvitals(exceptfever)didnotseemtoplayaroleinour primaryoutcomes.
Inourstudyweobservednosignificantdifferenceinthe percentageoroutcomesofpediatricnon-transportduringthe COVID-19pandemiccomparedtopre-pandemic.Itis importanttonotethatEMSprotocolsdidnotchange betweenthesetwostudyperiods.WhilemanyEMSagencies adoptedmorepermissive “non-transport” policiesin anticipationofhigherEMScallvolumesand9-1-1overuse forminor, flu-likeillnesssymptoms,oursystemdidnotadopt anysuchpolicy;thus,itisatruercomparison.
Ourreasonsfornon-transportaresimilartothose previouslyreportedintheliterature.5–7,23 Duringthe pandemic,therewasapproximatelya10%increasein “parentsfeelanambulanceisnotnecessary.” Itisunclear
VariableCategoryOddsRatio95%C.I.forOddsRatio P-value GenderFemale Male1.645(0.219,1.899)0.42 Race/EthnicityBlack/AfricanAmerican White4.765(0.276,82.237)0.28 Hispanic5.532(1.009,30.320)0.04 Other1.573(0.087,28.284)0.75 Patientage(yrs)0–2 2–53.835(0.639,23.029)0.14 5–122.526(0.678,9.415)0.16 >=123.932(1.058,14.618)0.04 EMSchiefcomplaint/diagnosisFever Gastrointestinal1.897(0.263,13.667)0.52 MentalHealth0.414(0.050,3.417)0.41 Neurological3.467(0.0489,24.553)0.21 Other5.952(1.419,24.965)0.01 Pain7.784(0.383,158.149)0.18 Respiratory4.435(0.842,23.346)0.07 Trauma9.223(1.319,64.478)0.02 EMSvitalsigns Temp0.378(0.144,0.993)0.04 HR1.001(0.971,1.032)0.92 RR0.977(0.947,1.009)0.16 Sat1.035(0.985,1.088)0.17
Table5. Non-transportedoutcomeduringtheCOVID-19pandemic-transportedtoEDonrepeatEMSactivation.
,temperature; HR
RR
Sat
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 251 Pandyaetal. PediatricOutcomesofEMSNon-TransportBeforeandDuringCOVID
,heartrate;
,respiratoryrate;
,oxygensaturation.
whetherthiswasdirectlyrelatedtothepandemicandfearof COVID-19exposureortomissingdata.
LIMITATIONS
Therearecertainlimitationsofthisstudy.Althoughwe aretheprimarychildren’shospitalandurgentcarewithinthe jurisdictionservedbytheEMSsystem,therewasthe potentialtomissrepeatEDvisitsatanon-affiliatedadult ED/urgentcare.Futurestudieswillincludeaphonecall follow-upwiththepatient/family.Weselectedasingle month,duetoourhighvolumes,forthispilotstudy, assumingitwouldberepresentativeofothermonths.Data wasobtainedthroughmanualreviewofprehospital electronicpatientcarerecordsobtainedfromanautomated report,resultinginsomeoccasionalincompletedata. Hospitalrecords(EDandinpatient)werematchedusing nameandDOB,potentiallymissingsubjectsiftherewasan errorinnamespellingorprovidedDOB.Ifconcernsarose foramismatch,theprovidedaddresswasusedtoconfirman identity,butthisdatawasnotalwaysavailable. Furthermore,theEMSsystem’sclinicalpractice
guidelines(protocols)requireconsultationwithonline medicalcontrolforpatients <18yearsoldandforspecific conditionsandvitalsignparameters.Inthisstudywedidnot examinetheproportionofnon-transportedpatientswith onlinemedicalcontrolactuallycontacted.Itisnotknown whetherthisinfluencedthesafetyofnon-transports.Lastly, reasonfornon-transportwasmissinginupto50%ofdata, andthereasonwasasdocumentedbytheEMSclinician.
CONCLUSION
Inoursystem,non-transportofpediatricpatients occurredinover50%ofEMSactivationwithnosignificant adverseoutcome.Themostcommonreasonfornontransportwasparentsfeelingitwasnotnecessary.Age >12years,presenceoffever,andHispanicethnicitywere morecommoninrepeatedEMSactivations.Chief complaint/diagnosisdidnotseemtoplayaroleinrepeat EMSactivationsorsubsequentEDvisitsafternontransport.Weobservednosignificantdifferenceinthe percentageoroutcomesofpediatricnon-transportduringthe COVID-19pandemiccomparedtopre-pandemic.
VariableCategoryOddsRatio95%C.I.forOddsRatio P-value GenderFemale Male1.595(1.047,2.430)0.02 Race/EthnicityBlack/AfricanAmerican White0.684(0.292,1.602)0.38 Hispanic1.357(0.879,2.096)0.16 Other0.459(0.085,2.481)0.36 Patientage(yrs)0–2 2–51.215(0.604,2.446)0.58 5–121.298(0.695,2.421)0.41 >=121.011(0.556,1.838)0.97 EMSchiefcomplaint/diagnosisFever Gastrointestinal0.919(0.280,3.018)0.88 MentalHealth1.939(0.365,10.301)0.43 Neurological0.877(0.313,2.460)0.80 Other0.589(0.251,1.380)0.22 Pain0.545(0.158,1.885)0.33 Respiratory0.654(0.257,1.665)0.37 Trauma0.858(0.355,2.073)0.73 EMSvitalsigns Temp0.983(0.679,1.424)0.92 HR1.009(0.998,1.020)0.10 RR1.004(0.977,1.032)0.76 Sat0.982(0.940,1.027)0.42 Temp,temperature; HR
,respiratoryrate;
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 252 PediatricOutcomesofEMSNon-TransportBeforeandDuringCOVID Pandyaetal.
Table6. Non-transportedoutcomeduringtheCOVID-19pandemic-EDvisitwithin72hours.
,heartrate; RR
Sat,oxygensaturation.
AdditionalstudiesareneededtodevelopreliableEMS guidelinesforpediatricnon-transport.
AddressforCorrespondence:LoriPandya,MD,Children’s HealthcareofAtlantaatScottishRite,DivisionofEmergency Medicine,1001JohnsonFerryRoad,Atlanta,GA30342.
E-mail: lori.pandya@pemaweb.com
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.
Copyright:©2024Pandyaetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.DiggsLA,Sheth-ChandraM,DeLeoG.Epidemiologyofpediatric prehospitalBasicLifeSupportcareintheUnitedStates. PrehospEmerg Care.2016;20(2):230–8.
2.WardC,ZhangA,BrownK,etal.NationalcharacteristicsofnontransportedchildrenbyemergencymedicalservicesintheUnited States. PrehospEmergCare.2022;26(4):537–46.
3.RamgopalS,Owusu-AnsahS,Martin-GillC.Factorsassociatedwith pediatricnontransportinalargeemergencymedicalservicessystem. AcadEmergMed.2018;25(12):1433–41.
4.RichardJ,OsmondMH,NesbittL,etal.Managementandoutcomesof pediatricpatientstransportedbyemergencymedicalservicesina Canadianprehospitalsystem. CJEM,2006;8(1):6–12.
5.KannikeswaranN,MahajanPV,DunneRB,etal.Epidemiologyof pediatrictransportsandnon-transportsinanurbanemergencymedical servicessystem. PrehospEmergCare.2007;11(4):403–7.
6.GerlacherGR,SirbaughPE,MaciasCG.Prehospitalevaluationofnontransportedpediatricpatientsbyalargeemergencymedicalservices system. PediatrEmergCare.2001;17(6):421–4.
7.ConeDC,KimDT,DavidsonSJ.Patient-initiatedrefusalsofprehospital care:ambulancecallreportdocumentation,patientoutcome,andonlinemedicalcommand. PrehospDisasterMed.1995;10(1):3–9.
8.ZachariahB,BryanD,PepeP,etal.Follow-upandoutcomeofpatients whodeclineoraredeniedtransportbyEMS. PrehospDisasterMed. 1992;7(4):359–64.
9.KnightS,OlsonLM,CookLJ,etal.Againstalladvice:ananalysisofoutof-hospitalrefusalsofcare. AnnEmergMed.2003;42(5):689–96.
10.SeltzerAG,VilkeGM,ChanTC,etal.Outcomestudyofminorsafter parentalrefusalofparamedictransport. PrehospEmergCare 2001;5(3):278–83.
11.SattyT,RamgopalS,ElmerJ,etal.EMSresponsesandnon-transports duringtheCOVID-19pandemic. AmJEmergMed.2021;42:1–8.
12.JoyT,BossonN,ChangA,etal.86short-termoutcomesandpatient perceptionsafterEMSnon-transportduringtheCOVID-19pandemic. AnnEmergMed.2022;80(4):S43–4.
13.GloberN,HamiltonJ,MontelauroN,etal.Safetyofanalternativecare protocolforEMSnon-transportintheCOVID-19pandemic. Prehosp EmergCare.2023;27(3):315–20.
14.HainesCJ,LutesRE,BlaserM,etal.Paramedicinitiatednon-transport ofpediatricpatients. PrehospEmergCare.2006;10(2):213–9.
15.SinclairJE,AustinM,FroatsM,etal.Characteristics,prehospital management,andoutcomesinpatientsassessedforhypoglycemia: repeataccesstoprehospitaloremergencycare. PrehospEmergCare 2019;23(3):364–76.
16.CosterJ,O’CathainA,JacquesR,etal.Outcomesforpatientswho contacttheemergencyambulanceserviceandarenottransportedto theemergencydepartment:adatalinkagestudy. PrehospEmergCare 2019;23(4):566–77.
17.LernerEB,DayanPS,BrownK,etal.Characteristicsofthepediatric patientstreatedbythePediatricEmergencyCareAppliedResearch Network’saffiliatedEMSagencies. PrehospEmergCare 2014;18(1):52–9.
18.HewesHA,DaiM,MannNC,etal.Prehospitalpainmanagement: disparitybyageandrace. PrehospEmergCare.2018;22(2):189–97.
19.TopjianAA,RaymondTT,AtkinsD,etal.Part4:PediatricBasicand AdvancedLifeSupport:2020AmericanHeartAssociationGuidelines forCardiopulmonaryResuscitationandEmergencyCardiovascular Care. Circulation.2020;142(16_suppl_2):S469–523.
20.USAFactsOrganization.OurChangingPopulation:Dallas,County, Texas.Availableat: https://usafacts.org/data/topics/people-society/ population-and-demographics/our-changing-population/state/texas/ county/dallas-county?endDate=2020-01-01&startDate=2019-01-01 AccessedJanuary12,2023.
21.KahaléJ,OsmondMH,NesbittL,etal.Whatarethecharacteristicsand outcomesofnontransportedpediatricpatients? PrehospEmergCare 2006;10(1):28–34.
22.OulasvirtaJ,SalmiH,KuismaM,etal.Outcomesinchildrenevaluated butnottransportedbyambulancepersonnel:retrospectivecohortstudy. BMJPaediatrOpen.2019;3(1):e000523.
23.HartkaTandVacaFE.FactorsassociatedwithEMStransportdecisions forpediatricpatientsaftermotorvehiclecollisions. TrafficInjPrev 2020;21(sup1):S60–5.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 253 Pandyaetal. PediatricOutcomesofEMSNon-TransportBeforeandDuringCOVID
EvaluationofPeer-reviewQuality:APilotStudy
JeffreyN.Love,MD,MHPE,MSc*
AnneM.Messman,MD,MHPE†
JonathanS.Ilgen,MD,PhD‡ ChrisMerritt,MD,MPH,MHPE§∥ WendyC.Coates,MD¶ DouglasS.Ander,MD# DavidP.Way,MEd**
*GeorgetownUniversitySchoolofMedicine,DepartmentofEmergency Medicine,Washington,DistrictofColumbia
† WayneStateUniversity,DepartmentofEmergencyMedicine, Detroit,Michigan
‡ UniversityofWashington,DepartmentofEmergencyMedicine, Seattle,Washington
§ AlpertMedicalSchoolofBrownUniversity,DepartmentofEmergency Medicine,Providence,RhodeIsland
∥ AlpertMedicalSchoolofBrownUniversity,DepartmentofPediatrics& EmergencyMedicine,Providence,RhodeIsland
¶ DavidGeffenSchoolofMedicineatUCLA,DepartmentofEmergency Medicine,LosAngeles,California
# EmoryUniversity,DepartmentofEmergencyMedicine,Atlanta,Georgia
**OhioStateUniversityCollegeofMedicine,DepartmentofEmergency Medicine,Columbus,Ohio
SectionEditor:MarkLangdorf,MD,MHPE
Submissionhistory:SubmittedAugust16,2023;RevisionreceivedDecember1,2023;AcceptedDecember7,2023
ElectronicallypublishedFebruary28,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18432
Introduction: Despitetheimportanceofpeerreviewtopublications,thereisnogenerallyaccepted approachforeditorialevaluationofapeerreview’svaluetoajournaleditor’sdecision-making.The graduatemedicaleducationeditorsofthe WesternJournalofEmergencyMedicine SpecialIssuein EducationalResearch&Practice(SpecialIssue)developedandstudiedtheholisticeditor’sscoring rubric(HESR)withtheobjectiveofassessingthequalityofareviewandanemphasisonthedegreeto whichitinformsaholisticappreciationforthesubmissionunderconsideration.
Methods: Usingpeer-reviewguidelinesfromseveraljournals,theSpecialIssue’seditorsformulatedthe rubricasdescriptionsofpeerreviewsofvaryingdegreeofqualityfromtheidealtotheunacceptable. Onceareviewwasassessedbyeacheditorusingtherubric,thescorewassubmittedtoathirdpartyfor blindingpurposes.Wecomparedtheperformanceofthenewrubrictoapreviouslyusedsemantic differentialscaleinstrument.Kane’svalidityframeworkguidedtheevaluationofthenewscoringrubric aroundthreebasicassumptions:improveddistributionofscores;relativeconsistencyratherthan absoluteinter-raterreliabilityacrosseditors;andstatisticalevidencethateditorsvaluedpeerreviewsthat contributedmosttotheirdecision-making.
Results: Ninetypeerreviewswerethesubjectofthisstudy,allwereassessedbytwoeditors.Compared tothehighlyskeweddistributionofthepriorratingscale,thedistributionofthenewscoringrubricwasbell shapedanddemonstratedfulluseoftherubricscale.Absoluteagreementbetweeneditorswaslowto moderate,whilerelativeconsistencybetweeneditor’srubricratingswashigh.Finally,weshowed thatrecommendationsofhigherratedpeerreviewsweremorelikelytoconcurwiththeeditor’s formaldecision.
Conclusion: EarlyevidenceregardingtheHESRsupportstheuseofthisinstrumentindeterminingthe qualityofpeerreviewsaswellasitsrelativeimportanceininformingeditorialdecision-making.[WestJ EmergMed.2024;25(2)254–263.]
DevelopmentandValidationofaScoringRubricforEditorial
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 254 ORIGINAL RESEARCH
BACKGROUND
Peerreviewplaysacriticalroleinthetraditionalparadigm ofpublishedscholarship.Whilepeerreviewisthestandard forassessingscholarlysubmissionforpublication,themost appropriatemeansbywhichtoassessthequalityofpeer reviewremainsunclear.1–8 Thisissueisproblematicforall thestakeholdersofpublishedscholarship.Thedevelopment ofarigorousandvalidtoolforeditorstoassessthequalityof peerreviewscouldhelptoenhancethepeer-reviewprocess. Thiswouldimproveeditors’ abilitiestostratifythe contributionsoftheirreviewerpool,identifyingreviewers whodeserveoutstandingrecognitionaswellasthosewho couldbenefitfromdedicatedmentorship,andinform mechanismstoevaluatethedownstreamimpact ofinterventionstoimprovethequalityof peerreviews.
Effortstoassesspeer-reviewqualityhavebeen challenging.Priorstudieshavebeenbasedprimarilyonthe beliefthatreviewevaluationisanobjectiveprocess.9–12 Consequently,interventionshavebeenaimedatachieving ahighdegreeofabsolutereliabilityinscoringbetween editors.Theresultshavedemonstratedamodestdegreeof inter-raterreliability.9–12
Theinter-raterreliabilityofevaluationsofperformance byexpertsisconfoundedbyidiosyncraticperceptionsthat areshapedbyindividualexperiences,values,andpriorities. Indeed,thepreponderanceoftheliteraturearguesthat evaluationbyexpertsisoftensubjectiveand nuanced.2,4–6,8,10,13–17 Coleetalproposedthatthepotential divergentperspectivesamongpeerreviewersareoftenthe resultof “realandlegitimatedif ferencesofopinionamong expertsaboutwhatgoodscienceis, ”14 aconceptsupported byothers.15–17 Capturingthenuancedandpotentially divergentperspectivesofreviewersallowseditorstodevelop aholisticunderstandingofthevalueofamanuscript.15 This variabilityamongeditors’ perspectiveslimitsthedegree ofreliabilitythatcanbeachievedinassessing individualreviews.
TheSpecialIssue’seditorialevaluationofreviewshas traditionallydependeduponasingle,global five-pointscale withanchorsattheextremes(5 = highquality,1 = low quality).Anumberofissueshavebeenappreciatedbythe editorswiththisapproach:1)Thewebsitetemplateonly allowedforasingleeditor’sevaluationofareview;2)scores of1and2wereseldomused;and3)noguidancewas providedforeditorstodeterminehowtoscoreonthe fivepointscale,resultinginalackofvalidevaluationdataon whichtobasedecisionspertainingtothequalityofreviews.
Ourobjectiveinthisinitiativewastodevelopandstudya scoringrubricforeditorstoassessthequalityofareviewwith anemphasisonthedegreetowhichitinformsaholistic appreciationforthesubmissionunderconsideration.Herein wedescribethedevelopment,refinement,andpilot-testingof thisrubric.Additionally,ourreportingwasgroundedwithin
thevalidityevidenceframeworksuggestedbyKaneto informtheinterpretationsofscoresgeneratedby thistool.18
METHODS
HolisticEditor’sScoringRubricDevelopment
Thisstudyinvolvedgraduatemedicaleducation(GME) submissionstotheSpecialIssueandwasdeterminedtobe exemptbytheGeorgeWashingtonUniversityInstitutional ReviewBoard.
Thereareseveralrecurrentthemesidentifiedinthe literaturethatappearimportanttodevelopinganeffective peer-reviewevaluationsystem.Suchasystemshouldbe: 1)practicalandsimpletouse5,6,19,20;2)criterion referenced4,5,20;and3)beabletocapturedifferences inexpertreviewers’ perspectives.14,16,21–23 To successfullyoperationalizeanevaluationsystem,past worksalsosuggestthatratertrainingisnecessarytoensure properimplementation.3,10,13,20,24–28
Priortotheproductioncycleforthe2020–2021Special IssuetheCouncilofResidencyDirectorsinEmergency Medicine(CORD)guesteditorandthreeassociateeditors discussedtheneedforanimprovedsystemforevaluating peerreviews.Theuseofaglobal five-pointscorehasbeen showntobepracticalinassessingreviews.5,6 Byadding anchorstoeachpointonthe five-pointscalebasedonquality asthecriteriareference,Landkroonetalprovidedearly validityevidencesupportingitsuse.6 Todefine characteristicsimportanttohigh-qualitypeerreviewsin developinganchorsforthecurrentstudy’ s five-pointglobal scale,theeditorsreviewedthemissionandvisionstatements ofCORD,29 theliteraturerelevanttopeer-reviewscoring instrumentsandreviewerguidelinesfromfourmajormedical journals.30–33 Throughaniterativeprocess,theeditors definedqualitiesofanidealreviewasonethatprovidesthe following:1)–insightsthatreflectboththevaluetoreadership andalignmentwiththecurrentliterature;2)–consideration oftheappropriatenessofthestudymethod(s)andrelevant tenetsofeducationscholarship;and3)–feedbackthat providesmentorshiptoauthorsonhowtoimprovetheir manuscriptaswellastheirownskillset.
Throughthesameprocess,theeditorsdeterminedthatthe anchorsforthe five-pointscoringrubricshouldbebasedon thesethreeprovisionsofaqualityreviewaswellasthedegree towhichthereviewinformsthe finalevaluationofthe submissionunderreview(Figure1).Inotherwords,areview evaluationof fiveonthisholisticeditor’sscoringrubric (HESR)providesallthreeprovisionsofaqualityreviewand couldstandaloneasthe finalevaluationofthatsubmission (Goldstandardreview).
ValidityAssumptions
WeusedKane ’sframeworktogathervalidityevidence foruseofthisinstrument,whichinvolvedtesting
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 255 Loveetal. ScoringRubricforEvaluationofPeer-reviewQuality
Gold Standard Reviews are those that provide:
1)Insightful review (detailed and global)that reflects upon how the work under consideration may be of value to the readership and informs the current literature.
2)Appropriateness of the study method(s) employed anda reflection of the relevant tenets of education scholarship.
3)Feedback that provides mentorship to authors on how to improve their manuscript and their own skill set.
5-Exceptional: A model review that reflects each of the tenetsof the Gold Standard and could stand alone as a summary to the authors. Recommendations to the authors are appropriate, actionable and supportive with a basis in educational scholarship.The review provides an in-depth perspective which may include relevant citations, resources or specific suggestions for improving the manuscript and/or professional growth. An additional contributing factor includes instances where the reviewer makes an important observation or recommendation not previously considered by the editors.
4-Very Good: An excellent review that reflects the time, effort and expertise necessary tocontribute substantially to the formal decision but falls short in one or more of the 3 key areas that define the “Gold Standard”. For example, an excellent overall review that (1) misses 1-2 substantive points, (2) provides only cursory mention of educational scholarship concepts or (3) falls short of providing mentoring support when critiquing the authors work.
3-Good: The review meets the standard of an acceptable review. The analysis adds to the broader perspective in a measured way but is not as complete, organized, documented or is lacking adequate explanations for the authors. As a result, additional reviews are required to provide more extensive/actionable feedback to the authors.
2-Below Average: Though there may be some insights included the review provides a superficial evaluation of the submission. This may include lack of reasoning for the decisions rendered, comments are not actionable or there may be a general lack of critique for improvement. In essence, insights provided may reinforce other reviewers’ comments but are not substantive enough to shape editorial decision-making pertaining to the manuscript. The majority of components of a “Gold Standard review” are missing.
1-Unacceptable: The review is sparse and may provide 1-2 insights but either (1) provides a decision without explanation(accept/reject, like/dislike, good/bad)(2) provide praise without critique (no substantive feedback for how to improve the manuscript) and concludes “accept as is” when revisions are needed or rejects with minimal justification, (3) lacks meaningful insights or (4) conclusions are based on faulty reasoning based on the literature, opinions of the other reviewer(s) and the editor. In short, the review provides little if any substantive critique that contributes to consensus decision making.
Figure1. Theholisticeditor’sscoringrubricusedforevaluationofpeerreview.Theinitialversionofthisrubriccanbeappreciatedasthe unshadedcontent.Subsequentadditionsmadebasedonapilotofthe14initialreviewsaredenotedbytheshadedareas(See “Preliminary CalibrationExercise.”)
CORD,CouncilofResidencyDirectorsinEmergencyMedicine.
assumptionsaboutscoring,generalization,extrapolation, andimplication. 18 Our fi rstassumptioninvolvedthe distributionofratingsorscores.Inreviewingscoresfrom thepastfewyearswiththetraditional fi ve-pointscale,we observedlimiteduseoftheevaluationscalewithskewing towardshigherscores( >3);editorswerehesitanttoassign scoresof1or2whenappropriate.Thewidevariabilityof experienceandexpertiseamongthereviewerssuggested thatgreatervariabilityinscoresshouldhavebeenpresent. Thisskeweddistributioncouldbeattributedtoaleniency bias,whichisnotuncommoninmedicaleducation evaluations. 23, 34–37 OurlogicfollowedthatfortheHESRto beavalidre fl ectionofpeer-reviewperformance,thepeerreviewevaluationscoresmustre fl ectthefullrangeofpeerreviewperformance.Ifsuccessfullydevelopedand implemented,theHESRpeer-reviewevaluationscores
wouldhaveadistributionwhereallratingoptions wereused.
Oursecondassumptioninvolvedinferencesaboutscoring, namelythattheassignedHESRscoreforapeerreviewwould beanaccuraterepresentationoftheeditor’sperspectiveof thequalityandvalueofapeerreviewtoeditorialdecisionmaking.Iftrueateachscorelevel,theassociateandsenior editorswouldbeconsistentwitheachotherinapplyingthe HESRforanygivenreview.
Ourthirdassumptionhadtodowithimplicationsthat HESRscoreswouldbeusedtoinformtheeditorialdecisionmakingprocess.Inotherwords,highlyratedreviewswould havemorevalueindecision-making,andasaresultthe reviewer’sdecisionrecommendation(i.e.accept,revise, reject)wouldmorecloselyalignwiththeeditor’ s formaldecision.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 256 ScoringRubricforEvaluationofPeer-reviewQuality Loveetal.
StudySettingandParticipants
TheSpecialIssuewasestablishedin2014asanannual publicationof WestJEM dedicatedtoeducationalresearch practice.38 SubmissionsrelatedtoGMEweremanagedbya singleguesteditorandthreeassociateeditors.Peerreviewers fortheSpecialIssuewererecruitedbythesenioreditorvia theCORDandClerkshipDirectorsofEmergencyMedicine listservs.Theprerequisiteforbecomingareviewerincluded recognitionasanexperiencededucatorandauthorshipofat leastonescholarlyeducationalstudypublishedinapeerreviewedjournal.
Onceamanuscriptwassubmittedviathe WestJEM submissionportal,screeningeditorseitherapprovedthe manuscriptforpeerrevieworchoseto “desk-reject” themanuscriptwithoutreview.Manuscriptsthatpassedthe
screeningprocesswerethenassignedtotwoexternalpeer reviewers.Inaniterativeprocess,reviewersconcludedtheir reviewswitharecommendationtotheassociateeditorwhoin turnmadearecommendationtothesenioreditorfora formaldecision.Ateachstepinthisprocessthechoicewasto reject,revise,oracceptthemanuscript.Inthoseinstances whererevisionswererequestedandsubmitted,the final decisionwaseithertoacceptorrejectforpublication (Figure2).Reviewerassignmentwasrandomwithoutregard todefinedexpertise(eg,statistics,specificmethodology,topic underconsideration).
Throughthe first fiveeditionsoftheSpecialIssue,peer reviewswere “rated” byassociateandsenioreditorsusingthe methodsandinstrumentationadoptedfromtheparent WestJEM editorialboard:aclosed,internalevaluation
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 257
Figure2. Flowchartshowingthenumberofmanuscriptssubmittedandprocessedduringthe2020–21submissionscycleforthe Western JournalofEmergencyMedicine SpecialIssueinEducationalResearch&Practice. UGME,undergraduatemedicaleducation; CDEM,ClerkshipDirectorsofEmergencyMedicine; GME,graduatemedicaleducation; CORD, CouncilofResidencyDirectorsinEmergencyMedicine.
Loveetal. ScoringRubricforEvaluationofPeer-reviewQuality
systemthatusedaglobal five-pointsemanticdifferential scalewithlabelsattheextremes:5 = highquality and1 = low.
ImplementationoftheHolisticEditor’sScoringRubric
TheHESRwaspilotedduringproductionofthe 2020–2021SpecialIssue.Manuscriptsreviewedbythefour CORDeditorsoftheSpecialIssuewereincludedinthepilot. Tominimizeissuesofrecallandmaintainconsistency,the editorsallagreedtoreflectupontheHESRjustpriorto scoringeachpeerreview.Subsequently,eachpeerreview receivedanindependentscorefromanassociateeditorand thesenioreditor.Blindingwasaccomplishedbyhaving eacheditorreporttheirreviewscoretoathirdparty. Thethirdparty(AM),whowasnotinvolvedintheformal reviewprocess,maintainedthesecureddatabasethatlinked theeditor’sandassociateeditor’sHESRratingsfor eachreview.
PreliminaryCalibrationExercise(Pilot)
Theeditorspausedtoreviewtheirexperiencewiththe originalHESRafterthe first14peerreviewsofmanuscripts hadbeenscored.Comparisonsbetweenscoresassignedby theseniorandassociateeditorsraisedquestionsandconcerns abouttheclarityofthescoringrubric,whichwarranted anotherroundofrevisions.Changesweremadebasedon iterativediscussionsandconsensustoimprovetheclarityof therubricoptionsandratings.Inaddition,adjective descriptorsthatcharacterizedeachoptionwereaddedas follows:5-Excellent,4-Verygood,3-Good,2-Belowaverage, and1-Unacceptable(Figure1).Duringtheimplementation stage,the finalHESRwasusedtoscoretheremaining32 GMEsubmissionsduringthe2020-21SpecialIssue productioncycle.
DataAnalysis
Assumption1–DistributionofEvaluationScores
Our firstassumptionwasthatavalidscoringmechanism ofeditorialevaluationofpeerreviewsshouldreflectthe variabilityofqualityandvalueofthereviews.Theprevious semanticdifferentialratingsystemusedbytheSpecialIssue duringapriorproductioncycle(2019–2020)didnotreflecta highdegreeofvariabilityinpeer-reviewscores.Infact,the distributionofscoresfromthiscycleappearednegatively skewedwithscoresclusteredaround “4” onthe five-point semanticdifferentialscale.Accordingly,onegoalofthenew HESRwasforittomoreaccuratelyreflectthevariabilityof thereviewerpoolwithregardtoscholarlyexpertisethrough useoftheentireevaluationscale.Usingthe “Explore” featureinIBM-SPSSversion28(SPSS,Inc,Chicago,IL)to generatehistograms,frequencydistributions,andmeasures ofvariability,wecomparedthreesetsofpeer-reviewscores: semanticdifferentialratingsfromthe2019–2020Special Issue(baseline);thepilotCORDeditor’sevaluationofpeer
reviews;andthefullimplementationofCORDeditor’ s evaluationofpeerreviewsusingtherevisedHESRforthe 2020–2021edition.
Assumption2–Inter-raterReliabilityBetweenEvaluatorsof theSamePeerReviews
SincetheHESRprovidedclearcriteriafor fivedifferent levelsofpeer-reviewperformance,weexpectedtheHESRto generatereliablescoresacrosseditors.Accordingly,like Cicchetti,wecomparedinter-raterreliabilityamongeditors usingtheintraclasscorrelationcoefficient(ICC).39 To complementourreliabilityevaluation,wealsousedthree measuresofagreementbetweenassociateandsenioreditors’ ratingsofpeerreviewsfortheCORDeditorialteam:percent ofabsoluteandrelativeagreement,andtheSpearmanrho correlationforordinalleveldata.40 Thepercentageof identicalratingsisameasureofabsoluteagreementbetween raters,whilethepercentageofratingsincloseproximityof eachother(+1)isanindicatorofwithin-raterconsistency. TheSpearmanrhocorrelationprovidesanindicatorofthe strengthoftherelationshipbetweentheratingsacrossthe twotypesofraters.41 WeusedthecriteriafromSchoberetal. forinterpretingtheSpearmanrhocorrelation(rof
0–0.10 = negligible;rof0.10–0.39 = weak;rof0.40–0.69 = moderate;rof0.70–0.89 = strong,rof0.90–1-verystrong).40
TheICCmodelselectedforthisstudyisbasedonseveral assumptions.First,itisassumedthatassociateeditorswere randomlychosenfromalargerpoolandthatthesenior editorwas fixed.Second,thedesignwasnotfullycrossed, sincenoteveryreviewwasratedbythesameeditors.Third, sinceoneratingwasthefocus,ratherthanaseries,the absoluteagreementwasthoughttobethemostappropriate ICCmodel.A finalassumptionwasthatsincetheICCwas beingaskedtorepresenttheaverageofseveralcoders,the “ averagemeasures ” ICCwaschosen.Insummary,theICC formulachosenforthisstudyisaone-wayrandomeffects modelreflectingabsoluteagreementandtheunitofanalysis relatedtoaveragemeasures.42 Weappliedguidancefrom CicchettiforinterpretingtheresultingICCreliabilityindices (ICCof <.4 = poorreliability;ICCof.40–.59 = fair reliability;ICCof.60–.74 = goodreliability;ICCof .75–1.0 = excellentreliability).39 Unfortunately,wewerenot abletoperformcomparableinter-raterreliabilityanalyses forthepriorSpecialIssueproductioncycle(2019–2020)due tothetemplatedabilitytoprovideonlyoneeditor’ sscore permanuscript.
Assumption3–ImplicationsortheStatisticalRelationship BetweenPeer-ReviewRatingandtheEditorialDecision
Thecollectiveeditors’ evaluationsofthepeerreviewwere assumedtobeanindicatorofitsquality.Ifeditorsplaced valueonpeerreviewsduetotheirabilitytoinformthe decision-makingprocess,thenhigherqualitypeerreviews shouldhavebeenmorelikelytoagreewiththeeditorial
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 258 ScoringRubricforEvaluationofPeer-reviewQuality Loveetal.
decisionsthanlowerqualitypeerreviews.Inthisanalysis,the categoriesofyes/norefertowhetherthereviewer’ s recommendationagreedwiththeformalmanuscript decision. “Yes” designationswereappliedifthereviewer recommendedthearticlebeaccepted,rejectedorrevised andtheeditorialdecisionmadeagreedwiththat recommendation.Ifthereviewer’srecommendationdidnot agreewiththeeditor’sdecision,thiswascategorizedasa “No.” Thisisknownasaparallellineofvalidityevidence accordingtoKane.18 Wetestedthishypothesisbyaveraging theseniorandassociateeditors’ peer-reviewratingsandthen categorizingtheseaverageratingsinto fivecategories: (1–1.5);(2–2.5);(3–3.5);(4–4.5);and(5).Next,usingachisquaretestofassociationwetestedtherelationshipbetween thesummaryratingcategoryandthereviewer’sagreement withthe finaldecision(Didthereviewer’srecommendation agreewiththe finaldecision,yesorno?).Iftrue,theauthors positedthatthehighertheratingsbytheeditorsonthe qualityofthepeerreview,themorelikelytheir recommendationsforthemanuscriptsubmissionwould agreewiththeactualformaldecision.Weappliedthecriteria fromHahs-Vaughnetalforinterpretingtheassociatedeffect sizesfromthechisquaretestofassociation(smalleffects = <0.10,mediumeffects = 0.30;andlargeeffectsare >0.50).43
WealsoevaluatedtherelationshipbetweentheHESR scoreandthereviewer’sagreementwiththe finaldecision usinglogisticregressionanalysis.Forthisanalysis,we attemptedtopredictwhetherthereviewerrecommendation wouldmatchthe finaleditorialdecision(yesorno)fromthe HESRscoresassignedbyeachtypeofeditor.Resultsofthis testshouldprovidearelativestrengthoftherelationship betweeneachtypeofeditor’sratingandthe editorialdecision.
RESULTS
Thetotalnumberofmanuscriptssubmittedforthe2021 SpecialIssuewas163.Ofthese,85weremanagedbythe
CORDeditors.Thirty-eightsubmissionsweredesk-rejected bytheeditorialstaff.Subsequently,47(55.3%)manuscripts wereapprovedforpeerreviewand16werepublished,foran acceptancerateof18.8%.These47peer-reviewed manuscriptswerethesubjectofthisstudy,14duringthe pilotperiodand33duringfullimplementationofthe HESR(Figure2).
Eighty-fourpeerreviewersreviewedanaverageof1.84 manuscriptseach(SD1.34).Abouttwo-thirdsofpeer reviewersperformedonlyonereview(52/85;61.2%),while anadditional34%(29/85)completed2-4reviews,andfour individuals(4.8%)completed5– 7reviews.Theeditors performed91evaluationsofpeerreviews,27atthepilot stageand64atthefullimplementationstage.Thethree associateeditorsperformed95peer-reviewevaluations,32 atthepilotstageand63atthefullimplementationstage (Table1).Therewere90matchedpairsofevaluations onthesamepeerreviewfromboththeseniorand associateeditor.
DistributionofScores
Duringthepriorproductioncycle(2019–2020),163peer reviewswereratedusinga five-pointsemanticdifferential scale.Thedistributionofeditors’ ratingsofthesereviewswas showntobenegativelyskewed( 0.371),whichwascaused bytheunderuseofthe “1” ratingandoveruseofthe “4” and “5” ratings(Figure3).Contrastedwiththesemantic differentialscale,theHESRdistributionatboththepilotand fullimplementationstagehadskewnessclosertozero(0.005 and0.078,respectively).Thedistributionduringthepilot stageisconsideredaparalleldistribution,sincealmostall responseoptionswerechosenequally(exceptforthe “1” HESRrating).Duringthefullimplementation, negativeskewness(0.078)almostdisappearedasthe distributionbecamemorebellshaped,andkurtosis continuedtosuggestadistributionwithsymmetry (kurtosis = 0.967)(Figure3).44
Table1. Numberandpercentagesofsenioreditor,associateeditors,andreviewersinvolvedintheproductionofthe2021SpecialIssueby group.Includedarethenumbersofreviewevaluationsperformedandmanuscriptsprocessedbyseniorandassociateeditorsandthe numbersofpeerreviewersandmanuscriptstheyreviewed.
CORDHESRStudyTotal
Calibration-pilotImplementation
Associate editors
Reviewers*32N/A1469N/A3384N/A47
*Therewere84totalpeer-reviewerswhoreviewedmanuscriptsduringeitherthepilotorfullimplementationphaseofthisstudy.Fifteenof32 reviewersparticipatedonlyduringthepilotwhiletheother17contributedtoreviewingatbothstagesoftheproject(pilotandfull).Reviewers werenotinvolvedwithusingtheHolisticEditor’sScoringRubrictoassesstheirownpeer-reviews. CORD,CouncilofResidencyDirectorsinEmergencyMedicine; HESR,holisticeditor’sscoringrubric; evals,evaluations.
Personnel Review evalsManuscriptsPersonnel Review evalsManuscriptsPersonnel Review evalsManuscripts
Senioreditor127141633319047
332143633339547
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 259 Loveetal. ScoringRubricforEvaluationofPeer-reviewQuality
Figure3. Comparisonofthepeer-reviewratingsdistributionsfortwomethodsofeditorevaluationsacrosstwoSpecialIssueproduction cycles.The firstmethodinvolveda5-pointsemanticdifferentialscalewithlabelsonlyattheendpoints,whichwasusedduringthe2020cycle. Thesecondmethodinvolvedtheholisticeditor’sscoringrubricusedbytheCORDeditorsatthepilotandfullimplementationstagesofthe 2021productioncycle.
CORD,CouncilofResidencyDirectorsinEmergencyMedicine. *Datasignificantlydeviatefromnormaldistribution(P ≤ 0.001).
Resultsoflogisticregressionusingtheoutcome,whetherthereviewerrecommendationmatchedwiththe finaldecision(yesorno), regressedonthepredictors:associateandsenioreditor’speer-reviewratings.
Equation1:Associateeditor’sscoreaspredictorforreviewrec/finaldecmatch[93]
95%CIforEXP (B)
Equation2:Senioreditorscoreaspredictorforreviewerrec/ finaldecmatch[N = 91]
95%CIforEXP (B)
Equation3:Associate&senioreditorscoresaspredictorforreviewerrec/finaldecmatch[N = 90]
95%CIforEXP
Inter-raterAgreement
NinetypeerreviewswereassessedwiththeHESRbyboth thesenioreditorandoneofthethreeassociateeditors.The
percentageofabsoluteagreementbetweenthetwotypesof editors’ ratingsofpeerreviewswas37.8%(Table2).Nearly half(47.8%)ofratingswereindisagreementbyonlyone
Table2.
Associateeditorreviewscore.627.18411.5711 <.0011.8711.3042.68567.7 Constant 2.054.63110.5871.001.128
BSEWalddfSig.Exp(B)LowerUpper%CC
BSEWalddfSig.Exp(B)LowerUpper%CC Senioreditorreviewscore.935.24714.3501 <.0012.5481.574.13363.7 Constant 2.664.72113.6331 <.001.070
Associateeditorreviewscore.137.244.3141.5751.146.7111.84966.7 Senioreditorreviewscore.834.3137.0841.0082.3021.2464.254 Constant 2.789.76513.3001 <.001.061 Full-2021 Comp Editorial Rubric Pilot-2021 Comp Editorial Rubric 2020 Cycle -Diff Scale 137 73 163 N 2.91 3.32 3.64 Mean 1.26 1.20 1.05 Std Dev 3.00 3.00 4.00 Median .078 .005 -.371 Skew -.967 -1.13 -.681 Kurtosis .152* .165 .215* Normality
(B) BSEWalddfSig.Exp(B)LowerUpper%CC
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 260 ScoringRubricforEvaluationofPeer-reviewQuality Loveetal.
point.TheassociatedSpearmanrhocorrelationand r-squaredfortheratingsfromthetwotypesofassessorswas 0.703(R2 = 0.49).ASpearmanrhoofthismagnitudeis interpretedasborderingbetweenamoderateandstrong positivecorrelationorstatisticalrelationship.40 Finally,the ICCbetweentheassociateandsenioreditorswas0.795.
AsinterpretedbyCicchetti,anICCofthismagnitudeis consideredexcellentintermsofclinicalsignificance (between0.75–1.00).39
ImplicationofHESRScoresasAssociatedwithand PredictorsofManuscriptOutcomes
Thechi-squaretestofassociationfortherelationship betweenaveragepeer-reviewratingsandthepeerreviewer’ s recommendationwiththe finalmanuscriptdecisionwas statisticallysignificant(chi-square = 17.4,df = 4, P < 0.01, effectsize = 0.44).Theassociatedeffectsizeof0.44is classifiedasamediumeffectsizeaccordingtoHahs-Vaughn etalwhosuggestthatsmalleffectsarethose ≤ 0.10,medium effects = 0.30,andlargeeffects >0.50(Table3).43
LogisticRegression
Foralllogisticregressionanalyses,thetestsformodel coefficientsweresignificant,suggestingthatanyoneofthe threeformulaswouldimproveourestimateoftheprobability thatthepeer-reviewrecommendationmatchedtheeditorial decision.TheHosmer-Lemeshowtestswerenotsignificant, indicatingthatthemodelscouldbeagood fit,andanalysesof thescatterplotsofpredictedscoresandresidualscontributed totheconclusionthattheanalysesmettheassumptionsof normalityandequalvariance(statisticsnotshown).
Table3. Reviewersummaryratinggroupedinto5categoriescross tabulatedwithwhetherthereviewer’srecommendationagreedwith manuscript finaldecision(expectedvaluesareinparentheses)with chisquaretestofassociation*betweenthesetwovariables.
Didreviewer’srecommendation agreewith finaldecision Reviewersummary
Logisticregressionanalysesdemonstratedthatthe associateeditor’sHESRratingswereasignificantpredictor ofthemanuscriptoutcome:asuccessfulmatchbetweenthe reviewer’srecommendationandthe finaldecisionforthe manuscript.Thiswasalsotrueofthesenioreditor’sHESR ratings.However,becausetheratingsoftheassociateeditor andsenioreditorweresohighlycorrelatedwitheachother (Spearmanrhocorrelation = 0.703),oncecombinedintoone logisticregressionmodel,onlythesenioreditor’sratings surfacedasasignificantpredictor.
Interpretationofthesenioreditor’sHESRratingsasa predictorsuggeststhattheratingscontributedtoimproving thecorrectclassificationofpredictedvsobservedoutcomes from51.6%withnopredictorto63.7%.Theadjustedodds ratioExp(B) = 2.548(95%confidenceinterval[CI]
1.570–4.133]canbeinterpretedasfollows: “Foreveryone stepincreaseinthesenioreditor’sevaluationratings,therisk oftheoutcomeofasuccessfulmatchbetweenreviewer recommendationandthe finaldecisionincreasesbyafactor of2.548” (Table2).
DISCUSSION
UsingKane’sframeworkforvalidityevidence,thiswork testedthreeassumptionsregardingtheHESRasanovel meansforeditorstoassessthequalityofpeerreviewsof educationalscholarship.The firstassumptioninvolvedthe distributionofeditors’ ratingsofreviews, findingthatthe HESRdemonstratedgreatersymmetryinscorescompared toapriorinstrumentusedduringthe2019cycle.We concludethatleniencybiaswaslikelylimitedbythecriteriareferencingbasisofthisintervention,whichmayhave resultedfromclearerbehavioralanchorsoftheinstrument itself,5,6,13 improvementsinratertraining,3,10,13,24–28 and/or moreintentionalqualitycontrolamongeditorsduringthe reviewprocess.Thischangeinbehaviormayalsoreflectthe practicalityoftheHESRsinceitwasclearlybeingusedin editorialevaluationofreviews.
ThesecondassumptionmadeinfersthattheassignedpeerreviewscoresbasedontheHESRareanaccurate representationoftheeditors’ perspectiveonqualityand informaholisticperspectiveonthesubmission.Whilethe editorsoftheSpecialIssuehadlowerabsoluteagreement betweentheseniorandassociateeditors(37.8%),they demonstratedexcellentrelativeconsistencyreliability (ICC = 0.795)andcorrelations(Spearmanrho = 0.703) betweeneditors’ scores.Inotherwords,theirevaluations, whilenotidentical,wereinternallyconsistent.This finding relatedtoreliabilitysupportsthehypothesisthattheHESR capturestheeditorialperceptionofqualityaswellasthe degreetowhichpeerreviewinformsaholisticunderstanding ofasubmissionsvalue.
*Theminimumexpectedcountsare3.42.Cramer’sphieffect sizesareinterpretedas ≤ 0.10 = small;0.30 = medium;and ≥ 0.50 = largeeffects.
Finally,thethirdassumptionsmadehastodowiththe implicationthatscoresareusedtoinformtheeditorial decision-makingprocess.Thisissubstantiatedbythe finding
rating*NYesNo Percent agreement 1.00174(8)13(9)23.5 2.00216(10)15(11)28.6 3.002817(14)11(14)60.7 4.001710(8)7(9)58.8 5.0077(3)0(4)100 TOTAL9044(44)46(46)48.9 X2 = 17.4,df = 4,p = 0.006, es = 0.440
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 261 Loveetal. ScoringRubricforEvaluationofPeer-reviewQuality
thattheratingsontheHESRcorrespondedtothereviewer’ s agreementwiththemanuscript’ s finaldisposition.Thehigher thepeer-reviewevaluationofqualitybyHESRscoring,the higherthecorrelationbetweenthereviewerdisposition recommendationandthemanuscript’sformaloutcome. Althoughthisistobeexpected,thefactthatitholdstruein thisinstancedemonstratesthateditorsvalueandrate reviewshigherwhentheycontributesubstantiallytothe editorialdecision.18
LIMITATIONSANDFUTURESTUDIES
These findingsshouldbeinterpretedinthecontextof severallimitations.Traditionally,theSpecialIssuehasnot blindeditsseniorandassociateeditorstotheidentityofthe reviewers.Thisraisesthepotentialforbiasifeditors recognizedpeerreviewers’ names,whichcouldconceivably haveimpactedtheratingsofmorefamiliarpeerreviewers. Second,theeditorsinthisstudyhadregulardiscussions regardingtheuseandinterpretationofthisscoring instrument.Giventhecentralityofratertrainingintheuseof anyevaluationinstrument,futureworkwillhelpto determinewhethertheperformanceoftheHESRandlackof skewnessinscoringpersistsbeyondtheeditorialfocus associatedwiththisstudy.
Mostimportantly,thisstudyisbasedonasinglecycleofan annualspecialty-relatedpublicationfocusedonhealth professions-educationtopicswithasmallnumberofeditors andreviews.Futurestudiesshouldfocusonassessing additionalvalidityevidencesupportingtheHESR’ suse aswellasvaryingjournalenvironmentswithlargernumbers ofeditorsandreviews.Ourresultsaremostlikelytogeneralize tospecialty-specificeducationjournalswhoseapproachis similartothatpresentedinthisstudy.Our findingsareless likelytogeneralizetojournalsthattakeanalternative approachsuchasthosethatbringtogetheradiversesetof reviewersbasedonexpertise(eg,methodology, psychometrics,content,etc)toassessspecificcomponents ofthesubmission.
CONCLUSION
Aholisticunderstandingofthevalueofascholarly submissionrequiresaniterativeprocessinformedbythe expertperspectiveofreviewersthatisoftensubjectiveand nuanced.Theholisticeditor’sscoringrubricwasdevelopedas apracticalapproachtoeditorialevaluationofthequalityofa reviewandthedegreetowhichitinformstheformaleditorial decision.Bystudyingaprioriassumptionsrelatedtothe developmentanduseoftheHESR.includingdistributionsof evaluationscores,inter-raterreliabilitybetweenevaluatorsof thesamepeerreviewsandthestatisticalrelationshipbetween peer-reviewratingandtheeditorialdecision,thisstudy providesvalidityevidencesupportingtheuseoftheHESR. Futureworkshouldfocusonfurtherdefiningthevalueand limitationsoftheHESR.
AddressforCorrespondence:JeffreyN.Love,MD,4917HineDr., ShadySide,MD20764.Email: Jlove01@georgetown.edu
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.
Copyright:©2024Loveetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.LockS. ADifficultBalance:EditorialPeerReviewinMedicine Philadelphia,PA:ISIPress,1986.1–38,122–132.
2.CicchettiDV.Thereliabilityofpeerreviewformanuscriptandgrant submissions:across-disciplinaryinvestigation. BehavBrainSci. 1991;14(1):119–35.
3.StrayhornJrJ,McDermottJrJF,TanguayP.Aninterventiontoimprove thereliabilityofmanuscriptreviewsfortheJournaloftheAmerican AcademyofChildandAdolescentPsychiatry. AmJofPsychiatry. 1993;150(6):947–52.
4.HowardLandWilkinsonG.Peerreviewandeditorialdecisionmaking. BrJPsychiatry. 1998;173:110–3.
5.CallahamML,BaxtWG,WaeckerleJF,etal.Reliabilityofeditors’ subjectivequalityratingsofpeerreviewsofmanuscripts. JAMA. 1998;280(3):229–31.
6.LandkroonAP,EuserAM,VeekenH,etal.Qualityassessmentof reviewers’ reportsusingasimpleinstrument. ObstetGynecol. 2006;108(4):979–85.
7.BormannL,MutzR,DanielHD.Areliability-generalizationstudyof journalpeerreviews:amultilevelmeta-analysisofinter-raterreliability anditsdeterminants. PLoSOne. 2010;5(12):e14331.
8.SmithR.PeerReview:A flawedprocessattheheartofscienceand journals. JRSocMed. 2006;99(4):179–82.
9.FeurerID,BeckerGJ,PicusD,etal.Evaluatingpeerreviews:pilot testingofagradinginstrument. JAMA. 1994;272(2):98–100.
10.VanRooyenS,BlackN,GodleeF.Developmentofthereview qualityinstrument(RQI)forassessingpeerreviewofmanuscripts. JClinEpidemiol. 1999;52(7):625–9.
11.OnitiloAA,EngelJM,Salzman-ScottSA,etal.Acore-itemreviewer evaluation(CoRE)systemformanuscriptpeerreview. AccountRes. 2014;21(2):109–21.
12.DoiSAR,Salzman-ScottSA,OnitiloAA.ValidationoftheCoRE questionnaireforamedicaljournalpeerreview. AcctResear. 2016;23(1):47–52.
13.GingerichA,KoganJ,YeatesP,etal.Seeingthe “blackbox” differently: assessorcognitionfromthreeresearchperspectives. MedEduc. 2014;48(11):1055–68.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 262 ScoringRubricforEvaluationofPeer-reviewQuality Loveetal.
14.ColeS,ColeJR,SimonGA.Chanceandconsensusinpeerreview. JSci. 1981;214(4523):881–6.
15.GarfunkelJM,UlshenMH,HamrickHJ,etal.Problemsidentified bysecondaryreviewofacceptedmanuscripts. JAMA. 1990;263(10):1369–71.
16.KassirerJPandCampionEW.Peerreview:crudeandunderstudied,but indispensable. JAMA. 1994;272(2):96–7.
17.DeMariaAN.Whatconstitutesagreatreview? JACC. 2003;42(7):314–5.
18.CookDA,BrydgesR,GinsburgS,etal.Acontemporaryapproachto validityarguments:apracticalguidetoKane’sframework. MedEd. 2015;49(6):560–75.
19.ThompsonSR,AgelJ,LosinaE.TheJBJSpeer-reviewscoringscale: avalid,reliableinstrumentformeasuringthequalityofpeerreview reports. LearnPubl. 2016;29(1):23–5.
20.RajeshA,CloudG,HarisinghaniMG.Improvingthequalityof manuscriptreviews:impactofintroducingastructuredelectronic templatetosubmitreviews. AJR. 2013;200(1):20–3.
21.CaelleighAS,SheaJA,PennG.Selectionandqualitiesofreviewers. AcadMed. 2001;76(9):914–5.
22.VarsGF,WentworthJ,SebbersonD.Educationalconnoisseurship, criticism,andtheassessmentofintegrativestudies. IssuesStud. 2002;20:65–76.
23.MillerKH,WestDC,KaraniR.Foreword:sticksandstones:moving towardamoreproductivepeer-reviewprocessforauthorsand reviewers. AcadMed. 2017;92(11S):Si–iii.
24.AlbaneseJ,PruchaC,BarnetJH.Labelingeachresponseoption andthedirectionofthepositiveoptionsimpactstudentcourseratings. AcadMed. 1997;72(10Suppl1):S4–6.
25.SattlerDN,McKnightPE,NaneyMR.Grantpeerreview:improvinginterraterreliabilitywithtraining. PLoSOne. 2015;10(6):e0130450.
26.JacksonJS,BondM,LoveJN,etal.Emergencymedicinestandardized letterofevaluation(SLOE):datafrom2016-2017eSLOEelectronic website. JGradMedEduc. 2019;11(2):182–6.
27.RobbinsJ,FirthA,EvansM.Improvingwork-basedassessment: addressinggradeinflationnumericallyorpedagogically. Practitioner ResearchinHigherEducation. 2018;11(1):80–6.
28.Gomez-GaribelloCandYoungM.Emotionsandassessment: considerationforrater-basedjudgementsofentrustment. JMedEd. 2018;52(3):254–62.
29.CouncilofResidencyDirectorsofEmergencyMedicine:Visionand MissionStatements.Availableat: https://www.cordem.org/about-cord/ mission–vision/mission–vision/.AccessedFebruary17,2020.
30.TaskForceofAcademicMedicineandtheGEA-RIMECommittee. Checklistofreviewcriteriaforresearchmanuscripts. Academic Medicine. 2001;76(9):958–9.
31.MedicalEducation:GuidelineforAuthors.Availableat: https:// onlinelibrary.wiley.com/page/journal/13652923/homepage/ guidelines_for_reviewers.htm.AccessedFebruary17,2020.
32.WileyAuthorServices.Howtopeerreview:stepbystepguideto reviewingamanuscript.Availableat: https://authorservices.wiley.com/ Reviewers/journal-reviewers/how-to-perform-a-peer-review/ step-by-step-guide-to-reviewing-a-manuscript.html
AccessedFebruary17,2020.
33. JGradMedEduc.Instructionsfor JournalofGraduateMedical Education (JGME)PeerReviewers.2018.Availableat: https://meridian.allenpress.com/DocumentLibrary/GMED/ JGME-Peer-Reviewer-Instructions_April%202018.pdf
AccessedFebruary17,2020.
34.BandieraGandLendrumD.Dailyencounteredcardsfacilitate competency-basedfeedbackwhileleniencybiaspersists. CJEM. 2008;10(1):44–50.
35.ArcherJCandMcAvoyP.Factorsthatmightunderminethe validityofpatientandmulti-sourcefeedback. MedEduc. 2011;45(9):886–93.
36.LoveJN,DeiorioNM,Ronan-BentleS,etal.Characterizationofthe CORDstandardizedletterofrecommendationin2011-2012. Acad EmergMed. 2013;20(9):926–32.
37.Bing-YouR,VaraklisK,HayesV,etal.Thefeedbacktango: anintegrativereviewandanalysisofthecontentofthe teacher-learnerfeedbackexchange. AcadMed. 2018;93(4):657–63.
38.LoveJNandAnderDS.Buildingaspecialtyspecificcommunity ofpracticeineducationscholarship. WestJEM. 2015;16(6):799–800.
39.CicchettiDV.Guidelines,criteria,andrulesofthumbforevaluating normedandstandardizedassessmentinstrumentsinpsychology. PsycholAssess. 1994;6(4):284–90.
40.SchoberP,BoerC,SchwarteL.Correlationcoefficients:appropriate useandinterpretation. AnesthAnalg. 2018;126(5):1763–8.
41.KottnerJ,AudigéL,BrorsonS,etal.Guidelinesforreportingreliability andagreementstudies(GRRAS)wereproposed. IntJNursStud. 2011;48(6):661–71.
42.HallgrenKA.Computinginter-raterreliabilityforobservationaldata: anoverviewandtutorial. TutorQuantMethodsPsychol. 2012;8(1):23.
43.Hahs-VaughnDLandLomaxRG.Chapter8.In: AnIntroductionto StatisticalConcepts(312).(4th Edition).NewYork,NY:Routledge,2020.
44.GlassGVandHopkinsKD. StatisticalMethodsinEducationand Psychology,2ndEd.EnglewoodCliffs,NJ:Prentice-HallInc; 1984:68–70.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 263 Loveetal. ScoringRubricforEvaluationofPeer-reviewQuality
NovelScoringScaleforQualityAssessmentofLungUltrasound intheEmergencyDepartment
JessicaR.Balderston,MD*
TaylorBrittan,MD*
BruceJ.Kimura,MD†
ChenWang,MS‡
JordanTozer,MD*
SectionEditor:RyanGibbons,MD
*VirginiaCommonwealthUniversityMedicalCenter,DepartmentofEmergency Medicine,Richmond,Virginia
† ScrippsMercyHospital,DepartmentofCardiology,SanDiego,California
‡ VirginiaCommonwealthUniversity,DepartmentofBiostatistics,Richmond,Virginia
Submissionhistory:SubmittedMay15,2023;RevisionreceivedNovember28,2023;AcceptedDecember5,2023
ElectronicallypublishedFebruary28,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18225
Introduction: Theuseofareliablescoringsystemforqualityassessment(QA)isimperativetolimit inconsistenciesinmeasuringultrasoundacquisitionskills.ThecurrentgradingscaleusedforQA endorsedbytheAmericanCollegeofEmergencyPhysicians(ACEP)isnon-specific,applies irrespectiveofthetypeofstudyperformed,andhasnotbeenrigorouslyvalidated.Ourgoalinthisstudy wastodeterminewhetherasuccinct,organ-specificgradingscaledesignedforlung-specificQAwould bemoreprecisewithbetterinterobserveragreement.
Methods: ThiswasaprospectivevalidationstudyofanobjectiveQAscaleforlungultrasound(LUS)in theemergencydepartment.Weidentifiedthe first100LUSperformedinnormalclinicalpracticeinthe year2020.Fourreviewersatanurbanacademiccenterwhowereeitheremergencyultrasound fellowship-trainedorcurrentfellowswithatleastsixmonthsofQAexperiencescoredeachstudy, resultinginatotalof400.Theprimaryoutcomewasthelevelofagreementbetweenthereviewers.Our secondaryoutcomewasthevariabilityofthescoresgiventothestudies.Fortheagreementbetween reviewers,wecomputedtheintraclasscorrelationcoefficient(ICC)basedonatwo-wayrandom-effect modelwithasingleraterforeachgradingscale.Wegenerated10,000bootstrappedICCstoconstruct 95%confidenceintervals(CI)forbothgradingsystems.Atwo-sidedone-sample t-testwasusedto determinewhetherthereweredifferencesinthebootstrappedICCsbetweenthetwogradingsystems.
Results: TheICCbetweenreviewerswas0.552(95%CI0.40–0.68)fortheACEPgradingscaleand 0.703(95%CI0.59–0.79)forthenovelgradingscale(P < 0.001),indicatingsignificantlymore interobserveragreementusingthenovelscalecomparedtotheACEPscale.Thevarianceofscoreswas similar(0.93and0.92forthenovelandACEPscales,respectively).
Conclusion: Wefoundanincreasedinterobserveragreementbetweenreviewerswhenusingthenovel, organ-specificscalewhencomparedwiththeACEPgrading scale.Increasedconsistencyinfeedback basedonobjectivecriteriadirectedtothespecific,targetedorganprovidesanopportunitytoenhancelearner educationandsatisfactionwiththeirultrasoundeducation.[WestJEmergMed.2024;25(2)264–267.]
INTRODUCTION
Lungultrasound(LUS)isfrequentlyusedinthe emergencydepartment(ED)toassessbothmedicaland traumapatients.1,2 Qualityassessment(QA)ofultrasound imagesisoneofthesixrequiredelementsofdiagnostic
ultrasoundpertheAmericanCollegeofEmergency Physicians(ACEP)andisroutinelyperformedtoevaluate imagequality,ensuringappropriatepatientcare,and enablingreviewerstoassessuserperformance.2 The useofareliablescoringsystemforQAisimperative
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 264 BRIEF RESEARCH REPORT
tolimitinconsistenciesinmeasuringultrasound acquisitionskills.
ThecurrentQAgradingscaleendorsedbyACEPwas developedfromaconsensusreportofemergencyultrasound leaderstoprovideasystematicmethodtoreportand communicateultrasound findings.2 Itisanon-specificscale thatappliesirrespectiveofthetypeofstudyperformedand hasnotbeenrigorouslyvalidated.Similarlyformatted organ-specificQAgradingsystemsforcardiacandobstetric examshavebeendescribedbutarenotyetendorsedby ACEPandarenotwidelyused.2–5 AlternativeLUS assessmenttoolshavebeendeveloped;however,theyare extensiveandassuchimpracticalforroutineQAuseorare focusedonimageacquisitionskillsandnottailoredto anatomicfeedback.6,7 Ourgoalinthisstudywasto determinewhetherasuccinct,organ-specificgradingscale designedforQAwouldbemoreprecisewithbetter interobserveragreement.
METHODS
ThiswasaprospectivevalidationstudyofanobjectiveQA scaleforLUS.Wedevelopedanovel,lung-speci ficgrading scalebyarigorousreviewofexpert,publishedexperienceat anoutside,unaffiliatedinstitution(ScrippsMercyHospital, SanDiego,CA).Thisinstitutionroutinelyperformslung imagingandhaspublishedanassessmenttoolforthe evaluationofresident-performedbedsideultrasoundB-line interpretationinthoracicultrasound,aswellasananalogous cardiacqualityassessmentscale.3,7–13 Intheexpertreview, thecurrentavailable,organ-specificgradingscalefoundin theliteraturewasmodifiedtotheanatomyofthechest wall.3,5 Thegradationsofthescalewereempiricallyderived fromtheexperienceatthisinstitutioninadditiontoa rigorousreviewoftheliterature. 3,5,7–13 Theuseoffour criticallandmarks ribshadows,pleuralline,A/Blines,and technical flaws wererecognizedascommonalitiesinall publishedimagesinLUSstudies,includingexpert consensus.14,15 We,therefore,dividedtheselandmarksintoa pointscalethatprogressivelydefinesthepatternof acquisitionrequiredtoobtainanimage(ie,bones first, pleuralline,followedbyartifacts).Wedescribedtechnical flawsasnon-optimizeddepth/gain,distractingadjacent structures,inadequateaxis,orhandmovement.Wedeemed flawstobemajoriftheywerepresenttoadegreesignificant enoughtodecreasediagnosticcapabilities,orifmultiple flawswerepresent.
Thescalewasthenvalidatedatanurbanacademictertiary carecenterinRichmond,Virginia.Weidentifiedthe first100 LUSstudiescompletedaspartofregularclinicalpracticein theEDbyemergencyphysicianswithtwoormoreLUS videosperformedintheyear2020.Dedicatedthoracic ultrasoundexaminationsareingeneralperformedby residentphysicianswithattendingoversight.Studieswere obtainedusingSonositeXPorteultrasoundmachine
PopulationHealthResearchCapsule
Whatdowealreadyknowaboutthisissue?
Areliablemethodofqualityassessment(QA) ofultrasoundimagesisimperativetoassess userperformanceandlimitinconsistenciesin measuringultrasoundacquisitionskills.
Whatwastheresearchquestion?
IsthereaQAscoringscaleforlungultrasound (LUS)thatismoreprecisethanthe commonlyusedACEPscoringscale?
Whatwasthemajor findingofthestudy?
IntheQAofLUS,anovelscoringscale showedsigni fi cantlymoreinterobserver agreementcomparedtothe ACEPscale.
Howdoesthisimprovepopulationhealth?
AmoreindividualizedscoringscaleforQAof LUSresultsinlessgradingvarianceandmore objectivefeedbackwhencomparedtothe ACEPscale.
(FujifilmSonosite,Bethell,WA)usingeithertheC60XP5-2MHzcurvilineartransducer,L2513-6-MHzlineararray transducerortheP195-1-MHzphasedarrayprobe.Four reviewerswhowereeitheremergencyultrasoundfellowshiptrainedorcurrentfellowswithatleastsixmonthsofQA experiencescoredeachofthe100studiesresultinginatotal of400.TwoblindedreviewersusedthecurrentACEP gradingscale,2 andtwousedanovellung-specificgrading scale;therewasonefellowandoneultrasound-trained physicianineachgroup(Figure).Theprimaryoutcomewas thelevelofagreementbetweenthereviewers,indicatingthe reliabilityofthescoringsystem.Oursecondaryoutcomewas thevariabilityofthescoresgiventothestudies.Forthe agreementbetweenreviewers,wecomputedtheintraclass correlationcoefficient(ICC)basedontwo-wayrandomeffectmodelwithasingleraterforeachgradingscale.Ten thousandbootstrappedICCsweregeneratedtoconstruct 95%confidenceintervals(CI)forbothgradingsystems.We usedatwo-sidedone-sample t -testtodeterminewhether thereweredifferencesinthebootstrappedICCsbetweenthe twogradingsystems.
RESULTS
The first100LUSstudiescompletedintheEDby emergencymedicineresidents(postgraduateyear[PGY]-1,
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 265
NovelScoringScaleforQAofLUSintheED
Balderstonetal.
1 No recognizable features of ribs, pleural line, or A/B line artifacts No recognizable structures
2 1 or 2 rib shadows seen with minimal identifiable pleural line (off axis)
3 1 or 2 rib shadows and pleural line seen, with major technical flaws
4 2 rib shadows seen, framing either A- or Blines, but with minor technical flaws
5 2 rib shadows seen, framing either A- or Blines, with no technical flaws
Minimally recognizable structures but insufficient for diagnosis
Minimal criteria met for diagnosis, recognizable structures but with some technical or other flaws
Minimal criteria met for diagnosis; all structures imaged well
Minimal criteria met for diagnosis; all structures imaged with excellent image quality
Figure. Comparisonofthenovel,lungultrasoundquality assessmentscalewiththetraditionalAmericanCollegeof EmergencyPhysiciansscale.
ACEP,AmericanCollegeofEmergencyPhysicians; LUS, lungultrasound.
42%;PGY-2,14%;PGY-3,22%)andEDfaculty(22%)were reviewedbyfourblindedreviewers.Imageswereobtained usingthelinearprobe(27%),curvilinearprobe(32%),phased arrayprobe(28%),oracombinationofprobes(13%).Studies hadamedianofsixclips(IQR4–9).Thescoresgivenusing theACEPscaleandthenovelscalearesummarizedinthe Table.TheICCbetweenreviewerswas0.552(95%CI 0.4–0.68)fortheACEPgradingscaleand0.703(95%CI0.59,0.79)forthenovelgradingscale(P < 0.001),indicating significantlymoreinterobserveragreementusingthe novelscalecomparedtotheACEPscale.Thevarianceof scoreswassimilar(0.93and0.92forthenovelandACEP scales,respectively).
Summarytableofscoringsystems.
StatisticsNovelACEP
N*200200
Min,max1,51,5 Mean(SD)3.70(0.96)3.32(0.96)
Median43 Q1,Q33,43,4 ICC(95%CI)0.703(0.59,0.79)0.552(0.40,0.68)
Variance0.930.92
Varianceratio (Novel:ACEP) 1.01
*N = numberofscoresgiven.
ACEP,AmericanCollegeofEmergencyPhysicians; ICC,intraclass correlationcoefficient; CI,confidenceinterval.
DISCUSSION
ThecurrentACEPgradingscaleusedforQAwas developedfromaconsensusreportofemergencyultrasound leadersbuthasnotbeensystematicallyvalidated.2 Theuseof areliable,validatedscoringsystemforQAisimperativeto limitinconsistenciesandensureobjectivityinmeasuring ultrasoundacquisitionskill.Thevaguelanguageusedinthe ACEPscalemaycontributetovariableinterpretationby thoseassessingstudies,leadingtodiscrepanciesingrading ultrasoundskill.Inconsistentfeedbackmayconfusethe learnerandhindergrowthoftechnicalskill.Inourstudy,we foundthattherewasanincreasedinterobserveragreement betweenreviewerswhenusingthenovel,organ-specificscale whencomparedwiththeACEPgradingscale.Increased consistencyinfeedback,combinedwithdirectedfeedbackto thespecifictargetedorgan,providesanopportunityto enhancelearnereducationandsatisfactionwiththeir ultrasoundeducation.
Organ-specificcardiacandobstetricQAgradingsystems havebeendescribed,althoughtheyhavenotyetbeenwidely adoptedinclinicalpractice.3–5 Thisisthoughttobeduein parttothecomplexityofthesescalesand/orthattheywere validatedoutsidetheED,limitingtheexternalvalidity.3,4,6,7 Wesoughttodevelopascalethatwasconcise,organ-specific, andapplicabletothemostcommonsettinginwhichLUSis performed.Toimprovesuchvaguelanguageas “all structuresimagedwell,” wefoundbenefitinspecifically statingtheanatomiclandmarksneededtomaximize diagnosticimagingineachview.Byemphasizingproper imagingtechniquebeforediagnosticinterpretation,our assessmenttoolmayimproveerrorsinimagegradingand reducelearnerfeedbackvariability.
LIMITATIONS
OurstudywaslimitedbyitsevaluationofaQAexperience atasingle,academictertiary-carecenterinwhichthe validationtookplace.Patientdemographicswerenot collected.Theblindedreviewersalltrained(orcurrent trainees)atthesameclinicalultrasoundfellowshipand, therefore,weretaughttoperformQAusingtheACEP gradingscaleinasimilarmanner.Interestingly,thisperhaps mayhavecontributedtoahigheragreementwiththeACEP scalethanif,alternatively,reviewershadtrainedatdifferent institutions.Further,thescaleitselfwasdevelopedafteran extensivereviewoftheliterature,customizedintoafeasible scalethatisdirectlyapplicabletolearnerobjectives.Assuch, thisscalelackstherigorofalternativemethodological methodssuchasmodifiedDelphianalysis.Importantly,this scaledidnotvalidatewhetherthescorewasrelatedtothe diagnosisoroutcome,orwhetheritimprovedQAefficiency oreducationalfeedback,butratherthedegreeofagreement. Additionally,ourscalefocusesonpathologyrelatedtothe pleurallineitselfanddoesnotincludelanguagetoassessthe abilitytodiagnoseapleuraleffusion.Finally,ourstudy
Score Novel- LUS scale ACEP scale2
Table.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 266 NovelScoringScaleforQAofLUSintheED Balderstonetal.
involvedreviewerswithsixmonthsexperienceinQAand includedasmall(100)numberofstudies;consequently,our resultsmaybeunderstated.Furtherresearchiswarrantedto validatethisnovelscale,investigatelearnersatisfaction,and assessitsimpactoneducationalenhancement.
CONCLUSION
Wefoundthatamoreindividualizedqualityassessment scaleofultrasoundimagingtargetedtoaspecificorgan in thiscasethelung resultsinlessgradingvarianceandmore consistent,objectivefeedback.This findingmayhave implicationsonknowledgegainedandlearnersatisfaction. Futurestudiesarewarrantedpriortotheadoptionofthis novelscaleinclinicalpractice.
AddressforCorrespondence:JessicaR.Balderston,MD,VCU MedicalCenter,DepartmentofEmergencyMedicine, 1250E.MarshallSt,Box980401,Richmond,VA23298-0401. Email: Jessica.balderston@vcuhealth.org
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.
Copyright:©2024Baldersonetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.AmericanCollegeofSurgeons’ CommitteeonTrauma Advanced TraumaLifeSupport (ATLS);10th ed.Chicago,Illinois:American CollegeofSurgeons;2018.
2.Ultrasoundguidelines:emergency,point-of-careandclinicalultrasound guidelinesinmedicine. AnnEmergMed.2017;69(5):e27–54.
3.KimuraBJ,GilcreaseGW,ShowalterBK,etal.Diagnostic performanceofapocket-sizedultrasounddeviceforquick-lookcardiac imaging. AmJEmergMed.2012;30(1):32–6.
4.SalomonLJ,NassarM,BernardJP,etal.Ascore-basedmethodto improvethequalityofemergencygynaecologicalultrasound examinations. EurJObstetGynecolReprodBiol 2009;143(2):116–20.
5.DeMasiS,TaylorLA,WeltlerA,etal.Novelqualityassessment methodologyinfocusedcardiacultrasound. AcadEmergMed 2022;29(10):1261–3.
6.RavishankarSM,TsumuraR,HardinJW,etal.Anatomicalfeaturebasedlungultrasoundimagequalityassessmentusingdeep convolutionalneuralnetwork. IEEEIntUltrasonSymp.2021;2021: 10.1109/ius52206.2021.9593662
7.DessieAS,CalhounAW,KanjanauptomP,etal.Developmentand validationofapoint-of-careultrasoundimagequalityassessmenttool: thePOCUSIQscale. AnnEmergMed.2019;74(4):S45–6.
8.GaribyanVN,AmundsonSA,ShawDJ,etal.Theprognosticvalueof lungultrasound findingsinhospitalizedpatientsundergoing echocardiography. JUltrasoundMed.2018;37(7):1641–8.
9.KimuraBJ,LouMM,DahmsEB,etal.Prognosticimplicationsofa point-of-careultrasoundexaminationuponhospitaladmission. JUltrasoundMed.2020;39(2):289–97.
10.KimuraBJandNayakKR. “Asymptomatic” flashpulmonaryedemaby point-of-careultrasound:anovelbedside findingoftransientglobal ischemia. JACCCaseRep.2020;2(10):1545–9.
11.KimuraBJ,ShiR,TranEM,etal.Outcomesofsimplifiedlung ultrasoundexaminCOVID-19:Implicationsforself-imaging. JUltrasoundMed.2022;41(6):1377–84.
12.KimuraBJ,ResnikoffPM,TranEM,etal.Simplifiedlungultrasound examinationandtelehealthfeasibilityinearlySARS-CoV-2Infection. JAmSocEchocardiogr.2022;35(10):1047–54.
13.MaiTV,ShawDJ,AmundsonSA,etal.Learningtoapplythepocket ultrasounddeviceonthecriticallyill:comparingsix “quicklook” signsfor qualityandprognosticvaluesduringinitialusebynovices. CriticalCare 2013;17(5):448.
14.TaylorA,AnjumF,O’RourkeMC.Thoracicandlungultrasound. In:StatPearls[Internet].TreasureIsland,FL:StatPearlsPublishing; 2022.
15.MayoPH,CopettiR,Feller-KopmanD,etal.Thoracicultrasonography: anarrativereview. IntensiveCareMed.2019;45(9):1200–11.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 267 Balderstonetal. NovelScoringScaleforQAofLUSintheED
ORIGINAL RESEARCH
DiagnosticAccuracyofaHandheldUltrasoundvsaCart-based Model:ARandomizedClinicalTrial
RyanC.Gibbons,MD*
DanielJ.Jaeger,MD*
MatthewBerger,MD†
MarkMagee,MD*
ClaireShaffer,MD‡
ThomasG.Costantino,MD*
SectionEditor:MarkLangdorf,MD,MHPE
*LewisKatzSchoolofMedicineatTempleUniversity,DepartmentofEmergency Medicine,Philadelphia,Pennsylvania
† CapitalHealth,DepartmentofEmergencyMedicine,Pennington,NewJersey ‡ UniversityofPittsburghMedicalCenterHarrisburg,DepartmentofEmergency Medicine,Harrisburg,Pennsylvania
Submissionhistory:SubmittedJanuary1,2023;RevisionreceivedNovember16,2023;AcceptedNovember22,2023
ElectronicallypublishedFebruary9,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.17822
Introduction: Numerousstudieshavedemonstratedtheaccuracyofpoint-of-careultrasound(POCUS). Portable,handhelddeviceshaveexpandedtheclinicalscopeofPOCUSatafractionofthecostof traditional,cart-basedmodels.Thereisapaucityofdataassessingthediagnosticaccuracyofportable devices.Ourobjectiveinthisstudywastocomparethediagnosticaccuracyofaportabledevicewitha cart-basedmodel.
Methods: Thiswasaninstitutionalreviewboard-approved,observational,prospective,randomized clinicaltrial(NCT05196776)ofaconveniencesampleofadultpatientswhopresentedtoauniversitybasedhealthsystem.Patientswhorequiredacardiac,lung,renal,aorta,orbiliaryPOCUSwere randomizedtoaportabledeviceortoacart-basedmodel.Wehypothesizedthatthecart-basedmodel wouldhavea90%diagnosticaccuracyvs70%forthehandhelddevice.Todetecta20%difference,the samplesizewascalculatedtobe98,with49patientsrandomizedtoeacharm.Weusedstandard2x2 tablestocalculatetestcharacteristicswith95%confidenceintervals(CI).
Results: Atotalof110patientswereenrolled,with56patientsrandomizedtothecart-basedmodeland 54tothehandhelddevice.Thesensitivity,specificity,anddiagnosticaccuracyofthecart-basedvs handheldwere77.8%(40–97.2)vs92.9%(66.1–99.8),91.5%(79.6–97.6)vs92.3%(79.1–98.4%),and 89.3%(78.1–96)vs92.5%(81.8–97.9),respectively.
Conclusion: Thediagnosticaccuracyofaportable,handhelddeviceissimilartothatofacart-based model.[WestJEmergMed.2024;25(2)268–274.]
INTRODUCTION
Numerousstudieshavedemonstratedtheaccuracyof point-of-careultrasound(POCUS)todiagnosispathology andtoaugmentproceduralguidance.1–10 Portable,handheld deviceshaveexpandedtheclinicalscopeofPOCUSacross diversesettings,includingprehospital,resource-limited,and outpatientclinics.11–13 Themajorityofexistingliteraturehas assessedthetimelinessandimagequalityofhandhelddevices only.13–15 Todate,thereisapaucityofdataassessingthe
diagnosticaccuracyoftheseportabledevices.16–24 Toour knowledge,therearenorandomizedstudiescomparingthe diagnosticaccuracyofaportable,handhelddevice withatraditionalcart-basedmodel.Ourprimaryobjectivein thisstudywastocomparethediagnosticaccuracy ofthesetwodiagnosticimagingmodalities,specificallyfor cardiac,lung,biliary,renalandabdominalaorta imaging,Secondaryanalysisincludedassessment ofimagequality.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 268
METHODS
StudyDesign
Thiswasaninstitutionalreviewboard-approved, observational,prospective,randomizedclinicaltrial (NCT05196776)withparallelassignmentandanallocation ratioof1∶1.WefollowedtheCONSORTguidelinesand checklistsforclinicaltrials.ButterflyNetwork,Inc.provided fundingforthisstudy.
StudySettingandPopulation
BetweenOctober1–December31,2021weincludeda conveniencesampleofpatients ≥18yearsold,whopresented tooneofthreeclinicallydistinctemergencydepartments (ED)affiliatedwithanurban,LevelI,university-based healthsystemwith >200,000adultandpediatricvisits annually,andwhorequiredacardiac,lung,biliary,renal,or abdominalaortaPOCUSbasedonthediscretionofthe emergencyattendingphysician(EP).Studyinvestigators enrolledpatientscapableofprovidingwritteninformed consent.OurdepartmentcredentialsallEPsinthecore POCUSapplicationsasdefinedbytheAmericanCollegeof EmergencyPhysicians(ACEP).25 AllEnglish-andSpanishspeakingpatientsrequiringaPOCUSevaluationwere eligibleforenrollment.Weexcludedpatients unabletoconsent.
StudyProtocol
Weusedpermuted-blockrandomizationwithan allocationratioof1∶1.Allocationconcealmentincluded sequentiallynumbered,opaque,sealedenvelopes.Upon enrollment,blindedstudyinvestigatorsselectedanenvelope containingstudymaterialsandpre-randomizedselection intothehandhelddevice(HH)orcart-basedmodel(CB) usingResearchRandomizerversion4.0(www.randomizer. org).26 Patients,whorequiredacardiac,lung,renal,aorta,or biliaryPOCUS,wererandomizedtoaportabledevice,the ButterflyiQ(ButterflyNetwork,Inc,Guilford,CT) transducerconnectedtoa fifthgenerationAppleiPadMini
PopulationHealthResearchCapsule
Whatdowealreadyknowaboutthisissue?
Point-of-careultrasound(POCUS) enhancesourabilitytosafely,ef fi ciently,and accuratelydiagnoseandmanage ourpatients.
Whatwastheresearchquestion?
DoesahandheldPOCUSdevicehavesimilar diagnosticaccuracyasatraditional, cart-basedmodel?
Whatwasthemajor findingof thestudy?
AhandheldPOCUSdevicehassimilar diagnosticaccuracyasatraditional,cartbasedmodel(sensitivity77.8%vs.92.9%, speci fi city91.5%vs.92.3%andaccuracy 89.3%vs.92.5%).
Howdoesthisimprovepopulationhealth?
Giventhesimilardiagnosticaccuracies, handhelddevicesbroadentheavailabilityof POCUSandenhancepatientcareinresourcelimitedsettings.
(AppleInc,Cupertino,CA),ortoacart-basedmodel,theGE VenueGoorGELogiqE(GEHealthCare,Wauwatosa, WI).(Referto Image.)Westudiedthe fivemostcommonly performedPOCUSscansinourdepartment.
Usingthephasedarraytransducer(2–5mHz)forcardiac imagingorthecurvilineartransducer(1-mHz)forthelung, renal,aortic,andbiliaryscans,postgraduateyear1–3 emergencymedicine(EM)residentsperformedeachPOCUS priortoadvancedimaging.Performingphysiciansusedthe
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 269
Image. HandheldButterflyiQdeviceandcart-basedGEVenueGomodeldemonstratingparasternallongaxisview.
Gibbonsetal. HandheldUltrasoundsvsCart-basedModel
correspondingsettingsfortheHHdevice.AnattendingEP, credentialedinthecoreACEPPOCUSapplications, reviewedeachstudyconcurrently.Studyinvestigators blindedallresidentsperformingthescansandtheattending EPsreviewingthemtothestudyobjectiveanditsfunding.
Acardiologist-interpretedechocardiogram,performed within24hoursofpresentationtotheED,servedasthe referencestandardforcardiacimages.Forbiliarytract images,thereferencestandardwasaradiology-interpreted ultrasoundperformedduringtheEDvisit.Forlung,renal andaorticscans,thereferencestandardwascomputed tomographyimages(whenavailableandperformedduring theEDvisit),orPOCUSqualityassurance(QA)reviewby twoultrasoundfellowship-trainedphysicians(whennoCT wasavailable).Iftherewasdisagreement,athirdultrasound fellowship-trainedphysicianprovidedaninterpretation.The cardiologist,radiologist,andtheultrasoundfellowshiptrainedEPswereblindedtothereal-timePOCUSreads. However,theEPsperformingQAknewaboutthestudyand itsfunding.
Priortostartingtheirinternship,ourEMresidents participateinanintroductory five-hourIntroductionto POCUScoursetaughtbyouremergencyultrasoundfaculty. Additionally,eachresidentcompletesathree-week emergencyultrasoundrotationduringtheirinternshipin accordancewithAccreditationCouncilforGraduate MedicalEducation(ACGME)andACEPguidelines.24,27 Residentsreceivednoadditionaltrainingpriortotheirstudy participation.Nonetheless,eachparticipantcompletedmore than25ofeachscanpriortoparticipatinginthestudyto achievecompetencyperACEPandAGMEguidelines.24,27
Measurements
Priortostudycommencement,wedefinedthefollowing diagnosticendpoints:ejectionfraction(EF)(good >50%, moderate30–50%,poor <30%)andthepresenceorabsence ofthefollowing:gallstones;hydronephrosis(mild,moderate, orsevere);abdominalaorticaneurysm(>3centimeters),and B-lines(≥3inasinglelung fieldorasingle,confluentB-line occupying >1/3oftheintercostalwindow).28 Thepresenceof B-linesindicatesaninterstitialprocess,whetherlocalizedor diffuse,reflectsitsetiology.Wecomparedthistointerstitial findingsonCT(ifavailable)ofthecorrespondinglobe.We didnotcompareadditionalmeasurements(ie,gallbladder wallthickness,orassessM-modeorDoppler findings).The studyincludedB-mode findingsonly.Usingtheelectronic healthrecord(EpicSystemsCorp,Verona,WI),we performedchartabstractiononallpatientstoobtainresults ofcardiology-interpretedechocardiogramsandradiologyinterpretedultrasoundandCTstudies.
Diagnosticaccuracyofeachimagingmodalitycompared totheaforementionedgoldstandardsservedastheprimary endpoint.Imagequalityservedasthesecondaryendpoint. Threeultrasoundfellowship-trainedphysiciansuseda
previouslyvalidatedLikertscaletoassessimagequality.29 Ascoreof1indicatedunabletointerpret,andascoreof7 specifiedsuperiorimagingquality.
StatisticalAnalysis
PriorstudiesassessingPOCUSperformedusing traditionalCBtechnologyhavedemonstratedthefollowing sensitivitiesforrespectivepathologies:EF(89%); cholelithiasis(94%);abdominalaorticaneurysms(97%);Blines(92%);andhydronephrosis(75%),providinganaverage sensitivityof90%.28,30–37 Giventhelackofpre-existingdata comparingthemodalities,wehypothesizedthattheHH devicewouldhaveanoverallsensitivityof70%.We postulatedthattheHHwouldbeinferiorgiventhesmaller screensize,noveltechnologytogeneratesonographic images,andlimitedclinicianexperiencewiththedevice. Basedonapowerof80%andanalphaof0.05,wecalculated asamplesizeof98,with49patientsrandomizedtoeacharm, todetecta20%difference.Wereportcontinuousand categoricaldataasmedianswithinterquartileranges(IQR) orproportionswith95%confidenceintervals(CI),andwe usedstandard2 × 2tablestocalculatetestcharacteristics with95%CIsusingMedCalcversion19.1.6(MedCalc SoftwareLtd,Ostend,Belgium).Intraclasscorrelation coefficientassessedinter-raterreliabilitybetweenblinded expertreviewers,andweusedthe t -testtocomparemedian Likertscores.
RESULTS
Weenrolled110patientswith56patientsrandomizedto theCBmodeland54totheHHdevice(Figure1).Authors excludedoneHHpatientgiventherewerenosonographic imagesavailabletoreview. Table1 illustratesthesimilarity ofpatientcharacteristicsandthenumberofeachPOCUS typeacrossbothcohorts(Table1). Table2 portrays testcharacteristicsforeachdiagnosticmodality,while
Figure1. Patient flowchart. CB,cart-basedultrasoundmodel; HH,handhelddevice; POCUS, point-of-careultrasound; QA,qualityassurancereview; NCCT,noncontrastcomputedtomographyoftheabdomenandpelvis.
53HH 56 CB 110 patients approached (all enrolled) and randomized
Renal POCUS without QA or NCCT 50 correct 6 incorrect 48 correct 4 incorrect
1
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 270
HandheldUltrasoundsvsCart-basedModel Gibbonsetal.
Gibbonsetal. HandheldUltrasoundsvsCart-basedModel
CharacteristicCart-basedmodel(n = 56)Handhelddevice(n = 53)
Age,median(IQR),years57(18–90)60(18–89)
Gender,N(%)
Female60.751
Male39.349
Bodymassindex,median(IQR)30(22–64.9)27.9(15–42.2)
Point-of-careultrasoundscansAorta(9)Aorta(4)
Cardiac(11)Cardiac(17)
Gallbladder(10)Gallbladder(14)
Lung(11)Lung(7)
Renal(15)Renal(11) IQR,interquartilerange.
Cart-basedmodel(n = 56;95%CI)Handhelddevice(n = 53;95%CI)
Sensitivity77.8(40–97.2)92.9(66.1–99.8)
Specificity91.5(79.6–97.6)92.3(79.1–98.4)
Positivelikelihoodratio9.2(3.4–24.9)12.1(4.0–36.2)
Negativelikelihoodratio0.2(0.1–0.8)0.1(0–0.5)
Diseaseprevalence0.20.3
Positivepredictivevalue63.6(39.2–82.6)81.3(59.1–92.9)
Negativepredictivevalue95.6(86.3–98.7)97.3(84.5–99.5)
Accuracy89.3(78.1–96)92.5(79.3–96.9)
CI,confidenceinterval.
Tables3 and 4 depictthediagnosticcriterionreferenceused andthediagnosticinaccuracies,respectively.
Overall,therewere10incorrectdiagnoses,fourforthe HHdeviceandsixfortheCBmodel. Table4 highlightsthe diagnosticinaccuraciesbyscantype,diagnosticmodality, andcriterionreference.TheHHcorrectlyidentifiedthe following:sixinstancesofcholelithiasis;onecaseofmildand oneofmoderatehydronephrosis;fourindividualswith pulmonaryedema;andonepatientwithamoderateEF.The CBmodalitycorrectlyidentifiedthefollowing:twoinstances ofcholelithiasis;onecaseofseverehydronephrosis;two individualswithpulmonaryedema;andtwopatientswith poorEFs.ThemedianLikertscoreforCBwas5,and4 fortheHH.Intraclasscorrelationcoefficientsforthe HHandCBwere0.5(95%CI0.2–0.7)and 0.8(95%CI0.7–0.8),respectively.
DISCUSSION
Tothebestofourknowledge,oursisthe firstpublished randomizedtrialcomparingaportableHHdevicewitha
traditionalCBmodelinEDpatients.Giventhelackofpreexistingdata,wehypothesizedthatthetraditionalCBmodel wouldbesuperiorwithrespecttodiagnosticaccuracyand imagequality.Handhelddevicesarestillnovelandhavenot beenadoptedbroadly,limitingclinicianexperience. Moreover,noveltechnologytogeneratesonographic images,comparedtothetraditionalpiezoelectriccrystals, mayaffectimagequalityaswell.Similarly,weassumed screenresolutionandsizewouldlimitimagequalityand, subsequently,accuracy.However,asmallpilotstudyby MageeetaldemonstratedsimilarresultsbetweenHHand CBdeviceswheninterpretatingpre-recordedvideos assessingforfree fluidintherightupperquadrant.13
Wechose fivebasicPOCUSexaminationsthatourEPs haveconsiderableexperienceperformingwithappropriate diagnosticaccuracy.OurEPscurrentlyhavelessexperience withotherPOCUSindications,suchasregionalanesthesia andfractureassessment.Moreover,wedidnothaveaccessto aHHendocavitarytransducertoassessforpregnancyrelatedissues.Theseareasareripeforfutureresearch.
Table1. Patientcharacteristics.
Table2. Testcharacteristics.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 271
Cart-based model(n = 56)
Handhelddevice (n = 53)
QAEchoQAEcho
Cardiac(n = 27)5688
QARUQUSQARUQUS
Biliary(n = 24)100104
QACTQACT
Lung(n = 18)11052
Renal(n = 27)13293
Aorta(n = 13)6304
QA,qualityassurancereview; Echo,cardiology-performedand interpretedechocardiography; RUQUS,radiology-performed andinterpretedrightupperquadrantultrasound; CT,computed tomographyofthechestorabdomenandpelviswithor withoutcontrast.
Overall,wefoundnosignificantdifferenceinsensitivityor specificitybetweenCBandHHultrasoundimages. However,thislimitedoursamplesizeforeachmodality.
Althoughthestudytypesandindicationsvaried,theidea ofdiagnosticaccuracyshouldapplytoallPOCUSstudies.It isprobablyexpectedthatwhenthediagnosiswasthe objectivepresenceorabsenceofa finding,(ie,gallstones) therewerenomisdiagnoses.29,30 However,whenthe diagnosiswasmoresubjective(ie,estimatingEForthe degreeofhydronephrosis)thereweremoreinaccuracies acrossbothmodalities.Thisisconsistentwithprevious studiesshowingmoreoverlapofgoodandmoderateEFsand betweenpoorandmoderate.31 Inourstudy,therewasa tendencytooverestimatethepresenceordegreeof hydronephrosis,whichislikelyconfirmationalbiasinthe settingofapresumednephrolithiasisdiagnoses.
Asexpected,theCBdevicehadbetteroverallimage qualitythantheHH.However,thisdidnotaffectdiagnostic accuracy,asourresultssuggestthatitissimilarbetweenHH andCBmodalitiesinanacademicEMresidency.Superior imagequalitymaydetectmoresubtlepathology,suchas signsofcholecystitis.32 Eachdiagnosticmodalityservesa clinicalrole.Thisdatacanbeextrapolatedtothebroader EMcommunitywiththeincreasingprevalenceofultrasound competencyinpracticingEPsandavailabilityofportable devices.Furthermore,itsupportstheutilityofHHdevicesin resource-limitedsettings,outpatientclinics,andinpatient locationswithlimitedaccesstotraditionalsonographic machines,nottomentionpandemicsettingswhere disinfectionisparamount.2
LIMITATIONS
Thisstudysuffersfromthelimitationsofanobservational designwithconveniencesamplingatasinglehealthsystem resultinginaselectionbiasaswellasasmallersamplesize, whichlimitsthelevelofprecisiontoexcludeatypeIIerror. UsingthediscretionoftheattendingEPtodetermine whetherapatientneededaspecificPOCUSexamination createdaselectionbiasaswell.Wedidnotdefinespecific indicationstoperformoneoftheaforementionedPOCUS scans.Moreover,wehypothesizedthediagnosticaccuracyof theHHdevicegiventhelackofpre-existingdata.Thislimits thevalidityofourpoweranalysis.
ButterflyNetwork,Inc.fundedthestudy,whichmayhave introducedbias.However,physiciansperformingthe ultrasoundswereunawareofthisfunding.Furthermore, physiciansperformingtheultrasoundhadsignificantlymore experienceusingtheCBmodelcomparedtotheHHdevice, whichmayhaveintroducedbiasinfavorofthetraditional modality.Furthermore,wedidnotaccountforthe
Cart-basedmodel(n = 6)Handhelddevice(n = 4)
Cardiac(n = 5)2interpretedasnormalEF,readasmoderateduringQA1interpretedasnormalEF,readasmoderateduringQA 1interpretedasmoderateEF,readasnormalduringQA 1interpretedaspoorEF,readasnormalonecho
Biliary(n = 0)00
Lung(n = 0)00
Renal(n = 4)2interpretedasmildhydronephrosis,readasnormal duringQA
1interpretedasmoderatehydronephrosis,readas normalduringQA
1interpretedasmoderatehydronephrosis,readas normalonNCCT 1interpretedasmildhydronephrosis,readasnormalon NCCT
Aorta(n = 0)00
EF,ejectionfraction; QA,qualityassurancereview; Echo,cardiologyperformedandinterpretedechocardiography; RUQUS,radiology performed-andinterpretedrightupperquadrantultrasound; NCCT,non-contrastcomputedtomographyoftheabdomenandpelvis; CB, cart-basedmodel; HH,handhelddevice.
Table3. Diagnosticcriterionreferenceusedforcomparison.
Table4. Diagnosticinaccuraciesbyimagingmodality.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 272 HandheldUltrasoundsvsCart-basedModel Gibbonsetal.
experienceleveloftheresidentsperformingtheultrasound, whichcouldhaveimpactedqualityandaccuracy.
Presumably,seniorresidentshadmoreproficiency.
WedidnotcompareadditionaltypesofHHdevices.
Therefore,itisunclearwhetherourdataisapplicableto otherdevicesusingdifferenttechnology.Specifically,the ButterflyiQdeviceuseschiptechnologycomparedto traditionalpiezoelectriccrystals.Thismayimpactimage qualityanddiagnosticaccuracy.Presumably,theHH frequenciessettingsforeachstudyreflectthoseofthe traditionalCBmodalities.However,wedidnotaccountfor softwarefeatures,screensize,orresolutioninourstudy. Futurestudiesneedtovalidateour findingsacrossthearray ofHHdevicesandemergingtechnology.Furthermore,we limitedourstudytoonly fiveoftheACGMEcoreultrasound competencies.Therefore,additionalstudiesareneededto validateour findingstobroaderPOCUSapplications, includingvarioussettingssuchasM-modeandDoppler.
Usingthesubjectiveinterpretationofultrasound fellowship-trainedfacultyasthecriterionreferencewhen otherstandarddiagnosticsimagingmodalitieswerenotdone limitsthevalidityoftheresultsandcausesamisclassification bias.Specifically,wedidnotaccountforthepotentialfor inferiortechnology.Forexample,iftheHHorCBmodel providesinferiorimaging,notonlymaytheperforming physicianmisspathology,buttheEPsconductingQAmay overlookitaswell.Thisfalsenegativemaynotbemissedbya radiology-performedandinterpretedultrasound.Moreover, reviewerswerenotblindedtotheimagesource,HHvsCB, giventhateachmodalityusesuniquestoragemeans. Nonetheless,qualityassurancereviewiscommonpracticein academicEDswithanultrasounddivision,andconfirmatory studiesaretypicallyunnecessary.
Additionally,usingcardiologist-obtained echocardiogramsasareferencestandardintroducesthe potentialfortreatmenteffectsbetweenwhenthePOCUS imageswereobtainedandwhenthecardiologyimageswere obtained.Whileeachpatientreceivedacardiology echocardiogramwithin24hoursoftheEDvisittolimitsuch effects,thisisnonethelessalimitationtoourstudy.Finally, ourEDisnotrepresentativeofthebroaderEMcommunity. Wehaveanactiveultrasounddivisionwithnumerousfaculty andfellows.AllEPsarecredentialedinPOCUS.Inour department,residentsarethetreatingclinicians,who typicallyhavemorePOCUSexperiencecomparedtomost practicingEPs.Furthermore,ourdepartmenthasregular accesstoandexperiencewithportabledevices.
CONCLUSION
Thediagnosticaccuracyofaportable,handheld ultrasounddeviceissimilartotheaccuracyofatraditional, cart-basedmodelwhenperformingcardiac,lung,biliary, renal,orabdominalaortastudies.Futurelarger,multicenter studiesarerequiredtovalidatethese findings.
AddressforCorrespondence:RyanC.Gibbons,MD,LewisKatz SchoolofMedicineatTempleUniversity,DepartmentofEmergency Medicine,1017JonesHall,1316WestOntarioStreet,Philadelphia, PA19140.Email: ryan.gibbons@tuhs.temple.edu
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.ThisstudywasfundedbyButterfly Network,Inc.Thecompanyhadnoroleinconductionofthestudy, themethodofreportingordecisiontopublishthestudy.
Copyright:©2024Gibbonsetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.GibbonsRC,ZanaboniA,GenningerJ,etal.Ultrasound-versus landmark-guidedmedium-sizedjointarthrocentesis:arandomized clinicaltrial. AcadEmergMed. 2022;29(2):159–63.
2.GibbonsRC,MageeM,GoettH,etal.Lungultrasoundvs.chestX-ray studyfortheradiographicdiagnosisofCOVID-19pneumoniainahighprevalencepopulation. JEmergMed. 2021;60(5):615–25.
3.GibbonsRC,ZanaboniA,SaravitzSM,etal.Ultrasoundguidance versuslandmark-guidedpalpationforradialarteriallineplacementby noviceemergencymedicineinterns:arandomizedcontrolledtrial. J EmergMed. 2020;59(6):911–7.
4.BeckerBA,LahhamS,GonzalesMA,etal.Aprospective,multicenter evaluationofpoint-of-careultrasoundforsmall-bowelobstructioninthe emergencydepartment. AcadEmergMed. 2019;26(8):921–30.
5.Smith-BindmanR,AubinC,BailitzJ,etal.Ultrasonographyversus computedtomographyforsuspectednephrolithiasis. NEnglJMed. 2014;371(12):1100–10.
6.CostantinoTG,SatzWA,DehnkampW,etal.Randomizedtrial comparingintraoralultrasoundtolandmark-basedneedleaspirationin patientswithsuspectedperitonsillarabscess. AcadEmergMed. 2012;19(6):626–31.
7.BernardiE,CamporeseG,BüllerHR,etal.Serial2-point ultrasonographyplusD-dimervswhole-legcolor-codedDoppler ultrasonographyfordiagnosingsuspectedsymptomaticdeepvein thrombosis:arandomizedcontrolledtrial. JAMA. 2008;300(14):1653–9.
8.CostantinoTG,ParikhAK,SatzWA,etal.Ultrasonography-guided peripheralintravenousaccessversustraditionalapproachesinpatients withdifficultintravenousaccess. AnnEmergMed. 2005;46(5):456–61.
9.CostantinoTG,BrunoEC,HandlyN,etal.Accuracyofemergency medicineultrasoundintheevaluationofabdominalaorticaneurysm. J EmergMed. 2005;29(4):455–60.
10.BlaivasM,TheodoroD,SierzenskiPR.Astudyofbedsideocular ultrasonographyintheemergencydepartment. AcadEmergMed. 2002;9(8):791–9.
11.ColliA,PratiD,FraquelliM,etal.Theuseofapocket-sizedultrasound deviceimprovesphysicalexamination:resultsofanin-andoutpatient cohortstudy. PLoSOne. 2015;10(3):e0122181.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 273 Gibbonsetal. HandheldUltrasoundsvsCart-basedModel
12.ShorterMandMaciasDJ.Portablehandheldultrasoundinaustere environments:useintheHaitidisaster. Prehosp&DisastMed. 2012Apr;27(2):172–7.
13.MageeMA,AuAK,FlashnerL,etal.Theeffectofdisplaysizeon ultrasoundinterpretation. AmJEmergMed. 2022;51:285–9.
14.SabbadiniL,GermanoR,HopkinsE,etal.Ultrasoundhypotension protocoltime-motionstudyusingthemultifrequencysingletransducer versusamultipletransducerultrasounddevice. WestJEmergMed. 2021;22(3):775–81.
15.DewarZE,WuJ,HughesH,etal.Acomparisonofhandheldultrasound versustraditionalultrasoundforacquisitionofRUSHviewsinhealthy volunteers. JAmColEmergPhysOpen. 2020;1(6):1320–5.
16.RykkjeA,CarlsenJF,NielsenMB.Hand-heldultrasounddevices comparedwithhigh-endultrasoundsystems:asystematicreview. Diagnostics. 2019;9(2):61.
17.LeggettCB,NaqviM,EsakoffTF,etal.Incorporatingpersonal-devicebasedpoint-of-careultrasoundintoobstetriccare:avalidationstudy. AmJObsGyn. 2022;226(4):552.e1–e6.
18.VourvouriEC,PoldermansD,SchinkelAF,etal.Abdominalaortic aneurysmscreeningusingahand-heldultrasounddevice. “apilot study” EurJVascEndovascSurg. 2001;22(4):352–4.
19.Keil-RíosD,Terrazas-SolísH,González-GarayA,etal.Pocket ultrasounddeviceasacomplementtophysicalexaminationforascites evaluationandguidedparacentesis. InternEmergMed. 2016;11(3):461–6.
20.KimuraBJ,GilcreaseGW,ShowalterBK,etal.Diagnosticperformance ofapocket-sizedultrasounddeviceforquick-lookcardiacimaging. AmJ EmergMed. 2012;30(1):32–6.
21.BobbiaX,ChabannonM,ChevallierT,etal.Assessmentof fivedifferent probesforlungultrasoundincriticallyillpatients:apilotstudy. JEmerg Med. 2018;36(7):1265–9.
22.AmedaT,UebayashiK,WagaiK,etal.Assessmentoftherenal collectingsystemusingapocketsizedultrasounddevice. JMed Ultrasonics. 2018;45(4):577–81.
23.LauBC,MotamediD,LukeA.Useofpocket-sizedultrasounddevicein thediagnosisofshoulderpathology. ClinJSportsMed. 2020;30(1):20–4.
24.PujolS,LaurentJ,MarkarianT,etal.Compressionwithapocket-sized ultrasounddevicetodiagnoseproximaldeepveinthrombosis. AmJ EmergMed. 2018;36(7):1262–4.
25.AmericanCollegeofEmergencyPhysiciansPolicyStatement. UltrasoundGuidelines:Emergency,Point-of-care,and ClinicalUltrasoundGuidelinesinMedicine.2001.Availableat:
https://www.acep.org/patient-care/policy-statements/ultrasoundguidelines-emergency-point-of–care-and-clinical-ultrasoundguidelines-in-medicine/.AccessedDecember1,2022.
26.UrbaniakGCandPlousS.ResearchRandomizer(Version4.0) [Computersoftware].Availableat: http://www.randomizer.org/ AccessedOctober1,2021.
27.AccreditationCouncilofGraduateMedicalEducationEmergency MedicineDefinedKeyIndexProcedureMinimums.Availableat: https:// www.acgme.org/Portals/0/PFAssets/ProgramResources/ EM_Key_Index_Procedure_Minimums_103117.pdf?
ver=2017-11-10-130003-693.AccessedDecember1,2022.
28.ChiuL,JairamMP,ChowR,etal.Meta-analysisofpoint-of-carelung ultrasonographyversuschestradiographyinadultswithsymptomsof acutedecompensatedheartfailure. AmJCardiol. 2022;174:89–95.
29.BlaivasM,DeBehnkeD,SierzenskiPR,etal.Tissueharmonicimaging improvesorganvisualizationintraumaultrasoundwhencomparedwith standardultrasoundmode. AcadEmergMed. 2002;9(1):48–53.
30.KendallJLandShimpRJ.Performanceandinterpretationoffocused rightupperquadrantultrasoundbyemergencyphysicians. JEmerg Med. 2001;21(1):7–13.
31.MillerAH,PepePE,BrockmanCR,etal.EDultrasoundinhepatobiliary disease. JEmergMed. 2006;30(1):69–74.
32.MooreCL,RoseGA,TayalVS,etal.Determinationofleftventricular functionbyemergencyphysicianechocardiographyofhypotensive patients. AcadEmergMed. 2002;9(3):186–93.
33.SummersSM,ScruggsW,MenchineMD,etal.Aprospectiveevaluation ofemergencydepartmentbedsideultrasonographyforthedetectionof acutecholecystitis. AnnEmergMed. 2010;56(2):114–22.
34.RossM,BrownM,McLaughlinK,etal.Emergency physician–performedultrasoundtodiagnosecholelithiasis:a systematicreview. AcadEmergMed. 2011;18(3):227–35.
35.MarbachJA,AlmuflehA,DiSantoP,etal.Comparativeaccuracyof focusedcardiacultrasonographyandclinicalexaminationforleft ventriculardysfunctionandvalvularheartdisease:asystematicreview andmeta-analysis. AnnInternMed. 2019;171(4):264–72.
36.FernandoSM,TranA,ChengW,etal.Accuracyofpresenting symptoms,physicalexamination,andimagingfordiagnosisofruptured abdominalaorticaneurysm:systematicreviewandmeta-analysis. Acad EmergMed. 2022;29(4):486–96.
37.RiddellJ,CaseA,WopatR,etal.Sensitivityofemergencybedside ultrasoundtodetecthydronephrosisinpatientswithcomputed tomography-provenstones. WestJEmergMed. 2014;15(1):96–100.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 274 HandheldUltrasoundsvsCart-basedModel Gibbonsetal.
EDUCATIONAL ADVANCES
SpaceUltrasound:AProposalforCompetency-basedUltrasound TrainingforIn-flightSpaceMedicine
ChanelFischetti,MD*
EmilyFrisch,MD†
MichaelLoesche,MD,PhD*
AndrewGoldsmith,MD,MBA*
BenMormann,MD*
JosephS.Savage,MD‡ RogerDias,MD,PhD*
NicoleDuggan,MD*
SectionEditor:MarkLangdorf,MD,MHPE
*BrighamandWomen’sHospital,HarvardMedicalSchool,Departmentof EmergencyMedicine,Boston,Massachusetts
† ClevelandClinic,DepartmentofObstetricsandGynecology,Cleveland,Ohio ‡ YaleBridgeportHospital,DepartmentofEmergencyMedicine, Bridgeport,Connecticut
Submissionhistory:SubmittedAugust8,2023;RevisionreceivedNovember7,2023;AcceptedDecember6,2023
ElectronicallypublishedFebruary13,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18422
Spacetravelhastransformedinthepastseveralyears.Giventheburgeoningmarketforspacetourism, in-flightmedicalemergenciesarelikelytobeexpected.Ultrasoundisoneofthefewdiagnosticand therapeuticmodalitiesavailableforastronautsinspace.However,whilepoint-of-careultrasound (POCUS)isavailable,thereisnocurrentstandardoftrainingforastronautpreparation.Wesuggestan organizedandstructuredmethodologybywhichastronautsshouldbestprepareforspacewiththe medicalequipmentavailableonboard.Astechnologycontinuestoevolve,theassistanceofother artificialintelligenceandaugmentedrealitysystemsarelikelytofacilitatetraininganddynamicreal-time needsduringspaceemergencies.
Summary: Asspacetourismcontinuestoevolve,anorganizedmethodologyforPOCUSuseisadvised tobestprepareastronautsforspace.[WestJEmergMed.2024;25(2)275–281.]
BACKGROUND
Overthelastdecade,impressivetechnologicaladvances surroundingspacetravelhavemadespacetourismareality forthenot-too-distantfuture.Ascommercialindustry increasinglylaysdownastakeinthisnascentmarket,prior barrierstoprivate-passengerspacetravel,suchascostand accessibility,arerapidlyfalling.1,2 Proposedopportunities forprivatepassengersrangefromlong-haulglobaltravel throughspace,tobrieforbit,toprolongedstaywithhotel accommodations.3 Asoflate2021,lessthanahandfulofthe nearly600individualswhohavegonetospacewerecivilian passengers. 4 Despitesmallnumberscurrently,itisprojected thatby2030bothspacetourismandlong-haultravel byspacewillcapturenearly$20billionofthelarger spaceeconomy. 4
Professionalastronautsareoftenscreenedforbaseline healthconditionsthatcouldleadtoin-flightmedical emergenciesandpotentiallyjeopardizepersonnelsafetyor
themission.5 Therefore,truein-flightmedicalemergenciesto datehavebeenrare.However,withthegreaterdiversityof traveleranatomy,physiology,andmedicalhistory,which willinevitablyresultfromexpansionofprivate-passenger spacetravel,asignificantincreaseinin-flightmedical emergenciesisexpected.1,2,5 Forlongerdurationmissions,it isprojectedthatatleastonemedicalemergencywilloccur percrewofsixtravelers.5 Unlikeformedicalemergencies duringairtravel,emergencylandingsandreal-time conversationswithgroundcontrolarenotreliableoptionsin space.6,7 Additionally,asmore flightsdepartitwillbe increasinglyunlikelythatatrainedmedicaldoctor willbeavailableorpresentoneach flight.Infact,SpaceX justrecentlylaunchedanall-civilianmissioncrewwith onlyatrainedphysicianassistant.8 Thus,withthe expansionoftheprivatespace flightindustry,innovative medicalprotocolsandapproachesmust bedeveloped.9
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 275
OBJECTIVES
Point-of-careUltrasoundTrainingforSpaceMedicine CurrentTrainingStandards
Priortocurrentspacetravel, flightcrewsarerequiredto trainforanticipatedmechanical,mission,andengineering challenges.3 Medicalcareistheresponsibilityofthecrew medicalofficer(CMO)whotypicallyhaslimitedprior medicalknowledge.10 TheCMOtraininginvolves40–80 hoursofhands-ontrainingwithremediationandcontinuing virtualtrainingsasneeded.10 Someofthispreparatory trainingincludesrudimentarymedicaleducation (phlebotomy,vitalsignmeasurement,tonopenuse,panoptic use,andultrasound)11 designedinanticipationof coordinatedcarewithMissionControlfortelehealth interpretations.11 Thisoftenconsistsof “just-in-time” diagnosticalgorithmstofacilitateultrasoundinterpretation withtheaidoflivetelehealthguidance.10 Flightsurgeonsare frequentlyandregularlyonconsoleatFlightControlRoom 1-MissionControlandactivelyparticipateinmedical monitoringandguidance.
Point-of-careultrasound(POCUS)imagescanbe downloadedinrealtimeforevaluation,andbyusinga privatemedicalconferencechannelloop,onlytheultrasound operator(trainedandundernon-disclosureagreement),the physician,andthepatient/subjectareinvolved.Eventhe mission’ s flightdirectorwouldnothaveaccess.Inherently, telemedicinehasbeenapartoftheInternationalSpace Station(ISS)sinceitlaunched.Aslongerspaceduration missionsandinterplanetarytravelprogress,timelapsesof40 minutesorlongerareanticipatedforgroundcrewvirtual contact.6,7,12,13 Thesecommunicationdelayscouldleadto severemedicalconsequencesformissionswith flightcrew trainedaccordingtothecurrentstandardofcare.6,7,12
Becauseofitsportability,low-cost,andradiation-free, real-timeimagingforanimpressivearrayofmedical conditions,POCUShasademonstratedutilityinspace medicine,Incaseswhereultrasoundtrainingiscurrently provided,amaximumof2-3hoursisallottedthroughoutthe entirepre-flighttrainingcurriculum.11,14 ThisAdvanced DiagnosticUltrasoundinMicrogravity(ADUM) educationalprogramisusedontheISSwhere “cuecards” areusedtorapidlyguidenon-expertuserstoperform ultrasoundsonpatients,withmorethan90%accuracyafter justminutesoftraining.14 While “cuecards” canbeused, ADUMhasfoundthatnon-medicaloperatorscanobtain qualitydatawiththerightamountoftrainingand direction.14 Forthisreason,anon-boardproficiency enhancementhasalsobeencreatedbothinEnglishand Russian.14 Thecombinationofthisseveral-hourtraining coursewiththe “remoteexpertguidance” (availableby MissionControl)isthemosteffectivemeansbywhich in-flightultrasoundguidanceiscurrentlyconducted withattentiontolimitationsofultrasoundinspace(geluse, devicebatterylife,etc).14
Incontrast,truemasteryofPOCUSforhealthcare professionalstypicallyrequiresyearsofpracticeduring medicalresidencyandoftenanadditionalyearofdedicated trainingthroughanultrasoundfellowship.Whilemasteryof POCUSatthesamelevelofamedicalprofessionalisnot realisticformost flightsurgeontraining,introducingPOCUS tocrewmembersanda flightsurgeon’srepertoirethrougha structuredandsystematiccurriculumhasthepotentialto yieldsignificantbenefittobothprivatepassengersand potentiallytheentiretyofthemission.Additionally,inlonger duration flightmissionswhenemergencydecisionsneedtobe madeusingADUM’sproposedtelecommunicationand ultrasoundvideotransmission,timeandvideodelays haverealandsignificantlimitationsforastronautcare andoutcomes.
CURRICULARDESIGN
ProposedPoint-of-careUltrasoundTrainingSolutions
PriordataonPOCUSeducationsuggeststhatevennovice POCUSlearnerscanretainthebasicsofimageacquisition andinterpretationwithaminimumamountoffocused training.14–18 Corecompetencyinscanningeachorgan systemcanbeachievedwithatwo-hoursessionofcombined didacticsandhands-onscanning.19–22 Thus,asaconsortium ofmedicaldoctorsandexperts,weproposeastructured, competency-basedPOCUScurriculumforcommercialspace travelthatincludeswell-definedaimstargetingimage acquisitionandinterpretationforthemostcommonorgan systemsinvolvedinin-flightmedicalemergencies (Figure1A).Astronautstrainedforspaceshouldbe consideredtechniciansinthesescenarios,withphysicians supplementingthereal-timediagnosesandtreatments.
Thesevenmosthigh-yieldprocedureororgan-based systemsareidentifiedwithanadvised1–2hourstrainingper topic.Realistically,aone-daycourseofabout6–8hours wouldbesufficienttosatisfyafoundationforcompetency. However,aspre-missionastronautpreparationtimeisbusy and filledwithrequirements,thesepreparatorycoursescan beadjustedandelongatedastoleratedbyindividualmission schedulesandneeds.Thisstructuredformatwouldensure consistentandhomogenoustrainingforallastronauts anticipatedinspace.Eachaspectofthemissionisrehearsed, andeachastronaut(andback-upastronaut)isalsocrosstrainedforactivitiesoutsidetheirprimarymission designationscope,intheeventofastronautdrop-out. Trainingwouldbemission-specificandexpectedtobe intensiveandstartaboutthreeyearsbeforelaunch.Realtimeupdatescanbemadeifmissionrequirementschange atanypointwithinthethreeyearstolaunch,sothatthe mostup-to-dateequipmentandproceduresareused priortolaunch.
Corecompetencyisanappropriategoalformost flight surgeonsintrainingandshouldincludethebasicskills neededforPOCUSimageacquisitionandaproficientlevel
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 276 Competency-basedUltrasoundTrainingforIn-flightSpaceMedicine Fischettietal.
ofindependentinterpretation.Althoughfarfrommastery level,corecompetencyallowsforanappropriatebalanceof limitedinputoftrainingtimerequired.TheskillsofPOCUS acquisitionandinterpretationcanalwaysbesupplemented withadjuvanttoolssuchaslivetelehealthwithground
control,or(artificialreality/artificialintelligence[AI])tools duringlivemissions(Figure1B).Fortravelatloweraltitudes oforbit,lowertieredcompetencycoupledwithavailable telehealthguidancemaybesufficient.23–25 Forlong-haulor deeperspacetravel,however,completingtheentire
Figure1A. Corepoint-of-careultrasoundcompetenciesinastructuredultrasoundtrainingprogram.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 277
Figure1B. Suggestedtieredcompetency-basedultrasoundtraining.ImagesfromtheSTRATUSSpaceSimulationtraining.
Fischettietal.
Competency-basedUltrasoundTrainingforIn-flightSpaceMedicine
Physiological systemMedicalevents
Ocularand sensory organs
• Ocularforeignbody
• Increasedintracranial pressure
• Disequilibrium
Incidencein space (%reported)Pathophysiology
Upto42%32 Foreignbodiesfromexposureswithin thespacecapsuleororbit
CardiacArrhythmias0.2–9.55%Shifting fluidsanddynamicchangesin gravitationalmovementscancause compensatorychangesinboth pulmonaryandcardiacvolumesand potentiallyprovokingcardiac arrhythmiasandcardiacirritability33,34
Pulmonaryembolism***Lackofgravityandvenousstasisin spacecanpromotethromboticevents
Ultrasoundindication/ POCUS finding
• Identifyocularforeignbodies
• Measureopticnervesheath diameter
• Measureopticnervesheath diameter
Transthoracicechofor arrhythmias,wallmotion abnormalities,orcardiac standstill
Transthoracicechoforright ventricularstrain
Pulmonary
• Pneumothorax
• Respiratoryinfections
VascularVenous thromboembolism
GastrointestinalBowelobstructionand constipation
Genitourinary
• Acuteurinaryretention
• Renalstones
7.6–64%35 BarotraumaConfirmlungsliding Dysregulationoftheimmunesystem withpossibleconcurrentviral reactivation36
***37,38 Lackofgravityandvenousstasisin spacecanpromotethrombotic events31
***39 Constipationiscommoninspacebut symptomscanmimicbowel obstruction
~1.20%32
Oftenmultifactorial,pharmacologic, lossofgravitationalforces,and demandingscheduleswithlimited accesstovoidingareconsidered contributorstourinaryretention40
0–5%Bonelossandmusclewastingcan leadtoincreasedcalciumexcretion thatcanprecipitaterenalstones41
DermatologicalSofttissueinfections8–10%32 Inagravity-lessenvironment,bacteria andotherpathologic floracan potentiallylingerlongerontheskin’s surface
Traumatic injuries
• Intra-abdominal bleeding
• Fractures
• Jointinjuries
• Softtissueinjuries
11–26%32 Trauma27
IdentifypulmonaryB-linesor consolidations
Identifydeepveinthromboses
AbdominalPOCUSforbowel obstruction
Measurepost-voidbladder volume
Identifyhydronephrosis
Confirmabscessvscellulitis
• eFASTforintraperitonealfree fluid
• Identifybonyabnormalities
• Identifyjointeffusions
• Identifyhematomas,etc
***Indicatesdescribedreportsofpathophysiologyinspacewithoutdisclosednumericalvaluesinspaceorwithlittletonoepisodesinspace. Terrestrialincidencesareoftenusedforriskstratificationmodeling.
POCUS,point-of-careultrasound; eFAST,extendedfocusedassessmentwithsonographyfortrauma.
curriculum,perhapswithprogressiontoadvanced-level training,ishighlyrecommended.Foranyleveloftraining, learningcanbeconsolidatedbyremediationandspaced repetitionoftrainingthroughaugmentedrealityand recordedlectures.
IMPACTANDEFFECTIVENESS
Point-of-careUltrasoundandSpaceMedicine
WhilePOCUShasbeenusedbytheNationaland AeronauticsSpaceAdministration(NASA)astheprimary formofimagingaboardtheISSsince1982,originaldevices
Table1. Medicalemergencieswithrespectiveincidencesinspaceandrelatedutilityofpoint-of-careultrasound.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 278
flightSpaceMedicine Fischettietal.
Competency-basedUltrasoundTrainingforIn-
offeredonlyrudimentaryimagingcapabilities.12,27–29 Today POCUSdevicesarecapableofadvancedimagingwith multiplefrequenciesandmodalitiesforbothdiagnosticand therapeuticapplications.30 ManyPOCUSdevicesarenow hand-held,whichoffersauniqueadvantageoveralternative imagingmodalitiesinsettingswhereweightandvolume restrictionsarecritical,suchasinspacetravel.6–8 Consequently,POCUSrepresentsanidealimagingmodality forthegrowingspacemedicineindustry.8
Previouslydescribedmedicalemergenciesinspacespan nearlyallorgansystemsandreflecttheuniquephysiological stressplacedonthehumanbodybymicrogravityandother naturalrisksinspacesuchasdehydration(Table1).5,9,31 Similarly,therearescenariosinzerogravity,suchas scanningforfree fluidfortrauma(asincaseswithfocused assessmentwithsonographyfortraumaexams)orforpleural edema(pneumoniaorotherinfectiousorcardiacconditions), thathavealternativeinterpretationsgiventhegravity-less conditions.OnEarth,bloodor fluidwouldpoolincertain areasofthebody(thebladderrecessorinferioraspectsofthe lungs),butinzerogravity,thereisnoproclivityforpoolingin anyonespecificarea;hence,acompleteandthoroughexam isimportanttotrainfor.Asdemonstratedin Table1, POCUShasapotentialroleinassessingmedicalconditions associatedwithnearlyeveryorgansysteminspacetravel includingcardiac,pulmonary,genitourinary,andocular complaints.Althoughexpansive,thislistdoesnotaddressthe arrayofpotentialPOCUS-guidedcriticalprocedures.
Anticipatedin-flightproceduresincludethefollowing: establishingvascularaccess;regionalanesthesiaforacute paincontrol(orrarebutlife-savingprocedures);and pericardiocentesisandneedledecompressionfortension pneumothorax.Additionally,therearemanyother importantimplicationsforthepreparationofPOCUSusein spaceandadjustmentsthatmustbemadeandtrainedforin zero-gravityconditions.Forexample,ultrasoundgelisnot usedinspace,inpartbecausewaterisequallyaseffective andbecauseduplicateuseofitemsiscriticalforthecost andweightrestrictionsimposedforeachlaunch.
CONCLUSION
FutureofPoint-of-careUltrasoundinSpaceMedicine
Whilethereareavarietyofultrasoundapplicationsnot describedhere,theidentifiedorgansystemslistedwerechosen basedonfrequencyofemergenciesandtheanticipationofinflightmedicalneeds.42 Whiletheserecommendationshaveyet tobetestedandappliedforspacemedicinepractices,basedon similarPOCUSeducationmodels,skillretentionislikelytobe highamongastronauts.11,23 Spacedrepetitionand remediationwillhelpconsolidateskillsandcanbe instrumentalinmaintaining fluencylongterm.23,26 Alternativelearningmodalitiessuchasvirtualrealityand mixedin-persontrainingmodulescanassistwithskill retentioninrealtimewhenlivetele-consultsareunavailable.
Similarly,AIalgorithmshavethepotentialtoofferautomated imageinterpretationandclinical-decisionassistancewithout theneedforlivetele-support.
ImplementingastructuredPOCUScurriculumhasthe potentialtomaketangiblechangestoin-flighthealthcareand emergencyprocedures,whichwillbecrucialasthespace flightindustrycontinuestoevolve.Tomaximizetheutilityof thisdiagnosticandtherapeuticdevice,weproposethat POCUSeducationshouldbeaprerequisiteoftrainingfor space flightforbothnear-futureandfuturemissionsandcan beachievedthroughastructuredcurriculumtomakethe mostefficientuseofastronauttrainingandtime.11,29,43,44 Whiletimeallocationisanexceptionallyvaluableresourcein astronauttrainingandeducation,integratingPOCUS educationintothemandatoryspace-flighttraining curriculumwilllikelypayoffindividendsforfuture passengersandmissions.
AddressforCorrespondence:ChanelFischetti,MD,Brighamand Women’sHospital,DepartmentofEmergencyMedicine,75Francis St,Boston,MA02115.Email: cfischetti@bwh.harvard.edu
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Drs.EmilyFrisch,AndrewGoldsmith, andNicoleDugganallhavereceivedconsultingoradvisingfeesfrom CentaurLabs.Dr.ChanelFischettiwasformerlyemployedby CentaurLabsandownsequityinthecompany.Shealsoisa consultantforPhilipsUltrasoundandLevelEx.Selectauthors (Drs.Dias,Savage,LoescheandMormann)havereceivedfunding from,arecurrentlyemployedby,orreceiverelatedfunding/grant supportfromNASAorSpaceX.
Copyright:©2024Fischettietal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.ReddyMV,NicaM,WilkesK.Spacetourism:research recommendationsforthefutureoftheindustryandperspectivesof potentialparticipants. TourismManagement. 2012;33(5):1093–102.
2.ZubrinRandSz´okaBFollettC. TheCaseforSpace.Amherst,NY: PrometheusBooks;2019.
3.KingAK.(2020). TheFutureofSpaceTourism.Congressional ResearchService; reports.congress.gov;2020.
AccessedOctober1,2021.
4.FrielM.Tourismasadriverinthespaceeconomy. CurrIssuesTour. 2020;23(13):1581–6.
5.LandonLB,RokholtC,SlackKJ,etal.Selectingastronautsfor long-durationexplorationmissions. Reach. 2017;5:33–56.
6.FrankJ,SpirkovskaL,McCannR,etal.Autonomousmission operations. IEEE.2013:1–20.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 279 Fischettietal. Competency-basedUltrasoundTrainingforIn-flightSpaceMedicine
7.JanuaryO.Humanhealthandperformanceforlong-duration spaceflight. Future. 2008;79:629–35.
8.ChowD.SpaceXmakeshistorywith firstall-civilianspaceflight.NBC News.2021.Availableat: https://www.nbcnews.com/science/space/ spacex-makes-history-first-civilian-spaceflight-rcna2027 AccessedOctober1,2021.
9.RobertsonJM,DiasRD,GuptaA,etal.Medicaleventmanagement forfuturedeepspaceexplorationmissionstoMars. JSurgRes. 2020;246:305–14.
10.McFarlinK,SargsyanAE,MeltonS,etal.Asurgeon’sguidetothe universe. Surgery. 2006;139(5):587–90.
11.SargsyanAE,HamiltonDR,JonesJA,etal.FASTatMACH20:clinical ultrasoundaboardtheInternationalSpaceStation. JTrauma. 2005;58(1):35–9.
12.StewartLH,TrunkeyD,RebagliatiGS.Emergencymedicineinspace. JEmergMed. 2007;32(1):45–54.
13.KirkpatrickAW,McKeeJL,MoeiniS,etal.Pioneeringremotelypiloted aerialsystems(drone)deliveryofaremotelytelementoredultrasound capabilityforselfdiagnosisandassessmentofvulnerablepopulations. JDigitImaging. 2021;34(4):841–5.
14.DulchavskySA,GarciaK,Hamilton,etal.Advancedultrasoundforthe spaceprogramandonearth.In: InternationalSpaceStationBenefitsfor Humanity,3rd ed.Bookdevelopedcollaborativelybythemembersofthe InternationalSpaceStationProgramScienceForum.CreateSpace Publishing.2021;27–31.
15.NobleVE,NelsonBP,SutingcoAN,etal.Assessmentofknowledge retentionandthevalueofproctoredultrasoundexamsafterthe introductionofanemergencyultrasoundcurriculum. BMCMedEduc. 2007;7(1):1–5.
16.Osei-AmpofoM,TafoyaMJ,TafoyaCA,etal.Skillandknowledge retentionaftertrainingincardiopulmonaryultrasoundinGhana. EmergMedJ. 2018;35(11):704–7.
17.McKayGFMandWeerasingheA.Canwesuccessfullyteachnovice juniordoctorsbasicinterventionalultrasoundinasinglefocusedtraining session? PostgraduateMedJ. 2018;94(1111):259–62.
18.SteinmetzP,OleskevichS,LewisJ.Acquisitionandlong-termretention ofbedsideultrasoundskillsin first-yearmedicalstudents. JUltrasound Med. 2016;35(9):1967–75.
19.RussellFM,FerreR,EhrmanRR,etal.Whataretheminimum requirementstoestablishproficiencyinlungultrasoundtrainingfor quantifyingB-lines? ESCHeartFail. 2020;7(5):2941–7.
20.WatermanB,VanAarsenK,LewellM,etal.Qualityofabdominal ultrasoundimageacquisitionbynovicepractitionersfollowingaminimal trainingsessiononhealthyvolunteers. CJEM. 2020;22(S2):S74–8.
21.Situ-LaCasseE,AcunaJ,HuynhD,etal.Canultrasoundnovices developimageacquisitionskillsafterreviewingonlineultrasound modules? BMCMedEduc. 2021;21(1):175.
22.JensenJ,DyreL,JørgensenM,etal.Simulation-based point-of-careultrasoundtraining. ActaAnaesthesoilScand. 2018;62(6):811–9.
23.FinckeEM,PadalkaG,LeeD,etal.Evaluationofshoulderintegrityin space. Radiology. 2005;234(2):319–22.
24.ShackfordSR,RogersFB,OslerTM,etal.Focusedabdominal sonogramfortrauma. JTrauma. 1999;46(4):553–62.
25.ChiaoL,SharipovS,SargsyanAE,etal.Ocularexaminationfortrauma. JTrauma. 2005;58(5):885–9.
26.FischettiC,BhatterP,FrischE,etal.Theevolvingimportanceofartificial intelligenceandradiologyinmedicaltraineeeducation. AcadRadiol. 2021;29(Suppl5):S70–5.
27.WagnerMS,GarciaK,MartinDS.Point-of-careultrasoundinaerospace medicine. AviatSpaceEnvironMed. 2014;85(7):730–9.
28.PianL,GillmanLM,McBethPB,etal.Potentialuseofremote telesonographyasatransformationaltechnologyinunderresourced and/orremotesettings. EmergMedInt. 2013;2013:986160.
29.MartinDS,SouthDA,GarciaKM,etal.Ultrasoundinspace. Ultrasound MedBiol. 2003;29(1):1–12.
30.TheButterflyiQcatchesarideontheSpaceXdragontospace. BusinessWire.2021.Availableat: https://www.businesswire.com/ news/home/20210603005345/en/The-Butterfly-iQ-Catches-aRide-on-the-SpaceX-Dragon-to-Space.AccessedJuly1,2021.
31.SummersRL,JohnstonSL,MarshburnTH,etal.Emergenciesinspace. AnnEmergMed. 2005;46(2):177–84.
32.McPheeJCandCharlesJB.(2009). HumanHealthandPerformance RisksofSpaceExplorationMissions:EvidenceReviewedbytheNASA HumanResearchProgram.vol3405.USNationalAeronautics&Space Administration;2009.
33.Fritsch-YelleJM,LeuenbergerUA,D’AunnoDS,etal.Anepisodeof ventriculartachycardiaduringlong-durationspaceflight. AmJCardiol. 1998;81(11):1391–2.
34.D’AunnoDS,DoughertyAH,DeBlockHF,etal.Effectofshort-and long-durationspaceflightonQTcintervalsinhealthyastronauts. AmJCardiol. 2003;91(4):494–7.
35.ScheuringRAandJonesJ.Spacemedicineissuesandhealthcare systemsforspaceexplorationmedicine;NASATechnicalReport,20-7.
36.CrucianBE,ChoukèrA,SimpsonRJ,etal.Immunesystem dysregulationduringspaceflight:potentialcountermeasuresfordeep spaceexplorationmissions. FrontImmunology. 2018;9:1437.
37.Aun´on-ChancellorSM,PattariniJM,MollS,etal.Venousthrombosis duringspaceflight. NEnglJMed. 2020;382(1):89–90.
38.Marshall-GoebelK,LaurieSS,AlferovaIV,etal.Assessmentofjugular venousblood flowstasisandthrombosisduringspaceflight. JAMA NetworkOpen. 2019;2(11):e1915011.
39.ReyesDP,CarrollDJ,WaltonME,etal.Probabilisticriskassessmentof prophylacticsurgerybeforeextended-durationspaceflight. SurgInnov. 2021;28(5):573–81.
40.LiakopoulosV,LeivaditisK,EleftheriadisT,etal.Thekidneyinspace. IntUrolNephrol. 2012;44(6):1893–901.
41.CrucianB,Babiak-VazquezA,JohnstonS,etal.Incidenceofclinical symptomsduringlong-durationorbitalspaceflight. IntJGenMed. 2016;9:383–91.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 280 Competency-basedUltrasoundTrainingforIn-flightSpaceMedicine Fischettietal.
Fischettietal. Competency-basedUltrasoundTrainingforIn-flightSpaceMedicine
42.HodkinsonP,AndertonR,PosseltB,etal.Anoverviewofspace medicine. BrJAnaesth. 2017;119(suppl_1):i143–53.
43.ModiP,Glavis-BloomJ,NasrinS,etal.Accuracyofinferior venacavaultrasoundforpredictingdehydrationinchildren
withacutediarrheainresource-limitedsettings. PLoSOne. 2016;11(1):e0146859.
44.ChisholmCB,DodgeWR,BaliseRR,etal.Focusedcardiacultrasound training:howmuchisenough? JEmergMed. 2013;44(4):818–22.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 281
UltrasoundPerformedbyEmergencyPhysiciansforDeep VeinThrombosis:ASystematicReview
DanielHercz,MD*†
OrenJ.Mechanic,MD‡§
MarciaVarella,MD,PhD,MHS§
FranciscoFajardo,PhD,MLIS,MPA,MA§
RobertL.Levine,MD∥
*JacksonMemorialHospital,DepartmentofEmergencyMedicine, Miami,Florida
† UniversityofMiami,DepartmentofSurgery,Miami,Florida
‡ MountSinaiMedicalCenter,DepartmentofEmergencyMedicine, MiamiBeach,Florida
§ FloridaInternationalUniversity,HerbertWertheimCollegeof Medicine,DepartmentofMedicalEducation,Miami,Florida
∥ FloridaInternationalUniversity,HerbertWertheimCollegeof Medicine,DepartmentofEmergencyMedicineandCriticalCare, Miami,Florida
SectionEditors:JasonFields,MD,andPatrickMaher,MD,MS
Submissionhistory:SubmittedMay17,2023;RevisionreceivedOctober24,2023;AcceptedNovember22,2023
ElectronicallypublishedFebruary9,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18125
Introduction: Point-of-careultrasound(POCUS)performedbyemergencyphysicians(EP)has emergedasaneffectivealternativetoradiologydepartmentultrasoundsforthediagnosisoflower extremitydeepveinthrombosis(DVT).Systematicreviewssuggestedgoodsensitivityandspecificity overallforEP-performedPOCUSforDVTdiagnosis,yethighlevelsofheterogeneitywerereported.
Methods: Inthissystematicreviewandmeta-analysis,weaimedtoprovidethemostup-to-date estimatesoftheaccuracyofEP-performedPOCUSfordiagnosisofDVTandtoexplorepotential correlationswithtestperformance.WeperformedsystematicsearchesinMEDLINEandEmbasefor original,primarydataarticlesfromJanuary2012–June2021comparingtheef ficacyofPOCUS performedbyEPstothelocalstandard.QualityAssessmentofDiagnosticAccuracyStudies-2for individualarticlesarereported.Weobtainedsummarymeasuresofsensitivity,specificity,andtheir corresponding95%confidenceintervals(CI)usingbivariatemixed-effectsregressionmodels.We performedmeta-regression,subgroup,andsensitivityanalysesasplannedintheprotocol CRD42021268799submittedtoPROSPERO.
Results: Fifteenpublications fittheinclusioncriteria,totaling2,511examinations.Pooledsensitivityand specificitywere90%(95%CI82%–95%)and95%(CI91%–97%),respectively.Subgroupanalysesby EPexperiencefoundsignificantlybetteraccuracyforexamsperformedbyEPspecialists(93%,CI 88%–97%)vstrainees(77%,CI60%–94%).SpecificityforEPspecialists(97%,CI94%–99%)was higherthanfortrainees(87%,CI76%–99%, P = 0.01).Three-pointcompressionultrasound(CUS)was moresensitivethantwo-pointCUSbutwasonlystatisticallysignificantwhenlimitedtoEPspecialists (92%vs88%, P = 0.07,and95%vs88%, P = 0.02,respectively).
Conclusion: Point-of-careultrasoundperformedbyemergencyphysiciansissensitive andspecificforthediagnosisofsuspectedDVTwhenperformedbytrainedattendingEPs. Three-pointcompressionultrasoundexaminationmaybemoresensitivethantwo-point CUS.[WestJEmergMed.2024;25(2)282–290.]
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 282
SYSTEMATIC REVIEW
INTRODUCTION
Lowerextremitydeepvenousthrombosis(DVT)isan acutemedicalconditionthat,ifnoturgentlydiagnosedand treated,canresultinseveremorbidityandmortality.Left untreated,theassociatedone-monthmortalityofacuteDVT is10–15%.1 Postphlebiticsyndromeisseenin23–67%of patientsafterresolutionoftheinitialthrombosis.2 Further, DVTisacommonproblemrepresentingupto2%of diagnosesmadeintheemergencydepartment(ED),3,4 makingitacompelling “ can ’t-miss” urgentdiagnosis. Compressionultrasonography(CUS)hasbecomea widespreadtoolthatmakestheevaluationofDVTrapidand precise.Compressionultrasonographyisrecognizedbythe AmericanCollegeofEmergencyPhysiciansandthe AmericanCollegeofRadiologistsasthestandardofcarefor thediagnosisofDVT,supplantingoldertechniques.5 In additiontoradiologydepartment-performedCUS,point-ofcareultrasound(POCUS)performedintheEDhasemerged asaneffectivediagnosticmodality.6
TheregionofinterestformostED-basedDVTPOCUS protocolsextendsfromthecommonfemoralveintothe poplitealvein.MostDVTPOCUSprotocolsincludeCUSof thecommonfemoralvein,poplitealvein,andpossiblythe femoralvein.7 Thesearereferredtoastwo-pointorthreepointCUS,respectively,dependingonthenumberofsites interrogated.Theclinicalsignificanceofisolatedvenous thrombosisofthecalfiscontroversial;however,non-urgent outpatientsurveillanceisanacceptedtreatment.8 Finally, whileisolatedthrombosisoftheiliacveinisapotentiallylifethreateningcondition,itisrareanddifficulttodetectwith existingsonographictechniques.9 Thus,distalDVTand isolatediliacveinthrombosisarenotaddressedin thisreview.
WhileED-performedPOCUSisacceptedbyemergency physicians(EP)andradiologistsforthediagnosisof DVT,thereexistssubstantialvariabilityinthediagnostic accuracyofPOCUS.7 Factorsthatmayaffectdiagnostic accuracyincludetheexperienceandabilityofthe ultrasoundoperator,thenumberofanatomicalsites ofthelowerextremityscanned,10 whetheraugmentation techniquesareused(suchasDoppler)andimage interpretation(suchasvesselidentificationand partialcompressibility).11,12
StudiesandreviewscomparingtheaccuracyofEDperformedPOCUSforthediagnosisofDVTtoaradiology department-performedultrasoundspanmorethan20years. Earlierstudiesweresmall,morelikelybasedintheUnited States,andcomplicatedbyheterogeneousmethodsand results.13 Currently,toourknowledge,thereexistno guidelinesorbestpracticesforED-basedDVTPOCUS. Withthelastsystematicreviewpublishedalmostadecade ago,weperformedanupdatedsystematicreviewtoexplore thediagnosticaccuracyofED-basedPOCUScomparedto radiologydepartment-performedultrasound.Wealso
exploredfactorsaffectingthediagnosticaccuracyforthe diagnosisofDVTthroughsubgroupanalysisandmetaregressionofrecentstudies.
METHODS
Inthissystematicreviewweaimedtoassesstheaccuracy ofbedsidevenousultrasonographyasperformedbyEPs whencomparedtothoseperformedbytheradiology departmentforthediagnosisofDVToflowerextremitiesin adultpatients.Theprotocolforthisreviewwasaccepted andregisteredontheInternationalProspectiveRegister ofSystematicReviews(PROSPERO)underthe numberCRD42021268799.
SearchStrategy
WeconductedaliteraturesearchinMEDLINE(viaOvid MEDLINE)andEmbase(viaElsevier)forrelevant,original studiespublishedfromJanuary2012–June2021toupdate fromthelatestpublishedsystematicreviewonthetopic.13 Thedetailedlistofsearchtermsusedislistedinthe Appendix (supplementalmaterial).Weconsultedwithdomainexperts forunpublishedstudiesandconductedamanualsearchof publishedliteraturefromthereferenceslistedontheincluded articles.ThelanguagewasrestrictedtoEnglish.
StudySelection
Eligiblestudieswereoriginal,primarydata,collected usingcross-sectionalandlongitudinalstudydesigns(cohort orrandomizedcontrolledtrials),thatincludedadultpatients (age >18years)presentingtotheEDforwhichDVTwas listedasadifferentialdiagnosisandforwhich,aspart ofthediagnosticworkup,anultrasonographicexamwas performedbyanEPandanultrasoundwasperformed bytheradiologydepartment.Acontrastvenogram (angiography)wasanacceptablealternativetoaradiology department-performedultrasound.Weusedthesystematic reviewmanagementtool,Covidence,forthescreening oftitles/abstractsandqualityassessmentofstudies.At leasttwoinvestigators(DHandMVorOHandMV) independentlyreviewedthetitlesandabstractsofthe studiesforeligibility.
Discrepanciesintheeligibilitydecisionwereresolvedbya thirdinvestigator(RL)afterreviewingthefullarticle. Reasonsforexclusionwererecorded.Weexcludedreview articles,editorialsorletters,expertopinions,comments,and animalexperiments.Lastly,weexcludedarticlesforwhich noinformationwasavailableonthetotalnumberoftrue positives,truenegatives,falsepositives,orfalsenegatives.
DataExtraction
Atleasttworeviewersindependentlyextracteddataonthe selectedstudies(DHandMV,orOMandMV).Collected informationincludedthefollowing:countrywherethestudy wasperformed;thetypeofUSexamusedfortheindextest
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 283
Herczetal. UltrasoundforDeepVeinThrombosis
(two-pointorthree-point);clinicalexperience(attending and/ortrainee)anddescriptionoftheformaltrainingof physiciansperformingtheindextest;whethertheoriginal studyhadperformedriskstratificationofparticipantsprior totheuseoftheindextest;numbersoftruepositives,true negatives,falsepositives,andfalsenegatives,sensitivityand specificityasreported;andcorrespondingmeasuresof precision(confidenceintervals[CI]).Toassesspotential biasesinindividualstudies,weusedtheQualityAssessment ofDiagnosticAccuracyStudies2(QUADAS-2)checklist. Disagreementswereresolvedbyconsensus,orbya thirdreviewer.
StatisticalAnalyses
Summarymeasuresofsensitivity,specificity,andtheir corresponding95%CIswereobtainedusingbivariatemixedeffectsregressionmodels.WeestimatedI2 statisticassessing forstudyheterogeneity.Inaddition,inconsistencieswere furtherexploredthroughvisualinspectionofforestplots(for overlappingofsensitivityandspecificitypointestimatesand corresponding95%CI)andbysubgroupanalyses.Subgroup analyses,definedapriori,includedstratificationbythetype ofUSstudyperformed(two-pointCUSvsthree-pointCUS); experienceofphysiciansperformingtheindexPOCUS (completedspecialtyEMtrainingorspecialist/attendingvs EMtraineeorresidentstatus);prevalenceofDVT;sample size;riskofbias;andoutlierstatus.Weperformedall
Recordsidentified from Medline and EMBASE: Databases (n =230)
analyseswithSTATAv16(StataCorpLLC,College Station,TX).14
RESULTS
Weidentified230studiesinEMBASEandMedlinethat fit oursearchstrategy(Figure1).Afterremovingduplicates, titles,andabstracts,weretrieved38studiesforfurther evaluationbasedoninclusioncriteriaandabstractreview. Fifteenpublications15–29 remainedafterfulltextreviewwith reasonsforexclusionlistedin Figure1.Fourteenwerefulllengtharticleswithonemanuscriptreportingtwotrials.One additionalstudyreportingsufficientdataforinclusioninthe analysiswaspublishedasanabstract.Intwoinstances, weobtainedadditionalstudycharacteristicsviadirect authorcorrespondence.
CharacteristicsofStudies
Studiesvariedgreatlyingeographiclocations;three studiesweredoneintheUnitedStatesandCanada,twoin Australia,andthreeinIran,amongotherlocations(Table1). Thenumberofdiagnostictestscomparedrangedfrom 56–385.Moststudiesreporteddataperpatient,withtwo studiesthatreportedresultsbylimb.15,18 PrevalenceofDVT inthesamplesvariedfrom10%–79%.16,19,27 About50%of thestudiesusedtwo-pointultrasound,and50%usedthreepoint.Onepublicationtestedbothtwo-pointandthree-point UStothereferencestandard.22 Moststudiesusedthelocally
Recordsremoved before screening: Duplicate records removed. (n =6)
Recordstitles and abstract screened (n =224)
Reportssought for retrieval and assessed for eligibility. Full article screened: (n = 38)
Recordsexcluded(n =184)
Reason for exclusion(n=23):
Different index test (n=3)
Different outcome (n=2)
Different Reference (n=4)
Patient population (n=2)
Study design (n=2)
Review article (n=4)
Not English (n=1)
Duplicate data (n=2)
No data to construct 2x2 table (n=3)
Studies included in review. (n=15)
Figure1. PRISMA flowdiagramofthesearchandselectionprocessforstudiesincludedinthemeta-analysis.
Identification Screening Included
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 284 UltrasoundforDeepVeinThrombosis Herczetal.
Author,year
Experienceof physician performingthe indextest Torres-Macho,2012Spain76*34Two-pointUSUSdonebyradiologistAttending Abbasi,2012Iran8179Three-pointUS (withDoppler)
Patient’s country Number oftests*
DVT prevalence %IndextestReference
DuplexUSdonebya2nd-year radiologypostgrad EMresident Crowhurst,2013Australia178*13Three-pointUSDuplexUSdonebyradiologist (Dopplerusedifobesepatient)
Attending Poley,2014Canada22712Two-pointUSLCUSdonebyradiologistor medicalrecordreviewat6 monthsinthosewhohadno comprehensiveLCUS
LCUSdonebyradiologistAttending + EM resident
Attending + EM resident Zitek,2016UnitedStates385*10Two-pointUSUSdonebyradiologistEMresident Kim,2016UnitedStates29619Three-pointUS (withDoppler)
Pedraza-Garcia, 2017 Spain10954Three-pointUSUSdonebyradiologist (withDoppler) Attending
Zuker-Herman, 2018 Israel19526Two-andthreepointUS
DuplexUSdonebyradiologistAttending + EM resident Pujol,2018France5620Two-pointUSDuplexultrasounddonebya vascularcertifiedpractitioner. Attending Dehbozorgi,2019Iran24044Three-pointUSDuplexUSdonebyradiologistAttending + EM resident
Basaure,2019Chile10117Three-pointUSUSdonebyradiologistwith Doppler Attending + EM resident
Jahanian,2019Iran7238Three-pointUS (withDoppler)
USdonebyradiologist withDoppler EMresident
Howland,2019Australia10010Three-pointUSUnclearAttending Elsenga,2020Netherlands13821Two-pointUS (withDoppler) rCUSdonebyradiologistAttending + EM resident
Canakci,2020Turkey26626Two-pointUSUSdonebyradiologist orvenography EMresident
Diagnosticassessmentcouldbedoneperpatientsorperlimb(*markstudiesdoneperlimb).
DVT,deepveinthrombosis; US,ultrasound; ED,emergencydepartment; LCUS,limitedcompressionultrasound; rCUS,regional compressionultrasound; EM,emergencymedicine.
availableradiologydepartment-performedDVTUSasthe referencestandard.Lastly,pre-interventiontraining requirementsfortheEPoperatorsvariedgreatlybetween studies,rangingfrombriefdidacticstomulti-daypractical courses.Pre-existingexperiencewascategorizedaseither completionofanemergencymedicine(EM)postgraduate trainingprogramorbytraineestatus.
PrimaryOutcomes
Boththestudy-specificandpooledsensitivities, specificities,andrespective95%CIsareshownin Figure2. Comparedtothereferencestandard,thepooledsensitivity andspecificityoftheEP-performedUSfordiagnosisofDVT ofthelowerlimbwas90%(95%,CI82%–95%)and95% (95%,CI91%–97%),respectively.I2 andQ-teststatistics
suggestedsignificantheterogeneitybetweenstudies (Figure2).Thepooledpositiveandnegativelikelihoodratio forthesamecomparisonswere,respectively,19.1(95%,CI 10.2–35.8)and0.10(95%,CI0.06–0.19)(datanotshown).
SubgroupAnalysesandMeta-Regression
Weperformedexploratorymeta-regressionanalyseswith onlyoneexplanatoryvariableaddedtothemodel, consideringthelimitednumberofstudiesincluded.We assessedpresenceofbias,two-pointvsthree-pointCUS, priorexperienceoftheEP,prevalenceofDVTreported(less thanorgreaterthan30%),andsamplesize.Theexperienceof theEPandincreasedprevalenceofDVTinthesamplewere foundtobesignificantlyassociatedwithimprovedsensitivity andspecificity(meta-regressionjointmodel P = 0.01and
Table1. Characteristicsofstudiesselectedfordataabstraction.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 285
Herczetal. UltrasoundforDeepVeinThrombosis
Zuker-Herman
Zuker-Herman 2018*
Kim 2016
Abbasi 2012
Torres-Macho 2012
0.05,respectively)(Figure3).Traineesensitivitywas77%vs 93%withintheattendinggroup.Specificitywas87%and 97%,respectively.Thesensitivityoftwo-pointandthreepointCUSwere88%and92%,respectively.Whenassessing foraccuracythiswasanon-statisticallysignificant improvement(P = 0.07).
Heterogeneitywassubstantiallyreducedwithrespectto thepooledsensitivityandspecificityforthestudiesincluding onlyspecialistEPs.Giventhese findings,weperformed furthersubgroupanalysisonspecialistEP-performed studies.Two-pointCUSstudiesperformedbyspecialist EPshadapooledsensitivityof88%comparedtothe 95%foundforthree-pointCUSalsoperformedby specialistEPs(P = 0.02).SpecificityofUSperformed byEPspecialistwasnotdifferentwhencomparingtwotothree-pointsUS.
QualityAssessment
BasedontheQUADAS-2toolforassessmentofthe qualityoftheindividualstudies,therewereconcerns regardingtheriskofbias(Figure4).Theaggregateriskof
SPECIFICITY
Canakci 2020
Elsenga 2020
Howland 2019
Jahanian 2019
Dahbozorgi 2019
Basaure 2019
Zuker-Herman 2018**
Zuker-Herman 2018*
Pujol 2018
Garcia 2017
Kim 2016
Zitek 2016
Poley 2014
Crowhurst 2013
Abbasi 2012
Torres-Macho 2012
StudyId COMBINED
0.93 [0.89 - 0.96]
0.97 [0.92 - 0.99]
1.00 [0.96 - 1.00]
0.85 [0.71 - 0.94]
0.93 [0.88 - 0.97]
0.90 [0.82 - 0.96]
biasidentifiedthat40%ofstudieswereconsideredhighor unclearriskofbiasofpatientselectionduetotheuseof convenience,non-consecutivesampling.Concerns regardinghighorunclearriskofbiasesrelatedtotheindex test,thereferencestandard,blinding,orthe flowand timing(oftheindexprocedurerelativetothereferencetest) werefoundinfewerthan30%ofthestudiesincluded (Figure4A).Theratingofeachindividualstudy regardingtheQUADAS-2biasesassessedisshownin SupplementalTable1.
SensitivityAnalyses
Weperformedsensitivityanalysesexcluding studies16,19,24–27 thatwereoutliersbasedonmodel fittingand outliers’ assessment.Pooledaccuracyfortheremaining11 resultswasslightlylower,andheterogeneityreduced substantially(sensitivity89%,95%CI85%–92%,andI2 = 27.8;specificity96%,95%,CI93%–97%,I2 = 60.3).Lastly, analysesrestrictedtostudiesforwhichtheriskofbiaswas consideredlowforalldomainsyieldedsimilarpooled sensitivityandspecificity(datanotshown).
CI) Q = 93.98, df = 15.00, p = 0.00 I2 = 84.04 [77.14 - 90.93] 0.90[0.82 - 0.95] 0.92 [0.75 - 0.99] 0.86 [0.75 - 0.93] 0.78 [0.52 - 0.94] 0.88 [0.69 - 0.97] 0.57 [0.37 - 0.76] 0.85 [0.73 - 0.94] 0.93 [0.84 - 0.98] 1.00 [0.72 - 1.00] 0.90 [0.77 - 0.97]
[0.65 - 0.90] 1.00 [0.80 - 1.00] 1.00 [0.97 - 1.00] 0.54 [0.33 - 0.73] 1.00 [0.69 - 1.00]
[0.68 - 0.96] 0.93 [0.84 - 0.98]
SENSITIVITY (95%
0.79
0.86
COMBINED
2014
Crowhurst 2013 Poley
Zitek 2016
Garcia 2017
Pujol 2018
Jahanian 2019 Howland 2019 Elsenga 2020 Canakci 2020 0.2 1.0 SENSITIVITY * 3-point US and ** 2-point US
2018** Basaure 2019 Dahbozorgi 2019
SPECIFICITY (95% CI)
I2
- 0.97] 0.98 [0.89 - 1.00] 0.41 [0.18 - 0.67] 0.91 [0.85 - 0.96] 0.96 [0.92 - 0.99] 0.96 [0.93 - 0.98] 0.93 [0.89 - 0.96] 0.90 [0.78 - 0.97] 1.00 [0.92 - 1.00] 0.99 [0.95 - 1.00] 0.99 [0.95 - 1.00]
Q =147.05, df = 15.00, p = 0.00
= 89.80 [85.92 - 93.68] 0.95[0.91
0.2 1.0
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 286 UltrasoundforDeepVeinThrombosis Herczetal.
Figure2. Forestplotsofsensitivityandspecificityofultrasoundperformedbyemergencyphysicianforthediagnosisoflowerextremities deepveinthrombosis(DVT).
Figure3. Subgroupanalysesforsensitivityandspecificityaccordingtoselectedstudycharacteristics.I2 toassessheterogeneityandmetaregression P-valuesfordifferencesintheaccuracywithinsubgroups.Thedottedlinerepresentsreferencevaluesobtainedinthepooled sensitivityandspecificityofallstudies.
US,ultrasound; EP,emergencyphysician; DVT,deepveinthrombosis; CI,confidenceinterval.
UpdatedSearch
Weperformedanewliteraturesearchinlate2022.Only onenewrelevantstudyof100patientshadbeenpublished sinceJune2021.30 Anexploratoryanalysisaddingthisstudy tothepoolof16studiespreviouslyassessedshowedno differencesinthepooledresultsreported.
DISCUSSION
ThediagnosisofDVTintheEDevolvedfrom cumbersometestsperformedoutsidetheED,suchas impedanceplethysmographyandvenography,toeasily implementedPOCUSthatismainstaytrainingofcurrent EMcurriculumintheUnitedStatesandsomeother countries.31,32 DespitewidespreaduseofPOCUS,concerns persistregardingtheaccuracyoftestsdoneinwidely disparateEDs.Anearlierquantitativesystematicreviewof studiesperformedintheUSyieldedsensitivitiesgreaterthan 95%.33 However,asmorediversestudieswerepublished,a subsequentreviewdemonstratedamoremoderatepooled sensitivitycloseto90%.13 Bothreviewsdemonstratedvery highspecificity.Unfortunately,todateallmeta-analyses addressingthistopichavebeenplaguedbyhighlevelsof heterogeneity,aproblemidentifiedinarecentreviewby Lee.12 Nomodelhasbeenproposedtoreduceheterogeneity.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 287
Figure4. Aggregateassessmentofindividualstudyquality accordingtoQUADAS-2tool.
Herczetal. UltrasoundforDeepVeinThrombosis
Toourknowledgethisisthe firstsystematicreviewwitha focustomaximizeperformanceofED-basedDVTPOCUS withrecommendationsonoperatorandtechnique.We identifiedtrendsexplainingstudyvariabilityaswellaskey biaseswithintheliterature.Inthismeta-analysis,usingthe mostrecentstudiesontheuseofPOCUSinEDsfrom multiplecountries,wedemonstratedapooledsensitivityand specificityof90%and95%,respectively.Theseresultsare somewhatsimilartopriorsystematicreviewsonED-based DVTPOCUS.However,clinicallysignificantvariationin operatorandscanningprotocolexistedinthe subgroupsexamined.
Generaloperatorleveloftraining(trainee/residentvs attending/fellow/specialiststatus)wasanimportant predictorofperformancewith77%sensitivitynotedinthe traineegroupvs93%inthespecialistgroup.Specificityin thesegroupswas87%and97%,respectively.Thisisinsharp contrastwithtrainingprovidedaspartoftheincluded studies.Aquantitativeanalysisoftrainingimmediatelypreinterventionwasnotpossibleduetolackofdetailed information.Withwhathasbeenreported,itseffecton accuracyappearstobefarlessthangeneralleveloftraining/ specialization.CompletionofformalEMtrainingpathway appearstohaveastrongeffectonPOCUSDVT USperformance.
Thisreviewspans10nationsfrom2012-2022,representing differentapproachestoEMandultrasoundtrainingandis, therefore,broadlyapplicabletocontemporarypractice.
Whilespecialtytrainingisoftencountry-specific, 34–36 most ofthesecountriesnowincludededicatedPOCUStrainingas amandatoryrequirementforEMspecialistqualification withsubspecialistUScertificationavailableaswell. Ultrasoundtechniqueacrossallincludedcountriestendedto besimilar,witharelianceonCUSoftheproximallegveins, inaccordancewithinternationallypublishedguidelineson thediagnosisofDVT.32
AnotherunsettledquestionfortheEMapplicationof POCUSforthediagnosisofDVTiswhetherthree-pointUS issuperiorcomparedtothecommonlyimplementedtwopointexamination.A2018radiologistconsensusreport recommendsthree-pointratherthantwo-pointCUSasa baserequirementfordiagnosisofDVTbecausethree-point CUSdetectsisolatedfemoralveinthrombosesthatwould otherwisebemissedin5%–8%ofthosewithlowerextremity DVT.7 ThestudybyAdhikarietal,10 analyzingthreeyearsof radiology-performedCUSintheEDalsofoundthatthreepointCUSdetectedanadditional6%oflowerextremity DVTisolatedtothefemoralvein,withoutinvolvementofthe commonfemoralveinorpoplitealvein.Lastly,thestudyof Tabbutetalfoundasimilarrateofisolatedthrombifroma mixofPOCUSandradiology-performedstudies.37
Oneofthisreview’sstudiesexploredthesensitivityoftwo vsthree-pointUSexamsperformedbytraineesand specialistsasa within-patient analysis.Thesensitivityforthe
diagnosisofDVTincreasedby7%byincludingthethirdsite. Theseresultsareintuitiveevenincasesofnon-isolated femoralveinthrombi.Scanningmultiplesitesreducesthe probabilityoffalsenegativescansasjustasinglepositive findingisarequirementfordiagnosis.Ourpooledanalysisof two-pointvsthree-pointscanningyieldeda5%higherpoint estimateofsensitivityforthemorecomprehensivescan withoutlossinspecificity,whichiscongruentwithprior literature.Thedifferencewasnotstatisticallysignificant witha P -valueof0.07.Whenlimitedtoonlyspecialistperformedexams,thedifferencewasstatistically significant(P = 0.02).
Wefoundlargereductionsinheterogeneityinmultiple subgroupswhenlookingatstudiesofattendingphysicianperformedPOCUS.TheseincludespecialtytrainedEPperformedtwo-pointandthree-pointscansandstudies withouthighlevelsofbias.Thisimpliesahigherdegreeof confidenceintheconsistencyoftheintervention’ s performanceinqualifiedhands.Subgroupanalyseswith prevalencebeloworabove30%yieldedincreasedspecificity forstudieswithprevalenceabove30%.However,a30% prevalenceofDVTintheEDisunusuallyhighand unexpected.Differencesinpatientinclusioncriteria (Wellsscoringand/orD-dimer)mayhavecontributedtothis effect.Thepotentialeffectofhighprevalenceof DVTonthediagnosticaccuracystudiesisyetto beconfirmed.
LIMITATIONS
Thismeta-analysishassomelimitations.First,because only15studieswereidentified,morecomplexanalysescould notbeperformed.Furthermore,moststudiescontained elementsofbias,especiallyrelatedtopatientselection; recruitmentoftenoccurredasaconveniencesample, presumablyselectedbytheultrasoundoperator/clinician. Additionally,threestudiesincludedinconclusive results.18,20,28 Wefollowedbestpracticesandmadethe decisiontoclassifyinconclusivecasesasallpositiveorall negativedependingontheclinicalcontext.38 Basedonthe studydesignreportedbytheauthors,wecategorizedthe inconclusiveresultsasnegative.Asensitivityanalysiswas conducted,andthelimitednumberofinconclusiveresultsare unlikelytoaffectthepooledresultsherebyreported.
Anotherlimitationrelatestotheinabilitytobetter characterizethelevelofexperienceoftheUSoperatordueto limiteddetailedinformationonoperatortraining (SupplementalTable2).Lastly,restrictingpublicationsfrom 2012tothepresentlimitedthenumberofstudiesandthe powertoassesspotentialsubgroupdifferences.However, since2012formalizedtraininginPOCUShasbeen adoptedaspartofspecialisttraininginmostcountries includedinthisreview.Thus,webelievethatthis review’sresultsaremoregeneralizabletothebroad EMpopulation.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 288 UltrasoundforDeepVeinThrombosis Herczetal.
CONCLUSION
Thismeta-analysisofstudiesreportedsince2012 demonstratedexcellentperformanceofEMspecialistperformedthree-pointpoint-of-careultrasoundforthe diagnosisofdeepveinthrombosis.Boththepooled sensitivityandspecificitywere95%.Werecommendthat POCUS-trainedattendingEPsperformathree-point examinationintheEDtoeffectivelyandaccuratelydiagnose DVT.FuturegeneralstudiesonED-basedDVTPOCUSare unlikelytomodifythese findingsgiventhenumerousexisting studiesofatleastmoderatequality.Futurestudiesof rigorousmethodologyfurtheraddressingcertainsubgroups arerecommended.
AddressforCorrespondence:Robert,L.Levine,MD,Florida InternationalUniversity,HerbertWertheimCollegeofMedicine, DepartmentofEmergencyMedicineandCriticalCare,11200SW8th St.,AHC4suite261,Miami,FL33199.Email: rlevine@fiu.edu
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financialrelationshipswithanycompaniesthatarerelevanttothis study.Therearenoconflictsofinterestorsourcesoffunding todeclare.
Copyright:©2024Herczetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.GalanaudJP,MonrealM,KahnSR.Epidemiologyofthepostthromboticsyndrome. ThrombRes.2018;164:100–9.
2.WhiteRH.Theepidemiologyofvenousthromboembolism. Circulation 2003;107(23_suppl_1):I.4–8.
3.Carter-StorchR,OlsenUF,MogensenCB.Admissionstoemergency departmentmaybeclassifiedintospecificcomplaintcategories. Dan MedJ.2014;61(3):A4802.
4.KellyJandHuntBJ.Doanticoagulantsimprovesurvivalin patientspresentingwithvenousthromboembolism? JInternMed 2003;254(6):527–39.
5.ExpertPanelonVascularImaging,HanleyM,SteignerML,etal. ACRAppropriatenessCriteria® suspectedlowerextremitydeepvein thrombosis. JAmCollRadiol.2018;15(11S):S413–7.
6.UltrasoundGuidelines:Emergency,point-of-care,andclinical ultrasoundguidelinesinmedicine. AnnEmergMed 2023;82(3):e115–55.
7.NeedlemanL,CronanJJ,LillyMP,etal.Ultrasoundforlowerextremity deepvenousthrombosis:multidisciplinaryrecommendationsfromthe SocietyofRadiologistsinUltrasoundConsensusConference. Circulation.2018;137(14):1505–15.
8.GarryJ,DukeA,LabropoulosN.Systematicreviewofthecomplications followingisolatedcalfdeepveinthrombosis. BrJSurg 2016;103(7):789–96.
9.BarrellierMT,LezinB,MonsallierJM.Isolatediliacdeepvenous thrombosis.Studyof48casesseenin7yearsamong 18,297echo-Dopplerevaluationsofthelowerlimbs. JMalVasc 2001;26(5):290–8.
10.AdhikariS,ZegerW,ThomC,etal.Isolateddeepvenousthrombosis: implicationsfor2-pointcompressionultrasonographyofthelower extremity. AnnEmergMed.2015;66(3):262–6.
11.VarriasD,PalaiodimosL,BalasubramanianP,etal.Theuseofpoint-ofcareultrasound(POCUS)inthediagnosisofdeepveinthrombosis. JClinMed.2021;10(17):3903.
12.LeeJH,LeeSH,YunSJ.Comparisonof2-pointand3-pointpoint-ofcareultrasoundtechniquesfordeepveinthrombosisatthe emergencydepartment:ameta-analysis. Medicine(Baltimore) 2019;98(22):e15791.
13.PomeroF,DentaliF,BorrettaV,etal.Accuracyofemergencyphysicianperformedultrasonographyinthediagnosisofdeep-veinthrombosis:a systematicreviewandmeta-analysis. ThrombHaemost 2013;109(1):137–45.
14.StataCorp. StataStatisticalSoftware, 16thed.CollegeStation,Texas: StataCorp,LLC,2019.
15.Torres-MachoJ,Ant´on-SantosJM,García-GutierrezI,etal.Initial accuracyofbedsideultrasoundperformedbyemergencyphysiciansfor multipleindicationsafterashorttrainingperiod. AmJEmergMed 2012;30(9):1943–9.
16.AbbasiS,BolverdiE,ZareMA,etal.Comparisonofdiagnosticvalueof conventionalultrasonographybyemergencyphysicianswithDoppler ultrasonographybyradiologyphysiciansfordiagnosisofdeepvein thrombosis. JPakMedAssoc.2012;62(5):461–5.
17.CrowhurstTDandDunnRJ.Sensitivityandspecificityofthree-point compressionultrasonographyperformedbyemergencyphysiciansfor proximallowerextremitydeepvenousthrombosis. EmergMed Australas.2013;25(6):588–96.
18.PoleyRA,NewbiggingJL,SivilottiML.Estimatedeffectofanintegrated approachtosuspecteddeepvenousthrombosisusinglimitedcompressionultrasound. AcadEmergMed.2014;21(9):971–80.
19.ZitekT,BaydounJ,YepezS,etal.Mistakesandpitfallsassociatedwith two-pointcompressionultrasoundfordeepveinthrombosis. WestJ EmergMed.2016;17(2):201–8.
20.KimDJ,ByynyRL,RiceCA,etal.Testcharacteristicsofemergency physician-performedlimitedcompressionultrasoundforlower-extremity deepveinthrombosis. JEmergMed.2016;51(6):684–90.
21.PedrazaGarcíaJ,ValleAlonsoJ,CeballosGarcíaP,etal.Comparison oftheaccuracyofemergencydepartment-performedpoint-of-careultrasound(POCUS)inthediagnosisoflower-extremitydeepvein thrombosis. JEmergMed.2018;54(5):656–64.
22.Zuker-HermanR,AyalonDangurI,BerantR,etal.Comparisonbetween two-pointandthree-pointcompressionultrasoundforthediagnosisof deepveinthrombosis. JThrombThrombolysis.2018;45(1):99–105.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 289 Herczetal. UltrasoundforDeepVeinThrombosis
23.PujolS,LaurentJ,MarkarianT,etal.Compressionwithapocket-sized ultrasounddevicetodiagnoseproximaldeepveinthrombosis. AmJ EmergMed.2018;36(7):1262–4.
24.DehbozorgiA,DamghaniF,Mousavi-RoknabadiRS,etal.Accuracyof three-pointcompressionultrasoundforthediagnosisofproximaldeepveinthrombosisinemergencydepartment. JResMedSci.2019;24:80.
25.BasaureCE,RojasN,ClausdorffH,etal.Shorttrainingofcompressible ultrasoundtechniquefordiagnosisofdeepveinthrombosisby emergencyphysicians. AcadEmergMed.2019;26:S264.
26.JahanianF,KhatirIG,Bani-MostafaviES,etal.Diagnosticaccuracyofa three-pointcompressionultrasonographyperformedbyemergency medicineresidentforthediagnosisofdeepveinthrombosis:a prospectivediagnosticstudy. ActaInformMed.2019;27(2):119–22.
27.HowlandCandMurrayA.Canemergencyphysicianperformed threepointcompressionultrasonographyforlowerlimbdeepvein thrombosisreducelengthofstay? EmergMedAustralas.2019;31:48–9.
28.ElsengaHE,ColleeA,RosendaalAV.Agreementbetweenemergency physiciansandradiologistsforthediagnosisofdeepvenousthrombosis withcompressionultrasound:aprospectivestudy. EurJEmergMed 2021;28(1):25–8.
29.CanakciME,AcarN,BilginM,etal.Diagnosticvalueofpoint-of-care ultrasoundindeepveinthrombosisintheemergencydepartment. JClin Ultrasound.2020;48(9):527–31.
30.AhmedEl-GazzarEH,AlkafafyAM,El-SalamFathiHA,etal.Sensitivity andspecificityofthree-pointcompressionultrasonographytest performedbyemergencyphysiciansfordiagnosisoflowerlimbsdeep venousthrombosis. EgyptJAnaesth.2021;37(1):517–22.
31.AtkinsonP,BowraJ,LambertM,etal.InternationalFederationfor EmergencyMedicinepointofcareultrasoundcurriculum. CJEM 2015;17(2):161–70.
32.FischerLM,WooMY,LeeAC,etal.Emergencymedicinepoint-of-care ultrasonography:anationalneedsassessmentofcompetenciesfor generalandexpertpractice. CJEM.2015;17(1):74–88.
33.BurnsidePR,BrownMD,KlineJA.Systematicreviewofemergency physician-performedultrasonographyforlower-extremitydeepvein thrombosis. AcadEmergMed.2008;15(6):493–8.
34.DuchenneJ,MartinezM,RothmannC,etal.Premierniveaude compétencepourl’échographiecliniqueenmédecined’urgence. RecommandationsdelaSociétéfrançaisedemédecined’urgencepar consensusformalisé. Ann.Fr.Med.Urgence.2016;6(4):284–95.
35.LewisD,RangL,KimD,etal.Recommendationsfortheuseofpointof careultrasound(POCUS)byemergencyphysiciansinCanada. CJEM 2019;21(6):721–726.
36.AustralasianCollegeforEmergencyMedicine.TheUseofFocused UltrasoundinEmergencyMedicine.2022.Availableat: https://acem. org.au/getmedia/000b84ee-378f-4b65-a9a7-c174651c2542/
2023_2-P21-Policy-for-Focused-Ultrasound-in-Emergency-Medicine AccessedOctober10,2023.
37.TabbutM,EbersoleN,IckenL,etal.Two-pointcompressionultrasound techniquerisksmissingisolatedfemoralveinDVTs. WestJEmergMed 2022;23(4):597–600.
38.CohenJF,KorevaarDA,AltmanDG,etal.STARD2015guidelinesfor reportingdiagnosticaccuracystudies:explanationandelaboration. BMJOpen.2016;6(11):e012799.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 290
UltrasoundforDeepVeinThrombosis Herczetal.
ORIGINAL RESEARCH
andEmergencyContraceptioninEmergencyDepartments
intheUnitedStates:ANationalSurvey
ColleenCowdery,MD*
DianaHalloran,MD*
RebeccaHenderson,PhD*
MAKathleenM.Allen,MPH†
KellyO’Shea,MD*
KristenWoodward,MD*
SusanRifai,MD*
ScottA.Cohen,MPH†‡
MuhammadAbdulBakerChowdhury,MPH‡
CristinaZeretzke-Bien,MD‡
LaurenA.Walter,MD§
Marie-CarmelleElie-Turenne,MD‡§
SectionEditor:LauraWalker,MD
*UniversityofFloridaCollegeofMedicine,Gainesville,Florida
† UniversityofFloridaCollegeofPublicHealthandHealth Professions,Gainesville,Florida
‡ DepartmentofEmergencyMedicineUniversityofFlorida, Gainesville,Florida
§ DepartmentofEmergencyMedicine,SchoolofMedicine, UniversityofAlabamaatBirmingham,Birmingham,Alabama
Submissionhistory:SubmittedJuly19,2023;RevisionreceivedNovember28,2023;AcceptedDecember5,2023
ElectronicallypublishedFebruary28,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.18405
Background: Despitetheprevalenceofsexualassaultpresentationstoemergencydepartments(ED) intheUnitedStates,currentaccesstosexualassaultnurseexaminers(SANE)andemergency contraception(EC)inEDsisunknown.
Methods: Inthisstudyweemployeda “secretshopper,” cross-sectionaltelephonicsurvey.Ateam attemptedphonecontactwitharepresentativesampleofEDsandaskedrespondentsaboutthe availabilityofSANEsandECintheirED.Reportedavailabilitywascorrelatedwithvariablesincluding region,urban/ruralstatus,hospitalsize,faithaffiliation,academicaffiliation,andexistenceoflegislative requirementstoofferEC.
Results: Overatwo-monthperiodin2019,1,046callstohospitalswereattemptedand960were completed(91.7%responserate).Ofthe4,360eligiblehospitalslistedinafederaldatabase,960 (22.0%)werecontacted.AccesstoSANEsandECwerereportedtobeavailablein48.9%(95% confidenceinterval[CI]45.5–52.0)and42.5%(95%CI39.4–45.7)ofhospitals,respectively.Accessto ECwaspositivelycorrelatedwithSANEavailability.TheEDsreportingSANEandECavailabilitywere morelikelytobelarge,rural,andaffiliatedwithanacademicinstitution.ThosereportingaccesstoEC weremorelikelytobeintheNortheastandinstateswithlegislativerequirementstoofferEC.
Conclusion: Ourresultssuggestthatperceivedaccesstosexualassaultservicesandemergency contraceptioninEDsintheUnitedStatesremainspoorwithregionalandlegislativedisparities.Results suggestdisparitiesinperceivedaccesstoECandSANEintheED,whichhaveimplicationsforimproving EDpracticesregardingcareofsexualassaultvictims.[WestJEmergMed.2024;25(2)291–300.]
Keywords: emergencycontraception;sexualassaultnurseexaminer;sexualassault.
UserExperienceofAccesstoSexualAssaultNurseExaminer
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 291
INTRODUCTION
Theemergencydepartment(ED)isanimportantpointof entryforvictimsofrape,trafficking,andotherformsof sexualanddomesticviolence.IntheUnitedStates,sexual assaultpresentationstoEDSincreasedby1,533%from2006 to2019.1 Thecurrentstateofaccesstohigh-quality emergencysexualassaultcareintheUSisunclear.
SexualassaultcareinEDsintheUSincludestheneedfor forensicevidencecollection.Adirectedapproachtoprovide thisspecializedcareisthroughtheuseofsexualassault nurse(orforensic)examiners(SANE) 2;SANEsare registerednursesorclinicianswhohavecompleteda didacticandclinicalcurriculumapprovedbythe InternationalAssociationofForensicNursesorother certifyingbody.3 Theyperformforensicsexualassault examsandevidencecollectionwhilemeetingthemedical, psychological,andeducationalneedsofindividuals requiringservices.4 StudieshavedemonstratedthatSANEs providemore “humanizing ” carethannon-SANE emergencypractitionersfromthepatientperspective,5 more comprehensiveandconsistentmedicalservices, 4 andmore thoroughforensicexaminationstoimprovethecriminal justiceresponsetosexualassault. 6 Currently,thereareover 450SANEprogramsintheUS,approximately75%of whichareaf fi liatedwithanED.3, 4 However,nofederal regulationsdictatewhocanprovidesexualassaultcareor overseethequalityofcareforsexualassaultvictims,and requirementsvarybystate. 7 Thestateofnationalaccessto sexualassaultcare,includingtheknowledgeoffrontline healthcliniciansaboutaccessibility,isunclear.Despitethe effectivenessofSANE-ledcare,4 signi fi cantdisparitiesin accessarebelievedtopersist.7, 8
InadditiontoSANEaccessibility,emergency contraception(EC)isanimportantcomponenentofcare aftersexualassault,justasitisanimportantcomponent ofreproductivehealthcare.Endorsedbyleadingmedical organizations,ECisconsideredasafeandeffectivemeans ofpreventingpregnancy,includingincasesofsexual violence.9,10 ProvisionofECisimportantinthecareof survivorsofabuseordomesticviolence.11AccesstoECinthe EDisimportantbothasacomponentofappropriatecarefor sexualassaultandasaserviceforlow-incomeindividuals becausecostremainsabarrierforthem.Indeed,the AffordableCareActrequiresmostprivateinsurersandstate Medicaidprogramstocoverprescriptioncontraceptionbut notEC.12 In2017,thenationalaveragepricefortrade-name, one-doselevonorgestrelwas$49.48andgenericone-dose levonorgestrelwas$38.74.13 Inadditionto financialbarriers, only4.9%ofpharmaciesareopen24hoursperday/seven daysperweek.14 Otherpotentialbarrierstopatientaccess includerefusaltodispensebypharmacists,misinformation duetopersonalreligiousbeliefs,lackofclinicianexposure, andsocialstigma.15
PopulationHealthResearchCapsule
Whatdowealreadyknowaboutthisissue? In2005anestimated16%ofemergency departments(EDs)intheUSprovided unrestrictedaccesstoemergency contraception(EC).Shiftinglegislation mayhaveimpactedaccess.
Whatwastheresearchquestion?
Whatfactorsaffecttheuserexperienceof seekingECandsexualassaultnurseexaminer (SANE)careinUSEDs?
Whatwasthemajor findingofthestudy?
AccesstoSANEandECwerereported tobeavailablein48.9(95%CI45.5 – 52.0) and42.5%(95%CI39.4 – 45.7)of hospitals,respectively.
Howdoesthisimprovepopulationhealth?
AccesstoSANEcareandECinUSEDsis lowandwithcleardisparities.Resultshave implicationsforimprovingEDpolicies regardingcareofsexualassaultvictims.
A2005studyusinga “mysteryclient” surveyfoundthat onlyanestimated16%ofEDsintheUSprovideaccesstoEC withoutrestriction.16 However,thereisreasontobelievethat accesstoECintheEDhaschanged.Theabovestudywas performedpriortonotableexpansionsinECchoicesand accessintheUS.In2006,theUSFoodandDrug Administrationapprovedtheover-the-countersaleof levonorgestreltothose ≥18yearsofage,andthenin2013 expandedaccesstothose ≥15years.17 Newhormonal optionshavealsobecomeavailable.18 Further,since200514 statesandtheDistrictofColumbiahaverequiredEDsto dispenseECtosexualassaultvictimsuponrequest.19 Currentpenaltiesinclude finesorsuspensionorrevocationof hospitallicensure20;however,theabsenceofstrong enforcementmechanismshascorrelatedwithdecreased compliancerates.21 One2019reviewofliteratureonEC provisioninEDsintheUSfoundthat60%hadapolicyon EC,75%officiallyprovidedECcounseling,44%officially offeredEC,and62%officiallyhadECavailabletodispense onsite.22 Itisunknownhowthesestatisticcorrelate withpractice.
MoststudieshaveexaminedaccesstoSANEservicesand ECintheEDfromtheperspectiveofhospitalpersonnel,
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 292 SANEandECAccessinEDsintheUSA Cowderyetal.
basedoninstitutionalpolicy,orpriortochangesinEC legislation.Thestudiesincludedonlyidealcasesratherthan real-worldconditions;thosethatuseda “mysteryclient” approachshowedlowerratesofaccess.16,22 Thus,studies conductedfromtheperspectiveofthepatientorsexual assaultvictimareneededtodefinenationalaccessand ascertainpotentialdiscrepanciesbetweenpredicted (ie,reportedorpreviouslypublished)andobservedrates ofaccesstoSANEservicesandECintheED.
Giventherecentriseinpresentationsofsexualassaultin theUS1 andtheroleoftheEDasapivotalandtime-sensitive pointofaccessincasesofsexualviolence,wesoughtto evaluateSANEandECavailabilityinEDsintheUSfrom theperspectiveofapatientseekingtoknowtheavailability ofcareoverthephone.Oursurveyaddressesuserexperience, providingapragmaticexampleofpatientexperienceswhen attemptingtoaccesssexualassaultservicesandECthrough theED;thisstudyalsoexaminesdifferencesinperceived availabilityoftheseservicesonthebasisofgeographicand institutionalfactors.
MATERIALSANDMETHODS
Wesoughttoupdatethe2005telephone-based, “secret shopper” studyofhospitalsacrosstheUStoinvestigate patientaccesstosexualassaultcareusingtheavailability ofSANEservicesandECasaproxyforaccessto comprehensiveservicesfromtheperspectiveofaprospective patient.Toassessaccessibilityandperceivedavailability,we usedthereportoffrontlinehealthcareworkerslikelytobethe firstpointofcontactforpatientsintheEDasthesourceof informationregardingavailableservices.Wealsosoughtto determinewhethergeographicandinstitutionalfactorswere associatedwithreportedaccess.Moreover,giventhe influenceofgraduatemedicaleducationprogramson institutionalresources,wesoughttodeterminewhether teachingstatusimprovedaccess.Ourstudyincludeda demographicevaluationbasedonsize,ruralvsurbansetting, teachingstatus,andfaith-basedstatusofhospitals.
WeobtainedalistofEDsintheUSfromapublicly availabledatabaseoftheCentersforMedicare&Medicaid Services(CMS)inMarch2019.Thisdatabaseconsistedof 4,806hospitals.Exclusioncriteriaincludedfederal institutions,children’shospitals,tribalhospitals,hospitals withoutEDs,andhospitalslocatedinUSterritories.Ofthe remainingeligible4,360hospitals,25%wererandomly selectedandstratifiedbyregion(Northeast,South,West, andMidwest;see TableS1 forthelistofstatesperregion)and byteachingstatus.Weaimedtosurveygreaterthan20% ofeligiblehospitalswith21%representationofteaching institutions,whichwastheproportionofteaching institutionsintheoverallcohort.Hospitalswereclassifiedas teachinghospitalsonthebasisoftheirregistrationwiththe CMS.Eachregionalsamplewascheckedtoensure representationofhospitalsclassifiedashavingteaching
status.Ingeneral,foreverythreenon-teachinginstitutions, oneteachinginstitutionexistedintheanalysiswithin eachregion.
Fortheanalysis,hospitalswereclassifiedbyregionand stateassmall(<100beds),mid-sized(100–200beds),orlarge (>200);asurban(population ≥50,000)orrural(population <50,000);asacademicornon-academic;asfaith-ornonfaith-based;andbythepresenceofastatelegislative requirementtoofferECtosexualassaultvictims.Ateamof fivewomeninvestigatorssimulatingpotentialpatientscalled publiclyavailableEDphonenumbersforeachhospital betweenJune–September2019,seekingECasdescribedby Harrisonetal.16 CallerscontactedtheEDseekingmedical adviceandaskedaboutECandSANEaccess.The respondentwouldeitherprovidetheresponseortransferthe calltoamoreknowledgeablememberofthemedicalstaff includingadvancedpracticeprovidersandphysicians.
Callersreceivedstructuredtrainingwithstandardized scripts,whichwerethencalibratedthroughaseriesof simulatedcalls.Inaddition,5%ofcallswerescreenedfor fidelityandtoensurestandardizationbycompletingaseries ofobservedcallencounters.Thephonenumbersofthecallers wereconcealed,andthetimeofthedayanddayoftheweek wasrecorded;callstookplaceduringnormalbusinesshours (ie,9 AM-5 PM).Callers firstaskedaboutaccesstoECand thenaskedifitwasavailableinthecaseofsexualassault. TheythenaskedwhetheraSANEwasavailable.Thisscript wasmodeledontheprotocolofthemostrecentsurveyofEC access. 16 Followingthe first5%ofcalls,thescriptwasrevised andstandardizedforincreased fidelityindatacollection. Revisionsincludedminorchangesinwordingandorder ofquestions.
PrimaryoutcomeswerereportedaccesstoSANEsand ECintheED.ASANEwasconsideredavailableifthe respondentreportedthataSANEwasonsiteorcouldbeon sitewithinsixhours.ASANEwasconsiderednotavailableif respondentsweretoldtherewerenoSANEsavailablewithin sixhours.WedefinedECaccessasfull,conditional,orno access.Fullaccessincludedhospitalsthatreportedthatthey hadavailableECwithnorestriction.Conditionalaccesswas definedashospitalsthatreportedthattheyprovidedEConly inthecircumstanceofsexualassault;andnoaccesswas definedashospitalsthatreportedanabsenceofECprovision underanyconditionsorifthecallerwasreferredtoan outpatientpharmacyforaccess.Secondaryoutcomes includedtypeofECoptionsavailable,alternativemethodsof obtainingEC,accesstoreferraltoalternativehealthcare systems,andaccesstosexualassaultresources.Aswesought topragmaticallyimitatetheexperienceofaprospective patientcallingtheED,callersdidnotaskforthe qualificationsofrespondents,nordidtheyasktobe transferredtoaphysicianornurse,althoughtheytooksuch transfersiftheywereoffered.Theyrecordedthe first definitiveresponsetheyreceivedfromanystaffmember.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 293 Cowderyetal. SANEandECAccessinEDsintheUSA
WemanagedallstudydatainResearchElectronicData Capturev9.11,hostedattheUniversityofFlorida. StatisticalanalyseswereperformedusingSASv9.4(SAS Institute,Inc,Cary,NC).Weinitiallyuseddescriptive statistics,includingmeans,medians,frequencies,and proportions,toexaminesurveyresponserepresentation, variabledistribution,andmissingnesswhereappropriate. Wecalculatedexactconfidenceintervals(CI)usingthe Clopper–Pearsonmethod.Unadjustedandadjustedlogistic regressionmodelswereusedtoevaluatetherelationship betweenhospitalcharacteristicsandoutcomes.We performedanunconditionalhierarchicallogisticregression model,whereEDswerenestedinrespectivestates,toassess thepredictedprobabilityofanEDprovidingECforeach state.Eachstatewasaddedasarandomeffect.
Thisstudyreceivedapprovalforexemptionfromthe UniversityofFloridaInstitutionalReviewBoardprior toinitiation.
RESULTS
BetweenJuly2–September5,2019,callersattemptedto call1,046hospitalsandcompleted960calls(91.7%response rate).Eighty-sixofthecalls(8.2%)failedduetothefollowing reasons:failuretocontact(25,2.3%);refusaltoanswer
questions(13,1.2%);hospitalclosure(20,1.9%);noED(7, 0.6%),oranotherunclassifiedreason(21,2%).The Figure illustratesthe flowofhospitalinclusionorexclusionthrough thestudy. Table1 presentsthecharacteristicsofthe960 hospitalsthatweresuccessfullysurveyed.(See TableS2 for thebreakdownofnumberofhospitalsbystate.)Sexual assaultnurseexaminerswerereportedtobeavailablein 48.9%ofthe960hospitalssurveyed(Table2).
Afteradjustingforcovariates,thefollowingfactorswere independentpredictorsofSANEaccess:region;ECaccess; sizeofhospital;academicstatus;andurbanstatus(Table3). Seeunadjustedcomparisionsin TableS2).Regionwas associatedwithreportedSANEaccess,withhospitalsinthe Northeastbeing4.00timesmorelikely(95%CI2.38–7.14), 2.78timesmorelikel(95%CI1.59–4.76),and2.00times morelikely(95%CI1.19, 3.33)tohaveSANEaccessthan hospitalsintheSouth,West,andMidwest,respectively (Table3).ReportedECaccessincasesofsexualassaultwas alsoassociatedwithSANEpresence,withemployeesatthese hospitals3.94timesmorelikely(95%CI2.66–5.83)toreport
–15.0
Midwest284(29.6)26.7–32.6
South369(38.4)35.4–41.6
West185(19.3)16.8–21.9
Urban–ruralstatus – n(%)
Urbanarea319(33.2)30.3–36.3
Ruralarea641(66.8)63.7–69.8
Sizeofhospital – n(%)
Small469(48.9)45.7–52.1
Medium203(21.2)18.6–23.9
Large288(30.0)27.1–33.0
Numberofbeds – mean ± sd170.5 ± 206.3157.4–183.5
Faith-basedstatus – n(%)
Faithbased173(18.0)15.6–20.6
Non-faithbased787(82.0)79.4–84.4
Hospitaltype – n(%)
Academic237(24.7)22.0–27.5
Non-academic723(75.3)72.5–78.0
StaterequirementifSA – n(%)
In-staterequiringdispense284(29.6)26.7–32.6
Notrequiredtodispense612(63.8)60.6–66.8
Nostatelaw(Ohioand Pennsylvania) 64(6.7)5.2–8.4
CI,confidenceinterval; SA,sexualassault.
Figure. StandardsforReportingofDiagnosticAccuracydiagram reporting flowofparticipantsthroughthestudy.
Hospitalcharacteristics Total (N = 960)95%CI Region – n(%) Northeast122(12.7)10.7
Table1. Hospitalcharacteristicsofstudysample.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 294 SANEandECAccessinEDsintheUSA Cowderyetal.
Contraceptionoptions
= 408)
MethodofobtainingEC
= 408)
Accesstoreferrals(ifECnot available,n = 551)
*FullaccessvaluesarehospitalsthatansweredyestohavingEC wheninitiallyasked.
**Conditionalaccessvaluesarehospitalsthatrespondedno tohavingECavailableinitially,butyeswhensexualassault wasreported.
CI,confidenceinterval; EC,emergencycontraception; IUD, intrauterinedevice; SANE,sexualassaultnurseexaminer.
havingSANEswhencomparedtothoseathospitalswithout reportedECaccess(Table3).
Mid-sizedandlargehospitalswere2.96(95%CI 1.72–5.11)and2.43(95%CI1.63–3.61)timesmorelikely, respectively,toreporthavingSANEaccessthansmall hospitals.Priortoadjustingforcovariates,itappearedas thoughurbanhospitalsweremorelikelytoreporthaving SANEaccess(1.39timesmorelikely, Table3).Inthe
adjustedmodel,however,ruralhospitalswere1.48times morelikely(95%CI1.00–2.20)toreporthavingSANEs, illustratingareversaloftheassociationwithhospitalsize actingasthequalitativeconfounder(Table3).Faith-based andnon-faith-basedhospitalsreportedhavingSANEs availableatsimilarratesof51.7%and48.2%,respectively (Table3).Academichospitalswere2.18timesperlikely(95% CI1.42–3.34)toreporthavingSANEaccessthannonacademichospitals(Table3).
Ofthe960hospitalsincluded,551(57.4%)reportedno accesstoEC.Ofthe408(42.5%)reportingECaccess,215 (22.4%)hadfullaccess,and193(20.1%)hadconditional access(Table2).Ofthe551hospitalswithnoaccess,341 (61.9%)hadareferralsystemtoobtainEC.Ofthe408 hospitalswithreportedECaccess,196(48.0%)prescribed levonorgestrel,six(1.5%)prescribedulipristalacetate,and 216(52.9%)ofrespondentsdidnotknowtheavailable options.Nohospitalsreportedthecopperintrauterinedevice (IUD)asanoption.Themajorityofrespondentstoldcallers thatEDsleaveECprovisiontothediscretionofthephysician (62.7%),2.9%requireapregnancytest,and6.4%requirea pelvicexamination(Table2).Nationally,thepredicted probabilityofarespondentreportingthattheirhospitaldid notprovideanyECintheEDwas55.2%(TableS3).
Massachusetts,Oregon,NewJersey,NewYork, Washington,andWisconsinhadasignificantlygreater predictedprobabilityofreportedECaccessinEDsthanthe nationalaverage,whileFlorida,California,Kansas, Louisiana,Texas,andNebraskahadasignificantlylower chanceofhavingECthanthenationalaverage(TableS3). ThepresenceofastaterequirementtoprescribeECfor sexualassaultvictimswasthesecondstrongestpredictorof ECaccess(followingregion),at2.27timesmorelikely(95% CI1.59–3.22).Additionally,ruralhospitalswere1.65times morelikely(95%CI1.11–2.44)thanurbanhospitalstohave anyECaccess,andacademichospitalswere1.58timesmore likely(95%CI1.05–2.39)tthannon-academichospitalsto haveanyECaccess(Table4;seeunadjustedcomparisions in TableS4).
Afteradjustingforcovariates,reportedECaccesswas associatedwithhospitalregion,urbanstatus,academic status,andstaterequirementincasesofsexualassault (Table4).Afterexcludinghospitalswithreported conditionalaccessincasesofsexualassault,faith-based statusbecameanadditionalindependentpredictor,whilethe associationbetweenacademicstatusandECaccesswasno longersignificant(Table5).
DISCUSSION
Globally,ratesofsexualassault,gender-basedviolence, andhumantraffickingforsexualexploitationremainhigh, andaccesstoappropriatecarefollowingasexualassault remainsmarkedbysharpdisparities.23–25 Similarly,our studysuggeststhatthereisinconsistentaccesstoSANEsand
Surveyresponses (N = 960)Frequency(%)95%CI ECAccess Fullaccess*215(22.4)19.8–25.2 Noaccess551(57.4)54.2–60.5 Conditionalaccess**193(20.1)17.6–22.8
Table2. Emercencycontraceptionsurveyresponseby hospitalsample.
(ifavailable,n
Levonorgestrel(PlanB)196(48.0)43.1–53.0 Ulipristal(Ella)6(1.5)0.5–3.2 IUD0(0.0)0.0–0.0 Don’tknow216(52.9)48.0–57.9
Physiciandecision256(62.7)57.9–67.5 Pregnancytest12(2.9)1.5–5.1 Pelvicexam26(6.4)4.2–9.2 Don’tknow87(21.3)17.4–25.6 Other62(15.2)11.9–19.1
(ifavailable,n
Yes341(61.9)57.7–66.0 No207(37.6)33.5–41.8
resources Yes653(68.0)65.0–71.0 No281(29.3)26.4–32.3 Don’tknow23(2.4)1.5–3.6
–52.0 No458(47.8)44.5–50.9 Don’tknow32(3.3)2.3–4.7
Accesstosexualassault
AccesstoSANEs Yes468(48.9)45.5
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 295 Cowderyetal. SANEandECAccessinEDsintheUSA
Urban
HospitalcharacteristicsUnadjustedmodelOR(95%CI)AdjustedmodelOR(95%CI)
Sizeofhospital
–0.95)1.48(1.00–2.20)
SmallRefRef
Medium2.26(1.61–3.19)2.96(1.72–5.11)
Large2.76(2.03–3.76)2.43(1.63–3.61)
Numberofbeds(per250increase)1.49(1.24–1.80)0.91(0.70–1.18)
Faith-basedstatus
Non-faithbasedRefRef
Faithbased1.16(0.83–1.63)1.01(0.70–1.46)
HospitalType
Non-academicRefRef
Academic2.87(2.08–3.96)2.18(1.42–3.34)
StaterequirementifSA
In-staterequiringdispenseRefRef
Notrequiredtodispense0.65(0.49–0.87)0.95(0.66–1.38)
Nostatelaw(OhioandPennsylvania)1.42(0.81–2.52)0.81(0.42–1.53)
ECaccess
NoaccessRefRef
Fullaccess*2.87(2.05–4.00)2.33(1.62–3.34)
Conditionalaccess**4.82(3.33–6.97)3.94(2.66–5.83)
OR,oddsratio; CI,confidenceinterval; SA,sexualassault; EC,emergencyconception.
ECinEDsacrosstheUS.Whilethisstudydoesnotestablish thedistributionofabsoluteaccess,ourmethodology providesapragmaticdepictionofthepatientexperience whenattemptingtoaccesssexual-assaultservicesandEC throughanED.Thispicturereflectsstarkdisparitiesin accessaswellasoveralllowlevelsofaccesstoSANEsand ECnationally.Our findingshighlightthedifferencebetween policyandpractice,whichmaybeinfluencedbybias,lackof knowledgeofpolicybyclinicians,andotherfactors.
RoughlyhalfoftheEDssurveyedreportedthattheycould notprovideSANEsforsexualassaultvictimsonsitewithin sixhours,andrespondersintheSouthweretwiceaslikelynot toknowwhethertherewasaSANEavailable.This findingis incontrasttootherstudiesconductedintheSoutheasternUS thatreliedonclinicianandadministratorsurveys,which
foundthataccesstoSANEandECwasconsistentwiththe standardofcare.26 Itis,therefore,unclearwhetherthis regionaldifferencerepresentstrueavailabilityoragapin theeducationoffrontlineemergencycliniciansinthe southernUS.
Largeracademicinstitutionsweremorelikelytohavea SANEavailable,possiblybecauseforthoseinstitutionsit waslessofa financialburden.ThecosttodevelopaSANE programcanbeupto$40,000.27 Accordingtothe InternationalAssociationofForensicNursing(IAFN),only 1,200IAFN-certifiedSANEsforadultsandadolescentsare availableinternationally.28 Asaresult,disparitiesinaccess arelikely,andalthoughthereasonsarenotwellstudied,they likelyincludeanumberofvariablessuchashighcosts, limitedtrainingopportunities,andalackofsupportive
Table3. Unadjustedandadjustedoddsratiosofaccesstoasexualassaultnurseexaminerbyhospitalcharacteristics(availablevs notavailable).
–0.69)0.50(0.30–0.84)
–0.40)0.25(0.14–0.42)
–0.57)0.36(0.21–0.63)
Region NortheastRefRef Midwest0.43(0.27
South0.26(0.16
West0.35(0.21
–ruralstatus
Ruralarea0.72(0.55
UrbanareaRefRef
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 296 SANEandECAccessinEDsintheUSA Cowderyetal.
HospitalcharacteristicsUnadjustedmodelsOR(95%CI)AdjustedmodelOR(95%CI) Region
NortheastRefRef
Midwest0.31(0.20–
–0.65)
South0.16(0.10–0.26)0.25(0.15–0.43)
West0.32(0.20–
Urban–ruralstatus
UrbanareaRefRef
–0.57)
Ruralarea1.05(0.80–1.37)1.65(1.11–2.44)
Sizeofhospital
SmallRefRef
Medium1.31(0.94–1.84)1.35(0.91–2.00)
Large1.59(1.81–2.15)1.60(0.93–2.73)
Numberofbeds(per250increase)1.18(1.01–1.39)1.07(0.83–1.38)
Faith-basedstatus
Non-faithbasedRefRef
Faithbased0.91(0.65–1.28)0.90(0.62–1.30)
Hospitaltype
Non-academicRefRef
Academic1.67(1.25–2.25)1.58(1.05–2.39)
StaterequirementifSA
In-staterequiringdispenseRefRef
Notrequiredtodispense0.33(0.25–0.45)0.44(0.31–0.63)
Nostatelaw(OhioandPennsylvania)0.93(0.54–1.62)0.55(0.30–1.02)
Anyaccess*includeshospitalswithfullaccess**orconditionalaccess***.
Fullaccess**valuesarehospitalsthatansweredyestohavingemergencycontraceptionavailablewheninitiallyasked. Conditionalaccess***valuesarehospitalsthatrespondednotohavingECavailableinitially,butyeswhensexualassaultwasreported. OR,oddsratio; CI,confidenceinterval; SA,sexualassault.
resources,particularlyinalreadyunderservedareas.29 Our results,incombinationwiththeincreaseinthenumberof sexualassaultpatientsbeingseenintheED,1 highlightthe needforhospitalstobepreparedwithproperlytrainedstaff totreatthispatientpopulation.Onepossiblesolutiontothe costofSANEservicesforindividualhospitalsisto regionalizeresources.
Intheunadjustedmodel,ruralhospitalsappearedless likelytohaveSANEsavailable;however,onceadjustedfor hospitalsize,ruralhospitalsweremorelikelytoreport havingaSANE.Thisiscontrarytopreviousresearchin Pennsylvania,Washington,andOregon,which demonstratedthatprogramsinruralareaswerelackingin SANEsandfacilities,resultinginurbanprogramsabsorbing patientsfromunderservedareas,7,8 Thismaybearesultof theavailabilityofsexualassaultresourcesoutsidetheEDin urbanareas,orofthecentralizationofSANEsatasingle hospitalinanurbancenter.Iftheresultsofthisstudy representaccesstoSANEs,ratherthanalackofknowledge
amongfrontlinehealthcarepractitioners,thereisastrong disparityinSANEaccessforsexualassaultpatientsbasedon regionandhospitalsize.Thisdisparitymayaffectthequality ofcounselingandforensicevidencecollectionbasedonthe locationofthehospital,whichcouldhavelegalramifications forvictimsashospitalsindifferentlocationsmaynotequally facilitatethecollectionofhigh-qualityevidenceincasesof sexualassault.
Only22.4%ofEDfrontlinehealthcarepractitioners reportedthattheyprovideECwithoutrestriction; furthermore,anadditional20.1%reportedthatthey providedEConlyincasesofsexualassault.Ourresultsalign withthosereportedbyHarrisonetalin2005,withaminority (31.5%)ofsurveyedEDsfoundtoprovideEC.16 Thepoor accesstoECfoundinthisstudymayinpartreflectincreased accesstoalternativeresources,suchasover-the-counterEC atpharmaciesorwomen’sspecialtyclinics.Thelowrateof accessreportedbyEDpersonnelmayalsobeduetolackof knowledgeofhospitalpoliciesregardingECamongfrontline
Table4. Unadjustedandadjustedoddsratiosofemergencycontraceptionaccessbyhospitalcharacteristics(anyaccess*vsnoaccess).
0.50)0.39(0.24
0.53)0.33(0.19
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 297 Cowderyetal. SANEandECAccessinEDsintheUSA
HospitalcharacteristicsUnadjustedmodelsOR(95%CI)AdjustedmodelOR(95%CI) Region
NortheastRefRef
Midwest0.22(0.13
South0.14(0.08
West0.30(0.18–
Urban–ruralstatus
UrbanareaRefRef
Ruralarea1.14(0.81–1.60)1.74(1.05–2.87)
Sizeofhospital
SmallRefRef
Medium1.34(0.75–1.72)1.11(0.67–1.82)
Large1.42(0.99–2.04)1.28(0.67–2.46)
Numberofbeds(per250increase)1.16(0.97–1.39)1.16(0.86–1.55)
Faith-basedstatus
Non-faithbasedRefRef
Faithbased0.45(0.27–0.75)0.44(0.25–0.76)
Hospitaltype
Non-academicRefRef
Academic1.61(1.12–2.30)1.67(0.99–2.82)
StaterequirementifSA
In-staterequiringdispenseRefRef
Notrequiredtodispense0.32(0.22–0.45)0.42(0.27–0.67)
Nostatelaw(OhioandPennsylvania)0.93(0.54–1.62)0.54(0.25–1.14)
Fullaccess*valuesarehospitalsthatansweredyestohavingemergencycontraceptionwheninitiallyasked. OR,oddsratio; CI,confidenceinterval; SA,sexualassault.
EDstaff,especiallyaboutcostsandbarriersassociatedwith thesealternativeresources.27 SimilartowhatHarrisonetal reported,respondentsinourstudyfrequentlyprovided incorrectormisguidedcommentsregardingEC.Several respondentsreferredtoECasan “abortionpill,” possibly mifepristone,orahysterectomyduringthephonecall.
AccordingtotheEDnon-physicianpractitionerssurveyed, 63%ofECprovisionwasbasedonindividualphysician discretion,whichisnotrequiredinmanystates.
Studieshavedemonstratedthatlessthan50%ofvictimsof sexualassaultseekmedicalattention.Whilethereasonsare multifactorial,itisclearthatvictimsexperienceserious psychosocialandemotionalstressthatmaycontributetoa reluctancetobesubjecttoadditionalscrutiny,lossof privacy,orinvasiveexaminations.30 Inoursurvey,many reportedthatECwasdispensedonlyfollowingaphysician assessment,whichwouldincludeapelvicexamination. ManyrespondentsinourstudystatedthattheirEDdidnot takesexualassaultcasesandthatthepatientwouldneedto betransferredtoanotherfacilityorseekguidancefrom lawenforcement.
FewrespondentsprovidedthespecificbrandofEC available,andnoneofferedthecopperIUDasanoption. Manyrespondentscommonlyreferredpatientstoprivate pharmaciesforEC,aproblematicpracticegivencoverageof costsandpotentiallogisticaldifficultiesanddelays.
Importantly,thereferralofthoseseekingECtoprivate pharmacieslimitsaccesstoconsultationonsexually transmitteddisease,behavioralhealth,ortheopportunityto reporttolawenforcementinthecaseofvictimsofsexual assault,domesticviolence,ortrafficking.These findings underscoretheneedforincreasedtrainingforhealthcare practitionersresponsiblefortriageandresponsetoinquiries.
InstateswithlegislationrequiringaccesstoECincasesof sexualassault,EDsweremorethantwiceaslikelytoreport thatECwasavailablewithoutrestriction,demonstrating thatsuchlegislationmayhaveanimpact.Withcurrent enforcementmechanismsinplaceforonly13states, thereisroomforexpansionoflegislationtocoverthe remainingstates.
Perhapsunexpectedly,ECwasmorelikelytobeavailable inruralhospitalsafteradjustingforcovariates.Rural
Table5. Unadjustedandadjustedoddsratiosoffullemergencycontraceptionaccessbyhospitalcharacteristics(fullaccess*vsnoaccess).
–0.37)0.32(0.18–0.59)
–0.23)0.23(0.12–0.45)
0.54)0.33(0.18
–0.63)
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 298 SANEandECAccessinEDsintheUSA Cowderyetal.
hospitalsoftenserveascriticalaccesspointsforremoteor underservedcommunities.Non-faith-basedhospitalswere morethantwiceaslikelytoreportprovidingECthanfaithbasedhospitals,consistentwithapreviousstudyinwhich non-CatholichospitalsweremorelikelytoprovideECthan Catholichospitals.21 This findingmaybebasedon localinstitutionalpolicieslimitingaccessamong faith-basedinstitutions.
Thereisaneedforimprovededucationonsexualassault care,aswellasanincreaseinSANEaccessamonghospitals. HospitalsshouldconsiderbuildingSANEresourcesintoED protocols.Hospitaladministratorscancollaboratewithlocal rapecrisiscentersorapplyforfederalgrantsorfundingto defraythecostoftrainingandsupplies.
LIMITATIONS
Theprimarylimitationofthisstudywastheinconsistency inknowledgeofprotocolsrelatedtothistopicand willingnesstoprovideaccurateinformationoverthe telephone.Itisplausiblethatcallerswouldhavereceived differentinformationhadtheencounterbeeninperson. However,aphoneprotocolwasspecificallychosenasa pragmaticapproachusedbyapotentialmemberofthe communityseekingservices.
Thespeci ficinquiryregardingsexualassaultratherthan theinitialrequestforECmayhavein fl uencedthe respondent ’ sresponseregardingresourcesandaccess. Respondentsinthisstudymayhavebeenmoremotivatedto fi ndananswertoquestionswhenthetopicofsexualassault wasintroduced.Forexample,somerespondentswhostated ECwasnotavailablechangedtheirresponseuponthe callers’ disclosurethattherehadbeenasexualassault. WhenrespondentsendorsedSANEaccess,callersdidnot recordon-siteavailability,northehourswhenaccesswas available.Calltimingwasvariedrandomlybetween 9 AM– 5 PM butwasnotstandardized.Inaddition,asmany sexualassaultspresentoutsidenormalworkinghours, itispossiblethattheavailabilitycouldbeevenlower duringoff-hours.
CONCLUSION
Accesstoemergencycontraceptionandsexualassault nurseexaminersinEDsremainslimitedwithdisparitiesin accessacrossthenation.Variableaccessibilitydependingon thegeographiclocationofthehospitalorthelegislative statusofthestatesuggeststhatthoseseekingtheseresources mightreceivesubstandardqualityofhealthcaredepending ontheinstitutionwheretheyhavechosentoseekcare.Given theimportanceofECandsexualassaultservices,emergency physiciansmay finditworthwhiletoexaminetheirhospitals’ existingprotocolsregardingdispensingprescriptionsofthese medicationsandavailabilityofSANEs.Hospitalsshould considerprovidingtrainingforallEDstaff,especiallythose
who firstinteractwithpatients,topreventmisinformation aboutpatientaccesstoECorSANEs.
AddressforCorrespondence:Marie-CarmelleElie-Turenne,MD, UniversityofAlabamaatBirmingham,HeersinkSchoolofMedicine, DepartmentofEmergencyMedicine,1670UniversityBlvd, Birmingham,AL35233.Email: melie@uabmc.edu
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.
Copyright:©2024Cowderyetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
REFERENCES
1.VogtEL,JiangC,JenkinsQ,etal.(2022).TrendsinUSemergency departmentuseaftersexualassault,2006-2019. JAMANetwOpen 2022;5(10):e2236273.
2.NielsonMH,StrongL,StewartJG.Doessexualassaultnurse examiner(SANE)trainingaffectattitudesofemergencydepartment nursestowardsexualassaultsurvivors? JForensicNurs 2015;11(3);137–43.
3.CianconeAC,WilsonC,ColletteR,etal.Sexualassaultnurse examinerprogramsintheUnitedStates. AnnEmergMed 2000;35(4):353–7.
4.CampbellR,PattersonD,LichtyLF.Theeffectivenessofsexual assaultnurseexaminer(SANE)programs:areviewofpsychological, medical,legal,andcommunityoutcomes. TraumaViolenceAbuse 2005;6(4):313–29.
5.Fehler-CabralG,CampbellR,PattersonD.Adultsexualassault survivors’ experienceswithsexualassaultnurseexaminers(SANEs). JInterpersViolence.2011;26(18):3618–39.
6.SchmittT,CrossTP,AlderdenM.Qualitativeanalysisofprosecutors’ perspectivesonsexualassaultnurseexaminersandthecriminaljustice responsetosexualassault. JofForensicNurs.2017;13(2):62–8.
7.ClowersN.Sexualassault:informationontheavailabilityofforensic examiners. JusticeLawEnforcRep.2018;2019:279–92.
8.ThiedeEandMiyamotoS.Ruralavailabilityofsexualassaultnurse examiners(SANEs). JRuralHealth.2021;37(1),81–91.
9.PracticeBulletinNo.152:EmergencyContraception. ObstetGynecol 2015;126(3):e1–11.
10.AmericanCollegeofEmergencyPhysicians.Emergencycontraception forwomenatriskofunintendedandpreventablepregnancy. Annof EmergMed.2005;46(1):103.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 299 Cowderyetal. SANEandECAccessinEDsintheUSA
11.LewisNV,MooreTH,FederGS,etal.Useofemergencycontraception amongwomenwithexperienceofdomesticviolenceandabuse: asystematicreview. BMCWomensHealth, 2018;18(1):156.
12.GuttmacherInstitute.InsuranceCoverageofContraceptives.Available at: https://www.guttmacher.org/state-policy/explore/ insurance-coverage-contraceptives.AccessedMarch3,2020.
13.ContraceptionASoE.NotThereYet:ASEC’s2017ECAccessStudy. Availableat: https://www.americansocietyforec.org/_files/ugd/ 7f2e0b_0e4b5a581359421c84b67abafabf8e30.pdf AccessedMarch3,2020.
14.QatoDM,ZenkS,WilderJ,etal.Theavailabilityofpharmaciesinthe UnitedStates:2007–2015. PloSOne.2017;12(8):e0183172.
15.PragerS.Accesstoemergencycontraception. ObstetGynecol 2017;130(1):E48–52.
16.HarrisonT.Availabilityofemergencycontraception:asurvey ofhospitalemergencydepartmentstaff. AnnEmergMed 2005;46(2):105–10.
17.ClelandK,ZhuH,GoldstuckN,etal.Theefficacyofintrauterine devicesforemergencycontraception:asystematicreviewof35yearsof experience. HumReprod.2012;27(7):1994–2000.
18.AdministrationUSFaD.Ella(ulipristalacetate)Tablets.2015.Available at: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2015/ 022474Orig1s007TOC.cfm.AccessedMarch3,2020.
19.GuttmacherInstitute.EmergencyContraception.Availableat: https:// www.guttmacher.org/state-policy/explore/emergency-contraception
AccessedMarch3,2020.
20.NationalWomen’sLawCenter.ProvidingEmergencyContraceptionto SexualAssaultSurvivors:ElementsofaSuccessfulStateECintheER Law.2013.Availableat: https://nwlc.org/wp-content/uploads/2015/08/ providing_ec_to_sexual_assault_survivors_factsheet_6-28-13.pdf AccessedMarch3,2020.
21.PolisC,SchafferK,HarrisonT.Accessibilityofemergency contraceptioninCalifornia’sCatholichospitals. WomensHealthIssues 2005;15(4):174–8.
22.WangMJ,KhodadadiAB,TuranJM,etal.Scopingreviewofaccessto emergencycontraceptionforsexualassaultvictimsinemergency departmentsintheUnitedStates. TraumaViolenceAbuse 2021;22(2):413–21.
23.CepedaI,Lacalle-CalderonM,TorralbaM.Measuringviolence againstwomen:aglobalindex. JInterpersViolence 2022;37(19-20):NP18614–38.
24.DworkinER,KrahéB,ZinzowH.Theglobalprevalenceofsexual assault:asystematicreviewofinternationalresearchsince2010. PsycholViolence.2021;11(5):497–508.
25.UnitedNationsOfficeonDrugsandCrime.GlobalReportonTrafficking inPerson.2009.Availableat: https://www.unodc.org/documents/ Global_Report_on_TIP.pdf.AccessedSeptember15,2021.
26.HarrisonT,FosterAM,MartinSB,etal.Assessingsexualassault survivors’ accesstoemergencycontraception:resultsfromamixed methodsstudyinSouthCarolina. WomensHealthIssues 2010;20(4):248–53.
27.HoumesBV,FaganMM,QuintanaNM.Establishingasexualassault nurseexaminer(SANE)programintheemergencydepartment. JEmergMed.2003;25(1):111–21.
28.ColbertAMandSekulaLK.AmajorinvestmentintheUnitedStates sexualassaultnurseexaminerworkforce. JForensicNurs 2022;18(1):1–3.
29.IritaniK.Sexualassault:informationontraining,funding,andthe availabilityofforensicexaminers.2016.Availableat: dspacedirect.org AccessedFebruary4,2024.
30.ReevesEAandHumphreysJC.Describingthehealthcareexperiences andstrategiesofwomensurvivorsofviolence. JClinNurs 2018;27(5–6):1170–82.
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 300 SANEandECAccessinEDsintheUSA Cowderyetal.
LETTERSTOTHE EDITOR
FactorsAssociatedwithOverutilizationofComputedTomography
CervicalSpineImaging
TessyLaTorreTorres,DO
JonathanMcGhee,DO
ChristianaCareHospital,DepartmentofEmergencyMedicine,Newark,Delaware
SectionEditor:MarkLangdorf,MD,MHPE
Submissionhistory:SubmittedNovember12,2023;RevisionreceivedNovember12,2023;AcceptedDecember8,2023 ElectronicallypublishedFebruary28,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.18570
DearEditor:
Wearewritingtoprovidesomecommentsonthescientific paperrecentlypublishedinyourjournaltitled “Factors AssociatedwithOverutilizationofComputedTomography oftheCervicalSpine.”1
Firstly,wecommendtheauthorsforputtingtogethera relevantandwell-donemulticenterstudythatboth revalidatestheNEXUScriteriaandoffersinsightintothe overutilizationofcomputedtomography(CT)fortraumatic injuries.However,wedohavesomeconcernsaboutthe methodologyused.Havingasinglereviewercollectingchart dataonNEXUScriteria criteriathatweknowincludethe subjectivecomponentofadistractinginjuryordeficitnot attributabletopain introducesthepossibilityofbias;it would,therefore,bebeneficialtoseecongruenceofchart analysisbetweendifferentreviewers.Itisalsorecognizable thatthereweretimingconstraintsrelatedtofeasibility,thus allowingforonlyonepersontorevieweachchartforthe presenceofNEXUScriteria.Theprocessinvolvedcombing throughmorethan800individualrecordsthatincluded physiciandocumentation,imaging,labstudies,andnursing notes.Thisaddedsignificantlytotheworkloadofthesingle reviewer,whichcouldhaveimpactedtheoverallaccuracyof thedatacollected.Additionally,itwasunclearwhetherthe reviewerwasblindtotheresultoftheCTwhenreviewingthe chart,openingupfurtheropportunitiesforbias.
Secondly,theshorttimeframeinthiscaselinkedtoskiing/ wintersports-relatedinjuriesmayprovideonlyapartial picture,limitingtheapplicabilityofresults.Imaginethe studyhadbeenconductedforlongerthantwomonths outsidethewinterseason.Wouldtherebeadditional variablesregardingthemechanismofinjuryassociatedwith theoverutilizationofCTimagingnototherwiseuncoveredin theirinitialreview?Additionally,thebaselinecharacteristics forthepresentingmechanismofinjuryincludedfalls,which constitutedapproximately75%ofthetotalnumber.Further
characterizationofthemechanismofinjurymayalsohave beenbeneficial fallfromstandingvsfromaheight,or motorvehiclecollisionwithairbagdeploymentvswithout couldallrevealassociationsofinjurythatwouldcause physicianstobypasstheNEXUScriteriaaltogether.
Futurestudiesshouldlooktoinvestigatewhethera physician-in-triagestructureisassociatedwithincreasedCT overutilization.Weareseeingmoreprotocolsbeing implementedinemergencymedicine,includingwithinthe triageprocess,anditwould,therefore,beinterestingtosee howthisalternateworkflowwouldaffectresults.
Overall,wefoundtheauthors’ studytobeextremely informative,andweappreciatetheircontributiontotheever evolvingandhighlychallenging fieldofemergencymedicine.
AddressforCorrespondence:TessyLaTorreTorres,DO, ChristianaCareHospital,DepartmentofEmergencyMedicine, 4755Ogletown,StantonRd,Newark,DE19718.
Email: tessy.latorretorres@christianacare.org
ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.
Copyright:©2024LaTorreTorresetal.Thisisanopenaccess articledistributedinaccordancewiththetermsoftheCreative CommonsAttribution(CCBY4.0)License.See: http:// creativecommons.org/licenses/by/4.0/
REFERENCE
1.ChamberlinKT,CanellasMM,ReznekMA,etal.Factorsassociated withoverutilizationofcomputedtomographyofthecervicalspine. WestJEmergMed. 2023;24(5):967–73.
Volume25,No.2:March2024WesternJournal of EmergencyMedicine 301
LETTERSTOTHE EDITOR
Replyto “FactorsAssociatedwithOverutilizationofComputed TomographyCervicalSpineImaging”
KarlChamberlin,MD,MBA*† *UniversityofMassachusettsChanMedicalSchool,EmergencyMedicine, Worcester,Massachusetts
† UMassMemorialHealth,DepartmentofEmergencyMedicine, Worcester,Massachusetts
SectionEditor:MarkLangdorf,MD,MHPE
Submissionhistory:SubmittedDecember5,2023;RevisionreceivedDecember5,2023;AcceptedDecember8,2023 ElectronicallypublishedFebruary28,2024
Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.18614
November29,2023
DearEditor:
Weappreciatethefeedbackandcommentaryonour recentlypublishedstudy.First,wewouldliketo acknowledgethelimitationsofhavingasinglereviewer performingallchartreviews.Tomitigatepotentialbias, wecreatedobjectivebinarydefinitionsforeachNEXUS criterion.Forexample,weexplicitlydefineddistracting injuriestoincludeonlyradiographic findingsoflongbone fracturesormultipleribfractures,oranyinjuriesthat weredescribedexplicitlyas “distracting” intheclinical documentation.Similarly,alldocumentedneurological deficitswereassumedtofulfillthiscriterionunlesstheclinical documentationexplicitlyascribedthedeficittopain.While thesedefinitionsaremoreexplicitandobjectivethanthe originalNEXUScriteria,wefeltthismodificationwas necessaryduetotheretrospectivenatureofourstudy,asour methodologyinherentlyprohibitedusfromprospectively askingclinicianswhetheraninjurywasdistracting,for example.Furthermore,wefeltthatourdefinitionswere alignedascloselyaspossibletotheoriginalcriteriaand, therefore,providedanaccurateestimationofoverutilization. Ideally,futurestudiesonthetopicshoulduse multiplereviewersandprospectivelycollectdataonthe subjectivecriteria.
Secondly,weagreethatamoregranularanalysisof mechanismsofinjurycouldrevealadditionalassociations withoverutilization.Inouranalysis,noneofthemechanisms ofinjuryweresignificantlyassociatedwithCT overutilization,andseasonalmechanismsaccountedfora verysmallpercentage.Ourcategorizationofmechanismwas intentionallybroad,aswesoughttoidentifypotentially meaningfultargetsforfutureinterventionsaimedatreducing overutilization.Whileitispossiblethatasubsetof mechanismswithinagivencategorycouldbestatistically
significant,thiswouldrequirefurtherinvestigation,anditis notclearwhetherthose findingswouldyieldclinically importanttargetsforintervention.
Lastly,andmostimportantly,wewouldliketoemphasize theletterwriter’spointabout flowandtriageprocessesthat mayaffectoverutilizationofCTresources.Becausenoneof thesitesincludedinthisstudyuseaclinician-in-triage process,wewereunabletodirectlyassessthisassociation fromourdataset.However,strategiestoimproveemergency department(ED)throughput(eg,clinician-in-triagestaffing models,direct-to-CTprotocols,nurse-initiatedorders)have proliferatedacrossthecountryinrecentyears,despite thesestrategieshavinguncertainimpactsonresource overutilization.Aswedevelopandimplementnovel flowand triageprocesses,itisparamountthatweconsiderthe secondaryeffectsonhealthcarecosts,radiationexposure, incidental findings,andhospitalresources.Atatime whenEDcrowdingandboardinghavereachedcrisis levels,effectiveresourceutilizationisessentialfor operationalsuccess.
Thankyou.
KarlChamberlin,MD,MBA
AddressforCorrespondence:KarlChamberlin,MD,MBA,University ofMassachusettsChanMedicalSchool,DepartmentofEmergency Medicine,55LakeAvenueNorth,Worcester,MA01655.
Email: karl.chamberlin@umassmemorial.org
Copyright:©2024Chamberlin.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/
WesternJournal of EmergencyMedicineVolume25,No.2:March2024 302
CALIFORNIAACEPSECURED $200MANNUALLYTO INCREASEMEDI-CAL REIMBURSEMENTFOR EMERGENCYPHYSICIANS!
EMERGENCY MEDICINE IS THE ONLY SPECIALTY WITH ITS OWN BUDGET ALLOCATION.
@CALIFORNIAACEP www.californiaacep.org
Medi-cal Medi-cal rrateincrease ateincrease