ABSTRACT #31

TRANSITION FROM TIERED TO COMPETENCY-BASED GRADING IN THE OBSTETRICS AND GYNECOLOGY CLERKSHIP

Cynthia Abraham

PURPOSE AND GOALS: Traditionally, undergraduate medical education grades have been based on a tiered system. However, tier-based grading lends itself to comparison of medical students to the peers they are rotating with during the clerkship and a potentially stressful learning environment. This is in contrast to graduate medical eduation in which learners are evaluated against a set of requisite competencies, when combined with knowledge, skills and attitudes, that would prepare a physician for independent practice. In light of this current paradigm, initiatives have been constructed to have undergraduate medical education to move towards a competency-based evaluation system. In July 2022, the Icahn School of Medicine at Mount Sinai (ISMMS) transitioned from a tiered grading scheme to a competency-based one. Hence, the purpose of this study was to assess Obstetrics and Gynecology core clerkship grades and National Board of Medical Examiners (NBME) Shelf Exam scores after transitioning from tiered to competency-based grading.

METHODS: NBME scores and grade distribution between the first three quarters of academic year (AY) 2021-22 and the first three of AY 2022-2023 were compared.

EVALUATION PLAN: Between July 2021 and July 2022, the Icahn School of Medicine at Mount Sinai (ISMMS) adhered to tiered grading; 60 percent received Honors, 30% High Pass, 10% Pass. In July 2022, ISMMS transitioned to competency-based grading. Criteria delineating a score of “Pass” versus “Honors” for each competency were created. In the tier-based grading scheme, the following items were used for determining the final grade for the clerkship: clinical evaluations, oral and written case presentations, completion of clinical skills assessment card, NBME Shelf Exam score and direct observations exam performance. In the competency-based scheme, six objectives were created. Criteria to delineate achieving a score of “Pass” versus “Honors” for each specific objective were created.

Students needed to meet criteria for Honors for four out of six of the objectives in order to ultimately receive a grade of Honors for the clerkship.

SUMMARY OF RESULTS: Number of students studied in AY 2021-2022 and AY 2022-2023 were 55 and 52, respectively. A significantly higher percentage of students received Honors in AY 2022-2023 than in AY 2021-2022 (96% versus 60%, p < 0.01). Mean NBME scores were significantly higher for those receiving Honors in AY 2021-2022 than in AY 2022-2023 (p < 0.05); scores for AY 2021-2022 and AY 2022-2023 were 78.9, 95% CI 76.3, 80.9 and 75.1, 95% CI 73.3, 76.9, respectively. Mean NBME scores for all students were not significantly different between the two academic cohorts (77.0 versus 74.9, p = 0.11).

REFLECTIVE CRITIQUE: These findings support a model that compares learner performance to predefined measures as opposed to peer performance. Data collection is ongoing. Future directions include assessing student satisfaction ratings for the two schemes.