SAEM Fellowship Program Guidebook

From the Ground Up: Creating and Leading Fellowship Programs in Emergency Medicine

Fellowship Program Guide

Kern’s Model for Curriculum Development

General Overview of Kern’s Model

Accrediting bodies for undergraduate, graduate, and continuing medical education generally require a formal curriculum with explicitly stated goals, objectives, and evaluation strategies� David Kern, MD, MPH, developed a six-step systematic approach for medical educators that gives future fellowship directors the tools needed to clearly articulate educational goals and meet the needs of their learners, patients, and society�12 When developing or refining a fellowship curriculum, regardless of the field, Kern’s six-step model can serve as a useful road map� Over the course of this fellowship guide, the authors discuss Kern’s six-step model of curriculum development and its application in the construction, implementation, evaluation, and dissemination of a novel fellowship curriculum�

Step 1 Problem Identification and General Needs Assessment

The first stage in Kern’s six-step model is problem identification and a general needs assessment� The essential aim of a curriculum in medical education is to prepare learners to address a problem that affects the health of a specific population� To accurately identify the problem, one should consider two guiding questions: whom does the problem affect? And what does the problem affect? To answer the first question, one should examine a variety of

factors including the impact on patients, health care professionals, medical educators, and society�12 Secondly, how does the problem affect clinical outcomes, quality of life and health care, resource utilization, and societal function? For example, a toxicology fellowship may help improve the care of poisoned patients

Once a problem has been adequately defined, one can begin the general needs assessment� The purpose of the general needs assessment is to determine the gaps between current and desired or ideal conditions�12 When starting a new fellowship, it is essential to investigate other local fellowships as well as similar fellowships at other institutions�

For example, when implementing a global health fellowship, begin with a systematic review of all information available about existing global health fellowships� The information can be obtained from medical literature, accreditation organizations, professional organizations, as well as consultations with experts in the field� After completing a thorough review of the current body of knowledge, gather new information to gain a better understanding of what should be included in the ideal global health fellowship� Information gathering and data collection can be obtained by several methods such as the use of surveys, focus groups, and the Delphi method to obtain consensus on best practices for the fellowship’s curriculum� In this example, the goal of the general needs assessment is to understand the differences between current practices and desired practices for global health fellowships�

Step 2 Targeted Needs Assessment

A targeted needs assessment is the process of applying knowledge learned from the general needs assessment to obtain new information about the targeted learners and their environment�12 In the development of a new fellowship program, targeted learners will be the ideal future fellows� In addition, information can be gathered from other local sources of information to include core faculty, fellowship directors, departmental experts, and national content experts�

After identifying who the ideal learner is, the next step is to identify the needs of the ideal learner� It is helpful to begin with the end in mind to understand the scope of knowledge a fellow is expected to attain after completion of fellowship� For example, what position should they be able to fill after completing fellowship? To identify the learners needs, explore their prior experiences and learning strategies, perceived and measured deficiencies, attitudes, and motivations�

In addition to understanding the ideal learner, it is critical to identify the stakeholders involved in the learning environment to ensure the fellowship’s success� Additional stakeholders may include the chair, core faculty, departments, the sponsoring institution, and accrediting bodies (if applicable)�12 Targeted needs assessment should also include financial support, faculty support, and additional resources from the department�

Once the targeted learners and environment have been established, it is essential to gather information to focus one’s curricular efforts� Different methods can be applied including focus groups, surveys, formal interviews, and direct observation�14 Before deciding on a method of assessment, it is essential to review three guiding questions:

1 What standards of representativeness, validity, and accuracy will be required?

2� Will subjective or objective measures be used?

3� Will quantitative or qualitative data be preferable?13

In summary, the targeted needs assessment focuses curricular efforts on the ideal learner and environment to further elicit the resources and stakeholders required to build a successful curriculum

Step 3 Goals and Objectives

A goal is a broad, generalized overview of the direction a specific program or course plans to take� In contrast, learning objectives should be detailed aspects of goals that outline the process leading to desired outcome� Using the SMART framework, a learning objective should be specific, measurable, attainable, result-oriented, and time bound�15 The development of goals and objectives are vital in creating a fellowship curriculum as it guides curricular content and learning methods, and focuses the learner’s expectations for evaluation�

Learning objectives and goals can also facilitate distribution of important resources� Learning objectives should be evaluated routinely and refined based on Step 6, Evaluation and Feedback�

There are five basic elements to consider when developing learning objectives: time frame, the learner, the action verb, the degree, and assessment� Bloom’s taxonomy provides a useful theoretical framework in creating meaningful learning objectives for higher level learning�16,17 Objectives can be classified into three major categories including cognitive, affective, and psychomotor�12 When developing goals and learning objectives for a new fellowship program, it is important to create specific, measurable objectives that align with generalized goals of the program� An example illustrating the difference between goals and objectives is detailed below�

GOAL: By the end of this Medical Education Fellowship the learner should become a valued member of a Residency Program Leadership Team�

OBJECTIVES: By the end of Medical Education Fellowship the learner will be able to:

1� Develop and disseminate educational research as assessed by the number of presentations accepted at national conferences�

2� Design innovative conference didactic sessions as assessed by post lecture resident feedback�

3� Create longitudinal residency curriculum to address gaps in resident knowledge as assessed by exit interviews�

Step 4 Educational Strategies

After developing clear goals and measurable learning objectives, educational methods are selected that will most likely achieve the educational objectives and curricula goals� Prior to developing educational strategies for fellows, it is important to briefly review the needs of the adult learner�

Simulation

Lectures

Small Group Discussions

Advanced Degree Coursework

Leadership Experience Teaching

Flipped Classroom

Curriculum Development

Self-Directed Learning Scholarship

Understanding the Adult Learner

The adult learner is self-directed, intrinsically motivated, and uses life experiences to build an educational foundation�18 Malcom Knowles first described self-directed learning as a process where individuals take the initiative to identify their learning needs, formulate learning goals, identify resources, select and implement learning strategies, and monitor and evaluate their learning�19 As medical educators, it is important to find the right balance in guiding fellows to become self-directed learners For example, a director of an administrative fellowship may decide to make only 70% of the curriculum mandatory, allowing their fellow to select relevant areas of interest for the remaining 30%�

Determination of Content

The goal of curating content is to have the appropriate amount of material to achieve the desired objectives and outcomes� Constructing a syllabus for the curriculum will not only help the learner stay focused but also allow for learners to understand how they will be evaluated�12 The syllabus should include:

1� An explicit statement with learning objectives and methods�

2� A schedule of the curriculum events and practical information�

3� Curricular resources�

4� Plan for assessment�

Choice of Educational Methods

When determining educational methods, it is important to maintain congruence between objectives and the content� An engaging curriculum will utilize various educational methods that will be feasible in regard to time and resources� Examples of educational methods are provided in Table 1�

Educational Technology

To vary educational methods, implement educational technology including high fidelity simulation, free open access media, podcasts, and web-based learning modules� For example, an EMS fellow may benefit from medical control simulated cases or a mass casualty drill�

Table 1. Examples of Educational Strategies and Methods

Step 5 Implementation

During this step, the necessary actions are taken to make the fellowship a reality� This includes obtaining support, trialing the curriculum, initiating the program, and the dynamic process of constantly evaluating and refining the program�

Obtaining Support

Obtaining institutional and departmental support for the fellowship program is a crucial step during implementation� The initial step includes acquiring support and buy-in from additional faculty members in the subspecialty and the administrative team in the department� Once support has been established within the department, the fellowship program can be presented to the graduate medical education (GME) office and hospital leadership� A new program may be required to seek approval from additional departments if the training program impacts multiple departments�

Identifying and Procuring Resources

One potential barrier to long-term success of a fellowship is sustainable funding� Local, institutional practices generally determine funding sources and mechanisms� Funding for ACGME-accredited fellowship programs typically falls to the sponsoring institution, whereas non-ACGME fellowship program funding is complex, derived from departments or divisions, grants, or even partnership with public entities or private industry� This presents both additional opportunities and challenges�

Strategies for funding of non-ACGME accredited EM fellowships appear to be highly variable across institutions� No single, definitive best practice was identified, however some common themes did emerge in our exploration� In general, to support long-term viability, fellowship programs should at least be budget neutral Expenses to be considered include: fellow(s) salary and benefits, fellowship director(s) stipend and/or reduction in clinical commitment, fellow malpractice coverage if practicing clinically as part of the program, institutional overhead expenses calculated and allocated per local practice, fellow CME/travel expenses, fellow membership in relevant national/regional organizations and other resources to support fellow education and scholarly activity such as online access to journals, statistical analysis tools, etc�

In the majority of EM fellowship programs interviewed, funding to support the fellowship primarily derived from revenue generated by fellows’ practice in clinical settings affiliated with the sponsoring institution� In a common model, fellows are paid a trainee salary based on local post-graduate year norms for a predetermined clinical commitment� The difference between a faculty salary and a post-graduate trainee salary for that clinical commitment served to support the cost incurred for the fellows’ training� In many programs, fellows are given the opportunity to practice clinically beyond the base commitment in order to increase their overall salary� The additional clinical time appeared most often to be compensated at rates similar to junior faculty, although some models compensated fellows at a lower rate than faculty�

Alternative sources of funding were uncovered in our exploration, but they were rare� In a few cases, programs and/or fellows were able to secure scholarships to partially support salary or other fellowship related expenses� In general, these appeared to be one-time opportunities or of limited duration� Philanthropy represents another potential alternative source of revenue� One EM fellowship program reported securing an endowment sufficient enough to support a portion of fellowship-related expenses in perpetuity� If the fellowship can provide a paid service or partner with a business or non-governmental organization, it may be possible to generate funds to offset costs associated with the fellowship� This method is rare and can be complicated� It may require additional oversight, institutional buyin, and approval�

Pursuit of additional degrees (e�g� Master’s, PhD) as part of a fellowship program adds further complexity to the funding modeling for fellowship programs Funding for this portion of training appears to vary to an even greater degree among programs� Several EM fellowships report partnering with local institutional programs to arrange for a reduced tuition� Fellows may be required to work additional clinical shifts to offset the cost of an advanced degree� During investigation, rare cases of waived tuition were found due to established institutional policies or as a result of collaboration between the department and the program supporting the degree program�

Identifying and Addressing Barriers to Implementation

What are the barriers to implementing the fellowship? Potential barriers to consider are buyin from leadership, personnel including administrative, buy-down for faculty time spent on education, space for fellowship activities including both clinical and educational, equipment, potential impact on other training programs, and funding�

Piloting the Curriculum

One step in the implementation phase that may identify additional barriers is trialing the curriculum via a pilot program� For example: for a medical education fellowship, the curriculum may be trialed via a medical education elective for residents� This provides an opportunity to elicit feedback� Once the curriculum and infrastructure for the fellowship is in place, the program is ready for the first fellow�

Step 6 Evaluation and Feedback

Evaluation and feedback should be occurring regularly as part of the interconnected model� However, at this stage, a critical question in the development and implementation of a fellowship curriculum can be answered: were the desired goals and objectives met?12 Feedback should be collected from the learners, core faculty, and other stakeholders about both the individual’s and the curriculum’s performance� Various methods of collecting

feedback exist, including surveys, interviews, clinical metrics, and coaching meetings This information should be synthesized to modify and continuously improve the fellowship curriculum�

Program evaluation is the assessment of the performance of the individual and curriculum� Assessments can be both formative, allowing for ongoing feedback for learners and curriculum, as well as summative, allowing for a final grade of the performance of the learner and curriculum�20 The evaluation process includes the identification of users, resources, and ethical concerns�12 After the identification process, measurement methods can be designed to collect data� The analyzed data should be reported to important stakeholders involved in the fellowship process� This will allow for continued support and allotment of resources to support the fellowship program�

References

1 Nelson LS, Keim SM, Beeson MS, Chudnofsky CR, Gausche-Hill M, Gorgas DL, et al American Board of Emergency Medicine report on residency and fellowship training information (2018-2019) Annals of emergency medicine 2019 May 1;73(5):524-41

2 Kellermann A Are you considering an academic career? Emergency Medicine Residents’ Association, 2021 (Accessed on April 7, 2021)

3� Yarris L, Coates W, Lin M, Lind K, Jordan J, Clarke S, et al� A Suggested Core Content for Education Scholarship Fellowships in Emergency Medicine Academic Emergency Medicine 2012;19(12):1425-1433

4� Yarris L, Coates W� Creating Educational Leaders: Experiences with Two Education Fellowships in Emergency Medicine� Academic Emergency Medicine 2012;19(12):1481-1485�

5� Bayram J, Rosborough S, Bartels S, Lis J, VanRooyen M, Kapur B, et al� Core Curricular Elements for Fellowship Training in International Emergency Medicine� Academic Emergency Medicine 2010;17(7):748-757�

6 Klesick E, Hakmeh W Global Emergency Medicine Fellowships: Survey of Curricula and Pre-Fellowship Experiences Western Journal of Emergency Medicine 2021;22(1)�

7 Scott, B Administrative fellowships in emergency medicine Physician Executive, 1995; 21(8), 35+

8 Frallicciardi A, Vora S, Bentley S, Nadir N, Cassara M, Hart D, et al Development of an Emergency Medicine Simulation Fellowship Consensus Curriculum: Initiative of the Society for Academic Emergency Medicine Simulation Academy Academic Emergency Medicine 2016;23(9):1054-1060

9 Jahn H, Kwan J, O’Reilly G, Geduld H, Douglass K, Tenner A, et al Towards developing a consensus assessment framework for global emergency medicine fellowships BMC Emergency Medicine 2019;19(1)

10 Petinaux B, Douglass K, Lee J, Liu Y, Boniface K, Clancy C, et al Emergency Medicine Joint Fellowship Curriculum The Journal of Emergency Medicine 2012;43(2):351-355

11 Weiss P, Myers A, McGann K, Mason K, Kesselheim J, Fleming G, et al Funding Sources and Perceived Financial Insecurity in Pediatric Subspecialty Fellowship Programs Academic Pediatrics 2019;19(7):815-821

12 Thomas P, Kern M, Hughes M, Chen B Curriculum Development for Medical Education 3rd ed Baltimore, MD: Johns Hopkins University Press, 2016

13 SAEM Committee Descriptions Society for Academic Emergency Medicine, 2020 (Accessed February 20, 2021 at https://www saem org/ about-saem/saem-membership/committees/committee-descriptions)

14 Ratnapalan S, Hilliard R Needs Assessment in Postgraduate Medical Education: A Review Medical Education Online 2002;7(1):4542

15� Chatterjee D, Corral J� How to Write Well-Defined Learning Objectives� Journal of Education in Perioperative Medicine 2017;19(4)�

16 Bloom B, Engelhart M, Furst E, Hill W, Krathwohl D Taxonomy of educational objectives London: Longman; 1956

17 Anderson L, Krathwohl D A Taxonomy for Learning, Teaching, and Assessing Longman: New York; 2001

18 Knowles M The Adult Learner: A Neglected Species Houston, TX Gulf Publishing Company, 1973

19 Knowles, M Self-Directed Learning A guide for learners and teachers Englewood Cliffs: Prentice Hall/Cambridge, 1975

20� Pangaro L, ten Cate O� Frameworks for learner assessment in medicine: AMEE Guide No� 78� Medical Teacher 2013;35(6):e1197-e1210�

21 What We Do Accreditation Council for Graduate Medical Education, 2021 (Accessed April 7, 2021 at https://www acgme org/What-WeDo/Overview/Accrediation)

22 SAEM Fellowship Approval Program Society for Academic Emergency Medicine, 2021 (Accessed April 7, 2021 at https://www saem org/ resources/services/fellowship-approval-program)

23 SAEM Fellowship Directory Society for Academic Emergency Medicine, 2021 (Accessed April 7, 2021 at https://member saem org/ SAEMIMIS/SAEM_Directories/Fellowship_Directory/SAEM_Directories/P/FellowshipMap aspx?hkey=573c442c-03ee-4c33-934a5fda0b280835)

24 EMRA Match Emergency Medicine Residents’ Association, 2021 (Accessed April 7, 2021 at https://webapps acep org/utils/spa/match#/ search/map)

25 SAEM Residency and Fellowship Fair Society for Academic Emergency Medicine, 2021 (Accessed April 7, 2021 at https://www saem org/ annual-meeting/events/saem-rams-residency-fellowship-fair)