Companion Article: Masic I, Miokovic M, Muhamedagic B. Evidence Based Medicine-New Approaches and Challenges. Acta Informatica Medica. 2008;16(4):219-225.
Presented by Marc Imhotep Cray,1 M.D.
Learning Objectives By the end of this presentation the learner should: Understand the background, history, definition and importance of evidence-based medicine. Know how to formulate clinically relevant, answerable questions using the Patient Intervention Comparison Outcome (PICO) framework. Be able to systematically perform a literature search to identify relevant evidence. Understand the importance of assessing the quality and validity of evidence by critically appraising the literature. Know that different study designs provide varying types and levels of evidence. Marc Imhotep Cray, M.D.
2
Scope of Evidence-Based Medicine Term "evidence-based medicine" has two main tributaries First is insistence on explicit evaluation of evidence of effectiveness when issuing clinical practice guidelines and other population-level policies Second is introduction of epidemiological methods into medical education and individual patient-level decision-making o
This tributary had its foundations in clinical epidemiology a discipline that teaches medical students and physicians how to apply clinical and Marc Imhotep Cray, M.D. epidemiological research studies to their practices
3
Background, history and definition
EBM methods were published to a broad physician audience in a series of 25 "Users’ Guides to the Medical Literature" published in JAMA between 1993 and 2000 by the Evidence-based Medicine Working Group at McMaster University. Graphic from: Guyatt G et al. (Eds). Users’ Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice 3rd Ed. New York: McGraw-Hill Education-JAMA Network, 2015.
What is Evidence-Based Medicine? Evidence-based medicine (practice) is a systematic process primarily aimed at improving care of patients EBM Triad includes:
Clinical Judgement
EBM
Relevant Scientific Evidence
Patients’ Values and Preferences Redrawn after: Sackett DL, et al. BMJ. 1996; (7023): 71-72.
5
What is EBM? (2) Sackett and colleagues describe evidence-based medicine (a.k.a. evidence-based practice[EBP]) as “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients� Sackett DL, et al. BMJ. 1996; (7023): 71-72.
Marc Imhotep Cray, M.D.
6
What is EBM? (3) “Evidence-based medicine (EBM) is the process of systematically reviewing, appraising and using clinical research findings to aid the delivery of optimum clinical care to patients” Rosenberg W, Donald A. Evidence based medicine: an approach to clinical problem-solving. BMJ 1995; 310: 1122–1126.
Marc Imhotep Cray, M.D.
7
What is EBM ? (4) Capsule EBM is an approach to medical practice intended to optimize decision-making by emphasizing use of evidence from welldesigned and well-conducted research Although all medicine based on science has some degree of empirical support EBM goes further classifying evidence by its scientific strength and requiring that only strongest types evidence (i.e., meta-analyses, systematic reviews, randomized controlled trials) can yield strong recommendations o weaker types of evidence (such as from case-control studies) can yield only weak recommendations
Marc Imhotep Cray, M.D.
8
History of EBM 1940s-Formal assessment of medical interventions using controlled trials well established 1972-Prof. Archie Cochrane, director of Medical Research Council Epidemiology Research Unit in Cardiff expressed what later came to be known as evidence-based medicine (EBM) in his book Effectiveness and Efficiency: Random Reflections on Health Services Late 1980s and early 1990s-EBM concepts were developed into a practical methodology by groups at Duke University in North Carolina (David Eddy) and McMaster University in Toronto (Gordon Guyatt and David Sackett) Marc Imhotep Cray, M.D.
9
History of EBM (2) 1992- UK government funded establishment of Cochrane Centre in Oxford objective was to facilitate preparation of systematic reviews of randomized controlled trials of healthcare 1993-Cochrane Centre expanded into an international collaboration of centres, of which (as of 2009) there were thirteen, whose role is to co-ordinate activities of 11,500 researchers NB: Cochrane Collaboration considered as one of critical factors in spreading concept of EBM worldwide
Marc Imhotep Cray, M.D.
10
Main elements of EBM EBM is part of multifaceted process of assuring clinical effectiveness main elements are: Production of evidence through research and scientific review Production and dissemination of evidence-based clinical guidelines Implementation of evidence-based, cost effective practice through education and management of change Evaluation of compliance with agreed practice guidance through clinical audit and outcomesfocused incentives Marc Imhotep Cray, M.D.
11
Key principles of EBM EBM involves a number of key principles discussed in turn during course of presentation: Formulate a clinically relevant question Identify relevant evidence Systematically review and appraise evidence identified Extract most useful results and determine whether they are important in your clinical practice Synthesize evidence to draw conclusions Use clinical research findings to generate guideline recommendations which enable clinicians to deliver optimal clinical care to patients Evaluate implementation of EBM Marc Imhotep Cray, M.D.
12
Core of Evidence-Based Medicine At core of EBM is a care and respect for patients who will suffer if clinicians fall prey to muddled clinical reasoning and to neglect or misunderstanding of research findings Practitioners of EBM strive for a clear & comprehensive understanding of evidence underlying their clinical care and work w each pt. to ensure that chosen courses of action are in that pt’s best interest Practicing EBM requires clinicians to understand how uncertainty about clinical research evidence intersects w an individual pt’s predicament, values & preferences Marc Imhotep Cray, M.D.
13
Logic behind EBM To make EBM more acceptable to clinicians and to encourage its use best to turn a specified problem into answerable questions by examining:
Person or population in question Intervention given Comparison (if appropriate) Outcomes considered
Next, it is necessary to refine problem into explicit questions then check to see whether evidence exists Marc Imhotep Cray, M.D.
But where can we find information to help us make better decisions? (answers to follow) 14
What is involved in identification, appraisal and application of evidence summarized in reviews?
Framing questions ↓ Identifying relevant reviews ↓ Assessing quality of review and its evidence ↓ Summarizing the evidence ↓ Interpreting finding Marc Imhotep Cray, M.D.
15
https://www.healthcatalyst.com/5-reasons-practice-evidence-based-medicine-is-hot-topic 16
Where can we find information? Common sources include:
Personal experience for example, a bad drug reaction Reasoning and intuition Colleagues
Published evidence o meta-analyses, systematic reviews and randomized controlled trials NB: By becoming educated in strength of published evidence (and critical appraisal ), in contrast to more traditional--less rigorous--sources of information use of ineffective, costly or potentially hazardous interventions can be reduced Marc Imhotep Cray, M.D.
17
Formulating Clinical Questions In order to practice evidence-based medicine initial step = converting a clinical encounter into a clinical question A useful approach to formatting a clinical (or research) question Patient Intervention Comparison Outcome (PICO) framework
Marc Imhotep Cray, M.D.
18
Formulating Clinical Questions (2) Patient Intervention Comparison Outcome (PICO) framework Question is divided into four key components: 1. Patient/Population: Which pts. or popul. group of pts. are you interested in?  Is it necessary to consider any subgroups?
2. Intervention: Which intervention/treatment is being evaluated? 3. Comparison/Control: What is/are main alternative/s compared to intervention? 4. Outcome: What is most important outcome for patient?  Outcomes can include short- or long-term measures, intervention complications, social functioning or quality of life, morbidity, mortality or costs Marc Imhotep Cray, M.D.
19
PICO Framework illustrated Clinical Encounter Ali, 30 years old, was diagnosed with heart failure at 4 years old and prescribed a beta-blocker which dramatically improved his symptoms. Ali’s 5- year-old daughter, Leda, has been recently diagnosed with chronic symptomatic CHF. Ali asks you, whether his daughter should also be prescribed a beta-blocker. Question: Is there a role for beta-blockers in the management of heart failure in children? Patient Intervention Comparison Outcome Marc Imhotep Cray, M.D.
Children with congestive heart failure Carvedilol (a β-blockers ) No carvedilol Improvement of CHF symptoms 20
Formulating Clinical Questions (4) Types of research questions (=Tx/ Etio/ Dx/ Px) Not all research questions ask whether an intervention is better than existing interventions or no Tx at all From a clinical perspective EBM is relevant for three other key domains: 1. 2.
Etiology: Is exposure a risk factor for developing a certain condition? Diagnosis: How good is diagnostic test (history taking, physical examination, laboratory or pathological tests and imaging) in determining if a pt. has a particular condition?
3.
Questions usually asked about clinical value or diagnostic accuracy of test
Prognosis: Are there factors related to pt. that predict a particular outcome (disease progression, survival time after Dx of disease, etc.)?
Marc Imhotep Cray, M.D.
Px is based on characteristics of pt. (“prognostic factors”) 21
Formulating Clinical Questions (5) Important that pt. experience is taken into account when formulating clinical question (p)atient experience may vary depending on which pt. population is being addressed Following pt. views should be determined: o Acceptability of proposed (i)ntervention being evaluated o Preferences for Tx options already available (c) o What constitutes an appropriate, desired or acceptable (o)utcome NB: Incorporating above pt. views will ensure clinical question is patient-centered & thus, clinically relevant Marc Imhotep Cray, M.D.
22
Identifying Relevant Evidence Three Ways to Use the Medical Literature
Staying Alert to Important New Evidence Problem Solving Asking Background & Foreground Questions
Analyzing information In using evidence it is necessary to: o o o o o o o Marc Imhotep Cray, M.D.
Search for and locate it Appraise it Interpret it in context Implement it Store and retrieve it Ensure it is updated Communicate it 23
Ways to Use Medical Literature Medical student, in early training, seeing a patient with newly diagnosed type 2 diabetes mellitus She will ask questions such as:
What is type 2 diabetes mellitus? Why does this patient have polyuria? Why does this pt. have numbness & pain in his legs? What treatment options are available?
These questions address normal physiology and pathophysiology assoc. w a medical condition Traditional medical textbooks that describe underlying pathophysiology or epidemiology of a disorder provide an excellent resource for addressing these background questions Marc Imhotep Cray, M.D.
24
Ways to Use Medical Literature (2) …In contrast, sorts of foreground questions that experienced clinicians usually ask require different resources, namely using current medical literature for pt.-related problem solving Formulating a question is first step and critical skill for this evidence-based practice (EBP) Ways to use medical literature that follow provide an opportunity to start learning & practicing the skill Marc Imhotep Cray, M.D.
25
Ways to Use Medical Literature (3) “Clinicians do Problem Solving” Experienced clinicians managing a pt. w T2DM will ask questions such as: In pts w new onset T2DM, which clinical features or test results predict development of diabetic complications? In pts with T2DM requiring drug therapy, does starting w metformin Tx yield improved diabetes control and reduce long-term complications better than other initial treatments? NB: Here, clinicians are defining specific questions raised in caring for pts then consulting the medical literature to resolve these questions Marc Imhotep Cray, M.D.
26
Ways to Use Medical Literature (3) Most valuable single free access point is The Cochrane Library The Cochrane Library contains high-quality, independent evidence to inform all healthcare decision-making
An alternative to alerting systems are secondary evidence based journals
For example, in internal and general medicine, ACP Journal Club (http://acpjc.acponline.org) publishes synopses of articles that meet criteria of both high clinical relevance and methodologic quality
See: Haynes RB, Cotoi C, Holland J, et al; McMaster Premium Literature Service (PLUS) Project. Second-order peer review of the medical literature for clinical practitioners. JAMA. 2006;295(15):1801-1808 Marc Imhotep Cray, M.D.
27
Ways to Use Medical Literature (4) ď ąMost efficient strategy for ensuring you are aware of recent developments relevant to your practice is to subscribe to e-mail alerting systems, such as EvidenceAlerts  A free service w research staff screening approx. 45, 000 articles per year in more than 125 clinical journals for methodologic quality and a worldwide panel of practicing physicians rating them for clinical relevance and newsworthiness
Marc Imhotep Cray, M.D.
28
Asking Background & Foreground Questions One can think of first set of questions, those of medical student, as background questions and of browsing and problem-solving sets as foreground questions In most situations you need to understand background thoroughly before it makes sense to address foreground issues Guyatt G et al. (Eds). Users’ Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice 3rd Ed. New York: McGraw-Hill Education-JAMA Network, 2015. 29
Five Types of Foreground Clinical Questions In addition to clarifying population, intervention or exposure, and outcome, it is productive to label nature of question that you are asking Finding a Suitably Designed Study for Your Question Type You need to correctly identify category of study b/c to answer your question, you must find an appropriately designed study o For example, if you look for a randomized trial to inform properties of a diagnostic test, you will not find answer you seek Marc Imhotep Cray, M.D.
30
There are 5 fundamental types of clinical questions 1. Therapy: determining effect of interventions on patient important outcomes (symptoms, function, morbidity, mortality, and costs) 2. Harm: ascertaining effects of potentially harmful agents (including therapies from first type of question) on patient-important outcomes 3. Differential diagnosis: in patients with a particular clinical presentation, establishing the frequency of the underlying disorders 4. Diagnosis: establishing power of a test to differentiate Betw. those with and without a target condition or disease 5. Prognosis: estimating a patient’s future course ď ś We will now review study designs associated with 5 major types of questions. Marc Imhotep Cray, M.D.
31
Structure of Randomized Trials To answer questions about a therapeutic issue, we seek a randomized trial (group assignment analogous to flipping a coin) Once investigators allocate participants to treatment or control groups they follow them forward in time to determine whether they have, for instance, a stroke or myocardial infarction what we call outcome of interest
Guyatt G et al. (Eds). Users’ Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice 3rd Ed. New York: McGraw-Hill Education-JAMA Network, 2015.
Marc Imhotep Cray, M.D.
32
Structure of Observational Cohort Studies Ideally, we would also look to randomized trials to address issues of harm For most potentially harmful exposures, however, randomly allocating patients is neither practical nor ethical o For example , one cannot suggest to potential study participants that an investigator will decide by the flip of a coin whether or not they smoke during next 20 years
For exposures such as smoking, best one can do is identify observational studies (subclassified as cohort or case-control studies) provide less trustworthy evidence than randomized trials
Guyatt G et al. (Eds). Users’ Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice 3rd Ed. New York: McGraw-Hill Education-JAMA Network, 2015.
33
Structure of Studies of Differential Diagnosis For sorting out differential diagnosis investigators collect a group of patients with a similar presentation (eg, painless jaundice, syncope, or headache), conduct an extensive battery of tests, and if necessary follow patients forward in time
Ultimately, for each pt. investigators hope to establish underlying cause of symptoms and signs with which pt. presented
Guyatt G et al. (Eds). Users’ Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice 3rd Ed. New York: McGraw-Hill Education-JAMA Network, 2015.
Marc Imhotep Cray, M.D.
34
Structure of Studies of Diagnostic Test Properties In diagnostic test studies, investigators identify a group of patients among whom they suspect a disease or condition of interest exists (such as tuberculosis, lung cancer, or iron deficiency anemia) which we call the target condition
Pts. undergo new diagnostic test and a reference standard (also referred to as gold standard or criterion standard) Investigators evaluate diagnostic test by comparing its classification of pts. w that of reference standard
Guyatt G et al. (Eds). Users’ Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice 3rd Ed. Cray, New M.D. York: McGraw-Hill Education-JAMA Network, 2015. Marc Imhotep
35
Structure of Studies of Prognosis Final type of study examines a patient’s prognosis and may identify factors that modify that prognosis Here, investigators identify pts who belong to a particular group (such as pregnant women, pts. undergoing surgery, or pts w cancer) with or without factors that may modify their prognosis (such as age or comorbidity) The exposure here is time investigators follow up pts to determine if they experience the target outcome such as an adverse obstetric or neonatal event at end of a pregnancy, a myocardial infarction after surgery, or survival in cancer
Guyatt G et al. (Eds). Users’ Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice 3rd Ed. New York: McGraw-Hill Education-JAMA Network, 2015.
36
Finding Current Best Evidence Searching for Evidence is a Clinical Skill
Searching for current best evidence in medical literature has become a central skill in clinical practice On average, clinicians have 5 to 8 questions about individual patients per daily shift and regularly use online evidence-based medicine (EBM) resources to answer them
See: Chapter 4, Finding Current Best Evidence. In: Guyatt G et al. (Eds). Users’ Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice 3rd Ed. New York: McGraw-HillJAMA Network, 2015.
Marc Imhotep Cray, M.D.
37
Categories of EBM Resources Summaries and guidelines Nonpreappraised research
UpToDate DynaMed Clinical Evidence Best Practice US National Guidelines Clearinghouse
Preappraised research
ACP Journal Club McMaster PLUS DARE Cochrane Evidence Updates
PubMed (MEDLINE) CINAHL CENTRAL Filters: Clinical Queries in PubMed
Federated searches (engines)
ACCESSSS Trip SumSearch Epistimonikos
Abbreviations: ACCESSSS, ACCess to Evidence-based Summaries, Synopses, Systematic Reviews and Studies; CENTRAL, Cochrane Central Register of Controlled Trials; CINAHL, Cumulative Index to Nursing and Allied Health Literature; DARE, Database of Abstracts of Reviews of Effects. Marc Imhotep Cray, M.D.
38
Free EBM alerting system
http://www.cochranelibrary.com/
Marc Imhotep Cray, M.D.
39
Databases included in The Cochrane Library
Belsey J. What is evidence-based medicine? Hayward Medical Communications, 2009. Marc Imhotep Cray, M.D.
40
Free EBM alerting system
https://plus.mcmaster.ca/EvidenceAlerts/ Marc Imhotep Cray, M.D.
41
Free Medical Literature Research Portal
https://www.nlm.nih.gov/bsd/pmresources.html Marc Imhotep Cray, M.D.
42
Peer-Reviewed Publications, News, Alerts and CME
http://www.medscape.com/
Marc Imhotep Cray, M.D.
http://jama.jamanetwork.com/journal.aspx
43
Critically Appraising the Evidence Once all possible studies have been identified w literature search each study needs to be assessed for eligibility against objective criteria for inclusion or exclusion
Having identified those studies that meet inclusion criteria they are subsequently assessed for methodological quality using a critical appraisal framework
Despite satisfying inclusion criteria, studies appraised as being poor in quality should also be excluded
Marc Imhotep Cray, M.D.
44
Critical appraisal (2) ď ąCritical appraisal is process of systematically examining available evidence to judge its validity, and relevance in a particular context ď ąAppraiser should make an objective assessment of study quality and potential for bias Note: Methodological checklists for critically appraising study designs will be covered in a subsequent lecture
Marc Imhotep Cray, M.D.
45
Critical appraisal (3) Important to determine both internal validity and external validity of study: External validity: extent to which study findings are generalizable beyond limits of study to study’s target population. Internal validity: Ensuring that study was run carefully (research design, how variables were measured, etc.) and extent to which observed effect(s) were produced solely by intervention being assessed (and not by another factor) Three main threats to internal validity (confounding, bias and causality) for each of the key study designs are discussed in subsequent lectures Marc Imhotep Cray, M.D.
46
Levels of Evidence Table shows study designs that answer treatment questions  Further down list (or levels of evidence), greater risk of bias  Randomizing participants in a study reduces bias, b/c confounding factors (such as age, gender, smoking status, etc.) are evenly distributed betw. intervention and control arms of study
Kulkarni K. et al. (Eds.). Oxford Handbook of Key Clinical Evidence,2nd Ed. New York: Oxford University Press, 2016. Marc Imhotep Cray, M.D.
47
GRADE, A “Quality of evidence” framework Quality of evidence framework by Grading of Recommendations, Assessment, Development and Evaluation (GRADE) Working Group is very useful in appraising medical research studies Although developed mainly to help guideline developers make evidence-based recommendations its approach to assessing quality of evidence is widely used and makes important distinction betw. evidence quality and strength of a recommendation Helps to point out importance of looking at “body of evidence” for a clinical question
Marc Imhotep Cray, M.D.
48
A summary of GRADE’s approach to rating quality of evidence
Kulkarni K. et al. (Eds.). Oxford Handbook of Key Clinical Evidence,2nd Ed. New York: Oxford University Press, 2016. Marc Imhotep Cray, M.D.
49
https://www.healthcatalyst.com/5-reasons-practice-evidence-based-medicine-is-hot-topic 50
Evaluating the Literature: Capsule ‌reviewing medical literature poses a challenge to busy physicians. A willingness and ability to do so enhance quality of practice they bring to each of their patients. To save time, a brief primary survey of article of interest informs reader as to potential value of findings and to whether a more in-depth review is indicated. If so, this detailed analysis (secondary survey) allows reader to determine whether article's conclusion is supported by its results and whether these results are believable. Knowledge of standard anatomy of an article and idiosyncrasies of various types of studies will assist reader to intelligently review medical literature efficiently‌ Theodore J Gaeta et al. Evaluating the Literature. Available at http://emedicine.medscape.com/article/773527 Accessed 07-08-17 Marc Imhotep Cray, M.D.
51
Conclusion: Using the Medical Literature Provides for Optimal Patient Care.
Guyatt G et al. (Eds). Users’ Guides to the Medical Literature: Essentials of EvidenceBased Clinical Practice 3rd Ed. New York: McGraw-Hill Education-JAMA Network, 2015. Marc Imhotep Cray, M.D.
52
See next slide for links to tools and resources for further study.
53
Sources and further study: Textbooks Guyatt G et al. (Eds). Users’ Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice 3rd Ed. New York: McGraw-Hill Education-JAMA Network, 2015. Kaura A. Evidence-Based Medicine: Reading and Writing Medical Papers (Crash Course Series). Philadelphia: Mosby- Elsevier, 2012. Kulkarni K. et al. (Eds.). Oxford Handbook of Key Clinical Evidence, 2nd Ed. New York: Oxford University Press, 2016. Swiger KJ et al. (Eds). 50 studies every internist should know. New York: Oxford University Press, 2015. Cloud Folders EBM (Evidence Based Medicine), Reading the Medical Literature and Medical Writing Marc Imhotep Cray, M.D.
54
External Links Lefebvre, C., Manheimer, E., Glanville, J., 2011. Searching for studies. In: Higgins, J.P.T., Green, S. (Eds.), Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 (updated March 2011). The Cochrane Collaboration. National Institute for Health and Clinical Excellence, March 2012. The Guidelines Manual. National Institute for Health and Clinical Excellence, London. Available from: http:// www.nice.org.uk Sackett, D.L., Rosenberg, W.M.C., 1995. The need for evidence based medicine. J. R. Soc. Med. 88, 620–624. Sackett, D.L., Rosenberg, W.M.C., Gray, J.A.M., Haynes, R.B., Richardson, W.S., 1996. Evidence based medicine: What it is and what it isn’t. BMJ 312, 71–72.
Straus, S.E., McAlister, F.A., 2000. Evidence-based medicine: A commentary on common criticisms. CMAJ 163, 837–841.
Marc Imhotep Cray, M.D.
55