Jean Slutsky

An AHRQ Perspective on CER and Methodology Considerations Jean Slutsky Director, Center for Outcomes and Evidence New York City August 12, 2010

How Can We Make Better Healthcare Decisions?  Improve research base – Relevance – Quality

 Improve use of data in decision making

The Role of Clinical Decision Makers in CER CER +

=

Researcher looking through lens of decision maker to do research

EBM =

+

Decision maker looking through lens of research to make decisions

Comparing Evidence: Medical vs. Semiconductor Research “When I was doing semiconductor device research, it was expected that I would compare my results with other people's previously published results and that I would comment on any differences. But it seemed to be different in medicine.

“Medical practitioners primarily tended to publish their own data; they often didn't compare their data with the data of other practitioners, even in their own field, let alone with the results of other types of treatments for the same condition.”

Intel co-founder and prostate cancer patient Andy Grove Forbes 5/13/96

Questions for Setting Clinical Policy: A Systematic Process 1. What is the outcome I care most about?

2. How good is the evidence that the 3.

4.

5.

6.

interventions can improve those outcomes?

How sure am I that it will work in “real world”?

How do the potential benefits compare to possible harms and costs?

What constitutes “good enough” evidence?

What other considerations are relevant?

Some History on Comparative Effectiveness Research

Comparative Effectiveness: AHRQ Effective Health Care Program  Created in 2005, authorized by

Section 1013 of the Medicare Prescription Drug, Improvement, and Modernization Act (MMA) of 2003  To improve the quality, effectiveness, and efficiency of health care delivered through Medicare, Medicaid, and S-CHIP programs by focusing on – What is known now – What research gaps are critical to fill – Clinical effectiveness

Definition: HHS Comparative effectiveness research is the conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat and monitor health conditions in “real world� settings. The purpose of this research is to improve health outcomes by developing and disseminating evidence-based information to patients, clinicians, and other decision-makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances.

Recovery Act Spend Plan, Office of the Secretary, 11/09

A Framework for CER

Evidence Generation

Horizon Scanning Evidence Need Identification Evidence Synthesis

Strategies Interventions Conditions Populations

Dissemination Translation

Research Platform Infrastructure – Methods Development – Training

Improvements in Health Care

An Unprecedented Investment Allocations for the $1.1 billion in comparative effectiveness research funding in the American Recovery and Reinvestment Act of 2009:   

Research Data Infrastructure Dissemination and Adoption  Administrative support, inventory, evaluation

  

$681M (62%) $268M (24%) $132M (12%)

 $19M (2%)

AHRQ FY 2008 – 2010 (including ARRA) Investments

3%

3% 6%

13%

57%

12%

6%

Public Engagement  Topic nominations

for research  Comment period for key research questions  Comment period for draft reviews  Focus testing translation products www.effectivehealthcare.ahrq.gov

Categorical Investments in Evidence Generation  Distributed data networks and related

infrastructure development  Methods development  Non-experimental/observational studies, both retrospective and prospective  Sponsorship of pragmatic clinical trials addressing key foci of comparative effectiveness

Evidence Generation Activities Under ARRA  Request for Registries: Up to five awards for the creation or

enhancement of national patient registries, with a primary focus on 14 priority conditions defined under MMA ($48M)  DEcIDE Consortium Support: Expansion of multi-center research system and funding for distributed data network models that use clinically rich data from electronic health records ($24M)  Clinical and Health Outcomes Initiative in Comparative Effectiveness (CHOICE): Coordinated national effort to establish a series of ~10 pragmatic clinical comparative effectiveness studies ($100M)  iADAPT: Innovative dissemination and implementation grants for CER ($29.5M)

Underlying Principles Attentive to subgroups  Under-represented in clinical trials (elderly, children, racial and ethnic minorities)  Inclusive of comorbidities  Explore heterogeneity of treatment effects

Citizen’s Forum for Comparative Effectiveness Research  ARRA funding will support the Citizens’ Forum

Initiative to develop new mechanisms and refine existing approaches to eliciting public input  The Forum will increase our use of public input to inform health care policy, especially involving comparative effectiveness research for AHRQ’s Effective Health Care Program  It will expand AHRQ’s efforts to obtain professional and consumer input, build methods and capacity for obtaining public input and allow the program to obtain guidance and insight from a broader public

Horizon Scanning for CER ď Ž Horizon Scanning

methodology and reporting for CER

Registry of Patient Registries  The Recovery Act calls for AHRQ to fund

development of a registry that would serve broader research purposes. This type of registry would: – Benefit multiple stakeholders – Reduce unnecessary duplication of effort – Encourage communication between registry

developers

“For example, researchers undertaking a patient registry of cardiovascular disease among women under the age of 55 in a certain region may find several similar registries underway with which they might collaborate or choose to differentiate to better answer certain questions.” ARRA Spend Plan, Office of the Secretary

Educating Clinicians  Concise  Actionable  Paired with

consumer guides  Convey level of uncertainty/certainty of findings

Effective Health Care Web 2.0 and Social Media Tools Product Buttons Product Collections

RSS Feeds

Widgets Social Media

CME/CE Activities CME/CE activities are available for the following health conditions and methods areas:  Diabetes  Heart and Blood Vessel Conditions  Muscle Bone and Joint Conditions  AHRQ Training Modules for the Systematic Reviews Methods Guide  Other Topics

Emerging Methods in Comparative Effectiveness, EBM & Safety 

Variation in methods among systematic reviews undercuts transparency



Poorly done new research can be misleading Methods must continue to evolve and not remain stagnant AHRQ has and will continue to make investments in improving methods, esp. in understanding clinical heterogeneity.

 

New Resource on Comparative Effectiveness Research Methods  June 2010 supplement to Medical Care on

CER Methods

 22 original articles, special focus on: –

Ways to enhance the inclusion of clinically heterogeneous populations in clinical and comparative effectiveness studies

–

Methods for implementing longitudinal investigations that capture longer term health outcomes, including patientreported outcomes

 Printed copies available free of charge

through the AHRQ Publications Clearinghouse

www.effectivehealthcare.ahrq.gov AHRQ publication number: OM10-0067

Questions Related to Surrogate Endpoints  Easy to measure but..

– How well do they link to true meaningful

health outcomes – What options exist to capture distal outcomes of interest – How well do we understand the natural course of some conditions

http://effectivehealthcare.ahrq.gov