Are you at risk? falling victim to misleading presentations as a result of conflict of interest of investigators
Not at all A little Moderate risk High risk
Conclusions and Funding, Als-Nielsen, JAMA 2003 • 25 Cochrane reviews, 370 RCTs – –
random selection of 167 of 1081 (issue 2, 2001) excluded • • • •
101 < 5 studies 6 no concealment variability 2 no binary outcome 16 non-pharmacological
• abstracted – – – –
sources of funding effect on primary outcome adverse effects methodological quality (concealment, blinding)
Scale Used to Grade Conclusions in Trials 6 Points Experimental intervention highly preferred and should now be considered the standard intervention in all patients, or similar 5 Points Experimental intervention preferred to control, but further trials still indicated, experimental may be more costly, or similar 4 Points Experimental and control intervention about equal, but the experimental cheaper, easier to administer, or similar minor advantage 3 Points Experimental and control intervention about equal, but the control may be cheaper, easier to administer, or similar minor advantage 2 Points Control intervention preferred to experimental intervention, but experimental intervention might be promising under some circumstances, or similar 1 Point Control intervention highly preferred and should now be considered the standard intervention in all patients, or similar
Relation Between Funding Source and Conclusions in 370 Randomized Drug Trials Funding
# of Trials
Median Score
# (%) of Trials Scoring 6 Points+
Nonprofit organizations
67
(IQR)+ 4 (3-5)
Not reported
106
5 (3-6)
32 (30.1)
11 (16.4)
Nonprofit and forprofit organizations For-profit organizations
51
5 (4-6)
18 (35.2)
146
6 (5-6)
74 (50.6)
Total
370
5 (4-6)
135 (36.4)
Abbreviation: IQR, interquartile range. *Conclusions in trials were assessed by a 1-6 point scale. If the conclusions recommended the experimental drug as the treatment of choice without disclaimers, 6 points was assigned, and if not, 1-5 points was assigned. +p<.001, using Kruskal-Wallis test (medians) or X2 test (proportions)
Als-Nielson, JAMA, 2003 • bigger effect size, more likely stronger recommendation • blinding, more likely stronger recommendation • after adjustment, industry funding, odds ratio 5.3 (95% CI 2.0 to 14.4)
1 Read methods and results bypass the discussion
Use of placebo when active comparator optimal Do you manage patients with type II diabetes? Is there any agent you would routinely suggest for patients with diabetic nephropathy?
Angiotensin receptor blockers for diabetic nephropathy vs. ACE inhibitors Parving H-H et al. N Engl J Med 2001;345(12):870-878 Brenner BM N Engl J Med 2001;345(12):861-869 Lewis EJ et al. N Engl J Med 2001;345(12):851-60. Hostetter TH. N Engl J Med 2001;345(12):910-912
Incomparable comparators • 8 RCTS of 2nd generation neuroleptics vs. 20 mg/d haloperidol – –
fewer extrapyramidal symptoms standard dose haloperidol < 12 mg
• RCT paroxetine qd vs. amitriptyline bid – –
less daytime somnolence standard amitriptyline qhs
Safer J Nerv Ment Dis 2002;190(9):583-92. Geddes J et al BMJ 2000;321(7273):1371-6. Christiansen PE, et al. Acta Psychiatr Scand 1996;93(3):158-63
2
Beware faulty comparators
Irbesartan vs amlodipine in diabetic nephropathy • in comparison to amlodipine, irbesartan reduced the combined endpoint of all cause mortality, progression to end stage renal disease, and doubling of serum creatinine (RRR 20%, 95% CI 7.5% to 32%) • did irbesartan reduce all-cause mortality?
Risk reduction with irbesartan (vs. amlodipine)
Doubling of creatinine concentration
RRR 33% (16-47%)
End-stage renal disease
RRR 23% (-0.5-41%)
All-cause mortality
RRR -3% (-35-22%)
Composite endpoint
-40
RRR 20% (7.5-32%)
-24
-8
8
24
40
56
RRR (95% CI) RCT 1715 DM 2 nephropathy, HTN irbesartan vs amlodipine NEJM 2001; 345: 851
What has gone wrong? • widely varying importance • biggest effect on least important – most important no effect
• criteria for use of composite – similar patient-importance – similar effect
3 Beware composite endpoints
5,269 patients with abnormal glucose tolerance test randomized to lifestyle advice, or advice + rosiglitazone followed for 3 to 5 years
What is the authors’ message? • rosiglitazone to prevent diabetes: • strong indication (for all) • weak indication (for some) • not indicated
Doctor, what do I gain by taking rosiglitazone? • Doc: less chance of diabetes • Pt: what happens if I get diabetes • Doc: you have to take a drug • Pt: the same drug I’m taking to prevent diabetes? • Doc: I could give you a drug with less problems
drug
no drug
difference
30000
3650
26350
1060
2500
HR 0.38 (CI, 0.33-0.44)
Anxiety about diabetes
??
??
??
Costs inconvenience self-monitoring
??
??
??
Costs and inconvenience HbA1c, lipoprotein testing, retinal exam, etc.
??
??
??
MI, stroke, CV death at 3 years
120
90
HR 1.39 (CI 0.81-2.37)
Heart failure, at 3 years
50
10
HR 7.03 (CI 1.6-30.9)
kidney, eyes, neuropathy
??
??
??
Peripheral edema, at 3 years
680
490
RR 1.4 (CI 1.1-1.8)
Weight gain (kg), at 3 years
+ 1.1
-1.1
+ 2.2
??
??
??
diabetes medication 3 years new diagnosis of diabetes
Rare (fractures, macular edema)
What is your view? • rosiglitazone to prevent diabetes: • strong indication (for all) • weak indication (for some) • not indicated • what has gone wrong here?
Other problematic surrogates • tests of cognitive function instead of function and behavior in Alzheimer’s • bone density instead of fractures in osteoporosis • oxygenation instead of mortality in ARDS • asymptomatic DVT instead of symptomatic thromboembolism
4 Beware substitute endpoints
Five vs Four Courses of Therapy for Acute Myeloid Leukemia
Wheatley K, Clayton D. Controlled Clinical Trials 2003;24:66-70
Five vs Four Courses of Therapy for Acute Myeloid Leukemia
Wheatley K, Clayton D. Controlled Clinical Trials 2003;24:66-70
Five vs Four Courses of Therapy for Acute Myeloid Leukemia
Wheatley K, Clayton D. Controlled Clinical Trials 2003;24:66-70
Five vs Four Courses of Therapy for Acute Myeloid Leukemia
Wheatley K, Clayton D. Controlled Clinical Trials 2003;24:66-70
Stopping boundary True beneficial effect No effect
stop
Stopping boundary True beneficial effect No effect
Look after every patient or event
stop
stop
stop
Stopping boundary True beneficial effect No effect
Interim analyses every q patients or events
5 Beware trials stopped early
Users Guides to Spin 1. read methods and results; bypass the discussion section 2. beware faulty comparators 3. beware composite endpoints 4. beware of substitute endpoints
Wall St. Journal, November, 2006
Wall St. Journal, November, 2006
New York Times, May 9, 2007
Available at: www.mhprofessional.com/jama