I Congresso Basileiro De Pratica Clinica Basada em Evidencias Rio de Janeiro, August 28-30, 2010
Professional faking in treatment studies – the best prevention is to instruct everybody how to do it Franz Porzsolt MD, PhD Clinical Economics, University of Ulm D-89075 Ulm, Germany www.uniklinik-ulm.de/clinecs
Professional Faking in Treatment Studies
If everybody knows how to do it
The number of false studies will decrease The number of useless treatments will be reduced The quality of health care will increase The efficiency of healthcare will increase The satisfaction or patients and health care professionals will increase
This course should be mandatory to any young physician in our countries
Opinions are based on information Information is transferred by
Reading scientific literature Participation in training-programs Participation in quality circles Verbal communication
General remark
The most populare mistake: The study question is not defined as precisely as possible. Be suspicious if
The study protocol had to be chanched during the course of the study Complicated phrases or explanations are used which are difficult to understand Unknown „technical terms“ are introduced to present or discuss the results.
Specific remarks (take 10 min of your time to analyse the validity of your references)
Goal of the study
Does this study have a particular goal?
How can we find out?
Srategy & tactics
The Italian Plan: Iron defense, small presence in midfield, passes to striker.. and…Penalty
Strategie&&tactics Taktik Srategy
The German Plan Radical, ball may reach speed of 297 km/h
Srategy & tactics
Specific remarks (take 10 min of your time to analyse the validity of your references)
Goal of the study
Does this study have a particular goal or a hypothesis?
How can we find out?
Was the goal of the trial defined prospectively or retrospectively („sharpshooter“; check plausibility and compare with previous information on that topic; „Beckenbauer-trial“ >> not contaminated with a hypothesis >> „sterile“)
Question of the study
Is the study question consistent within the paper (Title, summary, introduction, discussion)
Design
What would be the ideal study design to answer the study question?
Identify the study question and design YOUR OPTIMAL DESIGN to answer this study question
Prepare a graph of the published study design. Compare the published study design with your „optimal study design“ If these are different, could you think of any reasons which may explain the difference?
Conduct I
Are there any indicators for a retrospective defini-tion of the inclusion and exclusion criteria?
Were the patients randomly allocated to the groups (was it a RCT)?
Compare double publications of same data (randomized and non-randomized)
Was the randomization concealed?
Age [23-87 yrs], laboratory data (BSR) [15-124 mm/hr], time intervals [0-14 months after surgery]
Consider the risk of monocentric trials …
Were adequate controls included in the study?
Examples: Immunotherapies, Zelen-Randomization, Waber experiment
Tablets
EFFECT OFCOMPLIANCE ON MORTALITY. NEJM 1980;303(18):1038-1042 Placebo Compliance with medication All patients Took => 80% Took < 80%
# of Patient 2789 ~2/3 ~1/3
Clofibrate 5 year mortalit y 20.9% 15.1% 28.3%
RRR ARR NNT
46.6% 13.2% 8
# of 5 year patients mortalit y 1103 ~2/3 ~1/3
20.0% 15.0% 24.6% 39.0% 9.6% 10
RRR
ARR
NNT
4% 0.9% 111 0% 0.1% 1000 13% 3.7% 27
Re-interpretation ... Randomization according to Zelen II
Response rate
The â&#x20AC;&#x17E;inverted vectorâ&#x20AC;&#x153; Experimentals
IC R Exp
Con Controls
yes or no
not asked
A
B
C
D
E
F
Conduct II
Were doctors and patients blindet?
Were all patients who participated in the study included in in the evaluation of the study evaluation?
Could „deblinding“ be prevented?
Which patients have the highest risk to be excluded from a trial?
Was the study closed before the scheduled endpoint?
Is the closing rule of a clinical trial comparable to the rules a socker game, or a horse race or a tennis match?
Report / Interpretation
Are the recommendations derived from the treatment confined to the investigated population?
Remember the inclusion- and exclusion criteria Compare the conclusions of the study with the in-excriteria
Are the results of this study clinically relevant?
Will your patients accept the side effects of the treatment to receive this „lousy“ benefit? Consider patients‘ values and preferences!
Alle
Treatment related death Local relapse Distant metastatses
Geheilte Patienten
Survival no difference Second cancer Stool incontinence Cardiovascular compl.
Kornmann M, Porzsolt F. Treatment Preferences of Physicians and Lay Persons: Lessons from a Study Analysing Neoadjuvant Treatment of Rectal Carcinoma. J Clin Oncol 2008;26:2866-4868
Report / Interpretation
Is the principle of „perceived safety“ considered“
Is „perceived safety“ ethically justified in this situation (talking about possible cure in an incurable cancer??)
Is the application of the the new principle justified when ethical, medical, epidemiological, and economic aspects are considered?
Further readings
Evidence Based Medicine. How to practice and teach EBM. Churchill Livingstone 3 ed. 2005 Portugese translation by ARTMED editors There will be many more updated books in Portugese language on EBM …
Further thinking
For 96% of the population ( ) it is rather difficult to use „conditional probabilities“ Example of the „GOAT PROBLEM“
Solution after Aberdura
incluing myself
Summary
Physicians and managers have to know how to to identify valid information. The methods to identify ‚at a glance‘ are easy to adapt. Don‘t try it by yourself – find friends to form small groups Try to get feedback from experienced people by joining a Brasilian network of EBM