Professional faking in treatment studies – the best prevention is to instruct everybody how to do it

I Congresso Basileiro De Pratica Clinica Basada em Evidencias Rio de Janeiro, August 28-30, 2010

Professional faking in treatment studies – the best prevention is to instruct everybody how to do it Franz Porzsolt MD, PhD Clinical Economics, University of Ulm D-89075 Ulm, Germany www.uniklinik-ulm.de/clinecs



Professional Faking in Treatment Studies 

If everybody knows how to do it     



The number of false studies will decrease The number of useless treatments will be reduced The quality of health care will increase The efficiency of healthcare will increase The satisfaction or patients and health care professionals will increase

This course should be mandatory to any young physician in our countries

 

Opinions are based on information Information is transferred by    

Reading scientific literature Participation in training-programs Participation in quality circles Verbal communication



General remark 



The most populare mistake: The study question is not defined as precisely as possible.

Be suspicious if 





The study protocol had to be chanched during the course of the study Complicated phrases or explanations are used which are difficult to understand Unknown „technical terms“ are introduced to present or discuss the results.



Specific remarks (take 10 min of your time to analyse the validity of your references) 

Goal of the study 

Does this study have a particular goal? 

How can we find out?

Srategy & tactics

The Italian Plan: Iron defense, small presence in midfield, passes to striker.. and…Penalty

Strategie&&tactics Taktik Srategy

The German Plan Radical, ball may reach speed of 297 km/h

Srategy & tactics



Specific remarks (take 10 min of your time to analyse the validity of your references) 

Goal of the study 

Does this study have a particular goal or a hypothesis? 



How can we find out?

Was the goal of the trial defined prospectively or retrospectively („sharpshooter“; check plausibility and compare with previous information on that topic; „Beckenbauer-trial“ >> not contaminated with a hypothesis >> „sterile“)



Question of the study 

Is the study question consistent within the paper (Title, summary, introduction, discussion)



Design 

What would be the ideal study design to answer the study question? 

 



Identify the study question and design YOUR OPTIMAL DESIGN to answer this study question

Prepare a graph of the published study design.

Compare the published study design with your „optimal study design“ If these are different, could you think of any reasons which may explain the difference?



Conduct I 

Are there any indicators for a retrospective defini-tion of the inclusion and exclusion criteria? 



Were the patients randomly allocated to the groups (was it a RCT)? 



Compare double publications of same data (randomized and non-randomized)

Was the randomization concealed? 



Age [23-87 yrs], laboratory data (BSR) [15-124 mm/hr], time intervals [0-14 months after surgery]

Consider the risk of monocentric trials …

Were adequate controls included in the study? 

Examples: Immunotherapies, Zelen-Randomization, Waber experiment

Tablets

EFFECT OFCOMPLIANCE ON MORTALITY. NEJM 1980;303(18):1038-1042 Placebo Compliance with medication All patients Took => 80% Took < 80%

# of Patient 2789 ~2/3 ~1/3

Clofibrate 5 year mortalit y 20.9% 15.1% 28.3%

RRR ARR NNT

46.6% 13.2% 8

# of 5 year patients mortalit y 1103 ~2/3 ~1/3

20.0% 15.0% 24.6% 39.0% 9.6% 10

RRR

ARR

NNT

4% 0.9% 111 0% 0.1% 1000 13% 3.7% 27

Re-interpretation ...

Randomization according to Zelen II

Response rate

The â&#x20AC;&#x17E;inverted vectorâ&#x20AC;&#x153; Experimentals

IC R Exp

Con Controls

yes or no

not asked

A

B

C

D

E

F



Conduct II 

Were doctors and patients blindet? 



Were all patients who participated in the study included in in the evaluation of the study evaluation? 



Could „deblinding“ be prevented?

Which patients have the highest risk to be excluded from a trial?

Was the study closed before the scheduled endpoint? 

Is the closing rule of a clinical trial comparable to the rules a socker game, or a horse race or a tennis match?



Report / Interpretation 

Are the recommendations derived from the treatment confined to the investigated population?  



Remember the inclusion- and exclusion criteria Compare the conclusions of the study with the in-excriteria

Are the results of this study clinically relevant? 

Will your patients accept the side effects of the treatment to receive this „lousy“ benefit?

Consider patients‘ values and preferences!

Alle

Treatment related death Local relapse Distant metastatses

Geheilte Patienten

Survival no difference Second cancer Stool incontinence Cardiovascular compl.

Kornmann M, Porzsolt F. Treatment Preferences of Physicians and Lay Persons: Lessons from a Study Analysing Neoadjuvant Treatment of Rectal Carcinoma. J Clin Oncol 2008;26:2866-4868



Report / Interpretation 

Is the principle of „perceived safety“ considered“ 



Is „perceived safety“ ethically justified in this situation (talking about possible cure in an incurable cancer??)

Is the application of the the new principle justified when ethical, medical, epidemiological, and economic aspects are considered?



Further readings 



Evidence Based Medicine. How to practice and teach EBM. Churchill Livingstone 3 ed. 2005 Portugese translation by ARTMED editors There will be many more updated books in Portugese language on EBM …



Further thinking 



For 96% of the population ( ) it is rather difficult to use „conditional probabilities“ Example of the „GOAT PROBLEM“

Solution after Aberdura

incluing myself



Summary 







Physicians and managers have to know how to to identify valid information.

The methods to identify ‚at a glance‘ are easy to adapt.

Don‘t try it by yourself – find friends to form small groups Try to get feedback from experienced people by joining a Brasilian network of EBM