Searching for Gold Standards

Why David Healy is Wrong

WHY RANDOMISE?

Why RCTs are the best…I really mean it… cross my heart and hope to die….

Insert Picture of Sliced Bread

Why do we bother with RCTs?

Why bother to randomise?

• If they fail to demonstrate any differences between a placebo and a drug which everybody knows to be effective, this means only that the work has not been done well enough

Slater and Sargeant, 1963

We are never going to learn how to treat depressions properly from double blind sampling in an MRC statistician’s office

Sargeant, BMJ, 1963

If you can believe fervently in your treatment, even though controlled tests show it is quite useless, then your results are much better, your patients are much better and your income is much better too. I believe this accounts for the remarkable success of some of the less gifted, but more credulous members of our profession, and also for the violent dislike of statistics and controlled test which fashionable and successful doctors are prone to display

Richard Asher, 1972

So what is unique about the randomised controlled trial?

So what is unique about the RCT? • Is it having a control group?

So what is unique about the RCT? • •

Is it having a control group?

Is it the blindness?

So what is unique about RCT? • • •

Is it having a control group?

Is it the blindness?

Is it controlling for the effects of chance?

Have the Samaritans Reduced the Suicide Rate?

RANDOMISATION IS THE ONLY WAY TO CONTROL CONFOUNDERS

And what happens if you don’t randomise?

YOU GET THE WRONG ANSWER

Randomization: the Gold Standard Randomization

Proportion of studies with p < 0.05

Non random assignment

Che ating difficult (%)

Cheating e asy (% )

8.8

24.4

58.1

Chalmers et al, NEJM, 1983

Boys behaving badly

Cambridge Somerville Youth Study • • • •

750 delinquent boys randomised “sustained friendly counselling” Intervention lasted five years

5, 10, 20 and 30 years later, one group had more • • • • •

criminal convictions unemployment premature death alcoholism Serious mental illness McCord, 1978

Men behaving badly

“FORWARD PSYCHIATRY” – The PIE Principles • Proximity • Immediacy • Expectancy

RFA Canberra

What else can we do about psychological trauma?

What is debriefing?

• • • • • • •

Introduction Describing traumatic incident Psychological reactions to incident Emotional processing/catharsis Identifying symptoms Teaching - symptoms are normal Re entry/closure

Debriefing has been used for….

•• •• •• •• •• •• •• •• •• ••

Police Police after after shooting shooting incidents incidents Sailors Sailors after after maritime maritime collisions collisions Red Red Cross Cross personnel personnel Medical Medical students students whose whose patients patients have have died died Families Families whose whose children children are are undergoing undergoing transplants transplants Soldiers Soldiers on on grave grave registration registration duties duties Train Train drivers drivers who who have have witnessed witnessed suicides suicides Jurors Jurors in in murder murder trials trials Air Air Force Force personnel personnel on on whose whose Base Base there there has has been been aa fatal fatal accident accident Casualty Casualty staff staff after after trauma trauma incidents incidents

•• •• •• •• •• •• •• •• •• •• ••

Burns Burns victims victims accident accident victims victims Hospital Hospital staff staff after after failed failed resuscitations resuscitations Nurses Nurses in in cancer cancer care care patients patients who who have have recovered recovered from from cancer cancer Rape Rape victims victims Rescue Rescue workers workers after after any any natural natural disaster disaster The The entire entire New New York York Fire Fire Department Department The The entire entire New New York York Police Police Department Department Children Children attending attending aa school school where where aa pupil pupil has has died died Ward Ward staff staff where where aa suicide suicide has has occurred occurred

Does it work? How dare you even ask? “the experiences of 700 CISM teams in more than 40,000 debriefings cannot be ignored, especially so when the overwhelming majority of reports are extremely positive” “numerous studies have already shown positive results….proves the clinical effectiveness beyond reasonable doubt”

Mitchell & Everly, 2003

Single session psychological debriefing definitely does not work… …..and increases the risk of PTSD

Rose, Wessely, Bisson – Cochrane Review 2003: Emmerink et al, Lancet, 2002

So why doesn’t debriefing work?

• • • • • •

Not everyone wants or needs to talk “retraumatisation” Interferes with natural recovery processes Suggestion Professionalisation of distress Impedes people talking to who they want, when they want

Everyone behaving badly

This canâ&#x20AC;&#x2122;t be right

Graded Exercise Therapy (GET) for CFS: (Fulcher & White, BMJ 1997) • Setting:

National Sports Centre

• Design:

RCT

• Treatment:

12 weeks Graded Exercise Therapy

• Control

Flexibility exercises

• Patients:

66 CFS patients

• Results:

GET superior

• Conclusions:

GET safe and effective

Percentage improved with GET 70 60 50 40

GET Control

30 20 10 0 Fulcher

Wearden

Powell

Percentage improved with GET 60 50 40 30

GET Control

20 10 0 MossMorris

Wallman

What is CBT? • • • • •

Collaborative Identifies cognitive barriers Sets behavioural targets, not exercise Emphasises predictability/consistency Does relapse prevention

CBT for CFS: the Oxford RCT (Sharpe et al, BMJ 1997)

• Setting:

Medical out patient clinic

• Treatment:

16 sessions CBT given by skilled therapists

• Patients:

60 CFS patients

• Results:

CBT superior at 12 months on symptoms and function

1996

CBT for CFS: The King’s RCT (Deale et al, 1997) • Setting:

CFS Clinic

• Treatment:

12 sessions CBT

• Control

12 sessions relaxation

• Patients:

72 patients with CFS

• Results:

CBT better on symptoms and disability

• Five year follow up Differences less, but still present

Outcome at 6 months follow up CBT

Relaxation

16 (53%)

3(10%)

Improved

8 (27%)

11 (37%)

Unimproved

6 (20%)

16 (53%)

Much improved (Increase of 50 or more on MOS score)

(incl drop outs)

CBT for CFS: a multi-centre RCT (Prins et al Lancet 2001) • Setting:

Medical out patient clinic

• Treatment:

CBT given by newly trained therapists; Support group or Natural course.

• Patients:

278 patients with CFS

• Results:

CBT group: 35% improved on fatigue; 49% physical functioning; 50% self rated improvement.

• Conclusions:

CBT was more effective than guided support and natural course with non specialist therapists.

RCT of patient education to encourage graded exercise in CFS (Powell et al BMJ 2001) • Setting:

Medical out patient clinic

• Patients:

148 patients with CFS

• Treatment:

standard medical care; GET; telephone advice + GET: face to face advice + GET

• Results:

69% improved in the intervention groups 6% of controls improved

• 4 years later

Improvements maintained

Routine Clinical Practice (Chalder et al) •

Setting:

King’s

•

Design:

Prospective uncontrolled study

•

Treatment:

Routine CBT

•

Patients:

293 patients with CFS

•

Results:

•

Conclusions:

58% rated themselves as very/much or much better; 26% were a little better; 16% were the same or worse on global outcome Outcomes only slightly less than in the RCTS

Game, set and match ……….

Action for ME Website (www.afme.org.uk) Q. What treatments are available?

A. Research has proven two therapies bring relief for many people with ME…………...

Action for ME Website (www.afme.org.uk)

â&#x20AC;Śâ&#x20AC;Ś.magnesium injections and the dietary supplement Efamol (a combination of evening primrose oil and marine oil)

Action for ME Self help Group Survey

COM P ARIS O N OF T R EATM E NT AP P RO A CH ES GRO UP S RE S E A R C H S E V E RE LY A FFE C TE D S URV E Y

s am ple

he lpe d

m a de no diffe re nce

m a de w orse

P ACI NG GRO UP S 257 S E V E RE LY A FFE C TE D 2,180

88% 89%

9% 9%

3% 1%

GR AD E D EX E RCIS E TH E R AP Y GRO UP S 209 S E V E RE LY A FFE C TE D 1,214

39% 34%

22% 15%

39% 50%

55% 7%

32% 67%

13% 26%

COG NITIV E B EH AV IOU R AL TH E R AP Y GRO UP S S E V E RE LY A FFE C TE D

113 285

Psychiatry is the dustbin of the medical profession (Clare Francis, 1988)

Psychiatry is opinion dressed up as science, stupid and hypocritical (Clare Francis, 1996)

“People with postviral fatigue syndrome often have to put with a lot of disbelief-there were many doctors who diagnosed this as a psychiatric disorder although on the whole it is taken much more seriously now” “Watchdog to look into ME resources”: Dundee Courier and Advertiser, 11.11.1994

â&#x20AC;&#x153;ME is an imaginary illness, for which the best treatment is psychiatricâ&#x20AC;?

Unnamed physician, cited in Steincamp, 1989

Without the RCT we would never have been able to improve the care of patients

People don’t do clinical trials in mental health because........

• • •

I know my treatment works My treatment is too individual The results are not generalisable

Whatâ&#x20AC;&#x2122;s the real reason?

They are bloody difficult

Problems with Psychiatry Trials: Our Fault • • • •

Not big enough Not good enough Too many rating scales Testing wrong kind of things

Cochrane Depression/Anxiety Group- Interventions 10% 23%

Other Psychotherapy

67%

Pharmacotherapy

Problems with Psychiatry Trials: Not Our Fault • • • • • •

Problems with follow up/drop outs Fuzzy outcome measures Large non specific effects Complex interventions Ethical dilemmas (capacity, coercion) Bureaucratic nightmare which is getting worse

So what is the moral of the story?

When in doubtâ&#x20AC;Ś.

Randomise!!