Spectrum bias of studies on diagnostic tests

I Congresso Brasileiro de Prática Clínica Baseada em Evidências Viés de espectro em estudos que avaliam a performance de testes diagnósticos.

Spectrum bias in studies on diagnostic tests Luz M. Letelier MD; FACP Associate Professor Pontificia Universidad Católica de Chile

OBJECTIVES Analyze how an inappropriate spectrum of disease may bias the results of a study on the performance of a diagnostic test.

Understand spectrum of disease.

Spectrum of disease • When do we use a diagnostic test? • What do we expect a test to do, when we apply it?

Likelihood of a disease 1)

2)

3)

Maria A: 26 year old, feeling “nervous” for 2 months, she’s lost 10 kilos, has tremor, increased sweating and 5 bowel movements/day. On exam: tremor, tachycardia 110/min, goiter, thyroid bruit and lid lag.

Maria B: 26 year old, feeling “nervous” for 2 months, feels short episodes of palpitations every day and has insomnia. PA 142/88.

Thyroid somewhat enlarge.

Maria C: 26 year old woman feeling “nervous” for 2 months, no other symptoms, but tells you she is moving to another country due to her husband’s new job. No remarkable findings on exam.

Diagnostic Process

Diagnostic threshold

0%

Uncertainty area

Therapy threshold

100%

Diagnostic process for the likelihood of disease

Maria C

Maria A Maria B Maria C

Hyperthyroidism Maria B

Maria A

Uncertainty area

Diagnostic threshold

Therapy threshold

Diagnostic test • We use a diagnostic test to move the probability of having a condition or not having the condition when there is uncertainty.

Diagnostic test • Assuming we use a test when we are uncertain regarding the condition status. • We need the test to discriminate between those who do from those who do not have the condition.

Spectrum of disease Distribution of people without hyperthyroidism

Distribution of people with hyperthyroidism

How can an inappropriate spectrum of patients influence the results of a study on the performance of a diagnostic test?

Let’s assume we want to validate a new test for morbid obesity: THE RING SIZE We prospectively recruit 100 women Compare “the ring size” (our test) to the Gold Standard: Body Mass Index (BMI).

If the ring size is > 10 the test is (+) If the ring size is <10 the test is (-)

THE RING SIZE TEST Suzana

BMI

Luz

40 Of the 100 women recruited: 70 look like Suzana 30 and look like Luz

Luz

Suzana

CONDITION MORBID OBESITY

TEST RING SIZE > 10

SI

NO

30

70

(+) (-) 100

CONDITION MORBID OBESITY

TEST RING SIZE > 10

SI

NO

(+)

30

0

30

(-)

0

70

70

30

70

100

CONDITION MORBID OBESITY

TEST RING SIZE

SI

NO

(+)

30

0

30

(-)

0 30

70 70

70

0 False positive and 0 false negative SENSITIVITY = 100% SPECIFICITY = 100% PERFECT TEST!!!!

RING SIZE TEST Now we have a perfect test that correctly identifies all morbid obese patients from all of those who are not.

BUT will it perform as good in a sample of women with real uncertaintyâ&#x20AC;Śâ&#x20AC;Ś.?

TRUE OR FALSE • The ring test is the perfect test. F

• We chose an inappropriate spectrum of V patients.

Studies on diagnostic tests might render biased results when: An inappropriate spectrum of patients is chosen.

ESPECTRUM OF DISEASE AND TEST RESULTS D-Dimer

D-Dimer Normal individuals

400

Very sick patients

600

D Dimer Possibly sick

Not severe sickness

D-Dimer Possibly sick

400

Non severe sick

600

Spectrum of disease

BREAST CANCER DIAGNOSIS

Diagnostic threshold

Therapy threshold

TO REMEMBER

Diagnostic tests should be validated on patients with diagnostic uncertainty.

REMEMBER â&#x20AC;˘ The appropriate spectrum of patients is crucial to reduce bias in a study on the performance of a diagnostic test. â&#x20AC;˘ The appropriate spectrum of patients are those who have diagnostic uncertainty regarding the condition under study.

OBRIGADA

CHEN JS, CHEN KT et al. Combined analysis of survivin autoantibody and carcinoembryonic antigen biomarkers for improved detection of colorectal cancer.

Clin Chem Lab Med. 2010 Feb 24.

Background: Survivin is a member of the family of inhibitor of apoptosis proteins that is overexpressed in several human tumors.

Previous studies have found that overexpression of survivin in cancer cells induces an antibody response. Methods: We compared 232 serum samples from colorectal cancer (CRC) patients and 365 samples from healthy volunteers using an in vitro enzyme-linked immunosorbent assay to evaluate the survivin autoantibody response in patients.

Conclusions Our results suggest that analysis of both parameters would assist in screening patients with CRC.