Sensationalizing the statistics

Media coverage of many medical stories often over-promote claims and exaggerate benefits, says Emory oncologist and American Cancer Society chief medical officer Otis Brawley.

“Gone from much of the media these days are professional science writers who really understand the material,” he says. “Recently I talked with a reporter from a major newspaper and had to explain the difference between statistically significant and clinically significant.”

Study results may be statistically significant, he explains, without being clinically so. For example, suppose we have a study showing that taking aspirin can decrease risk of heart attack in people who have already had one heart attack. Enough patients in the study were found to benefit from aspirin, so the results are statistically significant. But how much does any one person in the study benefit? If the study found only a 2% reduction in the occurrence of heart attack, its results would not be clinically significant. The percentage of reduction for any one person was small enough to lack practical relevance. Statistical significance is important to help researchers interpret data but should be used with care by reporters who don’t really understand what they mean, Brawley says.

Reporters need to know how to read findings of research studies. For example, this past June, a story about an experimental vaccine to prevent breast cancer grabbed the media’s attention. The headline, “Cleveland Clinic doctor reports a possible vaccine to prevent breast cancer,” appeared in many newspapers and on broadcasts across the country. But dig a little deeper, and the story seems less captivating, says Brawley.

The experimental vaccine was tested only in mice. Although clinical trials in humans may be years down the road, researchers have been curing breast cancer in mice for years but have found few breast cancer drugs that succeed in humans. On average, only one out of every 250 drugs in lab studies or animal models gets approved.