14 Clnical
Pharmacy Practice News • March 2022
COVID-19 Pandemic
Many COVID-19 Trials Rife With Design Flaws By David Wild
O
ne problem that infectious disease physicians, pharmacists and other specialists had throughout the COVID-19 pandemic was accessing rigorous scientific data to guide clinical decision making. Such efforts were crucial to achieving at least some successful outcomes as case counts and mortality rates soared during the pandemic. To this end, many clinical trials were started. However, not all had that scientific rigor, and findings were often contradictory, making it difficult for healthcare providers and public health officials to advise patients and families. Early in the pandemic, COVID-19 treatment trials were hampered by enrollment of small study populations, use of surrogate markers and open-label design, according to researchers from the Johns Hopkins University School of Medicine. Since the beginning of the pandemic, more than 200 papers about COVID-19 have been retracted or withdrawn, and journals voiced concern about several others that have not been retracted, according to the website Retraction Watch. An early study by investigators at Johns Hopkins, perhaps, sheds light on why so many papers have been retracted. In 2020, the Hopkins investigators looked at some of the early studies that focused on treatment (BMJ Open 2020;10:e039978). (The retracted papers are not just about treatment, but touch on a wide number of COVID-19 topics.) The Hopkins study painted a picture of more than 200 disparate and disconnected trials carried out at dozens of sites around the world, leading to findings of limited clinical applicability.
Troubling Numbers 24% of studies were not randomized
29% were single-arm studies
Only 10% of the multi-arm studies were blinded
>1/3 of the trials used surrogate clinical end points or biomarkers Source: BMJ Open 2020;10:e039978.
other key websites that list clinical trials. The researchers found 201 clinical trials examining 92 drugs or convalescent plasma. Sixty-four trials studied monotherapy and 28 included a variety of treatment combinations. All but eight of the studies examined already-approved molecular entities. According to Dr. Mehta’s team, 75.7% (152/201) of the trials included randomization to treatment or a comparator. Thirty-six percent (55/152) of the trials had some form of blinding and 97 were open-label studies. Of the 24% (49/201) of studies that were not randomized, 29 were singlearm studies and 20 had one or two comparator arms, with only 10% of these multi-arm studies using a blinded design. When they looked at the geographic
‘You just need one or two trials to make a dramatic impact on the management of COVID-19, if they are well designed and find a treatment is useful.’ —Aaron E. Glatt, MD “We understand the urgency of clinical research on COVID-19, but this is a time when we need rigorous science to inform policy and clinical decision making,” lead researcher Hemalkumar Mehta, PhD, told Infectious Disease Special Edition, a sister publication to Pharmacy Practice News, shortly after publishing the report in June 2020. Dr. Mehta, an assistant professor in the Department of Epidemiology at the Johns Hopkins Bloomberg School of Public Health, in Baltimore, and colleagues scoured the World Health Organization’s clinical trials registry network as well as ClinicalTrials.gov, documenting studies registered up to March 26, 2020. They also searched major medical journals and
location of the trials, the researchers found that 49.8% (100/201) of the trials were registered in China, while 37.8% (78/201) were registered in the United States. More than half (110/201) of the studies were sponsored by hospitals or universities, 19.4% (39/201) were funded by governments, and the remainder were industry sponsored. Although 66.7% of the trials (134/201) included one or more clinical end points—such as COVID-19 symptoms, death, recovery, need for intensive care or hospital discharge—the remainder used surrogate end points or biomarkers, most commonly looking at viral load. Roughly 27% (54/201) of the studies aimed to enroll 50 or fewer patients,
‘Unfortunately, by generating a bunch of highly biased, heterogeneous and underpowered studies, [many COVID-19 study] results have been wildly contradictory.’ —C. Michael White, PharmD while 46.8% (94/201) had the goal of enrolling 100 or more patients. “Poor trial designs, lack of hard clinical end points and small trials may limit usefulness of data in guiding clinical practice,” said Dr. Mehta, noting the number of trials studying COVID-19 treatment increased since they conducted their analysis. As of early January 2022, there were more than 7,300 registered studies about COVID-19 with ClinicalTrials.gov, but not all were examining drugs or plasma. “Since our study, government and pharma sponsorship of these trials has also increased, which will make trials more collaborative, bigger and hopefully with better-designed elements,” he said.
A Pharmacist’s Take C. Michael White, PharmD, the head of the Department of Pharmacy Practice at the University of Connecticut School of Pharmacy, in Storrs, acknowledged that although the research and clinical community did not have the luxury to wait for conclusive studies during the early days of the pandemic, the community “could have done research so much better if we had a true pandemic response infrastructure in place. Instead, everyone did their own thing instead of running a logical cluster randomized trial, in which some sites would have looked at standard of care and others studied standard of
care plus a set number of different interventions,” explained Dr. White, who was not involved in the Hopkins study but is no stranger to medical research, having published more than 410 peer-reviewed publications during his pharmacy career. Using such a standard-of-care lens in COVID-19 research would have led to a balanced data set and more reliable but still rapid results, he said. “Unfortunately, by generating a bunch of highly biased, heterogeneous and underpowered studies, [many COVID-19 study] results have been wildly contradictory,” Dr. White stressed. Aaron E. Glatt, MD, the chair of the Department of Medicine at Mount Sinai South Nassau, in Oceanside, N.Y., had a different perspective. Although there are “a tremendous number of differences in the studies, the critical and heartening thing is that there are so many studies underway looking at so many different interventions, both old and new,” he said. Although some studies were not very helpful, “you just need one or two trials to make a dramatic impact on the management of COVID-19, if they are well designed and find a treatment is useful,” said Dr. Glatt, who was not involved in the research. The sources reported no relevant financial disclosures.
