Real World Research

New models for evidence generating care

By Dr Michael Winlo, Researcher, West Leederville

Ever looked at an old photo and winced at your clothes? For me – hyper-colour pants in primary school. Only later can we see how awkward our choices are. Ideologies and traditions can be fashions which encourage us to look for fashions in conservative places like clinical research. The limitations of our most trusted evidence-generating apparatus – the randomised control trial (RCT) are being exposed by recent studies.

Clinicians face the challenge of trying to apply the take-aways from an RCT manuscript to the patient who, not infrequently, neither looks, nor behaves, like the carefully selected trial participants. And, what if the trial, itself, is flawed?

Diana Herrera-Perez et al released a study in 2019 examining the stability of the findings of more than 3000 RCTs published across three major medical journals. Her team found 396 cases of “medical reversal” – that is, where a newer and methodologically superior clinical trial produced a result contradicting existing clinical practice and the older trials on which it was based; 13% of published studies were eventually reversed.

Key messages

RCTs are not the only form of evidence The digitisation of health data provides new sources of evidence Real world evidence opens doors.

Evolving statistical models play a role. A 2018 study exposed the lack of concordance in modern statistical analysis. The research team gave the same data to 29 teams of statisticians; 69% produced analysis that was for the null hypothesis but 21% of teams produced data supporting the opposite result. Remarkably, all teams presented internally validated and rational statistical approaches. Deciding whether to include data outliers is a good example of where differences can emerge (are they aberrations or ‘real’ phenomena worth including?). The study demonstrated how subjective, yet defensible, analytical choices could influence research results. In response to these tensions, major regulators such as the FDA and EMA are increasingly interested

in real world evidence (RWE) as a complement to, or in lieu of, traditional clinical trial evidence.

RWE is clinical evidence derived from the careful analysis of realworld data (RWD) such as insurance claims, electronic health records, disease registries and patientgenerated data. It is an exciting trend. Firstly, it’s becoming easier to incorporate digital health technology into our own practices and thereby contribute to the body of RWD. Secondly, as the analysis of this data improves, we can more effectively evaluate the efficacy and safety of treatments directly in the line-ofcare and provide patients access to new treatments sooner.

As the limitations of RCTs become better understood, more pragmatic evidence-generating methods that absorb the real-world variances of patients and care delivery are welcome and, with luck, this kind of evidence-generating care can help expose our fashions a little earlier.

– References on request

Author competing interests - nil

Queensgate Specialist Centre

Specialist sessional suites NOW OPEN in Canning Vale

Want to start or build your practice where there is high demand for specialists? Join our multi-disciplinary specialist team in our new purpose built sessional suites. We offer a low-risk and low-cost option to consult in our new sessional rooms in the rapidly growing south eastern corridor of Perth. Free multi-level parking and only 8kms from Murdoch Hospital.