The anatomy of an awesome randomised trial

Ben A Marson and Daniel C Perry

Ben Marson is an Orthopaedic Registrar from the East Midlands North rotation who is currently out of programme and studying for a PhD at the University of Nottingham. He has been awarded a NIHR Doctoral Fellowship to investigate outcomes following childhood fractures.

Daniel Perry is a Professor of Orthopaedic Surgery and Honorary Consultant Orthopaedic Surgeon at Alder Hey Children’s Hospital. He is the chief investigator for several multicentre clinical trials of orthopaedic interventions including FORCE, SCIENCE and CRAFFT which have been funded by the NIHR. Dan is one of the three BOA Surgical Specialty Leads for Clinical Trials.

In the past decade, orthopaedics has emerged as an academic speciality able to deliver high-quality randomised trials. We are experiencing an explosion of engagement in research trials from both within our speciality and from external sources including research funders. This has resulted in the publication of many trials, including many in the top medical journals. However, there remain many uncertainties within our speciality that need proper investigation.

Randomised trials are the gold standard to compare healthcare interventions. However, conducting randomised trials is expensive, with major trials typically costing in excess of £1million. The cost of these is accrued through the rigour in design and conduct of these studies – to ensure that the trial is a definitive answer to a question, and ultimately to avoid the dreaded conclusion that “more evidence is required”.

As a community, we need to continue to ask clinical questions that can be answered through trials to continue to advance our understanding of the interventions we recommend. The purpose of this article is to unpick some of the features that convert these clinical questions into a trial that can be delivered for the benefit of our patients. This process starts when the clinical question is translated into a research question that underpins the clinical trial.

Where to start: the clinical question

There are several sources of questions abundant in orthopaedic clinical practice. There are so many questions and uncertainties that the issue is more where to start?

The panacea of a research question is to identify an issue with broad relevance. The question needs to be important to: 1) you, as you need to invest your own passion and energy into it; 2) your subspeciality colleagues, as you need their buy-in to deliver the study, and eventually to ensure your research findings have meaningful impact; 3) patients, who will be part of the study; 4) funding bodies to support the trial; 5) the public, who will ultimately benefit from the trial and 6) policy makers who use the research findings to shape commissioning decisions.

The need for research can be established through systematic reviews. Cochrane reviews highlight needs for future research, which often clarifies uncertainties found following literature searching. The National Institute for Health and Clinical Excellence (NICE) prioritise a list of research questions with each of their guidelines which can form the basis of further study. For example, the FORCE distal radius study was inspired by the 2016 non-complex fractures guidelines requesting an evaluation of the effectiveness of interventions for torus fractures.

Priority setting studies have also been completed by subspecialty groups including the AOUK10 and British Society of Childrens Orthopaedic Surgeons. Several of these questions have already been addressed with multicentre trials and others are being developed to address these priority uncertanities. However, the use of these surgeon-focused lists potentially omits the important views of patients, the public and policy makers.

The Gold Standard in research priority setting is the James Lind Alliance Priority Setting Partnership. This is based on a collaborative methodology to encourage meaningful engagement between researchers, clinicians and patients in setting research priorities. This programme is part funded by the UK medical research funding body, the NIHR. Currently, eleven lists of priority questions have been published that are relevant to orthopaedics, with three further priority sets in development (Table 1).

Figure 1: The orthopaedic randomised controlled trial bridge with the keystone research question. Image supplied courtesy of the University of Nottingham ©

Crafting the keystone: the research question

Where the clinical question identifies the problem to solve, the research question crystalises the problem into something that can be solved. If we imagine the trial as a bridge, safely transporting patients (with their surgeons) from illness or injury to recovery or rehabilitation, then the research question is the keystone from which the structure of the bridge depends (Figure 1). Crafted well and trial will endure, crafted poorly then the trial may collapse.

A common language is used to structure a research question is the PICO format, initially attributed to Richardson et al. in 1995 and subsequently included in the 2008 Cochrane handbook. The PICO format structures the question into the following four elements:

Participants-(P): the population to be studied as defined through clear inclusion and exclusion criteria.

Intervention-(I): the intervention or interventions of interest that are being trialled.

Comparison-(C): the control intervention being used as a comparison. Historically, the ideal comparison has been with a placebo therapy. For surgery, this can be a sham procedure such as used to evaluate arthroscopic surgery. However, for many orthopaedic trials this is less desirable, such as trauma surgery. An alternative is to use the agreed current standards of care.

Outcomes-(O): the primary outcome and the timing of the primary outcome is the key to the trial. The study is usually designed to answer a single question related to the primary outcome at a single time point. Secondary outcomes are supplementary information to support the ‘story’ of the primary outcome.

How pragmatic should a trial be?

A trial can be designed as an efficacy or effectiveness trial. An efficacy trial is often a proof-of-concept study, conducted using strict criteria for participants, interventions, and comparisons. It asks, “Under perfect conditions, does an intervention work?”. Carefully selected participants who meet trial inclusion criteria receive a standardised intervention that may be evaluated to ensure the intervention was completed as instructed. These studies are often single centre to reduce operator variability and demonstrate the utility of an intervention in these controlled circumstances. This design has the advantage that effects are likely to be due to the intervention, but may overestimate the effect size in the wider population and is limited in generalisability to other settings.

In contrast, an effectiveness trial tests the intervention in a ‘real-world’ setting, with broader inclusion criteria and more relaxed definitions of the interventions and comparisons to reflect current clinical practice. Effectiveness trials are termed as pragmatic to highlight the greater flexibility permitted in the definition of participants, interventions and comparisons.

The reality is that every trial sits somewhere on a continuum between these two designs. For example, the UK FASHIoN trial of hip arthroscopy compared to physiotherapy led non-surgical management had many features of an effectiveness trial. The trial recruited patients from 23 different centres and included patients with cam or pincer anatomy in the absence of pre-existing hip pathology. Surgery was completed by 27 different surgeons. However, the selection of comparison was less pragmatic in the form of a physiotherapy package for which practitioners had to attend specific training to maintain the consistency of the therapy protocol provided by the 47 physiotherapists within the trial6. This was selected to provide a standardised comparison for arthroscopy but may not reflect the real-world care delivered in every hospital.

What and how to measure

Each trial requires a primary outcome for which the sample size will be calculated to enhance our understanding of the interventions. The selection of outcomes will be driven by the clinical factors and should be informed by surgeons, patients, and other relevant stakeholders. Outcomes need to be relevant and are useful in helping surgeons and patients implement the trial findings.

To develop trial findings into treatment guidelines and policy decisions, trials are aggregated through systematic review and metanalyses. By pooling results uncertainties in treatment effects are reduced. However, a common criticism of trials in these systematic reviews is a lack of consistency in the selection of outcomes limiting this pooling. To address this, core outcomes sets have been developed in orthopaedics.

A core outcome set is a group of outcomes that have been agreed to be measured in all trials relating to a specific condition. A core set is not a comprehensive set as additional outcomes may be measured in addition to those identified in the core group. While the methodology varies between studies for the development of a core outcome set, the process typically focused on first agreeing the outcome domains that need to be measured in all trials. Once these broad outcomes are agreed, then candidate outcome measures (such as patient reported measures or other tools) can be evaluated for validity and reliability.

So far, there are six core outcome sets in orthopaedics covering arthroplasty, osteoarthritis, hip fracture, shoulder disorders, Perthes disease and back pain with sets in development for disorders including childhood fractures, childhood elbow fractures, hip avascular necrosis.

Onto the trial

With a keystone question sorted, the next step (ignoring the getting it funded) is to design and write a study protocol, with technical considerations such as randomisation schedules and plans for data monitoring, statistical analysis, and health economics etc. For all of this, help is at hand. In the UK we have support from the regional NIHR funded Research Design Service, and the BOA funds access to experts in clinical trials units as part of the BOA Orthopaedic Surgery Research Centres and the BOA Surgical Specialty Leads for Clinical Trials (Dan Perry, Paul Baker and Xavier Griffin). These resources are available to support you in making the technical decisions, win funding and deliver a successful, awesome trial.

References

References can be found online at www.boa.ac.uk/publications/JTO.