How should we introduce new technologies into surgical practice?

Part 1: The challenges of how we innovate now

Chetan Khatri, Steve Gwilym, Keith Tucker, Edward T Davis, Xavier L Griffin, David Beard and Andrew Metcalfe

Some new surgical technologies and procedures continue to revolutionise outcomes for patients, as they have done throughout the history of surgery [1]. There are many examples of innovations which have led to a vast improvement in the quality and length of peoples’ lives [2,3]. As a result, innovation and new technology often feels inevitable and desirable, with all new procedures feeling like progress towards better care.

This two-part series sets out the contemporary challenges and opportunities for surgical innovation in the UK. The first part describes the current processes and potential pitfalls of current device regulation, and the limitations in our ability to appraise these innovations. It describes the potential risks this poses to patients and healthcare funders. Furthermore, we discuss why case series and cohort data should not be relied upon by surgeons seeking to introduce new techniques into their practice. The second part (published in the next issue) describes potential solutions to this problem, including a national registries programme, initiatives by the orthopaedic community and the role of randomised trials in this critical and valuable area of orthopaedics.

Innovation has many dangers, and problems often arise that we did not anticipate. There have been well-documented examples of orthopaedic innovations that have caused unexpected harm. By their nature, these events are unpredictable, with new problems potentially representing ‘unknown unknowns’.

We may not feel that we can consent patients for these risks. Consent should include all material risks, whereby it is considered that a new procedure or device has an unknown safety profile. Experience tells us that undertaking new procedures may bring serious yet unanticipated problems, and as surgeons, it would be wise to be transparent about this.

While technological development causing harm is the most commonly cited concern, there is a much greater risk that many simply don’t infer benefit yet cost more. In this context, they represent poor value for money. Scrutiny of value is currently poorly conducted in healthcare generally and at the spearhead of medical innovation specifically. With a fixed budget healthcare system, investing in poor value healthcare in one area necessarily removes finance from another.

While we learn important lessons from each event, we also need to learn wider lessons about innovation in general. Each innovation cycle may bring benefits for patients, but also additional risk. Therefore, it is important that we consider as a community how we can protect patients while continuing to improve the treatments we offer them. The aim of this article is to explore how we currently innovate, how practicing surgeons can innovate safely and in the second part of this series, where we as a community might improve for the future.

What is an innovative procedure?

An innovative procedure has been described as ‘a new or modified surgical procedure that differs from currently accepted local practice, the outcomes of which have not yet been described and which may entail risk to the patient’ [4]. Surgeons are intuitively innovators, constantly seeking ways to improve the care they provide. Sometimes, this might be a stepchange, such as a new implant or procedure. In other cases, innovation involves an iterative adaption or modification of a procedure, where a clear point of change is difficult to pinpoint. For example, we may not appreciate that a software update can represent a major innovation. This grey zone of innovation is difficult to assess, but patients may still be exposed to uncertainties and risk.

The introduction of a new implant or a new instrument are more easily defined and understood. The introduction of the total hip replacement has transformed the lives of those with degeneration of the hip. As such, it has been labelled as the operation of the 20th century [2]. Subsequently there were serial modifications of the low friction arthroplasty without simultaneous rigorous evaluation, with disastrous results of high failure rates, such as the 3M capital hip replacement [5]. At the time, without a centralised, co-ordinated, monitoring programme, a call to mandate a national joint registry was triggered across the medical profession [6].

Anterior cruciate ligament reconstruction has been shown robustly to improve patient reported functional outcomes, compared with rehabilitation alone [7]. In the search of the optimum graft, surgeons looked to synthetic options. While early results supported their use in the short to medium term [8], further evidence emerged to suggest a higher-than-expected failure rate [9]. Within foot and ankle surgery, a hemiarthroplasty for the first metatarsal-phalangeal joint was developed to have the biomechanical properties of healthy cartilage. It promised a solution to hallux rigidus. However, its early evidence showed a higher-than-expected failure rate [10].

These examples describe only a fraction of the increasing problem with device safety [11]. Between 2006 and 2010, the Medicines and Healthcare products Regulatory Agency (MHRA) issued 2,124 safety notices, representing an 1,220% increase in volume of reports over a five-year period [12].

How are devices regulated?

The majority of devices and implants are developed within the United States, the UK and the European Union. Within the United States, there are two main routes for approval. The 510(k) process allows devices to be approved if they show equivalence to an existing product. In this scenario, clinical trials are not required. However, the definition of equivalence can be interpreted loosely, although this criterion is likely to change [13]. The second, more stringent method, Pre-Market Approval, requires controlled trials. However, of the devices approved, fewer than one-third of devices have undergone randomised trials and only half the trials included a control group [14]. The obvious brake that randomised trials apply to innovation are the costs associated with running a trial and the timeline to generate a meaningful result. Furthermore, clinical trials that focus on efficacy are also almost always underpowered to detect harm.

Until recently, both the UK and European Union relied upon on the Conformité Européenne (CE) approval and this process has often been regarded as a lighter touch than FDA Pre-Market Approvals, meaning many enterprises would seek CE approval well before costly approvals in the United States. For example, EU approval for a guidewire used in angioplasty required a 22-patient cohort study, while the FDA regulators required an 800-patient multicentre randomised controlled trial [15]. Carl Heneghan, a professor of evidence-based medicine, exposed a number of weaknesses in the CE marking system by gaining approval for mesh bag used to carry fruit to be implanted in bodies [16]. The regulatory environment in the EU and UK is in flux; the European Union has just introduced the new Medical Devices Regulations and we in the UK, the Medicines and Medical Devices Act 2021. The details of how these will impact orthopaedics are only just being understood. This may be particularly challenging for smaller enterprises or for devices with low usage (such as for rare diseases) where the costs of meeting the new requirements may outweigh the financial rewards of licencing them.

While regulation is an important step in maintaining safety, it represents a low bar for determining patient safety and can be slow to respond to emerging data and experience. If we are going to effectively prevent harm from new innovations, we need to go beyond this to understand how we can use outcomes and data to protect our patients.

Interpreting early data

Historically, surgeons innovated in using ‘see one, do one, teach one’ principles, with successes or failures reported only as anecdotes or personal experience. The essential need for data on all patients undergoing innovation is now wellunderstood. The Cumberlege report – written in response to several medical and surgical scandals including the problems of trans-vaginal mesh implants – was clear that we should all be engaging with data collection and reporting of outcomes [17]. Failing to collect or report data means that problems cannot be identified, and patients are exposed to harm in each new centre that takes on a new technique.

Traditionally, surgeons have reported clinical outcomes of innovative approaches as caseseries. While such papers often provide early data on an intervention, suboptimal outcomes may not be published for fear of personal or department reputation, leading to mistakes being repeated elsewhere. They are also inherently flawed in their ability to determine risks, the effectiveness of procedures and value to the healthcare system.

With many musculoskeletal conditions, patient reported outcome measures (PROMs) improve over time, irrespective of treatment allocated to them. The natural history of PROMs to improve over time is a remarkably common feature of outcomes in orthopaedics, with good evidence that this happens in rotator cuff tears, back pain, meniscal tears and osteoarthritis, a manifestation of the principle of regression to the mean [18-22]. This also leads to false reassurance when assessing case reports, as an improvement in PROMs is almost inevitable and does not necessarily mean that the operation itself was of benefit.

A recent example of how this can mislead is the subacromial spacer balloon, where multiple case reports showed improvements from baseline and were reported as showing evidence that the device was effective. However, such improvements over time are typical for many treatments for rotator cuff disease [19]. A subsequent randomised trial found worse results when the balloon was used compared to when it was not [23]. Females had particularly poor results, something that had not been identified in case series. A reliance on early case series data resulted in 30,000 patients across Europe receiving a device which was shown to have no meaningful benefit, and possible harm, in subsequent robust studies.

Case series may have a role in the early stages of a new innovation [24,25] and can be used to describe the refinement of a technique or explore the indications for use of a technique in development centres. They may be considered proof-of-concept records, describing innovation. Larger case series may provide some understanding of risks, although effective registries are much better for this due to the scale they can achieve. However, the greatest risk to case series data may be that they encourage others to adopt similar practice and provide false reassurance, potentially multiplying harm, with a cessation of the interrogation of the innovation so that it is considered ‘standard practice’.

Reducing our reliance on case series reports is a major challenge to surgeons. However, with increasing sophistication in our methods for collecting data routinely, we can innovate safely. In part two of this article, we will explore how routine and collaborative data collection, registries and trials can all be used to innovate effectively and safely.

Summary

While surgeons are constantly seeking to embrace new technology or devices that may improve the health and function of the people they serve, they should be aware of the potential risks this innovation may also bring. Stifling innovation is undesirable. However, without careful consideration and evaluation of new interventions, the next disaster is a case of when, not if.

Before adopting a new device, clinicians and health funders should be cautious of the regulatory approvals and evidence base behind it. Case-series or cohort data are appealing, but have many hidden limitations. Improvements in PROMs may provide false reassurance that an operation is effective and poor results may not be reliably reported.

We need better systems and studies before routinely adopting new devices and techniques into practice. The second part in this series goes on to describe ways in which surgical communities are developing systems to safely innovate through the appropriate use of registries, routine data collection and randomised trials, and ways we may continue to develop in the future.

References

References can be found online at www.boa.ac.uk/publications/JTO.