All Evidence Is Not Created Equal: What Makes Good Clinical Data?

Page 1

December - January 2023

All Evidence Is Not Created Equal: What Makes Good Clinical Data?

Editorial Summary

There were an estimated 3.8 million patients with a wound managed by the NHS in 2017/ 2018, of which 70% healed in the study year.1 The annual NHS cost of wound management was £8.3 billion, of which £2.7 billion and £5.6 billion were associated with managing healed and unhealed wounds, respectively.1 The annual prevalence of wounds increased by 71% between 2012/2013 and 2017/ 2018. This article explores clinical data in wound care.

Introduction

One cause of chronic wounds are diabetic foot ulcers (DFUs) - there were an estimated 326, 000 diabetic foot ulcers, which equates to 9% of all adult diabetic patients having a foot ulcer in the study year 2017/ 2018. Additionally, many of those patients with other causes of chronic wounds (e.g. non-healing venous leg ulcers) are also diabetic.1 In the United States, the total number of prevalent cases of DFU was 4,551,498 cases in the year 2020. Unhealed ulcers and foot infections are the leading cause of diabetes related amputations, with diabetic foot ulcers preceding 85% of amputations.2 In the United States, DFU patients are twice as costly to US Medicare as those with diabetes alone. The rate at which major amputations occur in a population with diabetes can be used as a good overall proxy measure of the effectiveness of health care and the foot care system for patients with diabetes.3 Survival rates have been found to be poor following a major amputation –the five year mortality for a diabetic patient following major amputation is 68% (compared to only 15% for breast cancer).4-5 There were 7,957 major lower-limb amputation procedures for patients with diabetes in England between 2017/ 2018 and 2019/ 2020 – this made up 51% of all lower limb amputations in England in this period.3 There were 21,738 minor lowerlimb amputation procedures for patients with diabetes in England between in the same period, with diabetic patients representing 69% of patients in this group.3

The cost of health care for ulceration and amputation in diabetes in 2014-2015 is

estimated at between £837 million and £962 million; 0.8% to 0.9% of the National Health Service (NHS) budget for England.6 More than 90% of expenditure was related to ulceration.6

How Is Evidence Graded?

The evidence pyramid is a useful visual representation of the internal validity of different study designs; designs of low internal validity are at the base of the pyramid and designs of high internal validity are at the top (Figure 1).6 While the evidence pyramid is a useful guide, it is important to recognise it has limitations.6 When considering evidence in the context of the evidence pyramid, it is important to consider the goal of the research project: to understand the effects of treatment where high internal validity is a key requirement or to seek to make new discoveries and find explanations for the causes of disease.6 Where the goal is to understand the disease aetiology, the traditional research pyramid may be reversed, with case reports and case series providing useful data to start an exploration of disease causation.6 This may be especially valid in the case of rare diseases or harms where there are few patients with the condition available for recruitment into high internal validity studies, such as randomised controlled trials (RCTs).6 However, the evidence pyramid provides a simple overview of study designs that may have high internal validity and, as such, may impact or change clinical practice where a clear relationship is found between a treatment and clinical outcome.6

16 Wound
Vol
Masterclass -
1 - December 2022
© Copyright. Wound Masterclass. 2023

Individual randomised controlled trials

Non-randomised controlled trials

Systematic reviews (with homogeneity) of cohort studies

Individual cohort studies, low-quality randomised controlled trials (e.g., <80% follow up) and non-comparative, uncontrolled studies

“Outcomes” research

Systematic reviews (with homogeneity) of case-control studies

Individual case-control studies

Case-series (and poor-quality cohort and case-control studies)

Expert opinion without explicit critical appraisal, or based on physiology, bench research or ‘first principles’

Randomised Controlled Trials

The design of the randomised controlled trial (RCT) compared to other research methodologies offers the clearest understanding of the relationship between an intervention and clinical outcome.6 The RCT, given its methodological rigour, is generally preferable over non-randomised or observational study designs, and forms the main source upon which systematic reviews of interventions are based.6

Conduct bias refers to methodological flaws in a study design and conduct that lead to bias6. While the RCT offers methodological rigour, failure to adhere to the study protocol can introduce bias and reduce the confidence of clinicians in the trial results and conclusions. It is generally believed that there are four key sources of bias that could be reduced by details of design of RCTs: selection, performance, attrition or exclusion and detection bias (Table 1).

Reporting bias describes the bias that arises due to selective reporting of only the statistically significant study findings.6 Reporting bias can occur if authors overemphasise differences of marginal statistical significance (with perhaps limited clinical significance) and/or positive results of secondary analyses. Adverse outcomes of an intervention may be reported selectively by researchers, which will also exaggerate the beneficial outcome of the intervention.6

Publication bias can occur when the outcome of a study influences the decision to publish and typically results in negative findings not being published as they are less likely to be of interest to journals seeking to maintain a high impact factor.6 There are three forms of negative findings:

• Conclusive negative results: derived from well designed and conducted studies that show clear evidence of a neutral or negative effect (i.e., intervention is as good as the control, or even less effective than the control)

Exploratory negative results: derived from well designed and conducted studies, with exploratory data analysis suggesting the intervention was less effective than the control

Inconclusive negative results: poorly designed and conducted study, which is often too small to show the effect of the intervention.6

What Makes for Good Clinical Data in Wound Care? Wound Masterclass - Vol 1 - December 2022 17 Systematic reviews and meta-analyses
LoE 1a LoE 1b LoE 1c LoE 2a LoE 2b LoE 2c LoE 3a LoE 3b LoE 4 LoE 5
Figure 1: Levels of Evidence for studies on therapy, prevention, aetiology and harm in wound care6
Type of bias Stage of RCT How it occurs Selection bias Group selection Randomisation and allocation to treatment groups are flawed Performance bias Exposed or not exposed to the intervention Blinding to treatment allocation does not occur A change in treatment occurs as current intervention is not considered to be working Attrition or exclusion bias Follow-up period Number of patients lost to follow up are high or different between the treatment arms Detection bias Assessment of outcomes Outcome in one treatment arm is measured in a different way to the other arm © Copyright. Wound Masterclass. 2023
Table 1: Sources of bias in RCTs6

Validity Scoring

Some organisations specialising in the promotion of evidence-based practice offer simple validity checklists to assess internal and external validity (e.g., in the UK, the Centre for Evidence Based Medicine, the Joanna Briggs Institute, and the Scottish Intercollegiate Guidelines Network).A group representing both the European Wound Management Association and the International Working Group on the Diabetic Foot published a 21-point score designed to assess the validity of intervention studies relating to diabetic foot ulcers (DFUs; Figure 2).6

When assessing a study about the prevention and management of diseases of the foot in diabetes, the participants need to be appropriately selected – i.e., the participants are patients with diabetes who are at risk of developing a diabetic foot ulcer or whose disease is complicated by a diabetic foot ulcer.7 If more than one foot ulcer is present, only one (a specified index ulcer) should be included per participant.7

The type of ulcer chosen should be appropriate for the type of intervention – for example some trials of new interventions are carried out on those with uncomplicated neuropathic ulcers, for which cheap and effective treatments already exist.7 New treatments should be targeted to those ulcers that have failed to heal despite administration of good standard care in expert centres.7 When an intervention is administered for the prevention or treatment of diabetic foot ulcers, it will inevitably be given in conjunction with other aspects of care, and these other components must be described.7 Many participants in studies about DFU are lost to follow up, and the higher the loss to follow-up, the greater the likelihood of bias in any observations made.7 There is no consensus on the rate of retention or attrition that is acceptable in this population, although a figure of <25% loss to follow up with an intervention phase of 20 weeks or more is generally

studied in the trial based on an appropriate sample size calculation?

9. Was the chosen primary outcome of direct clinical relevance?

10. Was the person who assessed the primary outcome or outcomes blinded to group allocation?

11. Was either the clinical researcher who cared for the wound at research visits or the participant blinded to group allocation?

Study conduct

12. Did the study complete recruitment?

13. Was it possible to document the primary outcome in 75% or more of those recruited?

14. Were the results analysed primarily by intention to treat?

15. Were the appropriate statistical methods used throughout?

Outcomes

16. Was the performance of the control group of the order that would be expected in routine clinical practice?

17. Are the results from all participating centres comparable?

Answer 'Yes' if the study was done in only one centre.

Rationale: study design

The intervention should be the only difference between study groups, there should be no difference between the baseline characteristics of the participants, other than those that may be the result of chance. It is also important that all participants otherwise receive defined good standard case. The importance of this is to ensure that any intervention being studied is the only difference between groups, which could account for any observed difference. The method of randomisation (ideally by an independent agency) should be described, together with a sample size calculation, blinding/ masking (especially of the outcome observer) and a choice of an outcome measure that is clinically relevant.

Rationale: study conduct

The four questions relate to completion of recruitment and follow-up, as well as to statistical analysis.

Rationale: study outcomes

Question 16 checks that the differences observed between groups are not the result of unusually poor performance in the control group, as has been the case in a number of published trials reporting apparent benefit of an intervention. For question 17, as many multicentre studies have a core of high-recruiting centres and a majority in which recruitment was either moderate or low, it is important to ensure that the aggregate outcomes are not dominated by performance in a small number of high recruiting centres. For example, if usual care is different in different centres, any benefit could be by chance, but if randomisation is drafted by centre, then this could have less of an influence. While this can be minimised by randomising seperately by study centre, this can increase the total number of participants needed.

Study reporting

18. Is the report free from errors of reporting, e.g. discrepancies between data reported in different parts of the same report?

19. Are the important strengths and weaknesses of the study discussed in a balanced way?

20. Are the conclusions supported by the findings?

21. Is the report free from any suggestion that the analyses or the conclusions could have been substantially influenced by people with commercial or other personal interests in the findings?

considered acceptable.7

Rationale: study reporting

The four questions are designed to explore the possibility of reporting bias. Questions 19 - 21 aim to expose aspects of the report that reflect intentional or unintentional choice of words, which could either exaggerate or obscure some aspects of the findings.

Some expensive interventions have been widely used as a result of studies that would now be viewed as flawed.7 In some cases, the apparent benefit was based on a significant difference from the comparator group when the difference could be accounted for by poor performance in the comparator group receiving usual care.7 Therefore the outcome in the comparator group must be scrutinised to check that performance is similar to that used as the basis for the sample size calculation.7

What Makes for Good Clinical Data in Wound Care? 18 Wound Masterclass - Vol 1 - December 2022
“When assessing a study about the prevention and management of diseases of the foot in diabetes, the participants need to be appropriately selected.”
Study design 1. Are adequate definitions includes for the terms 'ulcer', 'healing', and all other required aspects of the population and the outcomes? 2. Was the choice of study population appropriate for the chosen intervention and the stated outcomes? 3. Was the control population managed at the same time as those in the intervention group? 4. Is the intervention sufficiently well described to enable another researcher to replicate the study? 5. Are the components of other aspects of care described for the intervention and comparator groups? 6. Were the participants randomised into intervention and comparator groups? 7. Were the participants randomised by an independent person or agency? 8. Was the number of participants
Figure 2: Required rationale and markers of quality: the 21-point scoring system for reports on clinical trials for the prevention and management of diseases of the foot in patients with diabetes6,7
© Copyright. Wound Masterclass. 2023

Example: Leucopatch System for Management of Hard-To-Heal Diabetic Foot Ulcers

This was a multicentre, observer-blinded RCT.8 There was a 4-week run in period, and 326 patients (out of 595 consented) were excluded during that run-in period.8 The primary outcome was the proportion of ulcers that healed within 20 weeks assessed in the intention-to-treat population (all participants with post-randomisation data collected), defined as complete epithelialisation (confirmed by an observer who was masked to randomisation group), and remained healed for 4 weeks.8 The randomisation process (computer-generated, web-based) resulted in well matched groups (leucopatch + standard care v standard care) in terms of age, sex, T2DM, mean duration of diabetes, diabetes related complications, mean baseline haemoglobin, estimated GFR and foot ulcer characteristics and related complications (ABPI, loss of sensation at two or more sites, area of ulcer, depth of ulcer, affected foot position and type of offloading).8 The authors found a statistically significant effect of the intervention.8

This can be viewed as particularly valuable information for a number of reasons. The study population was well chosen. The ulcer must have been non-responsive to treatment for more than four weeks, as those whose ulcers healed in the run-in period were excluded. Grade 3 wounds and patients with a ABPI down to 0.5, indicating severe disease were included, indicating that this study chose the important group – those with hard to treat DFUs which are unlikely to respond to standard care that is already available, therefore targeting those most likely to benefit from novel treatment. Both groups received defined best standard of care including debridement, offloading, NPWT and protease inhibitors.

Groups exist to review evidence relating to the management of DFU. As a result of the Leucopatch II trial, the International Working Group for the Management of the Diabetic Foot Ulcer recommended that clinicians should

consider the use of autologous combined leucocyte, platelet and fibrin as an adjunctive treatment, in addition to best standard of care, in non-infected diabetic foot ulcers that are difficult to heal, but with a weak strength of recommendation and moderate grading of the quality of evidence to support this recommendation.9 They felt that whilst the quality of this one available study was strong, the lack of cost effectiveness, applicability in daily practice and the importantly, the absence of additional supportive studies meant that the strength of their recommendation was weak9. Similarly, the National Institute for Clinical Excellence, NICE, does not recommend their use in the NHS as there are uncertainties around whether the evidence would generalise to current NHS practive because of how and when the treatment would be used.10 Cost analysis also showed that the clinical benefits seen in the trial are unlikely to lead to a cost saving in practice.10

Summary

Chronic ulcers, particularly DFUs, are a major economic burden to the NHS and often lead to amputation which is associated with a high mortality. Therefore, when assessing evidence it is important to assess if that evidence is of high quality and can therefore be implemented in to practice. Randomised control trials are the best quality of evidence, after systematic reviews and meta-analyses in the pyramid of evidence. It is important to ensure than an RCT is free from conduct, reporting and publication bias and scoring systems have been developed to help assess this. Groups exist which assess these studies to enable them to make recommendations about clinical practice based on their assessment of the quality of the study conducted – it is therefore important for investigators to make their study design as free from bias as possible.

What Makes for Good Clinical Data in Wound Care? Wound Masterclass - Vol 1 - December 2022 19 © Copyright. Wound Masterclass. 2023
“When assessing evidence it is important to assess if that evidence is of high quality and can therefore be implemented in to practice.”

What Makes for Good Clinical Data in Wound Care?

1. Guest JF, Fuller GW, Vowden P. Cohort study evaluating the burden of wounds to the UK’s National Health Service in 2017/2018: update from 2012/2013. BMJ Open 2020;10:e045253.

doi: 10.1136/bmjopen-2020-045253

2. Edmonds M, Manu C, Vas P. The current burden of diabetic foot disease. J Clin Orthop Trauma. 2021 Jun; 17: 88–93.Published online 2021 Feb 8. doi: 10.1016/j.jcot.2021.01.017

3. Office for Health Disparities and Improvement (above): National Diabetes Foot Care Report. 2022.https://fingertips.phe.org.uk/static-reports/diabetes-footcare/national-diabeticfootcare-report.html

4. Icks A et al. Time-Dependent Impact of Diabetes on Mortality in Patients After Major Lower Extremity Amputation: Survival in a population-based 5-year cohort in Germany. Diabetes Care.34(6). 1 June 2011

5. Office of National Statistics: Cancer survival in England – adults diagnosed.12/8/19 https:// www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/ datasets/cancersurvivalratescancersurvivalinenglandadultsdiagnosed

6. World Union of Wound Healing Societies (2020) Evidence in wound care. London: Wounds International.

7. Jeffcoat W et al on behalf of the International Working Group on the Diabetic Foot and the European Wound Management Association.Reporting standards of studies and papers on the prevention and management of foot ulcers in diabetes: required details and markers of good quality.Lancet Diabetes Endocrinol 2016; 4: 781–88

8. Game F et al for the Leucopatch II trial team. Leucopatch system for the management of hard-to-heal diabetic foot ulcers in the UK, Denmark and Sweden: an observer-masked, randomised controlled trial. The Lancet Diabetes and Endocrinology. 6(11): 870-8. 1/11/18

9. IWGDF Guidelines on the Prevention and Management of Diabetic Foot Disease Chapter: IWGDF Guideline on interventions to enhance healing of foot ulcers in persons with diabeteshttps://iwgdfguidelines.org/wp-content/uploads/2021/03/IWGDF-2019-final.pdf

10. National Institute for Clinical Excellence. 3D patch for treating diabetic foot ulcers.

7/3/22 https://www.nice.org.uk/guidance/mtg66/chapter/1-Recommendations

20 Wound Masterclass - Vol 1 - December 2022
woundmasterclass.com/Register Register for full access to the journal, educational resources, information about upcoming events and more woundmasterclass.com © Copyright. Wound Masterclass. 2023 Image licenced from Adobe Stock. Credit: Dragana Gordic
References

Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.