HOW TO ESTABLISH AND EVALUATE CLINICAL PREDICTION MODELS
An Academic presentation by Dr. Nancy Agnes, Head, Technical Operations, Statswork Group
www.statswork.com
Email: info@statswork.com
TODAY'S DISCUSSION Outline Introduction Clinical Prediction Model Steps to establishing a clinical prediction model Clinical prediction models CODE Future Scope:
INTRODUCTION The use of a parametric/semi-parametric/nonparametric mathematical model to estimate the probability that a subject currently has a certain condition or the possibility of a certain outcome in the future is referred to as a clinical predictive model. Various regression analysis approaches are used to model clinical prediction models, and the statistical nature of regression analysis is to find "quantitative causality."
To put it another way, regression analysis is a quantitative assessment of how much X impacts Y. Multiple linear regression models, logistic regression models, and Cox regression models are all widely used approaches. The secret to s tatistical analysis, data modelling, and project design is assessing and verifying prediction models' efficacy. It is also the most difficult aspect of data analysis technology.
CLINICAL PREDICTIO N MODEL
A clinical prediction model is a tool used in healthcare to measure estimates of the likelihood of the future course of a specific patient outcome using multiple clinical or nonclinical predictors. A realistic checklist for developing a valid prediction model is presented in a clinical prediction model. A clinical prediction model can be used in various clinical contexts, including screening for asymptomatic illness, forecasting future events such as disease, and assisting doctors in their decision-making and health education. Contd...
Despite the positive effects of c linical prediction models on practice, prediction modelling is a difficult process that necessitates meticulous statistical analysis and sound clinical judgments.
Contd...
STEPS TO ESTABLISHIN G A CLINICAL PREDICTION MODEL
There exist several types of research detailing the methods to construct clinical prediction models. However, there is no proper method to construct the prediction model in medicine. The construction and evaluation of prediction models are classified into five steps.
Contd...
STEP 1: GATHERING THE IDEATIONS AND QUESTIONS FOR ENHANCING THE MODEL. It incorporates structuring the research questions, such as finding the target variable for predicting which age group of the targeted people you want to predict. etc. For instance, gathering one patient details and use it as a trained data set to test the other data set of another patient’s details. [1].
Contd...
STEP 2: SELECTION OF DATA Data collection is a vital part of statistical or clinical research. Nevertheless, the perfect data and a perfect model can't exist. It would be nice to look for the most appropriate. The primary dataset with the endpoint of the study and all key predictors may not be available at all the time. Secondary or administrative data sources are mandatory. Contd...
Based on the various data types of datasets, prediction models can be utilized. [2] For instance, the epidemiology study is based on the Data Mining systematic approach. STEP 3: WAYS TO HANDLE VARIABLES Most of the time, researchers may face challenging situations where the variables are highly correlated to each other, excluded in the study. Variables don't show statistical significance or the petite effect size. But it will contribute to the predictive model. Researchers will handle the missing data problems, categorical data, etc., before getting the interference. Contd...
CLINICAL PREDICTION MODELS CODE
Contd...
The Bayesian network was implemented to manipulate the independent variables of some diseases in the crucial stage of treatment. This model predicts and offers a way to handle the disease along with preventive measures [3].
Contd...
STEP 4: GENERATING MODEL There are no proper rules to select a particular model for the statistical analysis. There are some standard methods to build a model using L inearreg ression a nalysis, logistic regression analysis, and Cox models. Sometimes the clinical data encounters over-fitting of the model and its results in as estimates. This over-fitting issue can be detected using Akaike Information Criteria or Bayesian Information Criteria. Contd...
The smaller AIC and BIC values result in a good fit for the model. [4] Using Multivariate prediction models for analyzing the different characteristics of various patients.
Contd...
STEP 5: EVALUATION AND VALIDATION OF THE MODEL After building the model, it is necessary to evaluate and validate the predictive power of the model. The
key components that evaluate the model are calibration which plots the proportion, and discrimination classifies the events like success or
failure. There are two types of data validation, namely internal and external validation of the model. Internal validation evaluates the model within the data, whereas external validation can be done using the re-sampling technique, usually through bootstrapping. Contd...
It means you are creating or generating new data sets with similar characteristics to the original data and validating the study's method through the newly created or bootstrapped data. Further, there are several statistical measures to evaluate the model. Some of them are ROC curve, AUC curve, sensitivity and specificity, likelihood ratio, R square value, calibration plot, c-index, Hosmer-Lemeshow test, AIC, BIC, etc.
Contd...
Figure 1: Slope of Calibration plot – Source: Stevens and Poppe (2020) Contd...
Besides, Stevens and Poppe (2020) suggested the Cox- calibration slope using a l ogistic regression model instead of using the predictive model's calibration slope. This suggestion has been made after the scrutiny of around 33 research articles and found that most of the validation is external validation and identified the validity using the calibration slope.
Contd...
Contd...
Figure 2: This flow diagram illustrates the progress through the various phases of the CARDAMON phase II clinical trial, including the impact of COVID‐19 on the 70 patients on maintenance K across the two treatment arms at the start of the lockdown period. The 15 patients who stopped K maintenance joined the 170 patients who were already on long‐term follow‐up on 24 March 2020, bringing the number up to a total of 185. SCT, stem cell transplantation; K, carfilzomib; C, cyclophosphamide; d, dexamethasone [6].
Contd...
FUTUR E SCOP E
Based on the patient details, we can predict the further severe causation of disease in the future. By gathering the data from a single patient may help to predict other similar patients for better treatment. Big data support for manipulating vast amounts of clinical trials, without complexity simultaneously with high accuracy.
Contd...
TABLE 1 Concepts and Techniques of Clinical prediction models:
Contact Us UNITED KINGDOM +44-1143520021
INDIA +91-4448137070
EMAIL info@statswork.com