OCT 10, 2019
Research paper
APPLICATION OF LOGISTIC REGRESSION USING R PROGRAMMING Experience of Statswork.com Tags: Statswork | Logistic regression | R programming | Python Expert | Programmers | Statistical Data Analysis | Data Analysis Services | Data Mining Services | Data Collection | Big Data Analytics | Statistics Services Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright © 2019 Statswork. All rights reserved
0 1 LOGISTIC REGRESSION ANALYSIS
0 2 0 3
Logistic regression is a type of predictive model when the target variable is a categorical variable with two categories.
Logistic regression is used for the prediction of the probability of occurrence of an event by fitting the data into a logistic curve.
It makes use of predictor variables either numerical or categorical.
Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright Š 2019 Statswork. All rights reserved
USES OF LOGISTIC REGRESSION ANALYSIS Predicts the presence or absence of a characteristic or outcome based on values 01 of a set of predictor variables.
02 Estimates odds ratios (OD) for each of the independent variables in the model.
Suitable for models where dependent variable is dichotomous.
03
04
Predicts the probability of occurrence of an event by fitting data to a logit function.
Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright Š 2019 Statswork. All rights reserved
EXAMPLE OF LOGISTIC REGRESSION ANALYSIS Question 1 Prediction among patient having high blood pressure (BP) or not along with other observations such as Age, Smoking habits, Weight, or Body mass Index BMI, blood cholesterol levels, Diastolic and Systolic BP value, Gender, etc. Table 1. Blood Pressure Range
B P
Low
Norma l
Borderlin e
High
Systoli c
<90
90130
131140
14 0
Diastoli c
<60
60180
8190
>90
•Source: Adapted from Nimmala et al., 2018 Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright © 2019 Statswork. All rights reserved
PROCEDURES INVOLVED IN CONDUCTING LOGISTIC REGRESSION ANALYSIS
01 02 03 04 05
Select the response or dependent variable (patients having high BP or not). Other variables are considered as explanatory or independent variables. Dependent variable needs to be coded as 0 and 1. Explanatory variable can be a continuous variable or ordinal variable. Outcome is predicted by applying logistic regression model.
RESULT Outcome is predicted by applying logistic regression model. Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright Š 2019 Statswork. All rights reserved
Question 2Prediction among child having high blood patient having high blood pressure (BP) or not along with other observations such as Age, Smoking habits, Weight, or Body mass Index BMI, blood cholesterol levels, Diastolic and Systolic BP value, Gender, etc.
Table 1. Prediction model for childhood high blood pressure
Adapted from Hamoen et al., 2018
Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright Š 2019 Statswork. All rights reserved
LOGISTIC REGRESSION IMPLEMENTATION USING LOGIT FUNCTION AND R PROGRAMMING Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright © 2019 Statswork. All rights reserved
LOGISTIC REGRESSION IMPLEMENTATION USING LOGIT FUNCTION Logistic regression generates the coefficients of a formula to predict a logit transformation of the probability of the presence of the characteristic of interest: Logit (p) = b0 + b1X1 + b2X2 +…+ bkXk where p- probability of the presence of the characteristic of interest. The logit transformation is defined as the logged odds:
Odds = p = probability of prthe esence of the characteristic 1-p probability of absence of the characteristic & Logit (p) = log (p/(1-p)) = log (p) – log (1-p) Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright © 2019 Statswork. All rights reserved
LOGISTIC REGRESSION IMPLEMENTATION USING R PROGRAMMING
Fitting of binary logistic model for the Titanic dataset that is available in Kaggle. Procedure • Frame the objective of the study as survival variable: 1 - survived, 0 - not survived and other - independent variables. • Loading the training data into the console using the function read.csv (). train.data<- read.csv('train.csv',header=T,na.strings=c("")) • Check missing data before fitting using sapply() function in R. sapply(train.data,function(x) sum(is.na(x))) •
Check other missing entries in the subset data. There are different ways to replace the NA’s with either the mean or median of the data data$Age[is.na(data$Age)] <- mean(data$Age,na.rm=T)
Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
LOGISTIC REGRESSION IN R PROGRAMMING
R is an easier platform to fit a logistic regression model using the function glm(). Question
Copyright © 2019 Statswork. All rights reserved
PassengerId
Survived
Pclass
Name
Sex
0
0
0
0
0
Age
SibSp
Parch
177
0
0
Cabin
Embarked
687
2
Ticket 0
Fare 0
“Cabin” and “PassengerId” variables are missing which are skipped and made as subset of the data as new with the subset () function. data <- subset(train.data,select=c(2,3,5,6,7,8,10,12)) where the numeric values is the columns in the data file.
Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright © 2019 Statswork. All rights reserved
FITTING LOGISTIC REGRESSION MODEL
•
Before fitting, Split the data into two sets: training set ( to fit the model) testing set (for testing). traindata<- data[1:800,] testdata<- data[801:889,]
•
Specify family = binomial in glm() function since our response is binary. lrmodel<- glm(Survived ~.,family=binomial(link='logit'),data=traindata)
•
The result of the model can be obtained using the following command: summary(lrmodel)
Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright © 2019 Statswork. All rights reserved
RESULT INTERPRETATION: From the p-value, • “Embarked”, “Fare”, and “SibSp” variables- not significant. • “sex “variable has lesser p-value so there is a strong association of passengers with the chance of survival. It is important to note that the response variable is log odds ln(odds) = ln(p/(1-p)) = a*x1 + b*x2 + … + z*xn. To analyse the deviance, use anova() function in R. anova(lrmodel, test="Chisq") Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright © 2019 Statswork. All rights reserved
•
Wider difference between the null deviance and the residual deviance indicates better model performance.
•
Large p-value indicates that the logistic regression model without that particular variable explains the same amount of variation.
Hence found that Akaike Information Criterion (AIC) is the best model.
Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright © 2019 Statswork. All rights reserved
PREDICT THE MODEL ON A NEW SET OF DATA R enables us to do with predict() function. Let threshold for the new response data be P(y=1|X) > 0.5 then y = 1 otherwise y=0. The threshold changes according to the researchers needs. The following Table presents the accuracy prediction formulas especially when machine learning algorithms are applied.
Table 4. Measures and formula fit.results<predict(lrmodel,newdata=subset(test,select=c( 2,3,4,5,6,7,8)),type='response') fit.results<- ifelse(fit.results> 0.5,1,0) Error<- mean(fit.results != test$Survived) print(paste('Accuracy',1-Error))
Contd..
Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright Š 2019 Statswork. All rights reserved
"Accuracy 0.842696629213483.“ •
Accuracy= 84.26% (here data is split manually).
Finally, we find the area under the curve by plotting Receiver Operating Characteristic (ROC) curve using the following commands: library(ROCR) A<-predict(lrmodel,newdata=subset(test,select=c(2,3,4,5,6,7,8)), type="response") Pre<- prediction (A, test$Survived) Pre1<- performance(Pre, measure = "tpr", x.measure = "fpr") plot(Pre1) auc<- performance(Pre, measure = "auc") auc<- auc@y.values[[1]] auc
0.8647186 Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright © 2019 Statswork. All rights reserved
RESULTS Since Area Under the Curve (AUC) is closer to 1, then this is an good predicting model.
ROC curve generates the true positive rate against false positive rate, similar to the sensitivity and specificity.
Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright Š 2019 Statswork. All rights reserved
Statswork Lab @ Statswork.com www.statswork.com
Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright Š 2019 Statswork. All rights reserved
PHONE NUMBER
UK
: +44-1143520021
INDIA
: +91-4448137070
EMAIL ADDRESS info@statswork.com
GET IN TOUCH WITH US
Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics
Copyright © 2019 Statswork. All rights reserved