Chapter 9 Correlation and Regression
1
Chapter Outline • • • •
9.1 Correlation 9.2 Linear Regression 9.3 Measures of Regression and Prediction Intervals 9.4 Multiple Regression
2
Section 9.1 Correlation
3
Correlation Correlation • A relationship between two variables. • The data can be represented by ordered pairs (x, y) x is the independent (or explanatory) variable y is the dependent (or response) variable
4
Correlation A scatter plot can be used to determine whether a linear (straight line) correlation exists between two variables. y Example: x 1 2 3 y –4 –2 –1
2
4 0
5 2
x 2
4
6
–2
–4
5
Types of Correlation y
y
As x increases, y tends to decrease.
As x increases, y tends to increase. x
Negative Linear Correlation y
Positive Linear Correlation y
x
No Correlation
x
x
Nonlinear Correlation 6
Example: Constructing a Scatter Plot A marketing manager conducted a Advertising Company sales study to determine whether there is expenses, ($1000), x ($1000), y a linear relationship between 2.4 225 money spent on advertising and 1.6 184 company sales. The data are shown 2.0 220 in the table. Display the data in a 2.6 240 scatter plot and determine whether 1.4 180 there appears to be a positive or 1.6 184 negative linear correlation or no 2.0 186 linear correlation. 2.2 215 7
Solution: Constructing a Scatter Plot Company sales (in thousands of dollars)
y
x Advertising expenses (in thousands of dollars)
Appears to be a positive linear correlation. As the advertising expenses increase, the sales tend to increase. 8
Example: Constructing a Scatter Plot Using Technology Old Faithful, located in Yellowstone National Park, is the world’s most famous geyser. The duration (in minutes) of several of Old Faithful’s eruptions and the times (in minutes) until the next eruption are shown in the table. Using Excel\SPSS , display the data in a scatter plot. Determine the type of correlation.
Duration
Time,
Duration
Time,
x
y
x
y
1.8
56
3.78
79
1.82
58
3.83
85
1.9
62
3.88
80
1.93
56
4.1
89
1.98
57
4.27
90
2.05
57
4.3
89
2.13
60
4.43
89
2.3
57
4.47
86
2.37
61
4.53
89
2.82
73
4.55
86
3.13
76
4.6
92
3.27
77
4.63
91
3.65
77
9
Correlation Coefficient Correlation coefficient • A measure of the strength and the direction of a linear relationship between two variables. • The symbol r represents the sample correlation coefficient. • A formula for r is r
n xy x y
n x 2 x
2
n y 2 y
n is the number 2
of data pairs
• The population correlation coefficient is represented by ρ (rho). 10
Correlation Coefficient • The range of the correlation coefficient is -1 to 1. -1 If r = -1 there is a perfect negative correlation
0 If r is close to 0 there is no linear correlation
1 If r = 1 there is a perfect positive correlation
11
Linear Correlation y
y
r = ď€0.91
r = 0.88
x
Strong negative correlation y
x
Strong positive correlation y
r = 0.42
x
Weak positive correlation
r = 0.07
x
Nonlinear Correlation 12
Calculating a Correlation Coefficient In Words 1. Find the sum of the xvalues. 2. Find the sum of the yvalues. 3. Multiply each x-value by its corresponding y-value and find the sum.
In Symbols x y
xy
13
Calculating a Correlation Coefficient In Words
In Symbols 2 x
4. Square each x-value and find the sum.
y2
5. Square each y-value and find the sum. 6. Use these five sums to calculate the correlation coefficient.
r
n xy x y n x 2 x
2
n y 2 y
2
14
Example: Finding the Correlation Coefficient Calculate the correlation coefficient Advertising Company for the advertising expenditures and expenses, sales ($1000), x ($1000), y company sales data. What can you 2.4 225 conclude? 1.6 2.0 2.6 1.4 1.6 2.0 2.2
184 220 240 180 184 186 215 15
Solution: Finding the Correlation Coefficient x
y
2.4 1.6 2.0 2.6 1.4 1.6 2.0 2.2 Σx = 15.8
225 184 220 240 180 184 186 215 Σy = 1634
xy
x2
y2
540 5.76 50,625 294.4 2.56 33,856 440 4 48,400 624 6.76 57,600 252 1.96 32,400 294.4 2.56 33,856 372 4 34,596 473 4.84 46,225 Σxy = 3289.8 Σx2 = 32.44 Σy2 = 337,558
16
Solution: Finding the Correlation Coefficient Σx = 15.8
r
Σy = 1634 Σxy = 3289.8 Σx2 = 32.44 Σy2 = 337,558
n xy x y n x x 2
2
n y y 2
2
8(3289.8) 15.81634 8(32.44) 15.82 8(337, 558) 1634 2 501.2 0.9129 9.88 30, 508
r ≈ 0.913 suggests a strong positive linear correlation. As the amount spent on advertising increases, the company sales also increase. 17
Example: Using Technology to Find a Correlation Coefficient Use a technology tool to calculate the correlation coefficient for the Old Faithful data. What can you conclude?
Duration
Time,
Duration
Time,
x
y
x
y
1.8
56
3.78
79
1.82
58
3.83
85
1.9
62
3.88
80
1.93
56
4.1
89
1.98
57
4.27
90
2.05
57
4.3
89
2.13
60
4.43
89
2.3
57
4.47
86
2.37
61
4.53
89
2.82
73
4.55
86
3.13
76
4.6
92
3.27
77
4.63
91
3.65
77
18
Solution: Using Technology to Find a Correlation Coefficient
r ≈ 0.979 suggests a strong positive correlation.
19
Correlation and Causation • The fact that two variables are strongly correlated does not in itself imply a cause-and-effect relationship between the variables. • If there is a significant correlation between two variables, you should consider the following possibilities. 1. Is there a direct cause-and-effect relationship between the variables? • Does x cause y? 20
Correlation and Causation 2. Is there a reverse cause-and-effect relationship between the variables? • Does y cause x? 3. Is it possible that the relationship between the variables can be caused by a third variable or by a combination of several other variables? 4. Is it possible that the relationship between two variables may be a coincidence?
21
Section 7.2 Linear Regression
22
Regression lines • After verifying that the linear correlation between two variables is significant, next we determine the equation of the line that best models the data (regression line). • Can be used to predict the value of y for a given value of x. y
x 23
Residuals Residual • The difference between the observed y-value and the predicted y-value for a given x-value on the line. For a given x-value, di = (observed y-value) – (predicted y-value) y
Observed y-value d3{ }d1
}d 2
d4
{
d6{
}d5
Predicted y-value
x 24
Regression Line Regression line (line of best fit) • The line for which the sum of the squares of the residuals is a minimum. • The equation of a regression line for an independent variable x and a dependent variable y is ŷ = mx + b y-intercept Predicted y-value for a given xvalue
Slope
25
The Equation of a Regression Line • ŷ = mx + b where n xy x y m 2 n x 2 x
y x b y mx m n n
• y is the mean of the y-values in the data • x is the mean of the x-values in the data • The regression line always passes through the point x, y 26
Example: Finding the Equation of a Regression Line Find the equation of the regression Advertising Company line for the advertising expenditures expenses, sales ($1000), x ($1000), y and company sales data. 2.4 1.6 2.0 2.6 1.4 1.6 2.0 2.2
225 184 220 240 180 184 186 215 27
Solution: Finding the Equation of a Regression Line Recall from section 9.1: x
y
2.4 1.6 2.0 2.6 1.4 1.6 2.0 2.2 Σx = 15.8
225 184 220 240 180 184 186 215 Σy = 1634
xy
x2
y2
540 5.76 50,625 294.4 2.56 33,856 440 4 48,400 624 6.76 57,600 252 1.96 32,400 294.4 2.56 33,856 372 4 34,596 473 4.84 46,225 Σxy = 3289.8 Σx2 = 32.44 Σy2 = 337,558 28
Solution: Finding the Equation of a Regression Line Σx = 15.8
Σy = 1634
Σxy = 3289.8 Σx2 = 32.44 Σy2 = 337,558
n xy x y 8(3289.8) (15.8)(1634) m 2 2 2 8(32.44) 15.8 n x x 501.2 50.72874 9.88
1634 15.8 b y mx 8 (50.72874) 8 204.25 (50.72874)(1.975) 104.0607
Equation of the regression line yˆ 50.729 x 104.061 29
Solution: Finding the Equation of a Regression Line
Company sales (in thousands of dollars)
• To sketch the regression line, use any two x-values within the range of the data and calculate the corresponding yvalues from the regression line. y
260
240
yˆ 50.729 x 104.061
220
200
180
160 1.2
1.4
1.6
1.8
2
2.2
2.4
2.6
x
2.8
Advertising expenses (in thousands of dollars) 30
Example: Using Technology to Find a Regression Equation Use a technology tool to find the equation of the regression line for the Old Faithful data.
Duration
Time,
Duration
Time,
x
y
x
y
1.8
56
3.78
79
1.82
58
3.83
85
1.9
62
3.88
80
1.93
56
4.1
89
1.98
57
4.27
90
2.05
57
4.3
89
2.13
60
4.43
89
2.3
57
4.47
86
2.37
61
4.53
89
2.82
73
4.55
86
3.13
76
4.6
92
3.27
77
4.63
91
3.65
77
31
Solution: Using Technology to Find a Regression Equation
100
50
1
5
32
Example: Predicting y-Values Using Regression Equations The regression equation for the advertising expenses (in thousands of dollars) and company sales (in thousands of dollars) data is 天 = 50.729x + 104.061. Use this equation to predict the expected company sales for the following advertising expenses. (Recall from section 9.1 that x and y have a significant linear correlation.) 1. 1.5 thousand dollars 2. 1.8 thousand dollars 3. 2.5 thousand dollars 33
Solution: Predicting y-Values Using Regression Equations ŷ = 50.729x + 104.061 1. 1.5 thousand dollars ŷ =50.729(1.5) + 104.061 ≈ 180.155 When the advertising expenses are $1500, the company sales are about $180,155. 2. 1.8 thousand dollars ŷ =50.729(1.8) + 104.061 ≈ 195.373 When the advertising expenses are $1800, the company sales are about $195,373. 34
Solution: Predicting y-Values Using Regression Equations 3. 2.5 thousand dollars š =50.729(2.5) + 104.061 ≈ 230.884 When the advertising expenses are $2500, the company sales are about $230,884. Prediction values are meaningful only for x-values in (or close to) the range of the data. The x-values in the original data set range from 1.4 to 2.6. So, it would not be appropriate to use the regression line to predict company sales for advertising expenditures such as 0.5 ($500) or 5.0 ($5000). 35
Section 7.3 Measures of Regression and Prediction Intervals
36
Variation About a Regression Line • Three types of variation about a regression line Total variation Explained variation Unexplained variation • To find the total variation, you must first calculate The total deviation The explained deviation The unexplained deviation 37
Variation About a Regression Line Total Deviation = yi y Explained Deviation = yˆi y Unexplained Deviation = yi yˆi y
(xi, yi)
Total deviation yi y
y
Unexplained deviation
yi yˆi
(xi, ŷi) (xi, yi)
x
Explained deviation yˆi y x 38
Variation About a Regression Line Total variation • The sum of the squares of the differences between the y-value of each ordered pair and the mean of y. Total variation = yi y
2
Explained variation • The sum of the squares of the differences between each predicted y-value and the mean of y. Explained variation = yˆi y
2
39
Variation About a Regression Line Unexplained variation • The sum of the squares of the differences between the y-value of each ordered pair and each corresponding predicted y-value.
Unexplained variation = yi yˆi
2
The sum of the explained and unexplained variation is equal to the total variation. Total variation = Explained variation + Unexplained variation 40
Coefficient of Determination Coefficient of determination • The ratio of the explained variation to the total variation. • Denoted by r2
Explained variation r Total variation 2
41
Example: Coefficient of Determination The correlation coefficient for the advertising expenses and company sales data as calculated in Section 9.1 is r ≈ 0.913. Find the coefficient of determination. What does this tell you about the explained variation of the data about the regression line? About the unexplained variation? Solution: r 2 (0.913) 2 0.834 About 83.4% of the variation in the company sales can be explained by the variation in the advertising expenditures. About 16.9% of the variation is unexplained. 42
The Standard Error of Estimate Standard error of estimate • The standard deviation of the observed yi -values about the predicted ŷ-value for a given xi -value. • Denoted by se.
( yi yˆi)2 n is the number of ordered pairs se in the data set n2 • The closer the observed y-values are to the predicted y-values, the smaller the standard error of estimate will be. 43
The Standard Error of Estimate In Words 1. Make a table that includes the column heading shown. 2. Use the regression equation to calculate the predicted y-values. 3. Calculate the sum of the squares of the differences between each observed y-value and the corresponding predicted y-value. 4. Find the standard error of estimate.
In Symbols xi, yi, yˆi, ( yi yˆi ), ( yi yˆi )2
yˆ mxi b ( yi yˆi )2
( yi yˆi)2 se n2 44
Example: Standard Error of Estimate The regression equation for the advertising expenses and company sales data as calculated in section 9.2 is 天 = 50.729x + 104.061 Find the standard error of estimate. Solution: Use a table to calculate the sum of the squared differences of each observed y-value and the corresponding predicted y-value.
45
Solution: Standard Error of Estimate x
y
ŷi
2.4 1.6 2.0 2.6 1.4 1.6 2.0 2.2
225 184 220 240 180 184 186 215
225.81 185.23 205.52 235.96 175.08 185.23 205.52 215.66
(yi – ŷ i)2 (225 – 225.81)2 = 0.6561 (184 – 185.23)2 = 1.5129 (220 – 205.52)2 = 209.6704 (240 – 235.96)2 = 16.3216 (180 – 175.08)2 = 24.2064 (184 – 185.23)2 = 1.5129 (186 – 205.52)2 = 381.0304 (215 – 215.66)2 = 0.4356 Σ = 635.3463
unexplained variation 46
Solution: Standard Error of Estimate • n = 8, Σ(yi – ŷ i)2 = 635.3463 ( yi yˆi)2 se n2
635.3463 10.290 82
The standard error of estimate of the company sales for a specific advertising expense is about $10.29.
47
Section 9.4 Multiple Regression
48
Multiple Regression Equation • In many instances, a better prediction can be found for a dependent (response) variable by using more than one independent (explanatory) variable. • For example, a more accurate prediction for the company sales discussed in previous sections might be made by considering the number of employees on the sales staff as well as the advertising expenses.
49
Multiple Regression Equation Multiple regression equation • ŷ = b + m1x1 + m2x2 + m3x3 + … + mkxk • x1, x2, x3,…, xk are independent variables • b is the y-intercept • y is the dependent variable * Because the mathematics associated with this concept is complicated, technology is generally used to calculate the multiple regression equation. 50
Example: Finding a Multiple Regression Equation A researcher wants to determine how employee salaries at a certain company are related to the length of employment, previous experience, and education. The researcher selects eight employees from the company and obtains the data shown on the next slide. Use Minitab \SPSS\Excel to find a multiple regression equation that models the data.
51
Example: Finding a Multiple Regression Equation Employment Experience Education Employee Salary, y (yrs), x1 (yrs), x2 (yrs), x3 A 57,310 10 2 16 B 57,380 5 6 16 C 54,135 3 1 12 D 56,985 6 5 14 E 58,715 8 8 16 F 60,620 20 0 12 G 59,200 8 4 18 H 60,320 14 6 17 52
Solution: Finding a Multiple Regression Equation • Enter the y-values in C1 and the x1-, x2-, and x3values in C2, C3 and C4 respectively. • Select “Regression > Regression…” from the Stat menu. • Use the salaries as the response variable and the remaining data as the predictors.
53
Solution: Finding a Multiple Regression Equation
The regression equation is 天 = 49,764 + 364x1 + 228x2 + 267x3 54
Predicting y-Values • After finding the equation of the multiple regression line, you can use the equation to predict y-values over the range of the data. • To predict y-values, substitute the given value for each independent variable into the equation, then calculate ŷ.
55
Example: Predicting y-Values Use the regression equation ŷ = 49,764 + 364x1 + 228x2 + 267x3 to predict an employee’s salary given 12 years of current employment, 5 years of experience, and 16 years of education. Solution: ŷ = 49,764 + 364(12) + 228(5) + 267(16) = 59,544 The employee’s predicted salary is $59,544. 56