Cmjv02i01p0083 by Journal of Computer and Mathematical Sciences

J. Comp. & Math. Sci. Vol.2 (1), 83-91 (2011)

Efficiency Properties of the Stein-Minimax Estimators under Actual Value Prediction MANOJ KUMAR1, R. TIWARI2, RASPAL SINGH3 and SAROJ TIWARI4 1

Deptt. of Statistics, Panjab University, Chandigarh – 14, India e-mail:man_tiwa@yahoo.com 2 Deptt. of Statistics, Central University of Bihar, BIT Campus, PO : BV College, Patna – 14, India e-mail:rjtiwari@yahoo.com 3-4 Deptt. of Statistics, University of Jammu, Jammu-180006, India ABSTRACT In this paper we formulate Stein–minimax type estimators and compare the performance properties with other estimators in case of actual value predictions. When the model is estimated by ordinary least squares it has been observed that least squares predicted is unbiased while minimax and Stein-minimax predictors are biased. The superiority conditions of the estimators have been derived by assuming error distribution to be non-normal. Keywords: Linear regression model, Minimax estimator, Stein-minimax estimator, Actual value prediction.

1. INTRODUCTION Prediction is an important aspect of relationship analysis in any scientific study. It not only reflects the adequacy of underlying model but also assist in making a suitable choice among the competing models. In the context of linear regression models, the predictions are traditionally obtained either for the average values or for actual values of the study variable. Section-2 describes the model specification and the predictors. Section-3 deals with the properties of the estimators. We have derived the expression for the prediction of actual value of the study variable for

minimax and Stein-minimax estimator separately and their results are presented in the form of theorems in Section-4. In Section-5 comparative study have been made. Proof of the theorems are given in Section-6. 2. MODEL SPECIFICATION AND ESTIMATORS Let the true linear regression model is (2.1) Y = Xβ + σ U where y is a (n × 1) vector of dependent

variables X is a (n × p ) matrix of independent variables with full column rank,

Journal of Computer and Mathematical Sciences Vol. 2, Issue 1, 28 February, 2011 Pages (1-169)

Manoj Kumar, et al., J. Comp. & Math. Sci. Vol.2 (1), 83-91 (2011)

a (P × 1) vector of regression coefficients and σ (> 0 ) is an unknown scalar and U is non-stochastic vector with mean zero and variance 1 and measure of skewness and kurtosis are γ 1 and (γ 2 + 3) respectively. Application of least squares to (2.1) yields the ordinary least squares estimator as

β is

(

)

−1

b0 = X X X Y (2.2) which is well known to be the best linear unbiased estimator of β , having variancecovariance matrix. 1

(

)

V(b 0 ) = σ X X (2.3) The Stein rule estimator of the regression co-efficient from (2.1) is given by (2.4) 2

(

)

(2.7)

b* = β + σ ( X ′W −1 X ) X/ ′W −1U − σ 2 −1

( X ′W

−1

X ) β −σ 3 −1

(

b *S = β + σ X ′ W −1 X

)

−1

(

)

−1

)

(

− X ′W −1 X

)

−2

)(

Using above defined quantities

T X ′ W −1 U − σ 2  X ′ W −1 X α

(

[(

X ′W −1U ( X ′W −1 X )

′  Y − Xb * Y − Xb *  (2.8) b = 1 − K ′  b * X ′Xb * 

)

−1

T U ′BU − σ 3  X ′ W −1 U X ′ W −1 X − 2 + K X ′ W −1 X β′X ′Xβ α 2K β′ X 1 X α

α −2

The Stein-minimax regression coefficient of β is obtained by interchanging b 0 by b * in (2.4) and is given by

* S

where K is any positive non-stochastic scalar characterizing the estimation.

−1

 (Y − Xb 0 )′ (Y − Xb 0 ) b S = 1 − K b0 b ′o X ′Xb 0    

(2.9)

The Minimax estimation in linear regression model (MILE) proposed by Kuks and Olman (1971, 1972) and Kuks (1972) is (2.5) b * = D −1 X ′W −1 Y where D = α −1 σ 2 T + C (2.6) Using above defined quantities

(

β+K

)

−1

U ′ BU  β β′X ′Xβ 

X ′ W −1 U

 ]βT′XX′′UXββ − 2K βU′X′BU β′X ′U.β ′Xβ 

σ 4  U ′BU  T 2K  + K β X ′ W −1 X.X ′U − 2 β′X ′Xβ  + .  β′X ′Xβ  β′X ′Xβ  α  β′X ′Xβ Journal of Computer and Mathematical Sciences Vol. 2, Issue 1, 28 February, 2011 Pages (1-169)

) b  

Manoj Kumar, et al., J. Comp. & Math. Sci. Vol.2 (1), 83-91 (2011)

(

)

(

−1  2  −1 −1 −1 −1 β′X ′U U ′BU X ′W X X ′W U + β′ (X ′X ) − X ′W X α   −1   −1 K β′ 2  + X ′W −1 X β TU ′BU − X ′W −1 X (X ′X )  α α   

(

)

{

(

+ 2 Kβ ′ (X ′X ) − X ′W −1 X −1

)

−2

)

 TX ′ Uβ  

)

}TX U ′.X ′W U ]

The Predictors To study the performance properties of these estimators for prediction purposes, let us postulate the following prediction vectors. ˆ b* = X b* (2.10) T (2.11)

)

Tˆ b *S = X b *S

where b * , b *S are defined earlier. 3. PROPERTIES OF ESTIMATOR In order to study the properties of the estimator, we observe that the exact expressions for the bias vectors, mean squared error matrices and risk functions of the least squares estimators can be easily obtained and readily understood. However, it is not so with the minimax linear estimator. Therefore, in order to derive the expressions for the bias vectors, mean squared error matrices and risk functions of the minimax linear estimators, let us introduce some new notation as −1 (3.1) PX = X (X ′X ) X ′ (3.2) M = 1 − PX Here it is easily seen that

M = M′ : M ′.M = M MX = X ′M = 0

Substituting (2.1) in (2.2), we observe that (b 0 ) is an unbiased estimator of β with variance – covariance matrix as

−1

′ (3.3) V ( bo ) = E ( bo − β ) ( bo − β ) = σ 2 PX

[

]

where PX = 1 − X (X ′X ) X ′ The distributional assumption regarding the disturbances does not have any effect on the properties of least squares estimator. Clearly, the minimax estimator and Steinminimax estimators are biased estimators. −1

4. PREDICTIONS In this section, we have considered prediction of actual value of the study variable separately. Actual Value Prediction To study the impact of the prediction of actual value of the study variable, let us first consider the predictor of y based on ordinary least squares (4.1) Tˆo = Xb o where b o is defined in (2.2) It is easy to see that (4.2) E Tˆo − T = E[Xβ + σ P X U − Xβ − σU ] = 0

[

]

So, the predictors based on ordinary least squares from the model are unbiased. But the predictors based on minimax and Stein-minimax estimator from the model are

Journal of Computer and Mathematical Sciences Vol. 2, Issue 1, 28 February, 2011 Pages (1-169)

Manoj Kumar, et al., J. Comp. & Math. Sci. Vol.2 (1), 83-91 (2011)

( )

predictor Tˆb * up to the order o σ 4 are given by

biased. The predictive variance associated with them are given by (4.3)

( ) (

)(

)

′ PV Tˆt = E Tˆt ` − T Tˆt − T = σ 2 (n − p )

(4.4)

For the predictors bases on minimax and Stein-minimax estimator procedure from the model we have the following results. Theorem 4.1: When disturbance are small, the bias vector and predictive risk of

(4.5)

( )

(

)

−1 T PB Tˆb * = −σ 2 X X ′W −1 X β α ˆ * = σ 2 (n − p ) + σ 4 T ′T PR Tb 2

( )

( X ′W

−1

X ) β ′ X ′X β

−2

Theorem 4.2: When disturbance are small the bias vector and predictive risk of predictor Tb *s up to the order 0(σ 4 ) are given by (4.6)

( )

 K ˆ * = −σ 2  T X ( X ′W −1 X )−1 β + PB Tb X β tr ( B)  S α β ′ X ′X β  

(

)

 X X ′W −1 X −1 X ′W −1  − σ 3 γ 1 (I n * B)e K  − 2 β ′X ′Xβ  



1  K T γ 1 (I n * B)eXβ X ′W −1 X.X ′ − 2K tr (B)  α  β′X ′Xβ  β′X ′Xβ

+ σ 4 

{

(

)

−1 2K β′X ′X (γ 2 (I n * B) + tr (B) + 2B) X ′W −1 X X ′W −1 β′X ′Xβ −1  −1 K β′β  2  X ′W −1 X Xβ T.tr (B) − X ′W −1 X (X ′X )  + α α   

(

((

)

{

(

+ 2 kβ ′ (X ′X ) − X ′W −1 X (4.7)

−1

( )

)

−2

)

T X ′X. X ′W −1

}]

PR Tˆ b *S = σ 2 ( n − p ) − σ 4 [4K{γ 2 ( I n * B) + trB + 2 B} 2K − {γ 2 (I n * B) + trB + 2B} + 2K X ′W −1X −1 trB T β′X ′Xβ α 2 −1 T ′T K + 2 X ′W −1 X β ′X ′Xβ + {γ 2 trB(I n * B) + tr (B).B + 2trB}] β′X ′Xβ α

(

)

From (4.4) and (4.6), it is clearly seen that both the predictors Tˆ b * and Tˆ b *S are biased. This is in contrast with the unbiasedness of Tˆo . Journal of Computer and Mathematical Sciences Vol. 2, Issue 1, 28 February, 2011 Pages (1-169)

Manoj Kumar, et al., J. Comp. & Math. Sci. Vol.2 (1), 83-91 (2011)

(Tˆ ) − Risk(Tˆb ) = σ {4K{(I

5. COMPARISON *On comparing the risk associated with ordinary least square estimator Tˆ0 and minimax estimator observe that

* S

(Tˆ ) * b

( )

respectively, we

( )

′ (5.1) R Tˆ0 − R(Tb* ) = −σ4 T T (X′W−1X)−2 β′X′Xβ α2 By observing the above expression, we find that Tˆ0 is more efficient than Tˆb* in case of actual value prediction. * On comparing the risk associated with ordinary least square estimator Tˆ0 and

(

Stein-minimax estimator Tˆ b respectively, we observe that

* S

)

( )

(

T X ′W −1X α

)

−1

trB −

* B) + trB + 2B}

−1 T′T X ′W −1X β′X ′Xβ α2

(

)

K2 {γ 2 trB(I n * B) + tr(B).B + 2trB}} β′X ′Xβ

By observing the above expression, we find that Stein-minimax estimator is more efficient than ordinary least square estimator so long as γ 1 and β have same sign otherwise reverse hold true. If either γ 1 = 0 or λ 1 = 1 then both the estimator have same risk. We find that Stein minimax Tˆ b *S dominate the OLS Tˆ0 with respect to risk criterion if K satisfies

( )

(5.2) Risk

2K {γ 2 (I n * B) + trB + 2B} β′X ′Xβ

− 2K

−

( )

  2 (γ 2 (I n * B) + trB + 2B) − 4(γ 2 (I n * B) + trB + 2B) 0 < K < σ4     β′X ′Xβ −1 T   1 (γ 2 trB(I n * B) + tr (B).B + 2trB) − 2 X ′W −1 X trB  α   β′X ′Xβ  * ˆ * On comparing the risk associated with minimax estimator Tb and Stein-minimax Tˆ b *S ,

(5.3)

(

)

(

( )

)

we observe that (5.4)

( ) ( )

R Tˆb* − R Tˆ b *S = σ 4 [4 K (γ 2 (I n * B ) + trB + 2 B ) 2K + (γ 2 (I n * B) + trB + 2B) − 2K X ′W −1 X ′ ′ β X Xβ

(

−

)

−1

trB.

T α

 K2 (γ 2 trB (I n * B ) + tr (B ).B + 2 trB ) β ′ X ′X β 

Hence we find that Tˆ b *S is more efficient than Tˆ b * and we also find that Tˆ b *S is more efficient than Tˆ b * with respect to risk criterion if K satisfies (5.5)

  2 0 < K < σ 4    (γ 2 (I n * B ) + trB + 2 B ) + 4(γ 2 (I n * B ) + trB + 2 B )     β ′ X ′X β Journal of Computer and Mathematical Sciences Vol. 2, Issue 1, 28 February, 2011 Pages (1-169)

Manoj Kumar, et al., J. Comp. & Math. Sci. Vol.2 (1), 83-91 (2011)

(

T − 2 X ′W −1 X α

)

−1

 4 1  (γ 2 trB (I n * B ) + tr ( B).B + 2 trB )  σ  ′ ′    β X Xβ

 trB 

where

6. PROOF OF THE THEOREMS

Using (2.7) in the expression (2.10), we observe that (6.1) Tˆb* = Xβ + σπ1 + σ 2 π 2 + σ 3 π 3 where

(

)

−1

π1 = X X ′W −1 X X ′W −1 U −1 T (6.3) π 2 = − X X ′W −1 X β α −2 T (6.4) π 3 = − X X ′W −1 X X ′W −1 U α Now, subtract (2.1) from (6.1), we get (6.5) Tˆb* − Y = σπ 1* + σ 2 π 2 + σ 3 π 3

(

)

(

)

(

]

)

−1

π1* = − U 1 − X X ′W −1 X X ′W −1 and π 2 and π 3 are defined previously as (6.3) and (6.4) respectively. The predictive bias of Tˆb* for actual value prediction given as (4.4) in the theorem (4.1) can be obtained from PB Tˆb* = E Tˆb* − Y (6.6)

Theorem (4.1)

(6.2)

[

−1

( ) [

]

[ ]

= σE π1* + σ 2 E[π 2 ] + σ 3 E[π 3 ] where

[ ]

E π1* = 0 = E[π 3 ] T E[π 2 ] = − X X ′W −1 X α

(

)

−1

Next we observe that

′ E  π1* π1*   

[

(

= 1 − X X ′W −1 X

(

X X ′W −1 X

)

−1

)

−1

X ′W −1

′ ′ E π1* π 2  = E  π 2 π1*  = 0    

(

X ′W −1 − W −1 X X ′W −1 X

]

)

−1

(

X ′ + W −1 X X ′W −1 X

)

−1

X′ .

′ −2 −1 −2 T T  E π1* π 3  =  X X ′W −1 X X ′W −1 − W −1 X X ′W −1 X X ′X X ′W −1 X X ′W −1  α   α  −2 −2 −1 T′  T′  E π13 π1* =  W −1 X X ′W −1 X X ′ − W −1 X X ′W −1 X X ′X X ′W −1 X X ′W −1  α α  −1 −1 T ′T ′ E π 2 π 2  = 2 β ′ X ′W −1 X X ′X X ′W −1 X β   α

(

[

)

]

(

)

(

)

(

)

(

)

(

)

Using these expressions in (6.7)

( ) (

)(

′ PR Tˆb* = E Tˆb* − Y Tˆb* − Y

)

Journal of Computer and Mathematical Sciences Vol. 2, Issue 1, 28 February, 2011 Pages (1-169)

Manoj Kumar, et al., J. Comp. & Math. Sci. Vol.2 (1), 83-91 (2011)

′





′



′

 ′



′



= σ 2 E π1* π1*  + σ 3  π1* π 2 + π 2 π1*  + σ 4 π1* π 3 + π 2 π 2 + π 3 π1*      

( )

we get the predictive risk up to order 0 σ 4 of our approximation as stated (4.5) in Theorem (4.1). Theorem (4.2) Using (2.9) in the expression (2.11), we observe that (6.8) Tˆb *S = Xβ + σθ1 + σ 2 θ 2 + σ 3 θ 3 + σ 4 θ 4 where (6.9)

(

θ1 = X X ′W −1 X

)

−1

X ′W −1 U

−1 T  U′BU θ 2 = −  X X′W −1 X β + K Xβ β′X′Xβ  α −2 T U′BU −1 −1 X X ′W −1 X (6.11) θ 3 = −  X ′W XU X ′W X + K ′ ′ β X Xβ α

(

(6.10)

)

(

[

)

(

X (X ′X ) − X′W −1 X (6.12)

θ4 =

−1

)

−2

(

)

−1

X′W −1 U +

 ]βT′XX′′UXββ − 2K βU′X′BU β′X ′U.Xβ ′Xβ 

2K β′ . α

1  U ′BU T   K Xβ X ′W −1 X.X ′U − 2 β′X ′Xβ   β′X ′Xβ  β′X′Xβ  α 

(

)

  U ′BU X ′W −1 X −1 X ′W −1 U    2K  + β ′X ′UX 2  − 2 −1 −1 β ′X ′Xβ   + Xβ′ (X ′X ) − X ′W X TX ′Uβ   α   −1  −1 K β′  2  + X′W −1 X Xβ TU′BU − X ′W −1 X (X′X ) β α α    

(

[

(

)

{

(

+ 2 K β ′ (X ′X ) − X ′W −1 X −1

)

−2

)

]

)

}T X ′U ′.X X ′W U ] −1

Now subtract (2.1) from (6.8), we get * (6.13) Tˆb S − Y = σθ 1* + σ 2 θ 2 + σ 3 θ 3 + σ 4 θ 4

[

(

where θ1* = − I − X X ′W −1 X

)

−1

]

X ′W −1 U and θ 2 , θ 3 and θ 4 are defined previously as (6.10), (6.11) & (6.12) respectively. The predictive bias of Tˆb * for actual value prediction given as (4.6) in theorem (4.2) can be from s

Journal of Computer and Mathematical Sciences Vol. 2, Issue 1, 28 February, 2011 Pages (1-169)

Manoj Kumar, et al., J. Comp. & Math. Sci. Vol.2 (1), 83-91 (2011)

90 (6.14)

( ) [

PB Tˆ b *S = E Tˆb *S − Y

[ ]

]

= σE θ + σ E[θ 2 ] + σ 3 E[θ 3 ] + σ 4 E[θ 4 ] * 1

where (6.15) E θ1* = 0

[ ]

(

)

−1 T  K −1 Xβ tr (B) E [θ 2 ] = −  X X ′W X β + β′X ′Xβ α   K  −1 X X ′W −1 X X ′W −1 γ 1 (I n * B)e − 2Kγ 1 (I n * B)e (6.17) E[θ 3 ] = −   β′X ′Xβ 

(6.16)

(

)



1  K T γ 1 (I n * B)eXβ X ′W −1 X.X ′ − 2K tr (B)  α  β′X ′Xβ  β′X ′Xβ

(6.18) E (θ 4 ) = 

{

(

)

(

((

)

{

(

+ 2 kβ ′ (X ′X ) − X ′W −1 X Next we observe that  ′  (6.19) E  θ1* θ1*  =  

[I − X(X ′W

(6.20)

−1

(

−1

−2

T X ′X. X ′W −1

}]

) X ′ + W X (X ′W X ) X ) X ′X (X ′W X ) X ′W ]

X ′W −1 X X ′W −1 X

(X ′W

)

−1

[

]

 ′ 

 =  T X′W −1 X X ′W −1 X   α 

[

]

E θ1* θ 2

(6.21) E θ1* θ 3

(6.22)

)

−1

)

−1

X ′.X ]

−1

(

 K K =  γ 1 (I n * B )eXβ − γ 1 ( I n * B)eW −1 X X ′W −1 X ′ ′ ′ ′ β X X β β X X β 

(

)

−1

 X ′Xβ 

1 [γ 2 (I n * B)tr (B) + 2B] . β′X ′Xβ −2 2K TX ′β −1 −1 β ′X (X ′X ) − (X ′W −1 X ) X (X ′W −1 X ) X ′W −1 + α β′X ′Xβ − 2K (γ 2 (I n * B ) + tr ( B) + 2B )]

E θ12 θ 2 =

T ′T X ′W −1 X α2

(

)

−2

+ K.

(

)

 T′ T  β′X ′Xβ + K  +  trB X ′W −1 X  α α

(

)

−1

Journal of Computer and Mathematical Sciences Vol. 2, Issue 1, 28 February, 2011 Pages (1-169)

Manoj Kumar, et al., J. Comp. & Math. Sci. Vol.2 (1), 83-91 (2011)

K2 + [γ 2 tr (B)(I n * B) + tr (B).B + 2trB)] β′X ′Xβ (6.23)

−2 T′ K X ′W −1 X X ′W −1 X + (γ 2 (I n * B) + tr (B) + 2B ) . α β′X ′XB −2 2K −1 −1 T (X ′X ) − (X ′W −1 X ) W −1 X (X ′W −1 X ) X ′ ] + α − 2 K (γ 2 (I n * B ) + tr ( B) + 2 B ) −2 −1 T′ − X ′W −1 X (X ′W −1 X ) X (X ′W −1 X ) X ′W −1 α 1 −K (γ 2 (I n * B) + trB + 2B)W −1X X ′W −1X −1 X ′X. β′X ′Xβ (X ′W −1 X )−1 X ′W −1 − 2αK T (X ′X )−1 − (X ′W −1X )−2 X(X ′W −1X )−1 X ′W −1 −1 + 2 K (γ 2 (I n * B ) + trB + 2 B )X (X ′W −1 X ) X ′W −1 ]

[

E θ13 θ1*

] =[

(

)

(

)

(

Using these above expression, we get

( ) (

)(

)

′ PR Tˆb *S = E Tˆb *S − Y Tˆb *Ss − Y ′ ′ ′ ′ = σ 2 E θ1* θ1*  + σ 3 E θ1* θ 2 + θ12 θ1*  + σ 4 E θ1* θ 3 + θ13 θ1* + θ *2 θ 2        4 We get the prediction risk up to order 0(σ ) of our approximation as stated (4.7) in theorem (6.24)

(4.2).

REFERENCES 1. Bunke , O. “Minimax linear ridge and shrunken estimators for linear parameters”. Math. Operationsforschung Statistik, 66,697-701 (1975). 2. Kuks, J. and W. Olman. “Minimax estimation in linear regression model” Journal of Statistical Planning and Inference, Volume 50, Issue 1, Pages 7789 (1972). 3. Toutenberg , H. “Minimax-linear estimation and 2 phase mmle in a restricted linear regression model”. Math. Operationsforschung Statistik, 6, 730-706 1975.

4. Toutenberg, H. “Minimax-linear and Mse-estimators in generalized regression”. Biometrische Zeitschrift, 18, 91100 (1976). 5. Srivastava, A. K. and Shalabh. “Predictions in Linear Regression Models with Measurement Errors". Indian Journal of Applied Economics, Vol. 4, No. 2, pp. 1-14 (1995). 6. Shalabh.“Performance of Stein - rule Procedure for Simultaneous Prediction of Actual and Average Values of Study Variable in Linear Regression Model". Bulletin of the International Statistical Institute, The Netherlands, pp. 13751390 (1995).

Journal of Computer and Mathematical Sciences Vol. 2, Issue 1, 28 February, 2011 Pages (1-169)