Rajesh Singh Department of Statistics, BHU, Varanasi (U.P.), India
Jayant Singh Department of Statistics Rajasthan University, Jaipur, India
Florentin Smarandache Department of Mathematics, University of New Mexico, Gallup, USA
Optimum Statistical Test Procedure
Published in: Rajesh Singh, Jayant Singh, F. Smarandache (Editors) STUDIES IN STATISTICAL INFERENCE, SAMPLING TECHNIQUES AND DEMOGRAPHY InfoLearnQuest, Ann Arbor, USA, 2009 (ISBN-10): 1-59973-087-1 (ISBN-13): 978-1-59973-087-5 pp. 5 - 20
Introduction Let X be a random variable having probability distribution P( X / T), T 4 . It is desired to test H 0 : T 4 0 against H1 : T 41 outcomes of an experiment and x
4 4 0 . Let S denote the sample space of
( x 1 , x 2 , , x n ) denote an arbitrary element of S. A test
procedure consists in diving the sample space into two regions W and S – W and deciding to reject H0 if the observed
x W . The region W is called the critical region. The function
J (T) P T ( x W ) = PT ( W) , say, is called the power function of the test.
5
We consider first the case where 40 consists of a single element, T0 and its complement 41 also has a single element T1 . We want to test the simple hypothesis H0 : T T0 against the simple alternative hypothesis H1 : T T1 . Let L0 = L(X/H0) and L1 = L(X/H1) be the likelihood functions under H0 and H1 respectively. In the Neyman – Pearson set up the problem is to determine a critical region W such that
J(T0 )
PT0 (W)
³ L dx
D, an assigned value
0
(1)
w
and J(T1 )
PT1 (W)
³ L dx 1
is maximum
(2)
w
compared to all other critical regions satisfying (1). If such a critical region exists it is called the most powerful critical region of level . By the Neyman-Pearson lemma the most powerful critical region W0 for testing ܪ ǣ ߠ ൌ ߠ against ܪଵ ǣ ߠ ൌ ߠଵ is given by
W0
{x :L1 t kL0 }
provided there exists a k such that (1) is satisfied. For this test J(T 0 )
D and J (T1 ) o 1 as n o D .
But for any good test we must have J(T 0 ) o 0 and J (T1 ) o 1 as n o f because complete discrimination between the hypotheses H0 and H1 should be possible as the sample size becomes indefinitely large.
6
Thus for a good test it is required that the two error probabilities D and E should depend on the sample size n and both should tend to zero as n o f . We describe below test procedures which are optimum in the sense that they minimize the sum of the two error probabilities as compared to any other test. Note that minimizing ( D E ) is equivalent to maximising 1 - ( D E ) = (1 E) D = Power – Size. Thus an optimum test maximises the difference of power and size as compared to any other test. Definition 1 : A critical region W0 will be called optimum if
Âł L dx Âł L dx t Âł L dx Âł L dx 1
w0
0
w0
1
0
w
(3)
w
for every other critical region W. Lemma 1: For testing H0: T
T0 against H1 : T T1 the region
W 0 {x : L1 t L 0 } is optimum. Proof : W0 is such that inside W0, L1 t L 0 and outside W0 , L1 L 0 . Let W be any other critical region.
Âł (L
W0
1
L 0 )dx Âł (L1 L 0 )dx W
=
Âł
W0 W c
(L1 L 0 )dx
Âł (L
1
L 0 )dx ,
W W0c
by subtracting the integrals over the common region W0 W.
7
t0 since the integrand of first integral is positive and the integrand under second integral is negative. Hence (3) is satisfied and W0 is an optimum critical region. Example 1 : Consider a normal population N(T, V 2 ) where V 2 is known. It is desired to test H0: T
n
T0 against H1 : T T1 , T1 ! T 0 . ( x i T) 2
§ 1 ¡ iŒ1 2V 2 L( x T )= ¨¨ ¸¸ e Š V 2S š n
L1 L0
ÂŚ ( x i T1 ) 2
ÂŚ (x i T0 ) 2
e e
2V 2 2V 2
The optimum test rejects Ho
if
L1 t1 L0
i.e. if
L log 1 t 0 L0
i.e. if
i.e. if
ÂŚ ( x i T1 ) 2 ÂŚ ( x i T 0 ) 2 2V 2
2V 2
t0
2T1 ÂŚ x i nT12 2P 0 ÂŚ x i nT 02 t 0
8
i.e. if
2T1 2T 0 ¦ x i t n T12 T02
¦ xi
i.e. if
i.e. if
n
T T t 1 0 2
T T0 xt 1 2
Thus the optimum test rejects H0
T T0 if x t 1 2 We have
D
T T0 º ª PH 0 « x t 1 2 »¼ ¬ = PH 0 ª x T 0
t « «¬ V / n
Under Ho,
n T1 T 0 º » 2V »¼
x T 0 follows N(0,1) distribution. §V · ¨ ¸ n¹ ©
§ n T1 T 0 · ¸ ‫ ׵‬D 1 )¨ ¸ ¨ 2V ¹ ©
where ) is the c.d.f. of a N(0,1) distribution.
E
ª º T T0 º ª « x T1 n T1 T 0 » P PH 1 « x 1 = H1 « » V 2 »¼ 2V ¬ «¬ »¼ n
9
Under H1,
x T1 follows N(0,1) distribution. §V · ¨ ¸ n¹ ©
§ n T1 T 0 · ¸ E 1 )¨ ¨ ¸ 2V © ¹
Here D
E.
It can be seen that D
E o 0 as n o f .
Example 2 : For testing H0: T
T0 against H1 : T T1 , T1 T0 , the optimum test rejects
T T0 H0 when x d 1 . 2
Example 3 : Consider a normal distribution N T, V 2 , T known.
It is desired to test H0 : V 2
V 02 against H1 : V 2
V12 , V12 ! V 02 .
We have
Lx V
2
n
L1 L0
§ V2 · 2 ¨ 0¸ e ¨ V2 ¸ © 1¹
¦ x i T
2
2V12
¦ x i T
2
2V 02
e
L log 1 L0
x i T 2
n
§ 1 · 2 ¦ 2V 2 ¨¨ ¸¸ e © 2SV 2 ¹
n ¦ x i T 2 ¦ x i T 2 log V12 log V 02 2 2V12 2V 02
10
=
n ÂŚ x i T V12 V 02 log V12 log V 02 2 2 V12 V 02
2
The optimum test rejects H0
Ý…Ý‚
â€ŤÜŽâ€ŹŕŹľ ŕľ’Íł â€ŤÜŽâ€ŹŕŹ´
V12 V 02 2V12 V 02
i.e. if
ÂŚ x i T 2 t 2 log V12 log V 02
n
ÂŚ x i T 2
i.e. if
V 02
t
nV12 V12
V 02
log V12 log V 02
2
§ x i T ¡ ¸¸ t nc Š V0 š
Œ ¨¨
i.e. if
where c
V12 V12 V 02
log V12 log V02
்೔ ିŕ°? ଶ
Thus the optimum test rejects H0 if Ďƒ ቀ
ఙఏ
በྒ ݊ܿ.
x T Note that under H 0 : i follows N(0,1) distribution. Hence V0
2
§ xi T ¡ ¸¸ follows, under Š V0 š
Œ ¨¨
H0, a chi-square distribution with n degrees of freedom (d. f.).
Here
D
ª §x T 0 PH 0 Œ ¨¨ i  Š V0 
2 º ¡ ¸¸ t nc  š Ÿ
>
P F (2n ) t nc
@
ª § x T ¡2 º ¸¸ t nc and 1 – = PH 1 Œ ¨¨ i  Š V0 š   Ÿ
11
ª § x T · 2 ncV 2 º 0» ¸ t = PH 1 «¦ ¨¨ i 2 » « © V1 ¸¹ V 1 ¬ ¼
ª ncV 02 º = PH1 «F (2n ) t » V12 ¼» ¬« ௫ ିఏ ଶ
ͳǡ σ ቀ
ఙభ
ቁ
follows a chi-square distribution with n d.f.
It can be seen that D o 0 and E o 0 as n o f. Example 4 : Let X follow the exponential family distributions f x T c T e Q T T ( x ) h ( x )
It is desired to test H0: T
L( x T)
T0 against H1 : T T1
>c(T)@n e Q(T)¦ T( x i ) h ( x i ) i
The optimum test rejects Ho when
݈݃
ܮଵ Ͳ ܮ
i.e. when >Q(T1 ) Q(T 0 )@¦ T( x i ) t n log
c( T 0 ) c(T1 ) i.e. when ¦ T( x i ) t >Q(T1 ) Q(T 0 )@
c(T 0 ) c(T1 )
n log
if Q(T1 ) Q(T0 ) ! 0
and rejects Ho,
12
c(T 0 ) c(T1 ) when ¦ T( x i ) d >Q(T1 ) Q(T 0 )@ n log
if Q(T1 ) Q(T0 ) 0
Locally Optimum Tests:
Let the random variable X have probability distribution ܲሺ ݔΤߠሻ. We are interested in testing H0: T
T0 against H1 : T !T0 . If W is any critical region then the power of the test
as a function of ߠ is ߛሺߠሻ ൌ ܲఏ ሺܹሻ ൌ ௐ ܮሺܺΤߠ ሻ݀ݔ
We want to determine a region W for which ߛሺߠሻ െ ߛሺߠ ሻ ൌ ௐ ܮሺ ݔΤߠ ሻ݀ ݔെ ௐ ܮሺ ݔΤߠሻ
is a maximum. When a uniformly optimum region does not exist, there is not a single region which is best for all alternatives. We may, however, find regions which are best for alternatives close to the null hypothesis and hope that such regions will also do well for distant alternatives. We shall call such regions locally optimum regions. Let ߛሺߠሻ admit Taylor expansion about the point ߠ ൌ ߠ . Then ߛሺߠሻ ൌ ߛሺߠ ሻ ሺߠ െ ߠ ሻߛƴ ሺߠ ሻ ߜ
where ߜ ՜ Ͳ as ߠ ՜ ߠ
ߛ ሺߠሻ െ ߛሺߠ ሻ ൌ ሺߠ െ ߠ ሻߛƴ ሺߠ ሻ ߜ
If ȁߠ െ ߠ ȁ is small ߛሺߠሻ െ ߛሺߠ ሻ is maximised when ߛƴ ሺߠ ሻis maximised.
13
Definition2 : A region W0 will be called a locally optimum critical region if ௐ ܮሖሺ ݔΤߠ ሻ݀ ݔ ௐ ܮሖሺ ݔΤߠ ሻ݀ݔ
(4)
బ
For every other critical region W. Lemma 2 : Let W0 be the region ൛ݔǣ ܮሖሺ ݔΤߠ ሻ ܮሺ ݔΤߠ ሻൟ. Then W0 is locally optimum. Proof: Let W0 be the region such that inside it ܮሖሺ ݔΤߠ ሻ ܮሺ ݔΤߠ ሻ and outside it ܮሖሺ ݔΤߠ ሻ ൏ ܮሺ ݔΤߠ ሻ. Let W be any other region. ௐ ܮሖሺ ݔΤߠ ሻ݀ ݔെ ௐ ܮሖሺ ݔΤߠ ሻ݀ݔ బ
=
ௐ
బתೈ
ܮሖሺ ݔΤߠ ሻ݀ ݔെ ௐ תௐ ܮሖሺ ݔΤߠ ሻ݀ݔ
=
ௐ
బתೈ
ܮሖሺ ݔΤߠ ሻ݀ ݔ ௐ ௐ ܮሖሺ ݔΤߠ ሻ݀ݔ
ௐ
బתೈ
ܮሺ ݔΤߠ ሻ݀ ݔ ௐ ௐ ܮሺ ݔΤߠ ሻ݀ݔ
బ
బ
(*)
బ
since ܮሖሺ ݔΤߠ ሻ ܮሺ ݔΤߠ ሻ inside both the regions of the integrals. 0,
Hence
since ܮሺ ݔΤߠ ሻ Ͳ
in all the regions.
ௐ ܮሖሺ ݔΤߠ ሻ݀ ݔ ௐ ܮሖሺ ݔΤߠ ሻ݀ ݔfor every other region W. బ
To prove (*):
We have ோ ܮሺ ݔΤߠ ሻ݀ ݔ ோ ܮሺ ݔΤߠ ሻ݀ ݔൌ ͳ for every region R. Differentiating we have
14
න ܮሖሺ ݔΤߠ ሻ݀ ݔ න ܮሖሺ ݔΤߠ ሻ݀ ݔൌ Ͳ ோ
ோ
න ܮሖሺ ݔΤߠ ሻ݀ ݔൌ െ න ܮሖሺ ݔΤߠ ሻ݀ ݔൌ Ͳ ோ
ோ
In (*), take ܴ =ܹ ܹ תand the relation is proved. Similarly if the alternatives are H1 : T T0 , the locally optimum critical region is ൛ݔǣ ܮሖሺ ݔΤߠ ሻ ܮሺ ݔΤߠ ሻൟ. Example 5: Consider ܰሺߠǡ ߪ ଶ ሻ distribution, ߪ ଶ known.
It is desired to test H0: T
ܮሺ ݔΤߠ ሻ ൌ ൬
ͳ ߪξʹߨ
൰ ݁
T0 against H1 : T !T0
ି σሺ௫ ିఏሻమ ଶఙ మ
ͳ σሺݔ െ ߠሻଶ ݈ܮ݃ሺ ݔΤߠሻ ൌ ݈݊ ݃൬ ൰െ ʹߪ ଶ ߪξʹߨ ܮሖሺ ݔΤߠ ሻ ߜ݈ܮ݃ሺ ݔΤߠሻ σሺݔ െ ߠሻ ݊ሺݔҧ െ ߠሻ ൌ ൌ ൌ ߜߠ ߪଶ ܮሺ ݔΤߠ ሻ ߪଶ
ܮሖሺ ݔΤߠ ሻ ݊ሺݔҧ െ ߠ ሻ ൌ ܮሺ ݔΤߠ ሻ ߪଶ
The locally optimum test rejects H0 ,if ݊ሺݔҧ െ ߠ ሻ ͳ ߪଶ
i.e. ݔҧ ߠ
ఙమ
Now,
15
ߙ ൌ ܲுబ ቈݔҧ ߠ
ߪଶ ݊
ݔҧ െ ߠ ߪ ൌ ܲுబ ߪ ൗ ξ݊ ൗ ݊ ξ ൌ ͳ െ Ȱ ൬ߪൗ ൰, since under H0, ξ݊
௫ҧ ିఏబ ఙ ൗ ξ
follows N(0,1) distribution.
ଶ
ͳ െ ߚ ൌ ܲுభ ݔҧ ߠ ߪ ൗ݊
ݔҧ െ ߠଵ ߠଵ െ ߠ ߪ ൌ ܲுభ ߪ െ ߪ ξ݊ ൗ ݊ ൗ ݊ ξ ξ െ൫ߠଵ െ ߠ ξ݊൯ ߪ ൌ ͳ െ Ȱቈ ߪ ξ݊
since under H1,
௫ҧ ିఏభ ఙ ൗ ξ
follows N(0,1) distribution.
Exercise : If ߠ ൌ ͳͲǡ ߠଵ ൌ ͳͳǡ ߪ ൌ ʹǡ ݊ ൌ ͳǡ then ߙ = 0.3085, 1-ߚ = 0.9337
Power - Size = 0.6252 ௫ҧ ିఏబ
If we reject H0 when ఙ
ൗ ξ
> 1.64, then ߙ = 0.05, 1-ߚ = 0.6406
Power – Size = 0.5906 Hence Power – Size of locally optimum test is greater than Power – size of the usual test. Locally Optimum Unbiased Test:
16
Let the random variable X follows the probability distribution ܲሺ ݔΤߠ ሻ. Suppose it is desired to test H0: T
T0 against H1 : T zT0 . We impose the unbiasedness restriction
ߛሺߠሻ ߛሺߠ ሻǡ ߠ ് ߠ
and ߛሺߠሻ െ ߛሺߠ ሻ is a maximum as compared to all other regions. If such a region does not exist we impose the unbiasedness restriction ߛƴ ሺߠ ሻ ൌ ͲǤ Let ߛሺߠሻ admit Taylor expansion about the point ߠ ൌ ߠ Ǥ Then ߛሺߠሻ ൌ ߛሺߠ ሻ ሺߠ െ ߠ ሻߛƴ ሺߠ ሻ
ሺఏିఏబ ሻమ ଶ
J cc T 0 +ߟ
where ߟ ՜ Ͳ as ߠ ՜ ͲǤ ߛ ሺߠሻ െ ߛሺߠ ሻ ൌ ሺߠ െ ߠ ሻߛƴ ሺߠ ሻ
ሺఏିఏబ ሻమ ଶ
J cc T 0 +ߟ
Under the unbiasedness restriction ߛƴ ሺߠ ሻ = 0, if ȁߠ െ ߠ ȁ is small ߛሺߠሻ െ ߛሺߠ ሻ is maximised when J cc T 0 is maximised. Definition 3 : A region W0 will be called a locally optimum unbiased region if ߛƴ ሺߠ ሻ ൌ ௐ ܮሖሺ ݔΤߠ ሻ݀ ݔൌ Ͳ
(5)
బ
and
J cc T 0 =
³ Lcc X T 0 dx t ³ Lcc X T 0 dx
W0
(6)
W
for all other regions W satisfying (5). Lemma 3 : Let W0 be the region
^x : Lcc x T0 t L x T0 `
Then W0 is locally optimum unbiased.
17
Proof : Let W be any other region ௐ ܮᇱᇱ ሺ ݔΤߠ ሻ݀ ݔെ ௐ ܮᇱᇱ ሺ ݔΤߠ ሻ݀ݔ బ
න ܮᇱᇱ ሺ ݔΤߠ ሻ݀ ݔ
ൌ
න ܮᇱᇱ ሺ ݔΤߠ ሻ݀ݔ ௐబ תௐ
ௐబ תௐ
by subtracting the common area of W and W0. න ܮᇱᇱ ሺ ݔΤߠ ሻ݀ ݔ
ൌ
ௐబ תௐ
න ௐబ
ܮᇱᇱ ሺ ݔΤߠ ሻ݀ݔǡ
תௐ
since ܮᇱᇱ ሺ ݔΤߠ ሻ ܮሺ ݔΤߠ ሻ inside W0 and outside W. Ͳ
since
ܮሺ ݔΤߠሻ ͲǤ
Example 6:
Consider ܰሺߠǡ ߪ ଶ ሻ distribution, ߪ ଶ known . ܪ ǣ ߠ ൌ ߠ ǡ ܪଵ ǣ ߠ ് ߠ
ܮൌ൬
ͳ ߪξʹߨ
൰ ݁
ି
σሺ௫ ିఏሻమ ଶఙమ
̶ܮሺ ݔΤߠ ሻ ݊ଶ ሺݔҧ െ ߠሻଶ ݊ ൌ െ ଶ ߪ ߪସ ܮሺ ݔΤߠሻ
Locally optimum unbiased test rejects H0 ݂݅
݊ଶ ሺݔҧ െ ߠ ሻଶ ݊ െ ଶͳ ߪ ߪସ
݅Ǥ ݁Ǥ
݊ሺݔҧ െ ߠ ሻଶ ߪଶ ͳ ଶ ߪ ݊
ܷ݊݀݁ܪ ݎǡ
ሺ௫ҧ ିఏబ ሻమ ఙమ
ଶ follows ߯ሺଵሻ distribution.
18
Testing Mean of a normal population when variance is unknown.
Consider ܰሺߠǡ ߪ ଶ ሻ distribution, ߪ ଶ known. For testing H0: T
T0 against H1 : T T1 , the critical function of the optimum test is
given by ߶ ሺݔሻ ൌ ቄ
ͳ ݂݅ ܮሺ ݔΤߠଵ ሻ ܮሺ ݔΤߠ ሻ Ͳ ݁ݏ݅ݓݎ݄݁ݐ
On simplification we get ߠ ߠଵ ݂݅ ߤଵ ߤ ߶ ሺݔሻ ൌ ൝ͳ ݂݅ ݔҧ ʹ Ͳ ݁ݏ݅ݓݎ݄݁ݐ Ʌ Ʌଵ ρଵ ൏ ρ Ԅ୫ ሺ ሻ ൌ ൝ ͳ ത ൏ ʹ Ͳ Consider the case when ࣌ is unknown.
For this case we propose a test which rejects H0 when ܮሺ ݔΤߠଵ ሻ ͳ ܮሺ ݔΤߠ ሻ
where ܮሺ ݔΤߠ ሻ, (i=0,1) is the maximum of the likelihood under Hi obtained from ܮሺ ݔΤߠ ሻ by replacing ߪ ଶ by its maximum likelihood estimate ଶ ଵ ଶ ߪ ప ൌ σୀଵ൫ݔ െ ߠ ൯ ; i=0,1.
Let ߶ ሺݔሻ denote the critical function of the proposed test, then ሺ ݔΤߠଵ ሻ ܮሺ ݔΤߠ ሻ ߶ ሺݔሻ ൌ ൜ͳ ݂݅ ܮ Ͳ ݁ݏ݅ݓݎ݄݁ݐ
19
On simplification we get
߶ ሺݔሻ ൌ ቊ
ͳ ݂݅ ݔҧ
ఏబ ାఏభ ଶ
݂݅ ߠଵ ߠ
݁݅ݏݓݎ݄݁ݐ ఏ ାఏ
and
ͳ ݂݅ ݔҧ ൏ బ భ ݂݅ ߠଵ ൏ ߠ ߶ ሺݔሻ ൌ ቊ ଶ ݁݅ݏݓݎ݄݁ݐ
Thus the proposed test ߶ ሺݔሻ is equivalent to ߶ ሺݔሻ which is the optimum test that minimizes the sum of two error probabilities ሺߙ ߚሻ. Thus we see that one gets the same test which minimises the sum of the two error probabilities irrespective of whether ߪ ଶ is known or unknown. Acknowledgement : Authors are thankful to Prof. Jokhan Singh for his guidance in writing
this article. References
Pandey, M. and Singh,J. (1978) : A note on testing mean of a normal population when variance is unknown. J.I.S.A. 16, pp. 145-146. Pradhan,M. (1968) : On testing a simple hypothesis against a simple alternative making the sum of the probabilities of two errors minimum. J.I.S.A. 6, pp. 149-159. Rao, C.R.(1973) : Linear Statistical Inference and its Applications. John Wiley and Sons Singh,J. (1985): Optimum Statistical Test Procedures. Vikram Mathematical Journal Vol V, pp. 53-56.
20