Exam zone
Optimum Nonlinear Estimation Considertworandom variables X ( µ) and Y ( µ) deßnedonthesamesample spaceS. We have previously looked at the optimal linear MMSE estimate of Y in terms of X . In thisexercise we will look at the form of the optimal nonlinear estimate of Y giventhe observation X . Specißcally letus look at the conditional mean estimate given by : Ý = g( X ) = E f Y j X g = Z Y
1
_
yf Y j X ( yj x ) dy:
Note that this estimate does not depend on Y and depends only on the conditioned value of X , since the variable y has been integrated out. The mean-squared error incurred by this estimate is given by: ¯2g = E f ( Y
g( X )) 2g
Ý2 = h( X ). The corresponding Consider another estimate of Y given by Y error incurred by this estimate is given by: ¯2h = E f ( Y
h( X )) 2g
Relating this error to the error incurred by the conditional mean estimator we have: ¯2h = E f ( Y
g( X ) + g( X )
h( X )) 2g:
This can further be written as the sum of three terms: ¯2h = ¯2g + 2E f ( Y |
g( X ))( g( X ) {z T2
h( X )) g + E f ( g( X ) h( X )) 2g : } | {z } T3
The third term is always positive as is the ßrst term. We will look at the second term in detail. Using iterated expectation we can rewrite this : E f (Y
g( X ))( g( X )
h( X )) g = E X ¢E Y j X f ( Y
g( X ))( g( X )
h( X )) g£
Evaluating the inner expectation we have: EY jX f (Y
g( X ))( g( X )
h( X )) g =( g( X )
h( X )) E Y j X f Y
g( X ) g =0 :
We can therefore write the error incurred by the estimate h( X ) as ¯2h = ¯2g + E f ( g( X )
h( X )) 2gÀ!
¯2h µ ¯2g:
The statement made above has two important implications:
1. The conditional mean estimator is the best non-linear estimate of Y given the observationsX . 2. Anyotherestimatorbeitlinearornonlinearwillalwayshaveanestimation error larger than the conditional mean estimator. 3. In the special case whereX and Y are jointly Gaussian then the optimal non-linear estimate becomes the optimal linear estimate.