A New Approach to the Indefinite LQ Optimal Control for a Kind of Stochastic Bilinear System with Control Dependent Noises Xing Guo-jing School of Control Science and Engineering, Shandong University, Jinan, Shandong, China xgjsdu@126.com Abstract This article discusses the indefinite LQ optimal state feedback control problem for a kind of stochastic bilinear system with control dependent noises. First, a backward dual system in Krein space of the original system is constructed, and the dual theorem between the original indefinite LQ problem and a backward stochastic filtering problem is obtained. Then, the stationary point of the control and the condition for the stationary point to be the minimum are derived. Keywords Multiplicative Noises; Stochastic Bilinear System; Duality; Indefinite LQ Control
Introduction There exist extensively multiplicative noises in practical engineering areas such as electronic communication and mathematical economy. The stochastic systems with multiplicative noises are also called bilinear stochastic systems(BLSS)[1-2]. BLSS approximates the actual nonlinear stochastic system, and it provides better tools to depict stochastic uncertainties in nature. Meanwhile, stochastic linear quadratic (LQ) problem is an important branch of stochastic optimal control, and it has attracted extensive attentions of researchers. For linear stochastic systems, a necessary assumption for the LQ problem to be well-posed is that the state weighting matrices are semi-positive definite, and the control weighting matrices are positive definite. Otherwise, the LQ problem is considered to be trival or meaningless. Early researches on the LQ problem used this assumption, and many results which are completely parallel with deterministic systems were derived [3-5]. However, it’s found that the LQ problem for BLSS is also well defined even if this assumption is not fulfilled. It shows the intrinsic difference between the stochastic and deterministic optimal control problems, and leads to studies on stochastic indefinite LQ optimal control problems [6, 9]. X. Y. Zhou made systematic research on indefinite LQ problem of BLSS. The sufficient and necessary condition for the problem to be solvable was obtained via dynamic programming methods [7, 11]. However, it is needed to solve complicate partial differential equation, and the results are not unique. It is well known that there is duality between classical LQ problem and least mean square error estimation problem. So, the LQ optimal control problem can be translated into more simple state estimation problem to solve. But, the classical Kalman duality is not satisfied for the LQ problem of BLSS. Hassibi proposed a kind of quadratic optimization approach in Krein space, and established the duality between indefinite LQ optimal control and Kalman filter in Krein space [10]. However, the result is only appropriate for linear stochastic systems with addictive noises. H. S. Zhang proposed a new linear estimator for multiplicative-noise system, and established the duality between a stochastic LQ control problem and the estimation problem [12]. The purpose of this paper is to establish the duality between state estimation and indefinite LQ problem of BLSS with control dependent multiplicative noises. First, the backward dual system of the original system is constructed. Then, the duality theorem is obtained. Finally, the stationary point of optimal control and the condition for the stationary point to be the maximum point are derived. International Journal of Automation and Control Engineering, Vol. 4, No. 2—October 2015 2325-7407/15/02 077-6 Š 2015 DEStech Publications, Inc. doi:10.12783/ijace.2015.0402.03
77
78
Xing Guo-jing
Problem Statement (1) is states vector, and is initial state. * + is control vector. matrices. are standard Gaussian white noises, and are not correlated with each other. ]=0, ,
, -=E[ Where , - denotes expectation, and
-
, ,
are deterministic
-
is the Dirac Delta function satisfying *
Consider the optimal control problem of linear discrete time stochastic system (1) with control dependent noises and quadratic cost function (
)
,∑
(
)
-,
(2)
Where, the state weighting matrices and the control weighting matrices are appropriate symmetric matrices. The target of optimal control design is to find control sequence in order to minimize the cost function (2). It is only assumed that the weighting matrices are symmetric, but the semi-positive definiteness of and positive definiteness of are not assumed in advance. So, this problem is called indefinite LQ optimal control problem. Pseudo Deterministic Problem When the states of system (1) are completely observable, let , - be the covariance matrices of the states, and the problem (1-2) completely equals to a Pseudo deterministic problem. Lemma 1. If states of (1) are completely observable and the linear optimal state feedback control is , then the stochastic indefinite LQ problem completely equals to the following Pseudo deterministic problem. The system equation is shown as (3), and the cost function is shown as (4). (3) *
+
∑
,(
)
-
,
-
(4)
Where, , - denotes the trace of matrices M. After the Pseudo deterministic problem has been derived by lemma.1, the original indefinite LQ problem can be solved using lemma.2. Lemma 2. If the generalized difference Riccati equation (GDRE) (5) admits a solution, then the unique optimal state feedback control of the Pseudo deterministic problem (3-4) and the original indefinite LQ problem (1-2) is (6) [12]. {
}
(5) (6)
Duality in Krein Space Krein space is a kind of indefinite linear space which is different from Hilbert space. The greatest difference between them is that there are some nonzero vectors named neutral vectors whose length is zero and some nonzero vectors called isotropic vectors which are orthogonal to all vectors in the space. Existing results show that the filtering problem, LQ problem and risk sensitive problem can be solved effectively based on Krein space estimation theory. Compared with Dynamic Programming approach, it is also more effective to solve the indefinite LQ problem considered in this paper using Krein space estimation theory. For this, it is necessary to establish the duality between them. At first, the definition of Krein space projection and the existence and uniqueness condition are given.
A New Approach to the Indefinite LQ Optimal Control for a Kind of Stochastic Bilinear System with Control Dependent Noises
79
Definition 1 (Projections in Krein space). Given the elements and * + in the Krein space , * + * + denotes the linear subspace generated by * +. We define ̂ to be the projection of onto * + if ̂
* + ̃
̃ ̂
* +
〈̃
〉
Lemma 3. In the Hilbert space, projections always exist and are unique. However, in the Krein space, only if the 〈 〉 Gramian matrix , ̂ can be given uniquely by 〈
̂
〉〈
〉
(7)
For the indefinite LQ problem (1-2) considered in this paper, the cost function (2) is an indefinite quadratic index because and are indefinite. Generally speaking, the maximum point with respect to the control sequence * + does not always exist. So, we should find the stationary point firstly, and then find conditions for the stationary point to be the maximum point. The first step to solve this problem using projections theory in Krein space is to construct the dual system model corresponding to original system. We define new variables * + and * + , following equations can be obtained according to (1)
Let * *
and define the stochastic matrix
+
[
],
+ where [
].
Then, we can get the following equations
Where ,
,
-,
, [
]
(
-
, [
* + and We define the inner terms of the expectation are ∑
=,
] *
[
]
+, and substitute (14) and (15) into the cost function (2), then )
Expanding equation (8), the following equation can be derived.
(8)
80
Xing Guo-jing
Then, taking an expectation over J, it can be obtained that ,
-[
][ ]
(9)
Where
We define such an equation below {
(10)
In which 〈[
] [
]〉
[
〈[
]
] [
]〉
[
]
[
]
By these equations, the indefinite quadratic cost function can be rewritten as ,
-[
][ ]
(11)
Based on projections theorem in Krein space, it is obvious that the maximum point of the indefinite quadratic cost with respect to the control is of the form following (12) It can be seen from equation (12) that the optimal state feedback control gain is actually the minus conjugate transpose of the gain of the filter which is used to estimate from . In order to get the final result, we should + via the minus conjugate transpose of the project onto , and determine the optimal control sequence * gain matrices. The backward dual state space model, whose form is like equation (10), of equation (1) can be given by Lemma. 4. Lemma 4. For the BLSS (1) with control dependent multiplicative noises, there exists backward dual state space model with state dependent multiplicative noises as following. {
(13)
in which
〈
〉 [
] [
[
]
]
Lemma 5. For the backward system (13), its linear least square error estimator is given by ̂
̂
̃
̃
̂
̂
(14)
̂
(15)
(
)
(16)
(
)
(17)
It can be seen from Lemma 5 that the estimator gain is actually the minus conjugate transpose of the state feedback gain given by Lemma 2, and the same GDRE should be solved. Until now, the duality theorem in forms between the indefinite LQ problem and the filtering problem in krein space has been established.
A New Approach to the Indefinite LQ Optimal Control for a Kind of Stochastic Bilinear System with Control Dependent Noises
81
Optimal State Feedback Control based on Duality Theorem In this section, we will give the optimal state feedback control based on the duality theorem. Theorem 1. Considering the indefinite LQ optimal control problem (1-2), the optimal state feedback control be given by
can (18)
and the sufficient and necessary condition for this solution to be maximum is that (19) Proof: From Lemma 5, the linear least square error estimator equation (14) of the dual system (13) can be rewritten as ̂
̂
(20)
based on the observation sequence {
Then, the backward estimation of
̂
[
} is
]
(21) [
]
According to the duality relation obtained in last section, the optimal control sequence in terms of the following equation
[
]
[ [
][
should satisfy
]
(22)
]
By Lemma 3, the condition for the uniqueness and existence of projection in Krein space is
̃
, ie. Equation
(19). If the optimal control exists and is unique, this condition should also be satisfied because the optimal control gain is just the minus conjugate transpose of the gain of the filter. Conclusion In this paper, the Krein space dual state space model of a kind of BLSS with control dependent noises was constructed, and a Krein space approach for indefinite LQ problem was derived. The result extended the existing LQ control theory for BLSS. Moreover, the results here may be used to solve and optimal control problems of BLSS. ACKNOWLEDGEMENT
This research was supported by the National Natural Science Foundation of China (Grant no. 61304130). REFERENCES
[1]
P. J. Mclane. Optimal stochastic control of linear systems with stateand control-dependent disturbances. IEEE Transactions on Automatic Control, 1971, 16(6): 793-798.
[2]
S. G. Loo. Statistical design of optimum multivariable systems whose inputs are corrupted by multiplicative noise. IEEE Transactions on Automatic Control, 1968, 13(3): 300-301.
[3]
W. M. Wonham. Optimal stationary control of a linear system with state-dependent noise. SIAM Journal on Control, 1967, 5(3): 486-500.
[4]
D. L. Kleinman. Optimal stationary control of linear systems with control-dependent noise. IEEE Transactions on Automatic Control, 1969, 14(6): 673-677.
82
[5]
Xing Guo-jing
D. E. Gustafson, J. L. Speyer. Linear minimum-variance estimation and control in systems with state-dependent noise. Proceedings of the 10th IEEE Conference on Decision and Control, 1971, 10(1): 395-400.
[6]
S. P. Chen, X. J. Li, X. Y. Zhou. Stochastic linear quadratic regulators with indefinite control weight costs I. SIAM Journal On Control and Optimization, 1998, 36(5): 1685-1702.
[7]
J. B. Moore, X. Y. Zhou, A. E. B. Lim. Discrete time LQG controls with control dependent noise. System & Control letter, 1999, 36: 199-206.
[8]
M. Athan. The matrix minimum principle. Information& Control, 1968, 11: 592-606.
[9]
S. P. Chen, J. M. Yong. Stochastic linear quadratic optimal control problems. Applied Mathematics & Applications, Springer, 2001, 43: 21-45.
[10] B. Hassibi, A. H. Sayed, T. Kailath. Indefinite-quadratic estimation and control: a unified approach to H2 and H1 theories. Society for Industrial and Applied Mathematics Philadelphia, PA, USA, 1999. [11] M. A. Rami, X. Chen, X. Y. Zhou. Discrete-time indefinite LQ control with state and control dependent noises. Journal of Global Optimization, 2002, 23, pp.245-265. [12] H.S. Zhang, X.M. Song. New estimator and duality of control for multiplicative noise systems with application to packet dropping and multiple input delay systems. IEEE International Conference on Control and Automation, 2009, pp. 75-80.