2004-01-3132
Proposed Real Time Performance Evaluation of Two Types of Adaptive Neural Network Controllers Steve Rogers Institute for Scientific Research, Fairmont, WV srogers@isr.us Copyright © 2004 SAE International
ABSTRACT This paper addresses the problem of real time assessment or evaluation of an adaptive neural network controller performance. Complex control systems require robust controllers that handle a large variety of operating conditions. In these circumstances controllers must be adaptive or robust with respect to inaccurate plant models and changes in the plant dynamics. One of the major difficulties in adaptive controller is proof of stability of the update mechanism. In the case where a conventional controller is available it is convenient to toggle it based on controller performance. This paper presents a proposed approach to evaluate the performance of an adaptive sigma pi neural network controller.
NN’s are used to generate command signals to compensate for errors in the estimates and from the model inversion. A Lyapunov stability proof is said to limit 2 tracking error and bound network weights . The block diagram of the architecture is presented in Figure 1. The main blocks of the architecture are: aircraft or model, PID (proportional, integral, and derivative) controller, inverted dynamics in the model inverse, and adaptive neural network.
Keywords – adaptive neural network controller, real time controller performance evaluation
INTRODUCTION Adaptive neural networks have become popular as an opportunity to reduce flight control development time. Over the past several years, various adaptive control techniques have been investigated with the hope of obtaining a generic flight control architecture that could 1 replace tedious gain scheduling designs . Although gain scheduling has proved very successful, they have also been shown to be aircraft specific. A popular adaptive control approach is to combine direct adaptive control with dynamic inversion. Due to real time constraints, a simplified linear model is normally used for the dynamic inversion. A more sophisticated neural network (NN) is occasionally used for the direct adaptive controller. The NN then must compensate for the model errors of the dynamic inversion and supplement it by improving the tracking performance. A neural flight control architecture has been developed 1,2 by Rysdyk and Calise . This architecture integrates feedback linearization with on-line learning NN’s. The
Figure 1 Block diagram of an Adaptive Neural Network with feedback linearization Feedback linearization is described in numerous texts & papers. This paper contains a brief overview of the sigma pi and the single hidden layer neural networks in conjunction with feedback linearization. It then describes an approach toward real time stability evaluation of them both. The paper is organized as: 1) introduction, 2) sigma pi neural network, 3) single hidden layer neural network, 4) performance indicators, and 5) simulation results.
SIGMA PI NEURAL NETWORK If the dynamic inversion were perfect the closed loop performance would be as good as the reference model. Due to inaccuracies in the model inversion errors are
introduced into the system. Thus, the adaptive neural network controller must learn the inversion bias and dynamic errors caused by model inaccuracies. One such neural network control augmentation approach is called 1,2 a two-layer sigma pi neural network . Inputs into the network consist of control commands, sensor feedback, 1 and bias terms. Table 1 below lists the inputs to the neural network. P-Network
Q-Network
2
2
.1, vt, vt , h
.1, vt, vt , h
C2
.1, r, p, pc,
.1, q, qc,
.1, p, r, rc,
1 − e −(Up −Upad )
1 − e −(Uq −Uqad )
1 − e −(Ur −Urad )
1+ e
1+ e
C3
.1, α, β
.1, α, β
Another option is to take the cubic power or kurtosis of both W and Ue. This provides an evaluator that is more sensitive to changes. The implementation is shown in simulink code in Figure 2.
2
.1, vt, vt , h
− (Uq −Uqad )
xT x
R-Network
C1
− (Up −Upad )
norm( x) =
1 + e −(Ur −Urad )
.1, α, β
Table 1 Sigma Pi input table The output of the sigma pi neural network is the control augmentation command. T
Uad = W B(C1, C2, C3) The vector of basis functions () is computed from the inputs in each signal category using a nested kronecker 2 product. The kronecker product is given below . β = kron(kron(C1, C2 ), C3 )
kron( x, y ) = [x1 y1
x1 y2 L xm yn ]T
Note that Ci is defined in Table 1. The network weights (W) are computed by an adaptation law, which incorporates an adaptation gain (G) and deadband (L), and the command augmentation error (Ue) computed by 1,2 the error handling system .
(
W& = −G U e B + L U e W Ue =
1 2Ki
1 + Ki
∫ e + 2K K i
)
Figure 2 Simulink implementation of real time evaluators for 3 channels (roll-pitch-yaw) In Figure 2 the three types of evaluators are shown. On the left is the discrete pole calculation. On the center right the error norms are calculated. On the lower right the weight norms are computed. The smooth pole block in the upper right is used for smoothing the final evaluators when needed.
e
p
By inspection of the equation for W several evaluators or indicators come to mind. The pole of the differential equation may be immediately written as –GL|Ue| in its continuous form. A simple euler transformation gives the pole in its discrete form (1- ∆GL|Ue|). As long as the pole is in the left half plane or within the unit circle, as applicable, the neural network will be stable. In most applications, G, L, and Ue are scalars or diagonal matrices, which simplifies the computations and makes it feasible for real time applications. Taking a norm of both W and Ue give us two more real time evaluators. The norm is shown below. Figure 3 Low pass smoothing filters
Signal smoothing is implemented by simple low pass filters as shown in Figure 3. Another simple upgrade is to use a Lyapunov energy function, such as L = e T e + W T W , where L is the Lyapunov energy function, e is the tracking error, and W is the weight vector as described above. L combined with L& , which can be computed numerically from L will give two more anomaly indicators. A simulink code block that generates L and L& is shown in Figure 4.
Figure 5 Neural Networks with a single hidden layer 3
The map of Figure 5 may be expressed below . n2
ν adk = bwθ wk +
∑w
wkσ j
(z j ), k = 1,K, n3
j =1
Figure 4 Lyapunov and Lyapunov-Derivative Calculations for 3-channels
SINGLE HIDDEN LAYER (SHL) NEURAL NETWORK The shl neural network is also used as a direct adaptive neural network in conjunction with a feedback linearization technique. Figure 5 presents the 3 architecture of the single hidden layer NN . In the figure, n1 is the number of inputs to the NN, n2 is the number of hidden layer neurons, and n3 is the number of outputs of the NN.
( )
σ j z j = σ bvθ vj +
n1
∑ v x ij i
i =1
See reference 3 for further details. The weight adaptation laws are:
{( & Vˆ = −{x ςWˆ
)
}
& Wˆ = − σˆ − σˆ zVˆ T x ς + λ ς Wˆ Γw T
}
σˆ z + λ ς Vˆ Γv
Γw > 0, Γv > 0, λ > 0
By inspection we may derive a few shl evaluators. They include ς , W + V , and the poles of the differential equations. Since the poles are coupled with transposed matrices, it is difficult to extract the exact poles. However, one may approximate the poles by ignoring the coupling and we have continuous pole estimates,
{ } polesV ≅ −{λ ς }Γv
polesW ≅ − λ ς Γw
. With these 3 calculations we may
evaluate in real time the stability of the shl NN. The simulink implementation is in Figure 6.
low pass filter design using an ARMA (autoregressive moving average) model. The representative equation is n
H ( z) =
∑ (1 − α ) i =1
z1−i n
m
z −i 1− α m i =1
∑
,α = [.85L.99] ,
where
n
is
the
number of numerator coefficients (moving average) and m is the number of denominator coefficients (autoregressive). Note that each of the numerator coefficients are equal and, thus, equally weighted. The same is true with the denominator. If this is not desirable other choices for coefficient weightings are possible as long as they are unity gain. For example, it may be desirable to weight recent data more heavily. Trending represents the rate of the original signal and 4 can be obtained from the output of a Hilbert filter or other type of derivative filter. A typical derivative filter is shown below. H (s) =
sω 2 s 2 + 2ζωs + ω 2
, where s is the Laplace operator, ζ
is the damping coefficient, and ω is the natural frequency of the filter.
Figure 6 Simulink implementation of shl evaluators On the left the discrete time approximate pole for each of the three channels is calculated. At the center right is the error norm. On the lower right is the weight norm. These may be modified as before to represent the kurtosis, which is more sensitive than squaring.
Prediction may be implemented as an FIR (finite impulse 5 response or all zeros) adaptive filter . The number of lags will be somewhat determined by how far ahead the prediction horizon is and the signal frequency content. The number of lags should be sufficient to capture the dominant frequency content of interest. An adaptive filter prediction architecture is shown in Figure 7.
PERFORMANCE INDICATORS As shown above we derived 3 basic types of indicators – poles, error norms, and weight norms. However, these are merely raw indicators. They must be smoothed, trended, and predicted for practical usage in a diagnostic system. Smoothing is usually the result of low pass filtering to remove the higher frequency content of the signal. Trending may be accomplished by band pass 4 filtering of the smoothed signal to remove the DC gain . The trended signal should indicate the direction it is headed. Prediction will indicate a future value of the signal. A predicted value of the signal can be compared to a threshold for stability evaluation. Figure 3 illustrates the simulink code for a typical unity gain IIR (infinite impulse response or having both zeros and poles in the transfer function) low pass filter. IIR filters have transfer functions of the form −1 −2 α + α1 z + α 2 z + L . Figure 3 shows a simple H ( z) = 0 1 − β 1 z −1 − β 2 z − 2 − L
Figure 7 Adaptive prediction filter Threshold logic will be determined based on operational experience and further analyses. The expectation is to develop algorithms that will provide advance notice of performance degradation so that the pilot may take appropriate steps. Only the smoothing part of the above discussion has been implemented to date.
SIMULATION RESULTS
A simulation at a linearized operating point of 0.75 mach and 20k feet altitude of an F15 was used to test the scheme. The typical performance of the pqr response is shown in Figure 8. The input is a series of doublets.
Figure 8 Nominal PQR responses The neural networks controllers are working with the inverse dynamics controllers to provide the tracking as shown in Figure 8.
Figure 9 Nominal sigma pi poles for pqr
The sigma pi NN pole plots for the same simulation are st shown in Figure 9. Poles plots for roll are in the 1 two nd plots. The top plot is the continuous roll pole and the 2 is the discrete roll pole. Discrete pole plots for pitch and yaw are given in the bottom two plots respectively. By inspection, the nominal poles are near perfect integrators. If the sigma pi NN degrades toward instability, we expect the pole to act as an indicator. By arbitrarily increasing the magnitudes of the update law gains, the discrete poles may progress out of the unit circle in the negative direction. The sigma pi NN weight norms for nominal operation are plotted in Figure 10. From top to bottom they correspond to roll, pitch, and yaw channels. They are usually small under nominal conditions, but expand at the onset of instability.
Figure 10 Nominal sigma pi weight norms for pqr
The sigma pi NN error norm is given in Figure 11. As with the weight norms, it is also small in nominal conditions, but expands as performance deteriorates.
Several evaluation criteria for real time monitoring of two types of neural network controllers have been proposed. The criteria for nominal conditions for the sigma pi NN have been plotted. Further research is needed in order to develop the concept, including, shl simulation, testing the evaluators when NN performance is unstable, development/testing of the smoothing, trending, and prediction algorithms.
ACKNOWLEDGMENTS This research was supported by NASA Cooperative Agreement: NCC4-00130.
REFERENCES
Figure 11 Error norm for the sigma pi NN The evaluation criteria for the shl NN is similar during nominal operations and is not shown.
CONCLUSION
1. Kaneshige, J., etal, ‘Generic Neural Flight Control and Autopilot System,’ AIAA Guidance, Navigation, and Control Conf, 2000, Denver, AIAA-2000-4281 2. Rysdyk, R. and Calise, A., ‘Fault Tolerant Flight Control via Adaptive Neural Network Augmentation’ AIAA–4483, August, 1998. 3. Calise, A., etal, ‘Development of a Reconfigurable Flight Control Law for the X-36 Tailless Fighter Aircraft’, AIAA GNC Conference, Aug., 2000, Denver, CO, AIAA-2000-3940. 4. Rogers, S., ‘Sensor Noise Fault Detection’, ACC June, 2003, Denver, CO, FA17-04 5. Haykin, S., Adaptive Filter Theory, Prentice Hall, 1996, ISBN 0-13-322760-X