Scientific Journal of Control Engineering June 2013, Volume 3, Issue 3, PP.106-110
Prediction of Ship Roll Based on Second Diagonal Recurrent Neural Network Liang Xu 1, Zhanying Li 2, Yuzhi Song 3, Yanping Wang 2 1. Hull Workshop, COSCO (Dalian) ship yard Co., Ltd, Dalian 116113, China 2. School of Electronic Engineering and Automation, City Institute, Dalian University of Technology, Dalian 116600, China 3. Det Norske Veritas (China) Co., Ltd. Dalian Branch 116011, China #
Email: l_zy1979@126.com
Abstract An optimized second diagonal recurrent neural network is proposed to develop a model of prediction of ship rolling motion. This approach is based on an algorithm of optimization second diagonal recurrent neural networks (OSDRNN). The stochastic gradient descent algorithm is used to optimize parameters of this network. Using this model to predict the situation of one certain type of ship sailing in the beam sea condition, simulation results show that the optimization of this network improves network performance and the generalization performance of the network, and it has higher prediction on the accuracy and forecast rate. The presented network model used in contrast to SDRNN model can quickly and accurately predict the time series of ship rolling. Keywords: Second Diagonal Recurrent Neural Network (SDRNN); Stochastic Gradient Descent Algorithm; Time Series Prediction; Ship Rolling Motion
1 INTRODUCTION Recently, there have been increasing researches interests of artificial neural networks and many efforts have been made on applications of neural networks of various fields [1-4]. Neural network was extensively used to study on nonlinear control system by many scholars. Most researchers used feed-forward neural network, combined with tapped delays and the back propagation training algorithm (BP) to identify dynamical systems. Then recurrent neural networks and fully connected recurrent neural networks were developed, but these methods needed long time to converge at weights. Diagonal Recurrent Neural Network (DRNN) is proposed by Ku Chao-chee and Lee K wang in 1995[5], DRNN has simple structure and easy realization training method. The self-feedback of the hidden neurons ensures that the outputs of DRNN contain the whole past information of the system even if the inputs of DRNN are only the present states and inputs of the system. The structure of DRNN may be simple than that of recurrent neural networks, so it attracted attention by many scholars. However, the recurrent weights are updated only using the previous state and can’t use other states directly. Many scholars started to make the improvement on the basis of it [612] . In the literature [12], second diagonal recurrent neural network that contains two recurrent weights for every hidden neuron was proposed by Ali Kazemy in 2007, more historical states of neurons can be incorporated directly into the training algorithm in this network. It also raises concern in time series prediction [13-16]. Due to increasing one feedback weight, the structures and parameters of network become complex, thereby it is not easy to select the parameters for network during actual system’s prediction. This article tries to optimize the parameters of secondorder diagonal recurrent neural network on the basis of literature [16], so that improve the network performance. It is used for the ship rolling motion prediction, and by contrast of experiment of non-optimized second-order diagonal recurrent neural network. The accuracy, time and other characters of prediction after optimization of second-order diagonal recurrent neural network can be evaluated. This paper is organized as follows. In section II, one SDRNN model is developed. In Section III, the optimized parameter was considered as to train various weights based on stochastic gradient descent algorithm. In Section IV, a simulation prediction of ship rolling motion will be given. Finally, concludes the paper. - 106 http://www.sj-ce.org/
2 SECOND DIAGONAL RECURRENT NEURAL NETWORKS The diagonal recurrent neural network proposed by Ku Chao-chee [5] cannot use previous state more due to only one feedback weight in the hidden layer. It can be only updated based on the current state so that a certain error exists in the real-time prediction to dynamic system. Against this defect, Ali Kazemy proposed one improved diagonal recurrent neural network with two feedback weights based on DRNN network, called second diagonal recurrent neural network (SDRNN). This network includes two feedback weights, and it can bring more historical states into learning methods. Compared to DRNN network, it has more rapid and accurate identifications. It is also concerned with respect to prediction of time series. SDRNN network structure is shown in Fig. 1. I
W ij
O
Wj
u i（k）
O （k）
D1
Wj
TD
S j （k）
TD D2
ρ
Wj
C j （k） FIG.1 SECOND DIAGONAL RECURRENT NEURAL NETWORK STRUCTURE
Mathematical model for this network is given by：
y(k ) O(k ) W o S (k ) S (k ) f (C(k ))
C (k ) W U W S (k 1) W S (k 2) I
D1
D2
(1)
Where f function. It consists of three layers: input layer, hidden layer and output layer. The number of neurons of each layer is n , h and 1 respectively. W I is n h dimensional matrix, W D1 , W D2 is h h dimensional matrix, W O is h 1 dimensional matrix, dynamic training algorithm for second Diagonal recurrent neural network is in reference [12]，and not described in this article.
3 PARAMETER OPTIMIZATION OF SDRNN When applying SDRNN to forecast practical problems, the choice of parameters is one important factor to affect the accuracy. The selection of network parameters will be different due to different objects of prediction. The current parameter choice is normally based on the previous experiences, but it will ignore the unique character of predicted objects with fully depending on experiences. Once the objects are changed, it results in the poor forecasting. So it is very important to select optimized parameters for model when carrying out the model prediction. To verify the performance of new network, the optimization of parameters must be considered. In SDRNN neural network, its main parameters include input weights matrix W I , hidden layer weights matrix, W D1 , W D2 and output layer weights matrix W O . W I , W D1 and W D2 are built before network learning, W O is to be calculated after learning. Therefore, optimizing network parameters is optimization of W I , W D1 and W D2 . Directly optimize matrix is more difficult, so the article improves formula (1) in order to solve this problem.
C ( k ) W U 1W S ( k 1) 2W S ( k 2) I
D
1
D
2
(2)
W , W and W are unit matrix, is scale factor of input layer weights, 1 and 2 are scale factors of hidden layer feedback weights. The network hidden layer after modification has three more parameters compared with original SDRNN network. The structures of network can be optimized through optimizing these three parameters. By using method of stochastic gradient descent to optimize these three parameters, let y (k ) and yd (k ) be the real and desired outputs respectively. e( k ) is the error between the plant and the network response. The error cost function is defined by E(k ) . I
D1
D2
e( k ) y ( k ) y d ( k ) - 107 http://www.sj-ce.org/
E (k )
1
y ( k ) yd ( k )
2
2 According to LMS rule, , 1 and 2 are optimized by stochastic gradient descent algorithm. Taking as example, is parameter update rate of , the parameter update formula is as follows: ( k 1) ( k ) ( E ( k )
S (k )
E ( k )
e ( k ) W [ T
f ' (C (k )) (W I U 1W
O
D1
)
S ( k )
S (k 1)
]
2W D 2
S (k 2)
In the same way S (k ) 1 S (k ) 2
f ' (C (k )) (W S (k 1) 1W D1
f ' (C (k )) ( 1W
D1
S (k 1) 2
D1
S (k 1) 1
2W D 2
W S (k 2) 2W D 2 D2
S (k 2) 1 S (k 2) 2
4 SIMULATION OF PREDICTION OF SHIP ROLLING MOTION A ship has six degrees of free movements in waves: rolling, pitching, yawing, swaying, surging, and heaving. Among them, rolling has the biggest influence on ship’s movements, if the rolling angle under the function of wind, wave and flows can be predicted beforehand, which will be very important because if one can look 5 to 10 seconds ahead of time, then one can time the landing to avoid a serious crash in the case of aircraft landing. However it is very difficult to forecast and analyze the time series due to the serious effect on non-linear factor of ship motion. Based-on NN models have been successfully applied and well accepted in numerous practical problems [17-20]. Literature [16] proposes one way that using second-order diagonal recurrent neural network to forecast the ship rolling motion, and achieve better results. On this basis, the optimized SDRNN network used for simulation prediction of ship rolling motion has been provided in article. Meanwhile, the results compared with original SDRNN network are also given. The proposed approach was programmed based on the platform Matlab7.1. Below is one typical example of time series prediction of ship rolling motion, using SDRNN neural network and OSDRNN neural network separately to carry on the prediction of ship rolling motion. The data used for this prediction are from experimental data of one certain vessel navigating on the beam sea condition. There are 1012 data (506s), 400s of which are used for learning and 401s-420s are used for predicting the efficiency of test model and algorithm. Structure of the network is selected 2-15-1. The network has two inputs which are roll angles of the current time measured and the moment before network output; Select 15 delay self-feedback neurons in hidden layer; network output is predicted roll angle in the next time. Using of parameter optimization SDRNN to predict, and compare the results with SDRNN network. Parameter update rate =0.1 , Based on least squares criterion, the stochastic gradient descent algorithm is used to optimize W I , W D1 and W D2 , the initial value of , 1 , 2 is 1, optimized parameters by calculation is 0.6232 , 1 0.7561 , 2 0.0231 . Prediction of Ship rolling motion based on SDRNN and OSDRNN are shown in Fig. 2, Fig.3 shows predicted error curves. In Fig. 2, the red real line is actual output from system, the blue dot line is predicted output of SDRNN, and the black dash line is predicted output of OSRDNN. In Fig. 3, the blue real line is predicted error curves of SDRNN, and the black dash line is predicted error curves of OSDRNN. From the comparison of the two forecasting results, the optimized SDRNN network has a better prediction in accuracy and speed. Taking forecast of 40 steps and 20 seconds for example, the RMS error predicted by optimized SDRNN network is 0.0085 and increases three time compared with RMS error 0.0295 by original SDRNN network. And in contrast to the same environment with network drilling 2000 steps and MSE 0.001, the optimized SDRNN - 108 http://www.sj-ce.org/
network uses only 18.5601s and is less than 3s with regard to 21.9197 s by non-optimized SDRNN from the beginning of training to the end of prediction. Taking the minimum training error as target to optimize the network parameters, which can be provided based on different objects. It improves performance and generalization functions of network. But the forecast accuracy is still too large after 17 seconds, which relates to the characteristics of memory fading in recurrent network. prediction error curve 0.3
0.5
0.2
0
0.1
-0.5
0
error
roll angle/°
prediction of ship rolling motion 1
-1 -1.5
-0.1 -0.2
-2
-0.3
real prediction of SDRNN prediction of OSDRNN
-2.5 -3
SDRNN OSDRNN
OSDRNN
0
5
10
15
20 t/s
25
30
35
FIG.2 PREDICTION RESULTS OF SDRNN AND OPTIMIZED SDRNN
SDRNN -0.4
40
-0.5
0
5
10
15
20 t/s
25
30
35
40
FIG.3 PREDICTION ERROR CURVES OF SDRNN AND OPTIMIZED SDRNN
5 CONCLUSIONS In the work presented here, this article come up with one new method to predict time series for ship rolling motion. An application of optimized second diagonal recurrent neural networks to the prediction of ship roll motion has been described. The optimization of this network improves network performance and the generalization performance of the network, and it has better predicted accuracy and forecast rate contrast to SDRNN model. The main goal of this paper is to provide ship roll motion data to a detection forecasting results based on optimized SDRNN. Although the study has been limited to predicting of ship rolling motion, the application of this method to any other type of ship motion. Our future work in this line will be focused on improving prediction time. At the same time, it will also be necessary to compare the forecast results obtained with the recurrent neural network method to other time series forecasting approaches that could be appropriate for dealing with the highly nonlinear phenomena in ship rolling motion.
REFERENCES [1]
Pena, Fernando Lopez; Gonzalez, Marcos Miguez; Casas, Vicente Diaz; Duro, Richard J.” Ship roll motion time series forecasting using neural networks”. IEEE International Conference on Computational Intelligence for Measurement Systems and Applications (CIMSA). 2011, 9:1-6
[2]
De Masi, G., Gaggiotti, F., Bruschi, R.,Venturi, M. “Ship motion prediction by radial basis neural networks.” IEEE Workshop on Hybrid Intelligent Models and Applications (HIMA). 2011, 4:28-32
[3]
Liu L.S., Peng X.F.. “Diagonal recurrent neural network with output feedback and its application”. The 6th International Conference on Computer Science & Education (ICCSE 2011). 2011, 8: 286-288
[4]
Mu YQ, Sheng AD, Guo Z. “Evolutionary diagonal recurrent neural network for nonlinear dynamic system identification”. Proc. 2008 IEEE International Conference on Networking, Sensing and Control. 2008: 837-841
[5]
Ku Chao-Chee, Lee Kwang. ”Diagonal Recurrent Neural Networks for Dynamic Systems Control”.IEEE Transactions on Neural Networks. 1995, 6(1):144-156
[6]
P. A. Mastorocostas and J. B. Theocharis, “On stable learning of block-diagonal recurrent neural networks, part I: the RENNCOM algorithm”. IEEE International Joint Conference on Neural Networks, vol. 2, 2004: 815-820
[7]
P. A. Mastorocostas and J. B. Theocharis, “On stable learning of block-diagonal recurrent neural networks, part II: application to the analysis of lung sounds”. IEEE International Joint Conference on Neural Networks, vol. 2, 2004: 821-826
[8]
P. A. Mastorocostas and J. B. Theocharis, “A stable learning algorithm for block-diagonal recurrent neural networks: application to the analysis of lung sounds”. IEEE Trans. Syst., Man, Cybern. – Part B: Cybernetics, vol. 36. no. 2, 2006: 242-254 - 109 http://www.sj-ce.org/
[9]
S. C. Sivakumar, W. Robertson and W. J. Philips, “On-line stabilization of block-diagonal recurrent neural networks,” IEEE Trans. Neural Nets., vol. 10, no.1, 1999:167-175
[10] P. A. Mastorocostas, D. Varsamis, C. Mastorocostas and I. Rekanos, “An accelerating learning algorithm for block-diagonal recurrent neural networks”. Intern. Conf. Comput. Intelligence for Modelling, Contr. And Automat., Vienna, Austria, 2005 [11] Zheng Liying. “Study and Application on Chaotic Neural Network and Fuzzy Chaotic Neural Network”. Harbin Engineering Universty, 2002: 61-64 [12] Kazemy Ali, Hosseini Seyed Amin and Farrokhi Mohammad.”Second Order Diagonal Recurrent Neural Network”. IEEE International Symposium on Industrial Electronics, Vigo, 2007: 251-256 [13] Wang Kejun, Li Guobin.”Time Series Prediction of Ship Roll Using Diagonal Recurrent Neural Network”..Journal of Harbin Engineering University, 1997, 18(1):39-45 [14] Azemy, A.,Hosseini, S.A., Farrokhi, M.. “Second Order Diagonal Recurrent Neural Network”. 2007 IEEE International Symposium on Industrial Electronics (ISIE 2007). Vigo, Spain, 2007: 251-256 [15] Shen Yan, Ju Xianlong, Liu Chunxue. “Application of Second Order Diagonal Recurrent Neural Network in Nonlinear System Identification”. 2010 International Conference on Web Information Systems and Mining (WISM). Harbin, China, 2010: 420-424 [16] Li Zhanying, Wang Kejun, Zhang Mingjun, etal. “Second Diagonal Recurrent Neural Network Approach to Ship Roll Prediction”. Journal of Huazhong University of Science and Technology, 2011, 6(39):125-128 [17] Viorel Nicolau, Dorel Aiordachioaie, Rustem Popa Wiener. ”Neural Network Prediction of the Wave on the Yaw Motion of a Ship Influenc”. IEEE International Joint Conference on Neural Networks, Budapest, Hungary. 2004: 2081-2086 [18] Pena, Fernando Lopez, Gonzalez, Marcos Miguez, Casas, Vicente Diaz, Duro Richard J.” Ship roll motion time series forecasting using neural networks”. IEEE International Conference on Computational Intelligence for Measurement Systems and Applications (CIMSA). 2011, 9:1-6 [19] De Masi, G., Gaggiotti, F., Bruschi, R., Venturi, M. “Ship motion prediction by radial basis neural networks”. IEEE Workshop on Hybrid Intelligent Models and Applications (HIMA). 2011, 4: 28-32 [20] FU Huixuan, WANG Yuchao, DU Chunyang . “Ship Course Prediction Based on PSO Combined with BP Neural Network”. Proceedings of the 30th Chinese Control Conference, Yantai, China: IEEE, 2011, 7: 2748-2752
AUTHORS 1
3
on March 31, 1978. Bachelor's degree in
on January 17, 1980. Master's degree in
Shipbuilding Engineering at the Harbin
ship and ocean structure's design and
Engineering
Liang Xu was born in Shandong, China,
University,
in
Yuzhi Song was born in Harbin, China,
2002.
construction at the Harbin Engineering
Engineer. Mr. Xu’s areas of interest is
University, in 2006. Assistant Engineer.
Hull Design.
Mr. Song’s areas of interest are Hull Design and homonized common structure
2
Zhanying Li was born in Jilin, China,
on August 1, 1979. Ph.D. degree in Pattern
Recognition
and
Intelligent
Systems at the Harbin Engineering University, in 2012. Associate professor. Dr. Li’s areas of interests are intelligent control, neural networks and time series prediction.
- 110 http://www.sj-ce.org/
rules.