Dilruba Sharmin et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 2, Issue No. 1, 099 - 103
A study on the parameters of backpropagation artificial neural network temperature prediction model Dilruba Sharmin,Farzana Hussain,Md.Shafiqur Rahman,Susmita Ghose,T.K.Yousufzai,Mahfuja Akter Dept. of Applied Physics Electronics & Communication Engineering University of Dhaka Dhaka,Bangladesh Email:shafiqrahman50@yahoo.com II.
BASIC CONCEPTS OF NEURAL NETWORK
T
An Artificial Neural Network (ANN) is an information processing paradigm that is inspired by the way biological nervous systems, such as the brain, process information. The key element of this paradigm is the novel structure of the information processing system. It is composed of a large number of highly interconnected processing elements (neurons) working in unison to solve specific problems. ANNs, like people, learn by example. An ANN is configured for a specific application, such as pattern recognition or data classification, through a learning process. Learning in biological systems involves adjustments to the synaptic connections that exist between the neurons. This is true of ANNs as well.
I.
A
ES
Abstract—The aim of this work is to study the modeling process of artificial neural networks (ANN) to predict the maximum and minimum temperature of Dhaka, the capital of Bangladesh. A feed forward multilayer ANN namely Temperature predicting Neural Network (TPNN) has been developed and trained using back propagation learning algorithm to test its prediction capability. The TPNN was constructed, trained, and tested with the software developed in Visual C++. The parameters of the neural network have been varied and the corresponding predictive results were recorded. The parameters studied in this research were the learning rate, momentum factor and number of neurons in the hidden layer. The model was trained and tested using nine years (1992-2000) meteorological data. Inputs of the neural network were daily maximum temperature, minimum temperature, average temperature, rainfall, humidity, sunshine hours and wind speed of the previous day, and the output was the maximum or minimum temperature of the day. The data for the years 1992-1999 were used in training phase while that for the year 2000 were used to test the model. The accuracy of the model was calculated and the mean relative percentage error for the TPNN model was 7.45826% for maximum temperature and 8.655804% for minimum temperature prediction. The result shows that the proposed TPNN introduced a good accurate prediction for the daily maximum and minimum temperature.
INTRODUCTION
IJ
Weather prediction is a complex process and a challenging task for researchers. It includes expertise in multiple disciplines [1], [2]. The prediction of atmospheric parameters is essential for various applications. Some of them include climate monitoring, drought detection, severe weather prediction, agriculture and production, planning in energy industry, aviation industry, communication, pollution dispersal etc. Accurate prediction of weather parameters is a difficult task due to the dynamic nature of atmosphere. Various techniques like linear regression, auto regression, Multi Layer Perception, Radial Basis Function networks are applied to predict atmospheric parameters like temperature, wind speed, rainfall, meteorological pollution etc. [3], [4], [5], [6], [7], [8]. It was found that the non linear operator equations governing the atmospheric system are the ones who can better understand the dynamics of atmosphere. In the recent past many forecast methods have been developed using Artificial Neural Networks.
Fig-1: Neural Network
It consists of Input layer, Hidden Layer(s) and output layer. Can be trained with input-output data pattern. Can be tested for new input data
A. Model of an artificial neuron The Model of a simple artificial neuron is shown in figure: 2.It receives n inputs x1, x2… xn with weights w1, w2… wn attached to the input links. The weighted sum of inputs I=∑wi.xi is computed to be passed on to a nonlinear filter Ф, called activation function to release the output Ф (I). Here Ф could be a step function, signum function, sigmoidal function, or hyperbolic tangent function.
Our study was based on Multi Layer Perception (MLP) which trained and tested using past nine years (1992-2000) meteorological data. The objective of this study is to develop ANN-based model by using meteorological data of Dhaka city located in Center of Bangladesh for one year ahead forecasting of temperature of this area.
ISSN: 2230-7818
@ 2011 http://www.ijaest.iserp.org. All rights Reserved.
Page 99
Dilruba Sharmin et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 2, Issue No. 1, 099 - 103
III.
DATA COLLECTION & PROCESSING
Meteorological data of Dhaka station were collected for a period of nine years(1992-2000) Data of 1992-1999 were used to train All data of the year 2000 were used to test Normalized data were used (Data) normalized = [(Data) actual – (Data) min)] / [(Data) max–(Data) min]
B. Application of ANN
Non-linear system modeling. Forecasting and risk assessment. Pattern recognition (PR)/image processing. Neural networks in medicine. Neural networks in business & marketing.
SYSTEM MODEL
The network receives the value of maximum temperature, minimum temperature, average temperature, bright sunshine hour, humidity, rainfall, wind speed of 1992 to 1999 as input, and predicts the value of maximum and minimum temperature of 2000 as output. The training process of the NNs used a set of input-output data pairs. In this work Maximum &minimum temperature of a day is predicted based on the maximum and minimum temperature of previous n days. The available data is divided into training, and test sets. Training set is used to train the model, and test set is used to evaluate the output A.
TPNN modelling
Input layer: consists of seven nodes that are maximum temperature, minimum temperature, average temperature, bright sunshine hour, humidity, rainfall, wind speed. The input signals to the input layer are directly passed to the next layer (hidden layer) without any computation or modifications. It uses linear activation function. Hidden layer: which receive signal from the input layer through the weights and send their outputs to the nodes of the output layer. The nodes of hidden layer use the sigmoidal activation function.
ES
C. Backpropagation learning Consider the network as shown in figure: 3 where the subscripts I, H and O denote input, hidden and output neurons.
IV.
T
Fig 2: Simple model of an artificial neuron
ISSN: 2230-7818
Max temp 45.00 40.00 35.00 30.00 25.00 20.00 15.00 10.00
@ 2011 http://www.ijaest.iserp.org. All rights Reserved.
20-03-1998
03-03-1999
Max temp
23-04-1996
E. Backpropagation algorithm The basic algorithm loop structure is given below: Initialize the weights Repeat For each training pattern Train on that pattern End Until the error is acceptably low A computer program was developed using Visual C++ 6.0 for training the data and inferring the result using the algorithm.
B. Data for Training
06-04-1997
IJ
Forward pass: input signal propagate from the network input to the output. Reverse pass: calculated error signals propagate backwards through the network to adjust the weight.
28-05-1994
Output layer: contains only one node that receives its input from the output of neurons in the hidden layer through the weights. The node of output layer also uses the sigmoidal activation function.
11-05-1995
D. Training of an ANN involves two passes
01-07-1992
Fig 3: Multilayer feedforward backpropagation network [9].
14-06-1993
A
Page 100
Dilruba Sharmin et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 2, Issue No. 1, 099 - 103
RF (mm)
MIN TFMP
AVG TEMP 35.00 30.00 25.00 20.00 15.00
AvgT
10.00 5.00
RH (% ) 120.000
Rh (%)
60.000 40.000 20.000
23-06-1999
09-12-1998
27-05-1998
12-11-1997
30-04-1997
16-10-1996
03-04-1996
20-09-1995
08-03-1995
24-08-1994
09-02-1994
28-07-1993
13-01-1993
01-07-1992
0.000
V.
WSP 12.000 10.000 8.000
WP
22-12-1999
23-06-1999
23-12-1998
24-06-1998
24-12-1997
25-06-1997
28-06-1995
28-12-1994
29-06-1994
29-12-1993
30-06-1993
30-12-1992
01-07-1992
0.000
25-12-1996
2.000
26-06-1996
4.000
27-12-1995
A
6.000
IJ
BSSH
01-08-1999
14-01-1999
29-06-1998
12-12-1997
27-05-1997
09-11-1996
24-04-1996
VARRIATION OF PARAMETER
A. Effect of no of neuron in hidden layer The Visual C++ program that was developer allows 1 to 21 neurons in the hidden layer. But the error in learning process varies for different number neurons in the hidden layer. In the experiment, it is found that error is minimum for maximum temperature using 3 neurons and for minimum temperature using 10 neurons in the hidden layer but the variations are negligible if more neurons are added in the hidden layer.
BSSH
18-08-1999
13-02-1999
11-08-1998
06-02-1998
04-08-1997
30-01-1997
28-07-1996
24-01-1996
22-07-1995
17-01-1995
15-07-1994
10-01-1994
08-07-1993
03-01-1993
16.00 14.00 12.00 10.00 8.00 6.00 4.00 2.00 0.00
01-07-1992
D. Network parameter issues This research attempts to understand the network parameters by varying them and observing their effect to on the network. The parametric effect of varying the No of hidden neuron(HN) Momentum factor (MF). Learning rate coefficient (LR). No of iteration.
ES
100.000 80.000
C. Training of TPNN Training inputs: Previous day’s maximum temperature. Minimum temperature. Average temperature. Bright sunshine hour. Humidity. Rainfall. Wind speed. Training output: Maximum temperature of the day or Minimum temperature of the day.
T
22-12-1999
23-06-1999
23-12-1998
24-06-1998
24-12-1997
25-06-1997
25-12-1996
26-06-1996
27-12-1995
28-06-1995
28-12-1994
29-06-1994
29-12-1993
30-06-1993
30-12-1992
01-07-1992
0.00
08-10-1995
05-10-1999
14-03-1999
23-08-1998
31-01-1998
11-07-1997
19-12-1996
29-05-1996
07-11-1995
17-04-1995
25-09-1994
05-03-1994
13-08-1993
21-01-1993
01-07-1992
0.00
23-03-1995
01-07-1992
10.00 5.00
RF (mm)
05-09-1994
MnT
18-02-1994
20.00 15.00
03-08-1993
30.00 25.00
16-01-1993
35.00
160.00 140.00 120.00 100.00 80.00 60.00 40.00 20.00 0.00
Fig: Effect of no of neuron in hidden layer (for maximum temperature)
ISSN: 2230-7818
@ 2011 http://www.ijaest.iserp.org. All rights Reserved.
Page 101
Dilruba Sharmin et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 2, Issue No. 1, 099 - 103 Fig: Effect of adding a momentum term (for minimum temperature)
VARIATION OF HN 2 HN
3 HN
4 HN
7 HN
10 HN
12 HN
0.054000 0.053000 0.052000
The effect of momentum factor during training the NN is shown in figure From above figure it is found that the optimum value of momentum factor is 0.5 for maximum temperature and 0.2 for minimum temperature.
0.051000 0.050000 0.049000
D.
0.048000
Convergence error for NN training process
0.047000 1
4
7
10 13 16 19 22 25 28 31 34 37 40 43 46 49
Fig: Effect of no of neurons in hidden layer (for minimum temperature)
B. Effect of learning rate coefficient () VARIATION OF LR 0.1 LR
0.3 LR
0.5 LR
0.7 LR
0.9 LR
0.054 0.053
Fig: Convergence error for the NN (for maximum temperature)
0.052
T
0.051 0.05 0.049 1
4
7
10 13 16 19 22 25 28 31 34 37 40 43 46 49
Fig: Effect of learning rate coefficient (for maximum temperature) VARIATION OF LR 0.3 LR
0.4 LR
0.7 LR
Series6
ES
0.2 LR 0.050000 0.049500 0.049000
Fig: Convergence error for the NN (for minimum temperature)
0.048500 0.048000 0.047500 0.047000 1
4
7
10 13 16 19 22 25 28 31 34 37 40 43 46 49
Fig: Effect of learning rate coefficient (for minimum temperature)
A
The figure shows the effect of learning rate coefficient during training process of NN. It shows that the error is minimum when the learning rate coefficient for max temp is 0.3 and for min temp is 0.2.
C. Effect of adding a momentum term The value of momentum coefficient should be positive but less than 1. Typical values lie in the range 0.5-0.9.
From figure, it is found that the error decreases if the training process takes more iteration. E. Adjusted parameter Parameter No of hidden layer No of hidden layer neuron Learning rate coefficient Momentum factor No of iteration Sigmoid function VI.
IJ
VARIATION OF MF
0.2 MF
0.5 MF
0.6 MF
0.7 MF
0.9 MF
1.0 MF
0.054000
0.053000 0.052000
0.051000 0.050000
Max temp 1 3
Min temp 1 10
0.3 0.5 1000 1
0.7 0.2 1000 1
TESTING TPNN
After training process, TPNN with adjusted has been used for testing. After testing process, predicted output for new testing data sets has been provided by TPNN. The performances of ANN are then compared with actual data.
0.049000
1
4
7
10 13 16 19 22 25 28 31 34 37 40 43 46 49
Fig: Effect of adding a momentum term (for maximum temperature) VARIATION OF MF 0.2 MF
0.5 MF
0.9 MF
0.6 MF
0.7 MF
1.0 MF
0.052000000 0.051000000 0.050000000 0.049000000 0.048000000 0.047000000 1
4
7 10 13 16 19 22 25 28 31 34 37 40 43 46 49
ISSN: 2230-7818
@ 2011 http://www.ijaest.iserp.org. All rights Reserved.
Page 102
Dilruba Sharmin et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 2, Issue No. 1, 099 - 103 Engineering Applications of Artificial. [8] Wei-Zhen Lu. Wen-Jian Wang. “Potential assessment of the support Vector machine method in forecasting ambient air Pollutant trends” Chemosphere, 59, pp.693-701. 2005ntelligence, 20, pp.745-755. 2007, Algorithm and Applications, Prentice Hall, Englewood cliffs. [9] Rajsekaran & G.A. Bijayalaksmi pai, neural networks, Fuzzy logic and Genetic Algorithms, synthesis and Applications.
A. Performance of TPNN
VII. RESULT
ES
Fig: Comparison of relative percentage errors while predicting closing minimum temperature.
T
Fig: Comparison of relative percentage errors while predicting closing maximum temperature.
The accuracy of the model was calculated. Mean relative percentage error: 7.45826% for maximum temperature prediction and 8.65580% for minimum temperature prediction Result shows that the ANN introduces a good accurate prediction. VIII. CONCLUSION
Neural network could be an important tool for temperature prediction. For better performance Long term data should be used. Related more input variables may be used. Number of hidden layer may varied.
IJ
A
References
[1] Denis Riordan, and Bjarne K Hansen, “A fuzzy case-based system for weather prediction.”Engineering Intelligent Systems, Vol .10, No.3. 2002. [2] Guhathakurtha, P., “Long-Range monsoon rainfall prediction of 2005 for the districts and sub-division Kerala with artificial neural network. .” Current Science, Vol.90, No.6, 25. 2006. [3] Jae H.Min., Young-chan Lee. “Bankruptcy prediction using support Vector machine with optimal choice of kernel function parameters. .”Expert Systems with Applications, 28, pp.603-614. 2005. [4] Mohandes, M.A., Halawani, T.O., Rehman, S and Ahmed Hussain, A. “Support vector machines for wind speed prediction.” Renewable Energy, 29, pp.939-947. 2004 [5] Pal, N.R., Srimanta Pal, Jyotirmoy Das, and Kausik Majumdar, “SOFM-MLP: Hybrid Neural Network for Atmospheric Temperature Prediction...”IEEE Transactions on Geoscience and Remote Sensing, Vol.41, No, 12, pp.2783-2791. 2003. [6] Pao-Shan Yu., Shein-sung Chen., I-Fan Chang. “Support vector regression for real- time flood stage forecasting.” Journal of Hydrology, 328, pp. 704-716. 2006. [7] Stanislaw Osowski and Konrad Garanty, “Forecasting of daily Meteorological pollution using wavelets and support vector machine”
ISSN: 2230-7818
@ 2011 http://www.ijaest.iserp.org. All rights Reserved.
Page 103