Statistical Model and Neural Networks based Weather Forecasting

GRD Journals- Global Research and Development Journal for Engineering | Volume 4 | Issue 10 | September 2019 ISSN: 2455-5703

Statistical Model and Neural Networks based Weather Forecasting A. Velayudham Department of Computer Science and Engineering Jansons Institute of Technology

M. S. Krishna Priya Department of Computer Science and Engineering Jansons Institute of Technology

Abstract Weather conditions change endlessly and it is very essential for people to know these changes and predict the weather in order to decide there day to day task. A person may confuse prediction with the forecast, the main difference is forecast is statistical and scientific and free from intuitions whereas prediction is subjective in nature. Traditionally atmosphere is modeled as fluid. And future weather is predicted using fluid dynamics and thermodynamics, but since machine learning techniques are more robust to perturbations, so in this paper we have decided to explore its application to weather forecasting to generate more accurate result.

This paper discusses various methods used in deep neural networks and statistical models to find the most appropriate method for forecasting weather based on a very minimalistic weather related data set to reduce data collection overheads. It also outlines an IOT based pipeline to collect data in real time to update existing models and forecast accordingly.

Keywords- Deep Neural Networks, Statistical Models, Minimalistic Weather Data, IoT, Pipeline, Weather Forecasting, Prediction, Atmosphere

I. INTRODUCTION Weather forecasting can be defined as the process of predicting the state of atmosphere at a particular time in future for a specified location. Traditionally we used to model the atmosphere as fluid. And future weather is predicted using fluid dynamics and thermodynamics, but the problem is weather depends on various factors and these traditional models are not robust enough to perturbations and are hence less accurate and are unable to make predictions of a long period.

Machine learning techniques changed this process completely and made more accurate reliable predictions as they are more robust to perturbations. Initially major weather forecasting agencies were using machine learning as their primary analytics tool but were unable to predict weather over a long time period. A more suitable approach towards such data’s analysis came with the rise of deep neural networks which is being used at a very high scale these days.

The paper discusses and evaluates various prominently used machine learning and deep learning models e.g. ridge regression model, Support Vector Regression, ARIMA time series model, Holt Winters model and LSTM Recurrent Neural Network model to predict the average temperature, based on a very minimalistic data set comprising of maximum temperature and minimum temperature of previous day. The historical weather data was collected for New Delhi from National Oceanic and Atmospheric Administration. The input to these models was the historical weather data of last 3 years of average temperature, maximum temperature and minimum temperature, and the output was the prediction of average temperature of next 1 year.

II. METHODOLOGY A. Linear Regression Model Linear regression is a linear approach to model the relationship between a dependent variable and one or more than one independent variables. Linear regression models uses the least squares approach for fitting purpose but when the problem has substantial uncertainties in the independent variable (the x variable), then linear regression and least-squares methods have problems. The linear regression model is in the form –

Where x0 is min temp and x1 is max temperature.

To speed up the calculation we can use Ordinary least squares using following equation –

Here X is 2xN matrix having each column as features min temp and max temp.

For unbiased result an estimator is used whereas,

All rights reserved by www.grdjournals.com

6

Statistical Model and Neural Networks based Weather Forecasting (GRDJE/ Volume 4 / Issue 10 / 002)

B. Ridge Regression Model Ridge regression is most commonly used method of regularization of ill-posed problems (If no independent variable satisfy the equation or more than one independent variable does that is, the solution is not unique).

For giving weightage to the desired attributes a regularization term can be used in form of Tikhonov Regularization – It is same as Linear regression but with a Tikhonov Matrix as. If alpha is the regularization parameter we can choose Tikhonov Matrix as a scalar multiplication with a 2I identity matrix as – We used 1 year data of average temperature to train our model and predicted the average temperature of next 1 year and compared it with the test data. This comparison is shown below in graph.

Fig. 1: Forecasted vs. Actual Data Table 1: Model Details Location MUMBAI, IN Model Ridge Regression Data Average Temperature Training Data period 1 August 2017- 31 July 2018 Testing Data period 1 August 2018- 31 July 2019

As from the above graph, the accuracy of the model was 88.84%, but there was a problem with this model as soon as we provided data of past 10 years our model was unable to predict and accuracy dropped to a vast extent. So we used another model. That is ARIMA model.

C. ARIMA Model In Auto Regressive Integrated Moving Average model data is fitted to time series format to better understand the data and to predict future points in the series. Generally ARIMA models are applied in cases where data show evidence of non-stationary, where an initial differencing step can be applied one or more times to eliminate the non-stationarity. Here: Auto Regressive (AR) indicates that the evolving variables (feature: average temperature) of our interest is regressed on its own prior values.

Integrated (I) indicates that the data values have been replaced with the difference between their values and the previous values.

Moving Average (MA) indicates that the regression error is actually a linear combination of error terms whose values occurred contemporaneously and at various times in the past.

ARIMA model is generally denoted ARIMA (p,d,q) where parameters p,d, and q are non-negative integers. ‘p’ is the order (number of time lags) of the autoregressive model. ‘d’ is the degree of differencing (the number of times the differencing is done). ‘q’ is the order of the moving-average model.

If we consider θ1 and Θ1 parameterization terms then the corresponding equation for the ARIMA 2,0,1 can be written as following ϕ0(B)(1−B)2Φ0(B12)(1−B12)0Yt=θ1(B)Θ1(B12)et ϕ0(B)=Φ0(B12)=1, θ1(B)=1+θ1B Θ1(B12)=1+Θ1B12 And the result is Yt−2Yt−1+Yt−2=et+Θ1et−12+θ1et−1+θ1Θ1et−13

All rights reserved by www.grdjournals.com

7

Statistical Model and Neural Networks based Weather Forecasting (GRDJE/ Volume 4 / Issue 10 / 002)

We used past 6 years data of average temperature to train our model and predicted the average temperature of next 1 year and compared it with the test data. As seen in graph.

Fig. 2: Data Log Difference

Fig. 3: Forecast vs. Actual Data Table 2: Model Details Location MUMBAI, IN Model ARIMA Data Average Temperature Training Data period 1 August 2011- 31 July 2018 Testing Data period 1 August 2018- 31 July 2019

As of the above graph, the accuracy of our model was 82.19%, but accuracy is not enough. So we used another model. That is LSTM model.

D. Holts Winters Model Holt-winter is used for exponential smoothing to make short term forecasts by using multiplicative models with increasing or decreasing trend and seasonality. Smoothing is measured by beta and gamma parameters in holt’s model.

If the beta parameter is set FALSE, the function performs exponential smoothing.

The gamma parameter is used for the seasonal component. If the gamma parameter is set to FALSE, a non-seasonal model is fitted.

All rights reserved by www.grdjournals.com

8

Statistical Model and Neural Networks based Weather Forecasting (GRDJE/ Volume 4 / Issue 10 / 002)

E. LSTM Neural Networks Model When we consider weather data a person may think that the current weather condition depends only on data of past one week instead of data of past few years but it is false sometimes older data helps the system to recognize trends that recent data is not able to show. So a problem of Long-Term Dependencies arises to which the general RNNs are not robust.

Thus LSTMs are used, these type of RNNs are capable of dealing with Long-term Dependencies. LSTM remembers the information for the longer period and has a total control over its value and can change its information any time.

The working of LSTM in the context of our dataset can be represented with following equations –

Here xt are our features for tth iteration and yt-1 is the corresponding actual average temperature in the previous iteration. bi ,bf and bo are corresponding biase used for input, forget and output gate. C t is the corresponding cost for tth iteration. And Wx is the weight used in xth layer.

The final output is in form of 1x365x1 matrix for next year.

We used past years data of average temperature, minimum temperature and maximum temperature to train our model and predicted the average temperature, minimum temperature and maximum temperature of next year and compared it with the test data. As seen in graph below.

Fig. 4: Weather data (Average Temperature) 2011-2018 Table 3: Model Details Location MUMBAI, IN Model LSTM Average Temperature Data Minimum Temperature Maximum Temperature Training Data period 1 August 2011- 31 July 2018 Testing Data period 1 July 2018 - 31 July 2018

All rights reserved by www.grdjournals.com

9

Statistical Model and Neural Networks based Weather Forecasting (GRDJE/ Volume 4 / Issue 10 / 002)

Fig. 5: Error reduces as the program iterates

As of the above graph, the accuracy of our model was 96.03%.

F. IoT based Pipeline to Continuously Predict Temperature 1) Arduino, Temperature Sensor Arduino are small micro-controllers which are available in a form of DIY (do it yourself) kits which can be placed at some open ground or a park and temperature sensor is connected to it which collects the data of temperature on daily basis and provide it to the Arduino.

2) Flask server The data from Arduino is then send to our server which stores the data captured by the sensors and also consist of our trained model.

3) Pre-trained LSTM model on Flask server The server will consist of a pre trained LSTM model which is trained using past 4-5 years data and is designed to forecast future temperature on the basis of previous data.

4) Data collection using Arduino The Arduino works for whole day continuously and collects the data which is transmitted to server at some point in a day.

5) Data Processing The data that is transmitted to the server is processed at a certain time to find minimum and maximum temperature of that day and average temperature of that today.

6) Server contains last day's data Our server already contains the data of the previous day and now the data that is collected on present day is integrated to it and used for prediction of next day data.

7) Model retraining Our model is now trained with yesterday's temperature and today's actual temperature which make next day prediction more accurate and a system which performs online training is being developed.

8) Model’s Prediction Now after retraining our model forecasts today's temperature and returns its values to the dashboard and stored into the server which can be used for prediction of next day.

All rights reserved by www.grdjournals.com

10

Statistical Model and Neural Networks based Weather Forecasting (GRDJE/ Volume 4 / Issue 10 / 002)

III. CONCLUSION Below is the comparative study of all three models that are used in this paper: Table 5: Comparison Details No 1 2

3

Model Ridge Regression Time Series ARIMA LSTM

Features Avg. Temp Avg. Temp.

Max. Temp, Min. Temp.

Training Span

Testing Span

1 August 2017- 31 July 2018 1 August 2011- 31 July 2018

1 August 2018- 31 July 2019 1 August 2018- 31 July 2019

1 August 2011- 31 July 2018

1 July 2018 - 31 July 2018

Accuracy (%)

Flaws

88.84

Overfitting occurs in this model.

82.19

Model is inefficient for this kind of data.

96.03

Accuracy varies as and when training data changes.

Based on the table of results above we can conclude that amongst various models of machine learning that can be used to predict weather LSTM is best suitable for this purpose. Amongst these models the ridge regression model is mostly used for regularization of ill-posed problems so it is good for forecasting problems and thus showed a very high accuracy but there is problem of over fitting which is not good for our kind of application data set. So we used Time Series ARIMA model, in ARIMA model data is fitted to time series format to better understand the data and to predict future points in the series.

As now as our data set comprises of temperature of different dates so it can be well represented using the time series model which work quite close to real world environment, but the problem with this model was that it showed a very less accuracy.

In order to increase the accuracy and give more weightage to recent information we used LSTM neural network model, these type of RNNs are capable of dealing with long-term Dependencies. LSTM remembers the information for the longer period and has a total control over its value and can change its information any time, which is well suited for our kind of data set and out of all models we have used for our paper LSTM model outperformed all of them and showed the best accuracy.

REFERENCES [1] [2] [3] [4] [5] [6]

Mohamed Akram Zaytar, Chaker El Amrani. “Sequence to Sequence Weather Forecasting with Long Short-Term Memory Recurrent Neural Networks”, International Journal of Computer Applications, ISSN: 0975-8887 vol.143-No.11, June 2016.

Chandar Sahu, “An intelligent application of fuzzy ids to forecast seasonal runoff”, International Journal on Cybernetics & Informatics, vol.2, No.1, February 2018.

Gurbrinder kaur, “Meteorological data mining techniques: A survey”, International Journal of Emerging Technology and Advanced Engineering, vol. 2, Issue 8, August 2017.

Divya Chauhan, Jawahar Thakur, “Data mining techniques for weather prediction: A review”, International Journal on Recent and Innovation Trends in Computing and Communication, ISSN: 2321-8169 vol.2 Issue 8.

Mark Holmstorm, Dylan Liu, Christopher Vo. Stanford University (December 15, 2016).

Ron Holmes, “Using a decision tree and neural network to identify severe weather radar characteristics”

All rights reserved by www.grdjournals.com

11