Andreas Teneketzoglou, Nikolaos G. Paterakis, João P. S. Catalãο
Presentation Outline 1. The photovoltaic power 2. PV power forecasting 3. Now-casting PV power 4. Neural Networks 5. Extreme Learning Machines & Time Delay Neural Networks (ELMs & TDNNs) 6. Wavelet Analysis 7. Forecasting Model Description 8. Results 9. Summary
2/20
The photovoltaic power The fact PV power grows rapidly
The problem PV power variability and uncertainty
The solution PV power forecasting
3/20
PV power forecasting Applications Baseline Ramps, regulation, load following, unit commitment, regional power prediction Methods Depend on application, timescales involved, available data & tools 4/20
Now-casting PV power • Very short-term forecasting in timescales of minutes up to 3-4 hours ahead. • Important for grid operators to guarantee grid stability. • Many Independent System Operators (ISOs) are considering the implementation of now-casting PV power services. • Now-casting benchmark model is the Persistence. Persistence assumes:
5/20
Neural Networks • Machine Learning, data-driven models • Ability to handle non linear approximations • Successfully applied in forecasting field
6/20
ELMs & TDNNs Extreme Learning Machines (ELMs) • Recent algorithm for training single hidden layer Feed-Forward Neural Networks. • Input weights and hidden biases randomly assigned. • Output weights analytically determined by simple matrix calculation. • No need for iterative training. Time Delay Neural Networks (TDNNs) • Based on Multi-Layer-Perceptron(MLP), the most widely used Feed Forward Neural Network. • Adopts back-propagation training algorithm based on the iterative steps: feed- forward simulated output, error back-propagation and weights update. 7/20
Wavelet analysis • Discrete Wavelet Transform and Multi-Resolution Analysis decompose time series at different frequencies with different resolutions. • Reconstruction of original time-series is achieved by adding the timefrequency components. • Effective data pre-process technique for time-frequency domain neural network time-series forecasting.
8/20
Now-casting Model Description Objective 5 minutes ahead forecasting with high accuracy, simplicity and computational efficiency. Input data Historical PV power measurements on 5 minutes interval (NREL public dataset). Preprocess Technique Evaluation Maximum Overlap Discrete Wavelet Transform(MODWT) and Multi-Resolution Analysis. Time-frequency versus time domain forecasting. Forecasting Tools Extreme Learning Machine versus Time-delay Neural Network and Persistence model. Development Environment R programming language (elmNN, neuralnet, wmtsa packages).
9/20
Now-casting Model Description Data Preprocess for time-domain forecasting
• Normalization to improve convergence efficiency (sigmoid transfer function for neural networks). • Data Partition to Train, Validation and Test sets performed based on sequence of “day blocks” to assist neural network auto-regression task. • Time Sliding Window converts a static neural network to dynamic for time series forecasting. Time series embedding dimension 2 lags (trial and error).
10/20
Now-casting Model Description Data Preprocess for time-frequency domain forecasting • Wavelet Filter: extremal phase (daublet - d4). • Decomposition levels: 3. • Highest wavelet detail (3) includes smooth (low frequency). • Data partition and time sliding window with same settings as in time domain forecasting.
11/20
Now-casting Model Description •Neural Network Evaluation Algorithm • ELM: 25 initializations, 2-50 hidden nodes search range (1 hidden layer). • TDNN: 5 initializations, 2-25 hidden nodes search range (1 hidden layer). Error Evaluation Criteria Root Mean Square Error (RMSE) an easy interpretable metric that captures both variability and magnitude of a set of errors and normalized RMSE (nRMSE).
12/20
Now-casting Model Description Dataset statistical properties. Boxplots of each month (M1-M12) for Test, Training and Validation Sets. M1
M2
M3
M4
M5
M6
M7
M8
M9
M10
M11
M12
•
150
Watts
100
•
50
• 0 Test Sets
13/20
Train Sets
Validation Sets
Training different neural networks for each month due to the differences between profiles of the months. Diversity between “unknown” to NN test set and the “known” trainvalidation sets of a month is the NN performance challenge. Many outliers = Volatile set
Now-casting Model Description A1
Hidden nodes
15
B1
10 5
Hidden nodes
NN evaluation algorithm resulting architectures and runtime analysis (ELMs: A1,A2 & TDNNs:B1,B2)
0 1
2
3
4
5
6
7
Month
8
9
Minutes
Hidden nodes
40
20 10 0
14/20
1
2
Wavelet Details(A2)
3
• ELMs perform best for smaller number of hidden nodes resulting less NN complexity than TDNNs. • ELMs are more than 10 times faster to train than TDNNs. • Training an ELM or TDNN in time-frequency domain is faster.
15 10 5 0
10 11 12
30
20
1
2
Wavelet Details(B2)
3
30 25 20 15 10 5 1 0 A1
A2
B1
NN evaluation algorithm
B2
Results Normalized RMSE [%] results for each month ELMs (A1,A2); TDNNs(B1,B2) ; Persistence (P)
%
A1
B1
P
A2
7.5
7.5
5.0
5.0
2.5
2.5
0.0
0.0 1
2
3
4
5
6
7
Month
8
9
10 11 12
Time domain forecasting 15/20
Average nRMSE results
10.0
10.0
B2
P
Persistence: 6.6% Time domain • ELMs: 6.63% • TDNNs: 6.6% Time-frequency domain • ELMs: 1.95% • TDNNs: 1.68%
1
2
3
4
5
6
7
Month
8
9
10
11
12
Time-frequency domain forecasting
Results Forecasting plots of 9th day of May and November in time domain forecasting Normalized RMSE values 100
A1
A1
B1
B1
May
M
•
ELM:3.3%
•
TDNN:3.3%
•
Persistence:3%
100
Watts
M
50
50
November 0
0 5:00
16/20
8:00
11:00
14:00
Hour
17:00
19:55
5:00
8:00
11:00
14:00
Hour
17:00
19:55
•
ELM:8%
•
TDNN:8.4%
•
Persistence:7.7%
Results Forecasting plots of 9th day of May in time-frequency domain A2
B2
M
Watts
Watts
0.01 0.00
-0.01
B2
0.00
M
-0.25 5:00
8:00
11:00
14:00
Hour
A2
17:00
B2
19:55
M
5:00
0.000
-0.005 -0.010
8:00
11:00
14:00
Hour
17:00
19:55
A2
100
Watts
0.005
Watts
A2
0.25
B2 50
M
0 5:00
8:00
11:00
14:00
Hour
17:00
19:55
5:00
8:00
11:00
14:00
Hour
17:00
19:55
Normalized RMSE values Wavelet Detail 1 •
ELM:6.2%
•
TDNN:6.6%
Wavelet Detail 2 •
ELM:13.3%
•
TDNN:8.2%
Wavelet Detail 3 •
ELM:0.1%
•
TDNN:0.2%
Reconstructed Forecast 17/20
•
ELM:1%
Results Forecasting plots of 9th day of November in time-frequency domain A2
0.1
B2
M
Watts
Watts
0.2
0.0 -0.1 8:00
11:00
14:00
Hour
0.15
A2
0.10
17:00
B2
19:55
M
5:00
M
Watts
Watts
B2
0.25 0.00
5:00
0.05 0.00 -0.05
8:00
11:00
14:00
Hour
17:00
19:55
A2
100
B2 50
M
0 5:00
18/20
A2
0.50
8:00
11:00
14:00
Hour
17:00
19:55
5:00
8:00
11:00
14:00
Hour
17:00
19:55
Normalized RMSE values Wavelet Detail 1 •
ELM: 6.2%
•
TDNN: 5.2%
Wavelet Detail 2 •
ELM: 16%
•
TDNN: 6.6%
Wavelet Detail 3 •
ELM: 1.3%
•
TDNN: 1.4%
Reconstructed Forecast
Summary • Non-linearity of high resolution PV power time series cannot be approximated effectively by NN. Time-domain NN forecasting degrades to persistence model performance. • Wavelets Analysis is a powerful technique to assist neural network forecasting that increases forecasting accuracy and computational performance. • ELMs computation performance is at least 10 times faster than traditional back-propagation based NN without significant performance degradation. 19/20
Thank you for your attention!