Adaptive Wavelet Neural Network Based Wind Speed Forecasting Studies

Abstract

Wind has been a rapidly growing renewable power source for the last twenty years. Since wind behavior is chaotic in nature, its forecasting is not easy. At the same time, developing an accurate forecasting method is essential when wind farms are integrated into the power grid. In fact, wind speed forecasting tools can solve issues related to grid stability and reserve allocation. In this paper 30 hours ahead wind speed profile forecast is proposed using Adaptive Wavelet Neural Network (AWNN). The implemented AWNN uses a Mexican hat mother Wavelet, and Morlet Mother Wavelet for seven, eight and nine levels decompositions. For wind speed forecasting, the time series data on wind speed has been gathered from the National Renewable Energy Laboratory (NREL) website. In this work, hourly averaged 10-min wind speed data sets for the year 2004 in the Midwest ISO region (site number 7263) is taken for analysis. Data sets are normalized in the range of [-1, 1] to improve the training performance of forecasting models. Total 8760 samples were taken for this forecasting analysis. After the forecasting phase, statistical parameters are calculated to evaluate system accuracy, comparing different configurations.

1. Introduction

In recent days the importance of renewable energy sources has increased. Renewable energy is the best alternative to conventional energy sources. In fact, they are abundant in nature, where as conventional energy sources are exhausting day by day, they are non-pollutant and freely available in our environment. Wind power prediction is very much essential in the present day world. As wind power is proportional to the cube of wind speed, accurate wind speed forecasting can play a vital role in present and future wind power market. With the integration of wind power into the power system, forecast of wind power is gaining much more importance for proper grid operations. In the literature several methods and tools are proposed for wind speed forecasting. In this paper the authors will discuss 30 hours ahead wind speed forecasting, which will be helpful for one day ahead wind power market.

Wind speed forecasting can be performed by Ensemble Empirical Mode Decomposition (EEMD) [1] and in combination with support vector machine (SVM). In this method wind data decomposed by EEMD can be forecasted individually by using SVM. Neural networks are massively used in wind speed forecasting [2-5]. Neural networks can be supported by a Back propagation algorithm (BPA), Tabu search algorithm for forecasting application. In [6] Adaptive Neuro Fuzzy Inference Systems [ANFIS] is used for wind speed forecasting for power generation in Tasmania, Australia. Hybrid methods are also used for wind speed forecasting [7]. Hybrid methods can be a combination of Wavelet transforms (WT), Particle Swarm Optimization (PSO) and ANFIS. In [8] many wind speed forecasting methods have been discussed. In [9] two statistical based methods, namely Autoregressive Moving Average (ARMA) and Neural networks (NN) have been discussed for wind speed forecasting. Novel approaches like empirical mode decomposition (EMD) and time series analysis [10] are recently used for wind speed prediction for a practical data in North China. Wavelets are also used for wind speed forecasting [11, 12]. Here wind speed series are decomposed and each decomposed signal is forecasted individually. Later on all these signals are then re-combined to get final forecasted signals [13]. Wavelet methods are also used for energy price forecasting [14] and for practical applications in power systems [16]. Data mining algorithms are also useful to predict wind speed and wind power as suggested in [17]. In [18], 48 hours ahead forecasting tool has been developed using a statistical method. In [19] a hybrid method is again used for wind speed forecasting to improve forecast accuracy. Finally, in [20] wind speed forecasting and its impacts on the generation system reliability have been discussed.

In this paper an Adaptive Wavelet Neural Network (AWNN) wind speed forecasting study has been conducted. Both Mexican hat and Morlet wavelets have been used as mother wavelets and analyses with 7, 8, 9 levels decompositions of wind speed series have been done in the forecasting studies. In each case decomposed signals are forecasted for 30 hours ahead and later all these signals are added to get forecasted wind speed. All these forecasting results are compared among themselves based on statistical parameters and their respective individual forecasting results are tabulated and commented in section III. Among these methods Morlet eight levels, nine level decompositions have shown best performances. AWNN Multi level decomposition for different wavelets are first time used in this paper.

 

2. Wavelets

A wavelet is a tiny wave which can increase and decrease its amplitude and width in a fixed time period. Wavelet properties make them more suitable for many problems. In wavelets translation, dilation parameters (generally represented as a, b) reflect length and breadth of a wavelet. These parameters can be adjusted according to the problem type, and varied accordingly. They are easily adaptable, flexible and easily fit into any complex problems. Compared to neural networks wavelets training is accurate since wavelets consist of translation, dilation parameters. Wavelet analysis is advantageous when compared to Fourier series analysis. In Fourier analysis every signal can be expressed either in sine or cosine waveforms, where as in wavelet analysis a suitable wavelet can be chosen from a family of wavelets. Fourier analysis is suitable to analyze either frequency or time but not both at the same time, where as in wavelet analysis this operation is possible. In other words wavelets can be better adoptable to time varying frequency analysis. Wavelet, as shown below, satisfies two fundamental properties by which it can be said that wavelets are also like ordinary waves [12, 15]:

Several types of wavelets exist in literature. Depending on the type of the problem, a suitable wavelet can be chosen. Here in this paper Mexican hat wavelet and Morlet wavelets have been chosen for wind speed forecasting. One complete year (2004) of Wind speed data has been collected from National Renewable Energy Laboratory (NREL) website [13]. This paper is based on Adaptive Wavelet Neural Networks, where the term “Adaptive” means they can be suitable or adjustable to new conditions. In the present problem hidden layer consists of wavelet function where wavelet function output value is based on network weights as well as translation a and dilation b parameters.

These translation and dilation parameters are also updated at every iteration like network weights so that network convergence is faster and forecasting result is more accurate. Another thing which can differentiate the present paper with remaining literature is an additional direct connection between input and output layer which leads to better input-output relation there by forecast accuracy improves. The above mentioned hidden layer and additional direct input-output relation not only improves forecast accuracy but also distinguishes this network structure with previous networks existing in literature. In wavelets Multi Resolution Analysis (MRA) technique has been used to decompose wind signal to find approximated and detailed coefficients. Decomposition makes signal clearer to visualize and noise can be then easily eliminated to improve prediction accuracy. Generally approximated coefficients are used to analyze low frequency signals and detailed coefficients are used to analyze high frequency signals. Finally these two coefficients combination is used to analyze signal at all levels so that signal can be analyzed accurately. Each of these coefficients are forecasted for next thirty hours ahead and all these forecasted signals are re-combined to get original signal by using Wavelet Methods for Time series Analysis (WMTSA).

Thus in this paper WMTSA (Wavelet methods for Time series Analysis) has been used. This is a wavelet tool kit designed in MATLAB to analyze time series data. Wavelet methods can be easily implemented by using this toolkit. MATLAB code is implemented for mentioned algorithm using MATLAB R2009a version to evaluate wind speed forecasting. Here a back propagation algorithm is used to train the network. Wavelet networks are the combination of wavelet decomposition and neural networks, and they posses neural network characteristics too. Wavelet neural network is similar to back propagation network except that input layer is connected to hidden layer as well as output layer. Here an AWNN method is used for wind speed forecasting [12, 15].

2.1 Mexican hat wavelet

Fig. 1 shows [12] wavelet neural network where u1, u2 … u50 show input wind velocities, z1, z2, z3 are hidden nodes, v1, v2, … v50 represent weights connected between input to output and w1, w2 … … wm are weights of connection between hidden and output node, where m denotes number of weights and here m=3. In Wavelet neural network, hidden layer consists of wavelet function. In this section Mexican hat has been used as a mother wavelet and is shown in Fig. 2 [15]. It is obtained after derivation Gaussian function twice, where Gaussian function is defined as:

Fig. 1.Wavelet neural network

Fig. 2.Mexican hat wavelet

This function has some special properties like symmetry in shape, explicit in expression, providing exact time frequency analysis. All these properties make Mexican hat more suitable for forecasting applications. Here input patterns have been set as u = [u1, u2, … … … … . un]T. Where n denotes dimension, that is number of wind samples n=50, and ‘u’ is a pattern; similarly p such patterns are used to train the network. In each pattern the elements are lag hours of different decomposed signals. The wavelet family generates the entire input space using translating and dilating the mother wavelet as:

The input data in the input layer is directly transferred to the wavelet layer. Finally, the n-dimensional wavelet basis function is calculated based on tensor product of all one-dimensional wavelets, so output Zj of hidden layer neurons is given by

To map the linear input-output relation, it is tradition to have additional direct connections from input to output layer, since wavelets are not used for reconstructing linear terms. The representation of the Wavelet Neural Network (WNN), for hour-ahead forecast of the decomposed signal is calculated as:

Where wj indicates the weight between the jth wavelon and output node, vi represents the weight between the ith input node and output node, and g is the bias at output node. Standard back propagation algorithm with gradient descent technique is then used to train the wavelet neural network and output function is computed by the Wavelet Neural network (WNN), which is differentiable with respect to unknown parameters like translation, dilation parameters, weights and bias of the network. The main goal of training is minimization of cost function which is also known as mean square error (E). Where E can be represented as:

Where y( p) is the calculated (forecasted) output and y(d) ( p) is the actual output required for a given pth input pattern. A free parameter update is given as:

where Γ is a unknown free variable, η and α are learning rate and momentum parameters. The change in free parameters using (7) can be found as:

Algorithm for Wind Speed Forecasting using AWNN Network

Training: 1. After normalizing, first 50 wind samples (from 1to 50) have been used as input for AWNN network and immediate sample (51st wind sample) is the target wind sample. 2. Next 2 to 51 wind samples have been used as input for the network and next immediate (52nd sample) is the target wind sample. 3. This process is used recursively for next 60 patterns (Each pattern consists of 50 wind samples). 4. Similar procedure is used for D1 to Dn and Sn, till problem converges (here ‘D’ and ‘S’ indicates detailed and absolute coefficients where ‘n’ represents level of decomposition).

Testing: 5. Once the problem converges, then that weights, translation a, dilation b parameters have been used to test wind speed data. 6. Forecasting (testing) has been done for 30 hours ahead from 2551 to 2580 wind sample and corresponding wind input samples are from 2501 to 2550, in each instant forecasted output has taken input for the next pattern. This recursive procedure repeated for next 30 samples. 7. This procedure is applicable to D1 to Dn and Sn. 8. Finally all these coefficients are added to get forecasted wind speed. 9. Decomposition of wind speed signal at different levels is shown below from Figs. 3-5. These individual decomposed signals are helpful to analyze the complete wind speed series. All these decomposed signals are forecasted individually and later added to get forecasted wind speed.

Fig. 3.Decomposition of wind signal up to seven levels using MRA

Fig. 4.Decomposition of wind signal up to eight levels using MRA

Fig. 5.Decomposition of wind signal up to nine levels using MRA

2.2 Morlet wavelet

Fig. 6 [15] shows Morlet wavelet, which has been used for wind speed forecasting in this section. Generally Morlet wavelts are used for rapid variations in the signals. Wind speed variations are drastic so morlet wavelet is best suitable for wind speed forecasting applications.

Fig. 6.Morlet wavelet

In this part of work Morlet mother wavelet is used. The following equations can explain forecasting processor by using AWNN.

Morlet mother wavelet is defined as:

The translation and dilation version of Morlet wavelet is as follows:

As we already discussed in equations from 5-9, we repeat the same procedure as in Morlet wavelet and finally Δaij , Δbij is given by

Decomposition of wind signal at different levels has been shown in the following figures from 7-9. Similar algorithm is used to forecast the wind signal using again Morlet wavelet in the above mentioned Mexican hat wavelet neural network.

Fig. 7.Decomposition of wind signal up to seven levels usin MRA

Fig. 8.Decomposition of wind signal up to eight levels using MRA

Fig. 9.Decomposition of wind signal up to nine levels using MR

 

3. Results and Discussions

In this section wind speed forecasting results using AWNN are analyzed and discussed. In Adaptive Wavelet Neural Network (AWNN), Mexican hat mother wavelet and Morlet mother wavelet have been used. Both these wavelets are used to forecast at seven, eight and nine levels of decomposition. Tables 1 to 4 have shown comparison of actual and forecasted wind speed for 2 different wavelets at various levels of decomposition. Table 7 shows comparison among all forecasting methods based on their respective statistical measures. Among all these results Morlet wavelet 8-level, 9-levels decomposition are giving the best results. Morlet wavelet 8-level decomposition shows a better accuracy when APE, MAPE, MAE are considered and 9-level decomposition is relatively better when RMSE, Correlation coefficient (R) are considered. Figs. 10 to 15 have shown graphical representation between actual and forecasted wind speed. This comparison has been done for 30 hours ahead wind speed prediction.

Table 1.Comparison of actual and forecasted wind speed for mexican hat with seven level decomposition

Table 2.Comparison of actual and forecasted wind speed for mexican hat with eight level decomposition

Table 3.Comparison of actual and forecasted wind speed for mexican hat with nine level decomposition

Table 4.Comparison of actual and forecasted wind speed for morlet with seven levels decomposition

Table 7.Comparison of statistical measures for different wavelets at various levels of decomposition

Fig. 10.Actual and forecasted wind speed time series using Mexican hat Wavelet as a mother wavelet for seven levels of decomposition

Fig. 11.Actual and forecasted wind speed time series using Mexican hat Wavelet as a mother wavelet for eight levels of decomposition.

Fig. 12.Actual and forecasted wind speed time series using Mexican hat Wavelet as a mother wavelet for nine levels of decomposition.

Fig. 13.Actual and forecasted wind speed time series using Morlet Wavelet as a mother wavelet for seven levels of decomposition.

Fig. 14.Actual and forecasted wind speed time series using Morlet Wavelet as a mother wavelet for eight levels of decomposition.

Fig. 15.Actual and forecasted wind speed time series using Morlet Wavelet as a mother wavelet for nine levels of decomposition.

Selection of mother wavelet function surely depends on problem type, smoothness and reconstruction. Here, for wind speed forecasting problem, Morlet wavelet shows itself suitable in terms of above mentioned factors so that forecasting with this Mother wavelet leads to more accurate results. Besides, since if as levels of decomposition increases then resolution, accuracy of forecast will increase accordingly. 8-levels decomposition is generally more accurate than 7-levels decomposition. Coming to 9-levels decomposition results, as network parameters reached to saturation, the analysis did not show any further improvement with respect to 8-levels; in fact 8- levels decomposition shows relatively better performance. Various statistical parameters are evaluated by using following formulae in order to evaluate forecasting accuracy:

Table 5.Comparison of actual and forecasted wind speed for morlet with eight level decomposition

Table 6.Comparison of actual and forecasted wind speed for morlet with nine level decomposition

Mean Absolute Percentage Error

Root Mean Square Error

In above equations AW=Actual wind speed, FW= Forecasted wind speed, n=Number of wind samples, RAF= Covariance between Actual and Forecasted wind speed, Std (A) = Standard deviation of Actual wind speed, Std (F) = Standard deviation of Forecasted wind speed.

Summary of parameters used to train AWNN Network:

Network: Learning rate (η) = 0.5 Momentum coefficient (∝) = 0.5 Tolerance (ℰ ) = 0.0001 Number of training patterns = 60 and each training pattern consists of 50 input samples. Number of input nodes = 50 Number of hidden nodes = 3

Average computation time to converge (from D1 to Dn and Sn) is varying from 4 seconds to 60 seconds and it also depends on wavelet type and decomposition level.

 

4. Conclusions

In this paper wind speed forecasting has been carried out for 30 hours ahead. Wind speed forecasting performance by AWNN method is analyzed and discussed using Mexican hat and Morlet mother wavelets. The wind signal is decomposed for seven, eight and nine levels of decomposition. In all these decomposed methods signals are forecasted individually, later they are re-combined by using WMTSA (Wavelet Method for Time Series Analysis). Among three different levels of decomposition 8-levels decomposition has given better result in their respective wavelets, depending on the number of wind samples that we have taken. Among all these methods Morlet wavelet with eight levels of decomposition gives minimum MAPE and (7.92%), MAE (0.433). Morlet 9-level decomposition has resulted in better correlation Coefficient (R) 0.940 and minimum Root Mean Square Error (RMSE) 0.544. Thus Morlet Generally Morlet wavelets are better suited to analyze sudden variations so here for wind speed forecasting this kind of wavelet was best suited. Therefore AWNN with Morlet wavelet both 8-levels and 9-levels of decomposition have shown better results. Among these two Morlet wavelet 8-level decomposition wind forecasting results were relatively more accurate.