# LSSVR Model of G-L Mixed Noise-Characteristic with Its Applications

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Bayesian Principle to Mixed Noise Empirical Risk Loss

## 3. $\mathit{LSSVR}$ Model of G-L Mixed Noise-Characteristic

**Definition**

**1**

#### 3.1. $LSSVR$ Model of G-L Mixed Homoscedastic Noise-Characteristic

**Proposition**

**1.**

**Theorem**

**1.**

**Proof.**

#### 3.2. $LSSVR$ Model of G-L Mixed Heteroscedastic Noise-Characteristic

**Proposition**

**2.**

**Theorem**

**2.**

**Proof.**

## 4. Solution from ALM

## 5. Case Study

#### 5.1. G-L Mixed-Noise-Characteristic of Wind-Speed

#### 5.2. Prediction Performance Evaluation Criteria

#### 5.3. Short-Term Wind-Speed Forecasting with Real dataset

**$\overline{{A}_{i}}=({x}_{i-11},{x}_{i-10},\cdots ,{x}_{i-1},{x}_{i})$**, ${x}_{j}$ is the actual observed data of wind speed at moment $j(j=i-11,i-10,\cdots ,i)$, and the forecasting value is ${x}_{i+step}$, where $step=1,3,6$. That is, the above models are used to forecast wind speed of each point ${x}_{i}$ after 10, 30 and 60 min, respectively. Figure 5, Figure 6, Figure 7, Figure 8, Figure 9, Figure 10, Figure 11, Figure 12 and Figure 13 describe the forecasting results given by models $\nu -SVR$, $GN-SVR$, $LSSVR$, and $GLM-LSSVR$.

## 6. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## Abbreviations

LR | Linear regression model |

$\nu $-SVR | $\nu $-Support vector regression |

GN-SVR | $\nu $-SVR model of Gaussian homoscedastic-noise |

LSSVR | Least squares support vector regression model |

GLM-LSSVR | LSSVR model of Gaussian–Laplacian mixed homoscedastic-noise |

ALM | Augmented Lagrange multiplier method |

## References

- Bishop, C.M. Pattern Recognition and Machine Learning; Springer: New York, NY, USA, 2006. [Google Scholar]
- Tikhonov, A.A.; Arsenin, V.Y. Solutions of Ill-Posed Problems; New York Wiley: New York, NY, USA, 1977. [Google Scholar]
- Gonen, A.; Orabona, F.; Shalev-Shwartz, S. Solving Ridge Regression using Sketched Preconditioned SVRG. In Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA, 19–24 June 2016. [Google Scholar]
- Hoerl, A.E. Application of ridge analysis to regression problems. Chem. Eng. Prog.
**1962**, 58, 54–59. [Google Scholar] - Zhang, Z.H.; Dai, G.; Xu, C.F. Regularized Discriminant Analysis, Ridge Regression and Beyond. J. Mach. Learn. Res.
**2010**, 11, 2199–2228. [Google Scholar] - Sun, L.; Wang, L.; Ding, W.; Qian, Y.; Xu, J. Feature Selection Using Fuzzy Neighborhood Entropy-Based Uncertainty Measures for Fuzzy Neighborhood Multigranulation Rough Sets. IEEE Trans. Fuzzy Syst.
**2020**. [Google Scholar] [CrossRef] - Jiao, L.C.; Bo, L.F.; Wang, L. Fast Sparse Approximation for Least Squares Support Vector Machine. IEEE Trans. Neural Netw.
**2007**, 18, 685–697. [Google Scholar] [CrossRef] - Völgyesi, L.; Palánc, B.; Fekete, K.; Popper, G. Application of Kernel Ridge Regression to Network Levelling via Mathematica. Geophys. Res. Abstr.
**2005**, 73, 263–276. [Google Scholar] - Sun, L.; Zhang, X.; Qian, Y.; Xu, J.; Zhang, S. Feature selection using neighborhood entropy-based uncertainty measures for gene expression data classification. Inf. Sci.
**2019**, 502, 18–41. [Google Scholar] [CrossRef] - Douak, F.; Melgani, F.; Benoudjit, N. Kernel ridge regression with active learning for wind-speed prediction. Appl. Energy.
**2013**, 103, 328–340. [Google Scholar] [CrossRef] - Alexiadis, M.C.; Dokopoulos, P.S.; Sahsamanoglou, H.S.; Manousaridis, I.M. Short term forecasting of wind speed and related electrical power. J. Sol. Energy
**1998**, 63, 61–68. [Google Scholar] [CrossRef] - Negnevitsky, M.; Potter, C.W. Innovative short-term wind generation prediction techniques. In Proceedings of the power systems conference and exposition, Atlanta, GA, USA, 29 October–1 November 2006. [Google Scholar]
- Torres, J.L.; Garcia, A.; De Blas, M.; De Francisco, A. Forecast of hourly average wind speed with ARMA models in Navarre (Spain). J. Sol. Energy
**2005**, 79, 65–77. [Google Scholar] [CrossRef] - Kavasseri, R.G.; Seetharaman, K. Day-ahead wind-speed forecasting using f-ARIMA models. Renew. Energy
**2009**, 34, 1388–1393. [Google Scholar] [CrossRef] - Li, G.; Shi, J. On comparing three artificial neural networks for wind speed forecasting. Appl. Energy
**2010**, 87, 2313–2320. [Google Scholar] [CrossRef] - Hu, Q.; Zhang, R.; Zhou, Y. Transfer learning for short-term wind-speed prediction with deep neural networks. Renew. Energy
**2016**, 85, 83–95. [Google Scholar] [CrossRef] - Salcedo-Sanz, S.; Ortiz-Garcı, E.G.; Pérez-Bellido, Á.M.; Portilla-Figueras, A.; Prieto, L. Short term wind-speed prediction based on evolutionary support vector regression algorithms. Expert Syst. Appl.
**2011**, 38, 4052–4057. [Google Scholar] [CrossRef] - Zhou, J.; Shi, J.; Li, G. Fine tuning support vector machines for short-term wind speed forecasting. Energy Convers. Manag.
**2011**, 52, 1990–1998. [Google Scholar] [CrossRef] - Liu, H.; Tian, H.-Q.; Chen, C.; Li, Y.-F. A hybrid statistical method to predict wind speed and wind power. Renew. Energy
**2010**, 35, 1857–1861. [Google Scholar] [CrossRef] - Wang, Y.; Hu, Q.; Li, L.; Foley, A.M.; Srinivasan, D. Approaches to wind power curve modeling: A review and discussion. Renew. Sustain. Energy Rev.
**2019**, 116, 109422. [Google Scholar] [CrossRef] - Suykens, J.; Vandewalle, J. Least Squares Support Vector Machine Classifiers. Neural Process. Lett.
**1999**, 9, 293–300. [Google Scholar] [CrossRef] - Suykens, J.; Lukas, L.; Vandewalle, J. Sparse approximation using least square vector machines. In Proceedings of the IEEE International Symposium on Circuits and Systems, Geneva, Switzerland, 28–31 May 2000; pp. 757–760. [Google Scholar]
- Suykens, J.; De Brabanter, J.; Lukas, L.; Vandewalle, J. Weighted least squares support vector machines: robustness and sparse approximation. Neurocomputing
**2002**, 48, 85–105. [Google Scholar] [CrossRef] - Du, P.; Wang, J.; Guo, Z.; Yang, W. Research and application of a novel hybrid forecasting system based on multi-objective optimization for wind speed forecasting. Energy Convers. Manag.
**2017**, 150, 90–107. [Google Scholar] [CrossRef] - Sun, L.; Wang, L.; Ding, W.; Qian, Y.; Xu, J. Neighborhood multi-granulation rough sets-based attribute reduction using Lebesgue and entropy measures in incomplete neighborhood decision systems. Knowl. -Based Syst.
**2020**, 192, 105373. [Google Scholar] [CrossRef] - Jiang, Y.; Huang, G.Q. A hybrid method based on singular spectrum analysis, firefly algorithm, and BP neural network for short-term wind-speed forecasting. Energies
**2016**, 9, 757. [Google Scholar] - Jiang, Y.; Huang, G. Short-term wind speed prediction: Hybrid of ensemble empirical mode decomposition, feature selection and error correction. Energy Convers. Manag.
**2017**, 144, 340–350. [Google Scholar] [CrossRef] - Zhang, S.; Zhou, T.; Sun, L.; Wang, W.; Wang, C.; Mao, W. ν-Support Vector Regression Model Based on Gauss-Laplace Mixture Noise Characteristic for Wind Speed Prediction. Entropy
**2019**, 21, 1056. [Google Scholar] [CrossRef] [Green Version] - Shevade, S.; Keerthi, S.S.; Bhattacharyya, C.; Murthy, K. Improvements to the SMO algorithm for SVM regression. IEEE Trans. Neural Netw.
**2000**, 11, 1188–1193. [Google Scholar] [CrossRef] [Green Version] - Klaus-Robert, M.; Sebastian, M. An introduction to kernel-based learning algorithms. IEEE Trans. Neural Netw.
**2001**, 12, 181–202. [Google Scholar] - Chu, W.; Keerthi, S.; Ong, C.J. Bayesian Support Vector Regression Using a Unified Loss Function. IEEE Trans. Neural Netw.
**2004**, 15, 29–44. [Google Scholar] [CrossRef] [Green Version] - Rockafellar, R.T. Augmented Lagrange Multiplier Functions and Duality in Nonconvex Programming. SIAM J. Control
**1974**, 12, 268–285. [Google Scholar] [CrossRef] - Boyd, S.; Vandenberghe, L. Convex Optimization; Cambridge University Press: Cambridge, UK, 2004; pp. 521–620. [Google Scholar]
- Wang, S.; Zhang, N.; Wu, L.; Wang, Y. Wind speed forecasting based on the hybrid ensemble empirical mode decomposition and GA-BP neural network method. Renew. Energy
**2016**, 94, 629–636. [Google Scholar] [CrossRef] - Bordes, A.; Bottou, L.; Gallinari, P. SGD-QN: Careful quasiNewton stochastic gradient descent. J. Mach. Learn. Res.
**2009**, 10, 1737–1754. [Google Scholar] - Bludszuweit, H.; Dominguez-Navarro, J.; Llombart, A. Statistical Analysis of Wind Power Forecast Error. IEEE Trans. Power Syst.
**2008**, 23, 983–991. [Google Scholar] [CrossRef] - Fabbri, A.; Román, T.G.S.; Abbad, J.R.; Quezada, V.H.M. Assessment of the cost associated with wind generation prediction errors in a liberalized electricity market. IEEE Trans. Power Syst.
**2005**, 20, 1440–1446. [Google Scholar] [CrossRef] - Guo, Z.; Zhao, J.; Zhang, W.; Wang, J. A corrected hybrid approach for wind speed prediction in Hexi Corridor of China. Energy
**2011**, 36, 1668–1679. [Google Scholar] [CrossRef] - Wang, J.Z.; Hu, J.M. A robust combination approach for short-term wind-speed forecasting and analysis-Combination of the ARIMA, ELM, SVM and LSSVM forecasts using a GPR model. Energy
**2015**, 93, 41–56. [Google Scholar] [CrossRef] - Abdoos, A.A. A new intelligent method based on combination of VMD and ELM for short term wind power forecasting. Neurocomputing
**2016**, 203, 111–120. [Google Scholar] [CrossRef] - Chalimourda, A.; Schölkopf, B.; Smola, A.J. Experimentally optimal ν in support vector regression for different noise models and parameter settings. Neural Netw.
**2004**, 17, 127–141. [Google Scholar] [CrossRef] - Cherkassky, V.; Ma, Y. Practical selection of SVM parameters and noise estimation for SVM regression. Neural Netw.
**2004**, 17, 113–126. [Google Scholar] [CrossRef] [Green Version] - Kwok, J.T.; Tsang, I.W. Linear dependency between and the input noise in ϵ-support vector regression. IEEE Trans. Neural Netw.
**2003**, 14, 544–553. [Google Scholar] [CrossRef] [Green Version] - Schölkopf, B.; Smola, A.J.; Williamson, R.C.; Bartlett, P. New Support Vector Algorithms. Neural Comput.
**2000**, 12, 1207–1245. [Google Scholar] - Wu, Q. A hybrid-forecasting model based on Gaussian support vector machine and chaotic particle swarm optimization. Expert Syst. Appl.
**2010**, 37, 2388–2394. [Google Scholar] [CrossRef] - Wu, Q.; Law, R. The forecasting model based on modified SVRM and PSO penalizing Gaussian noise. Expert Syst. Appl.
**2011**, 38, 1887–1894. [Google Scholar] [CrossRef]

Model | MAE (m/s) | RMSE (m/s) | MAPE (%) | SEP (%) |
---|---|---|---|---|

$\nu -SVR$ | 0.4280 | 0.5833 | 8.02 | 7.12 |

$GN-SVR$ | 0.4256 | 0.5789 | 7.92 | 7.07 |

$LSSVR$ | 0.4219 | 0.5768 | 7.94 | 7.06 |

$GLM-LSSVR$ | 0.4190 | 0.5711 | 7.91 | 7.05 |

Model | MAE (m/s) | RMSE (m/s) | MAPE (%) | SEP (%) |
---|---|---|---|---|

$\nu -SVR$ | 0.7979 | 1.0116 | 23.36 | 12.53 |

$GN-SVR$ | 0.7368 | 0.9886 | 19.93 | 11.89 |

$LSSVR$ | 0.7109 | 0.9226 | 17.17 | 11.43 |

$GLM-LSSVR$ | 0.6185 | 0.8241 | 10.71 | 10.19 |

Model | MAE (m/s) | RMSE (m/s) | MAPE (%) | SEP (%) |
---|---|---|---|---|

$\nu -SVR$ | 0.9994 | 1.2580 | 33.93 | 15.66 |

$GN-SVR$ | 0.9728 | 1.2355 | 31.78 | 15.37 |

$LSSVR$ | 0.9646 | 1.2177 | 29.01 | 15.16 |

$GLM-LSSVR$ | 0.8835 | 1.1180 | 25.72 | 13.97 |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Zhang, S.; Zhou, T.; Sun, L.; Wang, W.; Chang, B.
*LSSVR* Model of G-L Mixed Noise-Characteristic with Its Applications. *Entropy* **2020**, *22*, 629.
https://doi.org/10.3390/e22060629

**AMA Style**

Zhang S, Zhou T, Sun L, Wang W, Chang B.
*LSSVR* Model of G-L Mixed Noise-Characteristic with Its Applications. *Entropy*. 2020; 22(6):629.
https://doi.org/10.3390/e22060629

**Chicago/Turabian Style**

Zhang, Shiguang, Ting Zhou, Lin Sun, Wei Wang, and Baofang Chang.
2020. "*LSSVR* Model of G-L Mixed Noise-Characteristic with Its Applications" *Entropy* 22, no. 6: 629.
https://doi.org/10.3390/e22060629