# Hourly Urban Water Demand Forecasting Using the Continuous Deep Belief Echo State Network

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

^{2}), normalized root-mean-square error (NRMSE), and mean absolute percentage error (MAPE) are adopted as assessment criteria. Forecasting results obtained in the testing stage indicate that the CDBESN model has the largest r

^{2}value of 0.995912 and the smallest NRMSE and MAPE values of 0.027163 and 2.469419, respectively. The prediction accuracy of the proposed model clearly outperforms those of the models it is compared with due to the good feature extraction ability of CDBN and the excellent feature learning ability of ESN.

## 1. Introduction

## 2. Methodology

#### 2.1. Continuous Deep Belief Network

^{l}represents the weight matrix of the layers l and l−1. A typical CRBM model is marked by blue dashed lines in Figure 1, which is constructed from a hidden layer h and a visible layer v. Symmetric connections of the weight matrix exist between the two layers, but no such connections are present within a layer.

_{j}and s

_{i}denote the states of the hidden layer unit j and the visible layer unit i, respectively; moreover, w

_{ij}denotes the interconnected weights of the units j and i. A group of samples is randomly chosen as input data, and the update rule of the states s

_{j}of the hidden layer unit is given as follows:

#### 2.2. Echo State Network

^{in}represents the weight matrix of the input layer, W is the internal weight matrix of the reservoir, W

^{o}is the weight matrix of the output layer, and W

^{b}is the feedback weight matrix. The values of W

^{in}, W, and W

^{b}are randomly produced during the initialization process and cannot be changed after generation. Only the value of W

^{o}must be adjusted during the training process of the reservoir.

#### 2.3. CDBESN Model

## 3. Application Example

#### 3.1. Study Area and Data Collection

^{3}/h, and supplies water to about 600,000 urban residents and factories in that region with an area of about 500 km

^{2}. The original hourly water demand data were divided into two parts: 84% of the data (the first 6552 hourly data, from 1 January 2016 to 30 September 2016) were used to train the CDBESN model, and the remaining 16% were applied for the testing dataset. Figure 4 shows the original hourly water demand records obtained from the urban waterworks.

#### 3.2. Performance Index

^{2}), normalized root mean-square error (NRMSE), and mean absolute percentage error (MAPE) were employed to measure the prediction accuracy of the hourly urban water demand forecasting model. The respective equations were defined as follows:

## 4. Results and Discussions

#### 4.1. CDBESN Modeling

_{ij}in Equation (1) need to be set. A set of random initial values of w

_{ij}was used in the first CRBM, and the weight matrix was constantly adjusted until it reached stability. Then, the next CRBM’s weight matrix was initialized by using the previously trained CRBM’s weight matrix, and layer-wise training was performed until all CRBMs were trained completely. Fixed values of the parameters θ

_{min}and θ

_{max}in Equations (2) and (4) were adopted, and set to be the minimum and maximum values of the original hourly water demand data, respectively. The constant σ in Equations (3) and (5), the learning rates η

_{w}in Equation (6), η

_{a}in Equations (7) and (8), and the noise-control parameters a

_{j}and a

_{i}in Equations (7) and (8), respectively, were determined by utilizing the fivefold cross-validation strategy and were also considered.

^{in}, W, and W

^{b}were randomly initialized and remained constant until the ESN training was complete. The relevant optimal parameters of the reservoir were determined by the grid search method and fivefold cross-validation method.

^{2}, NRMSE, and MAPE) were used to assess the learning performance of the CDBESN model with different parameters and select the parameters with the best learning performance. According to the method described above, the optimal architecture of the CDBN is 10–5–10; that is, 10 input layer units, 5 units in the first hidden layer, and 10 units in the second hidden layer. The optimal parameters of the ESN are the reservoir units N = 1000, the spectral radius λ = 0.9, and the leaking rate α = 0.3. The results of three performance indexes of the CDBESN model for the hourly water demand prediction in the training stage are r

^{2}= 0.995753, NRMSE = 0.027649, and MAPE = 2.354166.

#### 4.2. Prediction and Results

^{2}= 0.995912, NRMSE = 0.027163, and MAPE = 2.469419.

#### 4.3. Comparison Experiment

^{2}shown in the figure reveal that the CDBESN model slightly outperforms the comparison models in predicting the hourly water demand during the testing stage.

^{2}, NRMSE, and MAPE are employed to estimate the forecasting performances of the ESN, CDBNN, and SVR models by using the same testing dataset, as shown in Table 1. The CDBESN model has the best predictive performance, having the largest r

^{2}value and the smallest NRMSE and MAPE values among all models. Compared with the ESN, CDBNN, and SVR models, the proposed CDBESN model shows increases in r

^{2}of approximately 0.27%, 0.53%, and 1.12%; reductions in NRMSE of 21.91%, 33.28%, and 55.05%; and reductions in MAPE of 25.18%, 36.20%, and 56.55%. The CDBESN approach also has a higher prediction accuracy than the other comparison models in predicting the hourly water demand during the testing stage, partly because of the excellent feature extraction capabilities of the CDBN model and the good regression performance of ESN model in the new deep learning architecture.

## 5. Conclusions

^{2}, NRMSE, and MAPE, are used to estimate the forecasting performances of these models. The empirical results show that the proposed CDBESN model more accurately predicts the hourly urban water demand of the urban waterworks in Zhuzhou, China than the other models. The excellent performance of the proposed CDBESN model is due to the powerful feature extraction capacity of the CDBN model and the good feature regression ability of the ESN model.

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Msiza, I.S.; Nelwamondo, F.V.; Marwala, T. Artificial neural networks and support vector machines for water demand time series forecasting. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Montreal, QC, Canada, 7–10 October 2007; pp. 638–643. [Google Scholar]
- Candelieri, A.; Soldi, D.; Archetti, F. Short-term forecasting of hourly water consumption by using automatic metering readers data. Procedia Eng.
**2015**, 119, 844–853. [Google Scholar] [CrossRef] - Chen, G.; Long, T.; Xiong, J.; Bai, Y. Multiple random forests modelling for urban water consumption forecasting. Water Resour. Manag.
**2017**, 31, 1–15. [Google Scholar] [CrossRef] - Adamowski, J.; Karapataki, C. Comparison of multivariate regression and artificial neural networks for peak urban water-demand forecasting: Evaluation of different ann learning algorithms. J. Hydrol. Eng.
**2010**, 15, 729–743. [Google Scholar] [CrossRef] - Gagliardi, F.; Alvisi, S.; Kapelan, Z.; Franchini, M. A probabilistic short-term water demand forecasting model based on the markov chain. Water
**2017**, 9, 507. [Google Scholar] [CrossRef] - Bai, Y.; Wang, P.; Li, C.; Xie, J.; Wang, Y. A multi-scale relevance vector regression approach for daily urban water demand forecasting. J. Hydrol.
**2014**, 517, 236–245. [Google Scholar] [CrossRef] - Candelieri, A. Clustering and support vector regression for water demand forecasting and anomaly detection. Water
**2017**, 9, 224. [Google Scholar] [CrossRef] - Brentan, B.M.; Luvizotto, E., Jr.; Herrera, M.; Izquierdo, J.; Pérez-García, R. Hybrid regression model for near real-time urban water demand forecasting. J. Comput. Appl. Math.
**2017**, 309, 532–541. [Google Scholar] [CrossRef] - Shabani, S.; Candelieri, A.; Archetti, F.; Naser, G. Gene expression programming coupled with unsupervised learning: A two-stage learning process in multi-scale, short-term water demand forecasts. Water
**2018**, 10, 142. [Google Scholar] [CrossRef] - Donkor, E.A.; Mazzuchi, T.A.; Soyer, R.; Roberson, J.A. Urban water demand forecasting: Review of methods and models. J. Water Resour. Plan. Manag.
**2014**, 140, 146–159. [Google Scholar] [CrossRef] - Jain, A.; Kumar Varshney, A.; Chandra Joshi, U. Short-term water demand forecast modelling at IIT Kanpur using artificial neural networks. Water Resour. Manag.
**2001**, 15, 299–321. [Google Scholar] [CrossRef] - Adamowski, J.F. Peak daily water demand forecast modeling using artificial neural networks. J. Water Resour. Plan. Manag.
**2008**, 134, 119–128. [Google Scholar] [CrossRef] - Bennett, C.; Stewart, R.A.; Beal, C.D. Ann-based residential water end-use demand forecasting model. Expert Syst. Appl.
**2013**, 40, 1014–1023. [Google Scholar] [CrossRef] - Al-Zahrani, M.A.; Abo-Monasar, A. Urban residential water demand prediction based on artificial neural networks and time series models. Water Resour. Manag.
**2015**, 29, 3651–3662. [Google Scholar] [CrossRef] - Hinton, G.E.; Osindero, S.; Teh, Y.W. A fast learning algorithm for deep belief nets. Neural Comput.
**2006**, 18, 1527–1554. [Google Scholar] [CrossRef] [PubMed] - Sarikaya, R.; Hinton, G.E.; Deoras, A. Application of deep belief networks for natural language understanding. IEEE/ACM Trans. Audio Speech Lang. Process.
**2014**, 22, 778–784. [Google Scholar] [CrossRef] - Chen, Y.; Zhao, X.; Jia, X. Spectral–spatial classification of hyperspectral data based on deep belief network. IEEE J. Select. Top. Appl. Earth Obs. Remote Sens.
**2015**, 8, 2381–2392. [Google Scholar] [CrossRef] - Zhao, Z.; Jiao, L.; Zhao, J.; Gu, J.; Zhao, J. Discriminant deep belief network for high-resolution sar image classification. Pattern Recognit.
**2017**, 61, 686–701. [Google Scholar] [CrossRef] - Shao, H.; Jiang, H.; Zhang, H.; Liang, T. Electric locomotive bearing fault diagnosis using novel convolutional deep belief network. IEEE Trans. Ind. Electron.
**2017**, 65, 2727–2736. [Google Scholar] [CrossRef] - Zheng, J.; Fu, X.; Zhang, G. Research on exchange rate forecasting based on deep belief network. Neural Comput. Appl.
**2017**. [Google Scholar] [CrossRef] - Fu, G. Deep belief network based ensemble approach for cooling load forecasting of air-conditioning system. Energy
**2018**, 148, 269–282. [Google Scholar] [CrossRef] - Bai, Y.; Chen, Z.; Xie, J.; Li, C. Daily reservoir inflow forecasting using multiscale deep feature learning with hybrid models. J. Hydrol.
**2016**, 532, 193–206. [Google Scholar] [CrossRef] - Bai, Y.; Sun, Z.; Zeng, B.; Deng, J.; Li, C. A multi-pattern deep fusion model for short-term bus passenger flow forecasting. Appl. Soft Comput.
**2017**, 58, 669–680. [Google Scholar] [CrossRef] - Xu, Y.; Zhang, J.; Long, Z.; Chen, Y. A novel dual-scale deep belief network method for daily urban water demand forecasting. Energies
**2018**, 11, 1068. [Google Scholar] [CrossRef] - Kuremoto, T.; Kimura, S.; Kobayashi, K.; Obayashi, M. Time series forecasting using a deep belief network with restricted boltzmann machines. Neurocomputing
**2014**, 137, 47–56. [Google Scholar] [CrossRef] - Qin, M.; Li, Z.; Du, Z. Red tide time series forecasting by combining arima and deep belief network. Knowledge-Based Syst.
**2017**, 125, 39–52. [Google Scholar] [CrossRef] - Xu, Y.; Zhang, J.; Long, Z.; Lv, M. Daily urban water demand forecasting based on chaotic theory and continuous deep belief neural network. Neural Process. Lett.
**2018**. [Google Scholar] [CrossRef] - Ding, S.; Su, C.; Yu, J. An optimizing bp neural network algorithm based on genetic algorithm. Artif. Intell. Rev.
**2011**, 36, 153–162. [Google Scholar] [CrossRef] - Sun, X.; Li, T.; Li, Q.; Huang, Y.; Li, Y. Deep belief echo-state network and its application to time series prediction. Knowledge-Based Syst.
**2017**, 130, 17–29. [Google Scholar] [CrossRef] - Jaeger, H. The “Echo State” Approach to Analysing and Training Recurrent Neural Networks—With an Erratum Note; GMD Report 148; German National Research Center for Information Technology: Bonn, Germany, 2001. [Google Scholar]
- Jaeger, H. Short Term Memory in Echo State Networks; GMD Report 152; German National Research Center for Information Technology: Bonn, Germany, 2002. [Google Scholar]
- Jaeger, H.; Haas, H. Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science
**2004**, 304, 78–80. [Google Scholar] [CrossRef] - Jaeger, H.; Lukoševičius, M.; Popovici, D.; Siewert, U. Optimization and applications of echo state networks with leaky- integrator neurons. Neural Netw.
**2007**, 20, 335–352. [Google Scholar] [CrossRef] - Lun, S.-X.; Yao, X.-S.; Qi, H.-Y.; Hu, H.-F. A novel model of leaky integrator echo state network for time-series prediction. Neurocomputing
**2015**, 159, 58–66. [Google Scholar] [CrossRef] - Chouikhi, N.; Ammar, B.; Rokbani, N.; Alimi, A.M. Pso-based analysis of echo state network parameters for time series forecasting. Appl. Soft Comput.
**2017**, 55, 211–225. [Google Scholar] [CrossRef] - Chen, H.; Murray, A. A continuous restricted Boltzmann machine with a hardware- amenable learning algorithm. In Proceedings of the 12th International Conference on Artificial Neural Networks, Madrid, Spain, 28–30 August 2002; pp. 358–363. [Google Scholar]
- Hinton, G.E. Training products of experts by minimizing contrastive divergence. Neural Comput.
**2002**, 14, 1771–1800. [Google Scholar] [CrossRef] [PubMed]

**Figure 5.**Forecasting results and scatter plots by the CDBESN model for the training data: (

**a**) Shows the forecasting results, and (

**b**) shows the scatter plots.

**Figure 6.**Forecasting results and scatter plots by the CDBESN model for the testing data: (

**a**) Shows the forecasting results, and (

**b**) shows the scatter plots.

**Figure 7.**Prediction results and scatter plots of the hourly water demand in the testing stage using different models: (

**a**) and (

**b**) for ESN, (

**c**) and (

**d**) for CDBNN, and (

**e**) and (

**f**) for SVR.

Model | r^{2} | NRMSE | MAPE |
---|---|---|---|

CDBESN | 0.995912 | 0.027163 | 2.469419 |

ESN | 0.993212 | 0.034783 | 3.300566 |

CDBNN | 0.990701 | 0.040711 | 3.870726 |

SVR | 0.984903 | 0.060430 | 5.683949 |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Xu, Y.; Zhang, J.; Long, Z.; Tang, H.; Zhang, X.
Hourly Urban Water Demand Forecasting Using the Continuous Deep Belief Echo State Network. *Water* **2019**, *11*, 351.
https://doi.org/10.3390/w11020351

**AMA Style**

Xu Y, Zhang J, Long Z, Tang H, Zhang X.
Hourly Urban Water Demand Forecasting Using the Continuous Deep Belief Echo State Network. *Water*. 2019; 11(2):351.
https://doi.org/10.3390/w11020351

**Chicago/Turabian Style**

Xu, Yuebing, Jing Zhang, Zuqiang Long, Hongzhong Tang, and Xiaogang Zhang.
2019. "Hourly Urban Water Demand Forecasting Using the Continuous Deep Belief Echo State Network" *Water* 11, no. 2: 351.
https://doi.org/10.3390/w11020351