Comparison between Inverse Model and Chaos Time Series Inverse Model for Long-Term Prediction

Abstract

This paper presents an inverse model using chaotic behaviour. The chaos time series inverse model, which uses coupling methods between an inverse model and chaos theory can reconstruct a deterministic and low-dimensional phase space by transforming irregular behaviours of nonlinear time-varying systems into a strange attractor (e.g., a Rossler attractor or a Lorenz attractor), and it can then predict future states. For this study, the author used a training dataset measured in an existing high-rise building and examined the predictive abilities of the chaos time series inverse model modelled into phase spaces with strange attractors in comparison with those of the Support Vector Regression (SVR) out of the inverse model. The paper discusses the effective use of the chaos time series inverse model for multi-step ahead prediction.

1. Introduction

Recently, the Building Energy Management System (BEMS) has been used to operate various building systems (dynamic blinds, lights, boiler, chiller, fan, pump, etc.). Model Predictive Control (MPC), which has been used for robust BEMS, is based on sensor data, a mathematical model of the systems, and optimization techniques. In general, optimal control logic in the MPC is an iterative process to search for optimal control variables that minimize a cost function over a time horizon for a given system. The time horizon usually consists of multiple sampling times. Afram and Janabi-Sharifi [1] showed that MPC for Heating, Ventilation and Air Conditioning (HVAC) system control has the advantages of providing better performance and transient response than other control methods (on/off control, Proportional (P)/Proportional Integral (PI)/Proportional Integral Derivate (PID) control, etc.). They indicated that system models, disturbance predictions such as climate data, time horizon, and optimization techniques for accuracy, and reliable MPC runs are major considerations. In particular, the system model, which can be divided into a forward model and an inverse model [2], must be carefully chosen.

The forward model can predict the system’s behaviours using Building Performance Simulation (BPS) tools (e.g., EnergyPlus, ESP-r, and TRNSYS), which are based on first-principles (heat transfer and thermodynamics). Such BPS tools have been used for real-time MPC by coupling with middleware programs (MLE+ [3], BCVTB [4], and FMU [5], etc.). However, it is not easy to develop a robust first principles-based simulation model due to many unknown inputs. In particular, the unknown inputs inherited in the model cause model uncertainties. Calibration processes can be used to remedy this. Heo [6] showed that a stochastic calibration technique (i.e., Bayesian calibration) in an optimization routine could sufficiently improve performance qualities. However, it should be noted that the models coupled with the calibration technique may not be suitable in terms of real-time MPC runs, since the BPS tools integrated with the calibration techniques require extensive computation time.

In contrast, the inverse model has been accepted as a surrogate for the first-principle model due to the recent development of machine learning techniques. The inverse model is constructed based on observed data and machine learning algorithms. If measured data are available and reliable, it demands less computation time and modelling effort compared to the conventional BPS tools. In general, examples of the inverse model include the Auto Regressive Integrated Moving Average (ARIMA), Artificial Neural Network (ANN), Support Vector Regression (SVR), and Gaussian Process Model (GPM). Most previous studies have showed that the aforementioned inverse models are able to produce relatively accurate outputs, and can be used for reliable MPC runs. Niu et al. [7] presented a data-driven SVR with a data mining technology for forecasting short-term electric power load, and insisted that SVR is effective in power load forecasting. Kim and Park [8] showed real-time stochastic MPC formulations and processes of a chiller system using a coupling between Genetic Algorithm (GA) and GPM. They found that a stochastic GPM based on the correlation between observed data can account for model uncertainty, and can find reliable optimal control variables (outlet temperatures of chilled water loop and cooling tower loop) in feasible regions. Kiguchi et al. [9] suggested a GPM for predicting Photovoltaic (PV) generation and on-site consumption, and showed that the developed GPM in terms of an excess-generation model could produce much higher prediction performance than the decoupled value. However, it should be noted that their robustness significantly depends on the qualities of the observed data. When a system is influenced by complex and dynamic interactions with other HVAC systems and when dominant time-varying data are not measured (or are not available during the development of the model), the inverse models usually fail to produce a performance prediction. For example, occupants’ behaviour is a dominant unknown input influencing the output prediction, and is usually hard to reflect in the inverse models. Hence, the inverse models usually produce probabilistic outputs with high variances.

For reliable performance prediction of the inverse model, this study presents a chaos time series inverse model using a coupling method between SVR and chaos theory. The chaos time series inverse model can reconstruct a deterministic and low-dimensional phase space by transforming irregular behaviours of nonlinear time-varying systems into a strange attractor (e.g., a Rossler attractor or a Lorenz attractor), and can then predict future states. Karatasou and Santamouris [10] and Wang et al. [11] showed that a chaos time series inverse model based on phase space dynamics can provide accuracy and reliable performance predictions. Previous chaos studies presented remarkable results for forecasting the behaviours of systems. However, questions remain about multi-step ahead prediction.

With the aforementioned concepts in mind, this study aims to compare the inverse model (SVR) with the chaos time series inverse model (Chaos SVR) in terms of multi-step ahead prediction. The SVR has been widely used for the prediction of building systems. In particular, the SVR has excellent prediction capabilities compared with other inverse models [12]. For this study, total electric energy consumption acquired in an existing office building was chosen as the training data. The comparison was made by focusing on the multi-step ahead prediction. The prediction abilities of the models were assessed in terms of the Mean Bias Error (MBE) and Coefficient of Variance of the Root Mean Square Error (CVRMSE), as suggested by [13].

2. Chaos Time Series Inverse Model

The physical phenomena of real building systems are complex and transient, where a variation of the initial state of a certain system may influence overall systems. In other words, variation of the initial state of a system may cause a critical state or tipping-point of an interconnected system, and then have a significant effect on its performance. For example, occupants’ behaviours affect control of openings (windows or doors) and systems (blinds, lights, HVAC systems, etc.). Occupants move between rooms or respond to system information or environment such as indoor temperature. Such actions influence various building systems’ functions as well as performance indicators (e.g., energy consumption, indoor air quality). To deal with issues of occupants’ behaviours, various stochastic models (e.g., Humphreys algorithm [14], Hidden Markov Chain model [15], and probit model [16]) were developed and validated. However, Andersen et al. [17] insisted that “most validation processes do not go beyond assessing the performance predictions on the very same dataset the model was inferred from” (p. 106). They found that the validation results may lead to significant performance gaps between measured data and predicted outputs if previous stochastic occupant behaviour models were tested using external or other datasets. This means that stochastic behaviour models are suitable when occupants’ behaviours in a given target building are measured. However, in the real BEMS installed in various sensor devices, it is not easy to acquire the data for occupants’ behaviours due to privacy protection, high installation and maintenance costs of sensor devices, etc.

In these conditions, building systems are influenced by such occupants’ behaviours, and can result in an unexpected state or tipping-point according to variations in their initial states. Since, an individual action can have an important bearing on other occupants’ cognitive thinking, as well as on the physical states of building systems. In other words, it can be inferred that real systems are very sensitive to small changes (e.g., the control of openings). This sensitivity of the initial states is called the ‘butterfly effect’. However, most inverse models (e.g., ARIMA, ANN, SVR, and GPM) are not able to reflect the aforementioned unexpected phenomena incurred from the butterfly effect, and may provide predictions with high variances due to unmeasured dominant inputs (e.g., occupants’ behaviours). Otherwise, the aforementioned phenomena can be modelled as a chaos state in random time series data.

The chaos behaviours in time series data can be identified using a Lyapunov exponent [18] and a Hurst exponent [19]. The Lyapunov exponent is a method to quantify the sensitivity of the initial states, termed the butterfly effect, whereby great changes are generated from the small changes in a given condition. If the Lyapunov exponent is positive, the time series data include chaos behaviours having an exponential divergence of the trajectories. Otherwise, the negative Lyapunov exponents are characteristic of non-conservative systems. The Hurst exponent, which has a value in the range from 0.0 to 1.0, can be used to pick out a fractal geometry from time series data. If the Hurst exponent indicates between 0.5 and 1.0, the time series data include the chaos behaviours. Otherwise, it means a random walk against the chaos behaviours. The random walk is a mathematical formalization of a path that consists of a succession of random steps [20]. In other words, the Lyapunov exponent and Hurst exponent can be used to find manifestation of chaos behaviours in the given time series data.

Chaos theory reconstructs the complex irregular behaviours of nonlinear systems into a phase space with deterministic rules such as a strange attractor [10,11], if the time series data prove to be in a chaos state according to aforementioned methods (Lyapunov exponent and Hurst exponent). The chaos time series inverse model can be developed by coupling between the reconstructed phase space and the inverse model. According to Taken’s theory [21], one-dimensional time series data x(t) can be transformed into a state vector z(t) by calculating an embedding dimension d and time delay $\tau$, as shown in Equation (1). The embedding dimension and time delay are calculated using a False Nearest Neighbour (FNN) algorithm and an Average Mutual Information (AMI), respectively. The reconstructed d-dimensional phase space can be used to predict the next time horizon throughout the inverse model F(z_t), as shown in Equation (2).

$$\overrightarrow{z_t}=\left[x_t,x_{t-\tau },\dots ,x_{t+(d-1)\tau }\right]$$

(1)

$$\overrightarrow{z_{t+1}}=F(z_t)$$

(2)

The FNN algorithm proposed by [22] is used to estimate the minimal sufficient embedding dimension. When state vectors are reconstructed while a random embedding dimension is increased, the state vectors are divided into their nearest neighbours ($D_i^{nn}$ in Equation (3)) and false nearest neighbours ($D_i^{fnn}$ in Equation (4)) according to the Euclidean distance of each state vector. The closer to the embedding dimension FNN ratio ($R_i$ in Equation (5)) gets, the less the FNN ratio becomes. Based on the aforementioned characteristics, the embedding dimension can be obtained using the FNN ratio.

$$D_i^{nn}=\Vert \overrightarrow{z_i}-\overrightarrow{z_j}\Vert$$

(3)

$$D_i^{fnn}=\Vert \overrightarrow{z_{i+1}}-\overrightarrow{z_{j+1}}\Vert$$

(4)

$$R_i=\frac{D_i^{nn}}{D_i^{fnn}}$$

(5)

The AMI developed by [23] is used to calculate the time delay of nonlinear data. It can quantify the dependency between the pairs of random variables through the entropy of their joint probability density function, as shown in Equation (6). The AMI determines the delay time for the phase space from the first minimum time.

$$I=-{\displaystyle \sum _{ij}{p}_{ij}(\tau )\ln \frac{p_{ij}(\tau )}{p_i{p}_j}}$$

(6)

In this study, the one-dimensional time series data identified as a chaos state using the Lyapunov exponent and Hurst exponent are reconstructed as a d-dimensional phase space using the aforementioned FNN algorithm and AMI. The phase space is used to predict outputs of the next time horizon through a coupling with the inverse model.

3. Support Vector Regression (SVR)

To construct the inverse model and the chaos time series inverse model, SVR was chosen. As mentioned in previous studies [7,24,25,26], SVR can be formulated to a linear regression f(x) with a weight factor $\omega$, a high-dimensional feature space $\phi (x_i)$, and a threshold value b, as shown in Equations (7) and (9). $\omega$ is expressed as non-negative Lagrange multipliers $\lambda$, training input samples x_i, and target values y_i. $\phi (x_i)$ denotes a set of nonlinear transformations for mapping the training input samples x_i with target values y_i. The linear regression was reformulated using the Lagrange multipliers and kernel matrix $k(x_i,x_j)$, as shown in Equations (10) and (11).

$$f(x)=\omega \times \phi (x_i)+b$$

(7)

$$\omega ={\displaystyle \sum _{i=1}^n{\lambda }_i{y}_i\phi (x_i)}$$

(8)

$$b=-{\displaystyle \sum _{i=1}^n{\lambda }_i{y}_i\phi (x_i)\phi (x_i)}$$

(9)

$$f(x)={\displaystyle \sum _{i=1}^n({\lambda }_i-{\lambda }_i^{*})k(x_i,x_j)+b}$$

(10)

$$k(x_i,x_j)=\exp \left(-{\Vert x_i-x_j\Vert }^2/(2{\sigma }^2)\right)$$

(11)

To estimate the weight factor $\omega$ and threshold value b, the linear regression model is required. The linear regression model results in a quadratic programming optimization problem. In this study, the optimization problem is constrained by a set of inequality equations with canonical hyperplanes, as shown in Equations (12) and (13). $C$ denotes a penalty parameter which is a constant to determine the trade-off between training error and model flatness [26]. ${\xi }_i$ denotes a slack variable used to deal with noise errors in the data. With the Lagrange multipliers $\lambda$, the optimization problem can be expressed as shown in Equation (14). To maximize the Lagrange multipliers in the cost function, Equations (14)–(16) are obtained. It should be noted that the predictability of the SVR was influenced by estimates of three unknown parameters ($C$, $\sigma$, and $\epsilon$). To estimate such parameters, various optimization techniques (e.g., gradient based optimization, Genetic Algorithm (GA), Particle Swarm Optimization (PSO)) can be used. In the study, the gradient based optimization was used. It has the advantage of finding an optimal solution quickly in the feasible region.

$$\arg \underset{(\omega ,\xi ,b)}{\min }\left\{\frac{1}{2}{\Vert \omega \Vert }^2+C{\displaystyle \sum _{i=1}^n{\xi }_i}\right\}$$

(12)

$$\begin{array}{lll}s.t.& y_i-(\omega \phi (x_i))+b)\le \epsilon +{\xi }_i& \mathrm{for} {\xi }_i\ge 0\\ & (\omega \phi (x_i))+b)-y_i\le \epsilon +{\xi }_i& \mathrm{for} {\xi }_i\ge 0\end{array}$$

(13)

$$\arg \underset{(\omega ,\xi ,b)}{\min }\underset{({\lambda }_i,{\lambda }_i^{*})}{\max }\left\{\begin{array}{l}\frac{1}{2}{\Vert \omega \Vert }^2+C{\displaystyle \sum _{i=1}^n{\xi }_i}\\ -{\displaystyle \sum _{i=1}^n{\lambda }_i\left[(\omega x_i-b)-1+{\xi }_i\right]-{\displaystyle \sum _{i=}^n{\lambda }_i^{*}{\xi }_i}}\end{array}\right\}$$

(14)

$$\tilde{L}({\lambda }_I,{\lambda }_I^{*})={\displaystyle \sum _{i=1}^n{y}_i\left({\lambda }_j-{\lambda }_j^{*}\right)}-\epsilon {\displaystyle \sum _{i=1}^n\left({\lambda }_i-{\lambda }_i^{*}\right)}-\frac{1}{2}{\displaystyle \sum _{i,j=1}^n\left({\lambda }_i-{\lambda }_i^{*}\right)\left({\lambda }_j-{\lambda }_j^{*}\right)k({\mathrm{x}}_i,x_j)}$$

(15)

$$\begin{array}{ll}s.t.& {\lambda }_i,{\lambda }_i^{*}\in \left[0,C\right]\end{array}$$

(16)

4. SVR vs. Chaos Time Series Inverse Model

4.1. Target Building and Training Data

An existing high-rise office building was selected. The building has 10 stories above ground and three underground levels. In this study, the BEMS data were used including total electric energy consumption at a sampling time of 1 h during 20 weekdays in July (Figure 1). The BEMS data were used as training and validating data for constructing two inverse models (SVR and chaos time series inverse model) based on machine learning and chaos theory. The data were divided as follows: (1) 15 days for development of the model, (2) 5 days for validation of the model.

4.2. Comparison between SVR and Chaos Time Series Inverse Model

Figure 2 only shows an architecture of inverses models for formulating a SVR and Chaos time series inverse model in terms of 1-step ahead prediction. A training dataset D of n observations $D=\left\{(x_i,y_i)|i=1,\dots ,n\right\}$ consist of training input data $x={\left[x_1,\dots ,x_{n-k}\right]}^T$ and training output data $y={\left[x_{1+k},\dots ,x_n\right]}^T$ reflecting different time delays according to k-steps. With the training dataset, the inverse models were constructed using a linear regression model. When the new inputs $x_{1:k}^{*}={\left[x_1^{*},\dots ,x_k^{*}\right]}^T$ were given, it was able to predict outputs $y(x^{*})$ of next time horizon. If the k-step (2, 3, 4, 5, and 6) was increased, training input and output data were set to $x={\left[x_1,\dots ,x_{n-k}\right]}^T$ and $y={\left[x_{1+k},\dots ,x_n\right]}^T$, respectively. In the example, training input and output data for two-step ahead prediction were set to $x={\left[x_1,\dots ,x_{n-2}\right]}^T$ and $y={\left[x_3,\dots ,x_n\right]}^T$, respectively, and then it was possible to obtain the predicted outputs $x_{n+1:n+2}^{*}={\left[x_{n+1}^{*},x_{n+2}^{*}\right]}^T$ using the new data inputs $x_{n-1:n}^{*}={\left[x_{n-1}^{*},x_n^{*}\right]}^T$.

Before attempting development of the chaos time series inverse model, the Lyapunov exponent and Hurst exponent were calculated to identify whether or not the time series data were in the chaos state. The Lyapunov exponent and Hurst exponent results were 1.62 and 0.56, respectively. Since the Lyapunov exponent is positive, and the Hurst exponent is more than 0.5, the obtained time series data include characteristics of chaotic behaviour. Based on the chaos state in the given data, the embedding dimension and time delay were calculated using the FNN algorithm and AMI to reconstruct the d-dimensional phase space with state vectors in the chaos time series inverse model.

Figure 3 and Figure 4 show the results of the embedding dimension and time delay calculated by the FNN algorithm and AMI, respectively. The calculated embedding dimension and time delay were 5 and 6, respectively. Figure 5 shows the 3D projection (i.e., ‘strange attractor’) of 5D phase space having a delay time of 6 h, resulting from the measured one-dimensional time series data. The strange attractor suggested by [27], which can be represented geometrically in d-dimensional phase space, formulates repeating patterns in the d-dimensional phase space having the delay time, and the orbit or trajectory of the strange attractor moves in a deterministic but unpredictable manner [28]. Due to the deterministic manner, it can capture dynamic natures of nonlinear systems using the flow of a curve with the observed finite set of points, and such repeating patterns can be used to predict future states. In other words, the measured raw BEMS data, which were dependent on the embedding dimension and time delay, were transformed into new vector time series (1D → 5D phase space having a delay time of 6). The new vector times series were used as training data for the inverse model. In other words, the chaos time series inverse model is constructed by coupling between new vector time series and inverse model (SVR). Due to an increase of dimensionality, the chaos time series model (Chaos SVR) has a minor increase in computation cost in comparison to the inverse model (SVR). However, the difference in computation speed between Chaos SVR and SVR is imperceptible.

Figure 6 shows the predicted outputs of the inverse model (SVR) and chaos time series inverse model (Chaos SVR) with varying k-steps (1, 2, 3, 4, 5, and 6). When the number of steps (k) increased, the prediction accuracy of the SVR significantly decreased. It can be inferred that the SVR was unsuitable for multi-step ahead predictions (e.g., for multi-step ahead MPC). On the other hand, the prediction accuracy of the Chaos SVR using state orbits of the reconstructed d-dimensional phase space was good compared to the SVR. In other words, the chaos time series inverse model can provide more accurate long-term outputs than the inverse model. In light of the above, the chaos time series inverse model is advantageous for providing multi-step ahead predictions. This means that energy flows of nonlinear dynamical systems can be modelled using the strange attractor and greater future changes (total electric energy consumption) incurred from various unknown minor changes (e.g., occupants’ actions) can be sufficiently predicted.

Table 1. Comparison between SVR and Chaos SVR using Mean Bias Error (MBE) and Coefficient of Variance of the Root Mean Square Error (CVRMSE).

k-Step	SVR		Chaos SVR
	MBE	CVRMSE	MBE	CVRMSE
1-step	4.85	12.26	4.13	13.75
2-step	6.43	21.39	4.19	13.95
3-step	8.32	31.33	4.20	13.97
4-step	10.15	38.93	4.18	13.90
5-step	10.87	43.08	4.18	13.90
6-step	10.36	43.87	4.21	13.97

compares the SVR and the chaos SVR in terms of the MBE and CVRMSE. The predictions of the chaos time series inverse model (from one-step to six-step ahead predictions) are satisfactory when assessed by [13] (at the sampling time of 1 h, the recommended values of MBE and CVRMSE are less than 10% and 30%, respectively). Even if the CVRMSE result of SVR is only minorly better than that of the Chaos SVR in terms of one-step ahead prediction, the Chaos SVR should provide accurate and trustworthy long-term predicted outputs in terms of the multi-step predictions, as shown in Table 1. The chaos time series inverse model using such repeating patterns, which changes from the observed finite set of points to another according to a given delay time (refer to Figure 5), has significant prediction abilities within the limit of the delay time of 6. In other words, the Chaos SVR model based on state orbits of the reconstructed d-dimensional phase space can provide reliable multi-step ahead predictions.

In summary, real buildings in which various nonlinear systems and occupants’ actions are highly interconnected have been continuously affected by complex and dynamic factors (environmental/physical/psychological/cognitive/policy/cultural information, etc.). Such factors may be drawn to unstable or unpredictable system behaviours at some points as time goes by. Ultimately, the features interrupt long-term predictions for time-series analysis of the building systems, as shown in the prediction abilities of the inverse model (SVR) that resulted from a case study. On the other hand, the strange attractor derived from the chaos theory can be used for long-term predictions by coupling with the inverse model due to the deterministic manner (regular structure with repeated patterns) in the low-dimensional phase space. In this regard, this study presented an effective method to predict future states along the strange attractor in the field of building performance simulation and examined characteristics of the chaos time series inverse model (Chaos SVR) in comparison with the inverse model (SVR) for long-terms predictions. In the light of the multi-step MPC applications, the chaos time series model coupled with the optimization algorithm (e.g., genetic algorithm) can be used to obtain meaningful predictions and control variables within the short time horizon (1, 2, 3, 4, 5 min).

5. Conclusions and Future Work

This study compared the multi-step ahead predictability of the chaos time series inverse model and the inverse model (SVR). As the number of steps (k) increases, the prediction accuracy of the SVR model decreases. However, the chaos SVR model provides multi-step ahead predictions that are very close to the measured data.

Reconstructing the measured one-dimensional time series data into a phase space, which reduces prediction risks, enhances the chaos time series inverse model. In terms of multi-step ahead prediction, the chaos time series inverse model (SVR + chaos behaviour) can accurately forecast future states of a real system that has complex and irregular behaviours. The chaos inverse models perform better than the inverse model (SVR), and can be used for real-time MPC and other control applications. Future works may include the following:

●

Application of the chaos time series inverse model to a variety of building systems: the proposed chaos time series inverse model will be applied to various building systems (lights, blinds, HVAC system, plants, etc.).

●

Real-time optimal control: to implement real-time MPC, the validated chaos time series inverse model will be coupled with optimization techniques (e.g., genetic algorithm, particle swarm optimization) and used to find optimal control variables.