# Simulation-Based Evaluation and Optimization of Control Strategies in Buildings

^{1}

^{2}

^{3}

^{4}

^{5}

^{6}

^{7}

^{8}

^{*}

## Abstract

**:**

## 1. Introduction

- Utilise predictions instead of historical data: Here, instead of gathering and analysing historical data from the building to improve the efficiency of specific controllers, forecasts (e.g., on weather, occupancy, equipment gains, etc.) are utilised to determine the (near-)future state (e.g., thermal/visual comfort conditions) of the building. This can enable for a continuous adaptation of the control logic to the (predicted) needs of the building and the microclimatic conditions of each site.
- Automated controller tuning: Here, an optimisation process is defined (utilising the available predictions) to design efficient controllers in an automated and laborious-free manner.

^{®}, which provide a remote (cloud-based) layer for hosting computationally demanding services, such as supervisory-level control design.

- installing additional sensors/meters required for accurate state estimation of the building to minimise the model/reality mismatch;
- configuration of the overall MPC solution (including model calibration) requires significant amount of time and the involvement of high-qualified engineers; and
- monitoring and solution debugging times of MPC solutions are significantly higher compared to the deployment of traditional knowledge-based controllers.

## 2. Model Predictive Control in Buildings

- t is the discrete time-step (with its length depending on the application), T is the prediction horizon.
- ${\mathbf{u}}^{\ast}:=\left[{u}_{1},{u}_{2},\dots ,{u}_{T}\right]$ is the optimal control solution.
- ${x}_{t}$ is a vector of values for each time-step t for the states of the system (e.g., wall and air temperatures [3]).
- The admissible control actions ${u}_{t}$ are a vector of values for each time-step t, with each value corresponding to a control action (e.g., temperature set-point, hot water flow, etc.) for all controllable systems of the building.
- ${w}_{t}$ is a vector of exogenous, uncontrollable factors affecting the system, like weather conditions or user actions. As discussed previously, we have some (reliable) predictions ${\widehat{w}}_{t}$ for these disturbances.
- $\mathbb{U}\subseteq {\mathbb{R}}^{m}$ and $\mathbb{X}\subseteq {\mathbb{R}}^{n}$ are closed sets defining the admissible controls and states respectively, while $\mathbb{W}\subseteq {\mathbb{R}}^{v}$ is the closed set of exogenous uncorrelated factors.
- $f:{\mathbb{R}}^{(n+v)}\times {\mathbb{R}}^{m}\to {\mathbb{R}}^{n}$ is a (possibly non-linear) function that describes the system dynamics.
- ${g}_{t}\in \mathbb{R}$ is the value of the cost (or objective) function to be minimised at time t and represents a performance measure, estimating the efficiency of a given control strategy. In the buildings domain, this index can either be economical (e.g., minimise operational cost) or environmental (e.g., maximise the net energy produced or minimise CO${}_{2}$ emissions).
- $C\left({x}_{t}\right)\le 0$ is a function usually describing thermal comfort constraints for the building occupants.

**Physics-based models**These models are designed based on first principles, utilising knowledge of the underlying physical process of the building and are usually analogous to electrical Resistance–Capacitance (RC) networks [3,6,17,18,19,20,21]. In an effort to reduce the model design- and calibration-associated costs for these models, a number of MPC approaches based on system identification methods [18,50,51,55,56,57,58] have been proposed. The problem here is that some initial (quasi-)random actions need to be performed to the real building (called persistent excitation signals in system identification theory) [51,56,59,60] to satisfy key theoretical assumptions on reliable statistical identification [55,61]. In real operation, this requirement is almost always infeasible without compromising the comfort or the health of the occupants (note that this is a problem closely related to the safe exploration/exploitation dilemma that has been extensively studied in reinforcement learning [52,53,62,63]). It is conceivable that a poorly identified model can lead to severe compromise of occupant comfort conditions; the model will provide inaccurate predictions that will lead to unsuitable controllers [64].

**Data-driven models**Here, a linear or non-linear regression model is fitted using available data from the real building and then utilised for the MPC application. Several types of regression functions have been reported in the literature, e.g. neural networks, support vector regression, autoregressive moving average, regression trees, etc. (see [5,65] for an overview). These models have the same data requirements as the system identification methods described before while they also suffer from poor extrapolation properties.

**BEP simulation models**When these models are utilised for control design, the main concern is the optimisation algorithm to be used, as they are characterised by high simulation times. A common approach is to assume that the simulation model is an exact model of the real building and construct a surrogate regression model off-line; this model is orders of magnitude faster compared to the original model so it could be used for MPC-like on-line control design [55,66]. From our experience with real-building control experiments [2,4,67], the assumption that the original simulation model (and thus also the surrogate model derived from it) can actually predict all states of the real building, under all different combinations of weather conditions and occupant actions, usually does not hold in practice. Instead, what is required is a continuous calibration process, for example as defined in the digital twin paradigm [68].

## 3. The Proposed Approach

- Design Phase:
- The Control Improvement module evaluates candidate control strategies utilising the simulation model under a set of pre-defined objectives (e.g., energy consumption, thermal comfort in each controller space, etc).
- When the Design Phase finishes, the best resulting strategy is communicated and deployed to the real building.

- Application Phase:
- The deployed controllers are used to generate new control actions in each control time-step for each controllable system of the building (possibly also utilising in-building and weather sensor measurements).

#### 3.1. Simulation-Based Evaluation Using Multi-Criteria Decision Analysis Methods

- The positive outranking flow ${\varphi}^{+}\left({S}_{i}\right)={\sum}_{{S}_{j}}\frac{pr({S}_{i},{S}_{j})}{N-1}$ expresses the degree to which a control strategy ${S}_{i}$ is preferred over all the other control strategies ${S}_{j}$.
- The negative outranking flow ${\varphi}^{-}\left({S}_{i}\right)={\sum}_{{S}_{j}}\frac{pr({S}_{j},{S}_{i})}{N-1}$ expresses the degree to which all the other control strategies ${S}_{j}$ are preferred over this specific one.

#### 3.2. Simulation-Based Optimisation Using Gaussian Process State Space Models

- The Squared Exponential (SE) covariance function, defined as:$${k}_{\mathrm{SE}}(x,{x}^{\prime})=exp\left(-\frac{\left|\right|x-{x}^{\prime}{\left|\right|}^{2}}{2{d}^{2}}\right),$$
- The Rational Quadratic (RQ), defined as:$${k}_{\mathrm{RQ}}(x,{x}^{\prime})={\left(1+\frac{\left|\right|x-{x}^{\prime}{\left|\right|}^{2}}{2a{d}^{2}}\right)}^{-a},$$
- The Matérn covariance functions, defined as follows:$${k}_{\mathrm{Matern}}(x,{x}^{\prime})=\frac{{2}^{1-\nu}}{\mathsf{\Gamma}\left(\nu \right)}{\left(\frac{\sqrt{2\nu}\left|\right|x-{x}^{\prime}\left|\right|}{d}\right)}^{\nu}{H}_{\nu}\left(\frac{\sqrt{2\nu}\left|\right|x-{x}^{\prime}\left|\right|}{d}\right),$$

- t, ${\mathbf{u}}^{\ast}$, ${u}_{t}$, $\mathbb{U}$, $\mathbb{X}$, $\mathbb{W}$, ${g}_{t}$ and $C(\xb7)$ have the same definitions as in Equation (1).
- ${\widehat{w}}_{t}$ is a vector of available (weather, occupancy, etc.) predictions.
- ${\widehat{x}}_{0}$ is provided by the simulator.
- $GP$ represents a set of GPs that provide an estimate of the states ${\widehat{x}}_{t+1}$ at time $t+1$, taking into account the states ${\widehat{x}}_{t}$, exogenous predictions ${\widehat{w}}_{t}$ and applied actions ${u}_{t}$ at time t.
- $\mathit{s}\left[{\sigma}_{1},{\sigma}_{2},\dots ,{\sigma}_{T}\right]$ is a vector containing the prediction variance in all time-steps t.
- $\u03f5$ is a design parameter. By requiring ${\parallel \mathit{s}\parallel}_{\infty}\le \u03f5$, we can implicitly allow for more or less exploration in the optimisation algorithm (i.e., favouring or not control actions that lead to “uncertain” states), simply by changing the value of $\u03f5$.

- An initial set of random simulations is performed in the detailed thermal simulation model. Note that, since we are operating in the simulation world, we can safely explore random or drastic actions with no effect to the real-building occupants.
- A set of GPs is trained using the simulated state, action and disturbance data.
- The best parameters discovered are simulated in the “expensive” thermal simulation model and the GPs are re-trained in a new dataset augmented with the most recent simulation data.
- The process re-iterates until convergence or a time-out occurs.

#### 3.3. Closed-Loop Control Extension

## 4. Experiments and Results

#### 4.1. The Example Building

#### 4.2. The Experimental Setup

- The building is controlled only during the occupancy period (08:00–17:00).
- The control actions are hourly setpoints, which are the same for the four control offices. Thus, the control time-step is one hour.
- The thermal comfort constraints are defined according to Fanger PMV index and are posed as illustrated in Equation (11).
- The (predicted) exogenous factors in our case are the weather conditions (provided by the weather file) and the occupancy patterns (which are assumed fixed and known in advance).
- Since we consider that the predictions for the disturbances are accurate, there is no need to re-design our control strategy at each control time-step (i.e., every hour), so for the work presented here we design the entire control strategy once for the entire day. Of course, in future application in the real building frequent control, (re-)design will be required to compensate for all unpredicted disturbances.
- The evolution of the system dynamics is provided by the EnergyPlus simulator; as described earlier we do not have access to the internal equations of the simulator but we can only observe the input-output relationship.
- The objective/cost function to be minimised is the total energy consumption measured at the VRF5 unit, which serves the four controllable offices.

#### 4.3. The Software Framework

#### 4.4. Results

#### 4.4.1. Approach Using Multi-Criteria Decision Analysis

- Weights: ${w}_{1}=10$ and ${w}_{2}=0$.
- Reasoning: This is a weighting combination that gives priority to the energy consumption.
- Result: The ranking of the strategies from PROMETHEE II algorithm is as expected, since the algorithm prioritises the solutions taking into account only the energy consumption criterion.

- Weights: ${w}_{1}=0$ and ${w}_{2}=10$.
- Reasoning: This is a weighting combination that gives priority to the thermal comfort conditions in the building.
- Result: The algorithm favours solutions that maintain comfortable interiors, but the highest ranking controller is the one with the minimum consumption among all the controllers that achieve comfort. This is one of the properties of the algorithm: even though we indicated that the energy consumption has no priority for us, the fact that this criterion needs to be minimised too leads to this sorting.We have to note here that, even though Strategy #9 leads to limited discomfort, it is not among the high-ranking strategies. This is due to the high priority given to the comfort criterion, which results into treating the constraint violations as hard constraints.

- Weights: ${w}_{1}=5$ and ${w}_{2}=5$.
- Reasoning: This is a weighting combination that gives equal priority to both criteria.
- Result: Here, the comfort violations are treated as soft constraints and the ranking of strategies is quite balanced, but the highest ranking strategy determined by the algorithm (Strategy #10) leads to significant discomfort in Office 87.

#### 4.4.2. Approach Using Gaussian Process State Space Models

- Input Features:
- ${\widehat{w}}_{t}^{1}$: Outdoor Air Temperature
- ${\widehat{w}}_{t}^{2}$: Global Solar Radiation in the Horizontal plane
- ${\widehat{x}}_{t}^{1-4}$: Indoor Air Temperature of the four controlled rooms
- ${\widehat{x}}_{t}^{5-8}$: Indoor Relative Humidity of the four controlled rooms
- ${\widehat{x}}_{t}^{9-12}$: Fanger PMV value of the four controlled rooms
- ${\widehat{x}}_{t}^{13}$: Energy consumption of VRF5 in the time-interval $(t-1,t)$
- ${u}_{t}$: the control setpoint

- Target values: all states at time-step $t+1$, i.e., ${\widehat{x}}_{t+1}^{1-13}$.

## 5. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Roth, K.W.; Westphalen, D.; Feng, M.Y.; Llana, P.; Quartararo, L. Energy Impact of Commercial Building Controls and Performance Diagnostics: Market Characterization, Energy Impact of Building Faults And Energy Savings Potential; Final Report; TAIX LLC: Cambrige, MA, USA, 2005; p. 412. [Google Scholar]
- Kontes, G.D. Model-Assisted Control for Energy Efficiency in Buildings. Ph.D. Thesis, School of Production Engineering and Management, Technical University of Crete, University Campus, Chania, Greece, 2017. [Google Scholar]
- Oldewurtel, F.; Parisio, A.; Jones, C.N.; Gyalistras, D.; Gwerder, M.; Stauch, V.; Lehmann, B.; Morari, M. Use of model predictive control and weather forecasts for energy efficient building climate control. Energy Build.
**2012**, 45, 15–27. [Google Scholar] [CrossRef] [Green Version] - Kontes, G.D.; Valmaseda, C.; Giannakis, G.I.; Katsigarakis, K.I.; Rovas, D.V. Intelligent BEMS design using detailed thermal simulation models and surrogate-based stochastic optimization. J. Process Control
**2014**, 24, 846–855. [Google Scholar] [CrossRef] - Afram, A.; Janabi-Sharifi, F. Theory and applications of HVAC control systems–A review of model predictive control (MPC). Build. Environ.
**2014**, 72, 343–355. [Google Scholar] [CrossRef] - Sturzenegger, D.; Gyalistras, D.; Morari, M.; Smith, R.S. Model predictive climate control of a swiss office building: Implementation, results, and cost–benefit analysis. IEEE Trans. Control Syst. Technol.
**2016**, 24, 1–12. [Google Scholar] [CrossRef] - Liang, W.; Quinte, R.; Jia, X.; Sun, J.Q. MPC control for improving energy efficiency of a building air handler for multi-zone VAVs. Build. Environ.
**2015**, 92, 256–268. [Google Scholar] [CrossRef] - Mei, J.; Xia, X. Energy-efficient predictive control of indoor thermal comfort and air quality in a direct expansion air conditioning system. Appl. Energy
**2017**, 195, 439–452. [Google Scholar] [CrossRef] - Godina, R.; Rodrigues, E.M.; Pouresmaeil, E.; Catalão, J.P. Optimal residential model predictive control energy management performance with PV microgeneration. Comput. Oper. Res.
**2018**, 96, 143–156. [Google Scholar] [CrossRef] - Hilliard, T.; Swan, L.; Qin, Z. Experimental implementation of whole building MPC with zone based thermal comfort adjustments. Build. Environ.
**2017**, 125, 326–338. [Google Scholar] [CrossRef] - Ascione, F.; Bianco, N.; De Stasio, C.; Mauro, G.M.; Vanoli, G.P. A new comprehensive approach for cost-optimal building design integrated with the multi-objective model predictive control of HVAC systems. Sustain. Cities Soc.
**2017**, 31, 136–150. [Google Scholar] [CrossRef] - Razmara, M.; Maasoumy, M.; Shahbakhti, M.; Robinett, R., III. Optimal exergy control of building HVAC system. Appl. Energy
**2015**, 156, 555–565. [Google Scholar] [CrossRef] - Fiorentini, M.; Wall, J.; Ma, Z.; Braslavsky, J.H.; Cooper, P. Hybrid model predictive control of a residential HVAC system with on-site thermal energy generation and storage. Appl. Energy
**2017**, 187, 465–479. [Google Scholar] [CrossRef] [Green Version] - Godina, R.; Rodrigues, E.M.; Pouresmaeil, E.; Matias, J.C.; Catalão, J.P. Model predictive control home energy management and optimization strategy with demand response. Appl. Sci.
**2018**, 8, 408. [Google Scholar] [CrossRef] - Gwerder, M.; Tödtli, J.; Lehmann, B.; Dorer, V.; Güntensperger, W.; Renggli, F. Control of thermally activated building systems (TABS) in intermittent operation with pulse width modulation. Appl. Energy
**2009**, 86, 1606–1616. [Google Scholar] [CrossRef] - Tödtli, J.; Gwerder, M.; Lehmann, B.; Renggli, F.; Dorer, V. TABS-control: Steuerung und Regelung von thermoaktiven Bauteilsystemen. In Handbuch für Planung, Auslegung und Betrieb, Schriftenreihe Technik, Faktor-Verl, Zürich; FAKTOR Verlag AG: Zurich, Switzerland, 2009. [Google Scholar]
- Moroşan, P.D.; Bourdais, R.; Dumur, D.; Buisson, J. Building temperature regulation using a distributed model predictive control. Energy Build.
**2010**, 42, 1445–1452. [Google Scholar] [CrossRef] - Cigler, J.; Prívara, S. Subspace identification and model predictive control for buildings. In Proceedings of the 2010 11th International Conference on Control Automation Robotics & Vision (ICARCV), Marina Bay Sands Resort, Singapore, 7–10 December 2010; pp. 750–755. [Google Scholar]
- Širokỳ, J.; Oldewurtel, F.; Cigler, J.; Prívara, S. Experimental analysis of model predictive control for an energy efficient building heating system. Appl. Energy
**2011**, 88, 3079–3087. [Google Scholar] [CrossRef] - Nghiem, T.X.; Pappas, G.J. Receding-horizon supervisory control of green buildings. In Proceedings of the American Control Conference (ACC), San Francisco, CA, USA, June 29–July 1 2011; pp. 4416–4421. [Google Scholar]
- Ma, Y.; Anderson, G.; Borrelli, F. A distributed predictive control approach to building temperature regulation. In Proceedings of the American Control Conference (ACC), San Francisco, CA, USA, June 29–July 1 2011; pp. 2089–2094. [Google Scholar]
- Buderus, J.; Dentel, A. Generalization Approach for Models of Thermal Buffer Storages in Predictive Control Strategies. In Proceedings of the 15th IBPSA Conference, San Francisco, CA, USA, 7–9 August 2017. [Google Scholar]
- Sturzenegger, D.; Gyalistras, D.; Semeraro, V.; Morari, M.; Smith, R.S. BRCM Matlab toolbox: Model generation for model predictive building control. In Proceedings of the American Control Conference (ACC), Portland, OR, USA, 4–6 June 2014; pp. 1063–1069. [Google Scholar]
- Cauchi, N.; Abate, A. Benchmarks for cyber-physical systems: A modular model library for building automation systems (Extended version). arXiv, 2018; arXiv:1803.06315. [Google Scholar]
- Kontes, G.D.; Giannakis, G.I.; Horn, P.; Steiger, S.; Rovas, D.V. Using thermostats for indoor climate control in office buildings: The effect on thermal comfort. Energies
**2017**, 10, 1368. [Google Scholar] [CrossRef] - Fisk, W.J. Health and productivity gains from better indoor environments and their relationship with building energy efficiency. Annu. Rev. Energy Environ.
**2000**, 25, 537–566. [Google Scholar] [CrossRef] - Akimoto, T.; Tanabe, S.i.; Yanai, T.; Sasaki, M. Thermal comfort and productivity-Evaluation of workplace environment in a task conditioned office. Build. Environ.
**2010**, 45, 45–50. [Google Scholar] [CrossRef] - International Organization for Standardization. ISO 7730:2005 -Ergonomics of the Thermal Environment: Analytical Determination and Interpretation of Thermal Comfort Using Calculation of the PMV and PPD Indices and Local Thermal Comfort Criteria; International Organization for Standardization: Geneva, Switzerland, 2005. [Google Scholar]
- Standard, A. Standard 55-2013. In Thermal Environmental Conditions for Human Occupancy; ASHRAE: Atlanta, GA, USA, 2013; pp. 9–11. [Google Scholar]
- Zavala, V.M. Real-time optimization strategies for building systems. Ind. Eng. Chem. Res.
**2012**, 52, 3137–3150. [Google Scholar] [CrossRef] - Klauco, M.; Kvasnica, M. Explicit MPC approach to PMV-based thermal comfort control. In Proceedings of the 53rd IEEE Conference on Decision and Control, Los Angeles, CA, USA, 15–17 December 2014; pp. 4856–4861. [Google Scholar]
- Cigler, J.; Prívara, S.; Váňa, Z.; Žáčeková, E.; Ferkl, L. Optimization of predicted mean vote index within model predictive control framework: Computationally tractable solution. Energy Build.
**2012**, 52, 39–49. [Google Scholar] [CrossRef] - Mařík, K.; Rojíček, J.; Stluka, P.; Vass, J. Advanced HVAC control: Theory vs. reality. IFAC Proc. Vol.
**2011**, 44, 3108–3113. [Google Scholar] [CrossRef] - Chen, X.; Wang, Q.; Srebric, J. Model predictive control for indoor thermal comfort and energy optimization using occupant feedback. Energy Build.
**2015**, 102, 357–369. [Google Scholar] [CrossRef] [Green Version] - Lee, S.; Bilionis, I.; Karava, P.; Tzempelikos, A. A Bayesian approach for probabilistic classification and inference of occupant thermal preferences in office buildings. Build. Environ.
**2017**, 118, 323–343. [Google Scholar] [CrossRef] - Yan, D.; Hong, T.; Dong, B.; Mahdavi, A.; D’Oca, S.; Gaetani, I.; Feng, X. IEA EBC Annex 66: Definition and simulation of occupant behavior in buildings. Energy Build.
**2017**, 156, 258–270. [Google Scholar] [CrossRef] [Green Version] - Ghahramani, A.; Tang, C.; Becerik-Gerber, B. An online learning approach for quantifying personalized thermal comfort via adaptive stochastic modeling. Build. Environ.
**2015**, 92, 86–96. [Google Scholar] [CrossRef] [Green Version] - Malavazos, C.; Tsatsakis, K.; Tsitsanis, A. Towards a “context aware” flexibility profiling mechanism for the energy management environment. In Proceedings of the MedPower Conference, Athens, Greece, 2–5 November 2014; pp. 2–5. [Google Scholar]
- Nagy, Z.; Yong, F.Y.; Schlueter, A. Occupant centered lighting control: a user study on balancing comfort, acceptance, and energy consumption. Energy Build.
**2016**, 126, 310–322. [Google Scholar] [CrossRef] - Crawley, D.B.; Lawrie, L.K.; Winkelmann, F.C.; Buhl, W.F.; Huang, Y.J.; Pedersen, C.O.; Strand, R.K.; Liesen, R.J.; Fisher, D.E.; Witte, M.J.; et al. EnergyPlus: creating a new-generation building energy simulation program. Energy Build.
**2001**, 33, 319–331. [Google Scholar] [CrossRef] - Wetter, M.; Zuo, W.; Nouidui, T.S.; Pang, X. Modelica buildings library. J. Build. Perform. Simul.
**2014**, 7, 253–270. [Google Scholar] [CrossRef] - Lilis, G.; Giannakis, G.; Kontes, G.; Rovas, D. Semi-automatic thermal simulation model generation from IFC data. In Proceedings of the European Conference on Product and Process Modelling, Vienna, Austria, 17 September 2014; pp. 503–510. [Google Scholar]
- Bazjanac, V.; Maile, T.; O’Donnell, J.; Rose, C.; Mrazovic, N. Data environments and processing in semi-automated simulation with EnergyPlus. In Proceedings of the CIB W078-W102: 28th International Conference, Sophia Antipolis, France, 26–28 October 2011. [Google Scholar]
- Lilis, G.N.; Giannakis, G.I.; Rovas, D.V. Automatic generation of second-level space boundary topology from IFC geometry inputs. Autom. Constr.
**2017**, 76, 108–124. [Google Scholar] [CrossRef] [Green Version] - Lilis, G.N.; Katsigarakis, K.; Giannakis, G.I.; Rovas, D. A cloud-based platform for IFC file enrichment with second-level space boundary topology. In Proceedings of the Joint Conference on Computing in Construction (JC3), Heraklion, Greece, 4–7 July 2017. [Google Scholar]
- Andriamamonjy, A.; Saelens, D.; Klein, R. An automated IFC-based workflow for building energy performance simulation with Modelica. Autom. Constr.
**2018**, 91, 166–181. [Google Scholar] [CrossRef] - Lilis, G.N.; Giannakis, G.; Katsigarakis, K.; Rovas, D. District-aware Building Energy Performance simulation model generation from GIS and BIM data. In Proceedings of the BSO 2018: 4th Building Simulation and Optimization Conference, Cambridge, UK, 11–12 September 2018. [Google Scholar]
- Giannakis, G. Development of Methodologies for Automatic Thermal Model Generation Using Building Information Models. Ph.D. Thesis, School of Production Engineering and Management, Technical University of Crete, University Campus, Chania, Greece, 2015. [Google Scholar]
- Dean, S.; Mania, H.; Matni, N.; Recht, B.; Tu, S. On the sample complexity of the linear quadratic regulator. arXiv, 2017; arXiv:1710.01688. [Google Scholar]
- Nghiem, T.X.; Jones, C.N. Data-driven demand response modeling and control of buildings with gaussian processes. In Proceedings of the American Control Conference (ACC), Seattle, WA, USA, 24–26 May 2017; pp. 2919–2924. [Google Scholar]
- Jain, A.; Nghiem, T.; Morari, M.; Mangharam, R. Learning and control using Gaussian processes. In Proceedings of the 2018 ACM/IEEE 9th International Conference on Cyber-Physical Systems (ICCPS), Porto, Portugal, 11–13 April 2018; pp. 140–149. [Google Scholar]
- Sutton, R.S.; Barto, A.G. Reinforcement Learning: An Introduction; MIT Press: Cambridge, MA, USA, 1998. [Google Scholar]
- Deisenroth, M.P.; Fox, D.; Rasmussen, C.E. Gaussian processes for data-efficient learning in robotics and control. IEEE Trans. Pattern Anal. Mach. Intell.
**2015**, 37, 408–423. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Bertsekas, D.P. Dynamic Programming and Optimal Control, 4th ed.; Approximate Dynamic Programming; Athena Scientific: Nashua, NH, USA, 2012; Volume II. [Google Scholar]
- Prívara, S.; Váňa, Z.; Gyalistras, D.; Cigler, J.; Sagerschnig, C.; Morari, M.; Ferkl, L. Modeling and identification of a large multi-zone office building. In Proceedings of the 2011 IEEE International Conference on Control Applications (CCA), Denver, CO, USA, 28–30 September 2011; pp. 55–60. [Google Scholar]
- Váňa, Z.; Kubeček, J.; Ferkl, L. Notes on finding black-box model of a large building. In Proceedings of the 2010 IEEE International Conference on Control Applications (CCA), Yokohama, Japan, 8–10 September 2010; pp. 1017–1022. [Google Scholar]
- Garnier, A.; Eynard, J.; Caussanel, M.; Grieu, S. Low computational cost technique for predictive management of thermal comfort in non-residential buildings. J. Process Control
**2014**, 24, 750–762. [Google Scholar] [CrossRef] - Kolokotsa, D.; Pouliezos, A.; Stavrakakis, G.; Lazos, C. Predictive control techniques for energy and indoor environmental quality management in buildings. Build. Environ.
**2009**, 44, 1850–1863. [Google Scholar] [CrossRef] - Žáčeková, E.; Váňa, Z.; Cigler, J. Towards the real-life implementation of MPC for an office building: Identification issues. Appl. Energy
**2014**, 135, 53–62. [Google Scholar] [CrossRef] - Žáčeková, E.; Pcolka, M.; Sebek, M. On Satisfaction of the Persistent Excitation Condition for the Zone MPC: Numerical Approach. IFAC Proc. Vol.
**2014**, 47, 2848–2853. [Google Scholar] [CrossRef] - Van Overschee, P.; De Moor, B. Subspace iDentification for Linear Systems: Theory—Implementation—Applications; Springer Science & Business Media: Berlin, Germany, 2012. [Google Scholar]
- Deisenroth, M.; Rasmussen, C.E. PILCO: A model-based and data-efficient approach to policy search. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), Bellevue, WA, USA, 28 June 2011; pp. 465–472. [Google Scholar]
- Schulman, J.; Levine, S.; Abbeel, P.; Jordan, M.; Moritz, P. Trust region policy optimization. In Proceedings of the International Conference on Machine Learning, Lille, France, 6–11 July 2015; pp. 1889–1897. [Google Scholar]
- Recht, B. A Tour of Reinforcement Learning: The View from Continuous Control. arXiv, 2018; arXiv:1806.09460. [Google Scholar]
- Schmidt, M.; Åhlund, C. Smart buildings as Cyber-Physical Systems: Data-driven predictive control strategies for energy efficiency. Renew. Sustain. Energy Rev.
**2018**, 90, 742–756. [Google Scholar] [CrossRef] - Eisenhower, B.; O’Neill, Z.; Narayanan, S.; Fonoberov, V.A.; Mezić, I. A methodology for meta-model based optimization in building energy models. Energy Build.
**2012**, 47, 292–301. [Google Scholar] [CrossRef] - Katsigarakis, K.; Kontes, G.; Giannakis, G.; Rovas, D. Sense-Think-Act Framework for Intelligent Building Energy Management. Comput.-Aided Civ. Infrastruct. Eng.
**2016**, 31, 50–64. [Google Scholar] [CrossRef] - Glaessgen, E.; Stargel, D. The digital twin paradigm for future NASA and US Air Force vehicles. In Proceedings of the 53rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference 20th AIAA/ASME/AHS Adaptive Structures Conference 14th AIAA, Honolulu, HI, USA, 23–26 April 2012; p. 1818. [Google Scholar]
- Coffey, B.; Haghighat, F.; Morofsky, E.; Kutrowski, E. A software framework for model predictive control with GenOpt. Energy Build.
**2010**, 42, 1084–1092. [Google Scholar] [CrossRef] - Henze, G.P.; Kalz, D.E.; Liu, S.; Felsmann, C. Experimental analysis of model-based predictive optimal control for active and passive building thermal storage inventory. HVAC R Res.
**2005**, 11, 189–213. [Google Scholar] [CrossRef] - Baldi, S.; Michailidis, I.; Kosmatopoulos, E.B.; Ioannou, P.A. A “plug and play” computationally efficient approach for control design of large-scale nonlinear systems using cosimulation: A combination of two “ingredients”. IEEE Control Syst. Mag.
**2014**, 34, 56–71. [Google Scholar] - Korkas, C.D.; Baldi, S.; Michailidis, I.; Kosmatopoulos, E.B. Intelligent energy and thermal comfort management in grid-connected microgrids with heterogeneous occupancy schedule. Appl. Energy
**2015**, 149, 194–203. [Google Scholar] [CrossRef] - Karagevrekis, A.; Baldi, S.; Michailidis, I.; Kosmatopoulos, E.B. Interconnected microgrids: An energyplus simulation test case. Mach. Rev.
**2014**, 1, 7–13. [Google Scholar] - Pichler, M.F.; Dröscher, A.; Schranzhofer, H.; Kontes, G.; Giannakis, G.; Kosmatopoulos, E.B.; Rovas, D. Simulation-assisted building energy performance improvement using sensible control decisions. In Proceedings of the Third ACM Workshop on Embedded Sensing Systems for Energy-Efficiency in Buildings, Seattle, WA, USA, 1–4 November 2011; pp. 61–66. [Google Scholar]
- Olesen, B. International standards for the indoor environment. Indoor Air
**2004**, 14, 18–26. [Google Scholar] [CrossRef] [PubMed] - Luenberger, D.G.; Ye, Y. Linear and Nonlinear Programming; Springer: Berlin, Germany, 1984; Volume 2. [Google Scholar]
- Calandra, R.; Gopalan, N.; Seyfarth, A.; Peters, J.; Deisenroth, M.P. Bayesian gait optimization for bipedal locomotion. In Proceedings of the International Conference on Learning and Intelligent Optimization, Gainesville, FL, USA, 16–21 February 2014; pp. 274–290. [Google Scholar]
- Balaji, B.; Weibel, N.; Agarwal, Y. Managing Commercial HVAC Systems: What do Building Operators Really Need? arXiv, 2016; arXiv:1612.06025. [Google Scholar]
- Diakaki, C.; Panagiotidou, N.; Pouliezos, A.; Kontes, G.D.; Stavrakakis, G.S.; Belibassakis, K.; Gerostathis, T.P.; Livanos, G.; Pagonis, D.N.; Theotokatos, G. A decision support system for the development of voyage and maintenance plans for ships. Int. J. Decis. Support Syst.
**2015**, 1, 42–71. [Google Scholar] [CrossRef] - Greco, S.; Figueira, J.; Ehrgott, M. Multiple Criteria Decision Analysis; Springer: Berlin, Germany, 2016. [Google Scholar]
- Brans, J.P.; Mareschal, B. PROMETHEE methods. In Multiple Criteria Decision Analysis: State of the Art Surveys; Springer: Berlin, Germany, 2005; pp. 163–186. [Google Scholar]
- Moffett, A.; Sarkar, S. Incorporating multiple criteria into the design of conservation area networks: A minireview with recommendations. Divers. Distrib.
**2006**, 12, 125–137. [Google Scholar] [CrossRef] - Mateo, J.R.S.C. Multi Criteria Analysis in the Renewable Energy Industry; Springer Science & Business Media: Berlin, Germany, 2012. [Google Scholar]
- Kocijan, J.; Girard, A.; Banko, B.; Murray-Smith, R. Dynamic systems identification with Gaussian processes. Math. Comput. Model. Dyn. Syst.
**2005**, 11, 411–424. [Google Scholar] [CrossRef] - Rasmussen, C.E.; Williams, C.K. Gaussian Processes for Machine Learning; The MIT Press: Cambridge, MA, USA, 2006; Volume 38, pp. 715–719. [Google Scholar]
- Schölkopf, B.; Smola, A.J.; Bach, F. Learning With Kernels: Support Vector Machines, Regularization, Optimization, and Beyond; MIT Press: Cambridge, MA, USA, 2002. [Google Scholar]
- Brochu, E.; Cora, V.M.; De Freitas, N. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv, 2010; arXiv:1012.2599. [Google Scholar]
- Duvenaud, D.; Lloyd, J.R.; Grosse, R.; Tenenbaum, J.B.; Ghahramani, Z. Structure discovery in nonparametric regression through compositional kernel search. arXiv, 2013; arXiv:1302.4922. [Google Scholar]
- May-Ostendorp, P.; Henze, G.P.; Corbin, C.D.; Rajagopalan, B.; Felsmann, C. Model-predictive control of mixed-mode buildings with rule extraction. Build. Environ.
**2011**, 46, 428–437. [Google Scholar] [CrossRef] - Domahidi, A.; Ullmann, F.; Morari, M.; Jones, C.N. Learning decision rules for energy efficient building control. J. Process Control
**2014**, 24, 763–772. [Google Scholar] [CrossRef] - Le, K.; Bourdais, R.; Guéguen, H. From hybrid model predictive control to logical control for shading system: A support vector machine approach. Energy Build.
**2014**, 84, 352–359. [Google Scholar] [CrossRef] - Tindale, A. Designbuilder Software; Design-Builder Software Ltd.: Gloucestershire, UK, 2005. [Google Scholar]
- Remund, J.; Kunz, S. METEONORM: Global Meteorological Database for Solar Energy And Applied Climatology; Meteotest: Bern, Switzerland, 1997. [Google Scholar]
- Giannakis, G.; Pichler, M.; Kontes, G.; Schranzhofer, H.; Rovas, D. Simulation speedup techniques for computationally demanding tasks. In Proceedings of the BS 2013: 13th Conference of the International Building Performance Simulation Association, Chambery, France, 26–28 August 2013; pp. 3761–3768. [Google Scholar]
- Georgescu, M.; Mezić, I. Building energy modeling: A systematic approach to zoning and model reduction using Koopman Mode Analysis. Energy Build.
**2015**, 86, 794–802. [Google Scholar] [CrossRef] - Giannakis, G.; Kontes, G.; Korolija, I.; Rovas, D. Simulation-time Reduction Techniques for a Retrofit Planning Tool. In Proceedings of the BS 2017: 15th Conference of the International Building Performance Simulation Association, San Francisco, CA, USA, 7–9 August 2017; pp. 2014–2023. [Google Scholar]
- Wetter, M. Co-simulation of building energy and control systems with the Building Controls Virtual Test Bed. J. Build. Perform. Simul.
**2011**, 4, 185–203. [Google Scholar] [CrossRef] [Green Version] - GPy. GPy: A Gaussian Process Framework in Python. 2012. Available online: http://github.com/SheffieldML/GPy (accessed on 27 November 2018).
- Kraft, D. A software package for sequential quadratic programming. In Forschungsbericht- Deutsche Forschungs- und Versuchsanstalt fur Luft- und Raumfahrt; DFVLR: Braunschweig, Köln, 1988. [Google Scholar]
- Perez, R.E.; Jansen, P.W.; Martins, J.R. pyOpt: A Python-based object-oriented framework for nonlinear constrained optimization. Struct. Multidiscip. Optim.
**2012**, 45, 101–118. [Google Scholar] [CrossRef] - Zhang, T.; Kahn, G.; Levine, S.; Abbeel, P. Learning deep control policies for autonomous aerial vehicles with mpc-guided policy search. arXiv, 2015; arXiv:1509.06791. [Google Scholar]
- Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A.A.; Veness, J.; Bellemare, M.G.; Graves, A.; Riedmiller, M.; Fidjeland, A.K.; Ostrovski, G.; et al. Human-level control through deep reinforcement learning. Nature
**2015**, 518, 529. [Google Scholar] [CrossRef] [PubMed] - Silver, D.; Huang, A.; Maddison, C.J.; Guez, A.; Sifre, L.; Van Den Driessche, G.; Schrittwieser, J.; Antonoglou, I.; Panneershelvam, V.; Lanctot, M.; et al. Mastering the game of Go with deep neural networks and tree search. Nature
**2016**, 529, 484. [Google Scholar] [CrossRef] [PubMed] - Rusu, A.A.; Vecerik, M.; Rothörl, T.; Heess, N.; Pascanu, R.; Hadsell, R. Sim-to-real robot learning from pixels with progressive nets. arXiv, 2016; arXiv:1610.04286. [Google Scholar]
- Tobin, J.; Fong, R.; Ray, A.; Schneider, J.; Zaremba, W.; Abbeel, P. Domain randomization for transferring deep neural networks from simulation to the real world. In Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), San Francisco, CA, USA, 7–9 August 2017; pp. 23–30. [Google Scholar]

**Figure 3.**Overview of simulation-based evaluation of building control strategies using multi-criteria decision analysis methods.

**Figure 5.**The Mafra City Hall Building: (

**a**) front view of the Mafra City Hall Building; (

**b**) simulation of Mafra City Hall Building using DesignBuilder Software [92]; (

**c**) the Mafra City Hall Building cut-off; and (

**d**) the Mafra City Hall Building floor-plan.

**Figure 6.**The experimental results for the controlled/occupied period within one simulation day: (

**a**) the performance of the simple rule-based controller with the setpoint 24.5${\phantom{\rule{0.222222em}{0ex}}}^{\circ}\mathrm{C}$; (

**b**) he performance of the best controller ranked by the MCDA method with the setpoint 25.5${\phantom{\rule{0.222222em}{0ex}}}^{\circ}\mathrm{C}$; and (

**c**) the performance of the optimised controller discovered by the GP_SS method with the setpoint varying per timestep, attaining values between 25.4${\phantom{\rule{0.222222em}{0ex}}}^{\circ}\mathrm{C}$ and 26.1${\phantom{\rule{0.222222em}{0ex}}}^{\circ}\mathrm{C}$.

**Table 1.**The evaluation of different control strategies in the simulation model. Here, the thermal comfort constrained violations are calculated using Equation (11) for the entire occupied period.

Strategy | Setpoint | Energy | Thermal Comfort | MCDA Results | |||||
---|---|---|---|---|---|---|---|---|---|

Index | Value | Consumption | Constraint Violation | ${\mathit{w}}_{1}=10$ | ${\mathit{w}}_{1}=0$ | ${\mathit{w}}_{1}=5$ | |||

(${\phantom{\rule{0.222222em}{0ex}}}^{\circ}\mathbf{C}$) | ($\mathbf{kWh}$) | R21 | R22 | R87 | R88 | ${\mathit{w}}_{2}=0$ | ${\mathit{w}}_{2}=10$ | ${\mathit{w}}_{2}=5$ | |

1 | 21.5 | 163.1 | 0.3 | 0.9 | 0.5 | 11.8 | 12 | 8 | 10 |

2 | 22.0 | 157.9 | 0.1 | 0.5 | 0.1 | 4.9 | 11 | 7 | 9 |

3 | 22.5 | 152.5 | 0.0 | 0.1 | 0.0 | 0.8 | 10 | 6 | 8 |

4 | 23.0 | 146.9 | 0.0 | 0.0 | 0.0 | 0.0 | 9 | 5 | 7 |

5 | 23.5 | 137.3 | 0.0 | 0.0 | 0.0 | 0.0 | 8 | 4 | 6 |

6 | 24.0 | 122.2 | 0.0 | 0.0 | 0.0 | 0.0 | 7 | 9 | 5 |

7 | 24.5 | 107.5 | 0.0 | 0.0 | 0.0 | 0.0 | 6 | 3 | 4 |

8 | 25.0 | 93.5 | 0.0 | 0.0 | 0.0 | 0.0 | 5 | 10 | 3 |

9 | 25.5 | 79.8 | 0.0 | 0.0 | 0.01 | 0.0 | 4 | 2 | 11 |

10 | 26.0 | 67.5 | 0.1 | 0.2 | 1.0 | 0.0 | 3 | 1 | 2 |

11 | 26.5 | 57.7 | 2.1 | 2.7 | 4.9 | 0.0 | 2 | 11 | 1 |

12 | 27.0 | 48.9 | 6.6 | 8.1 | 11.3 | 0.1 | 1 | 12 | 12 |

Energy Consumption RB (kWh) | Energy Consumption MCDA (kWh) | Energy Consumption GP_SS GPSS (kWh) |
---|---|---|

107.5 | 79.8 | 72.1 |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Kontes, G.D.; Giannakis, G.I.; Sánchez, V.; De Agustin-Camacho, P.; Romero-Amorrortu, A.; Panagiotidou, N.; Rovas, D.V.; Steiger, S.; Mutschler, C.; Gruen, G.
Simulation-Based Evaluation and Optimization of Control Strategies in Buildings. *Energies* **2018**, *11*, 3376.
https://doi.org/10.3390/en11123376

**AMA Style**

Kontes GD, Giannakis GI, Sánchez V, De Agustin-Camacho P, Romero-Amorrortu A, Panagiotidou N, Rovas DV, Steiger S, Mutschler C, Gruen G.
Simulation-Based Evaluation and Optimization of Control Strategies in Buildings. *Energies*. 2018; 11(12):3376.
https://doi.org/10.3390/en11123376

**Chicago/Turabian Style**

Kontes, Georgios D., Georgios I. Giannakis, Víctor Sánchez, Pablo De Agustin-Camacho, Ander Romero-Amorrortu, Natalia Panagiotidou, Dimitrios V. Rovas, Simone Steiger, Christopher Mutschler, and Gunnar Gruen.
2018. "Simulation-Based Evaluation and Optimization of Control Strategies in Buildings" *Energies* 11, no. 12: 3376.
https://doi.org/10.3390/en11123376