# Optimal Dispatch of Aggregated HVAC Units for Demand Response: An Industry 4.0 Approach

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

#### 1.1. Context and Motivation

#### 1.2. Related Work

#### 1.3. Contributions

#### 1.4. Structure

## 2. Models and Assumptions

_{A}(t) is assumed to be well approximated by a first-order differential equation, as follows:

_{T}is the equivalent thermal gain of the converter. A model such as this has previously been studied in the context of HVAC control for demand response applications, and time constants of 10–30 min and dead-times between 0–5 min are typical of most buildings [17,18,19,20]. In Equation (1), the sign of K

_{T}distinguishes between a heating (positive) or cooling (negative) application. Neglecting the standby power consumption, the variable electrical power consumption of the HVAC unit E(t) is determined by the state of the control signal and the electrical gain of the converter K

_{E}, such that E(t) = u(t) K

_{E}. Hence, as is well known, the HVAC unit may participate in STOR and DTU DR programs by modulating the electrical power consumption through deliberate under- or over-supply of thermal power, at the possible expense of some thermal comfort of the building occupants. Digitizing the Equation (1) for a time-step of T

_{s}seconds, using k as the (integer) discrete-time index, yields:

_{s}/τ) > 0 and the integer d = 1 + (D/T

_{s}), which if fractional is assumed rounded to the nearest whole integer. For the thermal model, Equation (1), it is suggested that the time step be chosen to be not more than 5% of the time constant τ. The reasoning behind this is that basic systems theory gives the open loop rise time of the model, Equation (1), as 2.2τ, and hence choosing T

_{s}as suggested gives at least 44 time steps in the rise time, which should be adequate for good on/off control without leading to excessively short time steps (which could be problematic from an implementation viewpoint). For example, a 30-min time constant gives step times of not more than 90 s. The digitized model, Equation (2), provides a means in which the temperature evolution may easily be predicted over a future time horizon, enabling predictive optimization to be applied at each time step. The proposed optimization approaches are described in the following section.

## 3. Technical Development

#### 3.1. Individual Unit Dispatch

_{R}(k) represents the temperature setpoint at time step k, and the deadzone of the relay (during which the output is held at its last value) is given by 2Δ. Such a mode of operation is simple to implement; in the following, it is wished to extend the basic relay-based approach to situations in which pre-heating (cooling) can be optimally managed for explicit DR events. For this, a rolling-horizon non-linear MPC employing a finite discrete input set is developed [20]. The optimization problem to be solved at each discrete-time step k is the minimization of the following multi-stage quadratic cost function:

_{R}(k) represents the zone temperature reference (setpoint) at time slot k and T

_{A}(k) represents the ambient temperature at time slot k. The binary decision variable u(k) represent the control input at time slot k and Δ is the difference operator such that Δu(k) = u(k) − u(k − 1). The scalar term λ is used to penalize changes in the control input and can be considered a ‘move suppression’ or regularization term. Integer d ≥ 1 represents the time delay and w(k) ∊ {0, 1} are indicator variables for defining a demand response window, such that DR is active during time slot k if w(k) is ‘1’ and not active otherwise. The integer M represents the length of the prediction horizon, and integers U

_{U}and U

_{L}represent upper and lower bounds on the allowed input activity during a defined DR window. Henceforth in this paper, only an upper bound on control activity (e.g., for handling STOR-style DR events) is considered, with the understanding that adding the lower bound (e.g., for DTU-style DR events) follows directly from the described methods. As with other predictive control problems, an appropriate length for the prediction horizon would be the number of samples required to capture the open-loop setting time of the process [21]. However, with reference to Figure 2, this should be extended somewhat to allow for preparatory control actions to also take place. For the thermal model, Equation (1), basic systems theory gives the settling time as 4.6τ, giving a suggested horizon length M = ⌈(4.6τ + W)/T

_{s}⌉, where W is the length of time required for preparatory control. For example, a 30-min time constant and preparatory window W of one hour with time step of 90 s gives a horizon length M of 132 steps.

_{U}are used in combination to penalize electricity consumption during a specific defined window of the prediction horizon (corresponding to DR events). DR events are enabled for a particular stage by setting the indicator variable w(k) equal to ‘1’. As with all rolling horizon predictive control schemes, once the optimization has been solved for the current time step, only the first control (corresponding to u(k)) is applied. At time step k + 1, the process is repeated with repeated measurement information.

_{R}(k + d) and actual temperature T(k + d) and the second is the previous state of the control input, u(k − d − 1). In this situation, the control defaults to a predictive relay-based controller with deadzone Δ ≈ √λ(k). In the presence of upcoming DR events, however, the effective switching surface can change considerably to provide optimal pre-heating (or cooling). In addition, the solution of the optimization problem contains useful predicted quantities regarding upcoming DR events. In particular, since the future control signals can only be ‘1’ or ‘0’, the predicted electricity consumption of the HVAC unit during any upcoming DR event—denoted as URC(k)—is easily derived from either inspection of the solution vector, or from the left hand side (l.h.s.) of constraint (8), as follows:

_{U}on the right hand side (r.h.s.) of constraint (8) can be used to limit URC(k) and specify a load curtailment/reduced consumption from the HVAC unit. These observations will be exploited in the sequel, during which the DR allocation mechanism will be presented. The optimization problem of Equations (4)–(8) is easily formulated as a mixed-integer quadratic program (MIQP), or alternatively as an approximate mixed-integer linear program (MILP) [16,21]. In order to improve the run-time efficiency of the approach (and enable bounded worst-case execution times), the DP method is chosen. DP (see Bellman, [23]) is a computational method for solving optimal control problems with separable additive performance indices. It is based on the recursive application of Bellman’s ‘Principle of Optimality’ [23,24]:

‘An optimal policy has the property that whatever the initial state and the initial decision are, the remaining decisions must form an optimal policy with regard to the state resulting from the first decision.’

_{k}(x

_{k}) is the cost of entering stage k with state x

_{k}, g

_{k}(x

_{k}) is the cost for entering stage k with state x

_{k}, U

_{k}(x

_{k}) is the set of allowed controls for the input u

_{k}when the state x

_{k}is entered at stage k, and f

_{k}(x

_{k}, u

_{k}) is a function which maps the state x

_{k}onto state x

_{k}

_{+1}when control u

_{k}is applied at step k. In discrete DP (DDP), the state vector is mapped onto a grid of size S and the controls onto a grid of size U. By iterating through the recursion and trying all admissible control values at each admissible set of state values, a vastly reduced search space is explored when compared to a pure brute-force search; at the end of the minimization, a solution grid is obtained and the optimal control is obtained from the position in the grid corresponding to the current state. In the current context, the state variable is the temperature T(k) plus the previous applied control u(k − 1). The admissible controls are the current control u(k), having two possibilities (U = 2) at each stage in the absence of DR, but some possibilities may not be admissible if they invalidate constraint (8) when DR is active. In the approach taken in this paper, constraint (8) is translated into a penalty function inserted into the objective; with an iterative tightening of an applied weight upon constraint violation (a maximum of 10 iterations is employed). The temperature is mapped into a discrete grid of over a suitable working range, e.g., 10–30 °C; the control is already discrete in nature. The transition function f

_{k}(x

_{k}, u

_{k}) is given by the linear Equation (6). In the case that the HVAC dynamics are actually non-linear, then this can easily be captured by an appropriate choice of transition function. During the recursion, the minimal cost function is stored for each admissible state along with the corresponding partial sum of the l.h.s of constraint (8). After the recursion, the value of UDR(k) is readily computable using Equation (9) and the final value of this partial sum. In many situations, temperature sensors either have 10-bit resolution—or can easily be cast or truncated into this range—giving a full state size S = 2

^{11}. As the run-time complexity of the DDP algorithm is given by O(M.U.S), the algorithm runs efficiently even for a relatively large prediction horizon M, which may be needed for best results. During the backwards recursion, only the current and next stage costs and the partial l.h.s. of constraint (8) are actually required, reducing the memory requirement to O(U.S). As sampling time requirements of approximately one minute are sufficient in many instances (due to the comparatively slow thermal dynamics), then the algorithm is clearly suitable for deployment without undue problems in a small/low-cost embedded system.

#### 3.2. Generation of Predicted Baseline and Reduced Consumption

_{U}= ∞. As discussed in the previous section, the control law resulting from the solution of Equations (4)–(8) in the absence of any DR signals is a predictive relay-based controller. Therefore to simplify the process of generating the baseline controls u(k + i|k) and y(k + i|k) (i.e., i-step predictions of the asset input and output, incorporating measured disturbances such as weather forecast), Equation (3) may be employed at each stage with deadzone Δ = λ

^{2}/2. Thus, both quantities can be computed in a straightforward procedure integrating both unit DR controls and unit baseline/reduced consumption predictions in an ITU-based edge device.

#### 3.3. Coordinated Dispatch Scheme

_{j}(k). Similarly, let the predicted baseline electricity consumption at time step k for an upcoming DR event involving HVAC unit j be given as UBL

_{j}(k). This quantity is computable as detailed in the last section. Since for each HVAC unit j and for every time step k it must hold that UBL

_{j}(k) ≥ URC

_{j}(k) (i.e., the predicted unit baseline consumption is not less than the unit DR consumption during a DR event), at step k the explicit unit predicted reduction in load—denoted as UPR

_{j}(k)—for an upcoming DR event is given by: UPR

_{j}(k) = UBL

_{j}(k) − URC

_{j}(k). This quantity is easily computed in each ITU as detailed in the previous section. Then the predicted aggregated explicit reduction in load for an upcoming DR event at time step k—denoted APR(k)—for N participating HVAC units can be computed by the aggregator as:

_{j}(k). Since it has been assumed that the HVAC controls are binary in nature, in a given DR window the available load curtailment is limited between zero and the maximum available from the predicted baseline in discrete steps. As such, assume that each HVAC unit j ∊ N offers an agreed price-schedule for load curtailment X

_{j}, such that a specific load L

_{j}

_{,l}may be reduced for a price p

_{j}

_{,l}by selecting one of l ∊ X

_{j}different discrete price/curtailment options. Only one (or none) of the price/curtailment options can be selected for a given HVAC unit, for any given DR event. The objective for the DR coordinator is then to (i) at the start of a preparatory event, allocate individual HVAC unit load curtailments to meet the aggregate DR requirements; and (ii) should an HVAC unit opt-out or become unavailable—or environmental/pricing conditions otherwise change leading to an invalid initial allocation—reallocate curtailments to best suit the new conditions. The allocation/reallocation problem for N available units at time-step k can be written as a variant of a standard integer knapsack problem, as follows:

_{j}

_{,l}∊ {0, 1} are binary variables that indicate whether load reduction level L

_{j}

_{,l}is active for HVAC unit j. Equation (12) defined the main objective, to minimize DR costs while meeting the target reduction in demand (constraint (13)). Constraint (14) allocates load reduction level to individual unit targets, while constraint (15) ensures that individual unit load reductions are not greater than the individual unit baselines for the upcoming event. Constraint (16) enforces mutual exclusion in the choice of load reductions to individual units based upon the available price/curtailment options for that unit. Note that the presence of this constraint allows that any costs for activating a particular HVAC unit for DR purposes can be added into the corresponding costs for discrete load curtailment. The presence of the constraints also suggests that the problem can be efficiently solved using standard DP techniques, using a slight variant of the standard knapsack DP approach [26]. The time and space complexity of this approach is O(N.D.X), where N is the number of participating units, D is the level of load curtailment requested and X is the largest cardinality of curtailment choices among the participating units [23,24]. Alternatively, efficient Branch-And-Bound techniques are known and can be applied [26]. Should the problem defined by Equations (12)–(17) prove infeasible, this will be due a violation of constraint (13), indicating that there is not enough capacity to achieve the aggregated DR value. In this case, either further units should be brought into the DR scenario, or the aggregator should report that it might not be able to deliver the required target to the market.

## 4. Evaluation

#### 4.1. Computational Study

#### 4.2. Experimental Evaluation

## 5. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Isaksson, A.J.; Harjunkoski, L.; Sand, G. The impact of digitalization on the future of control and operations. Comput. Chem. Eng.
**2018**, 114, 122–129. [Google Scholar] [CrossRef] - Mahdavinejad, M.S.; Rezvan, M.; Barekatain, M.; Adibi, P.; Barnaghi, P.; Sheth, A.P. Machine learning for internet of things data analysis: A survey. Digit. Commun. Netw.
**2018**, 4, 161–175. [Google Scholar] [CrossRef] - Perez-Lombard, L.; Ortiz, J.; Pout, C. A review on buildings energy consumption information. Energy Build.
**2008**, 40, 394–398. [Google Scholar] [CrossRef] - Crosbie, T.; Short, M.; Dawood, M.; Charlesworth, R. Demand response in blocks of buildings: Opportunities and requirements. J. Entrep. Sustain. Issues
**2017**, 4, 271–281. [Google Scholar] [CrossRef] - Arteconi, A.; Polonara, F. Demand side management in refrigeration applications. Int. J. Heat Technol.
**2017**, 35, S58–S63. [Google Scholar] [CrossRef] - Proffitt, E. Profiting from Demand Side Response; Technical Publication; The Major Energy Users’ Council in Association with National Grid: London, UK, 2016. [Google Scholar]
- Shi, Q.; Li, F.; Hu, Q.; Wang, Z. Dynamic demand control for system frequency regulation: Concept review, algorithm comparison, and future vision. Electr. Power Syst. Res.
**2018**, 154, 75–87. [Google Scholar] [CrossRef] - Luo, F.; Ranzi, G.; Dong, Z. Rolling horizon optimization for real-time operation of thermostatically controlled load aggregator. J. Mod. Power Syst. Clean Energy
**2017**, 5, 947–958. [Google Scholar] [CrossRef] - Williams, S.; Short, M.; Crosbie, T. On the use of thermal inertia in building stock to leverage decentralised demand side frequency regulation services. Appl. Therm. Eng.
**2018**, 133, 97–106. [Google Scholar] [CrossRef] - Ramanathan, B.; Vittal, V. A framework for evaluation of advanced direct load control with minimum disruption. IEEE Trans. Power Syst.
**2008**, 23, 1681–1688. [Google Scholar] [CrossRef] - Hao, H.; Sanandaji, B.; Poolla, K.; Vincent, T.L. Aggregate flexibility of thermostatically controlled loads. IEEE Trans. Power Syst.
**2015**, 30, 189–198. [Google Scholar] [CrossRef] - Vrettos, E.; Anderson, G. Combined load frequency control and active distribution network management with thermostatically controlled loads. In Proceedings of the IEEE International Conference on Smart Grid Communications (SmartGridComm), Vancouver, BC, Canada, 21–24 October 2013. [Google Scholar]
- Mathieu, J.L.; Koch, S.; Callaway, D.S. State estimation and control of electric loads to manage real-time energy imbalance. IEEE Trans. Power Syst.
**2013**, 28, 430–440. [Google Scholar] [CrossRef] - Wang, H.; Meng, K.; Luo, F.; Dong, Z.Y.; Verbič, G.; Xu, Z.; Wong, K.P. Demand response through smart home energy management using thermal inertia. In Proceedings of the Australia Universities Power Engineering Conference (AUPEC 2013), Hobart, TAS, Australia, 29 September–3 October 2013. [Google Scholar]
- Luo, F.; Zhao, J.; Wang, H.; Wang, H.; Tong, X.; Chen, Y.; Dong, Z.Y. Direct load control by distributed imperialist competitive algorithm. J. Mod. Power Syst. Clean Energy
**2014**, 2, 385–395. [Google Scholar] [CrossRef] - Drgona, J.; Klauco, M.; Kvasnica, M. MPC-based reference governors for thermostatically controlled residential buildings. In Proceedings of the IEEE 54th Annual Conference on Decision and Control (CDC 2015), Osaka, Japan, 15–18 December 2015. [Google Scholar]
- Callaway, D.S. Tapping the energy storage potential in electric loads to deliver load following and regulation, with application to wind energy. Energy Convers. Manag.
**2009**, 50, 1389–1400. [Google Scholar] [CrossRef] - Zhou, L.; Li, Y.; Wang, W.; Hu, X. Provision of supplementary load frequency control via aggregation of air conditioning loads. Energies
**2015**, 8, 14098–14117. [Google Scholar] [CrossRef] - Afram, A.; Janabi-Sharifi, A. Review of modelling methods for HVAC systems. Appl. Therm. Eng.
**2014**, 67, 507–519. [Google Scholar] [CrossRef] - Mathews, E.; Richards, P.; Lombard, C. A first-order thermal model for building design. Energy Build.
**1994**, 21, 133–145. [Google Scholar] [CrossRef] - Camacho, E.F.; Bordons, C. Model Predictive Control, 2nd ed.; Springer: Berlin, Germany, 2004. [Google Scholar]
- Djongyang, N.; Tchinda, R.; Njomo, D. Thermal comfort: A review paper. Renew. Sustain. Energy Rev.
**2010**, 14, 2626–2640. [Google Scholar] [CrossRef] - Bellman, R.E. Dynamic Programming; Princeton University Press: Princeton, NJ, USA, 1957. [Google Scholar]
- Bertsekas, D.P. Dynamic Programming and Optimal Control; Athena Scientific: Belmont, MA, USA, 2005. [Google Scholar]
- Chelmis, C.; Aman, S.; Saeed, M.R.; Frincu, M.; Prasanna, V.K. Estimating reduced consumption for dynamic demand response. In Proceedings of the AAAI Workshop on Computational Sustainability, Austin, TX, USA, 1 April 2015. [Google Scholar]
- Martello, S.; Toth, P. Knapsack Problems: Algorithms and Computer Implementations; John Wiley and Sons: Hoboken, NJ, USA, 1990. [Google Scholar]
- Short, M.; Crosbie, T.; Dawood, M.; Dawood, N. Load forecasting and dispatch optimization for decentralized co-generation plant with dual energy storage. Appl. Energy
**2017**, 186, 304–320. [Google Scholar] [CrossRef] - Ogwumike, C.; Short, M.; Abugchem, F. Heuristic optimization of consumer electricity costs using a generic cost model. Energies
**2016**, 9, 6. [Google Scholar] [CrossRef][Green Version] - Short, M. Optimal HVAC dispatch for demand response: A dynamic programming approach. In Proceedings of the International Conference on Innovative Applied Energy (IAPE’19), Oxford, UK, 15 March 2019. [Google Scholar]

**Figure 12.**Recorded temperatures in °C (red trace, DR case; black trace, baseline case; green trace, ambient temperature).

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Short, M.; Rodriguez, S.; Charlesworth, R.; Crosbie, T.; Dawood, N. Optimal Dispatch of Aggregated HVAC Units for Demand Response: An Industry 4.0 Approach. *Energies* **2019**, *12*, 4320.
https://doi.org/10.3390/en12224320

**AMA Style**

Short M, Rodriguez S, Charlesworth R, Crosbie T, Dawood N. Optimal Dispatch of Aggregated HVAC Units for Demand Response: An Industry 4.0 Approach. *Energies*. 2019; 12(22):4320.
https://doi.org/10.3390/en12224320

**Chicago/Turabian Style**

Short, Michael, Sergio Rodriguez, Richard Charlesworth, Tracey Crosbie, and Nashwan Dawood. 2019. "Optimal Dispatch of Aggregated HVAC Units for Demand Response: An Industry 4.0 Approach" *Energies* 12, no. 22: 4320.
https://doi.org/10.3390/en12224320