Centralized and Decentralized Optimal Control of Variable Speed Heat Pumps

Abstract

Utility service providers are often challenged with the synchronization of thermostatically controlled loads. Load synchronization, as a result of naturally occurring and demand-response events, has the potential to damage power distribution equipment. Because thermostatically controlled loads constitute most of the power consumed by the grid at any given time, the proper control of such devices can lead to significant energy savings and improved grid stability. The contribution of this paper is the development of an optimal control algorithm for commonly used variable speed heat pumps. By means of selective peer-to-peer communication, our control architecture allows for the regulation of home temperatures while simultaneously minimizing aggregate power consumption, and aggregate load volatility. An optimal centralized controller is also explored and compared against its decentralized counterpart.

1. Introduction

The goal of a 100% renewable electric supply system presents significant challenges to the organizations responsible for maintaining the reliability and resilience of electric grids. Although grid-level battery storage is often touted as the solution to integrating high penetrations of variable generating resources [1,2] (e.g., solar PV and wind generators), there is a growing body of research pointing to the potential for flexibility and control of demand to play a significant role in grid operations going forward. In this regard, thermostatically controlled loads (TCLs) are some of the most sensible areas to explore because they make up a significant portion of demand on any given electric grid and there is a great deal of flexibility in when they actually draw power.

To realize the true potential of TCLs as a resource to aid in grid operations, it is widely understood that numerous loads must be aggregated and controlled in a coordinated fashion. Many researchers have proposed methods of centralized control of aggregated TCL’s and the problem of synchronization often arises [3]. The most common example of synchronization occurs during load-shedding demand-response events. Many utilities have programs which homeowners agree to allow the grid operators to occasionally turn off their air conditioning compressor in return for a rebate or reduction in rates. When the grid operator anticipates that they will have a difficult time meeting load requests, then many compressors can be shut off, thus shedding that load. The problem arises when the demand-response event is released. At this point, a large portion of the compressors will cycle “on” because they have no doubt risen in temperature during the demand-response event. This often results in an immediate peak that is higher than the peak they were attempting to avoid. Subsequently, a period of oscillations occurs, during which time the aggregate load experiences large oscillations.

This paper explores innovative approaches to controlling an aggregation of TCLs by applying a combination of optimum control theory and localized communication between individual TCL’s which serve to prevent inadvertent synchronization while minimizing peak demand.

The work presented in this paper builds on the body of research dedicated to smart grid systems. A smart grid system, defined by its ability to efficiently balance various energy streams, has been shown to increase both stability and efficiency of electric grids. Due to the vastness of smart grid applications, its often helpful to define a smart grid by its constitutive subcategories. These categories include, but are not limited to, demand-response events, agent-based modeling, dynamic price control, and  thermostatically controlled loads (TCLs).

Much of the body of literature in demand response and TCLs stems from Malhamé and Chong’s seminal paper [4]. This paper introduced a stochastic control framework designed to aggregate a large population of homogeneous TCLs (electric space heaters) that are modeled with a pair of coupled Focker Plank equations (CFPEs). Two decades later, Callaway [5] built upon Malhamé and Chong’s work not only by deriving an exact solution to the CFPEs, but also devised a method to aggregate TCLs about the variably produced renewable energy sources.

As the field of smart grid systems matured, so too followed unique control methods designed to regulate aggregate power consumption. As it pertains to TCLs, two primary classifications exist: centralized and decentralized control. Various centralized control approaches are used to aggregate TCLs. For instance, various state-bin transition techniques are proposed in [6,7,8]. These binning techniques stochastically characterize the flow of TCLs between their respective off and on states. Through the systematic transitioning of states, an aggregate power reference signal is therefore tracked. Among [7,8], including  [9,10], a model predictive control framework is used to optimally schedule a population of control actions. Machine learning techniques have also been adopted into the control of smart grid systems. In [11,12,13], a  reinforcement learning control framework is used to learn the complex action space of a population of TCLs. Other notable approaches to the aggregation of TCLs are the priority-stack-based controllers presented in  [1,14], and the unique geometric approach proposed in [9]. Lastly, several publications have been dedicated to the reduction aggregate power consumption via dynamic price control [15,16].

Another classifier for TCLs, under the umbrella of smart grid systems, is decentralized control. In terms of large-scale aggregation of TCLs, a  centralized command structure is inherently burdened with the computation and communication complexities associated with its operation. Both scalability and cyber-security are often cited as the primary concerns of a centralized network topology when governing the actions of TCLs  [17,18]. Of the literature results in decentralized control, such as that of the stochastic framework proposed in  [19], methods of distributing the computational complexity to its participating patrons is explored. In both  [20,21], decentralized controllers have been shown to resist various cyber-attacks and communication failures caused by network dropout.

The rest of this paper is organized as follows. In Section 2 a second-order equivalent thermal parameter model of a TCL is presented along with its state space representation. A parameter estimation method is presented thereafter using the recursive least squares algorithm. Lastly, justifications for the network architecture and demand response event is provided. In Section 3 an optimization program is provided for both the decentralized and centralized control frameworks. Simulation results are presented Section 4 followed by closing remarks made in Section 5.

2. Background

In the United States, including many other modernized countries, the majority of generated electricity is therefore consumed by thermostatically controlled loads (TCLs). These TCLs, defined by their ability to store thermal energy (and are controlled as such), loosely resemble that of a leaky battery. For instance, water-heaters, a type of TCL, heats inlet water in an insulated storage tank to within a prescribed temperature range where it awaits its use. Including water-heaters, other prominent TCLs, defined by their high energy consumption, are HVAC systems and refrigerator units. It should be noted these TCL devices, including many others, operate on a similar premise. Typically, TCLs are governed by simplistic toggle condition, like that of an on/off controller, also referred to as a Bang-Bang control. Because thermal energy is stored within a medium, there exists a level flexibility as to when such energy is consumed/replenished. Through the proper control of such devices, as will be demonstrated in this work, certain beneficial aggregate characteristics may be achieved for instance the reduction in aggregate power consumption and its associated volatility. In the following sections a mathematical model for a particular TCL, that being a variable speed heat pump (VSHP) is developed. This mathematical model provides a causal relationship between the control action of a population of VSHPs and the aggregate effects of power consumption. Although this research focuses on a particular TCL, its application may be expanded other similarly governed devices.

2.1. Dynamics

This paper uses the well-established second-order equivalent thermal parameter (EPT) model to describe the temperature dynamics of a residential home. By means of the thermal circuit shown in Figure 1, two coupled first order differential Equations are formed.

$$C_A{\dot{T}}_A=Q_A-\frac{1}{R_1}\left(T_A-T_o\right)-\frac{1}{R_2}\left(T_A-T_M\right),$$

(1)

and,

$$C_M{\dot{T}}_M=Q_M+\frac{1}{R_2}\left(T_A-T_M\right),$$

(2)

where $T_A$ and $T_M$ denote the interior air and lumped mass temperatures of a residential home. Adjacent to the residential control volume is the surrounding outside air temperature $T_o$. With regard to control theory, both the outside temperature $T_o$ and the internal heat generating elements, denoted $Q_M$, represent disturbances to this thermal system. Rejection of such disturbances is accomplished via the home’s HVAC system, denoted $Q_A$. In particular, $Q_A$ represents heat being removed from the interior air. The elegance of this ETP model is that it is fully defined by four measurable parameters, that being the thermal capacities $C_A$, $C_M$ and the conductive properties $R_1$, $R_2$. As will be further discussed in Section 2.4, a parameter estimation model will be presented, which provides systematic method to estimate these thermal parameters.

Through the manipulation of Equations (1) and (2) a second-order differential equation is formed in terms of $T_A$ and its derivatives ${\dot{T}}_A$ and ${\ddot{T}}_A$,

$$\begin{array}{c}{C}_M{C}_A{R}_2{\ddot{T}}_A+\left(C_M\left(\frac{R_2}{R_1}+1\right)+C_A\right){\dot{T}}_A+\frac{1}{R_1}{T}_A\\ =\frac{C_M{R}_2}{R_1}{\dot{T}}_o+\frac{1}{R_1}{T}_o+C_M{R}_2\eta \dot{m}+\eta m.\end{array}$$

(3)

Within Equation (3), the terms $Q_A$ and ${\dot{Q}}_A$ are replaced with $\eta m$ and $\eta \dot{m}$ respectively. The negative constant term $\eta <0$ represents the heat removal capacity of the homes HVAC system and m, a controllable parameter, scales $\eta$ according to the governing control algorithm. $Q_M$, similar to that of $Q_A$, represents alternative heat sources/sinks. In a typical residential setting, $Q_M$ includes, but is not limited to, heat generated by home appliances, solar radiation, and the dissipation of heat between the home’s lumped mass and ground. Based on the minimal effects of $Q_M$, especially with regard to $Q_A:=\eta m$, its inclusion will be neglected in this study.

Equation (3) is further abbreviated by replacing each constant term with elements of a parameter vector $\theta \in {\mathbb{R}}^5$.

$${\theta }_1{\ddot{T}}_A+{\theta }_2{\dot{T}}_A+{\theta }_3{T}_A={\theta }_4{\dot{T}}_o+{\theta }_3{T}_o+{\theta }_5\eta \dot{m}+\eta m$$

(4)

For the sake of clarity, a VSHP regulates the indoor air temperature, $T_A$, by adjusting its cooling capacity, m, between being fully on or completely off. This range of values may be mathematically defined as the continuous set $m\in [0,1]$. A goal of this research is the development of a controller which provides certain beneficial characteristics to an electric utility service provider. These characteristics are the minimization of aggregate power consumption, defined as

$$P_{agg}=\sum _{p=1}^N{\eta }_p{m}_p,$$

(5)

and the instantaneous rate of change of aggregate power consumption, ${\dot{P}}_{agg}$. Along with the aggregate power design constraints, this controller must also maintain indoor air temperatures at, or near, its associated set-point temperature $T_{sp}$. To satisfy the combination of these three control requirements, a novel approach is taken. Instead of directly controlling the cooling capacity of a VSHP by means of its control variable m, this study opts to additionally control its derivative $\dot{m}$. To distinguish $\dot{m}$ as a controllable parameter, moving forward we redefine it as $\sigma$. A faithful state space representation of Equation (4), incorporating this novel control approach, is defined as follows,

$$\begin{array}{c}\dot{\mathbf{x}}=A\mathbf{x}+B\left(\eta (m+{\theta }_5\sigma )+{\theta }_4{\dot{T}}_o+{\theta }_3{T}_o\right)\end{array}$$

(6)

where $\mathbf{x}={\left[\begin{array}{cc}{T}_A& {\dot{T}}_A\end{array}\right]}^{\top }$ and,

$$\begin{array}{cc}\hfill A& =\left[\begin{array}{cc}0& 1\\ -\frac{{\theta }_3}{{\theta }_1}& -\frac{{\theta }_2}{{\theta }_1}\end{array}\right],B=\left[\begin{array}{c}0\\ \frac{1}{{\theta }_1}\end{array}\right].\hfill \end{array}$$

Through the application of backward Euler method, Equation (6) is therefore rewritten in a discrete format as,

$$\begin{array}{c}{\mathbf{x}}^{k+1}=(I+\Delta tA){\mathbf{x}}^k+\Delta tB\left(\eta \left(m^k+{\theta }_5{\sigma }^k\right)+{\theta }_4{\dot{T}}_o^k+{\theta }_3{T}_o^k\right),\\ {\sigma }^k=\frac{m^{k+1}-m^k}{\Delta t},\end{array}$$

(7)

where $\Delta t$ and k denote the step-length and simulation time-step respectively.

2.2. Decentralized Network Communication

Much of the proposed literature in smart grid systems, particularly TCLs, are constructed using a centralized framework. In such a framework, a utility service provider determines the control action of a population of TCLs, typically by means of tracking an aggregate reference signal. This methodology, including other centralized controllers, have shown great performance benefits. However, there are several glaring drawbacks to a centralized framework. Some of the more prominent challenges include its scalability, vulnerability to cyber-attack, and inherent lack of consumer privacy. To address these challenges, a decentralized framework is introduced in this paper. A decentralized framework relies on the autonomy of participating agent/device to calculate their own control action. As will be demonstrated, the performance of a decentralized controller is further improved through the localized communication between neighboring TCLs.

To address the privacy concerns of participating end-users, communication between VSHPs is limited to what might be ascertained if one were to open their window and listen to when a neighbor’s compressor is cycled off/on. For this reason, each VSHP can only communicate with four other neighboring VSHPs. As a result of this communication, a VSHP’s control action m is partially influenced by the previous control actions of its neighbor set ${\mathcal{N}}_p$, where ${\mathcal{N}}_p$ is the set of all VSHPs connected to the $p^{th}$ agent. The entire population set is similarly defined as $\mathcal{N}=\{1,\cdots ,N\}$, which is the set all homes in the given population, ${\mathcal{N}}_p\subseteq \mathcal{N}$.

Conventionally, communication is modeled via the elements of an adjacency matrix, $\mathcal{A}\in {\{0,1\}}^{N\times N}$. The ${(i,j)}^{\mathrm{th}}$ entry of $\mathcal{A}$ is nonzero if node i can communicate with node j. Based on the communication constraints proposed above, a random regular graph with a connection degree $N_{cd}=4$, such as the one shown in Figure 2, is used to simulate a population of VSHPs.

2.3. Demand Response

In addition to the implementation of structured communication, this study also observes the effects of a demand-response event on the proposed controller(s) defined Section 3. A demand-response event, also referred to as a conservation event, is a circumstance where power consumption is systematically controlled by the electric utility to maintain both power generating equipment and the means of distribution within safe operating conditions. Many types of demand-response events exist ranging from emergency protocol to adaptive price control and other related ancillary services. Correspondingly, this study implements the following demand-response event in an effort to test the governing controller with strenuous and unpredictable conditions. Over the time-span $\Delta t_{DR}$, centered about the warmest outside temperature peak $T_{o,max}$, all VSHPs are prevented from conditioning their associated home. This demand-response event, aside from long-term power outages, represents a worst-case scenario regarding aggregate power consumption. During this demand-response event indoor air temperatures will no doubt rise. Upon reinstatement of power, all VSHPs will independently begin cooling their respective homes resulting in a characteristically large spikes in aggregate power consumption.

To prime the readers of the challenges associated with this demand-response event, a population of VSHP’s are simulated, whose control action, m, is governed by a typical proportional-integral-derivative (PID) controller. This PID controller represents what would likely be used to regulate a population of independently operated VSHP devices. With the system diagram of Figure 3, in conjunction with the controller defined by Equation (8), a sample population of VSHPs are simulated. The results of this simulation are graphically presented as the subplots of Figure 4. As previously mentioned, a spike in aggregate power consumption is observed. This spike in aggregate power consumption is further reduced at the expense of a larger temperature deviation from the customers set-point preference. The gains of the PID controller in Equation (8) are judicially selected to balance each home’s indoor temperature and the aggregate power of the population. This means that we performed an empirical optimization of the PID gains such that the temperature and power response of the system is the best that we could achieve. This study aims to find a more optimal solution which minimizes spikes in aggregate power consumption while simultaneously maintaining home temperatures at or near their associated temperature set-point, $T_{sp}$.

$$u\left(t\right)=K_{P}e\left(t\right)+K_I{\int }_0^{t}e\left(\tau \right)d\tau +K_D\frac{d}{dt}e\left(t\right)$$

(8)

As depicted in Figure 3, a control action $u\left(t\right)$, is determined based on the closed loop error signal $e\left(t\right)$ prior to being constrained via the saturation limits of the maximum cooling capacity, $m\in [0,1]$.

2.4. Parameter Estimation

With regard to a physical setting, the thermal parameters $C_A$, $C_M$, $R_1$, and $R_2$, which define the ETP model, are likely to be measured with some statistical error. Given the proposed ETP model is capable of accurately predicting the dynamics of a residential home, these measurement errors will, by deduction, result in suboptimal controller performance. To account for these initial measurement errors, including other errors that might arise, a recursive least squares (RLS) algorithm is employed to systematically update the thermal parameter vector $\theta \in {\mathbb{R}}^5$. Using similar notation presented in  [22], an RLS algorithm is constructed by first redefining Equation (4) as,

$$y_k={\mathit{\phi }}_k^{\top }{\mathit{\theta }}_0,$$

(9)

where the control input, $y_k$, and regressor, ${\phi }_k$, terms are respectively defined as,

$$\begin{array}{c}{y}_k=\eta m^k,\\ {\mathit{\phi }}_k^{\top }=\left[\begin{array}{ccccc}{\ddot{T}}_A^k& {\dot{T}}_A^k& (T_A^k-T_o^k)& -{\dot{T}}_o^k& -\eta {\sigma }^k\end{array}\right].\end{array}$$

(10)

Whether physically measured, or in the case of this simulation, generated about a known statistical distribution, all elements within the parameter vector ${\mathit{\theta }}_0$ must be known prior to running the RLS algorithm.

$$\begin{array}{c}{\mathit{\theta }}_0={\left[\begin{array}{ccccc}{\theta }_1^0& {\theta }_2^0& {\theta }_3^0& {\theta }_4^0& {\theta }_5^0\end{array}\right]}^{\top }\in {\mathbb{R}}^5,\end{array}$$

(11)

Before the RLS algorithm can predict thermal parameters with sufficient accuracy, both control input and regressor terms are to be collected over the set of initial time-steps $k\in \{1,\cdots ,k_s\}$ to form the following control input and regressor matrices,

$$\begin{array}{c}{Y}_{k_s}=\left[\begin{array}{c}{y}_1\\ ⋮\\ y_{k_s}\end{array}\right],{\mathsf{\Phi }}_{k_s}=\left[\begin{array}{c}{\phi }_1^{\top }\\ ⋮\\ {\phi }_{k_s}^{\top }\end{array}\right].\end{array}$$

(12)

The last time-step, $k_s\in \mathbb{N}$, is chosen such that ${\mathsf{\Phi }}_{k_s}^{\top }{\mathsf{\Phi }}_{k_s}$ is non-singular. Given the starting value ${\theta }_0$ and the adequately populated control input and regressor matrices, which are used to form the inverse covariance matrix, $P_{k_s}={\left({\mathsf{\Phi }}_{k_s}^{\top }{\mathsf{\Phi }}_{k_s}\right)}^{-1}$, the RLS algorithm has the following sequence of events,

Within Algorithm 1, $\lambda \in \mathbb{R}$ denotes the exponential forgetting factor. Unlike the sustained plant dynamics of this study, the thermal properties of a residential home are expected to change over time. The inclusion of this forgetting factor allows for long-term adaptation to the present dynamics. It should be noted, as $\lambda \to 1$ the RLS algorithm with exponential forgetting becomes the vanilla RLS algorithm. Like many other feedback systems, rejection of system level noise is an important step to achieve stable and robust performance. A low-pass filter is applied, via software, to the newly estimated thermal parameters therefore damping system level noise.

Algorithm 1. RLS with Exponential Forgetting.

1: Initialize $P_{k_s}={\left({\mathsf{\Phi }}_{k_s}^{\top }{\mathsf{\Phi }}_{k_s}\right)}^{-1}$ 2: while$k_s<k\le K$do 3:     $S_k=P_{k-1}{\phi }_k{(\lambda +{\phi }_k^{\top }{P}_{k-1}{\phi }_k)}^{-1}$ 4:     $P_k=(I-S_k{\phi }_k^{\top })P_{k-1}/\lambda$ 5:     ${\theta }_k={\theta }_{k-1}+S_k(y_k-{\phi }_k^{\top }{\theta }_{k-1})$. 6: end while

3. Controller

The culmination of this research is the development of a decentralized controller, and to a lesser extent, centralized controller for a population of VSHPs. In brief, two frameworks are presented in Section 3.1 and Section 3.2, which optimally schedule the control actions of a population of VSHPs by means of model predictive control (MPC). As shown in Figure 5, an MPC solves for the control action(s) $m^i\forall i\in \underline{\mathrm{H}}$ such that the model dynamics are satisfied and the objective penalty is minimized. Next, the first control action $m^{i_1:=k}$ is therefore sent to the VSHP device. Assuming the system’s response is observable, the measured/predicted states are returned to the controller serving as the initial conditions for the following iteration. This sequence is then repeated until a termination condition is met.

Two indices are used in Section 3.1 and Section 3.2 to denote time, that being k and $i_k$. Of the two indices, k denotes the current time-step of the simulation, while $i_k$ represents the step(s) of the controllers predicted horizon which symbolically begins at the $k^{th}$ index.

3.1. Decentralized Controller (Dc)

In this section, a decentralized controller is developed, whose control action, m, is partially influenced by control actions of its neighbor set ${\mathcal{N}}_p$. The controller depicted in Figure 5 is mathematically defined as the following quadratic optimization program,

$$\begin{array}{cc}\underset{\mathbf{x},m,\sigma }{min}\hfill & J\left(k,{\mathcal{X}}_o\right)={\displaystyle \sum _{i_k\in \underline{\mathrm{H}}}}{\ell }_{DC}\left(i_k,{\mathbf{x}}^{i_k},m^{i_k},{\sigma }^{i_k}\right)\hfill \\ \mathrm{s}.\mathrm{t}.\hfill & {\mathbf{x}}^{i_k=1}={\mathcal{X}}_o,\hfill \\ \hfill & m^{i_k\in [0,1]},\hfill \\ \hfill & m^{i_k+1}=m^{i_k}+\Delta t{\sigma }^{i_k},\hfill \\ \hfill & {\mathbf{x}}^{i_k+1}=f_{DC}\left(i_k,{\mathbf{x}}^{i_k,}{m}^{i_k,}{\sigma }^{i_k},{\theta }^k\right),\hfill \\ \hfill & \forall i_k\in \underline{\mathrm{H}}.\hfill \end{array}$$

(13)

This optimization program minimizes the objective function, $J(·)$, with respect to the state, $\mathbf{x}$, and control, m and $\sigma$, decision variables. Through the manipulation of these decision variables, a horizon of control actions are calculated such that the model dynamics, initial conditions, and saturation limits are upheld. To ensure continuity between time-steps, an equality constraint is placed between the initial state values, ${\mathbf{x}}^{i_k=1}$, and the measured/predicted response of the plant, ${\mathcal{X}}_o$. Likewise, m is constrained between zero and one, therefore preserving its associated VSHP devices within safe operating conditions. Lastly, the model dynamics of Equation (7), is expressed as a set of linear equality constraints, where the function $f_{CD}(·)$ is defined as,

$$\begin{array}{c}{f}_{DC}\left(i_k,{\mathbf{x}}^{i_k},m^{i_k},{\sigma }^{i_k},{\theta }^k\right)=\left(I+\Delta tA\right){\mathbf{x}}^{i_k}\\ +\Delta tB\left(\eta \left(m^{i_k}+{\theta }_5{\sigma }^{i_k}\right)+{\theta }_4{\dot{T}}_o^{i_k}+{\theta }_3{T}_o^{i_k}\right).\end{array}$$

(14)

As shown in Equation (13), the objective function, $J(·)$, is the summation of each time-step’s cumulative objective penalty, ${\ell }_{DC}(·)$, defined as,

$$\begin{array}{c}{\ell }_{DC}\left(i_k,{\mathbf{x}}^{i_k},m^{i_k},{\sigma }^{i_k}\right)=\alpha {\left(m^{i_k}-\frac{{\sum }_{j\in {\mathcal{N}}_p}{m}_j^{i_k}}{|{\mathcal{N}}_p|}\right)}^2\\ -\beta \eta m^{i_k}+\gamma {\tilde{T}}^{i_k}+\zeta {\left({\dot{T}}_A^{i_k}\right)}^2+\tau {\left({\sigma }^{i_k}\right)}^2.\end{array}$$

(15)

Each penalty term of Equation (15) is dedicated to a certain attribute of the desired response. Based the preferences of an electric utility service provider and its end-users alike, this controller must maintain indoor air temperatures while simultaneously minimizing the aggregate power consumption signal and its associated volatility.

The first objective penalty term,

$$\alpha {\left(m^{i_k}-\frac{{\sum }_{j\in {\mathcal{N}}_p}{m}_j^{i_k}}{|{\mathcal{N}}_p|}\right)}^2$$

(16)

provides a means for optimizer to group aggregate effects via peer-to-peer (P2P) communication. This is done so by minimizing the difference in control action, m, between the $p^{th}$ agent and its neighbor set ${\mathcal{N}}_p$. In an effort to reduce aggregate power consumption, defined by Equation (5), individual power consumption is minimized with the penalty term $-\beta \eta m^{i_k}$. In addition, a home’s indoor air temperature, $T_A$, is maintained at or near its set-point temperature, $T_{sp}$, via the soft constraint $\gamma {\tilde{T}}^{i_k}$. As depicted in Figure 6, the double hinge function, ${\tilde{T}}^{i_k}$, is mathematically expressed as,

$${\tilde{T}}^{i_k}=max\left({\delta }^{-}-T_A^{i_k},0,T_A^{i_k}-{\delta }^{+}\right)$$

If the indoor air temperature, $T_A$, deviates above ${\delta }^{+}$ or below ${\delta }^{-}$, a proportional cost will be accrued. Similarly, the minimization of the penalty term, $\zeta {\left({\dot{T}}_A^{i_k}\right)}^2$, reduces the rate at which temperature changes in the resulting simulation. Lastly, to ensure the ramp-rate of aggregate power consumption is maintained, the penalty term, $\tau {\left({\sigma }^{i_k}\right)}^2$, is minimized. This term results in a smoothing effect in aggregate power consumption.

Each of the five objective penalty terms are accompanied by a real numbered objective constant, that being $\alpha$, $\beta$, $\gamma$, $\zeta$, and $\tau$. Much like knobs on a dial, these objective constants scale their respective objective penalty term. The qualitative performance of the controller is determined by the relative scale of each objective parameter regarding one another. By deduction, said performance may be tuned in accordance to the attributes described above.

3.2. Centralized Controller (Cc)

Much like the decentralized controller defined above, the preceding centralized controller uses a similarly structured optimization program to determine the control actions of a population of VSHPs. However, unlike the decentralized controller, this centralized controller has omniscient knowledge of the states and control actions of its population. Furthermore, during simulation, this optimization program simultaneously solves for the control actions for an entire population set. In this formulation, the notion of network communication is less meaningful as all information is distributed to and from the electric utility service provider, much like a star graph. This quadratic optimization program has the form,

$$\begin{array}{cc}\hfill \underset{\mathbf{X},\mathbf{M},\mathsf{œ}}{min}& J\left(k,{\mathcal{X}}_0\right)={\displaystyle \sum _{i_k\in \underline{\mathrm{H}}}}{\ell }_{\mathrm{CC}}\left(i_k,{\mathbf{X}}^{i_k},{\mathbf{M}}^{i_k},{\mathsf{œ}}^{i_k}\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\mathbf{x}}_p^{i_k=1}={\mathcal{X}}_{o,p},\hfill \\ & m_p^{i_k}\in [0,1],\hfill \\ & m_p^{i_k+1}=m_p^{i_k}+\Delta t{\sigma }_p^{i_k},\hfill \\ & {\mathbf{x}}_p^{i_k+1}=f_{CC}\left(p,i_k,{\mathbf{x}}_p^{i_k},m_p^{i_k},{\sigma }_p^{i_k},{\theta }_p^k\right),\hfill \\ & \forall p\in \mathcal{N},\forall i_k\in \underline{\mathrm{H}}\hfill \end{array}$$

(17)

where ${\mathbf{X}}^{i_k}={[{\mathbf{x}}_1^{i_k},\dots ,{\mathbf{x}}_N^{i_k}]}^{\top }$ and ${\mathbf{M}}^{i_k}={[m_1^{i_k},\dots ,m_N^{i_k}]}^{\top }$ are defined as the collection of all state and control decision variables at the $i_k^{th}$ time-step. As may become apparent, the number of decision variables is now directly proportional to the size of the population being simulated. Due to this increased number of decision variables, the associated complexity of the controller so to rises.

The dynamic model is simulated with a set of equality constraints between the incremented state values and the function $f_{CC}(·)$. Similar to $f_{DC}(·)$ of Equation (14), $f_{CC}(·)$ is now an expressed function of the $p^{th}$ home, for brevity, its redefinition is omitted. Furthermore, the objective penalty function, ${\ell }_{CC}(·)$, which is summed over each time-step $i_k$, has the form,

$$\begin{array}{c}{\ell }_{CC}\left(i_k,{\mathbf{X}}^{i_k},{\mathbf{M}}^{i_k},{\mathsf{œ}}^{i_k}\right)=\alpha \left|{\displaystyle \sum _{p\in \mathcal{N}}}\left(m_p^{i_k+1}+m_p^{i_k}\right)\right|\\ +{\displaystyle \sum _{p\in \mathcal{N}}}\left(-\beta {\eta }_p{m}_p^{i_k}+\gamma {\tilde{T}}_p^{i_k}+\zeta {\left({\dot{T}}_{A,p}^{i_k}\right)}^2+\tau {\left({\sigma }_p^{i_k}\right)}^2\right).\end{array}$$

(18)

The last four objective penalty terms of Equation (18), are similar to that of Equation (15). However, the penalty term,

$$\alpha \left|\sum _{p\in \mathcal{N}}\left(m_p^{i_k+1}+m_p^{i_k}\right)\right|$$

(19)

is used to minimize the difference in aggregate control effort between adjacent time-step. Observably, this penalty term smooths aggregate power consumption among VSHP devices. Because Equations (16) and (19) focus on the grouping of control actions, they both are scaled the objective constant $\alpha$. The other objective constants $\beta$, $\gamma$, $\zeta$, and $\tau$ scale their associated objective penalty term.

4. Case Study

HVAC units are typically sold in half ton increments, where one ton of cooling is defined as the amount of heat required to freeze/melt 2000 pounds of water in a 24-h period. Prior to installing a VSHP, the discrete tonnage is chosen according to thermodynamic properties of the space required to condition. Similar to how a residential HVAC system is chosen, in this study a VSHP’s cooling capacity, $\eta \le 0$, is determined based on the time required to cool that home’s indoor air temperature, $T_A$, from the upper dead-band, ${\delta }^{+}$, to the lower dead-band, ${\delta }^{-}$. To assure population heterogeneity, the thermal parameters $C_A$, $C_M$, $R_1$, and $R_2$ are generated about a known statistical distribution. This Gaussian distribution is defined by the mean and standard deviation values listed in

Table A1. Gaussian distribution of thermal parameters.

Thermal Parameter	Mean	Std.
$C_A\left[\frac{Btu}{{}^{\circ }F}\right]$	1080	54
$C_M\left[\frac{Btu}{{}^{\circ }F}\right]$	4280	214
$\frac{1}{R_1}\left[\frac{Btu}{{}^{\circ }F·hr}\right]$	520	26
$\frac{1}{R_2}\left[\frac{Btu}{{}^{\circ }F·hr}\right]$	7050	353

. All other simulation parameters, including objective constants, are therefore listed in

Table A2. Simulation parameters of decentralized (DC) and centralized (CC) controllers.

		(DC)	(CC)
Objective Constants	$\alpha$	200	300
	$\beta$	${10}^{-3}$	${10}^{-3}$
	$\gamma$	5000	8000
	$\zeta$	100	100
	$\tau$	1000	1500
Simulation Parameters	Homes (N)	50
	Time-step (K)	4000
	Horizon (H)	20
	Step-length ($\Delta t$)	23.4 [s]
	D.R. length	20 [min]

.

As previously stated, $T_o$ and its derivative ${\dot{T}}_o$ represent a disturbance to the thermal system. The proposed controllers, shown in Section 3, use a receding horizon approach to determine control action, m, for a corresponding VSHP. This receding horizon controller necessarily requires that $T_o$ and ${\dot{T}}_o$ be known prior to simulation. Based on modern metrological forecasting techniques, it is reasonable to assume this outside temperature data are known over the current finite simulation horizon $\underline{\mathrm{H}}$. Conversely, in a simulated environment, like the one presented in this paper, outside temperature data are queued from the National Solar Radiation Database (NSRDB) in the form of a Typical Meteorological Year (TMY) [23]. This TMY dataset, among other qualitative properties, provides hourly ambient outside temperature data. As the name suggests, this dataset represents the most usual weather conditions for a given region and is well suited for the application of weather prediction.

The simulation results generated via Algorithms 2 and 3 are compared using several quantitative metrics. These performance metrics provide a key insight into the load-shedding capability of each control framework. The first two metric, denoted $P_i$ and $P_f$, describe the peak power drawn before and after the simulated demand response event. Each term is expressed as a ratio between the peak power demand and the total consumable power within the system. For clarity, the total consumable power is the amount of power demanded, assuming all VSHPs operate at full capacity, i.e., $m_p^k=1\forall k,p\in K,\mathcal{N}$. The next performance metric is a measurement of the total energy associated with the aggregate power consumption signals observed in Figure 7b and Figure 8b. In the context of this paper the energy consumed is defined as,

$$E_T={\int }_{t_i}^{t_f}{P}_{agg}dt.$$

(20)

To ensure this consumed energy is intrinsically represented, it too is expressed, in

Table 1. Simulation results.

Metric	(DC)	(CC)	(PID)
$P_i$	35.6%	35.3%	34.7%
$P_f$	30.7%	31.3%	61.2%
$E_T$	14.25%	14.48%	14.44%

, as a ratio by dividing it with the total consumable energy within the system. As a point of reference, the standard PID controller, shown in Figure 4, is also compared using the three performance metrics described above.

Algorithm 2. Decentralized MPC Sequence.

Initialize $\mathcal{P}\leftarrow \varnothing ,\mathcal{D}\leftarrow \varnothing$

$\mathcal{P}\leftarrow$ Generate ${\theta }_0$

$\mathcal{D}\leftarrow$ Set Initial Conditions

for$k=1$ to K do

for $p=1$ to N do

$m_p^{i_k\in \underline{\mathrm{H}}}\leftarrow Controller(\mathcal{D},\mathcal{P},k,p)$

$x_p^k\leftarrow Plant(\mathcal{D},\mathcal{P},k,p,m_p^k)$

$\mathcal{D}\leftarrow x_p^k,m_p^{i_k\in \underline{\mathrm{H}}}$

$\mathbf{end}\mathbf{for}$

$\mathbf{if}k\ge 2\mathbf{then}$

$\mathcal{P}\leftarrow \mathrm{Algorithm} 1$

$\mathbf{end}\mathbf{if}$

$\mathbf{end}\mathbf{for}$

Algorithm 3. Centralized MPC Sequence.

Initialize $\mathcal{P}\leftarrow \varnothing ,\mathcal{D}\leftarrow \varnothing$

$\mathcal{P}\leftarrow$ Generate ${\theta }_0$

$\mathcal{D}\leftarrow$ Set Initial Conditions

for$k=1$ to K do

${\mathbf{M}}^{i_k\in \underline{\mathrm{H}}}\leftarrow Controller(\mathcal{D},\mathcal{P},k,p)$

${\mathbf{X}}^k\leftarrow Plant(\mathcal{D},\mathcal{P},k,p,{\mathbf{M}}^k)$

$\mathcal{D}\leftarrow {\mathbf{X}}^k,{\mathbf{M}}^{i_k\in \underline{\mathrm{H}}}$

$\mathbf{if}k\ge 2\mathbf{then}$

$\mathcal{P}\leftarrow \mathrm{Algorithm} 1$

$\mathbf{end}\mathbf{if}$

$\mathbf{end}\mathbf{for}$

Based on the simulation results of Figure 7 and Figure 8, in addition the quantitative metrics listed in Table 1, both control frameworks are observed to similar performance attributes. Unlike the PID controller defined in Section 2.3, both optimal control frameworks reduce the load synchronization effects caused by the demand-response event. Furthermore, Figure 7b and Figure 8b show a smooth gradual increase in aggregate power consumption. This reduction in $P_f$ is partially attributed to the scheduling of control actions accomplished via the minimization the objective penalty terms (16) and (19). For the other quantitative metrics, $P_i$ and $E_T$, little change is observed between control frameworks.

A key difference between the centralized and decentralized controllers defined in Section 3, is the computational complexity associated with each framework. For the centralized controller, the number of decision variables within the optimization program of Equation (17) is directly proportional to the number of homes being simulated, N. As the number of simulated homes increases, so too increases the time required to compute the population of control actions, assuming all auxiliary features remain constant. An advantageous feature of the decentralized framework is that the computational complexity required to solve a control action for a VSHP remains constant. This assertion is exemplified by the simulation runtimes listed in

Table 2. Resultant Simulation times.

Simulation Time	(DC)	(CC)	(PID)
Single VSHP [s]	$5.4\times {10}^{-3}$	-	$1.8\times {10}^{-5}$
Population [s]	-	$1.9\times {10}^{-2}$	-
Algorithm Runtime [s]	1022	747	18

. The proposed decentralized controller necessarily requires that each VSHP compute its own control action.

By way of Algorithm 1, elements of the thermal parameter vector, $\theta \in {\mathbb{R}}^5$, are systematically updated to improve the accuracy and resilience of the predictive model. These incremental updates provide an ETP model the wherewithal to accurately mimic the dynamics of its respective plant. To show convergence between the model and plant dynamics an error signal is generated for each element of the thermal parameter vector. This error signal is defined as the difference between the plant and model parameter values, i.e., ${\theta }^p-{\theta }^m$. Each error signal is graphically depicted by the subplots of Figure 9 and Figure 10. Due to its initialization process, these thermal parameters show a tendency to deviate from the plant values during early time-steps of the simulation. We also observe that before the demand-response event, these thermal parameters tend not to converge upon their respective plant value. This failure to converge is partially attributed to the lack of persistence of excitation. After the demand-response event, parameters then converge upon their desired plant value. In a real setting, the true thermal parameters of the proposed ETP model, which defines the plant dynamics, are likely uncharacterizable by a linear approximation. Moreover, it is reasonable to assume the thermal characteristics of a home are capable of change by way of renovation or degradation. For these reasons, two update condition are employed to help determine periods of stable prediction. The first update condition is $\kappa \left(P_k\right)\le c_1$, where $\kappa (·)$ is the condition number of $P_k$, as defined in Algorithm 1, and $c_1$ is an empirically determined constant. In Figure 9 and Figure 10, it may be observed that the system parameters change rapidly during the initial stages of the system response. This may be attributed to the condition number $\kappa \left(P_k\right)$ being relatively large. The second indicator used to predict convergence is the rate at which each thermal parameter varies. Thermal parameters are therefore updated when the condition, ${\dot{\theta }}_k\le c_2$, holds. similar to the value of $c_1$, the value of $c_2$ is empirically determined. The combination of these two conditions attempts to predict when the thermal parameters are capable of being updated. Each thermal parameter must be initialized within an approximate region of the true parameter characteristics.

Remark 1.

It is a standard result of adaptive control theory that when the inputs to a control system do not excite sufficient modes of the system, parameter estimation does not converge to their correct values. For a detailed exposure, the reader is invited to see Chapter 2.4 (page 63) of [22].

5. Conclusions

In this paper, we develop a decentralized control framework to optimally schedule the control actions of a population of VSHPs. Albeit contradictory, this control framework balances end-user temperature requirements with an electric utility service provider’s desire to supply smooth predictable power. We show through minimal network communication that our decentralized control framework performs on par with a similarly structured centralized controller with omniscient knowledge of the state and control actions of its population. Based on the quantitative metrics presented in Section 4, we also show this decentralized control framework alleviates the load aggregations experienced by more traditional PID controllers. In an effort to improve the accuracy of our controller, we implement a recursive least squares algorithm to adaptively update the parameters defining the second-order ETP model. Although failing to converge under certain simulated conditions, this RLS algorithm proved useful in the convergence of parameters when stimulated with a demand response event. Over longer simulated time periods, by means of persistence of excitation, we conclude that thermal parameters will eventually approach a desired value such that the dynamic model mimics the response of the plant. Further experimental validation is needed to show the efficacy of our simulation. Such experiments will be the focus of future research. Due to the approvals needed by local utilities, regulators, and participants alike, significant planning is required to perform the necessary experiments. This research serves as an important precursor to future experimental studies.