2. Problem Formulation
In order to use occupancy for demandbased HVAC control, we first must illustrate our HVAC model settings and control strategy. MPC is a class of algorithms designed to exploit building models and forecasts of interior and exterior disturbance signals. An MPC algorithm then computes openloop optimal control actions by optimizing a cost function over a finite time horizon.
The reference temperature set point will serve as a bridge to relate occupancy prediction with MPC control strategy via the temperature set algorithm proposed in [
34].
2.1. Building Thermal Model
In this section, we describe the typical onedimensional resistancecapacitance (RC) model used in MPC design. The model stems from a physicsbased continuoustime model, which captures the dynamics of indoor temperature, interiorwall surface temperature, as well as exteriorwall core temperature. This building thermal model has been widely applied in dozens of researches [
32,
36,
37] for simulating residential and commercial buildings. It is described by
where the variables are defined in
Table 1, and the parameter values are provided in
Table 2.
The system states are ${x}_{1}$, ${x}_{2}$, and ${x}_{3}$. The model inputs consist of control decision variables and exterior disturbance signals. The control decision variables are the demands sent to HVAC systems (with ${u}_{1}$ represents the cooling power, while ${u}_{2}$ corresponds to the heating power). The disturbances are ${\delta}_{1},{\delta}_{2},$ and ${\delta}_{3}$.
To translate the model into an MPCfriendly model, we must define the state vector
x, the control signal vector
u, and the environment stochastic disturbance vector
$\omega $ as:
The continuoustime statespace model can then be described compactly as:
where
We then consider the discretetime (sampled) version of Equation (
1) described by
where
k is the discretetime index,
${x}_{k}={\left[{x}_{1,k}\phantom{\rule{0.166667em}{0ex}}{x}_{2,k}\phantom{\rule{0.166667em}{0ex}}{x}_{3,k}\right]}^{T}$ and the parameters
$\left[{A}_{d},{B}_{d},{C}_{d}\right]$ are computed from the continuoustime model parameters in Equation (
2).
It should be mentioned that such statespace matrices $A,B,G$ can be easily generated for any given buildings through either physicsbased or datadriven modeling techniques.
2.2. Baseline Control Strategy (or RBC)
The performance of the proposed adaptive control scheme will be compared with a baseline rulebased on/off control (RBC) scheme commonly used by thermostats in residential homes. These RBC algorithms represent the core logic behind the most popular mechanical and digital controls of thermostats in residential homes.
Figure 2a,b describe the overall schemes for summer and winter cases, respectively. Basically, the RBC rules compare the indoor temperature
T with a given reference temperature
${T}_{ref}$, which is allowed to drift by a cooling/heating dead band
$\Delta {T}_{c}$ or
$\Delta {T}_{h}$, respectively.
2.3. System Model
The system states are the room air temperature ${t}_{1}$, interior wall surface temperature ${t}_{2}$, and exteriorwall core temperature ${t}_{3}$. u represents the control input, which is heating power ${S}_{h}$ in this scenario (${S}_{h}$ comes as electrical power kW, since power conversion coefficients from heating load are absorbed in the model), and v is the outside disturbance including: the outdoor drybulb temperature ${T}_{outdoor}$ ${}^{\circ}\mathrm{C}$, the approximated sky temperature ${T}_{sky}$ ${}^{\circ}\mathrm{C}$, the internal load of space ${Q}_{internal}$ [W], and the solar radiation on the nodes ${Q}_{solar}$ [W]. All variables with subscripts H correspond to the HVAC.
The thermal model for any given building can then be described as:
where
States$X=\left[{t}_{1},{t}_{2},{t}_{3}\right],$Inputs$U=\left[{S}_{h}\right],$
Disturbance$V=\left[{T}_{outdoor},{T}_{sky},{Q}_{internal},{Q}_{solar}\right].$
We assume the following constraints are imposed on the temperature and control inputs (for winter heating):
where
${U}_{k}\le 0$ means cooling (we can consider similar heating case when
${U}_{k}$ positive). It should be mentioned that subindex
k represents the
kth time step. Additionally, we consider a variable speed HVAC system, where 0 represents HVAC totally OFF, and 1 (
$1$) means working at the maximum heating (cooling) power. It should be mentioned that while the temperature comfort interval has been chosen by ASHRAE, it can be adjusted to any other values based on user’s preference.
2.4. Cost Function
In this MPC problem, we desire to minimize both the temperature deviation from the reference setpoints and the energy consumption while simultaneously enforcing a performance guarantee that ensures the indoor temperature always falls in a predefined comfort zone. We can assign setpoints for all three temperature states, where
${x}_{r}$ represents the indoor temperature setpoints (dominated by current and future occupancy information). Ultimately, our objective is to find the finitehorizon control sequences which minimize the following finite horizon objective function:
where
$P\ge 0,\phantom{\rule{0.166667em}{0ex}}\phantom{\rule{0.166667em}{0ex}}Q\ge 0$ (i.e., semidefinite positive matrices),
$R>0$ (i.e., positive definite matrix),
N is the prediction horizon, and
Thus, once we have accurate occupancy predictions available provided in
Section 3, we can adaptively adjust temperature setpoints according to the novel temperature set algorithm proposed in [
34]. From this, we will be able to achieve occupancybased optimal control (as shown in
Figure 3) to improve energy efficiency of the buildings.
2.5. Temperature Set Algorithm
In conventional practice, the HVAC operates under a fixed deadband (chosen by the users) for indoor temperature. Currently, most temperature setpoints are predefined by the building owner or administrator, and do not change frequently during the regular operation periods.
We recall our own simple yet effective algorithm (shown in Algorithm 1) to assign reference temperature setpoints for each half hour of the day based on the realtime occupancy [
34]. Inside this temperature set algorithm, we need first define the maximum (max(
${O}_{h}$)) and minimum(min(
${O}_{h}$)) occupancy values (based on previous field measurement or survey) during normal operation. Similarly, we also must define the comfort band chosen by the customers. Obviously, the larger the band is, the more energy savings will be achieved. The beauty of Algorithm 1 is its ability to identify a temperature setpoint depending on the occupancy information. The temperature setpoints are then assigned to each half hour of the day based on the range in which the occupancy number of corresponding half hours fall in.
Following [
38], a discomfort tolerance index
$\alpha $ is defined to characterize building users’ different choices/tolerance on thermal comfort (discomfort). Discomfort tolerance is considered high when
$\alpha >0$, and low when
$\alpha <0$.
Algorithm 1 Temperature Setting Algorithm [34] 
 1:
Step 1:  2:
Initialize $\alpha $  3:
Step 2:  4:
$n\leftarrow \frac{{T}_{max}{T}_{min}}{k}+1$  5:
for all hour $h=1$ to 48 do  6:
$Range\leftarrow max\left({O}_{h}\right)min\left({O}_{h}\right)$  7:
${r}_{0}\leftarrow min\left({O}_{h}\right)$  8:
end for  9:
if$\alpha =0$then  10:
Go to step 3  11:
else  12:
Go to step 4  13:
end if  14:
Step 3:  15:
for all setpoint j ($j=1$ to n) do  16:
${r}_{j}\leftarrow {r}_{j1}+\frac{Range}{n}$  17:
Go to Step 5  18:
end for  19:
Step 4:  20:
for all setpoint j ($j=1$ to n) do  21:
${r}_{j}\leftarrow {r}_{j1}+Range\ast \frac{{2}^{\alpha (j1)}(1{2}^{\alpha})}{(1{2}^{\alpha n})}$  22:
end for  23:
Step 5:  24:
for all hour $h=1$ to 48 do  25:
${T}_{h}^{set}\leftarrow k[argmin\{j:{O}_{h}\le {r}_{j}\}1]+{T}_{min}$  26:
end for

3. Occupancy Prediction Algorithms
In this section, we introduce our algorithm for the detailed estimation of future occupancy, and we show how we could predict the number of occupants in the room. An overview of all the developed algorithms is summarized in
Figure 4.
In order to study a practical occupancy model, we collected real occupancy data from an occupancy sensor for a room. We randomly pick a segment of data dated “13 October 2010–5 April 2011”, i.e., 174 days. The sampling interval is 30 min, so each individual sensor collects 48 occupancy samples each day. i.e., we have 8352 samples for each sensor. For simplicity, we consider the 8352 samples from a single sensor.
Natural questions which arise are: what is the probability for this room to be occupied; and how many people will be in the room?
To answer the first question, we could compute the probability for the room to be occupied by observing historic data. This has been reported in our earlier paper [
34], so we skip this model here due to space constraints.
Due to occupancy’s stochastic characteristic, it is not realistic to expect the real occupancy of the room to exactly follow the given schedule. Therefore, we should desire to accurately predict the occupancy information based on the most recent measurement. Moreover, detailed occupancy estimation considers not only whether the building is occupied or not, but also takes into account the number of occupants in the building. Here we will introduce some background and several alternative techniques that may be adequate for occupancy estimation.
3.1. Expectation Maximization (EM)
The first occupancy modeling algorithm relies on the statespace model, which has been very popular in both societies of control systems and signal processing due to its advantage of online recursive implementation. The EM algorithm is a datadriven approach that builds and updates the estimated occupancy relying purely on collected measurements. Its core mechanism consists of two simple equations, i.e., a state
${x}_{k}$ Equation (
7) and a measurement
${y}_{k}$ Equation (
8).
A standard EM model in discrete time can be written as:
where
${x}_{k+1}\in {\mathbb{R}}^{n\times 1}$ (
${\mathbb{R}}^{n\times 1}$ denotes the space of real vectors with dimension
$n\times 1$) is the state that characterizes the occupancy; it is a variable of the time series
$\left\{{x}_{k}\right\}$ determined by the previous state
${x}_{k}$ and the noise term
${w}_{k}\in {\mathbb{R}}^{m\times 1}$ introduced at each
k.
${A}_{k}\in {\mathbb{R}}^{n\times n}$ and
${B}_{k}\in {\mathbb{R}}^{n\times m}$ are corresponding coefficients.
The beauty of the EM algorithm is the time varying nature inherited in the statespace model, which enables the model to adapt dynamically to the most recent occupancy model. Moreover, it takes into account the noise terms ${w}_{k}$ and ${v}_{k}$ that capture small perturbations or uncertainties introduced at each time k. This greatly alleviates the challenges associated with occupancy uncertainties in the model.
We can estimate the unknown system parameters
${\beta}_{k}=\{{A}_{k},{B}_{k},{C}_{k},{D}_{k}\}$ and states
$\left\{{x}_{k}\right\}$ through a finite set of received signal measurement data
$Y=\{{y}_{1},{y}_{2},\dots \}$ following [
39]. Finally, we can achieve a onestep prediction of the occupancy by [
40]:
where
${\widehat{y}}_{k+1}$ denotes the predicted occupancy at
$k+1$ and
${\widehat{K}}_{k}$ is the Kalman gain.
3.2. Finite State Automata (FSA)
Probabilistic FSA [
41] have previously been introduced to describe distributions over strings. FSA has been used quite successfully to address several complex sequential pattern recognition problems, such as continuous speech recognition, computational biology [
42] and linguistics [
43]. The general Systems Problem Solver (GSPS) proposed in [
44], provides a novel and highly effective method for reconstructing the input/output behavior of FSA. The detailed algorithm and system formulation can be found in [
45].
Given a system and a sequence of length n generated by that system, we may posit a relation between its variables. This relation takes the form of a function f that maps observations of some variables at times $n,n1,\dots ,k$ with $k\ge 1$ to observations of other variables at time n.
In this occupancy problem, we want to know the exact number of people in the room. So we assume different numbers of occupants as different states in the FSA. As long as finite states are involved, the general form of the rule and the methods for constructing and forecasting are the same.
Such a relation is called a rule. Typically this purpose is forecasting; i.e., to predict future observations from past and current observations. This relation takes the form of a function f that maps observations of some variables at times $t,t1,\dots ,tn$ to observations of other variables at time t. Rules are posited by the observer for some particular purpose.
Continuing the example above, consider strings comprised of the letters a, b, c. Take these strings to be our system and the variable ${v}_{1}$ to be a single character in a string. Observations of this variable are indexed by position in the string ordered from left to right so that the first character in the string is ${v}_{1}\left(1\right)$, the second is ${v}_{1}\left(2\right)$, and so forth. One example of a sequence for this system is $abcabcabc$.
A possible rule that predicts the next letter in this string is ${v}_{1}\left(n\right)=f\left({v}_{1}(n1)\right)$. This rule relates the characters in subsequences of length two such that the leftmost character predicts the rightmost character. Interpreting the sequence $abcabcabc$ with this rule yields the following function: $f\left(a\right)=b$ which occurs three times, $f\left(b\right)=c$ which occurs three times, and $f\left(c\right)=a$ which occurs two times.
3.3. Simplified Binary States FSA
In general, the length of the “look back depth” used is decided by actual problem. For our specific occupancy prediction problem, the system of interest has a single variable with two possible values: occupied (
b) or not occupied (
a). After tail and error check, we posit a rule that looks back three steps and also considers the time of day, i.e.,
${v}_{1}\left(n\right)=f({v}_{1}(n1),{v}_{1}(n2),{v}_{1}(n3),t\left(n\right))$. The time of day
$t\left(n\right)={t}_{n}$ is used to characterize the different rules for different time period during a day; for the available data use there are 48 times that can be considered, as the sampling interval of the sensor is 30 min. Given the data and a rule, we can build a model for forecasting with the simple procedure described by Klir [
44]. More details about working mechanisms for FSA can be found in [
34].
To illustrate this procedure, consider the mask and sequence in
Table 3. The system that generated this sequence has time variable
${t}_{n}$ and a single variable
v that can take the value
b or
a. The rule for this mask built with 3000 data points is shown in
Table 4. It should be noted that in order to simplify the explaination, we decouple the time variable from the rule. To be specific,
Table 4 represents the rule for 3:00 p.m. using 3000 data points. Considering the sampling interval is every 30 min, we could generate 48 such rule tables for the whole day (24 h).
To illustrate how this model is applied for forecasting, suppose we begin with the latest observation sequence as $bab$ at 3:00 p.m. The next output is a with probability $0.33$ or b with probability $0.67$. If we were at a different time step other than 3:00 p.m., then the output is selected according to the corresponding rule table at that time step. Once the time step is fixed, then the second output is a with some probability p or b with probability $1p$. Continuing in this fashion, we can construct a tree of possible futures and use these possible futures to inform the control system.
3.4. Estimating Number of Occupants
We consider three methods for anticipating the actual number of occupants within a room.
3.5. Basis Function
Finally, we introduce the third powerful algorithm, i.e., the uncertain basis functions. If we consider
${y}_{k}$ as the current occupancy measurements detected by occupancy sensor, it can be well represented by the following basis functions:
where
$\{{\varphi}_{1,k},\dots ,{\varphi}_{p,k}\}$ are the basis functions, and
$p<N$, and
${A}_{i}$ is the corresponding coefficient of each basis function [
46,
47].
However, this trivial representation may not be able to capture all the dynamics of occupancy due to the limitation of fixed basis functions, which eliminates the uncertainty nature. An alternative way to cast this formulation is to assume that each basic function depends on some unknown parameter vectors
${\mathit{\theta}}_{i}$, where only some statistics of the distribution of
${\mathit{\theta}}_{i}$ are known. Then, the corresponding coefficients can be estimated by minimizing an expected cost function following the technique developed in [
48].
Following [
48], we assume that each
${\mathit{\theta}}_{i}$ is independent. The basic functions are further represented by
${\varphi}_{i,k}\left({\mathit{\theta}}_{i}\right)$. The main objective here is to find the best coefficients to minimize the expected cost function
$\widehat{J}\left(\mathit{A}\right)$ defined as:
where
${\mathbb{E}}_{\mathit{\theta}}$ is the expectation with respect to
${\mathit{\theta}}_{i}$. The measured values are real, so we could estimate the coefficients
$\mathit{A}$ following steps presented in [
39].
We focus on predicting future occupancy using the basis function:
This means that we can predict occupancy values with
p random basis functions. Similarly as in [
39], we assume the basis functions to be governed by three different distributions, i.e.,
Gaussian, Laplace and Uniform. Consequently, we are able to compute occupancy predictions under each distribution.
4. Case Studies
To illustrate the effectiveness of the occupancy prediction techniques proposed in the last section, we assess their performance using the aforementioned occupancy measurement (at the beginning of
Section 3). In the first part, performance of three occupancy prediction techniques are examined, and corresponding accuracy comparison are provided. In the second part, different temperature set trajectories s are obtained using our temperature set algorithm. The algorithm is employed to assign reference temperature setpoints for each hour of the day. Lastly, these reference temperature setpoints are utilized in the standard HVAC control strategies. In essence, different energy saving benefits are studied for traditional ON/OFF control and advanced MPC control, respectively.
4.1. Definition of the Performance Indexes
We define the estimation accuracy as the total number of correct predictions divided by the total number of predictions. The Root Mean Squared Error (RMSE), which characterizes the absolute estimation errors.
More formally, if we let ACC represent the accuracy, then using indicator functions, we obtain the accuracy of the estimator:
where
O and
$\widehat{O}$ are true and estimated occupancy, respectively. Let us denote the characteristic function of estimation error
$\U0001d7d9\left(EO\right(k\left)\right)$ as:
Here, the RMSE is defined as the square root of the mean square error:
Therefore, RMSE will be the square root of (
15).
4.2. Occupancy Prediction Performance
4.2.1. GSPS Model
In this GSPSbased scenario, we assume we have access to a large enough historical data set. We trained the model using the last 3000 and 5000 data points, respectively. The prediction results are shown in
Figure 5a,b. As expected, the more data involved in training the FSA, the better the prediction results. This is also confirmed by the estimation error comparison shown in
Table 5.
4.2.2. EM Method
Next, the elegant EM algorithm is applied to occupancy prediction. The performance is depicted in
Figure 6, where only the last 20 sample points are used to predict the occupancy information at the very next time step.
4.2.3. Uncertain Basis Functions
In order to the show robustness of the proposed uncertain basis technique, we tested it with three different distributions for
${\theta}_{i}$. It should be mentioned that only 10 last sampling points are used to build the optimal basis for each distribution.
Figure 7 shows prediction comparison results using three different distributions.
4.3. Temperature Set Points
Through the above simulation results, we achieve the desired occupancy prediction. However, our goal is to design temperature setpoints based on these occupancy sequences. In order to further determine the effectiveness of these occupancy prediction results, we integrate the occupancy prediction results into the temperature set algorithms. The one corresponding to basic functions using
uniform distribution is presented in
Figure 8.
4.4. OccupancyBased Control
To show the energysaving performance using the proposed stochastic models for occupancy and temperature set algorithm, we insert the obtained temperature reference into simple ON/OFF switching control framework. A fixed reference temperature $23{\phantom{\rule{3.33333pt}{0ex}}}^{\circ}\mathrm{C}$ is chosen for the baseline scenario. An occupancydependent reference temperature generated via our temperature set algorithm replaces original fixed schedule. This simple step will adaptively tune the setpoint of HVAC systems according to current occupancy information, avoiding wasting energy for empty rooms.
In order to quantify the performance using different algorithms, we utilize the energy cost using traditional ON/OFF control strategy without any occupancy information as a benchmark. It should be mentioned that energy cost is defined as 2norm, i.e., ${U}_{total}=\sqrt{{\sum}_{k=1}^{T}{U}_{k}^{2}}$.
Next, a comparison between different algorithms is made by changing only the occupancy information. Detailed numerical result is given in the right end column of
Table 5. We are expected to save approximately 13% and 20% energy consumptions for traditional ON/OFF and MPC control strategies, respectively.
4.5. Summary of the Results
In this section we compare four occupancy prediction algorithms, all trained using the same training set described at the beginning of
Section 3.
Figure 8 shows the realizations of occupancy predictions can be applied to the corresponding test set. Ideally, we can increase temperature set point when less occupants will be present in the room. As expected, we notice higher temperature set points are achieved corresponding to a larger tolerance index, as we are studying a summer cooling case in this paper.
Table 5 summarizes the achieved numerical performance and accuracy comparison for the three algorithms. Generally speaking, both EM and Uncertain basis methods can provide reliable predictions with just a few historical data points. The GSPS method meanwhile, requires many more points to build a reliable model. Additionally, GSPS and Uncertain basis methods achieve a higher accuracy, while the EM method provides a degraded prediction result. Each method may find its own suitable application scenarios depending on the accuracy requirement and data structure.
It should be remarked that, although some mismatches exist for nonzero jumps in
Figure 5a,
Figure 6 and
Figure 7c, all prediction algorithms track the 0 base line (nonoccupied status) perfectly. Therefore, all prediction techniques are effective for identifying empty rooms, with an over 90 percent accuracy rate. Moreover, the accuracy conditions we set are extremely restrictive. In other words, the accuracy is said to be satisfied only when the estimated number of occupants is exactly the same as the real measurement. So in this case, the accuracy is void if the estimated number is 13 while actual number is 12.
Furthermore, the obtained occupancy status is successfully applied into the temperature set algorithm which dominates energy consumption in building climate systems. In this final experiment, we applied the designed temperature setpoints into two different algorithms  basic control and MPC (designed in
Section 2), and compare their energy consumption. The building thermal zone model we picked has also been introduced in
Section 2. The detailed energy consumption data of the simulation has been given in
Table 6. These control tests complete our occupancybased control framework, which showcases up to 20% energy saving benefits via the proposed corresponding control framework.
5. Conclusions and Future Work
In this paper, we propose three different occupancy prediction methods for demandbased HVAC control. All three proposed shortterm stochastic modeling methods, GSPS, EM and uncertain basis, achieved more than 70% accuracy in the experimental studies. Furthermore, we have designed a novel temperature set algorithm to correctly assign temperature set points based on the instantaneous occupancy information. To complement the occupancybased framework, we have integrated the temperature set point naturally into the MPC algorithm. Finally, detailed comparisons are provided for energy consumptions with various occupancy estimation algorithms and without any occupancy information. This paper delivered a novel endtoend solution, which connects a reliable stochastic occupancy modeling study with the occupancybased control design. Consequently, we have seen up to 20% energy saving by integrating the proposed technique into two standard HVAC control strategies.
A great number of increasingly complex models are being developed for HVAC systems. However, the limited number of implementations of such models in demandbased control and the lack of occupants’ effects limits their ability to improve efficiency while guaranteeing a comfortable temperature environment in buildings. Our near future work will involve detailed internal heat gain subject to different occupancy situations and various application scenarios, particularly the hot topic of buildingtogrid integration. Another interested direction is to perform the sensitivity analysis for changing the set point. Basically, to answer the question, when is the best time to change the setpoint; and how long/much will the electricity consumption reflect the change. This knowledge is critical for doing demandresponse using buildings.