Dynamic Residential Energy Management for Real-Time Pricing

: A home energy management system (HEMS) was designed in this paper for a smart home that uses integrated energy resources such as power from the grid, solar power generated from photovoltaic (PV) panels, and power from an energy storage system (ESS). A fuzzy controller is proposed for the HEMS to optimally manage the integrated power of the smart home. The fuzzy controller is designed to control the power rectifier for regulating the AC power in response to the variations in the residential electric load, solar power from PV panels, power of the ESS, and the real-time electricity prices. A self-learning scheme is designed for the proposed fuzzy controller to adapt with short-term and seasonal climatic changes and residential load variations. A parsimonious parameterization scheme for both the antecedent and consequent parts of the fuzzy rule base is utilized so that the self-learning scheme of the fuzzy controller is computationally efficient.

One of the most attractive functions of the HEMS is that it saves electricity cost in response to the dynamic electricity prices without affecting users' comfort. The users participating in DR programs uses the HEMS to schedule the use of their residential controllable appliances to the time at which electricity prices are lower to reduce their electricity cost [5][6][7][8]. Conversely, the HEMS can also manage the connected energy supplying systems such as PV panels [9,10] and ESSs [11,12] to provide the necessary energy for powering appliances instead of using only the power from grids.
In order to reflect the fluctuated wholesale electricity prices to end customers, timedifferentiated pricing models such as real-time pricing (RTP), critical-peak pricing (CPP), day-ahead pricing (DAP), time-of-use pricing (TOUP), etc., have been adopted [13] in different DR programs. The end customers tend to adjust their electricity consumption in response to these timedifferentiated prices. Among these pricing models, RTP is the most effective model in terms of reducing the peak reduction and load curve flattening [14,15]. It is common for the HEMS to optimize the appliance operation scheduling under the time-differentiated electricity pricing models. The electricity prices in the next 24 h for DAP or TOUP are acquired by the end customers. The operation scheduling of most residential appliances can thus be optimized based on the received price profile using HEMS. However, if the RTP is adopted, the price signals for the next 24 h or next long enough period of time are no longer available because the end customer can only receive the prices in realtime. Some simple linear schemes such as ARMA model [16] is usually utilized to predict the electricity prices. Other complex machine learning approaches such as support vector machine [17], neural networks [18], or genetic algorithm [19] are also applied to the price prediction in order to increase prediction accuracy. The electricity prices are predicted from the current step to the end of optimization horizon for the appliance scheduling optimization in HEMS. The scheduling optimization accuracy can be reasonably attained under the conditions that enough number of price data are collected for the price prediction, and the fluctuations of future prices are within a reasonable range. The intervals updating real-time prices usually range from 10 min to 1 h. The appliances scheduling and energy management for HEMS with RTP relies on the price prediction scheme and prediction horizon. Different price prediction schemes are with different statistical variance and bias.
In this paper, an efficient HEMS-based residential energy management approach is proposed for RTP but without price prediction. A real-time approach capable of calculating the amount of power drawn from the grid while managing all energy resources and minimizing electricity cost is a viable solution of energy management for the RTP without price prediction. A fuzzy controller is proposed for the HEMS to control the rectifier that dynamically determines the power drawn from the grid. The fuzzy controller is designed with a simple computational structure so that the amount of power from the grid required to balance the overall residential energy resources is calculated in real time. In other words, the power drawn from the grid can be determined instantaneously in response to residential power consumption, solar energy generation from PV panels, the status of charging (SOC) of the ESS, and the real-time electricity prices. The proposed fuzzy controller dynamically determines the power drawn from the grid to reduce the electricity cost of smart homes.
Both load scheduling and energy management allow the HEMS to save energy cost and participate in DR programs under dynamic electricity prices. In [20][21][22][23], a household's electricity cost was minimized using the HEMS to automate the energy usage in smart homes in response to dynamic prices. The partially observable Markov decision process optimization approach was utilized in [20] to optimize appliance scheduling. In [21], a stochastic dynamic programming was applied to optimize a set of appliances to be controlled while considering the uncertainty in real-time prices. Dynamic programming and mixed integer linear programming were utilized in [22] and [23], respectively, to optimize appliance scheduling for time-varying electricity prices. The HEMS considers the aforementioned factors while optimizing load scheduling and energy management in order not to affect residential users' comfort or convenience of using appliances [24][25][26]. In [24], users' discomfort levels were defined in terms of the volume of energy curtailed by the HEMS. The quality of experience based on the knowledge of the annoyance suffered by users was defined in [25] as the discomfort level along with the load scheduling optimization. The inverse of the total waiting time of appliances was utilized as the comfort level in [26]. As air conditioners and heaters are one of most power consuming appliances, they are categorized into the list of controlled appliances by the HEMS [27][28][29]. However, directly calculating the power consumption of air conditioners and heaters is a challenging work. In [27,28], the mathematic models to determine indoor temperatures associated with air conditioners and heaters are utilized for the scheduling optimization.
Along with the appliance scheduling optimization, the power that is drawn from the grid can also be optimized using an optimization tool such as mixed-integer linear programming (MILP) [30,31], meta-heuristic approach including genetic algorithm (GA) [32,33], and particle swarm optimization (PSO) [34,35]. It is common for smart homes to integrate renewable energy resources such as solar energy, wind energy, and the energy obtained from the power grid. To overcome the uncertainty of using renewable energy intermittently, stochastic programming was utilized as the optimization approach in the HEMS [36,37]. However, optimization is a computationally expensive approach and occupies most of the computational resource and time of the central processing unit (CPU) in the HEMS. In addition to scheduling optimization, the HEMS is designed to conduct monitoring and control functions for all connected meters, devices, appliances, etc.
A self-learning scheme is designed in the proposed fuzzy controller to allow the HEMS to adapt to the variations of the residential power consumption, historical price data, as well as short-term climatic changes. The fuzzy rule base needs to be refined from time to time through the self-learning. Numerous self-learning approaches have been explored for fuzzy controllers. In [38] and [39], a neural fuzzy controller was proposed using the gradient descent approach. The self-learning approach for adaptive fuzzy controller proposed in [40] and [41] requires a mathematical model of the system to be controlled although sometimes it is not easy to be accurately measured. Other metaheuristic optimization approaches such as GA [42,43], ant colony optimization [44,45], and PSO [46,47] have also been applied to fuzzy controller learning. However, low computational power CPUs are generally designed in the HEMS for cost reduction. The self-learning approach proposed in this paper needs to be computationally simple but efficient. A fuzzy controller with orthogonal modulated triangular membership functions (MTMF) [48,49] is designed in this paper. The MTMF is a parsimonious parameterization scheme for membership functions that only requires two parameters to define the membership functions for every input in the antecedent part of the entire fuzzy rule base. An efficient approach for designing the membership functions in the consequent part of the fuzzy rule base will also be proposed in this paper. The computational efficiency of self-learning scheme for the fuzzy controller can be greatly improved. The GA is applied to optimize the fuzzy controller in this paper. With a simplified organization of the fuzzy rule base, very limited time is required to learn the fuzzy controller in the HEMS with lower computational power of the CPU. The proposed fuzzy controller is a fast online controller that continuously controls the rectifier output. However, the self-learning of the fuzzy controller is an offline scheme and yet is computationally efficient due to the parsimonious parameterization design of fuzzy rule base. The same idea with online fuzzy controller and computationally efficient off-line self-learning can be applied to the systems such as data centers [50] or tele-operation of robots [51,52] that requires complex real-time control.
The technical novelty and main contribution of this paper are summarized as follows: 1. A HEMS integrating energy resources from the grid, ESS, and PV panels is designed and implemented for RTP without price prediction. A self-learning fuzzy controller is designed in the HEMS dynamically controlling the rectifier output to reduce the electricity cost. To the best of our knowledge, no similar studies addressing this issue have been proposed in literature. 2. To reduce the computational effort required for self-learning, a novel membership function known as a MTMF is proposed to parameterize the fuzzy rule base so that the number of parameters required to be learned for the entire fuzzy rule base is greatly reduced. 3. The GA is applied to the learning of the fuzzy rule base to minimize the electricity cost paid to the utility company. Although the GA-based learning is conducted offline, it does not require too much time to obtain convergent solutions because the number of parameters to be learned and the search range of every parameter have been greatly reduced. 4. Although GA is a random-search based learning approach, the search range of every parameter is mathematically analyzed before learning. The search range of every parameter is pinpointed and focused to a small range.
The rest of this paper is organized as follows. Section 2 introduces the structure of the HEMS investigated in this paper. Section 3 describes the fuzzy rule base and analyzes the characteristics of the proposed fuzzy controller with MTMF. Section 4 introduces the self-learning scheme of the proposed fuzzy controller. The experiments conducted using the proposed fuzzy controller are presented in Section 5, and the conclusions are drawn in Section 6.

Home Energy Management System
The structure of the power system with the home area network (HAN) for a smart home is shown in Figure 1 where all power sources are controlled and monitored by the HEMS through the HAN. All power sources are integrated and connected to the DC bus and the DC power is converted to the AC power through an inverter. All residential loads and converted AC power are connected to the AC bus. The DC power mainly comes from the PV panels, the lead-acid battery based ESS, and the power from the grid that is converted to DC through a rectifier. Since the output resistance of the ESS is much smaller than that of other power sources in Figure 1, the ESS discharges as soon as the voltage on the DC bus is below a preset threshold. Therefore, the ESS and PV panel are usually installed together because the ESS can compensate for the sudden voltage drops on the DC bus due to an unexpected decrease in the DC power or abrupt residential load increase. To compensate for the unexpected insufficiency of renewable DC energy sources, AC power from the grid is fed into the DC bus through a rectifier. The amount and scheduling of AC power is determined by the HEMS in response to the variations in the solar energy generated from PV panels, load demands at home, SOC of the ESS, and the RTP. A fuzzy controller controlling the rectifier output is designed in the HEMS. The fuzzy controller is a tailor-made controller because both the solar energy generated from PV panels and load demands vary for every smart home. It takes time to collect the information of the solar energy generated by PV panels and the load demands for every home. Engineers need to put in great efforts to fine tune the fuzzy controller for the HEMS in every home. A fast and effective self-learning fuzzy controller is designed to overcome this practical environmental uncertainties for HEMS installation at different homes. The fuzzy controller in every newly implemented HEMS comes with an initial parameter setting that results in reasonable energy management performance. After gaining reasonable amount of data such as daily solar energy profile data, daily load demand profiles, and historical data of RTP for a certain number days, the HEMS conducts self-tuning for the parameters in the fuzzy controller. The self-learning of the fuzzy controller can be conducted periodically or in response to environmental changes.
The fuzzy controller in the HEMS controls the rectifier output at every sampling interval s T .
Denote the power sources j pv P , j g P , and j ess P as the output power of the PV panels, rectifier, and ESS, respectively, j d P as the load demand, at the j-th time step. The power sources and the load demand are balanced as follows: , if P P , otherwise.
; (2) It is shown in Equation (2) that the HEMS does not need power from the grid if the solar energy is sufficient to self-support the load demand. If j g P is more than j s P , the additional power can be used to charge the ESS as long as the SOC < 1. Conversely, the ESS discharges to the DC bus maintaining the load balance stated in Equation (1) if j g P is not sufficient to support j s P due to a sudden increase in the load demand j d P or a sudden decrease in the solar power. The power charging ESS from the DC bus or the power ESS discharges to the DC bus is defined as follows: The ESS is in the charging mode if j ess P > 0 and is in the discharging mode otherwise. The SOC of the ESS can be updated at every j-th time step as follows: where s T is the sampling interval, and cap E is the capacity of the ESS. Moreover, the efficiency

Fuzzy Controller with Orthogonal Modulated Membership Functions
The fuzzy controller     in Equation (2) is designed in this section. The proposed fuzzy controller is to control the rectifier in real-time and to continuously determine the rectified power into the smart home from the grid. The design also puts the need of off-line self-learning for the fuzzy controller into consideration. A computationally efficient parsimonious parameterization scheme is designed in the fuzzy controller. The orthogonal MTMFs provide excellent parsimonious parameterization effect. The antecedent parts of the fuzzy controller proposed are defined with MTMFs.

Fuzzy Controller Design
A fuzzy controller is utilized to control the rectified AC power from the grid as in Equation (2).
Assume that 3 3 i A are the antecedent fuzzy sets for the inputs j s P ,  j , and  j , respectively, and 1 2 3 i i i g is the consequent fuzzy singleton for the output j g P at every j-th time step. The proposed fuzzy controller can be described by the following fuzzy rules: and is and is then is x is defined as a symmetric orthogonal triangular function: otherwise.
The universe of discourse of all membership functions in Equations (6) It is obvious for the orthogonal membership function that: Therefore, Substituting (12) into (9)  x at the j-th time step. The calculation of defuzzification can be greatly simplified as in Equation (13).

Fuzzy Rule Base
There are totally     m m m fuzzy rules in the fuzzy rule base as shown in (5). A selflearning scheme is proposed in this paper to learn parameters of every fuzzy rule in the fuzzy rule base. It is obvious that considerable computation effort is necessary to learn the entire fuzzy rule base in (5). The MTMFs are utilized so that the number of parameters required to be learned for the fuzzy rule base can be greatly reduced.
The centers of the MTMF defined in (14) are modulated using a modulation factor  a and the exponent  a in (15) as follows: The centers of the MTMF can be added an offset factor 0 1    a except for the first one that is set at the origin. Therefore, the centers of the MTMF are defined as follows: Referring to Equations (14)- (17), the fuzzy rule base with a total of    (17) form a strictly increasing sequence if the following theorem is satisfied. (17)

Theorem 1. The centers of the MTMF defined in
The condition in Equation (19)     m m m fuzzy rules in Equation (5)   The design of the consequent part of the fuzzy controller aims to reduce electricity cost. Referring to the characteristics of the HEMS and the load balance equation, the design of the consequent part of fuzzy controller is mainly based on following considerations.
1. The fuzzy controller should allow more rectified power j g P into DC bus from the grid as the net load difference j s P increases because increase in j s P implies either that the net load demand increases or that the solar energy decreases.
2. The fuzzy controller should allow less rectified power j g P into DC bus as ESS has a higher amount of stored energy, i.e., as the SOC of the ESS  j increases. Using stored energy in the ESS in a smart way could result in reducing electricity cost.
3. The fuzzy controller should allow less rectified power j g P into DC bus as the RTP increases because the main function of this fuzzy controller is to reduce the total electricity cost paid to the utility.   m m m fuzzy rules in (5) are defined as: where:

Self-Learning of Fuzzy Controllers
As the HEMS is initially installed and operated with the initial parameter settings for a certain period of time, the fuzzy controller that controls the rectifier output can be optimized or updated on the basis of the recorded historical data. The previously optimized fuzzy controller has to be updated from time to time when the environmental settings or home load variations are updated.
The output of the fuzzy controller at every j-th time step parameterized by   i,k ρ is defined as: where     j is the penalty function that prevents the ESS from over discharging. Denote  as the lower bound of the SOC that allows the ESS to discharge. Define     j as: where  is a large number. Let   where   1 0 1   , is a random real number. Every chromosome is selected for mutation with probability  . The mutation is conducted by randomly perturbing every parameter in the chromosome within a given range. Without loss of generality, let  a in the j-th chromosome be selected for mutation, then: where   2 1 1    , is a random real number, and  w is a constant denoting the perturbation range for mutation. If the mutated parameter is out of the searching range, then the mutation is repeated until the mutated result is within the allowed range. The elitist scheme is applied to the proposed GA, i.e., the best chromosome associated with the minimum fitness value in every generation is passed to the next generation. Assume that it takes g N generations for the GA to converge, the best parameter * ρ for the fuzzy rule base is the best chromosome obtained in the

Experiments
An environment practically simulating a smart home with a home power system controlled by the HEMS is developed and used to test the effectiveness and efficiency of the proposed self-learning fuzzy controller. The appliances and their associated power ratings in the simulated smart home are shown in Table 1. The rated output power of the PV panel is 4kW. The sampling interval is set as 5 min. The self-learning fuzzy controller is automatically learned based on the 14 days of previously recorded load profiles. Referring to Equation (29), the total number of time steps for evaluation L are 4032. The fuzzy controller was initially set in the HEMS to control the grid power g P to the home power system. After collecting 14 days of load profiles, the fuzzy controller parametrized by the MTMF was optimized through self-learning. The performance of the optimized fuzzy controller on both sunny and cloudy days is shown in Figures 2 and 3 The first scenario of the experiment was to test the performance of the proposed fuzzy controller with the HEMS on a sunny day. The average theoretical output power of the PV panels th P = 1868 W. It is shown in Figure 3 that the solar power pv P was greater than the household load demand d P from 7:00 to 16:00, thus leading to negative net load demand     s d pv P P P within this period. The additional solar power can also charge the EES so that the SOC  increased before 16:00.
No additional power from the grid was required during this period. Therefore, the fuzzy controller allowed almost no power from the grid g P . Although the net load demand s P became positive, both the SOC  and the RTP  were still high from 16:00 to 17:30. g P was controlled to be approximately 0 during this period. Since the RTP  received at every time step was still high from 17:30 to 20:00, the fuzzy controller controlled g P to a reasonably small value so that the stored energy in the ESS can be used as much as possible. The RTP  dropped to reasonably small values after 20:00. The g P controlled by the fuzzy controller increased to reasonably larger values, thus providing sufficient power to cope with net load demand. The net load demand significantly reduced to much smaller values after 01:00, and g P was controlled to be sufficiently large enough providing the required electricity because the RTP  were comparatively low during this period.   Figure 3. Output of fuzzy controller for the first scenario (sunny day). The second scenario of the experiment was to test the performance of the proposed fuzzy controller with the HEMS on a cloudy day. Referring to Figure 4, the solar power was not sufficient to provide the net load demand on a cloudy day. The net load demand s P varied from 6:00 to 15:00 due to insufficient solar power during this period. By integrating the ESS and solar power, the fuzzy controller dynamically controlled g P to cope with the dynamic net load demand. On the other hand, the controller attempted to minimize the use of g P when the RTP  were high. The RTP  were high from 12:00 to 15:00, the fuzzy controller only allowed reasonably small amount of g P used in the home power system. The net load demand s P was high from 18:00 to 20:00. However, the fuzzy controller allowed reasonably small amount of g P , thus leading to a higher amount of discharging power from the ESS due to the high RTP  during this period. The RTP reduced after 21:00, and the fuzzy controller allowed a higher amount of g P from the grid following the variation of net load demand s P .   Figure 4. Output of fuzzy controller for the second scenario (cloudy day).

Conclusions
A computationally efficient self-learning fuzzy controller for the HEMS has been proposed for reducing the electric energy cost for smart homes. With the parsimonious parameterization design of the fuzzy rule base, the fuzzy controller can efficiently regulate AC power from the grid. Moreover, the fuzzy controller only requires six and three parameters to automatically learn the antecedent and consequent part of the entire fuzzy rule base, respectively, for the three-input/one-output fuzzy controller. The proposed design of the fuzzy controller for the HEMS has a good commercialization potential because it was intentionally designed to fit the embedded system by using low computational power CPUs. As long as the power balance equation in Equation (1) is satisfied, the same control scheme of the proposed fuzzy controller can be applied to a smart home with more number of integrated energy sources such as fuel cells or electric vehicle charging and discharging.
Moreover, the problem to be solved in this paper is a system with varying conditions. More uncertainty quantification methodologies can be applied to this problem. The self-learning fuzzy controller proposed in this paper can be classified as a non-probabilistic approach solving the forward uncertainty propagation problem. Other approaches for the forward uncertainty propagation problem such as the one in this paper are worth investigating.