A Distributed Demand Side Energy Management Algorithm for Smart Grid

This paper proposes a model predictive control (MPC) framework-based distributed demand side energy management method (denoted as DMPC) for users and utilities in a smart grid. The users are equipped with renewable energy resources (RESs), energy storage system (ESSs) and different types of smart loads. With the proposed method, each user finds an optimal operation routine in response to the varying electricity prices according to his/her own preference individually, for example, the power reduction of flexible loads, the start time of shift-able loads, the operation power of schedulable loads, and the charge/discharge routine of the ESSs. Moreover, in the method a penalty term is used to avoid large fluctuation of the user’s operation routines in two consecutive iteration steps. In addition, unlike traditional energy management methods which neglect the forecast errors, the proposed DMPC method can adapt the operation routine to newly updated data. The DMPC is compared with a frequently used method, namely, a day-ahead programming-based method (denoted as DDA). Simulation results demonstrate the efficiency and flexibility of the DMPC over the DDA method.


Introduction
In recent years, the smart grid has undergone significant innovation and moved from conceptual to operational.It has been repeatedly demonstrated to provide more reliable, environment-friendly and economically efficient power systems [1].
With the development of advanced information and communication technologies and smart metering infrastructures, the two-way digital communication system now enables the utility company to share information, e.g., the time-dependent electricity price, with users.Moreover, the users can adopt related adjustment strategies to minimize their operation cost, and further to improve the overall energy system performance [2].This is referred to as demand-side management (DSM) [3,4], which has attracted more and more attention.
Usually, candidate control actions that a user can perform include, for example, delaying the start time of shift-able loads, decreasing the demand of power flexible loads, modifying the power schedule of schedulable loads, ESS charging/discharging, selling extra electricity (produced by distributed wind and/or PV generators) to utility companies.By the DSM, a "win-win" situation between users and utility companies is expected [5].

1.
A novel DSM is proposed that considers minimizing the operation cost of each user as well as the discomfort caused by the change of load operation schedule.The DSM can also accommodate different types of user-preferences.

2.
A distributed optimization algorithm based on games theory is proposed to coordinate the users' operation schedules to minimize their own operation costs.Meanwhile, an iteration mechanism is proposed to accelerate the convergence speed.

3.
An MPC framework is implemented to integrate the user operation management model and distributed optimization algorithm.The MPC framework, featuring a rolling up and feedback mechanism, is shown to be able to handle the negative impacts caused by the forecast uncertainty of the RESs output and load demand.
The rest of this paper is organized as follows.Section 2 reviews a selection of representative works that are related.Section 3 presents the model formulation of the smart grid, including the system overall model, the model of components such as the smart loads, the ESS, the pricing policy, and the operation optimization objectives for each user.Section 4 introduces the MPC-based distributed control scheme to coordinate the operations of all users.Numerical simulations are presented in Section 5. Section 6 concludes the paper.

Literature Review
There have been several studies investigating the DSM for smart grid.These methods have their own merits and disadvantages.Next, we review a selection of representative ones.
Since large commercial or industrial centers usually have large energy consumption, Aalami, et al. [12] discussed the DSM for large users.Setlhaolo, et al. [13] further studied the optimal scheduling scheme for typical home appliances.Zhang et al. [14] investigated the energy management strategy of a smart home with the consideration of the demand response (DR).Erdinc [15] evaluated the economic impacts of the ESS, distributed generators (DG) and shift-able loads under different DR strategies.These works consider DSM for either large aggregators or a single user only.Also, discomfort cost as well as different user preferences, altogether, which has greatly limited the application of existing methods.

System Model and Problem Formulation
This study considers a smart grid consisting of a utility company and multiple (M) energy users as shown in Figure 1.For brevity, utility company is abbreviated as utility in the following sections.A DSM control center is deployed in the utility and an EMS is deployed on each user's side.The EMS has the following functions: (i) communicating with the smart meters in the user side and the DSM control center in utility; (ii) predicating the power generations of RESs and load demand; (iii) optimizing operation routines of the dispatchable units of the user based on forecasts, electricity price and so on.The control orders for dispatchable units will be implemented through the programmable logic controller (PLC) in the system.For each user, he/she cannot only purchase electricity energy from the utility but also sell extra power generation back to the utility to make revenue.Moreover, for users with an ESS unit, both the selling time and the amount of selling power can be optimized.This study considers a smart grid consisting of a utility company and multiple () energy users as shown in Figure 1.For brevity, utility company is abbreviated as utility in the following sections.A DSM control center is deployed in the utility and an EMS is deployed on each user's side.The EMS has the following functions: (i) communicating with the smart meters in the user side and the DSM control center in utility; (ii) predicating the power generations of RESs and load demand; (iii) optimizing operation routines of the dispatchable units of the user based on forecasts, electricity price and so on.The control orders for dispatchable units will be implemented through the programmable logic controller (PLC) in the system.For each user, he/she cannot only purchase electricity energy from the utility but also sell extra power generation back to the utility to make revenue.Moreover, for users with an ESS unit, both the selling time and the amount of selling power can be optimized.

Model of Loads
According to the feature of different loads, they are classified as base loads, power flexible loads, shift-able loads and schedulable loads [26,27].Therefore, the total load demand in period  is computed as Equation (1).Also, the forecast error of load demand is assumed to fit the Gaussian distribution [28].
For based load, the operation time and power demand must be satisfied, and cannot be changed.The power demand cannot excess the defined capacity limit.
For flexible loads, the power demand can be curtailed by users to save money or to use in emergency conditions.However, the curtailed power cannot exceed a certain range to avoid the dissatisfaction caused by adjustment actions.In the model,  power flexible appliances are considered, denoted as For shift-able load, the start time can be delayed in a certain time window, but its operation power is fixed and cannot be adjusted.Also, once the task is started, it cannot be stopped before the Prior to describing the system model, it is assumed that the whole control and prediction horizon is divided into T time intervals.Duration of each time interval is ∆t,

Model of Loads
According to the feature of different loads, they are classified as base loads, power flexible loads, shift-able loads and schedulable loads [26,27].Therefore, the total load demand in period t is computed as Equation (1).Also, the forecast error of load demand is assumed to fit the Gaussian distribution [28].
For based load, the operation time and power demand must be satisfied, and cannot be changed.The power demand cannot excess the defined capacity limit.
For flexible loads, the power demand can be curtailed by users to save money or to use in emergency conditions.However, the curtailed power cannot exceed a certain range to avoid the dissatisfaction caused by adjustment actions.In the model, F i power flexible appliances are considered, denoted as For shift-able load, the start time can be delayed in a certain time window, but its operation power is fixed and cannot be adjusted.Also, once the task is started, it cannot be stopped before the task is completed.In the model, A i power flexible appliances are considered, denoted as a ∈ For schedulable loads, both the start time and operation power can be adjusted if the total demand is satisfied before the deadline.However, once the load is started it should be operated till the task is completed.In this model, B i power flexible appliance is considered, denoted as T i,a +t Next, we model the discomfort cost.Specifically, the discomfort cost C i, f (t) by power reduction of power flexible loads f is modelled as a convex non-decreasing function, as shown in Equation (10).
The discomfort cost C i,a (t) by the delay of the start time of shift-able load a is computed as in Equation (11).It indicates that as the delay start time increases, more discomfort penalty cost must be paid.
where δ base i,a (t) is the user's baseline on/off status of appliance a in period t.The discomfort cost C i,b (t) caused by appliances, e.g., b in period t comes from two parts: the cost of power adjustment cost C where δ base i,b (t), l base i,b (t) denote the user's baseline on/off status and power demand of appliance a in period t, respectively.

Model of DERs
The application of distributed energy resources (DERs) including RESs and ESSs has attracted more and more attention due to the increase of energy demand and emissions of greenhouse gases.
Despite the advantages of RESs, the randomness and fluctuation of the RESs power generation make their applications in smart grid difficult.To guarantee safe use of RESs, their power outputs are constrained within a certain range.0 ≤ P i,PV (t) 0 ≤ P i,wind (t) Similarly, the forecast errors of wind and PV generation is described by Gaussian distributions [29,30].
As the most flexible unit in smart grid, the ESS model mainly consists of the system dynamic model, energy capacity model, power charge/discharge model and operation status model.
The initial energy level is denoted as . To effectively handle emergency conditions, the energy level of the ESS is reset to near the half energy capacity at the beginning of each day.Moreover, the operation and maintenance cost of ESS [31] is calculated as follows.

Energy Price Model
The base price p u (t) comes from the cost by thermal generators (we consider RESs all are deployed in the user side, and they are not the main power source) or a simple artificial cost tariff which is used by the utility to impose a proper DSM program.
As the price function is non-decreasing convex, we set the time-dependent base price function to be proportional to the first order derivative of the time-dependent generation cost [32].
where P u (t) is the power generated by the utility, which equals to the summary of the load demands for all users.
To promote the use of DERs as well as reduce the negative impacts of randomness of the RESs outputs, the selling electricity price p i,s (t) for users is set lower than the base price [31].Meanwhile, according to the rate-of-return regulations [33], the buying electricity price p i,b (t) for users is set higher than the base price. where The real-time electricity price mechanism defined in Equations ( 25) and ( 26) guarantees the user's benefits, that is, when the base price is high the selling price is also high.Meanwhile, the difference between the buying price and the selling price promote the users to consume the power generated by themselves, and to encourage the utility to buy energy from the users.
Furthermore, to reduce the negative impacts introduced by the intermittent and random RESs outputs to the power system, users are not allowed to sell energy back to the utility if the forecasted RESs generation cannot supply the energy demand of the user completely.
Without loss of the generality, we set the operation cost C u (P u (t)) as, where c 1 , c 2 are constants.

Power Interaction Model
Because of Equations ( 25) and ( 26), the buying/selling energy cost model for each user is not a continuous model, becoming a mixed logical dynamic (MLD) model [14].
The power interaction between users and utility is modelled as follows.
0 ≤ P i,gI (t) ≤ P max i,gI δ i,gI (t) (28) Equations ( 28)-(39) indicate that the buying and selling power of each user cannot exceed its capacity.Equation (30) indicates that a user cannot purchase and sell power at the same time.
The power flow in each user is shown as follows.

Cost Model
The overall cost Ψ i (t) as shown in Equation (33) consists of energy billing cost and the discomfort cost.The billing cost includes the energy purchasing cost, the energy selling revenue and the ESS operation cost.The discomfort cost includes the penalty cost of curtailing flexible loads, the delaying penalty cost of shift-able loads start time, and the adjustment penalty cost of schedulable loads.
Subject to (1)- (32) The cost Ψ u for the utility includes the fuel consumption cost and the buyback cost from the users, which is computed as follows.
According to Equations ( 33) and ( 34), the cost optimization model for users is a mixed integer programming (MIP) problem, it is non-convex.The scheme of each user is affected by other users since the electricity price is determined by the total load demand of the smart grid.To solve the centralized optimization problem, all users' specifications and preferences are required, which is often difficult to obtain in practice, considering the privacy protection and computation burden.Meanwhile, each user hopes to minimize his/her own operation cost.This is difficult to achieve in a centralized way.Alternatively, a reasonable way is to allow each user to optimize his/her own operation schedule in a distributed way

MPC Based Distributed User Energy Management Strategy
This section first presents the distributed optimization algorithm used to solve the cost optimization model and verifies the equilibrium of the proposed distributed algorithm.Second, the detailed process of the MPC-based distributed optimization method is described.

Distributed User Energy Scheduling Optimization
Game theory is adopted to capture the competition among users.All users are considered to be rational entities that are only interested in minimizing their own cost.Specifically, the game theory-based method is described as follows.
(i) Players: all users (M users) in the smart grid.(ii) Strategies: each user i ∈ M selects its strategy by scheduling the dispatchable units (smart loads and ESS) to minimize his/her own cost.(iii) Payoffs: the payoff P(y i , y − i ) for user i comprises two parts, see Equation ( 35): the actual operation cost Ψ k i described in Equation ( 33) at iteration k, and the penalty cost Φ k i caused by large fluctuant of the operation routine in two successive iterations.
Please note that we use X k i to denote X i in the k th iteration, where X can represent all the decision variables and objective functions in this paper.
Different from [21], the power varying of dispatchable units in two successive iterations is considered in this study.This is to avoid the situation that little difference for the power interaction but a large difference for the payoff in two successive iterations.Furthermore, Equation ( 36) can effectively cope with the power varying introduced by ESS.The detail distributed optimization process is shown in Table 1.
Energies 2019, 12, 426 9 of 19 When the changes of the user's payoff and power routine within the preset threshold in two successive iterations, the distributed optimization will be stopped and the equilibrium is achieved.
According to the distributed algorithm in Table 1, the user only needs to send its total power schedule to the utility control center and receive the electricity price over the control horizon.No communication is required among different users.Hence, the privacy of each user is protected effectively.
Table 1.Algorithm for distributed optimization algorithm for the energy scheduling of users.

Begin
Initialize the iteration counter k = 0; Initialize P 0 i,gI (t), P 0 i,gO (t) according to the base load scheme and forecasts; Send P 0 i,gI (t), P 0 i,gO (t) to the utility control center; Repeat update the received buying and selling electricity price p i,b (t), p i,s (t) from the utility control center; minimize the payoff shown in Equation ( 35) and calculate the newly operation schedule for all dispatchable units.send newly buying/selling power schedule P k+1 i,gI (t), P k+1 i,gO (t) to the utility control center; where ∇C opt , ∇L sh f , ∇L sch , ∇P ESS , ∇P grid are the total cost difference, shift-able load power schedule difference, schedulable load power schedule difference, ESS power schedule difference and the buying/selling power schedule difference in two consecutive iterations, respectively.
In the distributed optimization algorithm, λ k i is a user-defined parameter which affects the algorithm convergence.In this paper, λ k i is defined based on the following considerations.First, the change of operation routines becomes small as the algorithm progresses.Second, the greater the change, the higher the penalty cost.Third, a small penalty coefficient is used for large power consuming users while a large penalty coefficient is used for small power consuming users.This is because high consuming users tend to care more about the electricity price.Correspondingly, the change of their operations impacts more on the electricity price.Specifically, λ k i is computed as follows.
The parameter π is an auxiliary coefficient to adjust the convergence speed of the distributed optimization algorithm, and is set to 0.5.
Nash equilibrium refers to the best solution to a non-cooperative game, that is, no player can gain more credits by changing his/her own strategy only [34].In this paper, NE consists of a set of strategies y The game theory-based distributed user energy management strategy proposed in this paper satisfies the second fundamental welfare theorem [35] of the Walrasian equilibrium theory [36], this is because the base electricity price mechanism and game theory framework (p u , {P i } M i=1 ) meet the form of the Walrasian equilibrium theory.p u , P i are the base electricity price and power schedule vector, respectively, and are defined as follows.

MPC-Based Control Framework
In recent years, the MPC framework has attracted many attention in power system energy management [26,31] due to its feedback and rolling horizon mechanism being suitable to handle varying information and stochastic parameters.
In this section, we formulate an MPC-based user energy management problem, whose solution yields a trajectory of control actions and states into the future that satisfies the dynamics and constraints of the user and smart grid operation while optimizing some given criteria.Only the first sample of the control actions is implemented, and then the horizon is shifted.
The detailed coordination procedure for MPC-based distributed optimization is as follow: (i) At the end of period τ, the EMS of user i obtains the updated state of its related dispatchable units, including the energy level of ESS, E i,ESS (τ), the operation status of shift-able loads, δ i,a (τ), the operation status, δ i,b (τ) and power demand, l i,b (τ) of the schedulable loads.Then the EMS calculates the forecasted data of load demand, PV generation and wind production from period ).The first sample of the control sequence is then sent to local controllers.(iii) At the beginning of τ + 1, only the first sample of the control sequence is implemented.The insufficient power caused by forecast errors is compensated by the utility.On the contrary, the excess power will be sold back to the utility with a lower price.Finally, the EMS updates the parameters and forecast model with new data.(iv) Go to step (i) until the end of the simulation.
The designed control variables for the MPC controller is P i .The forecast error is FE i which contains (2 + F i ) × T variables.
It is expected that, by the MPC-based distributed optimization, an optimal plan can be obtained to potentially compensate the forecast errors.

Simulation and Results
In this section, we first compare our proposed MPC-based distributed DSM (DMPC) strategy with the traditional day-ahead programming-based distributed DSM (DDA) strategy [32].Then, the impact of the penalty cost term of the shift-able and schedulable loads of the system operation routine is discussed.Finally, we compare our parallel distributed optimization framework with the sequential distributed optimization framework [16].

Experiment Setup
We consider a smart grid with four users.Each user has wind and PV generators, ESS unit and different kinds of smart loads.Related parameters like the capacity of PV and wind generators, the maximum base load demand and the maximum power interaction for each user is listed in Table 2.The control and prediction horizon is T = 24 h, the duration of each time interval is 1 h.
The historical data of base load, wind and PV power generation for each user is collected and modified from Belgium's transmission system [37] as shown in Figure 2. The basic operation schedule is as follows.The shift-able loads and schedulable loads work as they are planed, no power is curtailed and no ESS is integrated.Since no specialized data set for power flexible loads, we only consider a total load demand for power flexible appliances, that is, F = 1.
Also, we assume that the demand of flexible load is 30% of the base load in each period.The maximum curtailment ratios for the four users are preset as [0.5, 0.4, 0.35, 0.3], meanwhile, the discomfort penalty coefficients for curtailing the flexible loads are set as [2.5, 1.8, 2.2, 2].
There are 8 shift-able appliances for each user, the power demand, available operation time window, time needed to complete the task and the discomfort penalty coefficient are all listed in Table 3. Different discomfort penalty coefficients are set for different users since the user preferences are usually different.Similarly, the parameters for schedulable loads are shown in Table 4. Since the schedulable loads can adjust both the start time and operation power, two penalty cost coefficients are adopted.
The parameters of the ESS are listed in Table 5.The operation and maintenance cost for each unit kWh is 0.1 $.The charging/discharging efficiency for each ESS is 0.95.The initial energy level for each ESS is half of its maximum energy level.
According to the rate-of-return regulations, we set the retail purchasing electricity price for each user at 1.2 times of the base price, and the retail selling price for each user is 0.8 times the base price.Furthermore, we set the extra power purchasing price in the real-time power compensation stage at 3 times the base price and 0.5 times of the base price to sell the extra power generation, due to the fact that the real-time adjustment of the utility needs more cost than a prescheduled plan.The cost parameters for the utility is c 1 = 0.000066 $/kWh 2 , c 2 = 0.18 $/kWh.The capacity of the utility is 5 MW, and the minimum power output is 50 kW.If the total power demand of all users is less than 50 kW, the utility will only act as a servicer.In other words, the energy coordination should be performed among users themselves.
The parameters for the stopping criteria of the distributed optimization algorithm in Table 1 is set as follows.

Simulation Results
All simulations are run on a laptop with Intel(R) Core (TM) i5-3210M CPU @2.5 GHz and 8.00 GB memory.The ILOG's CPLEX v.12 optimization solver is utilized for solving the MIP models, MATLAB 2013a and YALMIP toolbox [38] are used for linking the CPLEX solver and computing the optimization model.

Results of the DMPC Strategy and DDA Strategy
First, the DDA strategy is introduced briefly.It is a traditional two-stage-based open-loop distributed energy management algorithm [15,32], including the scheduling stage and the real-time power compensation stage.The detailed process is shown as follow.i).At the scheduling stage, the EMS determines the operation schedule of the smart load appliances and the ESS over the control horizon by implementing the distributed optimization algorithm of Table 1 at the beginning of the day with the forecasts of RESs generation and load demand.

Simulation Results
All simulations are run on a laptop with Intel(R) Core (TM) i5-3210M CPU @2.5 GHz and 8.00 GB memory.The ILOG's CPLEX v.12 optimization solver is utilized for solving the MIP models, MATLAB 2013a and YALMIP toolbox [38] are used for linking the CPLEX solver and computing the optimization model.

Results of the DMPC Strategy and DDA Strategy
First, the DDA strategy is introduced briefly.It is a traditional two-stage-based open-loop distributed energy management algorithm [15,32], including the scheduling stage and the real-time power compensation stage.The detailed process is shown as follow.
(i) At the scheduling stage, the EMS determines the operation schedule of the smart load appliances and the ESS over the control horizon by implementing the distributed optimization algorithm of Table 1 at the beginning of the day with the forecasts of RESs generation and load demand.The control sequence sent to the controllers of all dispatchable units should be implemented strictly.
(ii) At the real-time power compensation stage, for each user the insufficient power will be provided by the utility company at a higher price and the extra power generation will be sold at a lower price back to the utility.
The operations for the DMPC and DDA at the real-time power compensation stage are the same, aiming to reduce the negative impacts introduced by forecast errors and randomness of the RESs output.
The routines of power generation for the utility under the DMPC and DDA are shown in Figure 3. Please note that the basic power refers to the utility generation where no optimization is implemented and no ESS unit is used.The scheduling power refers to the utility generation based on the forecasts at the scheduling stage.The real-time power refers to the utility generation adjusted with the real-time data at the real-time power compensation stage.
Energies 2017, 10, x FOR PEER REVIEW 13 of 20 The control sequence sent to the controllers of all dispatchable units should be implemented strictly.ii).At the real-time power compensation stage, for each user the insufficient power will be provided by the utility company at a higher price and the extra power generation will be sold at a lower price back to the utility.
The operations for the DMPC and DDA at the real-time power compensation stage are the same, aiming to reduce the negative impacts introduced by forecast errors and randomness of the RESs output.
The routines of power generation for the utility under the DMPC and DDA are shown in Figure 3. Please note that the basic power refers to the utility generation where no optimization is implemented and no ESS unit is used.The scheduling power refers to the utility generation based on the forecasts at the scheduling stage.The real-time power refers to the utility generation adjusted with the real-time data at the real-time power compensation stage.The peak of the basic power is 3.459 × 10 3 kW in the 10th hour of the first day, and its average value is 2.032 × 10 3 kW.Thus, the peak-to-average ratio is 1.702.
The power peak of the DMPC strategy at the scheduling stage is 2.934 × 10 3 kW at the 11th hour of the first day, and its average power is 1.945 × 10 3 kW.Thus, the peak-to-average ratio is 1.510.Meanwhile, considering the forecast uncertainties, the real-time power generation of the utility at the real-time stage is different from that at the scheduling stage.The power peak of the DMPC strategy at real-time power compensation stage is 2.945 × 10 3 kW at the 11th hour of the first day, and its average power is 1.9455 × 10 3 kW, and hence its peak-to-average ratio is 1.514.Such results indicate that by the DMPC strategy, the impact caused by forecast uncertainties can be well handled.The peak-to-average ratio is reduced about 11.3% compared to the basic situation, and the peak power is reduced about 15% (more than 500 kW).This demonstrates that the DMPC strategy can effectively save social welfare for the society.
The power peak of the DDA strategy at the scheduling stage is 3.292 × 10 3 kW at the 11th hour of the first day, and its average power is 2.018 × 10 3 kW.Thus, the peak-to-average ratio is 1.63.This shows that the performance of the DDA strategy at the scheduling stage is inferior to that of the DMPC strategy.Moreover, the power peak of the DDA strategy at the real-time power compensation stage is 3.307 × 10 3 kW at the 11th hour of the first day, and its average power is 2.029 × 10 3 kW.The The peak of the basic power is 3.459 × 10 3 kW in the 10th hour of the first day, and its average value is 2.032 × 10 3 kW.Thus, the peak-to-average ratio is 1.702.
The power peak of the DMPC strategy at the scheduling stage is 2.934 × 10 3 kW at the 11th hour of the first day, and its average power is 1.945 × 10 3 kW.Thus, the peak-to-average ratio is 1.510.Meanwhile, considering the forecast uncertainties, the real-time power generation of the utility at the real-time stage is different from that at the scheduling stage.The power peak of the DMPC strategy at real-time power compensation stage is 2.945 × 10 3 kW at the 11th hour of the first day, and its average power is 1.9455 × 10 3 kW, and hence its peak-to-average ratio is 1.514.Such results indicate that by the DMPC strategy, the impact caused by forecast uncertainties can be well handled.The peak-to-average ratio is reduced about 11.3% compared to the basic situation, and the peak power is reduced about 15% (more than 500 kW).This demonstrates that the DMPC strategy can effectively save social welfare for the society.
The power peak of the DDA strategy at the scheduling stage is 3.292 × 10 3 kW at the 11th hour of the first day, and its average power is 2.018 × 10 3 kW.Thus, the peak-to-average ratio is 1.63.This shows that the performance of the DDA strategy at the scheduling stage is inferior to that of the DMPC strategy.Moreover, the power peak of the DDA strategy at the real-time power compensation stage is 3.307 × 10 3 kW at the 11th hour of the first day, and its average power is 2.029 × 10 3 kW.The peak-to-average ratio is thus 1.63.Compared with the DMPC strategy, the DDA strategy is inferior, Energies 2019, 12, 426 reducing only 4.23% of the peak-to-average ratio, and 4.4% of the peak power (about 150 kW) for the utility company The reason for the inferior performance of DDA compared to the DMPC is that in DDA both the forecast and the operation schedule are made at the beginning of the day; however, the forecast and the operation schedule of the DMPC are adaptively adjusted every hour according to the newly updated information.
Figure 4 shows the power adjustment at the real-time stage under the DMPC and DDA strategy.From the figure, we can see that the amount of power adjustment with the DMPC strategy is much smaller than that with the DDA strategy.That is, the DDA is inferior to the DMPC strategy.
Energies 2017, 10, x FOR PEER REVIEW 14 of 20 peak-to-average ratio is thus 1.63.Compared with the DMPC strategy, the DDA strategy is inferior, reducing only 4.23% of the peak-to-average ratio, and 4.4% of the peak power (about 150 kW) for the utility company The reason for the inferior performance of DDA compared to the DMPC is that in DDA both the forecast and the operation schedule are made at the beginning of the day; however, the forecast and the operation schedule of the DMPC are adaptively adjusted every hour according to the newly updated information.
Figure 4 shows the power adjustment at the real-time stage under the DMPC and DDA strategy.From the figure, we can see that the amount of power adjustment with the DMPC strategy is much smaller than that with the DDA strategy.That is, the DDA is inferior to the DMPC strategy.In addition to the comparison from the view of the utility, we next discuss the performance of the DMPC strategy from the view of users.The operation routines for each user and their dispatchable units with the DMPC strategy at the scheduling stage is illustrated in Figure 5.
User 1 buys 4.27 × 10 4 kWh power energy from the utility while sells no power back to the utility company as power generated by RESs is always less than the load demand.User 1 spends 4.314 × 10 4 $ purchasing energy from the utility.The ESS charges 1.248 × 10 3 kWh and discharges 1.127 × 10 3 kWh power over the whole time horizon.The overall charged energy is a little larger than the discharged energy.This is because the power efficiency of the ESS is not 100%.The operation and maintenance cost of the ESS cost is 237.5 $.Meanwhile, user 1 curtails 237 kWh power flexible load demand, which results in a penalty cost of 533.8 $.The starting time of shift-able appliances delayed about 10 hours, resulting in 0.39 $ penalty cost.The amount of energy adjustment for schedulable loads is 2.63 × 10 3 kWh, resulting in 54.124 $ penalty cost.Also, no schedulable appliance chooses to shift its operation time window.
User 2 buys 3.88 × 10 4 kWh power energy from the utility while sells no power back to the utility company.3.932 × 10 4 $ is spent to purchase energy from the utility.The ESS charges 1.239 × 10 3 kWh and discharges 1.117 ×10 3 kWh power over the horizon.The operation and maintenance cost of the ESS cost is 235.61 $.Meanwhile, user 2 curtails 88.01 kWh power flexible load demand, resulting in 121.26 $ penalty cost.Since the discomfort penalty cost of user 2 is lower than that of user 1, its penalty cost for curtailing loads is more effective than user 1.The shift-able appliances delay about 7 hours, and the penalty cost for the delay is 0.335 $.Like user 1, no schedulable appliance in user 2 chooses to shift its operation time window.The amount of energy adjustment for schedulable loads is 2.15 × 10 3 kWh, resulting in 47.76 $ penalty cost.In addition to the comparison from the view of the utility, we next discuss the performance of the DMPC strategy from the view of users.The operation routines for each user and their dispatchable units with the DMPC strategy at the scheduling stage is illustrated in Figure 5.
User 1 buys 4.27 × 10 4 kWh power energy from the utility while sells no power back to the utility company as power generated by RESs is always less than the load demand.User 1 spends 4.314 × 10 4 $ purchasing energy from the utility.The ESS charges 1.248 × 10 3 kWh and discharges 1.127 × 10 3 kWh power over the whole time horizon.The overall charged energy is a little larger than the discharged energy.This is because the power efficiency of the ESS is not 100%.The operation and maintenance cost of the ESS cost is 237.5 $.Meanwhile, user 1 curtails 237 kWh power flexible load demand, which results in a penalty cost of 533.8 $.The starting time of shift-able appliances delayed about 10 h, resulting in 0.39 $ penalty cost.The amount of energy adjustment for schedulable loads is 2.63 × 10 3 kWh, resulting in 54.124 $ penalty cost.Also, no schedulable appliance chooses to shift its operation time window.
User 2 buys 3.88 × 10 4 kWh power energy from the utility while sells no power back to the utility company.3.932 × 10 4 $ is spent to purchase energy from the utility.The ESS charges 1.239 × 10 3 kWh and discharges 1.117 ×10 3 kWh power over the horizon.The operation and maintenance cost of the ESS cost is 235.61 $.Meanwhile, user 2 curtails 88.01 kWh power flexible load demand, resulting in 121.26 $ penalty cost.Since the discomfort penalty cost of user 2 is lower than that of user 1, its penalty cost for curtailing loads is more effective than user 1.The shift-able appliances delay about 7 h, and the penalty cost for the delay is 0.335 $.Like user 1, no schedulable appliance in user 2 chooses to shift its operation time window.The amount of energy adjustment for schedulable loads is 2.15 × 10 3 kWh, resulting in 47.76 $ penalty cost.
User 3 buys 5.0 × 10 4 kWh power energy from the utility, and sells 15 kWh power back to the utility company.The money spent to purchase energy from the utility is 5.191 × 10 4 $ while the revenue made by selling energy back to the utility is 4.2 $.The ESS charges 1.00 × 10 3 kWh and discharges 0.916 × 10 3 kWh power over the horizon.The operation and maintenance cost of the ESS is 191.89 $.Meanwhile, user 3 curtails 92.182 kWh power flexible load demand, resulting in 117.85 $ penalty cost.Compared with user 2, the power curtailment action for user 3 is more effective.This is because that both the penalty cost coefficient and the curtailed power of user 2 are lower than those of user 3 but the penalty cost of user 2 is higher than user 3. The shift-able appliances delay about 8 h, and the penalty cost is 0.265 $.Again, no schedulable appliance in user 3 chooses to shift the operation time window.The amount of energy adjustment for schedulable loads is 1.985 × 10 3 kWh, resulting in 45.125 $ penalty cost.
User 4 buys 5.526 × 10 4 kWh power energy from the utility while sells no power back to the utility company.5.594 × 10 4 $ is spent to purchase energy from the utility.The ESS charges 0.887 × 10 3 kWh and discharges 0.7935 × 10 3 kWh power over the horizon.The operation and maintenance cost of the ESS is 191.89 $.User 4 curtails 90.51 kWh power flexible load demand, resulting in 135.578 $ penalty cost.The shift-able appliances delay about 14 h, resulting in 0.46 $ penalty cost.No schedulable appliance in user 4 chooses to shift its operation time window.The amount of energy adjustment for schedulable loads is 1.508 × 10 3 kWh, resulting in 31.98 $ penalty cost.
Though the rolling horizon and feedback mechanism can reduce the negative impact introduced by the random and intermittent outputs of RESs to some extent, the forecast uncertainties cannot be completely due to the forecast model is imperfect.Therefore, user 1 must purchase 67.19 kWh energy from the utility at a high electricity price with a total cost 169.97$.Correspondingly, user 1 sells 55.48 kWh energy back to the utility at a low electricity price, making revenue of 22.67 $.User 2 must purchase 72.03 kWh energy from the utility at a total cost 182.14$.Correspondingly, user 2 sells 45.41 kWh energy back to the utility, making revenue of 18.3 $.Due to the penetration level of RESs output in user 3 is the highest, the influence of the forecast uncertainty is the highest.User 3 has to purchase more energy, i.e., 93.72 kWh from the utility with a total cost of 231.62 $.Correspondingly, user 3 sells 72.5 kWh energy back to the utility, making revenue of 27.83 $.User 4 purchases 71.53 kWh energy from the utility with a total cost of 175.8 $.Correspondingly, user 4 sells 78.66 kWh energy back to the utility, making revenue of 31.85 $.
Energies 2017, 10, x FOR PEER REVIEW 15 of 20 User 3 buys 5.0 × 10 4 kWh power energy from the utility, and sells 15 kWh power back to the utility company.The money spent to purchase energy from the utility is 5.191 × 10 4 $ while the revenue made by selling energy back to the utility is 4.2 $.The ESS charges 1.00 × 10 3 kWh and discharges 0.916 × 10 3 kWh power over the horizon.The operation and maintenance cost of the ESS is 191.89 $.Meanwhile, user 3 curtails 92.182 kWh power flexible load demand, resulting in 117.85 $ penalty cost.Compared with user 2, the power curtailment action for user 3 is more effective.This is because that both the penalty cost coefficient and the curtailed power of user 2 are lower than those of user 3 but the penalty cost of user 2 is higher than user 3. The shift-able appliances delay about 8 hours, and the penalty cost is 0.265 $.Again, no schedulable appliance in user 3 chooses to shift the operation time window.The amount of energy adjustment for schedulable loads is 1.985 × 10 3 kWh, resulting in 45.125 $ penalty cost.
User 4 buys 5.526 × 10 4 kWh power energy from the utility while sells no power back to the utility company.5.594 × 10 4 $ is spent to purchase energy from the utility.The ESS charges 0.887 × 10 3 kWh and discharges 0.7935 × 10 3 kWh power over the horizon.The operation and maintenance cost of the ESS is 191.89 $.User 4 curtails 90.51 kWh power flexible load demand, resulting in 135.578 $ penalty cost.The shift-able appliances delay about 14 hours, resulting in 0.46 $ penalty cost.No schedulable appliance in user 4 chooses to shift its operation time window.The amount of energy adjustment for schedulable loads is 1.508 × 10 3 kWh, resulting in 31.98 $ penalty cost.
Though the rolling horizon and feedback mechanism can reduce the negative impact introduced by the random and intermittent outputs of RESs to some extent, the forecast uncertainties cannot be completely due to the forecast model is imperfect.Therefore, user 1 must purchase 67.19 kWh energy from the utility at a high electricity price with a total cost 169.97$.Correspondingly, user 1 sells 55.48 kWh energy back to the utility at a low electricity price, making revenue of 22.67 $.User 2 must purchase 72.03 kWh energy from the utility at a total cost 182.14$.Correspondingly, user 2 sells 45.41 kWh energy back to the utility, making revenue of 18.3 $.Due to the penetration level of RESs output in user 3 is the highest, the influence of the forecast uncertainty is the highest.User 3 has to purchase more energy, i.e., 93.72 kWh from the utility with a total cost of 231.62 $.Correspondingly, user 3 sells 72.5 kWh energy back to the utility, making revenue of 27.83 $.User 4 purchases 71.53 kWh energy from the utility with a total cost of 175.8 $.Correspondingly, user 4 sells 78.66 kWh energy back to the utility, making revenue of 31.85 $.The operation costs for the DPMC and DDA strategies at the scheduling and real-time power compensation stages are shown in Table 6.From the results, we can find that the operation cost of the DMPC and DDA is nearly the same at the scheduling stage while different at the real-time stage (the cost by the DDA is higher than that by the DMPC).This is because the forecasted electricity prices for the two strategies at the scheduling stage are similar.However, at the real-time stage, the closed-loop-based DMPC can adjust control actions with newly updated forecasts while the open-loop-based DDA cannot.This leads DDA to cost more.6 presents the utility power generation routine with and without considering the penalty cost term.We use the term no penalty to refer to the case that no penalty cost is considered.Under the no penalty case, the peak power is 2.898 × 10 3 kW, and the average power is 1.872 × 10 3 kW.The peak-to-average ratio is thus 1.548.Though the peak power and the average power are both lower than the case considering penalty cost, the peak-to-average for no penalty case is higher than penalty case.This clearly shows that penalty cost term has important impact on the utility.
The operation costs for the DPMC and DDA strategies at the scheduling and real-time power compensation stages are shown in Table 6.From the results, we can find that the operation cost of the DMPC and DDA is nearly the same at the scheduling stage while different at the real-time stage (the cost by the DDA is higher than that by the DMPC).This is because the forecasted electricity prices for the two strategies at the scheduling stage are similar.However, at the real-time stage, the closedloop-based DMPC can adjust control actions with newly updated forecasts while the open-loopbased DDA cannot.This leads DDA to cost more.6 presents the utility power generation routine with and without considering the penalty cost term.We use the term no penalty to refer to the case that no penalty cost is considered.Under the no penalty case, the peak power is 2.898 × 10 3 kW, and the average power is 1.872 × 10 3 kW.The peakto-average ratio is thus 1.548.Though the peak power and the average power are both lower than the case considering penalty cost, the peak-to-average for no penalty case is higher than penalty case.This clearly shows that penalty cost term has important impact on the utility.

Comparison of the Parallel and Sequential Optimization Algorithm
This section compares the performance of the sequential distributed optimization (SDO) algorithm [16] with the parallel distributed optimization (PDO) algorithm described in Table 1.The users apply the SDO algorithm to update their operation schedule sequentially, namely, the electricity price has to change after the optimization scheduling.The first day's data is used to present the comparison results as shown in Figures 7 and 8.
This section compares the performance of the sequential distributed optimization (SDO) algorithm [16] with the parallel distributed optimization (PDO) algorithm described in Table 1.The users apply the SDO algorithm to update their operation schedule sequentially, namely, the electricity price has to change after the optimization scheduling.The first day's data is used to present the comparison results as shown in Figures 7 and 8.  Figure 7 shows that the SDO algorithm needs to run 11 iterations to achieve an equilibrium for a user.Since there are four users, so the total required iterations are 44.However, the PDO algorithm reaches an equilibrium after 12 iterations.This indicates that the PDO algorithm converges much faster than the SDO algorithm.Moreover, it can be observed from Figure 8 that the peak power of the PDO algorithm is only a bit higher than the SDO algorithm.This further demonstrates the advantage of the PDO over the SDO algorithm.

Conclusions
In this study, an MPC-based distributed demand side management strategy is proposed to provide optimal control actions for energy users in a smart grid.The users are equipped with renewable energy source (RES) generators, energy storage system (ESS) units and different kinds of smart loads.For each user, an energy management system (EMS) is used to determine the operation This section compares the performance of the sequential distributed optimization (SDO) algorithm [16] with the parallel distributed optimization (PDO) algorithm described in Table 1.The users apply the SDO algorithm to update their operation schedule sequentially, namely, the electricity price has to change after the optimization scheduling.The first day's data is used to present the comparison results as shown in Figures 7 and 8.  Figure 7 shows that the SDO algorithm needs to run 11 iterations to achieve an equilibrium for a user.Since there are four users, so the total required iterations are 44.However, the PDO algorithm reaches an equilibrium after 12 iterations.This indicates that the PDO algorithm converges much faster than the SDO algorithm.Moreover, it can be observed from Figure 8 that the peak power of the PDO algorithm is only a bit higher than the SDO algorithm.This further demonstrates the advantage of the PDO over the SDO algorithm.

Conclusions
In this study, an MPC-based distributed demand side management strategy is proposed to provide optimal control actions for energy users in a smart grid.The users are equipped with renewable energy source (RES) generators, energy storage system (ESS) units and different kinds of smart loads.For each user, an energy management system (EMS) is used to determine the operation Figure 7 shows that the SDO algorithm needs to run 11 iterations to achieve an equilibrium for a user.Since there are four users, so the total required iterations are 44.However, the PDO algorithm reaches an equilibrium after 12 iterations.This indicates that the PDO algorithm converges much faster than the SDO algorithm.Moreover, it can be observed from Figure 8 that the peak power of the PDO algorithm is only a bit higher than the SDO algorithm.This further demonstrates the advantage of the PDO over the SDO algorithm.

Conclusions
In this study, an MPC-based distributed demand side management strategy is proposed to provide optimal control actions for energy users in a smart grid.The users are equipped with renewable energy source (RES) generators, energy storage system (ESS) units and different kinds of smart loads.For each user, an energy management system (EMS) is used to determine the operation scheme of dispatchable units and the interaction with the utility (i.e., purchasing power from or selling power back to the utility).Experimental results show that the proposed MPC-based distributed DSM (DMPC) strategy enables users to optimally control their own subsystems individually, and coordinate them properly when necessary.Moreover, its performance is demonstrated to be better than the traditional day-ahead programming-based distributed DSM (DDA) strategy [32].In addition, the proposed parallel distributed optimization method is also demonstrated to be superior to the sequential distributed optimization algorithm.
With respect to the future work, we would like to analyze the games among the users and the utility.Also, the convergence property of the parallel distributed optimization method needs to be theatrically analyzed.

Figure 1 .
Figure 1.Power and information flows among users and the utility.

Figure 1 .
Figure 1.Power and information flows among users and the utility.
po i,b (t), and the cost induced by the delay of start time C tm i,b (t).

Figure 2 .
Figure 2. PV and wind generation, base load demand and the basic buy/selling power for each user.

Figure 2 .
Figure 2. PV and wind generation, base load demand and the basic buy/selling power for each user.

Figure 3 .
Figure 3. Utility generation for DMPC and DDA strategies in different cases.

Figure 3 .
Figure 3. Utility generation for DMPC and DDA strategies in different cases.

Figure 4 .
Figure 4. Power adjusted for DMPC and DDA strategies at real-time stage.

Figure 4 .
Figure 4. Power adjusted for DMPC and DDA strategies at real-time stage.

Figure 5 .
Figure 5. Power routine for each user and each dispatchable units with DMPC strategy.

Figure 5 .
Figure 5. Power routine for each user and each dispatchable units with DMPC strategy.

Figure 6 .
Figure 6.Power generation of the utility with or without considering penalty cost terms.

Figure 6 .
Figure 6.Power generation of the utility with or without considering penalty cost terms.

Figure 7 .
Figure 7. Utility fuel cost and BESS power change for the PDO and SDO algorithms.

Figure 8 .
Figure 8. Utility power of the first day for PDO and SDO algorithms.

Figure 7 .
Figure 7. Utility fuel cost and BESS power change for the PDO and SDO algorithms.

Figure 7 .
Figure 7. Utility fuel cost and BESS power change for the PDO and SDO algorithms.

Figure 8 .
Figure 8. Utility power of the first day for PDO and SDO algorithms.

Figure 8 .
Figure 8. Utility power of the first day for PDO and SDO algorithms.

Table 1
is solved individually for each user, reaching a Nash Equilibrium (NS), i.e., (p * u , P * i M i=1

Table 2 .
Power parameters for each user.

Table 3 .
Parameter of shift-able loads.

Table 5 .
Parameters of ESS.

Table 5 .
Parameters of ESS.

Table 6 .
Operation costs for each user with different strategies.

Table 6 .
Operation costs for each user with different strategies.