Dynamic Optimization of Combined Cooling , Heating , and Power Systems with Energy Storage Units

In this paper, a combined cooling, heating, and power (CCHP) system with thermal storage tanks is introduced. Considering the plants’ off-design performance, an efficient methodology is introduced to determine the most economical operation schedule. The complex CCHP system’s state transition equation is extracted by selecting the stored cooling and heating energy as the discretized state variables. Referring to the concept of variable cost and constant cost, repeated computations are saved in phase operating cost calculations. Therefore, the most economical operation schedule is obtained by employing a dynamic solving framework in an extremely short time. The simulation results indicated that the optimized operating cost is reduced by 40.8% compared to the traditional energy supply system.


Introduction
Combined cooling, heating, and power (CCHP) systems follow the principle of cascade utilization of energy with high energy efficiency and have become a major research focus [1][2][3][4][5][6].It is verified that operation optimization can improve their performance to some extent [7][8][9][10].However, fluctuating energy demands might not always fall within the high efficiency region of CCHP systems [11,12].Satisfactory operation cannot be achieved easily without energy storage units, which can facilitate high-efficiency CCHP system operation and increase the energy conservation rate by approximately 21% [13].Meanwhile, the introduction of energy storage units makes CCHP system optimization very difficult [14,15].
The most common operating strategy is based on following the electric loads or following the thermal loads [16,17].Current studies solve the optimal operating strategy of CCHP systems with storage units in the following way: the outputs of different pieces of equipment in each stage are taken as equivalent optimization variables, which are limited by the plant capacity and energy balance.After setting an objective function, various kinds of algorithms are applied with the objective of seeking the optimal operating schedule.The current studies can be separated into the following two general categories based on their algorithms.
Nearly half of the published research papers employ intelligent optimization algorithms, which are mainly genetic algorithms (GAs) and particle swarm optimization (PSO) algorithms, to solve the CCHP system operation optimization problem.Wang et al. employed GA to optimize an electric load-following operating strategy of a CCHP system [18].Zeng et al. employed GA to determine the optimal operating solution of a CCHP system combined with ground source heat pumps [19].Wang et al. built a two-time scale optimized model of a CCHP system, and an improved PSO algorithm is proposed [20].Considering the co-optimization issue of CCHP system with ice-storage air-conditioners, Bao et al. introduced the Improved PSO algorithm to the solution of the day-ahead operating schedule [21].
Numerous examples of linear programming (LP) applications to CCHP system operation optimization can also be found.Shaneb et al. purposed an optimal online operation of residential CCHP systems using LP [22].Bischia et al. built a detailed nonlinear CCHP system model, which was piecewise approximated as several linear models, and introduced mixed-integer linear programming (MILP) to optimize the operating schedule [23].Gu et al. built a prediction control model of a CCHP system; its prediction errors and system deviations were corrected online by rolling optimization, and the dispatch schedule in each step of the rolling optimization was determined by using MILP [24].Luo et al. proposed two-stage optimization and control structure of the CCHP system, and employed MILP to search the operating schedule [25].
GA, PSO, and MILP can easily optimize the CCHP system operation as long as storage units are not introduced.However, the operation optimization of CCHP systems with storage units is more complex than that of systems without storage units [26], and the methods mentioned above cannot handle the optimization of such systems adequately.Difficulties arise not only from the numerous optimization variables corresponding to each stage, but also because of the correlation between adjacent stages due to the existence of storage units [27].To be more specific, the energy storage state of each stage depends on the energy supply of the previous stage, whereas the energy supply of each stage is influenced by its current energy storage.To describe the correlation between the adjacent stages, complex constraints must be applied.
Hence, it is not certain that GA or PSO can provide optimal solutions.This conclusion is derived from the fact that different results are obtained for the same problem when they are applied repeatedly.MILP is improved to be efficient when the optimization model is considered to be linear.However, to the best of our knowledge, there is no linear CCHP system that has already been developed, so the piecewise linearity model is constantly used when considering off-design performance.As a result, the computation load is large.
Very few studies have employed dynamic programming in CCHP system operating optimization.Facci et al. applied dynamic programming to a no-storage CCHP system.Considering that generator restart would require extra cost, the generator status in terms of starting and stopping was set as a 0-1 state variable and dynamic programming was employed [28].Based on previous work, Facci et al. built a CCHP system with storage units.Considering the off-design performance, a dynamic model was established.To reduce the difficulties of the non-linear optimizing problem, dynamic programming combined with meta-heuristic optimization is applied [29].
Their study represented a rare example of the application of dynamic programming to CCHP system operation optimization.However, existing studies maintain a relatively simple system structure.The computation will increase significantly as more plants are introduced, particularly storage units.Further research on dynamic programming applications should be conducted for CCHP systems with complex structure.
In summary, the operation optimization of CCHP systems with storage units should be solved dynamically.Traditional methods such as PSO, GA, and MILP cannot be utilized to tackle it successfully.By resolving the dynamic problem in stages, a dynamic solving framework is created.The computation reduction in complex systems needs significant research, though the prospect of dynamic programming has been confirmed preliminarily.
In this paper, a common CCHP system is proposed.The electric demand is supplied by a power generation unit (PGU) and the power grid.The excess electricity can be sold back to grid.The recovered thermal energy is used to satisfy heating and cooling demands.In addition, two separate heat pumps can also be used to satisfy the thermal demands.The difference between thermal the energy demand and supply can be offset using the thermal storage tanks.The state transition equation is extracted according to the dynamic relationship of the energy storage.A dynamic optimization is proposed to determine the most economical operating schedule.The CCHP system operating optimization is divided into small static problems based on the framework of dynamic programming.The economical concept of variable cost and constant cost are introduced to solve static problems, which can be expressed by the same mathematical model and then solved by the same method with very few computations.As the day-ahead optimization simulation shows, significant improvements over the traditional energy system have been achieved.

CCHP System Modeling
The structure and energy flux of the CCHP system are depicted in Figure 1.The power generation unit (PGU), which is connected to the grid, consumes natural gas to generate electricity and thermal energy simultaneously.The exhaust heat exchanger transfers heat from the exhaust gas to jacket water.The absorption chiller recovers energy from the jacket water to produce cooling water.Similarly, the domestic hot water heat exchanger recovers energy from the jacket water to produce domestic hot water.The chiller and exchanger are assisted by separate heat pumps.The thermal storage tanks store extra energy and supply it when necessary.In winter, the cooling demand changes to a space heating demand and the original cold storage tank is employed to store heating water.Meanwhile, the absorption chiller functions as a normal heat exchanger to satisfy the heating demand associated with the corresponding heat pump.It must be noted that each of the operations of the equipment obeys the solution for operating optimization.

State Transition Equation of CCHP System
The components enclosed within a rectangle with dashed borders in Figure 1 constitute the critical section of this system.The state transition equation of the dynamic relationships of the production, load, and stored energy between the kth hour and (k + 1)th hour can be expressed as follows: where η h is the heat storage efficiency, which represents the proportion of thermal energy remaining after one dissipation stage, and η c has a similar physical significance; H s and C s represent the quantities of stored heating and cooling energy, respectively.The heating and cooling contribution of heat pump are signified as H pumph and C pumpc , respectively.C br and H exc are the chiller and exchanger outputs, respectively.H load and C load are heating and cooling energy demands, respectively.f is the total operating cost and v is the phase cost.Equation ( 1) is the core of this paper, based on which the dynamic solving framework is established.Therefore, the huge dynamic problem of CCHP system operating optimization is dynamically broken up into smaller static problems.The operating cost function v is the key of static problem, which will be discussed in chapter three.

Plant Modeling
The PGU is a gas-fired small internal combustion generating set, whose data is listed in Table 1.
Table 1.Performance of a small naturally aspirated internal combustion engine generator [9].Note: PLR is the load rate, and η g and η i are the efficiencies of the generator and internal combustion engine, respectively.p j and p e are the energy ratios corresponding to the jacket water and exhaust, respectively.p l represents the heat loss rate.

PLR
Taking η re and l rj as the exhaust heat exchanger efficiency and the dissipated thermal energy ratio of jacket water in heat exchanging process, respectively, the waste heat recovery ratio η rw is given by: The recovered waste heat H r is used to drive the absorption chiller and domestic hot water heat exchanger, whose efficiency are η br and η exc , respectively.The contribution of the absorption chiller and heat exchanger are C br and H exc , respectively.The chilling and heating coefficient of performance (COP) of the electric heat pump are COP c and COP h , respectively.The heating and cooling contribution of heat pump are H pumph and C pumpc , respectively.The consumed electricity of heat pump is E pump .

Optimal Operation Model
The optimization of a CCHP system with storage units is dynamic in nature.Thus, the solution framework is based on dynamic programing.The state variable selection is the most important step in dynamic programing.Energy storage should be chosen because it serves as a link between adjacent stages (see Equations ( 1)).Thereupon, the optimization model can be established.The state variable discretization is as follows.
s k (H s , C s ) denotes the heating and cooling energy storage of stage k, where 0 ≤ H s ≤ N h and 0 ≤ C s ≤ N c .N h and N c are the capacities of the heating and cooling storage tanks, respectively.After setting m and n, s k can be discretized into (m + 1)•(n + 1) state points.The state point s k p N h m , q N c n can be expressed simply as s p,q k , where 0 ≤ p ≤ m, and 0 ≤ q ≤ n.The set of s p,q k is expressed as s k .Larger m and n lead to more accurate discretization and more state points.
According to the discretization described above, s k is arrayed as depicted in Figure 2.  k+1 is set to +∞.Suppose P is a path of D from the initial point s 1 to the end point s N+1 , and define the weight of P as the sum of each arc in P, represented as v(P).The objective of this shortest path problem is to find the minimum-weight path P 0 among all of the paths P from s 1 to s N+1 , where: P 0 is the shortest path from s 1 to s N+1 .The weight of P 0 is the distance from s 1 to s N+1 , represented as f (s 1 , s N+1 ).For CCHP system operation, f (s 1 , s N+1 ) is the minimum operating cost.Thus, the optimization problem can be solved by finding P 0 .

Shortest Path Determination Based on Dynamic Programming
The shortest path search is a multi-stage decision problem.The optimality principle was developed particularly to solve this kind of issue.Moreover, dynamic programming is proposed by transforming the multi-stage process into single stages.The result obtained by dynamic programming is certain to be optimal due to optimality principle.The best methods recognized for solving the shortest path problem involve dynamic programming without exception.The diagram of the dynamic programming flow used in this paper is provided in Figure 4.The shortest path P 0 from s 1 to s N+1 always starts from s 1 , passing through one state point s i,j N , and finally arriving at s N+1 .According to the optimality principle, the path from s 1 to s i,j N is the shortest.Hence, the dynamic programming equation of this model is obtained as: Using s * k−1 to signify the optimal state point selected from s k−1 , a more general expressions can be derived as: and: As shown in Equations ( 6) and ( 7), forward dynamic programming is applied.This problem is solved step by step.Meanwhile, the shortest distance and path selection are recorded.The optimization problem is solved when f N+1 (s 1 , s N+1 ) is obtained.
In addition, as can be seen in Figure 2, larger m and n lead to larger s k .To reduce the amount of unnecessary calculations, the discretization is separated into two steps.Firstly, the energy storage is discretized with rough accuracy and dynamic programming is applied to search the shortest path.Secondly, the energy storage is discretized with precise accuracy near the path obtained in the first optimization.The second optimizing result is precise to 1 kW•h.

Static Problem: Analysis of Stage Cost
The static problem is searching for the minimum cost resulting from the state transition.In other words, its objective is to determine v k s i k , s j k+1 according to state points s k (H s , C s ) and s k+1 (H s , C s ).According to Equation (1), the heating and cooling production of stage k can be represented by the energy storage of stages k and k + 1.Based on the required energy production, the most economical dispatch strategy and its corresponding cost can be determined by referring to the operation optimization of a no-storage CCHP system, which is a static problem.Although LP, GA, and PSO can be employed, it is time consuming to calculate the static problems repeatedly in dynamic programming.In this section, the operating cost is solved without any optimizations by introducing the concept of variable cost.
The operating cost of a CCHP system consists of electricity and gas costs.The stage cost v can be calculated as follows: where G is the consumed natural gas and G price is the gas price.The amount of electricity received from the power grid is given by: For the given state points s k (H s , C s ) and s k+1 (H s , C s ), the total heating and cooling demand, H and C, respectively, are fixed.
Based on the modeling of the PGU and exhaust heat exchanger given in Section 2.2, E pgu and G can be fitted as polynomial functions of H r .The required data are listed in Tables 1 and 2. Because H r is the function of C br and H exc , the conclusion can be drawn that both E pgu and G are functions of C br and H exc .Hence, v can be represented as a function of C br and H exc .Referring to the economics, the operating cost consists of constant cost v and variable cost ∆v: Assume that all of the heating and cooling energy is provided by the heat pump and that the electrical load is supplied by the power grid.The constant cost v is determined by E price , E load , H and C. In other words, v cannot be optimized: Starting the generator results in an extra gas cost, while the produced power offsets the power bought from the grid.The variable cost ∆v represents the change in cost resulting from generator operation at different power levels: The domain of this function is: v has no relationship with C br and H exc .To determine the minimum stage operating cost v, attention should be paid to ∆v, which is a function of C br and H exc .Hence, the essence of static problems is searching for the minimum value of ∆v.According to the expression for ∆v, E price is the most influential parameter.Its influence is shown in Figure 5.For clarity, C br and H exc are combined into H r .

Parameters Values
Generator  When E price is 1.1 ¥/(kW•h), all of the heating and cooling is supplied by the absorption chiller and domestic hot water exchanger.When E price is 0.7 ¥/(kW•h), the heat recovery of 144.7 kW•h corresponds the most economic operating strategy.When E price is 0.4 ¥/(kW•h), the heat recovery of 123 kW•h corresponds to the peak efficiency of the generator.The generator should work to ensure that the ∆v as small as possible, so long as ∆v is negative.Otherwise, it is more economical to stop the generator when the efficiency is low.
E pgu , H pumph , and C pumpc can be determined based on H exc and C br .Hence, the optimal dispatch strategy can be described as u (C br , H exc , E pgu , H pumph , C pumpc ).The operating cost v is also obtained.By referring to the concept of constant cost and variable cost, thousands of repeated computations can be eliminated.

Load Description and Basic Data
The energy demands can be obtained from [29,32].The building was simulated using EnergyPlus (5.0.0.035).The description of the simulated building is given in Table 3.For our test case, we selected two typical days in summer and winter, as reported in Figure 6.In addition, there is no cooling load in winter.Instead, extra hot water is required by the central air-conditioning system to keep the dormitory warm.This part of the hot water is separated from that consumed by bathing and so on.The cooling storage tank is employed to store this part of the hot water.
The electricity price (in Yuan ¥) per hour refers to [33]: The CCHP system parameters are listed in Table 4.

Parameters Values
Rated The parameters of a traditional energy system are listed in Table 5.The heating load is supplied by a gas boiler, the electrical load is supplied by the power grid, and the cooling load is supplied by an electrically driven air conditioning system.Table 5. Constant parameters of a traditional energy system [9].

Parameters Values
Efficiency of Power Plant 0.35 Efficiency of power transmission 0.92 COP of electrically driven air conditioning system 3 Efficiency of gas boiler 0.88 The fuel parameters employed for the traditional energy system and CCHP systems to calculate the operation targets are listed in Table 6.

Results and Analysis
The state variable discretization process was divided into two steps with accuracy at 10 kW•h and 1 kW•h.According to the discretization method described in Section 3.1, the amount of computations required was reduced by 98%.
The optimal results obtained using the loads in summer are presented in Figures 7 and 8.The negative power grid output values indicate generator feedback power to the grid.
As shown in Figure 7, the energy storage units store energy when the demand is low and then supply a substantial portion of the energy demand during the peak power consumption periods.As depicted in Figure 8, the generator operates at the load rate of about 80%, although the electrical load fluctuates sharply.The storage units serve to reduce the peaks and fill the valleys, which dramatically improves the energy utilization.Nevertheless, the energy demand tendencies remain observable in the generator operation tracking results.Moreover, the power track of the generator follows the power price of the grid.The generator operates at a high load rate when electrical power is expensive.An appropriate load rate is applied when power is modestly priced.The generator would stop at a low power price.
From 8:00 to 10:00, the generator operated at nearly full capacity, and some extra power was sold to the grid.It can be seen from Table 1 that the operating efficiency at full capacity is lower than the maximum efficiency.However, electrical power is so expensive that it is profitable to sacrifice some efficiency.Moreover, the storage units store considerable energy to prepare for the upcoming phase of peak energy consumption.
From 19:00 to 20:00, the thermal energy demand is low.Due to the high electricity price and large amount of electricity demand, the generator operated at nearly full capacity.Meanwhile, large quantity of thermal energy is stored to handle the next peak of thermal energy consumption.
The generator stopped at 23:00.The subsequent thermal demand can be supplied by the energy stored beforehand.If the generator continues operating, the stored energy would remain unutilized.The generator should stop operating although there was little power demand at 23:00.In summary, the operation optimization is influenced by three main factors.The most important factor is the power demand, which determines the general trend of the optimization results.The next factor is the price of electricity, which strongly affects the operating state of the generator.The last factor is the dissipation of stored energy, which restricts the energy storage time.These three factors jointly determine the optimization results.
Under the energy demands of a typical day in summer, the operating targets of the CCHP system obtained using dynamic programming and the traditional energy system targets are provided in Table 7.The operating cost is converted into dollars.The energy efficiency is the proportion of energy consumed by users and the fossil energy consumed by the power station, gas boiler, and CCHP system.The operating cost of the CCHP system is reduced by 40.8% compared to that in the traditional energy system.Furthermore, the fuel energy saving ratio is 22.0% and the carbon emission is decreased by 54.7% in the CCHP system.
The optimal results obtained considering the loads in winter are presented in Figures 9 and 10.As mentioned previously, the hot water required by the central air conditioning system to keep the dormitory warm was separated from that consumed by bathing and so on, and the cool storage tank was employed to store this part of the hot water.
Generally, the optimization result under winter conditions is influenced by the three factors discussed for summer conditions.However, a significant characteristic occurs at late night.Unlike in the results obtained for summer, the generator starts at night because the heating load is heavy in winter.Because the electricity is cheap late at night, the generator has to operate at the highest efficiency.Otherwise, it has to stop.Hot water is stored to supply heating.Under the energy demands of a typical day in winter.The operating targets are compared in Table 8.When compared with the traditional energy system, the operating cost is reduced by 35.8%, the fuel energy saving ratio is 16.7%, and the carbon emission is decreased by 39.5%.
The conclusion can be drawn that this optimization method not only ensures that the optimal operating cost is achieved, but also obviously improves the fuel energy saving and environment protection.Moreover, all the optimizing results were obtained in less than three seconds.

Conclusions
In this paper, a CCHP system with storage units was designed.Due to its complex structure and internal coupling relation, especially considering that its operation progress is essentially dynamic, traditional optimizing algorithms have some insufficiencies in optimizing its operating schedule.Recent research has improved the advantages of dynamic programming applied to CCHP system optimization.However, its application to a CCHP system with complex structure needs efficient planning to reduce computation.
In the proposed method, the optimization problem was split into a dynamic problem and an embedded static problem.The dynamic problem reflects the essence of the optimization problem, while the static problem provides the basis of the dynamic problem.Thousands of repeated computations were eliminated in economical optimization by introducing the concept of constant cost and variable cost.Compared to a traditional energy system, the operating cost was reduced by 40.8%, the fuel energy saving ratio was 22.0%, and the carbon emission was decreased by 54.7%.Moreover, the optimization of the whole day of a CCHP system requires about 3 s on an average desktop computer.This is a very short optimization time for a CCHP system with energy storage units.Thus, dynamic programming can be successfully employed to solve the optimization of CCHP system with complex structure.
In addition, the optimizing methodology applied in this paper implies a stochastic dynamic solving framework, which will probably contribute to CCHP system optimization.We have achieved some breakthrough and are trying to employ it in stochastic optimization of CCHP systems considering off-design performance.

Figure 2 .
Figure 2. Arrangement of s k .
capacity N pgu 90 kW Efficiency of domestic hot water heat exchanger η exc 0.95 Rated efficiency of absorption chiller η br 0.8 Efficiency of exhaust heat exchanger η re 0.8 Energy loss ratio of jacket water in exhaust heat exchanger l rj 0.2 Heating value of natural gas Q 35,500 kJ/m 3 Price of natural gas G price 2 ¥/m 3According to the expression for ∆v, E price is the most influential parameter.Its influence is shown in Figure5.For clarity, C br and H exc are combined into H r .

Figure 5 .
Figure 5. Relationship between H re and ∆v for different E price .

Figure 7 .
Figure 7. Changes in energy storage under summer conditions.

Figure 8 .
Figure 8. Power variations of system components under summer conditions.

Figure 9 .
Figure 9. Changes in energy storage under winter conditions.

Figure 10 .
Figure 10.Power variations of system components under winter conditions.

Table 7 .
Targets comparison of a whole day of operation in summer.

Table 8 .
Targets comparison of a whole day of operation in winter.