An Innovative Control Framework for District Heating Systems : Conceptualisation and Preliminary Results

This paper presents a holistic innovative solution for the transformation of the current district heating and cooling systems to automated more efficient systems. A variety of technological advancements have been developed and integrated to support the effective energy management of future district heating and cooling sector. First, we identify and discuss the main challenges and needs that are in line with the EU objectives and policy expectations. We give an overview of the main parts that our solution consists of, with emphasis on the forecasting tools and an advanced control system that addresses unit commitment and economic load dispatch problems. The proposed control approach employs distributed and scalable optimisation algorithms for optimising the short-term operations of a district heating and cooling plant subject to technical constraints and uncertainties in the energy demand. To test the performance and validate the proposed control system, a district heating plant with multiple energy generation units and real-life heat load data were used. Simulation experiments were also used to evaluate the benefits of using thermal storage units in district heating systems. The results show that the proposed method could achieve significant cost savings when energy storage is employed. The proposed control strategy can be applied for both operating optimally district heating plants with storage and supporting investment planning for new storage units.


Introduction
Europe, forced by climate change and the need for a sustainable economic and social growth, should focus on an energy revolution to reverse present-day unsustainable trends and live up to the ambitious policy expectations.A rational, consistent and far-sighted approach to heating and cooling is key for ensuring such transformation [1], which is also in line with the EU policies of almost zero carbon energy solutions by 2050.More precisely, the European objectives are distinguished between short-term (towards 2020) and medium term (towards 2030), while taking into account that any of the developments described must remain sustainable and future-proof also in the long-term (towards 2050).To achieve these goals and accelerate the development of technological advancements, research efforts should focus on integrated, flexible, highly efficient and environmentally friendly solutions.As previous research studies have shown, the district heating infrastructure has the potential to play a key role in sustainable energy systems [2][3][4][5].Towards this direction, future District Heating and Cooling (DHC) systems can help Europe achieve the objectives [6] reported in Table 1.
Table 1.European objectives for DHC of the future [6].by 2020 -Avoidance of 9.3% of all European CO 2 emissions by District Heating -Additional 40-50 million tons of annual CO 2 reductions by District Cooling -Decrease of primary energy consumption with 2.14 EJ (595 TWh) per year, corresponding to 2.6% of entire European primary energy demand -25% share of renewable energies in District Heating by 2030 -A smart energy exchange network, allowing for optimal resource allocation between the multiple low carbon energy sources feeding into the system and various temperature demands of customers by 2050 -Fully carbon neutral energy solutions through regional, integrated networks Unfortunately, due to the limited use of green industry, the first batch of objectives set for 2020 seem to be very far away from being accomplished.As of today, the DHC sector is considered as a green industry.This means that, on average, 86% of heat for DH in Europe is sourced from a combination of recycled and renewable heat sources [7].On the other hand, in an attempt to steer clear from fossil fuels and support a combination of more efficient renewable and competitive energy supplies, DHC emerges as a viable short-term solution for various European communities [8], as indicated in Table A2 (Appendix A).By no means however has the full potential of DHC been exploited and there is ample room to grow as, currently, it accounts for a limited 10% market share [9].Future DHC systems need to be more efficient, intelligent and cost-effective.It is necessary to develop and deploy intelligent systems using smart metering, including autonomous sensors supplied by their environment (energy harvesting), and control solutions for optimisation and consumer empowerment.Exploiting multiple energy resources and considering co-generation and integration with renewable energy systems will help towards that goal.Moreover, an open challenge is to roll-out intelligent solutions for the integration of thermal and cooling networks with the future smart electricity grids, the so-called tri-generation energy networks [10][11][12].

Scientific Challenges
Based on the future EU objectives and policy expectations outlined above for the DHC sector, we identify several technological requirements that have been emerged [13][14][15][16]: Need for transport, distribution and storage of thermal/cooling energy: To be as efficient as possible, the future DHC systems must even further decrease thermal losses in transport and distribution [17].The constant improvement of pipe technology in accordance with contextual developments is a prerequisite to minimise energy losses and to support new connections of low-energy buildings which require heat transport in smaller pipes.Thus, the development of plug-and-play pipe systems could help to decrease the construction/maintenance time [8] and the associated costs by ensuring quick localisation and problem identification.

(b)
Need to satisfy and analyse the heating/cooling demands: Energy balance between production and demand is the most crucial condition for reducing costs without sacrificing consumer satisfaction.
To fit with the evolving demand trends of end-users, the energy sector must better understand and analyse the present and future demand levels [18] (population developments, densification of cities vs. low-energy buildings, impact of new demands, and own production) [19,20].
Collection and efficient use of data are essential to that aim, since this information helps the network operator manage the grid efficiently, while at the same time the consumers will be better informed about their consumption and about any possibilities to decrease their energy bills [8,21].Thus, intelligent Advanced Metering Infrastructures (AMI) with data mining algorithms need to be developed to support the above prerequisite.

(c)
Need for short-term near real-time energy demand prediction: As already discussed for Scientific Challenge (b), to achieve energy savings and efficient operation of buildings, it is important to establish a plan for adequate energy supply and demand balance.This can be accomplished with the development of advanced forecasting tools that provide short-term predictions of the heating and cooling (H&C) needs for various types of buildings [22].Such tools are of paramount importance since accurate (near real-time) load demand forecasts are crucial for the performance of DHC control systems.Recent advancements in Artificial Intelligence (AI) technology [23] would help to fulfil that need.(d) Need for intelligent monitoring and control of DHC systems: In general, current DHC systems are controlled by conventional PID controllers and most adjustments are still performed manually based on the experience of the network operator.Hence, there is a strong need to support the transformation towards automated DHC systems by developing intelligent control strategies that guarantee efficient energy management and reduction of energy costs [24].Such control systems should be: (i) able to predict ahead in time consumer demand profiles; (ii) adaptive to any changes in the network status as well as copying with faults [25]; (iii) robust to stochastic uncertainties; and (iv) flexible and scalable.Moreover, monitoring tools for energy storage units and renewables power generation could assist the control system to take optimal decisions, whereas simulation tools help to evaluate the outcomes of such decisions.Data-driven artificial neural networks (ANN)-based models have proven to be suitable and could support the monitoring, simulation and development of decision-making algorithms for future DHC systems.(e) DHC systems as an integrated solution: The integration of the various technological advancements (control systems, prediction tools, monitoring tools, etc.) is essential to be able to provide a complete solution to fully achieve the efficient management and operation of DHC systems.
To address the challenges and the technological needs raised above, we propose a holistic approach integrating a variety of innovative solutions for the efficient management and operation of DHC systems.More precisely, an advanced automated decision-making system was developed employing a control framework with multiple decision layers.The proposed system is supported by a variety of tools, such as: (i) forecasting tools that provide short-term energy demand and weather predictions; (ii) modelling and monitoring tools that allow the network operator to know the level of energy stored in storage unit(s); and (iii) intelligent control algorithms for optimal scheduling of production units and distribution of loads that guarantee economic performance and energy efficiency.The aim of this paper is to stimulate a discussion on the future challenges of the DHC sector towards more energy efficiency and intelligent automated solutions.To accomplish such a transformation, a novel approach for optimal energy management in DHC systems that incorporates advanced control algorithms and mathematical programming, supported by AI-based modelling tools, is investigated and discussed.Research efforts are also devoted to the development of new insulation materials for DHC pipelines and intelligent AMIs, that addresses Scientific Challenges (a) and (b), but these will be discussed in a future work.
The rest of the paper is organised as follows: Section 2 reviews the main existing approaches for control and optimisation in DHC systems and gives an overview of the proposed modelling and control framework as well as the supporting technologies.The high-level control strategy is discussed here, along with a brief description of the decision-making and control problems for each layer of the control hierarchy.In Section 3, we report simulation results where the optimal decisions of the control strategy were applied to address unit commitment and economic load dispatch in a DH plant with and without energy storage.The results derived were used to evaluate the benefits of using storage units in DH systems.The integration and interconnection of the supporting technological modules is also discussed.Finally, conclusions are drawn in Section 4 together with ideas for future research activities.

Methodology
The ultimate target of modern control strategies is to guarantee increase of the overall energy efficiency of DHC systems and a fair distribution of heating and cooling based on energy demand of each building.Before we present the proposed control framework and its supporting technologies, we review the main control approaches for DHC systems with emphasis on model-based predictive control strategies.

Current Control Approaches in DHC Systems
Many optimisation models have been studied in the literature for operational planning and control in district heating systems (DHS).In [21,26], the authors presented energy management strategies to optimise the energy distribution of smart buildings in districts, with the aim to limit wastes and costs, while respecting the users' comfort preferences.Mixed-integer programming with mathematical models and Lagrangian relaxation based algorithms is proposed in [27], for short-term production planning in co-generation and power systems.A control strategy for the economic optimal operation in a combined heat and power (CHP) system with renewable sources, including wind energy, photovoltaic and heat recovery boiler, is reported in [28].Similar control strategies with linear programming models are analysed in [29,30] for optimising the operations in CHP systems and evaluating the benefit of using different sized storages.
Most of the works reported above employ a static optimisation model that is not suitable for capturing the time-evolution of energy vectors.A model predictive controller (MPC) overcomes the shortcomings of the static optimisation due to the use of receding horizon strategy and disturbance models.Mixed-integer linear programming (MILP) models combined with MPC strategies have been applied to several cases that require decision-making and scheduling, e.g., in microgrid power plants feeding both electrical and thermal loads [31,32].A similar formulation that tackles the unit commitment problem in thermal systems is presented in [33].A hybrid evolutionary linear programming optimisation algorithm is used in [34] to minimise the total operating costs of DH networks, whereas, in [35], structural, design and operational multi-objective optimisations are performed for distributed energy systems including heating networks.The work in [36] describes a decision support system based on a short-term optimal predictive control law considering the network's lifetime consumption.Comprehensive reviews of the most recent studies in control and optimisation for DHC systems can be found in [37,38], whereas the available software for production planning can be found in [39,40].

The Proposed Control Architecture
Coordinating heat distribution in a large number of buildings in a district under stochastic uncertainties in the demand, weather parameters and energy/fuel prices is a rather complex task from a control systems point of view and a centralised solution would be computationally intractable.To manage such numerical complexity, we employ a multi-layered structure that decomposes the overall decision problem into smaller and computationally tractable subproblems.Each control layer uses a different optimisation model with different time-scale resolution and has its own objectives.A hierarchical control structure with three layers (Figure 1) handles the interactions and coupling between the decision variables and the global constraints at the various levels.More precisely, starting from the top layer of the control hierarchy (i.e., higher-level (HiLe)), we have the so-called superstructure model that takes optimal decisions on the system level.It is implemented as a mixed-logical dynamical system, with both binary and continuous decision variables, suitable for modelling hybrid system operations.The superstructure optimisation model handles the long-term decisions, which is slower than on a basis of several hours, and therefore its time-scale can be set up from several hours to day(s).In our study, a HiLe prediction horizon of 24 h with hourly sampling time intervals was used.The long-term and short-term information from the forecasting units (supporting tools) on weather conditions and thermal/cooling power demand profiles are integrated at this layer as well.At the middle-level (MiLe), a slow time-scale model captures the continuous dynamics of the system, e.g., the transported water in the DH pipelines and optimises the continuous system variables, such as supply/return temperatures and mass flow at the central production units.Major advantages of including models based on physical laws at this layer are the higher modelling accuracy and the possibility to impose constraints on physical and operational variables of the real system.To function well, the MiLe prediction horizon should be longer than the slowest dynamic in the pipe network, i.e., the maximum transport delay.Having this in mind, the MiLe time-scale can be set-up from minutes to several hours, depending on the network under study.A typical MiLe prediction horizon is 6-12 h.The lower control layer corresponds to a fast time-scale model that handles the basic regulation of process variables, such as supply temperature, mass flow rate and pressure, at the substation/building level.

Supporting Machine Learning-Based Tools
Efficient forecast tools are necessary components for energy planning in energy networks playing a key role in operational decision making.Short term weather forecasting could be a crucial contributing factor for estimating future energy demand in DHC systems.Several weather parameters, such as temperature, solar radiation, atmospheric pressure, humidity and precipitation, are widely available nowadays.The availability of this information base sets the stage for a great adoption of more advanced data-driven computer modelling and automated processing of meteorological data.However, data-driven weather prediction remains a challenge due to the inherent uncertainties of the existing weather forecast methodologies.Big data and advances in sensors and smart meters have resulted in massive growth of energy datasets, thus creating new opportunities for energy prediction in DHC systems.Taking advantage of the vast availability of data, a machine learning (ML) approach is proposed that relies on readings from smart meters and contextual information such as weather data or forecasting weather parameters.The proposed two-stages prediction tool comprises of: (a) a data-driven weather forecasting unit that is trained on local weather data (many years of historical data) and predicts future weather parameters within a horizon of 24 h at maximum; and (b) a ML-based short-term, real-time energy demand prediction unit that uses both weather predictions and energy consumption historical data to predict accurately the forthcoming consumer energy needs within the same time horizon of 24 h.
Advanced deep learning (DL) models (Long Short Term Memory (LSTM)) are employed to implement the weather forecasting task.For the training phase of the models, a set of training examples is used in the form of input-output pairs.Historical values of different weather parameters (e.g., temperature, solar radiation, atmospheric pressure, humidity and precipitation) are used as inputs to the DL model, whereas temperature is the desired output parameter.The dimensionality and the format of the training and testing samples are dependent on three parameters: (i) the number of input features (N f ); (ii) the size of the look-back window (N LB ); and (iii) the size of the look-ahead window (N LA ) (or forecasting horizon).The first parameter, N f , denotes the number of weather parameters that are considered as inputs to the DL model (N f = 4 in our case).The second and the third parameters are measured in hourly time-steps.N LB denotes the number of previous hourly time-steps of the input parameters used from training, whereas N LA is the desired number of predicted temperature hourly time-steps.Therefore, the input dimensionality of the weather forecasting model is N LB × N f , while the dimension of the output vector is N LA × 1.Similarly, the energy demand prediction model receives as inputs the N LB historical energy consumption values, the N LA predicted temperatures from the weather forecasting model along with some extra parameters (such as day of the week and time of day).The N LB + N LA + 2 inputs parameters are fed into the DL model (LSTM) for training and learning to estimate the future energy consumption of the DHC network for the next N LA hours.It should be noted that the detailed performance analysis of the proposed forecasting tools is out of the scope of this paper.
As far as the necessary hardware, smart metering systems and wide-area communication infrastructure are needed at the physical level to achieve an efficient two-way communication that will allow commands to be sent from/to end-users.Smart meters will enable consumers to utilise energy in an intelligent way by providing them with information such as time-of-use, pricing information demand response actions or remote service disconnects.The main parameters that need to be measured by the smart meters to calculate the energy consumption are water temperatures and flow-rates in both supply and return pipelines.

The High-Level Optimal Control Strategy
The control algorithms employed in the control hierarchy provide a unit commitment (UC) and economic load dispatch (ELD) strategy for the overall system.The decision problems are solved in near real-time, in a sense that a new optimal energy plan is derived at each sampling time instant (1 h).The aim is to maximise plant efficiency and minimise running costs under strict demand and operational constraints, and uncertainties.The proposed hierarchical-based control framework allows for a decomposition of the large-scale optimal control problem into two smaller subproblems that can be solved efficiently, by separating the discrete and continuous optimisation variables of the system.Instead of using the standard MPC quadratic criterion, the cost function is designed based on some economic criteria that reflect the total energy production costs, the operational and maintenance costs and the variable running costs of the plant.The outcome of the control strategy is the on/off status trajectories and the thermal power profiles for the energy generation units (biomass, gas, oil boilers and auxiliary units) that gives the most economic viable plan.More precisely, at each time step, the optimisation algorithm that handles the UC takes decisions on main plant operations, such as when each production unit should be started and stopped, and when the storage device should be charged or discharged.The output (decision variables) of UC problem is propagated to the ELD one which aims to distribute the loads and optimise system parameters, such as temperatures and mass flows, based on physical models of the entire system, i.e., the network and plant characteristics.The coupling between the generated power that is being produced and the supply temperature from the plant is handled by the middle-level control algorithm, which selects the appropriate mass flow rate to guarantee the energy balance equation, and hence satisfy the consumers demand.Both problems require information on future energy loads over the entire prediction horizon, which is provided by integrating the forecasting tools with the control modules.
Apart from the main demand constraint that handles the energy balance between supply and consumer demand, other typical constraints that the decision algorithm takes into account are: (a) production units' energy generation limits (min/max capacity) (b) minimum up/down times that a boiler should be kept on/off (c) temperature limits per each generation unit (d) ramp limits (up/down) per each generation unit (e) thermal energy storage dynamics and physical constraints (f) reliability constraints Note that the above constraints include some of the typical constraints used in modelling of such systems.Depending on the network under investigation, some of them might not be applicable, or other constraints, such as plant layout and interconnection topology might be considered as well.To anticipate forecast inaccuracies and make the decision-control algorithms more robust to uncertainties, the optimisation model is embedded into a receding horizon control (RHC) framework with a moving-time horizon.With this approach, we create a feedback mechanism that can potentially compensate for any disturbances or uncertainties acted meanwhile.
The overall integrated system (optimisation, control module and supporting tools) can be viewed as a decision and support system (DSS) that provides with optimal suggestions the network operators for the optimal management of the DHC systems.The decision and support mechanism is designed to be flexible and can be used to any small, large-scale network by suitably adjusting the model inputs and constraints.

Simulation Results and Discussion
The main components of a typical DH system are: the plant that consists of the energy (heating and/or cooling) generation units, the distribution network (pipeline), and the consumers' substations.Typical units used for heat production include biomass boilers, heat-only boilers, CHP units, etc.The thermal power produced is then distributed through a network of pipeline to the connected consumer substations, where the thermal energy from the primary network is transferred to the end-users internal heating system, e.g., radiators.The proposed control strategy was applied to a pilot DHS to test its performance in a realistic simulation scenario with real-life energy demand data, as described subsequently.

Simulation Set-Up and Network Characteristics
The network and the plant considered in our analysis was based on a sub-district of a real DHS located in Vransko (Slovenia) [41] with small modifications in plant characteristics to obtain a more representative study.The actual DHN in total consists of 187 consumer substations connected via about 12, 000 km of pipeline (DN50-DN200) that distributes heating with an aggregate average base load 3.2 (MW).The DH plant we considered in our simulation setup consisted of the following: (i) a biomass boiler burning wood chips; (ii) a gas boiler; and (iii) an oil fuelled boiler (mostly used as a reserve unit).A thermal energy storage (TES) consisting of a 100 m 3 stratified water tank was also considered in our evaluation.Table 2 reports the main parameters for each production unit (referred to as B1-B3 for simplicity).The associated energy generation cost per unit was based on typical fuel consumption prices, as suggested by the network operator (see Table A1, Appendix A).We should note that the focus of this study is on the demonstration of the unit commitment and load dispatch problems, addressed at the top-layers of the proposed control hierarchy, by assuming that suitably tuned low-level controllers exist that regulate the local process variables at the desired levels.

Simulation Experiments
Two main scenarios were considered in our investigation, employing the TES unit or not, and we compared both cases with a reference case derived by assuming perfect forecast, i.e., the controller used the actual demand data in place of forecasts.To run the simulations, the control algorithms required as exogenous inputs the aggregated thermal demand and the corresponding weather conditions for the whole time horizon.Figure 2 shows the outside temperature (upper plot) and the actual versus the forecasted thermal power demand profile (bottom plot) from 90 connected consumers, based on real historical data from October 2017.In the following analysis, a simulation scenario of 14 days = 336 (h) with sampling time interval ∆t = 1 (h) was considered.The HiLe prediction horizon window was set to N h = 24 × ∆t = 24 (h) to provide the optimal energy management plan for the next 24 h.Table 3 summarises the simulation setup used in this study.The main results are discussed next.
As Figures 3 and 4 display, the biomass boiler (B1: ciel line) was mainly used to meet the consumers' energy demand due to its lower production cost compared to the other available energy production units.To increase the operational efficiency of the biomass boiler, the control algorithm suggested a working power at about 80% of its maximum capacity.When more heating power was required, the gas boiler (B2: orange line) was employed, working at low-medium power level.The oil boiler (B3) was only used to cover peak demands, as it was the most expensive production unit, and especially when the peaks were short, due to the longer minimum-up time constraints.We next considered the possibility of employing thermal energy storage (100 m 3 stratified water tank).By using as exogenous inputs the same weather data and demand profiles and assuming constant fuel prices, we applied the proposed control strategy.The optimal decisions on the scheduling and thermal power levels of the production units are reported in Figures 5 and 6.When TES was employed, the decision algorithm planning suggested reducing the usage of the biomass boiler (running at lower levels) and eliminating the usage of the gas boiler when available energy was stored at TES and the demand was low.The only drawback that might be undesirable here was the multiple start-stop actions of the gas boiler (B2), however this could be tackled by adjusting the corresponding constraints.
The average net operational costs associated with the optimal planning suggested by running the proposed control algorithm are shown in Figure 7, for both cases considered.It should be noted that these costs include the operational and variable running costs of the plant (e.g., start-up/shut-down, fixed maintenance costs, etc.) plus some demand penalties and slack variables necessary for the feasibility of the optimisation algorithm.Thus, the derived costs might seem higher than the real energy production costs.Table 4 shows that employing a thermal storage unit made the DHS more economically efficient.Moreover, the control algorithm could achieve about 85% cost savings compared to the reference case.

Discussion and Future Work
The applicability of our proposed control solution was demonstrated in a simulation setup based on energy demand data from a real-life DH network.By running several simulation experiments, we evaluated the optimal decisions of the control strategy and outlined the benefits of using thermal energy storages to the network efficiency.The optimal control strategy, aiming to keep the energy inside the TES as low as possible when the energy demand from the network is low, in combination with the forecasting information can anticipate the upcoming peak demands and use the TES properly to satisfy the consumer demand with the lowest possible energy production costs.
Two set of results are reported over a 14-day simulation horizon.The scenario without TES led to costs of 5997 e, i.e., 1278 e more expensive than the reference case, 4719 e, which defined the theoretical optimum that cannot be achieved in practice.The scenario with the TES employed yielded net operating costs of 4908 e, which implies cost savings of about 20% compared with the without-TES case.This implies that the control algorithm despite the forecast uncertainties can obtain about 85% of the theoretically possible savings derived based on perfect forecasts.From the results above, we may also conclude that the proposed strategy produced in both cases a solution close to the benchmark (optimal scenario).In addition, the fact that the performance of the control algorithm, when applied in a DH plant with TES, was better than the without-TES case (about 85% as good as the theoretical optimal case) validated the proposed methodology.Apart from the economic benefits, a list of practical benefits also exists by integrating energy storage units in DHC systems, e.g., TES can decouple heat production, allows for price-driven power production schemes and flexible demand, and improves the overall system efficiency.
As future activities, we plan to test the robustness of our proposed method by examining the forecasting error and its effect on the operational costs and the control performance.Robust or stochastic MPC variants will be examined in order to cope with the uncertainty.The thermal energy storage monitoring tool, which is currently under development, will be integrated with the control framework to provide information about the energy storage content ahead in time.A future goal we also consider as crucial is to integrate the advanced control framework and the supporting technologies in an online platform with a human-machine interface in order to build an intelligent decision and support system that will assist the network operators to take optimal planning decisions, evaluate their actions and anticipate any undesirable effects.

Figure 1 .
Figure 1.The proposed decision-making and control hierarchy for DHC systems.

Figure 2 .
Figure 2. (Top) outside temperature; and (Bottom) actual versus forecasted aggregated thermal load demand over a 14-day horizon.

Figure 3 .
Figure 3. Optimal heat production and load distribution amongst energy units without TES.

Figure 4 .
Figure 4. Optimal heat production and load distribution amongst energy units without TES.

Figure 5 .
Figure 5. Optimal heat production and load distribution amongst energy units with TES.

Figure 6 .
Figure 6.Optimal heat production and load distribution amongst energy units with TES.

Figure 7 .
Figure 7. Average daily net operational costs for both cases (with and without TES).

Table 2 .
Energy production units' main parameters.

Table 3 .
Simulation inputs and parameters.

Table 4 .
DH plant energy production costs for 14-day planning.