A Multi-Head Attention-Based TimesNet for Heat Production Planning Under Unknown Future Demands

Abstract

Efficient operational planning in district heating systems (DHSs) is essential for minimizing operating costs and maximizing energy efficiency. However, since practitioners must determine future production plans under unknown future demands and costs in real-world energy systems, it is challenging to solve the production planning problems of DHSs. In this paper, we propose a multi-head attention-based TimesNet (MATN) in which a transformer decoder is incorporated that operates solely on a 24 h lookback window without requiring any future information. Specifically, the model is trained in an end-to-end manner, for which the training dataset was built by solving a mixed integer programming (MIP) model. Experimental results demonstrate that the proposed MATN model significantly outperforms baseline deep learning-based methods. A qualitative analysis of the hourly production plans further indicates that MATN generates robust operational plans that mimic those generated by an MIP model, which suggests the effectiveness of the proposed approach in terms of economic efficiency and operational stability without depending on future information.

1. Introduction

A district heating system (DHS) is critical infrastructure for efficiently delivering thermal energy to urban areas [1,2]. The core operational task of a DHS is the heat production planning problem, which is responsible for determining the production levels of heating plants in a real-time manner to satisfy fluctuating demand while minimizing operational costs [3,4]. Unfortunately, it is challenging to effectively solve such problems, since operators must leverage a trade-off between generating heat, utilizing stored thermal energy, and transacting with external networks [5,6,7]. The economic incentives for these choices are driven by volatile energy prices, while the feasibility of any plan is constrained by the physical limitations of the production units, such as minimum uptime and downtime requirements, which create strong dependencies between time periods in the process of making decisions [8,9,10].

This planning complexity, established by temporal dependencies, is further amplified by operational uncertainty. Real world demand and market prices, driven by factors like weather and unplanned events, often deviate from forecasts. These discrepancies can render a pre-calculated optimal plan inefficient or even unfeasible when confronted with actual conditions, leading to avoidable costs and stability risks. The combination of strong temporal constraints and input uncertainty makes optimization plans sensitive to forecast errors, where deviations from predicted conditions can cascade through temporal dependencies and degrade plan quality [11,12,13,14,15,16].

This study focuses on determining heat production plans given unknown future information. Specifically, the production plans are obtained by utilizing the demand and production cost information together with a 24 h history window. This design aims to enable immediate replanning when new demands and costs are observed [17,18]. Mixed-integer programming (MIP) is a well-known approach to developing production plans, prized for its ability to deliver optimal solutions [19]. Its primary drawback, however, is a significant computational intensity that hinders rapid replanning in dynamic scenarios [20,21].

To overcome this limitation, prior work has employed deep learning-based approaches that were trained on the outputs of MIP [22,23]. These approaches successfully demonstrated that a trained model could generate multi-period production plans while requiring a short inference time. Yet, a drawback of these approaches is their dependence on complete, day-ahead forecasts for training. These dependencies on deterministic inputs make the models highly sensitive to the inevitable errors of real-world forecasting [24,25], which might lead to a significant performance gap between simulation and deployment [26].

Motivated by the above challenges, we propose a multi-head attention-based TimesNet (MATN) that aims to build heat production plans under the dynamic and uncertain conditions of real-world DHSs. The architecture of MATN is inspired by recent advances in time-series analysis and sequence-to-sequence modeling, integrating a TimesNet encoder for periodic pattern extraction with a Transformer-based decoder [27,28,29,30]. Specifically, MATN is trained on production plans generated by an MIP model [22]. The trained network operates by first encoding multi-period forecasts of demand and cost into a rich temporal embedding, and then the decoder autoregressively queries this embedding at each decision step to generate a feasible and coherent forward-looking plan. As a result, MATN quickly produces cost-effective schedules, bridging the gap between offline optimization and the demands of practical, online deployment [31].

The remainder of this paper is structured as follows. Section 2 reviews prior work in DHS optimization and relevant deep learning methodologies. Section 3 formally defines the heat production planning problem and presents the MIP formulation used to generate training data. Section 4 details the proposed MATN architecture and the cost-aware training framework. Section 5 presents a comprehensive experimental evaluation of our model against baseline methods, analyzing its performance in terms of accuracy, economic efficiency, and operational stability. Finally, Section 6 concludes the paper with a summary of our findings and a discussion of future research directions.

2. Literature Review

Classical optimization formulates district heating operation planning with MIP, which minimizes operating costs under realistic plant and storage constraints [11,31,32]. These models capture production limits, storage bounds, and unit commitment rules, including minimum uptime and minimum downtime, typically through mixed integer linear programming and in some cases mixed integer nonlinear programming, when efficiency or investment effects are represented [13,33,34,35,36,37]. These formulations offer clear optimality guarantees and represent complex feasibility sets with high fidelity. However the computational burden rises steeply with longer horizons and richer constraints, which limits their ability to replan quickly as conditions change in real time [20,38,39,40]. This limitation motivates approximations that deliver near-optimal plans at low latency while respecting engineering constraints.

Data driven forecasting improves the inputs to optimization by predicting heat demand and related variables from historical observations [16,17,41,42]. Early studies employed support vector machines and multilayer perceptrons, and subsequent work adopted recurrent models such as long short-term memory and gated recurrent units to capture temporal dependence among demand, temperature, and other covariates [27,43,44,45]. More recent time series architectures, including TimesNet, learn multi-periodic structures that often appear in the daily and weekly cycles of district heating data [29]. These advances increase predictive accuracy, yet most methods assume access to future exogenous signals such as weather forecasts, and thus still hand predictions to a separate optimizer [46]. Under information constraints where only past data are available at decision time, forecasting alone is insufficient for end-to-end planning [47,48].

Deep learning for planning seeks to learn policies that map observed histories to feasible schedules with low costs [49,50,51,52]. Supervised approaches train on optimal schedules generated offline by an MIP model but frequently assume look-ahead inputs over the next twenty-four hours and, therefore, diverge from real operations when future values are unknown [23]. Reinforcement learning has been explored for building and energy control, because it can learn long-horizon behavior from interaction, yet it requires a trusted simulator, stable training, and careful safety assurance before deployment in thermal networks [53,54,55,56]. These limitations define a clear gap. The present work addresses this gap by developing a real time planning policy from only past observations, using MATN to encode a multi-periodic temporal structure and cost-aware objective to internalize economic goals and switching stability [57]. This design retains the strengths of classical formulations by learning from optimal solutions while eliminating the need for future information at inference time [58,59].

3. Problem Description

The primary objective when operating a DHS is to satisfy time-varying heat demand reliably while minimizing total operational cost. For each time step, an operator must determine the optimal production level of the heat plant. This decision is made within a system comprising the production units, a thermal energy storage facility, and an external heat network, as illustrated in Figure 1.

This configuration provides crucial operational flexibility. Surplus heat can be stored or sold to the external network, while deficits can be met by discharging the storage or purchasing from the network. Consequently, an optimal production plan transcends simple demand-following, requiring a sophisticated strategy to leverage these components in the most economical way based on factors like production prices and the storage’s state of charge.

This complex, sequential decision-making problem is formally modeled as a MIP problem. Under the ideal assumption that all future information over the planning horizon is known, this model provides a theoretical optimal solution. In this study, this MIP formulation serves the two critical roles of generating expert-level training data and establishing a benchmark for evaluating the performance of our proposed MATN model.

Given the notation defined in

Table 1. Notations used in the proposed problem.

Notation	Description	Unit
t	Unit period for the planning
T	End of the planning periods
i	Production level
K	Maximum number of production levels
$x_t$	Heat production at time t	MWh
$u_{i,t}$	If production level i is selected at t, set to 1. Otherwise, 0.
$y_t$	If the facility is operated at time t, it is set to 1. Otherwise, 0.
$z_t$	If the facility starts up at time t, it is set to 1. Otherwise, 0.
$w_t$	If the facility shuts down at time t, it is set to 1. Otherwise, 0.
$F_i$	Heat production volume at the production level i	MWh
$q_t$	Heat inventory at the heat storage at time t	MWh
$s_t$	Heat supply from external network at time t	MWh
$r_t$	Heat sales to external network at time t	MWh
$D_t$	Heat demand at time t	MWh
$H_t$	Heat production cost at time t	USD/MWh
${EF}_t$	Heat supply cost from external network at time t	USD/MWh
${ET}_t$	Heat sales price to external network at time t	USD/MWh
C	Heat inventory holding cost	USD/MWh
$P_{min}, P_{max}$	Min and Max capacity at heat production facility	MW
${PC}_{min}, P{C}_{max}$	Min and Max capacity at heat storage	MW
$OT$	Min operation time of the production facility	Hour
${\alpha }_i$	Class weight for level $i$in the weighted cross-entropy
$p_{i,t}$	Softmax probability that the model selects level $i$at time $t$

, we adopt the MIP formulation as in a previous study [23] to generate production plans for training datasets. The formulation of the MIP model used in this study is based on [23]. This MIP model serves as our optimization-based benchmark, providing solutions for training and evaluation, which is defined as follows.

$$\mathrm{Minimize} {\sum }_{t=1}^T{H}_t·x_t+{\sum }_{t=1}^T{C}_t·q_t+{\sum }_{t=1}^T{EF}_t·s_t+{\sum }_{t=1}^T{ET}_t·r_t$$

(1)

$$\mathrm{Subject} \mathrm{to} q_{t-1}+x_t+r_t=q_t+s_t+D_t, t=1, \dots , T$$

(2)

$$x_t\le P_{max}·y_t, t=1, \dots , T$$

(3)

$$x_t\ge P_{min}·y_t,t=1, \dots , T$$

(4)

$$x_t={\sum }_{i=1}^K{F}_i·u_{i,t},t=1, \dots , T$$

(5)

$${\sum }_{i=1}^K{u}_{i,t}\le 1, t=1, \dots , T$$

(6)

$$y_t-y_{t-1}=z_t-w_t, t=1, \dots , T$$

(7)

$$y_{t+a}\ge z_t, t=1, \dots , T, 0\le a<\mathrm{O}T$$

(8)

$$y_{t+b}{\le (1-w}_t), t=1, \dots , T, 0\le b<\mathrm{O}T$$

(9)

$$q_t\le {PC}_{max}, t=1, \dots , T$$

(10)

$$q_t\ge {PC}_{min}, t=1, \dots , T$$

(11)

$$y_t,z_t,w_t\in \left\{\mathrm{0,1}\right\}, t=1, \dots , T$$

(12)

$$u_{i,t}\in \left\{\mathrm{0,1}\right\}, t=1, \dots , T,\mathrm{i}=1,\dots ,K$$

(13)

$$x_t,q_t,r_t,s_t\in {\mathbb{R}}^{+}, t=1, \dots , T$$

(14)

Equation (1) minimizes the total operating cost over the planning horizon by aggregating production costs $H_t{·x}_t$, inventory-holding costs $C_t·q_t$, and external purchase costs ${EF}_t·s_t$, offset by revenue from external sales ${ET}_t·r_t$. The intertemporal energy-balance constraint (2) links consecutive periods by ensuring that heat inventory consistently reflects production, demand, storage inflow/outflow, and external exchanges. Constraints (3)–(4) tie feasible output to the unit’s on/off status and enforce the minimum and maximum production capacity. Constraints (5)–(6) encode the discrete operating regime via a mutually exclusive (one-hot) level selection. Constraints (7)–(9) impose operational logic such as minimum up/downtime. Constraints (10)–(11) enforce the storage’s lower and upper bounds. Finally, Constraints (12)–(14) specify variable domains, yielding a standard MIP that can be solved with off-the-shelf optimizers.

4. Proposed Method

4.1. Overall Framework

The overall methodology follows a framework of two stages as presented in Figure 2. The primary objective of the first stage is to prepare a comprehensive and high-fidelity dataset to train the MATN to learn optimal decision patterns. Given that historical operating records often lack the breadth to cover sufficient situations for robust learning, we leverage a MIP model, as defined in Section 3, to compute theoretically optimal plans for diverse simulated scenarios, which then function as the ground truth for training. The methodology for generating these optimal plans follows the approach detailed in a prior work [23]. As depicted in the training dataset in Figure 2, each training instance maps an input structure to its corresponding optimal action. This structure combines the previous system state, namely the heat inventory and production levels, with a 24 h historical window of time-series data like demand and cost. Utilizing this dataset, the MATN iteratively updates its parameters to minimize the discrepancy between its predictions and these optimal solutions.

In the second planning stage, the trained model produces plans in real time that mimic actual operations. Unlike the MIP model, it does not rely on any forecasts of future demand or cost. Reflecting the data structure shown in the diagram, MATN receives only the available past time series combined with the immediately preceding system state as input and outputs the production level for the current time step. This decision is then used to update the next system state, and the procedure repeats across the planning horizon in an autoregressive rollout, yielding a complete operating schedule.

4.2. Data Configuration and Inference Method

Because the study targets a realistic setting without future information, the model inputs and outputs are restructured as in

Table 2. Model inputs and output features for MATN.

Categories	Descriptions	Number of Features
Inputs	${\widehat{q}}_{t-1}$, initial inventory at the beginning of time t	1
	${\widehat{x}}_{t-1}$, heat production at time t − 1	1
	$D_{t-23},\dots ,D_t,$ heat demand data for the past 24 h	24
	$H_{t-23},\dots ,H_t$, hourly production costs for the past 24 h	24
Output	${\widehat{u}}_{i,t}$, target heat production level indicator at time t	i

. At time t, the input consists of the preceding 24 h time series together with the immediately preceding system state. Specifically, it includes heat demand $D_t$ from $\left(D_{t-23}, \dots ,D_t\right),$ unit production price $H_t$ from ${(H}_{t-23}, \dots , H_t)$, and the previous inventory $q_{t-1}$ and previous production ${ x}_{t-1}$. The scalar states $q_{t-1}$ and $x_{t-1}$ are broadcast across the 24 time steps to match the temporal dimension of the time series. All inputs are min–max normalized to the range of zero to one.

The model output $u_{i,t}$ is defined as a classification over four discrete production levels to be executed at time t.

During deployment, the trained model operates in an autoregressive manner. The procedure follows the inference rules in constraints (15) through (21). At time t equals zero, the model takes the actual past data as input and predicts the first production level $u_{i,t}$. This process is repeated over the 120 h planning horizon to produce the full operating schedule. This inference methodology is adopted from the prior work [23].

$${\widehat{x}}_t=\sum _{i=1}^K{F}_i·{\widehat{u}}_{i,t}$$

(15)

$${\widehat{y}}_t=\left\{\begin{array}{c}1 if {\widehat{x}}_t>0\\ 0, otherwise\end{array}\right.$$

(16)

$${\widehat{z}}_t=\left\{\begin{array}{c} 1, if {\widehat{y}}_t=1 and {\widehat{y}}_{t-1}=0\\ 0, otherwise \end{array}\right.$$

(17)

$${\widehat{w}}_t=\left\{\begin{array}{c} 1, if {\widehat{y}}_t=0 and {\widehat{y}}_{t-1}=1\\ 0, otherwise \end{array}\right.$$

(18)

$${\widehat{q}}_t=\left\{\begin{array}{c}if {\widehat{q}}_{t-1}+D_t-{\widehat{x}}_t\ge {PC}_{max}\\ if {\widehat{q}}_{t-1}+D_t-{\widehat{x}}_t>{PC}_{max}\\ if {\widehat{q}}_{t-1}+D_t-{\widehat{x}}_t\le {PC}_{max}\end{array} and {\widehat{q}}_{t-1}+D_t-{\widehat{x}}_t<{PC}_{max}\right.$$

(19)

$${\widehat{r}}_t=\left\{ \begin{array}{l}{\widehat{q}}_{t-1}+D_t-{\widehat{x}}_t-{PC}_{max} if {\widehat{q}}_{t-1}+D_t-{\widehat{x}}_t\ge {PC}_{max}\\ 0 if {\widehat{q}}_{t-1}+D_t-{\widehat{x}}_t<{PC}_{max}\end{array}\right.$$

(20)

$${\widehat{s}}_t=\left\{\begin{array}{l}{PC}_{min}-\left({\widehat{q}}_{t-1}+D_t-{\widehat{x}}_t\right) if {\widehat{q}}_{t-1}+D_t-{\widehat{x}}_t\le {PC}_{min}\\ 0 if {\widehat{q}}_{t-1}+D_t-{\widehat{x}}_t>{PC}_{min}\end{array}\right.$$

(21)

These equations define the state-update rules used during the autoregressive rollout. Constraint (15) translates the model’s categorical output $u_{i,t}$into a physical production quantity $x_t$. Constraints (16) through (18) then determine the plant’s operational status, such as its on/off state $y_t$, and track startup $z_t$ and shutdown${ w}_t$ events. Finally, constraints (19) through (21) manage the energy balance. Constraint (19) updates the heat inventory $q_t$ while ensuring it remains within the storage capacity limits ${PC}_{min}, P{C}_{max}$. Any excess heat beyond the maximum capacity is sold to the external network $r_t$ as defined in constraint (20), while any shortfall below the minimum capacity triggers a purchase from the network $s_t$ as defined in constraint (21).

4.3. Proposed MATN-Based Heat Production Planning Model

This study designed MATN to analyze complex time series patterns and to make sequential decisions. MATN captures multi-periodic structure and temporal context, providing a solution tailored to the DHS planning problem. The overall structure is shown in Figure 3.

This study introduces an architecture specifically designed to analyze complex multi-periodic time series patterns and to make robust sequential decisions in the context of DHS planning, called MATN. MATN integrates advanced temporal feature extraction with a powerful sequence-to-sequence learning paradigm providing a tailored solution to the intricate challenges of DHS operation. The overall structure of the MATN model is conceptually illustrated in Figure 3. The MATN architecture processes an input time series over a lookback window to predict a forecast target.

The MATN encoder branch is meticulously designed to extract rich multi-periodic patterns from the lookback window. It begins by applying a fast Fourier transform (FFT) to compute a frequency spectrum. For each selected dominant period the time series data are dynamically reshaped into a two-dimensional representation. These reshaped features are processed by a parameter-efficient inception block [44]. The outputs from these period-specific processing streams are then combined, passed through a softmax layer, and subsequently projected with positional encoding [57]. This process creates a compressed memory sequence which effectively summarizes the critical temporal context and multi-periodic features of the input for the subsequent decoder.

The MATN decoder branch is responsible for sequentially generating the optimal production plan by leveraging the rich temporal context provided by the encoder’s memory sequence. As shown in Figure 3, the decoder first employs a masked self-attention mechanism to ensure its predictions are internally coherent without referencing future information. Subsequently, a multi-head attention mechanism queries the encoder’s memory, allowing the model to focus on the historical periodic patterns most relevant to the current decision-making step [57]. The output from the attention layers is then processed through a feed-forward network and, finally, a classifier head outputs a probability distribution over the discrete production levels for the target time step.

4.4. Cost-Aware Loss Function

To effectively train the MATN model and align its decisions with real world operational objectives, this study employs a multi objective cost-aware loss function. This function guides the model beyond simple imitation learning by integrating explicit penalties for operational inefficiencies and unstable production schedules. The total loss for training MATN as presented in Equation (25) is a weighted sum of three distinct components.

$${\mathcal{L}}_{CE}=-{\displaystyle \frac{1}{T}}\sum _{t=1}^T\sum _{i=1}^K{\alpha }_i u_{i,t} \mathit{log}{p}_{i,t}$$

(22)

$${\mathcal{L}}_{cost}={\displaystyle \frac{1}{T}}\sum _{t=1}^T(H_t{·\widehat{x}}_t+C_t·{\widehat{q}}_t+{EF}_t{·\widehat{s}}_t+{ET}_t·{\widehat{r}}_t)$$

(23)

$${\mathcal{L}}_{switch}={\displaystyle \frac{1}{T-1}}\sum _{t=2}^T\left|{\widehat{y}}_t-{\widehat{y}}_{t-1}\right|$$

(24)

$${\mathcal{L}}_{total}={{\mathcal{L}}_{CE}+\lambda }_{cost}·{\mathcal{L}}_{cost}+{\lambda }_{switch}·{\mathcal{L}}_{switch}$$

(25)

This training objective combines three loss terms, each targeting a specific goal. First, the classification loss, denoted as ${\mathcal{L}}_{CE}$, from Equation (22) utilizes weighted cross-entropy to train the model to replicate the optimal decisions generated by the MIP. This component provides a strong foundation in imitation learning. Second, the cost term, denoted as ${\mathcal{L}}_{cost}$, from Equation (23) incorporates a differentiable surrogate for the total operating cost. This addition makes the model directly sensitive to economic factors like production and inventory costs, aligning its decisions with the primary objective of cost minimization. Finally, the switching regularizer, denoted as ${\mathcal{L}}_{switch}$, from Equation (24) penalizes frequent toggling between production states. This component promotes operational stability and generates practical schedules that avoid equipment wear from constant adjustments. The final loss, called ${\mathcal{L}}_{total}$, in Equation (25) combines these three components. The coefficients, denoted as ${\lambda }_{cost}$ and ${\lambda }_{switch}$, aims to leverage the trade-off between economic efficiency and operational stability [52,53,54,55].

5. Computational Experiments

5.1. Datasets

To evaluate the performance of the proposed MATN model, we conducted a series of experiments comparing it against a deep neural network (DNN)-based approach [23] that uses 24 h look-ahead inputs. All models were evaluated on a separate set of test scenarios. To quantify overall performance, we evaluated the generated plans based on accuracy, cost, and robustness. Accuracy is the fraction of time steps at which the production level chosen by the AI policy matches the theoretical optimum from the MIP reference under the same scenario state. Economy is the total operating cost obtained by simulating each 120 h plan in an autoregressive rollout using the calculated demand and prices. Robustness is evaluated by repeating the training with different random seeds that control initialization and data shuffling and by reporting the mean and the standard deviation across runs. When appropriate we also performed paired per-scenario significance tests to verify that improvements over the baseline were consistent.

The foundation of our dataset was the operational records from a real-world district heating system in South Korea. As a representative sample in Figure 4 shows, this historical data exhibits clear daily and weekly periodicities. However, relying solely on these raw historical records is insufficient for robust model training, as they may not cover the full range of operational conditions and edge cases to be considered by models.

To overcome this limitation and create a comprehensive dataset, we used the historical data as a basis for generating a large-scale set of diverse, simulated scenarios. This process involved augmenting the underlying patterns of the historical demand with calibrated noise and pairing these augmented profiles with various simulated prices. The MIP model, as described in Section 3, then determined the theoretically optimal production plan for each of these simulated scenarios. This generation yielded a total of 1,800,000 h of optimal operational data, partitioned into a 1,440,000 h training set and a 360,000 h validation set. A separated test set of 12,000 h, structured as 100 independent 120 h scenarios, was also constructed to represent a balanced mix of seasonal and weekly patterns. We note that production costs were calculated using an exchange rate of 1388.89 KRW per 1 USD, as of 10:44 AM UTC on 17 July 2025. All experiments were conducted on a workstation with an AMD Ryzen 7 9700X 8-Core CPU (Advanced Micro Devices, Inc., Santa Clara, CA, USA), 81.6 GB RAM, and an NVIDIA GeForce RTX 4070 SUPER (NVIDIA Corporation, Santa Clara, CA, USA), utilizing Python 3.10.9, PyTorch 2.5.1, and Gurobi Optimizer 12.0.3 (Gurobi, Inc., Beaverton, OR, USA).

5.2. Results and Analysis

A systematic hyperparameter optimization was conducted to identify the most effective MATN configuration. This process evaluated key structural parameters, including the number of decoder layers from two to four, attention heads, and the Top-k dominant periods.

Table 3. Structural parameter ablation with test accuracy and total cost.

Index	Number of Layer	Number of Head	Number of Top-k	Accuracy	Cost
A1	2	2	1	77.50%	196,578.35
A2	2	2	3	79.26%	200,380.87
A3	2	2	5	78.23%	194,987.77
A4	2	4	1	80.13%	193,867.88
A5	2	4	3	79.17%	190,568.41
A6	2	4	5	79.67%	191,530.94
A7	2	8	1	78.11%	188,707.79
A8	2	8	3	78.01%	195,272.26
A9	2	8	5	77.98%	190,729.23
A10	3	2	1	79.34%	194,792.20
A11	3	2	3	79.13%	191,766.22
A12	3	2	5	78.44%	197,323.18
A13	3	4	1	78.40%	195,178.02
A14	3	4	3	76.22%	197,208.24
A15	3	4	5	78.06%	195,949.76
A16	3	8	1	78.45%	197,886.27
A17	3	8	3	78.83%	189,428.18
A18	3	8	5	78.47%	199,672.11
A19	4	2	1	79.26%	207,002.87
A20	4	2	3	77.72%	197,400.13
A21	4	2	5	76.14%	198,520.65
A22	4	4	1	79.92%	194,236.02
A23	4	4	3	77.22%	198,277.50
A24	4	4	5	78.75%	193,950.74
A25	4	8	1	77.89%	198,959.16
A26	4	8	3	78.99%	195,893.00
A27	4	8	5	77.98%	190,729.23

presents a selection of representative results, focusing on the Layer 2 configurations which consistently showed the most promising outcomes.

The results in Table 3 highlight two distinct optimal configurations based on different performance objectives. Bold indicates the highest accuracy and the lowest cost. First, the configuration at A4 achieved the highest accuracy of 80.13%. This result was obtained using two decoder layers, four attention heads, and a Top-k value of one. Second, for operational efficiency, the configuration at A7 yielded the lowest cost of 188,707.79 USD. This setup also used two decoder layers, but with eight attention heads and a Top-k value of one.

Based on this analysis, two candidate architectures were selected for further evaluation. The configuration from index 4 was chosen as the accuracy-optimized model, while the configuration from A7 was selected as the cost-optimized model. This outcome suggests that for this application, a moderate number of attention heads is best for accuracy, whereas a higher number may be more beneficial for cost efficiency, while focusing on only the single most dominant period proved to be the most effective strategy for both.

Figure 5 presents the results of a sensitivity analysis conducted on the cost-aware loss function weights. Following the previous experiment, this analysis was performed on two selected candidate configurations: the cost-optimized configuration (a) with parameters $N_L$ = 2, $N_H$ = 8, and $N_T$ = 1, and the accuracy-optimized configuration (b) with parameters $N_L$= 2, $N_H$ = 4, and $N_T$ = 1. The objective was to find the optimal values for the cost term weight and the switching penalty for each configuration. The search ranges for these weights were set based on values previously found to be effective in similar energy optimization and control tasks [52,53,54,55].

The heatmaps show that the two configurations respond differently to the weights. The cost-optimized configuration (a) exhibits a distinct optimal region, achieving the best balance of high accuracy and low operational cost at the specific coordinates of ${\lambda }_{cost}=0.3$ and ${\lambda }_{switch} = 0.1$. In contrast, for the accuracy-optimized configuration (b), increasing the weights generally leads to higher costs without a corresponding significant improvement in accuracy.

This analysis demonstrates that the cost-optimized architecture under $N_L$ = 2, $N_H$ = 8, and $N_T$ = 1 is highly responsive to the tuning of the cost-aware loss function, finding a clear optimum that enhances both performance metrics. Given this outcome, the cost-optimized configuration with the weights ${\lambda }_{cost}=0.3$ and ${\lambda }_{switch} = 0.1$ was selected as the final, most robust MATN model for all subsequent experiments.

As shown in Figure 6, the loss dropped quickly during the first few epochs and then flattened. Training and validation accuracy reached a stable plateau by epoch 10. This pattern shows that the model learns the most useful structure early and that longer training gives little gain in generalization, while the risk of overfitting increases. We therefore evaluated the saved checkpoints at epochs 10 to 50 using the same test protocol with autoregressive rollout and the deployment cost function. The checkpoint at epoch 10 gave the highest test accuracy and the lowest average cost among the evaluated epochs. Later checkpoints showed no improvement in accuracy and tended to increase the cost.

Table 4. Total cost and accuracy comparison.

Dataset	Total Operation Cost (USD)			Accuracy
	MIP	DNN	MATN	DNN	MATN
1	198,658.39	220,812.98	209,543.83	76.79%	80.00%
2	184,608.80	203,229.94	189,407.38	77.00%	80.58%
3	165,845.17	179,442.22	167,510.09	76.89%	82.25%
4	196,220.41	216,606.60	206,143.42	77.26%	78.42%
5	199,945.69	215,901.00	203,376.74	76.51%	79.25%
6	170,035.34	184,604.26	172,773.43	76.60%	81.00%
7	160,868.08	173,123.14	163,521.72	76.95%	83.33%
8	190,647.88	211,028.40	194,658.41	76.99%	82.08%
9	195,720.11	211,283.42	203,182.63	77.63%	78.33%
10	163,526.79	176,477.69	164,288.45	68.50%	82.08%
Avg	182,607.67	199,250.96	187,440.61	77.72%	80.73%

shows that across all ten test sets, MATN achieved a lower total operation cost and a higher decision accuracy compared to the DNN baseline. The average cost for MATN was 187,441 USD, significantly lower than the 199,251 USD for DNN, and approached the theoretical optimum of 182,608 USD achieved by the MIP model. On average, MATN reduced operational costs by 5.93% and improved accuracy by 3.01 percentage points over DNN.

This superior performance in both cost and accuracy was consistently observed across all test scenarios, demonstrating the model’s reliability. Bold indicates the best model for each metric, the lowest cost and the highest accuracy. These results indicate that the cost-aware training of MATN effectively produces economically efficient actions without compromising predictive quality. Ultimately, this approach yields plans that are not only more accurate but also substantially more cost-effective than the baseline DNN.

Figure 7 compares the average hourly heat production and corresponding operational costs for the MIP, DNN, and MATN models. The MIP benchmark demonstrates a binary production strategy, maintaining output near zero during low-demand periods by utilizing stored heat. In stark contrast, the DNN model shows significant deviation during these same periods. Its heat production fluctuates, reaching peaks of approximately 40 units when the MIP output is zero. As a direct consequence, shown in the bottom panel, this unnecessary production incurs substantial hourly costs, peaking near 1000 USD while MIP costs remain minimal. The proposed MATN model, however, closely mirrors the operational profile of the MIP benchmark. During low-demand intervals, its production remains stable and near zero. This stability results in a cost profile that is also consistently low, closely tracking the optimal trajectory set by MIP. These results demonstrate that MATN’s production strategy directly leads to lower and less volatile operational costs compared to the baseline DNN.

Table 5. Short horizon autoregressive accuracy.

Data Set	MATN			DNN
	H = 4	H = 12	H = 24	H = 4	H = 12	H = 24
1	87.50%	73.33%	82.92%	80.00%	74.16%	72.91%
2	65.00%	84.17%	82.92%	60.00%	69.16%	72.91%
3	87.50%	78.33%	82.92%	80.00%	70.00%	80.83%
4	82.50%	77.50%	83.33%	67.50%	86.66%	73.33%
5	77.50%	76.67%	81.25%	75.00%	66.67%	73.33%
6	70.00%	90.00%	82.08%	32.50%	75.83%	81.66%
7	82.50%	84.17%	83.75%	70.00%	67.50%	79.58%
8	90.00%	92.50%	84.58%	65.00%	80.83%	73.75%
9	80.00%	80.00%	77.50%	42.50%	57.5%	71.25%
10	82.50%	84.17%	82.50%	57.50%	77.5%	78.75%
Avg	80.50%	82.08%	82.38%	63.00%	72.58%	75.83%

presents the results of the short-horizon autoregressive accuracy evaluation, conducted to examine near-term control fidelity. The results show a consistent performance advantage for MATN across all tested horizons. For the shortest horizon (H = 4), MATN achieves an average accuracy of 80.50%, compared to 63.00% for the DNN. This trend continues for longer horizons, with MATN recording accuracies of 82.08% (H = 12) and 82.38% (H = 24), outperforming the DNN’s 72.58% and 75.83%, respectively. The largest performance gap between the two models is observed at the shortest horizon of H = 4.

This consistent outperformance suggests that MATN is more effective at making accurate near-term predictions. Bold indicates the highest accuracy in each row. The significant accuracy margin at H = 4, in particular, indicates that the model excels at capturing the immediate, local, intraday regularities that are most critical for short-term operational decisions.

6. Conclusions

This study proposed MATN, an AI-based methodology for real-time production planning in district heating systems that overcomes the critical limitation of reliance on future forecasts. Using only historical data, we designed a hybrid architecture that integrates a TimesNet-based encoder to capture periodic patterns from time series and a Transformer-based decoder to support robust sequential decision making. Furthermore, to move beyond simple classification accuracy and induce economically sound and stable operation, we introduced a cost-aware loss function that jointly accounts for operational costs and switching stability.

The experimental results comprehensively validate the effectiveness of the proposed methodology. Empirically, MATN achieved a 5.93% reduction in total operating cost and a 3.01 percentage point increase in accuracy relative to a prior DNN baseline [23] that required future look-ahead information. Qualitative analysis of the operational profiles revealed that MATN successfully learned to generate stable, conservative plans that closely mirrored the strategic discipline of the optimal MIP benchmark, effectively avoiding the volatile and inefficient behavior of the baseline DNN. These findings demonstrate that the cost-aware training approach enables the model to internalize the complex economic logic of DHS operations, leading to both enhanced stability and superior economic efficiency.

Nevertheless, since the proposed MATN was validated only on a single DHS plant, the training data were generated from one specific MIP configuration that reflects the operation and utility of the partnered plant. As a result, the empirical results do not yet show how well the approach would generalize to more complex multi-plant or network-coupled DHSs. In multiple production units, shared thermal storage and complicated network flows will introduce additional decision variables and constraints, which makes the planning problem more challenging. In future work, extending MATN to these settings will be tested. Moreover, this study does not make use of exogenous variables such as outdoor temperature, weather forecasts, or electricity prices, even though such signals can be informative for both prediction and planning. Including these variables as additional input channels and systematically assessing their impact is an important direction for future work.

From a practical deployment perspective, MATN is intended to serve as a decision support component within existing DHS operator workflows rather than as a fully autonomous controller. In real-world setting, the model would generate cost-aware production plans and scenario-based forecasts that are presented to operators together with the associated operating costs, switching patterns, and constraint margins. This type of information can improve interpretability by making the economic and technical trade-offs behind each recommendation more transparent. At the same time, additional work is required to ensure safe operation, including the use of hard safety constraints in downstream optimization, operator override mechanisms, and logging of accepted and rejected plans so that operator feedback can be incorporated into future model updates.