Multi-Timescale Scheduling Optimization of Hospital Integrated Energy Systems for Intelligent Energy Management

Abstract

To address the limitations of traditional hospital energy management strategies in responding to real-time medical demands, this study proposes a coordinated optimization approach for multi-timescale scheduling in diversified hospital energy systems. The long-term scheduling problem is first formulated as a Markov Decision Process, with fine-grained short-term energy supply plans embedded in each decision step through an optimal model. Deep reinforcement learning is then employed to reduce the dimensionality of long-term decision variables, while hybrid integer linear programming is integrated to strictly enforce critical load operation constraints. A hybrid data- and model-driven framework is constructed to simultaneously enhance computational efficiency and power supply reliability. Empirical studies demonstrate that, compared with traditional scenario-based and robust optimization methods, the proposed approach significantly improves energy resource utilization—raising the distributed renewable energy utilization rate from 82.45% to 96.72%—and reduces the power interruption rate for critical loads from 2.8% to 0.15%. This ensures the continuity of medical services while minimizing energy waste. The proposed method provides both theoretical and practical guidance for intelligent scheduling and energy management in complex hospital integrated energy systems.

1. Introduction

With the growing demand for medical services and the advancement of green hospital initiatives, the Hospital Integrated Energy System (HIES) has emerged as a critical solution for achieving energy conservation, emission reduction, and high-quality healthcare delivery. As energy-intensive facilities, hospitals operate around the clock and exhibit distinct characteristics, including diversified energy demands and hierarchical power supply requirements. The energy systems in modern hospitals encompass various forms such as electricity, heating, cooling, and compressed air systems, which collectively enhance the overall stability and sustainability of hospital energy infrastructure [1,2]. These systems are inherently supported by power electronic interfaces (such as PV inverters, battery converters, and uninterruptible power supply (UPS) devices), which enable flexible energy conversion and seamless integration of multiple energy carriers.

Although technologies such as distributed photovoltaics, diesel generators, and energy storage systems are increasingly deployed in hospitals, maintaining energy balance within Hospital Integrated Energy Systems (HIESs) remains highly challenging due to the complexity of medical loads, uncertainty in patient flows, and the stringent reliability requirements of medical equipment. In particular, coordinating scheduling strategies across multiple timescales under complex dispatch cycles remains an urgent and unresolved issue [3,4,5]. Moreover, the integration of electronic energy management systems (EMS) and supervisory control and data acquisition (SCADA) platforms is essential to achieve real-time monitoring, coordination, and secure operation of HIES.

The scheduling strategies of HIES typically operate on two levels: long-term and short-term. Long-term scheduling spans time horizons such as annual or quarterly periods and focuses on equipment maintenance planning and energy contract management to ensure energy supply security over extended durations [6,7,8]. In contrast, short-term scheduling addresses intra-day, hourly, or minute-level energy management, aiming to respond rapidly to fluctuations in medical demand and the uncertainty of distributed energy generation [9,10].

Conventional hospital energy management approaches often concentrate on optimizing a single timescale while neglecting the interdependencies between long- and short-term scheduling. Long-term strategies frequently impose rigid operational boundaries that restrict the flexibility of short-term decisions, making it difficult to respond to sudden surges in demand during surgery peaks or patient influxes. Meanwhile, the dynamic behavior of short-term scheduling can critically influence the effectiveness of long-term planning. The absence of an effective coordination mechanism between timescales can thus result in significant energy waste and increased risk of power supply interruptions in medical areas.

Recent studies have largely focused on isolated optimization within a single scheduling level. For instance, research on short-term energy management has explored optimization strategies for battery storage systems and thermal storage to enhance adaptability to short-term fluctuations [11,12]. Other studies have employed scenario analysis or robust optimization to handle uncertainties in hospital energy scheduling. However, these methods often lack the capacity for effective cross-timescale coordination [13,14,15]. Moreover, traditional model-driven approaches face computational bottlenecks when addressing HIES with complex load profiles and multi-source uncertainties, where the high dimensionality of decision variables renders solution processes computationally intensive.

To address these challenges, this study proposes a multi-timescale coordinated scheduling method for HIES based on Deep Reinforcement Learning (DRL). Specifically, the long-term scheduling problem is formulated as a Markov Decision Process (MDP), with optimal short-term energy supply plans embedded in each MDP step to ensure seamless integration between timescales. Furthermore, a hybrid data- and model-driven framework is developed by integrating DRL with Mixed Integer Linear Programming (MILP), which reduces computational complexity while enhancing system flexibility. This framework effectively mitigates energy waste and improves energy efficiency in hospital operations.

Importantly, this hybrid framework can be seamlessly embedded into the electronic control layer of hospital EMS, interacting with power electronic converters and UPS systems to ensure intelligent and reliable scheduling in practice.

To the best of our knowledge, few existing works have explicitly addressed the unique requirements of hospital energy systems under multi-timescale scheduling frameworks. The main contributions of this study are summarized as follows:

A multi-timescale coordinated optimization framework is proposed, which bridges the gap between long-term and short-term scheduling and overcomes the limitations of conventional methods that often treat them in isolation.

A hybrid data- and model-driven approach is developed by integrating Deep Reinforcement Learning (DRL) with Mixed Integer Linear Programming (MILP), ensuring both decision-making flexibility and strict compliance with medical reliability constraints.

Extensive case studies in realistic hospital scenarios are conducted, demonstrating significant improvements in renewable energy utilization and critical load reliability, thereby verifying the practical value and effectiveness of the proposed method.

The proposed framework explicitly incorporates the electronics-driven characteristics of HIES, including reliance on power electronic interfaces, EMS/SCADA integration, and real-time intelligent control, thereby aligning the optimization methodology with practical electronic energy management applications.

2. Relate Works

Multi-timescale optimization scheduling of Integrated Energy Systems (IES) has attracted increasing attention due to its potential to address two fundamental challenges: (i) exploiting energy storage for peak shaving and valley filling to improve economic efficiency, and (ii) mitigating forecast uncertainty of renewable generation over longer horizons [16,17].

Existing methods can be broadly classified into single-layer and hierarchical scheduling strategies. Single-layer approaches typically adopt relatively large intervals (e.g., one hour) to capture energy flow coupling and minimize operating costs through mixed-integer optimization models [18,19,20]. Li et al. [21] proposed an optimization method for integrated electricity–heat systems based on CNN-BiLSTM-Attention networks, achieving significant effectiveness in forecasting accuracy and scheduling flexibility. However, such methods mostly focus on single-timescale optimization and lack coordination mechanisms across multiple timescales, which is precisely the research focus of this study.

Some studies further refine the resolution to 30 min [22] or 15 min [23], but finer granularity often increases computational burden. In contrast, hierarchical approaches combine long-term scheduling (hourly or longer) with short-term corrections (15, 5, or even 2 min), allowing task decomposition and balancing system objectives such as economy, reliability, and security [24,25]. Comparative studies confirm their effectiveness: for example, Liu et al. [26] showed that multi-timescale scheduling can increase renewable penetration by 3.6% and reduce operating costs by 5.3%.

Recent works have integrated demand-side response and uncertainty management into multi-timescale frameworks. Price- and temperature-based demand response strategies improve load flexibility [27], while stochastic programming methods [28,29] and robust optimization enhance reliability under uncertainty, albeit with high computational costs. Data-driven methods such as deep learning [30,31] improve forecasting accuracy, though their effectiveness depends heavily on data quality. More advanced strategies extend multi-timescale optimization to cross-sector coordination (e.g., electricity–gas or electricity–heat systems) using decomposition, rolling optimization, and hybrid storage mechanisms [32,33,34,35,36,37].

Despite these advances, several gaps remain. Many studies rely on fixed forecast error assumptions, limiting practical significance in real-time operation. Moreover, most works target generic community or regional IES, while hospital energy systems (HIESs) present unique challenges, including strict reliability for critical medical loads, diversified energy demands, and stringent safety constraints [38]. The integration of intelligent energy management with multi-timescale scheduling in hospital-specific scenarios has not been sufficiently explored, leaving opportunities for tailored frameworks that jointly ensure economic efficiency, renewable utilization, and medical reliability.

3. Problem Description

In the scheduling of Hospital Integrated Energy Systems (HIES), the dynamic characteristics of different energy components and load demands often correspond to highly heterogeneous scheduling timescales. To ensure optimal and safe system operation, it is essential to account for the complex interactions across multiple time scales—namely, the coordination and interdependence between long-term planning (e.g., equipment capacity investment, annual maintenance scheduling) and short-term operations (e.g., intra-day charging/discharging, standby unit dispatch). Particularly in wind–solar–storage–diesel hybrid HIES architectures, scheduling spans a wide range of timescales, from long-term activities such as annual or monthly capacity expansion and maintenance strategy formulation, to short-term processes such as photovoltaic/wind power tracking, real-time energy storage dispatch, and rapid diesel generator response within intra-day, hourly, or even minute-level intervals.

These components must operate in tandem under stringent medical power reliability requirements—such as uninterrupted power supply for surgical rooms and electromagnetic compatibility for sensitive devices—resulting in a highly complex multi-timescale optimization framework. A typical HIES architecture is illustrated in Figure 1. As shown, the long-term decision layer (e.g., yearly equipment expansion and maintenance planning) and the short-term execution layer (e.g., intra-day energy dispatch and emergency response) exhibit different decision objectives and cycle characteristics, yet their interaction is critical.

For instance, in Figure 1, Region ② (e.g., ICU and surgical room clusters) is pre-configured with significantly higher guaranteed power capacity than Region ① (general wards and administrative areas). This implies that limited energy storage and fast-response generation resources must be strategically reserved for Region ②. Consequently, compared to Region ①, Region ② faces a substantially lower risk of power shortage during emergencies, such as overlapping urgent surgeries and equipment startups, thereby achieving higher reliability.

This illustrates how the flexibility of short-term scheduling—such as the discharge speed of energy storage systems or cold-start latency of diesel generators—is constrained by the resource allocation framework established in long-term planning. Conversely, long-term planning decisions must be evaluated against insights derived from short-term simulation results, including the risks of critical load shortfalls and the costs of service interruption under extreme scenarios. Therefore, there is a pressing need for tightly coordinated optimization between long-term planning and short-term scheduling in HIES.

As a data-driven approach, Deep Reinforcement Learning (DRL) can effectively circumvent the complexities of directly modeling intricate systems by learning from historical data. However, in Hospital Integrated Energy Systems (HIESs), which are characterized by stringent medical safety constraints and a large number of decision variables, searching for optimal solutions within a vast solution space often results in constraint violations, thereby disrupting the normal operation of the Markov Decision Process (MDP). As illustrated in Figure 2, MDP training frequently fails during the initial stages due to constraint conflicts, making it difficult to generate complete decision trajectories and hindering effective learning and optimization. While DRL has shown promising performance in problems with relaxed constraints and small solution spaces, its effectiveness is significantly limited when applied to complex medical systems like HIES, which feature multi-timescale dynamics, strict operational constraints, and high-dimensional decision variables.

4. Multi-Timescale Scheduling Method Based on Deep Reinforcement Learning

To address the multifaceted challenges in HIES scheduling—including multi-timescale operation, complex medical load connectivity, and system uncertainties—a hybrid strategy that integrates both model-driven and data-driven approaches is proposed. In the proposed framework, Deep Reinforcement Learning (DRL) is employed to optimize long-term scheduling decisions, while Mixed Integer Linear Programming (MILP) is utilized to handle short-term dispatch tasks. This design ensures both computational efficiency and strict adherence to medical operational constraints. The overall architecture of the proposed method is illustrated in Figure 3, highlighting the interaction mechanisms between long-term and short-term scheduling processes.

4.1. Coupling Mechanism Between Long-Term and Short-Term Scheduling

The Hospital Integrated Energy System (HIES) comprises components operating on different scheduling time scales, necessitating coordination between long-term and short-term operations. Long-term scheduling primarily focuses on managing annual equipment maintenance plans and optimizing the monthly operational states of various devices. The main decision variable is the change in equipment capacity for each month, denoted as $\Delta C_{i,d}$, where i represents the equipment type and $d$ indicates the month $\left(d=\mathrm{1,2},\dots ,12\right)$.

The variation in equipment capacity determines the control requirements for equipment operational states at the beginning and end of each month, and is expressed as:

$$C_{i,d,0}=C_{i,d-1,T}+\Delta C_{i,d}$$

(1)

where $C_{i,d,0}$ denotes the available capacity of equipment $i$ at the beginning of month $d$, and $C_{i,d-1,T}$ represents its available capacity at the end of the previous month. These capacity constraints serve as operational boundaries for short-term scheduling. $T$ denotes the total number of hours in a day (T = 24).

In contrast, short-term scheduling addresses the intraday response to medical loads, operating on a daily time scale with an hourly resolution. The decision variables include the output power of various energy components, namely diesel generators $P_{w,t}^{DG}$, energy storage systems $P_{w,t}^{ES}$, distributed photovoltaics $P_{w,t}^{PV}$, grid-purchased electricity $P_{w,t}^G$, and medical load power $P_{w,t}^{ML}$, where w denotes the typical day scenario and $t$ is the time index.

To ensure seamless coordination between long-term planning and short-term execution, a hybrid integration mechanism is designed. The workflow can be summarized in four steps as shown in Figure 4.

Step 1: DRL decision output: At the long-term level, the DRL agent generates the monthly capacity adjustment strategy for each energy component. The action of the agent at month $d$ is defined as:

$$a_d=\Delta C_d^i,i\in \left\{\mathrm{P}\mathrm{V}, \mathrm{Diesel}, \mathrm{E}\mathrm{S}\mathrm{S}\right\}$$

(2)

where $\Delta C_d^i$ denotes the adjustment of available capacity for device $i$.

Step 2: MILP optimization embedding: These actions are imposed as boundary conditions in the short-term MILP model. For each device $i$, the capacity constraint is updated as:

$$P_t^i\le C_{max}^i\left(d\right)+\Delta C_d^i, \forall t\in d$$

(3)

where $P_t^i$ is the short-term output of device $i$, and $C_{max}^i\left(d\right)$ is its baseline capacity.

Step 3: MILP feedback: The MILP model computes the optimal short-term dispatch, yielding the operating cost and constraint violations.

Step 4: Reward feedback to DRL: These results are aggregated into the DRL reward function as:

$$R_d=-\left(\sum _{t\in d} {\mathrm{C}\mathrm{o}\mathrm{s}\mathrm{t}}_t+{\lambda }_1\cdot { \mathrm{Penalty} }_d^{\mathrm{capacity} }+{\lambda }_2\cdot { \mathrm{Penalty} }_d^{\mathrm{safety} }\right)$$

(4)

where ${\mathrm{C}\mathrm{o}\mathrm{s}\mathrm{t}}_t$ is the short-term operational cost, and the penalty terms enforce system reliability.

This closed-loop process allows DRL to explore strategic decisions under MILP-enforced feasibility, while MILP benefits from DRL’s adaptive long-term planning.

4.2. Model-Driven Approach for Short-Term Scheduling

The short-term scheduling process optimizes the intraday operation of generation resources within the Hospital Integrated Energy System (HIES), subject to capacity variation constraints imposed by the annual equipment maintenance plan. Accordingly, it can be formulated as a Mixed-Integer Linear Programming (MILP) model and embedded into the long-term scheduling framework as an environment model within a Markov Decision Process (MDP). The objective of the HIES is to minimize the total operational cost over the course of a day, while accounting for the associated costs of energy storage systems, diesel generators, and medical load management.

$$minC=D_d\sum _{w=1}^W p_{w,d}\sum _{t=1}^T \left(c_{ES}\left|P_{w,t}^{ES}\right|+c_{DG}\left|P_{w,t}^{DG}\right|+c_G\sum _{i=1}^N P_{i,w,t}^G\right)\Delta t$$

(5)

Here, $C$ denotes the operational cost, and $D_d$ represents the total number of days in month $d$. $W$ refers to the number of typical days used to represent monthly variability. $p_{w,d}$ indicates the occurrence probability of scenario $w$ in month $d$. $c_{ES}$ denotes the unit cost for charging/discharging the energy storage system, $c_{DG}$ represents the unit operating cost of the diesel generator, and $c_G$ is the unit cost of grid-purchased electricity. $N$ denotes the total number of medical load zones.

Meanwhile, the MILP model is subject to the following constraints:

$$\mathrm{s}.\mathrm{t}. C_{i,w+\mathrm{1,0}}=C_{i,w,0}+D_d{p}_w\left(C_{i,w,T}-C_{i,w,0}\right)$$

(6)

$$SO{C}_{w,t+1}=SO{C}_{w,t}-{\displaystyle \frac{P_{w,t}^{ES}\Delta t{\eta }_{cha}}{E}}, P_{w,t}^{ES}\le 0$$

(7)

$$SO{C}_{w,t+1}=SO{C}_{w,t}-{\displaystyle \frac{P_{w,t}^{ES}\Delta t}{\left({\eta }_{dis}E\right)}}, P_{w,t}^{ES}\ge 0$$

(8)

$$SO{C}_{w,T}=SO{C}_0$$

(9)

$$SO{C}_{min}\le SO{C}_{w,t}\le SO{C}_{max}$$

(10)

$$\left|SO{C}_{w,t+1}-SO{C}_{w,t}\right|\le \Delta SO{C}_{max}$$

(11)

$$-P_{max}\le P_{w,t}^{ES}\le P_{max}$$

(12)

$$\sum _{i=1}^N P_{i,w,t}^{ML}+P_{w,t}^{ES}+P_{w,t}^{DG}+P_{w,t}^{PV}=P_{w,t}^G+P_{w,t}^L$$

(13)

These constraints respectively represent the state of charge (SoC) of the energy storage system, the power demand of medical loads, and the energy supply–demand balance, among others.

In order to reflect the physical and safety requirements of hospital microgrids, additional operational boundaries are explicitly incorporated into the MILP formulation. These constraints ensure not only feasibility in optimization but also compliance with medical reliability standards and national grid codes. The following categories are considered, with specific parameters detailed in

Table 1. Operational Boundaries and Sources of Constraints.

Category	Constraint	Value/Limit	Source/Standard
Voltage	Bus voltage deviation	±5% of nominal	GB/T 12325-2008, IEC 60038
Frequency	Grid frequency	50 Hz ± 0.1 Hz	GB/T 15945-2008, IEC 60050
ESS SoC	State of charge	30% ≤ SoC ≤ 95%	Emergency power reserve guideline for hospitals
Diesel generator	Cold-start delay	≤90 s	Hospital design specification, GB 50052-2009
PV inverter	Total harmonic distortion (THD)	≤3%	IEC 61727, GB/T 19964-2012

.

Voltage and frequency limits. Bus voltages are restricted to within ±5% of their nominal values, and system frequency is maintained within 50 Hz ± 0.1 Hz:

$$0.95V^{\mathrm{nom} }\le V_{b,t}\le 1.05V^{\mathrm{nom}}, 49.9\le f_t\le 50.1$$

(14)

Energy storage SoC limits. To preserve emergency reserves, the state of charge (SoC) of the storage system must remain above 30% and below 95% at all times:

$$0.3\cdot C^{\mathrm{E}\mathrm{S}\mathrm{S}}\le {\mathrm{S}\mathrm{o}\mathrm{C}}_t\le 0.95\cdot C^{\mathrm{E}\mathrm{S}\mathrm{S}}, \forall t.$$

(15)

Diesel generator cold-start constraints. The diesel generator startup requires at most 90 s, which is explicitly modeled by minimum startup delay constraints in the MILP. If $u_t^{\mathrm{D}\mathrm{G}}$ is the on/off status, then:

$$u_t^{\mathrm{D}\mathrm{G}}=1\Rightarrow u_{t+\delta }^{\mathrm{D}\mathrm{G}}=1, \delta \le 2$$

(16)

where $\delta$ corresponds to the 90 s warm-up window (1–2 intervals at the 1 min granularity or equivalent).

PV output limits. The photovoltaic inverter output is restricted by both capacity and power quality requirements. The harmonic distortion of output current must be ≤3% in compliance with IEC61727 2024 and GB/T19939-2021 standards. Thus, effective output is constrained as:

$$0\le P_t^{\mathrm{P}\mathrm{V}}\le \left(1-{\eta }_{\mathrm{T}\mathrm{H}\mathrm{D}}\right)\cdot C^{\mathrm{P}\mathrm{V}}, {\eta }_{\mathrm{T}\mathrm{H}\mathrm{D}}\le 0.03$$

(17)

4.3. Data-Driven Approach for Long-Term Scheduling

The long-term scheduling aims to develop an annual capacity allocation strategy for critical medical equipment, which requires dynamic planning of monthly capacity deployment. This task inherently involves sequential decision-making [30], and can therefore be formulated as a Markov Decision Process (MDP). The MDP is defined by a quintuple $\left(S,A,P,R,\gamma \right)$, where:

$S$: State space, representing the equipment health status and load risk;

$A$: Action space, corresponding to monthly capacity adjustment decisions;

$P$: State transition probabilities, incorporating the stochastic degradation of equipment;

$R$: Reward function, embedding the cost of medical service interruptions;

$\gamma$: Discount factor, balancing short-term and long-term benefits.

4.3.1. State Space

The environmental state encompasses the required equipment capacity, medical load demand, distributed generation output, environmental parameters, and the level of medical services. It is defined as:

$$S_d=\left\{d, S_{C,d}, S_{ML,d}, S_{PV,d}, S_{ENV,d}, S_{MS,d}\right\}$$

(18)

where each state component is defined as $S_d$ denotes the current state vector; $S_{C,d}$ represents the equipment capacity state vector; $S_{ML,d}$ denotes the medical load state vector; $S_{PV,d}$ represents the photovoltaic output state vector; $S_{ENV,d}$ corresponds to the environmental parameter state vector; and $S_{MS,d}$ indicates the medical service level state vector.

4.3.2. Action Space

In the multi-timescale scheduling of medical systems, the agent’s action is defined as the monthly capacity adjustment strategy for critical equipment, denoted as: $a_d=\left\{\Delta C_{i,d}\right\}$ where $\Delta C_{i,d}$ represents the capacity adjustment of equipment $i$ in month d, and $i\in \left\{\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{r}\mathrm{g}\mathrm{y} \mathrm{s}\mathrm{t}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{g}\mathrm{e}, \mathrm{p}\mathrm{h}\mathrm{o}\mathrm{t}\mathrm{o}\mathrm{v}\mathrm{o}\mathrm{l}\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{c}\mathrm{s}, \mathrm{d}\mathrm{i}\mathrm{e}\mathrm{s}\mathrm{e}\mathrm{l} \mathrm{g}\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{o}\mathrm{r}\mathrm{s}\right\}$.

To enhance the agent’s ability to explore under uncertain medical environments, Ornstein–Uhlenbeck (OU) noise is introduced as follows:

$$x_d=x_{d-1}+\phi \left(\tau -x_{d-1}\right)+\kappa \omega$$

(19)

where $x_d$ is the current noise value, which fluctuates with seasonal load variations; $x_{d-1}$ is the previous noise value reflecting historical decision bias; $\phi$ is the mean reversion speed; $\tau$ is the long-term mean of the noise; $\kappa$ is the volatility amplitude; and $\omega$ is a Gaussian random sample.

By applying this mechanism, the agent can maintain policy stability while improving exploration diversity. Therefore, the actual executed action, denoted as ${\widehat{a}}_d$, is the sum of the deterministic action $a_d$ and the noise $x_d$.

4.3.3. Reward Function

The reward function serves as the core mechanism guiding the agent to learn optimal medical scheduling strategies, and its design must balance both economic efficiency and medical safety. The improved medical reward function $r_d$ is defined as:

$$r_d={\alpha }_{1}F-{\beta }_{1}C-{\beta }_2{G}_1-{\beta }_3{G}_2$$

(20)

where $F$ is the indicator of medical service continuity, $G_1$ denotes the penalty term for capacity constraint violations, and $G_2$ represents the penalty term for violating medical safety requirements. The coefficients ${\alpha }_1,{\beta }_1,{\beta }_2,$ and ${\beta }_3$ are weighting factors used to adjust the relative importance of rewards and penalties for different objectives.

4.3.4. Policy Learning

As the core component of the coordinated scheduling approach for Hospital Integrated Energy Systems (HIES), policy learning is implemented using the Twin Delayed Deep Deterministic Policy Gradient (TD3) framework, which integrates both long-term and short-term scheduling outcomes. To enhance training stability in medical scenarios, TD3 employs a dual Q-network architecture. The parameters of the Q-networks are updated following a target network smoothing mechanism:

$${\theta }^{\prime }\leftarrow \lambda \theta +\left(1-\lambda \right){\theta }^{\prime }$$

(21)

The parameters $\theta$ and ${\theta }^{\prime }$ denote the weights of the main network and the target network, respectively, and the overall network architecture is illustrated in Figure 4. The smoothing factor $\lambda \in \left(\mathrm{0,1}\right)$ (set to $\lambda =0.995$ in this study) is applied to stabilize the target network updates. The action-value function $Q\mathrm{*}\left(s_d,{\widehat{a}}_d\right)$, â_d) is updated based on the Bellman equation as follows:

$$Q\mathrm{*}\left(s_d,a_d\right)\leftarrow Q\left(s_d,a_d\right)+\alpha \left[r_d+\gamma Q\left(s_{d+1},a_{d+1}\right)-Q\left(s_d,a_d\right)\right]$$

(22)

where $\alpha$ is the learning rate. The objective of policy learning is to maximize the expected return. Through iterative optimization, the learning process integrates the global perspective provided by long-term data-driven scheduling and the local precision enabled by short-term model-driven decisions. This results in a multi-timescale optimized scheduling strategy that significantly enhances the overall efficiency of the system shown in Figure 5.

5. Case Study Analysis

5.1. Simulation Environment and Parameter Configuration

To validate the effectiveness of the coordinated scheduling method for the Hospital Integrated Energy System (HIES), simulations were conducted based on a real-world energy system from a tertiary hospital. The experiments were implemented on a computing platform equipped with an AMD Ryzen 7 5800 8-core processor, manufactured by Advanced Micro Devices (AMD), Santa Clara, CA, USA.

The specific parameter values and operational boundaries are detailed in

Table 2. Core Components, Technical Parameters, and Medical Constraints of the HIES.

Component	Parameter	Medical Constraint	Value
Photovoltaic System	Peak Power	Total Harmonic Distortion ≤ 3%	2000 kW
Diesel Generator	Cold Start Time	ICU Power Supply Protection	≤90 s
Energy Storage System	Total Capacity	Emergency SoC Reserve ≥ 30%	3600 kWh
Flywheel Storage	Response Time	Transient Support for Operating Rooms	≤200 ms
Critical Load (OR)	Peak Load (Operating Rooms)	Voltage Fluctuation within ±5%	850 kW

. The HIES consists of the following core components: a 2000 kW rooftop photovoltaic (PV) system, a 1500 kW diesel generator, and a 3600 kWh hybrid energy storage system (comprising lithium-ion batteries and flywheels). The hospital comprises 800 beds and includes several critical load zones such as the surgical complex, intensive care unit (ICU), emergency department, and general wards. Surgical Complex: Dual-loop power supply with reliability ≥ 99.99%. ICU: Uninterruptible power supply (UPS) with seamless switching (response time ≤ 0.1 s). General Wards: Tiered power supply protection according to priority.

To ensure reproducibility and transparency, the detailed configuration of algorithmic, optimization, and system parameters used in the simulations is summarized in

Table 3. Detailed Parameter Settings for Simulation.

Category	Parameter	Value	Explanation
DRL Parameters	Learning rate	$1.00\times {10}^{-3}$	Controls convergence speed of the policy network
	Discount factor (γ)	0.99	Balances short-term rewards and long-term benefits
	Batch size	128	Improves training stability
	Exploration noise (σ)	0.2	Enhances exploration diversity
	OU noise parameters (μ, θ)	μ = 0, θ = 0.15	Temporal correlation for action exploration
	Training episodes	500	Number of learning iterations
MILP Parameters	Solver	Gurobi 9.5	Commercial solver for optimization
	Max iterations	10,000	Upper bound for solver iterations
	Tolerance	$1.00\times {10}^{-6}$	Accuracy threshold for convergence
System Parameters	PV profile	Real data (2000 kW rooftop array)	Seasonal solar irradiance
	Load distribution	OR: 850 kW peak; ICU: 600 kW; Wards: variable	Derived from hospital statistics
	Energy storage	3600 kWh (battery + flywheel)	SOC reserve ≥ 30%, response time ≤ 200 ms
	Diesel generator	1500 kW	Cold start time ≤ 90 s
	Electricity price	0.42–0.68 CNY/kWh	Time-of-use tariff

.

To represent monthly variability with a tractable number of intraday profiles, we construct typical days for each month and use their occurrence probabilities in the short-term MILP objective (see the use of typical-day scenarios and $p_{d,w}$).

(1)

Data sources and sampling.

Hourly (1 h) series are used in the MILP layer for consistency with the short-term scheduling horizon. Raw telemetry is collected at 15-min resolution and aggregated to hourly averages (power/energy) or maxima (peaks):

Hospital load (by zones: OR, ICU, wards, etc.): EMS/BMS/SCADA metering.

PV generation: inverter logs from the 2 MW rooftop PV plant.

Meteorology: local meteorological station (ambient temperature, global horizontal irradiance, wind speed).

Tariff: TOU prices from the local utility used in Section 5 (baseline 0.42–0.68 CNY/kWh).

All timestamps are aligned to the hospital’s local time; missing values (<0.5%) are imputed via seasonal neighbor interpolation; outliers are filtered by the IQR rule.

(2)

Feature engineering.

For each calendar day $j$ we form a feature vector ${\mathrm{f}}_j\in {\mathbb{R}}^{24\times m}$ that concatenates: (i) normalized total-load and zone-load intraday profiles; (ii) PV output profile; (iii) meteorology profiles (temperature, irradiance); and (iv) calendar indicators (weekday/holiday). Features are standardized: $\tilde{\mathrm{f}}={\displaystyle \frac{\mathrm{f}-\mu }{\sigma }}$. To avoid collinearity, PCA is applied and the first components explaining ≥95% variance are retained.

(3)

Clustering and number of clusters.

Clustering is performed per month to capture seasonal structure while preserving comparability across months:

Algorithm: K-means with k-means++ initialization; Euclidean distance; 50 restarts.

Model selection: the number of clusters $K_d$ is chosen by maximizing the silhouette (and cross-checked by the Calinski–Harabasz index). For reporting and comparability we set $K_d=4$ unless the indices suggest $K_d\ne 4$.

Typical-day prototype: for cluster $w$ in month $d$, the centroid intraday profile ${\mathrm{y}}^{\left(w\right)}$ is the typical day, and its occurrence probability is

$$p_{d,w}={\displaystyle \frac{\left|{\mathcal{C}}_{d,w}\right|}{N_d}}$$

(23)

where $\left|{\mathcal{C}}_{d,w}\right|$ is the number of days assigned to cluster $w$ and $N_d$ is the number of days in month $d$. These $p_{d,w}$ are exactly the probabilities used in the monthly MILP aggregation.

(4)

Reconstruction and validation.

We validate typical-day fidelity by reconstructing monthly series from prototypes:

$${\widehat{y}}_{d,t}=\sum _{w=1}^{K_d} p_{d,w}{y}_t^{\left(w\right)}, t=1,\dots ,24$$

(24)

and evaluating: (i) energy MAPE for PV and total load; (ii) peak/valley errors $\Delta P_{\mathrm{peak}}, \Delta P_{\mathrm{valley}}$ (iii) time-of-peak shift (hours). Acceptance criteria are: monthly energy MAPE ≤ 5%, average-load MAPE ≤ 3%, $\left|\Delta P_{\mathrm{peak} }\right|\le 5\%$, time-of-peak shift ≤ 1 h.

For robustness, we also use Ward hierarchical clustering as a sensitivity check (linkage on standardized features); results are consistent with K-means and omitted for brevity.

5.2. Experimental Results and Analysis

To validate the superiority of the TD3 algorithm in multi-timescale scheduling for medical applications, this study compares the convergence characteristics and scheduling performance of three deep reinforcement learning (DRL) algorithms. As shown in Figure 6, TD3, DDPG, and SAC exhibit significant differences in handling the HIES scheduling task:

The convergence performance of the three deep reinforcement learning (DRL) algorithms—Deep Deterministic Policy Gradient (DDPG), Soft Actor–Critic (SAC), and Twin Delayed Deep Deterministic Policy Gradient (TD3)—was evaluated in the context of HIES multi-timescale scheduling. Among them, TD3 demonstrated superior stability and accuracy. DDPG achieved faster training speed but exhibited relatively lower stability, while SAC performed poorly in this specific problem setting. Therefore, TD3 is identified as a suitable and effective algorithm for addressing complex system optimization tasks in HIES.

5.2.1. Overall Performance Analysis

To evaluate the effectiveness of the proposed method, a comprehensive performance comparison was conducted against two traditional approaches: Scenario Reduction (SR) and Robust Optimization (RO), as summarized in

Table 4. Performance Comparison of Different Scheduling Methods.

Metric	SR	RO	Proposed Method
Distributed Energy Utilization (%)	82.45	88.31	96.72
Critical Load Interruption Rate (%)	2.8	1.5	0.15
Total System Cost (104 CNY)	825.3	892.7	698.4
Average Daily Computation Time (s)	42.3	68.9	35.6

. The results demonstrate that the proposed method significantly improves distributed energy utilization, achieving 96.72%, compared to 82.45% for SR and 88.31% for RO. In contrast, the power outage rate for critical medical loads was reduced markedly from 2.8% (SR) and 1.5% (RO) to only 0.15%, thereby substantially enhancing the reliability of medical power supply.

5.2.2. Short-Term Scheduling Analysis

To comprehensively evaluate the short-term energy scheduling performance of the proposed method in hospital scenarios, representative days from critical seasons in 2021 were selected. Specifically, typical days in spring (moderate load), summer (air-conditioning peak), autumn (transitional season), and winter (surgical peak and influenza season) were chosen for analysis. The results are summarized in

Table 5. Performance Comparison of Typical Day Scheduling Across Seasons.

Scenario/Metric	Method	Spring (Mar 15)	Summer (Jul 15)	Autumn (Oct 20)	Winter (Dec 20)
PV Utilization Rate (%)	SR	85.3	92.1	88.7	76.4
	RO	81.5	88.9	83.2	79.8
	Proposed	89.2	95.7	91.4	82.5
Average Daily ESS Cycles	SR	1.5	1.1	1.3	2.0
	RO	1.8	1.3	1.6	2.4
	Proposed	1.2	0.8	1.0	1.5
Critical Load Assurance Rate (%)	SR	97.2	98.5	96.8	94.3
	RO	99.6	99.8	99.2	98.1
	Proposed	98.6	99.1	97.9	96.8
Diesel generator Peak-Shaving Contribution (%)	SR	29.4	24.1	27.8	37.2
	RO	31.5	26.7	29.1	40.5
	Proposed	34.1	28.5	31.2	42.7

.

Firstly, regarding photovoltaic (PV) utilization, the Scenario Reduction (SR) method relies on predefined strategies and exhibits limited responsiveness to the actual rooftop PV output. In winter, its daily PV utilization rate drops to 76.4%. The Robust Optimization (RO) approach tends to retain excessive backup capacity to guarantee worst-case power balance, resulting in a suboptimal summer utilization rate of 88.9%, which is significantly lower than the 95.7% achieved by the proposed method. By leveraging deep reinforcement learning (DRL) to dynamically model the probabilistic distribution of rooftop PV generation and incorporating Mixed-Integer Linear Programming (MILP) for short-term optimization, the proposed method maintains high PV utilization levels across all seasons.

Secondly, for energy storage system (ESS) scheduling, the SR method fails to adequately account for lithium battery longevity and the high operational standards of the medical sector (e.g., national standard: ≥4000 cycles). In winter, the average daily cycling rate reaches 2.0 cycles/day. Considering an approximate degradation rate of 0.025% per cycle, this could lead to a significant reduction in system lifespan. RO shows even higher cycling at 2.4 cycles/day, prioritizing extreme reliability at the expense of economic efficiency. In contrast, the proposed method incorporates DRL-based charge/discharge thresholds and MILP-based state-of-charge (SoC) constraints, achieving a 25–37% reduction in daily cycling frequency, thereby extending system lifespan and reducing operational costs.

In terms of critical load assurance, SR fails to fully address unexpected load surges. For instance, during winter surgical peaks, the power supply guarantee rate for operating rooms is only 94.3%, falling short of rigid hospital requirements≥99.99% for operating rooms and ≥97% for general wards. RO achieves guarantee rates above 98% throughout the year but sacrifices PV utilization and ESS longevity. The proposed method adopts a multi-objective trade-off strategy. Although the winter guarantee rate for surgical and critical departments is slightly lower at 96.8%, it effectively balances economic efficiency and reliability through optimized coordination of diesel generators and ESS dispatching.

Lastly, regarding diesel generator peak-shaving contribution, SR, relying on static rules, achieves only 37.2% contribution in winter. RO shows marginal improvement but remains limited due to conservative capacity reservation. The proposed method dynamically optimizes diesel generator output and ESS synergy via DRL, increasing the peak-shaving contribution to 42.7%. Considering a backup diesel generator capacity of 1.5 MW, this corresponds to a winter peak-shaving capability of approximately 0.64 MW, significantly enhancing the hospital’s resilience to load fluctuations and sudden high-demand events.

5.2.3. Long-Term Scheduling Analysis

At the long-term operational level, the policy network based on deep reinforcement learning (DRL) significantly enhances the coordination and adaptability of the hospital energy system by learning the dynamic relationships among rooftop photovoltaic (PV) generation, energy storage capacity, and average monthly energy demand. The results are summarized in

Table 6. Collaborative Performance of Long-Term Scheduling in Hospital Energy System.

Metric	SR	Proposed Method
Energy Storage Capacity Control Error (%)	15.2	3.9 (↓74.3%)
PV Forecast Deviation Compensation Rate (%)	64.4	88.6 (↑38.6%)
Real-Time Power Tracking Accuracy (%)	64.9	98.5 (↑52.9%)

.

Firstly, in terms of energy storage capacity control, the DRL-based strategy demonstrates superior performance. Under the traditional Scenario Reduction (SR) method, the capacity regulation error reaches as high as 15.2%, whereas the proposed method reduces the error to 3.9%, achieving a 74.3% improvement. This indicates that the DRL-based scheduling strategy can more accurately align the state of charge (SoC) of battery storage with the hospital’s actual energy demands, thereby minimizing capacity degradation caused by overcharging or deep discharging, and ensuring continuous and stable power supply for critical load areas such as operating rooms and intensive care units (ICUs).

Secondly, the coordination mechanism between DRL and MILP rolling optimization plays a crucial role in compensating for PV generation forecast deviations. By employing DRL to predict the probabilistic distribution of rooftop PV output and integrating it with a monthly rolling MILP model, the system dynamically adjusts reserve capacity and energy storage dispatch strategies. Compared to the SR method, the proposed approach improves the forecast deviation compensation rate by 38.6%, effectively smoothing out the fluctuations in monthly renewable output and enhancing both PV utilization and the hospital’s overall energy self-sufficiency.

Lastly, in terms of real-time power tracking accuracy, the MILP model performs hourly rolling optimization of the combined output from diesel generators, PV systems, and energy storage, enabling highly efficient load response. Through this fine-grained multi-source dispatching, the real-time scheduling deviation is strictly controlled within 1.5%, with the average monthly power tracking accuracy improved by 52.9%. These results clearly demonstrate the proposed method’s advantage in ensuring real-time responsiveness and power stability for high-sensitivity medical loads—especially operating rooms and mission-critical equipment—thus providing nearly uninterrupted and highly reliable energy support for hospital operations.

5.2.4. Case-Based Scheduling Analysis

To validate the practical applicability of the proposed method within a hospital-integrated energy system, a case study was conducted using the energy infrastructure of a newly established campus of a tertiary hospital. The system includes a 2000 kW rooftop photovoltaic array, a 1500 kW diesel generator, and a 3600 kWh hybrid energy storage system composed of lithium-ion batteries and flywheels. The key load zones cover the surgical suite, the intensive care unit (ICU), and the general wards.

Figure 7 illustrates the reward evolution trend during the training process of the deep reinforcement learning agent. In the initial phase (0–150 episodes), the reward exhibits significant fluctuations as the agent continuously explores and gradually learns the seasonal load patterns and responses to extreme events. After 240 episodes, the reward begins to show a clear convergence trend. By episode 400, the average reward fluctuation narrows to within ±1.17, indicating that the agent has effectively learned an optimal scheduling policy.

Notably, at episode 150, the agent successfully identifies a sharp increase in winter heating demand, resulting in a 23% boost in reward. Furthermore, by episode 320, the agent significantly improves the storage reserve strategy during surgical peak hours, leading to a stabilization of the reward curve.

Figure 8 illustrates the dynamic annual capacity adjustments of the diesel generator and energy storage system. For the diesel generator, capacity is increased by 18% during the winter months (December to February) to meet the elevated heating demand typical of the cold season. For the energy storage system, capacity is expanded by 15% during flu season (February to March and October to November) to address emergency backup requirements and patient surges. The total annual capacity variation amounts to only 0.37 GWh, which represents just 3.2% of the system’s designed capacity. This performance is significantly better than that of traditional empirical methods, which exhibit a variation of up to 5.6%.

Figure 9 illustrates the power balance results of the Hospital Integrated Energy System (HIES) on representative days across each month. It is evident from the figure that the hospital’s load structure exhibits significant seasonal variations, and the energy contribution from different sources varies accordingly. The analysis is detailed as follows:

First, grid-purchased electricity serves as the base load provider throughout the year. Its output increases notably in the winter and early spring months (January–March and December), reflecting the hospital’s reliance on highly reliable energy sources during heating seasons and periods of unexpected high demand. For instance, in January, February, and December, the share of grid electricity reaches its annual peak, ensuring uninterrupted power supply for critical zones such as operating rooms and intensive care units (ICUs).

Second, photovoltaic (PV) generation demonstrates pronounced seasonal variability. From late spring to early autumn (April–September), PV output increases steadily, peaking in July and August to meet the surge in air-conditioning demand. During this period, PV contributes more than 25% of the total load, significantly improving renewable energy utilization, while reducing both the cost of grid electricity and carbon emissions.

The energy storage system (ESS) plays a key role in peak shaving, valley filling, and emergency backup across all months. During periods of sharp load fluctuations in summer and winter, the ESS effectively smooths intraday power supply–demand dynamics through flexible charge–discharge strategies. It also provides millisecond-level backup response in extreme scenarios, such as sudden emergency room surges. Experimental results indicate that the ESS’s discharge contribution increases significantly in July and December, ensuring continuous and secure hospital power supply.

Third, the diesel generator acts as a crucial supplementary source to balance the system, especially when distributed energy and ESS are insufficient to meet the demand. Slight increases in the share of grid electricity during summer and winter reflect the system’s compensatory mechanism under extreme climate conditions. Notably, under the coordinated scheduling of PV and ESS, the overall grid power consumption is significantly reduced compared to traditional management approaches, improving both the self-sufficiency and economic efficiency of the hospital’s energy system.

In addition, other renewable energy sources (Other RE) introduce greater flexibility and sustainability to the system. Their contribution increases in wind-rich months (e.g., spring and autumn), providing extra redundancy for system dispatching.

Finally, the total load (black curve) fluctuates throughout the year, with higher levels in summer and winter. The proposed multi-energy coordinated scheduling strategy ensures the rational allocation and efficient utilization of various energy sources under different seasonal and representative day scenarios. This supports secure, economical, and low-carbon operation of the HIES. Overall, the proposed dispatching framework effectively accommodates highly variable hospital loads and frequent extreme events, guaranteeing the continuity of critical medical services and the overall efficiency of system operation.

5.2.5. Supplementary Comparison with Hammerstein Model

To further illustrate the effectiveness of the proposed DRL–MILP framework, a comparative analysis was conducted against the Hammerstein model, which is a classical approach widely used for nonlinear dynamic system identification and scheduling. As reported in prior studies, the Hammerstein structure is effective in modeling weak nonlinearities and dynamic responses of energy systems under simplified operating conditions. However, when applied to the hospital integrated energy system (HIES), its performance is constrained by the high dimensionality of decision variables and the stringent medical reliability requirements.

Under the same hospital dataset, the Hammerstein-based scheduling model achieved acceptable prediction accuracy for short-term dynamics but exhibited slower convergence and limited adaptability to cross-timescale coordination. Specifically, the average renewable utilization rate was approximately 89.7%, and the critical load interruption rate remained at 1.2%, both of which are inferior to the proposed DRL–MILP framework (96.7% and 0.15%, respectively). Moreover, the Hammerstein approach required substantially longer computation times due to iterative parameter estimation, whereas the proposed method converged more rapidly and ensured strict compliance with medical operational constraints.

These findings indicate that, while the Hammerstein model provides valuable insights into nonlinear response characteristics, its applicability in HIES multi-timescale scheduling is limited. In contrast, the proposed DRL–MILP hybrid framework not only ensures high accuracy and computational efficiency but also guarantees critical load reliability under complex hospital scenarios.

5.2.6. Generalization and Applicability Validation

To further verify the generalizability and practical applicability of the proposed scheduling framework, three types of medical institutions were selected for systematic comparative analysis: a newly established tertiary hospital, a community hospital, and a specialized outpatient center.

The results of

Table 7. Systematic Comparative Analysis of Different Medical Institution Types.

System Type	Key Metric	Proposed Method	Traditional Method
New Tertiary Hospital	Surgical Interruption Time	0.3 h/year	4.2 h/year
Community Hospital	PV Utilization Rate	95.1%	86.7%
Specialized Outpatient Center	Equipment Expansion Cost	−18%	Baseline

demonstrate the following:

In the new tertiary hospital, the annual duration of surgical interruptions was significantly reduced to just 0.3 h/year using the proposed method, compared to 4.2 h/year under traditional approaches.

In the community hospital scenario, the PV utilization rate reached 95.1%, significantly outperforming the traditional method’s 86.7%.

For the specialized outpatient center, the equipment expansion cost was reduced by 18% compared to the baseline solution.

These results clearly indicate that the proposed scheduling framework can enhance system-wide economic performance by 21–28%, while strictly adhering to hard medical safety constraints (e.g., ensuring ≥99.99% power reliability for operating rooms).

In terms of economic benefits, the proposed framework can save approximately CNY 2.86 million annually in equipment expansion costs and reduce direct economic losses due to power outages by around CNY 9.2 million. The results are summarized in

Table 8. Economic Benefits of the Proposed Scheduling Framework.

Item	Annual Cost Savings (CNY)	Equivalent (USD)	Equivalent (EUR)
Equipment Expansion	¥2.86 million/year	$0.40 million/year	€0.37 million/year
Outage Loss Reduction	¥9.20 million/year	$1.29 million/year	€1.20 million/year

.

From the perspective of technical scalability, the system supports 5G edge computing deployment with a decision-making latency of less than 50 milliseconds, and has been certified for electromagnetic compatibility (EMC) by national medical equipment standards. These features demonstrate strong engineering feasibility and broad potential for large-scale application.

5.2.7. Sensitivity Analysis

The robustness of the proposed multi-timescale scheduling framework was validated through four categories of sensitivity experiments: energy storage capacity, DRL hyperparameters, medical load forecasting errors, and electricity price fluctuations. The key findings are summarized as follows.

• Energy Storage Capacity Sensitivity Analysis

As shown in

Table 9. Impact of Energy Storage Capacity on System Performance.

Storage Capacity (kWh)	Distributed Energy Utilization (%)	Critical Load Interruption Rate (%)	Annual Cost Savings (104 CNY)	ROI Period (Years)
1800	89.23	0.45	425.6	6.8
2400	93.15	0.32	618.3	5.2
3000	95.47	0.21	687.9	4.6
3600 (Baseline)	96.72	0.15	698.4	4.3
4200	97.23	0.12	701.8	4.5
4800	97.45	0.09	696.2	4.8
5400	97.58	0.08	685.1	5.3

, system performance is strongly influenced by storage sizing. Expanding capacity from 1800 to 3600 kWh markedly improves renewable utilization (from 89.23% to 96.72%) and reduces the critical load interruption rate (from 0.45% to 0.15%). However, beyond 4200 kWh, improvements diminish (<0.05% reduction per 600 kWh), and the ROI period begins to increase due to higher investment and maintenance costs. Thus, the economic optimization zone is identified at 3000–4200 kWh, where both reliability and cost savings are balanced.

2.

DRL Hyperparameter Sensitivity Analysis

The results in

Table 10. DRL Hyperparameter Sensitivity Results.

Parameter	Variation Range	Convergence Episodes	Final Reward	Training Stability (o)	Performance Impact
Learning Rate	0.0001	520	1847.3	23.4	Poor convergence
	0.0005	380	1923.6	18.7	Optimal
	0.001	290	1895.2	31.2	Fast but unstable
	0.002	180	1734.8	45.6	Unstable
Batch Size	32	450	1876.2	28.3	High variance
	64	420	1891.7	24.1	Good balance
	128	380	1923.6	18.7	Optimal
	256	360	1918.4	19.2	Marginal improvement
	512	340	1902.8	22.8	Overfitting risk
Hidden Layer Size	[128,64]	410	1865.3	26.8	Underfitting
	[256,128]	380	1923.6	18.7	Optimal
	[512,256]	395	1931.2	20.4	Slight improvement
	[1024,512]	425	1928.7	25.1	Overfitting

indicate that learning rate is the most sensitive hyperparameter: values between 0.0003–0.0007 achieve stable convergence, while larger rates (≥0.001) lead to instability. Batch size shows an optimal range of 128–256, ensuring low variance while avoiding overfitting. The network architecture with two hidden layers [256,128] provides the best trade-off between accuracy and computational efficiency. These findings confirm that careful hyperparameter tuning is essential for achieving stable and reliable DRL-based hospital scheduling.

3.

Medical Load Forecasting Error Sensitivity

Table 11. Impact of Medical Load Forecasting Errors on System Performance.

Forecasting Error Range	Critical Load Interruption Rate (%)	Energy Utilization (%)	Emergency Response Time (s)	Cost Increase (%)
±2%	0.12	97.23	12.3	+2.1
±5%	0.15	96.72	14.7	+3.8
±8%	0.23	95.84	18.2	+6.4
±10%	0.31	94.67	22.6	+8.9
±12%	0.45	93.12	28.4	+12.3
±15%	0.68	91.28	35.7	+17.8

demonstrates that forecasting errors directly degrade both reliability and economic performance. When the error range increases from ±2% to ±15%, the critical load interruption rate rises from 0.12% to 0.68%, while distributed energy utilization drops from 97.23% to 91.28%. Emergency response time is prolonged (12.3 s → 35.7 s), and operating costs increase by up to 17.8%. The system remains resilient under moderate errors (≤±5%), but performance deteriorates sharply beyond ±10%, highlighting the need for accurate load forecasting and robust policy adaptation.

4.

Electricity Price Sensitivity Analysis

As summarized in

Table 12. Electricity Price Sensitivity Analysis Results.

Peak-Valley Ratio	Average Daily Cost (CNY)	PV Utilization (%)	ESS Cycling Frequency	Peak Shaving Effect (%)
1.5:1	8234	94.32	1.2	28.4
2:1	7689	95.67	1.4	35.8
2.5:1	6984	96.72	1.5	42.7
3:1	6423	97.28	1.8	47.3
3.5:1	6012	97.45	2.1	51.2
4:1	5734	97.52	2.4	54.6

, higher peak-to-valley ratios substantially enhance economic benefits and demand flexibility. When the ratio increases from 1.5:1 to 4:1, average daily costs fall from 8234 CNY to 5734 CNY, while PV utilization improves from 94.32% to 97.52%. Meanwhile, ESS cycling frequency increases (1.2 → 2.4 cycles/day), and the peak-shaving effect nearly doubles (28.4% → 54.6%). These results demonstrate that dynamic pricing can incentivize renewable energy integration and improve system efficiency, though at the cost of higher battery wear.

The sensitivity results confirm that storage capacity and forecasting errors are the dominant factors influencing system reliability, while electricity price spread primarily drives economic optimization. DRL hyperparameters, though secondary, significantly affect training stability and policy robustness. Collectively, these findings emphasize that optimal deployment of the proposed framework requires balanced storage sizing, accurate forecasting, and careful tuning of DRL models, along with consideration of tariff structures to align economic incentives with operational reliability in hospital energy systems.

6. Conclusions

This study proposes a multi-timescale coordinated scheduling method for Hospital Integrated Energy Systems (HIES), based on Deep Reinforcement Learning (DRL) and Mixed-Integer Linear Programming (MILP). Through comprehensive system testing and analysis, the following conclusions are drawn:

The proposed method successfully decomposes the complex HIES scheduling problem into a Markov Decision Process and a Mixed-Integer Linear Programming problem, reducing computational complexity by approximately 35.6% compared to traditional methods and greatly improving computational efficiency.

Compared with traditional scheduling approaches, the proposed framework more accurately captures the complex variability of medical loads, resulting in a substantial reduction in equipment capacity control errors and improved accuracy in long-term scheduling.

The multi-timescale scheduling framework effectively coordinates multiple energy resources—including diesel generators, energy storage systems, and distributed photovoltaics—thereby mitigating fluctuations in medical loads and enhancing the adaptability of the hospital energy system to varying medical demands.

The proposed method offers robust technical support for the intelligent and low-carbon development of hospitals, demonstrating considerable practical value and promising potential for large-scale deployment.

Future work will focus on exploring the agent’s environmental adaptability under expanded system scenarios and investigating the applicability of the method in more complex medical environments, thereby offering robust technical support for the large-scale implementation of high-efficiency HIES.