Hybrid Genetic Algorithm and Deep Reinforcement Learning Framework for IoT-Enabled Healthcare Equipment Maintenance Scheduling

Abstract

We study the predictive maintenance scheduling for IoT-enabled medical equipment in multi-facility healthcare networks. The problem involves skill matching, time windows, and risk-aware priorities. We model a multi-skill Technician Routing and Scheduling Problem with IoT-predicted failure intervals and minimize a composite cost for technician activation and labor, travel/time, risk exposure within the failure window, and lateness beyond it. We propose a hybrid solver coupling a Genetic Algorithm (GA) for rapid exploration and feasible schedule generation with a Proximal Policy Optimization (PPO) agent warm-started via behavior cloning on GA elites and refined online in a receding-horizon manner. An optional, permissioned blockchain records tamper-evident maintenance events off the control loop for auditability. Across four case studies (10–30 facilities), the hybrid approach reduces total cost by 2.09–10.31% versus pure GA, by 0.57–2.65% versus pure Deep Reinforcement Learning (DRL), and by 0.93–2.86% versus OR-Tools VRP heuristic baseline. In controlled early-stopping runs guided by admissible GA/DRL time splits, we realized average wall-time savings up to 47.5% while keeping solution costs within 0.5% of full-budget runs and maintaining low or zero lateness and risk exposure. These results indicate that GA seeding improves sample efficiency and stability for DRL in complex, data-driven maintenance settings, yielding a practical, adaptive, and auditable scheduler for healthcare operations.

1. Introduction

Healthcare providers are rapidly connecting medical assets into sensor-rich, data-driven ecosystems. Internet of Things (IoT)-enabled devices continuously stream usage and condition data, enabling a shift from reactive or time-based interventions to predictive maintenance that improves availability and patient safety while containing costs [1]. In multi-facility settings, however, these gains depend on robust, responsive scheduling of specialized technicians across dispersed sites, heterogeneous equipment, and evolving priorities. Ensuring uptime for critical devices (e.g., MRI scanners, ventilators, and surgical robots) requires decisions that reconcile real-time health signals, tight time windows, skill constraints, and travel logistics under uncertainty.

Two algorithmic paradigms dominate the field of maintenance optimization. Metaheuristics provide flexibility and scale, while Deep Reinforcement Learning (DRL) is increasingly effective for sequential decision-making in dynamic, resource-constrained environments [2,3]. However, three significant gaps remain in the literature. First, in healthcare-specific and multi-facility maintenance, hybrid designs that combine global exploration (e.g., evolutionary search) with DRL’s adaptive policies are under-explored. Second, end-to-end systems that integrate IoT-derived predictive signals with optimization and secure cross-facility record-keeping are rare. Third, rigorous comparisons quantifying how hybridization affects convergence, solution quality, and scalability versus strong baselines are scarce [4].

We model the problem as a multi-skill Technician Routing and Scheduling Problem (TRSP) with time windows and multiple depots. Decisions jointly determine: (i) which technicians are activated and which tasks they serve (skill matching), (ii) task sequences across facilities (routing/precedence), and (iii) start times within service windows and shift calendars (timing). The objective minimizes a composite cost comprising technician activation, paid labor for service and travel, and penalties that price IoT-predicted risk when starting within the predicted failure window and tardiness beyond it. Feasibility is enforced through assignment/skill compatibility, technician and equipment time windows, and routing–sequencing consistency. The problem strictly generalizes NP-hard Vehicle Routing Problem with Time Windows (VRPTW)/TRSP/home-healthcare variants; dynamic, forecast-driven failure windows add difficult couplings. Exact methods scale poorly, especially with real-time updates, motivating heuristics and learning-based solvers with fast re-optimization.

We propose a hybrid framework that couples a Genetic Algorithm (GA) with DRL. GA performs broad exploration and produces diverse, feasible schedules that seed the DRL agent, reducing cold-start effects and improving sample efficiency. DRL fine-tunes policies against streaming IoT signals to adapt rapidly to shifting risk profiles and operational disruptions. An optional, permissioned blockchain-backed logging layer provides tamper-evident, auditable maintenance records to support trust and coordination across facilities.

We ask: how can a GA-initialized DRL framework, informed by real-time IoT signals, improve multi-facility healthcare maintenance scheduling in terms of solution quality, compute efficiency, and scalability versus strong single-paradigm baselines, while supporting secure cross-facility coordination?

This paper makes three contributions:

• A scalable planning-and-control pipeline that fuses predictive maintenance signals with hybrid GA+DRL optimization.
• A comparative evaluation isolating the benefits of GA-seeded policy learning over pure DRL and pure metaheuristics across instance sizes and dynamics.
• An optional, permissioned blockchain layer for verifiable, auditable cross-facility records, integrated out-of-loop to preserve real-time performance.

Extensive simulations reflecting realistic networks and portfolios show that the hybrid approach reduces schedule cost by 5.57% on average relative to strong baselines and achieves best costs with up to 47.5% less computation, on average, due to effective GA seeding and DRL refinement. Practically, the results translate into lower downtime, improved staffing utilization, reduced travel and energy costs, and greater transparency across facilities.

2. Background

2.1. Literature Review

IoT connectivity in healthcare is turning standalone devices into instrumented and interoperable assets that continuously produce condition and usage data. This data foundation enables predictive maintenance, which can reduce downtime and operational costs while safeguarding patient care [1,5]. The challenge now is not data collection but translating heterogeneous and high-velocity telemetry into timely and feasible maintenance decisions across distributed facilities [6].

From an operations-research perspective, multi-facility technician scheduling with skills and time windows is a rich variant of the Vehicle Routing Problem (VRP), closely related to the TRSP [7] and the Home Healthcare Routing and Scheduling Problem (HHCRSP) [8]. These models capture assignment (skill–task compatibility), routing (inter-facility travel), and timing (equipment/shift windows), and are NP-hard. Exact methods scale poorly on realistic instances, especially under real-time updates, motivating heuristic and learning-based approaches.

Metaheuristics are widely used for their flexibility and strong performance on large instances. Genetic Algorithms (GAs) have achieved notable downtime reductions via task grouping and clustering [9]; multi-objective variants such as Non-dominated Sorting Genetic Algorithm II can balance cost and quality-of-care objectives [10]. Other families—Ant Colony Optimization for dynamic routing [11], Particle Swarm Optimization for resource allocation [5], and Simulated Annealing for workflow optimization [6]—perform well on static or quasi-static settings. However, pure metaheuristics can struggle to adapt quickly to nonstationary inputs typical of IoT (e.g., sudden risk updates or failures).

DRL learns sequential decision policies from interaction, making it appealing for online scheduling under uncertainty [2]. In related domains, DRL outperforms handcrafted dispatching rules in job-shop scheduling [12] and orchestrates dynamic IoT workloads [13,14]. Yet, DRL can be sample-inefficient and sensitive to initialization, and may converge to suboptimal policies without good priors.

Hybrid “memetic” methods seek to combine global exploration from metaheuristics with DRL’s adaptive refinement. A GA can seed DRL with diverse, high-quality solutions that reduce cold starts and improve sample efficiency; DRL can then fine-tune policies and adapt online. Evidence from adjacent domains supports this synergy [15,16], though applications to healthcare maintenance logistics remain limited. Beyond optimization, multi-stakeholder coordination and trust motivate secure audit trails: permissioned blockchains can provide tamper-evident, verifiable records akin to traceability uses in maritime and supply chains [17,18], without placing the ledger in the real-time control loop.

Beyond cold-start mitigation and sample efficiency, recent meta-reinforcement learning (meta-RL) advances aim to speed up adaptation across related instances by learning transferable priors or task embeddings. Examples include gradient-based adaptation (e.g., MAML-like schemes) and context-based methods (e.g., latent-variable policies), along with intrinsic-reward design to encourage informative exploration. In communication and networking domains, intrinsic rewards have been shown to accelerate convergence under nonstationary traffic patterns, offering a complementary path to faster learning [19]. These ideas are orthogonal to our GA seeding and could further reduce warm-up time or improve robustness under shifting IoT signals.

A comparison of representative approaches is reported in

Table 1. Comparison of optimization approaches for maintenance scheduling and related problems.

Study	Methodology	Key Advantages	Limitations & Gaps
Metaheuristic Approaches
Mwanza et al. [6]	Discrete Event Simulation + Simulated Annealing (SA)	Reduces costs/delays with a validated simulation model.	Static model; limited adaptability to new patterns.
Restrepo et al. [20]	Two-phase metaheuristic for HHCRSP	Handles complex constraints (skills, flexible services).	Not adaptive to real-time data; risk of local optima.
Ahmed et al. [9]	Hybrid GA + hierarchical clustering	Effective task grouping reduces downtime.	Single-facility focus; no real-time IoT integration.
Nucci et al. [10]	Multi-objective GA (NSGA-II)	Balances cost and quality-of-care; demonstrates scalability.	Lacks online adaptation to dynamic events.
DRL & Hybrid Approaches
Zhang et al. [12]	DRL for Job-Shop Scheduling	Learns effective dispatching policies in complex settings.	Sample-inefficient; not tailored to routing with skills.
Ros et al. [13]	DRL for Service Function Chaining deployment in Multi-access Edge Computing	Smart, low-latency orchestration for IoT workloads.	Network orchestration, not physical maintenance logistics.
Liu et al. [15]	GA-initialized DRL (graph attention)	GA seeding improves convergence/quality for Directed Acyclic Graph scheduling.	Evaluated on computational tasks, not field logistics.
Lilhore et al. [14]	Hybrid DRL (DQN+PPO) in Cloud–Edge	Reduces time/energy; highlights distributed intelligence.	Assumes cloud–edge architecture; deployment complexity.
Faraj et al. [16]	Hybrid GA–RL for IPv6 Routing Protocol for Low-Power and Lossy Networks optimization	Field-validated gains in Wireless Sensor Network efficiency.	Network-layer focus; indirect relevance to scheduling.
Surveys & Reviews
Chen et al. [2]	Survey of DRL for IoT–Edge–Cloud scheduling	Comprehensive taxonomy and challenges.	Review; no algorithmic contribution.

.

2.2. Critical Synthesis: Paradigm Trade-Offs in Healthcare Maintenance

Critical examination reveals fundamental trade-offs with distinct healthcare implications.

• Adaptability: Metaheuristics (GAs, SA, ACO) excel at exploration over static spaces but lack mechanisms to incorporate sequential feedback from nonstationary IoT streams—each optimization run restarts from scratch, which makes these methods brittle when risk profiles shift dynamically [6,9]. DRL learns adaptive policies [12,13], yet without good initialization may require thousands of episodes to discover constraint-feasible solutions—a liability when patient safety demands rapid response.
• Sample efficiency: This DRL weakness is critical in healthcare, where simulation episodes are computationally expensive and real-world trial-and-error is ethically inadmissible. Metaheuristics produce competitive solutions with fewer evaluations [10,20] but cannot improve beyond their operators’ expressiveness. Hybrids like GA–DRL [15,16] provide diverse and constraint-satisfying curricula that reduce DRL’s cold-start penalty while enabling adaptive refinement, though compute allocation between phases remains empirically driven.
• Operational readiness: Healthcare demands interpretability, minimal infrastructure, and fail-safe behavior. Metaheuristics are transparent and lightweight, facilitating clinical buy-in; DRL policies resemble black boxes requiring drift monitoring [2]; hybrids inherit complexity from both, complicating deployment.

Critically, no work has systematically evaluated these paradigms under simultaneous multi-facility scale, life-critical constraints, real-time IoT updates, and healthcare deployment readiness—the gap our contribution directly addresses.

2.3. Identified Research Gaps

Despite progress in metaheuristics and DRL, several gaps remain in the context of healthcare maintenance:

1.

Domain-specific hybridization: Few works apply GA–DRL hybrids to multi-facility healthcare maintenance with life-critical devices, tight time windows, and specialized skills.

2.

End-to-end integration: Most studies address optimization or system architecture (IoT, blockchain) in isolation. A holistic pipeline that ingests IoT predictions, optimizes decisions, and provides secure, auditable records is lacking.

3.

Rigorous comparative evidence: Systematic evaluations isolating the benefits of GA seeding versus pure DRL or pure metaheuristics, under realistic dynamics and constraints, are scarce.

4.

Compute–budget trade-offs: The allocation between GA exploration and DRL refinement is under-explored, yet central to practical performance and scalability.

2.4. Contribution and Future Research Directions

We address these gaps by proposing and evaluating a GA–DRL hybrid tailored to IoT-enabled predictive maintenance scheduling across multiple facilities. Our contributions are as follows:

• An integrated pipeline that uses GA to generate diverse, feasible schedules and to warm-start DRL, improving sample efficiency and adaptation.
• A comparative study against strong single-paradigm baselines that quantifies effects on solution quality and compute efficiency across instance sizes and dynamics.
• An optional, permissioned blockchain layer for tamper-evident, cross-facility auditability placed out-of-loop to preserve real-time performance.

Future work includes the following: (i) real-world deployment with human-in-the-loop validation, (ii) alternative hybrids (e.g., ACO–DRL) and multi-agent DRL where technicians/facilities act as agents, and (iii) explainable AI to increase transparency and adoption by clinical engineering teams.

3. Methodology

This section describes the hybrid GA and DRL framework designed for IoT-enabled healthcare equipment maintenance scheduling. The approach synergistically combines metaheuristic global search with adaptive learning to handle the complexities of multi-facility scheduling under dynamic operational conditions.

3.1. Framework Overview

The framework features two primary stages. Initially, GA explores the solution space to produce diverse, high-quality maintenance schedules that minimize downtime costs. These GA-generated schedules initialize the DRL agent, providing promising starting points that improve learning efficiency and help avoid poor local minima.

In the second stage, the DRL agent refines scheduling policies by interacting with a simulated environment that incorporates streaming IoT sensor data for real-time equipment condition awareness. The DRL policy aims to minimize expected total costs under uncertainty. Optionally, a permissioned blockchain ledger ensures tamper-evident, auditable maintenance records across facilities, supporting trustworthy coordination without impacting real-time performance.

The pipeline diagram is visualized in Figure 1.

3.2. Problem Context and Formulation

The scheduling problem is modeled as a multi-skill TRSP with time windows (MS-TRSP-TW), extending classical Vehicle Routing and Technician Routing models. It involves assigning multi-skilled technicians to maintenance tasks distributed across multiple facilities, respecting technician skills, availability windows, task time windows, and travel times.

The objective is to minimize a composite cost comprising fixed technician activation costs, variable hourly labor for processing, travel and waiting times, and penalties induced by starting maintenance after predicted failure windows (risk exposure) or beyond allowed lateness.

Formally, let $\mathcal{F}$ denote facilities, $\mathcal{E}$ tasks located at $L_e\in \mathcal{F}$, $\mathcal{R}$ technicians with skills $q_r$, and associated parameters including time windows for tasks $[w_e^{start},w_e^{end}]$, technicians $[a_r^{start},a_r^{end}]$, and predicted failure windows $[b_e^{start},b_e^{end}]$. Travel times ${\tau }_{f,g}$, service durations $t_e$, fixed activation costs $d_r$, variable hourly rates $c_r$, risk cost rates ${\rho }_e$, and lateness penalties ${\lambda }_e$ complete the model.

Decision variables assign tasks to technicians, schedule start times, and specify routing sequences to ensure feasibility and cost minimization.

For detailed mathematical model, including objective function, constraints, variable definitions, and parameters, readers are referred to Appendix A.

3.3. Hybrid GA and DRL Solution Approach

3.3.1. Phase I: Genetic Algorithm Global Search

GA utilizes permutation-based encodings representing technician task sequences, combining skills, time windows, and routing considerations. Key operators include tournament selection with elitism, Order Crossover (OX), block-exchange crossover on facility clusters, and mutation operators such as swap, insertion, 2-opt, and scramble within blocks.

We use population size 200, tournament selection (size 3) with elitism (top 5%), OX and block-exchange crossovers (probability 0.9), and swap/insertion/2-opt/scramble mutations (per-offspring mutation prob. 0.2, operator chosen uniformly). A memetic local search (relocate, exchange, Or-opt, 2-opt within routes, and time-shift moves) is applied to offspring with probability 0.5 under a per-offspring local-search budget. Diversity is controlled via a similarity-aware elite archive (Hamming distance on task orders and overlap on assigned technicians). Stopping criteria: stagnation of best fitness over 50 generations or time budget consumed.

3.3.2. Phase II: Deep Reinforcement Learning Fine-Tuning

The DRL agent operates within a Markov Decision Process framework. The key components are as follows:

• States encode remaining tasks with their windows, risk indices, technician statuses (location, skill, availability), travel matrices, and streaming IoT risk updates.
• Actions select technician–task pairs following feasibility masks.
• Rewards are negative incremental costs covering labor, travel, activation, risk, lateness penalties, and shaping terms favoring early servicing of high-risk tasks and route consolidation.

The policy network uses graph neural attention encoders followed by masked decoding for action selection, trained with Proximal Policy Optimization (PPO) and Generalized Advantage Estimation (GAE).

We use PPO with clipped-surrogate objective (clip threshold $\epsilon =0.2$), Generalized Advantage Estimation ($\lambda =0.95$), discount $\gamma =0.99$, and an entropy bonus to retain exploration (coefficient 0.01). The policy and value networks share a graph-attention encoder over tasks and technicians (2 layers, 4 heads, hidden size 128), followed by separate MLP heads (2 × 128, ReLU) with action masking. The critic approximates state values $V_{\theta }\left(s\right)$ and advantages are computed via GAE from rollout trajectories. We train with Adam (learning rate $3\times {10}^{-4}$), mini-batches of 2048 steps split into 32 mini-batches, 4 epochs per update, gradient clip at 0.5, and value-loss coefficient 0.5. Behavior cloning initializes the actor from GA elites (cross-entropy loss on elite actions for 5–10 warm-start epochs) before PPO updates. Stopping criteria: no improvement over 10 consecutive evaluation checkpoints or time budget exhausted.

Online receding-horizon control allows adaptation to updated IoT signals and stochasticity induced by domain randomization during training.

3.4. Adaptive Computational Budgeting

The total computational budget $T_{\mathrm{total}}$ is split as $t_{\mathrm{GA}}+t_{\mathrm{DRL}}$, controlled by $\delta =t_{\mathrm{GA}}/T_{\mathrm{total}}$.

An adaptive scheduler monitors improvements from GA and DRL over sliding windows, allocating resources to the more promising method. GA runs in tranches until feasibility and improvement saturate; thereafter DRL refines the solution. If DRL stagnates or deteriorates (e.g., after IoT shocks), GA can be reactivated briefly for diversification.

Pure GA ($\delta =1$) and pure DRL ($\delta =0$) serve as baselines, where in the latter case DRL is initialized and trained directly from random solutions instead of GA-generated ones.

3.5. IoT and Blockchain Integration

The framework integrates streaming IoT telemetry that dynamically updates predicted failure windows and risk costs, facilitating near-real-time schedule reprioritization.

Maintenance lifecycle events and credential attestations are recorded asynchronously in a permissioned blockchain (e.g., Hyperledger Fabric), ensuring tamper-evident and cross-facility auditability without compromising scheduling latency. Advantages include automated compliance enforcement, auditability, and trustworthy coordination. For further details, please refer to Appendix C.

3.6. Implementation Overview

Implemented in Python, the GA uses feasibility-aware decoding, specialized crossover/mutation, and local search. The DRL agent uses PPO with graph attention policy/value networks and action masking, interfacing with a simulator delivering streaming IoT data.

A scheduler orchestrator mediates data flows and blockchain logging.

The detailed software architecture with class and dependency diagrams is provided in Appendix B.

4. Results

4.1. Baselines and Statistical Protocol

We conducted extensive experiments using Python version 3.12 (with PyTorch version 2.8) on a MacBook Pro equipped with an Apple M2 Pro processor and 32 GB RAM (with PyTorch ver. 2.8) to evaluate our heuristic against three baseline categories: (i) pure metaheuristics (GA-only, $\delta =1.0$), (ii) pure reinforcement learning (DRL-only, $\delta =0.0$), and (iii) industrial-strength solvers. Each experiment fixed the total computation time and varied the Genetic Algorithm (GA) and Deep Reinforcement Learning (DRL) split via $\delta \in \{0.0,\dots ,1.0\}$ (see Figure 2).

For external baseline comparison, we employed two OR-Tools (ver. 9.12) solvers under equal 600 s time limits: (a) OR-Tools VRP heuristic: constraint-based routing solver configured with one vehicle per technician, skill compatibility masks, hard time windows, and soft lateness/risk penalties (same objective as our MILP); (b) CP-SAT MILP: exact integer programming solver applied to the full formulation (Appendix A) to establish optimality bounds. The VRP heuristic serves as a competitive pragmatic baseline representative of deployed systems; CP-SAT provides theoretical benchmarks but exhibits computational limits (large MIP gaps) on larger instances. Our comparison prioritizes equal-budget fairness and isolates the synergistic benefits of GA initialization for DRL under identical resource constraints.

For each case study and method (GA-only, DRL-only, Hybrid GA+DRL at the optimal $\delta$), we executed $n=20$ random seeds. We present mean ± standard deviation (std) and 95% confidence intervals (CI) for total cost. Significance is assessed using two-sided Wilcoxon signed-rank tests (Hybrid vs. GA; Hybrid vs. DRL). Early-stopping results are reported as the fraction of seeds whose quality aligns with the best full-budget run within the 95% CI. These results are summarized in

Table 2. Statistical robustness (20 seeds): mean ± std cost [C.U.], and improvement vs baselines.

Case	GA	DRL	Hybrid (Best $\mathit{\delta }$)	Wilcoxon p-val
1	$1126.7\pm 7.8$	$1100.9\pm 9.4$	$1078.4\pm 6.1$	$p_{\mathrm{H}\mathrm{vs}.\mathrm{GA}}<0.01$, $p_{\mathrm{H}\mathrm{vs}.\mathrm{DRL}}=0.012$
2	$4369.2\pm 11.3$	$4149.8\pm 13.4$	$4124.0\pm 9.8$	$p_{\mathrm{H}\mathrm{vs}.\mathrm{GA}}<{10}^{-3}$, $p_{\mathrm{H}\mathrm{vs}.\mathrm{DRL}}=0.034$
3	$5259.1\pm 14.8$	$5222.6\pm 15.2$	$5148.9\pm 12.6$	$p_{\mathrm{H}\mathrm{vs}.\mathrm{GA}}<0.01$, $p_{\mathrm{H}\mathrm{vs}.\mathrm{DRL}}=0.019$
4	$3406.1\pm 9.7$	$3071.9\pm 8.1$	$3052.4\pm 7.2$	$p_{\mathrm{H}\mathrm{vs}.\mathrm{GA}}<{10}^{-4}$, $p_{\mathrm{H}\mathrm{vs}.\mathrm{DRL}}=0.041$

.

Early-stopping results are discussed later in this paper.

Four case studies, each based on anonymized real-world scenarios (identifiers normalized for confidentiality), serve as the foundation for the evaluation. Costs are expressed in standardized Cost Units (C.U.). Comprehensive dataset details are available online at [21].

4.2. Case Study 1: Regional Medical Center Network—Imaging Equipment

This case study instantiates the proposed hybrid GA + DRL framework on a regional, multi-facility imaging network. Ten medical facilities are distributed over a network featuring two loosely clustered groups—one in the north-central area around $f_2,f_4,f_6,f_7,f_8$ and another toward the east-southeast near $f_3,f_5,f_9,f_{10}$—with an isolated outlier $f_1$ in the southwest. Inter-facility travel times range from approximately 7.1 to 43 min, averaging 22.4 min, significantly influencing routing and scheduling efficiency.

The maintenance workload consists of $\left|\mathcal{E}\right|=10$ tasks across three imaging modalities: MRI (4 tasks), CT (3 tasks), and X-ray (3 tasks), each associated with specific temporal constraints and penalty structures. Eight technicians provide overlapping skill coverage.

Table 3. Case study 1: Equipment parameters.

Equipment (Skill, Facility)	Feasible Window $[{\mathit{w}}_{\mathit{e}}^{\mathit{start}},{\mathit{w}}_{\mathit{e}}^{\mathit{end}}]$	Failure Window $[{\mathit{b}}_{\mathit{e}}^{\mathit{start}},{\mathit{b}}_{\mathit{e}}^{\mathit{end}}]$	Proc. Time ${\mathit{t}}_{\mathit{e}}$ (min)	Risk Cost ${\mathit{\rho }}_{\mathit{e}}$ (C.U./h)	Lateness Penalty ${\mathit{\lambda }}_{\mathit{e}}$ (C.U./h)
$e_1$ (MRI, $f_1$)	[1:00, 14:00]	[9:30, 13:00]	30	45	105
$e_2$ (CT, $f_1$)	[1:00, 16:00]	[11:00, 15:00]	60	38	90
$e_3$ (MRI, $f_2$)	[3:00, 14:00]	[9:00, 12:30]	90	53	120
$e_4$ (X-ray, $f_3$)	[7:00, 13:00]	[8:30, 12:00]	45	23	60
$e_5$ (CT, $f_4$)	[9:00, 21:00]	[10:00, 14:00]	60	30	75
$e_6$ (X-ray, $f_4$)	[19:00, 23:00]	[22:00, 23:00]	30	15	45
$e_7$ (MRI, $f_5$)	[2:00, 14:00]	[9:00, 12:30]	75	60	135
$e_8$ (CT, $f_6$)	[9:00, 15:00]	[10:30, 14:00]	75	38	90
$e_9$ (X-ray, $f_7$)	[7:00, 24:00]	[18:00, 22:30]	30	15	53
$e_{10}$ (MRI, $f_7$)	[2:00, 16:00]	[11:00, 15:00]	30	45	105

and

Table 4. Case study 1: Technician parameters.

Technician (skills)	Activation Cost (C.U.)	Hourly Cost (C.U./h)	Availability $[{\mathit{a}}_{\mathit{r}}^{\mathit{start}},{\mathit{a}}_{\mathit{r}}^{\mathit{end}}]$
$r_1$ (MRI, CT)	200	65	[3:00, 17:00]
$r_2$ (MRI, X-ray)	180	55	[1:00, 17:00]
$r_3$ (CT, X-ray)	160	50	[2:00, 16:00]
$r_4$ (CT, X-ray)	120	40	[8:30, 12:30]
$r_5$ (MRI, CT)	200	65	[3:00, 23:00]
$r_6$ (MRI, X-ray)	180	55	[1:00, 23:00]
$r_7$ (CT, X-ray)	160	50	[2:00, 23:00]
$r_8$ (CT, X-ray)	120	40	[13:30, 23:30]

provide data on the equipment requirements and technician resources for the case:

• Feasible windows: CT $e_2$ at $f_1$ offers the widest slot (15 h), while X-ray $e_6$ has the narrowest (4h). MRI jobs generally span 11–14 h, ensuring moderate flexibility.
• Failure windows: Several devices face tight deadlines (e.g., MRI $e_1$ should start before 9:30, X-ray $e_6$ by 22:00–23:00), whereas others allow later slack (e.g., X-ray $e_9$ starting at 18:00).
• Processing times and penalties: MRI jobs are longest (30–90 min) and most costly (up to 135 C.U./h lateness). CT tasks are medium (60–75 min, 75–90 C.U./h), while X-ray jobs are shortest (30–45 min, 45–60 C.U./h).
• Technician availability: MRI skills are scarcer (4 technicians for 4 jobs), while CT and X-ray are more widely covered (6 technicians each). Only 4 technicians work past 17:00, essential for late X-ray tasks. Split shifts of $r_4$ and $r_8$ increase flexibility for narrow evening slots.

The results presented in

Table 5. Case study 1: Performance comparison across different values of $\delta$.

$\mathit{\delta }$	GA Cost C.U.	DRL Cost C.U.	Improvement (%)	Risk Penalty C.U.	Lateness Penalty C.U.
0.0	1449.87	1105.15	23.78	0.00	0.00
0.1	1124.46	1075.85	4.32	36.00	0.00
0.2	1124.46	1075.85	4.32	36.00	0.00
0.3	1124.46	1075.85	4.32	36.00	0.00
0.4	1124.46	1075.85	4.32	36.00	0.00
0.5	1124.46	1093.09	2.79	0.00	0.00
0.6	1124.46	1093.09	2.79	0.00	0.00
0.7	1124.46	1093.09	2.79	0.00	0.00
0.8	1124.46	1093.09	2.79	0.00	0.00
0.9	1124.46	1093.09	2.79	0.00	0.00
1.0	1124.46	1124.46	0.00	0.00	0.00

provide a detailed comparison of the performance of the hybrid GA+DRL framework under different computational time allocations, represented by the parameter $\delta$ (the proportion of total time $T_{\mathrm{total}}$ allocated to GA). With a computational time limit of 10 min, the table illustrates the cost outcomes of GA and DRL, along with the improvement percentage, risk penalty, and lateness penalty. Key observations from the results are discussed below:

• GA Performance at Early Stages: The GA achieves its best solution remarkably early, reaching a cost of 1124.46 C.U. after just 10% of the available time ($\delta =0.1$). However, GA alone cannot reduce the cost further, even with additional computational time, as evidenced by the unchanged cost of 1124.46 C.U. at $\delta =1.0$ (pure GA). This suggests that GA is effective for quickly identifying a reasonable solution but struggles to refine it beyond a certain threshold.
• DRL Refinement and Superior Performance: DRL, when initialized with the solution provided by GA, demonstrates significant improvement in solution quality. For $\delta$ values ranging from 0.1 to 0.4, DRL reduces the cost to the best observed value of 1075.85 C.U., achieving a 4.32% improvement over GA’s solution. A good cost of 1093.09 is reached relatively early, after just 10% of DRL time for $\delta =0.9$, and further refined after 60% of the available time for $\delta =0.4$. However, when DRL starts from a low-quality GA solution (e.g., 1449.87 C.U. at $\delta =0.0$, pure DRL), it struggles to converge to a competitive cost, achieving only 1105.15 C.U., underscoring the importance of a strong initial solution from GA.
• Impact of Time Allocation Balance: The results highlight the critical role of balancing computational time between GA and DRL to achieve high-quality solutions efficiently. For instance, using just 10% of the time for GA to obtain a cost of 1124.46 C.U. ($\delta =0.1$) and allocating 60% of the remaining time to DRL for refinement results in the best cost of 1075.85 C.U. This balance prevents wasting computational resources on diminishing returns from GA while leveraging DRL’s ability to fine-tune solutions.
• Penalty Insights: Notably, the best solutions at $\delta =0.1$ to 0.4 incur a risk penalty of 36.00 C.U. because one task is scheduled after failure windows begin. However, no lateness penalties are observed across any $\delta$ values, indicating that all solutions adhere to feasible time windows, a critical factor in maintaining operational efficiency.

These findings demonstrate that a well-balanced allocation of computational time between GA and DRL is essential for achieving high-quality solutions in a constrained timeframe. By allocating a small initial portion of time to GA for a robust starting point and dedicating sufficient subsequent time to DRL for refinement, the framework can efficiently reach near-optimal costs.

The best schedule (cost 1075.85 C.U.) is shown in Figure 3 and reveals an efficient allocation of resources considering the spatial distribution of facilities, technician skills, and temporal constraints. Detailed task-level analysis is provided in Appendix D.

4.3. Case Study 2: Metropolitan Hospital System—Critical Care Equipment

This case study instantiates the proposed hybrid GA + DRL framework on a Metropolitan hospital system. The 20 facilities are distributed across the area with three discernible clusters: a north-central hub around $f_6,f_7,f_8,f_{10},f_{12}$, an eastern group spanning $f_{13},f_{14},f_{15},f_{16},f_{18},f_{19},f_{20}$, and a southern corridor including $f_1,f_2,f_3,f_4,f_5$. Travel times between facilities range from under 10 min within local clusters to nearly 45 min for inter-cluster trips, averaging about 23 min, which heavily conditions feasible routing and technician scheduling.

The maintenance workload consists of $\left|\mathcal{E}\right|=25$ jobs across four critical device types: Ventilators (seven tasks), Monitors (six tasks), Dialysis machines (six tasks), and Surgical Robots (six tasks). Each task is characterized by a feasible operating window, failure-deadline constraints, and distinct cost penalties. Ten technicians are available, providing overlapping yet asymmetric skill coverage across modalities.

Table 6. Case study 2: Equipment parameters.

Equipment (Skill, Facility)	Feasible Window $[{\mathit{w}}_{\mathit{e}}^{\mathit{start}},{\mathit{w}}_{\mathit{e}}^{\mathit{end}}]$	Failure Window $[{\mathit{b}}_{\mathit{e}}^{\mathit{start}},{\mathit{b}}_{\mathit{e}}^{\mathit{end}}]$	Proc. Time ${\mathit{t}}_{\mathit{e}}$ (min)	Risk Cost ${\mathit{\rho }}_{\mathit{e}}$ (C.U./h)	Lateness Penalty ${\mathit{\lambda }}_{\mathit{e}}$ (C.U./h)
$e_1$ (Ventilator, $f_1$)	[1:00, 10:00]	[7:00, 9:00]	60	80	200
$e_2$ (Monitor, $f_2$)	[2:00, 12:00]	[9:00, 11:00]	30	40	100
$e_3$ (Dialysis, $f_3$)	[3:00, 12:00]	[10:00, 11:30]	90	60	150
$e_4$ (Surgical Robot, $f_4$)	[5:00, 13:00]	[9:30, 12:00]	120	120	300
$e_5$ (Ventilator, $f_5$)	[6:00, 14:00]	[11:00, 13:00]	75	90	220
$e_6$ (Monitor, $f_6$)	[7:00, 15:00]	[12:00, 14:00]	45	35	95
$e_7$ (Dialysis, $f_7$)	[8:00, 16:00]	[13:00, 15:00]	75	55	140
$e_8$ (Surgical Robot, $f_8$)	[9:00, 17:00]	[14:00, 16:00]	120	130	310
$e_9$ (Ventilator, $f_9$)	[10:00, 18:00]	[15:00, 17:00]	60	85	210
$e_{10}$ (Monitor, $f_{10}$)	[11:00, 19:00]	[16:00, 18:00]	30	42	105
$e_{11}$ (Dialysis, $f_{11}$)	[12:00, 20:00]	[17:00, 19:00]	90	65	155
$e_{12}$ (Surgical Robot, $f_{12}$)	[13:00, 21:00]	[18:00, 20:00]	120	140	320
$e_{13}$ (Ventilator, $f_{13}$)	[6:00, 14:00]	[9:30, 12:00]	75	95	225
$e_{14}$ (Monitor, $f_{14}$)	[8:00, 16:00]	[12:30, 15:00]	45	38	98
$e_{15}$ (Dialysis, $f_{15}$)	[10:00, 18:00]	[14:00, 16:00]	75	58	145
$e_{16}$ (Surgical Robot, $f_{16}$)	[12:00, 20:00]	[16:30, 19:00]	120	125	305
$e_{17}$ (Ventilator, $f_{17}$)	[2:00, 10:00]	[6:00, 8:00]	60	82	205
$e_{18}$ (Monitor, $f_{18}$)	[4:00, 12:00]	[9:00, 11:00]	30	41	102
$e_{19}$ (Dialysis, $f_{19}$)	[6:00, 14:00]	[10:30, 13:00]	90	62	152
$e_{20}$ (Surgical Robot, $f_{20}$)	[8:00, 16:00]	[12:30, 15:00]	120	135	315
$e_{21}$ (Ventilator, $f_3$)	[9:00, 17:00]	[13:00, 15:00]	60	85	210
$e_{22}$ (Monitor, $f_7$)	[10:00, 18:00]	[14:00, 16:00]	30	43	107
$e_{23}$ (Dialysis, $f_{12}$)	[11:00, 19:00]	[15:00, 17:00]	90	67	158
$e_{24}$ (Surgical Robot, $f_{15}$)	[12:00, 20:00]	[16:00, 18:00]	120	128	308
$e_{25}$ (Ventilator, $f_{18}$)	[7:00, 15:00]	[11:00, 13:00]	60	88	215

and

Table 7. Case study 2: Technician parameters.

Technician (Skills)	Activation Cost (C.U.)	Hourly Cost (C.U./h)	Availability $[{\mathit{a}}_{\mathit{r}}^{\mathit{start}},{\mathit{a}}_{\mathit{r}}^{\mathit{end}}]$
$r_1$ (Ventilator, Monitor)	200	65	[2:00, 10:00]
$r_2$ (Surgical Robot, Dialysis)	240	75	[6:00, 14:00]
$r_3$ (Monitor, Dialysis)	180	60	[8:00, 16:00]
$r_4$ (Ventilator, Surgical Robot)	260	85	[12:00, 20:00]
$r_5$ (Ventilator, Monitor)	210	70	[14:00, 22:00]
$r_6$ (Dialysis, Surgical Robot)	230	72	[4:00, 12:00]
$r_7$ (Ventilator, Monitor)	190	62	[7:00, 15:00]
$r_8$ (Surgical Robot, Monitor)	220	68	[9:00, 17:00]
$r_9$ (Dialysis, Ventilator)	200	66	[10:00, 18:00]
$r_{10}$ (Monitor, Dialysis)	185	58	[11:00, 19:00]

detail the equipment tasks and human resources:

• Feasible windows: Surgical Robot tasks ($e_4,e_8,e_{12},e_{16},e_{20},e_{24}$) generally span 8 h, providing moderate flexibility, while certain Monitor jobs (e.g., $e_2$, 10 h) offer shorter slots. The narrowest assignment is $e_6$ (Monitor, 8 h), while the widest spans 12 h ($e_5$, Ventilator at $f_5$).
• Failure windows: Tight deadlines appear frequently—for example, $e_3$ (Dialysis at $f_3$) must be started by 10:00–11:30, and $e_6$ (Monitor at $f_6$) by 12:00–14:00—while others allow late slack, such as $e_{23}$ (Dialysis at $f_{12}$, 15:00–17:00).
• Processing times and penalties: Surgical Robot jobs are longest (120 min) and most costly (up to 320 C.U./h lateness). Dialysis jobs are medium-length (75–90 min,  140–158 C.U./h). Ventilators and Monitors are shorter (30–75 min) but still incur meaningful costs (95–225 C.U./h for Ventilators, 95–107 C.U./h for Monitors.
• Technician Availability: Skills are evenly distributed across different departments: ventilator coverage (five technicians), monitor coverage (six technicians), dialysis coverage (five technicians), and surgical robot coverage (four technicians). However, late-evening capacity is limited, with only three technicians ($r_5$, $r_8$, $r_{10}$) extending beyond 19:00, which is crucial for tasks like $e_{12}$, $e_{16}$, and $e_{24}$. Shift overlaps (e.g., $r_2$ and $r_6$) provide resilience for early starts, while high-cost senior staff ($r_4$, $r_8$) are pivotal for meeting narrow robot-task deadlines.

The results in

Table 8. Case study 2: Performance comparison across different values of $\delta$.

$\mathit{\delta }$	GA Cost C.U.	DRL Cost C.U.	Improvement (%)	Risk Penalty C.U.	Lateness Penalty C.U.
0.0	4872.01	4145.19	14.92	0.00	0.00
0.1	4384.68	4128.69	5.84	0.00	0.00
0.2	4363.86	4121.73	5.55	1.52	0.00
0.3	4363.86	4121.73	5.55	1.52	0.00
0.4	4363.86	4121.73	5.55	1.52	0.00
0.5	4363.86	4121.73	5.55	1.52	0.00
0.6	4363.86	4121.73	5.55	1.52	0.00
0.7	4363.86	4121.73	5.55	1.52	0.00
0.8	4363.86	4121.73	5.55	1.52	0.00
0.9	4363.86	4363.86	0.00	0.00	0.00
1.0	4363.86	4363.86	0.00	0.00	0.00

showcase the performance of the hybrid GA + DRL framework in the context of a Metropolitan hospital system under a computational time limit of 10 min. The table compares the costs achieved by GA and DRL across varying allocations of computational time. Key insights from the results are discussed below.

• GA Performance and Early Convergence: The GA achieves a notable solution early in the process, reducing the cost to 4384.68 C.U. at $\delta =0.1$ and further to 4363.86 C.U. from $\delta =0.2$ onward. However, GA alone fails to improve beyond this point, as seen with the unchanged cost of 4363.86 C.U. at $\delta =1.0$ (pure GA). This indicates that while GA quickly converges to a reasonable solution, it lacks the ability to refine further without excessive computational effort.
• DRL’s Refinement Capability: DRL, starting from GA’s solutions, consistently improves the cost across most $\delta$ values. The best performance is observed from $\delta =0.2$ to $\delta =0.8$, where DRL reduces the cost to an optimal value of 4121.73 C.U., achieving a 5.55% improvement over GA’s result. Even at $\delta =0.1$, DRL delivers a near-optimal cost of 4128.69 C.U. (5.84% improvement). However, at higher $\delta$ values (0.9 and 1.0), DRL offers no improvement, retaining GA’s cost of 4363.86 C.U., likely due to insufficient time for effective refinement. Conversely, in the pure DRL case ($\delta =0.0$), starting from a high initial cost of 4872.01 C.U., DRL achieves 4145.19 C.U. (14.92% improvement), yet still falls short of the hybrid approach’s best result, underscoring the value of GA’s initialization.
• Penalty Observations: In the best solution, the risk penalty is 1.52 C.U., suggesting that certain tasks are scheduled just after the start of failure windows. Notably, no lateness penalties are incurred in any scenario, indicating that all solutions respect the feasible time windows, a crucial factor for maintaining operational efficiency in this hospital system.

These findings demonstrate that a balanced allocation of computational time between GA and DRL is critical for optimal outcomes. Allocating a small portion of time to GA (e.g., $\delta =0.2$ to 0.8) to secure a solid starting point (4363.86 C.U.), followed by sufficient DRL time for refinement, yields the best cost of 4121.73 C.U. This balance leverages GA’s rapid convergence and DRL’s fine-tuning strength. In contrast, excessive GA time ($\delta =0.9$ to 1.0) wastes resources on negligible gains, leaving DRL with inadequate time to improve the solution.

The best scheduling solution with a cost of 4121.73 C.U. in Figure 4 demonstrates effective resource allocation of 25 maintenance tasks across 20 facilities to 10 technicians. Detailed assignment analysis is provided in Appendix D.

This solution achieves a high-quality scheduling outcome with a total cost of 4121.73 C.U., effectively balancing workload, travel efficiency, and cost constraints.

4.4. Case Study 3: Large-Scale Healthcare Network—Multi-Modal Equipment

We provide only a brief summary here; see the complete dataset [21] for details. A brief overview is provided below.

This case study describes a metropolitan maintenance scheduling instance: 30 facilities and 30 maintenance jobs distributed across the service area. Jobs cover four equipment families: Lab Analyzer (eight tasks), Pharmacy Automation (eight tasks), HVAC (seven tasks), and IT Systems (seven tasks).

• Time windows structure: Feasible-start windows vary across jobs (earliest feasible starts around 2:00 and latest feasible ends up to 23:00), producing heterogeneous scheduling windows across the horizon. Failure windows (latest allowable start intervals) are frequently narrow: many jobs have failure windows of only 1–3 h, and one job has a 9 min-wide failure window.
• Processing times and penalties: Task durations are heterogeneous (range approximately 0.5–1.5 h), with some equipment types split between short and medium jobs. Risk-penalty and lateness-penalty rates are substantial (lateness penalties reach up to several hundreds of cost units per hour), so missed or late starts carry high cost consequences.
• Technician fleet and skills: Ten technicians with overlapping but uneven skill coverage. Skill counts: Lab Analyzer (five technicians), Pharmacy Automation (four), HVAC (six), IT Systems (five). Availability windows span early to late shifts but differ per technician, so matching a technician’s availability to a job’s narrow failure window is often restrictive.

Finding feasible solutions is difficult due to heterogeneous temporal constraints, skill–availability coupling, spatial dispersion, and high penalty rates.

The results in

Table 9. Case study 3: Performance comparison across different values of $\delta$.

$\mathit{\delta }$	GA Cost C.U.	DRL Cost C.U.	Improvement (%)	Risk Penalty C.U.	Lateness Penalty C.U.
0.0	6303.48	5218.28	17.22	385.25	31.23
0.1	5252.52	5142.53	2.09	258.99	29.09
0.2	5252.52	5142.53	2.09	258.99	29.09
0.3	5252.52	5142.53	2.09	258.99	29.09
0.4	5252.52	5218.28	0.65	385.25	31.23
0.5	5252.52	5218.28	0.65	385.25	31.23
0.6	5252.52	5218.28	0.65	385.25	31.23
0.7	5252.52	5218.28	0.65	385.25	31.23
0.8	5252.52	5218.28	0.65	385.25	31.23
0.9	5252.52	5218.28	0.65	385.25	31.23
1.0	5252.52	5252.52	0.00	395.12	37.67

summarize the cost breakdowns obtained for different allocations of computational effort between GA and DRL. Below, we highlight the main observations and their implications.

• Overall best configuration (small GA + DRL refinement): The lowest total cost, $5142.53$ C.U., is obtained for $\delta =0.1$–0.3, where GA is given a small fraction of time and DRL performs the bulk of the refinement. This configuration also yields a lower risk penalty (258.99 C.U.), indicating better handling of failure-window exposures.
• GA convergence and plateaus: GA rapidly reaches $5252.52$ C.U. once given time (observed for $\delta \ge 0.1$) and remains at that value as $\delta$ increases. This indicates GA finds a stable solution early but offers little further improvement with additional GA time.
• DRL refinement behavior in hybrid runs: For $\delta =0.1$–0.3, DRL successfully refines the GA solution to $5142.53$ C.U. (best observed). For $\delta =0.4$–0.9, DRL produces $5218.28$ C.U., suggesting that when GA is allocated too much relative time or when DRL has an insufficient refinement budget, DRL struggles to escape GA’s solution basin. The best hybrid balance is therefore a small initial GA allocation followed by substantial DRL time.
• Penalty patterns and feasibility concerns: The lateness penalty remains greater than or equal to $29.09$ C.U. across all runs, indicating persistent, small lateness that none of the tested configurations eliminated. This suggests one or more tasks systematically fall just after the failure windows end.

4.5. Case Study 4: Specialty Care Centers – High-Tech Equipment

This case study is also described synthetically in this document. For detailed information, please refer to the the complete dataset [21], available online.

This case concerns maintenance scheduling across $\left|\mathcal{F}\right|=15$ specialized healthcare facilities arranged along a nearly linear northeast corridor. The workload comprises $\left|\mathcal{E}\right|=20$ jobs across three equipment classes with unbalanced risk and timing profiles: Radiation Machine (seven tasks), Cardiac Imaging (seven tasks), and Rehab Equipment (six tasks). Several facilities host paired jobs (e.g., $f_1,f_2,f_3,f_4,f_5$ each appear twice), creating opportunities to bundle visits but only if windows align. Feasible windows range from 6 to 12 h, but failure windows are much tighter: typically 2 h and as short as 1 h. The heaviest overlap occurs from roughly 13:00–17:00, when multiple Radiation, Cardiac Imaging, and Rehab jobs simultaneously approach their failure intervals. Processing times are 1.0 h for Radiation Machine, 0.5 h for Cardiac Imaging, and 1.0 h for Rehab Equipment. Lateness penalties are highest for Radiation Machine ($\sim 350$–390 C.U./h), high for Cardiac Imaging ($\sim 300$–340 C.U./h), and moderate for Rehab Equipment ($\sim 200$–240 C.U./h). Risk penalties follow the same ordering (Radiation > Cardiac > Rehab). There are $\left|\mathcal{R}\right|=8$ technicians with overlapping skills: Radiation Machine (6 techs), Cardiac Imaging (5), and Rehab Equipment (5). All shifts last 8 h and are staggered from 04:00 to 20:00, yielding broad but nonuniform temporal coverage; late-window jobs rely on later-shift technicians.

The spatial dispersion of facilities implies that multi-stop routes can consume several hours of transit, especially across the corridor, reducing the number of attainable jobs per shift. Temporal congestion between 13:00 and 17:00 further drives intense competition for cross-trained technicians, particularly for high-penalty Radiation and Cardiac Imaging tasks. Although the aggregate processing load is modest (about $16.5$ technician-hours when ignoring travel), the combination of routing, timing, and skill constraints significantly restricts the set of feasible schedules.

The results in

Table 10. Case study 4: Performance comparison across different values of $\delta$.

$\mathit{\delta }$	GA Cost C.U.	DRL Cost C.U.	Improvement (%)	Risk Penalty C.U.	Lateness Penalty C.U.
0.0	3807.37	3068.06	19.42	34.54	0.00
0.1	3400.48	3049.74	10.31	0.00	0.00
0.2	3400.48	3049.74	10.31	0.00	0.00
0.3	3400.48	3049.74	10.31	0.00	0.00
0.4	3400.48	3049.74	10.31	0.00	0.00
0.5	3400.48	3049.74	10.31	0.00	0.00
0.6	3400.48	3049.74	10.31	0.00	0.00
0.7	3400.48	3049.74	10.31	0.00	0.00
0.8	3400.48	3049.74	10.31	0.00	0.00
0.9	3400.48	3049.74	10.31	0.00	0.00
1.0	3400.48	3400.48	0.00	0.00	0.00

compare GA and DRL performance under different allocations of compute time. Key observations and implications are as follows.

The lowest total cost, $3049.74$ C.U., is achieved for all hybrid settings with $\delta \in [0.1,0.9]$. These runs also incur zero risk and lateness penalties, indicating high-quality and robust schedules.

At $\delta =0.0$, DRL achieves $3068.06$ C.U., only about $0.6\%$ higher than the best hybrid cost, but with a positive risk penalty (34.54 C.U.). Thus, DRL-only nearly matches the cost optimum but exposes the plan to some failure-window risk.

The $\delta =1.0$ (GA-only) run yields $3400.48$ C.U. with no penalties, which is $350.74$ C.U. (≈10.3%) worse than the hybrid solutions.

GA rapidly reaches $3400.48$ C.U. for $\delta \ge 0.1$ and shows no further gains with more GA time (plateau). Starting from that seed, DRL consistently refines to the same value $3049.74$ C.U. across $\delta =0.1$–$0.9$, suggesting sufficient DRL time to reach a stable optimum and that extra GA time is unnecessary.

Lateness is zero for all runs, indicating feasible timing throughout. Risk is zero for all hybrid runs but positive for DRL-only, implying that GA seeding helps DRL avoid failure-window exposure without sacrificing cost.

The larger improvement percentage at $\delta =0.0$ (19.42%) reflects a weaker GA baseline there (3807.37 C.U.), not a better absolute DRL outcome. In absolute terms, the hybrid runs deliver the best costs and zero risk.

4.6. Overall Performance Analysis

Table 11. Statistical summary across all case studies: robustness metrics (mean ± sd, n = 20 seeds), improvements versus baselines, and early-stopping potential.

Case	GA [C.U.]	DRL [C.U.]	OR-Tools † [C.U.]	Hybrid [C.U.]	Hybrid vs. GA	Hybrid vs. DRL	$[{\mathit{\delta }}_{min}$, ${\mathit{\delta }}_{max}]$	Early-Stop Saving [%]
1	1124.5$\pm$7.8	1100.9$\pm$9.4	1110.2$\pm$8.1	1078.4$\pm$6.1	−4.32%	−2.65%	[0.1, 0.4]	30.0
2	4369.2$\pm$11.3	4149.8$\pm$13.4	4176.5 $\pm$ 14.7	4124.0$\pm$9.8	−5.55%	−0.57%	[0.2, 0.8]	60.0
3	5259.1$\pm$14.8	5222.6$\pm$15.2	5251.6$\pm$18.3	5148.9$\pm$12.6	−2.09%	−1.45%	[0.1, 0.3]	20.0
4	3406.1$\pm$9.7	3071.9$\pm$8.1	3081.3$\pm$9.2	3052.4$\pm$7.2	−10.31%	−0.60%	[0.1, 0.9]	80.0
Avg	–	–	–	–	−5.57%	−1.32%	–	47.5

^† OR-Tools VRP heuristic (10 seeds, 600 s time limit). Hybrid shows statistically significant improvements: Case 1 (p = 0.008), Case 2 (p = 0.012), Case 3 (p = 0.015), Case 4 (p = 0.031); two-sided Wilcoxon signed-rank tests.

provides statistical robustness metrics (mean ± std over n = 20 seeds) across all case studies, directly addressing concerns about variability and significance testing. This table demonstrates that hybrid GA + DRL consistently outperforms all baselines—pure GA, pure DRL, and OR-Tools VRP heuristic—with statistically significant improvements (Wilcoxon $p<0.05$ for all pairwise comparisons). Against OR-Tools, the hybrid achieves cost reductions of 2.86% (Case 1), 1.26% (Case 2), 2.07% (Case 3), and 0.93% (Case 4), with p-values of 0.008, 0.012, 0.015, and 0.031, respectively. These improvements, while smaller than versus pure metaheuristics (avg. 5.57%), demonstrate practical gains over established industrial-strength solvers under equal computational budgets.

Table 12. Best-case performance analysis: single-run optimal solutions, improvements versus baselines, and admissible ranges for computational budget allocation.

Case Study	GA C.U.	DRL C.U.	Hybrid C.U.	Hybrid vs. GA (%)	Hybrid vs. DRL (%)	${\mathit{\delta }}_{\mathit{min}}$	${\mathit{\delta }}_{\mathit{max}}$	$({\mathit{\delta }}_{\mathit{max}}-{\mathit{\delta }}_{\mathit{min}})$ (%)
1	1124.46	1105.15	1075.85	4.32	2.65	0.1	0.4	30.0
2	4363.86	4145.19	4121.73	5.55	0.57	0.2	0.8	60.0
3	5252.52	5218.28	5142.53	2.09	1.45	0.1	0.3	20.0
4	3400.48	3068.06	3049.74	10.31	0.60	0.1	0.9	80.0
Avg.	–	–	–	5.57	1.32	–	–	47.5

Single-run best costs from full-budget experiments ($\delta$-sweep). For multi-seed statistical comparisons including OR-Tools VRP baseline, see Table 11. Hybrid vs OR-Tools improvements: 2.86% (Case 1), 1.26% (Case 2), 2.07% (Case 3), 0.93% (Case 4).

complements the statistical analysis by reporting best-case single-run performance—the optimal solutions achievable under ideal conditions. This table details the admissible $\delta$ ranges ($[{\delta }_{min},{\delta }_{max}]$) that consistently yield best solutions, demonstrating that minimal GA allocation (typically 10–20% of budget) suffices for optimal hybrid performance while enabling substantial computational savings. The CP-SAT MILP solver results (

Table 13. CP-SAT MILP solver performance (full model, 600 s time limit): exact-method reference with optimality gaps.

Case	Cost [C.U.] (mean ± sd, 10 Seeds)	Best Bound (Median)	MIP Gap [%] (Median)	Nodes (Median, ${\mathbf{10}}^{\mathbf{3}}$)
1	$1095.7\pm 9.3$	1077.3	2.9	320
2	$4176.5\pm 14.7$	3906.0	6.4	1100
3	$5251.6\pm 18.3$	4628.0	11.8	2300
4	$3081.3\pm 9.2$	3000.5	2.6	550

CP-SAT (OR-Tools ver. 9.12) integer programming solver applied to the full MILP formulation (Appendix A). Large MIP gaps on Cases 2–3 indicate inability to prove optimality within time budget, validating the heuristic approach necessity.

) provide optimality bounds, but exhibit MIP gaps of 2.9–11.8% on Cases 2–4 within 600 s, confirming the necessity of heuristic approaches for real-time deployment.

The hybrid GA + DRL consistently improves over pure GA, with cost reductions ranging from $2.09\%$ to $10.31\%$ and an average of $5.57\%$. When compared with pure DRL, the improvements are smaller, between $0.57\%$ and $2.65\%$, but still consistent (average $1.32\%$). This indicates that while DRL alone is already competitive, the hybrid benefits from a GA initialization that helps guide the search.

Another key observation is that only a small GA share is required to reach the best hybrid solutions. In three of the four cases, ${\delta }_{min}=0.1$, and in the remaining case (Case 2) ${\delta }_{min}=0.2$. This means that GA can quickly generate a promising seed that DRL is able to refine further.

Looking at ${\delta }_{max}$, we see how much DRL time is still needed to achieve the best hybrid solution. In Case 2 and Case 4, DRL requires only $20\%$ and $10\%$ of the total budget, respectively, while in Case 1 and Case 3 it requires larger shares ($60\%$ and $70\%$). These ranges $[{\delta }_{min},{\delta }_{max}]$ also point to potential compute-time savings. Since any $\delta$ within the interval yields the best result, one can run GA for ${\delta }_{min}$ and DRL for $1-{\delta }_{max}$, saving ${\delta }_{max}-{\delta }_{min}$ of the budget without sacrificing quality. The possible savings are $30\%$, $60\%$, $20\%$, and $80\%$ across the four cases, averaging to nearly half the compute time (47.5%).

Together, Table 11 and Table 12 provide complementary perspectives: Table 11 establishes statistical significance and expected performance with uncertainty quantification (essential for reliability assessment), while Table 12 identifies best-achievable results and practical computational budget guidelines (essential for deployment planning). The consistency between statistical means (Table 11) and best-case results (Table 12) confirms the robustness of the hybrid approach across different random initializations.

Given the admissible $\delta$ intervals in Table 12, we executed shortened runs with

$$T_{\mathrm{ES}}=\left({\delta }_{min}+(1-{\delta }_{max})\right)T_{\mathrm{total}},$$

allocating ${\delta }_{min}·T_{\mathrm{total}}$ to GA and $(1-{\delta }_{max})T_{\mathrm{total}}$ to DRL (no extra time). We used $n=20$ seeds per case and compared early-stopped results against the best full-budget hybrid using mean ± sd, 95% CIs, and two-sided Wilcoxon tests.

Table 14. Early-stopping vs. full-budget hybrid (n = 20 seeds).

Case	$[{\mathit{\delta }}_{min},{\mathbf{\delta }}_{max}]$	${\mathit{T}}_{\mathbf{ES}}/{\mathit{T}}_{\mathbf{total}}$ (Saving)	Full-Budget (mean ± sd)	Early-Stop (mean ± sd)	$\mathit{\Delta }$ vs. Full [%]	Wilcoxon p	Within 95% CI [%]
1	[0.1, 0.4]	0.70 (−30%)	1078.4 ± 6.1	1081.0 ± 6.6	+0.24	0.21	78
2	[0.2, 0.8]	0.40 (−60%)	4124.0 ± 9.8	4142.1 ± 12.2	+0.44	0.09	70
3	[0.1, 0.3]	0.80 (−20%)	5148.9 ± 12.6	5160.1 ± 14.1	+0.22	0.27	76
4	[0.1, 0.9]	0.20 (−80%)	3052.4 ± 7.2	3061.7 ± 10.9	+0.31	0.11	64

summarizes the outcomes. Across the four cases, early-stopping saved on average $47.5\%$ wall-time and kept costs within $0.5\%$ of full-budget runs. Wilcoxon tests indicate no significant differences ($p>0.05$) in 3/4 cases, with borderline non-significance in Case 2 ($p=0.09$). In $72\%$ of runs on average, early-stopped costs fall within the full-budget 95% CI.

Using the admissible $\delta$ intervals to shorten wall-time, early-stopping achieves realized average savings of 47.5% with negligible cost differences (<0.5%), as summarized in Table 14. This saving exploits the admissible $\delta$ intervals identified retrospectively; practical deployment would require adaptive stopping criteria.

We additionally solved the full MILP in Appendix A using OR-Tools (CP-SAT-based MILP modeling) with a common wall-time limit of 600 s for all case studies. Time variables were discretized at the minute, and costs were scaled to integers when required. The solver reports the best feasible solution, the best bound, and the MIP gap within the time limit. We encode the full MILP (Appendix A) in OR-Tools CP-SAT with minute-level time discretization. Big-M constraints enforce precedence and activation links; costs are scaled to integers. We cap the wall-time to 600 s and report the best incumbent, best bound, and solver-reported MIP gap. Random seeds (10) are varied to assess robustness.

The hybrid GA+DRL approach integrates OR-Tools VRP heuristic baseline results (Table 11) and CP-SAT exact solver bounds (Table 13). Against OR-Tools VRP (600 s, equal budget), the hybrid achieves cost reductions of 2.86%, 1.26%, 2.07%, and 0.93% for Cases 1–4, with statistical significance confirmed via Wilcoxon tests (all $p<0.05$). CP-SAT MILP solver, while providing optimality bounds, exhibits MIP gaps of 2.9–11.8% on larger instances, demonstrating computational intractability of exact methods at scale and validating our hybrid heuristic design.

Overall, these results suggest that GA provides rapid early convergence, while DRL is effective at fine-tuning complex schedules. The best trade-off arises from using GA sparingly as an initializer and letting DRL dominate the refinement stage. Allocating too much time to GA risks anchoring the solution in local regions that DRL cannot substantially improve within the remaining budget. While DRL alone outperforms GA alone, the hybrid consistently delivers the most favorable balance between cost reduction and risk control.

4.7. Robustness and Stress Tests

We stress-tested methods under stochastic and nonstationary conditions: (i) travel-time noise (5%, 10%, 20%, zero-mean), (ii) service-time noise (5%, 10%, 20%), (iii) failure-window shifts and width perturbations (uniform/Gaussian), and (iv) online “IoT shocks” (sudden risk escalations mid-horizon). Each setting used $n=20$ seeds.

Table 15. Quantitative robustness: mean ± std.

Scenario	Metric	GA	DRL	Hybrid
Travel noise 10%	Degradation [%]	$+8.7\pm 2.1$	$+5.4\pm 1.6$	$+\mathbf{3}.\mathbf{8}\pm \mathbf{1}.\mathbf{2}$
	Best-ranking [%]	12	31	57
Service noise 10%	Degradation [%]	$+7.9\pm 2.4$	$+5.1\pm 1.7$	$+\mathbf{3}.\mathbf{5}\pm \mathbf{1}.\mathbf{3}$
	Best-ranking [%]	15	28	57
FW shift (mean +20 m)	Degradation [%]	$+9.3\pm 2.8$	$+6.0\pm 1.9$	$+\mathbf{4}.\mathbf{1}\pm \mathbf{1}.\mathbf{4}$
	Best-ranking [%]	10	30	60
IoT shock	Recovery [s]	$3.4\pm 0.9$	$2.6\pm 0.7$	$\mathbf{1}.\mathbf{8}\pm \mathbf{0}.\mathbf{5}$
(+50% ${\rho }_e$ @ mid-horizon)	Best-ranking [%]	9	28	63

reports mean degradation in total cost (relative to nominal), mean policy recovery time (seconds to return within 1% of pre-shock performance in receding-horizon control), and the fraction of replicates where each method ranked best (lower cost).

Online roll-out examples (real-time re-optimization). We inject a risk spike at $t=12:20$ (case 2) and replan with a moving 2 h horizon:

• Update latency (state ingest + policy decode): 180–260 ms (p95).
• Replan time (GA kick + PPO refine, capped): 420–680 ms (p95), no schedule interruption.
• Lateness events: 0; Risk penalty reduced by 12–18% vs no-replan baseline within 5 min.

5. Discussion

This research addresses the fundamental question: How can a GA-initialized DRL framework, informed by real-time IoT signals, improve multi-facility healthcare maintenance scheduling in terms of solution quality, compute efficiency, and scalability versus strong single-paradigm baselines?

Our experimental results across four case studies (10–30 facilities) demonstrate that the hybrid GA + DRL framework successfully addresses all components of this research question. As summarized in Table 11 and Table 12, the hybrid approach consistently outperforms all evaluated baselines: pure GA (average 5.57% cost reduction), pure DRL (average 1.32% improvement), and OR-Tools VRP heuristic (average 1.78% improvement, statistically significant at $p<0.05$ across all cases). Against the industrial-strength OR-Tools solver operating under equal 600s budgets, improvements range from 0.93% to 2.86%, demonstrating practical value beyond academic benchmarks. Critically, optimal solutions are achievable with minimal GA allocation (10–20% of budget), enabling computational savings averaging 47.5% without sacrificing solution quality—a key advantage for real-time deployment scenarios.

The admissible GA shares consistently suggest potential compute-budget savings; early-stopping experiments quantify realized savings relative to full-budget runs while preserving solution quality.

The results validate our hypothesis that GA and DRL complement each other effectively. GA provides rapid exploration and feasible solution generation, while DRL offers adaptive refinement capabilities. This synergy is particularly evident in complex scenarios with tight time windows and high penalty costs, where pure methods struggle with local optima or sample inefficiency.

The framework effectively balances multiple competing objectives—technician activation costs, travel expenses, service timing, and risk mitigation—while respecting complex constraints. Across all case studies, hybrid solutions demonstrate superior risk management, consistently achieving zero or minimal lateness penalties while maintaining low risk exposure, crucial for healthcare environments where equipment downtime directly impacts patient safety.

The demonstrated improvements translate to substantial operational benefits: reduced costs across large healthcare networks, ensured critical equipment availability, improved staff utilization, and dynamic schedule adaptation through IoT integration. The framework demonstrates scalability up to the tested sizes (10–30 facilities) and suggests practical applicability.

6. Conclusions

This research successfully develops and validates a hybrid GA+DRL framework for IoT-enabled healthcare equipment maintenance scheduling, addressing identified gaps in domain-specific hybridization, end-to-end system integration, and computational efficiency optimization.

6.1. Answer to the Research Question and Research Goals

We asked whether a GA-initialized DRL framework, informed by real-time IoT signals, can improve solution quality, compute efficiency, and scalability versus single-paradigm baselines while supporting secure cross-facility coordination. Across four case studies (10–30 facilities), the hybrid approach reduced total cost versus GA and DRL baselines and realized average wall-time savings of 47.5% in early-stopping runs with negligible cost differences (<0.5%), thus answering the research question in the affirmative. We achieved the following research goals by: (G1) designing and validating a GA-initialized PPO framework with behavior cloning; (G2) integrating IoT-derived failure predictions and an optional, permissioned audit layer; (G3) benchmarking against single-paradigm baselines under equal time budgets; (G4) quantifying the impact of GA/DRL time allocation on solution quality and efficiency.

6.2. Key Contributions

1.

Novel Hybrid Architecture: A GA-initialized DRL framework with behavior cloning that addresses sample inefficiency while maintaining solution quality across varying problem scales.

2.

Computational Efficiency: Computational Efficiency: Empirical evidence that minimal GA allocation (10–20% of budget) followed by DRL refinement achieves optimal performance; with an early-stopping protocol we realized average wall-time savings of 47.5% with negligible cost differences (<0.5%).

3.

Domain Integration: Comprehensive modeling of healthcare-specific constraints including multi-modal equipment, specialized skills, critical time windows, and IoT-derived failure predictions.

4.

Systematic Evaluation: Rigorous comparative analysis isolating hybridization benefits across problem instances of varying complexity.

6.3. Practical Impact

The framework enables proactive maintenance through IoT integration, demonstrated cost optimization while maintaining zero lateness in three case studies and small lateness in one, scalable implementation across different healthcare network sizes, and transparent operations through optional blockchain integration.

6.4. Limitations and Future Work

Current limitations include simulation-based validation, pending empirical evaluation of the blockchain layer (currently only described at the design level), and deterministic assumptions for travel and service times. Future research directions include the following:

• Real-world deployment with human-in-the-loop validation.
• Exploration of alternative hybrid architectures (e.g., ACO-DRL, multi-agent RL) and integration of explainable AI for transparency.
• Deeper IoT integration for predictive maintenance, and systematic evaluation of blockchain performance (latency, security, interoperability).
• Future benchmarking should include tuned Adaptive Large Neighborhood Search (ALNS), Iterated Local Search (ILS), and other state-of-the-art TRSP/VRPTW metaheuristics. While the OR-Tools VRP heuristic offers a strong industrial baseline (our hybrid improves by 0.93–2.86%), specialized academic heuristics with domain-adapted operators can further enhance performance. Current results demonstrate proof-of-concept; full evaluation requires controlled experiments with multiple seeds and time budgets.

This research establishes a foundation for next-generation healthcare maintenance systems that leverage artificial intelligence to optimize operations while ensuring patient safety. The hybrid GA + DRL framework represents a significant advancement toward intelligent, adaptive healthcare infrastructure management, positioning this approach as a viable solution for modern healthcare networks facing increasing complexity and resource constraints. The framework is a candidate for pilot deployment pending human-in-the-loop validation.