Search Results (1,349)

Distant-water squid-jigging operations require continuous route decisions that balance expected fishing returns and fuel consumption under spatially heterogeneous fishery resources and oceanographic conditions. This study proposes a Proximal Policy Optimization (PPO)-based dynamic routing framework for the Peruvian jumbo flying squid fishery. A data-informed gridded simulator integrates oceanographic variables, historical fishing-ground information, a wave-induced fuel adjustment, and an MVT-Inspired Local Depletion Mechanism. The routing task is formulated as an augmented-state Markov decision process with a multi-component reward function considering economic return, historical fishing-ground guidance, wave-related operating effects, and operational inefficiency. Vessel-level cross-fitting is used to reduce data reuse between reward-environment construction and historical-prior construction, and the policy is evaluated retrospectively in a held-out 2021 simulation environment. Across three independent training seeds, PPO achieved 257.42 ± 13.92 t of Cumulative Simulated Catch, 256.59 ± 7.83 t of Cumulative Fuel Consumption, and a Simplified Simulated Operating Margin of 129,376.80 ± 24,244.68 USD. Relative to the prespecified representative Historical Trajectory Replay, these results represent 13.77% higher simulated catch, 8.58% lower simulated fuel consumption, and 85.88% higher simulated operating margin. Additional sensitivity analyses showed that the qualitative catch–fuel–margin advantage was retained under moderate perturbations of the held-out catch field and across ±20% squid- and fuel-price variations. The results indicate that PPO provides a more favorable and comparatively stable catch–fuel–margin trade-off within the constructed retrospective simulation framework. Full article

Early-exit deep neural networks (DNNs) can reduce edge-inference latency, but abrupt variations in wireless and computing resources can destabilize split-inference policies. This paper proposes a meta-learning-driven adaptive control framework for joint backbone splitting and early-exit routing in MobileViT. The framework formulates multi-exit splitting as a constrained Markov decision process (CMDP) and introduces splitting-aware multi-dimensional adaptive proximal policy optimization (SMAPPO). SMAPPO combines nonlinear quality-of-service (QoS) penalties with topology-aware action masking, while cross-environment meta-initialization supports edge-local adaptation after resource disturbances. Under the stated simulation assumptions, SMAPPO reached the highest performance-index plateau among six methods in a representative 500-episode stationary trace and achieved the lowest normalized total cost across three latency–energy preference settings. Across ten seeds and nine stationary or disturbed scenarios, online SMAPPO achieved a 77.20% measured accuracy and 22.40 mJ of system energy. With an adaptation horizon of $K=14$, SMAPPO yielded a post-disturbance mean latency of 37.68 ms, a QoS-violation rate of 2.24%, and an on-time completion rate of 98.69%. These results indicate that combining meta-initialization, nonlinear constraint shaping, and topology-aware action masking improves stationary optimization and disturbance recovery within the controlled simulator. Full article

Sequential object-oriented software remains difficult to parallelize because several important decisions are not entirely local. A useful decomposition should consider architectural structure, execution frequency, communication latency, and how much communication can overlap with local computation. In contrast, many traditional modularization methods focus on cohesion, coupling, or communication cost without explicitly considering whether an unavoidable remote call can be issued early enough for part of its latency to be hidden by independent work. DTMC-GA addresses this problem by combining a Discrete-Time Markov Chain (DTMC) execution model, a Distributive Execution Time (DET) estimator, and a Genetic Algorithm (GA) search procedure. Guarded call-use displacement enlarges modeled overlap windows only when the relevant dependence, ordering, and safety conditions are satisfied. The public artifact contains five local benchmark fixtures, five comparison methods, 30 retained runs per algorithm–fixture pair, raw outputs, statistical scripts, regenerated figures, Docker support, continuous-integration verification, and a Zenodo DOI. Within this released five-method panel, DTMC-GA achieved the highest aggregate mean modeled speedup of 2.708 ± 0.329. A more recent NSGA-III comparator was also evaluated using matched fixtures, DTMC–DET inputs, constraints, random seeds, population size, and generation budget. NSGA-III achieved an aggregate mean modeled speedup of 2.737 ± 0.323, 1.07% above DTMC-GA, although none of the five fixture-level paired differences remained significant after Bonferroni correction. A separate sparse-graph scalability study covered 50, 100, 250, 500, 750, and 1000 classes; mean analysis time increased from about 0.024 to 0.286 s, while peak memory rose from about 119.1 to 120.9 MiB. The study also includes an executed OpenCV 4.7.0 calibration using a fixed five-convolution DNN workload. Across 30 matched pairs, native-parallel OpenCV achieved a measured ratio-of-means speedup of 1.8691×. The bounded empirical DTMC-GA/DET rerun over 30 seeds produced a mean modeled speedup of 1.0000× with an SD of 0.0000, giving an absolute modeled-versus-measured difference of 46.50%. Because the measured parallel comparator is OpenCV’s native threaded implementation rather than a DTMC-GA-generated source rewrite, this experiment is interpreted as real-runtime model calibration rather than direct measured DTMC-GA optimized speedup. Full article

Deep reinforcement learning (DRL) has become an important approach for manufacturing scheduling because it supports sequential decision-making under complex and changing production conditions. However, existing reviews primarily organize the literature by scheduling problem or learning method, providing less explicit support for tracing how manufacturing context, Markov Decision Process (MDP) formulation, scheduler architecture, and evaluation choices interact across heterogeneous studies. This study presents an evidence-based systematic literature review of DRL for manufacturing scheduling using a structured methodology for corpus construction, configuration-level coding, evidence traceability, study-quality assessment, and cross-study synthesis. The validated corpus comprises 52 primary studies and 54 independently coded DRL configurations. The evidence is synthesized across manufacturing scheduling characteristics, MDP design, DRL scheduler design, hybrid optimization, and empirical evaluation. The results show that scheduler design is context-dependent and architecturally diverse: manufacturing requirements are associated with differences in state, action, and reward formulation, while DRL schedulers combine different learning algorithms, representation architectures, control structures, and complementary optimization mechanisms. The evidence does not establish universal superiority for individual representations, algorithms, or hybrid architectures because reported outcomes remain strongly conditioned by problem formulation and experimental design. Evaluation evidence further highlights limited generalization, uneven statistical and component-level validation, and a continuing gap between benchmark or simulation studies and live industrial deployment. By linking study-, configuration-, and evidence-level information, this review provides a traceable basis for interpreting methodological relationships, identifying research gaps, and guiding the development and evaluation of DRL-based manufacturing scheduling systems. Full article

Addressing the dual threats of malicious jamming and time-varying fading faced by wireless communication links in complex dynamic electromagnetic adversarial environments, existing intelligent anti-jamming methods predominantly focus on single-dimensional resource optimization under quasi-static channels. This focus neglects the nonlinear superposition effects of multi-path deep fading and dynamic strong jamming in the time-frequency domain, making it challenging for systems to balance transmission reliability and system energy efficiency in physical environments where fading and suppression coexist. To address this issue, this study proposes a joint intelligent anti-jamming method for channel switching and transmit power control based on a Deep Q-Network (DQN). Initially, a composite communication environment model incorporating Markov time-varying fading and jamming is constructed. Subsequently, the joint resource scheduling problem is formulated as a Markov Decision Process. The environment state space is reconstructed by integrating continuous channel state estimation and jamming observation features, accompanied by the design of a highly aggregated two-dimensional discrete action space for both channel and power. Finally, a composite reward function evaluating both communication success rates and power consumption costs is proposed to guide the agent in multi-dimensional resource joint optimization. Simulation results demonstrate that the proposed algorithm effectively extracts implicit features under the composite state of fading and jamming. When encountering extreme deep fading or full-band blocking, the agent strategically triggers a silent mechanism to avoid exorbitant invalid energy consumption penalties, while precisely matching interference-free channels with the minimum effective transmit power during favorable communication windows. Simulation results show that compared with traditional xx algorithms, the proposed method significantly improves the dynamic successful transmission rate and system energy efficiency in complex, highly dynamic scenarios, achieving an effective optimization of anti-jamming reliability and low power overhead. Full article

Farmland in the hilly and mountainous regions of southern China is characterized by complex terrain and highly variable operating loads. Conventional diesel-powered tracked harvesters are constrained by high crop losses, excessive impurity rates, frequent blockages, and low overall energy-use efficiency. Distributed electric drive provides a promising solution; however, threshing cylinder blockage, high-frequency load transients, and slope operation make it difficult for conventional energy-management strategies to simultaneously ensure dynamic responses, fuel economy, and battery state of charge (SOC) stability. This study therefore proposes a deep deterministic policy gradient (DDPG)-based reinforcement learning energy-management strategy (RL-EMS) for a range-extended, distributed-drive hybrid tracked combine harvester. First, a full-vehicle dynamic model incorporating eight electric-drive units and strong electromechanical coupling is established. Second, power allocation is formulated as a Markov decision process (MDP), with a multi-objective reward function that accounts for fuel consumption, SOC tracking, and boundary violations; the load-rate-of-change is introduced as a feedforward state. Finally, a supervisory physical layer comprising feasible power projection, safety filtering, and rate limiting is inserted between the policy network output and the physical plant so that the executed command satisfies range extender power, battery SOC, current, and power-slew constraints. Under the standard 1000 s cycle, SOC-corrected energy-equivalent comparison shows that the RL-EMS reduces fuel consumption by 1.5% relative to the adaptive equivalent consumption minimization strategy (A-ECMS) and by 26.6% relative to the constant-torque energy-management strategy (CT-EMS). Under an unseen complex random cycle, the RL-EMS reduces fuel consumption by 5.1% relative to A-ECMS. It also suppresses DC-bus voltage sag during a threshing cylinder blockage transient, demonstrating favorable electromechanical transient response. The proposed method provides a modeling and control reference for the intelligent energy management of range-extended, distributed-drive agricultural machinery. Full article

Under the vehicle as a service (VaaS) paradigm, intelligent connected vehicles continuously generate large-scale, computation-intensive perception data processing tasks. However, limited onboard computing resources and power supply prevent vehicles from handling these tasks efficiently. Vehicular edge computing (VEC) extends available computing resources through vehicle–infrastructure collaboration. Nevertheless, continuous vehicle mobility causes intermittent communication links between vehicles and roadside units, posing new challenges for VEC task scheduling. Therefore, this paper proposes a reinforcement learning-based VEC task scheduling approach that integrates a fuzzy reward mechanism with multi-agent proximal policy optimization (FRMPPO). First, a system architecture is developed by integrating the directed acyclic graph task model, dynamic communication model, and computation model. The scheduling problem is formulated as a partially observable Markov decision process, with the objective of minimizing task completion latency and vehicle energy consumption. Second, a fuzzy reward mechanism is designed to guide the training of multi-agent proximal policy optimization. It takes edge node load pressure and communication state as inputs to adaptively combine local immediate rewards and the global reward, eventually guiding the agents toward a globally optimized cooperative policy. Finally, real-time scheduling decisions under communication intermittency are enabled through a centralized training and decentralized execution framework and gated recurrent unit-based state encoding. Simulation results demonstrate that FRMPPO effectively solves the VEC task scheduling problem, achieving significantly superior performance over existing algorithms in terms of both task completion latency and vehicle energy consumption. The proposed method thereby satisfies the real-time processing demands of perception tasks in VaaS scenarios. Full article

To address the challenge of satisfying strict Service-Level Agreement (SLA) requirements for concurrent smart video surveillance tasks in heterogeneous edge computing environments, an SLA-aware adaptive scheduling algorithm for heterogeneous computing collaboration is proposed. First, a mixed-task flow model is constructed, and a finite-state Markov chain is utilized to dynamically model the time-varying wireless channel. Second, a Dueling Double Deep Q-Network (Dueling DDQN) scheduling algorithm based on SLA awareness and channel adaptation is proposed, with a designed SLA action-masking mechanism. This mechanism advances hard delay constraints to the decision-generation stage, dynamically prunes the action space based on real-time channel conditions and node loads, and filters out actions predicted to violate the SLA before execution. Experimental results show that the proposed algorithm coordinates heterogeneous computing resources between the cloud center and the edge and exhibits earlier empirical reward stabilization and lower task-violation rates than the compared learning-based baselines under the tested workload conditions. Full article

Multi-UAV Cooperative Target Search (MCTS) is a critical task in low-altitude sensing applications, requiring agents to efficiently explore unknown environments under complex constraints. However, traditional search methods are mostly unscalable and perform poorly in dynamic multi-UAV environments. As a promising alternative, Reinforcement Learning (RL) has emerged to overcome these limitations by enabling agents to learn adaptive policies directly from environmental interactions. A key limitation is that current RL methods lack efficient exploration, which is a critical bottleneck preventing UAVs from finding more targets. To address this limitation, we propose a novel method named AEQMIX, which integrates trajectory entropy maximization into QMIX, an advanced Multi-Agent Reinforcement Learning (MARL) method, to encourage efficient exploration. We formulate the MCTS problem as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP) and design a multi-objective reward function. To mitigate the intractability of density estimation in high-dimensional spaces, we employ a nonparametric particle-based entropy estimator to quantify the spatial diversity of UAV trajectories. This entropy estimate is utilized as an intrinsic reward, incentivizing agents to maximize the distance between their trajectories and those of their neighbors. Extensive simulations demonstrate that AEQMIX significantly outperforms baseline reinforcement learning and traditional optimization methods in terms of search rate, coverage efficiency, and collision avoidance. Compared with DNQMIX, AEQMIX improves the search rate and coverage rate by 9.52% and 11.54%, respectively, while reducing the average collision count by 70.59% in the (40 × 40) environment. Full article

Cloud platforms must hold utilisation and latency targets while demand shifts and attack traffic arrive together. Reactive threshold scaling meets neither pressure, and an autoscaler blind to attacks funds the load an adversary requested. Recent work shows adversaries can drive this loop into economic denial of sustainability, so the controller sits inside the attack surface. No prior orchestrator couples workload forecasting, unsupervised anomaly detection and learned scaling in one loop, and none reports multi-seed significance testing. This article extends SARO, presented at IMCOM 2026, to close that gap. We formalise the problem as a Markov decision process with a corrected multi-objective reward, and replace the tabular agent with SARO-DQN, a continuous-state controller trained by three-step Double Q-learning. Across ten held-out days and five seeds, SARO-DQN reaches the highest composite reward (200.1 ± 16.5) and the highest utilisation (68.2%) against eight alternatives, and every reward difference is significant under Welch tests ($p<0.05$). Ablations attribute 23.6 reward points to the detector ($p=1.4\times {10}^{-6}$) and 8.9 to the forecast ($p=0.016$). On UNSW-NB15, the detector attains an AUC of 0.888. Two principles follow. Detectors must consume exogenous traffic-shape signals, and detector and policy must be trained as a coupled system. Full article

Maintenance planning for highway bridges without historical inspection records remains challenging because conventional deterioration models typically require long-term data for calibration. This study proposes an inspection-driven decision-support framework that integrates bridge-specific engineering calibration, Markov deterioration modelling, an independent condition-rating-based Remaining Service Life (RSL) assessment, and Markov Decision Process (MDP) optimization. The framework was demonstrated on a 26-year-old six-span composite highway bridge in Türkiye. A comprehensive inspection yielded a weighted Bridge Condition Index of 2.98, which was used to calibrate the bridge-specific Markov deterioration model. The model predicted attainment of the State-4 intervention threshold after approximately 15.71 years under a do-nothing scenario, while the independent condition-rating assessment estimated an RSL of approximately 17 years for the governing pier columns. The optimized finite-horizon MDP policy reduced the expected discounted life-cycle cost by 89.75% relative to the do-nothing strategy, while sensitivity analyses confirmed the stability of the principal maintenance policy under the examined modelling and economic perturbations. The proposed framework therefore provides a practical, transparent, and progressively updateable methodology for deterioration prediction and maintenance planning for bridges with limited historical inspection information. Full article

Cooperative communication is a promising technology for enhancing the reliability and capacity of free-space optical (FSO) systems. It achieves this by employing multiple FSO links involving relays to overcome environmental challenges. In this work, we consider a cooperative decode-and-forward (DF) system with energy-harvesting capabilities. The relays are equipped with rechargeable batteries with limited capacity. The relays harvest energy from the optical source and store it in their batteries for later use in forwarding information. The relay selection process in this cooperative system with energy-harvesting capabilities is modeled as a Markov decision process (MDP). To improve long-term system capacity under energy constraints, a deep reinforcement learning (DRL)-based relay selection algorithm is proposed. Specifically, a deep Q-network (DQN) is trained using a battery-aware reward function to guide relay-selection decisions while accounting for the relays’ current battery levels. We evaluate the algorithm’s performance through simulations, focusing on its convergence, performance under different numbers of relays, adaptability to varying energy availability, and robustness across diverse weather conditions. The simulation results indicate that the proposed DQN-based algorithm outperforms tabular Q-learning, Battery-Aware Greedy (BA-Greedy), and random selection as baseline methods. Full article

Scheduling at the printing bottleneck of plastic woven packaging production is complicated by asymmetric sequence-dependent setup times (SDST) caused by color transitions and by mandatory non-preemptive shutdown windows. This study focuses on assigning and sequencing orders on parallel printing machines under these coupled operational constraints. To address this problem, we propose an Attention-Enhanced Dual-Critic Deep Reinforcement Learning framework with Action Masking (AE-DMAC). The scheduling problem is formulated as a bi-objective parallel-machine Markov Decision Process that considers makespan and total setup time under a fixed preference setting. A cross-attention module models the compatibility between the current machine state and pending orders to capture asymmetric SDST effects. Two critics separately estimate the efficiency- and setup-related value signals before they are combined for policy optimization, reducing interference between the two objectives. In addition, a deterministic action-feasibility mask removes assignments that would overlap the known shutdown window before action sampling. The framework is evaluated on industrially calibrated synthetic instances with 50, 150, and 300 orders scheduled on eight parallel printing machines. Experimental results show that AE-DMAC consistently improves makespan and normalized setup time per machine compared with the implemented heuristic, meta-heuristic, and Vanilla PPO baselines under the tested operating conditions. In the large-scale instance, AE-DMAC achieves an average makespan of 223.8 h and a normalized setup time of 568.4 min per machine, corresponding to reductions of 7.4% and 27.4%, respectively, relative to Vanilla PPO. The feasibility mask maintains zero shutdown-window violations in the evaluated deterministic setting, while the attention mechanism substantially reduces high-cost sequence-dependent color transitions. These results indicate that the proposed framework is a promising scheduling approach for parallel printing systems with asymmetric changeovers and known machine-unavailability windows. Full article

Background: Imported halal meat supply chains face substantial challenges caused by fragmented certification systems, information gaps, and uncertainties. These factors challenge procurement management and hinder the assurance of halal compliance. Conventional Enterprise Resource Planning (ERP) systems are mainly designed for transaction processing and offer limited support for flexible decision-making. Methods: An Adaptive Bayesian-Feedback Framework was developed that integrates Digital Halal Authentication, Evidence-Based Decision Support, Bayesian inference, adaptive feedback learning, and the Markov Decision Process (MDP) within the ERP Purchasing Module. The framework was evaluated using 30 imported meat shipment cases, expanded into 300 procurement transaction records. Results: Implementation of the framework reduced halal authentication processing time from 2509 s to 290 s, representing an 88.44% decrease and an approximately 8.65-fold improvement in processing speed. The framework also enabled continuous revisions to evidence on vendors and procurement risks and improved the consistency of decision-making amid uncertain supply chain circumstances. Conclusions: The framework develops a unified decision-support system comprising adaptive halal authentication, evidence-based decision support, and intelligent ERP. This integration improves procurement governance, traceability, transparency, and uncertainty management in imported halal meat supply chains. Full article

Background/Objectives: To inform policy making for the German breast cancer (BC) screening program, we systematically evaluated the long-term benefits and harms of extended age limits compared to the current standard of biennial mammography at ages 50–69 years using a decision-analytic approach. Methods: We developed and applied a Markov state-transition model for mammography screening in Germany to systematically assess the benefit–harm trade-offs of various screening strategies varying in age at start and end of screening as well as in screening frequency. The model was populated with international data for sensitivity and specificity of mammography along with German epidemiological, clinical and age-specific quality-of-life data. In deterministic analyses, the following outcomes were projected: detected ductal carcinoma in situ (DCIS) and invasive BC, BC-related deaths, life years (LY), and quality-adjusted life years (QALY), number of positive, false-positive, and total mammograms, overdiagnosis, and the incremental harm–benefit ratio (IHBR). Results: In the base-case analysis, mammography at ages 45–79 (annual, age 45–49; biennial, 50–79) achieved the highest gain in LY (10.0 life years gained [LYG] per 100 women) compared with current screening. Biennial mammography at ages 45–74 resulted in the highest benefits considering both life expectancy and quality of life (3.5 QALYs gained/100 women). Compared to current biennial mammography screening at ages 50–69, lowering the start age from 50 to 45 years resulted in an IHBR of 47 additional mammograms/LYG. Compared to biennial mammography at ages 45–69, biennial mammography at age 45–74 results in 96 additional mammograms/LYG. Further extended screening results in substantially less favorable IHBRs. Conclusions: Based on our results, extending biennial mammography screening to women aged 45 to 74 years may prevent additional BC deaths and increase remaining life expectancy at an acceptable benefit–harm ratio, and improve quality-adjusted life expectancy. Full article