Search Results (319)

This study applies reinforcement learning to search parameter regimes that yield chaotic dynamics across six systems: the Logistic map, the Hénon map, the Lorenz system, Chua’s circuit, the Lorenz–Haken model, and a custom 5D hyperchaotic design. The largest Lyapunov exponent (LLE) is used as a scalar reward to guide exploration toward regions with high sensitivity to initial conditions. Under matched evaluation budgets, the approach reduces redundant simulations relative to grid scans and accelerates discovery of parameter sets with large positive LLE. Experiments report learning curves, parameter heatmaps, and representative phase portraits that are consistent with Lyapunov-based assessments. Q-learning typically reaches high-reward regions earlier, whereas SARSA shows smoother improvements over iterations. Several evaluated systems possess equation-level symmetry—most notably sign-reversal invariance in the Lorenz system and Chua’s circuit models and a coordinate-wise sign pattern in the Lorenz–Haken equations—which manifests as mirror attractors and paired high-reward regions; one representative is reported for each symmetric pair. Overall, Lyapunov-guided reinforcement learning serves as a practical complement to grid and random search for chaos identification in both discrete maps and continuous flows, and transfers with minimal changes to higher-dimensional settings. The framework provides an efficient method for identifying high-complexity parameters for applications in chaos-based cryptography and for assessing stability boundaries in engineering design. Full article

Cooperative navigation is a fundamental enabling technology for unlocking the full potential of Unmanned Underwater Vehicle (UUV) clusters in GNSS-denied environments. However, the severe constraints of the underwater acoustic channel, such as high latency, low bandwidth, and non-Gaussian noise, pose significant challenges to designing robust and efficient state estimation and information fusion algorithms. While numerous surveys have cataloged the available techniques, they have remained largely descriptive, lacking a rigorous, quantitative comparison of their performance trade-offs under realistic conditions. This paper provides a comprehensive and critical review that moves beyond qualitative descriptions to establish a novel quantitative comparison framework. Through a standardized benchmark scenario, we provide the first data-driven, comparative analysis of key frontier algorithms—from recursive filters like the Maximum Correntropy Kalman Filter (MCC-KF) to batch optimization methods like Factor Graph Optimization (FGO)—evaluating them across critical metrics including accuracy, computational complexity, communication load, and robustness. Our results empirically reveal the fundamental performance gaps and trade-offs, offering actionable insights for system design. Furthermore, this paper provides in-depth technical analyses of advanced topics, including distributed fusion architectures, intelligent strategies like Deep Reinforcement Learning (DRL), and the unique challenges of navigating in extreme environments such as the polar regions. Finally, leveraging the insights derived from our quantitative analysis, we propose a structured, data-driven research roadmap to systematically guide future investigations in this critical domain. Full article

This study proposes a discrete multi-agent Q-learning framework for the online tuning of PID controllers in continuous dynamic systems with limited observability. The approach treats the adjustment of each PID gain ($k_p$, $k_i$, $k_d$) as an independent learning process, in which each agent operates within a discrete state space corresponding to its own gain and selects actions from a tripartite space (decrease, maintain, or increase its gain). The agents act simultaneously under fixed decision intervals, favoring their convergence by preserving quasi-stationary conditions of the perceived environment, while a shared cumulative global reward, composed of system parameters, time and control action penalties, and stability incentives, guides coordinated exploration toward control objectives. Implemented in Python, the framework was validated in two nonlinear control problems: a water-tank and inverted pendulum (cart-pole) systems. The agents achieved their initial convergence after approximately 300 and 500 episodes, respectively, with overall success rates of $49.6\%$ and $46.2\%$ in 5000 training episodes. The learning process exhibited sustained convergence toward effective PID configurations capable of stabilizing both systems without explicit dynamic models. These findings confirm the feasibility of the proposed low-complexity discrete reinforcement learning approach for online adaptive PID tuning, achieving interpretable and reproducible control policies and providing a new basis for future hybrid schemes that unite classical control theory and reinforcement learning agents. Full article

Image captioning, as a representative cross-modal task, faces significant challenges, including high annotation costs and modality alignment difficulties. To address these issues, this paper proposes CMSA, an image captioning framework that does not require paired image-text data. The framework integrates a generator, a discriminator, and a reward module, employing a collaborative multi-module optimization strategy to enhance caption quality. The generator builds multi-level joint feature representations based on a contrastive language-image pretraining model, effectively mitigating the modality alignment problem and guiding the language model to generate text highly consistent with image semantics. The discriminator learns linguistic styles from external corpora and evaluates textual naturalness, providing critical reward signals to the generator. The reward module combines image-text relevance and textual quality metrics, optimizing the generator parameters through reinforcement learning to further improve semantic accuracy and language expressiveness. CMSA adopts a progressive multi-stage training strategy that, combined with joint feature modeling and reinforcement learning mechanisms, significantly reduces reliance on costly annotated data. Experimental results demonstrate that CMSA significantly outperforms existing methods across multiple evaluation metrics on the MSCOCO and Flickr30k datasets, exhibiting superior performance and strong cross-dataset generalization ability. Full article

Active suspension systems have gained significant attention for their capability to improve vehicle dynamics and energy efficiency. However, achieving consistent control performance under diverse and uncertain road conditions remains challenging. This paper proposes a prior-guided residual reinforcement learning framework for active suspension control. The approach integrates a Linear Quadratic Regulator (LQR) as a prior controller to ensure baseline stability, while an enhanced Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm learns the residual control policy to improve adaptability and robustness. Moreover, residual connections and Long Short-Term Memory (LSTM) layers are incorporated into the TD3 structure to enhance dynamic modeling and training stability. The simulation results demonstrate that the proposed method achieves better control performance than passive suspension, a standalone LQR, and conventional TD3 algorithms. Full article

We study the predictive maintenance scheduling for IoT-enabled medical equipment in multi-facility healthcare networks. The problem involves skill matching, time windows, and risk-aware priorities. We model a multi-skill Technician Routing and Scheduling Problem with IoT-predicted failure intervals and minimize a composite cost for technician activation and labor, travel/time, risk exposure within the failure window, and lateness beyond it. We propose a hybrid solver coupling a Genetic Algorithm (GA) for rapid exploration and feasible schedule generation with a Proximal Policy Optimization (PPO) agent warm-started via behavior cloning on GA elites and refined online in a receding-horizon manner. An optional, permissioned blockchain records tamper-evident maintenance events off the control loop for auditability. Across four case studies (10–30 facilities), the hybrid approach reduces total cost by 2.09–10.31% versus pure GA, by 0.57–2.65% versus pure Deep Reinforcement Learning (DRL), and by 0.93–2.86% versus OR-Tools VRP heuristic baseline. In controlled early-stopping runs guided by admissible GA/DRL time splits, we realized average wall-time savings up to 47.5% while keeping solution costs within 0.5% of full-budget runs and maintaining low or zero lateness and risk exposure. These results indicate that GA seeding improves sample efficiency and stability for DRL in complex, data-driven maintenance settings, yielding a practical, adaptive, and auditable scheduler for healthcare operations. Full article

This paper proposes a stepwise expert-teaching reinforcement learning framework for intelligent frequency control in hydro–thermal–wind–solar–compressed air energy storage (CAES) integrated systems under high renewable energy penetration. The proposed method addresses the frequency stability challenge in low-inertia, high-volatility power systems, particularly in Southwest China, where large-scale renewable-energy-based energy bases are rapidly emerging. A load frequency control (LFC) model is constructed to serve as the training and validation environment, reflecting the dynamic characteristics of the hybrid system. The stepwise expert-teaching PPO (SETP) framework introduces a stepwise training mechanism in which expert knowledge is embedded to guide the policy learning process and training parameters are dynamically adjusted based on observed performance. Comparative simulations under multiple disturbance scenarios are conducted on benchmark systems. Results show that the proposed method outperforms standard proximal policy optimization (PPO) and traditional PI control in both transient response and coordination performance. Full article

The rapid development of the Internet of Things (IoT) poses significant problems in securing heterogeneous, massive, and high-volume network traffic against cyber threats. Traditional intrusion detection systems (IDSs) are often found to be poorly scalable, or are ineffective computationally, because of the presence of redundant or irrelevant features, and they suffer from high false positive rates. Addressing these limitations, this study proposes a hybrid intelligent model that combines quantum computing, chaos theory, and deep learning to achieve efficient feature selection and effective intrusion classification. The proposed system offers four novel modules for feature optimization: chaotic swarm intelligence, quantum diffusion modeling, transformer-guided ranking, and multi-agent reinforcement learning, all of which work with a graph-based classifier enhanced with quantum attention mechanisms. This architecture allows as much as 75% feature reduction, while achieving 4% better classification accuracy and reducing computational overhead by 40% compared to the best-performing models. When evaluated on benchmark datasets (NSL-KDD, CICIDS2017, and UNSW-NB15), it shows superior performance in intrusion detection tasks, thereby marking it as a viable candidate for scalable and real-time IoT security analytics. Full article

Securing Industrial Control Systems (ICSs) is critical, but it is made challenging by the constant evolution of cyber threats and the scarcity of labeled attack data in these specialized environments. Standard intrusion detection systems (IDSs) often fail to adapt when transferred to new networks with limited data. To address this, this paper introduces an adaptive intrusion detection framework that combines a hybrid Convolutional Neural Network and Long Short-Term Memory (CNN-LSTM) model with a novel transfer learning strategy. We employ a Reinforcement Learning (RL) agent to intelligently guide the fine-tuning process, which allows the IDS to dynamically adjust its parameters such as layer freezing and learning rates in real-time based on performance feedback. We evaluated our system in a realistic data-scarce scenario using only 50 labeled training samples. Our RL-Guided model achieved a final F1-score of 0.9825, significantly outperforming a standard neural fine-tuning model (0.861) and a target baseline model (0.759). Analysis of the RL agent’s behavior confirmed that it learned a balanced and effective policy for adapting the model to the target domain. We conclude that the proposed RL-guided approach creates a highly accurate and adaptive IDS that overcomes the limitations of static transfer learning methods. This dynamic fine-tuning strategy is a powerful and promising direction for building resilient cybersecurity defenses for critical infrastructure. Full article

The field of underwater image enhancement (UIE) has advanced significantly, yet it continues to grapple with persistent challenges stemming from complex, spatially varying optical degradations such as light absorption, scattering, and color distortion. These factors often impede the efficient deployment of enhancement models. Conventional approaches frequently rely on uniform processing strategies that neither adapt effectively to diverse degradation patterns nor adequately incorporate physical principles, resulting in a trade-off between enhancement quality and computational efficiency. To overcome these limitations, we propose a Dual-Path Physics-Guided Mamba Network (DPPGM), a lightweight framework designed to synergize physical optics modeling with data-driven learning. Extensive experiments on three benchmark datasets (UIEB, LSUI, and U45) demonstrate that DPPGM outperforms 13 state-of-the-art methods, achieving an exceptional balance with only 1.48 M parameters and 25.39 G FLOPs. The key to this performance is a symmetry-constrained architecture: it incorporates a dual-path Mamba module for degradation-aware processing, physics-guided optimization based on the Jaffe–McGlamery model, and compact subspace fusion, ensuring that quality and efficiency are mutually reinforced rather than competing objectives. Full article

Hierarchical Reinforcement Learning (HRL) for UAV Coverage Path Planning (CPP) is hindered by the “subgoal space explosion”, causing inefficient exploration. To address this, we propose a two-stage framework, Hierarchical Reinforcement Learning Guided by Landmarks (HRGL), which synergistically combines HRL with a multi-scale observation space. The framework provides a low-resolution global map for the high-level policy’s strategic planning and a high-resolution local map for the low-level policy’s execution. To bridge the information gap between these hierarchical views, the first stage, ACHMP, introduces a learned Adjacency Network. This network acts as an efficient proxy for local feasibility by mapping coordinates to an embedding space where distances reflect true reachability, allowing the high-level policy to select feasible subgoals without processing complex local data. The second stage, HRGL, further introduces a landmark-guided global guidance mechanism to overcome local myopia. Extensive experiments on a variety of simulated grid-world maps demonstrate that HRGL significantly outperforms baseline methods in terms of both convergence speed and final coverage rate. Full article

In response to growing pressures for sustainability in tourism, this paper examines the techno-economic evaluation of green innovations in small and medium-sized tourism enterprises (SMEs). Focusing on a single case study of a hotel in Greece, the research investigates how and why specific sustainability interventions were implemented and assesses their operational and economic impacts. The study adopts an interpretivist approach, combining process tracing with thematic analysis. The analysis is guided by innovation diffusion theory, supported by organizational learning perspectives, to explain the stepwise adoption of sustainability practices and the internal adaptation processes that enabled them. The techno-economic evaluation draws on quantitative indicators and qualitative assessments of perceived benefits and implementation challenges, offering a broader view of value beyond purely financial metrics. Data were collected through semi-structured interviews, on-site observations, and internal documentation. The findings reveal a gradual, non-linear path to innovation, shaped by adoption dynamics and organizational learning, reinforced by leadership commitment, contextual adaptation, supply chain decisions, and external incentives. Key interventions, including solar energy adoption, composting, and the formation of zero-waste partnerships, resulted in measurable reductions in energy use and landfill waste, along with improvements in guest satisfaction, operational efficiency, and local collaboration. Although it is subject to limitations typical of single-case designs, the study demonstrates how even modest sustainability efforts, when integrated into daily operations, can generate multiple types of outcomes (economic, environmental, and operational). The paper offers practical implications for tourism SMEs and policymakers and formulates propositions for future testing on sustainable innovation in the tourism sector. Full article

The filarioid nematode Dirofilaria immitis is the causative agent of heartworm disease, a major parasitic infection of canids, felids and occasionally humans. Current prevention relies on macrocyclic lactone-based chemoprophylaxis, but the emergence of drug resistance highlights the need for new intervention strategies. Here, we applied a machine learning (ML)-based framework to predict and prioritise essential genes in D. immitis in silico, using genomic, transcriptomic and functional datasets from the model organisms Caenorhabditis elegans and Drosophila melanogaster. With a curated set of 26 predictive features, we trained and evaluated multiple ML models and, using a defined threshold, we predicted 406 ‘high-priority’ essential genes. These genes showed strong transcriptional activity across developmental stages and were inferred to be enriched in pathways related to ribosome biogenesis, translation, RNA processing and signalling, underscoring their potential as anthelmintic targets. Transcriptomic analyses suggested that these genes are associated with key reproductive and neural tissues, while chromosomal mapping revealed a relatively even genomic distribution, in contrast to patterns observed in C. elegans and Dr. melanogaster. In addition, initial evidence suggested structural variation in the X chromosome compared with a recently published D. immitis assembly, indicating the importance of integrating long-read sequencing with high-throughput chromosome conformation capture (Hi-C) mapping. Overall, this study reinforces the potential of ML-guided approaches for essential gene discovery in parasitic nematodes and provides a foundation for downstream validation and therapeutic target development. Full article

The flexible job shop scheduling problem (FJSP) with transportation resources such as automated guided vehicles (AGVs) is prevalent in manufacturing enterprises. Multi-type AGVs are widely adopted to transfer jobs and realize the collaboration of different machines, but are often ignored in current research. Therefore, this paper addresses the FJSP with multi-type AGVs (FJSP-MTA). Considering the difficulties caused by the introduction of transportation and the NP-hard nature, the artificial bee colony (ABC) algorithm is adopted as a fundamental solution approach. Accordingly, a Q-learning hybrid multi-objective ABC (Q-HMOABC) algorithm is proposed to deal with the FJSP-MTA. First, to minimize both the makespan and total energy consumption (TEC), this paper proposes a novel mixed-integer linear programming (MILP) model. In Q-HMOABC, a three-layer encoding strategy based on operation sequence, machine assignment, and AGV dispatching with type selection is used. Moreover, during the employed bee phase, Q-learning is employed to update all individuals; during the onlooker bee phase, variable neighborhood search (VNS) is used to update nondominated solutions; and during the scout bee phase, a restart strategy is adopted. Experimental results demonstrate the effectiveness and superiority of Q-HMOABC. Full article

This research explores the phenomenon of plate-end (PE) debonding in reinforced concrete (RC) beams strengthened with fiber-reinforced polymer (FRP) composites. This type of failure represents a key mechanism that undermines the structural performance and efficiency of FRP reinforcement systems. Despite the widespread use of FRP in structural repair due to its high strength and corrosion resistance, PE debonding—often triggered by shear or inclined cracks—remains a major challenge. Traditional computational models for predicting PE debonding suffer from low accuracy due to the nonlinear relationship between influencing parameters. To address this, the research employs machine learning techniques and SHapley Additive exPlanations (SHAP), to develop more accurate and explainable predictive models. A comprehensive database is constructed using key parameters affecting PE debonding. Machine learning algorithms are trained and evaluated, and their performance is compared with existing normative models. The study also includes parameter importance and sensitivity analyses to enhance model interpretability and guide future design practices in FRP-based structural reinforcement. Full article