Search Results (476)

Space-Air-Ground Integrated Network (SAGIN), which is considered a network architecture with great development potential, exhibits significant cross-domain collaboration characteristics at present. However, most of the existing works ignore the matching and adaptability of differential tasks and heterogeneous resources, resulting in significantly inefficient task execution and undesirable network performance. As a consequence, we formulate a category attribute-oriented resource allocation and task offloading optimization problem with the aim of minimizing the overall scheduling cost. We first introduce a task–resource matching matrix to facilitate optimal task offloading policies with computation resources. In addition, virtual queues are constructed to take the impacts of randomized task arrival into account. To solve the optimization objective which jointly considers bandwidth allocation, transmission power control and task offloading decision effectively, we proposed a deep reinforcement learning (DRL) algorithm framework considering type matching. Simulation experiments demonstrate the effectiveness of our proposed algorithm as well as superior performance compared to others. Full article

This paper studies deep reinforcement learning (DRL)-based joint resource allocation and three-dimensional (3D) trajectory optimization for unmanned aerial vehicle (UAV)–ground access point (GAP) cooperative non-orthogonal multiple access (NOMA) communication in Industrial Internet of Things (IIoT) systems. Cooperative and non-cooperative users adopt different signal transmission strategies to meet diverse, task-oriented, quality-of-service requirements. Specifically, the DRL framework based on the Soft Actor–Critic algorithm is proposed to jointly optimize user scheduling, power allocation, and UAV trajectory in continuous action spaces. Closed-form power allocation and maximum weight bipartite matching are integrated to enable efficient user pairing and resource management. Simulation results show that the proposed scheme significantly enhances system performance in terms of throughput, spectral efficiency, and interference management, while enabling robustness against channel uncertainties in dynamic IIoT environments. The findings indicate that combining model-free reinforcement learning with conventional optimization provides a viable solution for adaptive resource management in dynamic UAV-GAP cooperative communication scenarios. Full article

To address the challenges of high complexity in modeling the correlation of multi-dimensional stochastic variables and the difficulty of solving long-term scheduling models in continuous action spaces in multi-energy complementary systems, this paper proposes a long-term optimization scheduling method based on Deep Deterministic Policy Gradient (DDPG). First, an improved C-Vine Copula model is used to construct the multi-dimensional joint probability distribution of water, wind, and solar energy, and Latin Hypercube Sampling (LHS) is employed to generate a large number of water–wind–solar coupling scenarios, effectively reducing the model’s complexity. Then, a long-term optimization scheduling model is established with the goal of maximizing the absorption of clean energy, and it is converted into a Markov Decision Process (MDP). Next, the DDPG algorithm is employed with a noise dynamic adjustment mechanism to optimize the policy in continuous action spaces, yielding the optimal long-term scheduling strategy for the water–wind–solar multi-energy complementary system. Finally, using a water–wind–solar integrated energy base as a case study, comparative analysis demonstrates that the proposed method can improve the renewable energy absorption capacity and the system’s power generation efficiency by accurately quantifying the uncertainties of water, wind, and solar energy and precisely controlling the continuous action space during the scheduling process. Full article

The integration of Building Information Modeling (BIM) and Digital Twin (DT) technologies offers new opportunities for enhancing reinforcement design and on-site constructability. This study addresses a current gap in DT applications by introducing an intelligent framework that simultaneously automates rebar layout generation and reduces rebar cutting waste (RCW), two challenges often overlooked during the construction execution phase. The system employs heuristic algorithms to generate constructability-aware rebar configurations and leverages Industry Foundation Classes (IFC) schema-based data models for interoperability. The framework is implemented using Autodesk Revit and Dynamo for rebar modeling and layout generation, Microsoft Project for schedule integration, and Autodesk Navisworks for clash detection. Real-time scheduling synchronization is achieved through IFC schema-based BIM models linked to construction timelines, while embedded clash detection and constructability feedback loops allow for iterative refinement and improved installation feasibility. A case study on a high-rise commercial building demonstrates substantial material savings, improved constructability, and reduced layout time, validating the practical advantages of BIM–DT integration for RC construction. Full article

The dynamic scheduling optimization of sports facilities faces challenges posed by real-time demand fluctuations and complex interdependencies between facilities. To address the adaptability limitations of traditional centralized approaches, this study proposes a decentralized multi-agent reinforcement learning framework based on graph neural networks (GNNs). Experimental results demonstrate that in a simulated environment comprising 12 heterogeneous sports facilities, the proposed method achieves an operational efficiency of 0.89 ± 0.02, representing a 13% improvement over Centralized PPO, while user satisfaction reaches 0.85 ± 0.03, a 9% enhancement. When confronted with a sudden 30% surge in demand, the system recovers in just 90 steps, 33% faster than centralized methods. The GNN attention mechanism successfully captures critical dependencies between facilities, such as the connection weight of 0.32 ± 0.04 between swimming pools and locker rooms. Computational efficiency tests show that the system maintains real-time decision-making capability within 800 ms even when scaled to 50 facilities. These results verify that the method effectively balances decentralized decision-making with global coordination while maintaining low communication overhead (0.09 ± 0.01), offering a scalable and practical solution for resource management in complex built environments. Full article

Container terminals are facing significant challenges in meeting the increasing demands for volume and throughput, with limited space often presenting as a critical constraint. Key areas of concern at the quayside include the berth allocation problem, the quay crane assignment, and the scheduling problem. Effectively managing these issues is essential for optimizing port operations; failure to do so can lead to substantial operational and economic ramifications, ultimately affecting competitiveness within the global shipping industry. Optimization models, encompassing both mathematical frameworks and metaheuristic approaches, offer promising solutions. Additionally, the application of machine learning and reinforcement learning enables real-time solutions, while robust optimization and stochastic models present effective strategies, particularly in scenarios involving uncertainties. This study expands upon earlier foundational analyses of berth allocation, quay crane assignment, and scheduling issues, which have laid the groundwork for port optimization. Recent developments in uncertainty management, automation, real-time decision-making approaches, and environmentally sustainable objectives have prompted this review of the literature from 2015 to 2024, exploring emerging challenges and opportunities in container terminal operations. Recent research has increasingly shifted toward integrated approaches and the utilization of continuous berthing for better wharf utilization. Additionally, emerging trends, such as sustainability and green infrastructure in port operations, and policy trade-offs are gaining traction. In this review, we critically analyze and discuss various aspects, including spatial and temporal attributes, crane handling, sustainability, model formulation, policy trade-offs, solution approaches, and model performance evaluation, drawing on a review of 94 papers published between 2015 and 2024. Full article

Complex scenario analysis and evaluation simulations often involve multiple sets of simulation applications with different combinations of parameters, thus resulting in high computing power consumption, which is one of the factors that limits the efficiency of multi-sample parallel simulations. Cloud computing provides considerable amounts of cheap and convenient computing resources, thus providing efficient support for multi-sample simulation tasks. However, traditional simulation scheduling methods do not consider the collaborative parallel scheduling of multiple samples and multiple entities under multi-objective constraints. Deep reinforcement learning methods can continuously learn and adjust their strategies through interactions with the environment, demonstrating strong adaptability in response to dynamically changing task requirements. Therefore, herein, a parallel simulation multi-sample task scheduling method based on deep reinforcement learning in a cloud computing environment is proposed. The method collects cluster load information and simulation application information as state inputs in the cloud environment, designs a multi-objective reward function to balance the cost and execution efficiency, and uses deep Q-networks (DQNs) to train agents for intelligent scheduling of multi-sample parallel simulation tasks. In a real cloud environment, the proposed method demonstrates runtime reductions of 4–11% and execution cost savings of 11–22% compared to the Round-Robin algorithm, Best Fit algorithm, and genetic algorithm. Full article

Modern database systems require autonomous CPU scheduling frameworks that dynamically optimize resource allocation across heterogeneous workloads while maintaining strict performance guarantees. We present a novel hierarchical deep reinforcement learning framework augmented with graph neural networks to address CPU scheduling challenges in mixed database environments comprising Online Transaction Processing (OLTP), Online Analytical Processing (OLAP), vector processing, and background maintenance workloads. Our approach introduces three key innovations: first, a symmetric two-tier control architecture where a meta-controller allocates CPU budgets across workload categories using policy gradient methods while specialized sub-controllers optimize process-level resource allocation through continuous action spaces; second, graph neural network-based dependency modeling that captures complex inter-process relationships and communication patterns while preserving inherent symmetries in database architectures; and third, meta-learning integration with curiosity-driven exploration enabling rapid adaptation to previously unseen workload patterns without extensive retraining. The framework incorporates a multi-objective reward function balancing Service Level Objective (SLO) adherence, resource efficiency, symmetric fairness metrics, and system stability. Experimental evaluation through high-fidelity digital twin simulation and production deployment demonstrates substantial performance improvements: 43.5% reduction in p99 latency violations for OLTP workloads and 27.6% improvement in overall CPU utilization, with successful scaling to 10,000 concurrent processes maintaining sub-3% scheduling overhead. This work represents a significant advancement toward truly autonomous database resource management, establishing a foundation for next-generation self-optimizing database systems with implications extending to broader orchestration challenges in cloud-native architectures. Full article

Cantonese Automatic Speech Recognition (ASR) is hindered by tonal complexity, acoustic diversity, and a lack of labelled data. This study proposes a phoneme-aware hierarchical augmentation framework that enhances performance without additional annotation. A Phoneme Substitution Matrix (PSM), built from Montreal Forced Aligner alignments and Tacotron-2 synthesis, injects adversarial phoneme variants into both transcripts and their aligned audio segments, enlarging pronunciation diversity. Concurrently, a semantic-aware SpecAugment scheme exploits wav2vec 2.0 attention heat maps and keyword boundaries to adaptively mask informative time–frequency regions; a reinforcement-learning controller tunes the masking schedule online, forcing the model to rely on a wider context. On the Common Voice Cantonese 50 h subset, the combined strategy reduces the character error rate (CER) from 26.17% to 16.88% with wav2vec 2.0 and from 38.83% to 23.55% with Zipformer. At 100 h, the CER further drops to 4.27% and 2.32%, yielding relative gains of 32–44%. Ablation studies confirm that phoneme-level and masking components provide complementary benefits. The framework offers a practical, model-independent path toward accurate ASR for Cantonese and other low-resource tonal languages. This paper presents an intelligent sensing-oriented modeling framework for speech signals, which is suitable for deployment on edge or embedded systems to process input from audio sensors (e.g., microphones) and shows promising potential for voice-interactive terminal applications. Full article

The rapid growth of electric vehicle (EV) adoption necessitates advanced energy management strategies to ensure sustainable, reliable, and efficient operation of charging infrastructure. This study proposes a hybrid AI-based framework for optimizing residential EV charging systems through the integration of Reinforcement Learning (RL), Linear Programming (LP), and real-time grid-aware scheduling. The system architecture includes smart wall-mounted chargers, a 120 kWp rooftop solar photovoltaic (PV) array, and a 60 kWh lithium-ion battery energy storage system (BESS), simulated under realistic load conditions for 800 residential units and 50 charging points rated at 7.4 kW each. Simulation results, validated through SCADA-based performance monitoring using MATLAB/Simulink and OpenDSS, reveal substantial technical improvements: a 31.5% reduction in peak transformer load, voltage deviation minimized from ±5.8% to ±2.3%, and solar utilization increased from 48% to 66%. The AI framework dynamically predicts user demand using a non-homogeneous Poisson process and optimizes charging schedules based on a cost-voltage-user satisfaction reward function. The study underscores the critical role of intelligent optimization in improving grid reliability, minimizing operational costs, and enhancing renewable energy self-consumption. The proposed system demonstrates scalability, resilience, and cost-effectiveness, offering a practical solution for next-generation urban EV charging networks. Full article

Wildfires often spread rapidly and cause significant casualties and economic losses. Firefighting drones carrying water capsules provide an efficient way for wildfire extinguishing, but their operational capabilities are limited by their payloads. This weakness can be compensated by using ground vehicles to provide mobile water supply. To this end, this paper presents an optimization problem of scheduling multiple drones and water supply trucks for wildfire fighting, which allocates burning subareas to drones, routes drones to perform fire-extinguishing operations in burning subareas and reload water between every two consecutive operations, and routes trucks to provide timely water supply for drones. To solve the problem within the limited emergency response time, we propose a deep reinforcement learning method, which consists of an encoder for embedding the input instance features and a decoder for generating a solution by iteratively predicting the subarea selection decision through attention. Computational results on test instances constructed upon real-world wilderness areas demonstrate the performance advantages of the proposed method over a collection of heuristic and metaheuristic optimization methods. Full article

Differential reinforcement of compliance (DRC) and functional communication training (FCT) are two effective treatments for escape-maintained behavior. They each, however, have unique limitations. This study aimed to replicate and extend past work by isolating the effects of each treatment and assessing for treatment preference. FCT produced larger reductions in challenging behavior and lower levels of compliance relative to DRC, which produced elevated levels of both compliance and challenging behavior. Additionally, all participants preferred FCT to DRC. Overall, challenging behavior was low and compliance was high when both treatments were embedded within a chained schedule, and these reductions were maintained throughout fading. Full article

The COVID-19 pandemic prompted a major shift in project management practices, offering a unique opportunity to assess organizational resilience and sustainability. This study explores how project professionals in Quebec adapted to the early months of the pandemic, focusing on emergent practices in communication, decision making, stakeholder engagement, resource management, and scheduling. Using the most significant change (MSC) method, we collected and analyzed 114 stories from practitioners operating at both strategic and operational levels across multiple sectors. The findings reveal how project contributors reconfigured their practices to sustain value delivery amid disruption—adopting digital tools, modifying governance structures, and redefining engagement strategies. Operational contributors showed greater adaptability, while strategic actors experienced challenges with control and oversight. These stories illustrate not only reactive adaptations but also the foundations of more resilient and sustainable governance frameworks. By surfacing lived experiences and perceptions, this research contributes methodologically through its use of MSC and conceptually by linking crisis response with long-term sustainability in project contexts. Our study invites reflection on how temporary adaptations may evolve into embedded practices, reinforcing the interconnection between adaptability, resilience, and sustainability in the governance of project-based organizations. Full article

Maritime vision sensing can be applied to maritime unmanned systems to perform search and rescue (SAR) missions under complex marine environments, as multiple unmanned aerial vehicles (UAVs) and unmanned surface vehicles (USVs) are able to conduct vision sensing through the air, the water-surface, and underwater. However, in these vision-based maritime SAR systems, collaboration between UAVs and USVs is a critical issue for successful SAR operations. To address this challenge, in this paper, we propose a long-endurance collaborative SAR scheme which exploits the complementary strengths of the maritime unmanned systems. In this scheme, a swarm of UAVs leverages a multi-agent reinforcement-learning (MARL) method and probability maps to perform cooperative first-phase search exploiting UAV’s high altitude and wide field of view of vision sensing. Then, multiple USVs conduct precise real-time second-phase operations by refining the probabilistic map. To deal with the energy constraints of UAVs and perform long-endurance collaborative SAR missions, a multi-USV charging scheduling method is proposed based on MARL to prolong the UAVs’ flight time. Through extensive simulations, the experimental results verified the effectiveness of the proposed scheme and long-endurance search capabilities. Full article

Leak management in water distribution networks (WDNs) is essential for minimising water loss, improving operational efficiency, and supporting sustainable water management. However, effectively identifying, preventing, and locating leaks remains a major challenge owing to the ageing infrastructure, pressure variations, and limited monitoring capabilities. Leakage management generally involves three approaches: leakage assessment, detection, and prevention. Traditional methods offer useful tools but often face limitations in scalability, cost, false alarm rates, and real-time application. Recently, artificial intelligence (AI) and machine learning (ML) have shown growing potential to address these challenges. Deep Reinforcement Learning (DRL) has emerged as a promising technique that combines the ability of Deep Learning (DL) to process complex data with reinforcement learning (RL) decision-making capabilities. DRL has been applied in WDNs for tasks such as pump scheduling, pressure control, and valve optimisation. However, their roles in leakage management are still evolving. To the best of our knowledge, no review to date has specifically focused on DRL for leakage management in WDNs. Therefore, this review aims to fill this gap and examines current leakage management methods, highlights the current role of DRL and potential contributions in the water sector, specifically water distribution networks, identifies existing research gaps, and outlines future directions for developing DRL-based models that specifically target leak detection and prevention. Full article