Search Results (673)

The fast expansion of cloud computing has raised carbon emissions and energy usage in cloud data centers, so creative solutions for sustainable resource management are more necessary. This work presents a new algorithm—Carbon-Aware, Energy-Efficient, and SLA-Compliant Virtual Machine Placement using Deep Q-Networks (DQNs) and Agglomerative Clustering (CARBON-DQN)—that intelligibly balances environmental sustainability, service level agreement (SLA), and energy efficiency. The method combines a deep reinforcement learning model that learns optimum placement methods over time, carbon-aware data center profiling, and the hierarchical clustering of virtual machines (VMs) depending on resource constraints. Extensive simulations show that CARBON-DQN beats conventional and state-of-the-art algorithms like GRVMP, NSGA-II, RLVMP, GMPR, and MORLVMP very dramatically. Among many virtual machine configurations—including micro, small, high-CPU, and extra-large instances—it delivers the lowest carbon emissions, lowered SLA violations, and lowest energy usage. Driven by real-time input, the adaptive decision-making capacity of the algorithm allows it to dynamically react to changing data center circumstances and workloads. These findings highlight how well CARBON-DQN is a sustainable and intelligent virtual machine deployment system for cloud systems. To improve scalability, environmental effect, and practical applicability even further, future work will investigate the integration of renewable energy forecasts, dynamic pricing models, and deployment across multi-cloud and edge computing environments. Full article

Emergencies can occur unexpectedly and require immediate action, especially in aviation, where time pressure and uncertainty are high. This study focused on improving emergency evacuation in airport and aircraft scenarios using real-time decision-making support. A system based on the Asynchronous Advantage Actor–Critic (A3C) algorithm, an advanced deep reinforcement learning method, was developed to generate faster and more efficient evacuation routes compared to traditional models. The A3C model was tested in various scenarios, including different environmental conditions and numbers of agents, and its performance was compared with the Deep Q-Network (DQN) algorithm. The results showed that A3C achieved evacuations 43.86% faster on average and converged in fewer episodes (100 vs. 250 for DQN). In dynamic environments with moving threats, A3C also outperformed DQN in maintaining agent safety and adapting routes in real time. As the number of agents increased, A3C maintained high levels of efficiency and robustness. These findings demonstrate A3C’s strong potential to enhance evacuation planning through improved speed, adaptability, and scalability. The study concludes by highlighting the practical benefits of applying such models in real-world emergency response systems, including significantly faster evacuation times, real-time adaptability to evolving threats, and enhanced scalability for managing large crowds in high-density environments including airport terminals. The A3C-based model offers a cost-effective alternative to full-scale evacuation drills by enabling virtual scenario testing, supports proactive safety planning through predictive modeling, and contributes to the development of intelligent decision-support tools that improve coordination and reduce response time during emergencies. Full article

In recent years, deep learning-based end-to-end Text-To-Speech (TTS) models have made significant progress in enhancing speech naturalness and fluency. However, existing Variational Inference Text-to-Speech (VITS) models still face challenges such as insufficient pitch modeling, inadequate contextual dependency capture, and low inference efficiency in the decoder. To address these issues, this paper proposes an improved TTS framework named Q-VITS. Q-VITS incorporates Rotary Position Embedding (RoPE) into the text encoder to enhance long-sequence modeling, adopts a frame-level prior modeling strategy to optimize one-to-many mappings, and designs a style extractor based on a diffusion model for controllable style rendering. Additionally, the proposed decoder ConfoGAN integrates explicit F0 modeling, Pseudo-Quadrature Mirror Filter (PQMF) multi-band synthesis and Conformer structure. The experimental results demonstrate that Q-VITS outperforms the VITS in terms of speech quality, pitch accuracy, and inference efficiency in both subjective Mean Opinion Score (MOS) and objective Mel-Cepstral Distortion (MCD) and Root Mean Square Error (RMSE) evaluations on a single-speaker dataset, achieving performance close to ground-truth audio. These improvements provide an effective solution for efficient and controllable speech synthesis. Full article

Routing in flying ad hoc networks (FANETs) is hindered by high mobility, trajectory-induced topology dynamics, and energy constraints. Conventional topology-based or position-based protocols often fail due to stale link information and limited neighbor awareness. This paper proposes a trajectory-informed routing protocol enhanced by Q-learning: Traj-Q-GPSR, tailored for mission-oriented UAV swarm networks. By leveraging mission-planned flight trajectories, the protocol builds time-aware two-hop neighbor tables, enabling routing decisions based on both current connectivity and predicted link availability. This spatiotemporal information is integrated into a reinforcement learning framework that dynamically optimizes next-hop selection based on link stability, queue length, and node mobility patterns. To further enhance adaptability, the learning parameters are adjusted in real time according to network dynamics. Additionally, a delay-aware queuing model is introduced to forecast optimal transmission timing, thereby reducing buffering overhead and mitigating redundant retransmissions. Extensive ns-3 simulations across diverse mobility, density, and CBR connections demonstrate that the proposed protocol consistently outperforms GPSR, achieving up to 23% lower packet loss, over 80% reduction in average end-to-end delay, and improvements of up to 37% and 52% in throughput and routing efficiency, respectively. Full article

With the advancement of Agriculture 4.0, intelligent systems and data-driven technologies offer new opportunities for pork supply-demand balance regulation, while also confronting challenges such as production cycle fluctuations and epidemic outbreaks. This paper introduces a knowledge-driven smart system for pork supply-demand regulation, which integrates essential components including a knowledge base, a mathematical-model-based expert system, an enhanced optimization framework, and a real-time feedback mechanism. Around the core of the system, a nonlinear constrained optimization model is established, which uses adjustments to newly retained gilts as decision variables and minimizes supply-demand squared errors as its objective function, incorporating multi-dimensional factors such as pig growth dynamics, epidemic impacts, consumption trends, and international trade into its analytical framework. By harnessing dynamic decision-making capabilities of reinforcement learning (RL), we design an optimization architecture centered on the Q-learning mechanism and dual-strategy pools, which is integrated into the honey badger algorithm to form the RL-enhanced honey badger algorithm (RLEHBA). This innovation achieves an efficient balance between exploration and exploitation in model solving and improves system adaptability. Numerical experiments demonstrate RLEHBA’s superior performance over State-of-the-Art algorithms on the CEC 2017 benchmark. A case study of China’s 2026 pork regulation confirms the system’s practical value in stabilizing the supply-demand balance and optimizing resource allocation. Finally, some targeted managerial insights are proposed. This study constructs a replicable framework for intelligent livestock regulation, and it also holds transformative significance for sustainable and adaptive supply chain management in global agri-food systems. Full article

This study presents a reinforcement learning–based approach to optimize replenishment policies in the presence of uncertainty, with the objective of minimizing total costs, including inventory holding, shortage, and ordering costs. The focus is on single-level assembly systems, where both component delivery lead times and finished product demand are subject to randomness. The problem is formulated as a Markov decision process (MDP), in which an agent determines optimal order quantities for each component by accounting for stochastic lead times and demand variability. The Deep Q-Network (DQN) algorithm is adapted and employed to learn optimal replenishment policies over a fixed planning horizon. To enhance learning performance, we develop a tailored simulation environment that captures multi-component interactions, random lead times, and variable demand, along with a modular and realistic cost structure. The environment enables dynamic state transitions, lead time sampling, and flexible order reception modeling, providing a high-fidelity training ground for the agent. To further improve convergence and policy quality, we incorporate local search mechanisms and multiple action space discretizations per component. Simulation results show that the proposed method converges to stable ordering policies after approximately 100 episodes. The agent achieves an average service level of 96.93%, and stockout events are reduced by over 100% relative to early training phases. The system maintains component inventories within operationally feasible ranges, and cost components—holding, shortage, and ordering—are consistently minimized across 500 training episodes. These findings highlight the potential of deep reinforcement learning as a data-driven and adaptive approach to inventory management in complex and uncertain supply chains. Full article

To reduce the significant energy consumption in data centers, virtual machine scheduling optimization and server consolidation are deployed. However, existing server power load (SPL) models typically adopt linear approximations for model developments, which results in inaccuracy with actual SPL characteristics, hindering the optimal solution of virtual machine scheduling. Therefore, intelligent virtual machine scheduling (IVMS) is proposed based on a CPU temperature-involved server load model for data center energy conservation. The IVMS establishes a novel server power load model considering the influence of CPU temperature to capture the actual server load characteristics. Based on the model, the Q-learning method is utilized to solve the problem with the advantage of global optimization to obtain the scheduling solution that further improves calculation accuracy. The performance of the proposed IVMS is evaluated and compared to existing methods by both simulation and experiments in data centers, proving that the IVMS can better predict SPL characteristics and further reduce server energy consumption. Full article

Background/Objectives: Knee osteoarthritis (KOA) is a prevalent disorder affecting both older adults and younger individuals, leading to compromised joint function and mobility. Early and accurate detection is critical for effective intervention, as treatment options become increasingly limited as the disease progresses. Traditional diagnostic methods rely heavily on the expertise of physicians and are susceptible to errors. The demand for utilizing deep learning models in order to automate and improve the accuracy of KOA image classification has been increasing. In this research, a unique deep learning model is presented that employs autoencoders as the primary mechanism for feature extraction, providing a robust solution for KOA classification. Methods: The proposed model differentiates between KOA-positive and KOA-negative images and categorizes the disease into its primary severity levels. Levels of severity range from “healthy knees” (0) to “severe KOA” (4). Symptoms range from typical joint structures to significant joint damage, such as bone spur growth, joint space narrowing, and bone deformation. Two experiments were conducted using different datasets to validate the efficacy of the proposed model. Results: The first experiment used the autoencoder for feature extraction and classification, which reported an accuracy of 96.68%. Another experiment using autoencoders for feature extraction and Extreme Learning Machines for actual classification resulted in an even higher accuracy value of 98.6%. To test the generalizability of the Knee-DNS system, we utilized the Butterfly iQ+ IoT device for image acquisition and Google Colab’s cloud computing services for data processing. Conclusions: This work represents a pioneering application of autoencoder-based deep learning models in the domain of KOA classification, achieving remarkable accuracy and robustness. Full article

The rapid increase in the deployment of Internet of Things (IoT) sensor networks has led to an exponential growth in data generation and an unprecedented demand for efficient resource management infrastructure. Ensuring end-to-end communication across multiple heterogeneous network domains is crucial to maintaining Quality of Service (QoS) requirements, such as low latency and high computational capacity, for IoT applications. However, limited computing resources at multi-access edge computing (MEC), coupled with increasing IoT network requests during task offloading, often lead to network congestion, service latency, and inefficient resource utilization, degrading overall system performance. This paper proposes an intelligent task offloading and resource orchestration framework to address these challenges, thereby optimizing energy consumption, computational cost, network congestion, and service latency in dynamic IoT-MEC environments. The framework introduces task offloading and a dynamic resource orchestration strategy, where task offloading to the MEC server ensures an efficient distribution of computation workloads. The dynamic resource orchestration process, Service Function Chaining (SFC) for Virtual Network Functions (VNFs) placement, and routing path determination optimize service execution across the network. To achieve adaptive and intelligent decision-making, the proposed approach leverages Deep Reinforcement Learning (DRL) to dynamically allocate resources and offload task execution, thereby improving overall system efficiency and addressing the optimal policy in edge computing. Deep Q-network (DQN), which is leveraged to learn an optimal network resource adjustment policy and task offloading, ensures flexible adaptation in SFC deployment evaluations. The simulation result demonstrates that the DRL-based scheme significantly outperforms the reference scheme in terms of cumulative reward, reduced service latency, lowered energy consumption, and improved delivery and throughput. Full article

Fifth-generation Vehicle-to-Everything (V2X) networks have ushered in a new set of challenges that negatively affect seamless connectivity, specifically owing to high user equipment (UE) mobility and high density. As UE accelerates, there are frequent transitions from one cell to another, and handovers (HOs) are triggered by network performance metrics, including latency, higher energy consumption, and greater packet loss. Traditional HO mechanisms fail to handle such network conditions, requiring the development of Intelligent HO Decisions for V2X (IHD-V2X). By leveraging Q-Learning, the intelligent mechanism seamlessly adapts to real-time network congestion and varying UE speeds, thereby resulting in efficient handover decisions. Based on the results, IHD-V2X significantly outperforms the other mechanisms in high-density and high-mobility networks. This results in a reduction of 73% in unnecessary handover operations, and an 18% reduction in effective energy consumption. On the other hand, it improved handover success rates by 80% from the necessary handover and lowered packet loss for high mobility UE by 73%. The latency was kept at a minimum of 22% for application-specific requirements. The proposed intelligent approach is particularly effective for high-mobility situations and ultra-dense networks, where excessive handovers can degrade user experience. Full article

We propose a probabilistic shaping (PS) scheme based on a single-layer lookup table (LUT) that employs only one LUT for symbol mapping while achieving favorable system performance. This scheme reduces the average power of the signal by adjusting the symbol distribution using a specialized LUT architecture and a flexible shaping proportion. The simulation results indicate that the proposed PS scheme delivers performance comparable to that of the conventional constant-composition distribution-matching-based probabilistic shaping (CCDM-PS) algorithm. Specifically, it reduces the bit error rate (BER) from 1.2376 $\times {10}^{-4}$ to 6.3256 $\times {10}^{-5}$, corresponding to a 48.89% improvement. The radial basis function neural network (RBFNN) effectively compensates for nonlinear distortions and further enhances transmission performance due to its simple architecture and strong capacity for nonlinear learning. In this work, we combine lookup-table-based probabilistic shaping (LUT-PS) with RBFNN-based nonlinear equalization for the first time, completing the transmission of 16-QAM OFDM signals over a photonic terahertz-over-fiber system operating at 400 GHz. Simulation results show that the proposed approach reduces the BER by 81.45% and achieves a maximum Q-factor improvement of up to 23 dB. Full article

Autonomous Underwater Vehicles (AUVs) operating in dynamic, constrained underwater environments demand sophisticated navigation and detection fusion capabilities that traditional methods often fail to provide. This paper introduces a novel hybrid framework that synergistically fuses a Multithreaded Energy-Optimized Batch Informed Trees (MEO-BIT*) algorithm with Deep Q-Networks (DQN) to achieve robust AUV autonomy. The MEO-BIT* component delivers efficient global path planning through (1) a multithreaded batch sampling mechanism for rapid state-space exploration, (2) heuristic-driven search accelerated by KD-tree spatial indexing for optimized path discovery, and (3) an energy-aware cost function balancing path length and steering effort for enhanced endurance. Critically, the DQN component facilitates dynamic obstacle detection and adaptive local navigation, enabling the AUV to adjust its trajectory intelligently in real time. This integrated approach leverages the strengths of both algorithms. The global path intelligence of MEO-BIT* is dynamically informed and refined by the DQN’s learned perception. This allows the DQN to make effective decisions to avoid moving obstacles. Experimental validation in a simulated Achao waterway (Chile) demonstrates the MEO-BIT* + DQN system’s superiority, achieving a 46% reduction in collision rates (directly reflecting improved detection and avoidance fusion), a 15.7% improvement in path smoothness, and a 78.9% faster execution time compared to conventional RRT* and BIT* methods. This work presents a robust solution that effectively fuses two key components: the computational efficiency of MEO-BIT* and the adaptive capabilities of DQN. This fusion significantly advances the integration of navigation with dynamic obstacle detection. Ultimately, it enhances AUV operational performance and autonomy in complex maritime scenarios. Full article

Unmanned Aerial Vehicle (UAV) path planning is critical for ensuring flight safety and enhancing mission execution efficiency. This problem is typically formulated as a complex, multi-constrained, and nonlinear optimization task, often addressed using meta-heuristic algorithms. The Crested Porcupine Optimizer (CPO) has become an excellent method to solve this problem; however, the standard CPO has limitations, such as the lack of adaptive parameter tuning to adapt to complex environments, slow convergence, and the tendency to fall into local optimal solutions. To address these issues, this paper proposes an algorithm named QCPO, which integrates CPO with Q-learning to improve UAV path optimization performance. Q-learning is employed to adaptively adjust the key parameters of the CPO, thereby overcoming the limitations of traditional fixed-parameter settings. Inspired by the porcupine’s defense mechanisms, a novel audiovisual coordination strategy is introduced to balance visual and auditory responses, accelerating convergence in the early optimization stages. A refined position update mechanism is designed to prevent excessive step sizes and boundary violations, enhancing the algorithm’s global search capability. A B-spline-based trajectory smoothing method is also incorporated to improve the feasibility and smoothness of the planned paths. In this paper, we compare QCPO with four outstanding heuristics, and QCPO achieves the lowest path cost in all three test scenarios, with path cost reductions of 30.23%, 26.41%, and 33.47%, respectively, compared to standard CPO. The experimental results confirm that QCPO offers an efficient and safe solution for UAV path planning. Full article

To address the fact that changes in network topology can have an impact on the performance of routing, this paper proposes an Elastic Routing Algorithm based on Multi-Agent Deep Deterministic Policy Gradient (ERA-MADDPG), which is implemented within the framework of Multi-Agent Deep Deterministic Policy Gradient (MADDPG) in deep reinforcement learning. The algorithm first builds a three-layer architecture based on Software-Defined Networking (SDN). The top-down layers are the multi-agent layer, the controller layer, and the data layer. The architecture’s processing flow, including real-time data layer information collection and dynamic policy generation, enables the ERA-MADDPG algorithm to exhibit strong elasticity by quickly adjusting routing decisions in response to topology changes. The actor-critic framework combined with Convolutional Neural Networks (CNN) to implement the ERA-MADDPG routing algorithm effectively improves training efficiency, enhances learning stability, facilitates collaboration, and improves algorithm generalization and applicability. Finally, simulation experiments demonstrate that the convergence speed of the ERA-MADDPG routing algorithm outperforms that of the Multi-Agent Deep Q-Network (MADQN) algorithm and the Smart Routing based on Deep Reinforcement Learning (SR-DRL) algorithm, and the training speed in the initial phase is improved by approximately 20.9% and 39.1% compared to the MADQN algorithm and SR-DRL algorithm, respectively. The elasticity performance of ERA-MADDPG is quantified by re-convergence speed: under 5–15% topology node/link changes, its re-convergence speed is over 25% faster than that of MADQN and SR-DRL, demonstrating superior capability to maintain routing efficiency in dynamic environments. Full article

This paper presents a novel deep learning-based automatic modulation recognition (AMR) model, designed to classify ten modulation types from complex I/Q signal data. The proposed architecture, named CV-CNN-TCN, integrates Complex-Valued Convolutional Neural Networks (CV-CNNs) with Temporal Convolutional Networks (TCNs) to jointly extract spatial and temporal features while preserving the inherent phase information of the signal. An enhanced variant, CV-CNN-TCN-DCC, incorporates dilated causal convolutions to further strengthen temporal representation. The models are trained and evaluated on the benchmark RadioML2016.10b dataset. At SNR = −10 dB, the CV-CNN-TCN achieves a classification accuracy of 37%, while the CV-CNN-TCN-DCC improves to 40%. In comparison, ResNet reaches 33%, and other models such as CLDNN (convolutional LSTM dense neural network) and SCRNN (Sequential Convolutional Recurrent Neural Network) remain below 30%. At 0 dB SNR, the CV-CNN-TCN-DCC achieves a Jaccard index of 0.58 and an MCC of 0.67, outperforming ResNet (0.55, 0.64) and CNN (0.53, 0.61). Furthermore, the CV-CNN-TCN-DCC achieves 75% accuracy at SNR = 10 dB and maintains over 90% classification accuracy for SNRs above 2 dB. These results demonstrate that the proposed architectures, particularly with dilated causal convolutional enhancements, significantly improve robustness and generalization under low-SNR conditions, outperforming state-of-the-art models in both accuracy and reliability. Full article