Search Results (475)

The disorderly charging of a large number of electric vehicles (EVs) intensifies the operational pressure on the distribution network and negatively impacts the users’ charging experience. This paper proposes an orderly-charging optimization strategy based on the Deep Deterministic Policy Gradient (DDPG) algorithm. First, a comprehensive EV charging behavior model is developed, incorporating regional functional characteristics, vehicle categories, and user behavioral diversity to more accurately reflect real-world charging patterns. Second, a closed-loop control architecture is designed, integrating charging load forecasting, dynamic energy storage regulation, and real-time power allocation. Finally, the DDPG algorithm is applied to enable intelligent dynamic power allocation, which effectively flattens peak–valley load disparities and minimizes user charging costs. The simulation results demonstrate that the proposed strategy significantly enhances distribution network performance and user satisfaction. Specifically, the strategy reduces peak load by 17.08% and achieves a total cost saving of USD 511.49 (17.08%). By considering real-world zones and diverse EV types, this strategy provides substantial engineering value for practical implementation in multi-zone charging systems. Full article

In the process of steel rolling production, the speed reduction compensation of the rolling mill is a key link to ensure the stability of slab rolling and product quality. This paper proposes a hybrid compensation method that integrates motor dynamic modeling with reinforcement learning to minimize mass flow error between adjacent rolling mills during slab rolling. A two-stage compensation strategy is designed, consisting of a constant-gain compensation phase followed by a decaying compensation phase, which explicitly accounts for the repetitive and consistent rolling conditions in batch slab production. Based on a motor dynamics-based theoretical model, an initial estimation of compensation parameters is first obtained, providing a physically interpretable starting point for optimization. Subsequently, a Deep Deterministic Policy Gradient (DDPG) algorithm is employed to iteratively refine the compensation parameters by learning from the mass flow error of each rolled slab, enabling data-driven adaptation while preserving physical consistency. Simulation results demonstrate that the proposed hybrid approach significantly reduces the mass flow error and achieves stable convergence, outperforming strategies with randomly initialized parameters. The results verify the effectiveness and novelty of the proposed method in combining model-based insight with reinforcement learning for intelligent and adaptive rolling mill speed drop compensation. Full article

Global warming and increasing heat events necessitate long-term assessments of passive design strategies to ensure thermal resilience under future climatic conditions. Although machine-learning-based Surrogate Models (SMs) offer timely approximation of building performance compared to conventional simulation-based approaches, the lack of uncertainty quantification raises concerns about the reliability of their design optimization outcomes. This study aims to develop a robust surrogate-assisted optimization framework, based on a probabilistic Bayesian Neural Network (BNN) model and supported by an uncertainty-aware objective function. The framework is applied to an affordable housing case study in Surakarta, Indonesia, evaluating its generalizability under current and future climatic scenarios for 2050, 2070, and 2090. Thermal resilience is assessed through overheating hours exceeding acceptability limits in Southeast Asian context, using a parametric workflow implemented in Ladybug-tools and Grasshopper 3D. Compared to simulated test data, the BNN model demonstrates reliable predictive accuracy and probabilistic inference (R² = 0.99, MAE = 0.52%, CRPS = 0.38%). Furthermore, validation against re-evaluated optimal solutions shows low error ranges (RMSE = 0.43%, MAE = 0.33%), outperforming the deterministic SM optimization approach—using Artificial Neural Networks—by a factor of five. Overall, the uncertainty-aware framework provides a feasible, overconfidence-resistant, and reliable surrogate-assisted optimization method, identifying optimal solutions closely matching those from simulation-based optimization while reducing computational time by 96%. Full article

Decision-making for maneuvering in the presence of long-range threats is crucial for enhancing the safety and reliability of autonomous aerial platforms operating in beyond-line-of-sight environments. This study employs the Deep Q-Network (DQN) method to investigate maneuvering strategies for simultaneously avoiding incoming high-speed threats and re-establishing tracking of a maneuvering target platform. First, kinematic models for the aerial platforms and the approaching interceptor are developed, and a DQN training environment is constructed based on these models. A DQN framework is then designed, integrating scenario-specific state representation, action space, and a hybrid reward structure to enable autonomous strategy learning without prior expert knowledge. The agent is trained within this environment to achieve near-optimal maneuvering decisions, with comparative evaluations against Q-learning and deep deterministic policy gradient (DDPG) baselines. Simulation results demonstrate that the trained model outperforms the baselines on key metrics by effectively avoiding approaching threats, re-establishing robust target tracking, reducing maneuver time, and exhibiting strong generalization across challenging scenarios. This work advances Beyond-Visual-Range (BVR) maneuver planning and provides a foundational methodological framework for future research on complex multi-stage aerial pursuit–evasion problems. Full article

Rainfall infiltration is one of the main factors inducing slope instability, while the spatial heterogeneity and uncertainty of soil parameters have profound impacts on slope response characteristics and stability evolution. Traditional deterministic analysis methods struggle to reveal the dynamic risk evolution process of the system under heavy rainfall. Therefore, this paper proposes an uncertainty analysis framework combining Karhunen–Loève Expansion (KLE) random field theory, Polynomial Chaos Kriging (PCK) surrogate modeling, and Monte Carlo simulation to efficiently quantify the probabilistic characteristics and spatial risks of rainfall-induced slope instability. First, for key strength parameters such as cohesion and internal friction angle, a two-dimensional random field with spatial correlation is constructed to realistically depict the regional variability of soil mechanical properties. Second, a PCK surrogate model optimized by the LARS algorithm is developed to achieve high-precision replacement of finite element calculation results. Then, large-scale Monte Carlo simulations are conducted based on the surrogate model to obtain the probability distribution characteristics of slope safety factors and potential instability areas at different times. The research results show that the slope enters the most unstable stage during the middle of rainfall (36–54 h), with severe system response fluctuations and highly concentrated instability risks. Deterministic analysis generally overestimates slope safety and ignores extreme responses in tail samples. The proposed method can effectively identify the multi-source uncertainty effects of slope systems, providing theoretical support and technical pathways for risk early warning, zoning design, and protection optimization of slope engineering during rainfall periods. Full article

This study presents a unified parametric optimization framework for the thermal design of microchannel spreaders used in high-power processor cooling. The fin geometry is expressed in a shape-agnostic parametric form defined by fin thickness, top and bottom gap widths, and channel height, without prescribing a fixed cross-section. This approach accommodates practical fin profiles ranging from rectangular to tapered and V-shaped, allowing continuous geometric optimization within manufacturability and hydraulic limits. A coupled analytical–numerical model integrates conduction through the spreader base, interfacial resistance across the thermal interface material (TIM), and convection within the coolant channels while enforcing a pressure-drop constraint. The optimization uses a deterministic continuation method with smooth sigmoid mappings and penalty functions to maintain constraint satisfaction and stable convergence across the design space. The total thermal resistance ($R_{\mathrm{tot}}$) is minimized over spreader conductivities $k_s=400–2200$ W m⁻¹ K⁻¹ (copper to CVD diamond), inlet fluid velocities $U_{\mathrm{in}}=0.5–5.5$ m s⁻¹, maximum pressure drops of 10–50 kPa, and fluid pass counts $N_p\in \{1,2,3\}$. The resulting maps of optimized fin dimensions as functions of $k_s$ provide continuous design charts that clarify how material conductivity, flow rate, and pass configuration collectively determine the geometry, minimizing total thermal resistance, thereby reducing chip temperature rise for a given heat load. Full article

The growing deployment of connected and autonomous vehicles (CAVs) requires coordinated control strategies that jointly address safety, mobility, and energy efficiency. This paper presents a novel two-stage cooperative control framework for autonomous hybrid electric vehicle (HEV) platoons based on machine learning. In the first stage, a metric learning-based distributed model predictive control (ML-DMPC) strategy is proposed to enable cooperative longitudinal control among heterogeneous vehicles, explicitly incorporating inter-vehicle interactions to improve speed tracking, ride comfort, and platoon-level energy efficiency. In the second stage, a multi-agent twin-delayed deep deterministic policy gradient (MATD3) algorithm is developed for real-time energy management, achieving an optimal power split between the engine and battery while reducing Q-value overestimation and accelerating learning convergence. Simulation results across multiple standard driving cycles demonstrate that the proposed framework outperforms conventional distributed model predictive control (DMPC) and multi-agent deep deterministic policy gradient (MADDPG)-based methods in fuel economy, stability, and convergence speed, while maintaining battery state of charge (SOC) within safe limits. To facilitate future experimental validation, a dSPACE-based hardware-in-the-loop (HIL) architecture is designed to enable real-time deployment and testing of the proposed control framework. Full article

This work proposes a non-singular fast terminal sliding mode control (NFTSMC) strategy based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm and a nonlinear disturbance observer (NDO) to address the issues of modeling errors, motion disturbances, and transmission friction in robotic manipulators. Firstly, a novel modular serial 5-DOF robotic manipulator configuration is designed, and its kinematic and dynamic models are established. Secondly, a nonlinear disturbance observer is employed to estimate the total disturbance of the system and apply feedforward compensation. Based on boundary layer technology, an improved NFTSMC method is proposed to accelerate the convergence of tracking errors, reduce chattering, and avoid singularity issues inherent in traditional terminal sliding mode control. The stability of the designed control system is proved using Lyapunov stability theory. Subsequently, a deep reinforcement learning (DRL) agent based on the TD3 algorithm is trained to adaptively adjust the control gains of the non-singular fast terminal sliding mode controller. The dynamic information of the robotic manipulator is used as the input to the TD3 agent, which searches for optimal controller parameters within a continuous action space. A composite reward function is designed to ensure the stable and efficient learning of the TD3 agent. Finally, the motion characteristics of three joints for the designed 5-DOF robotic manipulator are analyzed. The results show that compared to the non-singular fast terminal sliding mode control algorithm based on a nonlinear disturbance observer (NDONFT), the non-singular fast terminal sliding mode control algorithm integrating a nonlinear disturbance observer and the Twin Delayed Deep Deterministic Policy Gradient algorithm (TD3NDONFT) reduces the mean absolute error of position tracking for the three joints by 7.14%, 19.94%, and 6.14%, respectively, and reduces the mean absolute error of velocity tracking by 1.78%, 9.10%, and 2.11%, respectively. These results verify the effectiveness of the proposed algorithm in enhancing the trajectory tracking accuracy of the robotic manipulator under unknown time-varying disturbances and demonstrate its strong robustness against sudden disturbances. Full article

The primary objective of this research is to develop a probabilistic nonlinear model predictive control structure (NMPC) that efficiently operates uncrewed surface vehicles (USVs) in an environment that has probabilistic disturbances, such as wind, waves, and currents of the water, while simultaneously maneuvering the vehicle in a way that avoids stationary or moving stochastic obstacles in its path. The proposed controller structure considers the mean and covariances of the inputs or state variables of the vehicle in the cost function to handle probabilistic disturbances, where an extended Kalman filter (EKF) is utilized to calculate the mean, and the covariances are calculated dynamically via a linear matrix equality based on this mean and obtained system matrices with successive linearization for every sampling instance. The proposed control structure deals with non-zero-mean probabilistic disturbances such as water current via an innovative approach that treats the mean of the disturbance as a deterministic part, which is estimated by a disturbance observer and eliminated by a control term in the controller in addition to the control signal obtained via MPC optimization; the effect of the remaining zero-mean part is handled over its covariance during the probabilistic MPC optimization. The probabilistic constraints are also dealt with by converting them to deterministic constraints, as in linear probabilistic MPC. However, unlike the linear MPC, these constraints updated each sampling instance with the information obtained via successive linearization. The control structure incorporates the velocity obstacle (VO) method for collision avoidance. In order to ensure stability, the proposed NMPC adopts a dual-mode strategy, and a stability analysis is presented. In the second mode, an LQG design that ensures stability in the existence of non-zero mean disturbance is also provided. The simulation results demonstrate that the proposed probabilistic NMPC framework effectively handles probabilistic disturbances as well as both stationary and moving obstacles, ensuring collision avoidance while reaching the desired position and orientation through optimal path tracking, outperforming the conventional NMPC. Full article

Fused Filament Fabrication (FFF) plays a critical role in several application fields due to its affordability and manufacturing versatility. However, FFF reliability remains vulnerable to rapid environmental and operational variations, which directly influence the dimensional precision and mechanical properties of printed parts. To address these challenges, this study presents a simulation-based computational framework for the real-time early-warning supervision of FFF systems. The proposed multilayer architecture integrates high-throughput data acquisition, distributed computing, and dynamic analysis to proactively detect deviations from optimal conditions. Architectural verification follows a simulation-first methodology designed to replicate the operational dynamics of standard FFF hardware. By employing telemetry streams to test the decision-making pipeline, the study isolates computational performance, such as throughput and latency, from the confounding variables of physical hardware. This approach enables a precise, deterministic assessment of the system’s responsiveness, serving as a foundational de-risking step prior to empirical implementation. Numerical results of this study show that the integrated distributed computing model successfully manages high-frequency telemetry with a response time within the operational safety margins, confirming the architectural viability of the proposed solution. By providing insights into system behavior prior to physical deployment, this simulation-first strategy mitigates implementation risks and offers practical guidance for developing autonomous additive manufacturing workflows, advancing the transition toward intelligent industrial FFF. Full article

This article presents the design, simulation, and real-time implementation of an intelligent water level control system using Deep Reinforcement Learning (DRL) with the Deep Deterministic Policy Gradient (DDPG) algorithm. The control policy was initially trained in a MATLAB-based simulation environment, where actor–critic neural networks were trained and optimized to ensure accurate and robust performance under dynamic and nonlinear conditions. The trained policy was subsequently deployed on a low-cost embedded platform (Arduino Uno), demonstrating its feasibility for real-time embedded applications. Experimental results confirm the controller’s ability to adapt to external disturbances. Quantitatively, the proposed controller achieved a steady-state error of less than 0.05 cm and an overshoot of 16% in the physical implementation, outperforming conventional proportional–integral–derivative (PID) control by 22% in tracking accuracy. The combination of the DDPG algorithm and low-cost hardware implementation demonstrates the feasibility of real-time deep learning-based control for intelligent water management. Furthermore, the proposed architecture is directly applicable to low-cost Internet of Things (IoT)-based water management systems, enabling autonomous and adaptive control in real-world hydraulic infrastructures. This proposal demonstrates its potential for smart agriculture, distributed sensor networks, and scalable and resource-efficient water systems. Finally, the main novelty of this work is the deployment of a DRL-based controller on a resource-constrained microcontroller, validated under real-world perturbations and sensor noise. Full article

This study investigates the neutronic performance of burnable poisons (BPs) in an epithermal spectrum supercritical carbon dioxide (S-CO₂)-cooled reactor. Twelve candidate BP materials are systematically evaluated, including rare-earth oxides (e.g., HfO₂, Er₂O₃, Eu₂O₃, etc.) and boron-based compounds (B₄C and PACS). The deterministic neutron transport code KYLIN-I with the ENDF/B VI 45-group cross-section library is employed for analysis. According to the calculation results, Eu₂O₃ effectively suppresses the initial k_inf of the epithermal-spectrum fuel assembly to ~1.2 with a relatively low weight fraction (~2.6%) while maintaining a total temperature coefficient (TTC) lower than −1.4 pcm/K throughout the entire burnup period. HfO₂ and Er₂O₃, at approximately 15% weight fraction, achieve TTC values better than −2 pcm/K. Furthermore, both Eu₂O₃ and HfO₂ contribute to maintaining a low, stable power peaking factor (PPF) below 1.24 throughout the burnup process. This study provides a theoretical foundation and technical support for designing an efficient and safe S–CO₂-cooled nuclear reactor. It highlights the importance of selecting BP materials that are well-adapted to the neutron spectrum and optimizing the fuel assembly configuration accordingly. Full article

Sixth-generation (6G) wireless systems aim to integrate terrestrial, aerial, and satellite networks to support large-scale remote sensing and service delivery. In such non-terrestrial networks (NTNs), channels change quickly and the multi-tier architecture is heterogeneous, which makes real-time channel state acquisition and cooperative resource scheduling difficult. This paper proposes an FMA-MADDPG framework that combines a channel prediction module with a constraint-based multi-agent deep deterministic policy gradient scheme. The Fusion of Mamba and Attention (FMA) predictor uses a Mamba state-space backbone and a multi-head self-attention block to learn both long-term channel evolution and short-term fluctuations, and forecasts future CSI. The predicted channel information is added to the agents’ observations so that scheduling decisions can take expected channel variations into account. A constraint-based reward is also designed, with explicit performance thresholds and anti-idle penalties, to encourage fairness, avoid free-riding, and promote cooperation among heterogeneous agents. In a representative NTN uplink scenario, the proposed method achieves higher total reward, efficiency, load balance, and cooperation than several DRL baselines, with relative gains around 10–20% on key metrics. These results indicate that prediction-aware cooperative reinforcement learning is a useful approach for resource optimization in future 6G NTN systems. Full article

This study presents the design, real-time implementation, and full-scale experimental validation of a rule-based Energy Management Strategy (EMS) for a dual-stack Fuel Cell Hybrid Electric Vehicle (FCHEV) developed on a Jeep Wrangler platform. Unlike previous studies, predominantly focused on simulation-based analysis or single-stack architectures, this work provides comprehensive vehicle-level experimental validation of a deterministic real-time EMS applied to a dual fuel cell system in an SUV-class vehicle. The control algorithm, deployed on a National Instruments CompactRIO embedded controller, ensures deterministic real-time energy distribution and stable hybrid operation under dynamic load conditions. Simulation analysis conducted over eight consecutive WLTC cycles shows that both fuel cell stacks operate predominantly within their optimal efficiency range (25–35 kW), achieving an average DC efficiency of 68% and a hydrogen consumption of 1.35 kg/100 km under idealized conditions. Experimental validation on the Wrangler FCHEV demonstrator yields a hydrogen consumption of 1.67 kg/100 km, corresponding to 1.03 kg/100 km·m² after aerodynamic normalization (Cd·A = 1.624 m²), reflecting real-world operating constraints. The proposed EMS promotes fuel-cell durability by reducing current cycling amplitude and maintaining operation within high-efficiency regions for the majority of the driving cycle. By combining deterministic real-time embedded control with vehicle-level experimental validation, this work strengthens the link between EMS design and practical deployment and provides a scalable reference framework for future hydrogen powertrain control systems. Full article

This research develops and implements a novel reinforcement learning (RL) architecture to address the trajectory-tracking problem in bipedal robotic systems under articulated-joint constraints. The proposed RL framework extends previously designed adaptive controllers characterized by state-dependent gain structures. The learning mechanism comprises two hierarchical adaptation layers: the first employs an adaptive dynamic programming (ADP) formulation to approximate the Bellman value function using a class of continuous-time dynamic neural networks. In contrast, the second uses an iterative optimization scheme based on the deep deterministic policy gradient (DDPG) algorithm. The resulting control strategy minimizes a robust performance index defined over the tracking trajectories of a system with uncertain and nonlinear dynamics representative of bipedal locomotion. The dynamic programming formulation ensures robustness to bounded parametric uncertainties and external perturbations. By approximating the Hamilton–Jacobi–Bellman (HJB) value function using neural network structures, a closed-loop controller design is systematically established. Numerical simulations demonstrate the convergence of the tracking error to a region centered at the origin with a size that depends on the approximation quality of the selected neural network. To assess the effectiveness of the proposed approach, a conventional state-feedback control design is adopted as a benchmark, revealing that the suggested method produces a lower cumulative tracking error norm (0.023 vs. 0.037 rad·s) in the trajectory-tracking control problem for all robotic joints while simultaneously reducing the control effort required to complete motion tasks. Full article