Search Results (1,019)

The rapid integration of electric vehicles (EVs) and decentralized renewable energy sources is transforming urban power systems, while simultaneously increasing the complexity of real-time coordination across charging infrastructure, distributed energy resources, and grid-support devices. This systematic review synthesizes recent research on multi-agent reinforcement learning (MARL) for EV–grid coordination, with emphasis on four emerging dimensions: forecasting-informed control, safety-constrained learning, explainability and interpretability, and trustworthy decentralized coordination. A systematic literature search was conducted in IEEE Xplore, Scopus, Web of Science, ScienceDirect, MDPI, and arXiv, covering primarily the period 2016–2025, with selected early-2026 studies retained where relevant, with selected earlier foundational studies retained for context. The review was conducted and reported in accordance with the PRISMA 2020 framework. A total of 412 records were identified through database searching; after duplicate removal and screening, 58 studies were included in the final qualitative synthesis. The reviewed literature shows that MARL is increasingly being applied to EV charging coordination, demand-side management, community energy systems, transactive energy, and ancillary grid services. The evidence further indicates that forecasting integration improves anticipatory control, safety-aware formulations enhance operational reliability, and explainability-oriented designs help address transparency and trust barriers in safety-critical grid environments. However, the literature remains limited by heterogeneous benchmarks, inconsistent evaluation metrics, and a lack of real-world deployment evidence. This review provides a structured synthesis of current methodologies, identifies critical research gaps, and outlines future directions for the development of safe, interpretable, and deployment-ready MARL frameworks for urban energy systems. Full article

In this paper, we present a study on the automated design optimization of a wideband low-noise amplifier (LNA) operating in Ku-band (12 to 18 GHz) using proximal policy optimization (PPO), one of the widely applied reinforcement learning (RL) algorithms for engineering problems. As a target microwave active circuit, we select a two-stage LNA architecture, where transmission lines (TLs) are dominantly used for impedance matching and gain/noise optimization. For simplicity, all widths of TLs were fixed so that the characteristic impedance is 50 Ω, with lengths of TLs being set as design parameters. In addition, dimension variables of capacitors were treated as design parameters and, in total, we optimized 29 parameters. For target specifications, we set both $S_{11}$ and $S_{22}$ to be below −10 dB over the 12–18 GHz band and the noise figure (NF) to be below 2 dB. A total of 20,140 simulations were performed for training and the overall process took about 24 h. The results show that both the reward and the loss converged appropriately, achieving the target specifications successfully. For the final results, we performed up to 25 predictions, and the prediction process was terminated early if a solution meeting all target specifications was found within the given number of attempts. The device model used was a commercial 150 nm GaN high-electron-mobility transistor (HEMT) process technology. Full article

Metallurgical batching—governing raw material proportioning across sintering, blast furnace ironmaking, converter steelmaking, and non-ferrous smelting—critically determines product quality, energy consumption, and production cost throughout the full process chain. Its inherent complexity, characterized by strong nonlinear physicochemical coupling, measurement delays of up to 1.5 h, and multi-source raw material disturbances, renders conventional linear programming and empirical methods inadequate for dynamic, multi-objective industrial environments. This review systematically examines 98 representative studies (2020–2026) on intelligent algorithms applied to metallurgical batching optimization. A two-dimensional analysis framework of the fusion algorithm function and metallurgical scene is established. All kinds of methods are divided into three categories: prediction-oriented, optimization-oriented and decision-oriented, covering four typical scenes of sintering burdening, blast furnace ironmaking, converter steelmaking and non-ferrous metal smelting. Traditional machine learning models achieve sintering burn-through point prediction with R² ≈ 0.85 and offer superior interpretability via SHAP analysis. Deep learning architectures deliver blast furnace silicon content prediction with RMSE ≈ 0.04%, while multi-objective evolutionary algorithms provide mature Pareto optimization for batching cost and carbon objectives. Reinforcement learning holds long-term potential for closed-loop adaptive control but remains constrained by Sim-to-Real safety barriers. Converter steelmaking and non-ferrous smelting are identified as underexplored domains. Three priority directions are proposed: domain-adaptive predictive modeling for cross-plant generalization, real-time re-optimization embedding mechanism constraints, and safe reinforcement learning transfer via high-fidelity digital twins. Full article

This paper introduces a policy-search-based reinforcement learning algorithm aimed at generating optimal set-points of wind turbines in wind farms. The proposed approach addresses the problem of multivariable optimization in systems where the objective function is unknown or difficult to model. The algorithm is a model-free framework and relies solely on measured performance of the system. Namely, it does not require gradient information of the objective function or an explicit model of the aerodynamic interaction between wind turbines. The proposed scheme utilizes stochastic policy perturbations to explore the search space and update the policy parameters directly based on the observed reward signal. In this way, the algorithm progressively drives the control variables toward optimal operating conditions. The proposed policy-search reinforcement learning framework is analyzed to establish its connection with gradient-free optimization methods. The proposed method is applied to wind farm power optimization, where multiple turbine control variables must be adjusted in the presence of wake interactions cooperatively. The performance of the proposed approach is evaluated through extensive simulations under both steady-state and time-varying wind conditions. The proposed algorithm is compared with an extremum-seeking control method that was previously suggested for the same problem. The results demonstrate that the proposed approach is able to effectively maximize power production in wind farms while maintaining a simple and model-free optimization structure. Full article

Bird strikes are a key threat to aircraft wing leading edges. This investigation evaluates a honeycomb block reinforcement concept to improve bird strike resistance while maintaining structural efficiency. A validated simulation was developed using an explicit dynamic finite element approach, in which the bird was modeled as a soft body using smoothed particle hydrodynamics, and the wing leading edge was represented with a honeycomb block reinforcement concept. A design of experiments based on McKay Latin hypercube sampling was applied to comprehensively examine the effects of the geometric parameters on the maximum von Mises stress and maximum deformation. Response surface regression models were then constructed to approximate the impact responses and analyze the model correctness. These models were subsequently integrated into a constrained optimization methodology using sequential quadratic programming and population-based integrated learning to minimize deformation while limiting stress below the material yield threshold. The optimized honeycomb and skin configuration demonstrated a noticeable optimization of the maximum deformation within the yield stress limit compared with the baseline design. The results confirm that the proposed honeycomb block reinforcement concept, combined with a regression-based optimization strategy, constitutes a practical, computationally effective approach to improving bird strike resistance and provides a feasible design option for future impact-resistant wing leading-edge designs. Full article

Background/Objectives: The integration of machine learning (ML) within model-informed drug development (MIDD) represents a rapidly evolving paradigm in pharmacometrics, enabling improved prediction, optimization, and regulatory decision-making across drug development pipelines. However, the extent to which ML methods are explicitly integrated into regulatory decision-making remains limited and unevenly characterized. This study aims to systematically map the ML-MIDD scholarly landscape, identify core sources and contributors, assess thematic evolution, and summarize key methodological advancements through an integrative bibliometric and systematic review. Methods: A comprehensive literature search was conducted across Web of Science, Scopus, and PubMed (2015–2025), followed by metadata harmonization and deduplication. Bibliometric analysis was performed using Bibliometrix, VOSviewer, and PRISMA guidelines to characterize publication trends, collaboration patterns, thematic structures, and representative methodological contributions. Results: A total of 770 records were initially retrieved, with Scopus contributing the largest share (n = 343; 44.5%), followed by Web of Science (n = 322; 41.8%) and PubMed (n = 105; 13.6%). After deduplication, 607 unique publications remained (78.8% of total), and 560 were included in the final systematic review (97.6% of full texts). Publications spanned 269 sources, with core journals accounting for 28% of output. The United States led in volume (n = 665; 20.8%) and international collaboration (16.47%). Thematic evolution revealed transitions from foundational PK/PD methods (2016–2018) to applied ML-driven precision pharmacology (2022–2025). Conclusions: Emerging methods included deep learning, reinforcement learning, and hybrid mechanistic–ML models. ML-MIDD is a rapidly maturing interdisciplinary field, evidenced by expanding methodological diversity and increasing use of ML-enabled components within regulatory-relevant modeling workflows, rather than formal regulatory endorsement of ML-MIDD as a standalone methodology, indicating growing translational relevance but continued need for validation and regulatory clarity. Full article

When performing nighttime passive visible remote sensing of non-emissive land surfaces, illumination is typically dominated by weak moonlight that varies with lunar phase, producing low-radiance images with degraded textures and thus motivating low-radiance visible remote sensing image enhancement. We propose a Complementary Illumination–Semantic Prompt Diffusion framework (CISPD) that incorporates a semantic-invariant prompt and a self-learned illumination-aware prompt to guide diffusion-based low-light remote sensing image enhancement. During denoising, we sequentially inject two complementary prompts. We first retrieve a self-learned illumination-aware prompt from a learnable pool conditioned on the current latent context to correct non-uniform brightness, and then apply a semantic-invariant prompt extracted from a vision foundation model to reinforce geometric structures and suppress artifacts. To keep the two prompts complementary rather than redundant, we introduce a contrastive constraint that encourages their representations to remain distinct, and the dual prompts jointly steer the diffusion trajectory toward well-exposed results with faithful structures. Experiments on iSAID-dark and darkrs, together with LOLv1 and LOLv2, demonstrate that CISPD achieves the best PSNR and SSIM on iSAID-dark, strong qualitative generalization on darkrs, and competitive quantitative performance on LOLv1 and LOLv2. Full article

Multilingual learning is key in natural language processing, but is challenged by the transfer–interference trade-off, where positive transfer benefits certain languages, while negative interference affects others. Prior methods, including linguistic-based and embedding-based language clustering, have attempted to address this; yet, they remain constrained by their static design and lack of task-specific feedback. In this study, we propose a novel computational strategy inspired by molecular design that constructs molecules with targeted properties. Languages are modeled as nodes in an undirected graph, with edges representing the transfer strength. This language molecule is optimized via Reinforcement Learning to adjust edge connections and weights to enhance positive transfer and minimize interference, where graph clustering is applied, and clusters are then evaluated on the Named Entity Recognition and POS tagging tasks using 25 languages from the WikiANN dataset and 12 typologically diverse languages from the UDPOS dataset. Compared to linguistic and embedding-based language clustering baselines, our method yields substantial improvements, especially for low-resource languages, with some showing over 35% increase in F1 score, while high-resource languages benefit moderately, confirming reduced transfer–interference trade-off. Our atom–language model offers a novel path for multilingual learning, inspired by molecular principles from physical sciences. Full article

Digitalizing essential services opens up a new risk of exposing critical infrastructure to botnet infections. In a grid topology network, the neighbor-to-neighbor paths can be used by the malicious botnet to spread the infection. Previous white-hat worm launchers used heuristics and supervised learning to exterminate botnets, which demand specific conditions or a suitable dataset to be effective. Although reinforcement learning addressed these issues, it requires a longer time to train. This article proposes a framework to shorten training and improve the effectiveness of reinforcement learning. The framework applies four key principles: (1) surveying the network status with multi-tensor input, (2) removing irrelevant actions via a novel Chebyshev-based masking strategy, (3) reinforcing key actions with rewards, and (4) optimizing rewards for winning. Four reinforcement learning algorithms are implemented to evaluate the framework, which are vanilla policy gradient, deep Q-network, proximal policy optimization, and MuZero in a stylized grid topology network simulation. An ablation study indicates that the masking used in identify accounts for the majority of the improvement, whereas multi-channel in Survey alone can reduce performance without complementary masking, rewards, and optimization. With the mean winning rate improved by 49.129% and mean win efficiency improved by 118.8031% against our previous work, the framework effectiveness is confirmed in stylized simulations. Full article

To improve vehicle user service quality and address data privacy and security issues in intelligent transportation vehicle networking systems, a three-tier communication architecture with cloud-edge-end collaboration was designed in this paper. A Bayesian decision criterion was utilized to divide user data segments into fine-grained slices based on their privacy levels, and differential privacy techniques were applied to protect the offloaded data. To achieve multi-objective optimization between user service quality and data privacy and security, the problem was formulated as a constrained Markov decision process. A communication model, a caching model, a latency model, an energy consumption model, and a data-fragment privacy protection model were designed. Additionally, a deep reinforcement learning algorithm based on the actor–critic approach was proposed for the collaborative and centralized training of multiple intelligent agents (CTMA-AC), enabling multi-objective optimization decision-making for the protection of offloaded private user data. Simulation experiments demonstrate that the proposed multi-agent collaborative privacy data offloading protection strategy can effectively safeguard private user data while ensuring high service quality. Full article

Mixed-signal simulation is indispensable for verifying modern integrated circuits that tightly couple analog and digital subsystems, yet the field lacks a unified framework for systematically comparing its diverse methodologies. This paper addresses that gap by proposing a novel three-axis taxonomy that classifies simulation methods along abstraction level, solver methodology, and analysis type, together with a comparative evaluation framework based on five quantitative metrics: accuracy, throughput, capacity, convergence reliability, and scalability. Applying this framework, we systematically compare thirteen classical method categories—spanning SPICE, FastSPICE, RF/periodic steady-state, behavioral modeling, co-simulation, and model order reduction—and eight AI/ML approaches including Gaussian process surrogates, graph neural networks, physics-informed neural networks, Bayesian optimization, and reinforcement learning. Our analysis reveals a clear maturity stratification: classical methods remain the only signoff-accurate approaches, Bayesian optimization represents the most industrially validated AI contribution with integration across all three major EDA platforms, while Neural ODE solvers and LLM-based design tools remain at the research stage. We identify a persistent academic-to-industry gap driven by foundry model complexity, limited benchmark diversity, and topology-specific overfitting. The proposed taxonomy and comparative framework provide practitioners with structured guidance for simulation method selection and highlight specific research directions needed to bridge the gap between AI promise and industrial deployment. Full article

Portfolio optimization is one of the most difficult sequential decision problems, as uncertainty and the non-stationary nature of financial markets hinder the development of robust strategies. Reinforcement learning is an attractive framework for addressing this problem, as it allows agents to learn market-adaptive strategies through data-driven interactions. However, existing risk-neutral reinforcement learning solutions for portfolio management are oblivious to downside risk and are mainly concerned with maximizing returns. To address this limitation, this paper proposes a novel risk-sensitive reinforcement learning framework for risk-aware portfolio optimization based on a conditional value-at-risk-based learning objective that explicitly controls extreme loss events. It formulates the portfolio optimization problem as a Markov decision process and solves it using a linearized actor–critic architecture. It also develops theoretical results to analyze important aspects of the learning process, specifically proving that the convexity of the conditional value-at-risk-based formulation and convergence of learning hold under standard assumptions. The proposed algorithm is applied in a realistic investment setting using NIFTY 50 market data. Quantitative results from a rolling window backtesting methodology show that the proposed model achieves the best risk-adjusted portfolio performance, i.e., a Sharpe ratio (0.610), while significantly reducing tail risk, as measured by the conditional value-at-risk (−0.121) and maximum drawdown (−0.198), compared to classical strategies and risk-neutral reinforcement learning solutions. Overall, the results demonstrate that integrating coherent risk measures into reinforcement learning provides an effective approach for developing robust and risk-aware portfolio optimization strategies in dynamic financial environments. Full article

Adaptive traffic signal control (ATSC) remains a critical challenge for urban mobility. In this direction, deep reinforcement learning (DRL) has been widely investigated for ATSC, showing promising improvements in simulated environments. However, a noticeable gap remains between simulation-based results and practical implementations, due to reward formulations that do not address phase instability. Stochastic variations may trigger premature phase changes (“flickers”), affecting signal behavior and potentially limiting deployment in real scenarios. Although several works have examined delay, queues, and decentralized coordination, stability-focused variables remain comparatively less explored, particularly in single yet complex intersections. This study proposes a decentralized DRL model for ATSC with noise injection (ATSC-DRLNI) applied to a single intersection, introducing a stability-oriented reward function that integrates flickers, queue length, and advantage actor-critic (A2C) learning feedback. The model is evaluated in the Simulation of Urban MObility (SUMO) platform and compared against seven baseline methods, using real traffic data from a Mexican city for calibration and validation. Results suggest that penalizing flickers may contribute to more stable phase transitions, while reductions of up to 40% in queue length were observed in heavy-traffic scenarios. These findings indicate that incorporating stability-related variables into reward functions may help in implementing DRL-based ATSC studies. Full article

Freeze–thaw damage significantly reduces the performance and durability of airport pavements in cold regions. Traditional assessment methods, such as the F300 freeze–thaw test, are time-consuming and hinder rapid optimisation of mix design. In addition, previous studies have mostly relied on long-term laboratory testing and have evaluated phase-change concrete (PCC) independently, without considering synergistic effects. These approaches lack fast, synergy-aware predictive capability and interpretable tools for mix proportion design, resulting in a gap between laboratory research and practical engineering applications. To address this issue, this study proposes an intelligent and explainable framework for predicting freeze–thaw damage and guiding mix design of steel fibre-reinforced phase-change concrete (SF–PCC). A boundary-controlled experimental programme was first conducted, varying steel fibre (SF) content from 0 to 1.2% and phase-change material (PCM) content from 0 to 12% under fixed mixture conditions. The freeze–thaw test results were recorded sequentially and used to construct a supervised learning dataset. Then, an XGBoost model was developed to predict two key durability indicators: relative dynamic modulus of elasticity (RDEM) and mass loss. SHAP (SHapley Additive exPlanations) analysis was further applied to quantify feature importance and interaction effects. The model achieved high predictive accuracy (R² = 0.9938 for mass loss and R² = 0.9935 for RDEM) under controlled experimental conditions. After 300 freeze–thaw cycles, the reference mix exhibited an RDEM of 61.2%, while optimised configurations showed improved performance. The economical design (9% PCM + 0.9% SF) achieved an RDEM of 66.8%, and the high-performance design (12% PCM + 1.2% SF) reached 72.6%. These results demonstrate that the proposed framework can effectively enhance durability and support rapid preliminary decision-making. The framework significantly accelerates freeze–thaw performance evaluation by enabling near-instant prediction and serves as an efficient supplementary tool for mix design optimisation alongside conventional laboratory testing. It also provides interpretable, data-driven insights for the design of freeze–thaw-resistant airport pavement concrete in cold regions. Full article

Point cloud semantic segmentation is a core component in indoor scene understanding and autonomous driving. Under weak point-level supervision, only a small subset of points is annotated, making effective use of unlabeled points critical yet non-trivial. Many existing approaches rely on prediction confidence to filter pseudo labels or enforce consistency, which can bias training toward easy points and amplify early mistakes. Consequently, confidently wrong predictions may be reinforced, while uncertain points around class boundaries or in geometrically complex regions are less utilized, limiting further gains. An evidential uncertainty decomposition framework is introduced for weakly supervised point cloud semantic segmentation. Network outputs are interpreted as evidential distributions, and uncertainty is decomposed to separate lack-of-knowledge uncertainty from boundary-related ambiguity, providing a more informative reliability signal for unlabeled points. Based on this signal, different constraints are applied to different subsets: reliable points are trained with pseudo labels together with prototype-based regularization to encourage intra-class compactness; boundary-ambiguous points are guided by evidential consistency to improve boundary learning; and points with high epistemic uncertainty are excluded from pseudo-label-based supervision to mitigate error reinforcement. In addition, an uncertainty calibration term on sparsely labeled points helps stabilize training. Experiments on S3DIS, ScanNet-V2, and SemanticKITTI yield 67.7%, 59.7%, and 53.3% mIoU, respectively, with only 0.1% labeled points, comparing favorably with prior weakly supervised point cloud segmentation methods. Full article