Search Results (1,509)

The opto-mechanical automated manufacturing process, characterized by stringent process constraints, dynamic disturbances, and conflicting optimization objectives, presents significant control challenges for traditional scheduling and control approaches. We formulate the scheduling problem within a closed-loop control paradigm and propose a novel bi-level intelligent control framework integrating Deep Reinforcement Learning (DRL) and Bayesian Optimization (BO). The core of our approach is a bi-level intelligent control framework. An inner DRL agent acts as an adaptive controller, generating control actions (scheduling decisions) by perceiving the system state and learning a near-optimal policy through a carefully designed reward function, while an outer BO loop automatically tunes the DRL’s hyperparameters and reward weights for superior performance. This synergistic BO-DRL mechanism facilitates intelligent and adaptive decision-making. The proposed method is extensively evaluated against standard meta-heuristics, including Genetic Algorithm (GA) and Particle Swarm Optimization (PSO), on a complex 20-jobs × 20-machines flexible job shop scheduling benchmark specific to opto-mechanical automated manufacturing. The experimental results demonstrate that our BO-DRL algorithm significantly outperforms these benchmarks, achieving reductions in makespan of 13.37% and 25.51% compared to GA and PSO, respectively, alongside higher machine utilization and better on-time delivery. Furthermore, the algorithm exhibits enhanced convergence speed, superior robustness under dynamic disruptions (e.g., machine failures, urgent orders), and excellent scalability to larger problem instances. This study confirms that integrating DRL’s perceptual decision-making capability with BO’s efficient parameter optimization yields a powerful and effective solution for intelligent scheduling in high-precision manufacturing environments. Full article

Microservices as an emerging architectural approach have been widely applied in the development of online applications. However, in large-scale service systems, frequent data communications, complex invocation dependencies, and strict latency requirements pose significant challenges to efficient microservice orchestration. In addition, microservices need to frequently access the database to achieve data persistence, creating a mutual dependency between the two, and this symmetry further increases the complexity of service orchestration and coordinated deployment. In this context, the strong coupling of service deployment, database layout, and request routing makes effective local optimization difficult. However, existing research often overlooks the impact of databases, fails to achieve joint optimization among databases, microservice deployments, and routing, or lacks fine-grained orchestration strategies for multi-instance models. To address the above limitations, this paper proposes a joint optimization framework based on the Database-as-a-Service (DaaS) paradigm. It performs fine-grained multi-instance queue modeling based on queuing theory to account for delays in data interaction, request queuing, and processing. Furthermore this paper proposes a proximal policy optimization algorithm based on multi-stage joint decision-making to address the orchestration problem of microservices and database instances. In this algorithm, the action space is symmetrical between microservices and database deployment, enabling the agent to leverage this characteristic and improve representation learning efficiency through shared feature extraction layers. The algorithm incorporates a two-layer agent policy stability control to accelerate convergence and a three-level experience replay mechanism to achieve efficient training on high-dimensional decision spaces. Experimental results demonstrate that the proposed algorithm effectively reduces service request latency under diverse workloads and network conditions, while maintaining global resource load balancing. Full article

Lexicographic preference trees (LP-Trees) provide a compact and expressive representation for modeling complex decision-making scenarios, yet measuring similarity between complete or partial structures remains a challenge. This study introduces PLPSim, a novel metric for quantifying alignment between partial lexicographic preference trees (PLP-Trees) and develops three coalition formation algorithms—HRECS1, HRECS2, and HRECS3—that leverage PLPSim to group agents with similar preferences. We further propose ContractLex and PriceLex protocols (comprising CLF, CFB, CFW, CFA, CFP) for coalition-based contract and pricing strategies, along with a new evaluation metric, F@Lex, which is designed to assess satisfaction under lexicographic preferences. To illustrate the framework, we generate a synthetic dataset (PLPGen) contextualized in a hybrid renewable energy market, where consumers’ PLP-Trees are aggregated and matched with suppliers’ tariff contracts. Experiments across 162 market scenarios, evaluated using Normalized Discounted Cumulative Gain (nDCG), Davies–Bouldin dispersion, and F@Lex, demonstrate that PLPSim-based coalitions outperform baseline approaches. The combination HRECS3 + CFP yields the highest consumer satisfaction, while HRECS3 + CFB achieves balanced satisfaction for both consumers and suppliers. While electricity tariffs and renewable energy contracts—static and dynamic—serve as the motivating example, the proposed framework generalizes to diverse multi-agent systems, offering a foundation for preference-driven coalition formation, adaptive policy design, and sustainable market optimization. Full article

Automated driving technologies have the capability to significantly increase road safety by decreasing accidents and increasing travel efficiency. This research presents a decision-making strategy for automated vehicles that models both lane changing and double lane changing maneuvers and is supported by a Deep Reinforcement Learning (DRL) algorithm. To capture realistic driving challenges, a highway driving scenario was designed using the professional multi-body simulation tool IPG Carmaker software, version 11 with realistic weather simulations to include aspects of rainy weather by incorporating vehicles with explicitly reduced tire–road friction while the ego vehicle is attempting to safely and perform efficient maneuvers in highway and merged merges. The hierarchical control system both creates an operational structure for planning and decision-making processes in highway maneuvers and articulates between higher-level driving decisions and lower-level autonomous motion control processes. As a result, a Duel Deep Deterministic Policy Gradient (Duel-DDPG) agent was created as the DRL approach to achieving decision-making in adverse driving conditions, which was built in MATLAB version 2021, designed, and tested. The study thoroughly explains both the Duel-DDPG and standard Deep Deterministic Policy Gradient (DDPG) algorithms, and we provide a direct performance comparative analysis. The discussion continues with simulation experiments of traffic complexity with uncertainty relating to weather conditions, which demonstrate the effectiveness of the Duel-DDPG algorithm. Full article

Plastic pollution in aquatic environments is a major ecological problem requiring scalable autonomous solutions for cleanup. This study addresses the coordination of multiple Autonomous Surface Vehicles by formulating the problem as a Partially Observable Markov Game and decoupling the mission into two tasks: exploration to maximize coverage and cleaning to collect trash. These tasks share navigation requirements but present conflicting goals, motivating a multi-objective learning approach. The proposed multi-agent deep reinforcement learning framework involves the utilisation of the same Multitask Deep Q-network shared by all the agents, with a convolutional backbone and two heads, one dedicated to exploration and the other to cleaning. Parameter sharing and egocentric state design leverages agent homogeneity and enable experience aggregation across tasks. An adaptive mechanism governs task switching, combining task-specific rewards with a weighted aggregation and selecting tasks via a reward-greedy strategy. This enables the construction of Pareto fronts capturing non-dominated solutions. The framework demonstrates improvements over fixed-phase approaches, improving hypervolume and uniformity metrics by 14% and 300%, respectively. It also adapts to diverse initial trash distributions, providing decision-makers with a portfolio of effective and adaptive strategies for autonomous plastic cleanup. Full article

Unmanned aerial vehicle (UAV) swarms offer an efficient solution for data collection from widely distributed ground users (GUs). However, incomplete environment information and frequent changes make it challenging for standard centralized planning or pure reinforcement learning approaches to simultaneously maintain global solution quality and local flexibility. We propose a hierarchical data collection framework for heterogeneous UAV-assisted wireless sensor networks (WSNs). A small set of high-capability UAVs (H-UAVs), equipped with substantial computational and communication resources, coordinate regional coverage, trajectory planning, and uplink transmission control for numerous resource-constrained low-capability UAVs (L-UAVs) across power-Voronoi-partitioned areas using multi-agent deep reinforcement learning (MADRL). Specifically, we employ Multi-Agent Deep Deterministic Policy Gradient (MADDPG) to enhance H-UAVs’ decision-making capabilities and enable coordinated actions. The partitions are dynamically updated based on GUs’ data generation rates and L-UAV density to balance workload and adapt to environmental dynamics. Concurrently, a large number of L-UAVs with limited onboard resources perform self-organized data collection from GUs and execute opportunistic relaying to a remote access point (RAP) via H-UAVs. Within each Voronoi cell, L-UAV motion follows a weighted Vicsek model that incorporates GUs’ age of information (AoI), link quality, and congestion avoidance. This spatial decomposition combined with decentralized weak-swarm control enables scalability to large-scale L-UAV deployments. Experiments demonstrate that the proposed strong and weak agent MADDPG (SW-MADDPG) scheme reduces AoI by 30% and 21% compared to No-Voronoi and Heuristic-HUAV baselines, respectively. Full article

To address the autonomous traversability challenge of an Unmanned Articulated Road Roller (UARR) operating on harsh terrains where low-adhesion slipperiness and geometric irregularities are coupled, and traction capacity is severely limited, this paper proposes a Terrain-Adaptive Maximum-Entropy Policy Optimization (TAMPO). A unified multi-physics simulation platform is constructed, integrating a high-fidelity vehicle dynamics model with a parameterized terrain environment. Considering the prevalence of geometric irregularities in construction sites, a parameterized mud-pit model is established—generalized from a representative case—as a canonical physical model and simulation carrier for this class of traversability problems. Based on this model, a family of training and test scenarios is generated to span a broad range of terrain shapes and adhesion conditions. On this foundation, the TAMPO algorithm is introduced to enhance vehicle traversability on complex terrains. The method comprises the following: (i) a Terrain Interaction-Critical Reward (TICR), which combines dense rewards representing task progress with sparse rewards that encourage terrain exploration, guiding the agent to both climb efficiently and actively seek high-adhesion favorable terrain; and (ii) a context-aware adaptive entropy-regularization mechanism that fuses, in real time, three feedback signals—terrain physical difficulty, task-execution efficacy, and model epistemic uncertainty—to dynamically regulate policy entropy and realize an intelligent, state-dependent exploration–exploitation trade-off in unstructured environments. The performance and generalization ability of TAMPO are evaluated on training, interpolation, and extrapolation sets, using PPO, SAC, and DDPG as baselines. On 90 highly challenging extrapolation scenarios, TAMPO achieves an average success rate (S.R.) of 60.00% and an Average Escape Time (A.E.T.) of 17.56 s, corresponding to improvements of up to 22.22% in S.R. and reductions of up to 5.73 s in A.E.T. over the baseline algorithms, demonstrating superior decision-making performance and robust generalization on coupled slippery and irregular terrains. Full article

This paper addresses the critical challenge of multi-objective optimization in residential Home Energy Management Systems (HEMS) by proposing a novel framework based on an Improved Multi-Agent Proximal Policy Optimization (MAPPO) algorithm. The study specifically targets the low convergence efficiency of Multi-Agent Deep Reinforcement Learning (MADRL) for coupled Battery Energy Storage System (BESS) and Air Source Heat Pump (ASHP) operation. The framework synergistically integrates an action constraint projection mechanism with an economic-performance-driven dynamic learning rate modulation strategy, thereby significantly enhancing learning stability. Simulation results demonstrate that the algorithm improves training convergence speed by 35–45% compared to standard MAPPO. Economically, it delivers a cumulative cost reduction of 15.77% against rule-based baselines, outperforming both Independent Proximal Policy Optimization (IPPO) and standard MAPPO benchmarks. Furthermore, the method maximizes renewable energy utilization, achieving nearly 100% photovoltaic self-consumption under favorable conditions while ensuring robustness in extreme scenarios. Temporal analysis reveals the agents’ capacity for anticipatory decision-making, effectively learning correlations among generation, pricing, and demand to achieve seamless seasonal adaptability. These findings validate the superior performance of the proposed centralized training architecture, providing a robust solution for complex residential energy management. Full article

This study examines decision-making in intelligent virtual agents (IVAs) and formalizes the distinction between tactical decisions (individual actions) and strategic decisions (composed of sequences of tactical actions) using a mathematical model based on set theory and the Bellman equation. Although the equation itself is not modified, the analysis reveals that the discount factor (γ) influences the type of decision: low values favor tactical decisions, while high values favor strategic ones. The model was implemented and validated in a proof-of-concept simulated environment, namely the Snake Coin Change Problem (SCCP), using a Deep Q-Network (DQN) architecture, showing significant differences between agents with different decision profiles. These findings suggest that adjusting γ can serve as a useful mechanism to regulate both tactical and strategic decision-making processes in IVAs, thus offering a conceptual basis that could facilitate the design of more intelligent and adaptive agents in domains such as video games, and potentially in robotics and artificial intelligence as future research directions. Full article

Empirical debates about a “crisis of trust” highlight long-lived pockets of high trust and deep distrust in institutions, as well as abrupt, shock-induced shifts between the two. We propose a probabilistic model in which such phenomena emerge endogenously from social learning on hierarchical networks. Starting from a discrete model on a directed acyclic graph, where each agent makes a binary adoption decision about a single assertion, we derive an effective influence kernel that maps individual priors to stationary adoption probabilities. A continuum limit along hierarchical depth yields a degenerate, non-conservative logistic–diffusion equation for the adoption probability $u(x,t)$, in which diffusion is modulated by $(1-u)$ and increases the integral of u rather than preserving it. To account for micro-level uncertainty, we perturb these dynamics by multiplicative Stratonovich noise with amplitude proportional to $u(1-u)$, strongest in internally polarised layers and vanishing at consensus. At the level of a single depth layer, Stratonovich–Itô conversion and Fokker–Planck analysis show that the noise induces an effective double-well potential with two robust stochastic phases, $u\approx 0$ and $u\approx 1$, corresponding to persistent distrust and trust. Coupled along depth, this local bistability and degenerate diffusion generate extended domains of trust and distrust separated by fronts, as well as rare, Kramers-type transitions between them. We also formulate the associated stochastic partial differential equation in Martin–Siggia–Rose–Janssen–De Dominicis form, providing a field-theoretic basis for future large-deviation and data-informed analyses of trust landscapes in hierarchical societies. Full article

Household adoption of residential photovoltaic systems in community microgrids is shaped by economic, infrastructural, and social factors. Previous studies have shown that agent-based modelling can help analyse adoption, but it often lacks clear mathematical foundations, systematic validation, and reproducibility. This study presents an open-source agent-based model with two advances: (1) a fuzzy-utility method for household decision-making and (2) combined modelling of financial incentives, grid reliability, infrastructure access, and peer effects as adoption drivers. The model explores adoption under diverse policy and technical scenarios, validates results against Bass diffusion and discrete choice models, and applies a Sobol-based sensitivity analysis to identify key parameters. Results clarify how incentives, barriers, and social influence shape adoption trajectories. By demonstrating cost-sharing dynamics and peer network effects and openly sharing model code and data, this study provides a transparent and reproducible benchmark for future community microgrid research. Full article

People with intellectual disabilities frequently use psychotropic and other medications, sometimes inappropriately. To promote shared decision-making, they require accessible information about their medication. This study combined data from two similar intervention studies, conducted in two different settings, to assess the appropriateness of medication use and the shared decision-making process among adults with mild intellectual disabilities who used psychotropic medication. The intervention consisted of a structured, multidisciplinary medication review, including the provision of accessible psychotropic medication leaflets, and a discussion of the pharmacotherapeutic treatment plan with the patient by either a pharmacist or physician, depending on the setting. Outcomes included medication use, pharmacotherapeutic problems, implementation of recommendations, and perceived shared decision-making, measured with the Shared Decision-Making Questionnaire Q9. The 15 included participants used an average of nearly seven medications, which were mainly neurotropic, gastrointestinal, cardiovascular, and respiratory agents. On average, two pharmacotherapeutic problems were identified; the most common were overtreatment, side effects, and administration difficulties. Recommendations often involved dose reduction or tapering, and about 75% were fully or partially implemented. Both participants and clinicians reported high satisfaction with shared decision-making. Multidisciplinary, structured medication reviews, incorporating accessible medication leaflets, may enhance appropriate medication use and shared decision-making, but more research is needed. Full article

Background: Early administration of antibiotics is commonly recommended for pneumonia in intensive care unit (ICU) patients. However, the clinical benefit of very early empirical treatment remains uncertain and may reflect differences in illness severity, baseline risk, or care pathways, particularly in non-septic or hemodynamically stable ICU populations. Methods: We performed a retrospective cohort study using the Medical Information Mart for Intensive Care IV (v2.2) database to evaluate the observational association between antibiotic timing and in-hospital mortality among adult ICU patients with pneumonia. Patients were categorized as receiving early (<3 h) or delayed (≥3 h) antibiotic therapy after ICU admission. A multivariable logistic regression model adjusted only for age and sex. Given the absence of detailed severity-of-illness measures, no causal inference was intended, and all analyses were considered hypothesis-generating. Additional analyses exploring antibiotic class, dosing frequency, and combination therapy were conducted in an exploratory manner, given substantial variation in sample sizes and a high risk of confounding by indication, misclassification, immortal-time, and survivorship bias. Results: Among 7569 ICU patients with pneumonia, 56.5% received antibiotics within three hours of ICU admission. Early antibiotic initiation was associated with higher in-hospital mortality than delayed therapy (26.1% vs. 21.5%; OR 1.30, 95% CI 1.16–1.44; p < 0.001). Because validated severity-of-illness measures were unavailable, residual confounding and confounding by indication are likely and may largely explain this association. A potential signal of increased mortality was observed in patients receiving ≥3 doses of levofloxacin (OR 4.39, 95% CI 1.13–17.02); however, this subgroup was small and the finding is highly susceptible to survivorship and indication bias. Mortality appeared lower in patients receiving two or three antibiotics compared with monotherapy, but marked group imbalances, lack of restriction or stratification, and clinical selection effects limit interpretability. Regimens involving ≥4 agents were rare and primarily associated with prolonged ICU length of stay rather than a clear mortality difference. Conclusions: In this large retrospective ICU cohort, very early antibiotic administration for pneumonia was observationally associated with higher in-hospital mortality. Causality cannot be inferred, and early treatment likely represents a marker of higher baseline risk or clinical urgency rather than a harmful exposure. These findings challenge the assumption that earlier antibiotic initiation is uniformly beneficial and underscore the importance of individualized, stewardship-aligned, and context-dependent decision-making regarding antimicrobial timing and intensity in critically ill patients. Full article

Autonomous vehicles (AVs) are expected to operate safely and efficiently in complex urban environments characterized by dynamic and uncertain elements such as pedestrians, cyclists and adverse weather. Although current neural network-based decision-making algorithms, fuzzy logic and reinforcement learning have shown promise, they often struggle to handle ambiguous situations, such as partially hidden road signs or unpredictable human behavior. This paper proposes a new hybrid decision-making framework combining multi-agent reinforcement learning (MARL) and explainable artificial intelligence (XAI) to improve robustness, adaptability and transparency. Each agent of the MARL architecture is specialized in a specific sub-task (e.g., obstacle avoidance, trajectory planning, intention prediction), enabling modular and cooperative learning. XAI techniques are integrated to provide interpretable rationales for decisions, facilitating human understanding and regulatory compliance. The proposed system will be validated using CARLA simulator, combined with reference data, to demonstrate improved performance in safety-critical and ambiguous driving scenarios. Full article

Effective knowledge management is critical for preserving institutional expertise and improving the efficiency of workforce training in state transportation agencies. Traditional approaches, such as static documentation, classroom-based instruction, and informal mentorship, often lead to fragmented knowledge transfer, inefficiencies, and the gradual loss of expertise as senior engineers retire. Moreover, given the enormous volume of technical manuals, guidelines, and research reports maintained by these agencies, it is increasingly challenging for engineers to locate relevant information quickly and accurately when solving field problems or preparing for training tasks. These limitations hinder timely decision-making and create steep learning curves for new personnel in maintenance and construction operations. To address these challenges, this paper proposes a Retrieval-Augmented Generation (RAG) framework with a multi-agent architecture to support knowledge management and decision-making. The system integrates structured document retrieval with real-time, context-aware response generation powered by a large language model (LLM). Unlike conventional single-pass RAG systems, the proposed framework employs multiple specialized agents for retrieval, answer generation, evaluation, and query refinement, which enables iterative improvement and quality control. In addition, the system incorporates an open-weight vision-language model to convert technical figures into semantic textual representations, which allows figure-based knowledge to be indexed and retrieved alongside text. Retrieved text and figure-based context are then provided to an open-weight large language model, which generates the final responses grounded in the retrieved evidence. Moreover, a case study was conducted using over 500 technical and research documents from multiple State Departments of Transportation (DOTs) to assess system performance. The multi-agent RAG system was tested with 100 domain-specific queries covering pavement maintenance and management topics. The results demonstrated Recall@3 of 94.4%. These results demonstrate the effectiveness of the system in supporting document-based response generation for DOT knowledge management tasks. Full article