Search Results (23)

The multi-valued dependency matrix (MVD matrix) is an important testability modeling approach, which can deliver more comprehensive testability information than the traditional dependency matrix (D-matrix). However, existing testability strategy optimization algorithms perform poorly in handling the MVD matrix, and the high-dimensional MVD matrix further aggravates these limitations as system complexity increases. To address these problems, a novel testability strategy optimization method under multi-valued dependency conditions based on deep reinforcement learning (DRL) is proposed. Firstly, the sets of elements and two reward functions to minimize test sequence length and test cost are established from the MVD matrix. Subsequently, the algorithm for selecting test points based on Deep Q-Network (DQN) is proposed. The DQN parameters are updated to fit the Q-value of test points. Thirdly, Double DQN (DDQN) and the prioritized experience replay (PER) mechanism are introduced to address the overestimation problem and sample redundancy problem, respectively, in high-dimensional matrix environments. The experimental results show that the testability strategy generated by this method can isolate all faults with fewer steps or at a lower cost. In a high-dimensional matrix environment, it can reduce test costs compared with the other heuristic algorithms while maintaining a good level of stability. Full article

Temples are tourist attractions that represent the history and culture of Thailand, especially in Chiang Mai province—a city with a rich history that has become a prominent destination attracting visitors from around the world. Many temples remain undiscovered yet are ready for tourists to visit; however, due to unfamiliarity, tourists tend to visit only the well-known temples, as other visitors do, missing great opportunities to engage with new cultural heritage tourism experiences. To address this issue, we propose a Hybrid Three-Stage Route Planning Recommendation (HTS-RPR), a novel method for tourist route planning that delivers recommended routes based on tourists’ preferred constraints. This model contains three-stage route recommendations providing an optimal single-day route with mandatory and recommended points of interest (POIs) through a metaheuristic integrating Mixed Integer Programming (MIP), heuristic-based POI recommendation filtering, and Genetic Algorithm route optimization with Bayesian reward and peak-time awareness, ensuring that users can effectively travel cultural routes with high popularity and satisfaction while avoiding attractions during periods of high traffic. To validate the efficacy of the proposed model, experiments with three baseline methods were conducted. The results demonstrate that HTS-RPR achieves the best fitness score in 55 out of 60 scenarios and the best reward in 54 out of 60 scenarios, with a median fitness score 28.34% and 103.67% higher than the Genetic Algorithm and Multi-Start Simulated Annealing baselines, respectively, and a median total reward exceeding all three baselines by up to 40.74%. Although HTS-RPR’s median execution time is approximately 2.6 times that of the Genetic Algorithm, it remains 84.5% faster than the Multi-Start Simulated Annealing baseline, offering a favorable trade-off between solution quality and computational cost. Moreover, the framework’s pluggable reward function enables destination managers to configure recommendation priorities, including the promotion of undiscovered tourist attractions, while the peak-time-aware optimization mitigates congestion at specific POIs. Full article

The purpose of this study is to examine whether analyst-level credit can be assigned quantitatively in a lightweight human-feedback decision-support pipeline. In intelligence and national security workflows, analysts often provide edits, comments, and evaluative feedback during the production of analytic products, yet these intermediate contributions are usually discarded, leaving no auditable record of how individual feedback shaped the final output. To address this problem, this study proposes a proof-of-concept Analyst-of-Record framework that combines synthetic analyst feedback, a linear ridge reward model, first-order influence functions, and additive Shapley aggregation to estimate both feedback-item and analyst-level contribution scores. The research design uses the Fact Extraction and VERification (FEVER) fact-verification dataset under controlled experimental settings. The pipeline retrieves evidence with Best Matching 25 (BM25), generates a grounded template-based response, derives three synthetic analyst feedback channels from FEVER annotations, trains a reward model on simple claim–answer and analyst-identity features, and aggregates per-feedback influence scores into an Analyst Contribution Index (ACI). The main experiments are conducted on a 500-claim subset across five random seeds, with additional ablation and bootstrap analyses used to assess sensitivity and stability. The findings show that the reward model achieves a mean validation $R^2$ of $0.801\pm 0.037$, indicating that the synthetic feedback signals are learnable under the selected featureization. The analyst-level contribution scores remain stable across random seeds, with approximately half of the total influence magnitude attributed to the explanation-quality channel and the remainder split across the other two channels. Ablation results further show that removing the explanation-quality channel collapses validation fit, while bootstrap resampling demonstrates tight concentration of absolute ACI magnitudes. Theoretically, this study extends attribution research beyond document-only grounding by showing how analyst feedback itself can be modeled as an object of contribution analysis. It also demonstrates that influence functions and Shapley-style aggregation can be adapted into a tractable framework for estimating interpretable analyst-level credit in a reproducible experimental setting. Practically, the proposed framework offers an initial foundation for more traceable and accountable decision-support workflows in which intermediate analyst contributions can be preserved rather than lost. The results also provide a feasible implementation path for future systems that incorporate stronger generators, richer evidence representations, and real analyst annotations. Full article

Although the elitist genetic algorithm (EGA) is an approach for the optimal design of pixelated metasurfaces, it is necessary to convert a two-dimensional (2D) metasurface to a one-dimensional array. This ignores the effects of the mutation on neighboring data in 2D metasurfaces, and hinders the rapid convergence of the algorithms. Therefore, we propose the 2D mutation-based EGA (2DM-EGA) to optimally design the linear-to-circular (LTC) polarization conversion metasurface (PCM). Compared with EGA, 2DM-EGA can significantly improve the convergence rate. Furthermore, combined with the proposed intuitive reward-based fitness function and circular polarization discrimination pertaining to an ellipticity angle β, 2DM-EGA, programmed in Python (2023 version), is used to accomplish optimal targets. Finally, the simulated operating band of the optimized metasurface varies from 8.16 GHz to 11.5 GHz with a reduced ellipticity angle β/π ≥ 0.15 and a relative bandwidth of 33.5%, which suggests that the optimized metasurface realizes the broadband LTC polarization conversion. The measured results are in excellent accord with the simulations validating 2DM-EGA for the optimal design of transmission-type wideband LTC PCMs. Additionally, the physical mechanism of the design is expounded. Full article

Urban logistics face complexity due to traffic congestion, fleet heterogeneity, warehouse constraints, and driver workload balancing, especially in the Heterogeneous Multi-Trip Vehicle Routing Problem with Time Windows and Time-Varying Networks (HMTVRPTW-TVN). We develop a mixed-integer linear programming (MILP) model with dual-peak time discretization and exact linearization for heterogeneous fleet coordination. Given the NP-hard nature, we propose a Hyper-Heuristic based on Cumulative Reward Q-Learning (HHCRQL), integrating reinforcement learning with heuristic operators in a Markov Decision Process (MDP). The algorithm dynamically selects operators using a four-dimensional state space and a cumulative reward function combining timestep and fitness. Experiments show that, for small instances, HHCRQL achieves solutions within 3% of Gurobi’s optimum when customer nodes exceed 15, outperforming Large Neighborhood Search (LNS) and LNS with Simulated Annealing (LNSSA) with stable, shorter runtime. For large-scale instances, HHCRQL reduces gaps by up to 9.17% versus Iterated Local Search (ILS), 6.74% versus LNS, and 5.95% versus LNSSA, while maintaining relatively stable runtime. Real-world validation using Shanghai logistics data reduces waiting times by 35.36% and total transportation times by 24.68%, confirming HHCRQL’s effectiveness, robustness, and scalability. Full article

In complex dynamic environments, robotic arms face multiple challenges such as real-time environmental changes, high-dimensional state spaces, and strong uncertainties. Trajectory planning tasks based on deep reinforcement learning (DRL) suffer from difficulties in acquiring human expert strategies, low experience utilization (leading to slow convergence), and unreasonable reward function design. To address these issues, this paper designs a neural network-based expert-guided triple experience replay mechanism (NETM) and proposes an improved reward function adapted to dynamic environments. This replay mechanism integrates imitation learning’s fast data fitting with DRL’s self-optimization to expand limited expert demonstrations and algorithm-generated successes into optimized expert experiences. Experimental results show the expanded expert experience accelerates convergence: in dynamic scenarios, NETM boosts accuracy by over 30% and safe rate by 2.28% compared to baseline algorithms. Full article

Complex scenario analysis and evaluation simulations often involve multiple sets of simulation applications with different combinations of parameters, thus resulting in high computing power consumption, which is one of the factors that limits the efficiency of multi-sample parallel simulations. Cloud computing provides considerable amounts of cheap and convenient computing resources, thus providing efficient support for multi-sample simulation tasks. However, traditional simulation scheduling methods do not consider the collaborative parallel scheduling of multiple samples and multiple entities under multi-objective constraints. Deep reinforcement learning methods can continuously learn and adjust their strategies through interactions with the environment, demonstrating strong adaptability in response to dynamically changing task requirements. Therefore, herein, a parallel simulation multi-sample task scheduling method based on deep reinforcement learning in a cloud computing environment is proposed. The method collects cluster load information and simulation application information as state inputs in the cloud environment, designs a multi-objective reward function to balance the cost and execution efficiency, and uses deep Q-networks (DQNs) to train agents for intelligent scheduling of multi-sample parallel simulation tasks. In a real cloud environment, the proposed method demonstrates runtime reductions of 4–11% and execution cost savings of 11–22% compared to the Round-Robin algorithm, Best Fit algorithm, and genetic algorithm. Full article

Ship trajectory tracking controllers based on deep reinforcement learning (DRL) are widely applied in various fields such as autonomous driving and robotics due to their strong adaptive learning capabilities and optimization decision-making ability. However, ship trajectory control faces challenges such as long training cycles and poor convergence performance. These issues are primarily caused by the unreasonable design of algorithm models and reward functions, which limit the performance optimization and energy efficiency improvements in real-world navigation. In this paper, we propose a ship trajectory tracking control algorithm based on deep reinforcement learning. The proposed algorithm introduces maximum entropy theory and experience replay techniques. Additionally, it enhances the reward function module by adding reward terms and fitting weight designs. A three-dimensional simulation environment is constructed to validate the proposed method. The results demonstrate that the controller designed in this study outperforms traditional DRL controllers in terms of fast convergence, convergence stability, and final reward values. The controller meets the requirements for tracking conventional trajectories and shows stable and efficient performance in both wide-area water search experiments and river channel traversal experiments. These experimental results provide valuable insights for future research directions. Full article

Planning and decision making are closely interconnected processes that often occur in tandem, influence and informing each other. Planning usually precedes decision making in the chronological sequence, and it can be viewed as a strategy to make decisions. A comprehensive planning or decision strategy can facilitate effective decisions. Thus, understanding and learning human decision-making strategies has drawn intensive attention from the AI community. For example, applying planning algorithms into reinforcement leaning (RL) can simulate the consequence of different actions and select optimal decisions based on learned models, while inverse reinforcement learning (IRL) learns a reward function and policy from expert demonstration and applies them into new scenarios. Most of these methods work based on learning human decision strategies by using modeling of a Markovian decision-making process (MDP). In this paper, we argue that the property of MDP is not fit for human decision-making processes in the real-world and it is insufficient to capture human decision strategies. To tackle this challenge, we propose a new approach to identify the characteristics of human decision-making processes as a decision map, where the decision strategy is defined by the probability distribution of human decisions that are adaptive to the dynamic changes in the environment. The proposed approach was inspired by imitation learning (IL) but with fundamental differences: (a) Instead of aiming to learn an optimal policy based on expert’s demonstrations, we aimed to estimate the distribution of decisions of any group of people. (b) Instead of modeling the environment by an MDP, we used an ambiguity probability model to consider the uncertainty of each decision. (c) The participant trajectory was obtained by categorizing each decision of a participant to a certain cluster based on the commonness in the distribution of decisions. The result shows a feasible way to capture human long-term decision dependency, which provides a complement to the existing machine learning methods for understanding and learning human decision strategies. Full article

This paper studies the tactical decision-making model of short track speed skating based on deep reinforcement learning, so as to improve the competitive performance of corresponding short track speed skaters. Short track speed skating, a traditional discipline in the Winter Olympics since its establishment in 1988, has consistently garnered attention. As artificial intelligence continues to advance, the utilization of deep learning methods to enhance athletes’ tactical decision-making capabilities has become increasingly prevalent. Traditional tactical decision techniques often rely on the experience and knowledge of coaches and video analysis methods that require a lot of time and effort. Consequently, this study proposes a scientific simulation environment for short track speed skating, that accurately simulates the physical attributes of the venue, the physiological fitness of the athletes, and the rules of the competition. The Double Deep Q-Network (DDQN) model is enhanced and utilized, with improvements to the reward function and the distinct description of four tactics. This enables agents to learn optimal tactical decisions in various competitive states with a simulation environment. Experimental results demonstrate that this approach effectively enhances the competition performance and physiological fitness allocation of short track speed skaters. Full article

Collision avoidance planning has always been a hot and important issue in the field of unmanned aircraft research. In this article, we describe an online collision avoidance planning algorithm for autonomous underwater vehicle (AUV) autonomous navigation, which relies on its own active sonar sensor to detect obstacles. The improved particle swarm optimization (I-PSO) algorithm is used to complete the path planning of the AUV under the known environment, and we use it as a benchmark to improve the fitness function and inertia weight of the algorithm. Traditional path-planning algorithms rely on accurate environment maps, where re-adapting the generated path can be highly demanding in terms of computational cost. We propose a deep reinforcement learning (DRL) algorithm based on collision avoidance tasks. The algorithm discussed in this paper takes into account the relative position of the target point and the rate of heading change from the previous timestep. Its reward function considers the target point, running time and turning angle at the same time. Compared with the LSTM structure, the Gated Recurrent Unit (GRU) network has fewer parameters, which helps to save training time. A series of simulation results show that the proposed deep deterministic policy gradient (DDPG) algorithm can obtain excellent results in simple and complex environments. Full article

This paper presents a multi-agent reinforcement learning (MARL) algorithm to address the scheduling and routing problems of multiple automated guided vehicles (AGVs), with the goal of minimizing overall energy consumption. The proposed algorithm is developed based on the multi-agent deep deterministic policy gradient (MADDPG) algorithm, with modifications made to the action and state space to fit the setting of AGV activities. While previous studies overlooked the energy efficiency of AGVs, this paper develops a well-designed reward function that helps to optimize the overall energy consumption required to fulfill all tasks. Moreover, we incorporate the e-greedy exploration strategy into the proposed algorithm to balance exploration and exploitation during training, which helps it converge faster and achieve better performance. The proposed MARL algorithm is equipped with carefully selected parameters that aid in avoiding obstacles, speeding up path planning, and achieving minimal energy consumption. To demonstrate the effectiveness of the proposed algorithm, three types of numerical experiments including the ϵ-greedy MADDPG, MADDPG, and Q-Learning methods were conducted. The results show that the proposed algorithm can effectively solve the multi-AGV task assignment and path planning problems, and the energy consumption results show that the planned routes can effectively improve energy efficiency. Full article

Currently, most multi-target data association methods require the assumption that the target motion model is known, but this assumption is clearly not valid in a real environment. In the case of an unknown system model, the influence of environmental clutter and sensor detection errors on the association results should be considered, as well as the occurrence of strong target maneuvers and the sudden appearance of new targets during the association process. To address these problems, this paper designs a target tracking and data association algorithm based on reinforcement learning. First, this algorithm combines the dynamic exploration capability of reinforcement learning and the long-time memory function of LSTM network to design a policy network that predicts the probability of associating a point with its various possible source targets. Then, the Bayesian network and the multi-order least squares curve fitting method are combined to predict the location of target, and the results are fed into the Bayesian recursive function to obtain the reward. Simultaneously, some corresponding mechanisms are proposed for possible problems that interfere with the association process. Finally, the simulation experimental results show that this algorithm associates the results with higher accuracy compared to other algorithms when faced with the above problem. Full article

Individuals spend time on online video-sharing platforms searching for videos. Video summarization helps search through many videos efficiently and quickly. In this paper, we propose an unsupervised video summarization method based on deep reinforcement learning with an interpolation method. To train the video summarization network efficiently, we used the graph-level features and designed a reinforcement learning-based video summarization framework with a temporal consistency reward function and other reward functions. Our temporal consistency reward function helped to select keyframes uniformly. We present a lightweight video summarization network with transformer and CNN networks to capture the global and local contexts to efficiently predict the keyframe-level importance score of the video in a short length. The output importance score of the network was interpolated to fit the video length. Using the predicted importance score, we calculated the reward based on the reward functions, which helped select interesting keyframes efficiently and uniformly. We evaluated the proposed method on two datasets, SumMe and TVSum. The experimental results illustrate that the proposed method showed a state-of-the-art performance compared to the latest unsupervised video summarization methods, which we demonstrate and analyze experimentally. Full article

For achieving seamless global coverage and real-time communications while providing intelligent applications with increased quality of service (QoS), AI-enabled space–air–ground integrated networks (SAGINs) have attracted widespread attention from all walks of life. However, high-intensity interactions pose fundamental challenges for resource orchestration and security issues. Meanwhile, virtual network embedding (VNE) is applied to the function decoupling of various physical networks due to its flexibility. Inspired by the above, for SAGINs with non-Euclidean structures, we propose a graph-convolution virtual network embedding algorithm. Specifically, based on the excellent decision-making properties of deep reinforcement learning (DRL), we design an orchestration network combined with graph convolution to calculate the embedding probability of nodes. It fuses the information of the neighborhood structure, fully fits the original characteristics of the physical network, and utilizes the specified reward mechanism to guide positive learning. Moreover, by imposing security-level constraints on physical nodes, it restricts resource access. All-around and rigorous experiments are carried out in a simulation environment. Finally, results on long-term average revenue, VNR acceptance ratio, and long-term revenue–cost ratio show that the proposed algorithm outperforms advanced baselines. Full article