We propose a Markov Decision Process Model that blends ideas from Psychological research and Economics to study decision-making in individuals with self-control problems. We have borrowed a dual-process of decision-making with self-awareness from Psy...
In a general Markov decision progress system, only one agent’s learning evolution is considered. However, considering the learning evolution of a single agent in many problems has some limitations, more and more applications involve multi-agent...
Maintenance decision-making is essential to achieve safe and reliable operation with high performance for equipment. To avoid unexpected shutdown and increase machine life as well as system efficiency, it is fundamental to design an effective mainten...
In this paper, in order to enhance the autonomous operation capabilities of aircraft and ensure their operational safety and efficiency, we propose an autonomous decision-making framework based on target motion state prediction combined with the Mark...
During the last decades, collaborative robots capable of operating out of their cages are widely used in industry to assist humans in mundane and harsh manufacturing tasks. Although such robots are inherently safe by design, they are commonly accompa...
LEO satellite mega-constellation projects have been proposed by many countries or commercial organizations in recent years. With more than 2000 satellites launched by SpaceX to configure the Starlink system, the orbital resources are more constrained...
This paper introduces a pioneering model that merges coupled Markov chains (CMC) with the fuzzy analytic hierarchy process (FAHP) to enhance multi-criteria decision making (MCDM) amidst the dynamic interplay of criteria. Traditional MCDM frameworks o...
Picking the reflection horizon is an important step in velocity inversion and seismic interpretation. Manual picking is time-consuming and no longer suitable for current large-scale seismic data processing. Automatic algorithms using different seismi...
Corrosion is one of the major causes of failure in pipelines for transporting oil and gas products. To mitigate the impact of this problem, organizations perform different maintenance operations, including detecting corrosion, determining corrosion g...
In the world of human–robot coexistence, ensuring safe interactions is crucial. Traditional logic-based methods often lack the intuition required for robots, particularly in complex environments where these methods fail to account for all possi...
Autonomous vehicles (AVs) are increasingly operating in complex traffic environments where safe and efficient decision-making is crucial. Merging into roundabouts is a key interaction scenario. This paper introduces a decision-making approach for rou...
In the context of distributed defense, multi-sensor networks are required to be able to carry out reasonable planning and scheduling to achieve the purpose of continuous, accurate and rapid target detection. In this paper, a multi-sensor cooperative...
Federated Learning (FL), as an emerging paradigm in distributed machine learning, has received extensive research attention. However, few works consider the impact of device mobility on the learning efficiency of FL. In fact, it is detrimental to the...
Recently, studies on early-exit mechanisms have emerged to reduce the computational cost during the inference process of deep learning models. However, most existing early-exit architectures simply determine early exiting based only on a target confi...
The design optimization of the grading of a resonator array for energy harvesting in sensors is described. Attention is paid to set the resonator heights, possibly removing resonators whenever convenient. Instead of employing time-consuming heuristic...
Inducing learning strategies is a crucial component of intelligent tutoring systems. Previous research has predominantly focused on the induction of offline learning strategies. Although the existing offline learning strategy induction methods can al...
Whether for institutional investors or individual investors, there is an urgent need to explore autonomous models that can adapt to the non-stationary, low-signal-to-noise markets. This research aims to explore the two unique challenges in quantitati...
The introduction of medical Internet of Things (IoT) for biomedical applications has brought about the era of proactive healthcare. Such advanced medical supervision lies on the foundation of a network of energy-constrained wearable or implantable se...
Online multi-object tracking (MOT) has broad applications in time-critical video analysis scenarios such as advanced driver-assistance systems (ADASs) and autonomous driving. In this paper, the proposed system aims at tracking multiple vehicles in th...
The multimode underwater acoustic communication network is a novel form of underwater acoustic communication that adjusts its communication mode to enhance overall performance. Current performance analysis methods are primarily applied to single-mode...
This paper proposes a continuous state partially observable Markov decision process (POMDP) model for the corrosion maintenance of oil and gas pipelines. The maintenance operations include complex and extensive activities to detect the corrosion type...
Many path planning algorithms developed for land or air based autonomous vehicles no longer apply under the water. A time-optimal path planning method for autonomous underwater vehicles (AUVs), based on a Markov decision process (MDP) algorithm, is p...
The visual analysis of trajectory topics is helpful for mining potential trajectory patterns, but the traditional visual analysis method ignores the evolution of the temporal coherence of the topic. In this paper, a novel visual analysis method for d...
The Ka-band and higher Q/V band channels can provide an appealing capacity for the future deep-space communications and Space Information Networks (SIN), which are viewed as a primary solution to satisfy the increasing demands for high data rate serv...
A real-time path planning algorithm based on the Markov decision process (MDP) is proposed in this paper. This algorithm can be used in dynamic environments to guide the wheeled mobile robot to the goal. Two phases (the utility update phase and the p...
This work focuses on optimizing energy dispatch in a hydroelectric power plant (HPP) with a large number of generating units (GUs) and uncertainties caused by sediment accumulation in the water intakes. The study was realized at Jirau HPP, and integr...
In a network of low-powered wireless sensors, it is essential to capture as many environmental events as possible while still preserving the battery life of the sensor node. This paper focuses on a real-time learning algorithm to extend the lifetime...
Trajectory planning plays a crucial role in ensuring the safe navigation of ships, as it involves complex decision making influenced by various factors. This paper presents a heuristic algorithm, named the Markov decision process Heuristic Algorithm...
With the rapid growth of the new energy vehicle market, efficient management of the closed-loop supply chain of power batteries has become an important issue. Effective closed-loop supply chain management is very critical, which is related to the eff...
The first part of a two-part series of papers provides a survey on recent advances in Deep Reinforcement Learning (DRL) applications for solving partially observable Markov decision processes (POMDP) problems. Reinforcement Learning (RL) is an approa...
The two-part series of papers provides a survey on recent advances in Deep Reinforcement Learning (DRL) for solving partially observable Markov decision processes (POMDP) problems. Reinforcement Learning (RL) is an approach to simulate the human’s na...
In this paper, we present a decentralized unmanned aerial vehicle (UAV) swarm formation control approach based on a decision theoretic approach. Specifically, we pose the UAV swarm motion control problem as a decentralized Markov decision process (De...
Local search metaheuristics like tabu search or simulated annealing are popular heuristic optimization algorithms for finding near-optimal solutions for combinatorial optimization problems. However, it is still challenging for researchers and practit...
We present REGA, a new adaptive-sampling-based algorithm for the control of finite-horizon Markov decision processes (MDPs) with very large state spaces and small action spaces. We apply a variant of the
Due to its ability to deal with non-determinism and partial observability, represent goals as an immediate reward function and find optimal solutions, planning under uncertainty using factored Markov Decision Processes (FMDPs) has increased its impor...
Traditional backward recursion methods face a fundamental challenge in solving Markov Decision Processes (MDP), where there exists a contradiction between the need for knowledge of optimal expected payoffs and the inability to acquire such knowledge...
This study establishes a novel mathematical framework for stochastic maintenance optimization in production systems by integrating Markov decision processes (MDPs) with convex programming theory. We develop a Z-transformation-based dual-space decompo...
Current electronic warfare jammers and radar countermeasures are characterized by dynamism and uncertainty. This paper focuses on a decision-making framework of radar anti-jamming countermeasures. The characteristics and implementation process of rad...
Autonomous shuttles have been used as end-mile solutions for smart mobility in smart cities. The urban driving conditions of smart cities with many other actors sharing the road and the presence of intersections have posed challenges to the use of au...
This research investigates the joint optimization of maintenance and spare unit management for series systems composed of multiple heterogeneous units. With advancements in communication and sensing technologies, condition-based maintenance has gaine...
In order to make wind energy more competitive, the big expenses for operation and maintenance must be reduced. Consistent decisions that minimize the expected costs can be made based on risk-based methods. Such methods have been implemented for maint...
The increasing demand for sustainable development necessitates sophisticated computational frameworks that facilitate intricate, long-term decision-making in circular economy systems. Therefore, we developed a novel machine learning based Markov deci...
Concept drift, which refers to changes in the underlying process structure or customer behaviour over time, is inevitable in business processes, causing challenges in ensuring that the learned model is a proper representation of the new data. Due to...
Numerous investigations assess the technical, technological, and managerial aspects of disaster response related to large technical systems. This paper deals with the possibility of synthesizing these aspects in a disaster response methodology, thus...
Background: Current obstructive sleep apnea treatment relies on manual PAP titration, but it has limitations. Complex interactions during titration and variations in SpO2 data accuracy pose challenges. Patients with co-occurring chronic hypercapnia m...
In Space–Air–Ground Integrated Networks (SAGIN), computation offloading technology is a new way to improve the processing efficiency of node tasks and improve the limitation of computing storage resources. To solve the problem of large delay and ener...
The evolution of commodity hardware makes it possible to use this type of equipment to implement traffic monitoring systems. A preliminary empirical evaluation of a network traffic probe based on Linux indicates that the system performance has signif...
As network systems become larger and more complex, there is an increasing focus on how to verify the security of systems that are at risk of being attacked. Automated penetration testing is one of the effective ways to achieve this. Uncertainty cause...
The utilization of reinforcement learning (RL) within the field of education holds the potential to bring about a significant shift in the way students approach and engage with learning and how teachers evaluate student progress. The use of RL in edu...
With the development of the convergence of IT and automotive technology, platoon-based driving systems are getting more attention and how to disseminate messages in the platoon is an important issue. In this paper, to enhance the energy efficiency an...
of 18