Search Results (7)

This paper investigates the robust tracking control of unmanned aerial vehicles (UAVs) against external time-varying disturbances. First, by introducing a virtual position controller, we innovatively decouple the UAV dynamics into independent position and attitude error subsystems, transforming the robust tracking problem into two zero-sum differential games. This approach contrasts with conventional methods by treating disturbances as strategic “players”, enabling a systematic framework to address both external disturbances and model uncertainties. Second, we develop an integral reinforcement learning (IRL) framework that approximates the optimal solution to the Hamilton–Jacobi–Isaacs (HJI) equations without relying on precise system models. This model-free strategy overcomes the limitation of traditional robust control methods that require known disturbance bounds or accurate dynamics, offering superior adaptability to complex environments. Third, the proposed recursive Ridge regression with a forgetting factor (${\mathrm{R}}^3{\mathrm{F}}^2$ ) algorithm updates actor-critic-disturbance neural network (NN) weights in real time, ensuring both computational efficiency and convergence stability. Theoretical analyses rigorously prove the closed-loop system stability and algorithm convergence, which fills a gap in existing data-driven control studies lacking rigorous stability guarantees. Finally, numerical results validate that the method outperforms state-of-the-art model-based and model-free approaches in tracking accuracy and disturbance rejection, demonstrating its practical utility for engineering applications. Full article

Mixed zero-sum games consider both zero-sum and non-zero-sum differential game problems simultaneously. In this paper, multiplayer mixed zero-sum games (MZSGs) are studied by the means of an integral reinforcement learning (IRL) algorithm under the dynamic event-triggered control (DETC) mechanism for completely unknown nonlinear systems. Firstly, the adaptive dynamic programming (ADP)-based on-policy approach is proposed for solving the MZSG problem for the nonlinear system with multiple players. Secondly, to avoid using dynamic information of the system, a model-free control strategy is developed by utilizing actor–critic neural networks (NNs) for addressing the MZSG problem of unknown systems. On this basis, for the purpose of avoiding wasted communication and computing resources, the dynamic event-triggered mechanism is integrated into the integral reinforcement learning algorithm, in which a dynamic triggering condition is designed to further reduce triggering times. With the help of the Lyapunov stability theorem, the system states and weight values of NNs are proven to be uniformly ultimately bounded (UUB) stable. Finally, two examples are demonstrated to show the effectiveness and feasibility of the developed control method. Compared with static event-triggering mode, the simulation results show that the number of actuator updates in the DETC mechanism has been reduced by 55% and 69%, respectively. Full article

This paper employs an integral reinforcement learning (IRL) method to investigate the optimal tracking control problem (OTCP) for nonlinear nonzero-sum (NZS) differential game systems with unknown drift dynamics. Unlike existing methods, which can only bound the tracking error, the proposed approach ensures that the tracking error asymptotically converges to zero. This study begins by constructing an augmented system using the tracking error and reference signal, transforming the original OTCP into solving the coupled Hamilton–Jacobi (HJ) equation of the augmented system. Because the HJ equation contains unknown drift dynamics and cannot be directly solved, the IRL method is utilized to convert the HJ equation into an equivalent equation without unknown drift dynamics. To solve this equation, a critic neural network (NN) is employed to approximate the complex value function based on the tracking error and reference information data. For the unknown NN weights, the least squares (LS) method is used to design an estimation law, and the convergence of the weight estimation error is subsequently proven. The approximate solution of optimal control converges to the Nash equilibrium, and the tracking error asymptotically converges to zero in the closed system. Finally, we validate the effectiveness of the proposed method in this paper based on MATLAB using the ode45 method and least squares method to execute Algorithm 2. Full article

In this paper, adaptive fuzzy fault-tolerant control (AFFTC) for the attitude control system of a hypersonic gliding vehicle (HGV) experiencing an actuator fault is proposed. Actuator faults of the HGV are considered with respect to its actual structure and actuator characteristics. The HGV’s attitude system is firstly represented by a T–S fuzzy model, and then a normal T–S fuzzy controller is designed. A reinforcement learning (RL)-based policy iterative solution algorithm is proposed for the solving of the T-S fuzzy controller. Then, based on the normal T–S controller, a fuzzy FTC controller is proposed in which the control matrices can improve themselves according to the special fault. An integral reinforcement learning (IRL)-based solving algorithm is proposed to reduce the dependence of the design methods on the HGV model. Simulations on three different kinds of actuator faults show that the designed IRL-based FTC can ensure a reliable flight by the HGV. Full article

In this paper, an event-triggered adaptive dynamic programming (ADP) method is proposed to deal with the $H_{\infty }$ problem with unknown dynamic and constrained input. Firstly, the $H_{\infty }$-constrained problem is regarded as the two-player zero-sum game with the nonquadratic value function. Secondly, we develop the event-triggered Hamilton–Jacobi–Isaacs(HJI) equation, and an event-triggered ADP method is proposed to solve the HJI equation, which is equivalent to solving the Nash saddle point of the zero-sum game. An event-based single-critic neural network (NN) is applied to obtain the optimal value function, which reduces the communication resource and computational cost of algorithm implementation. For the event-triggered control, a triggering condition with the level of disturbance attenuation is developed to limit the number of sampling states, and the condition avoids Zeno behavior by proving the existence of events with minimum triggering interval. It is proved theoretically that the closed-loop system is asymptotically stable, and the critic NN weight error is uniformly ultimately boundedness (UUB). The learning performance of the proposed algorithm is verified by two examples. Full article

This paper proposes an approximate optimal curve-path-tracking control algorithm for partially unknown nonlinear systems subject to asymmetric control input constraints. Firstly, the problem is simplified by introducing a feedforward control law, and a dedicated design for optimal control with asymmetric input constraints is provided by redesigning the control cost function in a non-quadratic form. Then, the optimality and stability of the derived optimal control policy is demonstrated. To solve the underlying tracking Hamilton–Jacobi–Bellman (HJB) equation in consideration of partially unknown systems, an integral reinforcement learning (IRL) algorithm is utilized using the neural network (NN)-based value function approximation. Finally, the effectiveness and generalization of the proposed method is verified by experiments carried out on a high-fidelity hardware-in-the-loop (HIL) simulation system for fixed-wing unmanned aerial vehicles (UAVs) in comparison with three other typical path-tracking control algorithms. Full article

Socially-Assistive Robotics (SAR) has been extensively used for a variety of applications, including educational assistants, exercise coaches and training task instructors. The main goal of such systems is to provide a personalized and tailored session that matches user abilities and needs. While objective measures (e.g., task performance) can be used to adjust task parameters (e.g., task difficulty), towards personalization, it is essential that such systems also monitor task engagement to personalize their training strategies and maximize the effects of the training session. We propose an Interactive Reinforcement Learning (IRL) framework that combines explicit feedback (task performance) with implicit human-generated feedback (task engagement) to achieve efficient personalization. We illustrate the framework with a cognitive training task, describing our data-driven methodology (data collection and analysis, user simulation) towards designing our proposed real-time system. Our data analysis and the reinforcement learning experiments on real user data indicate that the integration of task engagement as human-generated feedback in the RL mechanism can facilitate robot personalization, towards a real-time personalized robot-assisted training system. Full article