Submit to Applied Sciences Review for Applied Sciences Propose a Special Issue

Journal Browser

Adaptive Dynamic Programming and Control Application in Intelligent Systems

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Applied Sciences (ISSN 2076-3417). This special issue belongs to the section "Robotics and Automation".

Deadline for manuscript submissions: closed (20 November 2023) | Viewed by 7917

Share This Special Issue

Special Issue Editors

Dr. Pingli Lu

E-Mail Website
Guest Editor

School of Automation, Beijing Institute of Technology, Beijing, China
Interests: multi-agent cooperative control; fault diagnosis and fault tolerance of aircraft

Dr. Weinan Gao

E-Mail Website
Guest Editor

State Key Laboratory of Synthetical Automation for Process Industries, Shenyang 110004, China
Interests: robotic control; reinforcement learning; adaptive dynamic programming; output regulation; optimal control; cooperative control; sampled-data systems; intelligent transportation systems; connected vehicles & autonomous vehicles

Special Issue Information

Dear Colleagues,

Adaptive dynamic programming (ADP) affords a methodology for learning optimal control actions online in real time based on system performance without necessarily knowing the system dynamics. It successfully combines the theories of neural networks, adaptive evaluation design, reinforcement learning and classical dynamic programming, effectively avoids the “curse of dimensionality”, and receives much attention and research. As a result, recent years have witnessed a tremendous growth in its application in intelligent systems, such as robotic unmanned vehicles. The aim of the proposed Special Issue is to introduce the latest research results on adaptive dynamic programming and control applications in intelligent systems, including ADP for optimal regulation, ADP for game theory, ADP for cooperative control, etc. This Special Issue provides an opportunity for both academic researchers, particularly in the systems and control community, and the automotive industry to acquire frontier knowledge on ADP technologies.

This Special Issue aims to provide up-to-date research concepts, theoretical findings, and practical solutions that could help advance ADP technologies. Topics of interest include, but are not limited to, the following:

ADP-based intelligent control methods;
Deep ADP technology;
ADP for optimal regulation problems;
ADP for game theory;
ADP for large-scale systems;
ADP for multi-agent systems;

Dr. Pingli Lu
Dr. Weinan Gao
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Applied Sciences is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

ADP-based intelligent control methods
deep ADP technology
ADP for optimal regulation problems
ADP for game theory
ADP for large-scale systems
ADP for multi-agent systems

Published Papers (7 papers)

Download All Papers

Research

18 pages, 4219 KiB

Open AccessArticle

An Improved Data-Driven Integral Sliding-Mode Control and Its Automation Application

by Feng Xu, Zhen Sui, Yulong Wang and Jianliang Xu

Appl. Sci. 2023, 13(24), 13094; https://doi.org/10.3390/app132413094 - 8 Dec 2023

Viewed by 720

Abstract

Circulating fluidized bed (CFB) boilers are widely used in industrial production due to their high combustion efficiency, low pollutant emissions and wide load-adjustment range. However, the water-level-control system of a CFB boiler exhibits time-varying behavior and nonlinearity, which affect the control performance of the industrial system. This paper proposes a novel data-driven adaptive integral sliding-mode control (ISMC) method for the CFB control system with external disturbances. Firstly, the scheme designs a discrete ISMC law based on the full-format dynamic linearization (FFDL) data model, which is equivalent to a nonlinear system. Furthermore, a new reaching law is proposed to quickly drive the system state onto the sliding-mode surface. The improved ISMC control scheme only utilizes the input–output data during the design process and does not require model information. After theoretically verifying the stability of the method proposed in this paper, it is further applied in MIMO systems. Finally, the control and practical effects of this method are evaluated by using the DHX25-1.25 CFB boiler installed in the special-equipment testing center. The experimental results show that, compared with the traditional sliding-mode control (SMC) and model-free adaptive-control (MFAC) methods, the improved control method can quickly track the given signal and exhibit resistance to noise interference. Furthermore, it can rapidly respond to changes in the working conditions of the CFB system. Full article

(This article belongs to the Special Issue Adaptive Dynamic Programming and Control Application in Intelligent Systems)

► Show Figures

Figure 1

23 pages, 2211 KiB

Open AccessArticle

Adaptive-Dynamic-Programming-Based Robust Control for a Quadrotor UAV with External Disturbances and Parameter Uncertainties

by Shaoyu Yang, Fang Yu, Hui Liu, Hongyue Ma and Haichao Zhang

Appl. Sci. 2023, 13(23), 12672; https://doi.org/10.3390/app132312672 - 26 Nov 2023

Viewed by 762

Abstract

Thiswork addresses the trajectory-tracking-control problem for a quadrotor unmanned aerial vehicle with external disturbances and parameter uncertainties. A novel adaptive-dynamic-programming-based robust control method is proposed to eliminate the effects of lumped uncertainties (including external disturbances and parameter uncertainties) and to ensure the approximate optimal control performance. Its novelty lies in that two radial basis function neural network observers with fixed-time convergence properties were first established to reconstruct the lumped uncertainties. Notably, they tune only the scalar parameters online and have low computational complexities. Subsequently, two actor–critic neural networks were designed to approximate the optimal cost functions and control policies for the nominal system. In this design, two new actor–critic neural network weight update laws are proposed to eliminate the persistent excitation condition. Then, two adaptive-dynamic-programming-based robust control laws were obtained by integrating the observer reconstruction information and the nominal control policies. The uniformly ultimately bounded stability of the closed-loop tracking control systems was ensured using the Lyapunov methodology. Finally, numerical results are shown to verify the effectiveness and superiority of the proposed control scheme. Full article

(This article belongs to the Special Issue Adaptive Dynamic Programming and Control Application in Intelligent Systems)

► Show Figures

Figure 1

16 pages, 7045 KiB

Open AccessArticle

Research on Unmanned Surface Vessel Aggregation Formation Based on Improved A* and Dynamic Window Approach Fusion Algorithm

by Ge-An Wei and Jian-Qiang Zhang

Appl. Sci. 2023, 13(15), 8625; https://doi.org/10.3390/app13158625 - 26 Jul 2023

Cited by 1 | Viewed by 856

Abstract

The traditional A* and DWA fusion algorithm has three problems in the task of aggregation formation: one is the lack of meeting coordination strategy, the other is the inability to unify the terminal course, and the third is too many turning points in the global path. To solve the above problems, an improved algorithm is proposed. Firstly, by smoothing the global planning path, the stability of a USV heading in the navigation is improved. Then, by adding the key points of the global path, a guide path is formed to unify the terminal heading range. Finally, by adding an encounter coordination strategy, aggregation efficiency is improved. Simulation experiments were carried out in the Python environment based on this algorithm. The results show that the improved algorithm can improve the navigation obstacle avoidance ability of USVs and guide multiple USVs to finish the task of aggregation formation. Full article

(This article belongs to the Special Issue Adaptive Dynamic Programming and Control Application in Intelligent Systems)

► Show Figures

Figure 1

19 pages, 8275 KiB

Open AccessArticle

Retrospective-Based Deep Q-Learning Method for Autonomous Pathfinding in Three-Dimensional Curved Surface Terrain

by Qidong Han, Shuo Feng, Xing Wu, Jun Qi and Shaowei Yu

Appl. Sci. 2023, 13(10), 6030; https://doi.org/10.3390/app13106030 - 14 May 2023

Viewed by 1112

Abstract

Path planning in complex environments remains a challenging task for unmanned vehicles. In this paper, we propose a decoupled path-planning algorithm with the help of a deep reinforcement learning algorithm that separates the evaluation of paths from the planning algorithm to facilitate unmanned vehicles in real-time consideration of environmental factors. We use a 3D surface map to represent the path cost, where the elevation information represents the integrated cost. The peaks function simulates the path cost, which is processed and used as the algorithm’s input. Furthermore, we improved the double deep Q-learning algorithm (DDQL), called retrospective-double DDQL (R-DDQL), to improve the algorithm’s performance. R-DDQL utilizes global information and incorporates a retrospective mechanism that employs fuzzy logic to evaluate the quality of selected actions and identify better states for inclusion in the memory. Our simulation studies show that the proposed R-DDQL algorithm has better training speed and stability compared to the deep Q-learning algorithm and double deep Q-learning algorithm. We demonstrate the effectiveness of the R-DDQL algorithm under both static and dynamic tasks. Full article

(This article belongs to the Special Issue Adaptive Dynamic Programming and Control Application in Intelligent Systems)

► Show Figures

Figure 1

16 pages, 6183 KiB

Open AccessArticle

Intelligent Control of a Space Manipulator Ground Unfold Experiment System with Lagging Compensation

by Xiao Zhang, Zainan Jiang, Zhen Zhao, Yun He, Zhigang Xu and Yong Liu

Appl. Sci. 2023, 13(9), 5508; https://doi.org/10.3390/app13095508 - 28 Apr 2023

Cited by 1 | Viewed by 816

Abstract

In ground testing of space manipulators, gravity compensation is a critical testing requirement. The objective of this paper was to design a space manipulator gravity compensation test platform for ground tests and solve the problems of force control oscillation and precision degradation caused by the execution lag encountered in the development process. An intelligent PID controller was designed for this active-suspension gravity compensation experimental mechanism of a space manipulator on the ground, and a specially designed second-order method was used to solve the problem of the execution lag in this mechanism. The intelligent controller was developed based on adaptive dynamic programming and redesigned to improve its transient performance. The simulation was carried out, and its results were compared with the results on a real machine to demonstrate the effectiveness of this set of experimental controllers. This paper compares in detail the results of the designed method on system input and output and shows the effectiveness of this method in dealing with the execution lag of the mechanism. In conclusion, in this work, we successfully designed and implemented an intelligent PID controller for an active-suspension gravity compensation experimental mechanism of a space manipulator on the ground, and the experimental results demonstrate the effectiveness of the proposed method. Full article

(This article belongs to the Special Issue Adaptive Dynamic Programming and Control Application in Intelligent Systems)

► Show Figures

Figure 1

27 pages, 8596 KiB

Open AccessArticle

Adaptive Dynamic Programming-Based Cross-Scale Control of a Hydraulic-Driven Flexible Robotic Manipulator

by Xiaohua Wei, Jiangang Ye, Jianliang Xu and Zhiguo Tang

Appl. Sci. 2023, 13(5), 2890; https://doi.org/10.3390/app13052890 - 23 Feb 2023

Cited by 3 | Viewed by 1809

Abstract

This paper focuses primarily on adaptive dynamic programming (ADP)-based tracking control of the hydraulic-driven flexible robotic manipulator system (HDFRMS) with varying payloads and uncertainties via singular perturbation theory (SPT). Firstly, the dynamics is derived using a driven Jacobin matrix, which represents the coupling between the hydraulic servo-driven system and rigid–flexible manipulator established using the assumed mode method and Lagrange principle. Furthermore, the whole dynamic model of the manipulator system is decoupled into a second slow subsystem (SSS), a second fast subsystem (SFS) and a first fast subsystem (FFS). The three subsystems can describe a large range of movement, flexible vibration and electro-hydraulic servo control, respectively. Hereafter, an adaptive dynamic programming trajectory tracking control law with a critic-only policy iteration algorithm is presented in the second slow timescale, while both robust optimal control (ROC) in the second first timescale and adaptive sliding mode control (ASMC) in the first fast timescale are also designed using the Lyapunov stability theory. Finally, the numerical simulations are carried out to illustrate the rightness and robustness of the singular perturbation decomposition and proposed composite control algorithm. Full article

(This article belongs to the Special Issue Adaptive Dynamic Programming and Control Application in Intelligent Systems)

► Show Figures

Figure 1

17 pages, 969 KiB

Open AccessArticle

A Gradient-Based Particle-Bat Algorithm for Stochastic Configuration Network

by Jingjing Liu, Yefeng Liu and Qichun Zhang

Appl. Sci. 2023, 13(5), 2878; https://doi.org/10.3390/app13052878 - 23 Feb 2023

Cited by 1 | Viewed by 970

Abstract

Stochastic configuration network (SCN) is a mathematical model of incremental generation under a supervision mechanism, which has universal approximation property and advantages in data modeling. However, the efficiency of SCN is affected by some network parameters. An optimized searching algorithm for the input weights and biases is proposed in this paper. An optimization model with constraints is first established based on the convergence theory and inequality supervision mechanism of SCN; Then, a hybrid bat-particle swarm optimization algorithm (G-BAPSO) based on gradient information is proposed under the framework of PSO algorithm, which mainly uses gradient information and local adaptive adjustment mechanism characterized by pulse emission frequency to improve the searching ability. The algorithm optimizes the input weights and biases to improve the convergence rate of the network. Simulation results over some datasets demonstrate the feasibility and validity of the proposed algorithm. The training RMSE of G-BAPSO-SCN increased by

5.57 \times 10^{- 5}

and

3.2 \times 10^{- 3}

compared with that of SCN in the two regression experiments, and the recognition accuracy of G-BAPSO-SCN increased by 0.07% on average in the classification experiments. Full article

(This article belongs to the Special Issue Adaptive Dynamic Programming and Control Application in Intelligent Systems)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Adaptive Dynamic Programming and Control Application in Intelligent Systems

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Published Papers (7 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI