Adaptive Predefined-Time Tracking Control for Robotic Manipulator Based on Actor-Critic Reinforcement Learning

Abstract

This paper proposes a novel predefined-time adaptive neural tracking control method for uncertain manipulator systems based on Actor-Critic reinforcement learning framework. The proposed control scheme integrates the advantages of predefined-time stability theory and reinforcement learning to achieve fast convergence with guaranteed settling time bounds while handling unknown system dynamics. An Actor neural network is designed to approximate the unknown nonlinear functions and generate control inputs, while a Critic neural network evaluates the cost-to-go function to guide the learning process. The predefined-time convergence is ensured by incorporating specially designed terms into both the control law and the neural network weight update laws. The upper bound of the settling time can be explicitly preset by a single design parameter, independent of initial conditions and system parameters. Rigorous stability analysis based on Lyapunov theory proves that all closed-loop signals are bounded and the tracking error converges to a small neighborhood of the origin within the predefined time. Simulation results on a single-link manipulator system demonstrate the effectiveness and superiority of the proposed control scheme compared with conventional PID control.

1. Introduction

Robotic manipulators have been extensively deployed in industrial manufacturing, medical surgery, space exploration, and military applications due to their high flexibility, precision, repeatability, and efficiency [1,2]. In these applications, the control system must achieve accurate trajectory tracking while adapting to varying operating conditions and task requirements. However, the control of robotic manipulators remains challenging due to their inherent nonlinearities arising from trigonometric functions in the dynamic equations, strong coupling effects between joints, and inevitable uncertainties stemming from unmodeled dynamics, parameter variations, friction, and external disturbances [3]. Therefore, developing advanced control strategies that simultaneously guarantee tracking performance, fast convergence, and robustness against uncertainties has become a critical research topic in the field of robotics and control engineering.

To address the challenges posed by system uncertainties, numerous advanced control strategies have been developed over the past decades. Adaptive control provides an effective approach to handle parametric uncertainties through online parameter estimation, enabling the controller to adjust its parameters in real-time based on system behavior [4,5]. Neural network (NN)-based control has gained significant attention for its universal approximation capability, which allows it to deal with unknown nonlinear functions without requiring explicit mathematical models [6]. The combination of adaptive control and neural networks, known as adaptive neural network control, has demonstrated excellent performance in handling both parametric and functional uncertainties, and has been successfully applied to various robotic systems [7,8]. Despite these advances, most existing adaptive neural control methods only guarantee asymptotic or exponential convergence, where the settling time depends on initial conditions and system parameters, which may not satisfy the strict timing requirements in practical applications.

In practical robotic applications, fast convergence is often a critical requirement, particularly in time-critical tasks such as assembly operations, surgical procedures, and emergency response scenarios. To achieve convergence in finite time, finite-time control and fixed-time control have been developed based on nonsmooth Lyapunov analysis [9,10]. Finite-time control ensures that the system states converge to the equilibrium within a finite settling time, but this settling time depends on initial conditions, making it difficult to predict or prescribe in advance. Fixed-time control addresses this limitation by ensuring that the settling time is bounded regardless of initial conditions [3]. However, the relationship between the settling time bound and control parameters in fixed-time control is implicit and complex, typically involving multiple design parameters in a nonlinear manner, which complicates the controller tuning process for achieving desired convergence speed.

Recently, predefined-time control has emerged as a promising approach that allows designers to explicitly preset the upper bound of the settling time through a single design parameter [11,12,13]. This feature is particularly attractive for applications with strict timing requirements, as the maximum convergence time can be directly specified according to task demands without complex parameter calculations. Several predefined-time control schemes have been proposed for various systems including rigid spacecraft attitude stabilization [14] and robotic manipulators [15]. However, most existing predefined-time control methods require accurate system models or assume that the system uncertainties are bounded with known bounds, which significantly limits their practical applicability to real-world robotic systems where model parameters are often unknown or time-varying.

On the other hand, reinforcement learning (RL) has shown great potential in control applications due to its ability to learn optimal control policies through interaction with the environment without requiring explicit system models [16,17]. Among various RL architectures, the Actor-Critic (AC) framework is particularly well-suited for continuous control problems, where the Actor network generates control actions and the Critic network evaluates the performance by estimating the value function or cost-to-go [18,19]. The combination of Actor-Critic reinforcement learning and neural network approximation has been successfully applied to various robotic control problems, demonstrating improved adaptability and optimality compared to conventional methods [20,21]. The Actor-Critic structure offers several advantages: the Critic provides a global performance metric for guiding the Actor’s learning, the dual-network architecture separates policy evaluation from policy improvement for enhanced learning efficiency, and the framework naturally accommodates online learning in real-time control scenarios.

Despite the significant progress in each individual area, there remains a gap in the literature regarding the unified treatment of predefined-time convergence, adaptive learning capability, and optimal control for uncertain robotic systems. Most existing predefined-time control methods lack the ability to handle unknown nonlinearities adaptively, while conventional adaptive neural control schemes cannot guarantee predefined-time convergence. The integration of predefined-time stability with Actor-Critic reinforcement learning presents unique theoretical challenges: the predefined-time convergence mechanism must be incorporated into both the control law and the neural network weight update laws in a compatible manner, and the stability analysis must account for the coupled dynamics of tracking errors and weight estimation errors within the predefined-time framework. To the best of the authors’ knowledge, the problem of predefined-time adaptive neural control using Actor-Critic reinforcement learning for robotic manipulators has not been adequately addressed in the existing literature.

Motivated by the above observations, this paper proposes a novel predefined-time adaptive neural tracking control scheme for uncertain single-link manipulator systems based on the Actor-Critic reinforcement learning framework. The main contributions of this paper are summarized as follows:

• A novel control framework that synergistically integrates predefined-time stability theory with Actor-Critic reinforcement learning is proposed. The Actor neural network approximates unknown system dynamics and generates control inputs, while the Critic neural network evaluates the cost-to-go function to guide the learning process, achieving both guaranteed convergence time and online learning capability.
• Predefined-time neural network weight update laws are designed with specially constructed terms that incorporate the predefined-time convergence mechanism. These update laws ensure the convergence of both tracking errors and weight estimation errors within the predefined time while maintaining the learning and approximation capabilities of the neural networks.
• The upper bound of the settling time can be explicitly preset by a single design parameter $T_c$, satisfying $T_P<\sqrt{2}{T}_c$, which is independent of initial conditions and system parameters. This explicit relationship between the design parameter and settling time bound greatly simplifies the controller design process for applications with specific timing requirements.

The remainder of this paper is organized as follows. Section 2 presents the single-link manipulator system model, introduces the Actor-Critic neural network framework, and provides necessary mathematical preliminaries including the predefined-time stability lemma. Section 3 details the controller design procedure, including the predefined-time virtual controller, the Actor-Critic reinforcement learning controller, and the predefined-time weight update laws. Section 4 provides the rigorous stability analysis based on Lyapunov theory. Section 5 presents comprehensive simulation results to validate the effectiveness and superiority of the proposed control scheme. Finally, Section 6 concludes the paper and discusses future research directions.

2. Preliminaries and Problem Formulation

2.1. System Model

Consider a single-link robotic manipulator system described by the following dynamic equation:

$$I\ddot{q}+c_0\dot{q}+mglcos\left(q\right)=\tau +d$$

(1)

where $q\in \mathbb{R}$ denotes the joint angle, $\dot{q}\in \mathbb{R}$ is the angular velocity, $\ddot{q}\in \mathbb{R}$ is the angular acceleration, $\tau \in \mathbb{R}$ represents the control torque, $I={\displaystyle \frac{4}{3}}m{l}^2$ is the moment of inertia, m is the link mass, l is the link length, $c_0$ is the viscous friction coefficient, g is the gravitational acceleration, and d represents the bounded external disturbance satisfying $\left|d\right|\le \overline{d}$ with $\overline{d}$ being a known positive constant.

Define the state variables $x_1=q$ and $x_2=\dot{q}$. The system (1) can be rewritten in the following state-space form:

$$\left\{\begin{array}{c}{\dot{x}}_1=x_2\hfill \\ {\dot{x}}_2=f\left(\mathit{x}\right)+g_{1}u\hfill \\ y=x_1\hfill \end{array}\right.$$

(2)

where $u=\tau$ is the control input, $y=x_1$ is the system output, $g_1={\displaystyle \frac{1}{I}}$ is a known positive constant, and

$$f\left(\mathit{x}\right)=-{\displaystyle \frac{1}{I}}\left[c_0{x}_2+mglcos\left(x_1\right)-d\right]$$

(3)

is an uncertain nonlinear function.

Assumption 1.

The desired reference trajectory $y_d$ and its derivatives ${\dot{y}}_d$, ${\ddot{y}}_d$ are continuous and bounded, i.e., there exist positive constants ${\overline{y}}_d$, ${\overline{y}}_{d1}$, ${\overline{y}}_{d2}$ such that $|y_d|\le {\overline{y}}_d$, $|{\dot{y}}_d|\le {\overline{y}}_{d1}$, $|{\ddot{y}}_d|\le {\overline{y}}_{d2}$.

2.2. Control Objective

The control objective is to design an adaptive neural tracking controller based on Actor-Critic reinforcement learning such that:

(i)

The joint angle $x_1$ tracks the desired trajectory $y_d$ with the tracking error converging to a small neighborhood of the origin within a predefined time $T_P<\sqrt{2}{T}_c$, where $T_c$ is a preset design parameter.

(ii)

All signals in the closed-loop system remain bounded within the predefined time.

(iii)

The Actor-Critic neural networks learn to compensate for the unknown system dynamics online.

2.3. Actor-Critic Neural Network Framework

To handle the unknown nonlinear functions in the system and achieve adaptive optimal control, this paper employs an Actor-Critic reinforcement learning framework. This framework consists of two cooperatively working neural networks: the Actor network is responsible for approximating unknown dynamics and generating control policies, while the Critic network evaluates the control performance and guides the Actor’s learning process.

2.3.1. RBF Basis Function

Both neural networks adopt Radial Basis Functions (RBFs) as basis functions due to their universal approximation capability. For a continuous function $h\left(Z\right):{\mathbb{R}}^n\to \mathbb{R}$ defined on a compact set ${\mathsf{\Omega }}_Z\subset {\mathbb{R}}^n$, it can be approximated by an RBF neural network as:

$$h\left(Z\right)=W^{*T}S\left(Z\right)+ϵ\left(Z\right)$$

(4)

where $Z={[Z_1,Z_2,\dots ,Z_n]}^T\in {\mathsf{\Omega }}_Z$ is the input vector, $W^{*}={[w_1^{*},w_2^{*},\dots ,w_l^{*}]}^T\in {\mathbb{R}}^l$ is the ideal weight vector, l is the number of neural network nodes, $S\left(Z\right)={[s_1\left(Z\right),s_2\left(Z\right),\dots ,s_l\left(Z\right)]}^T$ is the basis function vector, and $ϵ\left(Z\right)$ is the approximation error satisfying $\left|ϵ\left(Z\right)\right|\le \overline{ϵ}$.

The Gaussian function is employed as the basis function:

$$s_i\left(Z\right)=\exp \left(-{\displaystyle \frac{\parallel Z-{\mu }_i{\parallel }^2}{{\eta }^2}}\right),i=1,2,\dots ,l$$

(5)

where ${\mu }_i={[{\mu }_{i1},{\mu }_{i2},\dots ,{\mu }_{in}]}^T$ is the center of the i-th basis function, and $\eta >0$ is the width of the Gaussian function.

2.3.2. Critic Network Structure

The Critic network is designed to evaluate the long-term performance of the current control policy. The long-term cost function is defined as:

$$I\left(t\right)={\int }_t^{\infty }{e}^{-{\displaystyle \frac{m-t}{\psi }}}\varphi \left(m\right)dm$$

(6)

where $\psi >0$ is the discount factor, and the instantaneous cost function is defined as:

$$\varphi \left(t\right)=z_1^{T}D{z}_1+{\tau }^{T}R\tau$$

(7)

where $D>0$ and $R>0$ are positive definite weight matrices that penalize the tracking error and control effort, respectively.

Using the RBF neural network to approximate the cost function:

$$I=W_c^{*T}{S}_c\left(Z_c\right)+{ϵ}_c$$

(8)

$$\widehat{I}={\widehat{W}}_c^T{S}_c\left(Z_c\right)$$

(9)

where $Z_c=z_1$ is the Critic network input, $W_c^{*}\in {\mathbb{R}}^{l_c}$ is the ideal weight vector, ${\widehat{W}}_c$ is the estimated weight vector, $S_c\left(Z_c\right)$ is the basis function vector, and ${ϵ}_c$ satisfies $|{ϵ}_c|\le {\overline{ϵ}}_c$.

When $\psi \to \infty$, based on the Bellman equation, the temporal difference (TD) error can be expressed as:

$$\gamma \left(t\right)=\varphi \left(t\right)+\dot{\widehat{I}}\left(t\right)=\varphi \left(t\right)+{\widehat{W}}_c^T\Lambda$$

(10)

where $\mathsf{\Lambda }=-{\displaystyle \frac{S_c}{\psi }}+\nabla S_c{\dot{Z}}_c$. The learning objective of the Critic network is to minimize the TD error.

2.3.3. Actor Network Structure

The Actor network is designed to approximate the unknown nonlinear functions in the system and assist in generating control inputs. Define the composite unknown function:

$$\phi =f\left(\mathit{x}\right)-{\dot{\alpha }}_1+z_1+{\displaystyle \frac{1}{2}}{z}_2$$

(11)

where $f\left(\mathit{x}\right)$ is the unknown nonlinear term of the system, and ${\dot{\alpha }}_1$ is the derivative of the virtual control.

Using the RBF neural network to approximate $\phi$:

$$\phi =W_a^{*T}{S}_a\left(Z_a\right)+{ϵ}_a$$

(12)

$$\widehat{\phi }={\widehat{W}}_a^T{S}_a\left(Z_a\right)$$

(13)

where $Z_a={[x_1,x_2,y_d,{\dot{y}}_d,{\ddot{y}}_d]}^T$ is the Actor network input, $W_a^{*}\in {\mathbb{R}}^{l_a}$ is the ideal weight vector, ${\widehat{W}}_a$ is the estimated weight vector, $S_a\left(Z_a\right)$ is the basis function vector, and ${ϵ}_a$ satisfies $|{ϵ}_a|\le {\overline{ϵ}}_a$.

2.3.4. Actor-Critic Cooperative Learning Mechanism

The cooperative learning mechanism of the Actor-Critic framework operates as follows:

(1)

Critic evaluates policy performance: The Critic network computes the estimated cost function $\widehat{I}$ based on the current state and control input, evaluating the quality of the Actor’s current policy. A larger $\widehat{I}$ indicates poorer policy performance that requires improvement.

(2)

Actor improves control policy: The Actor network utilizes the evaluation information $\widehat{I}$ provided by the Critic as feedback to adjust its weights ${\widehat{W}}_a$, thereby improving the control policy to minimize the long-term cost.

(3)

Online cooperative update: The weights of both networks are updated in real-time during the control process. Through continuous “evaluation-improvement” cycles, the control performance is progressively optimized.

Define the weight estimation errors as:

$${\tilde{W}}_c=W_c^{*}-{\widehat{W}}_c,{\tilde{W}}_a=W_a^{*}-{\widehat{W}}_a$$

(14)

The specific weight update laws for the Actor-Critic networks will be designed in Section 3, incorporating the predefined-time stability requirements.

Remark 1.

Compared with traditional single neural network adaptive control, the Actor-Critic framework possesses the following advantages: (i) The value function evaluation provided by the Critic offers a global performance metric for the Actor, rather than relying solely on local error information; (ii) The dual-network structure separates policy evaluation from policy improvement, enhancing learning efficiency and stability; (iii) This framework is naturally suited for integration with predefined-time control, allowing the predefined-time convergence mechanism to be incorporated into the weight update laws of both networks.

2.4. Technical Lemmas

Lemma 1

([22]). For any $\xi \in \mathbb{R}$ and $\rho >0$, the following inequality holds:

$$0\le \left|\xi \right|-\xi tanh\left({\displaystyle \frac{\xi }{\rho }}\right)\le \delta \rho$$

(15)

where $\delta =0.2785$.

Lemma 2

([23]). For $x_i\ge 0$ ($i=1,2,\dots ,n$) and $\gamma >0$, the following inequalities hold:

$$\sum _{i=1}^n{x}_i^{\gamma }\ge {\left(\sum _{i=1}^n{x}_i\right)}^{\gamma },0<\gamma <1$$

(16)

$$\sum _{i=1}^n{x}_i^{\gamma }\ge n^{1-\gamma }{\left(\sum _{i=1}^n{x}_i\right)}^{\gamma },\gamma >1$$

(17)

Lemma 3.

For $y\ge x$ and $v>1$, the following inequality holds:

$$x{(y-x)}^v\le {\displaystyle \frac{v}{1+v}}\left(y^{1+v}-x^{1+v}\right)$$

(18)

Lemma 4

([24]). (Predefined-Time Stability) Consider the system $\dot{x}=f(t,x)$. If there exists a continuous positive definite function $V\left(x\right)$ and parameters $0<\beta <1$, $T_c>0$, $0<\sigma <\infty$ such that

$$\dot{V}\left(x\right)\le -{\displaystyle \frac{\pi }{\beta T_c}}\left(V^{1-{\displaystyle \frac{\beta }{2}}}+V^{1+{\displaystyle \frac{\beta }{2}}}\right)+\sigma$$

(19)

then the system is practically predefined-time stable (PPTS), and the convergence region is

$$\left\{\underset{t\to T_P}{lim}x\mid V\le min\left\{{\left({\displaystyle \frac{2\beta T_c\sigma }{\pi }}\right)}^{{\displaystyle \frac{2}{2-\beta }}},{\left({\displaystyle \frac{2\beta T_c\sigma }{\pi }}\right)}^{{\displaystyle \frac{2}{2+\beta }}}\right\}\right\}$$

(20)

where $T_P$ is the settling time satisfying $T_P<T_{max}=\sqrt{2}{T}_c$.

Remark 2.

Lemma 4 is fundamental to predefined-time stability theory. The key feature is that the upper bound of the settling time $T_{max}=\sqrt{2}{T}_c$ can be explicitly preset through the parameter $T_c$, independent of the initial conditions and system parameters. This is in contrast to finite-time control where the settling time depends on initial conditions, and fixed-time control where the settling time bound is implicitly determined by multiple parameters.

Lemma 5

([25]). For any $z\in \mathbb{R}$ and $\kappa >0$:

$$0\le \left|z\right|-{\displaystyle \frac{z^2}{\sqrt{z^2+{\kappa }^2}}}<\kappa$$

(21)

Lemma 6.

(Power Function Inequality) For any $x>0$ and $0<\beta <1$, the following inequality holds:

$$x^2\ge x^{2-\beta }-C_{\beta }$$

(22)

where $C_{\beta }={\displaystyle \frac{\beta }{2}}{\left({\displaystyle \frac{2-\beta }{2}}\right)}^{{\displaystyle \frac{2-\beta }{\beta }}}$ is a positive constant depending only on β.

Proof. 

Define $f\left(x\right)=x^2-x^{2-\beta }$ for $x>0$. Taking the derivative:

$$f\prime \left(x\right)=2x-(2-\beta )x^{1-\beta }=x^{1-\beta }\left(2x^{\beta }-(2-\beta )\right)$$

Setting $f\prime \left(x\right)=0$ yields the critical point $x^{*}={\left({\displaystyle \frac{2-\beta }{2}}\right)}^{{\displaystyle \frac{1}{\beta }}}$. Since $f″\left(x^{*}\right)>0$, this is a minimum point. The minimum value is:

$$f\left(x^{*}\right)=-{\displaystyle \frac{\beta }{2}}{\left({\displaystyle \frac{2-\beta }{2}}\right)}^{{\displaystyle \frac{2-\beta }{\beta }}}=-C_{\beta }$$

Therefore, $f\left(x\right)\ge -C_{\beta }$, which completes the proof. □

3. Actor-Critic Predefined-Time Controller Design

In this section, we present the design of the predefined-time adaptive neural tracking controller based on the Actor-Critic reinforcement learning framework. The control system architecture is illustrated in Figure 1. The Actor network receives system states and reference signals, outputs the control signal to compensate for unknown dynamics. The Critic network evaluates the cost-to-go and provides feedback to guide the Actor’s learning process. Both networks are updated using predefined-time weight update laws.

3.1. Predefined-Time Virtual Controller Design

Define the tracking error variables as:

$$z_1=x_1-y_d$$

(23)

$$z_2=x_2-{\alpha }_1$$

(24)

where ${\alpha }_1$ is the virtual control law to be designed.

The time derivative of $z_1$ is:

$${\dot{z}}_1={\dot{x}}_1-{\dot{y}}_d=x_2-{\dot{y}}_d=z_2+{\alpha }_1-{\dot{y}}_d$$

(25)

Design the predefined-time virtual controller as:

$${\alpha }_1=-{\displaystyle \frac{z_1{\check{\alpha }}_1^2}{\sqrt{z_1^2{\check{\alpha }}_1^2+{\epsilon }_1^2}}}+{\dot{y}}_d$$

(26)

where ${\epsilon }_1>0$ is a small positive constant, and

$${\check{\alpha }}_1={\displaystyle \frac{\pi }{\beta T_c}}\left[2^{\beta }{\left({\displaystyle \frac{1}{2}}\right)}^{1+{\displaystyle \frac{\beta }{2}}}{\mathrm{sig}}^{1+\beta }\left(z_1\right)+{\left({\displaystyle \frac{1}{2}}\right)}^{1-{\displaystyle \frac{\beta }{2}}}{\mathrm{sig}}^{1-\beta }\left(z_1\right)\right]$$

(27)

with $0<\beta <1$, $T_c>0$ being the predefined time parameter, and ${\mathrm{sig}}^{\gamma }\left(a\right)={\left|a\right|}^{\gamma }\mathrm{sgn}\left(a\right)$.

Remark 3.

The virtual controller (26) is specifically designed to achieve predefined-time convergence. The structure $-{\displaystyle \frac{z_1{\check{\alpha }}_1^2}{\sqrt{z_1^2{\check{\alpha }}_1^2+{\epsilon }_1^2}}}$ ensures that the derivative ${\dot{\alpha }}_1$ remains bounded even when $z_1$ approaches zero, thus avoiding the singularity issue that commonly arises in traditional finite-time control designs where terms like ${\mathit{sig}}^{1-\beta }\left(z_1\right)$ with $0<\beta <1$ would cause unbounded derivatives.

Consider the Lyapunov function candidate:

$$V_1={\displaystyle \frac{1}{2}}{z}_1^2$$

(28)

Taking the time derivative of $V_1$ and substituting (25) and (26):

$${\dot{V}}_1=z_1{\dot{z}}_1=z_1(z_2+{\alpha }_1-{\dot{y}}_d)=z_1{z}_2-{\displaystyle \frac{z_1^2{\check{\alpha }}_1^2}{\sqrt{z_1^2{\check{\alpha }}_1^2+{\epsilon }_1^2}}}$$

(29)

Applying Lemma 5:

$$-{\displaystyle \frac{z_1^2{\check{\alpha }}_1^2}{\sqrt{z_1^2{\check{\alpha }}_1^2+{\epsilon }_1^2}}}<{\epsilon }_1-\left|z_1{\check{\alpha }}_1\right|<{\epsilon }_1-{\displaystyle \frac{\pi }{\beta T_c}}{\left({\displaystyle \frac{z_1^2}{2}}\right)}^{1-{\displaystyle \frac{\beta }{2}}}-{\displaystyle \frac{\pi }{\beta T_c}}{2}^{\beta }{\left({\displaystyle \frac{z_1^2}{2}}\right)}^{1+{\displaystyle \frac{\beta }{2}}}$$

(30)

Therefore:

$${\dot{V}}_1<-{\displaystyle \frac{\pi }{\beta T_c}}{\left({\displaystyle \frac{z_1^2}{2}}\right)}^{1-{\displaystyle \frac{\beta }{2}}}-{\displaystyle \frac{\pi }{\beta T_c}}{2}^{\beta }{\left({\displaystyle \frac{z_1^2}{2}}\right)}^{1+{\displaystyle \frac{\beta }{2}}}+z_1{z}_2+{\epsilon }_1$$

(31)

3.2. Actor-Critic Reinforcement Learning Controller Design

The time derivative of $z_2$ is:

$${\dot{z}}_2={\dot{x}}_2-{\dot{\alpha }}_1=f\left(\mathit{x}\right)+g_{1}u-{\dot{\alpha }}_1$$

(32)

Define the unknown nonlinear function:

$$\phi =f\left(\mathit{x}\right)-{\dot{\alpha }}_1+z_1+{\displaystyle \frac{1}{2}}{z}_2$$

(33)

3.2.1. Critic Network Design

The Critic network is designed to approximate the cost-to-go function and evaluate the control performance. Define the long-term cost function:

$$I\left(t\right)={\int }_t^{\infty }{e}^{-{\displaystyle \frac{m-t}{\psi }}}\varphi \left(m\right)dm$$

(34)

where $\psi >0$ is a discount factor, and the instantaneous cost function is defined as:

$$\varphi \left(t\right)=z_1^{T}D{z}_1+{\tau }^{T}R\tau$$

(35)

with $D>0$ and $R>0$ being positive definite weight matrices.

The cost-to-go function is approximated by the Critic neural network:

$$I=W_c^{*T}{S}_c\left(Z_c\right)+{ϵ}_c$$

(36)

$$\widehat{I}={\widehat{W}}_c^T{S}_c\left(Z_c\right)$$

(37)

where $Z_c=z_1$ is the Critic network input, $W_c^{*}$ is the ideal weight vector, ${\widehat{W}}_c$ is the estimated weight vector, $S_c\left(Z_c\right)$ is the basis function vector, and ${ϵ}_c$ is the approximation error.

When $\psi \to \infty$, the temporal difference error can be expressed as:

$$\gamma \left(t\right)=\varphi \left(t\right)+\dot{\widehat{I}}\left(t\right)=\varphi \left(t\right)+\nabla \widehat{I}\left(t\right){\dot{Z}}_c$$

(38)

The predefined-time Critic network weight update law is designed as:

$${\dot{\widehat{W}}}_c=-{\sigma }_c(\varphi \left(t\right)+{\widehat{W}}_c^T\Lambda )\Lambda -{\varsigma }_c{\widehat{W}}_c-c_c{\left|{\widehat{W}}_c\right|}^{\beta }{\widehat{W}}_c$$

(39)

where $\mathsf{\Lambda }=-{\displaystyle \frac{S_c}{\psi }}+\nabla S_c{\dot{Z}}_c$, ${\sigma }_c>0$ is the learning rate, ${\varsigma }_c={\displaystyle \frac{\pi ·2^{\frac{\beta }{2}}·r_c^{\frac{\beta }{2}}}{\beta T_c}}$, $c_c={\displaystyle \frac{\pi (2+\beta )}{2\beta T_c(1+\beta )r_c^{\frac{\beta }{2}}}}$ and $r_c>0$ is a design parameter.

3.2.2. Actor Network Design

The Actor network is designed to approximate the unknown function $\phi$ and generate control inputs. Using RBFNN approximation:

$$\phi =W_a^{*T}{S}_a\left(Z_a\right)+{ϵ}_a$$

(40)

where $Z_a={[x_1,x_2,y_d,{\dot{y}}_d,{\ddot{y}}_d]}^T$ is the Actor network input, $W_a^{*}$ is the ideal weight vector, $S_a\left(Z_a\right)$ is the basis function vector, and $|{ϵ}_a|\le {\overline{ϵ}}_a$.

The predefined-time Actor network weight update law is designed as:

$${\dot{\widehat{W}}}_a=r_a{z}_2{g}_1{S}_{a}tanh\left({\displaystyle \frac{z_2{g}_1\parallel S_a\parallel }{\rho }}\right)-{\varsigma }_a{\widehat{W}}_a-c_a{\left|{\widehat{W}}_a\right|}^{\beta }{\widehat{W}}_a+K_I\widehat{I}{S}_a$$

(41)

where $r_a>0$ is the learning rate, ${\varsigma }_a={\displaystyle \frac{\pi ·2^{\frac{\beta }{2}}·r_a^{\frac{\beta }{2}}}{\beta T_c}}$, $c_a={\displaystyle \frac{\pi (2+\beta )}{2\beta T_c(1+\beta )r_a^{\frac{\beta }{2}}}}$, $K_I>0$ is the Critic feedback gain and $\rho >0$ is a design parameter.

Remark 4.

The weight update laws (39) and (41) ensure both learning capability and predefined-time convergence by incorporating three essential terms: the first is the standard gradient descent term, which minimizes the approximation or temporal difference error; the second term, $-\varsigma \widehat{W}$, introduces a damping effect to prevent weight drift; and the third term, $-c|\widehat{W}{|}^{\beta }\widehat{W}$, acts as the predefined-time convergence component, guaranteeing that the weights converge within the specified time frame.

3.2.3. Predefined-Time Actual Controller

The actual control law is designed as:

$$u={\displaystyle \frac{{\alpha }_2}{g_1}}$$

(42)

where

$${\alpha }_2=-{\displaystyle \frac{z_2{\check{\alpha }}_2^2}{\sqrt{z_2^2{\check{\alpha }}_2^2+{\epsilon }_2^2}}}$$

(43)

with ${\epsilon }_2>0$ and

$$\begin{array}{cc} {\check{\alpha }}_2=\hfill & {\displaystyle \frac{\pi }{\beta T_c}}\left[2^{\beta }{\left({\displaystyle \frac{1}{2}}\right)}^{1+{\displaystyle \frac{\beta }{2}}}{\mathrm{sig}}^{1+\beta }\left(z_2\right)+{\left({\displaystyle \frac{1}{2}}\right)}^{1-{\displaystyle \frac{\beta }{2}}}{\mathrm{sig}}^{1-\beta }\left(z_2\right)\right]\hfill \end{array}$$

(44)

$$\begin{array}{c}+{\widehat{W}}_a^T{S}_{a}tanh\left({\displaystyle \frac{z_2\parallel S_a\parallel }{\rho }}\right)+K_2{z}_2+z_1\hfill \end{array}$$

(45)

where $K_2>0$ is a feedback gain.

Remark 5.

The control law defined in Equations (42)–(45) comprises three key components: a predefined-time convergence term that ensures the tracking error converges within the specified time $T_c$; a neural network compensation term, ${\widehat{W}}_a^T{S}_{a}tanh(·)$, which provides online compensation for unknown system dynamics; and stabilizing feedback terms, $K_2{z}_2+z_1$, designed to enhance closed-loop stability.

Remark 6.

The proposed Actor-Critic framework is rooted in the Adaptive Dynamic Programming (ADP) paradigm [16,17,18]. Specifically, the Critic network approximates the value function $I\left(t\right)$ associated with the Hamilton–Jacobi–Bellman equation:

$$0=\phi \left(t\right)+\nabla I^T\dot{x}$$

(46)

where $\phi \left(t\right)=z_1^{T}D{z}_1+{\tau }^{T}R\tau$ is the instantaneous cost that penalizes both tracking error and control effort. The TD error $\gamma \left(t\right)=\phi \left(t\right)+\dot{\widehat{I}}\left(t\right)$ measures the discrepancy between the current value estimate and the Bellman optimality condition. Minimizing ${\gamma }^2$ drives the Critic toward the true value function.

The term $K_I\widehat{I}{S}_a$ in the Actor update law (41) can be interpreted as an approximate policy gradient step: it adjusts the Actor weights in a direction that reduces the estimated long-term cost $\widehat{I}$, analogous to the policy improvement step in policy iteration methods. Together with the error-driven gradient term $r_a{z}_2{g}_1{S}_{a}tanh(·)$, the Actor update simultaneously ensures Lyapunov stability (via error reduction) and approximate optimality (via cost minimization).

It should be noted that due to the integration with predefined-time stability requirements, the damping term $-{\varsigma }_a{\widehat{W}}_a$ and the predefined-time term $-c_a{\left|{\widehat{W}}_a\right|}^{\beta }{\widehat{W}}_a$ modify the pure policy gradient direction. Therefore, the optimality guarantee is approximate rather than exact, representing a meaningful design trade-off between guaranteed predefined-time convergence and strict optimality. This is consistent with the ADP literature where stability-constrained policy optimization yields near-optimal rather than globally optimal policies.

4. Stability Analysis

Theorem 1.

Consider the single-link manipulator system (2) satisfying Assumption 1. Under the virtual controller (26), the actual controller (42), and the Actor-Critic neural network weight update laws (39) and (41), if the design parameters satisfy:

• $0<\beta <1$, $T_c>0$
• ${\epsilon }_1,{\epsilon }_2,\rho >0$
• ${\sigma }_c,r_a,K_I,K_2>0$

then the closed-loop system is practically predefined-time stable (PPTS). Specifically:

(i) 

The error signals $\mathit{\kappa }={[z_1,z_2,{\tilde{W}}_a,{\tilde{W}}_c]}^T$ converge to a compact set within the predefined time $T_P<T_{max}=\sqrt{2}{T}_c$.

(ii) 

All signals in the closed-loop system remain bounded.

(iii) 

The convergence region is given by:

$$\Delta =\left\{\underset{t\to T_P}{lim}\mathit{\kappa }\mid V\le min\left\{{\left({\displaystyle \frac{2\beta T_c\sigma }{\pi }}\right)}^{{\displaystyle \frac{2}{2-\beta }}},{\left({\displaystyle \frac{2\beta T_c\sigma }{\pi }}\right)}^{{\displaystyle \frac{2}{2+\beta }}}\right\}\right\}$$

(47)

Proof. 

Consider the following Lyapunov function candidate:

$$V={\displaystyle \frac{1}{2}}{z}_1^2+{\displaystyle \frac{1}{2}}{z}_2^2+{\displaystyle \frac{1}{2r_a}}{\tilde{W}}_a^T{\tilde{W}}_a+{\displaystyle \frac{1}{2r_c}}{\tilde{W}}_c^T{\tilde{W}}_c$$

(48)

where ${\tilde{W}}_a=W_a^{*}-{\widehat{W}}_a$ and ${\tilde{W}}_c=W_c^{*}-{\widehat{W}}_c$ are the weight estimation errors.

From (31), we have:

$${\dot{V}}_1<-{\displaystyle \frac{\pi }{\beta T_c}}{\left({\displaystyle \frac{z_1^2}{2}}\right)}^{1-{\displaystyle \frac{\beta }{2}}}-{\displaystyle \frac{\pi }{\beta T_c}}{2}^{\beta }{\left({\displaystyle \frac{z_1^2}{2}}\right)}^{1+{\displaystyle \frac{\beta }{2}}}+z_1{z}_2+{\epsilon }_1$$

(49)

Taking the derivative of $V_2={\displaystyle \frac{1}{2}}{z}_2^2$:

$${\dot{V}}_2=z_2{\dot{z}}_2=z_2(f-{\dot{\alpha }}_1+g_{1}u)=z_2(\phi -z_1-{\displaystyle \frac{1}{2}}{z}_2+g_{1}u)$$

(50)

Using the neural network approximation (40):

$$z_2\phi =z_2{W}_a^{*T}{S}_a+z_2{ϵ}_a$$

(51)

Applying Lemma 1:

$$z_2{g}_1{W}_a^{*T}{S}_a\le |z_2|g_1\parallel W_a^{*}\parallel \parallel S_a\parallel \le g_1\parallel W_a^{*}\parallel \left(\delta \rho +z_2\parallel S_a\parallel tanh\left({\displaystyle \frac{z_2{g}_1\parallel S_a\parallel }{\rho }}\right)\right)$$

(52)

Define ${\theta }_a={\displaystyle \frac{\parallel W_a^{*}\parallel }{g_1}}$. Substituting the control law (42):

$$z_2{g}_{1}u=z_2{\alpha }_2=-{\displaystyle \frac{z_2^2{\check{\alpha }}_2^2}{\sqrt{z_2^2{\check{\alpha }}_2^2+{\epsilon }_2^2}}}$$

(53)

By Lemma 5:

$$z_2{g}_{1}u<{\epsilon }_2-\left|z_2{\check{\alpha }}_2\right|$$

(54)

Expanding ${\check{\alpha }}_2$ and combining terms:

$$\begin{array}{cc}\hfill {\dot{V}}_2& <-{\displaystyle \frac{\pi }{\beta T_c}}{\left({\displaystyle \frac{z_2^2}{2}}\right)}^{1-{\displaystyle \frac{\beta }{2}}}-{\displaystyle \frac{\pi }{\beta T_c}}{2}^{\beta }{\left({\displaystyle \frac{z_2^2}{2}}\right)}^{1+{\displaystyle \frac{\beta }{2}}}-z_1{z}_2\hfill \\ \hfill & +z_2{g}_1{\tilde{W}}_a^T{S}_{a}tanh\left({\displaystyle \frac{z_2{g}_1\parallel S_a\parallel }{\rho }}\right)+{\sigma }_2\hfill \end{array}$$

(55)

where ${\sigma }_2={\epsilon }_2+\delta {\theta }_a{g}_1\rho +{\displaystyle \frac{1}{2}}{\overline{ϵ}}_a^2$.

Taking the derivative of $V_a={\displaystyle \frac{1}{2r_a}}{\tilde{W}}_a^T{\tilde{W}}_a$:

$${\dot{V}}_a=-{\displaystyle \frac{1}{r_a}}{\tilde{W}}_a^T{\dot{\widehat{W}}}_a$$

(56)

Substituting the Actor weight update law (41):

$$\begin{array}{cc}\hfill {\dot{V}}_a& =-z_2{g}_1{\tilde{W}}_a^T{S}_{a}tanh\left({\displaystyle \frac{z_2{g}_1\parallel S_a\parallel }{\rho }}\right)+{\displaystyle \frac{{\varsigma }_a}{r_a}}{\tilde{W}}_a^T{\widehat{W}}_a+{\displaystyle \frac{c_a}{r_a}}{\tilde{W}}_a^T{\left|{\widehat{W}}_a\right|}^{\beta }{\widehat{W}}_a-{\displaystyle \frac{K_I}{r_a}}{\tilde{W}}_a^T\widehat{I}{S}_a\hfill \end{array}$$

(57)

Using Young’s inequality for ${\tilde{W}}_a^T{\widehat{W}}_a={\tilde{W}}_a^T(W_a^{*}-{\tilde{W}}_a)$:

$${\tilde{W}}_a^T{\widehat{W}}_a\le -{\displaystyle \frac{1}{2}}\parallel {\tilde{W}}_a{\parallel }^2+{\displaystyle \frac{1}{2}}{\parallel W_a^{*}\parallel }^2$$

(58)

Using Lemma 3 for ${\tilde{W}}_a^T{\left|{\widehat{W}}_a\right|}^{\beta }{\widehat{W}}_a$:

$${\tilde{W}}_a^T|{\widehat{W}}_a{|}^{\beta }{\widehat{W}}_a\le {\displaystyle \frac{1+\beta }{2+\beta }}\parallel W_a^{*}{\parallel }^{2+\beta }-{\displaystyle \frac{1+\beta }{2+\beta }}{\parallel {\tilde{W}}_a\parallel }^{2+\beta }$$

(59)

Therefore:

$${\dot{V}}_a\le -z_2{g}_1{\tilde{W}}_a^T{S}_{a}tanh(·)-{\displaystyle \frac{{\varsigma }_a}{2r_a}}\parallel {\tilde{W}}_a{\parallel }^2-{\displaystyle \frac{c_a(1+\beta )}{r_a(2+\beta )}}{\parallel {\tilde{W}}_a\parallel }^{2+\beta }+{\sigma }_a$$

(60)

From the definition $V_a={\displaystyle \frac{1}{2r_a}}{\parallel {\tilde{W}}_a\parallel }^2$, we have:

$$\parallel {\tilde{W}}_a{\parallel }^{2+\beta }={\left(2r_a{V}_a\right)}^{1+{\displaystyle \frac{\beta }{2}}}={\left(2r_a\right)}^{1+{\displaystyle \frac{\beta }{2}}}{V}_a^{1+{\displaystyle \frac{\beta }{2}}}$$

(61)

Substituting into the third term of (60):

$$\begin{array}{cc}-{\displaystyle \frac{c_a(1+\beta )}{r_a(2+\beta )}}{\parallel {\tilde{W}}_a\parallel }^{2+\beta }& =-{\displaystyle \frac{c_a(1+\beta )}{r_a(2+\beta )}}{\left(2r_a\right)}^{1+{\displaystyle \frac{\beta }{2}}}{V}_a^{1+{\displaystyle \frac{\beta }{2}}}\hfill \\ \hfill & =-{\displaystyle \frac{c_a(1+\beta )·2^{1+\frac{\beta }{2}}·r_a^{\frac{\beta }{2}}}{2+\beta }}{V}_a^{1+{\displaystyle \frac{\beta }{2}}}\hfill \end{array}$$

(62)

To achieve the target form $-{\displaystyle \frac{\pi }{\beta T_c}}{2}^{{\displaystyle \frac{\beta }{2}}}{V}_a^{1+{\displaystyle \frac{\beta }{2}}}$, we require:

$${\displaystyle \frac{c_a(1+\beta )·2^{1+\frac{\beta }{2}}·r_a^{\frac{\beta }{2}}}{2+\beta }}={\displaystyle \frac{\pi }{\beta T_c}}{2}^{{\displaystyle \frac{\beta }{2}}}$$

(63)

Solving for $c_a$:

$$c_a={\displaystyle \frac{\pi (2+\beta )}{2\beta T_c(1+\beta )r_a^{\frac{\beta }{2}}}}$$

(64)

With this choice of $c_a$, we obtain:

$$-{\displaystyle \frac{c_a(1+\beta )}{r_a(2+\beta )}}{\parallel {\tilde{W}}_a\parallel }^{2+\beta }=-{\displaystyle \frac{\pi }{\beta T_c}}{2}^{{\displaystyle \frac{\beta }{2}}}{V}_a^{1+{\displaystyle \frac{\beta }{2}}}$$

(65)

From the definition $V_a={\displaystyle \frac{1}{2r_a}}{\parallel {\tilde{W}}_a\parallel }^2$, we have:

$$\parallel {\tilde{W}}_a{\parallel }^{2-\beta }={\left(2r_a{V}_a\right)}^{1-{\displaystyle \frac{\beta }{2}}}={\left(2r_a\right)}^{1-{\displaystyle \frac{\beta }{2}}}{V}_a^{1-{\displaystyle \frac{\beta }{2}}}$$

(66)

Applying Lemma 6 with $x=\parallel {\tilde{W}}_a\parallel$:

$$\parallel {\tilde{W}}_a{\parallel }^2\ge {\parallel {\tilde{W}}_a\parallel }^{2-\beta }-C_{\beta }$$

(67)

Multiplying both sides by ${\displaystyle \frac{{\varsigma }_a}{2r_a}}$:

$${\displaystyle \frac{{\varsigma }_a}{2r_a}}\parallel {\tilde{W}}_a{\parallel }^2\ge {\displaystyle \frac{{\varsigma }_a}{2r_a}}{\parallel {\tilde{W}}_a\parallel }^{2-\beta }-{\displaystyle \frac{{\varsigma }_a{C}_{\beta }}{2r_a}}$$

(68)

Substituting (66):

$$\begin{array}{cc}\hfill {\displaystyle \frac{{\varsigma }_a}{2r_a}}{\parallel {\tilde{W}}_a\parallel }^2& \ge {\displaystyle \frac{{\varsigma }_a}{2r_a}}{\left(2r_a\right)}^{1-{\displaystyle \frac{\beta }{2}}}{V}_a^{1-{\displaystyle \frac{\beta }{2}}}-{\displaystyle \frac{{\varsigma }_a{C}_{\beta }}{2r_a}}\hfill \\ \hfill & ={\displaystyle \frac{{\varsigma }_a{\left(2r_a\right)}^{1-\frac{\beta }{2}}}{2r_a}}{V}_a^{1-{\displaystyle \frac{\beta }{2}}}-{\sigma }_{a1}\hfill \\ \hfill & ={\displaystyle \frac{{\varsigma }_a}{{\left(2r_a\right)}^{\frac{\beta }{2}}}}{V}_a^{1-{\displaystyle \frac{\beta }{2}}}-{\sigma }_{a1}\hfill \end{array}$$

(69)

where ${\sigma }_{a1}={\displaystyle \frac{{\varsigma }_a{C}_{\beta }}{2r_a}}$ is a bounded positive constant.

Therefore:

$$-{\displaystyle \frac{{\varsigma }_a}{2r_a}}{\parallel {\tilde{W}}_a\parallel }^2\le -{\displaystyle \frac{{\varsigma }_a}{{\left(2r_a\right)}^{\frac{\beta }{2}}}}{V}_a^{1-{\displaystyle \frac{\beta }{2}}}+{\sigma }_{a1}$$

(70)

To achieve the target form $-{\displaystyle \frac{\pi }{\beta T_c}}{V}_a^{1-{\displaystyle \frac{\beta }{2}}}$, we require:

$${\displaystyle \frac{{\varsigma }_a}{{\left(2r_a\right)}^{\frac{\beta }{2}}}}={\displaystyle \frac{\pi }{\beta T_c}}$$

(71)

Solving for ${\varsigma }_a$:

$${\varsigma }_a={\displaystyle \frac{\pi {\left(2r_a\right)}^{\frac{\beta }{2}}}{\beta T_c}}={\displaystyle \frac{\pi ·2^{\frac{\beta }{2}}·r_a^{\frac{\beta }{2}}}{\beta T_c}}$$

(72)

With this choice of ${\varsigma }_a$, we obtain:

$$-{\displaystyle \frac{{\varsigma }_a}{2r_a}}{\parallel {\tilde{W}}_a\parallel }^2\le -{\displaystyle \frac{\pi }{\beta T_c}}{V}_a^{1-{\displaystyle \frac{\beta }{2}}}+{\sigma }_{a1}$$

(73)

Therefore, substituting (73) and (65) into (60):

$${\dot{V}}_a\le -z_2{g}_1{\tilde{W}}_a^T{S}_{a}tanh(·)-{\displaystyle \frac{\pi }{\beta T_c}}{V}_a^{1-{\displaystyle \frac{\beta }{2}}}-{\displaystyle \frac{\pi }{\beta T_c}}{2}^{{\displaystyle \frac{\beta }{2}}}{V}_a^{1+{\displaystyle \frac{\beta }{2}}}+{\sigma }_a\prime$$

(74)

where ${\sigma }_a\prime ={\sigma }_a+{\sigma }_{a1}$ is a bounded positive constant.

By the definitions of ${\varsigma }_a$ and $c_a$, and applying Lemma 6:

$${\dot{V}}_a\le -z_2{g}_1{\tilde{W}}_a^T{S}_{a}tanh(·)-{\displaystyle \frac{\pi }{\beta T_c}}{\left({\displaystyle \frac{\parallel {\tilde{W}}_a{\parallel }^2}{2r_a}}\right)}^{1-{\displaystyle \frac{\beta }{2}}}-{\displaystyle \frac{\pi }{\beta T_c}}{2}^{{\displaystyle \frac{\beta }{2}}}{\left({\displaystyle \frac{\parallel {\tilde{W}}_a{\parallel }^2}{2r_a}}\right)}^{1+{\displaystyle \frac{\beta }{2}}}+{\sigma }_a\prime$$

(75)

From ${\dot{V}}_2$, the cross term involving the Actor network is $z_2{g}_1{\tilde{W}}_a^T{S}_{a}tanh\left({\displaystyle \frac{z_2{g}_1\parallel S_a\parallel }{\rho }}\right)$, which arises because the control law $u={\alpha }_2/g_1$ yields $z_2{g}_{1}u=z_2{\alpha }_2$ and the neural network compensation term in ${\alpha }_2$ contains ${\widehat{W}}_a^T{S}_{a}tanh(·)$. From ${\dot{V}}_a=-{\displaystyle \frac{1}{r_a}}{\tilde{W}}_a^T{\dot{\widehat{W}}}_a$, substituting the Actor update law (41), the first term is $-z_2{g}_1{\tilde{W}}_a^T{S}_{a}tanh\left({\displaystyle \frac{z_2{g}_1\parallel S_a\parallel }{\rho }}\right)$, these two terms cancel exactly for any $g_1>0$.

Similarly, for the Critic network:

$${\dot{V}}_c\le -{\displaystyle \frac{\pi }{\beta T_c}}{\left({\displaystyle \frac{\parallel {\tilde{W}}_c{\parallel }^2}{2r_c}}\right)}^{1-{\displaystyle \frac{\beta }{2}}}-{\displaystyle \frac{\pi }{\beta T_c}}{2}^{{\displaystyle \frac{\beta }{2}}}{\left({\displaystyle \frac{\parallel {\tilde{W}}_c{\parallel }^2}{2r_c}}\right)}^{1+{\displaystyle \frac{\beta }{2}}}+{\sigma }_c\prime$$

(76)

where ${\sigma }_c\prime ={\sigma }_c+{\sigma }_{c1}$ is a bounded positive constant.

Combining all terms:

$$\dot{V}={\dot{V}}_1+{\dot{V}}_2+{\dot{V}}_a+{\dot{V}}_c$$

(77)

Note that the cross terms cancel:

• $z_1{z}_2$ from ${\dot{V}}_1$ cancels with $-z_1{z}_2$ from ${\dot{V}}_2$.
• $z_2{g}_1{\tilde{W}}_a^T{S}_{a}tanh(·)$ from ${\dot{V}}_2$ cancels exactly with $-z_2{g}_1{\tilde{W}}_a^T{S}_{a}tanh(·)$ from ${\dot{V}}_a$, since the Actor weight update law (41) explicitly includes the factor $g_1$ in the gradient term, and the control law $u={\alpha }_2/g_1$ ensures that $z_2{g}_{1}u=z_2{\alpha }_2$. This exact cancellation holds for any $g_1=1/I>0$ without requiring any approximation.

Therefore:

$$\begin{array}{cc}\hfill \dot{V}& <-{\displaystyle \frac{\pi }{\beta T_c}}\left[{\left({\displaystyle \frac{z_1^2}{2}}\right)}^{1-{\displaystyle \frac{\beta }{2}}}+{\left({\displaystyle \frac{z_2^2}{2}}\right)}^{1-{\displaystyle \frac{\beta }{2}}}+{\left({\displaystyle \frac{\parallel {\tilde{W}}_a{\parallel }^2}{2r_a}}\right)}^{1-{\displaystyle \frac{\beta }{2}}}+{\left({\displaystyle \frac{\parallel {\tilde{W}}_c{\parallel }^2}{2r_c}}\right)}^{1-{\displaystyle \frac{\beta }{2}}}\right]\hfill \\ \hfill & -{\displaystyle \frac{\pi }{\beta T_c}}{2}^{{\displaystyle \frac{\beta }{2}}}\left[{\left({\displaystyle \frac{z_1^2}{2}}\right)}^{1+{\displaystyle \frac{\beta }{2}}}+{\left({\displaystyle \frac{z_2^2}{2}}\right)}^{1+{\displaystyle \frac{\beta }{2}}}+{\left({\displaystyle \frac{\parallel {\tilde{W}}_a{\parallel }^2}{2r_a}}\right)}^{1+{\displaystyle \frac{\beta }{2}}}+{\left({\displaystyle \frac{\parallel {\tilde{W}}_c{\parallel }^2}{2r_c}}\right)}^{1+{\displaystyle \frac{\beta }{2}}}\right]+\sigma \hfill \end{array}$$

(78)

where $\sigma ={\epsilon }_1+{\sigma }_2+{\sigma }_a\prime +{\sigma }_c\prime$ is a positive constant.

Applying Lemma 2, for $0<1-{\displaystyle \frac{\beta }{2}}<1$:

$$\sum _i{\left({\displaystyle \frac{x_i^2}{2}}\right)}^{1-{\displaystyle \frac{\beta }{2}}}\ge {\left(\sum _i{\displaystyle \frac{x_i^2}{2}}\right)}^{1-{\displaystyle \frac{\beta }{2}}}=V^{1-{\displaystyle \frac{\beta }{2}}}$$

(79)

for $1+{\displaystyle \frac{\beta }{2}}>1$:

$$\sum _i{\left({\displaystyle \frac{x_i^2}{2}}\right)}^{1+{\displaystyle \frac{\beta }{2}}}\ge 4^{-{\displaystyle \frac{\beta }{2}}}{V}^{1+{\displaystyle \frac{\beta }{2}}}$$

(80)

Therefore:

$$\dot{V}\le -{\displaystyle \frac{\pi }{\beta T_c}}\left(V^{1-{\displaystyle \frac{\beta }{2}}}+V^{1+{\displaystyle \frac{\beta }{2}}}\right)+\sigma$$

(81)

By Lemma 4, the system is practically predefined-time stable with settling time $T_P<T_{max}=\sqrt{2}{T}_c$.

From the predefined-time stability, $z_1$, $z_2$, ${\tilde{W}}_a$, ${\tilde{W}}_c$ are all bounded.

This completes the proof. □

Remark 7.

By adjusting the predefined time parameter $T_c$, the upper bound of the settling time can be explicitly preset: $T_P<T_{max}=\sqrt{2}{T}_c$. A smaller $T_c$ leads to faster convergence but may require larger control efforts.

Remark 8.

The predefined-time parameters for both Actor and Critic networks are derived from the requirement that the weight estimation error dynamics satisfy the predefined-time stability condition in Lemma 4. The key insight is:

• The damping term $-\varsigma \widehat{W}$ generates the $V^{1-{\displaystyle \frac{\beta }{2}}}$ component through Lemma 6, which dominates when V is large.
• The predefined-time term $-c|\widehat{W}{|}^{\beta }\widehat{W}$ directly generates the $V^{1+{\displaystyle \frac{\beta }{2}}}$ component through algebraic substitution, which dominates when V is small.
• The combination of both terms ensures predefined-time convergence for all values of $V>0$.

5. Simulation Results

In this section, numerical simulations are conducted to verify the effectiveness of the proposed Actor-Critic predefined-time control scheme. The simulations are performed on a single-link manipulator system using MATLAB R2025a with Runge–Kutta 4th order integration.

5.1. Simulation Setup

The initial conditions are set as $x_1\left(0\right)=1.0$ rad and $x_2\left(0\right)=-1.0$ rad/s. The simulation runs for 20 s with a step size of $\Delta t=0.001$ s. For the Actor network with 100 nodes ($l_a=100$) processing 5-dimensional input $[x_1,x_2,y_d,{\dot{y}}_d,{\ddot{y}}_d]$, the basis centers are uniformly sampled from the hypercube ${[-3,3]}^5$ with width parameter ${\eta }_a=1.2$. All weights are initialized to zero, ${\widehat{W}}_a\left(0\right)=\mathbf{0}\in {\mathbb{R}}^{100}$, and bounded by $|{\widehat{W}}_{a,i}|\le 200$ via saturation clipping. The Critic network uses 64 nodes ($l_c=64$) with 2-dimensional input $[z_1,z_2]$. Centers are placed on an $8\times 8$ uniform grid over ${[-2,2]}^2$ with width ${\eta }_c=1.0$. Weights are similarly initialized as ${\widehat{W}}_c\left(0\right)=\mathbf{0}\in {\mathbb{R}}^{64}$ and bounded by $|{\widehat{W}}_{c,i}|\le 100$. Regarding the discount factor, we implement $\psi =10$ rather than the theoretical limit $\psi \to \infty$. This is a standard simplification in ADP literature [18], where using a sufficiently large $\psi$ makes the term $-S_c/\psi$ negligible compared to $\nabla S_c·Z_c$, effectively approximating the infinite discount case while maintaining numerical stability. The system and control parameters are given in

Table 1. System and Control Parameters.

Parameter	Description	Value	Unit
System Parameters
m	Link mass	1.0	kg
l	Link length	0.5	m
$c_0$	Friction coefficient	1.0	N·m·s/rad
g	Gravitational acceleration	9.8	m/s2
$d\left(t\right)$	External disturbance	$0.5sin\left(t\right)+0.3cos\left(2t\right)$	N·m
$y_d\left(t\right)$	Reference trajectory	$0.5sin\left(t\right)$	rad
Predefined-Time Parameters
$T_c$	Predefined time parameter	2.0	s
$T_{max}$	Maximum settling time	$\sqrt{2}{T}_c=2.83$	s
$\beta$	Convergence parameter	0.6	-
Controller Parameters
$K_2$	Feedback gain	100	-
${\epsilon }_1,{\epsilon }_2$	Small constants	${10}^{-4}$	-
$\rho$	Smoothing parameter	0.05	-
Neural Network Parameters
$l_a$	Actor network nodes	100	-
$l_c$	Critic network nodes	64	-
${\sigma }_a,{\sigma }_c$	Learning rates	100, 50	-
$K_I$	Critic feedback gain	2.0	-
${\eta }_a$, ${\eta }_c$	RBF widths	1.2, 1.0	-
$\psi$	Discount factor	10	-
$W_a^{max}$, $W_c^{max}$	Weight bounds	200, 100	-
PID Controller
$K_p,K_d,K_i$	PID gains	25, 12, 5	-

.

To verify that the performance is not an artifact of a specific initial condition, we additionally conducted 20 independent Monte Carlo simulations with randomized initial conditions uniformly drawn from $x_1\left(0\right)\in [0.5,1.5]$ rad and $x_2\left(0\right)\in [-1.5,-0.5]$ rad/s. The statistical results are reported in

Table 2. Statistical Performance over 20 Monte Carlo Runs (Mean ± Std).

Performance Metric	AC-PT	PID	Improvement
Total RMSE (rad)	$0.0417\pm 0.0129$	$0.1266\pm 0.0275$	67.0%
SS RMSE (rad)	$0.0014\pm 0.0004$	$0.0454\pm 0.0042$	96.9%
Max SS Error (rad)	$0.0030\pm 0.0009$	$0.1229\pm 0.0108$	97.5%
Settling Time (s)	$0.2031\pm 0.0068$	$13.824\pm 0.063$	98.5%
$T_P<T_{max}$ Satisfied	20/20 (100%)	N/A	—

N/A: Since the PID controller’s response is slow and does not meet the basic premise for evaluating this time constraint, this metric is not applicable.

.

5.2. Tracking Performance Analysis

Figure 2 compares the tracking performance of the proposed AC-PT controller and the conventional PID controller. As shown in Figure 2a, both controllers track the reference trajectory $y_d=0.5sin\left(t\right)$, but the AC-PT controller achieves stabilization within approximately 0.23 s, well within the theoretical upper bound $T_{max}=2.83$ s. In contrast, the PID controller requires approximately 13.84 s to reach the $\pm 0.01$ rad tolerance band (Figure 2b). The zoomed steady-state view in Figure 2c confirms that the AC-PT controller maintains the tracking error consistently within the specified tolerance, whereas the PID controller exhibits noticeable residual oscillations. The logarithmic-scale convergence plot in Figure 2d further illustrates the characteristic rapid error decay before $T_{max}$, corroborating the predefined-time stability guarantee of Theorem 1. The quantitative comparison is summarized in

Table 3. Performance Comparison: AC-PT vs PID Control ($T_c=2.0$ s). Single-run results with baseline initial condition $x_1\left(0\right)=1.0$ rad, $x_2\left(0\right)=-1.0$ rad/s.

Performance Metric	AC-PT	PID	Improvement
Total RMSE (rad)	0.0467	0.1333	65.0%
Steady-State RMSE (rad)	0.0014	0.0465	96.9%
Max Steady-State Error (rad)	0.0037	0.1259	97.1%
Settling Time to $\pm 0.01$ rad (s)	0.229	13.841	98.3%
Time Within $\pm 0.01$ rad (%)	100.0	12.2	-

: the AC-PT controller achieves 96.9% reduction in steady-state RMSE and 98.3% reduction in settling time compared to PID control.

5.3. Neural Network Learning Process

The online learning behavior of the Actor-Critic neural networks is shown in Figure 3. Both the Actor and Critic weight norms (Figure 3a,b) converge to bounded values and remain stable throughout the simulation, confirming that the predefined-time weight update laws incorporating the $|\widehat{W}{|}^{\beta }\widehat{W}$ terms successfully prevent weight divergence. The adaptive parameter $\widehat{\theta }$ (Figure 3c) increases during the transient phase to compensate for system uncertainties and subsequently stabilizes as the tracking error diminishes. Figure 3d shows that both the cost-to-go estimation $\widehat{I}$ and the instantaneous cost $\varphi$ decrease rapidly during the initial phase, indicating that the Actor-Critic framework effectively optimizes the control policy while compensating for unknown system dynamics.

5.4. Effect of Predefined Time Parameter

The influence of the predefined time parameter $T_c$ on control performance is investigated through simulations with $T_c\in \{1.5,2.0,3.0\}$ s, as shown in Figure 4. Smaller $T_c$ values lead to faster error convergence (Figure 4a), with the system converging before $T_{max}=2.12$ s for $T_c=1.5$ s. However, this faster convergence comes at the cost of larger initial control effort (Figure 4b), presenting a trade-off between convergence speed and actuator requirements. Figure 4c demonstrates that all tested $T_c$ values achieve comparable steady-state accuracy, indicating that $T_c$ primarily governs the transient response rather than the ultimate tracking precision. The Lyapunov function evolution in Figure 4d confirms that $V\left(t\right)$ decreases below its corresponding $T_{max}$ bound in all cases, thereby validating the predefined-time stability theory of Theorem 1 across different parameter settings.

5.5. Comparison with State-of-the-Art Methods

To further substantiate the contributions, the proposed AC-PT controller is compared with two representative methods from the literature: a disturbance-observer-based fixed-time sliding mode controller (FxT-SMC) based on [26], and a predefined-time robust controller without neural networks (PT-Robust) based on [24]. The tracking error comparison in Figure 5a shows that all three advanced controllers significantly outperform PID, with AC-PT and PT-Robust achieving comparable transient performance. The steady-state error detail in Figure 5c reveals that AC-PT achieves the smallest residual error among all methods. The quantitative results demonstrate that the proposed AC-PT method provides competitive convergence speed while offering two key advantages: online learning capability for unknown dynamics compensation (absent in PT-Robust) and an explicit, user-tunable settling time bound $T_P<\sqrt{2}{T}_c$ (which FxT-SMC cannot directly prescribe).

5.6. Robustness Evaluation

To evaluate the robustness of the proposed AC-PT controller under model uncertainties, we conduct simulations under three categories of perturbations: (i) mass uncertainty (m varied by $\pm 30\%$), (ii) friction coefficient uncertainty ($c_0$ varied by $\pm 50\%$), and (iii) increased external disturbance (amplitude scaled to $2\times$, $3\times$, and $5\times$ nominal). All tests use the baseline initial condition $x_1\left(0\right)=1.0$ rad, $x_2\left(0\right)=-1.0$ rad/s with all controller parameters unchanged from Table 1.

The results are summarized in

Table 4. Robustness Evaluation under Parameter Perturbations ($T_c=2.0$ s, $T_{max}=2.83$ s).

Scenario	Total RMSE	SS RMSE	Settling Time	${\mathit{T}}_{\mathit{P}}<{\mathit{T}}_{max}$
	(rad)	(rad)	(s)	Satisfied
Nominal ($m=1.0$, $c_0=1.0$)	0.0450	0.0015	0.209	Yes
Mass Uncertainty
$m=0.7$ kg ($-30\%$)	0.0429	0.0015	0.207	Yes
$m=1.3$ kg ($+30\%$)	0.0469	0.0016	0.212	Yes
Friction Uncertainty
$c_0=0.5$ ($-50\%$)	0.0448	0.0015	0.208	Yes
$c_0=1.5$ ($+50\%$)	0.0453	0.0015	0.210	Yes
Increased Disturbance
$2\times$ disturbance	0.0450	0.0015	0.209	Yes
$3\times$ disturbance	0.0450	0.0015	0.209	Yes
$5\times$ disturbance	0.0450	0.0015	0.209	Yes

. The AC-PT controller satisfies the predefined-time guarantee $T_P<T_{max}=2.83$ s in all tested scenarios without any parameter re-tuning. The settling time remains within the narrow range $[0.207,0.212]$ s, and the steady-state RMSE is maintained at approximately $0.0015$ rad across all cases.

This strong invariance is theoretically grounded: the predefined-time convergence rate in Lemma 4 depends on the control gains $K_{1,\mathrm{pt}}$, $K_{2,\mathrm{pt}}$ and the design parameter $T_c$, which are independent of the physical parameters. The adaptive parameter $\widehat{\theta }$ and the Actor neural network compensate for the parametric variations and disturbance changes online, as predicted by Theorem 1. The representative tracking error trajectories in Figure 6 confirm that the convergence behavior is qualitatively preserved under all perturbation conditions.

6. Conclusions

This paper has presented a predefined-time adaptive neural tracking control framework for uncertain single-link manipulator systems, integrating predefined-time stability theory with an Actor-Critic reinforcement learning architecture. The main contribution lies in the synergistic design where the predefined-time convergence mechanism is incorporated into both the control law and the neural network weight update laws, enabling a single parameter $T_c$ to explicitly prescribe the settling time upper bound as $T_P<\sqrt{2}{T}_c$, independent of initial conditions and system parameters.

The current work has several limitations that motivate future research. First, the single-link manipulator setting does not capture the coupling effects present in multi-DOF systems; extending the framework to multi-link and redundant manipulators with inter-joint coupling is a natural next step. Second, the current validation is simulation-based; experimental validation on physical robot platforms is essential to assess real-world applicability. Additional future directions include: incorporating input saturation constraints and actuator dynamics; developing event-triggered implementations to reduce computational and communication overhead.