Robust Design of Two-Level Non-Integer SMC Based on Deep Soft Actor-Critic for Synchronization of Chaotic Fractional Order Memristive Neural Networks

Abstract

In this study, a model-free  $P{I}^{\phi }$-sliding mode control ( $P{I}^{\phi }$-SMC) methodology is proposed to synchronize a specific class of chaotic fractional-order memristive neural network systems (FOMNNSs) with delays and input saturation. The fractional-order Lyapunov stability theory is used to design a two-level  $P{I}^{\phi }$-SMC which can effectively manage the inherent chaotic behavior of delayed FOMNNSs and achieve finite-time synchronization. At the outset, an initial sliding surface is introduced. Subsequently, a robust  $P{I}^{\phi }$-sliding surface is designed as a second sliding surface, based on proportional–integral (PI) rules. The finite-time asymptotic stability of both surfaces is demonstrated. The final step involves the design of a dynamic-free control law that is robust against system uncertainties, input saturations, and delays. The independence of control rules from the functions of the system is accomplished through the application of the norm-boundedness property inherent in chaotic system states. The soft actor-critic (SAC) algorithm based deep Q-Learning is utilized to optimally adjust the coefficients embedded in the two-level  $P{I}^{\phi }$-SMC controller’s structure. By maximizing a reward signal, the optimal policy is found by the deep neural network of the SAC agent. This approach ensures that the sliding motion meets the reachability condition within a finite time. The validity of the proposed protocol is subsequently demonstrated through extensive simulation results and two numerical examples.

1. Introduction

Delayed neural networks (DNNs) represent an approach to designing networks specifically tailored for tasks where the order and timing of information play a crucial role. Unlike networks that process data in a linear fashion, DNNs incorporate deliberate delays in their connections. This feature enables the network to consider occurrences over timeframes, enhancing its ability to detect evolving patterns. For instance, when forecasting the word in a sentence or predicting stock values, DNNs shine by leveraging and recalling the timing of inputs. This capacity to analyze and understand sequences positions DNNs as tools in areas such as speech recognition, financial prediction, and any field where grasping the sequence of events is vital. By capturing the subtleties of dynamics, DNNs provide an advanced method for modeling and forecasting time-related phenomena [1,2,3].

Delayed fractional-order memristive neural network systems (FOMNNSs) are advanced neural networks that integrate fractional calculus, memristive components, and time delays to enhance their processing capabilities [4]. The fractional-order aspect allows these networks to model more complex dynamics with memory effects, while the memristive elements, which remember past voltages, provide non-volatile storage and adaptive learning capabilities [5]. This combination makes these systems highly suitable for applications requiring sophisticated temporal pattern recognition and adaptive learning, such as advanced signal processing, adaptive control, and complex system modeling [6,7,8,9].

Time delays are introduced to manage and utilize temporal dependencies in the data [10,11]. In delayed FOMNNSs, control and synchronization are imperative for ensuring stability, accuracy, and operational efficiency. The integration of complex dynamics, memory effects, and time delays necessitates precise control mechanisms to prevent erratic behavior and ensure reliable performance. Synchronization enhances the coordination of interconnected subsystems, thereby facilitating accurate information processing, which is critical for real-time applications. Furthermore, effective control and synchronization optimize resource utilization, enhance robustness to environmental changes, and support coherent adaptive learning [3]. These contributions collectively ensure that the network remains reliable and effective in handling complex, dynamic tasks. Applications benefiting from these controlled and synchronized systems include advanced signal processing, adaptive control systems, and real-time data analysis, where precise timing and coordination are crucial for optimal performance [12,13,14].

Various control techniques have been devised to mitigate undesirable behaviors in FO nonlinear systems. These methods include the fuzzy technique [15,16], Proportional–Integral–Derivative technique [17], adaptive technique [18], backstepping technique [19], sliding mode technique [20], optimal technique [21], and so on. Sliding mode control (SMC) has become popular among these techniques because of its precision, simplicity of execution, robustness against uncertainties, and adaptability. The PID control with SMC, known as PID-SMC, offers significant benefits including high precision, robust performance under system uncertainties, and ease of implementation [22]. It enhances stability and provides a fast response to dynamic changes, while mitigating the chattering issue often associated with traditional SMC. This combined approach results in improved control accuracy, adaptability, and overall performance, making it a versatile solution for managing complex systems effectively. For instance, in [1], a sliding mode control method is investigated for uncertain delayed fractional-order reaction–diffusion memristor neural networks. Unlike most existing literature, this study constructs a linear sliding mode switching function and designs the corresponding control law. In [23], the asymptotic stability of fractional-order (FO) systems with uncertainty and time-varying delay is studied using sliding mode control (SMC). An integral type fractional-order sliding mode surface (FOSMS) is designed, and the stability condition is established using inequality techniques. Ref. [24] explores fixed-time synchronization in delayed FO complex-valued neural networks. It proposes a sliding mode surface, based on sliding mode control and Lyapunov stability.

In [25], the research focuses on the finite-time synchronization of uncertain delayed FOMNNs, which exhibit leak, and an adaptive-SMC method is proposed to approximate the upper bounds of uncertainties. Ref. [26], investigates synchronizing FO chaotic systems with input delay. To optimize communication resources and achieve synchronization, an adaptive neural network backstepping sliding mode controller is proposed using an event-triggered scheme without Zeno behavior. In [27], fixed-time stabilization of fuzzy neural networks with distributed delay is achieved using an adaptive sliding mode controller. A new fixed-time stability theorem provides the settling time, and an integral sliding mode surface is designed. In [28], a FO hyperbolic neuro-fuzzy backstepping sliding mode controller is designed for chaotic systems with time-varying delays and uncertainties. The controller uses adaptive rules for estimating system dynamics and updating uncertainty bounds, and employs hyperbolic tangential FO sliding surfaces to minimize tracking errors. The authors of [29] have studied the consensus problem for FO multi-agent systems with input delay and nonlinear dynamics. A dynamic event-triggered SMC scheme without Zeno behavior reduces communication usage. In [30], fixed- and preassigned-time synchronization of stochastic FOMNNs with uncertain parameters and mixed delays is addressed using SMC. A sliding surface and adaptive laws are constructed for fixed-time stability, and a preassigned-time SMC is proposed for fast synchronization. In [31], generalized projective synchronization of Caputo FOMNNs with multiple delays is studied, and a new function-matrix-projection-based SMC method is introduced. Also, a mixed controller, combining open-loop feedback and robust control, is introduced to derive the scheme using state and sampling information.

However, the cited research works exhibit one or more of the following limitations:

○

Existing research often relies heavily on either linear or nonlinear components in the suggestion of control techniques, limiting the comprehensiveness of the approaches.

○

In many instances, the utilization of SMC methods is accompanied by chattering phenomena that are not acceptable.

○

Most of these works simplify system definitions by neglecting uncertainties, external distributions, and input saturations, which are essential aspects of real-world systems.

○

Almost all these methods have selected the control parameters by classic error trials for simulation.

Recently, reinforcement learning (RL) was developed for the optimization problem of various engineering problems [32,33]. Q-learning is one of the most prevalent off-policy algorithms which can solve RL problems in a tabular manner. In Q-learning, agent actions and system states are stored in a Q-table and then the best action is selected among the possible actions. However, Q-learning fails to handle high-dimensional problems with a large number of states. To address this issue, deep neural nets (DNNs) were incorporated into RL algorithms as approximators, which is called a deep Q-network (DQN). In this regard, the Deep Deterministic Policy Gradient (DDPG) was introduced which can handle the system with continuous action space. The problem of overestimation was the main reason for the emergence of twin delayed DDPG, called TD3 [34]. However, both the DDPG and TD3 have high sensitivity to their hyper-parameters which limits their application. More recently, the soft actor-critic (SAC) was introduced which exploits a stochastic policy and maximum entropy RL. As a result, a superior level of exploration and convergence can be obtained by SAC than previous RL algorithms [35,36].

To address these issues, we propose a novel finite-time two-level SMC technique. This method aims to develop a chattering-free  ${\mathrm{PI}}^{\mathsf{\phi }}$-SMC technique that is dynamic and robust against unpredictability, external disturbances, and input saturations. Beginning at the first step, sliding surface will be introduced. Then, based on the PI rules, a robust  ${\mathrm{PI}}^{\mathsf{\phi }}$-sliding surface will be designed as the second sliding surface. The finite-time asymptotic stability of both surfaces will be proven. Also, the final step in this part is designing a dynamic-free control law, which is robust against system uncertainties, input saturations, and system delay.

Furthermore, to optimize parameters for SMC, the Q-learning (QL) algorithm is integrated with the SMC strategy. This eliminates constraints on the initial conditions of the control input, ensuring that the sliding motion meets the reachability condition within a finite time.

The main contributions and innovations of this research are as follows:

•

Development of a Novel Control Methodology: The study proposes a model-free  ${PI}^{\phi }$-SMC methodology, specifically designed for synchronizing chaotic with delays and input saturation.

•

Two-Level Sliding Surface Design: The methodology introduces a two-level control approach. The initial sliding surface addresses chaotic system dynamics, while the second  ${PI}^{\phi }$-sliding surface, based on proportional-integral (PI) rules, enhances system stability and robustness.

•

Finite-Time Synchronization: The proposed control method guarantees finite-time synchronization of delayed FOMNNSs, overcoming the challenge of delays, input saturations, and system uncertainties that previous approaches have struggled to handle effectively.

•

System Independence: The control laws are designed to be independent of the specific functions governing the system’s behavior. This is achieved through the norm-boundedness property of chaotic system states, making the methodology more adaptable to a wide range of chaotic systems.

•

Optimization via Soft Actor-Critic (SAC) Algorithm: The study utilizes SAC algorithm-based deep Q-learning to optimally adjust the controller parameters. This approach improves the adaptability and performance of the controller by maximizing a reward signal through the deep neural network of the SAC agent.

•

Extensive Validation: The effectiveness of the proposed control strategy is demonstrated through comprehensive simulation results and two numerical examples, confirming its applicability in practical engineering scenarios.

•

Practical Relevance: The proposed method’s robustness in handling chaotic dynamics, delays, and input saturation shows significant potential for real-world engineering applications, particularly in areas requiring advanced control of complex systems.

The rest of the study is organized as follows: Section 2 provides the necessary concepts about FO initials, stability theorems, and presents the problem statement and system description. Section 3 introduces a novel model-free P$I^{\phi }$-SMC technique designed to synchronize of different delayed FOMNNSs, in three steps. Section 4 details the deep SAC method employed in this study and elucidates the process for determining the optimal parameters. Section 5 includes numerical simulations and illustrates the analytical results through two case studies involving the synchronization of unknown delayed FOMNNSs. Section 6 concludes the paper with a discussion of the findings and draws final conclusions based on the results.

2. Preliminary Concepts and Description of the Problem

This section is divided into two subsections. Initially, fundamental definitions and principles concerning FO calculus and stability are outlined. Following that, the problem statement regarding synchronization is introduced.

2.1. Preliminaries

Definition 1

([37]). Suppose  $\Xi \left(t\right)$ is a continuous function in real space   $\mathbb{R}$. The Rieman–Liouville non-integer-order integral of   $\Xi \left(t\right)$ for a fractional constant   $\mathit{\phi }\in \mathbb{R}$ is defined as

$${}_{t_0}I_{t_1}\Xi \left(t\right)={\mathit{D}}^{-\mathit{\phi }}\Xi \left(t\right)={\displaystyle \frac{1}{\Gamma \left(\mathit{\phi }\right)}}{\int }_{t_0}^{t_1} {\displaystyle \frac{\Xi \left(\mathcal{E}\right)}{(t-\mathcal{E}{)}^{1-\mathit{\phi }}}}d\mathcal{E},$$

(1)

where   $t_0\in \mathbb{R}$ represents the starting time, and   $\Gamma (.)$ is the Gamma function.

Definition 2

([37,38]). Suppose  $\Xi \left(t\right)$ is a continuous function in real space  $\mathbb{R}$. The Caputo-defined fractional-order derivative of   $\Xi \left(t\right)$ for a non-integer number  $\mathit{\phi }\in \mathbb{R}$ is given by

$${}_{t_0}{}^c\mathit{D}_{t_1}^{\mathit{\phi }}\Xi \left(t\right)={\mathit{D}}^{\mathit{\phi }}\Xi \left(t\right)={\displaystyle \frac{1}{\Gamma \left(q-\mathit{\phi }\right)}}{\int }_{t_0}^t{\displaystyle \frac{{\Psi }^{\left(q\right)}\left(\mathcal{E}\right)}{(t-\mathcal{E}{)}^{\mathit{\phi }-q+1}}}d\mathcal{E},$$

(2)

in which   $q-1<\mathit{\phi }\le q\in \mathbb{N}$ and   ${\mathit{D}}^{\mathit{\phi }}$ represents the Caputo derivative operator throughout the rest of this study.

Property 1

([39]). If  $\mathit{\phi }\in \left(\mathrm{0,1}\right)$ and  $l$ is a real number, then  ${\mathit{D}}^{\mathit{\phi }}l=0$.

Property 2

([39]). Consider  $\mathit{\phi }\in \left(\mathrm{0,1}\right)$ and   $\mathcal{H}\left(t\right)\in C^n[0,T]$ , then

$$\dot{\mathcal{H}}\left(t\right)={\mathit{D}}^{1-\mathit{\phi }}\left({\mathit{D}}^{\mathit{\phi }}\mathcal{H}\left(t\right)\right)$$

(3)

$${\mathit{D}}^{\mathit{\phi }}\left({\mathit{I}}^{\mathit{\phi }}\mathcal{H}\left(t\right)\right)={\mathit{D}}^{\mathit{\phi }}\left({\mathit{D}}^{-\mathit{\phi }}\mathcal{H}\left(t\right)\right)=\mathcal{H}\left(t\right)$$

(4)

Lemma 1

([40]). Assuming  $\mathit{\phi }\in \left(\mathrm{0,1}\right),$ consider the fractional-order system

$${\mathit{D}}^{\mathit{\phi }}\aleph \left(t\right)=\mathcal{g}\left(t,\aleph \left(t\right)\right)$$

(5)

which satisfies the Lipschitz condition and has an equilibrium point at   $\aleph =0$. Under these conditions, let us examine the existence of a Lyapunov function   $V\left(t,\aleph \left(t\right)\right)$ that adheres to the following criteria:

$$b_1\left(‖q‖\right)\le V\left(t,\aleph \left(t\right)\right)\le b_2\left(‖q‖\right),$$

(6)

$$\dot{V}\left(t,\aleph \left(t\right)\right)\le -b_3\left(‖q‖\right).$$

(7)

Here,   $b_1$,  $b_2$ and  $b_3$represent positive constants. If these conditions are met, the equilibrium point of the fractional-order system   ${\mathit{D}}^{\mathit{\phi }}\aleph \left(t\right)=\mathcal{g}\left(t,\aleph \left(t\right)\right)$ will demonstrate (asymptotic) stability as per the Mittag–Leffler specifications.

2.2. Description of the Problem

For  $i=1,2, \dots , n$, let us examine the subsequent group of delayed FOMNNSs.

$${\mathit{D}}^{\mathit{\phi }}{z}_i\left(t\right)=-k_i{z}_i\left(t\right)+\sum _{j=1}^n{p}_{ij}{\psi }_j\left(z_j\left(t\right)\right)+\sum _{j=1}^n{q}_{ij}{\psi }_j\left(z_j\left(t-\tau \right)\right)+{\wp }_i,$$

(8)

where  $n$ denotes the number of neural network units, and  $z_i\left(t\right)$ represents the state of the i-th neuron within the delayed FOMNNSs.  ${\psi }_{1j}$ and  ${\psi }_{2j}$ denote the activation functions of the j-th neuron, while  $p_{ij}$ and  $q_{ij}$ signify the elements of connection weight matrices corresponding to the j-th neuron’s influence on the i-th neuron.  $k_i>0$ denotes an unspecified parameter indicating the rate at which the i-th neuron restores its prospective to the resting state upon disconnection from the network.  ${\wp }_i$ represents the i-th element of external distributions, and τ signifies the transmission delay.

To tackle the challenge of synchronization/stabilization in delayed FOMNNSs, we designate system (8) as the drive system. Subsequently, the ensuing system serves as the response system.

$${\mathit{D}}^{\mathit{\phi }}{w}_i\left(t\right)=-d_i{w}_i\left(t\right)+\sum _{j=1}^n{r}_{ij}{\mu }_j\left(w_j\left(t\right)\right)+\sum _{j=1}^n{l}_{ij}{\mu }_j\left(w_j\left(t-\tau \right)\right)+{\mathcal{g}}_i+{\varphi }_i\left(u_i\left(t\right)\right)$$

(9)

In this context,  $w_i\left(t\right)$ denotes the i-th state of neurons within the delayed FOMNNSs, while  ${\mu }_{1j}$ and  ${\mu }_{2j}$ represent the activation functions of the j-th neuron. The parameters  $r_{ij}$ and  $l_{ij}$ correspond to the components of connection weight matrices, where  $d_i$ is analogous to  $k_i$ in Equation (8). Additionally,  ${\mathcal{g}}_i$ denotes the i-th component of external distributions, and τ represents the transmission delay.

Furthermore,  $u_i\left(t\right)$ signifies the control input, and  ${\varphi }_i\left(u_i\left(t\right)\right)$ represents the input-saturation function, defined as

$${\varphi }_i\left(u_i\left(t\right)\right)=u_i\left(t\right)+\Lambda \left(u_i\left(t\right)\right),i=1,\dots , n,$$

(10)

where

$$\Lambda \left(u_i\left(t\right)\right)=\left\{\begin{array}{c}{u}^{b_1}-u_i\left(t\right) if { u}_{b_2}>u_i\left(t\right)\\ \left(\mho -1\right)u_i\left(t\right) if u^{{\theta }_1}<u_i\left(t\right)<u^{b_1}\\ {u^{{\theta }_1}-u}_i\left(t\right) if u_i\left(t\right)\ge u_{{\theta }_2.}\end{array}\right.,i=1,\dots , n$$

(11)

In (11),   $u^{{\theta }_1},$  $u_{{\theta }_2}\in R^{+}$and  $u^{b_1},$  $u_{b_2}\in R^{-}$ represent the upper and lower bounds, respectively, of the input saturation process in Equation (10). The parameter  $\mho \in \mathbb{R}$ denotes the slope saturation.

By specifying the error function, we obtain the following:

$$\begin{gathered} E=Z-W={\left[z_1\left(t\right), z_2\left(t\right), \dots , z_n\left(t\right)\right]}^T-{\left[w_1\left(t\right), w_2\left(t\right), \dots , w_n\left(t\right)\right]}^T \\ ={\left[e_1\left(t\right), e_2\left(t\right), \dots , e_n\left(t\right)\right]}^T. \end{gathered}$$

(12)

Hence, for  $i=1,\dots , n$, the dynamics to synchronize the error parameters can be derived by

$${\mathit{D}}^{\mathit{\phi }}{e}_i\left(t\right)={\mathit{D}}^{\mathit{\phi }}{z}_i\left(t\right)-{\mathit{D}}^{\mathit{\phi }}{w}_i\left(t\right)$$

(13)

$$\begin{gathered} =-\left(k_i{z}_i\left(t\right)-d_i{w}_i\left(t\right)\right)+\sum _{j=1}^n\left[p_{ij}{\psi }_j\left(x_j\left(t\right)\right)-r_{ij}{\mu }_j\left(w_j\left(t\right)\right)\right] \\ +\sum _{j=1}^n\left[q_{ij}{\psi }_j\left(z_j\left(t-\tau \right)\right)-l_{ij}{\mu }_j\left(w_j\left(t-\tau \right)\right)\right]+\left({\wp }_i-{\mathcal{g}}_i\right)-{\varphi }_i\left(u_i\left(t\right)\right). \end{gathered}$$

(14)

The overarching aim is to design a sliding mode control mechanism that is both pertinent and operational, ensuring that, for  $i=1,\dots , n$, the following convergence holds in the limit as time approaches infinity:

$$\underset{t\to \infty }{\mathit{lim}}‖e_i\left(t\right)‖=\underset{t\to \infty }{\mathit{lim}}‖z_i\left(t\right)-w_i\left(t\right)‖=0.$$

(15)

If Equation (15) can be derived, it will pave the way for the comprehensive resolution and implementation of error suppression between the drive system (8) and the response system (9).

Assumption 1.

The error states’ trajectories in chaotic systems are often constrained within certain regions of phase space [41,42], owing to the irregular attractors produced by chaotic behavior. As a result, there are positive constants  $a_{1i}$ and   $b_{1i}$ that fulfill the conditions outlined in the subsequent relations.

$$\left|-\left(k_i{z}_i\left(t\right)-d_i{w}_i\left(t\right)\right)+\sum _{j=1}^n\left[p_{ij}{\psi }_j\left(x_j\left(t\right)\right)-r_{ij}{\mu }_j\left(w_j\left(t\right)\right)\right]\right|\le a_{1i}$$

(16)

and

$$\left|\sum _{j=1}^n\left[q_{ij}{\psi }_j\left(z_j\left(t-\tau \right)\right)-l_{ij}{\mu }_j\left(w_j\left(t-\tau \right)\right)\right]\right|\le b_{1i}.$$

(17)

Moreover, it is anticipated that the uncertainty terms   ${\wp }_i$ and   ${\mathcal{g}}_i$ will be bounded. Consequently, there exists a positive constant   $c_{1i}$  such that

$$\left|\left({\wp }_i-{\mathcal{g}}_i\right)\right|\le c_{1i}, i=1,2, \dots , n.$$

(18)

As a result of (16), (17), and (18),

$$\left|-\left(k_i{z}_i\left(t\right)-d_i{w}_i\left(t\right)\right)+\sum _{j=1}^n\left[p_{ij}{\psi }_j\left(x_j\left(t\right)\right)-r_{ij}{\mu }_j\left(w_j\left(t\right)\right)\right]\right|+\left|\sum _{j=1}^n\left[q_{ij}{\psi }_j\left(z_j\left(t-\tau \right)\right)-l_{ij}{\mu }_j\left(w_j\left(t-\tau \right)\right)\right]\right|+\left|\left({\wp }_i-{\mathcal{g}}_i\right)\right|\le {\xi }_i$$

(19)

where   ${\xi }_i<\infty$ is a positive constant for   $i=1,2, \dots , n.$

Assumption 2.

Ensuring the boundedness property of the control method is among the most crucial prerequisites for the relevance of a control law. Therefore, it is desirable that  $\Lambda \left(u_i\left(t\right)\right)$ remains bounded. Consequently, one has

$$\left|\Lambda \left(u_i\left(t\right)\right), \right|\le N_i\le \infty , i=1,2, \dots , n.$$

(20)

Remark 1.

The input saturation function   ${\varphi }_i\left(u_i\left(t\right)\right)$ was introduced into the control input to address the practical constraints associated with real-world systems where the control signals are limited by the physical capabilities of actuators. By incorporating this function, the control strategy effectively models and manages these input constraints, ensuring that the control signals remain within allowable bounds and do not exceed the system’s limits. This approach helps prevent issues such as actuator overload and maintains system stability and performance under realistic operating conditions. It ensures that the proposed control methodology remains feasible and effective even in the presence of input saturation, which is crucial for practical applications where input constraints are an inherent aspect of system design.

3. Finite Time  $\mathit{P}{\mathit{I}}^{\mathit{\phi }}$-SMC Technique Design

This section will first outline the design of a robust sliding surface, and then introduce the  $P{I}^{\phi }$ sliding surface. Moreover, a finite time control scheme will be presented to synchronize FOMNNSs (8) and (9). Also, the related theorems and proofs are provided.

Hence, for  $i=1,2, \dots , n$, the hypothesized equation for the sliding surface in the field-oriented control system is proposed as follows:

$$s_i\left(t\right)=e_i\left(t\right)+{\mathit{D}}^{-1}\left[m_i\left|e_i\left(t\right)\right| sign\left(e_i\left(t\right)\right)\right],$$

(21)

in which  $m_i$ is a positive constant.

During the occurrence of sliding motion, it is widely acknowledged that the condition  $s_i\left(t\right)=0$ is met; thus, employing Property 1,

$$s_i\left(t\right)=0\Rightarrow {\mathit{D}}^{\mathit{\phi }}{s}_i\left(t\right)=0$$

(22)

$$\Rightarrow {\mathit{D}}^{\mathit{\phi }}{s}_i\left(t\right)={\mathit{D}}^{\mathit{\phi }}{e}_i\left(t\right)+{\mathit{D}}^{\mathit{\phi }-1}\left[m_i\left|e_i\left(t\right)\right| sign\left(e_i\left(t\right)\right)\right]=0$$

(23)

$$\Rightarrow {\mathit{D}}^{\mathit{\kappa }}{e}_i\left(t\right)=-{\mathit{D}}^{\mathit{\phi }-1}\left[m_i\left|e_i\left(t\right)\right| sign\left(e_i\left(t\right)\right)\right]$$

(24)

Theorem 1.

The dynamic equation governing sliding (Equation (24)) will exhibit stability, and the states of the fractional-order error neural network (Equation (14)) will asymptotically converge to the origin within finite time.

Proof of Theorem 1.

By taking into account the Lyapunov structure

$$V_{1i}=\left|e_i\left(t\right)\right|.$$

(25)

and applying an integer derivative to Equation (25), and utilizing Equation (3) in Property 2, one obtains the following:

$${\dot{V}}_{1i}=\dot{e_i}\left(t\right)sign\left(e_i\left(t\right)\right)= {\mathit{D}}^{1-\mathit{\phi }}\left({\mathit{D}}^{\mathit{\phi }}{e}_i\left(t\right)\right)sign\left(e_i\left(t\right)\right).$$

(26)

Substituting  ${\mathit{D}}^{\mathit{\phi }}{e}_i\left(t\right)$ from Equation (24) into Equation (26), we have

$${\dot{V}}_{1i}={\mathit{D}}^{1-\mathit{\phi }}\left(-{\mathit{D}}^{\mathit{\phi }-1}\left[m_i\left|e_i\left(t\right)\right| sign\left(e_i\left(t\right)\right)\right]\right)sign\left(e_i\left(t\right)\right)$$

(27)

$$=-\left[m_i\left|e_i\left(t\right)\right| sign\left(e_i\left(t\right)\right)\right]sign\left(e_i\left(t\right)\right)$$

(28)

$$=-m_i\left|e_i\left(t\right)\right|<0.$$

(29)

Hence, according to Equation (29), the inequality  ${\dot{V}}_{1i}\le -m_i\left|e_i\left(t\right)\right|$ holds true, ensuring  ${\dot{V}}_1<0$. Consequently, the stability criteria outlined in Lemma 1 are satisfied, leading to the attainment of asymptotic stability for the sliding surface dynamics (Equation (24)).

Now, in accordance with Equations (26) and (29),

$${\dot{V}}_{1i}={\left(\left|e_i\left(t\right)\right|\right)}^{\prime }={\displaystyle \frac{d}{dt}}\left|e_i\left(t\right)\right|\le -m_i\left|e_i\left(t\right)\right|$$

(30)

$$\Rightarrow dt\le -{\displaystyle \frac{1}{m_i}}{\displaystyle \frac{d \left|e_i\left(t\right)\right|}{\left|e_i\left(t\right)\right|}}$$

(31)

Integrating both sides of Equation (31) yields

$${\int }_{t_0^s}^{t_1^s}dt\le -{\displaystyle \frac{1}{m_i}}{\int }_{t_0^s}^{t_1^s}{\displaystyle \frac{ d\left|e_i\left(t\right)\right|}{\left|e_i\left(t\right)\right|}}$$

(32)

$$\Rightarrow t_1^s-t_0^s\le -{\displaystyle \frac{1}{m_i}}\ln \left(\left|e_i\left(t\right)\right|\right)\left|\begin{array}{c}{t}_1^s\\ t_0^s\end{array}\right.$$

(33)

Since  $e_i\left(t_1^s\right)=0,$ one obtains

$$\Rightarrow t_1^s-t_0^s\le {\displaystyle \frac{1}{m_i}}\ln \left(\left|e_i\left(t_0^s\right)\right|\right)$$

(34)

$$\Rightarrow t_1^s\le \frac{1}{m_i}\ln \left(\left|e_i\left(t_0^s\right)\right|\right)+t_0^s$$

(35)

Hence, the states of  $e_i\left(t\right)$ converge to zero within a finite time  $t_1^s\le {\displaystyle \frac{1}{m_i}}\ln \left(\left|e_i\left(t_0^s\right)\right|\right)+t_0^s$, thus completing the proof. □

Remark 2.

Equation (26) is valid for   $0<\phi <1.$ Also, it holds true specifically when   $\phi =1$, as the right-hand side simplifies to the first derivative   $\dot{e_i}\left(t\right)$. However, for   $\phi >$1, whether fractional or integer, the equation does not generally hold. This is because   $D^{\phi }{e}_i\left(t\right)$ involves a fractional derivative and applying   $D^{1-\phi }$ results in a fractional integral of order   $\phi -1$, which does not simplify back to the first derivative [39]. Thus, the equation does not accurately represent   $\dot{e_i}\left(t\right)$ for   $\phi >1$.

Now, let us introduce the second proportional–integral (PI) sliding surface. The PI sliding surface offers several advantages in control systems. By integrating both proportional and integral control actions, the PI sliding surface enhances system robustness by mitigating steady-state errors and improving tracking performance. It reduces overshooting, ensures steady-state accuracy, and simplifies controller tuning compared to more complex strategies. The continuous integration of error signals over time leads to smoother responses and improved transient behavior. Additionally, its versatility allows for application across a wide range of control problems, making it a practical choice for diverse industrial applications.

Here’s the second sliding surface, implemented as a PI equation:

$${\delta }_i\left(t\right)=L_P{s}_i\left(t\right)+L_I{\mathit{D}}^{-1}\left[s_i\left(t\right)+{\left|s_i\left(t\right)\right| }^{\gamma }sign\left(s_i\left(t\right)\right)\right],$$

(36)

in which  $L_P$ and  $L_I$ are PI parameters and  $0<\gamma <1$ is a real number.

As it mentioned before, in the event of sliding motion, it is widely recognized that the condition  ${\delta }_i\left(t\right)=0$ is satisfied, thereby leveraging Property 1:

$${\delta }_i\left(t\right)=0\Rightarrow {\mathit{D}}^{\mathit{\phi }}{\delta }_i\left(t\right)=0$$

(37)

$$\Rightarrow {\mathit{D}}^{\mathit{\phi }}{\delta }_i\left(t\right)=L_P{\mathit{D}}^{\mathit{\phi }}{s}_i\left(t\right)+L_I{\mathit{D}}^{\mathit{\phi }-1}\left[s_i\left(t\right)+{\left|s_i\left(t\right)\right| }^{\gamma }sign\left(s_i\left(t\right)\right)\right]=0$$

(38)

$$\Rightarrow {\mathit{D}}^{\mathit{\phi }}{s}_i\left(t\right)=-{\displaystyle \frac{L_I}{L_P}}{\mathit{D}}^{\mathit{\phi }-1}\left[s_i\left(t\right)+{\left|s_i\left(t\right)\right| }^{\gamma }sign\left(s_i\left(t\right)\right)\right]$$

(39)

Theorem 2.

The dynamic equation governing sliding (Equation (39)) is expected to demonstrate stability, with the states of the PI sliding dynamics (Equation (36)) asymptotically converging to the origin within finite time.

Proof of Theorem 2.

Examine the following Lyapunov framework:

$$V_{2i}=\left|s_i\left(t\right)\right|, i=1,2, \dots , n.$$

(40)

Now, through the application of an integer derivative to Equation (40) and attention to Equation (3) in Property 2, one obtains

$${\dot{V}}_{2i}=\dot{s_i}\left(t\right)sign\left(s_i\left(t\right)\right)= {\mathit{D}}^{1-\mathit{\phi }}\left({\mathit{D}}^{\mathit{\phi }}{s}_i\left(t\right)\right)sign\left(s_i\left(t\right)\right).$$

(41)

Substituting  ${\mathit{D}}^{\mathit{\phi }}{s}_i\left(t\right)$ from Equation (39) into Equation (41) yields the following:

$${\dot{V}}_{2i}={\mathit{D}}^{1-\mathit{\phi }}\left(-{\displaystyle \frac{L_I}{L_P}}{\mathit{D}}^{\mathit{\phi }-1}\left[s_i\left(t\right)+{\left|s_i\left(t\right)\right| }^{\gamma }sign\left(s_i\left(t\right)\right)\right]\right)sign\left(e_i\left(t\right)\right)$$

(42)

$$=-{\displaystyle \frac{L_I}{L_P}}\left[s_i\left(t\right)+{\left|s_i\left(t\right)\right| }^{\gamma }sign\left(s_i\left(t\right)\right)\right]sign\left(e_i\left(t\right)\right)$$

(43)

$$=-{\displaystyle \frac{L_I}{L_P}}\left[\left|s_i\left(t\right)\right|+{\left|s_i\left(t\right)\right| }^{\gamma }\right]<0.$$

(44)

Hence, based on Equation (44), the inequality  ${\dot{V}}_{2i}\le -{\displaystyle \frac{L_I}{L_P}}\left[\left|s_i\left(t\right)\right|+{\left|s_i\left(t\right)\right| }^{\gamma }\right]$ always holds true, and  ${\dot{V}}_{2i}<0$. Consequently, the stability criteria outlined in Lemma 1 are met, resulting in the attainment of asymptotic stability for dynamic Equation (39) and the PI sliding dynamics (Equation (36)).

Now, for  $i=1,2, \dots , n$, based on Equations (40) and (44),

$${\dot{V}}_{2i}={\left(\left|s_i\left(t\right)\right|\right)}^{\prime }={\displaystyle \frac{d}{dt}}\left|s_i\left(t\right)\right|$$

(45)

$$\le -{\displaystyle \frac{L_I}{L_P}}\left[\left|s_i\left(t\right)\right|+{\left|s_i\left(t\right)\right| }^{\gamma }\right]<-{\displaystyle \frac{L_I}{L_P}}\left|s_i\left(t\right)\right|$$

(46)

$$\Rightarrow dt\le -\frac{L_P}{L_I}\frac{d \left|s_i\left(t\right)\right|}{\left|s_i\left(t\right)\right|}$$

(47)

Integrating both sides of Equation (31) yields

$${\int }_{t_0^{\delta }}^{t_1^{\delta }}dt\le -{\displaystyle \frac{L_P}{L_I}}{\int }_{t_0^{\delta }}^{t_1^{\delta }}{\displaystyle \frac{ d\left|s_i\left(t\right)\right|}{\left|s_i\left(t\right)\right|}}$$

(48)

$$\Rightarrow t_1^{\delta }-t_0^{\delta }\le -\frac{L_P}{L_I}\ln \left(\left|s_i\left(t\right)\right|\right)\left|\begin{array}{c}{t}_1^{\delta }\\ t_0^{\delta }\end{array}\right.$$

(49)

Since  $s_i\left(t_1^{\delta }\right)=0,$ one obtains

$$\Rightarrow t_1^{\delta }-t_0^{\delta }\le {\displaystyle \frac{L_P}{L_I}}\ln \left(\left|s_i\left(t_0^{\delta }\right)\right|\right)$$

(50)

$$\Rightarrow t_1^{\delta }\le \frac{L_P}{L_I}\ln \left(\left|s_i\left(t_0^{\delta }\right)\right|\right)+t_0^{\delta }$$

(51)

Thus, the convergence of  $s_i\left(t\right)$ states to zero within a finite time  $t_1^{\delta }\le {\displaystyle \frac{L_P}{L_I}}\ln \left(\left|s_i\left(t_0^{\delta }\right)\right|\right)+t_0^{\delta }$ concludes the proof. □

Remark 3.

In this study, the sliding surfaces for the   $P{I}^{\phi }$-SMC method are carefully designed to achieve robust synchronization of chaotic FOMNNSs. The design process involves the following key elements:

• Initial Sliding Surface:

  •

  State Trajectories: The initial sliding surface is introduced to guide the state trajectories of the chaotic system towards a more controlled behavior. This surface is a function of the system’s state variables and is typically designed to simplify the system’s dynamics, making it easier to manage the chaotic behavior.

  •

  Robustness: Although the initial sliding surface provides some level of control, it primarily serves as the foundation for more robust control mechanisms. It reduces the complexity of the system’s chaotic dynamics, but further refinement is needed to handle uncertainties and external disturbances effectively.

• $P{I}^{\phi }$-Sliding Surface:

  •

  Proportional-Integral (PI) Rules: The ${PI}^{\phi }$-sliding surface is designed using PI rules, which combine proportional and integral actions to improve control performance. The addition of the fractional-order element (denoted by φ) allows the surface to better manage the memory and hereditary effects inherent in fractional-order systems, resulting in more precise control over the state trajectories.

  •

  Robustness Against Uncertainties: This surface is designed to be robust against system uncertainties, input saturations, and delays. By incorporating both PI control and fractional-order dynamics, the  ${PI}^{\phi }$-sliding surface ensures that the system can reach and maintain desired states even in the presence of significant disturbances and nonlinearities.

• Lyapunov Stability:

  •

  Finite-Time Asymptotic Stability: The stability of both the initial and  ${PI}^{\phi }$-sliding surfaces is analyzed using the fractional-order Lyapunov stability theory. This theory helps demonstrate that the system’s state trajectories will converge to the sliding surfaces within a finite time and remain there, ensuring that the chaotic behavior is effectively controlled and synchronization is achieved.

  •

  Lyapunov Function: A Lyapunov function is typically constructed to prove that the sliding surfaces are stable. The function is chosen such that it decreases over time, guaranteeing that the system’s state will move towards the sliding surface and stay there, leading to finite-time synchronization of the delayed FOMNNS.

Remark 4.

The decision to design two sliding surfaces—a primary sliding surface and a   $P{I}^{\phi }$ sliding surface—was made to enhance both the robustness and precision of the control system for FOMNNSs with delays and input saturation. The primary sliding surface ensures initial stabilization by guiding the chaotic system’s trajectory towards a stable region, while the  $P{I}^{\phi }$ sliding surface introduces proportional–integral control to refine the system’s response, compensating for steady-state errors and further improving robustness against uncertainties. This two-level structure also better handles input saturation and accelerates convergence to finite-time synchronization, offering significant advantages over a single sliding surface, which may struggle with control precision and only achieve asymptotic synchronization. This dual-layer approach thus optimizes both stability and performance, making it more suitable for complex system dynamics.

Now, for  $i=1, \dots , n,$ a novel and dynamic-free control method will be introduced in the following theorem, formulated as:

$$\begin{array}{c}{u}_i\left(t\right)=[{\xi }_i{+\mathit{D}}^{\mathit{\phi }-1}\left[m_i\left|e_i\left(t\right)\right| sign\left(e_i\left(t\right)\right)\right]+N_i+\frac{L_I}{L_P}{\mathit{D}}^{\mathit{\phi }-1}\left[s_i\left(t\right)+{\left|s_i\left(t\right)\right|}^{\gamma }sign\left(s_i\left(t\right)\right)\right]\\ +{\varpi }_i{\mathit{D}}^{\mathit{\phi }-1}\left[{\delta }_i\left(t\right)+{\left|{\delta }_i\left(t\right)\right| }^{\mathfrak{z}}sign\left({\delta }_i\left(t\right)\right)\right]\left]sign\right({\delta }_i\left(t\right)).\end{array}$$

(52)

The structure of the suggested controller applied to the FOMNNs is depicted in Figure 1.

Theorem 3.

Consider the fractional-order error mechanism (Equation (14)). Utilizing the subsequent control approach (Equation (52)), the trajectories of the error FOMNNs (Equation (14)) will achieve asymptotic stability within a finite duration.

Proof of Theorem 3.

Examine the following Lyapunov framework:

$$V_{3i}=\left|{\delta }_i\left(t\right)\right|.$$

(53)

By applying a first order derivative to Equation (53), and utilizing Equation (3) in Property 2, one obtains:

$${\dot{V}}_{3i}=\dot{{\delta }_i}\left(t\right)sign\left({\delta }_i\left(t\right)\right)= {\mathit{D}}^{1-\mathit{\phi }}\left({\mathit{D}}^{\mathit{\phi }}{\delta }_i\left(t\right)\right)sign\left({\delta }_i\left(t\right)\right).$$

(54)

By inserting  ${\mathit{D}}^{\mathit{\phi }}{\delta }_i\left(t\right)$ from Equation (38) into Equation (54), we obtain

$${\dot{V}}_{3i}={\mathit{D}}^{1-\mathit{\phi }}\left(L_P{\mathit{D}}^{\mathit{\phi }}{s}_i\left(t\right)+L_I{\mathit{D}}^{\mathit{\phi }-1}\left[s_i\left(t\right)+{\left|s_i\left(t\right)\right| }^{\gamma }sign\left(s_i\left(t\right)\right)\right]\right)sign\left({\delta }_i\left(t\right)\right).$$

(55)

Now, by inserting  ${\mathit{D}}^{\mathit{\phi }}{s}_i\left(t\right)$ from Equation (23) into Equation (55), we obtain

$${\dot{V}}_{3i}={\mathit{D}}^{1-\mathit{\phi }}\left(L_P\left[{\mathit{D}}^{\mathit{\phi }}{e}_i\left(t\right)+{\mathit{D}}^{\mathit{\phi }-1}\left[m_i\left|e_i\left(t\right)\right| sign\left(e_i\left(t\right)\right)\right]\right]+L_I{\mathit{D}}^{\mathit{\phi }-1}\left[s_i\left(t\right)+{\left|s_i\left(t\right)\right| }^{\gamma }sign\left(s_i\left(t\right)\right)\right]\right)sign\left({\delta }_i\left(t\right)\right).$$

(56)

Here, by substituting  ${\mathit{D}}^{\mathit{\phi }}{e}_i\left(t\right)$ from Equation (14) into Equation (56), and according to Equations (19) and (20), one has

$$\begin{array}{cc}\hfill {\dot{V}}_{3i}=& {\mathit{D}}^{1-\mathit{\phi }}(L_P[(-\left(k_i{z}_i\left(t\right)-d_i{w}_i\left(t\right)\right)+{\displaystyle \sum _{j=1}^n}\left[p_{ij}{\psi }_j\left(x_j\left(t\right)\right)-r_{ij}{\mu }_j\left(w_j\left(t\right)\right)\right]\hfill \\ & \hspace{1em}\hspace{1em}\hspace{1em}+{\displaystyle \sum _{j=1}^n}\left[q_{ij}{\psi }_j\left(z_j\left(t-\tau \right)\right)-l_{ij}{\mu }_j\left(w_j\left(t-\tau \right)\right)\right]+\left({\wp }_i-{\mathcal{g}}_i\right)\hfill \\ & \hspace{1em}\hspace{1em}\hspace{1em}-{\varphi }_i\left(u_i\left(t\right)\right))+{\mathit{D}}^{\mathit{\phi }-1}\left[m_i\left|e_i\left(t\right)\right| sign\left(e_i\left(t\right)\right)\right]]\hfill \\ & \hspace{1em}\hspace{1em}\hspace{1em}+L_I{\mathit{D}}^{\mathit{\phi }-1}\left[s_i\left(t\right)+{\left|s_i\left(t\right)\right| }^{\gamma }sign\left(s_i\left(t\right)\right)\right])sign\left({\delta }_i\left(t\right)\right)\hfill \end{array}$$

(57)

$$\begin{array}{c}\le {\mathit{D}}^{1-\mathit{\phi }}(L_P[\left|-\left(k_i{z}_i\left(t\right)-d_i{w}_i\left(t\right)\right)\right|+\left|{\displaystyle \sum _{j=1}^n}\left[p_{ij}{\psi }_j\left(x_j\left(t\right)\right)-r_{ij}{\mu }_j\left(w_j\left(t\right)\right)\right]\right|\\ \hspace{1em}\hspace{1em}\hspace{1em}+\left|{\displaystyle \sum _{j=1}^n}\left[q_{ij}{\psi }_j\left(z_j\left(t-\tau \right)\right)-l_{ij}{\mu }_j\left(w_j\left(t-\tau \right)\right)\right]\right|+\left|{\wp }_i-{\mathcal{g}}_i\right|\\ \hspace{1em}\hspace{1em}\hspace{1em}+\left|∆\left(u_i\left(t\right)\right)\right|]-L_P{u}_i\left(t\right)+L_P{\mathit{D}}^{\mathit{\phi }-1}\left[m_i\left|e_i\left(t\right)\right| sign\left(e_i\left(t\right)\right)\right]\\ \hspace{1em}\hspace{1em}\hspace{1em}+L_I{\mathit{D}}^{\mathit{\phi }-1}\left[s_i\left(t\right)+{\left|s_i\left(t\right)\right| }^{\gamma }sign\left(s_i\left(t\right)\right)\right])sign\left({\delta }_i\left(t\right)\right)<0.\end{array}$$

(58)

$$\le {\mathit{D}}^{1-\mathit{\phi }}\left(L_P\left[{\xi }_i+N_i\right]-L_P{u}_i\left(t\right)+L_P{\mathit{D}}^{\mathit{\phi }-1}\left[m_i\left|e_i\left(t\right)\right| sign\left(e_i\left(t\right)\right)\right]+L_I{\mathit{D}}^{\mathit{\phi }-1}\left[s_i\left(t\right)+{\left|s_i\left(t\right)\right| }^{\gamma }sign\left(s_i\left(t\right)\right)\right]\right)sign\left({\delta }_i\left(t\right)\right)<0.$$

(59)

By substituting  $u_i\left(t\right)$ from Equation (52), one obtains the following:

$${\dot{V}}_{3i}\le {\mathit{D}}^{1-\mathit{\phi }}\left(-{\varpi }_i{\mathit{D}}^{\mathit{\phi }-1}\left[{\delta }_i\left(t\right)+{\left|{\delta }_i\left(t\right)\right| }^{\mathfrak{z}}sign\left({\delta }_i\left(t\right)\right)\right]\right)sign\left({\delta }_i\left(t\right)\right)$$

(60)

$$\le -{\varpi }_i\left(\left|{\delta }_i\left(t\right)\right|+{\left|{\delta }_i\left(t\right)\right| }^{\mathfrak{z}}\right)<0.$$

(61)

Hence, based on Equation (61), the inequality  ${\dot{V}}_{3i}\le -{\varpi }_i\left(\left|{\delta }_i\left(t\right)\right|+{\left|{\delta }_i\left(t\right)\right| }^{\mathfrak{z}}\right)$ always holds true, and  ${\dot{V}}_{3i}<0$. Consequently, the stability criteria outlined in Lemma 1 are met, resulting in the attainment of asymptotic stability for error FOMNNs (14).

Now, for  $i=1,2, \dots , n$, based on the Equations (53) and (61)

$${\dot{V}}_{3i}={\left(\left|{\delta }_i\left(t\right)\right|\right)}^{\prime }={\displaystyle \frac{d}{dt}}\left|{\delta }_i\left(t\right)\right|$$

(62)

$$\le -{\varpi }_i\left(\left|{\delta }_i\left(t\right)\right|+{\left|{\delta }_i\left(t\right)\right| }^{\mathfrak{z}}\right)<-{\varpi }_i\left|{\delta }_i\left(t\right)\right|$$

(63)

$$\Rightarrow dt<-{\displaystyle \frac{1}{{\varpi }_i}}{\displaystyle \frac{d \left|{\delta }_i\left(t\right)\right|}{\left|{\delta }_i\left(t\right)\right|}}$$

(64)

Integrating both sides of Equation (64) yields

$${\int }_{t_0^u}^{t_1^u}dt\le -{\displaystyle \frac{1}{{\varpi }_i}}{\int }_{t_0^u}^{t_1^u}{\displaystyle \frac{ d\left|{\delta }_i\left(t\right)\right|}{\left|{\delta }_i\left(t\right)\right|}}$$

(65)

$$\Rightarrow t_1^u-t_0^u\le -\frac{1}{{\varpi }_i}\ln \left(\left|{\delta }_i\left(t\right)\right|\right)\left|\begin{array}{c}{t}_1^u\\ t_0^u\end{array}\right.$$

(66)

Dou to the fact that  ${\delta }_i\left(t_1^u\right)=0,$ one obtains

$$\Rightarrow t_1^u-t_0^u\le {\displaystyle \frac{1}{{\varpi }_i}}\ln \left(\left|{\delta }_i\left(t_0^u\right)\right|\right)$$

(67)

$$\Rightarrow t_1^u\le \frac{1}{{\varpi }_i}\ln \left(\left|{\delta }_i\left(t_0^u\right)\right|\right)+t_0^u$$

(68)

Thus, the convergence of  ${\delta }_i\left(t\right)$ states to zero within a finite time  $t_1^u\le {\displaystyle \frac{1}{{\varpi }_i}}\mathit{ln}\left(\left|{\delta }_i\left(t_0^u\right)\right|\right)+t_0^u$ concludes the proof. □

4. Design of Finite Time  $\mathit{P}{\mathit{I}}^{\mathit{\phi }}$-SMC Based on Deep SAC Learning

4.1. Principle Concepts of MDP and RL Algorithm

In many problems, all states of the system are not observable by the agent which are necessary to describe by Markov decision process (MDP) learning. By utilizing the MDP, the dynamics of any practical system can be captured in a straightforward manner where the agent policy is obtained according to the inherent characteristics of the system. According to the definition of the MDP, the next observation of a system only depends on the current observation [43]. Typically, the MDP is elaborated by a five-tuple  $〈\mathcal{O},A,\mathcal{P},r,\gamma 〉$ with the following expressions:

•

$\mathcal{o}\in \mathcal{O}$ is the set of system observations (states).

•

$w\in A$ is the set of actions.

•

$\mathcal{P}:\mathcal{O}\times A\times \mathcal{O}\to [0, 1)$ is the transition function, where this function indicates the probability of mapping from current observation  ${\mathcal{o}}_t$ to next step observation  ${\mathcal{o}}_{t+1}$ under action  $w_t$.

•

$r\in \mathcal{O}\times A\to \mathcal{R}$ is a reward signal which is emitted from the environment after applying the action  $w_t$.

•

A history  ${\mathcal{h}}_t=\left({\mathcal{o}}_0,w_1,\dots ,w_{t-1},{\mathcal{o}}_t\right)$ stores all the transitions of actions and observations.

At each step, a signal is produced by the RL agent under current state  ${\mathcal{o}}_t$ to obtain a reward signal  $r_t$ from the environment (system). To select the appropriate action, the policy  $\pi \left(w\right|\mathcal{o})$ is defined which represents the behavior of the RL agent.

In RL problems, the agent aims to maximize the sum of discounted returns, given as

$$G_t=r_{t+1}+{\gamma r}_{t+2}+\dots ={\sum }_{k=0}^{\infty }{\gamma }^k{r}_{t+k+1}$$

(69)

To complete the RL task, the state-value function  $V_{\pi }\left(\mathcal{o}\right)$ and action-value function  ${\mathcal{Q}}_{\pi }(\mathcal{o},w)$ are defined according to Equation (70) and Equation (71), respectively.

$$V_{\pi }\left(\mathcal{o}\right)={\mathbb{E}}_{\pi }\left[G_t|{\mathcal{O}}_t=\mathcal{o}\right]$$

(70)

$${\mathcal{Q}}_{\pi }(\mathcal{o},w)={\mathbb{E}}_{\pi }\left[G_t|{\mathcal{O}}_t=\mathcal{o},A_t=w\right]$$

(71)

4.2. Soft Actor-Critic Strategy

The soft actor-critic is one of the RL algorithms which can solve high-dimensional problems using the training capability of deep neural nets (DNNs). In the SCA, the concept of entropy regularization is utilized where the policy of this algorithm is characterized by preserving a balance between expected reward signals and entropy. This policy mechanism prevents premature convergence while improving the exploration capability. The SAC algorithm training is realized using entropy and reward terms [43,44].

$$J\left(\pi \right)=\sum _{t=0}^T{\mathbb{E}}_{\left({\mathcal{o}}_t,w_t\right)~{\rho }_{\pi }}\left[r_t\left({\mathcal{o}}_t,w_t\right)+\beta \mathcal{H}\left(\pi \left(·|{\mathcal{o}}_t\right)\right)\right]$$

(72)

where  $\beta$ is the hyper-parameter to adjust the entropy. The SAC algorithm is constructed from three DNNs including Q-net, V net, and Policy net. Also, an experience buffer is utilized to store all historical transitions during the training process.

The training Q-net parameterized by  $\theta$ is conduced by minimizing a soft Bellman equation, given as follows:

$$J_{\mathcal{Q}}\left(\theta \right)={\mathbb{E}}_{\left({\mathcal{o}}_t,w_t\right)~D}\left[\frac{1}{2}{\left({\mathcal{Q}}_{\theta }\left({\mathcal{o}}_t,w_t\right)-\widehat{\mathcal{Q}}\left({\mathcal{o}}_t,w_t\right)\right)}^2\right]$$

(73)

where

$$\widehat{\mathcal{Q}}\left({\mathcal{o}}_t,w_t\right)=r\left({\mathcal{o}}_t,w_t\right)+\gamma {\mathbb{E}}_{{\mathcal{o}}_{t+1}~{\rho }_{\pi }}\left[V_{\overline{\varnothing }}\left({\mathcal{o}}_{t+1}\right)\right]$$

(74)

where  $V_{\overline{\varnothing }}$ is the gradient of V net parameterized by  $\varnothing$.

The value function V is defined using the following expression:

$$J_V\left(\varnothing \right)={\mathbb{E}}_{{\mathcal{o}}_t~D}\left[\frac{1}{2}{\left(J_V\left({\mathcal{o}}_t\right)-{\mathbb{E}}_{{\mathcal{o}}_t~\pi }\left[{\mathcal{Q}}_{\theta }\left(w_t,{\mathcal{o}}_t\right)\right]-log\pi \left(w_t\right|{\mathcal{o}}_t)\right)}^2\right]$$

(75)

where  $D$ denotes the distribution of experience buffer.

The policy net parameterized by  $\varpi$ is trained by

$$J_{\pi }\left(\varpi \right)={\mathbb{E}}_{{\mathcal{o}}_t~D}\left(D_{KL}\left(\pi \left(·|{\mathcal{o}}_t\right)‖\frac{\mathit{exp}\left({\mathcal{Q}}_{\theta }\left({\mathcal{o}}_t,·\right)\right)}{{\mathcal{Z}}_{\theta }\left({\mathcal{o}}_t\right)}\right)\right)$$

(76)

Moreover, two additional nets including target Q-net and target policy net are adopted in the SAC algorithm which are updated by the soft updates of main nets.

4.3. Design of  $P{I}^{\phi }$-Sliding Model Control Based on SAC

The parameters of SMC play a critical role in the overall performance of the system. Thus, tunable coefficients embedded in the control law of SMC are adjusted by the soft actor-critic learning. The actor net generates regulatory signals to adaptively update the parameters of the control’s law to improve system performance. In this application, the terms of  $\Xi =\{{\xi }_i$,  $\mathfrak{z}$,  $L_P$,  $L_I$,  $\gamma$,  $m_i$} are selected as the tunable parameters of the controller which will be automatically adjusted by the deep SAC. The overall schematic of suggested  $P{I}^{\phi }$-SMC designed by soft actor-critic is depicted in Figure 2.

By training the ability of the DNNs, the actor and critic nets are trained to maximize the reward function. The reward function is defined as follows:

$$r=\frac{1}{\left|e\right|}$$

(77)

The deep neural nets (Q-net, V net, Policy net and their target nets) are made from two hidden layers (HLs) with 200 and 300 neurons. The rectified linear units (ReLU) are adopted as the activation function of the DNNs. The weights of three DNNs are trained by ADAM optimizer. The hyper-parameters of the deep SAC are listed in

Table 1. Hyper-parameters of SAC learning.

Description	Value	Description	Value
Learning rate (actor)	0.001	Discount factor (γ)	0.98
Learning rate (critic)	0.0001	Activation function	ReLU
Target smoothing coefficient	0.005	Batch size	256
Number of hidden layers	2	Mini-batch size	64
Number of neurons in hidden layers	200  $\times$ 300	Optimizer	Adam

.

Remark 5.

The SAC algorithm was chosen to optimize the parameters of the  $P{I}^{\phi }$-SMC controller due to its unique advantages in handling complex and high-dimensional control problems. SAC combines the benefits of both value-based and policy-based reinforcement learning approaches, which makes it particularly effective for optimizing control parameters in dynamic systems with challenging conditions.

Unique Advantages of SAC Compared to Other Optimization Algorithms:

• Sample Efficiency: SAC is known for its sample efficiency, meaning it requires fewer interactions with the environment to learn effective policies. This is crucial for optimizing control parameters in systems where obtaining data can be costly or time-consuming.
• Stability and Robustness: SAC utilizes off-policy learning and stable target networks, which contribute to its robustness and stability during training. This is particularly advantageous for managing the inherent uncertainties and chaotic behavior of FOMNNSs, ensuring reliable parameter optimization.
• Continuous Action Space: SAC is well suited for continuous action spaces, making it an ideal choice for optimizing control parameters in scenarios where control inputs are continuous rather than discrete. This aligns well with the needs of fine-tuning the PI^φ-SMC controller.
• Entropy Regularization: SAC incorporates entropy regularization to encourage exploration and prevent premature convergence to suboptimal solutions. This feature helps in finding more robust and optimal control policies by balancing exploration and exploitation.
• High Dimensionality Handling: SAC effectively manages high-dimensional state and action spaces, which is beneficial for complex systems like FOMNNSs where the control problem may involve numerous parameters and intricate dynamics.

Overall, SAC’s combination of sample efficiency, stability, suitability for continuous action spaces, and robust exploration makes it a superior choice for optimizing the  $P{I}^{\phi }$—SMC controller compared to other optimization algorithms such as genetic algorithms or particle swarm optimization, which may not handle the high-dimensional and continuous nature of the control problem as effectively.

5. Numerical Simulations

Here, two different numerical situations involving delayed FOMNNs are examined to demonstrate how the multi-level SMC method can be effectively applied. The numerical simulations utilized the algorithm described in [45,46], with a time-step of h = 0.01 implemented in MATLAB R2023bsoftware.

5.1. Scenario 1: In the Case of 2D Complex Delayed FOMNNSs

In this exemplar, we contemplate the illustration of two-dimensional unknown delayed FOMNNSs. Based on the delayed FOMNN (8), the following delayed FOMNN is regarded as the driving system:

$${\mathit{D}}^{\mathit{\phi }}{z}_i\left(t\right)=-k_i{z}_i\left(t\right)+\sum _{j=1}^n{p}_{ij}{\psi }_j\left(z_j\left(t\right)\right)+\sum _{j=1}^n{q}_{ij}{\psi }_j\left(z_j\left(t-\tau \right)\right)+{\wp }_i.$$

(78)

where,  $i,j=1, 2$,  $\mathit{\phi }=0.97$,  $k_1=k_2=1, \tau =1$,  $P_{2\times 2}=\left[p_{ij}\right]=\left[\begin{array}{cc}1& 6\\ 0.9& 1.8\end{array}\right]$, and  $Q_{2\times 2}=\left[q_{ij}\right]=\left[\begin{array}{cc}-1.35& 0.9\\ 0.9& -1.5\end{array}\right]$. Also,  ${\psi }_1\left(z_1\left(t\right)\right)=sin\left(\left|z_1\left(t\right)\right|-1\right)$  ${\psi }_2\left(z_2\left(t\right)\right)=tanh\left(\left|z_2\left(t\right)\right|-1\right)$, and  $z_1\left(0\right)=-4$, and  $z_2\left(0\right)=3$.

$$\begin{array}{c}{\wp }_1=0.2 sin\left({3z}_1\left(t\right)\right)+0.1 cos\left(3t\right), \\ {\wp }_2=-0.3 cos\left(z_2\left(t\right)\right)-0.1 cos\left(2t\right),\end{array}$$

(79)

and the response system is as follows:

$${\mathit{D}}^{\mathit{\phi }}{w}_i\left(t\right)=-d_i{w}_i\left(t\right)+\sum _{j=1}^n{r}_{ij}{\mu }_j\left(w_j\left(t\right)\right)+\sum _{j=1}^n{l}_{ij}{\mu }_j\left(w_j\left(t-\tau \right)\right)+{\mathcal{g}}_i+{\varphi }_i\left(u_i\left(t\right)\right)$$

(80)

where,  $d_1=d_2=1.9, R_{2\times 2}=\left[r_{ij}\right]=\left[\begin{array}{cc}-1.5& -0.2\\ -20.9& -1.8\end{array}\right]$, and  $L_{2\times 2}=\left[l_{ij}\right]=\left[\begin{array}{cc}-1.2& 0.3\\ -1.4& -1.75\end{array}\right]$. Also,  ${\psi }_j\left(w_j\left(t\right)\right)=0.5 tanh\left(\left|w_j\left(t\right)\right|\right)$, and  $w_1\left(0\right)=5$ and  $w_2\left(0\right)=-4$.

$$\begin{array}{c}{\mathcal{g}}_1=-0.15 cos\left(w_1\left(t\right)\right)+0.1 sin\left(t\right),\\ {\mathcal{g}}_2=-0.2 cos\left(w_2\left(t\right)\right)-0.1 cos\left(2t\right).\end{array}$$

(81)

The control technique parameters in Equation (52) are established as

The parameters of the control methodology (52) are set as  ${\mathcal{p}}_1={\mathcal{p}}_2=3, \nu =2.5, r=2, l=1.2,$ and  $\mathfrak{z}=1.8$. Furthermore, the parameters of the SS (21) are selected by  ${\mu }_1={\mu }_2=2$,  $q=0.92$. Moreover, the nonlinear input  ${\varphi }_i\left(u_i\left(t\right)\right)$ is introduced as

$${\varphi }_i\left(u_i\left(t\right)\right)=\left\{\begin{array}{c}3 if u_i\left(t\right)>5 \\ 0.97u_i\left(t\right) if-5\le u_i\left(t\right)\le 5 \\ -3 if u_i\left(t\right)<-5 \end{array}\right. i=1, 2.$$

(82)

Figure 3 and Figure 4 illustrate the synchronization and controlled error of the states of FO drive-response in delayed FOMNNSs (78) and (80). It is clear that the chaotic attractors of the complex FO error mechanisms quickly converged to stability points. Additionally, Figure 4 presents the time-history of the  $P{I}^{\phi }$-SMC method (52) used to synchronize the delayed FOMNNSs. The control input (52) is seen to reach equilibrium without any signs of the chattering phenomenon. This demonstrates the  $P{I}^{\phi }$-SMC method can effectively synchronize the two-dimensional delayed FOMNNSs (78) and (80). Moreover, as depicted in Figure 5, when the signals of the control laws approach the saturation boundaries, they are suppressed by the saturation condition, leading to leaping phenomena. Thus, jumping and switching states can be easily applied, especially when relays and specified saturation conditions are employed.

Figure 6 shows the time-response of the SS (21) for both function and surface plots. Clearly, each component of the SS (21) converges to zero, and there are no signs of the chattering phenomenon in the SS (21).

5.2. Scenario 2: In the Case of 3D Unknown Delayed FOMNNSs

Here, we examine the representation of two-dimensional unfamiliar delayed FOMNNSs. Using the delayed FOMNN (8) as a reference point, the subsequent delayed FOMNN is considered the leading system.

$${\mathit{D}}^{\mathit{\phi }}{z}_i\left(t\right)=-k_i{z}_i\left(t\right)+\sum _{j=1}^n{p}_{ij}{\psi }_j\left(z_j\left(t\right)\right)+\sum _{j=1}^n{q}_{ij}{\psi }_j\left(z_j\left(t-\tau \right)\right)+{\wp }_i.$$

(83)

where,  $i,j=1, 2,3$,  $\mathit{\phi }=0.98$,  $k_1=k_2=k_3=1, \tau =1.2$,  $P_{3\times 3}=\left[p_{ij}\right]=\left[\begin{array}{ccc}-2.8& 0.165& 0.29\\ 0.45& -0.9& 0.38\\ 0.18& 0.09& -2.15\end{array}\right]$, and  $Q_{3\times 3}=\left[q_{ij}\right]=\left[\begin{array}{ccc}-0.43& -0.31& 0.235\\ 0.485& -0.55& 0.25\\ 0.185& -0.225& -1.1\end{array}\right]$. Furthermore,  ${\psi }_1\left(z_1\left(t\right)\right)=-0.2 tanh\left(\left|z_1\left(t\right)\right|-1\right)+0.3 sin\left(z_1\left(t\right)\right)$,  ${\psi }_2\left(z_2\left(t\right)\right)=tanh\left(0.5z_2\left(t\right)\right)$, and  ${\psi }_3\left(z_3\left(t\right)\right)=0.5tanh\left(z_3\left(t\right)\right)$ and  $z_2\left(0\right)=3$.

$$\begin{array}{c}{\wp }_1=0.2 cos\left({3z}_1\left(t\right)\right)-0.2 cos\left(2t\right),\\ \begin{array}{c}{\wp }_2=-0.3 sin\left(z_2\left(t\right)\right)+0.1 sin\left(3t\right), \\ {\wp }_3=-0.2 sin\left({2z}_3\left(t\right)\right)+0.15 cos\left(4t\right).\end{array}\end{array}$$

(84)

And for the response system,

$${\mathit{D}}^{\mathit{\phi }}{w}_i\left(t\right)=-d_i{w}_i\left(t\right)+\sum _{j=1}^n{r}_{ij}{\mu }_j\left(w_j\left(t\right)\right)+\sum _{j=1}^n{l}_{ij}{\mu }_j\left(w_j\left(t-\tau \right)\right)+{\mathcal{g}}_i+{\varphi }_i\left(u_i\left(t\right)\right)$$

(85)

where,  $d_1=6, d_2=7, d_3=5.5,$  $R_{3\times 3}=\left[r_{ij}\right]=\left[\begin{array}{ccc}3& -2.5& 1.8\\ -1& 1.5& 2\\ -2.5& 2& -2\end{array}\right]$, and  $L_{3\times 3}=\left[l_{ij}\right]=\left[\begin{array}{ccc}1.4& -0.5& -2.8\\ -1.5& -2& 2\\ 3& -1.4& 1.6\end{array}\right]$. Also,  ${\mu }_j\left(w_j\left(t\right)\right)=tanh\left(\left|w_j\left(t\right)\right|\right)$, and  $w_1\left(0\right)=5$,  $w_2\left(0\right)=-4$ and  $w_3\left(0\right)=-4$.

$$\begin{array}{c}{\mathcal{g}}_1=-0.1 cos\left(w_1\left(t\right)\right)+0.1 sin\left(t\right),\\ \begin{array}{c}{\mathcal{g}}_2=0.2 cos\left(w_2\left(t\right)\right)-0.15 cos\left(2t\right),\\ {\mathcal{g}}_3=-0.1 sin\left(2w_3\left(t\right)\right)-0.1 cos\left(t\right).\end{array}\end{array}$$

(86)

And, for  $i=1, 2,3$, the structure of the input controller is

$${\varphi }_i\left(u_i\left(t\right)\right)=\left\{\begin{array}{c}5 for u_i\left(t\right)>5 ,\\ u_i\left(t\right) for-5\le u_i\left(t\right)\le 5, \\ -5 for u_i\left(t\right)<-5. \end{array}\right.$$

(87)

Also,  $y_1\left(0\right)=-3,y_2\left(0\right)=4$, and  $y_2\left(0\right)=-3$ show the starting values.

The parameters of the control signal (52) were selected as  ${\mathcal{p}}_1={\mathcal{p}}_2=2.5$ and  ${\mathcal{p}}_3=4, \nu =3, r=1.8, l=2.3,$ and  $\mathfrak{z}=2.6$. Plus, the parameters of the SS (21) were  ${\mu }_1=4,{\mu }_i=2$,  $q=0.94$ and  $\rho =5$.

Figure 7 and Figure 8 depict the synchronization and controlled error of the states of FO drive-response in three-dimensional delayed FOMNNSs (83) and (85). The chaotic attractors of the chaotic FO error system are quickly stabilized. Additionally, Figure 8 presents the time-history of the  $P{I}^{\phi }$-SMC method controller (52) used for synchronizing the delayed FOMNNSs (83) and (85). The control input (52) achieves equilibrium without exhibiting any signs of the chattering phenomena, demonstrating the effectiveness of the  $P{I}^{\phi }$-SMC method in synchronizing the three-dimensional delayed FOMNNSs (83) and (85). Moreover, as shown in Figure 9, when the control law signals approach the saturation limits, they are controlled by the saturation protocol, leading to leaping happening. Therefore, jumping and switching states can be easily operated, particularly when using transmits and specified saturation criteria.

Figure 10 illustrates the time-response of the sliding surface (21) used for synchronizing the delayed FOMNNSs (83) and (85). Clearly, each parameter of the sliding surface (21) converges to zero, and there are no indications of the chattering phenomenon. The sliding surfaces surface (21) converges to zero, and there are no indications of the chattering phenomenon on the sliding surfaces.

6. Discussion and Conclusions

This study presents a model-free  $P{I}^{\phi }$-SMC methodology for achieving synchronization in chaotic delayed FOMNNSs with input saturation. The approach leverages fractional-order Lyapunov stability theory to develop a two-tiered  $P{I}^{\phi }$-SMC framework, effectively addressing the inherent chaotic behavior of delayed FOMNNSs and ensuring finite-time synchronization. The methodology began with the introduction of a primary sliding surface. This was followed by the design of a robust  $P{I}^{\phi }$ sliding surface, grounded in proportional-integral (PI) rules, and the demonstration of finite-time asymptotic stability for both sliding surfaces. Subsequently, a dynamic-free control law was formulated, which is robust against system uncertainties, input saturations, and delays. This approach decouples the control signals from both nonlinear and linear components of the system, taking advantage of the norm-boundedness of chaotic states. To further enhance the performance of the  $P{I}^{\phi }$-SMC, the deep SAC learning is integrated with the control strategy, effectively removing constraints on the initial conditions of the control input and ensuring that the sliding motion adheres to the reachability condition within finite time. The proposed methodology’s validity is confirmed through comprehensive simulation results and two numerical examples, demonstrating its robustness and effectiveness in practical scenario.

For future work, we plan to extend the  $P{I}^{\phi }$-SMC methodology to other complex systems, including higher-dimensional and nonlinear systems. Future research will also focus on integrating adaptive control techniques to handle time-varying uncertainties and input saturations, and on conducting experimental validations to bridge theoretical results with practical applications. Additionally, we will explore methods to reduce computational complexity for real-time implementation and consider incorporating machine learning for system identification and adaptive control. Finally, we will perform a sensitivity analysis to better understand the robustness of our approach under varying conditions. Additionally, we plan to design and conduct experiments to calibrate the model parameters using real-world data. This can be realized through collecting data from physical systems similar to those modeled in our study and applying advanced parameter estimation techniques to fine-tune the control law coefficients.