Joint Parameter and State Estimation of Fractional-Order Singular Systems Based on Amsgrad and Particle Filter

Abstract

This article investigates modeling issues of fractional-order singular systems. The state estimation can be solved by using the particle filter. An improved Adaptive Moment Estimation (Adam) method—the Amsgrad algorithm can handle the optimization problem caused by parameter estimation. Thus, a hybrid approach that combines the particle filter and Amsgrad is proposed to estimate both parameters and states in fractional-order singular systems. This method leverages the strengths of the particle filter in handling nonlinear and high-dimensional problems, as well as the stability of the Amsgrad algorithm in optimizing parameters for dynamic systems. Then, the identification process is concluded to achieve a more accurate joint estimation. To validate the feasibility of the proposed hybrid algorithm, simulations involving three-order and four-order fractional-order singular systems are conducted. A comparative analysis with other algorithms demonstrates that the proposed method behaves better than the standard particle filter, Amsgrad and Gravitational search algorithm-Kalman filter algorithms.

1. Introduction

Recently, singular systems have been widely applied in power systems, economic systems, and electronic networks [1]. The parameter identification, control and stability analysis of singular systems are important in analyzing their dynamic indicators [2,3]. For example, Hong et al. derived a preconditioning-based method for solving singular systems [4]; a nonlinear singular density inhibition system with globally analytic solutions was analyzed in [5]; Zhang et al. developed a novel $H_{\infty }$ scheme to control singular fractional-order interval models [6]; Shu et al. investigated the optimal control strategy for uncertain discrete-time singular models under the criterion of expected value [7]; Liu et al. proposed a dissipative control scheme based on dynamic quantization for switched nonlinear singular systems [8]; and Zhang et al. introduced an event-triggered output quantization method for discrete-time Markovian singular models [9]. Thus, current research on the control methods of singular systems has reached relative maturity [10]. However, few studies have addressed the parameter identification of singular systems, not to mention the joint estimation of fractional-order singular systems. This paper aims to propose a novel method for simultaneously identifying the parameters and states of fractional-order singular systems.

The particle filter (PF) algorithm is a nonlinear filtering approach on the basis of Monte Carlo simulation. It approximates probability density functions by propagating random samples in the state space. This approach obtains minimum variance estimates of system states [11]. Thus, the PF algorithm has been widely applied in system state estimation [12]. For instance, Xie et al. developed an adaptive PF for change-point detection in profile data within manufacturing systems [13]; Michalski et al. investigated multi-particle filtering techniques for nonlinear state estimation [14]; Abolmasoumi et al. proposed a robust PF design and demonstrated its application in the state estimation of power systems [15]; and Bao et al. developed the implicit method to solve nonlinear filtering [16]. In summary, particle filters have demonstrated practical applications in the system modeling [17], but they suffer from sample degeneracy and initialization dependency issues. Thus, this paper proposes a hybrid PF algorithm to compensate for these deficiencies when estimating states of fractional-order singular systems.

The adaptive moment estimation (Adam) algorithm enhances the optimization accuracy and efficiency by adaptively adjusting learning rates and incorporating momentum [18]. Thus, Adam has been widely used in optimization issues, but it still faces challenges, including high initialization sensitivity and strong dependence on learning rate updates. To address these limitations, Wang et al. constructed the multistage artificial neural network by an optimization method which combined particle swarm optimization and Adam [19]; and Chang et al. developed the novel model using wavelet transforms and Adam-optimized Long Short-Term Memory neural networks [20]. It can be observed that the Adam algorithm currently exhibits extensive applications. However, Adam’s inherent characteristics may compromise the model’s performance and stability. This paper employs the improved Amsgrad algorithm to establish a stable learning rate update strategy for system identification, and simultaneously reduce identification errors.

This paper proposes a hybrid algorithm combining PF with Amsgrad optimization for joint parameter and state estimation of fractional-order singular systems. This method integrates the advantages of PF in handling nonlinear high-dimensional problems with the stability of Amsgrad for dynamic system parameter optimization, enabling the accurate estimation of the fractional-order singular system through a joint optimization process.

Our main contributions are listed as follows: (a) A novel identification model is developed for a class of fractional-order singular systems. (b) An enhanced Adam algorithm is implemented by incorporating the Amsgrad optimizer to improve the stability and accuracy during parameter optimization. (c) A novel algorithm is proposed by integrating PF with Amsgrad optimization. This combined approach simultaneously enhances the stability of dynamic parameter optimization and enables state correction, thus it can achieve significant reduction in parameter identification errors and good fitting of system states.

The rest of the sections of our work are shown here as follows: The system description of fractional-order singular systems is given in Section 2. The analytical derivations of both PF and Amsgrad algorithms, along with the identification process of the proposed Amsgrad-particle filter (Ams-PF) hybrid method for fractional-order singular systems, are given in Section 3. The numerical simulations of both third-order and fourth-order singular systems, and the comparative analysis are presented in Section 4. Section 5 shows some conclusions.

2. System Description

The Grünwald–Letnikov (GL) definition is defined as [21]

$$\begin{array}{c}\hfill D^{\alpha }x\left(k\right)=\underset{h\to 0}{lim}{\displaystyle \frac{1}{h^{\alpha }}}\sum _{i=0}^{\infty }{(-1)}^i\left(\begin{array}{c}\alpha \\ i\end{array}\right)x(k-ih),\end{array}$$

here $\alpha$ and h are the fractional order and sampling time. Theoretically, the upper limit for summation needs to range from 0 to ∞. However, infinite summation is not feasible, and it usually requires truncation to a finite number of terms in numerical calculations. Thus, directly setting a fixed memory length m and only retaining the most recent $m+1$ items yield [22]

$$\begin{array}{c}\hfill D^{\alpha }x\left(mh\right)\approx {\displaystyle \frac{1}{h^{\alpha }}}\sum _{i=0}^m{(-1)}^i\left(\begin{array}{c}\alpha \\ i\end{array}\right)x\left((m-i)h\right).\end{array}$$

Based on empirical values, m is usually taken as a large number (such as $m=$ 1000–5000) to make the truncation error acceptable. The Newton’s binomial factor $\left(\begin{array}{c}\alpha \\ i\end{array}\right)$ is computed by Euler’s function as [21]

$$\begin{array}{c}\hfill \left(\begin{array}{c}\alpha \\ i\end{array}\right)={\displaystyle \frac{\Gamma (\alpha +1)}{\Gamma (i+1)\Gamma (\alpha -i+1)}},\end{array}$$

where $\Gamma (·)$ is

$$\begin{array}{c}\hfill \Gamma \left(\alpha \right)={\int }_0^{\infty }{e}^{-k}{k}^{\alpha -1}dk.\end{array}$$

When h is 1 in the GL definition, the discrete fractional difference of the state vector $D^{\alpha }x(k-i)$ turns into [22]

$$D^{\alpha }x(k-i)=\sum _{j=0}^{k-i}{(-1)}^j\left({\displaystyle \genfrac{}{}{0pt}{}{\alpha }{j}}\right)x(k-i-j).$$

The fractional-order singular systems considered in this paper are formally defined by the following mathematical representation [23]:

$$\begin{array}{ccc}\hfill \mathit{E}{D}^{\alpha }\mathit{x}(k+1)& =& \mathit{A}{D}^{\alpha }\mathit{x}\left(k\right)+\mathit{B}u\left(k\right),\hfill \end{array}$$

(1)

$$\begin{array}{ccc}\hfill y\left(k\right)& =& \mathit{C}{D}^{\alpha }\mathit{x}\left(k\right)+v\left(k\right),\hfill \end{array}$$

(2)

$\mathit{x}\left(k\right)$ denotes the state vector of the singular system containing all state variables, $u\left(k\right)$ is the input signal, $y\left(k\right)$ is the output signal, $v\left(k\right)$ represents the measurement noise and chooses the Gaussian noise with zero mean and variance ${\sigma }^2$. $\mathit{E}$ is a singular matrix, typically of non-full rank. $\mathit{A}$, $\mathit{B}$, $\mathit{C}$ represent the state transition, control and output matrices as follows:

$$\begin{array}{ccc}\hfill \mathit{E}& =& \left[\begin{array}{ccccc}1& 0& 0& \dots & 0\\ 0& 1& 0& \dots & 0\\ ⋮& \ddots & \ddots & \ddots & ⋮\\ 0& \dots & 0& 1& 0\\ e_1& e_2& \dots & e_{n-1}& 0\end{array}\right]\in {\mathbb{R}}^{n\times n},\hfill \\ \hfill \mathit{A}& =& \left[\begin{array}{ccccc}0& 1& 0& \dots & 0\\ ⋮& \ddots & \ddots & \ddots & ⋮\\ 0& \dots & 0& 1& 0\\ a_1& a_2& \dots & a_{n-1}& 1\\ 0& 0& \dots & 0& 1\end{array}\right]\in {\mathbb{R}}^{n\times n},\hfill \\ \hfill \mathit{B}& =& {[b_1,b_2,\dots ,b_{n-1},1]}^{\mathrm{T}}\in {\mathbb{R}}^n,\hfill \\ \hfill \mathit{C}& =& [1,0,0,\dots ,0]\in {\mathbb{R}}^{1\times n}.\hfill \end{array}$$

Let $\underline{\mathit{x}}(k+1)=D^{\alpha }\mathit{x}(k+1)$. According to the input-output relationship described by (1) and (2), along with the definitions of $\mathit{E}$, $\mathit{A}$ and $\mathit{B}$, we can derive the following:

$$\begin{array}{c}\hfill \left[\begin{array}{c}{\underline{x}}_1(k+1)\\ ⋮\\ {\underline{x}}_{n-2}(k+1)\\ {\underline{x}}_{n-1}(k+1)\\ e_1{\underline{x}}_1(k+1)+\dots +e_{n-1}{\underline{x}}_{n-1}(k+1)\end{array}\right]=\left[\begin{array}{c}{\underline{x}}_2\left(k\right)\\ ⋮\\ {\underline{x}}_{n-1}\left(k\right)\\ a_1{\underline{x}}_1\left(k\right)+\dots +a_{n-1}{\underline{x}}_{n-1}\left(k\right)+{\underline{x}}_n\left(k\right)\\ {\underline{x}}_n\left(k\right)\end{array}\right]+\left[\begin{array}{c}{b}_{1}u\left(k\right)\\ ⋮\\ b_{n-2}u\left(k\right)\\ b_{n-1}u\left(k\right)\\ u\left(k\right)\end{array}\right].\end{array}$$

(3)

Then, expanding the matrix Equation (3) yields the following:

$$\begin{array}{ccc}\hfill {\underline{x}}_1(k+1)& =& {\underline{x}}_2\left(k\right)+b_{1}u\left(k\right),\hfill \\ \hfill ⋮\\ \hfill {\underline{x}}_{n-2}(k+1)& =& {\underline{x}}_{n-1}\left(k\right)+b_{n-2}u\left(k\right),\hfill \\ \hfill {\underline{x}}_{n-1}(k+1)& =& a_1{\underline{x}}_1\left(k\right)+\dots +a_{n-1}{\underline{x}}_{n-1}\left(k\right)+{\underline{x}}_n\left(k\right)+b_{n-1}u\left(k\right),\hfill \\ \hfill {\underline{x}}_n\left(k\right)+u\left(k\right)& =& e_1{\underline{x}}_1(k+1)+\dots +e_{n-1}{\underline{x}}_{n-1}(k+1).\hfill \end{array}$$

(4)

According to the last equation of (4), we can obtain the following:

$$\begin{array}{c}\hfill {\underline{x}}_n\left(k\right)=e_1{\underline{x}}_1(k+1)+\dots +e_{n-1}{\underline{x}}_{n-1}(k+1)-u\left(k\right).\end{array}$$

(5)

Substituting (5) into the (n − 1)-th equation of (4) yields the following:

$$\begin{array}{ccc}\hfill {\underline{x}}_{n-1}(k+1)& =& a_1{\underline{x}}_1\left(k\right)+\dots +a_{n-1}{\underline{x}}_{n-1}\left(k\right)+e_1{\underline{x}}_1(k+1)+\dots +e_{n-1}{\underline{x}}_{n-1}(k+1)\hfill \\ & & +(b_{n-1}-1)u\left(k\right).\hfill \end{array}$$

Based on the recursive relationship, we can derive the following key results:

$$\begin{array}{ccc}\hfill {\underline{x}}_1\left(k\right)& =& {\underline{x}}_2(k-1)+b_{1}u(k-1)\hfill \\ & & ⋮\hfill \\ & =& {\underline{x}}_{n-1}(k-n+2)+b_{n-2}u(k-n+2)+\dots +b_{2}u(k-2)+b_{1}u(k-1)\hfill \\ & =& [a_1{\underline{x}}_1(k-n+1)+\dots +a_{n-1}{\underline{x}}_{n-1}(k-n+1)+e_1{\underline{x}}_1(k-n+2)+\dots \hfill \\ & & +e_{n-1}{\underline{x}}_{n-1}(k-n+2)+(b_{n-1}-1)u(k-n+1)]+b_{n-2}u(k-n+2)\hfill \\ & & +\dots +b_{2}u(k-2)+b_{1}u(k-1).\hfill \end{array}$$

(6)

Substituting the output matrix $\mathit{C}$ into (2) yields the following:

$$\begin{array}{c}\hfill y\left(k\right)={\underline{x}}_1\left(k\right)+v\left(k\right).\end{array}$$

The input-output relationship is finally obtained as follows:

$$\begin{array}{ccc}\hfill y\left(k\right)& =& a_1{\underline{x}}_1(k-n+1)+\dots +a_{n-1}{\underline{x}}_{n-1}(k-n+1)+e_1{\underline{x}}_1(k-n+2)+\dots \hfill \\ & & +e_{n-1}{\underline{x}}_{n-1}(k-n+2)+(b_{n-1}-1)u(k-n+1)+b_{n-2}u(k-n+2)+\dots \hfill \\ & & +b_{2}u(k-2)+b_{1}u(k-1)+v\left(k\right).\hfill \end{array}$$

If the parameter and information vectors are defined as follows:

$$\begin{array}{ccc}\hfill \mathit{\theta }& =& {\left[a_1,\dots ,a_{n-1},e_1,\dots ,e_{n-1},b_1,\dots ,b_{n-1}-1\right]}^{\mathrm{T}}\in {\mathbb{R}}^{3(n-1)},\hfill \end{array}$$

$$\begin{array}{ccc}\hfill \mathit{\phi }\left(k\right)& =& [{\underline{x}}_1(k-n+1),\dots ,{\underline{x}}_{n-1}(k-n+1),\hfill \\ \hfill & & {\underline{x}}_1(k-n+2),\dots ,{\underline{x}}_{n-1}(k-n+2),\hfill \end{array}$$

(7)

$$\begin{array}{ccc}& & u(k-1),\dots ,u(k-n+1){]}^{\mathrm{T}}\in {\mathbb{R}}^{3(n-1)}.\hfill \end{array}$$

(8)

The corresponding identification model is obtained as follows:

$$\begin{array}{c}\hfill y\left(k\right)={\mathit{\phi }}^{\mathrm{T}}\left(k\right)\mathit{\theta }+v\left(k\right).\end{array}$$

3. Algorithm Derivation

In practical applications, many dynamic systems face the challenge of incomplete parameter knowledge or singular system matrices. Thus, traditional parameter estimation methods may fail to obtain accurate results. To address these issues, this study proposes a combined approach using particle filter (PF) and Amsgrad optimization. The PF demonstrates strong nonlinear processing capabilities, which can effectively handle state estimation in complex dynamic systems. The Amsgrad algorithm, as an improved version of Adam optimization, can enhance the stability in the parameter identification [24]. By integrating these two methods, this research aims to obtain a joint estimation of both parameters and states in fractional-order singular systems.

3.1. Particle Filter (PF)

The PF is particularly suitable for state estimation in linear and nonlinear systems because it approximates posterior probability distributions using particles. This approach does not rely on system linearization or Gaussian noise assumptions. Thus, it is suitable for fractional singular systems because the long-term memory effect introduced by fractional derivatives gives the system certain nonlinear characteristics. In this study, the PF algorithm is primarily used to estimate system states. The main procedures are as follows:

(1) The particle initialization is performed first. The particle number is set as N. The states are initialized according to the distributions of input $u\left(k\right)$ and noise $v\left(k\right)$. Each state of the particle is randomly initialized through a Gaussian distribution. Every particle represents a potential state solution in the system state space as

$${\widehat{\underline{\mathit{x}}}}^i\left(0\right)={\mathbf{1}}_n+\sqrt{\sigma }·\mathrm{randn}(n,1),$$

(9)

${\widehat{\underline{\mathit{x}}}}^i\left(0\right)$ represents the initial state of the i-th particle, which is also the estimated value of the initial state vector in the singular system. The notation ${\mathbf{1}}_n$ is an n-dimensional column vector with all elements equal to 1. $\sigma$ is the noise variance and $\mathrm{randn}(n,1)$ generates a column vector of random numbers following the standard normal distribution.

(2) Assume the data length is K. On the basis of Bayesian filter technique, the estimation for unknown states ${\underline{x}}_l\left(k\right),l=1,\dots ,n$ is transformed into processing the posterior probability density function $p\left({\underline{x}}_l^i\left(k\right)\right|y(1:K),u(1:K),\widehat{\mathit{\theta }}(k-1))$ [25]. According to the maximum a posteriori criterion and Monte Carlo methods, the posterior probability density function of the intermediate term calculated from the estimated parameter vector $\widehat{\mathit{\theta }}(k-1)$ is approximated as follows [11]:

$$\begin{array}{c}\hfill p\left({\underline{x}}_l^i\left(k\right)\right|y(1:K),u(1:K),\widehat{\mathit{\theta }}(k-1))\approx \sum _{i=1}^N{w}_l^i\left(k\right)\Delta ({\underline{x}}_l^i\left(k\right)-{\widehat{\underline{x}}}_l^i\left(k\right)),\end{array}$$

(10)

where $w_l^i\left(k\right)$ represents the normalized weight of particles during recursion, and $\Delta (·)$ indicates the Dirac delta function. ${\widehat{\underline{x}}}_l^i\left(k\right)$ denotes the state of i-th particle that is drawn from the posterior probability density function $p\left({\underline{x}}_l^i\left(k\right)\right|y(1:K),u(1:K),\widehat{\mathit{\theta }}(k-1))$.

(3) Nevertheless, directly extracting samples from the true posterior probability density function may be hard. To generate new sampling particles, the importance probability density function $d(·)$ is introduced and replaces the true posterior probability density function [26]. Besides, $d(·)$ must be easily sampled and can be chosen as the following measurement [12]:

$$\begin{array}{c}\hfill d\left({\underline{x}}_l^i\left(k\right)\right|y(1:K),u(1:K),\widehat{\mathit{\theta }}(k-1))=p\left({\underline{x}}_l^i\left(k\right)\right|{\underline{x}}_l^i(k-1),\widehat{\mathit{\theta }}(k-1)).\end{array}$$

Let $X_l^i\left(k\right)=\{{\widehat{\underline{x}}}_l^i\left(k\right),{\widehat{\underline{x}}}_l^i(k-1),\dots ,{\widehat{\underline{x}}}_l^i\left(1\right)$}, $Y\left(k\right)=\left\{y\left(k\right),y(k-1),\dots ,y\left(1\right)\right\}$. Based on the above equation, new particles are generated and their weights $w_l^i\left(k\right)$ can be recursively computed as

$$\begin{array}{ccc}\hfill w_l^i\left(k\right)& =& {\displaystyle \frac{p(X_l^i\left(k\right),Y\left(k\right))}{d\left(X_l^i\left(k\right)\right|Y\left(k\right))}}\hfill \\ & =& {\displaystyle \frac{p({\widehat{\underline{x}}}_l^i\left(k\right),y\left(k\right),X_l^i(k-1),Y(k-1))}{d\left({\widehat{\underline{x}}}_l^i\left(k\right)\right|X_l^i(k-1),Y\left(k\right))d\left(X_l^i(k-1)\right|Y\left(k\right))}}\hfill \\ & =& {\displaystyle \frac{p\left(y\left(k\right)\right|{\widehat{\underline{x}}}_l^i\left(k\right),X_l^i(k-1),Y_l(k-1))p\left({\widehat{\underline{x}}}_l^i\left(k\right)\right|X_l^i(k-1),Y(k-1))}{d\left({\widehat{\underline{x}}}_l^i\left(k\right)\right|X_l^i(k-1),Y\left(k\right))}}\hfill \\ & & \times {\displaystyle \frac{p\left(X_l^i(k-1)\right|Y(k-1))}{d\left(X_l^i(k-1)\right|Y(k-1))}}\hfill \\ & =& {\displaystyle \frac{p\left(y\left(k\right)\right|{\widehat{\underline{x}}}_l^i\left(k\right))p\left({\widehat{\underline{x}}}_l^i\left(k\right)\right|y(k-1),\dots ,y\left(1\right),u\left(1\right),\dots ,u\left(k\right),\widehat{\mathit{\theta }}(k-1))}{d\left({\widehat{\underline{x}}}_l^i\left(k\right)\right|y\left(1\right),\dots ,y\left(k\right),u\left(1\right),\dots ,u\left(k\right),\widehat{\mathit{\theta }}(k-1))}}{w}_l^i(k-1)\hfill \\ & =& p\left(y\left(k\right)\right|{\widehat{\underline{x}}}_l^i\left(k\right))w_l^i(k-1).\hfill \end{array}$$

(11)

Then, the probability density function $p\left(y\left(k\right)\right|{\widehat{\underline{x}}}_l^i\left(k\right))$ is computed as [15]:

$$p\left(y\left(k\right)\right|{\widehat{\underline{x}}}_l^i\left(k\right))={\displaystyle \frac{1}{\sqrt{2\pi }\sigma }}exp\left[-{\displaystyle \frac{{(y\left(k\right)-{\widehat{\underline{x}}}_l^i\left(k\right))}^2}{2{\sigma }^2}}\right].$$

By substituting $p\left(y\left(k\right)\right|{\widehat{\underline{x}}}_l^i\left(k\right))$ from (11) into the weight calculation formula, we can obtain the following:

$$w_l^i\left(k\right)={\displaystyle \frac{1}{\sqrt{2\pi }\sigma }}exp\left[-{\displaystyle \frac{{(y\left(k\right)-{\widehat{\underline{x}}}_l^i\left(k\right))}^2}{2{\sigma }^2}}\right]w_l^i(k-1).$$

(12)

All particles can be directly extracted from the importance probability density function $p\left({\underline{x}}_l(k-1)\right|{\widehat{\underline{x}}}_l^i(k-1),\widehat{\mathit{\theta }}(k-1))$. Then, weights can be normalized and calculated as follows [11]:

$${\tilde{w}}_l^i\left(k\right)={\displaystyle \frac{w_l^i\left(k\right)}{{\displaystyle \sum _{i=1}^N}{w}_l^i\left(k\right)}}.$$

(13)

The normalized weight ${\tilde{w}}^i\left(k\right)$ ensures the rationality of particle weights and the validity of the probability distribution. Through normalization, it prevents numerical instability and error accumulation in computations, thereby improving the accuracy of state estimation.

(4) After some recursions of the above-mentioned methods, the particles have shown a certain degree of degradation. Then, a resampling approach can be introduced. The core idea of resampling is to replicate and eliminate particles based on their weights. Specifically, particles with higher weights are duplicated, while those with lower weights are discarded. After resampling, the total number of particles remains unchanged, but their distribution becomes more concentrated in high-weight regions. All particle weights are then reset to identical values. This process effectively prevents particle degeneracy by preferentially selecting particles with larger weights. The resampled particles are subsequently used as the initial particles for the next state update. Through the aforementioned process, the particle weights become ${\tilde{w}}_l^i\left(k\right)={\displaystyle \frac{1}{N}}$ and the updated state can be obtained. The estimated state value becomes

$${\widehat{\underline{x}}}_l\left(k\right)={\displaystyle \frac{1}{N}}\sum _{i=1}^N{\widehat{\underline{x}}}_l^i\left(k\right).$$

The above steps are repeated until the predetermined recursion count is reached or the stopping condition is satisfied. In this way, the state estimation of fractional-order singular systems can be achieved.

3.2. Amsgrad Optimization

As is well known, the Adam algorithm is an improved method based on the traditional gradient descent. The gradient descent can maintain a constant learning rate throughout the parameter optimization process, which is relatively simplistic. In contrast, Adam dynamically modifies the learning rate during parameter identification based on some ongoing results. By computing both first-order and second-order moments of gradient, it enables an adaptive learning mode [27]. Thus, the Adam algorithm has been widely applied in various fields due to the efficient optimization capability [28]. The Amsgrad algorithm employed in this paper is an improved version of the Adam optimizer. Its primary advantage lies in adjusting the second-order moment of the gradient, thereby effectively preventing gradient explosion or vanishing issues. Thus, the Amsgrad can provide more stable optimization during dynamic system parameter optimization than the Adam method.

The number of parameters to be identified is $3(n-1)$. The update rules of Amsgrad are implemented through the following steps:

(1) First, the optimization parameters ${\beta }_1$, ${\beta }_2$, V, S, $\epsilon$ are initialized. Let k denote the current recursion time. These parameters are determined based on parameter vector estimates. Thus, the criterion function is defined as follows [29]:

$$J\left(\widehat{\mathit{\theta }}\left(k\right)\right)={\displaystyle \frac{1}{2}}{v}^2\left(k\right)={\displaystyle \frac{1}{2}}{[y\left(k\right)-{\mathit{\phi }}^{\mathrm{T}}\left(k\right)\widehat{\mathit{\theta }}\left(k\right)]}^2,$$

$\widehat{\mathit{\theta }}\left(k\right)$ is the estimate of the parameter vector at the k-th recursion. ${\mathit{\phi }}^{\mathrm{T}}\left(k\right)\widehat{\mathit{\theta }}\left(k\right)$ calculates estimated outputs and $y\left(k\right)-{\mathit{\phi }}^{\mathrm{T}}\left(k\right)\widehat{\mathit{\theta }}\left(k\right)$ represents errors between estimated outputs and actual outputs. The gradient is computed as the partial derivative of the criterion function with respect to the estimated parameter vector as follows:

$$\nabla J\left(\widehat{\mathit{\theta }}\left(k\right)\right)={\displaystyle \frac{\partial J\left(\widehat{\mathit{\theta }}\left(k\right)\right)}{\partial \widehat{\mathit{\theta }}\left(k\right)}}=\mathit{\phi }\left(k\right)[y\left(k\right)-{\mathit{\phi }}^{\mathrm{T}}\left(k\right)\widehat{\mathit{\theta }}\left(k\right)].$$

(2) The first-order moment estimate $V\left(k\right)$ and its bias-corrected version $\widehat{V}\left(k\right)$ are defined as follows [30]:

$$V\left(k\right)={\beta }_1\left(k\right)V(k-1)+(1-{\beta }_1\left(k\right))\nabla J\left(\widehat{\mathit{\theta }}\left(k\right)\right),({\beta }_1\left(k\right)={\displaystyle \frac{{\beta }_1}{\sqrt{k}}}),$$

(14)

$$\widehat{V}\left(k\right)={\displaystyle \frac{V\left(k\right)}{1-{\beta }_1^k\left(k\right)}},$$

(15)

where ${\beta }_1$ is the factor that controls the first-order moment estimate utilizing exponential weighted averaging. The above Equation (14) employs the first-order moment estimate of the criterion function to counteract oscillations in non-optimal directions during gradient descent and maintains continuity in optimal directions. During the initial optimization stages, $V\left(k\right)$ tends to be biased towards its initial value, thus it requires bias compensation through the above operation (15).

(3) The second-order moment estimate $S\left(k\right)$ and its bias-corrected version $\widehat{S}\left(k\right)$ are calculated as follows [30]:

$$S\left(k\right)={\beta }_{2}S(k-1)+(1-{\beta }_2)\nabla J^{\mathrm{T}}\left(\widehat{\mathit{\theta }}\left(k\right)\right)\nabla J\left(\widehat{\mathit{\theta }}\left(k\right)\right),({\displaystyle \frac{{\beta }_1\left(k\right)}{\sqrt{{\beta }_2}}}⩽\sqrt{c}<1),$$

(16)

$$\widehat{S}\left(k\right)={\displaystyle \frac{S\left(k\right)}{1-{\beta }_2^k}},$$

(17)

where ${\beta }_2$ is the factor that controls the second-order moment estimate utilizing exponential weighted averaging. The above Equation (16) employs the second-order moment estimate of the criterion function to achieve adaptive control of the learning rate [29]. At the beginning of optimization, $S\left(k\right)$ tends to be biased towards its initial value, thus it requires bias compensation through (17). To address the limitations in the Adam algorithm, the Amsgrad algorithm makes an improvement by modifying (17) as follows [31]:

$$\widehat{S}\left(k\right)=max(\widehat{S}(k-1),{\displaystyle \frac{S\left(k\right)}{1-{\beta }_2^k}}).$$

(18)

By taking the maximum value operation, we improve the second-moment estimate in the Adam algorithm to prevent it from becoming too small or unstable. This enhances the robustness of the algorithm, avoids premature learning rate decay, and adapts to non-stationary objective functions. Thus, the parameter update process is stabilized. The proposed strategy is applicable to parameter optimization tasks that require stable training and avoid excessively large or small learning rates.

(4) The synergistic effect between the first-moment and second-moment estimates enables faster and better parameter optimization than Adam. The update of the parameter vector is [31]

$$\widehat{\mathit{\theta }}\left(k\right)=\widehat{\mathit{\theta }}(k-1)-{\displaystyle \frac{\lambda \left(k\right)\widehat{V}\left(k\right)}{\sqrt{\widehat{S}\left(k\right)}+\epsilon }},(\lambda \left(k\right)={\displaystyle \frac{\lambda }{\sqrt{k}}}),$$

(19)

where $\lambda \left(k\right)$ denotes the learning rate, and $\epsilon$ is a small constant used to prevent the occurrence of a zero error. At last, the parameter estimation of fractional-order singular systems is completed.

Remark 1.

The adjustment method of hyperparameters for Amsgrad are as follows:

• Learning rate λ: Initial values should be higher for smooth problems and lower for noisy or nonlinear systems. Time decay ($\lambda \left(k\right)=\lambda /\sqrt{k}$) is implemented to balance rapid convergence in the early stage and the precise refinement in the later stage.
• First-order moment factor ${\beta }_1$: For fractional-order systems with long memory effects, dynamic ${\beta }_1$ decay (${\beta }_1\left(k\right)={\beta }_1/\sqrt{k}$) is used to mitigate historical gradient interference. And its initial value is close to 1.
• Second-order moment factor ${\beta }_2$: Maintaining ${\beta }_2\approx 0.999$ with max operation of Amsgrad can prevent premature learning rate decay while preserving gradient variance information.
• Stability constant ε: Setting ε to a tiny number such as ${10}^{-6}$ can prevent the denominator of (19) from reporting an error when the second-order moment estimate $\widehat{S}\left(k\right)$ is 0.

3.3. Joint Estimation Based on Amsgrad-Particle Filter (Ams-PF)

In the gradient calculation of Amsgrad, the vector $\mathit{\phi }\left(k\right)$ contains the unknown states

${\underline{x}}_1(k-n+1),\dots ,{\underline{x}}_{n-1}(k-n+1),x_1(k-n+2),\dots ,{\underline{x}}_{n-1}(k-n+2)$. These states can be estimated through PF. Meanwhile, the state update in PF includes the matrices $\mathit{E},\mathit{A},\mathit{B}$, whose unknown parameters can be identified by the Amsgrad method. Thus, the PF and the Amsgrad optimization operate alternately during the recursive process to achieve accurate estimation of both system states and parameters.

(1) During the state update process, the parameter estimates based on Amsgrad are obtained. The optimized parameter vector $\widehat{\mathit{\theta }}\left(k\right)$ from Amsgrad contains estimates of all parameters, which means

$$\begin{array}{c}\hfill \widehat{\mathit{\theta }}\left(k\right)={[{\widehat{a}}_1\left(k\right),\dots ,{\widehat{a}}_{n-1}\left(k\right),{\widehat{e}}_1\left(k\right),\dots ,{\widehat{e}}_{n-1}\left(k\right),{\widehat{b}}_1\left(k\right),\dots ,{\widehat{b}}_{n-1}\left(k\right)-1]}^{\mathrm{T}}.\end{array}$$

These parameter estimates are used to update the states in PF, that is

$$\widehat{\mathit{E}}\left(k\right)D^{\alpha }{\widehat{\underline{\mathit{x}}}}^i(k+1)=\widehat{\mathit{A}}\left(k\right){\widehat{\underline{\mathit{x}}}}^i\left(k\right)+\widehat{\mathit{B}}\left(k\right)u\left(k\right).$$

where $\widehat{\mathit{E}}\left(k\right),\widehat{\mathit{A}}\left(k\right),\widehat{\mathit{B}}\left(k\right)$ are estimates of the state transition, control and output matrices, respectively. They are constructed as follows:

$$\begin{array}{ccc}\hfill \widehat{\mathit{E}}\left(k\right)& =& \left[\begin{array}{ccccc}1& 0& 0& \dots & 0\\ 0& 1& 0& \dots & 0\\ ⋮& \ddots & \ddots & \ddots & ⋮\\ 0& \dots & 0& 1& 0\\ {\widehat{e}}_1\left(k\right)& {\widehat{e}}_2\left(k\right)& \dots & {\widehat{e}}_{n-1}\left(k\right)& 0\end{array}\right]\in {\mathbb{R}}^{n\times n},\hfill \\ \hfill \widehat{\mathit{A}}\left(k\right)& =& \left[\begin{array}{ccccc}0& 1& 0& \dots & 0\\ ⋮& \ddots & \ddots & \ddots & ⋮\\ 0& \dots & 0& 1& 0\\ {\widehat{a}}_1\left(k\right)& {\widehat{a}}_2\left(k\right)& \dots & {\widehat{a}}_{n-1}\left(k\right)& 1\\ 0& 0& \dots & 0& 1\end{array}\right]\in {\mathbb{R}}^{n\times n},\hfill \\ \hfill \widehat{\mathit{B}}\left(k\right)& =& {[{\widehat{b}}_1\left(k\right),{\widehat{b}}_2\left(k\right),\dots ,{\widehat{b}}_{n-1}\left(k\right),1]}^{\mathrm{T}}\in {\mathbb{R}}^n.\hfill \end{array}$$

The updated state vector ${\widehat{\underline{\mathit{x}}}}_{est}^i\left(k\right)$ is calculated as follows:

$${\widehat{\underline{\mathit{x}}}}_{next}^i\left(k\right)=\widehat{\mathit{A}}\left(k\right){\widehat{\underline{\mathit{x}}}}^i(k-1)+\widehat{\mathit{B}}\left(k\right)u(k-1),$$

(20)

$${\widehat{\underline{\mathit{x}}}}_{est}^i\left(k\right)=\mathrm{pinv}\left(\widehat{\mathit{E}}\left(k\right)\right)·{\widehat{\underline{\mathit{x}}}}_{next}^i\left(k\right),$$

(21)

$${\widehat{\underline{\mathit{x}}}}_{est}^i\left(k\right)={\left[{\widehat{\underline{x}}}_{est,1}^i\left(k\right),\dots ,{\widehat{\underline{x}}}_{est,l}^i\left(k\right),\dots ,{\widehat{\underline{x}}}_{est,n}^i\left(k\right)\right]}^{\mathrm{T}},l=1,\dots ,n.$$

The final estimated state is as follows:

$${\widehat{\underline{x}}}_l\left(k\right)={\displaystyle \frac{1}{N}}\sum _{i=1}^N{\widehat{\underline{x}}}_{est,l}^i\left(k\right).$$

(22)

(2) During the Amsgrad optimization process, we also utilize the updated $\widehat{\mathit{\phi }}\left(k\right)$ obtained from PF as

$$\begin{array}{cc}\hfill \widehat{\mathit{\phi }}\left(k\right)=[& {\widehat{\underline{x}}}_1(k-n+1),\dots ,{\widehat{\underline{x}}}_{n-1}(k-n+1),\hfill \\ \hfill & {\widehat{\underline{x}}}_1(k-n+2),\dots ,{\widehat{\underline{x}}}_{n-1}(k-n+2),\hfill \\ \hfill & {u(k-1),\dots ,u(k-n+1)]}^{\mathrm{T}}\in {\mathbb{R}}^{3(n-1)},\hfill \end{array}$$

(23)

The new criterion function is defined as follows:

$$\tilde{J}\left(\widehat{\mathit{\theta }}\left(k\right)\right)={\displaystyle \frac{1}{2}}{[y\left(k\right)-{\widehat{\mathit{\phi }}}^{\mathrm{T}}\left(k\right)\widehat{\mathit{\theta }}\left(k\right)]}^2,$$

(24)

The gradient expression of (24) with respect to $\widehat{\mathit{\theta }}\left(k\right)$ becomes

$$\nabla \tilde{J}\left(\widehat{\mathit{\theta }}\left(k\right)\right)={\displaystyle \frac{\partial \tilde{J}\left(\widehat{\mathit{\theta }}\left(k\right)\right)}{\partial \widehat{\mathit{\theta }}\left(k\right)}}=\widehat{\mathit{\phi }}\left(k\right)[y\left(k\right)-{\widehat{\mathit{\phi }}}^{\mathrm{T}}\left(k\right)\widehat{\mathit{\theta }}\left(k\right)].$$

(25)

The following optimization process is as described in Section 3.1.

(3) At each time step, the PF and the Amsgrad optimization alternate iteratively. The joint Amsgrad and particle filter (Ams-PF) algorithm operates through the following steps to identify fractional-order singular systems:

• Initialize the particle number N, the recursion count K, and set parameters ${\beta }_1,{\beta }_2,\lambda$ and $\epsilon$. Small initial variables are defined as $\widehat{\mathit{\theta }}\left(0\right)={\mathbf{1}}_{3(n-1)}/p_0,$$\mathit{x}\left(0\right)=\widehat{\underline{\mathit{x}}}\left(0\right)={\mathbf{1}}_n/p_0,$ $y\left(0\right)=1/p_0,w_l\left(0\right)=1/p_0$, $p_0$ is a relatively large number.
• Collect $u\left(k\right)$ and $y\left(k\right)$, then set $k=1$. Construct parameter and information vectors $\mathit{\theta }$, $\mathit{\phi }\left(k\right)$ by (7) and (8).
• The system states based on the PF phase begins. Firstly, solutions of N particles are initialized according to (9). Then, update both the particle weights and states using (13), (20) and (21), and implement the resampling strategy. Finally, the particle weights are normalized to recursively obtain the state estimates ${\widehat{\underline{x}}}_l\left(k\right)$ by (22).
• Construct the information vector estimate $\widehat{\mathit{\phi }}\left(k\right)$ based on (23). Design the criterion function $\tilde{J}\left(\widehat{\mathit{\theta }}\left(k\right)\right)$ according to (24) and compute its gradient value $\nabla \tilde{J}\left(\widehat{\mathit{\theta }}\left(k\right)\right)$ using (25).
• In the Amsgrad optimization phase, the system parameters are optimized. Firstly, the first moment $V\left(k\right)$ and second moment $S\left(k\right)$ are computed using (14) and (16), respectively. The parameter estimates are updated via gradient information, ensuring the system output converges closer to the true observed values. Then, compute the bias-corrected first moment estimate $\widehat{V}\left(k\right)$ by (15) and the bias-corrected second moment estimate $\widehat{S}\left(k\right)$ by (17). Besides, perform the maximum value operation based on (18). Finally, calculate the estimated parameter vector $\widehat{\mathit{\theta }}\left(k\right)$ by (19).
• Increase k by 1 and return to Step 3. Continue the recursive computation until k reaches the total data length K.

The combined approach benefits from the PF’s capability to efficiently estimate states in nonlinear and high-dimensional systems, while the Amsgrad optimization ensures stable and effective joint estimation through adaptive learning rate adjustment and gradient estimation. In summary, Figure 1 exhibits the flowchart of Ams-PF.

Remark 2.

The traditional gradient descent method is relatively sensitive to the initial parameters. If the initial particle distribution deviates too far from the true parameters, it may lead to convergence to a local optimum or divergence. Amsgrad retains the exponential moving average of the squares of historical gradients and prevents premature decay of the learning rate through maximum value constraints. In the parameter estimation stage, even if the initial particle distribution is poor, this mechanism can more stably adjust the parameter update step size and gradually correct the direction through long-term gradient information. In the joint optimization, the parameter estimation error propagates to the prediction stage of the particle filter through the state transition equation. Amsgrad reduces the dominant influence of the initial large gradient on subsequent updates by adaptively normalizing the parameter gradient, making the parameter estimation process more dependent on long-term statistical features rather than initial randomness.

3.4. Convergence and Error Bound Proof of the Ams-PF Algorithm

The following premise assumptions are given:

(1) For any k, $\tilde{J}\left(\widehat{\mathit{\theta }}\left(k\right)\right)$ is a convex function of $\widehat{\mathit{\theta }}\left(k\right)$;

(2) For any dimension j of the parameter vector, they are bounded as

$$\parallel {\theta }_j-{\widehat{\theta }}_j{\left(k\right)\parallel }_2⩽D_j,\forall {\theta }_j,{\widehat{\theta }}_j\left(k\right);$$

(3) For any dimension j of the gradient, they are bounded as

$$\parallel \nabla {\tilde{J}}_j\left(\widehat{\mathit{\theta }}\left(k\right)\right){\parallel }_2⩽G_j,\forall t.$$

Let ${\mathit{\theta }}^{*}=argmi{n}_{\widehat{\mathit{\theta }}}{\sum }_{k=1}^K\tilde{J}\left(\widehat{\mathit{\theta }}\left(k\right)\right)$. The indicator for determining convergence is the statistic Regret $R\left(K\right)$ [32]:

$$R\left(K\right)=\sum _{k=1}^K\left[\tilde{J}\left(\widehat{\mathit{\theta }}\left(k\right)\right)-\tilde{J}\left({\mathit{\theta }}^{*}\right)\right].$$

Because $\tilde{J}\left(\widehat{\mathit{\theta }}\left(k\right)\right)$ is a convex function, there is

$$\tilde{J}\left({\mathit{\theta }}^{*}\right)⩾\tilde{J}\left(\widehat{\mathit{\theta }}\left(k\right)\right)+\langle \nabla \tilde{J}\left(\widehat{\mathit{\theta }}\left(k\right)\right),\widehat{\mathit{\theta }}\left(k\right)-{\mathit{\theta }}^{*}\rangle .$$

Substituting in $R\left(K\right)$ yields

$$R\left(K\right)⩽\sum _{k=1}^K\langle \nabla \tilde{J}\left(\widehat{\mathit{\theta }}\left(k\right)\right),\widehat{\mathit{\theta }}\left(k\right)-{\mathit{\theta }}^{*}\rangle .$$

We further split the above equation according to the variable dimensions and change the summation order to obtain

$$R\left(K\right)⩽\sum _{j=1}^{3(n-1)}\sum _{k=1}^K\nabla {\tilde{J}}_j\left(\widehat{\mathit{\theta }}\left(k\right)\right)({\widehat{\theta }}_j\left(k\right)-{\theta }_j^{*}),$$

(26)

where $\nabla {\tilde{J}}_i\left(\widehat{\mathit{\theta }}\left(k\right)\right)$ is as follows:

$$\nabla {\tilde{J}}_j\left(\widehat{\theta }\left(k\right)\right)={\displaystyle \frac{\partial \tilde{J}\left(\widehat{\theta }\left(k\right)\right)}{\partial \widehat{{\theta }_j}\left(k\right)}}={\widehat{\phi }}_j\left(k\right)\left[y\left(k\right)-{\widehat{\phi }}_j^{\mathrm{T}}\left(k\right){\widehat{\theta }}_j\left(k\right)\right].$$

If we define a new term [33]

$$\gamma \left(k\right)=\lambda \left(k\right){\displaystyle \frac{1}{1-{\Pi }_{s=1}^k{\beta }_1\left(s\right)}},$$

and ignore the small constant $\epsilon$, the parameter update becomes

$${\widehat{\theta }}_j(k+1)={\widehat{\theta }}_j\left(k\right)-\gamma \left(k\right){\displaystyle \frac{{\beta }_1\left(k\right)V_j(k-1)+(1-{\beta }_1\left(k\right))\nabla {\tilde{J}}_j\left(\widehat{\theta }\left(k\right)\right)}{\sqrt{{\widehat{S}}_j\left(k\right)}}}.$$

Separating the summation terms of (26) can obtain the following:

$$\begin{array}{cc}\hfill \nabla {\tilde{J}}_j\left(\widehat{\mathit{\theta }}\left(k\right)\right)({\widehat{\theta }}_j\left(k\right)-{\theta }_j^{*})=& \underset{\left(1\right)}{\underbrace{{\displaystyle \frac{\sqrt{{\widehat{S}}_j\left(k\right)}\left[{\left({\widehat{\theta }}_j\left(k\right)-{\theta }_j^{*}\right)}^2-{\left({\widehat{\theta }}_j(k+1)-{\theta }_j^{*}\right)}^2\right]}{2\gamma \left(k\right)\left(1-{\beta }_1\left(k\right)\right)}}}}\hfill \\ \hfill & \underset{\left(2\right)}{\underbrace{-{\displaystyle \frac{{\beta }_1\left(k\right)}{1-{\beta }_1\left(k\right)}}{V}_j(k-1)\left({\widehat{\theta }}_j\left(k\right)-{\theta }_j^{*}\right)}}+\underset{\left(3\right)}{\underbrace{{\displaystyle \frac{\gamma \left(k\right)}{2\left(1-{\beta }_1\left(k\right)\right)}}{\displaystyle \frac{{\left(V_j\left(k\right)\right)}^2}{\sqrt{{\widehat{S}}_j\left(k\right)}}}}}.\hfill \end{array}$$

(27)

By expanding and contracting items (1), (2), (3) in (27), we derive that [34]

$$\sum _{k=1}^K{\displaystyle \frac{\sqrt{{\widehat{S}}_j\left(k\right)}\left[{\left({\widehat{\theta }}_j\left(k\right)-{\theta }_j^{*}\right)}^2-{\left({\widehat{\theta }}_j(k+1)-{\theta }_j^{*}\right)}^2\right]}{2\gamma \left(k\right)\left(1-{\beta }_1\left(k\right)\right)}}⩽{\displaystyle \frac{D_j^2{G}_j}{2\lambda \left(K\right)\left(1-{\beta }_1\left(1\right)\right)}},$$

$$\sum _{k=1}^K-{\displaystyle \frac{{\beta }_1\left(k\right)}{1-{\beta }_1\left(k\right)}}{V}_j(k-1)\left({\widehat{\theta }}_j\left(k\right)-{\theta }_j^{*}\right)⩽G_j{D}_j\sum _{k=1}^K{\displaystyle \frac{{\beta }_1\left(k\right)}{1-{\beta }_1\left(k\right)}},$$

$$\begin{array}{c}\hfill \sum _{k=1}^K{\displaystyle \frac{\gamma \left(k\right)}{2\left(1-{\beta }_1\left(k\right)\right)}}{\displaystyle \frac{{\left(V_j\left(k\right)\right)}^2}{\sqrt{{\widehat{S}}_j\left(k\right)}}}⩽\sum _{k=1}^K{\displaystyle \frac{\gamma \left(k\right)}{2(1-{\beta }_1\left(k\right))}}·\sum _{r=1}^k{\displaystyle \frac{{(1-{\beta }_1\left(r\right))}^2{\left({\prod }_{s=r+1}^k{\beta }_1\left(s\right)\right)}^2}{(1-{\beta }_2){\beta }_2^{k-r}}}·G_j.\end{array}$$

Thus, the upper bound of $R\left(K\right)$ can be sorted out

$$\begin{array}{ccc}\hfill R\left(K\right)⩽& & {\displaystyle \frac{{\sum }_{j=1}^{3(n-1)}{D}_j^2{G}_j}{2\lambda \left(K\right)\left(1-{\beta }_1\left(1\right)\right)}}+\left(\sum _{j=1}^{3(n-1)}{G}_j{D}_j\right)\sum _{k=1}^K{\displaystyle \frac{{\beta }_1\left(k\right)}{1-{\beta }_1\left(k\right)}}\hfill \\ & & +\left(\sum _{j=1}^{3(n-1)}{G}_j\right)\left[\sum _{k=1}^K{\displaystyle \frac{\gamma \left(k\right)}{2(1-{\beta }_1\left(k\right))}}·\sum _{r=1}^k{\displaystyle \frac{{(1-{\beta }_1\left(r\right))}^2{\left({\prod }_{s=r+1}^k{\beta }_1\left(s\right)\right)}^2}{(1-{\beta }_2){\beta }_2^{k-r}}}\right].\hfill \end{array}$$

(28)

According to the hyperparameters ${\beta }_1\left(k\right)$ in (14) and ${\beta }_2$ in (16), there are [24]

$$\left(\sum _{j=1}^{3(n-1)}{G}_j{D}_j\right)\left(\sum _{k=1}^K{\displaystyle \frac{{\beta }_1\left(k\right)}{1-{\beta }_1\left(k\right)}}\right)⩽\left(\sum _{j=1}^{3(n-1)}{G}_j{D}_j\right)\left({\displaystyle \frac{1}{1-{\beta }_1\left(1\right)}}\sum _{k=1}^K{\beta }_1\left(k\right)\right),$$

(29)

$$\begin{array}{ccc}\hfill & & \left(\sum _{j=1}^{3(n-1)}{G}_j\right)\left[\sum _{k=1}^K{\displaystyle \frac{\gamma \left(k\right)}{2(1-{\beta }_1\left(k\right))}}·\sum _{r=1}^k{\displaystyle \frac{{(1-{\beta }_1\left(r\right))}^2{\left({\prod }_{s=r+1}^k{\beta }_1\left(s\right)\right)}^2}{(1-{\beta }_2){\beta }_2^{k-r}}}\right]\hfill \\ & & ⩽\left(\sum _{j=1}^{3(n-1)}{G}_j\right)·{\displaystyle \frac{{\sum }_{k=1}^K\lambda \left(k\right)}{2{(1-{\beta }_1\left(1\right))}^2(1-{\beta }_2)(1-c)}}.\hfill \end{array}$$

(30)

From (28)–(30), we find that the order of magnitude of the upper bound of $R\left(K\right)$ is only related to ${\displaystyle \frac{1}{\lambda \left(K\right)}}$, ${\sum }_{k=1}^K{\beta }_1\left(k\right)$, and ${\sum }_{k=1}^K\lambda \left(k\right)$. Because $\lambda \left(k\right)={\displaystyle \frac{\lambda }{\sqrt{k}}}$, then

$${\displaystyle \frac{1}{\lambda \left(K\right)}}=O(K^{{\displaystyle \frac{1}{2}}}),\sum _{k=1}^K\lambda \left(k\right)=O(K^{{\displaystyle \frac{1}{2}}}).$$

(31)

Moreover, due to ${\beta }_1\left(k\right)={\displaystyle \frac{{\beta }_1}{\sqrt{k}}}$, there is

$$\sum _{k=1}^K{\beta }_1\left(k\right)=O(K^{{\displaystyle \frac{1}{2}}}).$$

(32)

It can be seen by (31) and (32) that the upper order of $R\left(K\right)$ is maintained at the optimal state $(K^{{\displaystyle \frac{1}{2}}})$. Thus, the average amortization value $R\left(K\right)/K=O(K^{-{\displaystyle \frac{1}{2}}})$ tends to 0 when $K\to \infty$ [35].

After Amsgrad optimization, the error of PF usually refers to the difference between the empirical distribution ${\widehat{p}}_N=p\left({\underline{x}}_l^i\left(k\right)\right|y(1:K),u(1:K),\widehat{\mathit{\theta }}(k-1))$ based on $\widehat{\mathit{\theta }}(k-1)$ and the true posterior distribution $p=p\left({\underline{x}}_l^i\left(k\right)\right|y(1:K),u(1:K))$. For a particle set ${\{{\underline{x}}_l^i(1:k),w_l^i\left(k\right)\}}_{i=1}^N$ before resampling, the empirical distribution is computed by (10). The weights follow [16]

$$w_l^i\left(k\right)\propto {\displaystyle \frac{p(X_l^i\left(k\right),Y\left(k\right))}{d\left(X_l^i\left(k\right)\right|Y\left(k\right))}}.$$

(33)

The resampling operation generates an equally weighted particle set ${\{{\underline{x}}_l^j(1:k),{\displaystyle \frac{1}{N}}\}}_{j=1}^N$ from ${\widehat{p}}_N$, corresponding to the resampled empirical distribution [17]:

$${\widehat{p}}_N^{\mathrm{res}}=\widehat{p}\left({\underline{x}}_l^j\left(k\right)\right|y(1:K),u(1:K),\widehat{\mathit{\theta }}(k-1))\approx {\displaystyle \frac{1}{N}}\sum _{j=1}^N\Delta ({\underline{x}}_l^j\left(k\right)-{\widehat{\underline{x}}}_l^j\left(k\right)).$$

Decompose the total error into

$$\parallel {\widehat{p}}_N^{\mathrm{res}}-p\parallel ⩽\parallel {\widehat{p}}_N^{\mathrm{res}}-{\widehat{p}}_N\parallel +\parallel {\widehat{p}}_N-p\parallel ,$$

where $\parallel {\widehat{p}}_N^{\mathrm{res}}-{\widehat{p}}_N\parallel$ is the resampling error and $\parallel {\widehat{p}}_N-p\parallel$ is the initial sampling error.

Lemma 1.

Under the condition that the importance weights have finite variance ($\mathrm{Var}\left(w_l^i\left(k\right)\right)<\infty$), and for any bounded function $\tilde{J}$ ($\parallel \tilde{J}{\parallel }_{\infty }⩽1$), the error of the classical PF is [36]

$$\mathbb{E}\left[{({\widehat{p}}_N\left(\tilde{J}\right)-p\left(\tilde{J}\right))}^2\right]⩽{\displaystyle \frac{\mathrm{Var}\left(w_l\left(k\right)\right)·{\parallel \tilde{J}\parallel }_{\infty }^2}{N}}.$$

Proof. 

Based on the properties of importance sampling and the law of large numbers, there is

$$\begin{array}{c}\hfill {\widehat{p}}_N\left(\tilde{J}\right)=\sum _{i=1}^N{w}_l^i\left(k\right)\tilde{J}\left({\underline{x}}_l^i(1:k)\right),\mathbb{E}\left[{\widehat{p}}_N\left(\tilde{J}\right)\right]=p\left(\tilde{J}\right).\end{array}$$

By combining (33), the variance is calculated as

$$\mathrm{Var}\left({\widehat{p}}_N\left(\tilde{J}\right)\right)={\displaystyle \frac{1}{N}}{\mathrm{Var}}_q\left(w_l\left(k\right)\tilde{J}\right)⩽{\displaystyle \frac{{\mathbb{E}}_q\left[w_l^2\left(k\right){\tilde{J}}^2\right]}{N}}⩽{\displaystyle \frac{\mathrm{Var}\left(w_l\left(k\right)\right)·{\parallel \tilde{J}\parallel }_{\infty }^2}{N}}.$$

□

Lemma 2.

For any bounded function $\tilde{J}(\parallel \tilde{J}{\parallel }_{\infty }⩽1)$, the empirical distribution after resampling satisfies [37]

$$\mathbb{E}\left[{({\widehat{p}}_N^{\mathrm{res}}\left(\tilde{J}\right)-{\widehat{p}}_N\left(\tilde{J}\right))}^2|{\widehat{p}}_N\right]⩽{\displaystyle \frac{\parallel \tilde{J}{\parallel }_{\infty }^2}{N}}.$$

Proof. 

After resampling, particle ${\underline{x}}_l^j(1:k)$ is conditionally independent and identically distributed from ${\widehat{p}}_N$, there is

$$\mathbb{E}\left[{\widehat{p}}_N^{\mathrm{res}}\left(\tilde{J}\right)|{\widehat{p}}_N\right]={\widehat{p}}_N\left(\tilde{J}\right),\mathrm{Var}\left({\widehat{p}}_N^{\mathrm{res}}\left(\tilde{J}\right)|{\widehat{p}}_N\right)={\displaystyle \frac{{\mathrm{Var}}_{{\widehat{p}}_N}\left(\tilde{J}\right)}{N}}⩽{\displaystyle \frac{\parallel \tilde{J}{\parallel }_{\infty }^2}{N}}.$$

□

Using Lemma 2 and the law of total expectation, the expectation of resampling error is

$$\mathbb{E}\left[{({\widehat{p}}_N^{\mathrm{res}}\left(\tilde{J}\right)-{\widehat{p}}_N\left(\tilde{J}\right))}^2\right]=\mathbb{E}\left[\mathbb{E}\left[{({\widehat{p}}_N^{\mathrm{res}}\left(\tilde{J}\right)-{\widehat{p}}_N\left(\tilde{J}\right))}^2|{\widehat{p}}_N\right]\right]⩽{\displaystyle \frac{\parallel \tilde{J}{\parallel }_{\infty }^2}{N}}.$$

The mean square bound of the total error is obtained by combining Lemma 1 and resampling error:

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{({\widehat{p}}_N^{\mathrm{res}}\left(\tilde{J}\right)-p\left(\tilde{J}\right))}^2\right]& & ⩽2\left(\mathbb{E}\left[{({\widehat{p}}_N^{\mathrm{res}}\left(\tilde{J}\right)-{\widehat{p}}_N\left(\tilde{J}\right))}^2\right]+\mathbb{E}\left[{({\widehat{p}}_N\left(\tilde{J}\right)-p\left(\tilde{J}\right))}^2\right]\right)\hfill \\ & & ⩽{\displaystyle \frac{2\left(1+\mathrm{Var}\left(w_l\left(k\right)\right)·{\parallel \tilde{J}\parallel }_{\infty }^2\right)}{N}}.\hfill \end{array}$$

The upper bound of the total variation error is derived through the Jensen inequality and the total variation norm (${\parallel ·\parallel }_{\mathrm{TV}}$):

$$\mathbb{E}\left[\parallel {\widehat{p}}_N^{\mathrm{res}}{-p\parallel }_{\mathrm{TV}}\right]⩽\sqrt{\parallel {\widehat{p}}_N^{\mathrm{res}}{-p\parallel }_{\mathrm{TV}}^2}⩽\sqrt{{\displaystyle \frac{2\left(1+\mathrm{Var}\left(w_l\left(k\right)\right)·{\parallel \tilde{J}\parallel }_{\infty }^2\right)}{N}}}.$$

If we define the difference between the initial particle distribution and the true distribution as $F_0$, and let $F=F_0·\sqrt{2\left(1+\mathrm{Var}\left(w_l\left(k\right)\right)·{\parallel \tilde{J}\parallel }_{\infty }^2\right)}$, the error constant $F\left(k\right)$ increases exponentially due to the mixality of state transitions for the k-step filtering:

$$F\left(k\right)⩽F·{\zeta }^k,\zeta >1.$$

In summary, the upper bound of Ams-PF error can be concluded as follows:

Theorem 1.

Let the resampling PF method under the following conditions:

(1) The variance of the importance weight is limited.

(2) The resampling algorithm is unbiased.

(3) The state transition model has exponential mixing.

Then, for any bounded function $\tilde{J}(\parallel \tilde{J}{\parallel }_{\infty }⩽1)$, there exists a constant F and $\zeta >1$, such that the upper bound of the resampling PF error is

$$\mathbb{E}\left[\parallel {\widehat{p}}_N^{\mathrm{res}}{-p\parallel }_{\mathrm{TV}}\right]⩽{\displaystyle \frac{F{\zeta }^k}{\sqrt{N}}}.$$

4. Illustrative Examples

In this section, two examples, including a third-order singular system and a fourth-order singular system with fractional orders, are presented to evaluate the effectiveness. After each joint estimation process of Ams-PF, the system’s parameter vector and state vector are updated. To assess the performance of parameter identification and state estimation, the updating process monitors optimization progress by calculating the parameter estimation error at each recursion step.

The calculations and graphs are performed on MATLAB^® 2022b in simulation. Finally, the optimization results are visualized through tables and graphical representations, demonstrating the variation trend of parameter estimation errors with the recursion time. These results prove that the joint estimation algorithm can stably converge to the vicinity of true values after multiple recursion cycles.

4.1. Three-Order Fractional Singular System

Consider the following three-order singular system with the state transition, control and output matrices defined, respectively, as follows:

$$\begin{array}{c}\hfill \mathit{E}=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 1.45& 7.62& 0\end{array}\right],\mathit{A}=\left[\begin{array}{ccc}0& 1& 0\\ 1.29& 7.5& 1\\ 0& 0& 1\end{array}\right],\mathit{B}=\left[\begin{array}{c}-12.23\\ -10.85\\ 1\end{array}\right].\end{array}$$

The fractional order is $\alpha =0.2$. The parameter vector of this three-order singular system is

$$\begin{array}{cc}\hfill \mathit{\theta }& ={[e_1,e_2,a_1,a_2,b_1,b_2]}^{\mathrm{T}}\hfill \\ \hfill & ={[1.45,7.62,1.29,7.5,-12.23,-10.85]}^{\mathrm{T}}.\hfill \end{array}$$

The variation range of parameters in the system matrix is [−20, 20], and they can be positive or negative during the optimization process. To quantify the discrepancy between identified results and true parameters, the parameter estimation error is defined as follows:

$$\delta ={\displaystyle \frac{\parallel \widehat{\mathit{\theta }}\left(k\right)-\mathit{\theta }\parallel }{\parallel \mathit{\theta }\parallel }},$$

where $\widehat{\mathit{\theta }}\left(k\right)$ represents the parameter identification results based on the Ams-PF, $\mathit{\theta }$ contains the true values of system parameters.

The input $u\left(k\right)$ of the three-order singular system is the uncorrelated measurable random signal sequence with zero mean and unit variance, and the noise $v\left(k\right)$ is the zero-mean Gaussian sequence with variance ${\sigma }^2=0.8^2$. The proposed Ams-PF is used to identify this three-order singular system.

Table 1. Parameter estimation and errors of the three-order fractional singular system (${\sigma }^2=0.8^2$).

Algorithm	Parameter	k = 100	k = 200	k = 500	k = 1000	k = 2000	k = 3000	True Value
Ams-PF	$e_1$	−0.61541	−0.02476	1.14192	4.72356	3.35476	1.35720	1.45000
	$e_2$	−1.06909	−0.92999	−0.24511	6.48938	7.48246	7.68150	7.62000
	$a_1$	−0.62677	−0.01994	1.18192	4.73852	3.42233	1.23715	1.29000
	$a_2$	−1.12186	−0.97711	−0.28772	6.38395	7.35942	7.56279	7.50000
	$b_1$	−4.49469	−5.55102	−6.21625	−11.97485	−12.32042	−12.28608	−12.23000
	$b_2$	−4.56647	−5.58235	−6.20746	−11.42739	−11.01381	−10.81964	−10.85000
	$\delta (\%)$	81.67777	75.76372	68.40623	25.73892	14.63015	0.77586	0.00000
Amsgrad	$e_1$	−0.00006	−0.00006	−0.00007	−0.00007	−0.00008	−0.00009	1.45000
	$e_2$	−0.00006	−0.00006	−0.00007	−0.00007	−0.00008	−0.00009	7.62000
	$a_1$	−0.00006	−0.00006	−0.00007	−0.00007	−0.00008	−0.00009	1.29000
	$a_2$	−0.00006	−0.00006	−0.00007	−0.00007	−0.00008	−0.00009	7.50000
	$b_1$	−0.32544	−0.58082	−2.51044	−5.70348	−10.32070	−13.04685	−12.23000
	$b_2$	−0.40543	−0.70195	−3.22139	−7.46451	−11.53878	−10.95777	−10.85000
	$\delta (\%)$	97.83775	96.21691	83.81867	66.83424	56.31184	55.51342	0.00000
GSA-KF	$e_1$	1.60228	1.87892	2.13652	0.55312	1.70922	0.71478	1.45000
	$e_2$	1.11366	0.46282	2.17060	1.04038	2.37803	2.59275	7.62000
	$a_1$	0.43591	−0.22426	0.78218	−0.10266	0.24588	−0.31317	1.29000
	$a_2$	0.99366	0.74029	−0.33393	0.82055	−0.26198	0.97589	7.50000
	$b_1$	−5.61093	−6.65074	−11.22424	−13.54027	−13.27158	−12.65576	−12.23000
	$b_2$	−3.68962	−4.99514	−9.76241	−8.89775	−10.09497	−8.62924	−10.85000
	$\delta (\%)$	68.43794	56.61571	45.42038	44.01233	40.03237	40.03237	0.00000

shows partial parameter estimates and error values. Figure 2 shows the variation curve of parameter estimation errors. Figure 3 presents the boxplot of each parameter estimation, and Figure 4 denotes a comparison between the estimated parameters and the true values of Ams-PF. The state and output fitting curves of the system are shown in Figure 5 and Figure 6, respectively. To prove the robustness of the algorithm, we conduct simulations at two higher noise levels, and the results are shown in

Table 2. Parameter estimation and errors of the three-order fractional singular system based on Ams-PF under different noises.

Noise	k	${\mathit{e}}_1$	${\mathit{e}}_2$	${\mathit{a}}_1$	${\mathit{a}}_2$	${\mathit{b}}_1$	${\mathit{b}}_2$	$\mathit{\delta }(\%)$
${\sigma }^2=0.9^2$	100	−0.61993	−1.07816	−0.63087	−1.12662	−4.51960	−4.59531	81.58529
	200	−0.02616	−0.93060	−0.02162	−0.97967	−5.58548	−5.62083	75.62647
	500	1.12818	−0.25172	1.16804	−0.29629	−6.24719	−6.24261	68.32139
	1000	4.68612	6.42875	4.69650	6.32621	−11.92001	−11.38289	25.59878
	2000	3.26999	7.44564	3.35706	7.30810	−12.31156	−11.00405	14.11934
	3000	1.56226	7.55420	1.38692	7.44336	−12.32993	−10.86013	1.01396
${\sigma }^2=1.0^2$	100	−0.62497	−1.07610	−0.63717	−1.13089	−4.57133	−4.65109	81.37875
	200	−0.03067	−0.93304	−0.02977	−0.98243	−5.64181	−5.68185	75.41060
	500	1.10890	−0.26537	1.14559	−0.31072	−6.29412	−6.29484	68.21139
	1000	4.61111	6.31260	4.61662	6.21463	−11.79260	−11.27850	25.36458
	2000	3.24096	7.30739	3.32794	7.16698	−12.27024	−10.98292	14.03265
	3000	1.63235	7.34108	1.42873	7.24212	−12.31576	−10.87492	2.30512
True Value		1.45000	7.62000	1.29000	7.50000	−12.23000	−10.85000	0.00000

and Figure 7. Furthermore, parameters and states of the above system are estimated using the Amsgrad, PF and Gravitational search algorithm-Kalman filter (GSA-KF) methods. The estimation results are also displayed in Table 1, Figure 2, Figure 5 and Figure 6.

4.2. Four-Order Fractional Singular System

Here we further consider a four-order singular system with the following state transition, control and output matrices:

$$\begin{array}{c}\hfill \mathit{E}=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ -4.35& -5.6& -3.22& 0\end{array}\right],\mathit{A}=\left[\begin{array}{cccc}0& 1& 0& 0\\ 0& 0& 1& 0\\ -4.30& -5.55& -3.17& 1\\ 0& 0& 0& 1\end{array}\right],\mathit{B}=\left[\begin{array}{c}-0.38\\ 6.5\\ 1.33\\ 1\end{array}\right].\end{array}$$

The fractional order is $\alpha =0.1$. The parameter vector of the four-order singular system is defined as follows:

$$\begin{array}{cc}\hfill \mathit{\theta }& ={[e_1,e_2,e_3,a_1,a_2,a_3,b_1,b_2,b_3]}^{\mathrm{T}}\hfill \\ \hfill & ={[-4.35,-5.6,-3.22,-4.30,-5.55,-3.17,-0.38,6.5,1.33]}^{\mathrm{T}}.\hfill \end{array}$$

Different from Example 4.1, the variance in noise $v\left(k\right)$ is set as ${\sigma }^2=0.6^2$. The proposed Ams-PF, Amsgrad, PF and GSA-KF algorithms are employed to estimate both parameters and states in this four-order singular system. The results are shown in

Table 3. Parameter estimation and errors of the four-order fractional singular system.

Algorithm	Parameter	k = 100	k = 200	k = 500	k = 1000	k = 2000	k = 3000	True Value
Ams-PF	$e_1$	−4.15262	−5.26322	−5.93478	−5.67812	−5.17051	−4.28444	−4.35000
	$e_2$	−3.89106	−4.56233	−4.68299	−4.80976	−5.13349	−5.62614	−5.60000
	$e_3$	−3.37188	−3.87983	−4.22355	−4.13936	−3.58811	−3.22194	−3.22000
	$a_1$	−4.02288	−5.03670	−5.81737	−5.54977	−5.08730	−4.22023	−4.30000
	$a_2$	−3.91768	−4.78959	−4.76688	−4.92696	−5.06612	−5.55220	−5.55000
	$a_3$	−3.42535	−3.75496	−4.09824	−3.97158	−3.57478	−3.16901	−3.17000
	$b_1$	−2.37429	−1.65605	0.06044	0.18526	−0.13507	−0.31770	−0.38000
	$b_2$	−1.52845	0.18578	4.08040	5.78942	6.28496	6.49679	6.50000
	$b_3$	−1.30907	−0.48574	1.46846	1.76074	1.57158	1.36158	1.33000
	$\delta (\%)$	70.32583	54.41356	29.41611	20.40704	11.59974	0.99473	0.00000
Amsgrad	$e_1$	−2.00252	−2.08265	−2.09940	−2.09940	−2.09940	−2.09941	−4.35000
	$e_2$	−2.00252	−2.08265	−2.09940	−2.09940	−2.09940	−2.09941	−5.60000
	$e_3$	−2.00252	−2.08265	−2.09940	−2.09940	−2.09940	−2.09941	−3.22000
	$a_1$	−2.00252	−2.08265	−2.09940	−2.09940	−2.09940	−2.09941	−4.30000
	$a_2$	−2.00252	−2.08265	−2.09940	−2.09940	−2.09940	−2.09941	−5.55000
	$a_3$	−2.00252	−2.08265	−2.09940	−2.09940	−2.09940	−2.09941	−3.17000
	$b_1$	2.40826	−0.55162	−0.33212	−0.45776	1.02542	−1.80296	−0.38000
	$b_2$	8.37223	6.67781	6.45143	6.78119	8.72513	5.86281	6.50000
	$b_3$	3.13573	0.18506	0.42633	0.29708	1.43440	−0.39791	1.33000
	$\delta (\%)$	64.55746	61.10604	61.10887	61.24270	62.02571	61.68410	0.00000
GSA-KF	$e_1$	0.82003	1.00473	2.03327	−2.68556	−3.48806	−3.42608	−4.35000
	$e_2$	1.67698	1.20865	0.93494	2.64296	1.08677	−2.40842	−5.60000
	$e_3$	0.46681	0.01001	−0.43139	−1.08602	1.30994	−2.98696	−3.22000
	$a_1$	0.75629	−0.19207	−0.36270	0.99605	−0.31427	−1.31883	−4.30000
	$a_2$	0.49804	−0.54719	−0.10848	−0.79584	−0.28896	−1.59978	−5.55000
	$a_3$	−0.25889	−0.73610	−0.09367	−0.08238	0.51672	−1.45601	−3.17000
	$b_1$	0.49905	−0.41279	−0.91987	−0.02331	−0.96026	−0.36071	−0.38000
	$b_2$	4.54039	5.34174	5.50382	6.17914	6.01744	6.21313	6.50000
	$b_3$	0.84229	0.78674	0.43152	2.44888	2.52241	0.42249	1.33000
	$\delta (\%)$	95.02548	88.95787	86.56646	85.57912	63.56020	46.87218	0.00000

and Figure 8, Figure 9, Figure 10 and Figure 11. To verify the robustness under parameter uncertainty, we conduct 10 Monte Carlo experiments under the initial value uncertainty. The average values and standard deviations of Ams-PF are demonstrated by

Table 4. Ams-PF average values and standard deviations over 10 Monte Carlo experiments for the four-order fractional singular system.

k	${\mathit{e}}_1$	${\mathit{e}}_2$	${\mathit{e}}_3$	${\mathit{a}}_1$	${\mathit{a}}_2$
100	$-4.15208\pm 0.01442$	$-3.89174\pm 0.01335$	$-3.37200\pm 0.01353$	$-4.02282\pm 0.01428$	$-3.91675\pm 0.01307$
200	$-5.26386\pm 0.01310$	$-4.56333\pm 0.01222$	$-3.88017\pm 0.01250$	$-5.03806\pm 0.01384$	$-4.78877\pm 0.01145$
500	$-5.93645\pm 0.01349$	$-4.68408\pm 0.01250$	$-4.22407\pm 0.01532$	$-5.81906\pm 0.01481$	$-4.76761\pm 0.01180$
1000	$-5.67426\pm 0.01358$	$-4.81180\pm 0.01372$	$-4.15230\pm 0.02347$	$-5.54824\pm 0.01513$	$-4.93122\pm 0.01286$
2000	$-5.16709\pm 0.02157$	$-5.13283\pm 0.03660$	$-3.60131\pm 0.01546$	$-5.09186\pm 0.01526$	$-5.06688\pm 0.03522$
3000	$-4.27683\pm 0.03262$	$-5.62203\pm 0.03032$	$-3.24030\pm 0.02420$	$-4.21272\pm 0.03236$	$-5.55482\pm 0.03117$
k	$a_3$	$b_1$	$b_2$	$b_3$	$\delta (\%)$
100	$-3.42702\pm 0.01334$	$-2.37493\pm 0.01038$	$-1.53010\pm 0.01246$	$-1.30937\pm 0.01086$	$70.34049\pm 0.09424$
200	$-3.75744\pm 0.01073$	$-1.65694\pm 0.00847$	$0.18407\pm 0.01311$	$-0.48606\pm 0.00945$	$54.43138\pm 0.12940$
500	$-4.10007\pm 0.01190$	$0.06292\pm 0.00704$	$4.08329\pm 0.01017$	$1.47155\pm 0.00722$	$29.41798\pm 0.13127$
1000	$-3.97786\pm 0.01550$	$0.18901\pm 0.00483$	$5.79624\pm 0.01559$	$1.76237\pm 0.00385$	$20.41996\pm 0.20451$
2000	$-3.58415\pm 0.01887$	$-0.14046\pm 0.01034$	$6.31104\pm 0.01556$	$1.56180\pm 0.01665$	$11.60933\pm 0.36103$
3000	$-3.18144\pm 0.02504$	$-0.32285\pm 0.00521$	$6.50560\pm 0.00940$	$1.35708\pm 0.00768$	$1.15654\pm 0.30091$

.

From Table 1, Table 2, Table 3 and Table 4 and Figure 2, Figure 3, Figure 4, Figure 5, Figure 6, Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11, we can draw the following conclusions:

• By horizontally looking at the values in Table 1 and Table 3, it can be observed that the parameters identified by Ams-PF gradually approach the true values as k increases. It can be seen from the last line of each method that the final identification errors of Ams-PF are $0.77586$ and $0.99473$, which are significantly smaller than those of the Amsgrad ($55.51342$ and $61.68410$) and GSA-KF ($40.03237$ and $46.87218$) methods. Thus, the Ams-PF algorithm can effectively identify fractional singular systems and its identification performance is satisfactory.
• In Figure 2, Figure 3 and Figure 4, Figure 8 and Figure 9, the Ams-PF algorithm exhibits excellent convergence speed and identification accuracy. Moreover, the system parameters identified by Ams-PF all converge near their true values with minimal estimation errors. Thus, it exhibits a stronger ability to escape local optima and has a faster convergence speed than Amsgrad and GSA-KF.
• In Figure 5 and Figure 10, the black line represents the true output, the red stars indicate the estimated outputs of Ams-PF, the blue dots represent the estimated outputs of PF, and the green forks denote the estimated outputs of GSA-KF. The system states in Figure 6 and Figure 11 are the same. As can be seen from these figures, the red stars are closest to the black line, which means that the estimated outputs and states of Ams-PF have a best fit with the true output and state values. These performances further demonstrate that the Ams-PF algorithm achieves satisfactory fitting between estimated and actual states, estimated outputs and actual outputs. The fitting performance is excellent and clearly superior to that of the PF and GSA-KF. Thus, the Ams-PF algorithm can effectively accomplish state estimation for fractional singular systems.
• From Table 2 and Figure 7, it can be seen that as the noise variance increases, the parameter estimation error becomes larger. However, this change is very minor. In Table 4, the final average value of Ams-PF identification error is 1.15654 and the standard deviation is 0.30091. Thus, we can conclude that the Ams-PF method can guarantee a certain robustness under different noise interferences or parameter uncertainty.

5. Conclusions

Singular systems have been widely applied in multiple fields, including control engineering, economic management, chemical processes, communication networks, and aerospace systems. Fractional-order singular systems are becoming increasingly important because they can approximate many systems with inconsistent speeds. Consequently, the accurate modeling of the fractional singular system becomes particularly important. This paper proposes an Ams-PF algorithm for joint parameter and state estimation of some fractional singular systems. The Amsgrad algorithm is computationally efficient and possesses strong optimization capability. Due to its powerful parallel search ability, it can effectively identify multiple parameters in singular systems. The PF algorithm approximates the probability density function by propagating random samples in the state space and uses the sample mean instead of the integration operation. Thus, minimum variance estimates of singular system states are obtained. By combining these advantages, the proposed Ams-PF algorithm can accelerate parameter convergence speed and improve estimation accuracy. Meanwhile, it can handle the relationship between states and parameters in fractional singular systems and enhance the overall joint estimation performance. Two simulation examples demonstrate that the Ams-PF achieves high estimation accuracy, fast convergence speed and excellent identification performance. Therefore, the Ams-PF algorithm can effectively identify fractional singular systems by delivering superior state estimation results and demonstrating significant practical application value.