Fuzzy Echo State Network-Based Fault Diagnosis of Remote-Controlled Robotic Arms

Abstract

This paper presents a novel fault diagnosis technique for remote-controlled robotic arm systems, utilizing deep fuzzy echo state networks (DFESNs) and applies the covariance matrix adaptation evolution strategy (CMA-ES) to optimize the hyperparameters of the DFESN model. The developed DFESN model, optimized via CMA-ES, efficiently performs online fault classification through small datasets and training. The method is evaluated through experiments on a leader–follower robotic arm system, demonstrating high accuracy and efficiency. The faults under consideration include leader sensor fault, communication fault, actuator fault, and follower sensor fault. Only follower sensor data are utilized for fault diagnosis. The DFESN model achieves a mean accuracy of 99.5% with the shortest training and online diagnosis times compared to other methods, making it suitable for real-time fault diagnosis applications.

1. Introduction

In recent years, remote-controlled robotic arms have been widely used in industrial production, hazardous environment detection, medical surgery, and other fields [1,2,3]. It can replace manual labor to complete complex and dangerous operation tasks, which greatly improves work efficiency and safety. However, due to the complexity and variability of the teleoperation environment [4,5], the high degree of coupling and the nonlinearity of the robotic arm system [6], and the long history of continuous work, the teleoperated robotic arm has a high incidence of failures in actual operation [7], which seriously affects its reliability and work performance. The reliability of the remote-controlled robotic arm system depends on the performance of each robotic arm and its communication quality [8]. Failure of a single arm can degrade the performance of its neighboring arms through communication, thus threatening the whole arm system [9]. Therefore, there is an urgent need for a method that can detect and diagnose abnormalities in teleoperated robotic arm systems at the initial stage of failure. Efficient and accurate fault diagnosis techniques [10] can not only buy valuable time for fault handling but also significantly reduce the system maintenance cost [11], reduce the risk of serious accidents leading to the paralysis of the whole system [12], and thus effectively guarantee the stable operation of the system and improve its reliability [13]. In this paper, we follow the terminology and methodologies outlined by Isermann and Ballé in their seminal work on model-based fault detection and diagnosis [14].

Most existing fault diagnosis methods for leader–follower robotic systems are based on analytical or observer-based models. These model-based approaches are known for their high accuracy and real-time performance under ideal conditions. For example, Quan et al. [15] proposed a distributed fault detection and isolation scheme using sliding-mode observers to estimate system states based on both local and neighboring agent information. Kang et al. [16] developed a fault diagnosis and fault-tolerant control strategy for multiple manipulators, which combined a learning observer with a radial basis function (RBF) neural network to approximate disturbances. Zhang et al. [17] introduced a robust distributed detection and isolation method leveraging Linear Matrix Inequality (LMI) techniques, while Zhong et al. [18] applied distributed PID control and observer-based techniques for fault detection within the context of multi-agent formation control.

However, despite their effectiveness, model-based methods depend heavily on the availability of accurate system models, which can be difficult to construct for complex, nonlinear, or dynamically changing systems [19]. These methods are often sensitive to model uncertainties and external disturbances, which can compromise their robustness and generalization capability [20]. Furthermore, the design and tuning of such systems typically require extensive expert knowledge and manual adjustment [21], making them less efficient and more difficult to adapt to unseen or evolving fault conditions.

In contrast, data-driven approaches, particularly those using deep learning, have emerged as powerful alternatives for fault diagnosis tasks [22,23]. These methods can automatically learn features from raw sensor data, model complex nonlinear relationships, and generalize well across a variety of operating scenarios. By eliminating the need for explicit modeling and manual feature extraction, deep learning techniques improve robustness, scalability, and adaptability, making them especially suitable for remote-controlled robotic arm systems where uncertainty and real-time requirements are prevalent.

Techniques such as the convolutional neural network (CNN) and recurrent neural network (RNN) have been successfully applied to various fault diagnosis tasks. Liu et al. [24] presented a deep-learning-based fault diagnosis method for leader–follower systems using CNN with multisensor data fusion. Xiao et al. [25] proposed a fault diagnosis method for asynchronous motors using Long Short-Term Memory (LSTM) neural networks to directly process raw vibration data, achieving high accuracy without feature engineering. Additionally, some researchers tried to incorporate the fuzzy method into deep learning models to handle data with high uncertainty and noise. For example, Deng et al. [26] proposed a hierarchical fused fuzzy deep neural network (FDNN) to enhance deep learning capability for data classification.

One significant challenge faced by typical deep neural networks during the backpropagation process is the vanishing gradient problem. This issue arises when gradients become exceedingly small during training, particularly in deep architectures, which hinders the effective updating of weights and slows down the learning process. The echo state network (ESN) is a novel type of recurrent neural network that forms part of reservoir computing models [27]. In ESN, the reservoir has fixed, random weights. This unique structure eliminates the need for backpropagation through the recurrent layers, thereby mitigating the vanishing gradient problem.

Covariance matrix adaptation evolution strategy (CMA-ES) is a continuous optimization algorithm designed to solve non-linear and non-convex optimization problems, particularly effective in exploring complex, high-dimensional search spaces without requiring gradient information [28]. The algorithm iteratively adapts a multivariate normal distribution, using the covariance matrix to guide the search towards promising regions, thereby enabling efficient exploration and exploitation of the hyperparameter space. Applying CMA-ES [28] to optimize deep fuzzy echo state networks (DFESNs) [29] helps address the complexity involved in tuning multiple hyperparameters that significantly influence model performance. The covariance matrix captures the dependencies among hyperparameters, generating well-suited candidates for improving model accuracy and efficiency.

In general, three challenging points remain in fault diagnosis for leader–follower robotic arm systems. First, model uncertainty due to the nonlinear and dynamic nature of the system poses significant difficulties. In addition, data variability and noise can obscure fault signals, complicating the diagnosis process. Finally, real-time diagnosis is critical as delays can result in significant performance degradation or system failures.

To address the aforementioned challenges, this paper proposes a fault diagnosis method for remote-controlled robotic arm systems based on a deep fuzzy echo state network (DFESN) [29], whose hyperparameters are optimized using the covariance matrix adaptation evolution strategy (CMA-ES) [28]. To overcome the difficulty of modeling nonlinear and dynamic systems, the proposed method does not rely on an explicit mathematical model of the robot but instead leverages the reservoir computing ability of the ESN to capture system dynamics directly from data. To tackle the challenge of data variability and noise, a fuzzy logic mechanism is incorporated into the ESN to enhance robustness and generalization. This allows the model to effectively identify fault patterns in noisy and uncertain teleoperation environments. The method is specifically designed to support online real-time fault diagnosis by maintaining low computational complexity, making it suitable for high-frequency feedback control in robotic systems. Compared to Zhang’s original DFESN framework [29], which focused on general machinery fault diagnosis, our approach expands the application to a more complex and interactive leader–follower architecture. Moreover, we introduce CMA-ES for the simultaneous optimization of multiple key hyperparameters—including reservoir size, spectral radius, sparsity, and the number of fuzzy clusters—which significantly improves diagnostic accuracy and adaptability. The model is capable of identifying diverse fault types, including sensor faults, actuator faults, and communication faults, under continuous operational conditions without requiring task interruption. The use of a fuzzy ESN further enables efficient processing of smaller three-dimensional datasets, ensuring robust performance under practical operating conditions. Experimental results on a leader–follower robotic arm system validate the effectiveness of the proposed technique, demonstrating its ability to accurately diagnose faults in real time.

The remainder of this paper is organized as follows. Section 2 provides a comprehensive overview of the system of remote-controlled robotic arms and details the types of faults considered in the study. Section 3 presents the DFESN for fault classification, including an explanation of the architecture, training algorithm, the use of CMA-ES for optimizing hyperparameters, and an analysis of the computational complexity compared to other methods. Section 4 showcases the experimental setup and results, validating the effectiveness of the proposed method. The paper concludes with Section 5, which discusses the conclusion.

2. System of Remote-Controlled Robotic Arms

This section provides a technical overview of the remote-controlled robotic arm system under investigation. It introduces the main hardware components, including the leader (Geomagic Touch) and the follower (QArm), as well as the communication protocol used between them. Additionally, the types of faults considered in this study—sensor faults, actuator faults, and communication faults—are defined. This background serves as a foundation for understanding the fault diagnosis methodology proposed in the subsequent sections.

2.1. Leader: Geomagic Touch

The leading arm is Geomagic Touch, which is also named Omni Bundle. It is a six-joint robot, with three active and three passive joints, enabling the end effector to move across the workspace. Position measurements are obtained using digital encoders and potentiometers. Integrated with QUARC software within MATLAB/Simulink, it allows precise control and feedback.

2.2. Follower: QArm

The QArm, acting as the follower in this robotic system, has a roll–tilt–roll configuration with four joints: base, shoulder, elbow, and wrist. Controlled by QUARC software within MATLAB/Simulink, it ensures precise synchronization with the leader robot. While the QArm follows the three active joints of the leader, the wrist joint remains inactive. This setup enables the QArm to reliably replicate the leader’s actions, ensuring coordination in complex tasks.

2.3. Communication Protocol Between Leader and Follower

The communication protocol uses the QUARC software’s Stream modules, employing TCP/IP for reliable data transmission. Data packets are sent every 0.02 s, with typical transmission delays below 10 ms. Zero matrices are used to initialize the data buffer, ensuring correct data alignment and handling. This allows the QArm to accurately synchronize with the Geomagic Touch, maintaining real-time coordination and operational efficiency.

2.4. The Type of Faults

In this remote-controlled robotic arm system, the three joint angles of the leader, ${\theta }_1$, ${\theta }_2$, and ${\theta }_3$, are transmitted to the follower through the controller designed in [30] to make corresponding joint angles of the follower, ${\theta }_1’$, ${\theta }_2’$, and ${\theta }_3’$, synchronize with ${\theta }_1$, ${\theta }_2$, and ${\theta }_3$, respectively. Sensor faults, actuator faults, and communication faults are incorporated into the system as illustrated in Figure 1. ${\phi }_i$ and ${\phi }_i^{\prime }$, where $i=1,2,3$, are real outputs of the leader and follower, respectively.

In this paper, three typical faults of the remote-controlled robotic arm system are under consideration: sensor faults, actuator faults, and communication interruption (see Figure 1). Sensor faults consist of leader sensor $f_{si}$ faults and follower sensor faults $f_{si}^{\prime }$, which are modeled as follows:

$$f_{si}=-{\theta }_i(i=1,2,3)$$

(1)

$$f_{si}’=-{\theta }_i’(i=1,2,3)$$

(2)

An actuator fault is represented as follows:

$$f_a=-u_r\cdot \tau$$

(3)

where $u_r$ is the input of the follower, and $\tau$ is the rate of loss from 0 to 1. This type of fault is more representative of the robotic arm, such as actuator wear, energy supply issues, or transmission efficiency losses, where the output performance degrades in proportion to the input signal. Such faults are more commonly observed in robotic manipulator systems [31,32,33].

The communication fault being investigated is classified as an interruption fault. When it occurs, the communication between the leader and the follower is disrupted, and they no longer work in coordination.

In the rest of the paper, an online fault diagnosis scheme is designed to identify the abovementioned faults, only through the follower output, that is ${\phi }_1^{\prime }$, ${\phi }_2^{\prime },$ and ${\phi }_3^{\prime }$.

3. Deep Fuzzy Echo State Networks for Fault Classification

In this section, a DFESN model [29] is introduced for fault classification. The fault types can occur in three sensors of the leader, three sensors of the follower, the actuator of the follower, and communication (see Figure 1). To achieve fault detection and fault diagnosis simultaneously, we make the normal working condition as another fault type. The classification process inherently identifies whether the system is in a healthy state or experiencing a fault; therefore, fault classification also effectively performs fault detection. The input of the training network utilized for training is $u\left(t\right)=\left[{\phi }_1^{\prime }\right(t)$ ${\phi }_2^{\prime }\left(t\right)$ ${\phi }_3^{\prime }\left(t\right)]$ from the data of the follower sensor.

3.1. A Fuzzy ESN

The ESN achieves fast training and inference speeds when handling small-scale datasets, which is crucial for online fault diagnosis. Original input data often contain anomalies or noise, which can impede subsequent feature extraction processes of ESN. To distinguish between principal signals and background noise, and to reveal the intrinsic distribution characteristics of the data, fuzzy clustering is introduced prior to feature extraction. The cluster centers were obtained using the Fuzzy C-Means clustering method [34]. Assuming that the input samples comprise $q$ categories, the optimal cluster centers are iteratively obtained by minimizing the objective function as

$$J_m={{\displaystyle {\sum }_{i=1}^N{\displaystyle {\sum }_{j=1}^q{\eta }_{ij}^m‖u(t)-c_j‖}}}^2$$

(4)

where $N$ denotes the number of data points, $m$ represents the fuzziness exponent, which in this paper is set to 2 [29], $u(t)\in R^N$ is the $i$-th input data point, and $c_j$ stands for the center of the $j$-th cluster, which is computed as follows:

$$c_j={\displaystyle \frac{{\displaystyle {\sum }_{i=1}^N{\eta }_{ij}^{2}u(t)}}{{\displaystyle {\sum }_{i=1}^N{\eta }_{ij}^2}}}$$

(5)

Moreover, ${\eta }_{ij}$ denotes the degree of membership of $u(t)$ in the $j$-th cluster, calculated as follows:

$${\eta }_{ij}={\displaystyle \frac{1}{{\displaystyle {\sum }_{k=1}^q{(\frac{‖u(t)-c_j‖}{‖u(t)-c_k‖})}^2}}}$$

(6)

By iteratively updating the membership values ${\eta }_{ij}$ and cluster centers $c_j$, the objective function $J_m$ is minimized to find the optimal cluster centers. First, initialize the membership matrix $U$ with random values such that the sum of the membership values for each data point is 1:

$${\displaystyle {\sum }_{j=1}^q{\eta }_{ij}=1}$$

(7)

Then, calculate the cluster centers $c$ with $\eta$ based on Equation (5). Next, update the membership values ${\eta }_{ij}$ using the newly computed cluster centers $c_j$ based on Equation (6). Repeat the calculation of cluster centers $c_j$ and the update of membership values ${\eta }_{ij}$ until the changes in the membership values between iterations are below a predefined threshold, indicating convergence:

$$‖U^{(t+1)}-U^{(t)}‖<\epsilon$$

(8)

where $U^{(t)}$ is the membership matrix at iteration $t$, and $\epsilon$ is a small positive number.

The fuzzy ESN structure is shown in Figure 2. The architecture of the Fuzzy-ESN consists of three parts: the input layer, the reservoir layer, and the output layer. The original data $u(t)$ undergo processing in the reservoir layer to extract its intrinsic distribution characteristics. The reservoir layer serves as a fixed, large-scale recurrent neural network that transforms the input data into a high-dimensional dynamic state space, capturing complex temporal patterns for subsequent analysis. In an ESN, the weights of the reservoir layer $W_2$ play a crucial role in determining the network dynamic behavior and its ability to capture and process temporal patterns in the input data, as follows:

$$W_2=\gamma \cdot {\displaystyle \frac{W}{\rho (W)}}$$

(9)

where $W\in {ℝ}^{N_{res}\times N_{res}}$ is a sparse matrix with its non-zero elements following a normal distribution. $N_{res}$ is the dimensionality of the reservoir. $\rho (W)$ denotes the spectral radius of the matrix $W$, which is the largest absolute value among the eigenvalues of $W$. The spectral radius is a critical factor in the context of ESN as it influences the memory capacity and stability of the network, and it is generally smaller than 1. $\gamma$ is a scaling factor that is used to adjust the influence of the spectral radius on the network’s dynamics. It is a predefined constant, typically smaller than 1.

As for the input weight matrix of the reservoir $W_1$, they are a randomly generated matrix following a standard normal distribution and are densely connected. After determining the values of $W_1$ and $W_2$, the state of the reservoir is updated according to the following equation:

$$x(t)=\tanh (W_{1}u(t)+W_{2}x(t-1))$$

(10)

where $\tanh (\cdot )$ is the hyperbolic tangent activation function applied element-wise.

The predicted values of the output layer in the fuzzy ESN are calculated as follows:

$$\widehat{y}(t)=W_{3}x(t)$$

(11)

where $W_3$ are the output weights that should satisfy the following:

$$h(t)=W_{3}x(t)$$

(12)

where $h(t)$ represents the output obtained after clustering, which can be calculated as follows:

$$h(t)=\left\{\begin{array}{ccc}{c}_j& if& u(t)\in j-th\begin{array}{c}\end{array}cluster\\ 0& & else\end{array}\right.$$

(13)

However, in practice, such $W_3$ may not exist, and the predicted values $\widehat{y}(t)$ can only be approximated to $h(t)$ through the continuous optimization of $W_3$. The formulation of the optimization problem for the output weight is as follows:

$$W_3=\underset{W_3}{\arg \min }({‖W_{3}X-h‖}_2^2+\lambda {‖W_3‖}_2^2)$$

(14)

where $\lambda$ is the regularization coefficient that introduces a penalty term to the optimization problem, balancing between reducing overfitting and fitting the training data well.

Generally, the solution to this optimization problem is addressed using ridge regression with the pseudoinverse as follows:

$$W_3=h{X}^T{(X{X}^T+\lambda I)}^{-1}$$

(15)

where $I$ denotes the identity matrix of the appropriate dimensions.

3.2. Architecture of DFESN

The aforementioned fuzzy ESN is a fundamental component of the DFESN illustrated in Figure 3. In the DFESN model, the original input signal passes through multiple fuzzy ESNs for feature extraction, ultimately being classified by a final ESN. Each stage of the DFESN plays a critical role in ensuring the robustness and accuracy of the model. Initially, fuzzy clustering is employed to reduce noise and manage uncertainty in the input data, providing a cleaner representation for further processing. The reservoir layer subsequently maps these inputs to a high-dimensional dynamic space, effectively capturing complex temporal patterns inherent in the data. State updates within the reservoir layer allow the model to maintain temporal dependencies over time, ensuring that relevant sequential information is preserved. The output layer is optimized using ridge regression, which helps minimize overfitting and enhances the overall classification performance. Finally, the softmax function is utilized to convert the output into probability distributions, enabling reliable classification. These components collectively contribute to the robustness of the DFESN in noisy environments and its capability to achieve accurate fault diagnosis.

The optimization of the final output weights $W_o$, depicted in Figure 3, can be optimized through solving an optimization problem as

$$W_o=\underset{W_o}{\arg \min }({‖W_{o}X-v‖}_2^2+{\lambda }_1{‖W_o‖}_2^2)$$

(16)

where $v\in {ℝ}^{N\times q}$ is the label matrix, which can be defined as follows:

$$v(t)=\left\{\begin{array}{ccc}1& if& u(t)\in j-th\begin{array}{c}\end{array}cluster\\ 0& & else\end{array}\right.$$

(17)

The solution to the optimization problem defined in Equation (16) is presented as follows:

$$W_o=v{X}^T{(X{X}^T+{\lambda }_{1}I)}^{-1}$$

(18)

Finally, during the testing, the output is obtained based on the model weights and classified using the Softmax function. The result calculated by the Softmax function represents the probability of the test samples being classified into a particular category. The establishment process of the DFESN model is shown in Algorithm 1.

Algorithm 1 Training Algorithm of DFESN

Require: Input signal $u\left(t\right)$, target classification y$\left(t\right)$, categories $c$, max iterative number $o$, spectral radius $\rho$, sparsity $\mu$, and regularization coefficient $\lambda$;

1:  for $i=1$ to $o$ do

2:      Calculate clustering results h$\left(t\right)$ by using Formulas (5), (6), and (13);

3:  end for

4:  Randomly generate the input weight matrix $W_1$ as a densely connected matrix from a standard normal distribution;

5:  Calculate the reservoir weight matrix $W_2$ by using Formula (9);

6:  for t $=1$ to T do

7:      Calculate the internal state x$\left(t\right)$ by using Formula (10), and include x$\left(t\right)$ in the state collection matrix X;

8:  end for

9:  Calculate the output weight matrix $W_3$ by using Formula (14);

10:  Randomly generate input weight matrix $W_{in}$ in uniform distribution;

11:  Calculate the reservoir weight matrix $W_{res}$;

12:  for t $=1$ to T do

13:     Calculate the internal state x$\left(t\right)$ and include x$\left(t\right)$ in the state collection matrix X;

14:  end for

15:  Calculate the output weight matrix $W_o$ by using Formula (16);

Ensure:

Output weight matrix $W_o$;

3.3. CMA-ES-Based Hyperparameter Optimization

The covariance matrix adaptation evolution strategy (CMA-ES) is used to iteratively search for the best combination of these hyperparameters by minimizing the model’s validation error [28]. In each generation, multiple sets of hyperparameters are sampled, and the DFESN model is trained and evaluated for each set. The covariance matrix adaptation helps to efficiently explore the hyperparameter space by adjusting the sampling distribution based on the performance of previous generations.

In this study, we employ the CMA-ES to optimize several critical hyperparameters of the DFESN used for fault diagnosis, which include the reservoir size ($N_{res}$), spectral radius ($\rho$), sparsity parameter ($\mu$), and the number of fuzzy clusters ($c$). The complexity of optimizing multiple hyperparameters lies in the high-dimensional and potentially non-convex nature of the search space. The primary advantage of using CMA-ES in this context is its ability to efficiently handle the high-dimensional optimization problem without requiring gradient information. CMA-ES iteratively adapts the covariance matrix of the sampling distribution, allowing it to effectively explore the search space and converge to an optimal solution even in complex, multimodal landscapes. This capability makes CMA-ES particularly suitable for optimizing the diverse set of hyperparameters in DFESN, ensuring that the network can achieve a balanced trade-off between expressiveness, stability, and computational efficiency.

The objective of the CMA-ES optimization is to minimize the classification error of the DFESN, which is measured by the Mean Squared Error (MSE) between the predicted output $\widehat{y}(t)$ and the true output $y(t)$. The objective function $J(\theta )$ is defined as follows:

$$J(\theta )={\displaystyle \frac{1}{N}}{\displaystyle \sum _{t=1}^N{‖y(t)-\widehat{y}(t)‖}^2}$$

(19)

where $\theta ={\left[N_{res},\rho ,\mu ,c\right]}^T$ is the vector of hyperparameters being optimized, representing the reservoir size ($N_{res}$), spectral radius ($\rho$), sparsity parameter ($\mu$), and the number of fuzzy clusters ($c$).

CMA-ES models the hyperparameters as a multivariate normal distribution, parameterized by a mean vector $m^{(g)}\in {ℝ}^d$ and a covariance matrix $C^{(g)}\in {ℝ}^{d\times d}$, where d is the number of hyperparameters to optimize. The algorithm iteratively updates these parameters to search for the optimal solution. At each generation $g$, a population of $\lambda$ candidate solutions $x_k^{(g+1)}$ is generated by sampling from the multivariate normal distribution:

$$x_k^{(g+1)}\sim \aleph (m^{(g)},{({\sigma }^{(g)})}^2{C}^{(g)})$$

(20)

where ${\sigma }^{(g)}$ is the global step size that controls the search radius.

The candidate solutions are evaluated using the objective function $J(\theta )$, and the best ${\mu }^{\prime }$ individuals are selected to update the mean vector:

$$m^{(g+1)}={\displaystyle \sum _{i=1}^{{\mu }^{\prime }}{w}_i{x}_{i:\lambda }^{(g+1)}}$$

(21)

where $w_i$ are positive weights that sum to one, and $x_{i:\lambda }$ represents the $i$-th best individual among the $\lambda$ candidates.

The covariance matrix $C$ is updated to capture the spread and orientation of the successful search directions, thus influencing future sampling:

$$C^{(g+1)}=(1-c_1-c_{{\mu }^{\prime }})C^{(g)}+c_1{p}_c^{(g+1)}(p_c^{(g+1)}{)}^T+c_{{\mu }^{\prime }}{\displaystyle \sum _{i=1}^{{\mu }^{\prime }}{w}_i(\frac{m^{(g+1)}-m^{(g)}}{{\sigma }^{(g)}})(\frac{m^{(g+1)}-m^{(g)}}{{\sigma }^{(g)}}{)}^T}$$

(22)

where $p_c$ is the evolution path, which accumulates information about the successful search directions, $c_1$ and $c_{{\mu }^{\prime }}$ are some empirical parameters that are typically defined based on the dimension of the problem.

The step size $\sigma$ is adapted to ensure efficient exploration and exploitation of the search space and is calculated as follows:

$${\sigma }^{(g+1)}={\sigma }^{(g)}\times \exp ({\displaystyle \frac{c_{\sigma }}{d_{\sigma }}}({\displaystyle \frac{‖p_{\sigma }^{(g+1)}‖}{E‖\aleph (0,I)‖}}-1))$$

(23)

where $c_{\sigma }$ and $d_{\sigma }$ are adaptation parameters, and $p_{\sigma }^{(g+1)}$ is the path for step-size control.

3.4. Computational Complexity Analysis

In this section, we analyze the computational complexity of the DFESN and compare it with the backpropagation (BP) algorithm. This comparison is essential to demonstrate the efficiency of DFESN in terms of computational resources.

The single-layer fuzzy ESN consists of three main steps: fuzzy clustering, state updating, and output weight optimization. The computational complexity for the fuzzy clustering process is

$${\kappa }_1=O(G\times N^2\times o)$$

(24)

where $\kappa$ denotes computing times, G represents the number of neurons in the hidden layer, $N$ is the number of samples, $o$ is the maximum number of iterations for fuzzy clustering, and $O(\ast )$ represents the computation complexity.

The computational complexity for the state updating step, as described in Equation (21), is

$${\kappa }_2=O(G\times (S+G)\times N)$$

(25)

where $S$ represents the input feature dimension.

The computational complexity for the output weight optimization step, as described in Equation (14), is

$${\kappa }_3=O(G^3\times N)$$

(26)

The total computational complexity of a single-layer fuzzy ESN is, therefore, given by

$$NU={\kappa }_1+{\kappa }_2+{\kappa }_3$$

(27)

In contrast, the computational complexity of the BP method in a single-layer neural network, for the forward propagation through the network, can be expressed as

$${\kappa }_4=O(G\times S\times N)$$

(28)

For the backward propagation process, one has

$${\kappa }_5=O(G^2\times N)$$

(29)

Including the iterative updates, the total computational complexity of the BP method with $k$ iterations is

$$NT=k\times ({\kappa }_4+{\kappa }_5)$$

(30)

A single-layer DFESN using layer-wise fuzzy tuning has a computational complexity of only $NU$. On the contrary, that of BP is $k\times NT$ due to the need for multiple iterations of feedback processing. To evaluate the computational loads of the two methods, the single-layer difference function $C$ between BP and DFESN can be derived by combining Equations (27) and (30) as follows:

$$\begin{array}{ll}C& =k\times NT-NU\\ & =k^2\times O(G\times S\times N)+k^2\times O(G^2\times N)\\ & -O(G\times N^2\times o)-O(G\times (S+G)\times N)-O(G^3\times N)\\ & =k^2\times O(G\times S\times N)-O(G\times (S+G)\times N)\\ & +k^2\times O(G^2\times N)-O(G\times N^2\times o)-O(G^3\times N)\end{array}$$

(31)

Assuming $S=\max \{G,S\}$ and $G=S$, one has

$$\begin{array}{ll}{C}_1& =k^2\times O(S^2\times N)-O(2\times S^2\times N)\\ & +k^2\times O(S^2\times N)-O(S\times N^2\times o)-O(S^3\times N)\\ & =(2k^2-2-S)O(S^2\times N)-O(S\times N^2\times o)\end{array}$$

(32)

where $C_1$ represents the minimum difference. Supposing $G=\min \{S,G\}$ and $S=G$, one has

$$\begin{array}{ll}{C}_2& =k^2\times O(G^2\times N)-O(2\times G^2\times N)\\ & +k^2\times O(G^2\times N)-O(G\times N^2\times o)-O(G^3\times N)\\ & =(2k^2-2-G)O(G^2\times N)-O(G\times N^2\times o)\end{array}$$

(33)

where $C_2$ denotes the maximum difference.

Based on the analysis above, it is clear that $C_1<C<C_2$ [29]. This outcome depends significantly on the values of $k$, $S$, and $G$. When $k\ll {[(2+G)/2]}^{1/2}$, the computational cost of the DFESN is typically not significantly lower than that of the BP; however, this case is relatively uncommon. The situation changes when $k\ge {[(2+G)/2]}^{1/2}$. Consequently, in most cases, the DFESN achieves a reduced total computational cost.

4. Experiment

4.1. Fault Description

In this section, the DFESN is applied for fault diagnosis in remote-controlled robotic arms to validate its effectiveness. The experimental scenario is shown in Figure 4. We first trained the DFESN by collecting data from the follower. The trained DFESN was then deployed to the remote-controlled robotic arm system for online fault diagnosis. The parameter of the sensor is shown in

Table 1. Parameter and meaning.

Parameter	Representative Meaning
${\theta }_1$	The angle of the base
${\theta }_2$	The angle of the shoulder
${\theta }_3$	The angle of the elbow

. The categories of faults under investigation are shown in

Table 2. The type of faults and category label.

Fault Type	Category Label
leader and follower work normally	1
communication fault	2
actuator fault	3
follower sensor1 fault	4
follower sensor2 fault	5
follower sensor3 fault	6
leader sensor1 fault	7
leader sensor2 fault	8
leader sensor3 fault	9

.

4.2. Data Collection

Different types of faults are introduced into the remote-controlled robotic arm system, with data recorded from three sensors in the follower. Real-time data are collected via Simulink, with a sampling interval of 0.02 s. In each fault scenario, real-time data from the follower robotic arm were collected over a period of 30 s, resulting in 1500 data samples per scenario. In total, 13,500 data samples were obtained across 9 different fault scenarios. Among these, 80% of the datasets were randomly selected for training, while the remaining 20% were used for testing in the experiments.

4.3. Experimental Results

In this experiment, the covariance matrix adaptation evolution strategy (CMA-ES) is used to optimize the hyperparameters of the deep fuzzy echo state network (DFESN). The four hyperparameters evaluated are reservoir size, spectral radius, sparsity parameter, and the number of fuzzy clusters, with respective ranges of [100–1000], [0–1], [0–0.5], and [5–50]. In the CMA-ES algorithm, as four parameters have been chosen as decision variables, which means $d=4$, and according to $\lambda =(4+3\ln d)\times 10$, $\lambda$ is found to be 80, i.e., the population size is 80, the search step length is set at 0.1, and the maximum number of iterations is 200. The values for step length and the maximum number of iterations are determined based on initial experiments to balance computational efficiency and optimization effectiveness. The step length of 0.1 was chosen to ensure a moderate exploration rate in the search space. A higher step length may lead to larger jumps and potentially miss local optima, while a lower step length might result in slow convergence. The maximum number of iterations was set to 200 to provide sufficient exploration without incurring excessive computational time, given the complexity of the model.

Figure 5 illustrates the variation in Mean Squared Error (MSE) of the DFESN throughout the optimization process. Despite some fluctuations in the curve, an overall decreasing trend in the MSE can be observed. The MSE reached its minimum value at iteration 124, at which point the best hyperparameters are found to be a reservoir size of 500, a spectral radius of 0.1, a sparsity parameter of 0.01, and a number of fuzzy clusters of 30.

In addition, to assist users intending to apply this network to other datasets, we recommend starting with an optimization strategy similar to CMA-ES. Initial hyperparameter values can be set using the same ranges as described in this study—reservoir size [100–1000], spectral radius [0–1], sparsity [0–0.5], and number of fuzzy clusters [5–50]. These ranges have proven effective across a variety of fault types in this study and provide a reasonable starting point.

The test sets for different fault scenarios were input into the trained DFESN model, and the results are shown in

Table 3. The results of different fault types (accuracy is defined as the percentage of correctly classified samples relative to all test samples.).

Label	1	2	3	4	5	6	7	8	9
Accuracy	99.67%	99.67%	100%	100%	99.67%	100%	99.65%	100%	100%

. DFESN model achieved high accuracy rates exceeding 90% across the 9 fault types. Notably, fault types 3, 4, 6, 8, and 9 achieved a classification accuracy of 100%.

The confusion matrix presented in Figure 6 provides a detailed evaluation of the DFESN model’s performance in fault classification across different fault types for the leader–follower robotic arm system. Each coordinate from 1 to 9 represents a distinct fault type as described in Table 3, including normal operation, communication faults, actuator faults, and different sensor faults. The DFESN model exhibits a high classification accuracy for most of the fault types, achieving a perfect accuracy of 1.00 for fault types 1, 3, 4, 5, 6, 8, and 9. This indicates that these faults were correctly identified without any misclassification, highlighting the model’s robustness in diagnosing these particular types of faults. Notably, the perfect classification for sensor faults (types 4, 5, 6, 8, and 9) shows that the model can effectively handle fault detection that involves complex sensor data.

However, for fault types 2 (communication fault) and 7 (leader sensor 2 fault), there are slight misclassifications, with accuracies of 0.97 and 0.99, respectively. These minor discrepancies suggest that there may be some overlap in the features representing these faults, possibly due to similarities in the observed data patterns under these conditions. The communication fault (type 2) seems particularly challenging for the model, possibly because interruptions in communication can manifest in a manner similar to other faults, making it harder for the model to distinguish.

To comprehensively evaluate the robustness of the DFESN model against various real-world conditions, we further analyzed its performance under different types of noise: Gaussian, Uniform, and Impulse noise. Figure 7 shows the classification accuracy across different signal-to-noise ratios (SNRs). At lower SNR levels (e.g., 5 dB), classification accuracy is significantly impacted by noise, dropping to around 60% for all noise types, highlighting the sensitivity of the model in highly noisy environments. However, as SNR increases to 10 dB and above, classification accuracy improves substantially, surpassing 80%. At an SNR of 20 dB and higher, the accuracy approaches nearly 100%, indicating excellent robustness of the DFESN model under typical operational noise conditions. Moreover, the results show no significant difference among Gaussian, Uniform, and Impulse noise types at higher SNR levels, demonstrating that the DFESN method maintains strong performance irrespective of the noise distribution encountered. This analysis confirms the suitability of the proposed DFESN model for practical scenarios where varying noise conditions are common.

Table 4. Main settings of the six models.

Model	Main Settings
ESN	$N_{res}$ = 1000, $\rho$ = 0.9, $\lambda$ = ${10}^{-8}$
LSTM	4 hidden units, 100 training cycles, learning rate is ${10}^{-3}$
CNN	2 Conv2D layers, 1 MaxPooling2D layer, 1 flattened layer, 2 dense layers
BP	2 hidden layers, 80, 25 neurons, 100 training cycles, learning rate is ${10}^{-2}$
DFESN	$N_{res1}$ = 100, $N_{res2}$ = 500, $\rho$ = 0.1, $\mu$ = 0.01, c = 30, $\lambda$=${10}^{-8}$

shows the parameter configurations for each approach, which were established through parameter tuning experiments. This ensures that all parameters were optimally adjusted to enable a fair comparison across methods. To further validate the effectiveness of DFESN, we performed comparative studies with commonly used methods including ESN, CNN, LSTM, and BP neural networks. DFESN outperformed these methods in terms of both accuracy and real-time efficiency, as shown in the

Table 5. Comparisons of training time and mean accuracy.

Model	Mean Accuracy	Training Time (s)	Online Diagnosis Time (s)
ESN	0.7374	1.45	0.0296
LSTM	0.9245	320.59	1.0137
CNN	0.9681	126.19	0.7658
BP	0.9925	15.84	1.4433
DFESN	0.9947	4.75	0.0105

. The training time of the ESN method was 1.45 s, which is the shortest of all of the methods. However, this method achieved the lowest accuracy. The LSTM, CNN, BP, and DFESN methods all achieve a mean accuracy exceeding 0.9, yet their training times vary significantly. The LSTM method has the longest training time at 320.59 s. Both the BP and DFESN methods achieve similarly high mean classification accuracies, with the DFESN method slightly outperforming at 0.9947 compared to BP’s 0.9925. Notably, the DFESN method has the shortest training time (4.75 s) and online diagnosis time (0.0105 s) among all the compared methods. This indicates that the proposed DFESN method is highly suitable for real-time fault diagnosis applications, providing both high accuracy and efficient processing times, which is crucial for handling the demands of real-time online diagnostics in complex systems.

4.4. Discussions

The aforementioned fuzzy ESN is a fundamental component of the DFESN illustrated in Figure 3. In the DFESN model, the original input signal passes through multiple fuzzy ESNs for feature extraction, ultimately being classified by a final ESN.

The results of this study confirm the effectiveness of our fault diagnosis approach for single follower robotic arm, supporting our hypothesis that advanced diagnostic algorithms enhance fault detection accuracy. This aligns with previous research but goes further by demonstrating robustness in dynamic scenarios. However, the approach’s scalability to systems involving multiple follower arms remains untested, particularly in complex environments requiring high inter-arm coordination. Additionally, the current reliance on a limited set of data sources may not capture all potential fault conditions, potentially leading to missed detections or false positives. Although the presented method is validated for the fault types listed, it can potentially be extended to other fault categories. For instance, the same architecture could be applied to detect faults in additional sensors or actuators not explicitly considered in this study. The DFESN model’s design, which allows for efficient adaptation through hyperparameter optimization, suggests that it could maintain high performance under different conditions, provided that appropriate training data are available.

Although the current results demonstrate promising performance, several avenues for further research remain open. Future studies should assess the generalizability of the proposed DFESN approach across diverse robotic platforms and multi-arm collaborative systems. It is worth noting that this study focuses primarily on fault classification, specifically identifying the component and type of fault (sensor, actuator, or communication). Future research will aim to extend this framework to include fault detection, isolation, and identification (FDII), enabling not only localization but also root cause analysis of faults. Additionally, while our current work provides comparisons between DFESN and standard methods such as ESN, CNN, LSTM, and BP neural networks, subsequent research could further extend these comparisons by exploring performance under more varied operational scenarios, including complex and combined fault conditions. While this study focused primarily on common fault types encountered in remote-controlled robotic arms—such as sensor faults, actuator faults, and communication faults—future investigations should incorporate more challenging fault scenarios, including simultaneous multi-sensor failures, sophisticated actuator degradations, and variable communication delay scenarios, to comprehensively evaluate the robustness of the proposed method. Finally, although real-time performance was evaluated primarily using online diagnosis time metrics, further research is necessary to explicitly examine other critical real-time factors, such as communication delays and tracking latency, thus ensuring more thorough insights into operational reliability under realistic working conditions.

5. Conclusions

In this work, we proposed a novel fault diagnosis technique for remote-controlled robotic arm systems, utilizing a neural network inspired by the DFESN. The DFESN model addresses the significant challenges posed by the high uncertainty and noise typical in teleoperated environments. By effectively handling smaller, three-dimensional datasets, the DFESN model proves to be both efficient and accurate in real-time fault diagnosis. Furthermore, we applied the covariance matrix adaptation evolution strategy (CMA-ES) to optimize the hyperparameters of the DFESN model, significantly enhancing its performance. This optimization was crucial in adapting the DFESN model for the complex dynamics involved in robotic fault diagnosis in real-time environments.

Our extensive experiments on a leader–follower robotic arm system demonstrate that the DFESN model achieves superior performance compared to traditional methods. Specifically, it achieves the highest mean accuracy of 99.5% while maintaining the shortest training time (4.75 s) and online diagnosis time (0.0105 s). These results highlight the DFESN model’s robustness and its ability to perform real-time fault diagnosis, which is critical for maintaining the reliability and safety of robotic arm systems in various applications.

The high classification accuracy across different fault types, with perfect classification for several fault categories, underscores the model’s effectiveness in distinguishing between various faults. The minimal misclassification rates further validate the DFESN model’s potential in practical applications where prompt and accurate fault detection is crucial.