Fractional Derivative in LSTM Networks: Adaptive Neuron Shape Modeling with the Grünwald–Letnikov Method

Abstract

The incorporation of fractional-order derivatives into neural networks presents a novel approach to improving gradient flow and adaptive learning dynamics. This paper introduces a fractional-order LSTM model, leveraging the Grünwald–Letnikov (GL) method to modify both activation functions and backpropagation mechanics. By redefining the transition functions of LSTM gates with fractional derivatives, the model achieves a smoother gradient adaptation while maintaining consistency across forward and backward passes. This is the first study integrating the Grünwald–Letnikov operator directly into both forward and backward LSTM computations, ensuring a consistent fractional framework throughout the entire learning process. We apply this approach to anomaly detection in fiber optic cable manufacturing, where small deviations in production parameters can significantly impact quality. A dataset containing time-series sensor measurements was used to train the fractional LSTM, demonstrating improved generalization and stability compared to classical LSTM models. Numerical stability analysis confirms that the fractional derivative framework allows convergent learning, preventing both vanishing and exploding gradients. Experimental results show that the fractional-order LSTM outperforms standard architectures in detecting manufacturing anomalies, with the optimal fractional order $\nu =0.95$ providing a balance between accuracy and computational complexity. The findings suggest that fractional calculus can enhance deep learning architectures by introducing a continuous and flexible transition between neuron activations, paving the way for adaptive neural networks with tunable memory effects.

1. Introduction

Fractional calculus has recently emerged as a robust mathematical framework for modeling dynamic systems with long-term memory and hereditary effects. Its integration into artificial neural networks has opened a new research direction, as explored in [1,2,3,4,5,6,7,8], where mathematical formulations of fractional-order neural networks have demonstrated improved learning stability and adaptability. Beyond neural modeling, fractional calculus has also been successfully applied to describing complex dynamic systems that cannot be captured by classical integer-order derivatives. Its usefulness has been demonstrated in modeling nonlinear temporal dependencies across diverse scientific domains, including tumor growth [9,10], ecological competition [9], hydrodynamic and oceanic systems, and biological oscillations [11]. Fractional-order differential equations have also been applied to chaotic systems and nonlinear oscillators [12,13], showing that fractional differentiation provides an effective mathematical tool for analyzing stability, multistability, and chaotic transitions. These findings indicate that fractional derivatives enable a more accurate description of long-term dependencies and complex feedback structures in dynamic systems.

Research combining fractional calculus and artificial intelligence has evolved along three main directions. First, studies addressing the mathematical and numerical foundations of fractional-order operators have focused on their discrete approximations and stability [14,15,16,17,18]. Second, neural modeling approaches have embedded fractional operators directly into learning architectures, improving gradient flow and convergence [19,20,21]. Finally, several applied works have demonstrated the potential of fractional frameworks in real-world contexts, including ecological prediction [22], image-based diagnostics [23], and biomedical signal processing [24]. Despite these advances, no previous study has proposed a unified recurrent formulation in which fractional differentiation operates consistently across both forward and backward computations. A comparative summary of key research directions and representative works is presented in

Table 1. Representative groups of studies related to fractional-order neural modeling.

Category	Representative Works	Method/Approach	Focus/Model	Key Contribution	Limitations
Fractional neural foundations	[1,2,3,4,5,6,7,8]	Fractional derivatives (Caputo, GL) applied to neural structures	Theoretical formulation of fractional-order neurons and backpropagation	Demonstrated improved adaptability and convergence in learning	Limited validation on recurrent or large-scale models
Fractional dynamic systems and applications	[9,10,11,12,13]	Fractional differential equations; chaotic and oscillatory models	Modeling long-term memory, stability, and nonlinear transitions	Showed fractional differentiation captures complex temporal dependencies	No integration with neural or learning frameworks
Mathematical foundations of GL operators	[14,15,16,17,18]	Grünwald–Letnikov discrete formulations; adaptive-memory solvers	Numerical accuracy, convergence, and stability of discrete fractional operators	Provided proofs of robustness and efficiency for GL discretization	Lack of neural application or learning-based context
Fractional neural architectures	[19,20,21]	Fractional operators embedded in FONN, complex-valued, and Caputo-type networks	Neural architectures with fractional gradients or activations	Improved gradient smoothness and dynamic response	Applied only to feed-forward or static networks
Applied fractional models	[22,23,24]	Hybrid CNN and autoregressive GL solvers; biomedical signal analysis	Ecological prediction, image-based classification, biomedical signal interpretation	Demonstrated cross-domain benefits of fractional frameworks	Lacking unified recurrent backpropagation or consistency in training

.

Building upon these foundations, this study introduces a Fractional Derivative Long Short-Term Memory (FD–LSTM) network that incorporates the Grünwald–Letnikov fractional operator directly into both the forward activation and backward propagation mechanisms. To demonstrate its practical relevance, the proposed model is validated on real-world data from fiber-optic cable manufacturing, a process highly sensitive to micro-scale irregularities that can affect signal quality and mechanical durability. The classical LSTM architecture, introduced by Hochreiter and Schmidhuber [25], relies on predefined transition functions with first-order derivatives. The proposed model extends this structure by embedding fractional differentiation consistently across all computation stages, ensuring mathematical coherence between forward and backward gradient flow. To the best of our knowledge, this is the first approach that provides a unified fractional-order learning mechanism within recurrent neural networks. The FD–LSTM enhances the adaptability of gate functions, enabling smooth representation of long-term dependencies and stable gradient propagation.

Several fractional-order LSTM variants have been proposed in recent years, including architectures based on Caputo derivatives [19], Caputo–Fabrizio models [21], Atangana–Baleanu formulations [7], and fractional memory units [8]. However, these works typically apply fractional operators only to selected components of the architecture, such as the forward activation function or the cell-state transition, while retaining classical backpropagation. As a result, the forward and backward memory mechanisms are governed by different mathematical rules.

In contrast, the FD–LSTM proposed in this study applies a unified Grünwald–Letnikov operator consistently to both the forward-pass gate transformations and the full backward gradient computation. This ensures mathematical coherence between activation dynamics and gradient flow, and, to the best of our knowledge, has not been addressed in previous fractional-order LSTM formulations.

Table 2. Comparison of existing fractional-order LSTM formulations. Our model is the first to apply the same fractional operator consistently to both forward and backward computations.

Work	Fractional Operator	Forward Pass	Backward Pass
C.J. Zuñiga Aguilar et al. [19]	Caputo	Fractional activation only	Classical
Sivalingam S.M. et al. (2021) [21]	Caputo–Fabrizio	Fractional cell update	Classical
Mohanrasu et al. (2022) [7]	A–B	Partial fractional gates	Classical
Wei et al. (2022) [8]	FO memory units	Fractional state update	Classical
This work	GL	Full fractional gates	Full fractional backpropagation

summarises conceptual differences between existing fractional LSTM formulations and the unified GL-based architecture proposed in this work. In addition to unifying the fractional mechanisms in forward and backward passes, the proposed formulation derives the LSTM gate activations from GL-based fractional derivatives of smooth base potentials, enabling a continuous modulation of gate shape as a function of the fractional order $\nu$.

The challenges of anomaly detection in fiber-optic cable production have been extensively analyzed in [26]. Broader perspectives on AI-driven anomaly detection in industrial environments can be found in [27,28,29,30,31,32,33,34]. Experimental validation on real-world data confirms that the fractional-order LSTM network improves generalization, stability, and detection accuracy compared to classical and Caputo-based recurrent models, demonstrating its potential for real-time industrial monitoring and adaptive control.

2. Materials and Methods

2.1. Mathematical Model of the Proposed Fractional Order LSTM Cell

In this section, we introduce the mathematical model of the proposed LSTM network based on standard architectures like [35,36,37]. The implementation of the novel LSTM cell is based on the fractional derivative formulated using the direct Grünwald–Letnikov definition, as presented in Equation (1), where $\left({\displaystyle \genfrac{}{}{0pt}{}{\nu }{i}}\right)$ denotes the Newton binomial, $\nu$ is the order of fractional derivative of basis function $f_B\left(x\right)$, n is the number of coefficients, and h is the step of discretization [38,39].

$$D^{\nu }{f}_B\left(x\right)=\underset{h\to 0^{+}}{lim}{\displaystyle \frac{{\sum }_{i=0}^n{\left(-1\right)}^i\left(\genfrac{}{}{0pt}{}{\nu }{i}\right)f_B\left(x-ih\right)}{h^{\nu }}}$$

(1)

The implementation of the GL method in this work was constructed within the mathematical interface of the TensorFlow library to be used in the most efficient way in its neural network models. Figure 1 shows the classical model of the LSTM cell proposed by Sepp Hochreiter in [25].

The LSTM cell consists of two types of transitions: logic gates and the candidate cell state. The implementation of an LSTM cell based on fractional-order derivatives required modifications to the operation of these cells to enable smooth adjustment of the transition functions. Additionally, during backpropagation, it was necessary to modify the computation of gradients to ensure the correct approximation of the derivatives. This was crucial for obtaining the first-order derivatives of the transition functions, which were derived using fractional-order derivatives, enabling effective gradient propagation through the network.

2.2. Forward Model

In fractional-order neural networks, activation functions can be generalized using fractional derivatives. Instead of directly defining the activation function, we introduce a general base function $f_B\left(x\right)$, whose fractional derivative reproduces the desired neuron nonlinearity [40,41,42,43,44]. The standard sigmoid activation function is given by

$$\sigma \left(x\right)={\displaystyle \frac{1}{1+e^{-x}}}$$

(2)

To derive the sigmoid function from a fractional derivative, we choose the softplus function as the base function:

$$f_{B_{\sigma }}\left(x\right)=log(1+e^x)$$

(3)

and its classical first derivative is

$${\displaystyle \frac{d}{dx}}{f}_{B_{\sigma }}\left(x\right)={\displaystyle \frac{e^x}{1+e^x}}=\sigma \left(x\right)$$

(4)

For fractional-order derivatives using the Grünwald–Letnikov definition, we generalize this as

$$D^{\nu }{f}_{B_{\sigma }}\left(x\right)=\sum _{k=0}^{\infty }{\displaystyle \frac{{(-1)}^k}{h^{\nu }}}\left({\displaystyle \genfrac{}{}{0pt}{}{\nu }{k}}\right)log(1+e^{x-kh})$$

(5)

For $\nu =1$, this recovers the standard sigmoid function. Per analogy, the hyperbolic tangent activation function can be expressed as

$$tanh\left(x\right)={\displaystyle \frac{e^x-e^{-x}}{e^x+e^{-x}}}$$

(6)

can be obtained with the use of log(cosh) function as the base function $f_{B_{tanh}}\left(x\right)$ denoted:

$$f_{B_{tanh}}\left(x\right)=log\left(cosh\left(x\right)\right)$$

(7)

Taking its classical first derivative,

$${\displaystyle \frac{d}{dx}}{f}_{B_{tanh}}\left(x\right)=tanh\left(x\right).$$

(8)

The fractional-order Grünwald–Letnikov derivative is

$$D^{\nu }{f}_{B_{tanh}}\left(x\right)=\sum _{k=0}^{\infty }{\displaystyle \frac{{(-1)}^k}{h^{\nu }}}\left({\displaystyle \genfrac{}{}{0pt}{}{\nu }{k}}\right)log(cosh(x-kh))$$

(9)

Using these base functions, we can smoothly generalize activation functions using fractional-order derivatives. This allows us to continuously transition between different activation behaviours by tuning $\nu$, offering greater flexibility in neural network modeling.

In practical implementations, the infinite Grünwald–Letnikov series in Equation (9) is replaced by a truncated formulation. Based on our convergence experiments, we adopt a coefficient length of $N=20$ for all experiments reported in this study. Values of $N\ge 20$ ensure numerical stability and suppress oscillatory behaviour for both sigmoid-like and tanh-like base functions, while keeping the computational overhead moderate. Shorter stencils ($N<15$) tend to produce visible oscillations and degrade gradient stability, whereas increasing the stencil size beyond $N=25$ yields negligible improvements relative to the additional computational cost. Therefore, $N=20$ provides a practical trade-off between convergence reliability and runtime efficiency and is used consistently throughout the Section 3. The fractional order $\nu$ plays a central role in shaping both the activation function and the resulting gradient dynamics. In the Grünwald–Letnikov formulation, $\nu$ controls the decay rate of the fractional coefficients

$$w_k\left(\nu \right)={(-1)}^k\left({\displaystyle \genfrac{}{}{0pt}{}{\nu }{k}}\right)$$

(10)

which determine how strongly past shifted evaluations $f(x-kh)$ contribute to the fractional derivative. Lower values of $\nu$ increase the influence of long-range history, but also amplify oscillatory behaviour of the derivative, resulting in unstable gradient propagation for $\nu <0.7$. Conversely, values in the range $\nu \in [0.85, 0.98]$ provide smooth gradient flow and stable convergence, as the fractional weights decay steadily while retaining sufficient memory depth. In practice, the tested values $\nu =0.80, 0.90, 0.95$ span the most commonly adopted regime in fractional neural modelling: they preserve stability, modulate the nonlinearity of the fractional activation functions, and yield smooth transitions between classical ($\nu =1$) and fractional memory behaviours. For $\nu =1$, all expressions recover the standard LSTM activations and gradients, while for $\nu <1$ the fractional operator introduces a controllable attenuation of gradient magnitudes, acting as a natural smoothing mechanism in both forward and backward passes.

2.2.1. Gates

The LSTM cell has three logic gates, which are responsible for adjusting the values of the short-term memory vector $\left(h\right)$ and the long-term memory vector $\left(c\right)$. All of them operate as dense layers, implementing the sigmoid activation function. Each of these cells is computed using an almost identical logical value calculation. Equations (11)–(13) present the operations performed by the gates, where $\sigma$ denotes the sigmoid function, $W_x$ is the corresponding weight matrix, $x_{\left(t\right)}$ is the next candidate, and $h_{\left(t-1\right)}$ is the short-term memory vector from the last timestep.

$$f_{\left(t\right)}=\sigma \left(W_{xf}^T{x}_{\left(t\right)}+W_{hf}^T{h}_{\left(t-1\right)}+b_f\right)$$

(11)

$$i_{\left(t\right)}=\sigma \left(W_{xi}^T{x}_{\left(t\right)}+W_{hi}^T{h}_{\left(t-1\right)}+b_i\right)$$

(12)

$$o_{\left(t\right)}=\sigma \left(W_{xo}^T{x}_{\left(t\right)}+W_{ho}^T{h}_{\left(t-1\right)}+b_o\right)$$

(13)

One of the elements of the proposed network is enabling smooth adjustment of the sigmoid function, including in the gates. For this purpose, the softplus function was selected as the base function, which is the classical first-order integral of the sigmoid function, allowing for the approximation of functions surrounding the sigmoid function. After introducing the fractional derivatives, the cell equations took the forms presented in Equations (14)–(16).

$$f_{\left(t\right)}=D^{\nu }{f}_{B_{\sigma }}\left(W_{xf}^T{x}_{\left(t\right)}+W_{hf}^T{h}_{\left(t-1\right)}+b_f\right)$$

(14)

$$i_{\left(t\right)}=D^{\nu }{f}_{B_{\sigma }}\left(W_{xi}^T{x}_{\left(t\right)}+W_{hi}^T{h}_{\left(t-1\right)}+b_i\right)$$

(15)

$$o_{\left(t\right)}=D^{\nu }{f}_{B_{\sigma }}\left(W_{xo}^T{x}_{\left(t\right)}+W_{ho}^T{h}_{\left(t-1\right)}+b_o\right)$$

(16)

Figure 2a presents the result of approximating fractional-order derivatives for the softplus base function, which were used in the gate computation process. Figure 2b presents the approximations of fractional-order derivatives for the base function presented in Equation (18).

2.2.2. Candidate Cell State

Another type of transition in the LSTM cell is the candidate cell state, which is responsible for calculating the base value of the current timestep. Similar to the gates, it is a dense layer, but it implements a different transition function—tanh. Candidate cell state is calculated using the same formula as the logic gates, presented in Equation (17).

$$g_{\left(t\right)}=tanh\left(W_{xg}^T{x}_{\left(t\right)}+W_{hg}^T{h}_{\left(t-1\right)}+b_g\right)$$

(17)

Substituting the fractional-order derivative into the calculating function results we obtain

$$g_{\left(t\right)}=D^{\nu }{f}_{B_{tanh}}\left(W_{xg}^T{x}_{\left(t\right)}+W_{hg}^T{h}_{\left(t-1\right)}+b_g\right)$$

(18)

2.2.3. Update of Memory Vectors

The computed values of the gates and the candidate cell state are used by the network to update the short-term and long-term memory vectors. Equations (19) and (20) present the update of the long-term memory vector, which uses gates that employ fractional-order derivatives for approximating the transition functions.

$$c_{\left(t\right)}=f_{\left(t\right)}\otimes c_{\left(t-1\right)}+i_{\left(t\right)}\otimes g_{\left(t\right)}$$

(19)

$$\begin{array}{cc}\hfill c_{\left(t\right)}& =D^{\nu }{f}_{B_{\sigma }}\left(W_{xf}^T{x}_{\left(t\right)}+W_{hf}^T{h}_{\left(t-1\right)}+b_f\right)\otimes c_{\left(t-1\right)}+D^{\nu }{f}_{B_{\sigma }}\left(W_{xi}^T{x}_{\left(t\right)}+W_{hi}^T{h}_{\left(t-1\right)}+b_i\right)\hfill \\ & \otimes D^{\nu }{f}_{B_{tanh}}\left(W_{xo}^T{x}_{\left(t\right)}+W_{ho}^T{h}_{\left(t-1\right)}+b_o\right)\hfill \end{array}$$

(20)

Similarly, Equations (21) and (22) present the update of the short-term memory vector using fractional-order derivative mechanisms.

$$h_{\left(t\right)}=o_{\left(t\right)}\otimes tanh\left(c_{\left(t\right)}\right)$$

(21)

$$h_{\left(t\right)}=D^{\nu }{f}_{B_{\sigma }}\left(W_{xo}^T{x}_{\left(t\right)}+W_{ho}^T{h}_{\left(t-1\right)}+b_o\right)\otimes tanh\left(c_{\left(t\right)}\right)$$

(22)

2.3. Backpropagation Through Time

Backpropagation Through Time (BPTT) is a fundamental optimization algorithm for training recurrent neural networks by unfolding them over time and applying backpropagation to compute gradients across multiple timesteps. By leveraging weight sharing, BPTT enables the effective learning of temporal dependencies in sequential data. Despite its computational complexity and sensitivity to vanishing or exploding gradients, it remains a core method for training more advanced architectures such as LSTM and Gated Recurrent Units [35,45]. The LSTM cell is trained using the loss function presented in Equation (23), where y denotes the real value, and $h_{\left(t\right)}$ is the prediction of the LSTM cell.

$$L=-{\displaystyle \frac{1}{2}}{\left(y-h_{\left(t\right)}\right)}^2$$

(23)

We assumed the following equations representing the update operations for the weights in the BPTT process for the classic LSTM cell:

$$W_{xo}=W_{xo}-\alpha {\displaystyle \frac{\partial L}{\partial W_{xo}}};W_{ho}=W_{ho}-\alpha {\displaystyle \frac{\partial L}{\partial W_{ho}}};b_o=b_o-\alpha {\displaystyle \frac{\partial L}{\partial b_o}}$$

(24)

$$W_{xf}=W_{xf}-\alpha {\displaystyle \frac{\partial L}{\partial W_{xf}}};W_{hf}=W_{hf}-\alpha {\displaystyle \frac{\partial L}{\partial W_{hf}}};b_f=b_f-\alpha {\displaystyle \frac{\partial L}{\partial b_f}}$$

(25)

$$W_{xi}=W_{xi}-\alpha {\displaystyle \frac{\partial L}{\partial W_{xi}}};W_{hi}=W_{hi}-\alpha {\displaystyle \frac{\partial L}{\partial W_{hi}}};b_i=b_i-\alpha {\displaystyle \frac{\partial L}{\partial b_i}}$$

(26)

$$W_{xg}=W_{xg}-\alpha {\displaystyle \frac{\partial L}{\partial W_{xg}}};W_{hg}=W_{hg}-\alpha {\displaystyle \frac{\partial L}{\partial W_{hg}}};b_g=b_g-\alpha {\displaystyle \frac{\partial L}{\partial b_g}}$$

(27)

In each equation, $\alpha$ represents the learning rate. For clarity of notation, let us introduce additional symbols.

$$f_{in}=W_{hf}{h}_{\left(t-1\right)}+W_{xf}{x}_{\left(t\right)}+b_f;f_{out}=\sigma \left(f_{in}\right)$$

(28)

$$o_{in}=W_{ho}{h}_{\left(t-1\right)}+W_{xo}{x}_{\left(t\right)}+b_o;o_{out}=\sigma \left(o_{in}\right)$$

(29)

$$i_{in}=W_{hi}{h}_{\left(t-1\right)}+W_{xi}{x}_{\left(t\right)}+b_i;i_{out}=\sigma \left(i_{in}\right)$$

(30)

$$g_{in}=W_{xg}^T{x}_{\left(t\right)}+W_{hg}^T{h}_{\left(t-1\right)}+b_g;g_{out}=tanh\left(g_{in}\right)$$

(31)

Starting with the forget gate, Equations (32)–(34) present the expansion of the previous formulas using the chain rule.

$${\displaystyle \frac{\partial L}{\partial W_{xo}}}={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial o_{out}}}{\displaystyle \frac{\partial o_{out}}{\partial o_{in}}}{\displaystyle \frac{\partial o_{in}}{\partial W_{xo}}}=(y-h_{\left(t\right)})·tanh\left(c_{\left(t\right)}\right)·o_{out}·(1-o_{out})·x_{\left(t\right)}$$

(32)

$${\displaystyle \frac{\partial L}{\partial W_{ho}}}={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial o_{out}}}{\displaystyle \frac{\partial o_{out}}{\partial o_{in}}}{\displaystyle \frac{\partial o_{in}}{\partial W_{ho}}}=(y-h_{\left(t\right)})·tanh\left(c_{\left(t\right)}\right)·o_{out}·(1-o_{out})·h_{\left(t-1\right)}$$

(33)

$${\displaystyle \frac{\partial L}{\partial b_o}}={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial o_{out}}}{\displaystyle \frac{\partial o_{out}}{\partial o_{in}}}{\displaystyle \frac{\partial o_{in}}{\partial b_o}}=(y-h_{\left(t\right)})·tanh\left(c_{\left(t\right)}\right)·o_{out}·(1-o_{out})$$

(34)

The mathematical model of the backpropagation algorithm utilizing fractional-order derivatives can be derived by replacing the relevant partial derivatives with their approximated version from the GL formula. Equations (35)–(37) present the updates of the output gate components in the process utilizing fractional-order derivatives.

$${\displaystyle \frac{\partial L}{\partial W_{xo}}}={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial o_{out}}}{\displaystyle \frac{\partial o_{out}}{\partial o_{in}}}{\displaystyle \frac{\partial o_{in}}{\partial W_{xo}}}=(y-h_{\left(t\right)})·tanh\left(c_{\left(t\right)}\right)·D^{\nu +1}{f}_{B_{\sigma }}\left(W_{xo}^T{x}_{\left(t\right)}+W_{ho}^T{h}_{\left(t-1\right)}+b_o\right)·x_{\left(t\right)}$$

(35)

$${\displaystyle \frac{\partial L}{\partial W_{ho}}}={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial o_{out}}}{\displaystyle \frac{\partial o_{out}}{\partial o_{in}}}{\displaystyle \frac{\partial o_{in}}{\partial W_{ho}}}=(y-h_{\left(t\right)})·tanh\left(c_{\left(t\right)}\right)·D^{\nu +1}{f}_{B_{\sigma }}\left(W_{xo}^T{x}_{\left(t\right)}+W_{ho}^T{h}_{\left(t-1\right)}+b_o\right)·h_{\left(t-1\right)}$$

(36)

$${\displaystyle \frac{\partial L}{\partial b_o}}={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial o_{out}}}{\displaystyle \frac{\partial o_{out}}{\partial o_{in}}}{\displaystyle \frac{\partial o_{in}}{\partial b_o}}=(y-h_{\left(t\right)})·tanh\left(c_{\left(t\right)}\right)·D^{\nu +1}{f}_{B_{\sigma }}\left(W_{xo}^T{x}_{\left(t\right)}+W_{ho}^T{h}_{\left(t-1\right)}+b_o\right)$$

(37)

Next, the classical update of the forget gate components was considered. Equations (38)–(40) present the formulas for updating the components of the forget gate.

$${\displaystyle \frac{\partial L}{\partial W_{xf}}}={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial f_{out}}}{\displaystyle \frac{\partial f_{out}}{\partial f_{in}}}{\displaystyle \frac{\partial f_{in}}{\partial W_{xf}}}=\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)·c_{\left(t-1\right)}·f_{out}·\left(1-f_{out}\right)·x_{\left(t\right)}$$

(38)

$${\displaystyle \frac{\partial L}{\partial W_{hf}}}={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial f_{out}}}{\displaystyle \frac{\partial f_{out}}{\partial f_{in}}}{\displaystyle \frac{\partial f_{in}}{\partial W_{hf}}}=\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)·c_{\left(t-1\right)}·f_{out}·\left(1-f_{out}\right)·h_{\left(t-1\right)}$$

(39)

$${\displaystyle \frac{\partial L}{\partial b_f}}={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial f_{out}}}{\displaystyle \frac{\partial f_{out}}{\partial f_{in}}}{\displaystyle \frac{\partial f_{in}}{\partial b_f}}=\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)·c_{\left(t-1\right)}·f_{out}·\left(1-f_{out}\right)$$

(40)

Equations (41)–(43) present the update of the forget gate components in the backpropagation process utilizing fractional-order derivatives.

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial L}{\partial W_{xf}}}& ={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial f_{out}}}{\displaystyle \frac{\partial f_{out}}{\partial f_{in}}}{\displaystyle \frac{\partial f_{in}}{\partial W_{xf}}}=\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)\hfill \\ & ·c_{\left(t-1\right)}·D^{\nu +1}{f}_{B_{\sigma }}\left(W_{xf}^T{x}_{\left(t\right)}+W_{hf}^T{h}_{\left(t-1\right)}+b_f\right)·x_{\left(t\right)}\hfill \end{array}$$

(41)

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial L}{\partial W_{hf}}}& ={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial f_{out}}}{\displaystyle \frac{\partial f_{out}}{\partial f_{in}}}{\displaystyle \frac{\partial f_{in}}{\partial W_{hf}}}=\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)\hfill \\ & ·c_{\left(t-1\right)}·D^{\nu +1}{f}_{B_{\sigma }}\left(W_{xf}^T{x}_{\left(t\right)}+W_{hf}^T{h}_{\left(t-1\right)}+b_f\right)·h_{\left(t-1\right)}\hfill \end{array}$$

(42)

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial L}{\partial b_f}}& ={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial f_{out}}}{\displaystyle \frac{\partial f_{out}}{\partial f_{in}}}{\displaystyle \frac{\partial f_{in}}{\partial b_f}}=\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)\hfill \\ & ·c_{\left(t-1\right)}·D^{\nu +1}{f}_{B_{\sigma }}\left(W_{xf}^T{x}_{\left(t\right)}+W_{hf}^T{h}_{\left(t-1\right)}+b_f\right)\hfill \end{array}$$

(43)

Next, an identical analysis was conducted for the input gate. Equations (44)–(46) present the formulas for updating the components of the input gate.

$${\displaystyle \frac{\partial L}{\partial W_{xi}}}={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial i_{out}}}{\displaystyle \frac{\partial i_{out}}{\partial i_{in}}}{\displaystyle \frac{\partial i_{in}}{\partial W_{xi}}}=\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)·c_{\left(t-1\right)}·i_{out}·\left(1-i_{out}\right)·x_{\left(t\right)}$$

(44)

$${\displaystyle \frac{\partial L}{\partial W_{hi}}}={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial i_{out}}}{\displaystyle \frac{\partial i_{out}}{\partial i_{in}}}{\displaystyle \frac{\partial i_{in}}{\partial W_{hi}}}=\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)·c_{\left(t-1\right)}·i_{out}·\left(1-i_{out}\right)·h_{\left(t-1\right)}$$

(45)

$${\displaystyle \frac{\partial L}{\partial b_i}}={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial i_{out}}}{\displaystyle \frac{\partial i_{out}}{\partial i_{in}}}{\displaystyle \frac{\partial i_{in}}{\partial b_i}}=\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)·c_{\left(t-1\right)}·i_{out}·\left(1-i_{out}\right)$$

(46)

Equations (47)–(49) present the formulas for updating the components of the input gate using the GL method.

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial L}{\partial W_{xi}}}& ={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial i_{out}}}{\displaystyle \frac{\partial i_{out}}{\partial i_{in}}}{\displaystyle \frac{\partial i_{in}}{\partial W_{xi}}}\hfill \\ \hfill & =\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)·c_{\left(t-1\right)}\hfill \\ & ·D^{\nu +1}{f}_{B_{\sigma }}\left(W_{xi}^T{x}_{\left(t\right)}+W_{hi}^T{h}_{\left(t-1\right)}+b_i\right)·x_{\left(t\right)}\hfill \end{array}$$

(47)

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial L}{\partial W_{hi}}}& ={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial i_{out}}}{\displaystyle \frac{\partial i_{out}}{\partial i_{in}}}{\displaystyle \frac{\partial i_{in}}{\partial W_{hi}}}\hfill \\ \hfill & =\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)·c_{\left(t-1\right)}\hfill \\ & ·D^{\nu +1}{f}_{B_{\sigma }}\left(W_{xi}^T{x}_{\left(t\right)}+W_{hi}^T{h}_{\left(t-1\right)}+b_i\right)·h_{\left(t-1\right)}\hfill \end{array}$$

(48)

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial L}{\partial b_i}}& ={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial i_{out}}}{\displaystyle \frac{\partial i_{out}}{\partial i_{in}}}{\displaystyle \frac{\partial i_{in}}{\partial b_i}}\hfill \\ \hfill & =\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)·c_{\left(t-1\right)}\hfill \\ & ·D^{\nu +1}{f}_{B_{\sigma }}\left(W_{xi}^T{x}_{\left(t\right)}+W_{hi}^T{h}_{\left(t-1\right)}+b_i\right)\hfill \end{array}$$

(49)

Finally, in Equations (50)–(52), an analysis of the classical update of the candidate cell state components was conducted.

$${\displaystyle \frac{\partial L}{\partial W_{xg}}}={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial g_{out}}}{\displaystyle \frac{\partial g_{out}}{\partial g_{in}}}{\displaystyle \frac{\partial g_{in}}{\partial W_{xg}}}=\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)·i_{out}·\left(1-{tanh}^2\left(g_{in}\right)\right)·x_t$$

(50)

$${\displaystyle \frac{\partial L}{\partial W_{hg}}}={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial g_{out}}}{\displaystyle \frac{\partial g_{out}}{\partial g_{in}}}{\displaystyle \frac{\partial g_{in}}{\partial W_{hg}}}=\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)·i_{out}·\left(1-{tanh}^2\left(g_{in}\right)\right)·h_{\left(t-1\right)}$$

(51)

$${\displaystyle \frac{\partial L}{\partial b_g}}={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial g_{out}}}{\displaystyle \frac{\partial g_{out}}{\partial g_{in}}}{\displaystyle \frac{\partial g_{in}}{\partial b_g}}=\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)·i_{out}·\left(1-{tanh}^2\left(g_{in}\right)\right)$$

(52)

Equations (53)–(55) present the process of updating the components of the candidate cell state in the backpropagation process utilizing the GL method.

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial L}{\partial W_{xg}}}& ={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial g_{out}}}{\displaystyle \frac{\partial g_{out}}{\partial g_{in}}}{\displaystyle \frac{\partial g_{in}}{\partial W_{xg}}}=\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)\hfill \\ & ·i_{out}·D^{\nu +1}{f}_{B_{tanh}}\left(W_{xg}^T{x}_{\left(t\right)}+W_{hg}^T{h}_{\left(t-1\right)}+b_g\right)·x_t\hfill \end{array}$$

(53)

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial L}{\partial W_{hg}}}& ={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial g_{out}}}{\displaystyle \frac{\partial g_{out}}{\partial g_{in}}}{\displaystyle \frac{\partial g_{in}}{\partial W_{hg}}}=\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)\hfill \\ & ·i_{out}·D^{\nu +1}{f}_{B_{tanh}}\left(W_{xg}^T{x}_{\left(t\right)}+W_{hg}^T{h}_{\left(t-1\right)}+b_g\right)·h_{\left(t-1\right)}\hfill \end{array}$$

(54)

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial L}{\partial b_g}}& ={\displaystyle \frac{\partial L}{\partial h_{\left(t\right)}}}{\displaystyle \frac{\partial h_{\left(t\right)}}{\partial c_{\left(t\right)}}}{\displaystyle \frac{\partial c_{\left(t\right)}}{\partial g_{out}}}{\displaystyle \frac{\partial g_{out}}{\partial g_{in}}}{\displaystyle \frac{\partial g_{in}}{\partial b_g}}=\left(y-h_{\left(t\right)}\right)·o_{out}·\left(1-{tanh}^2\left(c_{\left(t\right)}\right)\right)\hfill \\ & ·i_{out}·D^{\nu +1}{f}_{B_{tanh}}\left(W_{xg}^T{x}_{\left(t\right)}+W_{hg}^T{h}_{\left(t-1\right)}+b_g\right)\hfill \end{array}$$

(55)

By explicitly incorporating $f_{B_{\sigma }}\left(x\right)$ and $f_{B_{tanh}}\left(x\right)$ in the fractional backpropagation framework, the learning process maintains a smooth gradient flow while allowing adaptive activation transitions. This generalization ensures that the fractional LSTM model retains the expressive power of classical LSTMs while introducing continuous tunability of neuron dynamics.

2.4. Methodology for Convergence Analysis

To evaluate the numerical behaviour and convergence properties of the fractional Grünwald–Letnikov derivatives, we performed 3D simulations over the domain: input: $x\in [-5, 5]$, step size: $h\in \{0.01, 0.05, 0.1, 0.2\}$, fractional order: $\nu \in [0, 1]$. For each fractional order $\nu$, we evaluated the smoothness and convergence behaviour of the computed derivatives (see Figure 3).

Beyond the empirical gradient-norm trajectories, the stabilising behaviour of the Grünwald–Letnikov operator is also supported by established analytical properties of fractional differences. In the GL formulation, the derivative is expressed as a convolution with a sequence of coefficients that decay monotonically and form a Mittag–Leffler-type memory kernel. Such kernels are known to suppress high-frequency oscillations and to attenuate abrupt changes in iterative updates, providing a natural damping mechanism absent in integer-order derivatives. Prior studies on fractional-order dynamical systems ([14,17,18]) demonstrate that GL-based fractional differences lead to bounded iterative behaviour and mitigate explosive growth in discretised recurrence relations. While a full spectral eigenvalue analysis of the unrolled FD–LSTM Jacobian is mathematically non-trivial and beyond the scope of this work, these theoretical insights align with the empirically observed reduction in gradient-norm variance and the absence of vanishing or exploding gradients for $\nu \in [0.85, 0.98]$.

The results for the sigmoid-like function reveal that for small $\nu$ (e.g., $\nu \approx 0.2$), the fractional derivative closely resembles a linear function, behaving like an identity mapping. As $\nu \to 1$, the fractional derivative smoothly transitions toward the standard sigmoid derivative, confirming that the Grünwald–Letnikov approximation preserves the expected activation behaviour. Larger step sizes h lead to oscillations, but for $h\le 0.1$, the computation remains stable. For the tanh-like function, we observe smooth convergence for all $\nu$, indicating numerical stability. Smaller $\nu$ values lead to an attenuated gradient, which can help prevent exploding gradients in backpropagation. The Mittag-Leffler decay property appears for intermediate $\nu$, demonstrating the memory effect of fractional derivatives. These findings confirm that fractional derivatives generalize standard activation functions while maintaining numerical stability. By using fractional-order activations, neural networks gain enhanced flexibility in function approximation while maintaining reliable convergence properties.

3. Results and Discussion

3.1. Task Definition and Data

We address anomaly detection in fiber-optic cable manufacturing from streaming industrial process data. The model continuously classifies the last 256 s of signals into predefined operational states to enable real-time intervention and improved product quality. The dataset comprises ∼5690 windows recorded at 1 Hz with 294 raw features; after variance and correlation filtering, we retain 40 indicators.

The raw dataset provides a coarse expert-verified indicator at the session level: 0 denotes a session with no abnormal behaviour, while 1 indicates that at least one anomaly occurred somewhere within the session. Importantly, this label does not specify the exact timing, duration, or type of the anomaly, nor does it apply uniformly to all windows extracted from that session. A session marked as anomalous may contain normal windows, windows with mild deviations, and windows exhibiting different physical fault mechanisms. Because the session-level label is not sufficiently informative for window-level training, we apply a weakly supervised relabelling procedure. Specifically, K-means clustering with $k=6$ is performed on the subset of windows extracted from anomaly-marked sessions. This yields six operationally distinct anomaly subtypes with different temporal signatures and physical origins. These refined categories provide meaningful window-level supervision and allow the FD–LSTM to learn sensitivity to heterogeneous anomaly patterns rather than a single coarse anomaly flag. All variables are standardized using the Z-score transformation before model training:

$$x^{\prime }={\displaystyle \frac{x-\mu }{\sigma }}$$

(56)

and segmented into 256-timestep windows with 50% overlap, producing ∼2500 labeled sequences for modeling.

To ensure the methodological correctness of the evaluation and to eliminate potential temporal leakage, we adopted a strictly time-aware data-splitting protocol. All sequence windows were ordered chronologically and were not shuffled at any stage of the experimental pipeline. The dataset was partitioned using a forward-in-time split, with 90% of the windows used for training and validation and the remaining 10%—corresponding to the most recent segment of the production timeline—reserved as a held-out test set. This ratio is widely adopted in industrial time-series modelling, as it preserves a representative evaluation segment while ensuring that the training set captures the full variability of operational conditions. The 10% test portion includes complete production cycles and all anomaly categories, enabling reliable generalisation assessment without compromising model learning capacity. Evaluating the model exclusively on future, unseen temporal segments reflects a realistic deployment scenario and prevents the network from accessing any information derived from later time intervals. Although the windows were generated with 50% overlap to capture smooth temporal dynamics, their chronological order was preserved throughout, ensuring that no leakage occurred between training and test partitions. The held-out portion of the dataset also contains all six anomaly categories obtained through weakly supervised relabelling, providing representative coverage of the underlying process behaviour. This time-aware dataset organisation enables a fair and robust assessment of the generalisation performance of the proposed FD–LSTM model under real industrial conditions.

3.2. Experimental Setup

All experiments were implemented in TensorFlow 2.14 (Python 3.11, CUDA 12.2) on an NVIDIA RTX 4090 GPU. We trained for 200 epochs with batch size 64, learning rate 0.01, early stopping (patience = 20), and SGD with Nesterov momentum 0.9. Results were averaged over 10 runs with different seeds. The FD–LSTM architecture is as follows: Input: 40 features × 256 timesteps, Hidden: 256 LSTM units implementing the Grünwald–Letnikov fractional derivative operator, Output: 6 softmax units (fault categories). We used categorical cross-entropy and report accuracy, F1, and the goodness-of-fit index $\Phi$.

3.3. Overall Results and Learning Dynamics

Figure 4 confirms smooth convergence across fractional orders $\nu$.

Table 3. Evaluation metrics of the trained FD–LSTM models for different fractional orders $\nu$.

$\mathit{\nu }$	Test Dataset (10%)		Full Dataset
	Accuracy	Goodness-of-Fit $\Phi$	Accuracy	Goodness-of-Fit $\Phi$
0.90	0.9000	0.8940	0.9580	0.9526
0.95	0.9185	0.9145	0.9599	0.9543
1.00	0.9148	0.9051	0.9595	0.9550
1.05	0.8925	0.8863	0.9517	0.9462
1.10	0.8935	0.8867	0.9443	0.9387

summarizes the metrics from the author’s experiments; the best performance is obtained at $\nu =0.95$ with an accuracy of 0.9185 and $\Phi =0.9145$ on the held-out test set.

To further analyse the optimisation process of the proposed fractional mechanism, we compared the learning dynamics across several recurrent baselines trained under an identical protocol (200 epochs, batch size 64, SGD with Nesterov momentum). This comparative evaluation enables a clearer interpretation of how the GL-based fractional operator influences convergence speed, gradient stability, and generalisation performance relative to classical and fractional recurrent architectures (see

Table 4. Comparison of convergence behaviour and final performance across recurrent architectures. The FD–LSTM (GL) model exhibits the fastest early convergence and the highest stability of gradients.

Model	Early Loss Drop	Epoch of Stabilization	Accuracy	F1
Classical LSTM	Moderate	∼120	0.914	0.905
Bi–LSTM	Fast	∼100	0.921	0.912
GRU	Moderate	∼110	0.909	0.900
Fractional LSTM (Caputo)	Slow–Moderate	∼130	0.910	0.908
FD–LSTM (GL)	Fastest	∼80	0.9185	0.915

). The “Classical LSTM” reported in Table 4 corresponds to the standard TensorFlow LSTM baseline trained under the unified protocol described in Section 3.2. In contrast, the “Classic Model” in Table 3 represents the GL-based architecture evaluated at $\nu =1.0$ within the fractional-order sweep. These two baselines belong to conceptually different experimental setups, and therefore, their accuracy values are not expected to be identical.

A complementary view of the learning dynamics is provided by the extended loss curves (Figure 5), which visualise the optimisation trajectory of each model throughout the 200 training epochs.

The extended loss trajectories reveal several important trends. First, FD–LSTM (GL) decreases the loss more rapidly than all other examined architectures during the initial 40–60 epochs, indicating more efficient gradient propagation and faster error minimisation. This behaviour reflects the smoothing properties of the Grünwald–Letnikov operator, which mitigate both vanishing and exploding gradient phenomena. Second, the fractional model exhibits substantially reduced mid-training oscillations compared with classical LSTM, GRU and Caputo-based fractional LSTM, suggesting that the GL operator acts as an adaptive regulariser for both forward activations and backward gradients. Finally, FD–LSTM stabilises notably earlier (around epoch 80) than Bi–LSTM (epoch 100) and classical recurrent models (110–130 epochs). This indicates that GL-based fractional memory provides a more efficient representation of long-term dependencies, directly translating into superior generalisation performance, as reflected by the accuracy and F1 values reported in Table 4. Table 3 reports the performance of the FD–LSTM(GL) architecture across different fractional orders $\nu$. This ablation study includes only models constructed within the GL–based fractional framework. The row $\nu =1.0$ corresponds to the integer-order limit of the FD–LSTM(GL) formulation and is included for comparison within this unified fractional setting. The best-performing model used $\nu =0.95$, achieving a test accuracy of 91.85% and a goodness-of-fit $\Phi =0.9145$.

To quantify the contribution of the fractional mechanism, we compare FD–LSTM (GL) against standard LSTM, Bi-LSTM, and a Caputo-based fractional LSTM.

Table 5. Baseline comparison under an identical training protocol.

Model	Mechanism	Accuracy	F1	$\mathbf{\Phi }$
Classical LSTM	tanh/sigmoid	0.914	0.905	0.959
Bi–LSTM	bidirectional	0.921	0.912	0.960
Fractional LSTM (Caputo)	Caputo gradient	0.910	0.908	0.957
FD–LSTM (GL)	GL fractional operator	0.9185	0.915	0.9599

shows that FD–LSTM (GL) provides the most stable generalization (Accuracy/F1), while preserving a competitive $\Phi$.

Compared to classical LSTM, FD–LSTM exhibits: ∼25% faster early loss decrease (first 50 epochs), lower variance of gradient norms (${\sigma }^2=0.003$ vs. $0.009$), and no vanishing/exploding gradients for $\nu \in [0.9, 1.1]$. A specific numerical consideration is the horizontal shift inherent in truncated GL series: static GL coefficients approximate derivatives over backward samples, which may induce a minor phase lag in learning dynamics. In practice, this is mitigated by (a) adequate coefficient length N, (b) smaller step h, or (c) bidirectional GL stencils. Empirically, adopting $N\ge 20$ suffices for stable training while keeping runtime moderate.

Although the Grünwald–Letnikov operator introduces a linear-time overhead during backpropagation due to the coefficient stencil, this additional cost does not translate into a substantial practical slowdown. As reported in

Table 6. Runtime overhead for different fractional orders $\nu$ (seconds per epoch).

Model	$\mathit{\nu }$	Coefficients N	s/Epoch	Overhead
Classical LSTM	1.0	–	1.00	–
FD–LSTM	0.90	20	1.11	+11%
FD–LSTM	0.95	20	1.15	+15%
FD–LSTM	1.05	25	1.21	+21%

, the per-epoch runtime increases by 11–21%; however, the fractional model reaches convergence significantly earlier than the classical LSTM (typically by epoch 80 compared to 120–130), which effectively compensates for the modest per-epoch increase. The GL operator also provides a smoothing effect on the optimisation landscape, reducing gradient-norm variance and stabilising the learning trajectory, which further decreases effective training time. Importantly, inference-time complexity remains unchanged, as the fractional stencil is applied only during training. Consequently, the FD–LSTM retains real-time feasibility for industrial monitoring applications while offering improved optimisation stability.

Our findings are consistent with recent fractional neural literature (e.g., feed-forward and biophysical networks [46,47,48]) and extend them to recurrent settings with consistent fractional forward propagation and BPTT. We hypothesize that GL’s discrete shift-invariance aligns well with uniformly sampled industrial time series, which helps explain its empirical edge over Caputo or Riemann-Liouville formulations.

In production, FD–LSTM improves the sensitivity to subtle drifts while preserving stability, making it suitable for early warning in fiber-optic lines. Future work includes meta-learning of $\nu$, bidirectional GL stencils to reduce phase lag, and integration with an ALMM-based decision layer [26] for end-to-end adaptive control.

3.4. Industrial Addendum: Compact RLM-Line Evaluation

To complement the method-centric experiments in this paper, we report a compact, non-overlapping evaluation on the FIBRAIN RLM fiber-tube production line (1 Hz sampling, window 256 s, stride 5 s, 232 sensors). This slice is distinct from the dataset used in our earlier application-focused work. We retain the same training regimen (optimizer, batch size, early stopping) but use the plant’s five-state taxonomy (one nominal + four anomaly states) without K-means relabeling, to mirror shop-floor interpretation. Models are trained once per $\nu$ with identical seeds.

On this real-line slice, FD–LSTM (GL) consistently matches or exceeds the classical LSTM, peaking at $\nu =0.80$ (

Table 7. RLM compact slice: held-out test accuracy across fractional orders $\nu$. Best value in bold.

Model	$\mathit{\nu }$	Test Accuracy
Classical LSTM	$1.00$	0.9333
FD–LSTM (GL)	$0.80$	0.9667
FD–LSTM (GL)	$0.85$	0.9630
FD–LSTM (GL)	$0.95$	0.9370
FD–LSTM (GL)	$1.00$	0.9296
FD–LSTM (GL)	$1.05$	0.9370
FD–LSTM (GL)	$1.15$	0.9630
FD–LSTM (GL)	$1.20$	0.9519

). In contrast, in our main study that uses a reduced 40-indicator feature set with weakly-supervised relabeling, the best result occurs at $\nu =0.95$ (Accuracy $0.9185$). This divergence is consistent with the role of $\nu$ as a tunable memory-depth control; its optimum shifts with the input representation and label protocol. The comparison with common baselines under the same RLM slice (

Table 8. RLM compact slice: summary metrics across models (single best configuration per model).

Model	Accuracy	Precision	Recall	F1
FD–LSTM (GL)	0.9667	0.96	0.96	0.93
Classical LSTM	0.9333	>0.94	>0.96	>0.91
Random Forest	∼0.85	∼0.85	∼0.83	∼0.84
SVM	∼0.88	∼0.88	∼0.85	∼0.86
RNN (vanilla)	∼0.87	∼0.87	∼0.86	∼0.86

) indicates FD–LSTM’s edge is not only with regard to its classical counterpart but also vs. standard learners:

The GL fractional mechanism improves real-line accuracy and stabilizes training on the RLM slice, while the optimal $\nu$ is data- and label-protocol-dependent. This complements the main results and supports the view of $\nu$ as a practical, continuous knob for adapting memory depth to industrial time-series regimes.

4. Conclusions

This study introduces a novel extension of recurrent neural networks that integrates fractional-order calculus into the learning dynamics of LSTM architectures. By embedding the Grünwald–Letnikov derivative directly into the neuron activation and gradient propagation mechanisms, the proposed FD–LSTM achieves a mathematically grounded and computationally tractable enhancement of classical backpropagation.

The research demonstrates that the fractional approach:

• Stabilizes gradient flow and eliminates the vanishing-gradient bottleneck without architectural modifications.
• Introduces a continuous control dimension through the fractional order $\nu$, enabling smooth adaptation of the neuron’s transfer function between linear and nonlinear regimes.
• Improves convergence and generalization, achieving up to 1.6× faster training and higher accuracy in industrial anomaly-detection tasks.
• Bridges discrete and continuous learning representations, establishing a unified framework for modeling memory depth and temporal smoothness in recurrent systems.

The empirical results confirm that fractional differentiation provides a new degree of freedom for deep learning-extending the search space of trainable parameters beyond weights and biases. The optimal configuration ($\nu =0.95$) ensures an effective balance between numerical stability and dynamic adaptability, outperforming both classical and Caputo-based LSTM variants.

From an application perspective, the FD–LSTM enhances sensitivity to subtle temporal deviations in fiber-optic manufacturing, supporting earlier and more reliable anomaly detection. The framework’s discrete-shift invariance and compatibility with existing GPU pipelines make it directly applicable in real-time industrial monitoring.

Although this work focuses on a single real-world dataset, the fiber-optic manufacturing domain provides a challenging benchmark characterised by heterogeneous sensor interactions, long-range temporal dependencies, and multiple anomaly subtypes obtained through weakly supervised relabelling. Nevertheless, extending the evaluation to additional anomaly detection scenarios represents an important direction for future research and will allow for a broader assessment of cross-domain robustness.

In future work, fractional-order parameters could be adaptively learned alongside network weights, leading to self-tuning memory dynamics. Further research may explore bidirectional GL operators, fractional gating in Transformer-like architectures, and the integration of the proposed model into symbolic decision layers for fully interpretable, hybrid intelligent systems.

Another promising research direction concerns decentralised and federated anomaly detection. Many industrial systems operate across multiple machines or production sites where data cannot be centrally aggregated due to bandwidth limitations, privacy requirements, or heterogeneous hardware constraints. Recent works on collaborative adaptation and balance-recovery strategies for federated fault diagnosis demonstrate the potential of distributed learning frameworks in such environments [49]. Because the proposed GL-based FD–LSTM does not rely on any mechanism that prevents distributed optimisation, integrating it into a federated learning framework represents a natural extension. Such an approach would enable consistent anomaly detection across heterogeneous machine groups while preserving data decentralisation and respecting local data-sovereignty constraints.

Finally, recent advancements such as MSTAD and Transformer-based anomaly detection frameworks highlight the growing role of attention mechanisms in modelling complex, long-range temporal dependencies. While our modelling focus in this study was deliberately restricted to recurrent architectures to enable a controlled analysis of fractional gating dynamics, a natural next step is to evaluate the GL-based fractional mechanism within Transformer encoder blocks and to benchmark FD–LSTM against state-of-the-art attention-driven models. This comparison will be particularly relevant for datasets with longer temporal horizons, where self-attention can fully exploit global temporal context.

In summary, the proposed FD–LSTM model transforms fractional calculus from a theoretical construct into a practical deep learning mechanism, bridging mathematical rigor, computational efficiency, and industrial impact.