Fractional Optimizers for LSTM Networks in Financial Time Series Forecasting

Abstract

This study investigates the theoretical foundations and practical advantages of fractional-order optimization in computational machine learning, with a particular focus on stock price forecasting using long short-term memory (LSTM) networks. We extend several widely used optimization algorithms—including Adam, RMSprop, SGD, Adadelta, FTRL, Adamax, and Adagrad—by incorporating fractional derivatives into their update rules. This novel approach leverages the memory-retentive properties of fractional calculus to improve convergence behavior and model efficiency. Our experimental analysis evaluates the performance of fractional-order optimizers on LSTM networks tasked with forecasting stock prices for major companies such as AAPL, MSFT, GOOGL, AMZN, META, NVDA, JPM, V, and UNH. Considering four metrics (Sharpe ratio, directional accuracy, cumulative return, and MSE), the results show that fractional orders can significantly enhance prediction accuracy for moderately volatile stocks, especially among lower-cap assets. However, for highly volatile stocks, performance tends to degrade with higher fractional orders, leading to erratic and inconsistent forecasts. In addition, fractional optimizers with short-memory truncation offer a favorable trade-off between computational efficiency and modeling accuracy in medium-frequency financial applications. Their enhanced capacity to capture long-range dependencies and robust performance in noisy environments further justify their adoption in such contexts. These results suggest that fractional-order optimization holds significant promise for improving financial forecasting models—provided that the fractional parameters are carefully tuned to balance memory effects with system stability.

1. Introduction

Financial time series prediction remains one of the most challenging tasks in the field of quantitative finance due to the inherent noise, volatility, and dynamic behavior of financial markets. According to the semi-strong form of market efficiency [1], asset prices reflect all publicly available information, theoretically limiting the predictability of future price movements. However, numerous empirical studies have identified persistent market anomalies that challenge this notion, suggesting that certain patterns can indeed be exploited [1]. Over the past decade, financial time series forecasting—whether framed as a regression or a classification problem—has attracted significant academic and industrial interest. Traditional statistical and machine learning models, while useful, often fall short in capturing the nonlinear and temporal dependencies intrinsic to financial data [2,3]. In contrast, deep learning methodologies—particularly long short-term memory (LSTM) networks—demonstrate superior performance in financial time-series forecasting, attributable to their capacity for modeling sequential dependencies and capturing long-term temporal patterns [4,5,6]. This capability is increasingly critical given escalating financial market complexity and heightened economic uncertainty, which amplify the demand for robust, adaptive forecasting systems [3]. Consequently, recurrent neural network (RNN) architectures, notably LSTMs, have garnered significant scholarly and practical interest in recent years, proving particularly effective for applications such as equity price prediction [6]. However, the effectiveness of these models is strongly influenced by the choice and performance of the optimization mechanisms employed during training. Despite notable advancements, comprehensive guidance remains scarce for financial practitioners regarding optimal model selection, architecture design, and implementation strategies, thus underscoring the necessity for continued research in this domain [2].

Classical optimizers, including Stochastic Gradient Descent (SGD) [7], Momentum [8], Nesterov Accelerated Gradient (NAG) [9], Adagrad [10], Adadelta [11], RMSprop [12], and Adam [13] are widely adopted because of their effectiveness and robustness in addressing challenging optimization problem landscapes. Newer developments, including AMSGrad [14], AdamW [15], AdaBelief [16], and Rectified Adam (RAdam) [17], have enhanced convergence stability and generalization performance, particularly for deep and recurrent architectures.

Despite this progress, all the aforementioned optimizers are founded on classical integer calculus and may inherently limit their capacity to grasp the complex temporal contingencies and long-term memory influences that characterize financial time-series data. Such restrictions prompt the investigation of alternative mathematical frameworks that naturally incorporate memory and hereditary characteristics within the training dynamics.

Fractional calculus, which extends traditional differentiation and integration to non-integer (fractional) orders, provides a powerful approach to this challenge. By substituting fractional-order derivatives for conventional derivatives in the update rules of optimization algorithms, it becomes possible to construct fractional-order optimizers that better capture long-term dependencies and dynamic patterns in the data.

This paper investigates the theoretical foundations and empirical performance of fractional-order optimization algorithms for tuning LSTM models in stock price forecasting tasks. We introduce fractional variants of commonly used optimizers and measure their influence on forecast accuracy across a range of company stock price datasets, including AAPL, MSFT, GOOGL, AMZN, META, NVDA, JPM, V, and UNH. Our findings highlight that fractional-order optimization, when appropriately tailored, can significantly enhance model efficiency, especially in relatively stable financial markets, while also revealing its limitations in highly unstable market environments.

The principal contributions of this work can be summarized as follows:

• We introduce a novel class of fractional-order training optimizers that extend conventional optimization algorithms (Adam, RMSprop, SGD, Adadelta, FTRL, Adamax, and Adagrad) by integrating fractional computation into their update rules, allowing memory and hereditary effects to be embedded in the training process.
• We incorporate these fractional-order optimizers into the training phase of long short-term memory (LSTM) networks, establishing a new learning paradigm that enhances the model’s capacity to capture the long-term dependencies embedded in financial sequential data.
• We apply the proposed fractional optimization schemes to time-series forecasting tasks in the stock market domain, using leading technology stock prices (e.g., AAPL, MSFT, GOOGL, AMZN, META, NVDA, JPM, V, and UNH) as benchmark datasets.
• We conduct in-depth empirical comparisons between traditional integer-order optimizers (such as Adam, RMSprop, Adagrad, SGD, Nadam, Adadelta, and AMSGrad) and their fractional counterparts, highlighting the potential benefits and challenges associated with fractional-order processing in financial forecasting applications.

The rest of the paper is structured as follows. In Section 2, we provide a succinct background on fractional calculus. Section 3 introduces the development of fractional-order optimizers and their integration into the LSTM training framework. Section 4 outlines the experimental setup, including a detailed description of the dataset, implementation details, evaluation metrics, and tuning strategies. Finally, Section 5 concludes the paper and discusses promising directions for future work.

2. Fractional Calculus

Fractional calculus has recently attracted significant attention in the machine learning (ML) community, particularly for its ability to model memory effects and non-local behavior. These properties make it a powerful tool for enhancing the dynamics of optimization algorithms, especially in sequence modeling tasks such as stock price forecasting using LSTM networks. Unlike traditional integer-order methods, fractional-order algorithms can encode historical gradient information more effectively, offering improved convergence behavior and potentially better generalization in noisy or volatile environments.

Motivated by these theoretical advantages, this study proposes fractional variants of several widely used optimizers—including Adam, RMSprop, SGD, Adadelta, FTRL, Adamax, and Adagrad—by integrating fractional derivatives into their update mechanisms. Our approach is grounded in a growing body of recent work that demonstrates the efficacy of fractional operators in neural network training and optimization. Notable contributions in this field include both theoretical formulations and empirical validations of fractional-order gradient descent strategies and their convergence behavior [18,19,20,21,22,23].

Fractional calculus is a modern extension of standard calculus to non-integer orders of derivatives and integrals. For a function $f\left(t\right)$, the fractional derivative of order $\alpha$, referred to as D, is expressed as shown below [24]:

$$D^{\alpha }f\left(t\right)={\displaystyle \frac{d^{\alpha }f\left(t\right)}{d{t}^{\alpha }}}$$

(1)

where $\alpha \in {\mathbb{R}}^{+}$. Fractional derivatives have emerged as beneficial in numerous domains, covering signal treatment, monitoring theory, viscoelastic systems, and, progressively, machine learning and optimization, owing to their capability of encompassing the memory and inheritance characteristics of systems.

2.1. Definitions of Fractional Derivatives

There are different kinds of fractional derivatives, tailored to meet the needs of individual applications. The most frequently used are the following:

• Riemann–Liouville derivative [24]:

  $$D_{RL}^{\alpha }f\left(t\right)={\displaystyle \frac{1}{\Gamma (n-\alpha )}}{\displaystyle \frac{d^n}{d{t}^n}}{\int }_0^t{\displaystyle \frac{f\left(\tau \right)}{{(t-\tau )}^{\alpha -n+1}}}d\tau ,n-1<\alpha <n$$

  (2)
• Caputo derivative: [25]

  $$D_C^{\alpha }f\left(t\right)={\displaystyle \frac{1}{\Gamma (n-\alpha )}}{\int }_0^t{\displaystyle \frac{f^{\left(n\right)}\left(\tau \right)}{{(t-\tau )}^{\alpha -n+1}}}d\tau ,n-1<\alpha <n$$

  (3)
  Caputo’s version tends to be favored in physics and machine learning applications, since it enables starting assumptions to be handled in the equivalent form to integer-order differential systems.
• Grünwald–Letnikov derivative: [26]

  $$D_{GL}^{\alpha }f\left(t\right)=\underset{h\to 0}{lim}{\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^{⌊t/h⌋}{(-1)}^k\left({\displaystyle \genfrac{}{}{0pt}{}{\alpha }{k}}\right)f(t-kh)$$

  (4)
  This formulation is especially fruitful for numerical applications and represents the foundations of numerous discrete-time approximations of fractional derivatives.

2.2. Approximation Methods of Fractional Derivatives

Because fractional derivatives are defined as integrals over time or history, computing them exactly is often impractical, especially in online optimization. Thus, approximation methods play a vital role in applications.

• Short-memory principle: Computing the fractional derivative analytically is challenging, so we employ the short-memory principle, using only the most recent M steps to approximate it. The estimated fractional derivative is given by

  $$D^{\alpha }f\left(t_n\right)\approx {\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}f\left(t_{n-k}\right),$$

  (5)

  where ${\omega }_k^{\left(\alpha \right)}={(-1)}^k\left({\displaystyle \genfrac{}{}{0pt}{}{\alpha }{k}}\right)$ represents the binomial factor related to the fractional derivative, h denotes the time frame, and M denotes the number of current time frames considered.
• Lubich’s method: Founded on convolution squaring involving generative functions, this approach provides steady and rigorous approximations of fractional derivatives for hard differential equations and optimization procedures [27].
• Fractional backward differentiation formulas (FBDFs): These approaches expand conventional BDF procedures to fractional orders and are ideally matched for use in inflexible or memory-intensive systems.
• Adam–Bashforth–Moulton predictors: Fractional variants of ABM techniques offer successful estimator-corrector frameworks [28].
• Adaptive and data-driven approaches: Current research recommends approaching fractional operators by means of neural networks or kernel-based regressors that adjust to the function’s response with time [29].

2.3. Theoretical Rationale Behind Fractional Order Memory and Market Dynamics

To grasp why fractional LSTM models work better with small memory orders, it is important to understand the math behind fractional derivatives. In this subsection, we offer an intuitive and semi-formal interpretation of how the Caputo derivative introduces long-term memory and temporal smoothing, which aligns with empirical behaviors observed in financial markets such as volatility clustering and trend persistence. While a full theoretical treatment is beyond the scope of this empirical study, we summarize the key mechanisms that link fractional dynamics to financial signal structure.

2.3.1. Fractional Derivatives and Memory Effects

The use of Caputo fractional derivatives introduces non-local memory into the system via a convolution with a power-law kernel; see Equation (3).

This expression reveals that the fractional order $\alpha$ acts as a memory parameter, where lower $\alpha$ values increase the weight assigned to older values of $f^{\prime }\left(t\right)$. That is, the system retains long-term memory with a power-law decay governed by

$$w(t-\tau )\sim {(t-\tau )}^{-\alpha }$$

(6)

as compared to the exponential decay in traditional LSTM gates.

2.3.2. Temporal Filtering and Volatility Sensitivity

The long-memory behavior of the fractional derivative effectively acts as a temporal low-pass filter [30]. In the context of financial signals,

• When $\alpha \to 0$, the model heavily smooths the input, making it more robust to noise and transient fluctuations—ideal for steady stocks like AAPL.
• When $\alpha \to 1$, the filter approximates a first-order derivative, recovering a reactive model that is sensitive to short-term fluctuations—better suited for high-volatility assets like META.

This filtering effect can be interpreted using the frequency-domain response of the fractional derivative, where the Laplace transform of the Caputo derivative is

$$\mathcal{L}\left\{{}^{C}D_t^{\alpha }f\left(t\right)\right\}=s^{\alpha }\tilde{f}\left(s\right)-\sum _{k=0}^{n-1}{s}^{\alpha -k-1}{f}^{\left(k\right)}\left(0^{+}\right)$$

(7)

This reveals that the response attenuates higher-frequency components more aggressively as $\alpha$ decreases, thus suppressing noise.

2.3.3. Relation to Financial Market Structure

Financial time series are well known to exhibit volatility clustering and heteroskedasticity, as modeled in ARCH/GARCH models. The heterogeneous memory hypothesis [31] and empirical studies [32] suggest that smoother markets (with lower realized volatility) exhibit more persistent autocorrelations. In such cases, a fractional model with small $\alpha$ provides a more appropriate inductive bias.

This relationship has also been exploited in fractional Brownian motion (fBm) models, where the Hurst exponent H quantifies long-memory behavior:

$$\mathbb{E}\left[{(B_H(t+\tau )-B_H\left(t\right))}^2\right]\sim {\tau }^{2H}$$

(8)

Lower $\alpha$ values in our model correspond functionally to higher H values (i.e., smoother paths), and thus align well with trend-following markets.

Therefore, the enhanced performance of small $\alpha$ values for smooth equities is not coincidental but rooted in

• The power-law memory integration of the Caputo derivative.
• The temporal smoothing inherent in fractional orders.
• The market-dependent spectral properties of financial time series.

With these techniques, fractional-order strategies can be conveniently employed in optimization, especially for time-series training and recurrent neural networks such as LSTM, in which the detection of long-term dependencies is essential.

3. Fractional Optimizers to LSTM Learning

In this section, we introduce fractional-order variants of widely used optimization algorithms in machine learning. These fractional optimizers incorporate additional hyperparameters that influence training dynamics, memory effects, and convergence stability—factors that are particularly critical when adapting LSTM models to volatile and non-stationary financial time series. The overall methodology applied in this section is summarized in Figure 1.

3.1. Loss Function in LSTM for Time Series Forecasting

Recently, deep learning tools such as LSTM networks have attracted increasing attention in the field of time series forecasting. These models require powerful optimization methods for effective training, particularly when dealing with complex datasets such as financial time series. Standard optimization techniques, including Stochastic Gradient Descent (SGD) and its variants, often struggle to capture long-range dependencies in the data. To address this limitation, we explore the potential advantages of incorporating fractional calculus into optimization procedures, specifically through the use of fractional derivatives, to better model such dependencies in a meaningful way.

In time series prediction, the loss function measures the difference between the model’s predicted values and the actual observations. It serves as the objective that the optimizer minimizes during training by adjusting the model’s weights to improve forecasting accuracy.

The most widely used loss function in time series forecasting is the Mean Squared Error (MSE), defined as

$${\mathcal{L}}_{\mathrm{MSE}}={\displaystyle \frac{1}{T}}\sum _{t=1}^T{\left(y_t-{\widehat{y}}_t\right)}^2$$

(9)

where

• $y_t$ is the actual value at time step t;
• ${\widehat{y}}_t$ is the predicted value at time step t;
• T is the total number of time steps in the sequence.

The latter penalizes higher magnitudes of error to a greater extent, which is suitable for financial time-series data, in which extreme volatility and aberrant values have significant consequences. However, where robustness to aberrant values is demanded, the Mean Absolute Error (MAE) or the Huber loss can be favored:

$${\mathcal{L}}_{\mathrm{MAE}}={\displaystyle \frac{1}{T}}\sum _{t=1}^T|y_t-{\widehat{y}}_t|$$

(10)

The selection of the loss function is determined by the features of the data and the particular objectives of the prediction job, for example, monitoring a tendency or forecasting a particular position.

3.2. LSTM Architecture and Mathematical Formulation

LSTM networks are specifically designed to capture long-term dependencies in sequential data, overcoming the limitations of standard recurrent neural networks (RNNs) through the use of powerful gating mechanisms that control the flow of information within and between cells. The cell state $C_t$ serves as the memory component, retaining long-term information, and is updated at each time step according to the following equations; see Figure 2.

1. Forget Gate: The miss port specifies what data from the preceding time step should be deleted from the cell’s status. It provides a value ranging from 0 to 1, with 1 meaning “keep all” and 0 meaning “forget all”.

$$f_t=\sigma (W_f·[h_{t-1},x_t]+b_f)$$

(11)

2. Input Gate: The entry port manages the quantity of new information authorized in the status of the cell. It likewise generates a refresh quantity that changes the status of the cell.

$$i_t=\sigma (W_i·[h_{t-1},x_t]+b_i)$$

(12)

$${\tilde{C}}_t=tanh(W_C·[h_{t-1},x_t]+b_C)$$

(13)

3. Cell State Update: The status of the cell $C_t$ is actualized by merging the decision of the forget port to retain the preceding storage and the response of the input gate to upgrade the memory by adding new knowledge.

$$C_t=f_t·C_{t-1}+i_t·{\tilde{C}}_t$$

(14)

4. Output Gate: The exit port identifies the subsequent hidden status $h_t$, utilized in the succeeding time interval and also serves as the output of the LSTM.

$$o_t=\sigma (W_o·[h_{t-1},x_t]+b_o)$$

(15)

$$h_t=o_t·tanh\left(C_t\right)$$

(16)

These portals operate jointly to allow LSTM to conveniently omit, refresh, and send information, ensuring its ability to pick up long-term relationships in time-series data; see Figure 3.

3.3. State of the Art: Optimization Methods for LSTM Learning

Training long short-term memory (LSTM) networks efficiently remains a vital research challenge in the field of deep learning. Optimization algorithms play a critical role in ensuring convergence speed, robustness, and generalization. The choice of optimizer significantly affects the performance, particularly for time-series data where sequential dependencies are central.

Classical Methods: Stochastic Gradient Descent (SGD) has historically been the foundational optimizer in deep learning [33] but suffers from slow convergence and sensitivity to learning rates. Momentum-based methods partially address these issues by smoothing gradients [8].

Adaptive First-order Methods: The introduction of adaptive learning methods such as AdaGrad [10], RMSProp [12], and Adam [13] significantly advanced LSTM training. Adam combines the benefits of RMSProp and momentum, becoming a standard in sequence modeling due to its stability and speed.

Second-order Methods: More computationally intensive approaches such as Hessian-Free Optimization [34] and Natural Gradient Descent [35] have been explored for LSTM training, especially in settings where curvature information improves convergence.

Fractional-Order Optimization: Recently, fractional-order derivatives have been applied to gradient descent, giving rise to fractional optimizers such as FracSGD, FracAdam, and FracRMSProp [36,37]. These optimizers introduce memory and long-range dependency characteristics beneficial for sequence learning, and have shown empirical improvements in noisy and sparse environments.

Hybrid and Custom Strategies: Other methods like Lookahead [38], RAdam [17], and Yogi [39] aim to refine convergence trajectories and deal with unstable variance during training. Such techniques are often combined with LSTM architectures in financial forecasting, language modeling, and anomaly detection.

As LSTM models continue to be applied in more complex and data-intensive settings, research increasingly focuses on optimizer robustness, generalization under limited data, and convergence guarantees under non-convex loss landscapes.

3.4. Fractional Adam

Adam’s optimizer represents an optimization algorithm largely adopted in deep learning that fuses momentum and adaptive training factors. It performs two estimations of momentum—the primary momentum ($m_t$) and the secondary momentum ($v_t$)—and adjusts the training rates for every parameter according to these estimations. Adam’s classic adjustment rules consist of the following:

$$m_t={\beta }_1{m}_{t-1}+(1-{\beta }_1){\nabla }_{\theta }J\left(\theta \right)$$

(17)

$$v_t={\beta }_2{v}_{t-1}+(1-{\beta }_2){\left({\nabla }_{\theta }J\left(\theta \right)\right)}^2$$

(18)

where ${\nabla }_{\theta }J\left(\theta \right)$ represents the gradient of the loss function against the machine configuration; ${\beta }_1$ and ${\beta }_2$ denote the exponential decline factors at the first and second juncture, correspondingly.

Next, the resulting approximations are adjusted to incorporate the zero entry:

$${\widehat{m}}_t={\displaystyle \frac{m_t}{1-{\beta }_1^t}},{\widehat{v}}_t={\displaystyle \frac{v_t}{1-{\beta }_2^t}}$$

(19)

Lastly, the configuration is adjusted according to the following formula:

$$\theta =\theta -\eta {\displaystyle \frac{{\widehat{m}}_t}{\sqrt{{\widehat{v}}_t}+ϵ}}$$

(20)

where $\eta$ represents the training rate and $ϵ$ is a tiny fixed value to exclude the possibility of division by 0.

3.4.1. Fractional Adam with General Fractional Gradient

With the fractional version of Adam, rather than employing the traditional gradient ${\nabla }_{\theta }J\left(\theta \right)$, we call for a fractional gradient. With this operator approach, we are able to include long-term linkages in the optimization workflow, especially relevant for time-series foresight jobs. The usual adjustment rules for fractional Adam include the following:

$$m_t^{\left(\alpha \right)}={\beta }_1{m}_{t-1}^{\left(\alpha \right)}+(1-{\beta }_1)D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$$

(21)

$$v_t^{\left(\alpha \right)}={\beta }_2{v}_{t-1}^{\left(\alpha \right)}+(1-{\beta }_2){\left(D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)\right)}^2$$

(22)

Here, $D^{\alpha }$ denotes the fractional derivative of order $\alpha$, and records long-term storage and reliance in gradient upgrades.

3.4.2. Short-Memory Estimation of the Fractional Derivative

Based on the approximation given by Equation (5), the adjustment guidelines for the fractional Adam becomes

$$m_t^{\left(\alpha \right)}={\beta }_1{m}_{t-1}^{\left(\alpha \right)}+(1-{\beta }_1){\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J{\left(\theta \right)}_{t-k}$$

(23)

$$v_t^{\left(\alpha \right)}={\beta }_2{v}_{t-1}^{\left(\alpha \right)}+(1-{\beta }_2){\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J{\left(\theta \right)}_{t-k}\right)}^2$$

(24)

The upgrades have integrated short-memory estimation, boosting the performance of the Adam fractional calculator without sacrificing the advantages of fractional computing.

3.4.3. Training Regulation in the Framework of LSTMs

Within the context of LSTM nets, the parameters $\theta$ comprise the weights and skews regulating the hidden status and cell-state conversions in LSTM units. The goal of LSTM is to minimize the error function, quantizing the deviation from expected values of the time series. The fractional Adam optimizer, discussed earlier, serves to adjust the LSTM configuration while training.

In this case, the training guideline is given below:

$$\theta =\theta -\eta {\displaystyle \frac{{\widehat{m}}_t^{\left(\alpha \right)}}{\sqrt{{\widehat{v}}_t^{\left(\alpha \right)}}+ϵ}}$$

(25)

where ${\widehat{m}}_t^{\left(\alpha \right)}$ and ${\widehat{v}}_t^{\left(\alpha \right)}$ are the bias-adjusted primary and secondary estimates by means of fractional gradients. With these upgrades, the LSTM can more effectively incorporate long-term linkages in the time series data, rendering the machine better suited to predictive purposes like stock price forecasting.

Thanks to Adam’s fractional model, LSTM is able to effectively handle the time dependencies intrinsic to market data, thereby boosting forecast efficiency, notably in turbulent market circumstances.

3.5. Fractional Nadam

Nadam (Nesterov-accelerated Adam) combines Adam with Nesterov’s momentum. The principal distinction compared with Adam lies in the fact that Nadam expects the parameters to be adjusted by incorporating the momentum, allowing for quicker convergence. Nadam’s basic upgrade guidelines include the following:

$$m_t={\beta }_1{m}_{t-1}+(1-{\beta }_1){\nabla }_{\theta }J\left(\theta \right)$$

(26)

$$v_t={\beta }_2{v}_{t-1}+(1-{\beta }_2){\left({\nabla }_{\theta }J\left(\theta \right)\right)}^2$$

(27)

where ${\nabla }_{\theta }J\left(\theta \right)$ represents the gradient of the loss function versus the parameters $\theta$ and ${\beta }_1$ and ${\beta }_2$ represent the decreasing ratios of the two time estimates.

For Nadam, the upgrades feature

$${\tilde{m}}_t={\beta }_1{m}_{t-1}+(1-{\beta }_1)D^{\alpha }J\left(\theta \right),$$

(28)

where the $D^{\alpha }J\left(\theta \right)$ fractional gradient represents the loss function.

Skew compensation is calculated as follows:

$${\widehat{m}}_t={\displaystyle \frac{m_t}{1-{\beta }_1^t}},{\widehat{v}}_t={\displaystyle \frac{v_t}{1-{\beta }_2^t}}$$

(29)

The recommended rule for setting the configuration is described below:

$$\theta =\theta -\eta {\displaystyle \frac{{\beta }_1{\widehat{m}}_t+(1-{\beta }_1)D^{\alpha }J\left(\theta \right)}{\sqrt{{\widehat{v}}_t}+ϵ}}$$

(30)

where $\eta$ represents the training intensity and  $ϵ$ is a tiny value to avoid dividing by 0.

3.5.1. Fractional Nadam Using a Generalized Fractional Gradient

The generic upgrade rule at Fractional Nadam substitutes the fractional gradient in place of the conventional gradient $D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$:

$$m_t^{\left(\alpha \right)}={\beta }_1{m}_{t-1}^{\left(\alpha \right)}+(1-{\beta }_1)D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$$

(31)

$$v_t^{\left(\alpha \right)}={\beta }_2{v}_{t-1}^{\left(\alpha \right)}+(1-{\beta }_2){\left(D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)\right)}^2$$

(32)

where ${\beta }_1$ and ${\beta }_2$ represent exponential decay intensities and $D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$ represents the fractional gradient.

Skew adjustments are made using the same method as with conventional Nadam:

$${\widehat{m}}_t^{\left(\alpha \right)}={\displaystyle \frac{m_t^{\left(\alpha \right)}}{1-{\beta }_1^t}},{\widehat{v}}_t^{\left(\alpha \right)}={\displaystyle \frac{v_t^{\left(\alpha \right)}}{1-{\beta }_2^t}}$$

(33)

The latest configuration settings are processed according to the following rule:

$$\theta =\theta -\eta {\displaystyle \frac{{\beta }_1{\widehat{m}}_t^{\left(\alpha \right)}+(1-{\beta }_1)D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)}{\sqrt{{\widehat{v}}_t^{\left(\alpha \right)}}+ϵ}}$$

(34)

3.5.2. Short-Memory Approximation of Fractional Gradient

Based on the approximation, given by Equation (5), Nadam’s fractional adjustment takes the form of:

$$m_t^{\left(\alpha \right)}={\beta }_1{m}_{t-1}^{\left(\alpha \right)}+(1-{\beta }_1){\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J{\left(\theta \right)}_{t-k}$$

(35)

$$v_t^{\left(\alpha \right)}={\beta }_2{v}_{t-1}^{\left(\alpha \right)}+(1-{\beta }_2){\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J{\left(\theta \right)}_{t-k}\right)}^2$$

(36)

Therefore, the rule to adjust the fractional Nadam parameters using the short-memory estimation is as follows:

$$\theta =\theta -\eta {\displaystyle \frac{{\beta }_1{\widehat{m}}_t^{\left(\alpha \right)}+(1-{\beta }_1)\frac{1}{h^{\alpha }}{\sum }_{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J{\left(\theta \right)}_{t-k}}{\sqrt{{\widehat{v}}_t^{\left(\alpha \right)}}+ϵ}}$$

(37)

3.5.3. LSTM Training Policy

With LSTM nets, LSTM parameters $\theta$ are adjusted in training to reduce the loss function. The fractional Nadam tuner is used to tune LSTM parameters while learning, as shown below:

$$\theta =\theta -\eta {\displaystyle \frac{{\beta }_1{\widehat{m}}_t^{\left(\alpha \right)}+(1-{\beta }_1)\frac{1}{h^{\alpha }}{\sum }_{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J{\left(\theta \right)}_{t-k}}{\sqrt{{\widehat{v}}_t^{\left(\alpha \right)}}+ϵ}}$$

(38)

This adjustment rule integrates the fractional gradient, allowing LSTM to grasp long-term relationships effectively. This is especially beneficial in studies like time series prediction, where precise forecasting requires comprehending complex temporal dependencies.

The incorporation of fractional derivatives improves the optimizer’s capacity to manage complex time-varying dynamics, rendering it especially appropriate for predictive purposes, including stock market forecasting [40].

3.6. Fractional SGD (Frac-SGD)

Stochastic Gradient Descent (SGD) represents the basic optimizer and provides the foundation for a large number of others. The traditional upgrade rule for SGD is the following:

$$\theta =\theta -\eta {\nabla }_{\theta }J\left(\theta \right)$$

(39)

where $\theta$ values are the parameters (weights, biases), $\eta$ is the learning rate, and ${\nabla }_{\theta }J\left(\theta \right)$ is the standard gradient of the loss function $J\left(\theta \right)$.

3.6.1. General Fractional SGD (Frac-SGD)

Within Fractional SGD, the classical gradient ${\nabla }_{\theta }J\left(\theta \right)$ is substituted with the fractional derivative $D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$, incorporating a memory impact:

$$\theta =\theta -\eta D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$$

(40)

where $D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$ represents the fractional gradient of level $\alpha$, $0<\alpha \le 1$, and $\alpha =1$ retrieves classic SGDs.

The fractional order $\alpha$, which handles the impact of preview gradients, enables the optimizer to introduce past knowledge, which is particularly relevant for teaching long-term linkages in time-series data.

3.6.2. Short-Memory Approximation of Fractional Gradient

Based on the approximation, given by Equation (5), the Frac-SGD is given by:

$$\theta =\theta -\eta \left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)$$

(41)

This estimated fractional gradient aggregates historical contributions from the latest gradient history, offering a greater understanding of chronological dynamics instead of having to compute a complete fractional derivative across the full story.

3.6.3. Learning Rule in the Context of LSTM

During LSTM network training, Frac-SGD adjusts the LSTM parameters (weights and bias) according to the fractional gradient of the waste relative to those parameters:

$$\theta =\theta -\eta \left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)$$

(42)

In the case of this example, $\theta$ stands for all LSTM parameters, and $J\left(\theta \right)$ is commonly the cross-entropy loss or root mean square error (RMSE) depending on the task.

Fractional order $\alpha$ incorporates long-term storage into weight updates, which is particularly useful for learning LSTM models on financial time series or any other sequence data involving long-term dependencies [41].

3.7. Fractional RMSprop (Frac-RMSprop)

RMSprop (Root Mean Square Propagation) represents an evolutionarily adaptive training technique that standardizes gradient adjustments by means of a shifting means of quadratic gradients. The standardized RMSprop adjustment works as follows:

$$E{\left[g^2\right]}_t=\beta E{\left[g^2\right]}_{t-1}+(1-\beta ){\left({\nabla }_{\theta }J\left(\theta \right)\right)}^2$$

(43)

$$\theta =\theta -{\displaystyle \frac{\eta }{\sqrt{E{\left[g^2\right]}_t+ϵ}}}{\nabla }_{\theta }J\left(\theta \right)$$

(44)

where $\theta$ values represent the machine’s configuration parameters, $\eta$ stands for the learning rate, $\beta$ represents the decline rate of the shifting mean, and $ϵ$ represents a tiny fixed value to guarantee numerical consistency.

3.7.1. General Fractional RMSprop (Frac-RMSprop)

In Fractional RMSprop, the classical gradient ${\nabla }_{\theta }J\left(\theta \right)$ is substituted with the fractional gradient $D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$. Upgrades take the form of

$$E{\left[D^{\alpha }{g}^2\right]}_t=\beta E{\left[D^{\alpha }{g}^2\right]}_{t-1}+(1-\beta ){\left(D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)\right)}^2$$

(45)

$$\theta =\theta -{\displaystyle \frac{\eta }{\sqrt{E{\left[D^{\alpha }{g}^2\right]}_t+ϵ}}}{D}^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$$

(46)

where $D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$ represents the fractional derivative of order $\alpha$.

The fractional derivative incorporates a memory of previous experience within the optimization procedure, providing greater smoothness and reactivity to long-term models, which is especially beneficial in non-stationary or turbulent scenarios.

3.7.2. Short-Memory Approximation of the Fractional Gradient

Based on the approximation, given by Equation (5), in Frac-RMSprop, the fractional gradient term $D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$ and its squared value are approximated using recent gradient history:

$$D^{\alpha }{\nabla }_{\theta }J\left({\theta }_t\right)\approx {\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)$$

(47)

This practical formulation enables efficient training while still leveraging fractional dynamics.

3.7.3. Learning Rule in the Context of LSTM

During LSTM model training, the Frac-RMSprop algorithm upgrades the network’s parameters by integrating the fractional past gradient history:

$$E{\left[D^{\alpha }{g}^2\right]}_t=\beta E{\left[D^{\alpha }{g}^2\right]}_{t-1}+(1-\beta ){\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)}^2$$

(48)

$$\theta =\theta -{\displaystyle \frac{\eta }{\sqrt{E{\left[D^{\alpha }{g}^2\right]}_t+ϵ}}}\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)$$

(49)

In this case, $\theta$ stands for all LSTM parameters, and $J\left(\theta \right)$ is typically the loss function employed for workflow forecasting purposes. This fractional method enhances LSTM’s capacity to adjust to complex, time-dependent sequences in the data [29].

3.8. Fractional Adagrad (Frac-Adagrad)

Adagrad (Adaptive Gradient Algorithm) tunes the training rate of every parameter according to the previous gradient magnitude. The usual Adagrad upgrades consist of the following:

$$G_t=G_{t-1}+{\left({\nabla }_{\theta }J\left(\theta \right)\right)}^2$$

(50)

$$\theta =\theta -{\displaystyle \frac{\eta }{\sqrt{G_t+ϵ}}}{\nabla }_{\theta }J\left(\theta \right)$$

(51)

where $\theta$ values are the model parameters, $\eta$ is the initial learning rate, $G_t$ is the cumulative sum of squared gradients, and $ϵ$ is a small constant to avoid division by zero.

With increasing $G_t$, the actual training rate diminishes, rendering Adagrad ideally matched to sparse data challenges.

3.8.1. General Fractional Adagrad (Frac-Adagrad)

With Fractional Adagrad, the conventional gradient is substituted with the fractional gradient $D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$, introducing a storage aspect to the adjustment of training rates:

$$G_t^{\left(\alpha \right)}=G_{t-1}^{\left(\alpha \right)}+{\left(D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)\right)}^2$$

(52)

$$\theta =\theta -{\displaystyle \frac{\eta }{\sqrt{G_t^{\left(\alpha \right)}+ϵ}}}{D}^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$$

(53)

where $D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$ represents the fractional derivative of the loss against the parameter.

This fractional accumulation allows us to better grasp the long-term evolution of the gradient, providing an effective means of addressing non-stationary or time-varying dependencies in the data.

3.8.2. Short-Memory Approximation of the Fractional Gradient

Calculating the entire fractional gradient across the whole history is expensive. Utilizing the short memory assumption, the fractional gradient can simply be estimated from the latter M steps (using Equation (5)):

$$D^{\alpha }{\nabla }_{\theta }J\left({\theta }_t\right)\approx {\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)$$

(54)

Thus, the updates become

$$G_t^{\left(\alpha \right)}=G_{t-1}^{\left(\alpha \right)}+{\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)}^2$$

(55)

$$\theta =\theta -{\displaystyle \frac{\eta }{\sqrt{G_t^{\left(\alpha \right)}+ϵ}}}\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)$$

(56)

Such an estimation maintains the memory aspect but is still computable.

3.8.3. Learning Rule in the Context of LSTM

During LSTM nets training, the Frac-Adagrad scheduler changes parameter settings by incorporating fresh gradient records:

$$G_t^{\left(\alpha \right)}=G_{t-1}^{\left(\alpha \right)}+{\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)}^2$$

(57)

$$\theta =\theta -{\displaystyle \frac{\eta }{\sqrt{G_t^{\left(\alpha \right)}+ϵ}}}\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)$$

(58)

This fractional variant allows LSTM designs to tailor training rates accordingly across time, considering both newer and earlier gradient knowledge, thereby enhancing learning dynamics for time-series and sequential processes [40].

3.9. Fractional Adadelta (Frac-Adadelta)

Adadelta is an enhancement to Adagrad that restricts the collection of squared gradients within a fixed period, thereby restraining the training rate against indefinite decay. The default Adadelta upgrades include the following:

$$E{\left[g^2\right]}_t=\beta E{\left[g^2\right]}_{t-1}+(1-\beta ){\left({\nabla }_{\theta }J\left(\theta \right)\right)}^2$$

(59)

$$\Delta {\theta }_t=-{\displaystyle \frac{\sqrt{E{[\Delta {\theta }^2]}_{t-1}+ϵ}}{\sqrt{E{\left[g^2\right]}_t+ϵ}}}{\nabla }_{\theta }J\left(\theta \right)$$

(60)

$$\theta =\theta +\Delta {\theta }_t$$

(61)

where $E{\left[g^2\right]}_t$ is the decaying average of past squared gradients, $E{[\Delta {\theta }^2]}_t$ is the decaying average of past squared updates, $\beta$ is the decay rate, and $ϵ$ is a small constant for numerical stability.

In contrast to Adagrad, Adadelta automatically adjusts training rates depending on the latest gradient feedback, rather than necessitating a hand-set initial training rate $\eta$.

3.9.1. General Fractional Adadelta (Frac-Adadelta)

With the fractional variant, the conventional gradient is substituted by the fractional derivative $D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$, improving storage and dynamics:

$$E{\left[D^{\alpha }{g}^2\right]}_t=\beta E{\left[D^{\alpha }{g}^2\right]}_{t-1}+(1-\beta ){\left(D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)\right)}^2$$

(62)

$$\Delta {\theta }_t=-{\displaystyle \frac{\sqrt{E{[\Delta {\theta }^2]}_{t-1}+ϵ}}{\sqrt{E{\left[D^{\alpha }{g}^2\right]}_t+ϵ}}}{D}^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$$

(63)

$$\theta =\theta +\Delta {\theta }_t$$

(64)

where $D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$ represents the fractional derivative of the loss function relative to the parameters.

This extension incorporates a manageable memory impact through fractional order $\alpha$, catching deeper time dependencies.

3.9.2. Short-Memory Approximation of the Fractional Gradient

Utilizing the short memory idea, the fractional gradient $D^{\alpha }{\nabla }_{\theta }J\left({\theta }_t\right)$ can be approximated over the most recent M steps (using Equation (5)):

$$D^{\alpha }{\nabla }_{\theta }J\left({\theta }_t\right)\approx {\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)$$

(65)

Frac-Adadelta updates become

$$E{\left[D^{\alpha }{g}^2\right]}_t=\beta E{\left[D^{\alpha }{g}^2\right]}_{t-1}+(1-\beta ){\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)}^2$$

(66)

$$\Delta {\theta }_t=-{\displaystyle \frac{\sqrt{E{[\Delta {\theta }^2]}_{t-1}+ϵ}}{\sqrt{E{\left[D^{\alpha }{g}^2\right]}_t+ϵ}}}\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)$$

(67)

$$\theta =\theta +\Delta {\theta }_t$$

(68)

This approach preserves the benefits of Adadelta while efficiently incorporating fractional memory.

3.9.3. Learning Rule in the Context of LSTM

During LSTM network training, Frac-Adadelta adjusts training rates accordingly while integrating fractional memory:

$$E{\left[D^{\alpha }{g}^2\right]}_t=\beta E{\left[D^{\alpha }{g}^2\right]}_{t-1}+(1-\beta ){\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)}^2$$

(69)

$$\Delta {\theta }_t=-{\displaystyle \frac{\sqrt{E{[\Delta {\theta }^2]}_{t-1}+ϵ}}{\sqrt{E{\left[D^{\alpha }{g}^2\right]}_t+ϵ}}}\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)$$

(70)

$$\theta =\theta +\Delta {\theta }_t$$

(71)

This enables LSTM to adjust more smoothly to changing temporal dynamics over the course of training, especially during the treatment of long or noisy sequences [41].

3.10. Fractional FTRL (Frac-FTRL)

The Follow-The-Regularized-Leader (FTRL) approach is used extensively in e-learning and sparse optimization. It involves a combination of gradient summation and regularization to adjust the model configuration. The basic upgrades of the FTRL algorithm include the following:

$$g_t={\nabla }_{\theta }J\left(\theta \right)$$

(72)

$$z_t=z_{t-1}+g_t-{\displaystyle \frac{1}{\lambda }}\theta$$

(73)

$$\theta ={\mathrm{prox}}_{\lambda }\left(z_t\right)$$

(74)

where $g_t$ is the standard gradient at time t, $z_t$ accumulates the gradients with a shrinkage term, $\lambda$ is the regularization strength, and ${\mathrm{prox}}_{\lambda }$ denotes a proximal operator incorporating $L_1$ and/or $L_2$ regularization.

FTRL is especially efficient in high-dimensional applications, as it encourages dispersion and monitors the amplitude of weights.

3.10.1. General Fractional FTRL (Frac-FTRL)

With the fractional version, the classical gradient is substituted with a fractional derivative $D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$, enabling the algorithm to better grasp long-term relationships:

$$g_t^{\left(\alpha \right)}=D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$$

(75)

$$z_t^{\left(\alpha \right)}=z_{t-1}+g_t^{\left(\alpha \right)}-{\displaystyle \frac{1}{\lambda }}\theta$$

(76)

$$\theta ={\mathrm{prox}}_{\lambda }\left(z_t^{\left(\alpha \right)}\right)$$

(77)

where $g_t^{\left(\alpha \right)}$ represents the fractional derivative of the loss and $z_t^{\left(\alpha \right)}$ collects fractional gradients and adjustment correction.

Fractionation incorporates a deeper memory architecture within FTRL, which is helpful in nonstationary or volatile contexts.

3.10.2. Short-Memory Approximation of the Fractional Gradient

The fractional gradient $g_t^{\left(\alpha \right)}$ can be estimated with a limited window of fresh gradients:

$$g_t^{\left(\alpha \right)}\approx {\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)$$

(78)

Thus, the Frac-FTRL update becomes

$$z_t^{\left(\alpha \right)}=z_{t-1}+\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)-{\displaystyle \frac{1}{\lambda }}\theta$$

(79)

$$\theta ={\mathrm{prox}}_{\lambda }\left(z_t^{\left(\alpha \right)}\right)$$

(80)

With this approximation, Frac-FTRL can be practically deployed with no need for full access to the history.

3.10.3. Learning Rule in the Context of LSTM

With LSTM training, Frac-FTRL adjustments can dynamically equilibrate gradient information and regularization:

$$z_t^{\left(\alpha \right)}=z_{t-1}+\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)-{\displaystyle \frac{1}{\lambda }}\theta$$

(81)

$$\theta ={\mathrm{prox}}_{\lambda }\left(z_t^{\left(\alpha \right)}\right)$$

(82)

This allows LSTM models to preserve robustness and sparse representations while adapting to the nonstationary and highly dynamic time-series patterns typically found in financial time-series modeling and continuous data scenarios [41].

3.11. Fractional Adamax (Frac-Adamax)

Adamax is a further development of the Adam optimizer, which employs the infinite norm to compute parameter adjustments. The basic Adamax upgrade rule is the following:

$$m_t={\beta }_1{m}_{t-1}+(1-{\beta }_1){\nabla }_{\theta }J\left(\theta \right)$$

(83)

$$u_t=max({\beta }_2{u}_{t-1},|{\nabla }_{\theta }J\left(\theta \right)\left|\right)$$

(84)

$$\theta =\theta -{\displaystyle \frac{\eta }{u_t}}{m}_t$$

(85)

where $m_t$ denotes the momentum factor, $u_t$ denotes the running maximum of the gradient amplitude, $\eta$ denotes the training rate, and ${\beta }_1$ and ${\beta }_2$ manage the momentum and normalization behaviors.

3.11.1. General Fractional Adamax (Frac-Adamax)

With the fractional version, we substitute the classical gradient with the fractional gradient $D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$, introducing storage impacts, incorporating long-term dependencies into the model:

$$m_t^{\left(\alpha \right)}={\beta }_1{m}_{t-1}^{\left(\alpha \right)}+(1-{\beta }_1)D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$$

(86)

$$u_t^{\left(\alpha \right)}=max({\beta }_2{u}_{t-1}^{\left(\alpha \right)},|D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)\left|\right)$$

(87)

$$\theta =\theta -{\displaystyle \frac{\eta }{u_t^{\left(\alpha \right)}}}{m}_t^{\left(\alpha \right)}$$

(88)

Here, $m_t^{\left(\alpha \right)}$ represents the fractional momentum factor and $u_t^{\left(\alpha \right)}$ denotes the fractional running maximum of the gradient amplitudes.

This change enhances Adamax’s sensitivity and robustness in the face of frequent or erratic updates, especially in the case of non-stationary time series or noisy gradients.

3.11.2. Short-Memory Approximation of the Fractional Gradient

To compute the fractional derivatives, we estimate the fractional gradient by means of a bounded window of fresh gradients:

$$D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)\approx {\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)$$

(89)

Thus, the Frac-Adamax update rule becomes

$$m_t^{\left(\alpha \right)}={\beta }_1{m}_{t-1}^{\left(\alpha \right)}+(1-{\beta }_1)\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)$$

(90)

$$u_t^{\left(\alpha \right)}=max({\beta }_2{u}_{t-1}^{\left(\alpha \right)},\left|{\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right|)$$

(91)

$$\theta =\theta -{\displaystyle \frac{\eta }{u_t^{\left(\alpha \right)}}}{m}_t^{\left(\alpha \right)}$$

(92)

This estimation minimizes calculation complexity while maintaining the benefits of fractional derivatives.

3.11.3. Learning Rule in the Context of LSTM

With LSTM networks, Frac-Adamax provides a more robust and efficient optimization scheme, particularly in situations where training is sensitive to irregular time-series data. The update procedure for training LSTMs with Frac-Adamax is as follows:

$$m_t^{\left(\alpha \right)}={\beta }_1{m}_{t-1}^{\left(\alpha \right)}+(1-{\beta }_1)\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right)$$

(93)

$$u_t^{\left(\alpha \right)}=max({\beta }_2{u}_{t-1}^{\left(\alpha \right)},\left|{\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J\left({\theta }_{t-k}\right)\right|)$$

(94)

$$\theta =\theta -{\displaystyle \frac{\eta }{u_t^{\left(\alpha \right)}}}{m}_t^{\left(\alpha \right)}$$

(95)

This guarantees that LSTM networks are able to efficiently capture long-term dependencies while maintaining stability, even in the presence of strong gradients or significant parameter changes [40].

3.12. Fractional AdamW Lion

Fractional AdamW Lion combines the fractional derivative formalism with AdamW’s decoupled weight decay and Lion’s momentum-guided updates using the sign of gradients. This hybrid method leverages long-memory dynamics from fractional calculus while incorporating modern optimization strategies to enhance convergence and generalization, especially within sequential models like LSTMs.

3.12.1. Lion’s Momentum and Update Rule

The Lion (EvoLved Sign Momentum) optimizer is based on the following momentum and update equations:

$$m_t={\beta }_1{m}_{t-1}+(1-{\beta }_1){\nabla }_{\theta }J\left(\theta \right)$$

(96)

$$\theta =\theta -\eta ·\mathrm{sign}\left(m_t\right)$$

(97)

Lion uses a sign-based update direction rather than gradient magnitude. AdamW, meanwhile, applies decoupled weight decay as

$$\theta \leftarrow \theta -\eta ·\lambda ·\theta$$

(98)

where $\lambda$ is the weight decay coefficient.

3.12.2. Fractional AdamW Lion Update Equations

Using this approximation, the fractional momentum update becomes

$$m_t^{\left(\alpha \right)}={\beta }_1{m}_{t-1}^{\left(\alpha \right)}+(1-{\beta }_1)·{\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J{\left(\theta \right)}_{t-k}$$

(99)

Then, the parameter update rule is

$$\theta =\theta -\eta ·\mathrm{sign}\left(m_t^{\left(\alpha \right)}\right)-\eta ·\lambda ·\theta$$

(100)

This equation integrates three mechanisms: fractional memory via $m_t^{\left(\alpha \right)}$, sign-based adaptive direction, and decoupled regularization with weight decay.

3.12.3. Application to LSTM Training

In LSTM networks, the parameter vector $\theta$ includes weights and biases governing input, forget, and output gates, along with cell state transitions. Fractional AdamW Lion enables capturing long-range temporal dependencies, regularized and stable training, and directionally aware optimization using fractional history.

This makes the optimizer especially effective for time series tasks such as financial forecasting, anomaly detection, and biomedical signal analysis.

3.13. Fractional Yogi

Yogi is an adaptive optimization algorithm designed to address some of the limitations of Adam, particularly in controlling the variance of the second moment estimate $v_t$. Unlike Adam, which uses an exponential moving average, Yogi updates $v_t$ with a sign-based correction term that prevents rapid growth and promotes stability during training.

The classic Yogi update rules are

$$m_t={\beta }_1{m}_{t-1}+(1-{\beta }_1){\nabla }_{\theta }J\left(\theta \right)$$

(101)

$$v_t=v_{t-1}-(1-{\beta }_2)·\mathrm{sign}(v_{t-1}-{\left({\nabla }_{\theta }J\left(\theta \right)\right)}^2)·{\left({\nabla }_{\theta }J\left(\theta \right)\right)}^2$$

(102)

Here, ${\nabla }_{\theta }J\left(\theta \right)$ denotes the gradient of the loss function with respect to the parameters, while ${\beta }_1$ and ${\beta }_2$ are decay rates for the first and second moment estimates, respectively. The function $\mathrm{sign}(·)$ represents the element-wise sign operation that regulates the adjustment of $v_t$ to avoid uncontrolled variance growth.

The bias-corrected moment estimates are then computed as

$${\widehat{m}}_t={\displaystyle \frac{m_t}{1-{\beta }_1^t}},{\widehat{v}}_t={\displaystyle \frac{v_t}{1-{\beta }_2^t}}$$

(103)

Finally, the parameter update step is given by

$$\theta =\theta -\eta {\displaystyle \frac{{\widehat{m}}_t}{\sqrt{{\widehat{v}}_t}+ϵ}}$$

(104)

where $\eta$ is the learning rate and $ϵ$ a small constant ensuring numerical stability.

3.13.1. Fractional Yogi with General Fractional Gradient

In the fractional variant, the classical gradient ${\nabla }_{\theta }J\left(\theta \right)$ is replaced by its fractional derivative $D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$, with $D^{\alpha }$ representing a fractional derivative operator of order $\alpha$. This allows the optimizer to leverage long-term memory effects embedded in gradient information, which is particularly useful in sequential and time-dependent tasks.

The fractional moment updates are expressed as follows:

$$m_t^{\left(\alpha \right)}={\beta }_1{m}_{t-1}^{\left(\alpha \right)}+(1-{\beta }_1)D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)$$

(105)

$$v_t^{\left(\alpha \right)}=v_{t-1}^{\left(\alpha \right)}-(1-{\beta }_2)·\mathrm{sign}\left(v_{t-1}^{\left(\alpha \right)}-{\left(D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)\right)}^2\right)·{\left(D^{\alpha }{\nabla }_{\theta }J\left(\theta \right)\right)}^2$$

(106)

Bias correction factors adapt accordingly:  

$${\widehat{m}}_t^{\left(\alpha \right)}={\displaystyle \frac{m_t^{\left(\alpha \right)}}{1-{\beta }_1^t}},{\widehat{v}}_t^{\left(\alpha \right)}={\displaystyle \frac{v_t^{\left(\alpha \right)}}{1-{\beta }_2^t}}$$

(107)

The parameter update then becomes

$$\theta =\theta -\eta {\displaystyle \frac{{\widehat{m}}_t^{\left(\alpha \right)}}{\sqrt{{\widehat{v}}_t^{\left(\alpha \right)}}+ϵ}}$$

(108)

The fractional moment updates can be approximated by

$$m_t^{\left(\alpha \right)}={\beta }_1{m}_{t-1}^{\left(\alpha \right)}+(1-{\beta }_1){\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J{\left(\theta \right)}_{t-k}$$

(109)

$$v_t^{\left(\alpha \right)}=v_{t-1}^{\left(\alpha \right)}-(1-{\beta }_2)·\mathrm{sign}\left(v_{t-1}^{\left(\alpha \right)}-{\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J{\left(\theta \right)}_{t-k}\right)}^2\right)·{\left({\displaystyle \frac{1}{h^{\alpha }}}\sum _{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J{\left(\theta \right)}_{t-k}\right)}^2$$

(110)

3.13.2. Application in LSTM Training

When applied to LSTM networks, the parameter vector $\theta$ contains weights and biases controlling gating mechanisms and cell states. The fractional Yogi optimizer incorporates fractional-order memory into adaptive moment estimation, enabling the model to better capture long-range temporal dependencies, stabilize optimization, and improve convergence behavior.

The update rule remains

$$\theta =\theta -\eta {\displaystyle \frac{{\beta }_1{\widehat{m}}_t^{\left(\alpha \right)}+(1-{\beta }_1)\frac{1}{h^{\alpha }}{\sum }_{k=0}^M{\omega }_k^{\left(\alpha \right)}{\nabla }_{\theta }J{\left(\theta \right)}_{t-k}}{\sqrt{{\widehat{v}}_t^{\left(\alpha \right)}}+ϵ}}$$

(111)

This fractional enhancement helps LSTM models exploit memory effects effectively, boosting performance in complex sequential learning tasks.

In conclusion, incorporating fractional-order derivatives within optimization routines presents a promising approach to enhancing the learning capabilities of LSTM models for modeling financial time-series data. By embedding long-memory effects into the optimization process, these methods can more effectively capture long-term dependencies in the data, resulting in improved forecasting accuracy and overall predictive performance.

3.14. Why Caputo over Other Definitions?

Unlike the Riemann–Liouville derivative, which requires fractional-order initial conditions and is not zero on constants, the Caputo derivative maintains

$${}^{C}D_t^{\alpha }\mathrm{constant}=0,$$

(112)

and only requires standard (integer-order) initial values, which are naturally compatible with recurrent architectures like LSTMs. This property makes Caputo derivatives more suitable for learning systems where initial states are learned or well-defined [42]. Moreover, in our context, where training efficiency is essential, the Caputo formulation supports the short-memory principle—a practical approximation where the integration window is truncated to a finite horizon without significant loss of accuracy [30]. In addition, using a small value of $\alpha$, in Frac-LSTM, replicates this inductive bias, yielding smoother predictions and enhanced stability, especially in trend-following regimes. In fact, the Caputo derivative’s behavior resembles the Hurst exponent H in fractional Brownian motion [31]; see Equation (8). These properties collectively make the Caputo derivative the most suitable fractional formulation for integrating memory-based inductive biases in neural forecasting models for financial time series.

3.15. Computational Scalability and Runtime Analysis

Fractional-order optimization methods, such as Frac-RMSprop and Frac-Adam, introduce fractional derivatives into gradient-based training to incorporate long-term memory effects. While this provides a smoother and more stable convergence in noisy financial time series, it raises concerns regarding computational scalability, particularly for latency-sensitive applications like high-frequency trading (HFT).

3.15.1. Approximation of Fractional Derivatives

In our approach, we use the Grünwald–Letnikov (GL) approximation of the fractional derivative of order $\alpha \in (0,1]$ for a function $f\left(t\right)$ is given by Equation (5). In practice, M is chosen such that $M\ll T$ (entire series length), making the computation tractable.

3.15.2. Time Complexity Comparison

The inclusion of the truncated sum in Equation (5) implies a per-parameter update time complexity of $O\left(M\right)$ instead of $O\left(1\right)$ as in classical optimizers. However, the following comparative benchmark illustrates that for moderate values of $M\in \{30,60,90\}$, the observed time overhead remains modest; see

Table 1. Comparison of normalized runtimes per epoch for different optimizers.

Optimizer	Window Size M	Relative Runtime (Normalized)
Adam	–	1.00
RMSprop	–	1.01
Frac-RMSprop	30	1.05
Frac-RMSprop	60	1.09
Frac-RMSprop	90	1.12

.

3.15.3. Implications for Trading Frequency

For medium-frequency trading (daily to hourly horizons), the computational cost is acceptable and easily offset by gains in prediction stability and return performance. However, for ultra-high-frequency trading (UHFT), where millisecond latency is critical, even minor computational overheads may render fractional methods impractical without further optimization.

To address this, we propose two future directions:

• Development of GPU-accelerated implementations for fast convolutional approximations of fractional derivatives.
• Exploration of approximate memory kernels or fixed-window compression to reduce the sum in Equation (5) to a constant-time operation.

In conclusion, fractional optimizers with short-memory truncation introduce manageable computational costs in medium-frequency financial modeling. Their enhanced memory representation and robustness to noise justify their use in such settings. For HFT applications, ongoing research should focus on developing faster approximations or hybrid fractional–classical schemes.

4. Experimental Setup

This paper explores the benefits of developing fractional-order extensions of standard optimization techniques to more effectively capture long-term dependencies during training. The proposed fractional optimizers are integrated into LSTM networks and evaluated using stock market datasets within the context of financial time-series forecasting. To assess the performance of conventional versus fractional optimizers, we examine traditional SGD-based algorithms alongside their fractional counterparts. Through a series of experiments, we highlight the primary advantages of fractional optimization in enhancing the efficiency and stability of time-series predictions. This benchmarking study focuses in particular on stock price forecasting—where long-term dependencies play a critical role—and investigates the extent to which fractional-order LSTM models outperform their classical equivalents.

The implementation was performed using TensorFlow (via Keras) in Google Colab, which provides access to Tesla T4 GPUs. The key configuration details are as follows:

• Framework: TensorFlow 2.x (with Keras API);
• Hardware: Google Colab Tesla T4 GPU;
• Batch Size: 64;
• Sequence Length (Memory Window): Varied, tested with 30, 60, and 90;
• Epochs: 100 (with early stopping in some runs to prevent overfitting).

The environment also included custom implementations of fractional optimizers (e.g., Frac-Adagrad) based on fractional derivative approximations integrated into the optimizer update rules.

4.1. Data Description

The benchmark dataset employed in this study consists of daily stock price data for nine leading companies, sourced from Yahoo Finance: Apple Inc. (AAPL), Microsoft (MSFT), Google (GOOGL), Amazon (AMZN), Meta Platforms (META), NVIDIA (NVDA), JPMorgan Chase (JPM), Visa (V), and UnitedHealth Group (UNH). The data span from 1 January 2015 to 1 January 2023. The closing prices are selected as the primary feature for forecasting purposes. The raw input data are normalized using the MinMaxScaler technique, which scales the features to a range between 0 and 1. Sequences of 60 consecutive closing prices are used to train the models. The dataset is split into 80% for training and 20% for testing. This type of dataset is widely used in stock price forecasting tasks, as demonstrated in [43], where the objective is to predict future prices based on historical trends.

4.2. Algorithms

The fundamental model is a conventional LSTM neural network, designed to handle time-dependent sequential data. The architecture includes a 50-unit LSTM layer followed by a dense output layer for stock price forecasting. The model uses a 60-day sequence of closing prices to predict the next day’s closing price [44].

The fractional LSTM model extends the conventional LSTM by incorporating fractional memory effects. In this model, the LSTM output is combined with the weighted mean of the preceding input sequence, influenced by a fractional parameter $\alpha$. This modification aims to more accurately capture long-term dependencies, potentially improving forecasting performance. The value of $\alpha$ varies from 0.1 to 0.9 to assess the effects of different memory levels on model accuracy. Fractional LSTMs have also been explored in other time series prediction domains [45].

Table 2. The optimizer’s hyperparameters.

Optimizer	Learning Rate	Beta 1	Beta 2
Adam	0.001	0.9	0.999
RMSprop	0.001	0.9	–
SGD	0.01	–	–
Adagrad	0.01	–	–
Adadelta	1.0	0.95	0.999
Nadam	0.001	0.9	0.999
Ftrl	0.1	0.15	0.85
Adamax	0.002	0.9	0.999

lists the key hyperparameters used by the optimizers in this experiment. These hyperparameters are critical for model convergence and performance. The learning rate is the most important parameter, while the various optimizers include additional tuning factors such as momentum or decay.

Adam is an adaptive optimizer that combines the advantages of two extensions of gradient descent, AdaGrad and RMSProp, by utilizing moment estimates to adjust learning rates for individual parameters. Adam is commonly used for time series tasks because it is both efficient and robust [27].

RMSprop divides the learning rate by a moving average of the square root of recent gradients. This technique is particularly useful for non-stationary problems like stock price forecasting, where the data distribution may change over time [46].

SGD (Stochastic Gradient Descent) is a relatively simple optimizer that updates parameters based on a single mini-batch. Although SGD tends to be less effective for complex tasks such as time series prediction, it remains popular due to its simplicity and effectiveness when properly tuned.

Adagrad adapts the learning rate according to the frequency of parameter updates. It is well suited for sparse data scenarios where some features require more frequent updates than others.

Adadelta improves on Adagrad by mitigating its rapid learning rate decay using a running average of squared gradients to dynamically adjust the learning rate.

Nadam combines Adam with Nesterov’s accelerated gradient (NAG), providing the benefits of momentum alongside adaptive learning rates. This combination often results in faster convergence in practical applications.

FTRL (Follow-The-Regularized-Leader) is an optimizer designed for large-scale machine learning, commonly applied to sparse datasets such as those encountered in recommender systems or logistic regression tasks.

Adamax is a variant of Adam that uses the infinity norm, offering greater robustness in certain situations.

In addition, we have conducted a multi-dimensional sensitivity analysis varying key LSTM architectural parameters—hidden units ([20, 50, 100]) and dropout rates ([0.0, 0.2, 0.5])—across different memory window sizes ([30, 90]). The number of layers was fixed to 2 to limit complexity, but model performance was systematically evaluated under all other design variations; see

Table 3. Model performance metrics for the average different configurations on all stocks and all optimizers without the layer and stock columns.

Alpha	Window Size	Hidden Units	Dropout	Aver. Sharpe	Aver. Accuracy	Aver. Cumulative Return
0.3	30	20	0.0	0.023926	0.492414	0.735054
0.3	30	50	0.0	0.062335	0.503448	6.237091
0.3	30	100	0.0	0.070080	0.502069	4.549126
0.3	30	20	0.2	0.066320	0.507586	2.938690
0.3	30	50	0.2	0.069087	0.503448	4.410477
0.3	30	100	0.2	0.064119	0.500690	6.233910
0.3	30	20	0.5	0.071183	0.488276	3.204750
0.3	30	50	0.5	0.048990	0.508966	2.758423
0.3	30	100	0.5	0.061129	0.496552	6.540634
0.3	90	20	0.0	0.123771	0.499248	3.230061
0.3	90	50	0.0	0.093866	0.505263	6.305473
0.3	90	100	0.0	0.113093	0.485714	11.822601
0.3	90	20	0.2	0.095908	0.502256	6.796715
0.3	90	50	0.2	0.097023	0.508271	6.988266
0.3	90	100	0.2	0.134719	0.491729	7.455177
0.3	90	20	0.5	0.086247	0.514286	0.858746
0.3	90	50	0.5	0.138106	0.488722	11.097931
0.3	90	100	0.5	0.106046	0.493233	4.975113
0.4	30	20	0.0	0.064027	0.492414	3.721291
0.4	30	50	0.0	0.062604	0.496552	5.248947
0.4	30	100	0.0	0.058876	0.506207	5.469574
0.4	30	20	0.2	0.005598	0.504828	0.199921
0.4	30	50	0.2	0.068436	0.496552	7.558159
0.4	30	100	0.2	0.067228	0.493793	6.473572
0.4	30	20	0.5	0.066051	0.500690	4.153961
0.4	30	50	0.5	0.070924	0.511724	4.535126
0.4	30	100	0.5	0.060612	0.497931	6.441483
0.4	90	20	0.0	0.137105	0.512782	4.742943
0.4	90	50	0.0	0.070068	0.494737	4.054939
0.4	90	100	0.0	0.112514	0.500752	8.333633
0.4	90	20	0.2	0.092492	0.508271	5.157829
0.4	90	50	0.2	0.119598	0.503759	16.004845
0.4	90	100	0.2	0.106422	0.487218	6.550301
0.4	90	20	0.5	0.146293	0.508271	9.485603
0.4	90	50	0.5	0.074089	0.508271	1.592896
0.4	90	100	0.5	0.117845	0.484211	8.569008

. The results demonstrate that architectural choices significantly affect the Sharpe ratio and directional accuracy, confirming that both model design and optimizer behavior influence predictive performance.

The experimental results highlight that the best configuration in terms of Sharpe ratio is obtained with $\alpha =0.4$, a window size of 90, 20 hidden units, and a dropout rate of $0.5$, yielding a Sharpe ratio of $0.146293$. However, the highest cumulative return of $16.004845$ is achieved with the configuration $\alpha =0.4$, window size 90, 50 hidden units, and dropout rate of $0.2$, along with a solid Sharpe ratio of $0.119598$ and accuracy of $0.503759$. Although the maximum accuracy observed is $0.514286$ for $\alpha =0.3$, window size 90, 20 hidden units, and dropout $0.5$, this setting yields poor financial performance, with a cumulative return of only $0.858746$ and a modest Sharpe ratio of $0.086247$.

Overall, the results suggest that a window size of 90 consistently leads to better performance across all metrics compared to 30. Additionally, a higher value of $\alpha$ ($0.4$) outperforms $0.3$ in most top-performing configurations. Dropout values of $0.2$ or $0.5$ appear to enhance generalization and improve the Sharpe ratio and cumulative returns, particularly when combined with a larger window size. Lastly, the number of hidden units influences performance, with 50 or 100 units generally providing stronger returns, although the best Sharpe ratio is observed with only 20 units. These observations indicate that the configuration of $\alpha =0.4$, window size 90, hidden units 50, and dropout $0.2$ represents the most balanced setup for both profitability and risk-adjusted return.

This experimental framework provides a systematic approach to evaluating LSTM and fractional LSTM models for stock price forecasting. By exploring different optimizers and hyperparameter configurations, we aim to identify the optimal setup for accurate and robust predictions. The experimental results are expected to deepen our understanding of the effects of fractional memory and optimizer choices on forecasting accuracy.

4.3. Results and Analysis

This section offers a comprehensive analysis of the results of the standard optimization algorithms—SGD, Adam, RMSprop and Adagrad—and their fractional extensions, namely Frac-Adam, Frac-RMSprop and Frac-Adagrad, emphasizing their contrasting convergence behavior.

4.3.1. Analysis of Standard vs. Fractional Adam Optimization Results

Figure 4 shows the forecasts provided by the classic and fractional Adam methods.

The comparisons of conventional Adam and Fractional Adam (Frac-Adam) optimization at various fractional orders $\alpha$ for forecasting stock prices are presented and yield interesting insights regarding the effectiveness of fractional derivatives in the optimization procedure. A comprehensive discussion of the positive and negative aspects of the results is presented in this section.

The overall tendency noted in the results indicates that the behavior of the fractional Adam optimizer differs remarkably according to the choice of $\alpha$ and stock market ticker, highlighting that fractional derivatives used may not be generally suitable for all stock price forecasting tasks. Across the analyzed values, the standard Adam optimizer typically outperformed or closely matched the fractional Adam at smaller values of $\alpha$, with some variation between values. For GOOGL (Google)—see Figure 5—the conventional Adam optimizer scored a reasonable MSE of 62.64, and the fractional Adam with $\alpha =0.1$ improved this figure to 32.40, whereas $\alpha =0.3$ yielded a marginal enhancement with an MSE of 52.93. However, as $\alpha$ increases (e.g., to $\alpha =0.5$), the MSE deteriorates, although it improves marginally for $\alpha =0.9$.

For AMZN (Amazon)—see Figure 6—classical Adam reached an MSE of 74.65, but fractional Adam with $\alpha =0.1$ upped the MSE to 93.66, and additional increments in $\alpha$ resulted in significantly greater MSE values, peaking at 349.39 for $\alpha =0.9$.

META—see Figure 7—demonstrated considerable performance deterioration with fractional Adam, where the conventional Adam optimizer yielded an MSE of 148.65, while fractional optimization generated significant MSE increases, especially for larger values of $\alpha$ (e.g., $\alpha =0.9$, MSE = 11,691.78).

Finally, for NVDA (NVIDIA)—see Figure 8—the traditional Adam optimizer achieved the lowest MSE of 3.15, denoting excellent performance; however, the fractional Adam with smaller values of $\alpha$ led to only marginal increments in MSE, and larger values of $\alpha$ disrupted the optimization process, leading to a higher increment in MSE (e.g., $\alpha =0.5$, MSE = 55.91).

Smaller values of $\alpha$ (0.1 and 0.3) typically result in minor enhancements or slight deteriorations, while higher values $\alpha >0.3$ more frequently worsen performance, notably for stocks like META, AMZN, and NVDA, suggesting that fractional derivatives can disrupt the optimization workflow. The efficiency of fractional optimization differs depending on the stock: it performs remarkably well for AAPL and GOOGL at specific $\alpha$ values but significantly deteriorates performance for certain other stocks, such as META and UNH. In most cases, conventional Adam performs better than or similarly to fractional Adam, most notably for stocks like META, NVDA, and JPM, where larger values of $\alpha$ result in noticeable increases in MSE.

The results in

Table 4. Performance comparison of Adam-LSTM and Frac-Adam-LSTM models on AAPL stock price forecasting for various fractional orders $\alpha$.

Model	Alpha	Sharpe Ratio	Directional Accuracy	Cumulative Return
Adam-LSTM	0.0	0.7800	48.99%	89.04
Frac-Adam-LSTM	0.1	0.9266	49.71%	105.72
Frac-Adam-LSTM	0.2	0.7298	50.14%	83.32
Frac-Adam-LSTM	0.3	0.8818	50.14%	100.63
Frac-Adam-LSTM	0.4	0.5713	50.00%	65.25
Frac-Adam-LSTM	0.5	0.8330	49.86%	95.07
Frac-Adam-LSTM	0.6	1.0821	50.87%	123.38
Frac-Adam-LSTM	0.7	1.3051	51.73%	148.66
Frac-Adam-LSTM	0.8	1.0792	51.88%	123.06
Frac-Adam-LSTM	0.9	0.2846	48.55%	32.52

demonstrate the significant impact of the fractional order $\alpha$ on the forecasting and trading performance of the Frac-Adam-LSTM model for AAPL stock. Compared to the classical Adam-LSTM ($\alpha =0.0$), which achieves a Sharpe ratio of 0.78 and a cumulative return of 89.04, fractional models with intermediate $\alpha$ values generally yield enhanced risk-adjusted returns and directional accuracy. In particular, $\alpha =0.7$ results in the highest Sharpe ratio (1.305), the greatest cumulative return (148.66), and the best directional accuracy (51.73%), indicating improved profitability and predictive performance. However, very high fractional orders, such as $\alpha =0.9$, lead to a sharp decline in performance, underscoring the sensitivity of the model to excessive long-memory effects in the volatile AAPL market. Overall, the inclusion of fractional derivatives provides a clear advantage, suggesting that well-calibrated fractional orders can more effectively capture complex market dynamics and support more informed trading decisions.

Figure 9 presents the sensitivity analysis of the Frac-Adam-LSTM model on AAPL. The results show that the model’s performance is affected by both the learning rate ($\eta$) and the fractional order ($\alpha$). For $\alpha =0.3$, the Sharpe ratio peaks at $\eta ={10}^{-4}$, accompanied by a relatively high cumulative return of 66.59, while accuracy remains stable. In contrast, for $\alpha =0.5$ and $\alpha =0.7$, cumulative returns are more consistent across higher learning rates ($\eta ={10}^{-3}$ and ${10}^{-2}$), with $\alpha =0.7$ achieving the highest cumulative return of 71.12 at $\eta ={10}^{-2}$. Notably, $\alpha =0.9$ yields robust cumulative returns across all learning rates, indicating the fractional optimizer’s resilience; however, its peak Sharpe ratio of 0.2137 at $\eta ={10}^{-5}$ remains slightly lower than those at mid-range $\eta$ values. Interestingly, lower fractional orders such as $\alpha =0.5$ display instability at small learning rates, evidenced by negative Sharpe ratios and returns. This suggests that combinations of low $\alpha$ and low $\eta$ may lead to underperformance. Overall, higher fractional orders demonstrate greater tolerance and stability across a wider range of learning rates, reinforcing the idea that the fractional component serves as an implicit regularizer and enhances optimizer robustness.

Figure 10 gives the sensitivity analysis of Frac-Adam-LSTM on GOOGL memory window size. The sensitivity analysis shows that the best Sharpe ratio and cumulative return were achieved with $\alpha =0.7$ and a memory window of 60 days, yielding a Sharpe ratio of 0.1134 and a cumulative return of 32.29. Overall, models with a memory window of 60 consistently outperform others across all alpha values, indicating that a medium-term memory window provides a better trade-off between risk and return. Higher values of $\alpha$ (0.7 and 0.9) slightly improve accuracy, with the highest directional accuracy of 0.5103 observed for $\alpha =0.9$. This suggests that both the fractional learning rate and memory window size significantly influence the performance of the Frac-ADAM-LSTM model.

4.3.2. Analysis of Standard vs. Fractional RMSprop Optimization Results

Figure 11 gives the MSE associated with RMSprop and fracRMSprop used to learn LSTM on AAPL, MSFT, GOOGL, AMZN, META, NVDA, JPM, V, and UNH stocks.

Benchmarking the standard RMSprop optimizer and its fractional order variants on various stocks (AAPL, MSFT, GOOGL, AMZN, META, NVDA, JPM, V, and UNH) reveals considerable differences in performance depending on the chosen fractional order $\alpha$. In numerous cases, particularly for AAPL, GOOGL, JPM (see Figure 12), and UNH, fractional RMSprop with smaller $\alpha$ values (e.g., $\alpha =0.1$ or $\alpha =0.3$) outperforms the standard optimizer in terms of loss minimization. For example, AAPL’s loss decreases from 52.54 (standard) to 44.48 when using $\alpha =0.1$, while GOOGL shows a similar trend with a decline from 62.64 to 32.40. These improvements suggest that lower fractional orders may provide beneficial memory effects and smoother convergence on specific datasets, particularly those with low volatility or complex data patterns.

Nonetheless, when $\alpha$ exceeds 0.5, the loss margin tends to worsen significantly for many stocks, reflecting either volatile or divergent formation dynamics. This trend is particularly evident for META and UNH, where losses reach $11,691.78$ and $2798.63$, respectively, at $\alpha =0.9$, compared to their standard values of $148.65$ and $1281.37$. Such behavior suggests that high-order memory effects may excessively weight past gradients, causing delayed responses to new data or even leading to optimization divergence. Stocks with more complex or volatile time series, like META, seem especially vulnerable to this phenomenon, underscoring the importance of carefully tuning $\alpha$ to the characteristics of the data.

Conversely, for stocks such as MSFT and AMZN (see Figure 13), the classical RMSprop often outperforms the fractional versions at lower $\alpha$, with the best results occurring at intermediate values like 0.3 or 0.5.

Table 5. Performance comparison of RMSprop-LSTM and Frac-RMSprop-LSTM models for AMZN stock forecasting based on RMSE, Sharpe ratio, and directional accuracy. FracLSTM explores different fractional orders $\alpha$.

Model	RMSE	Sharpe Ratio	Directional Accuracy
Standard LSTM	7.8237	0.1949	50.14%
FracLSTM $\alpha =0.0$	5.4988	0.1949	50.00%
FracLSTM $\alpha =0.1$	6.8128	0.1949	50.29%
FracLSTM $\alpha =0.2$	7.2629	0.1949	51.30%
FracLSTM $\alpha =0.3$	7.1341	0.1949	50.43%
FracLSTM $\alpha =0.4$	7.9988	0.1949	49.42%
FracLSTM $\alpha =0.5$	7.5969	0.1949	50.43%

presents a performance comparison between the standard LSTM and FracLSTM models for AMZN stock price forecasting, highlighting the nuanced benefits of fractional memory integration. While the standard LSTM yields an RMSE of 7.8237, the FracLSTM with $\alpha =0.0$ (corresponding to classical LSTM behavior) achieves the lowest RMSE of 5.4988, indicating enhanced short-term prediction accuracy. As the fractional order $\alpha$ increases, RMSE generally deteriorates, peaking at $\alpha =0.4$, which may suggest diminishing returns of long-memory effects in the highly stochastic context of AMZN price movements. Despite fluctuations in RMSE, the Sharpe ratio remains constant at 0.1949 across all models, indicating comparable risk-adjusted performance. Notably, the highest directional accuracy (51.30%) is observed at $\alpha =0.2$, suggesting that a modest degree of fractional memory can improve the model’s ability to capture directional trends—an asset in trading strategies. These findings support the notion that careful tuning of the fractional order in FracLSTM can lead to improved predictive performance, especially for volatile assets such as AMZN.

Figure 13 gives the sensitivity analysis of Frac-Adagrad-LSTM on AAPL considering the learning rate. The sensitivity analysis of Frac-RMSprop-LSTM on AAPL reveals that learning rates around ${10}^{-3}$ combined with higher alpha values ($\alpha =0.7$ or $\alpha =0.9$) tend to yield the best trade-offs between Sharpe ratio and cumulative return. Notably, the configuration $(\alpha =0.7, \mathrm{LR}=0.001)$ achieves a high Sharpe ratio of 0.206 and the highest cumulative return of 73.19. In contrast, extremely low or high learning rates result in suboptimal performance regardless of alpha, suggesting a sweet spot in learning dynamics governed by fractional control. Accuracy remains relatively stable across configurations, centered around 51%, indicating modest directional predictability. Figure 14 gives the sensitivity analysis of Frac-RMSprop-LSTM on GOOGL memory window size. The sensitivity analysis of the Frac-RMSprop-LSTM model applied to GOOGL stock reveals that the performance varies with the memory window size and the fractional scaling parameter $\alpha$. The highest Sharpe ratio of 0.1308 is achieved at $\alpha =0.3$ and a memory window of 90, while the best cumulative return of 31.43 corresponds to $\alpha =0.3$ and window size 60. In general, intermediate memory windows (60 or 90) yield better Sharpe ratios and returns across all $\alpha$ values. Higher $\alpha$ values (e.g., 0.9) slightly improve accuracy but do not consistently enhance financial performance. This indicates a trade-off between memory depth and learning rate scaling, with moderate values offering the best predictive and financial outcomes.

4.3.3. Analysis of Standard vs. Fractional Adagrad Optimization Results

Figure 15 illustrates the mean squared error (MSE) associated with the standard Adagrad and Frac-Adagrad optimizers applied to LSTM models for forecasting stock prices of AAPL, MSFT, GOOGL, AMZN, META, NVDA, JPM, V, and UNH. Overall, the Frac-Adagrad-LSTM with a fractional order $\alpha =0.1$ outperforms the standard Adagrad-LSTM in terms of prediction accuracy, exhibiting notably lower errors for stocks such as AAPL, GOOGL, JPM, and UNH. However, as the fractional order $\alpha$ increases, performance deteriorates markedly. In particular, high values of $\alpha$ (e.g., $\alpha =0.5$ and $\alpha =0.9$) result in significantly elevated MSEs for several stocks, including META and UNH, indicating instability and poor generalization. These results highlight the data-dependent nature of fractional-order optimization: while a small fractional component can enhance learning by introducing mild memory effects, larger values may hinder convergence and negatively impact predictive accuracy.

The benchmarking investigation involving the traditional Adagrad optimizer and its fractional counterpart (Frac-Adagrad) at various values of the fractional order $\alpha \in \{0.1, 0.3, 0.5, 0.7, 0.9\}$ demonstrates a complex, asset-dependent response in terms of loss performance. In particular, for certain stocks such as AAPL (see Figure 16), GOOGL (see Figure 17), JPM, and UNH (see Figure 18), the fractional versions with smaller $\alpha$ values (e.g., $\alpha =0.1$) consistently outperform conventional Adagrad, indicating that the memory-retaining features of Frac-Adagrad are advantageous in such cases—possibly due to smoother loss surfaces or slower dynamic behaviors.

In contrast, intermediate $\alpha$ values such as $0.5$ or $0.7$ tend to yield substantially higher loss values, suggesting unstable or even divergent optimization behavior in many cases—especially for UNH and META, where losses spike to abnormally high levels.

In contrast, assets like MSFT, META and AMZN display a reversal of this tendency, with classical Adagrad outperforming or being comparable to Frac-Adagrad for smaller $\alpha$. As $\alpha$ grows, the loss values rise faster, especially for META and UNH, indicating a significant deterioration in optimization efficiency. This observation suggests a potential downside of using high memory effects in volatile or highly non-linear loss surfaces, which can magnify gradients and result in overrunning or mediocre convergence. NVDA displays outlier behavior with extremely low losses across all methods, while moderate rises at larger $\alpha$ still suggest that fractional dynamics require careful tuning.

In short, Frac-Adagrad provides encouraging enhancements over Adagrad for a number of stocks where fractional order is small, benefiting from its capacity to preserve long-term gradient memory without introducing excessive volatility. Nevertheless, it also induces instability at larger $\alpha$ values, particularly for complicated or volatile datasets. This underlines the relevance of the adaptive or data-driven screening of fractional order $\alpha$, which can change according to asset or training phase, to secure convergence and consistent performance.

We benchmarked the performance of the following optimization methods: Frac-SGD, Frac-Adadelta, Frac-Nadam, Frac-FTRL and Frac-Adamax. A comparable in-depth analysis would reveal the identical impact of $\alpha$ selection. However, to avoid crowding the paper with redundant analyses, we only supply the corresponding figures to provide a quick visual idea of these effects (

Table 6. Performance comparison of the Adagrad and Frac-Adagrad LSTM models on GOOGL stock price.

Model	Alpha	Sharpe Ratio	Directional Accuracy	Cumulative Return
Adagrad-LSTM	0.0	−0.339907	0.501445	−34.440094
Frac-Adagrad-LSTM	0.1	0.054720	0.514451	5.545547
Frac-Adagrad-LSTM	0.2	−0.136651	0.497110	−13.848495
Frac-Adagrad-LSTM	0.3	0.285306	0.513006	28.909805
Frac-Adagrad-LSTM	0.4	0.350022	0.523121	35.464493
Frac-Adagrad-LSTM	0.5	0.074067	0.518786	7.506226
Frac-Adagrad-LSTM	0.6	0.163547	0.520231	16.573822
Frac-Adagrad-LSTM	0.7	0.628578	0.527457	63.653625
Frac-Adagrad-LSTM	0.8	0.547424	0.523121	55.445953
Frac-Adagrad-LSTM	0.9	0.601502	0.528902	60.915802

).

The results clearly demonstrate the superiority of the Frac-Adagrad-LSTM models over the standard Adagrad-LSTM baseline. Notably, the configuration with $\alpha =0.7$ achieves the highest Sharpe ratio (0.6286) and final cumulative return (63.65), indicating that this fractional model offers the best risk-adjusted performance and profitability. Meanwhile, the highest directional accuracy (0.5289) is observed at $\alpha =0.9$, suggesting enhanced prediction of price movement direction with higher fractional influence. Overall, fractional memory improves the model’s effectiveness across all key financial metrics, making Frac-Adagrad a compelling alternative to traditional optimizers.

Figure 19 gives the Sensitivity analysis of Frac-Adagrad-LSTM on AAPL. Figure 19 presents the sensitivity analysis of the Frac-Adagrad-LSTM model applied to AAPL stock. The results indicate that higher learning rates (e.g., $\eta =0.01$) consistently lead to improved Sharpe ratios and cumulative returns across all values of the fractional order $\alpha$. The best performance is achieved with $\alpha =0.9$ and $\eta =0.01$, yielding a Sharpe ratio of 0.221 and a cumulative return of approximately 52.49. In contrast, very small learning rates ($\eta ={10}^{-5}$) produce poor performance regardless of the chosen $\alpha$, suggesting insufficient update magnitudes for effective learning. Notably, directional accuracy remains relatively stable across all configurations (approximately 48–52%), indicating that predictive directionality is less sensitive to variations in these hyperparameters compared to risk-adjusted and return-based metrics.

Figure 20 gives the sensitivity analysis of Frac-Adagrad-LSTM on GOOGL memory window size. The sensitivity analysis of the Frac-Adagrad-LSTM model on GOOGL stock data reveals that the optimal memory window size generally lies around 60 or 90 days. For all tested $\alpha$ values, a memory window of 60 consistently delivers strong performance across the Sharpe ratio, directional accuracy, and cumulative return. For instance, with $\alpha =0.7$, $M=60$ achieves a Sharpe ratio of $0.146$ and a cumulative return of $11.30$, outperforming other memory settings. Performance tends to decrease with either very short (30) or very long (120) memory windows, highlighting the importance of selecting an appropriate temporal context for modeling market dynamics.

4.4. Comparison to the ARIMA and GARCH Models

In this sub-section, we compare the performance of Frac-ADAM, Frac-Adagrad, Frac-RMSprop, ARIMA, and GARCH Models; see

Table 7. Performance of the Frac-ADAM, Frac-Adagrad, Frac-RMSprop, ARIMA and GARCH Models.

Model	Alpha	Sharpe Ratio	Directional Accuracy	Final Cumulative Return
Frac-ADAM	0.01	−0.4231	0.4812	−42.86
Frac-ADAM	0.06	−0.1869	0.4957	−18.94
Frac-ADAM	0.11	−0.4928	0.4769	−49.92
Frac-ADAM	0.16	0.1097	0.4971	11.12
Frac-ADAM	0.21	−0.4882	0.4754	−49.46
Frac-ADAM	0.26	−0.0111	0.5029	−1.12
Frac-ADAM	0.31	−0.3418	0.4870	−34.63
Frac-ADAM	0.36	−0.5918	0.4769	−59.94
Frac-ADAM	0.41	−0.1234	0.5014	−12.50
Frac-ADAM	0.46	−0.1559	0.4942	−15.80
Frac-ADAM	0.51	−0.3372	0.5014	−34.16
Frac-ADAM	0.56	−0.6069	0.4754	−61.47
Frac-Adagrad	0.01	−0.2910	0.5029	−29.49
Frac-Adagrad	0.06	−0.6234	0.4841	−63.13
Frac-Adagrad	0.11	−0.7387	0.4841	−74.79
Frac-Adagrad	0.16	−0.1376	0.5101	−13.95
Frac-Adagrad	0.21	−0.0700	0.5159	−7.10
Frac-Adagrad	0.26	0.0852	0.5058	8.64
Frac-Adagrad	0.31	0.3215	0.5116	32.58
Frac-Adagrad	0.36	0.7404	0.5275	74.95
Frac-Adagrad	0.41	0.4570	0.5188	46.30
Frac-Adagrad	0.46	−0.1770	0.5029	−17.94
Frac-Adagrad	0.51	0.6209	0.5303	62.88
Frac-Adagrad	0.56	0.3759	0.5231	38.08
Frac-RMSprop	0.01	−0.6345	0.4827	−64.25
Frac-RMSprop	0.06	−0.5548	0.4827	−56.19
Frac-RMSprop	0.11	−0.6762	0.4827	−68.47
Frac-RMSprop	0.16	−0.2074	0.4884	−21.01
Frac-RMSprop	0.21	−0.4239	0.4884	−42.95
Frac-RMSprop	0.26	−0.1025	0.5043	−10.39
Frac-RMSprop	0.31	−0.4032	0.4971	−40.85
Frac-RMSprop	0.36	−0.1236	0.4942	−12.53
Frac-RMSprop	0.41	0.1449	0.5029	14.68
Frac-RMSprop	0.46	0.1229	0.5014	12.46
Frac-RMSprop	0.51	−0.6096	0.4913	−61.73
Frac-RMSprop	0.56	−0.1501	0.4986	−15.21
ARIMA	N/A	−0.8201	0.0852	−0.0076
GARCH	N/A	1.067e6	0.0000	0.4740

.

The performance comparison reveals that among all models and fractional orders, the Frac-Adagrad optimizer with $\alpha =0.36$ achieves the highest Sharpe ratio (0.7404) and cumulative return (74.95), indicating superior risk-adjusted profitability. Frac-Adagrad consistently outperforms both Frac-Adam and Frac-RMSprop across most values of $\alpha$, exhibiting better directional accuracy and more stable returns. In contrast, Frac-RMSprop and Frac-Adam generally yield negative Sharpe ratios and cumulative returns, with only marginal improvements at certain fractional orders, but their overall performance remains inferior. Classical statistical models such as ARIMA and GARCH perform poorly in this context: ARIMA provides low directional accuracy and negligible returns, while GARCH exhibits unrealistic Sharpe ratios and fails to predict directional movements altogether. These findings highlight the effectiveness of fractional adaptive optimizers—particularly Frac-Adagrad—in capturing the complex dynamics of financial time series and enhancing predictive and trading performance.

The RMSE histograms for the Frac-SGD, Frac-Adadelta, Frac-Nadam, Frac-FTRL, and Frac-Adamax methods—evaluated across stock prices of various companies—are presented in Appendix A.

Figure A1 gives the MSE associated with standard SGD and Frac-SGD used to train the LSTM on AAPL, MSFT, GOOGL, AMZN, META, NVDA, JPM, V, and UNH stocks. Fractional SGD-LSTM with $\alpha =0.1$ generally improves prediction accuracy compared to standard SGD-LSTM for some stocks (e.g., GOOGL, JPM, UNH), showing lower error values. However, as $\alpha$ increases beyond 0.3, performance often deteriorates drastically, leading to high prediction errors (e.g., META and UNH at $\alpha =0.9$), indicating instability and poor convergence. Thus, fractional SGD can be beneficial at small $\alpha$, but higher values are detrimental to model performance.

Figure A2 gives the MSE associated with standard Adadelta and Frac-Adadelta used to train LSTM on AAPL, MSFT, GOOGL, AMZN, META, NVDA, JPM, V, and UNH stocks. The comparison between the standard Adadelta-LSTM and its fractional variants across different stock price predictions reveals a mixed performance. For certain stocks such as AAPL, GOOGL, JPM, and UNH, the fractional Adadelta-LSTM with lower fractional orders (particularly $\alpha =0.1$) significantly outperforms the standard approach in terms of reduced MSE. For instance, the MSE drops from 52.54 to 44.48 for AAPL and from 62.64 to 32.40 for GOOGL. However, higher values of $\alpha$ tend to degrade performance, often substantially, as observed in META and UNH, where MSE reaches over 11,000 and 7800, respectively. These results suggest that fractional optimization with carefully chosen $\alpha$ values can enhance prediction accuracy, but inappropriate settings may lead to instability or overfitting.Figure A3 gives the MSE associated with standard Nadam and Frac-Nadam used to train the LSTM on AAPL, MSFT, GOOGL, AMZN, META, NVDA, JPM, V, and UNH stocks. The performance comparison between the standard Nadam-LSTM and its fractional counterparts across various stock price prediction tasks demonstrates that the fractional approach can offer improved accuracy, but only under specific fractional orders. For example, with $\alpha =0.1$, the Frac-Nadam-LSTM significantly reduces the MSE for stocks like AAPL (from 52.54 to 44.48), GOOGL (from 62.64 to 32.40), and JPM (from 48.86 to 25.89), indicating enhanced predictive capability. However, as $\alpha$ increases, performance generally degrades, with some extreme values—such as for META and UNH—reaching very high MSEs (e.g., 11,691.78 and 7890.24, respectively at $\alpha =0.5$ and beyond). This highlights the sensitivity of fractional Nadam to the choice of $\alpha$, where smaller values may lead to improved accuracy, while larger ones risk severe instability or overfitting.

Figure A4 gives the MSE associated with standard FTRL and Frac-FTRL used to train the LSTM on AAPL, MSFT, GOOGL, AMZN, META, NVDA, JPM, V, and UNH stocks. The evaluation of standard FTRL-LSTM against its fractional variants for stock price prediction reveals that fractional FTRL can enhance or degrade performance depending on the fractional order $\alpha$. For instance, with $\alpha =0.1$, improvements are observed in several cases, notably for AAPL, GOOGL, JPM, and UNH, where the mean squared error (MSE) is substantially reduced compared to the standard version. However, as $\alpha$ increases beyond 0.3, the performance tends to deteriorate significantly, with large error spikes, particularly for META and UNH at $\alpha =0.9$, reaching values as high as 11,691.78 and 2798.63, respectively. This suggests that while fractional FTRL introduces additional flexibility, its effectiveness is highly sensitive to the choice of $\alpha$, where lower values are generally more stable and beneficial.

Figure A5 gives the MSE associated with standard Adamax and Frac-Adamax used to train LSTM on AAPL, MSFT, GOOGL, AMZN, META, NVDA, JPM, V, and UNH stocks. The comparative results between standard Adamax-LSTM and its fractional variant across various stock price prediction tasks indicate that fractional Adamax can improve prediction performance when the fractional order $\alpha$ is carefully selected. For instance, with $\alpha =0.1$, a notable reduction in prediction error is observed for AAPL, GOOGL, JPM, and especially UNH, where the error drops from 1281.37 to 304.45. However, higher values of $\alpha$ generally lead to performance degradation, as evidenced by extreme error increases for META ($\alpha =0.9$: 11,691.78) and UNH ($\alpha =0.5$: 7890.24). These findings suggest that fractional Adamax introduces significant sensitivity to $\alpha$, where low-order memory effects (small $\alpha$) can be beneficial, while higher orders may introduce instability and reduced accuracy.

Toward the end, the RMSE analysis of fractional optimizers in LSTM-based stock price prediction reveals that variants with small memory parameters ($\alpha =0.1$) consistently outperform their standard counterparts. Improvements are notable for stocks like AAPL, GOOGL, JPM, and UNH, where RMSEs drop significantly. However, increasing $\alpha$ generally leads to performance degradation, with large errors observed for META and UNH. This trend holds consistently across Frac-SGD, Frac-Adadelta, Frac-Nadam, Frac-FTRL, and Frac-Adamax. The results highlight the potential of fractional optimization when low $\alpha$ values are chosen carefully. Nonetheless, inappropriate fractional orders may introduce instability, overfitting, or convergence failures.

4.5. Synthesis of Optimization Analysis Results

The examination of different optimization techniques, namely Adam, RMSprop, and standard Adagrad, compared to their fractional analogues (Frac-Adam, Frac-RMSprop, and Frac-Adagrad), has provided valuable insights into the effect of fractional derivatives on the training of LSTM models for stock price forecasting.

The comparison between standard Adam and fractional Adam (Frac-Adam) yielded mixed results. For certain stocks such as GOOGL, fractional derivatives with low $\alpha$ values (such as 0.1) typically resulted in improvements in root-mean-square error (RMSE). Nevertheless, as $\alpha$ increases, Frac-Adam’s performance declines, especially for stocks like META and AMZN. In most cases, the classic Adam optimizer outperformed or performed similarly to Frac-Adam, particularly at higher $\alpha$ values and when the optimization process was unstable.

A comparison of RMSprop with Frac-RMSprop also revealed performance disparities across stocks. Fractional RMSprop underperformed the conventional variant in certain stocks (such as AAPL and GOOGL) at lower $\alpha$ values, indicating that fractional order may promote smoother convergence and enhanced memory effects. However, as $\alpha$ rises above 0.5, the performance significantly deteriorates for stocks like META and UNH, with notably larger MSE values. This highlights the risk of overemphasizing historical gradients in volatile stocks, leading to instability.

For Adagrad and Frac-Adagrad, smaller fractional orders ($\alpha =0.1$) had a positive effect for stocks like AAPL and GOOGL, where a smoother gradient descent benefited the optimization process. However, for stocks like MSFT, META, and AMZN, larger $\alpha$ values resulted in worsened performance, suggesting the likelihood of excessive memory effects interfering with convergence. This indicates that too much memory due to high $\alpha$ values can disrupt convergence. Therefore, fractional Adagrad should be used carefully and adapted according to the characteristics of the asset and dataset.

The choice of optimizers, particularly the adoption of fractional orders in optimizers such as Adam, RMSprop, and Adagrad, can have a considerable influence on the performance of LSTM models for stock price forecasting. While fractional derivatives can offer advantages in certain cases—particularly for stocks whose price movements are smoother or less volatile—excessively high fractional orders tend to destabilize the optimization process. In general, conventional optimizers such as Adam and RMSprop are likely to produce better or comparable results to their fractional counterparts, especially when larger memory effects are present. Therefore, tuning the fractional order $\alpha$ is crucial for achieving optimal results, and it is recommended to employ fractional versions cautiously, tailoring the optimization approach to the specific characteristics of the financial asset being modeled.

4.6. Limitations and Deployment Considerations

Deploying fractional-order optimizers in live trading environments requires careful consideration beyond algorithmic design. Real-world deployment introduces additional challenges such as low-latency inference, infrastructure scalability, financial data integrity, and regulatory compliance. However, these constraints are predominantly engineering and system-integration issues rather than fundamental limitations of the fractional optimization paradigm itself. To this end, we analyze and propose practical strategies to mitigate these hurdles in the following section:

• Latency: While fractional derivatives typically require historical memory, our implementation adopts a short-memory truncation approach, which approximates the Grünwald–Letnikov operator with a limited window. Empirical profiling indicates that for window sizes $M\in [30,90]$, the computational overhead is limited to 5–12% compared to standard optimizers. This is acceptable for medium-frequency strategies with minute-to-hour granularity.
• GPU Acceleration: We propose a GPU-based implementation of fractional derivative convolutions via parallel tensor operations. Specifically, the discretized fractional derivative at time t is computed using

  $$D^{\alpha }f\left(t\right)\approx \sum _{k=0}^M{w}_k^{\left(\alpha \right)}f(t-k),w_k^{\left(\alpha \right)}={(-1)}^k\left({\displaystyle \genfrac{}{}{0pt}{}{\alpha }{k}}\right).$$

  (113)

  where the weights $w_k^{\left(\alpha \right)}$ are precomputed and the convolution is implemented as a single `torch.nn.functional.conv1d()’ call on GPU. This significantly reduces the inference latency and makes the method compatible with online deployment.

  Table 8. Performance metrics across various stocks under different hyperparameter settings.

  Stock	Alpha	Mem Win	Sharpe	Accuracy	CPU (s)	GPU (s)	Speedup
  AAPL	0.3	60	0.1867	0.5151	0.0007	0.0006	1.10
  AAPL	0.3	90	0.1765	0.5128	0.0006	0.0006	0.98
  AAPL	0.5	60	0.1938	0.5180	0.0005	0.0007	0.79
  AAPL	0.5	90	0.1719	0.5068	0.0006	0.0006	0.96
  MSFT	0.3	60	0.1631	0.4892	0.0005	0.0005	0.98
  MSFT	0.3	90	0.1454	0.4842	0.0007	0.0007	0.98
  MSFT	0.5	60	0.1573	0.4892	0.0005	0.0005	1.06
  MSFT	0.5	90	0.1181	0.4842	0.0006	0.0006	1.06
  GOOGL	0.3	60	0.1237	0.5036	0.0005	0.0005	1.04
  GOOGL	0.3	90	0.1067	0.4902	0.0006	0.0007	0.88
  GOOGL	0.5	60	0.1255	0.4921	0.0007	0.0007	0.93
  GOOGL	0.5	90	0.0848	0.4947	0.0007	0.0006	1.04
  AMZN	0.3	60	−0.0108	0.5108	0.0005	0.0005	0.87
  AMZN	0.3	90	−0.0614	0.5038	0.0010	0.0011	0.89
  AMZN	0.5	60	−0.0092	0.5050	0.0005	0.0005	1.02
  AMZN	0.5	90	−0.0575	0.5008	0.0006	0.0007	0.95
  META	0.3	60	−0.0422	0.4719	0.0007	0.0008	0.92
  META	0.3	90	−0.0819	0.4707	0.0006	0.0007	0.89
  META	0.5	60	−0.0443	0.4705	0.0005	0.0005	1.04
  META	0.5	90	−0.0826	0.4677	0.0010	0.0007	1.36
  NVDA	0.3	60	0.0929	0.4950	0.0005	0.0005	1.05
  NVDA	0.3	90	0.0841	0.4917	0.0007	0.0007	0.88
  NVDA	0.5	60	0.0947	0.4950	0.0005	0.0005	1.01
  NVDA	0.5	90	0.0760	0.4932	0.0006	0.0007	0.86
  JPM	0.3	60	0.1313	0.4863	0.0005	0.0005	0.91
  JPM	0.3	90	0.1389	0.4932	0.0006	0.0006	0.99
  JPM	0.5	60	0.1295	0.4921	0.0005	0.0005	0.99
  JPM	0.5	90	0.1315	0.4977	0.0007	0.0007	1.04
  V	0.3	60	0.1098	0.4835	0.0007	0.0007	0.98
  V	0.3	90	0.0803	0.4902	0.0006	0.0006	1.07
  V	0.5	60	0.1020	0.4950	0.0005	0.0005	1.03
  V	0.5	90	0.0741	0.4887	0.0007	0.0006	1.14
  UNH	0.3	60	0.3906	0.4806	0.0006	0.0006	0.90
  UNH	0.3	90	0.3649	0.4842	0.0007	0.0007	0.98
  UNH	0.5	60	0.4259	0.4878	0.0006	0.0006	1.01
  UNH	0.5	90	0.3301	0.4812	0.0007	0.0006	1.06

  reports performance metrics across various stocks under different hyperparameter settings, especially CPU and GPU time; in this context, Speedup is a measure of how much faster one processor is compared to another. In this context, it compares CPU and GPU times and is defined as

  $$\mathrm{Speedup}={\displaystyle \frac{\mathrm{CPU}\mathrm{Time}}{\mathrm{GPU}\mathrm{Time}}}$$
  Across all tested stocks, CPU execution times are remarkably low, consistently below 0.0011 s. The performance difference between CPU and GPU is minimal, indicating that for lightweight financial computations (e.g., a Sharpe ratio and accuracy over short memory windows), GPU acceleration does not provide significant speedups. In several cases (e.g., rows 2, 10, and 13), the CPU is actually faster than the GPU, suggesting that the overhead associated with GPU data transfer may outweigh any benefits from parallelization. Notably, speedups rarely exceed 1.1×, with the highest GPU speedup observed for META (alpha 0.5, memory window 90) at 1.36×, which still reflects only a modest advantage. In conclusion, while GPU acceleration remains valuable for large-scale or deep-learning-based models, for fast statistical metrics like those presented here, CPU execution is highly efficient and often preferable.
• Hybrid Optimizer Pipelines: For ultra high-frequency trading (UHFT), we suggest a hybrid optimizer framework where fractional optimizers are used during slower retraining phases, while faster, conventional optimizers (e.g., RMSprop) are deployed during live signal execution. This ensures stability and long-term learning benefits of memory-aware updates without sacrificing real-time responsiveness.
• Data Integrity and Compliance: Fractional optimizers are orthogonal to the issues of data authenticity and regulation. Nonetheless, our pipeline is fully compatible with institutional-grade backtesting engines and real-time monitoring dashboards. Adopting industry-standard logging and audit protocols ensures compliance and traceability.

Together, these adaptations demonstrate that fractional optimizers are not only theoretically sound but can also be engineered for live deployment in medium- to high-frequency financial systems. We identify further directions such as volatility-aware $\alpha$ scheduling and streaming memory management as future enhancements to expand real-time viability.

Note: While our experiments focused on financial time series, the proposed methodology is designed to be domain-agnostic. This paper provides a general roadmap for transforming standard gradient-based optimizers (e.g., Adam, RMSprop, Adagrad) into their fractional-order counterparts using Caputo-based memory-aware formulations. This transformation and its practical implementation—based on the Grünwald–Letnikov approximation and the short-memory principle—are independent of the specific statistical properties of the time series.

Therefore, fractional optimizers could be readily adapted to other domains such as energy consumption forecasting, physiological signal modeling in healthcare, or climate and weather prediction, especially when memory effects and long-range dependencies are relevant. The potential of these methods to enhance convergence stability and trend sensitivity may also be beneficial in these contexts.

5. Conclusions

This study presents both theoretical advancements and empirical validation for the integration of fractional-order optimization techniques into machine learning systems, particularly for financial time-series forecasting using LSTM-based architectures. By embedding fractional derivatives into established gradient-based optimizers—namely Adam, RMSprop, and Adagrad—we introduce a family of fractional optimizers (Frac-Adam, Frac-RMSprop, Frac-Adagrad) that incorporate long-range memory and temporal dependencies directly into the learning dynamics.

The core contribution lies in leveraging the Caputo derivative formulation, which is more suitable than classical definitions for deep learning contexts. Caputo’s compatibility with standard initial conditions and its adherence to the short-memory principle make it particularly well-suited for recurrent neural networks such as LSTMs. Moreover, its inductive behavior—analogous to the Hurst exponent in fractional Brownian motion—provides a principled approach to introducing temporal smoothness and trend sensitivity into optimization.

From a computational perspective, while the Grünwald–Letnikov approximation introduces an overhead of $O\left(M\right)$ per update due to memory convolution, our runtime analysis indicates that moderate memory horizons ($M=30, 60, 90$) yield a favorable trade-off between performance gains and computational efficiency. This is particularly relevant in medium-frequency or offline settings, where learning stability and forecast robustness outweigh real-time processing constraints.

Experimental results demonstrate that fractional-order optimization can significantly improve forecasting performance under appropriate configurations. Sensitivity analyses reveal that a memory window of 60 days combined with a fractional order $\alpha \in \{0.5, 0.7\}$ yields the highest Sharpe ratios and cumulative returns, with notable robustness for relatively stable assets such as GOOGL and AAPL. In contrast, highly volatile stocks such as META and UNH exhibit performance degradation at higher fractional orders, underscoring the need for adaptive tuning. Directional accuracy metrics further suggest that higher $\alpha$ values (e.g., $\alpha =0.9$) may slightly improve trend detection at the expense of increased prediction variance.

In summary, fractional-order optimizers offer a flexible and biologically inspired framework for embedding memory effects into financial learning systems. However, their efficacy is nuanced and asset-dependent, highlighting the necessity for dynamic adjustment mechanisms. Future work should explore adaptive strategies for online tuning of $\alpha$ and memory windows, as well as extend fractional techniques to other machine learning paradigms—such as attention mechanisms, graph neural networks, and reinforcement learning—for a broader understanding of their advantages and limitations across diverse data environments.