PortRSMs: Learning Regime Shifts for Portfolio Policy

Abstract

This study proposes a novel Deep Reinforcement Learning (DRL) policy network structure for portfolio management called PortRSMs. PortRSMs employs stacked State-Space Models (SSMs) for the modeling of multi-scale continuous regime shifts in financial time series, striking a balance between exploring consistent distribution properties over short periods and maintaining sensitivity to sudden shocks in price sequences. PortRSMs also performs cross-asset regime fusion through hypergraph attention mechanisms, providing a more comprehensive state space for describing changes in asset correlations and co-integration. Experiments conducted on two different trading frequencies in the stock markets of the United States and Hong Kong show the superiority of PortRSMs compared to other approaches in terms of profitability, risk–return balancing, robustness, and the ability to handle sudden market shocks. Specifically, PortRSMs achieves up to a 0.03 improvement in the annual Sharpe ratio in the U.S. market, and up to a 0.12 improvement for the Hong Kong market compared to baseline methods.

1. Introduction

High-frequency trading often relies on the modeling of the distribution of short-term asset returns. Many studies have shown the serial correlation of volatility in asset returns (LeBaron, 1992; Shiller, 1990), indicating that the distribution of asset returns may exhibit consistency over a period of time. Basic time-series models can effectively capture these properties (Bauwens et al., 2006; Bollerslev et al., 1994). However, due to structural changes, the statistical characteristics of asset price series can completely change from one period to another. For example, after policy or macroeconomic shocks, the volatility of stock prices may undergo drastic changes, leading to the failure of basic time-series models. On the other hand, Regime Shift Model (RSM) paradigms address the shortcomings in basic time-series modeling by dividing time series into different states, effectively dealing with such shocks. Therefore, short-term asset return distributions are widely modeled by RSMs (Cai, 1994; Haas et al., 2004; So et al., 1998).

Optimized in the Deep Reinforcement Learning (DRL) formulation, the portfolio policy network aims to generate an effective policy to guide high-frequency portfolio rebalancing trading strategies (Jiang & Liang, 2017). It is essential to let the portfolio policy network model the distribution of asset returns in each trading period. In previous work, modeling was often done by neural network models (Jiang & Liang, 2017; X. Li et al., 2022; Wang et al., 2021; Xu et al., 2021). These methods can effectively extract consistent distribution properties in a short period, but they do not follow the RSM modeling paradigm, making hem sub-optimal in financial series modeling. Constructing a policy network withing the RSM paradigm has been challenging due to the lack of previous research using deep neural networks, which is essential for DRL.

With the recent breakthroughs in State-Space Model (SSM) research in the field of deep learning (Gu et al., 2020, 2022; Schiff et al., 2024), we can now use neural networks to model regime shifts in financial series. This enables us to come up with new policy network designs. In our work, we use stacked SSMs to model multi-scale continuous regime shifts present in financial time series, serving as the backbone of the DRL policy network. This method excels at balancing the exploration of consistent distribution properties over short periods and sensitivity to sudden shocks in price sequences. We also perform regime fusion between different assets through hypergraph attention mechanisms (HGAMs) (X. Li et al., 2022), providing a more comprehensive state space for describing changes in asset correlations and co-integration. These features give our method better performance compared to previous methods. We call our method PortRSMs, which is the abbreviation of “Portfolio RSMs”.

Our contributions are summarized as follows:

• We propose a new portfolio policy network structure with an RSM paradigm. The new structure can model regime shifts present in financial time series to strike a balance between exploring consistent distribution properties over short periods and maintaining sensitivity to sudden shocks for better portfolio decision-making.
• We propose a method for cross-asset regime fusion through HGAM, providing a more comprehensive state space for describing changes in asset correlations and co-integration.
• We conducted experiments on two different trading frequencies in the United States and Hong Kong stock markets. The experimental results showed the superiority of our method compared with other methods in terms of profitability, risk–return balancing, robustness, and ability to deal with sudden market shocks.

The remainder of this paper is organized as follows. Section 2 reviews related work in financial time-series modeling and deep reinforcement learning. Section 3 introduces the DRL framework for portfolio management and establishes the key mathematical notations used throughout the paper. Section 4 presents our proposed PortRSMs method, including the formulation of regime shift modeling and the regime fusion mechanism. Section 5 reports and analyzes the experimental results for multiple stock markets under different trading frequencies. Finally, Section 6 concludes the paper and discusses future research directions.

2. Related Work

Markowitz first introduced modern portfolio theory to design portfolios of assets with fixed weights using mean-variance analysis (Markowitz, 1952). The general portfolio algorithms are portfolio selection algorithms that rebalance the portfolio at the end of each trading period rather than using fixed weights. The general portfolio algorithms can be roughly divided into “follow the winner” (Agarwal et al., 2006; Helmbold et al., 1998), “follow the loser” (Borodin et al., 2003; Lai et al., 2018; B. Li & Hoi, 2012; B. Li et al., 2011b, 2012), and “pattern matching” (Györfi et al., 2006; B. Li et al., 2011a). Traditional strategies have a good explanatory and mathematical foundation, but they achieve suboptimal results in the long run, for they fail to model the complex dynamics of financial markets.

DRL approaches are a series of extremely strong “pattern matching” strategies that leverage the strong ability of deep learning for feature representation and pattern recognition. They have attracted significant attention in recent years. The EIIE methods (Jiang & Liang, 2017) first proposed a general framework to apply DRL for portfolio management, and they initially used the Temporal Convolutional Network (TCN) (Lea et al., 2017) and Long Short-Term Memory (LSTM) model (Hochreiter & Schmidhuber, 1997) as the policy network structure. The RAT (Xu et al., 2021) method first used the Transformer (Vaswani et al., 2017) structure as the policy network structure to extract complex information from financial time series. Those structures have also become commonly used in follow-up work (J. Li et al., 2023; X. Li et al., 2022; Wang et al., 2021). However, the above-mentioned structures are not exported by RSMs. We start with continuous-time RSMs, derive a method using SSMs (Gu et al., 2020, 2022; Schiff et al., 2024) as the policy network structure, and optimize it according to the specific needs of portfolio management tasks.

3. DRL for Portfolio Management

In this section, we briefly introduce the DRL method for the portfolio management problem, which provides the foundation for our work and establishes key mathematical symbols. Our formulation follows the foundational framework introduced in EIIE (Jiang & Liang, 2017) for DRL-based portfolio management, sharing all constraints.

3.1. Action

The portfolio vector represents the weights distributed to each asset. Let

$${\mathbf{w}}_k=(w_{k,0},w_{k,1},w_{k,2},\dots ,w_{k,m})\in {\mathbb{R}}_{+}^{m+1}[0,1],s.t.\sum _{i=0}^m{w}_{k,i}=1$$

(1)

be the portfolio vector before trading period k, where $i=0,1,2,\dots ,m$ indexes the assets and $k\in {\mathbb{N}}_{+}$ indexes the trading periods. Note that $w_{k,0}$ indicates the weight allocated to the risk-free asset (e.g., cash). ${\mathbf{w}}_k$ directly defines the action to take in trading period k in the framework. ${\mathbf{w}}_k$ then transitions to ${\mathbf{w}}_k^{\prime }\in {\mathbb{R}}_{+}^{m+1}[0,1]$, i.e., the portfolio vector after period k, due to changes in the prices of the assets.

3.2. State

In the framework, the state comprises market environment features and current asset holdings. Research in investment science indicates that due to market inefficiency, asset price data over a historical period has a certain predictive power for future price changes (Bustos & Pomares-Quimbaya, 2020; Jegadeesh & Titman, 2023). Therefore, the most intuitive market features are asset price time series. Asset price time series are sampled with equal intervals, using techniques like candlestick charts. Let $t:=kT\in {\mathbb{N}}_{+}$ represent the sample timestamps, where $T\in {\mathbb{N}}_{+}$ represents how many timestamps there are in a trading period. The portfolio vector after period $k-1$, represented as ${\mathbf{w}}_{k-1}^{\prime }$, is also included in the state, since adjusting it to ${\mathbf{w}}_k$ incurs transaction fees. In summary, the state in the framework is $s_t=({\mathcal{P}}_k,{\mathbf{w}}_{k-1}^{\prime })$, where

• ${\mathcal{P}}_k:=({\mathbf{P}}_{kT-l+1},\dots ,{\mathbf{P}}_{kT})\in {\mathbb{R}}_{+}^{l\times (m+1)\times 4}$ are the $l\in {\mathbb{N}}_{+}$ latest samples in the price series in trading period k;
• ${\mathbf{P}}_t:=({\mathbf{P}}_{t,0},\dots ,{\mathbf{P}}_{t,m})\in {\mathbb{R}}_{+}^{(m+1)\times 4}$ represents the price samples of all assets at timestamp t;
• ${\mathbf{P}}_{t,i}:=(p_{t,i}^O,p_{t,i}^H,p_{t,i}^L,p_{t,i}^C)\in {\mathbb{R}}_{+}^4$ indicates the opening, highest, lowest, and closing prices of asset i in the sample with timestamp t.

Initially, ${\mathbf{w}}_0^{\prime }=(1,0,0,\dots ,0)$, i.e., the situation where all funds are in the risk-free asset.

3.3. State Transition Function

Let $r_{k,i}\in {\mathbb{R}}_{+}$ be the price change ratio of asset i in trading period k, and let ${\mathbf{r}}_k:=(r_{k,1},r_{k,2},\dots ,r_{k,m})\in {\mathbb{R}}_{+}^{m+1}$ represent the price change ratios of all assets in trading period k, where $r_{k,i}:={\displaystyle \frac{p_{(k+1)T,i}^C}{p_{kT,i}^C}}$ represents the price change ratio of a certain asset. The price change ratios (${\mathbf{r}}_k$) can be predicted from ${\mathcal{P}}_k$; namely, there exists a probability model, i.e., ${\mathcal{T}}^{\prime }\left({\mathbf{r}}_k\right|{\mathcal{P}}_k)$. Given ${\mathbf{r}}_k$, the portfolio vector (${\mathbf{w}}_k$) deterministically becomes ${\mathbf{w}}_k^{\prime }$. Concurrently, the new asset price series (${\mathbf{P}}_k$) generated deterministically by the price changes becomes observable. The state transition model ($\mathcal{T}\left(s_{k+1}\right|s_k,a_k):=\mathcal{T}\left(({\mathbf{P}}_k,{\mathbf{w}}_k^{\prime })\right|({\mathcal{P}}_k,{\mathbf{w}}_{k-1}^{\prime }),{\mathbf{w}}_k)$) can then be readily obtained from the probability model (${\mathcal{T}}^{\prime }$) and all the aforementioned deterministic relationships.

3.4. Reward Function

Under portfolio vector ${\mathbf{w}}_k$, the return generated by the price changes (${\mathbf{r}}_k$) is rewarded. Note that the transaction fees incurred as a result of adjusting the portfolio from ${\mathbf{w}}_{k-1}^{\prime }$ to ${\mathbf{w}}_k$ should be deducted from the reward. Let the transaction fee ratio be $c\in \mathbb{R}[0,1]$; then, the return can be calculated as follows:

$$r_t:={\mathbf{w}}_k^{\mathrm{T}}{\mathbf{r}}_k-c\left|\right|{\mathbf{w}}_k-{\mathbf{w}}_{k-1}^{\prime }{\left|\right|}_1,$$

(2)

where the first term is the initial return and the second term represents the trading costs incurred from transaction fees. The formulation of the reward function in trading period k is $R(s_k,a_k)=log(r_k+1)$, where $a_k={\mathbf{w}}_k$. Here, the return is taken as the logarithm to ensure the additivity of the reward function over time.

3.5. Deterministic Policy Gradient

In deterministic policy gradient algorithms, the policy network (${\pi }_{\theta }\left(s\right)$) is a neural network that maps the current state to an action (Silver et al., 2014). The policy network is parameterized by $\theta$, which refers to the trainable weights and biases of the neural network. During training, the reward function is differentiated directly, and gradient ascent is applied to update $\theta$ with a learning rate of $\alpha \in {\mathbb{R}}^{+}$, according to $\theta \to \theta +\alpha \nabla R(s,{\pi }_{\theta }\left(s\right))$, where $\nabla R(s,{\pi }_{\theta }\left(s\right))$ denotes the gradient of the reward function with respect to the policy parameters ($\theta$).

4. PortRSMs

In this section, we introduce how we establish the PortRSMs method step by step, including the mathematical form of RSMs and regime fusion, as well as the portfolio weight-generating method. Figure 1 is a data flow diagram providing an overview of PortRSMs.

4.1. SSMs

Recent research used time-series modeling with SSMs to describe regime shifts (Gu et al., 2020). For continuous-time signals, SSMs have the following form:

$$\begin{array}{cc}\hfill & \left\{\begin{array}{cc}{u}^{\prime }\left(t\right)=\mathbf{A}u\left(t\right)+\mathbf{B}x\left(t\right),\hfill & \hfill \left(TransitionModel\right)\\ y\left(t\right)=\mathbf{C}u\left(t\right)+\mathbf{D}x\left(t\right),\hfill & \hfill \left(EmissionModel\right)\end{array}\right.\hfill \\ & where,u\left(t\right)\in {\mathbb{R}}^{h\times 1},x\left(t\right)\in {\mathbb{R}}^{d\times 1},y\left(t\right)\in {\mathbb{R}}^{d\times 1},\hfill \\ & \mathbf{A}\in {\mathbb{R}}^{h\times h},\mathbf{B}\in {\mathbb{R}}^{h\times d},\mathbf{C}\in {\mathbb{R}}^{d\times h},\mathbf{D}\in {\mathbb{R}}^{d\times d}\hfill \end{array}$$

(3)

where $u\left(t\right)$ represents the hidden state with hidden dimension $h\in {\mathbb{R}}_{+}$. $x\left(t\right)$ describes the observable instantaneous information at time t, while $y\left(t\right)$ is the instantaneous information that cannot be observed up to time t. Information is described with the $d\in {\mathbb{R}}_{+}$ dimension vector. In addition to $\mathbf{A}$ for describing inherent transitions of hidden states, the transition model also uses matrix $\mathbf{B}$ to describe how current observable instantaneous information affects hidden states. The emission model then maps $u\left(t\right)$ and $x\left(t\right)$ to current $y\left(t\right)$ through matrices $\mathbf{C}$ and $\mathbf{D}$.

The follow-up research improved SSMs. On one hand, $\mathbf{A}$ and $\mathbf{D}$ are considered trainable neural network parameters (Gu et al., 2022). On the other hand, $\mathbf{B}$ and $\mathbf{C}$ are considered time-variant parameters (${\mathbf{B}}_t$ and ${\mathbf{C}}_t$, respectively)to improve performance (Schiff et al., 2024). This makes SSMs a linear time-variant system where ${\mathbf{B}}_t$ and ${\mathbf{C}}_t$ are obtained through nonlinear transformation:

$$\left\{\begin{array}{c}{\mathbf{B}}_t=\delta \left({\mathsf{\Gamma }}_B{x}^{\top }\left(t\right)\right),\hfill \\ {\mathbf{C}}_t=\delta \left({\mathsf{\Gamma }}_C{x}^{\top }\left(t\right)\right),\hfill \end{array}\right.where,{\mathsf{\Gamma }}_B,{\mathsf{\Gamma }}_C\in {\mathbb{R}}^{h\times d}$$

(4)

where $\delta (.)$ is an SiLU activation function1 and ${\mathsf{\Gamma }}_B$ and ${\mathsf{\Gamma }}_C$ are learnable projection matrices.

4.2. RSMs in Price Series

RSMs, based on the improvement of hidden Markov models, are widely used in modeling asset price change ratio distributions (Cai, 1994; Haas et al., 2004; So et al., 1998). They take instantaneous prices as the input information and the instantaneous price change ratios as the output.

In our work, we use SSMs to model the continuous regime shifts from the discretized price series. Therefore, RSMs are defined as follows:

$$\begin{array}{cc}\hfill & x_{t,i}=\delta \left({\mathsf{\Gamma }}_P{\mathbf{P}}_{t,i}\right),\left\{\begin{array}{c}{u}_{t,i}={\tilde{\mathbf{A}}}_{t,i}{u}_{t-1,i}+{\tilde{\mathbf{B}}}_{t,i}{x}_{t,i},\hfill \\ y_{k,i}=\delta \left({\mathsf{\Gamma }}_C{x}_{t,i}\right)u_{t,i}+\mathbf{D}{x}_{t,i},\hfill \end{array}\right.\hfill \\ \hfill & s.t.t=kT,{\tilde{\mathbf{A}}}_{t,i}=exp\left({\Delta }_{t,i}\mathbf{A}\right),{\tilde{\mathbf{B}}}_{t,i}={\left({\Delta }_{t,i}\mathbf{A}\right)}^{-1}(exp\left({\Delta }_{t,i}{\mathbf{A}}_{t,i}\right)-\mathbf{I}){\Delta }_{t,i}{\mathbf{B}}_{t,i},\hfill \\ & {\mathbf{B}}_{t,i}=\delta \left({\mathsf{\Gamma }}_B{x}_{t,i}\right),{\Delta }_{t,i}=\delta \left({\mathsf{\Gamma }}_{\Delta }{x}_{t,i}\right)\hfill \\ \hfill & where.{\mathsf{\Gamma }}_P\in {\mathbb{R}}^{d\times 4},{\mathsf{\Gamma }}_{\Delta }\in {\mathbb{R}}^{h\times d}\hfill \end{array}$$

(5)

where ${\mathsf{\Gamma }}_P$ is a projection matrix to restore input information ($x_{t,i}$) from ${\mathbf{P}}_{t,i}$, as defined in Section 3.2, while $y_{k,i}$ is a descriptor vector for the distribution characteristics of $r_{k,i}$, as defined in Section 3.3. ${\tilde{\mathbf{A}}}_{\tau ,i}$ and ${\tilde{\mathbf{B}}}_{\tau ,i}$ represent discretized versions of $\mathbf{A}$ and ${\mathbf{B}}_{\tau ,i}$. In the case of zero-order hold sampling, the relationships between ${\tilde{\mathbf{A}}}_{\tau ,i},{\tilde{\mathbf{B}}}_{\tau ,i}$ and $\mathbf{A},{\mathbf{B}}_{\tau ,i}$ have been shown in existing research (Gu et al., 2022; Schiff et al., 2024). ${\Delta }_{\tau ,i}$ describes the time scale that one sample interval maps to in continuous perspective. Note that ${\Delta }_{\tau ,i}$ is also a time-variant parameter obtained from $x_{t,i}$ by the non-linear transformation by projection matrix ${\mathsf{\Gamma }}_{\Delta }$ and the activation function. This property further enhances its ability to model the widespread fractal properties in financial time series (Evertsz, 1995; Ni et al., 2011; Peters, 1989). The time-variant ${\Delta }_{\tau ,i}$ can map different dynamics from discrete perspectives to the same dynamics in continuous perspectives to model the continuous regime shifts. During training, optimization is carried out for parameters $\mathbf{A},\mathbf{D},{\mu }_i,{\mathsf{\Gamma }}_B,{\mathsf{\Gamma }}_C,{\mathsf{\Gamma }}_{\Delta }$, and ${\mathsf{\Gamma }}_P$.

Notice that the absolute price ranges of different assets at different times may vary greatly. On the one hand, this can lead to similar dynamics being considered completely different, further causing a decrease in training data efficiency. On the other hand, it can result in uneven gradient values, making convergence difficult during training. Therefore, data normalization is important in data pre-processing. We adopt the same approach used in previous work (Jiang & Liang, 2017; Xu et al., 2021), using the latest close price in the state representation defined in Section 3.2, i.e., $p_{kT}^C$, as the denominator to normalize all price data in ${\mathcal{P}}_k$. To apply this method, in each trading period (k), the SSM is re-executed on ${\mathcal{P}}_k$, i.e., all samples of the latest l timestamps. The calculation can be performed in a high-speed way on modern hardware by converting recursion to convolution (Schiff et al., 2024). In each calculation, $u_{kT-l,i}$ is initialized as a zero vector.

4.3. Stacked SSMs

To further improve performance, considering the existence of multiple regime shifts of different scales in the financial time series, we use stacked SSMs to model them. The formulation of RSMs modeled by N-layer SSMs is expressed as follows:

$$\begin{array}{cc}\hfill & x_{t,i}^1=\delta \left({\mathsf{\Gamma }}_P{\mathbf{P}}_{t,i}\right),\left\{\begin{array}{cc}{u}_{t,i}^1={\tilde{\mathbf{A}}}_{t,i}^1{u}_{t,i}^1+{\tilde{\mathbf{B}}}_{t,i}^1{x}_{t,i}^1,\hfill & \\ x_{k,i}^2=y_{t,i}^1=\delta \left({\mathsf{\Gamma }}_C^1{x}_{t,i}\right)u_{t,i}^1+{\mathbf{D}}_{t,i}^1{x}_{t,i}^1,\hfill \end{array}\right.\hfill \\ & \left\{\begin{array}{cc}{u}_{t,i}^2={\tilde{\mathbf{A}}}_{t,i}^2{u}_{t,i}^2+{\tilde{\mathbf{B}}}_{t,i}^2{x}_{t,i}^2,\hfill & \\ x_{t,i}^3=y_{t,i}^2=\delta \left({\mathsf{\Gamma }}_C^2{x}_{t,i}\right)u_{t,i}^2+{\mathbf{D}}_{t,i}^2{x}_{t,i}^2,\hfill \end{array}\right.\dots \hfill \\ \hfill & \left\{\begin{array}{cc}{u}_{t,i}^N={\tilde{\mathbf{A}}}_{t,i}^N{u}_{t,i}^N+{\tilde{\mathbf{B}}}_{t,i}^N{x}_{t,i}^N,\hfill & \\ y_{k,i}^N=y_{t,i}^N=\delta \left({\mathsf{\Gamma }}_C^N{x}_{t,i}\right)u_{t,i}^N+{\mathbf{D}}_{t,i}^N{x}_{t,i}^N,\hfill \end{array}\right.\hfill \\ \hfill & s.t.t=kT,{\tilde{\mathbf{A}}}_{t,i}^n=exp\left({\Delta }_{t,i}^n{\mathbf{A}}^n\right),{\tilde{\mathbf{B}}}_{t,i}^n={\left({\Delta }_{t,i}^n{\mathbf{A}}^n\right)}^{-1}(exp\left({\Delta }_{t,i}^n{\mathbf{A}}_{t,i}^n\right)-\mathbf{I}){\Delta }_{t,i}^n{\mathbf{B}}_{t,i}^n,\hfill \\ & {\mathbf{B}}_{t,i}^n=\delta \left({\mathsf{\Gamma }}_B^n{x}_{t,i}^n\right),{\Delta }_{t,i}^n=\delta \left({\mathsf{\Gamma }}_{\Delta }^n{x}_{t,i}^n\right),n=1,2,\dots ,N\hfill \end{array}$$

(6)

Note that parameters are not shared between layers. In this case, we take the output ($y_{t,i}^{n-1}$) of the $n-1$th SSM layer as the input ($x_{t,i}^n$) of the nth SSM layer. Each SSM layer can model regime shifts over the regimes modeled by the previous layer on a more abstract scale, thereby mining more stable patterns and longer dependencies.

4.4. Hypergraph Attention for Cross-Asset Regime Fusion

So far, we have only discussed the situation of individual assets. In the portfolio management task, multiple asset price series should be modeled simultaneously. The correlation and co-integration between these series can shift over time. Modeling these properties has been shown to be crucial for the learning of portfolio policy in existing work (X. Li et al., 2022; Shi et al., 2022; Soleymani & Paquet, 2021; Xu et al., 2021).

We include considerations of relevance and co-integration in the RSMs by utilizing HGAM (X. Li et al., 2022). The HGAM aims to learn the differing importance of asset neighbors for information merging with the attention mechanism (Vaswani et al., 2017). For stock i and its neighbor ($j=0,1,\dots ,m$), quantifying the degree to which i is related to j based on the output ($y_{k,i}^n$) of the nth SSM layer, i.e.,

$$D(y_{k,i}^n,y_{k,j}^n)=\delta \left({\mathbf{b}}^n[{\mathsf{\Gamma }}_R^n{y}_{k,i}^n,{\mathsf{\Gamma }}_R^n{y}_{k,j}^n]\right),$$

(7)

where ${\mathsf{\Gamma }}_R^n\in {\mathbf{R}}^{d\times 1}$ is a projection matrix to be learned, ${\mathbf{b}}^n\in {\mathbf{R}}^{1\times d}$ is a shared attention vector, d is the model dimension defined in Section 4.2, $[.]$ denotes the concatenation operation, $\delta (.)$ denotes a nonlinear activation function like LeakyReLU2. Then, the softmax function is applied to obtain the importance weight (${\alpha }_{ij}$):

$${\alpha }_{ij}={\displaystyle \frac{exp\left(D(y_{k,i}^n,y_{k,j}^n)\right)}{{\sum }_{v=0,1,\dots ,m}exp\left(D(y_{k,i}^n,y_{k,v}^n)\right)}}.$$

(8)

After that, $y_{t,i}^n$ is aggregated across assets as follows:

$$y_{k,i}^n\to \delta \left(\sum _{j=0,1,\dots ,m}{\alpha }_{ij}{P}^n{y}_{k,j}^n\right)$$

(9)

In the case of stacked SSMs, information from different assets is aggregated layer by layer, thereby achieving cross-asset regime fusion, providing more comprehensive state spaces for the description of changes in asset correlations and co-integration.

4.5. Portfolio Generation

The last SSM-layer output ($y_{k,i}^N\in {\mathbb{R}}^{d\times 1}$) is a descriptor of $r_{k,i}$ distribution characteristics. ${\mathcal{T}}^{\prime }$, as defined in Section 3.3, can be established following previous practices (Jiang & Liang, 2017; Xu et al., 2021). The formulation is expressed as follows:

$$\begin{array}{cc}\hfill & {\mathbf{w}}_{\mathbf{k}}=Softmax\left([b,{\left({\mathsf{\Gamma }}_{\mathbf{w}}[{\mathbf{w}}_{k-1}^{\prime },{\mathbf{y}}_{\mathbf{k}}]\right)}^{\top }]\right),\hfill \\ \hfill & where.{\mathbf{y}}_{\mathbf{k}}={(y_{k,1}^N,y_{k,2}^N,\dots ,y_{k,m}^N)}^{\top }\in {\mathbb{R}}^{d\times m},{\mathbf{w}}_{k-1}^{\prime }\in {\mathbb{R}}^{1\times m},b\in {\mathbb{R}}^{1\times 1},{\mathsf{\Gamma }}_{\mathbf{w}}\in {\mathbb{R}}^{(d+1)\times 1}\hfill \end{array}$$

(10)

where [.] denotes the concatenation operation in the first dimension. After concatenating the distribution descriptors and portfolio vector (${\mathbf{w}}_{k-1}^{\prime }$, without risk-free asset weights), the absolute score of each asset can be obtained by applying a projection matrix (${\mathsf{\Gamma }}_{\mathbf{w}}$). Finally, ${\mathbf{w}}_{\mathbf{k}}$ can be generated from absolute scores by concatenating a cash bias (b) and applying a softmax function on the first dimension.

5. Experiments

5.1. Experimental Settings

5.1.1. Datasets

We conducted experiments with two different trading frequencies (i.e., 1 day and 1 week as the duration of the trading period) in the United States and Hong Kong stock markets. Specifically, we used the Yahoo Finance API (Yahoo, 2025) and AKShare API (King, 2019) to collect data on the Dow Jones Industrial Average (DJIA) and Hang Seng Index (HSI) constituents from 1 January 2004 to 1 January 2024. The data are sampled in daily frequency, which means $T=1$ when the trading period is 1 day and $T=5$ when the trading period is 1 week. Stocks with more than 70% missing data were excluded. When reporting the experimental results, we use DJIA1d/DJIA1w and HSI1d/HSI1w to represent the names of the dataset. Furthermore, 1d means 1 day as the duration of the trading period, while 1w means 1 week. For trading fees, we used an industry-standard round-trip cost of 0.06%. The training set and test set are divided chronologically in a ratio of 4:1.

5.1.2. Methods for Comparison

We use the constant rebalanced portfolio (CRP) and buy-and-hold (BAH) method as a benchmark. CRP allocates equal funds to all assets at all times, representing average market performance. BAH allocates equal funds to all assets in the first trading period, without any further trading actions. We also compare our method with traditional portfolio management algorithms (EG (Helmbold et al., 1998), OLMAR (B. Li & Hoi, 2012), RMR (Huang et al., 2016), BNN (Györfi et al., 2006), and CORN (B. Li et al., 2011a)), as well as state-of-the-art DRL-based methods with different policy network structures (bRNN EIIE (Jiang & Liang, 2017), CNN EIIE (Jiang & Liang, 2017), RAT (Xu et al., 2021), HGAM (X. Li et al., 2022), and LSRE-CAAN (J. Li et al., 2023)).

5.1.3. Evaluation Metric

Following previous works (Jiang & Liang, 2017; Wang et al., 2021; Xu et al., 2021), we use three metrics to evaluate performance: the annualized return (AR), annualized Sharpe ratio (ASR), and annualized Calmar ratio (ACR). AR measures compound annual portfolio growth over a period, with a higher AR indicating profitability. ASR measures volatility-adjusted return, with a higher ASR reflecting better risk–return balancing. ACR is a risk–return balancing metric similar to ASR that measures return adjusted by the maximum draw-down in the profit. We used five different random seeds $\{0,64,128,256,512\}$ for each experimental group to conduct repeated experiments and reported the mean and standard deviation of the results to reduce the randomness of the results and study the robustness of training. We plot the graph of cumulative log return over time in the testing period for qualitative analysis.

5.1.4. Implementation Details

Training was performed using the Adam optimizer on a single NVIDIA RTX A6000 GPU, with the learning rate set to $\alpha =1.92\times {10}^{-5}$ and a batch size of 32. We used a model dimension of $d=36$ and hidden dimension of $h=16$. By default, we use an SSM layer number of $N=2$ and sample count of $l=50$. Cross-asset regime fusion is also applied by default.

5.2. Ablation Studies

5.2.1. Modeling Paradigm

We conducted experiments by replacing the SSM module with LSTM (Hochreiter & Schmidhuber, 1997), TCN (Lea et al., 2017), and Transformer (Vaswani et al., 2017) decoder modules with the same model dimension. As for the quantitative results, we present the AR in

Table 1. AR performance comparison of different modules across various datasets, using different time-series sample counts (l). Bold values indicate the best result in each group of comparisons, and underlined values indicate the suboptimal result.

Dataset	Module	$\mathit{l}=20$	$\mathit{l}=50$	$\mathit{l}=100$
DJIA1d	LSTM	0.104 ± 0.003	0.105 ± 0.004	0.104 ± 0.003
	TCN	0.099 ± 0.011	0.046 ± 0.036	0.093 ± 0.054
	Transformer	0.104 ± 0.001	0.109 ± 0.003	0.119 ± 0.013
	SSMs	0.109 ± 0.008	0.121 ± 0.012	0.127 ± 0.013
DJIA1w	LSTM	0.124 ± 0.006	0.127 ± 0.008	0.131 ± 0.009
	TCN	0.158 ± 0.022	0.171 ± 0.035	0.298 ± 0.078
	Transformer	0.132 ± 0.010	0.123 ± 0.003	0.156 ± 0.029
	SSMs	0.161 ± 0.015	0.170 ± 0.026	0.157 ± 0.017
HSI1d	LSTM	0.218 ± 0.034	0.175 ± 0.036	0.201 ± 0.085
	TCN	0.236 ± 0.020	0.211 ± 0.054	0.160 ± 0.086
	Transformer	0.232 ± 0.034	0.179 ± 0.020	0.112 ± 0.052
	SSMs	0.296 ± 0.088	0.220 ± 0.029	0.247 ± 0.122
HSI1w	LSTM	0.225 ± 0.012	0.231 ± 0.015	0.233 ± 0.018
	TCN	0.215 ± 0.047	0.245 ± 0.048	0.183 ± 0.087
	Transformer	0.251 ± 0.023	0.180 ± 0.071	0.145 ± 0.064
	SSMs	0.292 ± 0.048	0.279 ± 0.048	0.251 ± 0.076

, ASR in

Table 2. ASR performance comparison of different modules across various datasets, using different time-series sample counts (l). Bold values indicate the best result in each group of comparisons, and underlined values indicate the suboptimal result.

Dataset	Module	$\mathit{l}=20$	$\mathit{l}=50$	$\mathit{l}=100$
DJIA1d	LSTM	0.558 ± 0.006	0.559 ± 0.010	0.557 ± 0.007
	TCN	0.506 ± 0.067	0.394 ± 0.136	0.532 ± 0.077
	Transformer	0.558 ± 0.002	0.565 ± 0.004	0.586 ± 0.028
	SSMs	0.573 ± 0.025	0.613 ± 0.036	0.630 ± 0.040
DJIA1w	LSTM	0.621 ± 0.011	0.619 ± 0.008	0.616 ± 0.008
	TCN	0.532 ± 0.081	0.514 ± 0.055	0.681 ± 0.099
	Transformer	0.623 ± 0.003	0.617 ± 0.005	0.642 ± 0.019
	SSMs	0.646 ± 0.024	0.662 ± 0.018	0.665 ± 0.014
HSI1d	LSTM	0.695 ± 0.066	0.635 ± 0.051	0.676 ± 0.121
	TCN	0.724 ± 0.044	0.652 ± 0.088	0.565 ± 0.179
	Transformer	0.776 ± 0.040	0.710 ± 0.037	0.503 ± 0.155
	SSMs	0.808 ± 0.142	0.768 ± 0.040	0.792 ± 0.147
HSI1w	LSTM	0.756 ± 0.013	0.779 ± 0.027	0.785 ± 0.027
	TCN	0.718 ± 0.128	0.714 ± 0.062	0.613 ± 0.214
	Transformer	0.854 ± 0.049	0.679 ± 0.193	0.537 ± 0.136
	SSMs	0.868 ± 0.060	0.899 ± 0.065	0.793 ± 0.113

, and ACR in

Table 3. ACR performance comparison of different modules across various datasets, using different time-series sample counts (l). Bold values indicate the best result in each group of comparisons, and underlined values indicate the suboptimal result.

Dataset	Module	$\mathit{l}=20$	$\mathit{l}=50$	$\mathit{l}=100$
DJIA1d	LSTM	0.307 ± 0.004	0.307 ± 0.006	0.306 ± 0.004
	TCN	0.235 ± 0.073	0.164 ± 0.122	0.325 ± 0.119
	Transformer	0.305 ± 0.001	0.309 ± 0.002	0.321 ± 0.015
	SSMs	0.313 ± 0.015	0.345 ± 0.025	0.354 ± 0.029
DJIA1w	LSTM	0.387 ± 0.017	0.388 ± 0.014	0.385 ± 0.066
	TCN	0.364 ± 0.139	0.393 ± 0.116	0.724 ± 0.279
	Transformer	0.405 ± 0.011	0.399 ± 0.007	0.473 ± 0.079
	SSMs	0.499 ± 0.075	0.555 ± 0.101	0.515 ± 0.066
HSI1d	LSTM	0.567 ± 0.102	0.488 ± 0.058	0.560 ± 0.182
	TCN	0.590 ± 0.08	0.488 ± 0.113	0.431 ± 0.210
	Transformer	0.661 ± 0.069	0.604 ± 0.060	0.354 ± 0.181
	SSMs	0.728 ± 0.217	0.688 ± 0.068	0.762 ± 0.245
HSI1w	LSTM	0.804 ± 0.053	0.884 ± 0.058	0.881 ± 0.060
	TCN	0.676 ± 0.262	0.688 ± 0.131	0.478 ± 0.225
	Transformer	0.979 ± 0.112	0.588 ± 0.252	0.346 ± 0.158
	SSMs	1.189 ± 0.142	1.282 ± 0.182	0.830 ± 0.160

. SSM achieves the best performance on most datasets with most of the time-series samples (l).

Insights into the performance of different modules can be gained based on how their performance changes with l and the stability of training performance with different random seeds. Note that when using the LSTM module, although the algorithm’s performance remains stable for most of the l, it cannot surpass our SSMs’ performance, for it has sub-optimal modeling paradigms relative to RSMs. TCN shows a larger variance in performance metrics compared to other modules when trained with different random seeds, likely due to the assignment of equal weights to each sample during learning, which can lead to overfitting on noise. The Transformer’s performance is highly unstable as l varies, especially on the HSI dataset, where its performance sharply declines as l increases because the attention mechanism tends to focus on over-long dependencies during training, affecting sensitivity toward sudden shocks.

With the RSM paradigm, the SSM module exhibits better stability as l changes and lower variance when trained with different random seeds compared to other models, indicating a balance between the extraction of consistent distribution properties and sensitivity towards sudden shocks.

Figure 2 and Figure 3 show a performance comparison of different modules from a qualitative analysis perspective. The SSM module has a stable and rapidly growing return curve in most periods compared with other modules. Specifically, as shown in Figure 3, during the COVID outbreak period, when overall market prices were falling rapidly, the SSM module did not incur excessive losses, and during periods of price rebounding, the SSM module quickly identified profit opportunities in the market, resulting in higher returns than other methods.

5.2.2. Stacked SSMs

In the ablation experiment, we analyze the role of each component in the proposed algorithm. First, we compare the effect of the SSM module with the RSM paradigm and other existing time-series modeling modules with other paradigms by replacing each layer with other modules with the same model dimension. Secondly, we study the effect of stacked SSMs by changing the number of SSM layers. Finally, we compare the performance of algorithms with and without cross-asset regime fusion to illustrate its effectiveness.

We study the relationship between different numbers of SSM layers and performance when the time-series sample count (l) varies. As for the performance metrics, we present the AR in

Table 4. Performance comparison of different numbers of stacked SSM layers across various datasets, using different time-series sample count (l), as evaluated by AR. Bold values indicate the best result in each group of comparisons, and underlined values indicate the suboptimal result.

Dataset	Layer Num	$\mathit{l}=20$	$\mathit{l}=50$	$\mathit{l}=100$
DJIA1d	1	0.100 ± 0.010	0.110 ± 0.014	0.114 ± 0.011
	2	0.109 ± 0.008	0.121 ± 0.012	0.127 ± 0.013
	3	0.107 ± 0.005	0.116 ± 0.015	0.125 ± 0.019
DJIA1w	1	0.140 ± 0.003	0.156 ± 0.006	0.164 ± 0.008
	2	0.161 ± 0.015	0.170 ± 0.026	0.157 ± 0.017
	3	0.142 ± 0.030	0.187 ± 0.052	0.190 ± 0.048
HSI1d	1	0.273 ± 0.060	0.186 ± 0.022	0.181 ± 0.022
	2	0.296 ± 0.088	0.220 ± 0.029	0.247 ± 0.122
	3	0.355 ± 0.096	0.262 ± 0.082	0.317 ± 0.164
HSI1w	1	0.284 ± 0.007	0.341 ± 0.020	0.315 ± 0.031
	2	0.292 ± 0.048	0.279 ± 0.048	0.251 ± 0.076
	3	0.282 ± 0.040	0.294 ± 0.071	0.272 ± 0.029

, ASR in

Table 5. Performance comparison of different numbers of stacked SSM layers across various datasets, using different time-series sample counts (l), as evaluated by ASR. Bold values indicate the best result in each group of comparisons, and underlined values indicate the suboptimal result.

Dataset	Layer Num	$\mathit{l}=20$	$\mathit{l}=50$	$\mathit{l}=100$
DJIA1d	1	0.559 ± 0.004	0.582 ± 0.014	0.589 ± 0.020
	2	0.573 ± 0.025	0.613 ± 0.036	0.630 ± 0.040
	3	0.569 ± 0.014	0.596 ± 0.046	0.614 ± 0.054
DJIA1w	1	0.635 ± 0.008	0.654 ± 0.006	0.661 ± 0.006
	2	0.646 ± 0.024	0.662 ± 0.018	0.665 ± 0.014
	3	0.639 ± 0.037	0.658 ± 0.026	0.670 ± 0.023
HSI1d	1	0.763 ± 0.093	0.713 ± 0.043	0.685 ± 0.036
	2	0.808 ± 0.142	0.768 ± 0.040	0.792 ± 0.147
	3	0.948 ± 0.141	0.818 ± 0.105	0.880 ± 0.213
HSI1w	1	0.859 ± 0.012	0.950 ± 0.033	0.886 ± 0.041
	2	0.868 ± 0.060	0.899 ± 0.065	0.793 ± 0.113
	3	0.856 ± 0.069	0.905 ± 0.052	0.827 ± 0.041

, and ACR in

Table 6. Performance comparison of different numbers of stacked SSM layers across various datasets, using different time-series sample counts (l), as evaluated by ACR. Bold values indicate the best result in each group of comparisons, and underlined values indicate the suboptimal result.

Dataset	Layer Num	$\mathit{l}=20$	$\mathit{l}=50$	$\mathit{l}=100$
DJIA1d	1	0.309 ± 0.004	0.310 ± 0.012	0.316 ± 0.010
	2	0.313 ± 0.015	0.345 ± 0.025	0.354 ± 0.029
	3	0.314 ± 0.006	0.336 ± 0.032	0.343 ± 0.041
DJIA1w	1	0.428 ± 0.016	0.483 ± 0.014	0.508 ± 0.026
	2	0.499 ± 0.075	0.555 ± 0.101	0.515 ± 0.066
	3	0.476 ± 0.089	0.604 ± 0.190	0.611 ± 0.169
HSI1d	1	0.633 ± 0.128	0.603 ± 0.080	0.553 ± 0.059
	2	0.728 ± 0.217	0.688 ± 0.068	0.762 ± 0.245
	3	0.914 ± 0.210	0.780 ± 0.186	0.908 ± 0.306
HSI1w	1	1.159 ± 0.047	1.479 ± 0.088	1.206 ± 0.178
	2	1.189 ± 0.142	1.282 ± 0.182	0.830 ± 0.178
	3	1.209 ± 0.220	1.291 ± 0.278	0.979 ± 0.200

. Note that on the DJIA1d and HSI1d datasets, the choice of layer number is independent of l. This indicates that in high-frequency trading, it is only necessary to model very short-term regime shifts, so increasing l does not lead to significant changes in regime-shift information. In contrast, on the DJIA1w dataset, increasing l results in richer regime-shift information for portfolio policy generation. Therefore, using more layers for RSM modeling results in better performance. Finally, note that on the HSI1w dataset, although performance improves with three layers compared to using two SSM layers as l increases, better performance is achieved with just one layer instead. This is because the HSI1w dataset exhibits a clear mean-reversion property that can be modeled effectively with only one SSM layer. The mean-reversion nature of HSI1w is also shown in the results in Section 5.3.

5.2.3. Cross-Asset Regime Fusion

We study the relationship between performance with and without cross-asset regime fusion when the time-series sample count (l) varies. As for the performance metrics, we present the AR in

Table 7. Performance comparison (AR) of algorithms with and without cross-asset regime fusion across various datasets and time-series sample counts (l). Bold values indicate the best result in each comparison group. Bold values indicate the best result in each comparison group. (✔) means with cross-asset regime fusion, and (×) means without.

Dataset	Fusion	$\mathit{l}=20$	$\mathit{l}=50$	$\mathit{l}=100$
DJIA1d	×	0.100 ± 0.004	0.103 ± 0.007	0.112 ± 0.005
	✔	0.109 ± 0.008	0.121 ± 0.012	0.127 ± 0.013
DJIA1w	×	0.145 ± 0.006	0.158 ± 0.003	0.166 ± 0.006
	✔	0.161 ± 0.015	0.170 ± 0.026	0.157 ± 0.017
HSI1d	×	0.192 ± 0.033	0.171 ± 0.008	0.171 ± 0.006
	✔	0.296 ± 0.088	0.220 ± 0.029	0.247 ± 0.122
HSI1w	×	0.244 ± 0.015	0.305 ± 0.015	0.257 ± 0.016
	✔	0.292 ± 0.048	0.279 ± 0.048	0.251 ± 0.076

, ASR in

Table 8. Performance comparison (ASR) of algorithms with and without cross-asset regime fusion across various datasets and time-series sample counts (l). Bold values indicate the best result in each comparison group. (✔) means with cross-asset regime fusion, and (×) means without.

Dataset	Fusion	$\mathit{l}=20$	$\mathit{l}=50$	$\mathit{l}=100$
DJIA1d	×	0.557 ± 0.002	0.563 ± 0.010	0.578 ± 0.013
	✔	0.573 ± 0.025	0.613 ± 0.036	0.630 ± 0.040
DJIA1w	×	0.642 ± 0.006	0.661 ± 0.004	0.668 ± 0.004
	✔	0.646 ± 0.024	0.662 ± 0.018	0.665 ± 0.014
HSI1d	×	0.647 ± 0.041	0.678 ± 0.013	0.655 ± 0.015
	✔	0.808 ± 0.142	0.768 ± 0.040	0.792 ± 0.147
HSI1w	×	0.816 ± 0.019	0.909 ± 0.019	0.836 ± 0.026
	✔	0.868 ± 0.060	0.899 ± 0.065	0.793 ± 0.113

, and ACR in

Table 9. Performance comparison (ACR) of algorithms with and without cross-asset regime fusion across various datasets and time-series sample counts (l). Bold values indicate the best result in each comparison group. Bold values indicate the best result in each comparison group. (✔) means with cross-asset regime fusion, and (×) means without.

Dataset	Fusion	$\mathit{l}=20$	$\mathit{l}=50$	$\mathit{l}=100$
DJIA1d	×	0.308 ± 0.001	0.307 ± 0.002	0.308 ± 0.007
	✔	0.313 ± 0.015	0.345 ± 0.025	0.354 ± 0.029
DJIA1w	×	0.446 ± 0.019	0.474 ± 0.049	0.524 ± 0.019
	✔	0.499 ± 0.075	0.555 ± 0.101	0.515 ± 0.066
HSI1d	×	0.485 ± 0.063	0.557 ± 0.024	0.521 ± 0.024
	✔	0.728 ± 0.217	0.688 ± 0.068	0.762 ± 0.245
HSI1w	×	1.006 ± 0.044	1.311 ± 0.062	0.963 ± 0.086
	✔	1.189 ± 0.142	1.282 ± 0.182	0.830 ± 0.160

. In 1d high-frequency trading, cross-asset regime fusion can consistently improve performance, indicating that asset correlation and co-integration are particularly important for the generation of high-frequency portfolio policies. However, in 1w low-frequency trading scenarios, whether cross-asset regime fusion can improve performance is closely related to l. In both DJIA and HSI, although cross-asset regime fusion can enhance performance at small values of l, it may lead to a decrease in performance as l increases. This suggests that the correlation and co-integration of assets in DJIA and HSI are relatively unstable, where interference from instability characteristics within large samples leads to decreased performance.

5.3. Comparison with Other Methods

We compare our method’s performance to that of other methods, showing the AR in

Table 10. Performance comparison (AR) of different methods across various datasets. The best and second best results are marked by bold and underlined values, respectively.

Method	DJIA1d	DJIA1w	HSI1d	HSI1w
CRP	0.104	0.104	0.021	0.012
BAH	0.093	0.092	0.014	0.007
EG	0.104	0.104	0.020	0.012
OLMAR	−0.102	−0.042	−0.240	0.316
RMR	−0.211	0.024	−0.229	0.508
BNN	0.079	−0.171	−0.095	−0.171
CORN	−0.075	−0.069	−0.184	−0.314
CNN EIIE	0.114 ± 0.012	0.138 ± 0.028	0.082 ± 0.065	0.130 ± 0.094
bRNN EIIE	0.097 ± 0.006	0.122 ± 0.003	0.043 ± 0.013	0.107 ± 0.040
RAT	0.103 ± 0.002	0.124 ± 0.003	0.149 ± 0.013	0.215 ± 0.045
HGAM	0.103 ± 0.000	0.170 ± 0.056	0.078 ± 0.061	0.241 ± 0.110
LSRE-CAAN	0.119 ± 0.000	0.102 ± 0.029	0.011 ± 0.007	0.009 ± 0.000
PortRSMs	0.121 ± 0.012	0.170 ± 0.026	0.220 ± 0.029	0.279 ± 0.048

, ASR in

Table 11. Performance comparison (ASR) of different methods across various datasets. The best and second best results are marked by bold and underlined values, respectively.

Method	DJIA1d	DJIA1w	HSI1d	HSI1w
CRP	0.561	0.595	0.204	0.156
BAH	0.520	0.551	0.171	0.134
EG	0.559	0.593	0.202	0.154
OLMAR	−0.034	0.177	−0.262	0.815
RMR	−0.353	0.297	−0.225	1.116
BNN	0.388	−0.328	0.059	−0.216
CORN	−0.092	−0.108	−0.185	−0.542
CNN EIIE	0.584 ± 0.022	0.635 ± 0.032	0.398 ± 0.193	0.523 ± 0.287
bRNN EIIE	0.557 ± 0.001	0.625 ± 0.005	0.308 ± 0.056	0.516 ± 0.081
RAT	0.560 ± 0.001	0.620 ± 0.002	0.644 ± 0.041	0.805 ± 0.102
HGAM	0.559 ± 0.000	0.669 ± 0.034	0.428 ± 0.216	0.814 ± 0.176
LSRE-CAAN	0.564 ± 0.000	0.508 ± 0.094	0.182 ± 0.004	0.149 ± 0.000
PortRSMs	0.613 ± 0.036	0.662 ± 0.018	0.768 ± 0.040	0.899 ± 0.065

, and ACR in

Table 12. Performance comparison (ACR) of different methods across various datasets. The best and second best results are marked by bold and underlined values, respectively.

Method	DJIA1d	DJIA1w	HSI1d	HSI1w
CRP	0.310	0.332	0.067	0.043
BAH	0.279	0.297	0.039	0.023
EG	0.309	0.330	0.065	0.042
OLMAR	−0.149	0.064	−0.294	0.963
RMR	−0.304	0.036	−0.286	1.305
BNN	0.135	−0.239	0.124	−0.235
CORN	−0.154	−0.118	−0.240	−0.377
CNN EIIE	0.312 ± 0.005	0.462 ± 0.070	0.268 ± 0.206	0.452 ± 0.326
bRNN EIIE	0.309 ± 0.002	0.395 ± 0.004	0.150 ± 0.047	0.399 ± 0.069
RAT	0.308 ± 0.001	0.402 ± 0.004	0.514 ± 0.045	0.856 ± 0.209
HGAM	0.308 ± 0.000	0.578 ± 0.191	0.274 ± 0.204	0.870 ± 0.316
LSRE-CAAN	0.307 ± 0.000	0.292 ± 0.078	0.062 ± 0.022	0.025 ± 0.000
PortRSMs	0.345 ± 0.025	0.555 ± 0.101	0.688 ± 0.068	1.282 ± 0.182

. Overall, the use of DRL methods generally outperforms traditional methods. However, on HSI1w, mean reversion-based methods such as OLMAR (B. Li & Hoi, 2012) and RMR (Huang et al., 2016) perform better than deep learning methods, indicating a clear mean reversion characteristic on HSI1w. Previous DRL method performances depended on the dataset. CNN EIIE (Jiang & Liang, 2017) achieves good results on the DJIA dataset. The LSRE-CAAN (J. Li et al., 2023) method performs well on the DJIA dataset. RAT (Xu et al., 2021) handles HSI datasets well. Compared with other methods, our PortRSMs achieved the best performance among almost all datasets using DRL methods, showing its effectiveness and robustness.

6. Conclusions

This paper presents an innovative portfolio policy network structure that effectively addresses the challenges in financial time-series modeling by combining the regime shift model (RSM) paradigm with recent advancements in deep learning techniques. The experimental results across multiple stock markets and trading frequencies confirm the superiority of the proposed approach. This paper offers new insights for the development of high-frequency trading strategies and contributes valuable perspectives to the field of financial time-series modeling.

However, we acknowledge certain limitations of the current study. Due to the varying characteristics of different financial markets and time periods, some architecture hyper-parameters of the proposed model need to be adjusted for different scenarios. This sensitivity limits the model’s direct generalization across markets.

In addition, our approach implicitly assumes that short-term asset return distributions are locally consistent within regimes and that regime shifts can be effectively captured by structural patterns in the data. This assumption is supported by prior work on volatility clustering and regime shift models (RSMs) but may not hold under all market conditions, such as in cases of highly irregular or non-stationary shocks.

To address this, future research could explore the integration of meta-learning or online learning techniques, which are capable of adapting to dynamic and heterogeneous market environments in a more automated and robust manner.

Future research could also explore the application of the proposed method in other financial markets and asset classes, as well as further optimize the model to adapt to more complex market environments.