Cryptocurrency Futures Portfolio Trading System Using Reinforcement Learning

Abstract

This paper proposes a cryptocurrency portfolio trading system (CPTS) that optimizes trading performance in the cryptocurrency futures market by leveraging reinforcement learning and timeframe analysis. By employing the advantage actor–critic (A2C) algorithm and analysis of variance (ANOVA) portfolios are constructed over multiple timeframes. Data corresponding to the trade of 18 major cryptocurrencies on Binance Futures––between January 2022 and December 2023––are used to show that trading strategies can be effectively categorized into those with high-frequency (10, 30, and 60 min) and low-frequency (daily) timeframes. Empirical results demonstrate statistically significant differences in returns between these timeframe groups, with major cryptocurrencies (e.g., Bitcoin and Ethereum) exhibiting higher returns in high-frequency trading (16–17%) than in daily trading (6–7%) during training. Performance evaluation during the test period revealed that the low-frequency group achieved a 43.06% average return, significantly outperforming the high-frequency group (5.68%). The ANOVA results confirm that both the frequency type and portfolio selection significantly influence trading performance at the 5% significance level. This study offers a novel approach to cryptocurrency trading that considers the distinct characteristics of different timeframes. The effectiveness of combining reinforcement learning with statistical analysis for portfolio optimization in highly volatile cryptocurrency markets is demonstrated.

1. Introduction

In recent years, the cryptocurrency market has expanded rapidly, establishing itself as a critical asset class in the global financial market. In particular, prominent cryptocurrencies, such as Bitcoin and Ethereum, offer investors new opportunities and challenges driven by their significant volatility and liquidity [1,2]. However, owing to the unique characteristics of the cryptocurrency market, traditional financial analysis methods struggle to adequately explain or predict its price volatility [3,4]. Consequently, novel analytical approaches that leverage machine learning have been applied.

Stablecoins play a pivotal role in the cryptocurrency market. They are designed to minimize price volatility by pegging their value to real-world assets, such as fiat currencies (e.g., the US dollar and euro) or commodities (e.g., gold) [5,6]. For instance, Tether USDt (USDT) is a representative stablecoin pegged 1:1 to the US dollar, which is widely utilized as a fundamental medium of exchange in cryptocurrency trading [5]. Moreover, stablecoins constitute a critical infrastructure in the cryptocurrency market because of their advantages in facilitating transactions, providing liquidity, and ensuring price stability [7]. They hedge against the volatility and unpredictability inherent in cryptocurrency markets, enabling investors to manage sudden market fluctuations, that would otherwise pose significant risks, more effectively [8]. Further, the circulating supply of stablecoins can serve as an indicator of market liquidity and investor sentiment [9].

Reinforcement learning (RL) has been demonstrated to be an effective machine-learning approach for optimizing decision-making in uncertain and complex environments [10]. It is primarily composed of four elements: state, agent, action, and reward. Through interactions with the environment, the agent learns a policy that maximizes cumulative rewards. This approach has been extensively studied in various domains including finance, robotics, and natural language processing [11,12]. A prominent example showcasing the potential of RL is Google DeepMind’s AlphaGo, which exhibits superhuman performance in the complex game, Go [13]. Recently, RL has been increasingly applied in dynamic systems, such as financial markets [14].

In this study, we propose a conceptual framework for a cryptocurrency portfolio trading system (CPTS) designed to generate profitability based on multiple timeframes by implementing RL on the Binance Futures market.

The proposed trading system employs analysis of variance (ANOVA) to create a portfolio over different timeframes and RL to learn the trading model. We employ the advantage actor–critic (A2C) algorithm, which is known for its effectiveness in single-asset trading, as the RL framework. Based on the actor–critic architecture, A2C learns policy and value functions simultaneously, enabling efficient policy exploration [15]. Thus, it combines the advantages of policy- and value-based learning and exhibits stable learning and fast convergence. These characteristics make it particularly useful for single-asset trading that exhibits high volatility.

The A2C algorithm is trained on input data comprising technical indicators derived from open, high, low, and close (OHLC) prices and trading volumes obtained from cryptocurrency futures markets, as well as funding rates, on-chain data, and bases. Four frequencies—10, 30, and 60 min daily—were considered as timeframes for portfolio composition. Portfolio construction is based on profitable cryptocurrencies and an ANOVA of returns generated over different timeframes during the training period. The performance of CPTS is compared with that of Binance Futures, which are not selected as benchmarks for the portfolio. This study also performed an ANOVA to evaluate the performance of the CPTS with respect to varying timeframes.

The remainder of this paper is organized as follows. The fundamental concepts of blockchain, cryptocurrency, RL, and system trading and portfolios are briefly discussed in Section 2, along with a brief review of the existing literature on these topics. The proposed CPTS is introduced in Section 3, and Section 4 presents an empirical study performed to evaluate its performance. Finally, the conclusions are presented in Section 5.

2. Background and Related Works

2.1. Blockchain and Cryptocurrency

In a blockchain, data are stored in blocks, and each block contains the hash value of the previous block, creating a chain structure that ensures data integrity and immutability [1]. In addition, by sharing and verifying transaction records across all nodes, the system enhances transparency and trustworthiness [16].

Bitcoin, the first cryptocurrency to leverage blockchain technology, implements a secure and reliable electronic payment system without the need for a central authority [1]. However, Bitcoin exhibits certain limitations, including restricted transaction processing speed and the lack of smart contract functionality [17]. To address these issues, Ethereum was introduced in 2015, offering a platform for smart contract development and decentralized applications, thereby expanding the application scope of blockchain technology [2].

Cryptocurrencies play a pivotal role in blockchain networks and extend beyond mere value exchange. They provide economic incentives to participants, encouraging network activity, while ensuring the maintenance and security of blockchain systems [18]. For instance, in Bitcoin, the proof-of-work mechanism rewards miners with Bitcoin to generate new blocks [1]. This incentivizes miners to allocate computational resources, thereby enhancing network stability [19].

In addition, cryptocurrencies serve as the fuel required to execute smart contracts and operate decentralized applications, thereby contributing to the expansion of the blockchain ecosystem [20]. In Ethereum, the Ether (ETH) fulfills this role by covering the execution costs of smart contracts [2] and enhancing resource allocation and security within the blockchain platform.

The cryptocurrency market is characterized by high volatility and liquidity, offering both opportunities and risks to investors [8]. The price dynamics of cryptocurrencies are influenced by a variety of factors, including supply and demand, regulatory developments, and technological advancements [21]. Therefore, accurate price predictions play a crucial role in developing investment strategies and managing risks in cryptocurrency markets [22]. Existing studies on cryptocurrency price prediction have employed various approaches, including statistical methods, machine-learning-, and deep-learning-based techniques [23,24].

In conclusion, blockchains and cryptocurrencies are transforming the paradigm of financial systems, and their interplay plays a pivotal role in the advancement of blockchain ecosystems. Cryptocurrencies are essential for maintaining the activity and security of blockchain networks while offering new opportunities to investors. In this context, the development of cryptocurrency-trading strategies using RL has emerged as a significant and meaningful research topic, and is described in greater detail in the following subsection.

2.2. Reinforcement Learning

RL has been applied in various domains including gaming, robotics, natural language processing, and recommendation systems [14]. In the financial sector, RL is used for portfolio optimization, algorithmic trading, and risk management [25,26]. In particular, given the uniquely uncertain and dynamic characteristics of the cryptocurrency market, RL has emerged as an effective alternative to traditional financial models for ensuring optimal decision-making in the trade of related assets [27]. RL-based cryptocurrency-trading strategies aim to maximize returns and minimize risks in highly volatile markets [28].

RL is distinguished from supervised and unsupervised learning in that its agents undergo independent training without explicit guidance or labeled data [10]. RL agents learn the optimal timing and positioning of trades using market data. [29] demonstrated that RL outperforms traditional approaches in this task. However, it also faces several challenges, such as instability during the learning process, difficulties in hyperparameter tuning, and complexity of real-world data [30]. To address these issues, potential improvements to algorithms have been investigated to enable stable learning and model design tailored to real-world data.

RL algorithms can be categorized into value-based, policy-based, and actor–critic methods [10]. Value-based methods, such as Q-learning [31] and deep Q-networks (DQN) [11], focus on learning the state-action value function (Q-function) to derive the optimal policy. Policy-based methods directly learn policy functions for action selection, making them particularly effective in continuous action spaces [32]. Actor–critic methods combine the advantages of both approaches by learning a policy (actor) and value functions (critic) simultaneously [33].

In particular, the A2C algorithm exhibits a stable learning performance and fast convergence in problems with relatively simple states and action spaces, such as single-asset trading [15]. In exploring the applicability of RL in financial markets, Fischer [27] highlighted that actor–critic algorithms, such as A2C, could be effective for single-asset trading. Additionally, Pricope [34] developed algorithmic trading strategies using A2C to achieve excellent performance in single-asset trading scenarios.

While other algorithms such as DDPG and PPO have shown promise in continuous control problems, A2C provides a balanced approach that is well-suited for our initial exploration of timeframe-dependent portfolio optimization.

In this study, the A2C algorithm was employed to trade a single cryptocurrency asset based on efficient policy exploration. The actor and critic networks in our A2C model are both composed of a two-layer LSTM network followed by a fully connected layer. The hyperparameters used for training are as follows: learning rate = 0.001, discount factor (gamma) = 0.99, and entropy coefficient = 0.01. In this framework, the actor learns the policy ${\pi }_{\theta }(a\mid s)$ to select an action $\mathrm{a}$ in a given state $s$, while the critic estimates the state value function $V_{\phi }\left(s\right)$. The fundamental component of A2C is the advantage function, $A(s,a)$, which is defined as follows:

$$A\left(s,a\right)=Q\left(s,a\right)-V\left(s\right).$$

(1)

Here, $Q(s,a)$ represents the expected return when executing action $a$ in state $s$, and $V\left(s\right)$ denotes the expected value of state $s$. The actor and critic optimize the two objective functions defined by (2) and (3), respectively.

$$J_{\theta }=E\left[log{\pi }_{\theta }\left(a_t∣s_t\right)A\left(s_t,a_t\right)\right],$$

(2)

$$J_{\phi }=E[\left(R_t-V_{\phi }\left(s_t\right){)}^2\right].$$

(3)

2.3. System Trading and Portfolio Management

System trading is a trading method that automatically executes buy-and-sell orders using predefined algorithms and rules. Automated trading systems utilize computer programs to analyze market data, generate trading signals, and execute orders [35]. System trading eliminates subjective human judgment, minimizes irrational decision-making driven by emotions, and enhances the efficiency and consistency of trading activities and strategies [36,37,38,39].

Early forms of system trading involved executing trades based on rules derived from technical analysis indicators, such as moving averages and momentum indicators [40,41]. These technical indicators played a crucial role in analyzing price fluctuations and determining trading timings by systematizing market patterns. However, as the complexity of financial markets has increased, quantitative trading, which employs statistical methodologies and mathematical modeling, has been introduced as an alternative [42,43]. Subsequently, advancements in machine learning and artificial intelligence have revealed new possibilities for system trading [44,45].

The system trading approach has also been applied in the field of portfolio management. Portfolio optimization aims to determine the allocation of investments corresponding to various assets to maximize expected returns while minimizing risk [46]. Traditional portfolio theory calculates optimal asset allocation based on asset returns and covariances. However, it struggles to capture the nonlinearity and complex interactions inherent in financial markets [47].

Consequently, machine-learning techniques, such as RL, have been studied to address portfolio optimization problems [4,28]. In particular, deep RL, which combines RL with deep learning, is advantageous for portfolio management because of its ability to capture complex patterns in high-dimensional data [27,48].

Applying system trading and RL to the analysis of cryptocurrency markets, which are characterized by high volatility and complex market structures, presents both challenges and opportunities [27,34]. Recent studies have focused on optimizing cryptocurrency portfolios and developing strategies to adapt to market volatility using RL [28,49].

Jiang and Liang [28] applied the Deep Deterministic Policy Gradient (DDPG) algorithm, which is an RL-based portfolio management framework, to enhance the returns of cryptocurrency portfolios.

While these studies demonstrate the potential of RL in cryptocurrency portfolio management, they do not explicitly consider the impact of different trading timeframes on portfolio performance. Our study differs from these works by employing ANOVA to statistically analyze the performance of cryptocurrencies across different timeframes and construct portfolios based on these findings.

However, [4] utilized ensemble RL to achieve stable portfolio performance across various market conditions. These studies suggest that RL is a promising tool for system trading in cryptocurrency markets. However, cryptocurrency markets present several challenges, including data noise, potential market manipulation, and regulatory uncertainty [50,51]. Therefore, the development of RL-based system trading strategies must consider these factors to ensure the stability and practicality of the model.

In conclusion, system trading is a critical methodology that enables automated trading and portfolio management in financial markets, with further advancements driven by RL integration. In particular, its application in the cryptocurrency market has garnered attention owing to its ability to leverage RL to handle high volatility and complex market dynamics.

3. Materials and Methods

This section provides a detailed description of the proposed CPTS framework, which includes three stages (Figure 1). In the first stage, an input dataset is constructed. The second stage involves training the model for portfolio construction. The third stage evaluates the performance of the CPTS using trading simulation.

3.1. Stage 1: Construction of Input Dataset

At this stage, the OHLC prices, volume, funding rate, and circulating supply data of USDT of cryptocurrencies {$C_1, C_2, \dots , C_n$} are collected. Technical indicators {${\mathrm{T}}_1, {\mathrm{T}}_2, {..., \mathrm{T}}_n$} are generated using trading data (i.e., OFLC prices and volumes). In addition, the basis indicators are calculated based on future and spot trading data and are defined as follows:

$${Basis}_{open}= {\displaystyle \frac{({Futures}_{open}-{Spot}_{open})}{{Spot}_{open}}}$$

(4)

$${Basis}_{high}= {\displaystyle \frac{({Futures}_{high}-{Spot}_{high})}{{Spot}_{high}}}$$

(5)

$${Basis}_{low}= {\displaystyle \frac{({Futures}_{low}-{Spot}_{low})}{{Spot}_{low}}}$$

(6)

$${Basis}_{close}= {\displaystyle \frac{({Futures}_{close}-{Spot}_{close})}{{Spot}_{close}}}$$

(7)

3.2. Stage 2: Model Training and Portfolio Construction

At this stage, a model for constructing cryptocurrency futures portfolios is trained using the RL algorithm, A2C. The proposed RL model primarily comprises four modules: the environment, data processing, learning, and agents. The environment module defines the market environment and exchanges rules between the input dataset and both the agent and learning modules. The interactions of the RL agent are based on these data. The data-processing module transforms the state variables (i.e., technical indicators, funding rates, basis indicators, and circulating supply data) received from the environment module into a form suitable for learning using the RL algorithm. Because the variables exhibit different value ranges and scales, inputting them directly may introduce a bias during the learning process. To prevent this, the distribution of the variables was adjusted by applying standardization or normalization techniques to the input data. The learning module trains the agent’s policy network based on the preprocessed data received from the data-processing module and the market environment information obtained from the environment module. A Long Short-Term Memory (LSTM) neural network was employed to capture the characteristics of time-series data [52]. LSTM promotes the learning of long-term dependencies, enabling the model to identify patterns in the financial time-series data. The actor and critic networks in our A2C model are both composed of a two-layer LSTM network followed by a fully connected layer. The hyperparameters used for training are as follows: learning rate = 0.001, discount factor (gamma) = 0.99, and entropy coefficient = 0.01. The policy network uses the state as the input and outputs the probabilities for each action, whereas the value network estimates the value function of each state. The agent module selects actions based on the learned policy and receives rewards by interacting with the environment. At each time step, the agent chooses one of four actions: buying (entering a long position), selling (entering a short position), holding, and closing (exiting a position). The reward for each action was calculated by considering the realized profits and losses from trade, transaction costs, and funding rate expenses. The agent updates the parameters of the policy and value networks based on the reward signals, thereby minimizing the loss function of the A2C algorithm. The following training procedure was implemented.

• The agent observes the current state $s_t$ and selects an action $a_t$ using the policy network.
• Based on the selected action, the agent receives the next state $s_{t+1}$ and reward $r_t$ from the environment.
• The reward comprises immediate and delayed components; the immediate reward reflects the instantaneous profit or loss resulting from the action, whereas the delayed reward considers the long-term performance.
• The agent calculates the loss function using its experienced states, actions, and rewards, and updates the parameters of the policy and value networks. The loss function is defined as follows:

$$Loss = {\sum }_{t=0}^T(\left(r_t+\gamma V\left(s_{t+1}\right)-V\left(s_t\right){)}^2- log \pi \left(a_t∣s_t\right)\left(r_t+\gamma V\left(s_{t+1}\right)-V\left(s_t\right)\right)\right).$$

(8)

Here, $V\left(s_t\right)$ represents the output of the value network, and $\pi (a_t,s_t)$ denotes the output probabilities of the policy network.

The reward function is based on realized profit and loss (PnL) arising from trade after accounting for transaction costs and funding rate expenses. During the training period, the agent incorporates factors beyond the realized profit and loss into the reward function, including volatility (applying penalties during sharp fluctuations in returns) and maximum drawdown prevention (applying penalties when the maximum drawdown is large). Using this approach, the agent learns strategies to maximize returns while minimizing risks.

To mitigate the risk of overfitting, we employed a standard training/testing split of the data, using a 70% to 30% ratio. The model is trained on the training set, and its performance is evaluated on the unseen test set. The inclusion of penalties for volatility and maximum drawdown in the reward function also helps to prevent the agent from learning overly aggressive and potentially overfitted strategies.

In addition, to evaluate the performance of CPTS over different time intervals, various timeframes (10, 30, and 60 min daily) were established, and datasets specific to each timeframe were constructed accordingly.

Cryptocurrencies with higher per-agent trading returns than the average benchmark interest rate during the training period were selected for testing. Subsequently, portfolios were constructed based on the results of the ANOVA and post hoc tests on returns over different timeframes. In particular, we aimed to compare the returns during the test period between timeframe groups with significant differences in returns and those without such differences for CPTS.

3.3. Stage 3: Evaluation of CPTS

Trading simulations were conducted on timeframe-specific portfolio cryptocurrencies obtained using the model trained in Step 2. The simulations were executed individually for each cryptocurrency, and the CPTS performance was evaluated in terms of the agent’s rate of return (ROR). Specifically, the effectiveness of portfolio selection was verified by analyzing the differences in returns between the groups selected and not selected for the portfolio.

4. Experiments

4.1. Data Description

The data are obtained from the application programming interface of Binance Futures and Glassnode. We selected the top 20 cryptocurrencies by market capitalization on Binance Futures. After excluding two cryptocurrencies with insufficient historical data for the full study period, we were left with a final set of 18 cryptocurrencies for our experiment (see

Table 1. Ticker symbols of cryptocurrencies used in this study.

Ticker Symbol	Cryptocurrency
BTCUSDT	Bitcoin
ETHUSDT	Ethereum
ADAUSDT	Cardano
SOLUSDT	Solana
THETAUSDT	Theta Network
NEARUSDT	NEAR Protocol
BNBUSDT	Binance Coin
AVAXUSDT	Avalanche
DOTUSDT	Polkadot
BCHUSDT	Bitcoin Cash
TRXUSDT	TRON
FTMUSDT	Fantom
ALGOUSDT	Algorand
EGLDUSDT	Elrond
ATOMUSDT	Cosmos
XTZUSDT	Tezos
KAVAUSDT	Kava
ENJUSDT	Enjin Coin

). The selected cryptocurrencies are all actively traded on Binance Futures and have sufficient liquidity for our analysis. Additionally, stablecoins are methodologically excluded from price prediction studies as they are specifically designed to maintain price stability rather than generate returns through price appreciation, making them unsuitable for our research objectives.

To this end, cryptocurrency exchange futures contracts are selected, and their daily prices are investigated. Among the Binance Futures products, USDT perpetual contracts, which exhibit the highest trading volumes, are selected, along with the top 20 assets in terms of market capitalization. Perpetual contracts are cryptocurrency-trading instruments linked to the value of the underlying assets and serve as futures contracts without expiration dates. Of these, two assets with significant data losses were excluded, yielding an aggregate of 18 assets.

For each cryptocurrency, the time-series data were structured with respect to the selected timeframes (10, 30, and 60 min daily). The tick data were converted into these respective timeframes, allowing the model to learn the patterns and trends of the time-series data.

The collected data were divided into training and testing datasets. The training period is from 1 January 2022, to 30 April 2023, and data collected over this period are used by the agent to learn the optimal policy. The testing period was defined as 1 May 2023, to 31 December 2023, and was used to evaluate the performance of the trained model.

For on-chain data, we collect the circulating supply of USDT from Glassnode, which we use as a macroscopic proxy for market liquidity that uniformly affects all cryptocurrencies. Since each cryptocurrency protocol has different characteristics and on-chain metrics that are not directly comparable (e.g., Bitcoin’s hash rate has no equivalent in Ethereum or other networks), we adopted this universal approach to ensure methodological consistency across our diverse asset portfolio. However, the utilization of on-chain data is limited. This study considers only the circulating supply of USDT and does not consider various other on-chain indicators, such as active addresses, transaction volumes, and protocol-specific metrics.

4.2. CPTS Experimental Results

The average ROR values of the CPTS corresponding to 10, 30, and 60 min, as well as the daily timeframes during the training period are listed in

Table 2. Average ROR corresponding to the 10 min, 30 min, and 60 min timeframes during the training period.

Ticker Symbol	10 min	30 min	60 min	Daily
ADAUSDT	6.700	5.939	6.515	2.829
ALGOUSDT	0.237	0.198	0.186	0.931
ATOMUSDT	7.458	6.901	7.043	3.191
AVAXUSDT	1.789	1.595	1.543	1.499
BCHUSDT	5.315	5.332	4.976	3.201
BNBUSDT	16.572	16.832	19.048	7.59
BTCUSDT	16.624	17.036	17.21	6.594
DOTUSDT	3.747	3.735	3.852	2.119
EGLDUSDT	2.322	2.362	2.353	1.881
ENJUSDT	1.614	1.496	1.407	1.103
ETHUSDT	12.522	13.552	13.618	6.107
FTMUSDT	2.572	2.128	2.241	1.105
KAVAUSDT	2.562	2.708	2.486	1.439
NEARUSDT	1.081	1.059	1.04	0.744
SOLUSDT	1.041	0.988	1.038	1.322
THETAUSDT	3.322	3.676	3.767	2.485
TRXUSDT	27.092	26.416	26.015	10.526
XTZUSDT	3.885	3.570	3.936	1.950

.

The 10, 30, and 60 min timeframes corresponded to higher performance metrics than those of the daily timeframe. Notably, BTCUSDT, a major cryptocurrency, recorded returns of 16.572%, 16.832%, and 19.048% in the 10, 30, and 60 min timeframes, respectively. In contrast, it exhibits a relatively low return of 6.594% in the daily timeframe. Similarly, ETHUSDT exhibits a clear trend, with returns of 12.522%, 13.552%, and 13.618% corresponding to the 10 min, 30 min, and 60 min timeframes, respectively, compared with 6.107% corresponding to the daily timeframe. A similar pattern was observed for platform tokens such as BNBUSDT. Notably, TRXUSDT exhibited the most significant difference, achieving a return of 27.092% in the 10 min timeframe and 10.526% in the daily timeframe.

To select the portfolio assets to be employed in the CPTS, an ANOVA was performed on the RORs corresponding to different timeframes. Before conducting the ANOVA, we verified the assumptions of normality and homogeneity of variances using the Shapiro–Wilk and Levene’s tests, respectively. As some of the data did not meet these assumptions, we also employed the non-parametric Fritaedman test and Dunn’s post hoc test.

Table 3. ANOVA results corresponding to portfolio construction during the training period.

Source of Variation	Sum of Squares	Degrees of Freedom	Mean Square	F	p-Value
Timeframe	216,834.9	3	72,278.29	15.563	0.000
Residual	6,669,306	1436	4644.36

lists the ANOVA results, which confirm that the differences depend on the timeframe, while

Table 4. Post-analysis results for the training period (Dunn’s test).

Comparison Group	Mean Difference (%)	p-Value
10–30 m	2.987	0.319
10–60 m	−0.545	0.989
30–60 m	−3.532	0.181
10 m–1 d	−48.991	0.000
30 m–1 d	46.005	0.000
60 m–1 d	49.537	0.000

presents the results of the post hoc tests.

No significant differences in ROR were observed among the 10, 30, and 60 min timeframes, but the daily timeframe exhibited statistically significant differences compared with the others (p < 0.05). In this study, the 10 min, 30 min, and 60 min timeframes comprised the high-frequency group, whereas the daily timeframe comprised the low-frequency group. Portfolios suitable for high- and low-frequency groups were selected. In addition, exceeding the average benchmark interest rate of 2.389% during the training period was considered a criterion for portfolio construction. We use the average interest rate as a benchmark as it represents the risk-free rate of return in our market context. We acknowledge that comparisons with buy-and-hold strategies and cryptocurrency market indices would provide more comprehensive performance evaluation but were unable to conduct these comparisons due to data limitations in our current study design.

Table 5. Cryptocurrencies in the high-frequency group selected for the portfolio.

Ticker Symbol	Average ROR (%)
TRXUSDT	26.51
BNBUSDT	17.48
BTCUSDT	16.96
ETHUSDT	13.23
ATOMUSDT	7.13
ADAUSDT	6.51
BCHUSDT	5.31
XTZUSDT	3.94
THETAUSDT	3.77
DOTUSDT	3.75

and

Table 6. Cryptocurrencies in the low-frequency group selected for the portfolio.

Ticker Symbol	Average ROR (%)
TRXUSDT	10.53
BNBUSDT	7.59
BTCUSDT	6.59
ETHUSDT	6.11
BCHUSDT	3.20
ATOMUSDT	3.19
ADAUSDT	2.83

present the cryptocurrencies in the high- and low-frequency groups selected for the portfolios. The 95% confidence intervals for the mean ROR are also reported.

More cryptocurrencies in the high-frequency group outperform the benchmark interest rate, indicating that trading strategies utilizing high-frequency data could be more effective given the high volatility of the cryptocurrency market.

Trading simulations for the CPTS were performed during the test period, based on the portfolios constructed during the training period.

Table 7. Average RORs of cryptocurrencies in the high-frequency group selected for the portfolio during the testing period.

Ticker Symbol	Average ROR (%)
	10 min (%)	30 min (%)	60 min (%)
ADAUSDT	3.256	3.329	3.356
ATOMUSDT	7.396	7.996	3.458
BCHUSDT	7.897	7.997	3.458
BNBUSDT	7.489	8.000	7.989
BTCUSDT	3.358	7.990	3.458
DOTUSDT	3.123	7.996	3.423
ETHUSDT	3.423	7.997	3.423
THETAUSDT	3.223	7.998	3.423
TRXUSDT	9.512	8.003	8.012
XTZUSDT	3.058	7.992	3.458
Averages	5.174	7.530	4.346

and

Table 8. Average RORs of the cryptocurrencies in the low-frequency group selected for the portfolio during the testing period.

Ticker Symbol	Average ROR (%)
	Daily
ADAUSDT	47.491
ATOMUSDT	32.114
BCHUSDT	69.761
BNBUSDT	28.571
BTCUSDT	49.616
ETHUSDT	35.049
TRXUSDT	38.843
Average	43.064

present the ROR values for the high- and low-frequency groups, respectively. In the high-frequency group, TRXUSDT exhibited the most stable and high-quality performance, with returns of 9.51%, 8.00%, and 8.01% at 10, 30, and 60 min, respectively. BNBUSDT also exhibits stable performance, with returns of 7.49%, 8.00%, and 7.99% at 10, 30, and 60 min, respectively. Additionally, BCHUSDT and ATOMUSDT recorded high returns of 7.90% and 7.40%, respectively, corresponding to the 10 min timeframe.

In the low-frequency group, all selected assets achieve high returns, exceeding 28%. Notably, BCHUSDT exhibited the highest return at 69.76%, followed by BTCUSDT (49.62%), and ADAUSDT (47.49%). Notably, top-market cap coins, such as ETHUSDT and ATOMUSDT, are also observed to exhibit stable returns of 35.05% and 32.11%, respectively.

Table 9. T-test results on the differences in ROR between the two groups.

Frequency	N	Mean ROR (%)	Standard Deviation	t-Test	p-Value
High	300	5.68	2.36	−19.377	0.000
Low	70	43.06	16.10

presents the results of the t-test performed to identify the differences between the RORs of the two groups (high and low frequencies) during the test period. ANOVA results indicated that the differences in RORs between the groups were statistically significant.

Additionally, to verify the effectiveness of portfolio selection, a full-factorial mixed-model ANOVA was performed on the returns of cryptocurrencies selected for the high- and low-frequency groups, and those that were not selected. The analysis reveals that type of frequency (A) and portfolio selection (B) exert significant effects on performance at the 5% level of significance (See

Table 10. Multifactor ANOVA results for the test problem.

Source of Variation	Sum of Squares	Degrees of Freedom	Mean Square	F	p-Value
Type of Frequency (A)	299,444.23	1	299,444.23	1157.75	0.001
Portfolio Selection (B)	2147.51	1	2147.51	8.30	0.004
AB Interaction	10,378.82	1	10,378.82	40.13	0.001
Residual	185,188.61	716	258.64

).

5. Conclusions

This study proposes a CPTS framework that analyzes the data characteristics of cryptocurrency futures markets by using portfolios and RL. CPTS utilizes technical indicators, basis indicators, and on-chain data corresponding to different timeframes to construct a portfolio of cryptocurrencies whose returns exceed the benchmark interest rates during the training period. The performance of the CPTS was evaluated in terms of profitability, considering different timeframes. The application of CPTS to the cryptocurrency futures market in our experiments reveals the effectiveness of dividing portfolio composition into high-frequency (10, 30, and 60 m) and low-frequency timeframes (daily). Additionally, the ANOVA results indicate that the strength of returns depends on the timeframe-based classification. For example, during the training period, BTCUSDT recorded returns of 16.624%, 17.036%, and 17.210% corresponding to high-frequency timeframes in contrast to the relatively low return of 6.594% corresponding to low-frequency timeframes. ETHUSDT also exhibits high returns corresponding to high-frequency timeframes, and reduced returns corresponding to the daily timeframe.

This study not only proposes a novel trading system for the cryptocurrency futures market based on high-frequency and low-frequency data analysis but also contributes to the existing literature by performing ANOVA and applying portfolio construction methods tailored to various timeframes. In the past, trading strategies did not sufficiently consider the impact of different timeframes on performance, leading to many cases where they could not be effectively applied to markets with diverse data characteristics. The proposed trading system overcomes these limitations using various analysis methods, such as RL, ANOVA, and post hoc tests, to distinguish between high- and low-frequency trading strategies, thereby reflecting the unique volatility and data patterns of the cryptocurrency market. The overall results indicate that CPTS exhibits immense potential as a complementary trading system in the cryptocurrency market, by accounting for varying timeframes during portfolio construction. Although this approach does not always guarantee optimal trading outcomes, it can assist traders and investors in exploring alternative perspectives while developing trading systems or decision-support tools.

However, several limitations should be acknowledged. First, the experimental and evaluation period was relatively short, and, consequently, long-term market trends and volatility may not have been sufficiently captured. Second, the utilization of on-chain data is significantly limited, as we consider only the circulating supply of USDT and do not incorporate various other critical on-chain indicators. Future research would benefit from incorporating a broader range of on-chain metrics such as active addresses (indicating network usage and adoption), transaction volume and velocity (reflecting network activity), hash rate (for proof-of-work cryptocurrencies, indicating network security), network value to transactions ratio (NVT, serving as a valuation metric), realized market capitalization, and social sentiment indicators from various platforms. These additional on-chain indicators could provide richer insights into cryptocurrency market dynamics, network health, adoption trends, and investor behavior, potentially enhancing the predictive power of our model. Third, as the cryptocurrencies used are not categorized during experimentation, developing strategies that reflect the characteristics of each category is challenging. Fourth, due to computational constraints and data limitations, we could not conduct comprehensive comparisons with buy-and-hold strategies or cryptocurrency market indices, which would provide more meaningful benchmarks for evaluating the practical effectiveness of our CPTS framework. Fifth, future research should include more extensive hyperparameter search for the A2C algorithm to optimize performance further. Sixth, the evaluation based solely on returns does not capture risk-adjusted performance, which is crucial in volatile cryptocurrency markets. Future research should incorporate comprehensive risk-adjusted performance metrics such as the Sharpe ratio, Sortino ratio, maximum drawdown analysis, and Value at Risk (VaR) to provide more thorough evaluation of trading strategies in highly volatile markets.

Future research should enhance the performance of the trading system by conducting training and testing over longer periods, incorporating the broader range of on-chain data indicators mentioned above, and experimenting with a larger and more diverse set of cryptocurrencies. Additionally, implementing comprehensive risk-adjusted performance analysis, including the Sharpe ratio and other risk metrics, would significantly enhance the evaluation framework. Furthermore, future studies should include meaningful benchmarks such as buy-and-hold strategies, equal-weight portfolios, and cryptocurrency market indices to provide more comprehensive performance evaluation and establish the practical value of algorithmic trading approaches.

Subsequent research efforts should also focus on conducting comprehensive comparisons with other reinforcement learning algorithms such as DDPG, PPO, and ensemble methods, along with extended training and testing periods. Additionally, exploring the concept of ‘market time’ could provide valuable insights into our timeframe-dependent results. The relationship between sampling frequency, market microstructure characteristics such as the Hurst exponent, and performance outcomes warrants investigation using market time scaling approaches.