Stock Market Bubble Warning: A Restricted Boltzmann Machine Approach Using Volatility–Return Sequences

Abstract

Combining unsupervised learning with Restricted Boltzmann Machines and supervised learning with Balanced Random Forest and Feedforward Neural Networks, we propose a warning system for the early detection of stock bubbles by analyzing daily returns and the volatility of a market index. We complement our method by detecting states of high volatility and very low returns, which are market states that immediately follow a stock market’s bubble-bursting point. We trained our detection model using the S&P500 as an empirical case study, using successive samples of well-known crises from 1987 to 2022. Our results achieve area-under-the-curve (AUC) rates of over 70% and false-positive rates of less than 20%. Our model’s generative nature enables the creation of synthetic samples to analyze market periods prone to forming a bubble. The model successfully alerts periods of bubbles and instability in the stock market. Capital markets’ interconnectedness enables the model to be trained with various shocks from other stock markets, providing further detection learning possibilities and improved detection rates. Our work helps investors, regulators, and practitioners in their stock market investment, supervision, and monitoring tasks.

1. Introduction

The traditional economic and financial perspective, grounded in the theory of efficient markets (EMH), posits that financial assets such as stocks or bonds should reflect their economic or fundamental value in their market prices. Accordingly, in a market with rational agents and frictionless conditions, the price of a financial asset should incorporate all the available information regarding the asset [1]. Under these conditions, asset price levels should not deviate from their present value relationships, and price bubbles—understood as deviations of market value from an asset’s intrinsic or fundamental value—should not be observed, especially in assets such as stocks and bonds. However, the persistent failure of present-value models to explain price bubbles throughout history led practitioners and academics to develop models of price bubbles [2].

The literature that studies the occurrence of price bubbles from a behavioral finance standpoint can be categorized into two main perspectives: the limited arbitrage view and the bounded rationality stance. The first posits that professional arbitrageurs—those who possess rational expectations and make decisions based on the fundamentals of financial assets—will not be able to eliminate the errors made by unsophisticated investors or noise traders; consequently, mispricing in financial assets persists [3,4,5].

The second perspective assumes that at least a segment of market agents is not entirely rational and, therefore, suffers from bounded rationality. This condition prevents them from making rational decisions due to their limited cognitive resources, time, and information. Furthermore, in the context of stock markets, these agents may be subject to investor sentiment [3,6]. This trait is understood as erroneous beliefs about future cash flows and risks. This condition can negatively impact the quality of investors’ decision-making, as they tend to make emotional decisions rather than rational ones [7]. As a consequence of the above, the simultaneous effect of investors’ sentiment and bounded rationality, in conjunction with heterogeneous belief formation and herd behavior, provoke financial markets to exhibit substantial price and trading volume fluctuations, as well as underreactions and overreactions in asset prices that could generate price bubbles [5,8].

The EMH asserts that returns are unpredictable and that bubbles are not feasible. However, ref. [9] suggests that the argument behind the EMH is invalid because it ignores the agents’ psychology and interactions at the origin of price fluctuations. Additionally, the theoretical literature outlines the conditions under which market price deviations from fundamentals and specific predictability windows in returns are likely to occur. Factors related to momentum trading, trend-chasing, herding, and agents’ traits, such as animal spirits, fads, fashions, and overconfidence, may lead to positive feedback loops. Consequently, under such conditions, psychological and behavioral elements of stock price determination may render stock returns predictable, at least in pockets of predictability associated with the end of bubble episodes or regime changes [10].

Economic openness and financial integration have fostered increased interconnectedness in financial markets, expanding capital markets with global economic growth. This evolution created a global financial market with complex system characteristics marked by cross-border interdependencies, in which shocks often escalate into international events due to herding, social imitation, and positive feedback, causing abrupt price changes, high volume of transactions, return synchronization, high volatility, and price bubbles [11]. As seen during the Great Financial Crisis of 2008–2009, these effects can persist long after the initial shock, drastically altering stock price patterns [12]. Events like the Great Financial Crisis and the COVID-19 pandemic demonstrate how interconnected markets amplify shocks, making risk management at both portfolio and systemic levels difficult. Persistent volatility spillovers exacerbate contagion across assets, underscoring the need for tools to comprehend market dynamics during turmoil, develop early warning systems during regime shifts, and detect the formation of price bubbles [13,14].

Financial crashes and stock price collapses jeopardize the stability of the entire economic system. They reveal persistent patterns shaped by the type of shock and uncertainty involved. A typical response is flight-to-quality behavior, where investors shift from riskier assets to safer options, such as cash or money market instruments. Under extreme conditions, this behavior disrupts markets, triggering liquidity shortages, fire sales, and heightened volatility, which erodes overall market stability [15,16]. Extreme events and shocks in stock markets result in substantial economic losses, reduced investor confidence, and household consumption, ultimately leading to economic depression. Consequently, despite the unpredictable nature of stock markets, there remains a pressing need for investors, regulators, and practitioners to develop methods and empirical models to anticipate and monitor disruptive events, regime changes, and the formation of price bubbles.

A complementary approach to understanding bubbles emerges from the field of complex systems. This theory posits that bubbles and crashes are rooted in imitation and herding as the most visible manifestation of human behavior in our social systems. In particular, as a complex system comprising several heterogeneous agents with repetitive interactions in financial markets, this behavior tends to self-organize its internal structure, leading to surprising emergent properties that are out of equilibrium. Under such conditions, financial markets, prone to cooperative herding and imitation, exhibit repetitive nonlinear interactions that could destabilize the financial system. Therefore, in a market with multiple agents influenced by exogenous information and events, as well as endogenous interactions, self-organized and self-reinforcing extreme behaviors could emerge, generating market changes such as bubbles and crashes [17,18,19].

In line with the above, we propose a warning system for the early detection of stock market bubbles. Our approach combines unsupervised learning with Restricted Boltzmann Machines and supervised learning with Balanced Random Forest and Feedforward Neural Networks. Machine learning methods are emerging as practical models for analyzing extreme events, regime changes, and bubble formation in financial markets. Other works also focused on detecting or warning of financial crises. For instance, [20] developed a financial condition indicator using Artificial Neural Networks (ANNs) based on daily data. Their approach combined multiple ANN-based experts through a genetic algorithm, offering an integrated solution for financial crisis warnings. Although these authors use predictor features based on returns and volatilities, their proposal does not provide objective performance measures to compare with other detection alternatives.

Our approach focuses on the early detection of financial bubbles—phases of instability that precede crises—rather than directly predicting financial crash events. This distinction is important: in contrast to conventional binary classification problems, the field lacks standardized definitions for financial bubbles and widely accepted evaluation benchmarks, making it challenging to assess models using uniform criteria. While studies forecasting discrete financial shocks (e.g., using logistic regression, decision trees, XGBoost) typically predict crash events at a defined future point, our model aims to detect the evolving structural features of bubbles through volatility–return patterns.

We contribute to the literature on machine learning in finance with a practical and user-oriented approach. Using daily data from the S&P500 index, we analyze returns and their volatility to identify periods of market instability and points near regime changes in market trends. We capture the complex relationships between stock returns and their volatility to train a Restricted Boltzmann Machine (RBM), which encodes this information in a high-dimensional space through its unsupervised learning process. In this way, these RBM representations provide essential inputs for a supervised learning process that evaluates the likelihood of entering a state of instability or nearing a turning point in the stock market. An advantage of this approach is that it is scalable to other financial indexes in different markets without the need to train the model from zero, allowing for the integration of data from bubbles and financial crises in a sole global market with indicators across various financial markets, making it a valuable model for general financial bubble warnings. Additionally, we trained our model to assess the proximity of a possible phase change and the market conditions that characterize the instability and high volatility triggered during a crisis period. In this way, we have two mutually exclusive event signals that assist the analyst in assessing the market condition.

Our approach presents many advantages. Integrating unsupervised learning via RBMs into the model provides greater flexibility in analyzing market conditions. RBMs demonstrate remarkable versatility across various scientific fields, especially in finance, where they hold significant potential due to recent methodological advancements. Research on RBM learning phases evaluates the accuracy–efficiency tradeoff, enhancing their application in forecasting financial time-series data while balancing computational demands [21]. RBMs and several classifiers can predict short-term stock market trends [22]. They have also been utilized to extract discriminative low-dimensional features from raw data, with dimensions reaching up to 324, for stock price forecasting [23]. A new generalization of the RBM, the p-RBM, can retain the memory of p-states and predict stock market direction [24]. The above highlights RBMs’ adaptability in addressing complex financial modeling challenges, from microstructure analysis to identifying systemic risk. Finally, our model’s training period represents an advance in the empirical literature, as it considers a variety of financial shocks and crashes associated with multiple origins. We tested our model not only with one or two crises; instead, we trained it in a stepwise fashion over time using four well-known crashes in the U.S. market, starting from the Black Monday crash of 1987 to the COVID-19 outbreak in 2020 (see Appendix A for a summary of these and other episodes).

Based on composite features from the S&P500 index returns and volatilities and using successive samples from financial crises and turmoil between 1987 and 2022, our results demonstrate that the model can effectively detect bubble formations and stock instability. Additionally, it identifies periods of abrupt stock market crashes. The model trains a hybrid neural network, Balanced Random Forest (BRF), to signal the formation of potential financial bubbles and proximity to critical tipping points. Furthermore, our method can recognize conditions indicative of potential overvaluation or price bubbles. The results show an AUC (area under the curve) rate exceeding 70% and false-positive rates of less than 20%. Notably, this work represents the first application of Gaussian–Gaussian Restricted Boltzmann Machines (GGRBMs) for financial market modeling, providing a flexible framework to analyze diverse market scenarios. Their use is novel for anomaly detection in capital markets and remains largely unexplored. The model is scalable across different financial markets and lays the groundwork for future improvements through feature expansion, dynamic recalibration, and transfer learning across markets.

The paper is structured as follows: Section 2 presents the conceptual basis for describing financial systems, periods of instability, and periods of abnormal price growth or bubbles. Section 3 presents the bubble and instability detection model based on Restricted Boltzmann Machines (RBMs), Balanced Random Forests (BRFs), and Artificial Neural Networks (ANNs), along with the strategies for training these algorithms. Section 4 analyzes the S&P500 stock market index as a case study to test the performance of the models. Finally, in Section 5, we present the conclusion and discussion.

2. Background

2.1. Instability Periods in Stock Markets

We identify two types of samples in the price time series: one called unstable (UNS), and the other that corresponds to periods just before the instability (BTC) or bubble times. The first is commonly referred to in the literature as market crashes: a sudden and severe price drop. Substantial negative price variations and high volatility characterize these phenomena [25]. Since we are trying to generate a warning of the advent of instability, the second type of samples are price variations and volatilities that are located in the dynamics just before the periods of instability. In these cases, price variations are usually positive and high but with low volatility. The exact location of crisis periods will depend greatly on how instability is defined. Our empirical approach suggests taking a large sample of the stock price series or indices of interest and categorizing the return/volatility pair according to the percentile in which it is found. The choice of percentile also serves as a basis for later binarizing the return and volatility values.

Previous studies of financial crises showed that certain patterns appear to be characteristic of infrequent events associated with abrupt and sudden drops in asset values. By examining the joint distribution of price variations (or returns) and volatility (volatility is defined here as the price variation in a given period), it is possible to distinguish ordered return–volatility pairs associated with normal times, times of up-markets or bubbles, and times of instability or crises. In the first case, price variations are typically positive with moderate volatility; in the second, there are high returns and low volatility; and during periods of instability, there are strong negative variations and high volatility [25].

The volatility–return distribution allowed constructing a crisis detector for cryptocurrencies [26]. In periods of normality, there is a positive dependence between volatility and returns, where the distribution concentrates its mass on the diagonal, which is comprised of the direct relationship between volatility and return. However, during a crisis such as COVID-19, the distribution’s mass changes, concentrating on the diagonal, comprising the indirect volatility–return relationship. In such a case, the lowest returns (negative price variations) are concentrated in areas of high volatility. At the same time, minor price variations are concentrated in areas of low volatility. With this knowledge, it is possible to develop a shock event detector [27], which successfully recognizes periods of instability.

Similarly to the above idea, our approach uses the same distinctive property of the joint volatility–return distribution to train a stacked BBRBM-ANN and detect whether we are in an unstable zone. By instability, we refer only to the statistical property of those samples that are at the highest percentile of volatility and, at the same time, the lowest of returns. This characteristic is present in periods of financial turmoil or turbulence.

We analyze the S&P500 index as an example to obtain a whole perspective. Figure 1 shows the empirical joint distribution of the volatility–return index from March 1982 to June 2024. One can distinguish, in particular in (a), three states: one of low positive returns and low to moderate volatility (normal times), another zone of meager returns and high volatility, times of instability or crisis (lower right zone), and another zone of positive returns and moderate to high volatility, times of up-market or bubbles (upper right zone).

The instability zone corresponds to the drawdown period, which is the decline in the stock market from the maximum point $t_c$ to the minimum index value. When considering the drawdowns of the previously mentioned four crises and comparing them with the rest of the samples by obtaining the median percentiles, the results are $5.2$ for volatility and $5.55$ for return. However, for the instability cases, the values are $8.07$ for volatility and $5.05$ for return, i.e., periods of very high volatility and returns below the mean.

2.2. Financial Bubbles: A Behavioral Perspective

Classical financial theory posits that rational investors strive to determine the market value of financial assets by considering unbiased expectations of future cash flows discounted at a properly risk-adjusted rate. In this manner, traditional financial assets, such as stocks and bonds, are valued based on the present value of their future cash flows [28]. In other words, market prices should reflect the rational expectations of the economic agents negotiating those assets. Under such conditions, market prices equal their economic value. However, for this to occur, two conditions must be met: the first states that there must be unlimited arbitrage in the market, and the second states that there must be fully rational agents among market participants.

The unlimited arbitrage condition asserts that professional arbitrageurs or sophisticated traders—those who possess rational expectations and make decisions based on the fundamentals of financial assets—will be able to correct the errors made by unsophisticated investors [29]. However, this is not always the case, as arbitrageurs are limited by the risks and market frictions they face. These include transaction costs and impediments, such as short-selling constraints, along with various other limitations [3,4,5]. Accordingly, potential arbitrageurs face the risk that when they bet against a given mispricing, this may worsen, or the ultimate correction may occur much later, affecting their ability to correct errors in asset valuations. A direct consequence of the above is that, as rational arbitrage cannot correct all instances of misalignment between market prices and fundamental values, stock prices could underreact to certain types of information in the short term but overreact in the long term, favoring the formation of price bubbles [30].

The rationality condition states that the behavior of all investors can be synthesized through a representative rational-agent model on which traditional asset-pricing models are grounded. Under this rational paradigm, economic agents behave as utility maximizers. In addition, they can correctly update their beliefs using Bayes’ rule as new information arrives, make decisions by comparing expected utilities, behave as risk-averse agents, and possess homogeneous expectations about future returns, risk, and probabilities. However, as the financial history shows, the behavior of economic agents deviates from the previous paradigm, particularly during the occurrence of financial bubbles [31]. In such conditions, market participants exhibit bounded rationality, which prevents them from making entirely rational decisions due to their limited cognitive resources, time, and information [32]. In addition, from a psychological perspective, economic agents are also prone to investors’ sentiment, a characteristic related to moods, beliefs, and behavioral biases [3,6]. Investor sentiment, understood in the context of stock markets as erroneous beliefs about future cash flows and risks, can negatively affect the quality of decision-making and significantly impact the prices of all assets and the relationship between risk and returns in financial markets [33].

A direct consequence of the above is the pervasive nature of emotional responses among economic agents at both the individual and aggregate levels. Emotional reactions occur because the utility derived from economic choices cannot be divorced from emotions triggered by changes in the environment under which those decisions were taken [7]. In particular, under regime changes in financial markets [12], the widespread speculation regarding the future performance of assets, along with investors’ sentiment and bounded rationality, generates significant emotional responses among market participants [5,34,35]. These emotions influence the behavior of all financial assets, which tend to exhibit swings disconnected from fundamentals [36,37]. Consequently, due to investors’ sentiment, bounded rationality, heterogeneous belief formation, and herd behavior, financial markets exhibit substantial price and trading volume fluctuations, as well as underreactions and overreactions in asset prices [5,8]. In particular, during regime changes, due to the uncertainty and biases such as over(under)confidence, conservatism, and representativeness, investors fail to accurately update their beliefs regarding a stock’s performance, owing to varying degrees of strength (salience and extremity) and weight (statistical informativeness) of available corporate information [8,38]. Consequently, episodes of underreaction and overreaction in prices may emerge in stock markets as individuals focus too much on the strength of the evidence and too little on its weight [8].

In summary, the above literature establishes the functioning conditions of the stock market and the rationality requirements of investors that can lead to episodes of price bubble formation and stock market crashes.

2.3. Regime Changes in Financial Markets

Abrupt market changes result from shifts in regulation, economic policies, or rare black swan events such as wars, terrorist attacks, or pandemics. These shocks disrupt economic outlooks and alter the traditional risk–return relationship, undermining diversification and risk management strategies [39,40,41]. Consequently, the literature has developed regime-switching models using econometric methods that capture the dynamics of regime changes resulting from shifts in regulation, politics, and economic and market conditions. For instance, in fixed-income markets, the interest rate behavior resulting from monetary policy is a significant factor in price dynamics [42]. In equities, different regimes correspond to periods of high and low volatility and long bull and bear market periods [43]. Regime-switching models can fit narratives and stories of changing fundamentals that sometimes can only be interpreted ex-post, but in a way that allows ex-ante decision-making, such as portfolio management, forecasting, risk surveillance, and other economic and financial market applications. Different regime models generate differences in asset returns in terms of means, volatilities, autocorrelations, and covariances, depending on good (low-risk) and bad (high-risk) states of the market conditions [44,45,46,47]. Notwithstanding the above, they cannot capture the complex, nonlinear interdependencies and large-scale collective behaviors that typically emerge under extreme events, such as black swans.

From a complementary standpoint, empirical work on markets’ tipping points finds that the transition point $t_c$ or the tipping points of a financial system can be measured using the eigenvalues of the similarity matrix, representing the distance between return correlation matrices. This approach enables the recognition of transitions between different states of the financial market [48,49]. Similarly, a warning system based on temporal networks is also possible, in which topological changes entail upcoming information about abrupt changes in market regimes [50]. In this case, the authors analyze the dynamics of the dyadic and triadic motifs characterizing the network, elaborating indicators for a possible regime shift or tipping point.

From another perspective, research on market crash warnings grounds their work in the idea that financial time series have a fractal structure. Consequently, there is the ability to detect critical transitions [51]. In these cases, warning signals based on the multifractality of the time series can help check a market’s approach toward a critical point. Despite adopting a practical approach, the above methods for market crash warning lack performance measures to compare the effectiveness of their models, and they have only tested them with up to two or three recognized financial crisis events.

Extending the scope of analysis, [52] developed an early warning model that uses 20 features related to the stock market, bond market, and foreign exchange market as inputs for an Extreme Gradient Boosting (XGBoost) training process. This model classifies the Chinese financial market into low, medium, and high-risk conditions. Despite considering other financial market indicators to build a broad financial stability indicator, this work is a more complex methodologically, as it involves estimating GARCH and Markov Regime Switching models to create the stability indicator and determine the levels of financial stability. A variety of other machine learning techniques, such as deep learning, XGBoost, and logistic regression, have also been used to identify financial indicators that precede stock market crashes, showing that Deep Neural Networks provide the best performance across several metrics in the evaluation of a classification problem [53].

2.4. Model of Speculative Financial Bubbles

The complex systems standpoint complements the financial and psychological perspective on bubble formation. This view states that bubbles and crashes are linked to specific behaviors of economic agents, particularly those related to imitation and herd behavior. Thus, in the case of capital markets, understood as a complex system comprising several heterogeneous agents with repetitive interactions, these behaviors lead to surprising emergent properties that are out of equilibrium. Under such conditions, financial markets, prone to herd behavior and imitation, exhibit repetitive nonlinear interactions that could destabilize the financial system. Therefore, as capital markets are influenced by both exogenous information and events, as well as endogenous interactions among multiple agents, self-organized and self-reinforcing extreme behaviors can emerge, generating price bubbles and their corresponding crashes [17,18,19].

From a market perspective, it is essential to recognize a state of financial instability; however, understanding the conditions that lead to it is even more critical. Examining the early warning signs and processes contributing to instability in this context becomes particularly valuable. This pre-crash phase, commonly called a financial bubble, warrants close analysis to determine whether the system is currently in such a state. An interesting hypothesis is that financial crashes originate from the slow accumulation of long-range correlations originating from the cooperative behavior of market agents, which culminates in a drastic and short-lived correction of asset prices [19]. This way, the exact moment in which this transition from bubble to drastic price correction occurs is a critical point $t_c$, similar to what in statistical physics is called the determination of the exact moment of a phase change, in which an observable of interest becomes infinite [54].

The end of a bubble is precisely the moment of the crash at a critical time. The mechanism that originates the bubble is understood through the self-reinforcement of positive feedback, such as the imitative behavior of investors or herding [55,56]. This process fuels a continuous and accelerating price rise, preparing the system for a phase shift into a stage of instability initiated by the impending crash [57]. In this context, these price bubbles can be understood as a manifestation of super-exponential power law acceleration of the price, characterized by the log-periodic precursor model [18,58] (LPPM). This precursor pattern of crashes has been effectively utilized to detect end-of-bubble signals [59] of the S&P500 index over nearly two centuries of data.

The above authors used statistical tests of models fit iteratively over long time windows to test directly whether the data fit the behavior of the general expression of the power law singularity given by

$$F\left(\tau \right)=A_0+A_1{({\tau }_c-\tau )}^{-\gamma }\cos \left(\omega \log ({\tau }_c-\tau )+\varphi \right).$$

(1)

In this model, $A_0$ and $A_1$ are constants with real values. The parameter $\omega$ is the log-angular frequency of the oscillations during the bubble. Then, $\varphi$ is the positive phase parameter. The term $\tau$ is time and ${\tau }_c$ is the critical time (or critical point) that determines the separation between the bubble and the instability or phase change. The expression ${({\tau }_c-\tau )}^{-\gamma }$ represents a hyperbolic power law and corresponds to the mechanism of positive feedback and $\cos \left(\omega \log ({\tau }_c-\tau )+\varphi \right)$ related to the herding behavior or possible hierarchical cascades in the the growth of the bubble [60,61]. The singularity occurs when both times are equal. Finally, the exponent $\gamma$ must be between 0 and 1 for it to be a super-exponential growth.

Moving in a distinct direction from the above work, we employ a different empirical approach, grounded in neural networks, to alert to the proximity of a regime change in the market. Thereby, instead of using the data to test the fit between the model and the data, we use the data to train a neural network, first in an unsupervised way (with the RBM machines) and then through supervised learning (with the BFR) that alerts us to the proximity of a regime change. This method has the advantage of not relying on estimating specific model curves. On the other hand, it enhances the bubble detection capabilities by leveraging the power of machine learning algorithms.

3. Materials and Methods

In this section, we describe the model’s components for early detection of a phase change and strategies for training it.

The system consists of two independent models. The first model allows us to detect if we are in a financial instability condition, which is defined as a sample located in a zone of high volatility and negative price variations. On the other hand, in periods before a situation of instability, the prices have positive or negative variations, which on average tend to be zero (daily basis), and normal volatility. This situation is essential since it allows us to recognize if we are already in a phase change. This model comprises a classical Bernoulli–Bernoulli RBM (BBRBM) and a traditional feedforward network. With a simple Naive model, we can generate as many unstable samples as we want and train the ANN as a classifier to detect whether or not the incoming sample corresponds to an unstable sample (See Section 3.2). Essentially, we capture the system’s state with two measures: the market index or stock price variations and the volatility.

The second model lets us detect whether we are in a bubble condition. A bubble consists of a prolonged and rising rate of price or index increase, accompanied by oscillations of increasing frequency until the critical moment $t_c$, when a phase change occurs, and the system enters a stage of significant negative price variations and high volatility. To detect this phenomenon, binary representations of volatility and price variations alone are insufficient because the bubble process is long-lasting and can take years [18]. Consequently, we need to include other features for this long-run accumulation process. This paper primarily considers two features: (a) the index slopes at time t and (b) the cumulative index changes at time t. The first represents the price rise (or fall) in a unit of time before the critical time $t_c$. These values are known to be very high when we approach the critical point, i.e.,  price acceleration increases more and more. The second feature type is the cumulative returns over more than 250 days (approximately one year). We know that these cumulative returns are unusually high near the critical point with a very low probability of occurrence. Unlike the first instability detection model, we trained a Gaussian–Gaussian RBM (GGRBM) in this instance. This choice is to avoid losing information in the discretization process. Therefore, all the features that enter this RBM are continuous, ensuring no information is lost.

Figure 2 shows the general warning model. The BBRBM’s mission is to capture complex representations of the system in the hope that it has some level of discriminative power to distinguish between a normal and unstable state. The GGRBM’s task goes in the same direction but to identify bubble periods or an approximation to a critical time $t_c$. The information in these high-level representations embedded and learned by the RBMs can help predict the properties of a specific set of financial index samples. However, since the RBM is a generative model, we can use it, mainly the GGRBM, to reproduce synthetic samples under certain fixed interest conditions and find the bubble probability, as seen in Section 4.5.

3.1. Unsupervised Models

As we can see in Figure 2, we have unsupervised processes involving Restricted Boltzmann Machines (RBMs), supervised processes involving a neural network (ANN) to detect a sudden price drop accompanied by high volatility, and a Balanced Random Forest (BRF) that allows us to warn if we are in a bubble.

The traditional RBM is the Bernoulli–Bernoulli RBM, or BBRBM [62,63], in which the m visible units of the input layer are binary $v=(v_1,...,v_m)$. The visible units represent the features that account for the system’s state. These units are connected to each of the n hidden units of the hidden layer $h=(h_1,....,h_n)$, which also take binary values. Thus, the machine is a bipartite network in which the visible units are fully connected to each of the hidden units, but with the restriction that there are no connections between units of the same layer. This restriction allows the hidden units to be conditionally independent of each other but conditional on the values of all other visible units and vice versa. For the BBRBM, $(v,h)\in {\{0,1\}}^{m+n}$, and the joint distribution of $(v,h)$ can be written as $p(\mathbf{v},\mathbf{h})={\displaystyle \frac{e^{-E(v,h)}}{Z}}$, where Z is the normalization constant or partition function, and the energy function $E(v,h)$ is as follows:

$$E(v,h)=-\sum _{i,j}{w}_{ij}{v}_i{h}_j-\sum _i^m{b}_i{v}_i-\sum _j^n{c}_j{h}_j$$

(2)

where $i\in \{1,...,n\}$, and $j\in \{1,...,m\}$ indicate the hidden and visible units connected by the weighted edges, represented in the matrix W matrix with dimension $n\times m$. The values $b_j$ and $c_i$ represent the biases of the visible and hidden units, respectively. The conditional probabilities are estimated from the propagation through the neural network via the following equations:

$$p\left(\mathbf{v}\right|\mathbf{h})=f\left(\mathbf{b}+\sum _{j=1}{h}_j{w}_{ij}\right),$$

(3)

$$p\left(\mathbf{h}\right|\mathbf{v})=f\left(\mathbf{c}+\sum _{i=1}{v}_i{w}_{ij}\right)$$

(4)

where $f(.)$ is the sigmoid activation function.

The values of the BBRBM’s hidden layer units will be the input for a classifier (ANN) to tell us if we are in the presence of instability. It is equivalent to having a thermometer that evaluates market conditions in real time. However, we need more complex features that consider real (non-discretized) market values to evaluate the cumulative process that can give evidence of inflated price conditions or a financial bubble. For this case, we consider a Gaussian–Gaussian RBM (GGRBM) in which both the visible and hidden units support continuous value data. For this case, the energy function is given by

$$E(v,h)=-\sum _{i,j}{w}_{ij}{\displaystyle \frac{h_j}{s_j}}{\displaystyle \frac{v_i}{{\sigma }_i}}-\sum _i^m{\displaystyle \frac{{(v_i-b_i)}^2}{2{\sigma }_i}}-\sum _j^n{\displaystyle \frac{{(h_j-c_j)}^2}{2s_j}}$$

(5)

where $v_i$ and $h_j$ are real values for the visible and hidden units, respectively. As before, $b_i$ and $c_j$ are biases to the visible and hidden units, respectively, connected through the weights $w_{ij}$. The parameters ${\sigma }_i$ and $s_j$ are the standard deviations of the visible and hidden units, respectively. The conditional probabilities follow the following equations:

$$p\left(\mathbf{v}\right|\mathbf{h})=N(\mathbf{v}|\mathbf{b}+\sum _j{h}_j{w}_{ij},{\mathrm{\Sigma }}^2),$$

(6)

$$p\left(\mathbf{h}\right|\mathbf{v})=N(\mathbf{h}|\mathbf{c}+\sum _i{v}_i{w}_{ij},S^2)$$

(7)

where $N(\mu ,\mathrm{\Sigma })$ represents the multivariate normal distribution with mean $\mu$ and covariance $\mathrm{\Sigma }$, respectively. We denote ${\mathrm{\Sigma }}^2$ and $S^2$ as the covariance matrix of the visible and hidden units. It must be satisfied that S is an $N\times N$ matrix where the diagonal of S is $\mathrm{diag}(s_1,...,s_N)$, satisfying that $S_{jj}=s_j$ and $S_{ij}=0$. For the matrix $\mathrm{\Sigma }$, it is a matrix of $M\times M$ whose diagonal is $\mathrm{diag}({\sigma }_1,...,{\sigma }_m)$.

In the GGRBM model, the conditional distributions over visible and hidden units require predefined covariance matrices $\mathrm{\Sigma }$ and S, respectively. Rather than learning these variances jointly with the weights—which can lead to convergence issues and increased model complexity—we adopted an empirical strategy inspired by [64,65]. Specifically, (a) the covariance matrix of the visible units, $\mathrm{\Sigma }$, is fixed using the empirical covariance of the training data, and (b) the covariance matrix of the hidden units, §, is set as a scaled identity matrix $S=s{I}_m$, assuming statistical independence between hidden units. We found that setting $s=0.5$ resulted in the most stable convergence behavior. Larger values tended to produce instability or divergent learning dynamics. Similar behavior was reported in [66], who noted that tuning variance parameters—especially in Gaussian–Bernoulli and Gaussian–Gaussian configurations—requires particular care to avoid learning divergence and unreliable sampling.

To estimate the set of model parameters, i.e., $\mathbf{b},\mathbf{c}$ and $\mathbf{W}$, we can use the contrastive divergence (CD) learning [62] approach that has been the standard for training RBMs. Learning via CD takes k steps to approximate the second term of the gradient of the log-likelihood function by sampling from the RBM distribution. Following the literature notation in RBM, we express expectation under the data distribution as ${\langle .\rangle }_d$ using k steps of Gibbs samplings, and the expectation under the model distribution as ${\langle .\rangle }_m$, which we call the expected gradients of the positive and negative phases, respectively. The parameter update rules are as follows:

$$\mathrm{\Delta }{w}_{ij}={〈{\displaystyle \frac{v_i}{{\sigma }_i^2}}{\displaystyle \frac{h_j}{s_j^2}}〉}_d-{〈{\displaystyle \frac{v_i}{{\sigma }_i^2}}{\displaystyle \frac{h_j}{s_j^2}}〉}_m$$

(8)

$$\mathrm{\Delta }{b}_i={〈{\displaystyle \frac{v_i}{{\sigma }_i^2}}〉}_d-{〈{\displaystyle \frac{v_i}{{\sigma }_i^2}}〉}_m$$

(9)

$$\mathrm{\Delta }{w}_{ij}={〈{\displaystyle \frac{h_j}{s_j^2}}〉}_d-{〈{\displaystyle \frac{h_j}{s_j^2}}〉}_m$$

(10)

Note that the values of ${\sigma }_i$ and $s_j$ are meaningful only for the GGRBM, and for the BBRBM case, we can leave them equal to 1. The negative phase estimation process can be carried out with only the $k=1$ step of Gibbs. However, we opted for using Persistent Contrastive Divergence (PCD) [67] as the training method for both the Bernoulli–Bernoulli RBM (BBRBM) and the Gaussian–Gaussian RBM (GGRBM). While traditional Contrastive Divergence (CD-k) starts the Markov chain at the data point for every gradient update, PCD maintains a persistent chain of fantasy particles across iterations. This allows the persistent samples to reflect the model’s evolving distribution more accurately, leading to better mixing and potentially improved feature detectors. This modification has been shown to improve the estimation of the model’s expectation over the data distribution, especially when the model is still far from convergence [68,69,70,71].

3.2. Instability Model Training Strategy

As shown in Figure 2 (left), the instability detector is composed of a BBRBM that creates a high-level representation in the hidden layer values of the input layer values. Then, these hidden layer values are the input to a Feedforward Neural Network. This network is the supervised stage of the process. The idea is that given an input vector $\mathbf{v}$, we can have a probability indicator that we are in the presence of an instability.

Consistent with what we know about periods of instability (see Section 2.1), the BBRBM input features will be slices of k volatility lags and k price variation lags. Then, the input vector $\mathbf{v}=\{v^v,v^r\}$ is a set of k volatility values $v^v=(v_{t-k},...,v_{t-1},v_t)$ and k values of $v^r=(r_{t-k},...,r_{t-1},r_t)$ returns. However, the BBRBM accepts only 0 or 1 discrete values as inputs, so our strategy will be to discretize the values of volatility and returns according to percentile. So, for example, since we are interested in detecting unstable samples, we can make $v_i^r=0$ if $v_i^r<P_2^r$, where $P_2^r$ is the 20th percentile of returns, and similarly, $v_i^v=1$ if $v_i^v>P_8^v$, where $P_8^v$ is the 80th percentile of volatility. All other values will otherwise be zero. This way, we recognize samples in the lower right-hand region of the joint volatility–return distribution in Figure 1 (See Appendix B for more details justifying these percentiles). For example, consider $k=3$, i.e., we look at today and two periods back. Then, if $v^v=(1,1,0)$ and $v^r=(0,0,1)$, it would indicate that before yesterday and yesterday, we had unstable samples, and the input vector to the BBRBM would be $\mathbf{v}=(1,1,0,0,0,1)$. This binarization arrangement allows us to identify abnormal system states, leading to a regime change in the market.

Figure 3b shows magnetization diagrams of different crises. The magnetization is the average of the last k states. Thus, in the example above, the magnetization of volatility at t when $v^v=(1,1,0)$, would be $2/3$; similarly, $1/3$ for the returns. Then, in Figure 3a, we presented four crises: Black Monday in 1987, the dot-com crisis in 2000, the Subprime crisis in 2008, and the COVID-19 crisis in 2020. Each time series represents the value of the index at the indicated dates. Applying the model to the four crises, we can see in Figure 3b that for $k=8$, one can obtain that after the critical moment $t_c$, the magnetization of the volatility is very close to 1, while that of the returns is close to zero, indicating a clear transition of the system to a state of high instability (high variability and negative returns).

To better understand the features used to train the BBRBM, we present in Figure 4 time slices with eight lags for volatility and returns of the S&P500, represented in $4\times$4 images. The first two columns of each image depict the states of 8 consecutive days of volatility, and the following two columns represent the same 8 days of returns. As we observe, samples corresponding to periods of instability tend to be in high volatility (yellow color or +1 state) and low/normal volatility (purple color or zero state). In comparison, periods just before instability (BTC) are characterized by states of low volatility (purple color or zero state) and medium or high returns (yellow color or +1 state).

The clear distinction between unstable (UNS) and non-stable samples (BTC) allows us to consider generating synthetic samples to train the BBRBM and ANN to detect unstable samples. So, we can have as many synthetic samples as we want, which frees us from using our few real, unstable samples for training purposes. To generate synthetic samples, we run extensive simulations for a given number of days. Being p, the probability of having discretized volatility equal to 1 (in the zone of instability) and $1-p$ equal to 0 (in the zone before instability), we generate random numbers from a Bernoulli distribution for periods before and during instability. The same procedure is repeated for the returns. More details regarding the synthetic data generation can be found in Appendix B.

For the supervised component, we employ a traditional feedforward ANN with two layers: the visible layer corresponding to the hidden units of the last BBRBM layer, an intermediate layer, and the output layer. The ANN weights are adjusted using the backpropagation algorithm. We utilized two independently generated synthetic datasets, one for the BBRBM and the other for ANN training.

3.3. Bubble Model Training Strategy

Equation (1) of Section 2.4 gives us clues about the features we can consider to feed the learning process. First, we have a cumulative process of positive price variations over time, and second, an acceleration of these variations as the critical time approaches.

To better understand the features, let us consider the trajectory of the S&P500 from 1982 to 2024. We compute two features: first, the long-range $\delta$ returns

$$r_t^{\delta }={\displaystyle \frac{r_t-r_{t-\delta }}{r_t}},$$

(11)

where $r_t$ is the long-range return of $\delta$ days at time t. Second, the slope is calculated as follows:

$$s_t^{\delta }={\displaystyle \frac{p_t-p_{t-\delta }}{\delta }}.$$

(12)

Now that we have defined the long-range return and the slope, we will show an example where we have calculated the feature every $\delta =500$ days, as shown in Figure 5. From the dynamics of the slopes and the long-range accumulated returns, we see that as the value of the index increases at a higher rate during accumulating profits, the slopes begin to diverge from the accumulated returns. Both seem to meet again in the middle of the instability zone, making the divergence virtually null. At the end of the sharp falls and the beginning of the recovery, we see that the divergence begins to increase in positive terms ($r_t^{\delta }>s_t^{\delta }$), and this condition is maintained until another cycle of instability is repeated again. This is best seen in the 2000 and 2008 crises (bands b and c in Figure 5). The divergence is close to zero when the market has bottomed out. In the 2022 crisis, amid the index decline, a trend of convergence of the cumulative returns and slopes can also be observed, making the divergence very small. Interestingly, in recent years, there is a sustained increase in divergence in negative terms ($r_t<s_t$), where the slopes are greater than the cumulative returns. Of course, this does not necessarily indicate a crisis, so it is necessary to consider measures of volatility, short-term returns, and slopes with other $\delta$ ranges. Similar graphs, but based on simulations, depicting periods of normality (simulated with Geometric Brownian Motion) followed by a gradual increase in the index following the LPPLS model, can be seen in Appendix C. In these simulations, we can see that divergence in bubble periods tends to be exacerbated. Naturally, this behavior does not consider the inherent complexity of the real S&P500 index. However, it is necessary to understand better how these predictors act in the presence of bubbles.

All the information comes from the stock or index prices. Let us call $p_t$ the price in period t. Let us consider the price change as $r_t=p_t-p_{t-1}$. Volatility, on the other hand, is the price variation occurring on a transaction day. Consider the volatility at day t as $v_t={\displaystyle \frac{p_{\max }}{p_{\min }}}$, where $p_{\max }$ and $p_{\min }$ are the maximum and minimum price occurred in the period. Thus, $v_t\ge 1$. When $v_t$ is close to 1, it indicates slight price fluctuation and conversely, when it is greater than 1, it indicates high intra-day fluctuation. So, the GGRBM is trained with continuous values of volatility $v_t$, continuous daily returns, using k lags, long-range returns $r_t$, and slopes $s_t$.

As with any classification problem, an ideally balanced set of samples from the positive and negative classes is necessary to train a supervised model. We are not in a situation where financial bubbles are abundant, so no matter how we define a sample as belonging to a bubble, there will always be a much smaller number of such samples than non-bubbles. In our approach, we define a bubble sample as any observation occurring within a fixed window of r trading days preceding the critical point $t_c$, which corresponds to the peak price before a major drop or crash. This labeling is central to training the supervised classifier, as it allows the model to learn the latent features of a pre-instability regime. Thus, samples very close to and before the critical point (which we know in advance) are labeled positive samples (1). On the other hand, all those samples that are well away from the critical point and after the critical time $t_c$ can be labeled as negative class (0). How close they are to the critical point is a matter of empirical matter, the r-parameter. However, throughout history, the number of critical points is smaller than on non-bubble days so that we will have a class imbalance problem.

This strategy is motivated by the conceptual framework of the Log-Periodic Power Law Singularity (LPPLS) model [18], which interprets bubbles as regimes characterized by super-exponential growth with log-periodic oscillations culminating at a singular point $t_c$, analogous to a phase transition in statistical physics. However, in contrast to the LPPLS literature, we do not estimate or calibrate the LPPLS model directly to infer $t_c$. Instead, we use domain knowledge to define $t_c$ ex-post as the local maximum in price before a well-defined market crash. This simplification avoids the difficulties of fitting the LPPLS model, which has many nonlinear parameters and often leads to unstable or sensitive estimates, as noted in Ausloos et al. [58].

The parameter r can be interpreted as the empirical horizon of warning or alert. While it is not derived from a specific theoretical threshold in the LPPLS dynamics, it is consistent with the idea that bubbles exhibit distinct dynamics shortly before their collapse. A key reference supporting this empirical perspective is Zhang et al. [59], which showed that LPPLS-based end-of-bubble signals successfully anticipate several historical crashes, including those in 1987, 2000, and 2008, but also generate warnings before more minor corrections or ’other’ drawdowns.

To tackle the class imbalance problem, we will consider the Balanced Random Forest (BRF) algorithm, which has proven to be satisfactory in other problems in various data domains [72], in churn prediction [73], modifications combining the BRF and nearest neighbor algorithm to identify critical areas in the dataset [74], and also in the identification of RNA codes in genetics [75]. The BRF works in the same way as the traditional Random Forest [76], with the difference that in each iteration, a bootstrap sample is taken from the minority class. The same number of samples is taken with replacement for the majority class (see [72] for more details).

To train the BRF-based classifier, we must generate the representations of the original volatility and return samples with the GGRBMs already trained. So, the BRF input features will be the values of the last hidden GGRBM units.

4. Results

This section presents the results of applying the early detection model to real data, specifically regarding the stock market as represented by the Standard and Poor’s 500 index, used as a case study.

4.1. S&P500 and Sequences of Volatility and Returns

For this case, we have 9920 daily prices of the S&P500 from 31 December 1984 to 9 May 2024. This series covers at least five episodes: the stock market crash of 1987 (Black Monday), the dot-com crisis of 2000, the 2007–2009 Global Financial Crisis (Subprime crisis), the COVID-19 pandemic of 2020, and the stock market decline of 2022. These episodes allow us to identify periods of initial instability (sharp price drops or high volatility) and periods of normality just before the instability.

4.2. Instability Detection

For the instability detection model, we generate sequences of non-stable and unstable samples with a duration of 100 days, as described in detail in Appendix B. We generate 32500 discretized volatility and return strips, which will consequently be the input vectors to the BBRBM. We use 80% of these as training and the remaining 20% as tests. This is conducted using lags of $k=3$, $k=8$, and $k=16$ days.

The BBRBM is trained with a learning rate of 0.01, a constant decay of 0.01, 500 epochs, and a batch size of 128 samples. We empirically test different configurations of these parameters, monitoring the evolution and ensuring a clear convergence of the log-likelihood function [77]. For the PCD, we use only $n=1$ step to run the Markov chain. In choosing the number of hidden units, we prioritize a richer and more detailed embedding, so we selected 9, 24, and 48 hidden units from the BBRBM for the cases of $k=3$, $k=8$, and $k=16$ lags, respectively. Since we use synthetic samples, we do not risk overfitting the empirical data.

The ANN is trained with the BBRBM hidden unit values obtained when the BBRBM volatility and synthetic return slices (not used in the BBRBM training) are presented to the BBRBM input layer. A hyperparameter grid search is used to select the best design combination. In particular, we test with different numbers of units of the ANN hidden layer, the optimizer (Adaptive Estimation of first-order and second-order moments, ADAM, or Stochastic Gradient Descent, SGD), the learning rate, and batch size. The parameters selected are those for which a better AUC is achieved on a test dataset. For $k=3$, the number of hidden units is 12, SGD, learning rate of 0.001, and batch size of 12. For $k=8$, the number of hidden units is 32, ADAM, learning rate of 0.001, and batch size of 8. For $k=16$, the number of hidden units is 24, SGD, learning rate of 0.001, and batch size of 8.

Figure 6 (Left) shows the performance results of different ANNs trained with the values of the BBRBM hidden units with different numbers of lags. It is worth considering that the classification is achieved only by detecting the patterns of zeros and ones in the volatility and return strips without specifying any time dependence in advance. This is as if the ANN had to recognize images of one type or another. This is not very far from reality since there is usually no time dependence in the return series, although there may be in the case of volatility. In any case, we note that increasing the length of the sequence strips helps to improve classification.

Once the BBRBM-ANN has been trained, the next step is to apply the binarized volatility and return slices of the S&P500 to the BBRBM, whose hidden neurons serve as input for the ANN to generate a prediction. From the ANN output neuron, we obtain the probability values to visualize what the model tells us regarding the presence of volatility. Figure 7 shows the probabilities of having unstable samples generated by the BBRBM-ANN, using them as inputs to the BBRBM slices of $k=16$ volatility and returns lags. The model consistently predicts a high probability (close to 1) in periods well known as crises (marked in the graph from letters a to d). On the contrary, in periods of long index rises, the model generates very low probabilities of instability. Another interesting observation is that the model occasionally generates probabilities above 0.5 (but not close to 1.0) that tend to occur in index crashes not part of a crisis, capturing some of the noise characteristics of market volatility.

We have also performed the same exercise of testing the BBRBM-ANN instability detection models, but with inputs to the BBRBM using $k=3$ and $k=8$ lags. The plots can be viewed in Appendix D.

4.3. Bubble Detection

The first step consists of training the GGRBM. The training inputs are slices of $k=8$ lags of volatility, return and cumulative slopes, and returns of $\delta =375$, $\delta =500$, and $\delta =1250$ days backward, giving a total of 22 features. The GGRBM was trained with a hidden layer with $n=16$ units, 1200 epochs, batch size of 64, Persistent Contrastive Divergence (PCD) using 1 step of the Gibbs sampling, and learning rate between 0.005 and 0.008 with a decay rate of 0.1 to make the learning rate decrease smoothly as the number of iterations progresses.

In the second step, we train the BRF with the values of the hidden units of the GGRBM. For this case, we need to construct the class variable that tells us when we are in the presence of a bubble and when we are not. As indicated in Section 3.3, we use a practical approach. We recognize the critical point $t_c$ corresponding to the highest index value reached just before the sudden drop in the index occurs. We leave samples that are r days behind $t_c$ as bubble samples ($y=1$) and all others at $y=0$. We test by training BRF with $r=50$ and $r=90$ days backward, which allows us to assess whether there is any difference in the estimated bubble probabilities. The BRF was trained using n_estimators = 500 (500 trees), max_features=sqrt, max_depth = 10, and of course, with 16 features, corresponding to the values of the hidden units of the GGRBM.

To evaluate the model’s ability to detect bubbles, we trained three sets of GGRBM-BRF with different training sets: the first one using only the 1987 Black Monday crisis, the second using the Black Monday crisis and the dot-com crisis of 2000, and the third using the two previous crises and the Subprime crisis of 2008. Figure 6 (right) shows the model’s performance on different classical measures to evaluate classifiers. It is possible to note interesting results despite the complexity of our problem. As we indicated in the Introduction, similar financial regime change detection studies do not usually provide objective performance measures, with a few exceptions. We have examined studies that provide relevant benchmarks using supervised learning models. For instance, ref. [78] uses univariate predictors, logit models, and SVMs to forecast financial shocks in the Eurozone. Their best model achieved an accuracy of approximately 90%, comparable to our Balanced Random Forest results. However, their true positive rate (TPR) was only 19%, while our model achieved TPRs exceeding 60%, highlighting a potential advantage in identifying pre-crisis conditions. Similarly, [53] implemented various machine learning classifiers (RF, SVM, XGBoost, and others) to forecast market crises using financial indicators. Their best model yielded an AUC of around 80%, which is in the same range as our results.

Figure 8 (Right) shows an idea of the performance achieved using the BRF-based bubble detector, using the 1987 (Black Monday), 2000 (dot-com), and 2008 (Subprime) crises. Overall, the results are promising. First, note that the overall performance decreases marginally as we train with more shocks. This happens because when we train the detector with only one crisis, it only receives samples from that one. However, as we increase the number of crises, there are also more samples. The imbalance between samples considered bubbles and non-bubbles also becomes more extensive, and consequently, the performance is slightly affected. Second, there is no noticeable change in performance if we change the number of days from the critical point we use to define bubble samples.

Let us point out some observations from Figure 8. First, we can see that bubble probabilities tend to be higher precisely in the days before the shock when the index drops considerably quickly. This is most noticeable in longer-lasting crises such as 2000 (b), 2008 (c), and 2022 (e). This is good because we want the system to alert us that there is potential for excessive growth and that, as a consequence, there is a possibility of a critical point in the short term. Conversely, amid instability, the bubble probability returns to its minimum. Second, particularly in the first row, we see that even though the BRF has been trained with only one crisis, the system warns of future crises. However, from 2013 onwards, the machine warned of several bubble probability increases, which did not produce a sharp decline in the index. Perhaps the years from 2017 onwards are more complicated to evaluate, considering the continuous and rapid rises of the index that have not necessarily ended in financial crises. However, by training the machine with more crises (including 2000 and 2008), there is more clarity, at least for the 2022 crisis (e), where it is clear that the system warns during 2021 of a high probability of a bubble. The case for the 2019 crisis (f) (COVID) is unclear. The machine fails to recognize a bubble satisfactorily just before this crisis. This may be due to the purely exogenous character of this event, which would be very difficult to notice with only intrinsic information from the S&P500 price series.

To ensure temporal consistency and to mimic real-time deployment, we adopted a stepwise forward training approach: the model is initially trained using data from the first known crisis (e.g., 1987) and then tested on all subsequent periods. As each new crisis is included (e.g., 2000, 2008), the model is retrained with the cumulative crisis data and re-evaluated for the remaining future periods. This strategy allows us to study how model generalization evolves with expanding crisis knowledge. Importantly, model performance, particularly AUC, tends to marginally decline as more crises are added to the training set. We interpret this as evidence of model drift, a phenomenon in which the statistical relationships between features and target labels change over time due to evolving market dynamics. This is expected in financial systems, where the conditions that lead to a bubble in one decade may differ significantly from those in another. Consequently, model recalibration becomes essential: retraining the BRF periodically with the most recent market data helps adapt to drift and maintain predictive relevance. While relative returns and volatility help mitigate some non-stationarity, they do not guarantee the temporal stability of input–output mappings. Moreover, as seen in the case of the COVID-19 shock, which lacks typical pre-crisis patterns, the model may not flag bubbles arising from exogenous or unprecedented events, highlighting the limits of supervised learning in capturing structurally novel crises.

4.4. Using the Model

To ascertain how to use the instability sample detector (BBRBM-ANN) and the bubble detector (GGRBM-BRF), we propose monitoring the probability of instability and bubbles on a daily basis. The BBRBM is presented with samples (images) of discretized volatility and price variations. The GGRBM is presented with volatilities, price changes, slopes, and cumulative returns. In this way, the trained ANN and BRF models can identify what kind of sample is involved. An example of this is presented in Figure 9, where we have plotted the S&P500 index between 1995 and 2012, encompassing the 2000 dot-com crisis and the 2008 Subprime crisis, and between 2013 to June 2024, encompassing the COVID outbreak in 2020 and the stock market decline in 2022. The probability of instability and bubble predictions are observed along with the index.

We suggest identifying two basic scenarios: (1) The instability zone—the probability of instability is high, while the probability of a bubble is low. It responds to the abrupt fall of the price or index characterized by high volatility and negative index changes. (2) The bubble zone—the probability of instability is low, while the probability of a bubble increases. In this case, these are samples preceding the critical point where high returns have accumulated, and the price increase rates (slopes) accelerate. The tipping point or regime change may be imminent.

Considering the above scenarios, we can see by looking at the graph in Figure 9a that for the dot-com crisis, starting in 1997, the model begins to warn of an increase in bubble probability above 0.5, reaching a peak close to 1.0 in late 1998 and early 2000. The decline in the index started in mid-2000. Thereafter, the bubble probability falls sharply, and the instability detector begins to warn of probabilities above 0.5, particularly in early 2001 and mid-2002. As the BRF has been trained with more crises, we have a better chance of a better bubble representation, observed in Figure 9c,e. After that, the system seems calm, as the bubble and instability probabilities remain very close to zero. However, bubble probabilities rose at the beginning of 2007, although not alarmingly. But, in the second half of 2007, bubble probabilities were clearly above 0.5, also embellished by some probabilities above 0.5 for instability. In this case, it can be observed that the instability probabilities were very close to 1 in the middle of the crises of 2008 and 2009 when the market hit bottom. These observations indicate that simultaneously monitoring instability and bubble probabilities is more informative than looking at only one.

Similarly, looking at Figure 9b,d,f reveals how the models behave with samples that the detectors have not previously seen. Note that in the COVID-19 outbreak, the bubble and instability probabilities (especially the latter) are high. In other words, the system recognizes instability but simultaneously has a greater than 0.5 probability of a bubble. It is worth mentioning that, unlike the other crises, the pandemic is exogenous to the market, so it would be tough to pretend that the model trained only with composite index variables could warn of such an episode. In any case, the system reasonably recognizes the instability caused by the stock market decline in 2022, which is preceded by a high probability of a bubble.

Finally, by mid-2024, the system clearly warns of a high probability of a bubble while, at the same time, indicating a very low probability of instability. Although we cannot accurately predict the timing of an impending tipping point, given the results, it is at least worth mentioning that investors and regulators should be very cautious about this situation.

4.5. Detection Using Synthetic Samples

If unsupervised models based on RBM are well trained, they can correctly represent the distribution of the input units. This is an advantage if we use the RBM as a synthetic sample-generating model. In this case, we can predict the values of the visible units for a desired target with predetermined fixed visible units. We will utilize this resource by using the trained GGRBM to analyze under which conditions the GGRBM-BRF bubble detector model gives us higher or lower bubble probabilities. In this way, we can use the trained unsupervised model as a generative sensitivity analysis.

Inspired by using RBMs as collaborative filters [79], we have a vector $\mathbf{v}$ with m units. Of these units, some we will leave fixed in advance, and others will be left free to adopt whatever the RBM determines. Given the vector $\mathbf{v}$, we want to find the probability distribution of the free units after a Gibbs sampling iteration. For this, we see the vector of the hidden layer ${\mathbf{h}}^{\prime }=p\left(\mathbf{h}\right|\mathbf{v})$ with Equation (7), and then recover the vector of visible units ${\mathbf{v}}^{\prime }=p\left(\mathbf{h}\right|{\mathbf{h}}^{\prime })$. We replace the values of the visible units in ${\mathbf{v}}^{\prime }$ that we determined as clamped with the original values of ${\mathbf{v}}^{\prime }$. In this way, we generate a synthetic sample. We can repeat this process several times to achieve a set of synthetic samples so that, for a subset of visible units that remain fixed (determined by the user), we obtain a distribution of the possible values of the free visible units.

Finally, we can treat these synthetic samples as real samples for the bubble detector to give us a bubble probability. Therefore, we must again submit these synthetic samples to the GGRBM to obtain the values of the hidden layer, which in turn submits them to the BRF. This exercise is interesting because it allows us to analyze under which conditions a higher bubble probability exists.

To describe the usefulness of this exercise, we will use a GGRBM trained with $k=8$ slices of volatility and returns, slopes, and cumulative returns for $\delta =\{375,500,1250\}$ days using data from 30 April 1987 to 25 June 2024, covering three major crises (1987, 2000, and 2008). We have created three base cases of analysis from which we generate synthetic samples. In all cases, we leave the volatility and return values of the $k=8$ lags fixed, while the features of slopes and cumulative returns are presented on six possible variants from A to F. On these variants, each slope feature and cumulative return can take the following condition: free to run or fixed to the case. For more details, refer to

Table A1. Six different generation simulations (A to F) of synthetic samples are created for each of the Cases (1, 2, and 3). Volatility and return values for the $k=8$ lags are set at high or low levels, depending on the case. For high and low levels, the 90th and 10th percentiles, respectively, of the empirical values of the GGRBM input data are used.

Variant	Slopes 350d	CumRet 350d	Slopes 500d	CumRet 500d	Slopes 1250d	CumRet 1250d
A	Free to run	Free to run	Free to run	Free to run	Free to run	Free to run
B	Fixed to the Case	Fixed to the Case	Fixed to the Case	Fixed to the Case	Free to run	Free to run
C	Fixed to the Case	Fixed to the Case	Free to run	Free to run	Fixed to the Case	Fixed to the Case
D	Free to run	Free to run	Fixed to the Case	Fixed to the Case	Fixed to the Case	Fixed to the Case
E	Free to run	Fixed to the Case	Free to run	Fixed to the Case	Free to run	Fixed to the Case
F	Fixed to the Case	Free to run	Fixed to the Case	Free to run	Fixed to the Case	Free to run

of Appendix E.

Case 1 consists of high volatility and low return values resembling instability periods; the slopes and cumulative returns remain low. Case 2 consists of low and high volatility and return values, respectively; the slopes and cumulative returns remain high, resembling bubble periods. Case 3 consists of volatility and return values at high levels; the slopes and cumulative returns remain high, resembling possible bubble periods. For example, if we want to study unstable samples (Case 1), we leave the volatility and return features fixed at high and low levels, respectively. Then, we obtain synthetic samples in which we let the values of slopes and cumulative returns run free (Variant A).

Figure 10 shows the bubble probability estimation result for various cases synthetically created with the GGRBM. We will focus on the most interesting results. The simulation shows that variant C in Case 2 (low volatility and high returns) indicates a median bubble probability greater than 0.5, while the opposite occurs in Case 1. This makes sense, as Case 1 is associated with unstable samples. In contrast, Case 2 is associated with high-return samples, so it is natural to consider that we are in a likely bubble condition. Conversely, variant D in Case 2 has low bubble probabilities with a median (although with high variability). This is also interesting and counterintuitive because it suggests that high returns do not always indicate a possible bubble. This can occur, for example, when prices rise due to better corporate results and not because we are in the middle of a bubble. When accompanied by lags with high volatilities, the model indicates that we are not in the presence of a bubble or close to a critical point. Using generative models, such as the GGRBM, allows us to analyze the conditions under which a greater or lesser propensity to price bubbles exists. In Appendix E, we add another example with three cases and eight experimental variants.

5. Conclusions

Our work on the early detection of price bubbles connects with behavioral finance and complex systems theory. Behavioral finance posits that economic agents are not entirely rational; they operate with bounded rationality and are subject to cognitive biases and irrational beliefs. The formation of price bubbles becomes likely as behavioral traits influence investor decision-making. Bounded rationality, biases, and beliefs influence investors’ decision-making and behavior, playing a significant role in bubble formation. The complex systems literature complements this perspective, highlighting how nonlinear positive feedback loops can emerge within financial markets, pushing valuations far beyond intrinsic values and eventually triggering bubbles and crashes. Finally, regime changes resulting from interest rate hikes, drops in corporate earnings, regulatory changes, or extreme events like pandemics and wars jeopardize the proper functioning of financial markets and economic outlooks.

Under the above conditions, detecting stock bubbles and their explosion is crucial for the stability of financial markets. This work contributes to that goal by providing regulators and other financial decision-makers with an early detection mechanism that identifies the current potential for a phase shift in financial markets, offering timely and actionable insights. Specifically, our work alerts us to the potential financial bubble, leading to a tipping point or regime shift. This significant change occurs when the rapidly rising trend of prices reverses dramatically, resulting in steep and swift declines. While we cannot predict the timing of these tipping points, capturing nonlinear patterns associated with the emergence of bubbles offers an interesting avenue for developing early warning systems in other financial market sectors.

Our approach to architecture is straightforward. While this might seem like a disadvantage regarding the model’s effectiveness, our tests with the S&P500 index showed that it performs satisfactorily. It captures periods in the stock market that align with well-known financial crises and accurately identifies times of high volatility or instability. The detection system comprises two components. The first identifies times of extreme volatility and negative returns, which correspond to events of significant short- or long-term corrections occurring immediately after a critical turning point or regime change. The second focuses on periods of extreme accumulation of returns and price increases over short intervals, which are indicative of financial bubbles, typically happening just before the critical moment. By combining these two complementary pieces of information, the system can alert users to the likelihood of reaching a tipping point, warning about the potential onset of a regime change.

Our model does not assume a specific functional form or singular behavior. Instead, it learns representations directly from return–volatility patterns using a hybrid of unsupervised and supervised machine learning (GGRBM + BRF), thus offering greater flexibility. Our labeling approach, while empirical, provides a pragmatic balance between theoretical rigor and operational feasibility, allowing the model to generalize across multiple crises without relying on unstable parameter estimation. In line with the above, future extensions of our method could incorporate LPPLS diagnostics or hybridize machine learning with parametric bubble models to improve interpretation capabilities and early warning.

Our main results indicate that when the probability of instability is very low or near zero, conditions exist for a bubble to start forming; that is, the market index or stock prices will gradually increase at an accelerating rate. The evidence indicates that high probabilities of financial bubbles do not materialize suddenly or spontaneously; instead, they rise incrementally over time, providing valuable insights to investors, regulators, and market participants. Conversely, when the probability of instability is very high (indicating the full development of the crisis), the likelihood of an emerging bubble diminishes to values close to zero.

An important contribution of this study lies in using the generative capabilities of the trained GGRBM to perform a novel form of sensitivity analysis. Rather than solely focusing on the direct interpretability of the latent representations, which remains an open challenge in neural-network-based models, we propose an alternative route that exploits the RBM’s ability to generate synthetic samples under controlled conditions. We can simulate and analyze hypothetical market scenarios by clamping certain input features (e.g., volatility and return lags) and allowing others to vary. These synthetic cases are then evaluated using the full RBM-BRF pipeline, offering a detailed view of the model’s responses under different economic configurations.

Our approach provides three key advantages. First, it allows us to explore the model’s behavior under stylized economic regimes, such as high-return/low-volatility environments often associated with speculative bubbles, without requiring explicit labeling of real data. Second, it offers an economically intuitive way to evaluate model outputs beyond standard statistical performance metrics, thus enhancing model transparency. Third, it opens the door to integrating unsupervised learning as a feature extraction technique and a diagnostic tool for stress-testing market hypotheses.

Notwithstanding the above, our strategy has limitations. While it reveals how the model responds to stylized combinations of stock market features, it does not provide a mechanistic or structural explanation of how specific hidden units encode economic behaviors such as volatility clustering, leverage effects, or herding. The analysis is scenario-based rather than grounded in the inner workings of the learned latent space. Therefore, further studies are necessary to deepen interpretability. These could include correlation analyses between hidden unit activations and canonical market indicators or the construction of event-aligned activation profiles to examine how hidden units behave in proximity to known financial crises or bubble periods. Such approaches would complement our generative analysis and help bridge the neural representations and economic insight gap.

We believe this generative sensitivity analysis, still underexplored in the financial modeling literature, offers a valuable tool for practitioners and researchers interested in detecting and understanding financial instability. It highlights the dual utility of unsupervised learning models as powerful input transformers and economic scenario simulators. While this generative sensitivity analysis does not yield a direct economic interpretation of individual hidden units, it nonetheless provides a robust framework for uncovering latent patterns consistent with known market behaviors.

A limitation of the present study lies in the challenge of benchmarking our model against alternative early warning systems. As noted in the literature, there is no widely accepted framework for evaluating the predictive accuracy of models that aim to detect financial bubbles or anticipate systemic regime shifts. This lack of consensus complicates direct comparisons, especially since many models define the target variable differently, ranging from predicting the precise onset of financial crashes (e.g., [53,78]) to identifying broader structural instabilities using system-level indicators (e.g., [48]). Our model, in contrast, is trained to detect the presence of bubble conditions preceding well-known crises, as defined by specific volatility–return dynamics. Furthermore, while some models are theory-driven (e.g., LPPLS), our approach is fundamentally data-driven, leveraging neural architectures to learn informative representations from empirical patterns. Future work should systematically evaluate our model’s performance against classical econometric models (e.g., logit, XGBoost, LPPLS) using standardized metrics such as AUC, precision, and recall. Additionally, the development of benchmark datasets and shared evaluation protocols for financial bubble detection is still required to facilitate robust and fair comparisons across methodologies.

Our system does not replace other criteria, which may be more qualitative or involve traditional financial–economic models. On the contrary, we view our approach as a complement to market surveillance and monitoring responsibilities. In this way, our model can be continuously trained with new market information to enhance the financial decision-making of a wide range of market participants. Compared to other models discussed in the Introduction section, a downside of our approach is the time required to train the RBMs and supervised models. However, as indicated in Section 4.5, we gain the ability to generate synthetic samples and conduct a what-if analysis that enables us to take actions in advance of the emergence of risks and significant changes in the stock markets.

The phenomenon of financial bubble formation and its finale exhibit similar patterns across all stock markets, not just in the case studied here with the S&P500 index. Therefore, our model can be trained with various other market stock indices. This characteristic allows for more robust modeling and better addressing of the problem related to limited samples of bubbles and crises for analysis. This work also focused on simple bubble predictors based on cumulative returns, index changes, and volatilities. However, more sophisticated stock market indicators that capture other underlying characteristics of bubbles, such as periodogram-based features and those based on networks and multifractality, could be included. This extra information would offer a more comprehensive and robust approach to bubble analysis, enhancing instability detection regardless of the data source or market for training the model.

A direct implication of our work is the need for stock market agents to monitor investor sentiment evolution, and that models of early warning detection bubbles incorporate these variables that capture the behavioral factor into their empirical approach. While predicting a bubble’s exact tipping point is impossible, monitoring investor sentiment indicators is feasible, as they are associated with irrational investor behavior and potential price bubbles. Although these indicators do not predict exact turning points, they are invaluable for assessing risk levels and bubble price dynamics. Indicators such as the VIX, the Put/Call ratio, activity over IPOs (Initial Public Offerings) and SPACs (Special Purpose Acquisition Companies), and corporate valuation multiples are widely used to capture sentiment among investors. Similarly, the CCI (Consumer Level Confidence Index) is a forward-looking economic indicator that reflects future household consumption and savings. Also, indicators that capture uncertainty, such as the EPU (Economic Public Uncertainty Index) and the WUI (World Uncertainty Index), track uncertainty levels across the economy. Finally, news and tone sentiment analysis related to financial news and corporate information released to the market serve as tools to gauge the sentiment levels of economic agents in the stock market.

Finally, we foresee significant advancements in early warning systems’ ability to detect financial bubbles. These systems, especially those utilizing machine learning algorithms, can deliver more effective and context-rich market warning signals, which can be complemented by other AI systems, such as LLMs.