Comparison between Information Theoretic Measures to Assess Financial Markets

Information theoretic measures were applied to the study of the randomness associations of different financial time series. We studied the level of similarities between information theoretic measures and the various tools of regression analysis, i.e., between Shannon entropy and the total sum of squares of the dependent variable, relative mutual information and coefficients of correlation, conditional entropy and residual sum of squares, etc. We observed that mutual information and its dynamical extensions provide an alternative approach with some advantages to study the association between several international stock indices. Furthermore, mutual information and conditional entropy are relatively efficient compared to the measures of statistical dependence.


Introduction
Financial markets are dynamic structures which exhibit nonlinear behavior. The correlation structure of time series data for financial securities incorporates important statistics for many real world applications such as portfolio diversification, risk measures, and asset pricing modeling [1,2]. Strict linear correlation studies are likely to neglect important financial system characteristics. Traditionally, non-linear correlations with higher order moments have been studied.
Some researchers considered the information measures theoretic approach to study this statistical dependence because of its potential to identify a stochastic relationship as a whole, including linearity and non-linearity (refer to [3][4][5][6]), and so making it a general approach. For example, network statistics based on mutual information can successfully replace statistics based on the coefficient of correlation [7]. Ormos et al. [8] defined continuous entropy as an alternative measure of risk in an asset pricing model. Mercurio et al. [9] introduced a new family of portfolio optimization problems called the return-entropy portfolio optimization (REPO), which simplifies the computation of portfolio entropy using a combinatorial approach. Novais et al. [10] described a new model for portfolio optimization (PO), using entropy and mutual information instead of variance and covariance as measurements of risk.
Ghosh et al. [11] presented a non-parametric estimate of the pricing kernel, extracted using an information-theoretic approach. This approach delivers smaller out-of-sample pricing errors and a better cross-sectional fit than leading factor models. Batra et al. [12] compared the efficiency of the traditional Mean-Variance (MV) portfolio model proposed by Markowitz with the models that incorporate diverse information theoretic measures such as Shannon entropy, Renyi entropy, Tsallis entropy, and two-parameter Varma entropy. Furthermore, they derived risk-neutral density functions of multi-asset to model the price of European options by incorporating a simple constrained minimization of the Kullback measure of relative information to evaluate the explicit solution of this multi-asset option pricing model [13].
In this paper, we propose a way of integrating non-linear dynamics and dependencies in the study of financial markets using several information entropic measures and some dynamical extensions using mutual information, such as the normalized mutual information measure, relative mutual information rate, and variation of information. We show that this approach leads to better results than other studies using a correlation-based approach on the basis of nine international stock indices which are traded continuously over eighteen years, i.e., (2001)(2002)(2003)(2004)(2005)(2006)(2007)(2008)(2009)(2010)(2011)(2012)(2013)(2014)(2015)(2016)(2017)(2018).
The paper is divided into four sections. In Section 1, we give the theoretical framework of the information measures and methodology used. In Section 2, we describe financial market daily closing price data of some international stock indices with comparative results. In Section 3, we give a detailed comparative study with the help of tables and figures based on our analysis of the several international stock indices networks. In Section 4, we conclude the paper with a discussion.

Shannon Entropy
Shannon [14] introduced the concept of entropy in the communication system. For the given probability distribution p i = P(X = x i ), i = 1, 2 . . . , n of a random variable X, the Shannon entropy H(X) is given by This entropy has found many applications in various diversified fields specifically as a measure of volatility in financial market [15].

Joint Entropy
The joint entropy is the uncertainty measure allied with a joint probability distribution of two discrete random variables, say X = {x 1 , x 2 , x 3 , . . . , x s } and Y = {y 1 , y 2 , y 3 , . . . , y t }, and is defined as where p ij = p(x i , y j ) = P(X = x i , Y = y j ) is the joint probability density function of random variables X and Y.

Conditional Entropy
The conditional entropy, also called the equivocation, of the random variable Y = {y 1 , y 2 , y 3 , . . . , y t } given X = {x 1 , x 2 , x 3 , . . . , x s } is defined as The joint and conditional entropy measures follow some properties as given below:

Mutual Information
The mutual information, also called the transinformation, of Y relative to X is defined as: This information measure follows some properties, as given below: • Non-negative: I(Y|X) ≥ 0 • Symmetric: I(Y|X) = I(X|Y).
It is easy to see that where p i = P(X = x i ) and q j = P(Y = y j ) are marginal probability distributions of X and Y, respectively; and p ij = p(x i , y j ) = P(X = x i , Y = y j ) is the joint probability density function of X and Y. Next, we can estimate the entropy of a specific stock X from the stock index Y, refer to [16], as Further, for Y as independent and X as dependent variables, the total variance of the stock can be subdivided into two parts: the explained sum of squares, i.e., systematic risk and the residual sum of squares, i.e., specific risk.
Next, we summarize some of the extensions of mutual information measure as follows:

Relative Mutual Information
The relative mutual information measure is directly comparable to the coefficient of determination (R 2 ), because both estimate the proportion of variability demonstrated by an independent variable in the dependent variable [16].

Normalized Mutual Information
For the two discrete random variables say, X and Y, the normalized mutual information is defined as: In addition to capturing strong linear relationships, this non-linear approach based on mutual information captures the non-linearity found in the volatile market data which could not be captured by the approach based on correlation coefficient [17].

Global Correlation Coefficient
This standardized measure of mutual information can be used as a predictability measure based on the distribution of empirical probability, and is defined as: It ranges from 0 to 1 and records the linear as well as the nonlinear dependence between two discrete random variables, say X and Y, and is easily comparable with the linear correlation coefficient [18].
, if there is a perfect relationship between the vectors X and Y.

Variation of Information
This information measure, denoted as V I XY , is a true estimate of distance between two discrete random variables, say X and Y and, as such it obeys the triangle inequality [19]. Furthermore, it is closely related with mutual information measure and is given as It is easy to see that , when X and Y are dependent where H(X) is the entropy of the discrete random variable X = {x 1 , x 2 , x 3 , . . . , x s } and I(X, Y) is mutual information between two discrete random variables X = {x 1 , x 2 , x 3 , . . . , x s } and Y = {y 1 , y 2 , y 3 , . . . , y t } and H(X, Y) is the joint entropy of X and Y.

Data and Result
The main objective of this paper is to show that information theoretic measures present a decent solution to the problem of dependence in the financial series data. For this purpose, we considered 14 time series comprising the daily closing price of the international stock indices and their selective stocks which were traded continuously between 1 January 2001 and 31 December 2018 (about 4433 observations in each time series).
To apply entropic measures in operation and to find their properties, we obtained daily closing price data over eighteen years of international stock market indices, namely IBOVESPA (Brazil), Hang Seng (Hong Kong), SSE Composite (China), Dow Jones Industrial Average (United States), CAC 40 (France), DAX (France), NASDAQ (United States), Nifty 50 (India) and BSE sensex (India), accessed on 1 August 2020 from in.finance.yahoo.com, the free financial data online platform. Furthermore, we make a comparison of two Indian market indices namely, NIFTY 50 and S&P BSE sensex with their common stocks. During the complete selection process, daily closing price data filtered to select stocks listed in their respective indices and as a result due to inadequate data, five stocks were selected from the study.
Our main emphasis is on determining the dependence level between each stock and its stock index, i.e., to select a stock which is more dependent on its relative index and this has been estimated by the information theoretic measures. Further, to compare two different indices with their stocks and measure their level of dependence, the concept of mutual information is used as analogous to covariance and normalized mutual information is calculated akin to the Pearson's coefficient of correlation.    We discussed the choice of bin counts for the nature of calculation of statistical and information theoretic measures, see Tables 6-11. In general, the count of bins should be selected on the basis of availability of the number of observations of time series data. In Tables 6 and 7 we show a count of total number of bins for the indices NIFTY 50 (342 bins) and BSE sensex (341 bins), respetively; we come to a conclusion that BSE sensex is more uncertain stock index as compared to NIFTY 50 and furthermore, Axis bank is the underlying stock which is more dependent compared with other stocks.    In other words, by observing the Axis bank stock, more information about NIFTY 50 and BSE sensex can be obtained. Furthermore, by observing ONGC stock, remaining uncertainty about the aforementioned indices is greater than other stocks. We have also come to the conclusion that the measure of global correlation coefficient can be useful if a lower count of bins are considered, otherwise this measure gives a similar output to all stock variables. Thus, in this case, the large count of data bins would not be a reasonable choice for comparative analysis of information theoretic measures with statistical measures, as even in very small change in the prices no depth-patterns are to be found. The coefficient of correlation is a long-standing estimator of the degree of statistical dependence; however mutual information has advantages over it. Furthermore, it is evident from from Tables 7 and 8 that the Shannon entropy, mutual information and the conditional entropy perform well in accordance with the systematic risk and the specific risk derived through the linear market model. Furthermore, from Table 8, as per maximum count of bins (i.e., 403), the index BSE sensex can be used as the appropriate variable for the dependency of the index NIFTY 50.

Tables and Figures
Furthermore, studies based on other stock market indices and foreign exchange markets can also be carried out to further examine the effectiveness of this approach. One cannot directly visualize dependency of a stock on its index using technical analysis; therefore the investor has to work on theoretical aspects too. In Table 9, as per maximum count of bins (i.e., 1132), the international index DAX can be used as the appropriate variable for the dependency of DOW Jones in comparison of other indices, such as CAC 40 and NASDAQ.    The distribution of each stock/index is shown on the diagonal; on the lower diagonal, the bi-variate scatter plots with the fitted higher order degree polynomial are shown and on the upper diagonal, the correlation value plus the significance level (p-values: 0.001) as stars ("***").

Discussion
We find that the information measures such as mutual information is an ideal way to expand the financial networks of stock market indices based on statistical measures, while the measure of variation of information presents a different side of the markets but produces networks with troubling characteristics. The proposed methodology is sensitive to the choice of data bin counts in which the daily closing prices of market data are separated and require a large sample size. Furthermore, the comparison of entropy measures with statistical measures of dependent stock indices, such as Shannon entropy vs. variance of a stock, conditional entropy vs. residual sum of squares, coefficient of determination vs. the relative mutual information rate, and correlation vs. normalized mutual information measures indicate the further existence of non-linear relationships that are not identified by the statistical correlation measure and hence present a different aspect of market analysis.
Information measures such as Shannon entropy and one parametric entropy, such as Renyi and Tsallis entropies, were studied extensively by many researchers for analyzing volatility in financial markets. However, this concept can still be explored further using various two parametric entropy measures. The non-linear fluctuations in a variety of markets (i.e., stock, future, currency, commodity, and cryptocurrency) require powerful tools such as entropy for extracting information from the market that is otherwise not visible in standard statistical methods. Further, we can extend the information theoretic approach used in this paper to highly volatile financial markets.

Conclusions
This paper uses information theoretic concepts such as entropy, conditional entropy, and mutual information and their dynamic extensions to analyze the statistical dependence of the financial market. We checked some equivalences between information theoretic measures and statistical measures normally employed to capture the randomness in financial time series. This approach has the ability to check the non-linear dependence of market data without any specific probability distributions. We used different international stock indices to address our problem. Mutual information and conditional entropy were relatively efficient compared to the measures of statistical dependence. From the results obtained, it is inferred that the information theoretic approach observes the dependency effectively and can be considered to be an efficient approach to study financial markets. In addition, these measures can provide some additional information for financial market investors.