Abstract
Current covariance modeling methods work well in longitudinal data analysis. In the analysis of data with no nature order, a common covariance modeling method would be inadequate. In this paper, a study is implemented to investigate the effects of permutations of data on the estimation of covariance matrix . Based on the Hyper-sphere decomposition method (HPC), this study suggests that the change of data’s permutation breaks the consistency of covariance estimation. An alternative Hyper-sphere decomposition method with permutation invariant is introduced later in this paper. The alternative method’s consistency and asymptotic normality are studied when the observations follow a normal distribution. These results are tested using some example studies. Furthermore, a real data analysis is conducted for illustration purposes.
1. Introduction
The covariance matrix is a simple and popular method to describe the variation and correlation between random variables in multivariate statistics. A valid covariance matrix must be symmetric and semi-positive definite. In some areas, like social sciences, finance, economics and geology, people are more interested in studying the covariance matrices than mean models. Meanwhile, an appropriate working covariance matrix could increase the estimator’s efficiency. Moreover, Pourahmadi (2013) [1] suggests a good estimate of the covariance matrix can lead to an accurate statistical inference and test results. Little and Rubin (2019) [2] show that an accurate estimate of covariance is essential in dealing with missing data problems. A sample covariance is usually used to estimate the population covariance matrix for its convenience. In some special cases, for example, high-dimensional situations where the number of repeated measurements n is less than that of unknown parameters in covariance matrix, using sample covariance matrix would end up with a biased estimator of mean model [3].
Pourahmadi (1999) [4] introduced the modified Cholesky decomposition method (MCD ) into the estimation of covariance matrix by applying regression-based ideas into matrix decomposition methods. However, the interpretation of the relationship between model coefficients and correlation/variance of population Y is indirect. Pan and Pan (2017) [5] proposed the alternative Cholesky decomposition method (ACD) in 2017 that improves the correlation-interpretation problem of MCD by applying Cholesky decomposition on the correlation matrix R,
where matrix D is a diagonal matrix with standard deviations of population Y, and T is a lower-triangular matrix with unit row vectors, , whose Euclidean norm is 1. The same model regression procedure as in MCD cannot be applied in ACD due to the extra unit-norm-restriction on row vectors in matrix T. Rebonato and Jäckel (2011) [6] projected T in ACD into a unit Hyper-sphere coordinate system and proposed the Hyper-sphere decomposition method (HPC), in which elements in T are
where are corresponding new angular coordinates of vector in a unit hyper-sphere coordinate system. As a result we have
where refer to . Comparing to ACD there is an additional matrix , which is also a lower triangular with diagonal elements 0 and with the lower off-diagonal being angles ,
This method can guarantee the unit-row vectors in T for any given estimator matrix ; meanwhile, this decomposition has a geometry meaning for [7]. In that sense, the regular model regression procedure can be applied in HPC just as in Pourahmadi (2000) [8]. To ensure we can do a further triangular transpose on the liner model:
where and are design matrices and vectors for angles and variances , respectively. Meanwhile, and in Equation (2) are unknown correlation and variance components.
The current popular methods like HPC work well in longitudinal data analysis in which there is a natural order of the sample data. While in analyses like geometric and causal data there is no natural permutation of data, these methods generate estimates that depend on the permutation of sample data. The purpose of this article is to illustrate the permutation variance of covariance estimator of HPC. After that, we redefine the translation between T and , then propose an alternative Hyper-sphere decomposition (AHPC) method that inherits most of the advantages of HPC while improving the permutation variance. Meanwhile, the AHPC has a more straightforward geometrical interpretation.
2. Interpretations of Hyper-Sphere Decomposition Method
Rapisarda, Brigo and Mercurio (2007) [7] explained HPC from the aspect of Jacobi rotation. In a p-dimensional right-hand system, axis are selected as follows:
- 1.
- Set the direction of as the first axis .
- 2.
- In the plane containing vectors and , make the vertical to in a right-hand system.
- 3.
- is perpendicular to the previous axis, and , and on the same side of their cross product .
- 4.
- After the signed j axis, the next axis is defined as vertical to all previous axes and on the same direction of , while .
Cross product determines a direction perpendicular to the plane containing and given by the right-hand rule.
With all axes having been set up, the row vectors shows us the coordinate of in this Cartesian system. According to [7], the relation between angles and coordinates of s is (Figure 1a):
Figure 1.
3-dimensional illustration of HPC. (a) Interpretation of HPC method based on the idea of angles between hyper-plane. (b) Interpretation of HPC method based on Jacobi rotation.
- 1.
- We begin with . Next we turn counter-clockwise in hyper-plane with angle . Then we have . The hyper-plane is built with and .
- 2.
- Turn in with angle ; then do another rotation in with angle . We then have .
- 3.
- is built up with times turn as: .
In a p-dimensional space, is a Jacobi rotation matrix in hyper-plane with counter-clockwise angle . Besides the geometric explanation from [7], we also propose a new interpretation of the correlation between matrices T and in the HPC method, regarding angles between two hyper-planes in a row space of T (Figure 1b):
- 1.
- Lower off-diagonal entries in the first column of , s, are angles between the first and ith row vectors in T.
- 2.
- The lower off-diagonal entries in second column of contains angles, s, between plane , which is defined by the first and second row vectors in T, and , which is defined by the first and ith row vectors in T.
- 3.
- After that, elements in kth column of matrix , , are angles between two hyper-planes and .
The ’s are built with the first, second and until the kth row vectors. The angle between hyper-planes is well known in the mathematics literature, see, e.g., [Chep. 3] of Murty [9]. The angle ’s between two hyper-planes, and can be calculated recursively via
According to [7], when ’s and ’s are in , the translation in Equation (3) is unique, .
3. Order-Dependence of Hyper-Sphere Decomposition Method
Based on both explanations of the HPC method above, we show that the HPC is order-dependent from two aspects in this section.
Setting R as a correlation matrix between random sample , and after a standard Cholesky decomposition , where , based on the definition of HPC decomposition, the correlation in R is actually the value of an angle between two unit vectors and , . So eventually we say that these unit vectors in lower-triangular matrix T represent the whole correlation information of corresponding random sample . In that sense, the order of row vectors in T has almost the same representation of the order of in correlation matrix R. For example, when , under the order , we have R and the lower triangular T matrix in (4) where, for example, is the coordinate on axis for vector , which corresponds to . Then we will have a matrix as the left one in (5) where in the matrix , from left to right, is the kth Jacobi rotation angle in hyper-plane , which is built with and . For example, is the second Jacobi rotation angle in .
From the aspect of angles between two hyper-planes, we have the table for matrix as in the right one in (5) where j stands for row number. In this table, for example, is the angle between and . Hyper-plane is built with the first, second and fifth row vectors in T.
Eventually we have the same equation between and as in [10]:
where and stand for and , respectively.
If we change the order of s in correlation matrix R, simultaneously we change the corresponding row vectors, s, in lower-triangular matrix T. While the elements in new matrix may change after changing order, the relationship between and remains the same. As illustrated before, elements in matrix T are coordinates. When the order of s is different, we set up a new set of axes. That leads us to have a new set of coordinates for s. If we change the order into , then the new correlation matrix can be obtained by swapping elements in R and doing the Cholesky decomposition on we get as in (7).
Since we changed the order from position , simultaneously a new set of axes , , and are selected. As a result we will have new value for the coordinate on , and , represented as *s. Meanwhile, the corresponding matrix will be like in (8).
From the aspect of Jacobi rotation, element in is a rotating angle in plane built by axes and . Because the definition of axis s depend on the order of , which is again also the order of , still depends on the order of . We can observe that , and which shows that and only relay on , and . This explains the reason why and remain the same, respectively. Since the third vector becomes , angles on the column 3 all will have a new value, except , for obvious reasons, the angle between and equals that between and .
We summarize this dependency in Table 1. Table 1 indicates how the value of entries in depend on the order of row vectors in matrix T.
Table 1.
Dependence between entries in and row vectors in T. For example, 1, 2, 3, p means element , at row p column 3 in depends on the first, second, third and pth row vectors in T.
After proving the permutation variation of the values in the matrix, we consider how this would affect our modeling process. If the HPC method were permutation invariant, the model assumption should always hold no matter how we change the order of data Y. However, a different order gives different values of , proven above. Then, under the model (2), with the same covariates and in which the corresponding values have been rearranged according to the new order, HPC would have a different estimator of and . In some cases, even the whole model assumption would be wrong under the new permutation of sample data.
Due to the definition of translation between matrices T and , according to our study, the data order affects element-wise. Furthermore, there is no way of transforming a new angle matrix under the new data order back to its previous one only by rearranging elements. In other words, a change of the order will give a different set of values for . The regression model on elements would have different parameter estimates under varied data order. Furthermore, even the model assumption may vary according to the data order. In that sense, we say the current HPC method is order dependent. As the conclusion, under both interpretations, the HPC method is permutation variate both in terms of the matrix and models for variance components.
4. Alternative Hyper-Sphere Decomposition Method
We notice that the order dependency of the HPC method is caused by the definition of the translation between two coordinate systems, the Cartesian coordinate system for matrix T and a unique angular coordinate system for . We now propose a new definition of matrix inspired by spherical parameterization [11], which would improve the relationship between matrices R and . Then eventually the newly proposed alternative Hyper-sphere decomposition (AHPC) would be permutation invariant.
4.1. The AHPC Model
The AHPC method differs from HPC in a new definition on , the lower triangular matrix with diagonal elements equal to 0. Instead of being angles between hyper-planes in HPC, here we define our elements in as angles between row vectors s, as illustrated in Figure 2,
where to ensure the uniqueness.
Figure 2.
A geometric representation of AHPC when . The s are angles between and , respectively.
For dimension , there is a three-dimensional restriction. Any pairs of vectors and must be in a two-dimensional plan, the angle between and , to guarantee the uniqueness. Furthermore, for angles between three vectors, there are only two proper situations. The first one is when vector , and are in the same plane, and is sitting between and . The second scenario is where three vectors are not in the same plane. Consequently, for any three vectors , and , angles between each pair of vectors must satisfy that the sum of any two angles be not less than the third one and the difference between any two angles be not greater than the third one,
where . Thus for the restriction on angles between vectors in three-dimensional spaces, the model assumption must satisfy the constraint . This is not only due to the geometric fact but also is sufficient for positive semi-definiteness of correlation matrix R. When , , and must be on the same plane, which suggests that there must exist constants a and b such that . As a result, in the ith row of correlation matrix R,
which indicates the ith, jth and kth rows are linearly dependent; then the corresponding correlation matrix R would be positive semi-definite, . On the other hand, if , R would be positive definite.
We model the variance components by:
where should be a monotonic decreasing function for stationary data whose correlation only depends on absolute distances between covariates . For angles between three vectors , and , the distance is less than ; then . Another option is optimization with nonlinear programming (NLP). We can use NLP methods [12], with nonlinear inequality constraints,
to calculate the maximum likelihood estimator. The are slack parameters in the Karush–Kuhn–Tucker (KKT) condition for NLP to translate inequality constraints into equality ones. Libraries like fmincon in MATLAB and program LINGO could deal with this NLP problem as well. However, this is more of a mathematical issue than a statistical problem. So no details on NLP will be offered and studied in this paper.
Under the new definition of , we have a direct relation between R and :
where is in . This is the key to guarantee permutation invariance. Comparing with Equation (6), in AHPC only depends on one corresponding component instead of depending on a series of in a triangular area over rows in HPC. The translations between T and are
and
where s are elements in T.
When changing R into by re-ordering Y in matrix form, actually we apply a transformation matrix A on R such that:
in which R and are non-negative definite symmetric and A is a square matrix. Under this setting, finding A solves the unique form of the algebraic Riccati equation
where , which means is the Cholesky decomposition of R and setting . Consequently, there must exist a transformation matrix A such that .
Meanwhile, based on the definition of the permutation matrix, P, when changing the order of rows in matrix R from to with new order ,
where P is a permutation matrix. Thus, permutation matrix P is a solution of transformation matrix A. In that sense, after Cholesky decomposition and , the equation below is true:
Eventually, we have , noting that we need to simplify into a lower triangular matrix. We turn into a lower-triangular matrix, equivalent to selecting a new set of the coordinate axis. Furthermore, that is what makes differ from L.
In the alternative hyper-sphere decomposition method, there is a direct link between correlation and its corresponding coefficient in function (13). Only in the proof below, we use the symmetric instead of the lower triangular one. Thus, we can have:
which also means , where is the angle matrix under the new order. For example, changing into :
We can observe that entries in and are identical with locations rearranged. This guarantees the matrix in AHPC is permutation invariant.
Make model assumption on variance components as in (2) with covariate matrices and H. After changing into the new order, correspondingly we rearrange our new as . As proven before, the value of remains the same. Its corresponding and under the model assumption also remains the same. We can see that the model assumption and the estimators & will remain the same after any change of the order.
As shown above, from both the aspect of entries in and the model for variance components, the AHPC method is permutation invariant.
4.2. Estimation Method for AHPC
We use the two algorithms to get the maximum likelihood estimator of mean parameter and covariance components and jointly. Giving response that with p measurements and each measurement has n observations, under the linear model assumption, we have:
where X is a design matrix, and is an unknown parameter vector. In AHPC without loss of generality, we can set up a joint regression model for mean and covariance components as:
where and () are, respectively, the mean and standard deviation for the jth measurement . Moreover, () are angles in matrix . Dimensions for , and are , respectively. Ignoring the constant , the minus twice log-likelihood function has the following representation:
where s are residuals and . In AHPC we have
where s are diagonal elements in T. We define as a matrix, and means the elements of this matrix are s ( ; ). Since is symmetric as R, . Taking the first derivative of the log-likelihood function above with respect to , and separately, we can obtain the following score functions:
where ( ; ) is the kth element in , s are the th elements in . Based on model assumption (20), , where is the mth element in vector ; and . Set score functions above or equal to zero to get the estimator of parameters. In general, the score function does not have explicit solutions. Numerical optimization procedures such as the Newton–Raphson algorithm can be used here.
By calculating the second derivatives of the log-likelihood function with respect to , and we can have the Fisher-information matrix I,
in which,
The expected Fisher-information matrix of is as follows:
Similarly to how we deal with the score function for ,
We have:
We use the same method to find the cross section parts with :
and
where . We can show that the expectations of off block diagonal matrices and are equal to . Detail of derivation of score functions and expected Fisher-information matrix can be found in Supplementary Materials.
Giving the expected Fisher-information matrix in the form:
The Fisher-scoring algorithm can be applied here. This algorithm conveniently gives us the asymptotic covariance matrix of the estimator , and , the inverse Fisher-information matrix evaluated at the MLEs.
We propose the simple regression algorithm based on the direct relationship between and as in function (13). We can use sample variance and the correlation matrix to estimate covariance coefficients and , even if the sample covariance matrix is singular. This approach cannot be applied to the HPC method for the requirement that the correlation matrix be positive-definite for the necessity of Cholesky decomposition. Otherwise there is no direct way to translate R into .
This estimation method is more computationally efficient compared to the Newton–Raphson algorithm. Since we use LSE and GWLSE, there is no optimal searching problem, and also the grand optimality is guaranteed. As we prove before, parameter in the mean model is independent of the covariance coefficients and , so there is no problem separating the estimation procedures as above.
| Algorithm 1:Newton–Raphson Algorithm |
![]() |
| Algorithm 2:Simple Regression Algorithm |
![]() |
5. Asymptotic Property
In this section, consistency and asymptotic normality of AHPC estimators under standard situation are proven, similar to the proof in [10], under some regularity conditions.
Theorem 1.
Under the regularity conditions when sample size n goes to ∞, the maximum likelihood estimator , and is consistent with the true value, .
We already have the score functions for , and . Based on the consistency of MLEs , and in Theorem 1 , we then have:
By simple calculation, we can prove that:
Then, similar to the proof in [13], we have the asymptotic normality of estimators.
Theorem 2.
Based on the regularity conditions and Liapounov form of the multivariate central limit theorem, is asymptotically normally distributed with variance :
Theorems 1 and 2 indicate the asymptotic consistency and normality of AHPC estimators , respectively. Regularity conditions and the proof of theorems above are included in Supplementary Materials.
6. Example Studies
In this section we set up several numerical examples to test the order-dependence of HPC and permutation invariance of AHPC. We directly decompose and model the correlation matrix using both methods instead of making estimations by fitting sample data. In this way we isolate the permutation variation, which is the main difference between the two methods, from randomness.
6.1. Order-Dependency of Entries in of HPC
Giving a set of , assume there is no nature order for s. Then, under order , we will have HPC decomposition of as and the corresponding matrix,
First, we change the order into by switching the position between and . That will make into by relocating entries’ positions in the dark shaded area:
The HPC decomposition of is as follows:
The values of entries on the left-hand side of the vertical line remain the same. Only the location of the entries changes accordingly. For the reasons we stated before, on the right-hand side are all new values, except 0s. This simple simulation study testifies that the HPC method on the correlation matrix is order dependent in terms of entries in . Further numeric example studies are in Supplementary Materials.
6.2. Order Dependency of Covariance Model of HPC
Next, we study how the change of order under HPC would affect model assumption. Setting dimension , and a cubic linear model for correlation component :
where . Under the starting sequential order , . Based on model (32) and we can have , and . We plot s out against their corresponding s; we can see they form a smooth cubic line in Figure 3a. Then we switch the order 1 and 50; under the new order we can have the new correlation matrix by re-allocating some relevant elements. By applying the HPC method, we have and . Then we rearrange the design matrix according to the new order. Again we plot against their corresponding in Figure 3b. It can be observed that all values are different since the order changed.
Figure 3.
Plot of s against their corresponding s. (a) Plot of s against their corresponding s, with under sequential order , and the solid line in the true model. (b) Plot of against under new order in dots. The solid line is the fitted line with , while the dotted line stands for the model.
We fit a new cubic liner model on the s under the new order. As a result, new is generated, . It is obvious that in Figure 3a is quite different from the original , which is plotted in Figure 3b.
Even worse, when using to create an estimate , we find is a poor estimator of with a large sum of the absolute difference (SAD), the Frobenius norm . That implies that the model assumption based on the original order may not even be appropriate under the new order. Furthermore, to test how the HPC method reacts to different kinds of order changes, we do a series of simulations as described below.
| Algorithm 3:HPC Reactions to Different Permutation Changes |
|
First, we test how the distance between the pair of data we switch would affect the SAD between the estimator and true value. Here we do 49 times different switches between order 1 and k (). The results can be seen in Figure 4, from which we can observe the SAD between estimators and the true value getting bigger with the increase of distance between switched positions. That indicates that the further position being switched, the more significantly biased the estimator becomes.
Figure 4.
Plot of the difference between estimators and true value when switching order 1 and k ().
Second, we test how the first position we switch would affect the estimators. This time we also use the procedure above, while setting and . Then we have Figure 5, from which we can see the bias decrease as further change happens from the first order. Furthermore, comparing with Figure 4, we can see the effect of distance between the pair of switched positions is much stronger than that of the minimum number being changed.
Figure 5.
Plot of differences before and after switching order j and ().
6.3. Permutation Invariance of Entries in in AHPC
Now we apply the same simulation study on AHPC. Giving the same starting correlation matrix as above, after Cholesky decomposition on , the same as the one before is generated. Based on Equation (14) we have for the AHPC. Then we swap positions 3 and 5, which gives and as those in HPC. According to the new proposed definition of the translation between and T in AHPC, we have for AHPC:
all those gray shaded parts have positions re-allocated, but the value remains the same as the original for AHPC. We can see that, after the order changes, all values in matrix under AHPC remain the same; only their position changes to fit the new order. In that sense, we say AHPC is order invariant in terms of matrix .
6.4. Permutation Invariance of Covariance Model in AHPC
Then to find out how the AHPC modeling reacts to the order change, we implement the same and cubic linear model assumption (32) under sequential order . Applying the AHPC method and under the model assumption, we have , and . After switching the order between 1 and 50, applying AHPC we have , and . This time we can see all values in are as same as those in . Plotting s against their corresponding s, we have Figure 6.
Figure 6.
Plots of s in the AHPC method against s under the original sequential order and switched order.
Furthermore, we fit a cubic linear model to that, under the new order, against their corresponding s, involves entries from . Then we get an estimator . Moreover, the also has zero bias from . In conclusion, the AHPC is order invariant in terms of both value in and covariance modeling.
The model assumptions under HPC and AHPC are not compatible. Continue the simulation above; under the cubic linear model assumption, when correlation matrix is generated from HPC, we decompose by AHPC and end up with a plot of against in Figure 7a. The other way around, when correlation matrix is created by AHPC we decompose by HPC and plot Figure 7b. As we can see, the model assumptions in HPC and AHPC are not compatible with each other. Based on the asymptotic research, the consistency only holds when the model assumption is correct. Thus there is no point to doing the cross method estimation comparison under one identical model assumption.
Figure 7.
Cross method results. (a) s against s from de-composited by the AHPC method. (b) s against s from de-composited by the HPC method.
7. Real Data Analysis
In this section, we analyze a set of weather data by HPC and AHPC. We focus on comparing the estimators of the correlation matrix. Furthermore, there is no natural order in this weather data set. In this section, jmcm package in R [5] is used for estimations of the HPC method. Function lm is applied in R to do the linear regression for AHPC. These weather data are collected from the UK government public web page, Met Office. We select different weather stations in the UK. They are allocated in Ballypartick Forest, Cambridge, Lewick, Leuchars and Sheffield. We cut off the data before 1962 and make this data set balance, leaving us with sample size. There is no missing value in this data set. The average minimum temperature in each month is recorded in Celsius degrees.
For comparison between HPC and AHPC, our interest is on the estimator of correlation matrices. Thus, sample variance is used as the estimator of the variance for station i. We set the initial order alphabetically: Ballypartick Forest, Cambridge, Lewick, Leuchars and Sheffield. Then we shift this initial order by relocating the Sheffield station to the beginning. Meanwhile, based on the non-parametric analysis of under both methods with respect to distances between climate stations i and j, solid lines are plotted in Figure 8. Combined with the consideration of AIC and BIC, we assume three different polynomial models for in HPC and AHPC under two orders, respectively.
Figure 8.
Solid and dotted lines are non-parametric smooth and fitted model of covariance components (circles in each plots) under HPC and AHPC methods separately with respect to distances between five stations, under initial and shifted order.
For s of the HPC method, we assume a linear model under the initial order and a quadratic one under the shifted order. On the other hand, to keep the model monotonic decreasing, we model angle parameters s in AHPC instead of s. We assume a linear model under both orders.
Under both orders, by applying HPC and AHPC, we can get four regression results. The fitted models are plotted with dot lines in Figure 8. Cross comparison of these results with the sample correlation matrices under these two orders, by their relative errors defined as , where is Euclidean norm, and and are the sample and estimated correlation matrix, respectively. We should notice that the norm of sample correlations remains the same, , and the norm of the difference between sample correlation matrices under both is order .
Observing the results in Table 2 and Table 3, it is obvious that the estimator of HPC depends on the order of the data, while AHPC presents a consistent estimating result. Even redoing the modeling process from the model selecting stage, we observe from the relative errors in Table 2 that the relative errors between sample and estimated correlation generated on are different in HPC. As one conclusion, under different permutations, the estimating results of HPC would vary. Consequentially, it is improper to use the HPC method to model the covariance matrices of data without a natural order for its failure to offer a permutation consistent estimator.
Table 2.
Resulting HPC estimates and standard error of correlation components , under initial and shifted order, respectively. Relative errors between sample and estimated correlation under different orders are cross compared.
Table 3.
Resulting AHPC estimates and standard error of correlation components , under initial and shifted order, respectively. Relative errors between sample and estimated correlation under different orders are cross compared.
On the other hand, for AHPC, Table 3 shows the consistency of these estimators under different orders. Moreover, there is an interpretation advantage in AHPC comparing to HPC. For example, in this real data analysis, the interpretation of the relationship between correlation and the distance between stations j and k are not obvious in the HPC model. Meanwhile, the AHPC model for is monotonic decreasing, suggesting increases with distance . Moreover, is a monotonic decreasing function. Thus decreases with respect to the distance between stations.
8. Conclusions and Discussion
In this paper, we addressed the permutation variation of HPC through its geometrical interpretations. Then AHPC for covariance modeling was proposed. AHPC was proven to improve the order-dependence issue of HPC. Furthermore, the direct relation between and R in AHPC provides an advantage in making model assumptions, parameter estimations and statistical interpretations. However, due to the limitation of the relation between angles, the model assumption for angles in AHPC must satisfy an extra constraint.
Both HPC and AHPC can only guarantee semi-positive definiteness. The reason behind this drawback of both methods is the same. From the geometrical interpretations above, we can see that the definition of angles in both methods can only ensure the symmetry and diagonal elements being 1s in the correlation matrix R. By changing the inequality constraint on angles in AHPC to strict less, we can simultaneously make sure the correlation matrix is positive definite.
These four covariance modeling methods we mentioned in this paper have their advantages. For the most accurate estimator, an appropriate model assumption is essential. Thus we may use a certain decomposition method based on the data, since different methods would generate their pattern against coefficients. Under an appropriate model assumption, estimators in all four methods are consistent. AHPC, MCD and ACD can do the model selection visually, while in some cases, model selection in HPC can only be based on statistical criteria.
There are some potential researches available on this AHPC method. As we see in simulation examples, sometimes the pattern is too complicated to fit with the linear model; the non-parametric and semi-parametric model could be applied in the AHPC method, similar to the studies of [14,15].
Supplementary Materials
The following are available online at https://www.mdpi.com/article/10.3390/math10040562/s1.
Author Contributions
Conceptualization, J.P. and Q.L.; methodology, J.P. and Q.L.; software, Q.L.; validation, J.P.; writing—original draft preparation, Q.L.; writing—review and editing, Q.L.; visualization, Q.L.; supervision, J.P.; project administration, J.P. All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by the National Natural Science Foundation of China grant number 11731011 and 11871357.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
Not applicable.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Pourahmadi, M. High-Dimensional Covariance Estimation; Wiley: Hoboken, NJ, USA, 2013. [Google Scholar]
- Little, R.J.; Rubin, D.B. Statistical Analysis with Missing Data; John Wiley & Sons: Hoboken, NJ, USA, 2019; Volume 793. [Google Scholar]
- Zhang, C.H.; Huang, J. The sparsity and bias of the lasso selection in high-dimensional linear regression. Ann. Stat. 2008, 36, 1567–1594. [Google Scholar] [CrossRef] [Scilit]
- Pourahmadi, M. Joint mean-covariance models with applications to longitudinal data: Unconstrained parameterisation. Biometrika 1999, 86, 677–690. [Google Scholar] [CrossRef] [Scilit]
- Pan, J.; Pan, Y. jmcm: An R package for joint mean-covariance modeling of longitudinal data. J. Stat. Softw. 2017, 82, 1–29. [Google Scholar] [CrossRef] [Scilit]
- Rebonato, R.; Jäckel, P. The Most General Methodology to Create a Valid Correlation Matrix for Risk Management and Option Pricing Purposes. 2011. Available online: https://ssrn.com/abstract=1969689 (accessed on 8 May 2012).
- Rapisarda, F.; Brigo, D.; Mercurio, F. Parameterizing correlations: A geometric interpretation. IMA J. Manag. Math. 2007, 18, 55–73. [Google Scholar] [CrossRef] [Scilit]
- Pourahmadi, M. Maximum likelihood estimation of generalised linear models for multivariate normal covariance matrix. Biometrika 2000, 87, 425–435. [Google Scholar] [CrossRef] [Scilit]
- Murty, K.G. Computational and Algorithmic Linear Algebra and n-Dimensional Geometry; World Scientific Publishing Company: Singapore, 2014. [Google Scholar]
- Zhang, W.; Leng, C.; Tang, C.Y. A joint modelling approach for longitudinal studies. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 2015, 77, 219–238. [Google Scholar] [CrossRef] [Scilit]
- Bates, D.M.; Watts, D.G. Nonlinear Regression Analysis and Its Applications; Wiley: New York, NY, USA, 1988; Volume 2. [Google Scholar]
- Sallan, J.M.; Lordan, O.; Fernandez, V. Modeling and Solving Linear Programming with R; OmniaScience: Barcelona, Spain, 2015. [Google Scholar]
- Chiu, T.Y.; Leonard, T.; Tsui, K.W. The matrix-logarithmic covariance model. J. Am. Stat. Assoc. 1996, 91, 198–210. [Google Scholar] [CrossRef]
- Wang, N.; Carroll, R.J.; Lin, X. Efficient semiparametric marginal estimation for longitudinal/clustered data. J. Am. Stat. Assoc. 2005, 100, 147–157. [Google Scholar] [CrossRef] [Scilit]
- Fan, J.; Huang, T.; Li, R. Analysis of longitudinal data with semiparametric estimation of covariance function. J. Am. Stat. Assoc. 2007, 102, 632–641. [Google Scholar] [CrossRef] [Scilit] [PubMed]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).









