A Vine Copula-Based Global Sensitivity Analysis Method for Structures with Multidimensional Dependent Variables

: For multidimensional dependent cases with incomplete probability information of random variables, global sensitivity analysis (GSA) theory is not yet mature. The joint probability density function (PDF) of multidimensional variables is usually unknown, meaning that the samples of multivariate variables cannot be easily obtained. Vine copula can decompose the joint PDF of multidimensional variables into the continuous product of marginal PDF and several bivariate copula functions. Based on Vine copula, multidimensional dependent problems can be transformed into two-dimensional dependent problems. A novel Vine copula-based approach for analyzing variance-based sensitivity measures is proposed, which can estimate the main and total sensitivity indices of dependent input variables. Five considered test cases and engineering examples show that the proposed methods are accurate and applicable.


Introduction
Variance-based global sensitivity analysis (GSA) is very commonly used in the area of structural safety [1][2][3]. It can calculate the contribution of each input variable or group variables to the variance of the output response, thus allowing for selection of the important variables that have great influence on the output, while the randomness of unimportant variables can be ignored. Several methods exist to perform variance-based GSA of model outputs when the inputs are independent [2][3][4][5][6], but it is still a challenge for the dependent cases.
For dependent input variables, there are some techniques proposed to solve Sobol' indices. Xu and Gertner [7] divided the contribution of inputs into two components: the independent and dependent contributions, and proposed a regression-based method for estimating different contributions of the inputs. Similarly, Li et al. [8] introduced a unified framework including covariance in the decomposition of the model output variance, which can distinguish the structural and correlative contribution of a nominal input. Kala [9] performed GSA based on entropy, and proposed a novel method from differential entropy to alternative measures. Kucherenko et al. [10] applied a Gaussian copula method to variance-based global sensitivity analysis, and introduced a more general approach. On this basis, Song et al. [11] proposed using an adaptive copula method to solve Sobol' indices, which can describe different types of correlations. The above methods are mainly aimed at the case of bivariate correlation. Yet, considering that many input variables are multidimensional and interrelated in engineering problems, it is necessary to propose an effective method with which to measure multidimensional correlation.
Over the past few years, the Nataf model proposed by Kiureghian et al. [12] has been widely used in structural safety engineering with multidimensional correlated variables. Lebrun and Dutfoy [13] proved that the Nataf model is essentially equivalent to multivariate Gaussian copula. Besides, Eryilmaz [14] studied multivariate Archimedean copula and applied multivariate Clayton and Gumbel copulas to the reliability analysis of weighted-k-out-of-n systems with dependent components. Daul et al. [15] discussed the grouped t-copula with an application to credit risk. Cossette et al. [16] reviewed the Farlie-Gumbel-Morgenstern (FGM) copula and expanded it to multidimensional situations. The above copula methods require that the correlations between different variables conform to the same copula function. However, many of the multidimensional variables in practical engineering have different correlations. For example, in the variable-amplitude load fatigue problem, the fatigue life under different stress levels has different correlations [17]. In flood frequency analysis, the flood peak value, total amount and duration are complexly correlated [18]. For such problems, the above multivariate copula models are subject to errors [19], and the correlations should be measured by different copula functions.
In this paper, we introduce Vine copula [20][21][22] to the variance-based global sensitivity analysis of complex multidimensional correlation problems. Vine copula can decompose the joint PDF of multidimensional variables into some bivariate copula functions of original variables and conditional variables [20], so as to measure different correlations among the multivariate variables with different copula functions. After the decomposition, the Akaike Information Criterion (AIC) method [23] or one of other methods [24,25] is required to select the optimal bivariate copula function, which can separately measure the correlation among the input variables. For engineering problems with multidimensional correlated variables, the Vine copula method can use optimal copula to separately measure the correlation of different variables. Thus, the complex dependence structure of variables can be fitted to the maximum extent, and the most accurate possible solution of S i and S tot i can be obtained. Due to the excellent properties of Vine copula, it has been applied in structural safety and reliability. Torre et al. [26] formalized the framework needed to build Vine copula models of multivariate inputs and combine them with virtually any uncertainty quantification method. Nazih et al. [27] studied the specific problem of structural reliability as an application of Vine copula. Sarazin et al. [28] used Vine copula to perform reliabilityoriented sensitivity analysis in the presence of data-driven epistemic uncertainty. Inspired by the above works, we applied Vine copula to perform variance-based sensitivity analysis. This paper is organized as follows: Section 2 presents a detailed introduction of Vine copula, including its basic theory and classification. The process of solving variance-based global sensitivity analysis based on Vine copula is introduced in Section 3. Section 4 analyzes some numerical and engineering tests. Conclusions are summarized in Section 5.

Vine Copula
Vine copula is an efficient mathematical tool to decompose the multidimensional joint PDF into the continuous product of several two-dimensional copula functions and marginal PDF [20]. Consider a set of n-dimensional correlated variables X = (X 1 , X 2 , · · · , X n ), the joint PDF f (x 1 , x 2 , · · · , x n ) of that can be decomposed as follows: where f X i (x i ), i = 1, 2, · · · , n is the marginal PDF of the input variable X i and f X j |X 1 ,X 2 ,··· X j−1 (x j ), j = 2, 3, · · · , n is the conditional PDF of X j under X 1 , X 2 , · · · , X j−1 constraint. For the multidimensional independent cases, f X j |X 1 ,X 2 ,··· ,X j−1 (x j ) = f X j (x j ), i.e., the joint PDF of the variables is equal to the product of marginal PDFs.
For the case of two-dimensional variables, the joint PDF is f X (x 1 , x 2 ) = f X 1 (x 1 ) f X 2 |X 1 (x 2 ) and the conditional PDF and be expressed as: where c 12 (·) is copula density function of input variables X 1 and For the case of high-dimensional variables, the decomposition of Equation (1) is not unique. Bedford and Cooke [20,21] proposed Vine copula to describe the different decomposition methods and introduced a regular Vine (R-Vine) structure, in which C-Vine and D-Vine are two common structures. C-Vine can be well used to describe correlations among variables when a particular variable is known to be a key variable that governs interactions in the dataset (e.g., X 1 is the key variable of C-Vine in Figure 1), while D-Vine is suitable for describing correlated variables of the same status. An n-dimensional Vine copula model consists of n − 1 trees T j (j = 1, 2, · · · , n − 1 , which has n − j + 1 nodes and n − j sides. Each node represents an unconditional or conditional variable, and each side represents a two-dimensional copula density function. The nodes in tree T j are made up of the sides in tree T j−1 and related only to the sides that have a common node in tree T j−1 . According to the logical structure of C-Vine and D-Vine models, the joint PDF of n-dimensional variables can, respectively, be expressed as Equations (3) and (4):  Figures 1 and 2 can be expressed separately as follows. It can be seen that the notation of D-Vine resembles independence graphs more than that of C-Vine. Therefore, we mainly adopted C-Vine in order to emphasize the influence of dependence on sensitivity indices. The above decomposition process involves conditional distribution F X j |X 1 ,··· ,X j−1 (x j ) and F X i |X i+1 ,··· ,X i+j−1 (x i ). For the convenience of expression, we introduce the function h(·) to express the binary conditional distribution, while h −1 denotes the inverse function of h(·).
where u i = F X i (x i ) and u j = F X j (x j ) are the marginal cumulative distribution functions (CDF) of X i and X j , C(u i , u j ) denotes copula function and α denotes the parameter of copula function, which can be estimated by the AIC method [23] or other methods [24,25]. Table 1 lists the h(·) and h −1 (·) functions of some common copula functions [29]. According to the decomposition process in Equation (3) or (4), the multidimensional joint PDF can be transformed into the product of marginal PDF and several binary copula functions. Thus, we can use the conventional binary copula functions for variance-based global sensitivity analysis with multidimensional correlated input variables. Table 1. The functions h and h −1 of some frequently-used copula functions.

Variance-Based GSA Based on Vine Copula
Considering a model function Y = g(X) defined in R n with an input vector X = (X 1 , X 2 , · · · , X n ), Sobol [1] and Saltelli [2] put forward the variance-based global sensitivity indices based on ANalysis Of VAriance (ANOVA) decomposition: where D denotes the variance of model output Y, while D i and D tot i are the conditional variances of Y under X i constraint. The full expressions for S i and S tot i are given by the following formulas: Kucherenko et al. [10] derived the formula of variance-based global sensitivity indices under correlated variables based on the conditional variance theory: where X and X are unconditional sample matrices generated according to the joint PDF of input variables, while (X 1 , · · · , X i−1 , X i+1 , · · · , X n ) and X i are conditional samples generated by conditional PDFs f X ∼i |X i (x ∼i ) and f X i |X ∼i (x i ), respectively. Tarantola and Mara [30] indicated that S i measures the amount of variance of X i due to X i and its dependence with X ∼i but does not include the interactions of them, while S tot i accounts for the contributions of X i and its dependence with X ∼i by ignoring the correlations among variables. Therefore, the dependence among variables may lead to S i > S tot i (dependent effects larger than interacted effects). By comparing the values of the two indices, we can get the independent contributions, dependent contributions and interacted contributions of the variables, so as to perform factor ranking/fixing, variance cutting, etc.
According to Equations (12) and (13), Song et al. [11] introduced two copula-based methods for solving S i and S tot i under the case of two-dimensional correlated variables. The first method uses the copula function to approximate the real joint distribution, while the other approach uses the copula function to decompose the joint PDF into the product of the marginal PDF and copula PDF. Song et al. [11] also concluded that the second method is less computationally efficient than the first method, so we mainly discuss the first copula-based method in this paper. For the sake of description, we denote the two methods based on Vine copula as VC1 and VC2. In this section, we extend VC1 to the case of multidimensional correlation, where there is a need to decompose the joint PDF of the input variables into the product of several two-dimensional copula functions and marginal PDF using the Vine copula function (details of VC2 are presented in Appendix A).
The basic idea of VC1 is to use the copula function to approximate the real joint distribution, and generate unconditional and conditional samples according to the selected optimal copula function. This method can rewrite (12) and (13) as follows: where N is the number of samples. After the decomposition of the joint PDF with the Vine copula function, each optimal two-dimensional copula function and its parameters needs to be inferred by using the known samples. According to the selected optimal copula function, we generate samples according to the following ideas and steps [20,31].
Assuming that (x 1 , x 2 , · · · x n ) is a group of samples of input variables X, according to Rosenblatt model [32], the marginal and conditional distribution of (x 1 , x 2 , · · · x n ) can be expressed as follows: where (r 1 , r 2 , · · · r n ) are independent and uniformly distributed between 0 and 1. Correspondingly, we can get the inverse transform of Equation (16).
Assuming that (u 1 , u 2 , · · · u n ) are the marginal CDF values of the input variables, i.e., , · · · , u n = F X n (x n ), the sampling procedure is as follows: 1.
Let u 1 = r 1 , and obtain Let Take the iteration repeatedly, and get a group of unconditional samples (x 1 , x 2 , · · · x n ).
Considering the above sampling method involves a conditional copula function, the generation of conditional samples is similar to the above procedure. There is just a need to construct independent uniform variables as (r 1 , r 2 , · · · , r i , · · · , r n ) and (r 1 , r 2 , · · · , r i , · · · , r n ), and thus the specific steps are omitted. The above procedure can be briefly summarized as in the following figure. After getting the unconditional and conditional samples, S i and S tot i can be computed according to Equations (14) and (15). The solution process is concluded as Figure 3.

Test Cases
In this section, the developed approach is illustrated by five different models with multidimensional correlated input variables.
The first test with lognormal distributed variables' numerical solutions is computed and compared with the reference results. We also compare the solutions obtained by the two copula-based methods and discuss the effect of different Vine copulas on the results. Test 2 compares the results obtained by the developed approach and traditional Nataf model by analyzing a four-dimensional dependent model. The third case's variables are uniformly distributed and the influence of Kendall τ on the results is analyzed. The last two tests are engineering cases with complex model functions and correlations, in which Test 5 combines the proposed method and the finite element models.

Test 1. Example with Complete Probability Information
Consider a model function Further, we assume the input variables to have lognormal distribution with parameters µ and σ. By exponent transformation of the above normally distributed samples X N , we can obtain the lognormal distributed samples X L with unknown correlations, which are used for copula parameter estimation, and the optimal copula functions can be selected by the AIC method. The results are shown in Table 2, and the AIC values of the optimal copula are marked in bold.  Table 2 shows that the optimal bivariate copula functions among all variables are Gaussian copula, so we use bivariate Gaussian copula to construct Vine copula. The results obtained by the two methods based on the Vine copula function are listed in Table 3, while the reference solutions obtained by exponential transformation and the approximate solutions based on the Nataf model are also listed in Table 3.  Table 3 shows that the two Vine copula methods obtain almost the same results as reference solutions, which illustrates the feasibility and accuracy of the developed methods. For this case, Vine copula consists only of Gaussian copula functions, which means Vine copula is essentially the same as the Nataf model (multivariate Gaussian copula) for this model. Therefore, the results obtained by Vine copula methods and Nataf methods are very close.
Furthermore, we notice that the results obtained by VC2 need more sample points than VC1 but have less accuracy. The convergence curve of the results obtained by the two methods with the number of samples is shown in Figure 4. We can see that the convergence of VC1 is better than that of VC2, which is consistent with the previous conclusion of Song et al. [11]. This may be because VC2 needs to calculate the copula density function when calculating Sobol' indices, which may lead to numerical errors and cause abrupt changes in the results. Therefore, we only discuss VC1 in the following tests.
The above method uses C-Vine to construct the joint PDF of input variables (X 1 as the key variable), i.e., the two-dimensional variables required for sampling are (X 1 , X 2 ), (X 1 , X 3 ) and (X 2 |X 1 , X 3 |X 1 ) . We change the key variable and use a different C-Vine structure to describe the joint distribution of variables. The values of g 0 and D obtained by C-Vine copulas with different key variables are listed in Table 4. It can be seen that almost the same results can be obtained by different C-Vine copulas, which means C-Vine copulas based on different key variables have little effect on the mean and variance of the output response for this model. Table 4. Values of g 0 and D obtained by different C-Vine copulas (N = 2 16 ).

Key Variable
Vine Structure g 0 D

Test 2. Portfolio Model
Consider the Portfolio model [10]: The input variables are normally distributed with µ = (0, 0, 2.5, 4) and σ = (4, 2, 2, 3). There are several known samples (N = 500) in Figure 5, which shows all the variables are correlated. According to Equation (5), the four-dimensional input variables are decomposed as six two-dimensional variables: (X 1 , X 2 ), (X 1 , X 3 ), (X 1 , X 4 ), (X 2 |X 1 , X 3 |X 1 ) , (X 2 |X 1 , X 4 |X 1 ) and (X 3 |X 1 X 2 , X 4 |X 1 X 2 ) . The AIC values of each alternative bivariate copula function are listed in Table 5, which were calculated by using the above 500 samples. We mark the AIC values of the optimal copula in bold and display the corresponding parameters in Table 5. It can be seen that the optimal copula functions of two-dimensional variables are Clayton copula, Gaussian copula and Frank copula. Thus, there will be unknown errors if using the traditional Nataf method to measure the correlation. Table 6 gives the values obtained by the Nataf method and VC1. The results obtained by the traditional Nataf method and the proposed Vine copula method are different, as well as the importance orders of input variables. According to the values of S i , the Nataf method gives the importance order of input variables as X 2 > X 1 > X 3 > X 4 , while X 2 > X 1 > X 4 > X 3 is obtained if using the Vine copula method. Considering that an accurate importance order of variables is vital in practical problems, if we only use the conventional Nataf method for this problem, inaccurate results may be obtained, thus affecting the structural safety of a design. Figure 6 shows the distribution of the samples (X 1 , X 2 ) of the two methods. It can be seen that the samples generated by the Nataf method have symmetry but no tail correlation, while the Vine copula samples show a significantly lower tail correlation, which perfectly matches the distribution of variables X 1 and X 2 in Figure 5. This model illustrates that the tail correlation of variables has great impacts on Sobol' indices. Since the traditional Nataf method cannot capture the tail correlation between variables, the estimated results of the Nataf method will have unknown errors if there is obvious tail correlation between the variables, which is also the reason for the large gap between the results of the Nataf method and the Vine copula method for this model. Furthermore, it can be seen that S tot i < S i for this model, which means the correlations among input variables have greater effects than interactions on the variance of the output. Therefore, we should focus on the main indices S i in practical engineering when analyzing correlations, especially for non-normally dependent and nonlinear problems.

Test 3. Ishigami Function
Consider the Ishigami function [10]: (20) with input variables being uniformly distributed: −π ≤ X i ≤ π, i = 1, 2, 3. This function is widely used as a benchmark for sensitivity analysis since it has strong non-linearity and non-monotonicity.
We also analyzed this model in [11] and considered that only the variables X 1 and X 3 are dependent. In this paper, we assume that the three input variables are all correlated, and the distribution of 500 samples of input variables is shown in Figure 7. It can be seen that there is upper tail correlation between X 1 and X 2 , and lower tail correlation between X 1 and X 3 . By using the above samples, the optimal copula functions and their parameters among the two-dimensional variables are selected by the AIC method, and the results are listed in Table 7 (where the AIC values of the optimal copula functions are in bold). The optimal copula functions between two-dimensional variables are mainly Clayton copula and Gumbel copula, the parameters of which are listed in Table 7. Table 8 gives the values obtained by the Nataf method and the proposed method. It is shown that the importance orders of variables obtained by the two methods are the same (X 2 > X 1 > X 3 ), but the specific values of S i and S tot i are different. This is because the Vine copula method proposed in this paper takes into account the tail correlation between variables, which cannot be measured by the Nataf method. We also analyze the influence of degree of correlation of input variables on the importance analysis results. It is assumed that the Kendall τ between the decomposed two-dimensional variables is the same, and the optimal copula functions in the previous step are used. The proposed Vine copula method and Nataf method are used for analysis, and the independent situation of the three variables is considered at the same time. The curve of the calculated results changing with Kendall τ is shown in Figure 8. It can be seen that the results obtained by the two methods show the same trend with Kendall τ and are significantly different from the results in the independent case. This shows that the correlation between variables has a great impact on the Sobol' indices. If the variables are simply assumed as independent variables for processing, this may cause a large error in the importance analysis and affect the safety of designs in engineering. Besides, the results obtained by the two methods are relatively close in the case of small and large Kendall τ, while the results are significantly different when 0.5 < τ < 0.8. This indicates that the Vine copula method with better correlation measurement should be selected for analysis within the correlation range 0.5 < τ < 0.8.

Test 4. Fatigue and Creep of Materials
Fatigue and the creep of materials are two common failure modes at high temperatures. Based on experimental data using the linear damage accumulation rule, Mao et al. [33] proposed a probabilistic model for reliability analysis of the creep and fatigue of materials as follows: where D c and D f denote the creep damage and fatigue damage, respectively, and θ 1 and θ 2 are the parameters obtained from the experimental results. Two instances of damage can be expressed as D c = n c N c and D f = Assuming that the creep and fatigue lifetimes of a material follow log-normal distribution, θ 1 and θ 2 are normally distributed. The assumed values of the distribution parameters are shown in Table 9. Table 9. Values and distribution parameters of input variables. The input variables N c , N f , n c and n f are dependent according to the theory of material mechanics. There are several experimental data, the distributions of which are shown in Figure 9. The sample values of D c and D f can be calculated by using these data. Figure 10 shows the curve of creep-fatigue failure.   Figure 10 shows that the relationship between creep damage and fatigue damage is nonlinear, i.e., the ultimate damage to the material is a nonlinear function of fatigue damage and creep damage, which means creep damage and fatigue damage affect each other. The contribution of each input variable to the limit state equation is analyzed. The samples in Figure 9 are statistically inferred to determine the optimal copula function and correlation parameters between the two-dimensional variables by the AIC method. The results are listed in Table 10.  Figure 11 gives the results of variance-based global sensitivity indices with the Nataf method and Vine copula method. It can be seen that the results obtained by the two methods are relatively similar, and the same importance order of variables is obtained. According to the values of S i in Figure 11a, the fatigue life N f and the fatigue load cycle n f have greater influences on the limit state function than N c and n c , which is consistent with the AIC values in Table 10. This means the effects of input variables can be predicted preliminarily after the correlation estimation by the AIC method. For the parameters in the experiment, the influence of θ 1 is greater than θ 2 . We can conclude that the fatigue life and fatigue load cycle of materials most affect the creep-fatigue failure of the structure.

Input Variables Mean µ Coefficient of Variation (COV) Distribution
In addition, the importance measurement results obtained by the two methods are different since the Vine copula method uses the optimal copula (Gaussian copula and Frank copula, respectively) to measure the correlation between variables, while the traditional Nataf method only uses a single Gaussian copula to compute.

Test 5. Three-Bay Five-Story Linear Elastic Frame Structure
Consider a three-bay five-story linear elastic frame structure [34,35], as shown in Figure 12, which is characterized by 21 random variables: 3 applied loads, 2 Young's moduli, 8 moments of inertia and 8 cross-sectional areas. The properties of the frame structures and the distribution types and parameters of random variables are given in Tables 11 and 12, respectively. Statistical dependence among the basic variables is considered. All loads are correlated with linear correlation coefficient ρ = 0.5, while the cross-sectional area and the moment of inertia of each element type are correlated with ρ = 0.95. All cross-sectional properties are correlated as We consider the horizontal displacement at node 1 as output function: where u X denotes the actual horizontal displacement as an implicit function of all basic variables. According to engineering qualitative analysis, input variables P 1 , P 2 , P 3 , E 4 , A 18 , A 19 and A 20 contribute more than 95% to the output variance. Thus, we mainly analyze the importance orders of these seven input variables. In the following, the frame structure is modeled and analyzed by ANSYS with element type BEAM188. Several samples (N = 2 15 ) are generated using the Nataf model with the linear correlation coefficients. According to these samples, Gaussian copula is selected as the optimal bivariate copula of the Vine structure through the AIC method, and the corresponding parameters are computed as well (given in Appendix B). The values of Sobol' indices given by the Nataf method and Vine copula method are listed in Table 13.    It can be seen that two methods can get the same importance ranking of input variables. According to the main indices, the load P 1 has the largest contribution to the uncertainty of the output of the frame structure, and the importance of P 2 is larger than P 3 , which is consistent with the conclusion of the engineering qualitative analysis. In all interface properties, the cross-section area A 20 of element B 3 contributes the most to the uncertainty of the output. Besides, the total effects of input variables are generally smaller than the main effects, which indicates that the correlations among variables have greater effects on the output than the interacted effects. Therefore, the correlations among variables must be considered in importance analysis in engineering.
Comparing the results of the two methods, we can see that the values of the mean and variance of the output as well as Sobol' indices are almost the same. This is because the optimal bivariate copula functions of Vine structure are all Gaussian copula functions, which means the Vine copula method is essentially equivalent to the Nataf method under these circumstances.

Conclusions
Two novel Vine copula approaches (VC1 and VC2) for estimating variance-based global sensitivity indices for models with multidimensional correlated input variables are presented. VC1 generates dependent unconditional and conditional samples according to copula instead of the real joint CDF, while VC2 solves the indices respectively by decomposing the joint PDF into independent and dependent parts. The approaches provide a new method for variance-based GSA of multidimensional cases with different correlations. By introducing Vine copula, the complex multidimensional related cases are transformed into two-dimensional related problems, and the optimal copula functions are used to measure different correlations separately. In the general case of arbitrary distributions with unknown multidimensional correlations, the proposed methods have a wider application than the traditional correlation analysis methods, such as the multi-Gaussian copula method (Nataf model) or multi-Archimedean copula method. VC1 is mainly used in five different test functions for testing and comparison. The results show the feasibility and accuracy of the proposed methods compared with the Nataf model. Through the calculation of the AIC method, the number of decomposed items of the joint PDF can be reduced preliminarily, which can help to improve the computational efficiency of reliability and sensitivity analyses based on the joint PDF for the further work.  g(X (j) )c(u (j) ) g(X (j) Unlike Equations (14) and (15), the variable samples in Equations (A2) and (A3) are independent samples generated according to marginal PDF. We can see that Equations (A2) and (A3) consist in weighting the different terms of the sum in the MC estimator formula with the copula density function values at the corresponding data point. The basic idea of this method is analogous to the one on which importance sampling (IS) estimators [36] are based to reduce the variance of MC estimators (the weighting by the copula density function can be seen as a likelihood ratio). The detailed procedure is as follows: i. generate independent samples (X 1 , X 2 , · · · , X n ) and (X 1 , X 2 , · · · , X n ) according to the marginal PDFs; ii.
compute the values of output function and copula density function, and get S i and S tot i according to Equations (19) and (20). 0.0152 I 10 A 18 |I 6 I 7 I 8 I 9 0.9474 I 11 I 12−13 |I 6 I 7 I 8 I 9 I 10 , I 11 A 20−21 |I 6 I 7 I 8 I 9 I 10 0.0785 I 11 A 14−18 |I 6 I 7 I 8 I 9 I 10 0.0139 I 11 A 19 |I 6 I 7 I 8 I 9 I 10 0.9470 I 12 I 13 |I 6 I 7 I 8 I 9 I 10 I 11 , I 12 A 21 |I 6 I 7 I 8 I 9 I 10 I 11 0.0728 I 12 A 14−19 |I 6 I 7 I 8 I 9 I 10 I 11 0.0129 I 12 A 20 |I 6 I 7 I 8 I 9 I 10 I 11 0.9467 I 13 A 14−20 |I 6 I 7 I 8 I 9 I 10 I 11 I 12 0.0120 I 13 A 21 |I 6 I 7 I 8 I 9 I 10 I 11 I 12 0.9464 Other conditional variables 0.0020