# An Entropy-Based Approach to Path Analysis of Structural Generalized Linear Models: A Basic Idea

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Practical Example

## 3. Log Odds Ratio and Information

**Remark 1.**In general, the systematic component can be extended to be a function of explanatory variable vector

**x**. Then, the model is referred to as a generalized nonlinear model. For the sake of simplicity, the function is denoted by$\theta =\theta \left(\mathbf{x}\right)$. The discussion below is applicable to this case.

**Remark 2.**Applying ECD to the linear regression model, ECD is the usual coefficient of determination${R}^{2}$.

## 4. Measuring the Total, Direct, and Indirect Effects in Recursive GLM Systems

**Remark 3.**The total effect of${X}_{i}={x}_{i}$ on${X}_{K}={x}_{K}$ is given by:

**Remark 4.**The direct effect of${X}_{i}={x}_{i}$ on${X}_{K}={x}_{K}$ is given by

**Remark 5.**The total effects of variables by Kuha and Goldthorpe [10] are defined with the marginal distributions of response variables and explanatory variables. Meanwhile the present approach defines the total effects of explanatory variables based on a recursive structure of all the variables concerned and we have (6).

**Remark 6.**Indirect effects are defined by the total effects minus the direct effects as (3), (4) and (7); however the interpretation can be done in terms of entropy. On the other hand, direct and indirect effects are defined in an approach by [10], though the sum of the effects does not equal to the total effect.

**Remark 7.**Assessing the model identification and testing the goodness-of-fit of the model are based on the discussion of GLMs.

## 5. Statistical Test for Effects

## 6. Path Analysis of the British Morbility Data

**u**. Then, from Table 4 in [10], the estimated regression parameters for men are calculated as follows:

- the total effect of $X=\mathrm{S}$ and $Z=5$ on $Y=\mathrm{S}$ is calculated as follows: 0.51;
- the total effect of $Z=5$ is 0.04;
- the total effect of $X=\mathrm{S}$ is 0.47 when $Z=5$;
- the direct effect of $X=\mathrm{S}$ is 0.16 when $Z=5$;
- the indirect effect of $X=\mathrm{S}$ is 0.31 when $Z=5$.

## 7. Discussion

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Asher, H.B. Causal Modelling; Sage Publications: Beverly Hills, CA, USA, 1976. [Google Scholar]
- Bentler, P.M.; Weeks, D.B. Linear structural equations with latent variables. Psychometrika
**1980**, 45, 289–308. [Google Scholar] - Jöreskog, K.G.; Sörbom, D. LISREL8: User’s Reference Guide, 2nd ed; Scientific Software International: Chicago, IL, USA, 1996. [Google Scholar]
- Goodman, L.A. Causal analysis of data from panel studies and other kinds of surveys. Am. J. Sociol.
**1973**, 78, 1135–1191. [Google Scholar] - Goodman, L.A. The analysis of multidimensional contingency tables when some variables are posterior to others: A modified path analysis approach. Biometrika
**1973**, 60, 179–192. [Google Scholar] - Goodman, L.A. The analysis of systems of qualitative variables when some of the variables are unidentifiable: Part I. A modified latent structure approach. Am. J. Sociol.
**1974**, 79, 1179–1259. [Google Scholar] - Goodman, L.A. Exploratory latent structure analysis using both identifiable and unidentifiable models. Biometrika
**1974**, 61, 215–231. [Google Scholar] - Hagenaars, J.A. Categorical causal modeling: Latent class analysis and directed loglinear models with latent variables. Sociol. Methods Res.
**1998**, 26, 436–489. [Google Scholar] - Eshima, N.; Tabata, M.; Geng, Z. Path analysis with logistic regression models: Effect analysis of fully recursive causal systems of categorical variables. J. Jpn. Stat. Soc.
**2001**, 31, 1–14. [Google Scholar] - Kuha, J.; Goldthorpe, J.H. Path analysis for discrete variables: The role of education in social mobility. J. R. Stat. Soc. A
**2010**, 173, 351–369. [Google Scholar] - Albert, J.M.; Nelson, S. Generalized causal mediation analysis. Biometrics
**2011**, 67, 1028–1038. [Google Scholar] - Nelder, J.A.; Wedderburn, R.W.M. Generalized linear model. J. R. Stat. Soc. A
**1972**, 135, 370–384. [Google Scholar] - McCullagh, P.; Nelder, J.A. Generalized Linear Models, 2nd ed; Chapman and Hall: London, UK, 1989. [Google Scholar]
- Eshima, N.; Tabata, M. Entropy correlation coefficient for measuring predictive power of generalized linear models. Stat. Probab. Lett.
**2007**, 77, 588–593. [Google Scholar] - Eshima, N.; Tabata, M. Entropy coefficient of determination for generalized linear models. Comput. Stat. Data Anal.
**2010**, 54, 1381–1389. [Google Scholar] - Eshima, N.; Tabata, M. Three predictive power measures for generalized linear models: Entropy coefficient of determination, entropy correlation coefficient and regression correlation coefficient. Comput. Stat. Data Anal.
**2011**, 55, 3049–3058. [Google Scholar] - Patnaik, P.B. The non-central χ
^{2}and F-distributions and their applications. Biometrika**1949**, 36, 202–232. [Google Scholar]

Sex | Parental Class X | Education Level Z | ||||||
---|---|---|---|---|---|---|---|---|

1 | 2 | 3 | 4 | 5 | 6 | 7 | ||

Men | S | 0.038 | 0.015 | 0.009 | 0.053 | 0.050 | 0.047 | 0.082 |

I | 0.069 | 0.017 | 0.015 | 0.051 | 0.032 | 0.037 | 0.025 | |

W | 0.189 | 0.018 | 0.037 | 0.088 | 0.055 | 0.051 | 0.028 | |

Women | S | 0.040 | 0.014 | 0.020 | 0.079 | 0.034 | 0.048 | 0.048 |

I | 0.072 | 0.013 | 0.023 | 0.075 | 0.021 | 0.034 | 0.021 | |

W | 0.216 | 0.018 | 0.046 | 0.110 | 0.023 | 0.032 | 0.009 |

X | Z | Effect on Y = S
| ||||
---|---|---|---|---|---|---|

(X,Z) (total) | Z (total) | X (total) | X (direct) | X (indirect) | ||

S | 1 | −0.79 | −0.89 | 0.10 | 0.22 | −0.12 |

2 | −0.49 | −0.65 | 0.19 | 0.23 | −0.04 | |

3 | −0.14 | −0.42 | 0.29 | 0.24 | 0.04 | |

4 | 0.19 | −0.19 | 0.38 | 0.17 | 0.21 | |

5 | 0.51 | 0.04 | 0.47 | 0.16 | 0.31 | |

6 | 0.84 | 0.28 | 0.56 | 0.16 | 0.40 | |

7 | 1.16 | 0.51 | 0.65 | 0.11 | 0.55 | |

I | 1 | −0.93 | −0.89 | −0.04 | −0.04 | −0.01 |

2 | −0.93 | −0.57 | −0.04 | −0.05 | 0.01 | |

3 | −0.28 | −0.25 | −0.03 | −0.05 | 0.02 | |

4 | 0.04 | 0.07 | −0.03 | −0.04 | 0.02 | |

5 | 0.37 | 0.39 | −0.02 | −0.05 | 0.03 | |

6 | 0.69 | 0.71 | −0.01 | −0.05 | 0.03 | |

7 | 1.02 | 1.02 | −0.01 | −0.05 | 0.04 | |

W | 1 | −0.99 | −0.84 | −0.15 | 0.17 | −0.32 |

2 | −0.67 | −0.45 | −0.21 | 0.01 | −0.23 | |

3 | −0.34 | −0.07 | −0.23 | 0.03 | −0.30 | |

4 | −0.02 | 0.32 | −0.34 | 0.06 | −0.39 | |

5 | 0.31 | 0.71 | −0.40 | 0.03 | −0.43 | |

6 | 0.63 | 1.09 | −0.46 | 0.03 | −0.49 | |

7 | 0.96 | 1.48 | −0.52 | 0.01 | −0.53 |

X | Z | Effect on Y = I
| ||||
---|---|---|---|---|---|---|

(X,Z) (Total) | Z (Total) | X (Total) | X (Direct) | X (Indirect) | ||

S | 1 | 0.07 | 0.60 | −0.52 | −0.07 | −0.45 |

2 | 0.01 | 0.44 | −0.43 | −0.09 | −0.34 | |

3 | −0.06 | 0.28 | −0.34 | −0.09 | −0.25 | |

4 | −0.12 | 0.13 | −0.25 | −0.10 | −0.15 | |

5 | −0.19 | −0.03 | −0.16 | −0.11 | −0.04 | |

6 | −0.25 | −0.19 | −0.06 | −0.12 | 0.06 | |

7 | −0.32 | −0.34 | 0.03 | −0.12 | 0.15 | |

I | 1 | 0.15 | 0.24 | −0.09 | 0.06 | −0.16 |

2 | 0.08 | 0.17 | −0.09 | 0.10 | −0.18 | |

3 | 0.02 | 0.10 | −0.08 | 0.10 | −0.18 | |

4 | −0.05 | 0.03 | −0.07 | 0.07 | −0.15 | |

5 | −0.11 | −0.05 | −0.07 | 0.09 | −0.15 | |

6 | −0.18 | −0.12 | −0.06 | 0.08 | −0.14 | |

7 | −0.24 | −0.19 | −0.05 | 0.09 | −0.15 | |

W | 1 | 0.32 | 0.01 | 0.31 | 0.07 | 0.24 |

2 | 0.25 | 0.00 | 0.25 | 0.00 | 0.25 | |

3 | 0.19 | 0.00 | 0.19 | 0.01 | 0.18 | |

4 | 0.12 | 0.00 | 0.13 | 0.01 | 0.12 | |

5 | 0.06 | −0.01 | 0.07 | 0.00 | 0.06 | |

6 | −0.01 | −0.01 | 0.00 | 0.00 | 0.00 | |

7 | −0.07 | −0.01 | −0.06 | 0.00 | −0.05 |

X | Z | Effect on Y = W | ||||
---|---|---|---|---|---|---|

(X,Z) (Total) | Z (Total) | X (Total) | X (Direct) | X (Indirect) | ||

S | 1 | 0.67 | 1.28 | −0.62 | −0.05 | −0.57 |

2 | 0.42 | 0.95 | −0.52 | −0.09 | −0.44 | |

3 | 0.18 | 0.61 | −0.43 | −0.09 | −0.34 | |

4 | −0.07 | 0.27 | −0.34 | −0.07 | −0.27 | |

5 | −0.31 | −0.06 | −0.25 | −0.09 | −0.16 | |

6 | −0.56 | −0.40 | −0.16 | −0.10 | −0.06 | |

7 | −0.80 | −0.74 | −0.06 | −0.08 | 0.01 | |

I | 1 | 0.74 | 0.65 | 0.09 | −0.02 | 0.11 |

2 | 0.49 | 0.39 | 0.10 | −0.04 | 0.14 | |

3 | 0.25 | 0.14 | 0.11 | −0.05 | 0.15 | |

4 | 0.00 | −0.11 | 0.11 | −0.03 | 0.14 | |

5 | −0.24 | −0.36 | 0.12 | −0.04 | 0.16 | |

6 | −0.49 | −0.61 | 0.12 | −0.04 | 0.16 | |

7 | −0.73 | −0.86 | 0.13 | −0.04 | 0.17 | |

W | 1 | 0.64 | 0.40 | 0.24 | −0.10 | 0.34 |

2 | 0.39 | 0.21 | 0.18 | −0.01 | 0.19 | |

3 | 0.15 | 0.03 | 0.12 | −0.03 | 0.14 | |

4 | −0.10 | −0.15 | 0.06 | −0.07 | 0.12 | |

5 | −0.34 | −0.34 | −0.01 | −0.05 | 0.04 | |

6 | −0.59 | −0.52 | −0.07 | −0.05 | −0.02 | |

7 | −0.83 | −0.70 | −0.13 | −0.03 | −0.10 |

Sex | Explanatory Variables | Direct Effect | Indirect Effect | Total Effect |
---|---|---|---|---|

Men | Parental Class X | 0.033 (0.004)* | 0.076 (0.07) | 0.109 (0.008) |

Education Z | 0.168 (0.010) | — | 0.168 (0.010) | |

(X,Z) | — | — | 0.276 (0.013) | |

Women | Parental Class X | 0.011 (0.002) | 0.068 (0.06) | 0.079 (0.007) |

Education Z | 0.210 (0.011) | — | 0.210 (0.011) | |

(X,Z) | — | — | 0.289 (0.012) |

^{*}The numbers in parentheses are the standard errors.

© 2015 by the authors; licensee MDPI, Basel, Switzerland This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Eshima, N.; Tabata, M.; Borroni, C.G.; Kano, Y. An Entropy-Based Approach to Path Analysis of Structural Generalized Linear Models: A Basic Idea. *Entropy* **2015**, *17*, 5117-5132.
https://doi.org/10.3390/e17075117

**AMA Style**

Eshima N, Tabata M, Borroni CG, Kano Y. An Entropy-Based Approach to Path Analysis of Structural Generalized Linear Models: A Basic Idea. *Entropy*. 2015; 17(7):5117-5132.
https://doi.org/10.3390/e17075117

**Chicago/Turabian Style**

Eshima, Nobuoki, Minoru Tabata, Claudio Giovanni Borroni, and Yutaka Kano. 2015. "An Entropy-Based Approach to Path Analysis of Structural Generalized Linear Models: A Basic Idea" *Entropy* 17, no. 7: 5117-5132.
https://doi.org/10.3390/e17075117