# Ergodic Subspace Analysis

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Definition of Ergodicity

#### Conditional Equivalence

## 3. Ergodic Subspace Analysis

## 4. Performance in Artificial Situations

#### 4.1. Computation Examples

#### 4.1.1. Manual Computational Example

#### 4.1.2. Larger-Scale Computational Example

#### 4.2. Simulations

#### 4.2.1. Simulation 1

#### 4.2.2. Simulation 2

#### 4.2.3. Simulation 3

## 5. Application to COGITO Data

## 6. Discussion

## Author Contributions

## Funding

## Conflicts of Interest

## Appendix A. ESA in R Code

`ergodicSubspaceAnalysis <-`

`function(betweenCovariance, withinCovariance, cutoff = 0.1) {`

`# BEGIN ESA`

`# Step 1: Combining the matrices`

`combinedCovariance <- 0.5 * (betweenCovariance + withinCovariance)`

`# Step 2`

`eigendecompositionResult1 <- eigen(combinedCovariance);`

`QWhite <- t(eigendecompositionResult1$vectors);`

`for (i in 1:nrow(QWhite))`

`QWhite[i,] <- QWhite[i,] / sqrt(eigendecompositionResult1$values[i])`

`# Step 3`

`transformedBetweenCovariance <- QWhite %*% betweenCovariance %*% t(QWhite)`

`eigendecompositionResult2 <- eigen(0.5*transformedBetweenCovariance)`

`Q2 <- t(eigendecompositionResult2$vectors)`

`ergodicity <-`

`eigendecompositionResult2$values - rev(eigendecompositionResult2$values)`

`QESA <- Q2 %*% QWhite`

`# Step 4`

`cutoff <- 0.1`

`VBetween <- QESA[ergodicity > cutoff,]`

`VErgodic <- QESA[(ergodicity < cutoff) & (ergodicity > -cutoff),]`

`VWithin <- QESA[ergodicity < -cutoff,]`

`# Prepare return element`

`result <- list("QESA" = QESA, "ergodicity" = ergodicity,`

`"VBetween" = VBetween, "VErgodic" = VErgodic,`

`"VWithin" = VWithin)`

`return(result)`

`}`

`# Manual example from the article`

`betweenCovariance <- matrix(c(1,0.5,0.5,1), nrow = 2, byrow = T)`

`withinCovariance <- matrix(c(1,-0.5,-0.5,1), nrow = 2, byrow = T)`

`ergodicSubspaceAnalysis(betweenCovariance, withinCovariance)`

`# Sports example from the article, with~a fixed x and using`

`# expected covariance`

`factors <- matrix(c(1,1,1,1,`

`3,-1,-1,-1,`

`0,1,-1,0,`

`0,1,1,-2), nrow = 4, byrow = T)`

`for (i in 1:4) factors[i,] <- factors[i,] / norm(as.matrix(factors[i,]),"2")`

`x <- 0 betweenLoadings <- diag(c(10,x,5,1))`

`betweenCovariance <- t(factors) %*% betweenLoadings %*% factors`

`withinLoadings <- diag(c(x,10,5,1))`

`withinCovariance <- t(factors) %*% withinLoadings %*% factors`

`ergodicSubspaceAnalysis(betweenCovariance, withinCovariance)`

## References

- Adolf, Janne K., and Eiko I. Fried. 2019. Erodicity is sufficient but not necessary for group-to-individual generalizability. Proceedings of the National Academy of Sciences 116: 6540–41. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Asparouhov, Tihomir, and Bengt Muthen. 2007. Computationally efficient estimation of multilevel high-dimensional latent variable models. In JSM Meeting. Salt Lake City: Section on Statistics in Epidemiology. [Google Scholar]
- Borkenau, Peter, and Fritz Ostendorf. 1998. The big five as states: How useful is the five-factor model to describe intraindividual variations over time? Journal of Research in Personality 32: 202–21. [Google Scholar] [CrossRef] [Green Version]
- Borsboom, Denny, and Conor V. Dolan. 2006. Why g is not an adaptation: A comment on Kanazawa (2004). Psycholocical Review 113: 433–37. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Borsboom, Denny, Gideon J. Mellenbergh, and Jaap Van Heerden. 2003. The theoretical status of latent variables. Psycholocical Review 110: 203–19. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Brose, Annette, Manuel C. Voelkle, Martin Lövdén, Ulman Lindenberger, and Florian Schmiedek. 2015. Differences in the between-person and within-person structures of affect are a matter of degree. European Journal of Personality 29: 55–71. [Google Scholar] [CrossRef] [Green Version]
- Carroll, John B. 1993. Human Cognitive Abilities: A Survey of Factor-Analytical Studies. New York: Cambridge University Press. [Google Scholar] [CrossRef]
- Fisher, Aaron J., John D. Medaglia, and Bertus F. Jeronimus. 2018. Lack of group-to-individual generalizability is a threat to human subjects research. Proceedings of the National Academy of Sciences 115: E6106–E6115. [Google Scholar] [CrossRef] [Green Version]
- Goldstein, Harvey. 2003. Multilevel Statistical Models, 3rd ed. London: Edward Arnold. [Google Scholar]
- Goldstein, Harvey, and William Browne. 2005. Multilevel factor analysis models for continuous and discrete data. In Contemporary Psychometrics. A Festschrift to Roderick P. Mcdonald. Edited by Alberto Maydeu-Olivares and Jack McArdle. Mahwah: Lawrence Erlbaum, pp. 453–75. [Google Scholar]
- Hamaker, Ellen L. 2012. Why researchers should think “within-person”: A paradigmatic rationale. In Handbook of Research Methods for Studying Daily Life. Edited by Matthias Mehl and Tamlin Conner. New York: The Guilford Press, pp. 43–61. [Google Scholar]
- Hamaker, Ellen L., Conor V. Dolan, and Peter C. M. Molenaar. 2005. Statistical modeling of the individual: Rationale and application of multivariate stationary time series analysis. Multivariate Behavioral Research 40: 207–33. [Google Scholar] [CrossRef]
- Hox, Joop J. 2010. Multilevel Analysis: Techniques and Applications, 2nd ed. New York: Lawrence Erlbaum. [Google Scholar]
- Hox, Joop J., Cora J. M. Maas, and Matthieu J. S. Brinkhuis. 2010. The effect of estimation method and sample size in multilevel structural equation modeling. Statistica Nederlandica 64: 453–75. [Google Scholar] [CrossRef]
- Koles, Zoltan J., Michael S. Lazar, and Steven Z. Zhou. 1990. Spatial patterns underlying population differences in the background EEG. Brain Topography 2: 275–84. [Google Scholar] [CrossRef]
- Lindenberger, Ulman, Timo Von Oertzen, Paolo Ghisletta, and Christopher Hertzog. 2011. Cross-sectional age variance extraction: What’s change got to do with it? Psychology and Aging 26: 34–47. [Google Scholar] [CrossRef] [Green Version]
- Mehta, Paras D., and Michael C. Neale. 2005. People are variables too: Multilevel structural equation modeling. Psychological Methods 10: 259–84. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Molenaar, Peter C. M. 2004. A manifesto on psychology as idiographic science: Bringing the person back into scientific psychology, this time forever. Measurement 2: 201–18. [Google Scholar] [CrossRef]
- Muthen, Bengt O. 1989. Latent variable modeling in heterogeneous populations. Psychometrika 54: 557–85. [Google Scholar] [CrossRef]
- Muthen, Bengt O. 1994. Multilevel covariance structure analysis. Sociological Methods and Research 22: 376–98. [Google Scholar] [CrossRef]
- Muthen, Bengt O. 2002. General latent variable modeling. Behaviormetrika 29: 81–117. [Google Scholar] [CrossRef]
- Neale, Michael C., Michael D. Hunter, Joshua N. Pritikin, Mahsa Zahery, Timothy R. Brick, Robert M. Kirkpatrick, Ryne Estabrook, Timothy C. Bates, Hermine H. Maes, and Steven M. Boker. 2016. Openmx 2.0: Extended structural equation and statistical modeling. Psychometrika 81: 535–49. [Google Scholar] [CrossRef]
- von Oertzen, Timo, Andreas M. Brandmaier, and Siny Tsang. 2015. Structural equation modeling with onyx. Structural Equation Modeling: An Interdisciplinary Journal 22: 148–61. [Google Scholar] [CrossRef]
- Papoulis, Anastassios. 1984. Proability, Random Variables, and Stochastic Processes. New York: McGraw-Hill. [Google Scholar]
- Pearson, Karl. 1901. On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 2: 559–72. [Google Scholar] [CrossRef] [Green Version]
- Petersen, Karl E. 1983. Ergodicity Theory, 5th ed. New York: Cambridge University Press. [Google Scholar]
- Schmiedek, Florian, Martin Lövdén, and Ulman Lindenberger. 2010. Hundred days of cognitive training enhance broad cognitive abilities in adulthood: Findings from the cogito study. Frontiers in Aging Neuroscience 2: 27. [Google Scholar] [CrossRef] [Green Version]
- Schmiedek, Florian, Martin Lövdén, and Ulman Lindenberger. 2013. Keeping it steady: Older adults perform more consistently on cognitive tasks than younger adults. Psychological Science 2: 1747–54. [Google Scholar] [CrossRef] [Green Version]
- Schmiedek, Florian, Martin Lövdén, Timo von Oertzen, and Ulman Lindenberger. 2018. Within-person structures of daily cognitive performance cannot be inferred from between-person structures of cognitive abilities. PeerJ Preprints 7: E27576v1. [Google Scholar] [CrossRef]
- Voelkle, Manuel C., Annette Brose, Florian Schmiedek, and Ulman Lindenberger. 2014. Toward a unified framework for the study of between-person and within-person structures: Building a bridge between two research paradigms. Multivariate Behavioral Research 49: 193–213. [Google Scholar] [CrossRef] [PubMed]
- Voelkle, Manuel C., Christian Gische, Charles C. Driver, and Ulman Lindenberger. 2018. The role of time in the quest for understanding psychological mechanisms. Behavioral Research Methods 53: 782–805. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Voelkle, Manuel C., Johan HL Oud, Timo von Oertzen, and Ulman Lindenberger. 2012. Maximum likelihood dynamic factor modeling for arbitrary N and T using SEM. Structural Equation Modeling 19: 329–50. [Google Scholar] [CrossRef] [Green Version]
- Wiskott, Laurenz, and Terrence J. Sejnowski. 2002. Slow feature analysis: Unsupervised learning of invariances. Neural Computation 14: 715–70. [Google Scholar] [CrossRef] [PubMed]

1 | Strictly speaking, ergodicity would even require the variances to be equal and then, consequently, the covariances instead of the correlations; however, it is sometimes also useful to concentrate on the correlations and allow differences in variances. |

2 | Equivalence in this article is used very similarly to the term ergodicity; we will use this term from here on to avoid confusion. |

**Figure 1.**Expectation of the first estimated ergodicity value for different x values in the simulation. The ergodicity values follow ${(1-x)}^{2}$.

**Figure 2.**The expectation of the first estimated ergodicity value for different sample sizes N and time point T.

**Figure 3.**Scree plot for three situations of ergodicity: fully ergodic (independently sampled covariance matrices), fully non-ergodic (identical covariance matrices), and an intermediate case with a 50% mixture.

**Figure 4.**Scree plot for three artificial situations of ergodicity, fully ergodic (independently sampled covariance matrices), fully non-ergodic (identical covariance matrices), and an intermediate case with a 50% mixture, together with the ergodicity values of the nine cognitive tasks from the COGITO study (raw and de-trended).

**Table 1.**Reconstruction precision of the factor with dominant variance between participants and its corresponding ergodicity value. The value x gives the degree to which the data is ergodic, with $x=0$ indicating strong differences between and within participants and $x=10$ indicating a perfectly ergodic situation. The angle $\alpha $ is the angle between the true first factor and the reconstructed factor; a cosine of one means perfect reconstruction, a cosine of 0 means orthogonal vectors.

x | $cos\left(\mathit{\alpha}\right)$ | ${\mathit{erg}}_{1}$ | stdv(${\mathit{erg}}_{1}$) |
---|---|---|---|

0 | 1 | 1 | 0.144 |

1 | 0.999 | 0.823 | 0.129 |

2 | 0.996 | 0.670 | 0.119 |

3 | 0.988 | 0.542 | 0.111 |

4 | 0.972 | 0.432 | 0.103 |

5 | 0.938 | 0.337 | 0.095 |

6 | 0.875 | 0.253 | 0.090 |

7 | 0.764 | 0.182 | 0.084 |

8 | 0.594 | 0.115 | 0.080 |

9 | 0.421 | 0.056 | 0.075 |

10 | 0.277 | 0.005 | 0.072 |

**Table 2.**Standard error of the ergodicity value of the factor with dominant variance between participants for different sample sizes and time series lengths. High values indicate uncertain measurements, while lower values indicate a more precise measurement of the degree of ergodicity in the data.

N | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|

$\mathit{T}$ | 25 | 50 | 75 | 100 | 125 | 150 | 175 | 200 | 225 | 250 |

25 | 0.197 | 0.139 | 0.114 | 0.098 | 0.087 | 0.078 | 0.075 | 0.068 | 0.065 | 0.061 |

50 | 0.197 | 0.139 | 0.113 | 0.094 | 0.085 | 0.079 | 0.071 | 0.069 | 0.066 | 0.060 |

75 | 0.194 | 0.136 | 0.111 | 0.097 | 0.085 | 0.079 | 0.071 | 0.066 | 0.064 | 0.061 |

100 | 0.194 | 0.137 | 0.109 | 0.097 | 0.085 | 0.077 | 0.072 | 0.067 | 0.063 | 0.060 |

125 | 0.194 | 0.133 | 0.111 | 0.094 | 0.086 | 0.078 | 0.072 | 0.068 | 0.064 | 0.060 |

150 | 0.193 | 0.131 | 0.109 | 0.097 | 0.084 | 0.078 | 0.073 | 0.067 | 0.062 | 0.060 |

175 | 0.194 | 0.135 | 0.110 | 0.095 | 0.085 | 0.078 | 0.070 | 0.067 | 0.063 | 0.060 |

200 | 0.195 | 0.135 | 0.111 | 0.096 | 0.084 | 0.078 | 0.070 | 0.066 | 0.063 | 0.060 |

225 | 0.193 | 0.133 | 0.111 | 0.095 | 0.086 | 0.076 | 0.072 | 0.068 | 0.064 | 0.060 |

250 | 0.194 | 0.136 | 0.109 | 0.094 | 0.085 | 0.078 | 0.073 | 0.066 | 0.063 | 0.060 |

**Table 3.**The ergodicity values and the corresponding nine factors of the ergodic subspace analysis (ESA) on the de-trended cognitive data from the COGITO study.

$\mathit{erg}$ | Processing Speed | Episodic Memory | Working Memory | ||||||
---|---|---|---|---|---|---|---|---|---|

Numerical | Verbal | Figural | Numerical | Verbal | Figural | Numerical | Verbal | Figural | |

0.314 | 0.200 | 0.155 | 0.130 | 0.144 | 0.191 | 0.141 | 0.113 | 0.166 | 0.214 |

0.181 | 0.125 | 0.201 | 0.165 | 0.040 | 0.009 | 0.194 | −0.370 | −0.334 | −0.037 |

0.004 | 0.246 | 0.076 | 0.206 | −0.331 | −0.237 | −0.379 | 0.140 | 0.092 | 0.066 |

−0.038 | −0.041 | −0.201 | 0.120 | −0.343 | −0.234 | 0.405 | −0.065 | −0.076 | 0.398 |

−0.081 | 0.032 | 0.170 | −0.229 | 0.393 | −0.551 | −0.019 | −0.004 | −0.009 | 0.163 |

−0.163 | 0.177 | 0.400 | −0.619 | −0.334 | 0.095 | 0.096 | −0.118 | 0.120 | −0.004 |

−0.264 | 0.119 | 0.016 | 0.062 | −0.088 | −0.245 | 0.421 | 0.282 | 0.116 | −0.58 |

−0.460 | 0.115 | 0.043 | −0.169 | −0.017 | 0.083 | −0.035 | 0.591 | −0.671 | 0.078 |

−0.542 | 0.785 | −0.652 | −0.251 | 0.108 | 0.039 | 0.009 | −0.119 | −0.020 | −0.013 |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

von Oertzen, T.; Schmiedek, F.; Voelkle, M.C.
Ergodic Subspace Analysis. *J. Intell.* **2020**, *8*, 3.
https://doi.org/10.3390/jintelligence8010003

**AMA Style**

von Oertzen T, Schmiedek F, Voelkle MC.
Ergodic Subspace Analysis. *Journal of Intelligence*. 2020; 8(1):3.
https://doi.org/10.3390/jintelligence8010003

**Chicago/Turabian Style**

von Oertzen, Timo, Florian Schmiedek, and Manuel C. Voelkle.
2020. "Ergodic Subspace Analysis" *Journal of Intelligence* 8, no. 1: 3.
https://doi.org/10.3390/jintelligence8010003