# Polynomial Representations of High-Dimensional Observations of Random Processes

## Abstract

**:**

## 1. Introduction

## 2. Background

#### 2.1. Random Processes

**Definition**

**1.**

**Lemma**

**1.**

**Proof.**

**Definition**

**2.**

**Lemma**

**2.**

**Proof.**

#### 2.2. Estimation Methods

#### 2.3. Generating Random Processes

**Lemma**

**3.**

**Proof.**

**Lemma**

**4.**

**Proof.**

#### 2.4. Polynomials and Multivariate Functions

**Lemma**

**5.**

**Proof.**

**Lemma**

**6.**

**Proof.**

**Lemma**

**7.**

**Proof.**

**Theorem**

**1.**

**Definition**

**3.**

## 3. Background Extensions

#### 3.1. Linear LS Estimation

#### 3.2. Generating Pairwise-Correlated Gaussian Processes

**Lemma**

**8.**

**Proof.**

**Corollary**

**1.**

**Corollary**

**2.**

**Lemma**

**9.**

**Proof.**

**Corollary**

**3.**

## 4. Polynomial Statistics and Sum-Moments for Vectors of Random Variables

**Lemma**

**10.**

**Proof.**

**Claim**

**1.**

#### 4.1. Related Concepts

**Definition**

**4.**

**Lemma**

**11.**

**Proof.**

#### 4.2. Multiple Random Processes

**Definition**

**5.**

**Lemma**

**12.**

**Proof.**

## 5. Illustrative Examples

#### 5.1. Linear Regression

#### 5.2. Comparison of Central Moments

#### 5.3. Signal Processing Problems for the 1st Order Markov Process

**Conjecture**

**1.**

## 6. Conclusions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## Abbreviations

1MP | 1st order Markov process |

2MP | 2nd order Markov process |

AR | autoregressive |

LMMSE | linear minimum mean square error |

LS | least squares |

MMSE | minimum mean square error |

MSE | mean square error |

MA | moving average |

TV | total variance |

$|\xb7|$ | absolute value, set cardinality, sum of vector elements |

$E\left[\xb7\right]$ | expectation |

$corr\left[\xb7\right]$ | correlation |

$cov\left[\xb7\right]$ | covariance |

$var\left[\xb7\right]$ | variance |

${(\xb7)}^{-1}$ | matrix inverse |

${(\xb7)}^{T}$ | matrix/vector transpose |

${f}_{x}$ | distribution of a random variable X |

f, $\dot{f}$, $\ddot{f}$ | function f, and its first and second derivatives |

${\mathbb{N}}_{+}$ | positive non-zero integers |

$\mathcal{R}$, ${\mathcal{R}}^{\phantom{\rule{-0.166667em}{0ex}}+}$ | real numbers, positive real numbers |

$\overline{X}$ | mean value of random variable X |

${X}_{ij}$ | j-th sample of process i |

${W}_{-1}$ | Lambert function |

## References

- Drezner, Z. Multirelation—A correlation among more than two variables. Comput. Stat. Data Anal.
**1995**, 19, 283–292. [Google Scholar] [CrossRef] - Dear, R.; Drezner, Z. On the significance level of the multirelation coefficient. J. Appl. Math. Decis. Sci.
**1997**, 1, 119–130. [Google Scholar] [CrossRef] [Green Version] - Geiß, S.; Einax, J. Multivariate correlation analysis—A method for the analysis of multidimensional time series in environmental studies. Chemom. Intell. Lab. Syst.
**1996**, 32, 57–65. [Google Scholar] [CrossRef] - Abdi, H. Chapter Multiple correlation coefficient. In Encyclopedia of Measurements and Statistics; SAGE: Los Angeles, CA, USA, 2007; pp. 648–655. [Google Scholar] [CrossRef]
- Székely, G.J.; Rizzo, M.L.; Bakirov, N.K. Measuring and testing dependence by correlation of distances. Ann. Stat.
**2007**, 35, 2769–2794. [Google Scholar] [CrossRef] - Merigó, J.M.; Casanovas, M. A New Minkowski Distance Based on Induced Aggregation Operators. Int. J. Comput. Intell. Syst.
**2011**, 4, 123–133. [Google Scholar] [CrossRef] - Nguyen, H.V.; Müller, E.; Vreeken, J.; Efros, P.; Böhm, K. Multivariate maximal correlation analysis. In Proceedings of the International Conference on Machine Learning, Beijing, China, 21–26 June 2014; pp. II-775–II-783. [Google Scholar] [CrossRef]
- Josse, J. Measuring multivariate association and beyond. Stat. Surv.
**2016**, 10, 132–167. [Google Scholar] [CrossRef] [PubMed] - Shu, X.; Zhang, Q.; Shi, J.; Qi, Y. A comparative study on weighted central moment and its application in 2D shape retrieval. Information
**2016**, 7, 10. [Google Scholar] [CrossRef] [Green Version] - Böttcher, B.; Keller-Ressel, M.; Schilling, R.L. Distance multivariance: New dependence measures for random vectors. Ann. Stat.
**2019**, 47, 2757–2789. [Google Scholar] [CrossRef] - Wang, J.; Zheng, N. Measures of correlation for multiple variables. arXiv
**2020**, arXiv:1401.4827. [Google Scholar] - Gardner, W.A. Introduction to Random Processes With Applications, 2nd ed.; McGraw-Hill: New York, NY, USA, 1990. [Google Scholar]
- Papoulis, A.; Pillai, S.U. Probability, Random Variables, and Stochastic Processes, 4th ed.; McGraw-Hill: New York, NY, USA, 2002. [Google Scholar]
- Giller, G.L. The Statistical Properties of Random Bitstreams and the Sampling Distribution of Cosine Similarity. SSRN Preprint
**2012**. [Google Scholar] [CrossRef] - Kay, S.M. Fundamentals of Statistical Signal Processing: Estimation Theory; Prentice Hall: Upper Saddle River, NJ, USA, 1993; Volume I. [Google Scholar]
- Oppenheim, A.V.; Schafer, R.W.; Buck, J.R. Discrete-Time Signal Processing, 3rd ed.; Prentice Hall: Upper Saddle River, NJ, USA, 2009. [Google Scholar]
- Boyd, S.; Vandenberghe, L. Convex Optimization; Cambridge University Press: Cambridge, UK, 2004. [Google Scholar]
- Folland, G.B. Higher-Order Derivatives and Taylor’s Formula in Several Variables. Available online: https://sites.math.washington.edu/~folland/Math425/taylor2.pdf (accessed on 1 December 2020).
- Apostol, T. Mathematical Analysis, 2nd ed.; Pearson: London, UK, 1973. [Google Scholar]
- Yedidia, J.S. chapter An idiosyncratic journey beyond mean field theory. In Advanced Mean Field Methods; MIT Press: Cambridge, MA, USA, 2001; pp. 21–36. [Google Scholar]
- Sobol, I.M. Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Math. Comput. Simul.
**2001**, 55, 271–280. [Google Scholar] [CrossRef] - Saltelli, A.; Ratto, M.; Tarantola, S.; Campolongo, F. Sensitivity analysis for chemical models. Chem. Rev.
**2005**, 105, 2811–2828. [Google Scholar] [CrossRef] [PubMed] - Shirali, S. A Concise Introduction to Measure Theory; Springer: Berlin/Heidelberg, Germany, 2018. [Google Scholar]
- Abramowitz, M.; Stegun, I.A. Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables; Dover: Mineola, NY, USA, 1974. [Google Scholar]

**Figure 1.**The exact ($L{S}_{\mathrm{opt}}$) and the reduced complexity ($L{S}_{\mathrm{apr}}$) linear LS regression.

**Figure 3.**The second central sum-moments of the 2nd order Markov process with parameter $\alpha $ and length N.

**Figure 4.**The Minkowski (blue), sum-moment (black), and sum-moment of absolute values (red) mean statistics for the 1st order Markov sequence of length N. Columns: different values $\alpha $. Rows: different values m.

**Figure 5.**The second central sum-moment as a function of time differences between $N=3$ observations of a stationary random process.

**Figure 6.**The MSE of approximating the (auto-) covariance of the 1st order Markov process at the output of length N MA filter by the (auto-) covariance of the 2nd order Markov process.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Loskot, P.
Polynomial Representations of High-Dimensional Observations of Random Processes. *Mathematics* **2021**, *9*, 123.
https://doi.org/10.3390/math9020123

**AMA Style**

Loskot P.
Polynomial Representations of High-Dimensional Observations of Random Processes. *Mathematics*. 2021; 9(2):123.
https://doi.org/10.3390/math9020123

**Chicago/Turabian Style**

Loskot, Pavel.
2021. "Polynomial Representations of High-Dimensional Observations of Random Processes" *Mathematics* 9, no. 2: 123.
https://doi.org/10.3390/math9020123