Measuring Dependencies between Variables of a Dynamical System Using Fuzzy Affiliations

Abstract

A statistical, data-driven method is presented that quantifies influences between variables of a dynamical system. The method is based on finding a suitable representation of points by fuzzy affiliations with respect to landmark points using the Scalable Probabilistic Approximation algorithm. This is followed by the construction of a linear mapping between these affiliations for different variables and forward in time. This linear mapping, or matrix, can be directly interpreted in light of unidirectional dependencies, and relevant properties of it are quantified. These quantifications, given by the sum of singular values and the average row variance of the matrix, then serve as measures for the influences between variables of the dynamics. The validity of the method is demonstrated with theoretical results and on several numerical examples, covering deterministic, stochastic, and delayed types of dynamics. Moreover, the method is applied to a non-classical example given by real-world basketball player movement, which exhibits highly random movement and comes without a physical intuition, contrary to many examples from, e.g., life sciences.

1. Introduction

Over the last few decades, detecting influences between variables of a dynamical system from time series data has shed light on the interplay between quantities in various fields, such as between genes [1,2], between wealth and transportation [3], between marine populations [4], or within the Earth’s climate [5,6]. Such analyses can reveal the driving forces behind complex phenomena and enable the understanding of connections that otherwise would have been left uncovered. For example, recently, novel candidates for cancer-causing genes have been detected by systematic dependency analyses using patient data [7,8,9].

There exist various conceptually different numerical methods that aim to detect relations between variables from data. For example, a prominent method is Convergent Cross-Mapping (CCM) [10], introduced in 2012, which uses the delay-embedding theorem of Takens [11]. Another method is Granger causality [12,13], which was first used in economics in the 1960s and functions based on the intuition that if a variable X forces Y, then values of X should help to predict Y. Much simpler than these methods is the well-known Pearson correlation coefficient [14], introduced already in the 19th Century and one of many methods that detect linear dependencies [15]. There exist many more, such as the Mutual Information Criterion [16], the Heller–Gorfine test [17], Kendall’s $\tau$ [18], the transfer entropy method [19], the Hilbert–Schmidt independence criterion [20], or the Kernel Granger test [21].

Recently, in [22], a summarising library comprising a broad range of statistical methods was presented together with many examples. This illustrates the large number of already existing statistical techniques to detect influences between variables of a dynamical system. However, as also pointed out in [22], such methods have different strengths and shortcomings, making them well-suited for different scenarios while ill-suited for others. For example, CCM requires data coming from dynamics on an attracting manifold and suffers when the data are subject to stochastic effects. Granger causality needs an accurate model formulation for the dynamics, which can be difficult to find. Pearson’s correlation coefficient requires statistical assumptions on the data, which are often not met. The authors of [22] therefore suggest to utilise many different methods on the same problem instead of only a single one to optimally find and interpret relations between variables.

In this article, a practical method is presented that aims at complementing the weaknesses of related methods and enriching the range of existing techniques for the detection of influences. It represents the data using probabilistic, or fuzzy, affiliations with respect to landmark points and constructs a linear probabilistic model between the affiliations of different variables and forward in time. This linear mapping then admits a direct and intuitive interpretation in regard to the relationship between variables. The landmark points are selected by a data-driven algorithm, and the model formulation is quite general, so that only very little intuition of the dynamics is required. For the fuzzy affiliations and the construction of the linear mapping, the recently introduced method Scalable Probabilistic Approximation (SPA) [23,24] is used, whose capacity to locally approximate highly nonlinear functions accurately was demonstrated in [23]. Then, the properties of this mapping are computed, which serve as the dependency measures. The intuition, which is further elaborated on in the article, now is as follows: if one variable has little influence on another, the columns of the linear mapping should be similar to each other, while if a variable has a strong influence, the columns should be very different. This is quantified using two measures, which extract quantities of the matrix, one inspired by linear algebra, the other a more statistical one. The former computes the sum of singular values of the matrix with the intuition that a matrix consisting of similar columns is close to a low-rank matrix for which many singular values are zero [25]. The latter uses the statistical variance to quantify the difference between the columns. It is shown that they are in line with the above interpretation of the column-stochastic matrix, and the method is applied to several examples to demonstrate its efficacy. Three examples are of a theoretical nature, where built-in influences are reconstructed. One real-world example describes the detection of influences between basketball players during a game.

This article is structured as follows: In Section 2, the SPA method is introduced, including a mathematical formalisation of influences between variables in the context of the SPA-induced model. In Section 3, the two dependency measures that quantify the properties of the SPA model are introduced and connected with an intuitive perception of the SPA model. In Section 4, the dependency measures are applied to examples.

To outline the structure of the method in advance, the three main steps that are proposed to quantify influences between variables of a dynamical system are (also see Figure 1):

• Representation of points by fuzzy affiliations with respect to landmark points.
• Estimation of a particular linear mapping between fuzzy affiliations of two variables and forward in time.
• Computation of the properties of this matrix.

2. Scalable Probabilistic Approximation

Scalable Probabilistic Approximation (SPA) [23] is a versatile and low-cost method that transforms points from a D-dimensional state space to K-dimensional probabilistic, fuzzy, affiliations. If K is less than D, SPA serves as a dimension reduction method by representing points as closely as possible. If $K>D$, SPA can be seen as a probabilistic, or fuzzy, clustering method, which assigns data points to landmark points in D-dimensional space depending on their closeness to them. For two different variables X and Y, SPA can furthermore find an optimal linear mapping between the probabilistic representations.

The first step, the transformation to fuzzy affiliations, will be called SPA I, while the construction of the mapping in these coordinates will be called SPA II.

2.1. Representation of the Data in Barycentric Coordinates

The mathematical formulation of SPA I is as follows: Let $\mathbf{X}=[X_1,\dots ,X_T]\in {\mathbb{R}}^{D\times T}$. Then, solve

$$\begin{array}{cc}\hfill [\Sigma ,\Gamma ]& ={arg\; min\parallel \mathbf{X}-\Sigma \Gamma \parallel }_F\hfill \\ \hfill \mathrm{subject}\mathrm{to}& \Sigma =[{\sigma }_1,\dots ,{\sigma }_K]\in {\mathbb{R}}^{D\times K},\Gamma =[{\gamma }_1,\dots ,{\gamma }_T]\in {\mathbb{R}}^{K\times T},\hfill \\ \hfill & {\left({\gamma }_t\right)}_i\ge 0,\sum _{i=1}^K{\left({\gamma }_t\right)}_i=1.\hfill \end{array}$$

(SPA 1)

It was discussed in [26] that for $K\le D$, the representation of points in this way is the orthogonal projection onto a convex polytope with vertices given by the columns of $\Sigma$. The coordinates $\gamma$ then specify the position of this projection with respect to the vertices of the polytope and are called Barycentric Coordinates (BCs). A high entry in such a coordinate then signals the closeness of the projected point to the vertex.

Remark 1.

A similar representation of points has already been introduced in PCCA+ [27].

For $K>D$, however, in [23], the interpretation of a probabilistic clustering was introduced. According to the authors, the entries of a K-dimensional coordinate of a point denote the probabilities to be inside a certain box around a landmark point. One can generalise this interpretation to fuzzy affiliations to these landmark points, again in the sense of closeness. A BC $\gamma$ then denotes the distribution of affiliations to each landmark point, and (SPA 1) can be solved without loss, i.e., so that

$$X_t=\Sigma {\gamma }_t$$

(1)

holds for all data points. Figure 2 shows the representation of a point in ${\mathbb{R}}^2$ with respect to four landmark points.

Remark 2.

From now on, regardless of the relation between K and D, mostly the term BC will be used instead of “fuzzy affiliations” for the ${\gamma }_t$ to emphasise that they are new coordinates of the data, which can always be mapped back to the original data as long as Equation (1) is fulfilled. Note that while commonly in the literature, the term “barycentric coordinate” refers to coordinates with respect to the vertices of a simplex (i.e., is $(K-1)$-dimensional with K vertices, contrary to the assumption $K>D+1$), in various publications, e.g., [28], the term generalised barycentric coordinates is used if the polytope is not a simplex. In any case, the term “generalised” will be omitted and only “BC” will be written.

For $K>D+1$ and given landmark points, the representation with barycentric coordinates is generally not unique (while the set of points that can be represented by $\Sigma \gamma$ is a convex polytope, some landmark points can even lie inside this polytope if $K>D+1$). Therefore, let us define the representation of a point X analogously to [26] in the following way:

$$\begin{array}{cc}\hfill {\rho }_{\Sigma }(X,\gamma )& :=\underset{{\gamma }^{*}}{arg\; min}{\parallel \gamma -{\gamma }^{*}\parallel }_2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.{\gamma }^{*}& =\underset{{\gamma }^{\prime }}{arg\; min}\parallel X-\Sigma {\gamma }^{\prime }{\parallel }_2\mathrm{with}{\gamma }_{•}^{\prime },{\gamma }_{•}^{*}\ge 0\mathrm{and}\parallel {\gamma }^{\prime }{\parallel }_1,{\parallel {\gamma }^{*}\parallel }_1=1.\hfill \end{array}$$

(2)

${\gamma }^{*}$ should be selected among all barycentric coordinates that represent X without loss so that it is closest to the reference coordinate$\gamma$.

Remark 3.

Note that the solution for the landmark points determined by SPA I is never unique if $K>1$ [23]. In order to solve (SPA 1), its objective function is iteratively minimised by separately solving for Σ and Γ. For this, initial values are randomly drawn. Therefore, for $K>D+1$, an exact solution can be achieved by placing $D+1$ points so that all data points lie inside their convex hull (all convex combinations of them) while the remaining $K-(D+1)$ landmark points can be chosen arbitrarily. As a consequence, the placement of landmark points depends strongly on the randomly chosen initial values of the optimisation process. In simple cases, e.g., when the state space of the data is one-dimensional, one could even manually place landmark points across an interval containing the data and solve only for Γ.

Remark 4.

Note that with a fixed Σ, the computations of all columns of Γ are independent of each other so that this step can easily be parallelised, making SPA a very efficient method for long time series and justifying the term “Scalable”. Moreover, the time complexity of the full (SPA 1) problem grows linearly with the number of data points [23], which is comparable to the K-means method [29], while still giving a much more precise, for $K\ge D+1$ even lossless, representation of points.

Let now a dynamical system be given by

$$X_t=F\left(X_{t-1}\right).$$

(3)

In this case, let us select ${\gamma }_{t-1}$ as the reference coordinate for the point $X_t$. Using that Equation (1) holds for $K>D$, this gives

$${\gamma }_t={\rho }_{\Sigma }(X_t,{\gamma }_{t-1})={\rho }_{\Sigma }(F\left(X_{t-1}\right),{\gamma }_{t-1})={\rho }_{\Sigma }(F(\Sigma {\gamma }_{t-1}),{\gamma }_{t-1})=:\mathbf{v}\left({\gamma }_{t-1}\right).$$

(4)

With this, ${\gamma }_t$ solely depends on ${\gamma }_{t-1}$. This establishes a time-discrete dynamical system in the barycentric coordinates. By choosing the reference coordinate as ${\gamma }_{t-1}$, it is asserted that the steps taken are as short as possible.

2.2. Model Estimation between Two Variables

Assuming that data from two different variables $X\in {\mathbb{R}}^D$ and $Y\in {\mathbb{R}}^E$ is considered and one strives to investigate the relationship between them, one can solve (SPA 1) for both of them, finding landmark points denoted by the columns of ${\Sigma }^X\in {\mathbb{R}}^{D\times K_X},{\Sigma }^Y\in {\mathbb{R}}^{E\times K_Y}$ and BCs ${\gamma }_t^X$ and ${\gamma }_t^Y$ for $t=1,\dots ,T$ and solve Equation (SPA 2), given by

$$\begin{array}{cc}\hfill \Lambda & =\underset{{\Lambda }^{*}\in {\mathbb{R}}^{K\times K}}{arg\; min}\parallel [{\gamma }_1^Y|\dots |{\gamma }_T^Y]-{\Lambda }^{*}[{\gamma }_1^X|\dots |{\gamma }_T^X{]\parallel }_F,\hfill \\ \hfill \mathrm{subject}\mathrm{to}& \Lambda \ge 0\mathrm{and}\sum _{k=1}^K{\Lambda }_{k,•}=1.\hfill \end{array}$$

(SPA 2)

$\Lambda$ is a column-stochastic matrix and, therefore, guarantees that BCs are mapped to BCs again. It is an optimal linear mapping with this property that it connects X to Y on the level of the BCs. One can easily transform back to the original state space by

$$Y_t=\Sigma {\gamma }_t^Y\approx \Sigma \Lambda {\gamma }_t^X.$$

(5)

Since, by construction, the prediction must lie inside the polytope again, it is a weighted linear interpolation with respect to the landmark points, similar as, e.g., in Kriging interpolation [30]. Note that is was demonstrated in [23] that this linear mapping applied to the BCs can accurately approximate functions of various complexity and is not restricted to linear relationships.

Remark 5.

In [23], a way to combine SPA I and SPA II into a single SPA I problem was shown. The landmark points are then selected so that the training error of the SPA II problem can be set to 0. In other words, optimal discretisations of the state spaces are determined where optimal means with respect to the exactness of the ensuing (SPA 2) model.

Estimation of Dynamics

A special case of (SPA 2) is the estimation of dynamics, i.e., solving

$$\begin{array}{cc}\hfill \Lambda & =\underset{{\Lambda }^{*}\in {\mathbb{R}}^{K\times K}}{arg\; min}\parallel [{\gamma }_2^X|\dots |{\gamma }_T^X]-{\Lambda }^{*}[{\gamma }_1^X|\dots |{\gamma }_{T-1}^X{]\parallel }_F,\hfill \\ \hfill \mathrm{subject}\mathrm{to}& \Lambda \ge 0\mathrm{and}\sum _{k=1}^K{\Lambda }_{k,•}=1.\hfill \end{array}$$

(6)

$\Lambda$ is therefore a linear, column-stochastic approximation of the function $\mathbf{v}$ from Equation (4). Such a matrix is typically used in so-called Markov State Models [31,32]. With $\Lambda$, one can construct dynamics in the BCs with the possibility to transform back to the original state space by multiplication with $\Sigma$, since

$$X_t=\Sigma {\gamma }_t=\Sigma \mathbf{v}\left({\gamma }_{t-1}\right)\approx \Sigma \Lambda {\gamma }_{t-1}.$$

(7)

2.3. Forward Model Estimation between Two Processes

Given two dynamical systems:

$$X_t=F\left(X_{t-1}\right)\in {\mathbb{R}}^D\mathrm{and}{Y}_t=G\left(Y_{t-1}\right)\in {\mathbb{R}}^E,$$

(8)

let us now determine a column-stochastic matrix that propagates barycentric coordinates from one variable to the other and forward in time. This mapping will be used to quantify the effect of one variable on future states of the other.

With landmark points in ${\Sigma }^X\in {\mathbb{R}}^{D\times K_X},{\Sigma }^Y\in {\mathbb{R}}^{E\times K_Y}$ and BCs ${\gamma }_t^X$ and ${\gamma }_t^Y$ for $t=1,\dots ,T$, let us find a column-stochastic matrix ${\Lambda }_{XY}$ that fulfils

$$\begin{array}{c}\hfill {\Lambda }_{XY}=\underset{{\Lambda }^{*}\in {\mathbb{R}}^{K_Y\times K_X}}{arg\; min}\parallel [{\gamma }_2^Y|\dots |{\gamma }_T^Y]-{\Lambda }^{*}[{\gamma }_1^X|\dots |{\gamma }_{T-1}^X{]\parallel }_F.\end{array}$$

(9)

${\Lambda }_{XY}$ represents a model from $X_{t-1}$ to $Y_t$ on the level of the BCs and tries to predict subsequent values of Y using only X.

Now, let us assume that X in fact has a direct influence on Y, meaning that there exists a function:

$$H(Y_{t-1},X_{t-1})=Y_t.$$

(10)

Then, similar as when constructing the dynamical system in the BCs previously in Equation (4), we can observe, using Equations (10) and (1),

$$\begin{array}{cc}\hfill {\gamma }_t^Y& ={\rho }_{{\Sigma }^Y}(Y_t,{\gamma }_{t-1}^Y)\hfill \\ \hfill & ={\rho }_{{\Sigma }^Y}(H(Y_{t-1},X_{t-1}),{\gamma }_{t-1}^Y)\hfill \\ \hfill & ={\rho }_{{\Sigma }^Y}(H({\Sigma }^X{\gamma }_{t-1}^X,{\Sigma }^Y{\gamma }_{t-1}^Y),{\gamma }_{t-1}^Y)\hfill \\ \hfill \mathrm{defined}\mathrm{as}& =:\mathbf{w}({\gamma }_{t-1}^X,{\gamma }_{t-1}^Y).\hfill \end{array}$$

(11)

${\gamma }_t^Y$ therefore directly depends on ${\gamma }_{t-1}^X$ and ${\gamma }_{t-1}^Y$, while ${\Lambda }_{XY}$ attempts to predict ${\gamma }_t^Y$ using only ${\gamma }_{t-1}^X$. Assuming that the approximation:

$${\Lambda }_{XY}{\gamma }_{t-1}^X\approx {\gamma }_t^Y$$

(12)

is close for each pair of ${\gamma }_{t-1}^X,{\gamma }_t^Y$, one can assert

$${\gamma }_t^Y\approx {\Lambda }_{XY}{\gamma }_{t-1}^X=\sum _{j=1}^{K_X}{\left({\Lambda }_{XY}\right)}_{|j}{\left({\gamma }_{t-1}^X\right)}_j,$$

(13)

where ${\left({\Lambda }_{XY}\right)}_{|j}$ is the jth column of ${\Lambda }_{XY}$. A prediction for ${\gamma }_t^Y$ is therefore constructed using a weighted average of the columns of ${\Lambda }_{XY}$. The weights are the entries of ${\gamma }_{t-1}^X$.

Remark 6.

Note that the same argumentation starting in Equation (9) holds if one chooses a time shift of length $\tau >0$ and considers information of $X_{t-\tau }$ about $Y_t$. If $\tau =1$, it will simply be written ${\Lambda }_{XY}$, but otherwise, ${\Lambda }_{XY}^{(\tau )}$.

Remark 7.

Since ${\Lambda }_{XY}{\gamma }_{t-1}^X$ estimates ${\gamma }_t^Y$ using only ${\gamma }_{t-1}^X$, although it additionally depends on ${\gamma }_{t-1}^Y$, one can interpret ${\Lambda }_{XY}$ as an approximation to the conditional expectation of ${\gamma }_t^Y$ given ${\gamma }_{t-1}^X$, i.e., assuming that ${\gamma }_{t-1}^Y$ is distributed by a function ${\mu }_Y$,

$${\Lambda }_{XY}\gamma \stackrel{}{\approx }{\mathbb{E}}_{{\mu }_Y}\left[{\gamma }_t^Y\right|{\gamma }_{t-1}^X=\gamma ].$$

(14)

In Appendix A.1, this intuition is formalised further.

In the original SPA paper [23], the ${\gamma }_t$ were interpreted as probabilities to “belong” to states σ that describe a discretisation of the state space. Λ then uses the law of total probability and the $ij$-entry denotes ${\left({\Lambda }_{XY}\right)}_{ij}=\mathbb{P}[Y_t={\sigma }_i^Y|X_{t-1}={\sigma }_j^X]$. Together with the combined solution of the (SPA 1) and (SPA 2) problems outlined in Remark 5, the authors of [23] described their method as finding an optimal discretisation of the state space to generate an exact model on the probabilities, thereby satisfying desired physical conservation laws. In this light, SPA then directly competes with the before-mentioned Markov State Models (MSMs) with the significant advantage that for MSMs with equidistant discretisations, i.e., grids, the number of “grid boxes” increases exponentially with the dimension and box size. This is further elaborated in Appendix A.3. While the probabilistic view is a sensible interpretation of the objects involved in SPA, one is not restricted to it, and the view taken in this article focuses on the exact point representations, which the barycentric coordinates yield. However, also acknowledging the probabilistic view point, it should be possible to make the arguments following in the next sections in a satisfying and intuitive way.

3. Quantification of Dependencies between Variables

In the following, two methods that quantify the strength of dependence of $Y_t$ on $X_{t-1}$ by directly computing the properties of ${\Lambda }_{XY}$ will be defined. The intuition can be illustrated as follows: If a variable X is important for the future state of another variable, Y, then the multiplication of ${\Lambda }_{XY}$ with ${\gamma }_{t-1}^X$ should give very different results depending on which landmark point $X_{t-1}$ is close to, i.e., which weight in ${\gamma }_{t-1}^X$ is high. Since ${\gamma }_t^Y$ is composed by a weighted average of the columns of ${\Lambda }_{XY}$ by Equation (13), this means that the columns of ${\Lambda }_{XY}$ should be very different from each other. In turn, if X has no influence and carries no information for future states of Y, the columns of ${\Lambda }_{XY}$ should be very similar to each other. In Appendix A.1, it is shown that given the independence of ${\gamma }_t^Y$ from ${\gamma }_{t-1}^X$, all columns of ${\Lambda }_{XY}$ should be given by the mean of the ${\gamma }^Y$ in the data. There, this is also connected to conditional expectations and the intuition given in Equation (14).

In the extreme case that the columns are actually equal to each other, ${\Lambda }_{XY}$ would be a rank-1 matrix. If the columns are not equal, but similar, ${\Lambda }_{XY}$ is at least close to a rank-1 matrix. One should therefore be able to deduce from the similarity of the columns of ${\Lambda }_{XY}$ if X could have an influence on Y. This is the main idea behind the dependency measures proposed in this section. The intuition is similar to the notion of the predictability of a stochastic model introduced in [33].

Remark 8.

Note that if there is an intermediate variable Z that is forced by X and forces Y while X does not directly force Y, then it is generally difficult to distinguish between the direct and indirect influences. In Section 4.2, an example for such a case is investigated.

3.1. The Dependency Measures

Now, the two measures are introduced that will be used for the quantification of dependencies between variables. Note that these are not “measures” in the true mathematical sense, but the term is rather used as synonymous with “quantifications”.

3.1.1. Schatten-1 Norm

For the first measure, let us consider the Singular Value Decomposition (SVD) [34] of a matrix $\Lambda$, given by $\Lambda =US{V}^T\in {\mathbb{R}}^{K_Y\times K_X}$. $S\in {\mathbb{R}}^{K_Y\times K_X}$ is a matrix that is only nonzero in the entries $(i,i)$ for $i=1,\dots ,min(K_X,K_Y)$, which are denoted by $s_1,\dots ,s_{min(K_X,K_Y)}\ge 0$. $U\in {\mathbb{R}}^{K_Y\times K_Y}$ and $V\in {\mathbb{R}}^{K_X\times K_X}$ fulfil certain orthogonality properties and consist of columns $u_i,v_i$. One can thus write $\Lambda$ as

$$\Lambda =\sum _{i=1}^r{u}_i{v}_i^T{s}_i$$

(15)

A classic linear algebra result asserts that $rank(\Lambda )=\#\{s_i>0\}$. As a consequence, if some of the $s_i$ are close to 0, then Equation (24) means that only a small perturbation is sufficient to make $\Lambda$ a matrix of lower rank. Therefore, the sum of singular values, the so-called Schatten-1 norm [35], will be used as a continuous measure of the rank and, thus, of the difference in the rows of $\Lambda$.

Definition 1

(Schatten-1 norm). Let the SVD of a matrix $\Lambda \in {\mathbb{R}}^{K_Y\times K_X}$ be given by $\Lambda =US{V}^T$ with singular values $s_1,\dots ,s_{min(K_X,K_Y)}$. Then, the Schatten-1 norm of Λ is defined as

$${\parallel \Lambda \parallel }_1:=\sum _{i=1}^{min(K_X,K_Y)}{s}_i.$$

(16)

3.1.2. Average Row Variance

As the second dependency measure, the difference of the columns of a matrix $\Lambda$ is directly quantified using the mean statistical variance per row. Therefore, every row is considered, and the variance between its entries is computed, thereby comparing the columns with respect to this particular row. Then, the mean of these variances is taken across all rows.

Definition 2

(Average row variance). For a matrix $\Lambda \in {\mathbb{R}}^{K_Y\times K_X}$, let ${\overline{\Lambda }}_{i-}$ denote the mean of the ith row of Λ. Let

$${\nu }_i:=\frac{1}{K_X-1}\sum _{j=1}^{K_X}{({\Lambda }_{ij}-{\overline{\Lambda }}_{i-})}^2$$

be the statistical variance of the entries of the ith row. Then, the average row variance is defined as

$$\nu (\Lambda ):=\frac{1}{K_Y}\sum _{i=1}^{K_Y}{\nu }_i.$$

(17)

The calculated values for ${\parallel ·\parallel }_1$ and $\nu$ will be stored in tables, respectively, matrices of the form

$$M_{{\parallel ·\parallel }_1}=\left(\begin{array}{cc}\parallel {\Lambda }_{XX}{\parallel }_1& \parallel {\Lambda }_{XY}{\parallel }_1\\ \parallel {\Lambda }_{YX}{\parallel }_1& \parallel {\Lambda }_{YY}{\parallel }_1\end{array}\right),M_v=\left(\begin{array}{cc}\nu \left({\Lambda }_{XX}\right)& \nu \left({\Lambda }_{XY}\right)\\ \nu \left({\Lambda }_{YX}\right)& \nu \left({\Lambda }_{YY}\right)\end{array}\right).$$

(18)

Then, for each of these matrices, the property $M-M^T$ should be interesting for us, because they contain the differences between dependency measures, stating how strongly X depends on Y compared to Y depending on X. Let us therefore define the following:

Definition 3

(Relative difference between dependencies). Let M be one of the matrices from Equation (18). The relative difference between dependencies in both directions is defined as

$$\delta {\left(M\right)}_{ij}=\frac{M_{ij}-M_{ji}}{max(M_{ij},M_{ji})}.$$

(19)

Remark 9.

In Appendix A.3, it is explained how this method differs from linear correlations, Granger causality, and the same approach using boxes instead of fuzzy affiliations.

When using the dependency measures to analyse which of two variables more strongly depends on the other, it is unclear at this point whether the dimension of the variables and the number of landmark points affects the outcome of the analysis. Hence, for all numerical examples in the next section, pairs of variables that have the same dimension are considered, and the same number of landmark points for them is used, i.e., it holds $K_X=K_Y$ in each example to make the comparison as fair as possible. In this light, the following theoretical results on the Schatten-1 norm and the average row variance are directly helpful.

3.2. Maximisers and Minimisers of the Dependency Measures

About ${\parallel ·\parallel }_1$ and $\nu$, one can prove properties that validate why they represent sensible measures for the strengths of dependency between two processes. For this, let us make the following definition.

Definition 4

(Permutation matrix). As a permutation matrix, a matrix $A\in {\{0,1\}}^{n\times n}$ is defined with the property that every row and column contains exactly one 1.

Then, one can prove the following results on the maximisers and minimisers of the Schatten-1 norm and average row variance (half of them only for $K_Y\ge K_X$ or $K_Y=K_X$), whose proofs can be found in Appendix A.2.

Proposition 1

(Maximal Schatten-1 norm, $K_Y\ge K_X$). Let $\Lambda \in {\mathbb{R}}^{K_Y\times K_X}$ with $K_Y\ge K_X$. Then, the Schatten-1 norm of Λ obtains the maximal value $K_X$ if deletion of $K_Y-K_X$ rows of Λ yields a $K_X\times K_X$ permutation matrix.

Proposition 2

(Minimal Schatten-1 norm). The Schatten-1 norm of a column stochastic $(K_Y\times K_X)$-matrix A is minimal if and only if $A_{ij}\equiv \frac{1}{n}$, and its minimal value is equal to 1.

For the average row variance, the following results can be derived:

Proposition 3

(Maximal average row variance, $K_Y=K_X$). The average row variance of a column-stochastic $(K_Y\times K_X)$-matrix Λ with $K_Y=K_X$ obtains the maximal value $\frac{1}{K_Y}$ if it is a $K_X\times K_X$ permutation matrix.

It seems likely that for $K_Y>K_X$, the maximisers of the Schatten-1 norm from Proposition 1 also maximise the average row variance with maximal value $\frac{1}{K_Y}$.

Proposition 4

(Minimal average row variance). The average row variance of a column-stochastic $(K_Y\times K_X)$-matrix Λ obtains the minimal value 0 if and only if all columns are equal to each other.

In order to analyse the dependencies between two variables for which one uses different numbers of landmarks, i.e., $K_X\ne K_Y$, it would be desirable if similar results as above could be inferred for the case $K_Y<K_X$ so that one could make valid interpretations of both ${\Lambda }_{XY}$ and ${\Lambda }_{YX}$. However, it was more difficult to prove them, so that only the following conjectures are made for the case $K_Y<K_X$:

Conjecture 1.

(Maximal Schatten-1 norm, $K_Y<K_X$) The Schatten-1 norm of a column-stochastic $(K_Y\times K_X)$-matrix $\Lambda$ with $K_Y<K_X$ is maximal if and only if $\Lambda$ contains a $K_Y\times K_Y$ permutation matrix and the matrix of the remaining $K_X-K_Y$ columns can be extended by $K_Y$ columns to a permutation matrix.

Conjecture 2.

(Maximal average row variance, $K_Y<K_X$)The average row variance of a column-stochastic $(K_Y\times K_X)$-matrix $\Lambda$ with $K_Y<K_X$ is maximal if and only if $\Lambda$ contains an $K_Y\times K_Y$ permutation matrix and the matrix of the remaining $K_X-K_Y$ columns can be extended by $K_Y$ columns to a permutation matrix.

In summary, the maximising and minimising matrices of $\parallel ·\parallel$ and $\nu$ are identical and are of the following forms:

$$\mathrm{Maximal}:\left(\begin{array}{ccc}0& 1& 0\\ 0& 0& 1\\ 1& 0& 0\\ 0& 0& 0\end{array}\right),\mathrm{Minimal}:\left(\begin{array}{ccc}\frac{1}{n}& \dots & \frac{1}{n}\\ ⋮& & ⋮\\ \frac{1}{n}& \dots & \frac{1}{n}\end{array}\right).$$

(20)

These results show that the two dependency measures ${\parallel ·\parallel }_1$ and $\nu$ fulfil important intuitions: they are minimal, when information about $X_{t-1}$ gives us no information about $Y_t$ because, in this case, all columns of $\Lambda$ should be identical and even equal to each other. Maximal dependence is detected if the information about $X_{t-1}$ is maximally influential on $Y_t$. This happens when $\Lambda$ is, respectively can be reduced or extended to, a permutation matrix. This is illustrated in Figure 3.

4. Numerical Examples

Now, the two dependency measures are demonstrated on examples of dynamical systems with the aim to cover different structures and properties. In order to assess their efficacy, unidirectional dependencies are explicitly installed in the formulation of the dynamics, and it is investigated whether these are detected. For the usage of the SPA method, the Matlab code provided at https://github.com/SusanneGerber/SPA (accessed 14 August 2019) was used. For two examples, the results are briefly compared with those of Convergent Cross-Mapping (CCM) for which a Matlab implementation provided in [36] was used.

4.1. Low-Dimensional and Deterministic: The Hénon System

The first example is the classical Hénon system [37]. It describes a two-dimensional dynamical system on an attracting set, meaning that all trajectories converge to a specific set of points. Its equations read

$$\begin{array}{cc}\hfill X_t& =1-a{X}_{t-1}^2+Y_t\hfill \\ \hfill Y_t& =b{X}_{t-1}.\hfill \end{array}$$

(21)

For the—typically used—values of $a=1.4,b=0.3$, this makes X largely autonomous, while Y is merely a scaled copy of the previous value of X. Hence, Y should depend more strongly on the previous value of X than vice versa. Please find an image of a trajectory of this dynamical system in Appendix A.4.

Dependency Analysis

For the dependency analysis, it is investigated whether the higher dependence of Y on X can be reproduced by the dependency measures. A time shift $\tau$ of one time step is chosen for the analysis. Note that for $K>D+1$, the (SPA 1) solution is not unique in both ${\sigma }_i$ and ${\gamma }_t$. As explained before in Remark 3, the solution can depend on the randomly chosen initial values of the optimisation process. Therefore, 20 different solutions for $K_X,K_Y=2,\dots ,10$ of length 2000 time steps are computed, the dependency analysis is performed, and the mean over the dependency results is taken for each value of $K_X,K_Y$. The starting values ${\gamma }_0^X,{\gamma }_0^Y$ of the trajectories with respect to the landmarks are for both variables always chosen as positive only for the two landmarks directly above and below the initial points $X_0,Y_0$. The result of the analysis is shown in Figure 4.

It can be observed that, consistently, the stronger influence of X on Y is reconstructed since, for all values of $K_X,K_Y$, the green graph corresponding to the direction X to Y lies above the purple one corresponding to the other direction. Surprisingly, the relative difference decreases for increasing values for $K_X,K_Y$ for both measures in this example. Note that with increasing dimension of the $\Lambda$ matrix, the sum of singular values should naturally increase in magnitude. This is not the case for the average row variance for which one divides by the number of columns and rows. It is, however, surprising that for only two landmarks, the difference is that pronounced by $\nu$ compared to a higher number of landmarks. Overall, this example gives a satisfying result for the dependency measures since both were able to detect the higher influence of X.

It was checked whether the results were consistent for different (SPA 1) solutions: only in 3 out of 180 cases (9 values for $K_X,K_Y$ and 20 repetitions for each), Y was assigned a higher influence than X by the average row variance, while this did not happen with the Schatten-1 norm. It was also checked whether using equidistant landmarks instead of (SPA 1) solutions, a similar result could be produced. In fact, this led to more strongly emphasised differences and the relative difference being very stable over the number of landmarks (see Appendix A.4). This can be interpreted as an even better result. However, for systems in higher dimensions, this approach entails the problem that the number of landmarks would scale exponentially with the dimension, thereby potentially making it infeasible. This is also discussed in Appendix A.3.

4.2. Low-Dimensional and Stochastic: Continuous Movement by Stochastic Diffusions

This model describes a continuous evolution of processes $A,B,C$ along solutions of a Stochastic Differential Equation (SDE) [38] where C acts autonomously and A and B hierarchically depend on each other. In short:

4.2.1. The Dynamics

The SDE is given by

$$\begin{array}{cc}\hfill d{C}_t& =G\left(C_t\right)dt+{\sigma }_{C}d{W}_{Ct}\hfill \\ \hfill d{B}_t& =(\alpha H\left(B_t\right)-10(B_t-C_t))dt+{\sigma }_{B}Idd{W}_{Bt}\hfill \\ \hfill d{A}_t& =(\alpha H\left(A_t\right)-5(A_t-B_t)-5(A_t-C_t))dt+{\sigma }_{A}Idd{W}_{At}\hfill \end{array}$$

(22)

where

$$G\left(x\right)=\left(\begin{array}{c}0\\ -10(x_2^3-x_2)\end{array}\right),H\left(x\right)=\left(\begin{array}{c}-(x_1^3-x_1)\\ -1\end{array}\right)$$

(23)

and $Id$ is the identity matrix.

The choice of the function G ensures a metastable behaviour of C between regions around the values $-1$ and 1 in the $x_2$-coordinate, meaning that it remains around one of the values for some time and, due to the randomness in the SDE, at some point in time, suddenly moves towards the other value. In the $x_1$-direction, the movement of C is determined by noise with covariance matrix ${\sigma }_C$. The movements of the other processes are partly governed by the function H, which gives a metastable behaviour in the $x_1$-coordinates, and by a difference function between themselves and C, respectively (Figure 5 for ${\sigma }_A={\sigma }_B=0.01$). The movement of A depends equally on its difference from C as on its difference from B. Since diffusion processes are attracted to low values of their governing potential, this urges B to move into the direction of C. Furthermore, it urges A into the directions of both C and B. The parameter $\alpha$ therefore governs how autonomous the processes A and B are.

Realisations of the processes are created with the Euler–Maruyama scheme [39] with a time step of size $\Delta t=0.1$ for 1000 time steps. The parameters were set to $\alpha =5$ and ${\sigma }_C=diag\left((0.01,0.05)\right)$. For the noise in the evolution of A and B, multiple different values ${\sigma }_B={\sigma }_A=0.01,0.2,0.5$ and 1 were used.

4.2.2. Dependency Analysis

For SPA I, again, $K_X=K_Y=10$ was used. Contrary to the previous example, here, the dependencies of the time-differences $\Delta A,\Delta B,\Delta C$ between time steps were computed, meaning that instead of ${\Lambda }_{XY}^{(\Delta t)}$, ${\Lambda }_{\Delta X\Delta Y}^{(\Delta t)}$ for $A,B,C=X,Y$ was computed. It was also attempted to reconstruct the dependencies using the process values instead of the time-differences, but, while one could still reconstruct directions of influences, this came with a lower emphasis in the quantitative results. This presumably is the case since the SDE governs not the next state, but rather, the next time-differenced value of each process, making the time-differences a better choice here.

For different values for the noise variances ${\sigma }_A,{\sigma }_B$, 50 realisations each of the SDE Equation (22) were created with the same initial conditions, and the analysis was performed for each. The SPA I solution was always computed anew, so that these solutions were generally different from each other, to test the robustness of the method with regard to the SPA I solution.

The results of the dependency analysis well reflected the hierarchical dependencies between the three processes. An exemplary result is given in Equations (24) and (25). The statistics of the overall analysis are given in

Table 1. Results of the dependency analysis for the diffusion processes in Equation (22). The third and fifth columns denote the average relative difference between the first variable (in the first row A) and the second variable (in the first row B) in the Schatten-1 norm and the average row variance. It is always negative, meaning that the first variable (with lower influence) was on average correctly measured as less influential than the second variable. The fourth and sixth columns denote the relative number of occurrences when one variable was falsely identified as more influential, e.g., A as more influential than B.

${\mathit{\sigma }}_{\mathit{A}},{\mathit{\sigma }}_{\mathit{B}}$	Variables	Average $\mathit{\delta }\left({\mathit{M}}_{{\parallel ·\parallel }_1}\right)$	Incorrect	Average $\mathit{\delta }\left({\mathit{M}}_{\mathit{\nu }}\right)$	Incorrect
0.1	$A,B$	−0.21	0.12	−0.36	0.2
	$A,C$	−0.22	0.08	−0.56	0.08
	$B,C$	−0.46	0	−0.83	0.00
0.2	$A,B$	−0.2	0.06	−0.4	0.1
	$A,C$	−0.17	0.16	−0.51	0.14
	$B,C$	−0.36	0	−0.76	0
0.5	$A,B$	−0.2	0	−0.4	0.02
	$A,C$	−0.12	0.14	−0.43	0.12
	$B,C$	−0.30	0.02	−0.71	0.04
1	$A,B$	−0.25	0.02	−0.65	0
	$A,C$	−0.04	0.28	−0.19	0.30
	$B,C$	−0.22	0.08	−0.62	0.02

. It shows that, in the vast majority of the realisations, the less influential out of two variables was correctly measured as such. The average row variance $\nu$ gives more clear results with the minimal relative difference at least $0.19$ for large noise, but generally around $0.4$. For the Schatten-1 norm, the relative differences were mostly around $0.2$. Note that the results were not strongly influenced by the strength of noise, which loosens the strict dependence of A and B on C, except for the dependencies between A and C for ${\sigma }_A={\sigma }_B=1$ for which often A was falsely measured to have the stronger influence between the two.

It was also checked whether the Pearson correlation coefficient was able to detect the directional influences between the variables. It turned out that for ${\sigma }_A,{\sigma }_B=0.01$, the time-differences between the variables were highly correlated according to the imposed hierarchy, e.g., $\Delta A_{t+\Delta t}$ was highly correlated with $\Delta B_t$, but for ${\sigma }_A,{\sigma }_B\ge 0.2$, mostly correlation coefficients below $0.1$ could be observed, indicating that the correlation coefficient was not well suited to discover directional dependencies for these dynamics, while the dependency measures could do so. CCM did not manage to correctly identify the direction of dependencies in most realisations of the SDE. By construction, the strengths of CCM rather lied in the domain of deterministic dynamics on an attracting manifold (since it uses the delay-embedding theorem of Takens [11]), but it is less suited for stochastic dynamics, such as this SDE.

$$M_{{\parallel ·\parallel }_1}:\begin{array}{|cccc|}\hline \mathrm{From}\downarrow \mathrm{to}\to \hfill & \Delta A_{t+\Delta t}\hfill & \Delta B_{t+\Delta t}\hfill & \Delta C_{t+\Delta t}\\ \Delta A_t\hfill & 5.51& 3.86\hfill & 3.46\\ \Delta B_t\hfill & 5.29& 4.11\hfill & 2.94\\ \Delta C_t\hfill & 4.51& 7.72\hfill & 4.26\\ \hline\end{array}\Rightarrow \delta \left(M_{{\parallel ·\parallel }_1}\right)=\left(\begin{array}{ccc}0& \text{−0.27}& \text{−0.23}\\ 0.27& 0& \text{−0.61}\\ 0.23& 0.61& 0\end{array}\right)$$

(24)

$$M_{\nu }:\begin{array}{|cccc|}\hline \mathrm{From}\downarrow \mathrm{to}\to \hfill & \Delta A_{t+\Delta t}& \Delta B_{t+\Delta t}\hfill & \Delta C_{t+\Delta t}\\ \Delta A_t\hfill & 0.039& 0.021\hfill & 0.018\\ \Delta B_t\hfill & 0.046& 0.024\hfill & 0.016\\ \Delta C_t\hfill & 0.039& 0.065\hfill & 0.026\\ \hline\end{array}\Rightarrow \delta \left(M_{\nu }\right)=\left(\begin{array}{ccc}0& \text{−0.54}& \text{−0.53}\\ 0.54& 0& \text{−0.75}\\ 0.53& 0.75& 0\end{array}\right)$$

(25)

4.3. Higher-Dimensional, Stochastic, and Delayed: Multidimensional Autoregressive Processes

In order to demonstrate that influence can be detected for processes whose evolution depends not only on present, but on past terms, as well, realisations of multidimensional linear Autoregressive processes (AR) [40,41] were simulated in which some variables were coupled with others. An n-dimensional linear AR(p) process is a dynamical system of the form

$$X_t=\sum _{i=1}^p{\varphi }_i{X}_{t-1}+{\epsilon }_t$$

(26)

where ${\varphi }_i\in {\mathbb{R}}^{n\times n}$ and ${\epsilon }_t$ is a stochastic term, which was set to be normally distributed with mean 0 and (positive semi-definite) covariance matrix $C\in {\mathbb{R}}^{n\times n}$.

4.3.1. The Dynamics

Let us now consider AR(p) processes of the form

$$\left(\begin{array}{c}{X}_t\\ Y_t\end{array}\right)=\sum _{i=1}^q\left(\begin{array}{cc}{\varphi }_i^{XX}& {\varphi }_i^{YX}\\ 0& {\varphi }_i^{YY}\end{array}\right)\left(\begin{array}{c}{X}_{t-i}\\ Y_{t-i}\end{array}\right)+{\epsilon }_t.$$

(27)

Specifically, let X and Y be variables in ${\mathbb{R}}^4$. Thus, ${\varphi }_i^{XX},{\varphi }_i^{YX}$ and ${\varphi }_i^{YY}$ are matrices in ${\mathbb{R}}^{4\times 4}$. Through the structure of the coefficient matrices, it was imposed that X is influenced by Y, but not vice versa. $p=3$ was set, and ${\varphi }_i$ were constructed randomly by drawing for each entry a normally distributed value with mean 0 and variance ${\sigma }_i$, where ${\sigma }_1=0.1,$${\sigma }_2=0.05,{\sigma }_3=0.03$. This should lead to the coefficients gradually decreasing in magnitude for increasing time lag, so that the most recent process values should have the most influence on future ones. ${\epsilon }_t$ is normally distributed with mean 0 and covariance matrix $0.01Id$ (where $Id$ is the identity matrix). Then, a realisation of length $T=1000$ was created for such a process. This procedure was executed 50 times and the dependency analysis performed on each realisation.

4.3.2. Dependency Analysis

$K_X=K_Y=10$ was chosen again, and ${\Lambda }_{XY}^{(\tau )}$ and ${\Lambda }_{YX}^{(\tau )}$ were computed for $\tau =1,3,10,50$ to investigate how the dependence between the processes evolve with increasing time shift (contrary to the previous example, the process values and not the time differences were used).

One can see in

Table 2. Results for dependency measures of X on Y and vice versa for 50 realisations of processes of the form of Equation (27). The third and fifth columns represent the average absolute value of the Schatten-1 norm and average row variance. They decrease with increasing time lag $\tau$, which is consistent with the reconstruction. The fourth and sixth columns denote the average relative differences, which for $\tau =1,3$ correctly indicate that Y is more influential on X than vice versa, while for $\tau =10,50$, this is no longer the case, which again is consistent with the construction.

$\mathit{\tau }$	Direction	Average ${\parallel ·\parallel }_1$	Average $\mathit{\delta }\left({\mathit{M}}_{{\parallel ·\parallel }_1}\right)$	Average $\mathit{\nu }·{10}^2$	Average $\mathit{\delta }\left({\mathit{M}}_{\mathit{\nu }}\right)$
1	$X_{t-1}\to Y_t$	1.94	−0.19	1.67	−0.58
	$Y_{t-1}\to X_t$	2.43	0.19	4.40	0.58
3	$X_{t-3}\to Y_t$	1.88	−0.15	1.43	−0.49
	$Y_{t-3}\to X_t$	2.22	0.15	3.20	0.49
10	$X_{t-10}\to Y_t$	1.82	−0.02	1.27	−0.06
	$Y_{t-10}\to X_t$	1.86	0.02	1.38	0.06
50	$X_{t-50}\to Y_t$	1.86	0.01	1.39	0.02
	$Y_{t-50}\to X_t$	1.86	−0.01	1.36	−0.02

that the stronger influence of Y on X is recovered for $\tau =1$ and $\tau =3$. For $\tau =1$, the relative differences were stronger, which is in line with the fact that the AR coefficients ${\varphi }_3$ were selected to be smaller in magnitude than for ${\varphi }_1$, so that $X_t$ should be more strongly influenced by $Y_{t-1}$ than by $Y_{t-3}$. Moreover, not only the relative differences were smaller for $\tau =3$, but also, the average absolute number of the measures, again correctly indicating a smaller cross-influence with bigger time shift. For $\tau \ge 10$, only negligible differences can be seen. This is consistent with the construction of the processes that include a direct influence up to $\tau =3$.

Again, it was checked if the Pearson correlation coefficient or CCM could recover the unidirectional dependencies, but this gave negative results: the correlation coefficient was again below $0.1$ for most realisations and did not indicate significantly stronger correlation in either direction. CCM indicated only very weak and no directional influence between the variables. By construction, the Granger causality method should perform very well here since the way the dynamics are defined in Equation (27) is perfectly in line with the assumptions of the Granger method. This, of course, cannot be expected for dynamics with an unknown model formulation, such as in the following real-world example.

4.4. Real-World Example: Basketball Player Movement

The dependency measures will now be applied to the movement of basketball players during a game and quantify influences between players in the same manner as in the previous examples. For this, player tracking data captured by the SportVU technology (Stats LLC, Chicago, IL, USA) and publicly provided by Neil Johnson [42] from a game of the 2015/16 NBA season between the Dallas Mavericks and the Cleveland Cavaliers, played on 12 January 2016 in Dallas, were used. The data contain the x- and y-coordinates of each player on the court in 25 time frames per second for most of the 48 min of play. The ball data sometimes seem out of sync with the positions of the players and are not always available; therefore, they were not used in the analysis. The positions were measured in the units feet (ft). Using the same data, although from other games, there are several scientific publications, e.g., [43,44], that focused on different problems than this article does.

In basketball, each team has five players on court at all times. Typically, all five players attack or defend simultaneously so that all ten players are in one half of the court for several seconds, around 10 to 20 s, before moving into the other half. The basketball court has a rectangular shape with a width (x axis) of 94 ft and a length (y axis) of 50 ft. Let us install a coordinate system whose origin is at the mid-point of both axes. If a player is sitting on the bench and not actively participating in the game, the coordinate $(-48.5,-27.5)$ is assigned to him/her, which is slightly outside of the court, but these time frames were excluded from the analysis.

Figure 6 shows the distribution of the positions of Cleveland player LeBron James depending on whether he is attacking or defending and his position on the court over time during the first half of play. One can see that during the attack, James can mostly be found around the three-point-line and, occasionally, closer to the basket, including often at the edge of the small rectangular area around the basket. On defence, he is typically positioned slightly to the left or right of the basket.

Applying the Dependency Analysis to the Basketball Data

The dependency analysis on player coordinate data during the first half of the game is now performed, considering only the ten players in the starting lineups of the teams. For the representation of each two-dimensional position of a player, instead of solving (SPA 1), landmark points are chosen in advance, and (SPA 1) is solved only for $\Gamma$, i.e., solely the barycentric coordinates of player coordinates with respect to the landmark points were computed. For the clarity of both visualisation and numerical computations, only the absolute value of the x-coordinates were considered, meaning that the coordinates are reflected along the half-court line. For this reason, only the right half of the court has to be taken into account, and the following landmark points were used:

$$\Sigma =\left[\begin{array}{ccccccc}48.5& 48.5& 0& 0& 20& 20& 40\\ -27.5& 27.5& -27.5& 27.5& 15& -15& 0\end{array}\right]$$

(28)

so that $K_X=K_Y=7$. The landmark points are depicted in Figure 7.

To measure dependencies between each two players, those points in time during the game were used at which both players were on the court, and ${\Lambda }_{XY}^{(\tau )}$ was computed for each pair of players $X,Y$. The time lag was chosen to be $\tau =1\mathrm{s}$, so that the influence of the position of a player X for the position of a player Y one second later was investigated.

Note that, as mentioned, basketball games are frequently interrupted for various reason such as fouls or time-outs. This was taken into account by defining an event as the part of play between interruptions and denoting the number of events considered by L. Then, the training data were constructed in the form of multiple short time series, i.e., by storing coordinates from the kth event, of length $T_k$ seconds, as ${\Gamma }_k^X=[{\gamma }_{k,1}^X,\dots ,{\gamma }_{k,T_k-\tau }^X],{\Gamma }_k^Y=[{\gamma }_{k,1+\tau }^Y,\dots ,{\gamma }_{k,T_k}^Y]$, and for ${\Lambda }_{XY}$ minimising $\parallel [{\Gamma }_1^Y,\dots ,{\Gamma }_L^Y]-\Lambda [{\Gamma }_1^X,\dots ,{\Gamma }_L^X]{\parallel }_F$ (vice versa for ${\Lambda }_{YX}$).

Furthermore, it was distinguished between which team was attacking since the decisions of players should be strongly influenced by whether they are attacking or defending. Let us therefore define three different scenarios:

$$\begin{array}{cc}\hfill \mathrm{Dallas}\mathrm{attacking}:& \iff \mathrm{at}\mathrm{least}\mathrm{nine}\mathrm{players}\mathrm{in}\mathrm{Cleveland}’\mathrm{s}\mathrm{half}\hfill \\ \hfill \mathrm{Cleveland}\mathrm{attacking}:& \iff \mathrm{at}\mathrm{least}\mathrm{nine}\mathrm{players}\mathrm{in}\mathrm{Dallas}’\mathrm{half}\hfill \\ \hfill \mathrm{Transition}:& \iff \mathrm{otherwise}\hfill \end{array}$$

The analysis on the transition phase is omitted since play is typically rather unstructured during this phase.

The full results of the dependency analysis are, for the sake of visual clarity, shown only in Appendix A.5. Here, in the main text, selected parts are shown, restricted to the average row variance. One can make the following observations:

• As

  Table 3. Dallas attacking: average row variance multiplied by 100 for ${\Lambda }_{XY}$ for X from attacking players (Dallas) and Y from defending players (Cleveland).

  From ↓ to →	Irving (PG)	Smith (SG)	James (SF)	Love (PF)	Thompson (C)
  Williams (PG)	2.73	1.15	1.1	0.48	0.75
  Matthews (SG)	0.56	2.49	1.04	0.86	0.27
  Parsons (SF)	1.25	1.83	2.45	0.57	1.21
  Nowitzki (PF)	1.35	0.83	1.01	1.29	0.97
  Pachulia (C)	1.2	1.77	1.21	1.09	1.02

  and

  Table 4. Cleveland attacking: average row variance multiplied by 100 for ${\Lambda }_{XY}$ for X from attacking players (Cleveland) and Y from defending players (Dallas).

  From ↓ to →	Williams (PG)	Matthews (SG)	Parsons (SF)	Nowitzki (PF)	Pachulia (C)
  Irving (PG)	2.48	1.05	0.9	1.53	0.35
  Smith (SG)	0.67	1.8	0.84	0.43	0.33
  James (SF)	1.16	0.29	1.29	0.81	0.38
  Love (PF)	0.63	0.79	0.74	1.02	0.24
  Thompson (C)	1.01	1.56	1.2	1.49	1.04

  show, according to the dependency measures, with either team attacking, offensive players seem to have the strong, often the strongest, influence on their direct opponent, i.e., the opposing player playing the same position (Point Guards (PGs) Williams and Irving; Shooting Guards (SGs) Matthews and Smith; Small Forwards (SFs) Parsons and James; Power Forwards (PFs) Nowitzki and Love). Exceptions are Centres (Cs) Pachulia and Thompson. This is intuitive, since in basketball, often times, each offensive player has one direct opponent that follows him.
• One can also see that, typically, the defending Point Guards and Shooting Guards seem to be more strongly influenced by multiple opposing players than the Power Forwards and Centres. The reason for this could be that PFs and Cs usually are more steadily positioned around the basket, while the Guards are more encouraged to actively chase opposing attackers.
  One can see that when Dallas attack, they have a much stronger influence on Cleveland than vice versa, according to the average row variance. The Schatten-1 norm does not strongly emphasise that. When Cleveland attack, the cumulated dependencies are very similar to each other. This can be checked using the full results’ data in Appendix A.5.

  Table 5. Dallas attacking: relative differences of average row variance for ${\Lambda }_{XY}$ for X from attacking players (Dallas) and Y from defending players (Cleveland).

  From ↓ to →	Irving (PG)	Smith (SG)	James (SF)	Love (PF)	Thompson (C)
  Williams (PG)	0.11	0.17	−0.27	−0.42	−0.47
  Matthews (SG)	−0.08	0.4	0.29	−0.38	−0.46
  Parsons (SF)	−0.12	0.34	0.1	−0.28	−0.37
  Nowitzki (PF)	0.82	0.59	0.72	0.26	0.3
  Pachulia (C)	0.55	0.62	0.57	0.16	0.21

  also indicates this, as it shows that the relative differences are mostly positive for Dallas’ players when Dallas attack.
• When Cleveland attack, Thompson has large positive relative differences over most other players, except for Pachulia (

  Table 6. Cleveland attacking: relative differences in the average row variance for ${\Lambda }_{XY}$ for X from attacking players (Cleveland) and Y from defending players (Dallas).

  From ↓ to →	Williams (PG)	Matthews (SG)	Parsons (SF)	Nowitzki (PF)	Pachulia (C)
  Irving (PG)	−0.08	−0.18	−0.31	−0. 21	−0.46
  Smith (SG)	−0.36	−0.15	−0.03	−0.16	−0.55
  James (SF)	0.02	−0.55	−0.3	0.07	−0.27
  Love (PF)	0.47	0.61	0.17	−0.43	−0.37
  Thompson (C)	0.69	0.61	0.38	0.5	0.13

  ). This could be explained by the fact that Thompson plays below the basket, giving him a lower position radius, and his position is less dependent on spontaneous or set-up moves by his teammates. Pachulia is his direct opponent from whom he might try to separate himself so that his position is in fact influenced by Pachulia.
• One can check if the players of the depending team orient themselves at the positions of the attacking players. To see this, let us sum over all dependency values between players from opposing teams, i.e., compute ${\sum }_{X\mathrm{in}\mathrm{team}1,Y\mathrm{in}\mathrm{team}2}{M}_{XY}$ for both dependency measures (see

  Table 7. Cumulated sum over the dependency measures considering only one team attacking, the other defending.

  Attacking Team	Measure	X in DAL, Y in CLE	X in CLE, Y in DAL
  Dallas	${\parallel ·\parallel }_1$	54.4	52.2
  Dallas	$\nu$	0.30	0.25
  Cleveland	${\parallel ·\parallel }_1$	51.4	51.3
  Cleveland	$\nu$	0.25	0.24

  ).

Due to inherent randomness in a sport such as basketball, one must be cautious not to overstate these results. This example is meant to showcase how the method presented in this article can be applied to a complex real-world example. The obtained results, however, seem plausible for the explained reasons. In the future, it would be interesting to see if the method could be tailored more specifically to the basketball context and be used to actually find so-far hidden structures of play.

5. Conclusions

In this article, a data-driven method for the quantification of influences between variables of a dynamical system was presented. The method deploys the low-cost discretisation algorithm Scalable Probabilistic Approximation (SPA), which represents the data points using fuzzy affiliations, respectively barycentric coordinates with respect to a convex polytope, and estimates a linear mapping between these representations of two variables and forward in time. Two dependency measures were introduced that compute the properties of the mapping and admit a suitable interpretation.

Clearly, many methods for the same aim already exist. However, most of them are suited for specific scenarios and impose assumptions on the relations or the dynamics, which are not always fulfilled. Hence, this method should be a helpful and directly applicable extension to the landscape of already existing methods for the detection of influences.

The advantages of the method lie in the following: it is purely data-driven and, due to the very general structure of the SPA model, requires almost no intuition about the relation between variables and the underlying dynamics. This is in contrast to a method such as Granger causality, which imposes a specific autoregressive model between the dynamics. Furthermore, the presented method is almost parameter-free, since only the numbers of landmark points $K_X$ and $K_Y$ for the representation of points and the time lag $\tau$ have to be specified. Additionally, the dependency measures, the Schatten-1 norm and average row variance, are straightforward to compute from the linear mapping and offer direct interpretation. The capacity of the method to reconstruct influences was demonstrated on multiple examples, including stochastic and memory-exhibiting dynamics.

In the future, it could be worthwhile to find rules for the optimal number of landmark points and their exact positions with respect to the data. Plus, it seems important to investigate how dependencies between variables with differing numbers of landmark points can be compared with the presented dependency measures. Moreover, one could determine additional properties of the matrix ${\Lambda }_{XY}$ next to the two presented ones. It should also be investigated why in some of the presented examples, the average row variance gave a clearer reconstruction of influences than the Schatten-1 norm and how the latter can be improved. In addition, it should be worthwhile to use the combined solution of the (SPA 1) and (SPA 2) problems as done in [23] and observe whether this improves the performance. Furthermore, constructing a nonlinear SPA model consisting of the multiplication of a linear mapping with a nonlinear function, as done in [26], could give an improved model accuracy and, therefore, a more reliable quantification of influences. Lastly, since in this article, variables were always expressed using a higher number of landmark points, it should be interesting to investigate whether for high-dimensional variables, projecting them to a low-dimensional representation using SPA and performing the same dependency analysis is still sensible. This could be of practical help to shed light on the interplay between variables in high dimensions.