Measuring Statistical Dependence via Characteristic Function IPM

Abstract

We study statistical dependence in the frequency domain using the integral probability metric (IPM) framework. We propose the uniform Fourier dependence measure (UFDM) defined as the uniform norm of the difference between the joint and product-marginal characteristic functions. We provide a theoretical analysis, highlighting key properties, such as invariances, monotonicity in linear dimension reduction, and a concentration bound. For the estimation of the UFDM, we propose a gradient-based algorithm with singular value decomposition (SVD) warm-up and show that this warm-up is essential for stable performance. The empirical estimator of UFDM is differentiable, and it can be integrated into modern machine learning pipelines. In experiments with synthetic and real-world data, we compare UFDM with distance correlation (DCOR), Hilbert–Schmidt independence criterion (HSIC), and matrix-based Rényi’s $\alpha$-entropy functional (MEF) in permutation-based statistical independence testing and supervised feature extraction. Independence test experiments showed the effectiveness of UFDM at detecting some sparse geometric dependencies in a diverse set of patterns that span different linear and nonlinear interactions, including copulas and geometric structures. In feature extraction experiments across 16 OpenML datasets, we conducted 160 pairwise comparisons: UFDM statistically significantly outperformed other baselines in 20 cases and was outperformed in 13.

1. Introduction

The estimation of statistical dependence plays an important role in various statistical and machine learning methods (e.g., hypothesis testing [1], feature selection and extraction [2,3], causal inference [4], self-supervised learning [5], representation learning [6], interpretation of neural models [7], among others). In recent years, various authors (e.g.,  [1,8,9,10,11,12,13,14]) have suggested different approaches to measuring statistical dependence.

In this paper, we focus on the estimation of statistical dependence using characteristic functions (CFs) and integral probability metric (IPM) framework. We propose and investigate a novel IPM-based statistical dependence measure, defined as the uniform norm of the difference between the joint and product-marginal CFs. After introducing core concepts, we conduct a short review of the previous work (Section 2). In Section 3, we formulate the proposed measure and its empirical estimator and perform their theoretical analysis. Section 4 is devoted to empirical investigation. Finally, in Section 5 we discuss results, limitations, and future work. Appendix A contains technical details, such as mathematical proofs, and auxiliary tables. The main contributions of this paper are the following:

• Theoretical and methodological contributions. We propose a new IPM-based statistical dependence measure (UFDM) and derive its properties. The main theoretical result of this paper is the structural characterisation of UFDM, which includes invariance under linear transformations and augmentation with independent noise, monotonicity under linear dimension reduction, vanishing under independence, and a concentration bound for its empirical estimator. We additionally propose a gradient-based estimation algorithm with an SVD warm-up to ensure numerical stability.
• Empirical analysis. We conduct an empirical study demonstrating the practical effectiveness of UFDM in permutation-based independence testing across diverse linear, nonlinear, and geometrically structured patterns, as well as in supervised feature-extraction tasks on real datasets.

In addition, we provide the accompanying code repository https://github.com/povidanius/UFDM (accessed on 4 December 2025).

1.1. IPM Framework

In the context of estimation of statistical dependence, the IPM is a class of metrics between two probability distributions $P_{X,Y}$ and $P_X{P}_Y$, defined for a function class $\mathcal{F}$:

$$\mathrm{IPM}(P_{X,Y},P_X{P}_Y|\mathcal{F})=\underset{f\in \mathcal{F}}{sup}|{\mathbb{E}}_{U}f\left(U\right)-{\mathbb{E}}_{V}f\left(V\right)|,$$

(1)

where $U\sim P_{X,Y}$, and $V\sim P_X{P}_Y$ [15].

1.2. Characteristic Functions

Let $X\in {\mathbb{R}}^{d_X}$, $Y\in {\mathbb{R}}^{d_Y}$, and ${(X^T,Y^T)}^T\in {\mathbb{R}}^{d_X+d_Y}$ be random vectors defined on a common probability space $(\Omega ,\mathcal{F},\mathbb{P})$. Let us recall that their characteristic functions are given by

$$\varphi \left(\alpha \right):={\mathbb{E}}_X{e}^{i{\alpha }^{T}X},\varphi \left(\beta \right):={\mathbb{E}}_Y{e}^{i{\beta }^{T}Y},\mathrm{and}\varphi (\alpha ,\beta ):={\mathbb{E}}_{X,Y}{e}^{i({\alpha }^{T}X+{\beta }^{T}Y)},$$

(2)

where $i^2=-1$, $\alpha \in {\mathbb{R}}^{d_X}$, and $\beta \in {\mathbb{R}}^{d_Y}$. Having n i.i.d. realisations ${(x_i,y_i)}_{i=1}^n$, the corresponding empirical characteristic functions (ECFs) are given by

$${\varphi }_n\left(\alpha \right):={\displaystyle \frac{1}{n}}\sum _{j=1}^n{e}^{i{\alpha }^T{x}_j},{\varphi }_n\left(\beta \right):={\displaystyle \frac{1}{n}}\sum _{j=1}^n{e}^{i{\beta }^T{y}_j},\mathrm{and}{\varphi }_n(\alpha ,\beta ):={\displaystyle \frac{1}{n}}\sum _{j=1}^n{e}^{i({\alpha }^T{x}_j+{\beta }^T{y}_j)}.$$

(3)

The uniqueness theorem states that X and Y have the same distribution if and only if their CFs are identical [16]. Therefore, CFs can be considered a description of a distribution. Alternatively, a CF $\varphi$ can be represented as a real vector $(\Re \varphi ,\Im \varphi )\in {\mathbb{R}}^2$, where ℜ and ℑ denote real and imaginary components [17]. This viewpoint avoids explicit reliance on the imaginary unit i and makes the geometric structure of CFs more transparent.

For convenience, let us define $\gamma ={({\alpha }^T,{\beta }^T)}^T$, $\psi \left(\gamma \right)=\varphi \left(\alpha \right)\varphi \left(\beta \right)$ and let ${\psi }_n\left(\gamma \right)={\psi }_n(\alpha ,\beta )$ be its empirical counterpart. In our study, we will utilise IPM framework for investigation of the statistical dependence via

$$\Delta \left(\gamma \right)=\varphi \left(\gamma \right)-\psi \left(\gamma \right)$$

(4)

and its empirical counterpart

$${\Delta }_n\left(\gamma \right)={\varphi }_n\left(\gamma \right)-{\psi }_n\left(\gamma \right).$$

(5)

2. Previous Work

Various theoretical instruments have been employed for statistical dependence estimation. For example, weighted $L^2$ spaces and CFs (e.g., distance correlation, [13]), reproducing kernel Hilbert spaces (RKHS) (HSIC [1], DIME [18]), information theory (mutual information [19], and generalisations such as MEF [20,21]) and copula theory ([10,22]), among others. Since our work is rooted in the CF-based line of research and IPM framework, and it is empirically evaluated for independence testing and representation learning, let us consider DCOR, HSIC, and MEF, because these three measures form the compact set of high-performing baselines that span CFs, IPMs, and information-theoretic methods, which are widely used in representation learning tasks.

• Distance correlation. DCOR [13] is defined as

$$\mathrm{DCOR}(X,Y)={\displaystyle \frac{\mathrm{DCOV}(X,Y)}{\sqrt{\mathrm{DCOV}(X,X)\mathrm{DCOV}(Y,Y)}}},$$

where the distance covariance (DCOV) is given by

$${\mathrm{DCOV}}^2(X,Y)={\int }_{{\mathbb{R}}^{d_X+d_Y}}{|\Delta \left(\gamma \right)|}^{2}w\left(\gamma \right)d\gamma ,$$

(6)

with weighting function $w\left(\gamma \right)=w(\alpha ,\beta )=(c_{d_X}{c}_{d_Y}{\left|\right|\alpha \left|\right|}^{1+d_X}{\left|\right|\beta \left|\right|}^{1+d_Y}{)}^{-1}$, where $c_{d_X}={\pi }^{(1+d_X)/2}/\Gamma ((1+d_X)/2)$, and $c_{d_Y}={\pi }^{(1+d_Y)/2}/\Gamma ((1+d_Y)/2)$, and $\Gamma (.)$ is the gamma function. This weighting function allows one to avoid the direct estimation of the integral, expressing it in terms of the covariance of distances between data points [13]. The later result of [23] generalises the distance correlation to multiple random vectors. Given the i.i.d. sample pairs $(x_i,y_i)$, $i=1,\dots ,n$, the empirical unbiased estimator of the squared distance covariances [24] is defined as

$${\mathrm{DCOV}}_n^2(X,Y)={\displaystyle \frac{1}{n(n-3)}}\sum _{i\ne j}{A}_{ij}{B}_{ij},$$

(7)

where matrices $A=\left(A_{ij}\right)$, $B=\left(B_{ij}\right)$ are given by

$$A_{ij}=a_{ij}-{\displaystyle \frac{1}{n-2}}\sum _{k=1}^n{a}_{ik}-{\displaystyle \frac{1}{n-2}}\sum _{k=1}^n{a}_{kj}+{\displaystyle \frac{1}{(n-1)(n-2)}}\sum _{k,\ell =1}^n{a}_{k\ell },$$

with Euclidean distance $a_{ij}=\parallel x_i-x_j\parallel$. The matrix B is defined analogously using distances $b_{ij}=\parallel y_i-y_j\parallel$. The empirical DCOR is then obtained as follows:

$${\mathrm{DCOR}}_n(X,Y)={\displaystyle \frac{{\mathrm{DCOV}}_n(X,Y)}{\sqrt{{\mathrm{DCOV}}_n(X,X){\mathrm{DCOV}}_n(Y,Y)}}}.$$

Note that the biased version of the empirical distance-based estimator Equation (7) is equivalent to the ECF-based estimator of Equation (6) (Theorem 1, [13]). While consistency is established for the biased estimator under the moment condition $\mathbb{E}(\parallel X\parallel +\parallel Y\parallel )<\infty$ (Theorem 2, [13]), the unbiased estimator Equation (7) differs only by a finite-sample correction and converges to the same population quantity Equation (6)  [24], implying consistency under the same moment condition.

• HSIC. For reproducing kernel Hilbert spaces (RKHS) $\mathcal{F}$ and $\mathcal{G}$ with kernels k and l, it is defined as

$$\mathrm{HSIC}(X,Y)=\parallel {\mathbb{E}}_{XY}k(X,·)\otimes l(Y,·)-{\mathbb{E}}_{X}k(X,·)\otimes {\mathbb{E}}_Y{l(Y,·)\parallel }_{\mathrm{HS}}^2,$$

where ${\parallel ·\parallel }_{\mathrm{HS}}$ denotes the Hilbert–Schmidt norm, and ⊗ is the tensor product [1]. Taking a product kernel $\kappa ((x,y),(x^{\prime },y^{\prime }))=k(x,x^{\prime })l(y,y^{\prime })$, HSIC is equal to the squared maximum mean discrepancy, which is an instance of an IPM with function class $\mathcal{F}=\{f:|\left|f\right|{|}_{H_{\kappa }}\le 1\}$, where $H_{\kappa }$ is RKHS generated by $\kappa$ [25]. Having a sample of paired n i.i.d. observations, the empirical estimator is

$${\mathrm{HSIC}}_n(X,Y)={\displaystyle \frac{1}{{(n-1)}^2}}\mathrm{tr}\left(KHLH\right)$$

with kernel matrices $K_{ij}=k(x_i,x_j)$, $L_{ij}=l(y_i,y_j)$, and centering matrix $H=I-{\displaystyle \frac{1}{n}}\mathbf{1}{\mathbf{1}}^T$. When both kernels k and l are translation-invariant (i.e., $k(x,x^{\prime })=k_0(x-x^{\prime })$ on ${\mathbb{R}}^{d_X}$ and $l(y,y^{\prime })=l_0(y-y^{\prime })$ on ${\mathbb{R}}^{d_Y}$, with $k_0,l_0$ positive definite functions such as the Gaussian $k_0\left(v\right)={exp(-\parallel v\parallel }^2/\left(2{\sigma }^2\right))$ with $\sigma >0$), the product kernel $\kappa ((x,y),(x^{\prime },y^{\prime }))=k(x,x^{\prime })l(y,y^{\prime })=k_0(x-x^{\prime })l_0(y-y^{\prime })$ is also translation-invariant on ${\mathbb{R}}^{d_X+d_Y}$. In this case, $\kappa (u,v)={\kappa }_0(u-v)$ for some positive definite function ${\kappa }_0$ on ${\mathbb{R}}^{d_X+d_Y}$, and HSIC can be expressed in the frequency domain as

$$\mathrm{HSIC}(X,Y)={\int }_{{\mathbb{R}}^{d_X+d_Y}}{|\Delta \left(\gamma \right)|}^2{F}^{-1}{\kappa }_0\left(\gamma \right)d\gamma ,$$

(8)

where $\gamma ={({\alpha }^T,{\beta }^T)}^T$, and $F^{-1}{\kappa }_0$ denotes the inverse Fourier transform of ${\kappa }_0$. Therefore, for translation-invariant kernels, HSIC is structurally analogous to distance covariance, since it also corresponds to the squared $L^2$ norm of $\Delta$ (Equation (4)), with weighting determined by $\kappa$.

• MEF. Shannon mutual information is defined by $\mathrm{MI}(X,Y)={\mathbb{E}}_{X,Y}log{\displaystyle \frac{p(X,Y)}{p\left(X\right)p\left(Y\right)}}$ [19]. The neural estimation of mutual information (MINE, [26]) uses its variational (Donsker–Varadhan) representation $\mathrm{MI}(X,Y)\approx ma{x}_{\theta }{\mathbb{E}}_{X,Y}f(x,y|\theta )-log\left({\mathbb{E}}_X{\mathbb{E}}_Y{e}^{f(x,y|\theta )}\right)$, since it allows avoiding density estimation (here $f(x,y|\theta )$ is a neural network with parameters $\theta$). In this case, the optimisation is performed over the space of neural network parameters, which often leads to unstable training and biased estimates due to the unboundedness of the objective and the difficulty of balancing the exponential term. The matrix-based Rényi’s $\alpha$-order entropy functional (MEF) [20,21,27] provides a kernel version of mutual information that avoids both density estimation and neural optimization. For random variables X and Y with distributions $P_X$, $P_Y$, and $P_{XY}$, it is defined as

$${\mathrm{MEF}}_{\alpha }(X,Y)={\mathbf{S}}_{\alpha }\left(P_X\right)+{\mathbf{S}}_{\alpha }\left(P_Y\right)-{\mathbf{S}}_{\alpha }\left(P_{XY}\right),$$

(9)

where ${\mathbf{S}}_{\alpha }\left(P_X\right)={\displaystyle \frac{1}{1-\alpha }}{log}_2\left(\mathrm{tr}\left(T_X^{\alpha }\right)\right)$ and $T_X$ is the normalised kernel integral operator on $L^2\left(P_X\right)$ [27]. Given i.i.d. samples ${\left\{(x_i,y_i)\right\}}_{i=1}^n$ with Gram matrices $K_{ij}=k(x_i,x_j)$ and $L_{ij}=l(y_i,y_j)$, the empirical estimator is

$${\mathrm{MEF}}_{\alpha ,n}(X,Y)={\mathbf{S}}_{\alpha ,n}\left(\frac{K}{\mathrm{tr}\left(K\right)}\right)+{\mathbf{S}}_{\alpha ,n}\left(\frac{L}{\mathrm{tr}\left(L\right)}\right)-{\mathbf{S}}_{\alpha ,n}\left(\frac{K\odot L}{\mathrm{tr}(K\odot L)}\right),$$

(10)

where ⊙ denotes the element-wise product, ${\mathbf{S}}_{\alpha ,n}\left(A\right)={\displaystyle \frac{1}{1-\alpha }}{log}_2\left({\sum }_i{\lambda }_i{\left(A\right)}^{\alpha }\right)$, and ${\lambda }_i$ are eigenvalues of $n\times n$ matrix A.

Motivation

The motivation of our work stems from the theoretical observation that applying the $L^{\infty }$ norm to $\Delta$ Equation (4) yields a novel, structurally simple IPM with some advantageous properties, such as the ability to detect arbitrary statistical dependencies, invariance under full-rank linear transformations and coordinate augmentation with independent noise, and monotonicity under linear dimension reduction (Theorem 1).

Since the $L^{\infty }$ norm isolates the most informative frequencies where dependence concentrates, we hypothesise that its empirical estimator could extract important structure from $\Delta$ that may be diluted by weighted $L^2$ or other global approaches such as DCOV, HSIC, and MEF.

3. Proposed Measure

Given two random vectors X and Y of dimensions $d_X$ and $d_Y$, and assuming possibly unknown joint distribution $P_{X,Y}$, we define our measure via IPM with function class $\mathcal{F}=\{f:f\left(z\right)=e^{i{\gamma }^{T}z};\gamma ,z\in {\mathbb{R}}^{d_X+d_Y},i^2=-1\}$, which corresponds to the following.

Definition 1.

Uniform Fourier Dependence Measure.

$$\mathrm{UFDM}(X,Y)={\left|\right|\Delta \left|\right|}_{L_{\infty }}=\underset{\gamma }{sup}|\Delta \left(\gamma \right)|.$$

(11)

Since CF is a Fourier transform of a probability distribution, and the norm in $L^{\infty }$ is called a uniform norm, we refer to it as Uniform Fourier Dependence Measure (UFDM).

Theorem 1.

$\mathrm{UFDM}$ has the following properties:

• $0\le \mathrm{UFDM}(X,Y)\le 1$.
• $\mathrm{UFDM}(X,Y)=\mathrm{UFDM}(Y,X)$.
• $\mathrm{UFDM}(X,Y)=0$ if and only if $X\perp Y$ (⊥ denotes statistical independence).
• For Gaussian random vectors $X\sim N(0,{\Sigma }_X)$, $Y\sim N(0,{\Sigma }_Y)$ with cross-covariance matrix ${\Sigma }_{X,Y}$ we have $\mathrm{UFDM}(X,Y)={sup}_{\alpha ,\beta }{e}^{-{\displaystyle \frac{1}{2}}({\alpha }^T{\Sigma }_X\alpha +{\beta }^T{\Sigma }_Y\beta )}|e^{-{\alpha }^T{\Sigma }_{X,Y}\beta }-1|$.
• Invariance under full-rank linear transformation: $\mathrm{UFDM}(AX+a,BY+b)=\mathrm{UFDM}(X,Y)$ for any full-rank matrices $A\in {\mathbb{R}}^{d_X\times d_X}$, $B\in {\mathbb{R}}^{d_Y\times d_Y}$ and vectors $a\in {\mathbb{R}}^{d_X}$, $b\in {\mathbb{R}}^{d_Y}$.
• Linear dimension reduction does not increase $\mathrm{UFDM}(X,Y)$.
• If $X\perp \mathcal{E}$, for any continuous function $f:{\mathbb{R}}^{d_X}\to {\mathbb{R}}^{d_Y}$, ${lim}_{\lambda \to \infty }\mathrm{UFDM}(X,f\left(X\right)+\lambda \mathcal{E})=0$, if $\mathcal{E}$ has a density.
• If X and Y have densities, then $\mathrm{UFDM}(X,Y)\le min\{1,\sqrt{2\mathrm{MI}(X,Y)}\}$, where $\mathrm{MI}(X,Y)$ is mutual information.
• Invariance to augmentation with independent noise: let $X,Y,Z$ be random vectors such that $Z\perp {(X^{\top },Y^{\top })}^{\top }$. Then $\mathrm{UFDM}\left({(X^{\top },Z^{\top })}^{\top },Y\right)=\mathrm{UFDM}(X,Y)$.

Proof. 

See Appendix A.1.    □

• Interpretation of UFDM via canonical correlation analysis (CCA). In the Gaussian case, the UFDM objective reduces analytically to CCA via a closed-form expression (Theorem 1, Property 4): after whitening (setting $u={\Sigma }_X^{1/2}\alpha$ and $v={\Sigma }_Y^{1/2}\beta$), it becomes ${max}_{u,v}{e}^{-{\displaystyle \frac{1}{2}}{\left(\right|u|}^2+{\left|v\right|}^2)}(1-e^{-u^{\top }Kv})$, where $K={\Sigma }_X^{-1/2}{\Sigma }_{XY}{\Sigma }_Y^{-1/2}$. By von Neumann’s inequality, the maximizers $(u,v)$ align with the leading singular vectors of K, corresponding to the top CCA pair. Note that since Gaussian independence is equivalent to the vanishing of the leading canonical correlation ${\rho }_1$ (as all remaining correlations $0\le {\rho }_j\le {\rho }_1$, $j>1$ must also vanish), UFDM’s focus on the leading canonical correlation entails no loss of discriminatory power.
• Interpretation of UFDM via cumulants. Let us recall that $\gamma ={({\alpha }^T,{\beta }^T)}^T$, $\varphi \left(\gamma \right)=\varphi (\alpha ,\beta )$, $\psi \left(\gamma \right)=\varphi \left(\alpha \right)\varphi \left(\beta \right)$. For general distributions, writing $\Delta \left(\gamma \right)=\psi \left(\gamma \right)(exp(C\left(\gamma \right))-1)$ offers a cumulant-series factorization, with $C\left(\gamma \right)=log{\displaystyle \frac{\varphi \left(\gamma \right)}{\psi \left(\gamma \right)}}={\sum }_{p,q\ge 1}{\displaystyle \frac{i^{p+q}}{p!q!}}⟨{\kappa }_{p,q},{\alpha }^{\otimes p}\otimes {\beta }^{\otimes q}⟩$, where ${\kappa }_{p,q}$ are cross-cumulants and ${\alpha }^{\otimes p}\otimes {\beta }^{\otimes q}$ are the $(p+q)$-order tensors formed by the tensor product of p copies of $\alpha$ and q copies of $\beta$. The leading term, corresponding to $p=q=1$, is ${\displaystyle \frac{i^2}{1!,1!}}⟨{\kappa }_{1,1},\alpha \otimes \beta ⟩=-{\alpha }^{\top }{\Sigma }_{XY}\beta$ (with ${\kappa }_{1,1}={\Sigma }_{XY}$ for centered variables), which aligns with the CCA interpretation, while higher-order ${\kappa }_{p,q}$ terms capture non-Gaussian deviations, interpreting UFDM as a frequency-domain approach that aligns $(\alpha ,\beta )$ with cross-cumulant directions under marginal damping by $\psi \left(\gamma \right)$.
• Remark on the representations of CFs. Since $\mathrm{UFDM}(X,Y)={sup}_{\gamma }{\parallel (\Re \Delta \left(\gamma \right),\Im \Delta \left(\gamma \right))\parallel }_2$, the UFDM objective naturally operates on the real two-dimensional vector formed by the real and imaginary parts of $\Delta \left(\gamma \right)$. This aligns with recent work on real-vector representations of characteristic functions [17] and shows that UFDM does not rely on any special algebraic role of the imaginary unit.

3.1. Estimation

Having i.i.d. observations $(X^n,Y^n)=(x_j,y_j)\sim P_{X,Y}$, $j=1,2,\dots ,n$, we define and discuss empirical estimators of UFDM. Recall that (Section 1.2) that $\gamma ={({\alpha }^T,{\beta }^T)}^T$ and let $\varphi \left(\alpha \right),\varphi \left(\beta \right)$, and $\varphi \left(\gamma \right)$ be CFs of X, Y, and $(X,Y)$, respectively ($\alpha \in {\mathbb{R}}^{d_X}$, $\beta \in {\mathbb{R}}^{d_Y}$, and $\gamma \in {\mathbb{R}}^{d_X+d_Y}$). Let us also denote norms ${\left|\right|f\left|\right|}_{L_{\infty }}^t={sup}_{\left|\right|\tau \left|\right|<t}\left|f\left(\tau \right)\right|$, ${\left|\right|f\left|\right|}_{L_{\infty }}={sup}_{\tau }\left|f\left(\tau \right)\right|$, for $t>0$ and multivariate $\tau$.

• Empirical estimator. Let us define the empirical estimator of $\mathrm{UFDM}$ for a fixed $t>0$:

$${\mathrm{UFDM}}_n^t(X^n,Y^n)=\left|\right|{\Delta }_n{\left|\right|}_{L_{\infty }}^t.$$

(12)

3.2. Estimator Convergence

The ECF is a uniformly consistent estimator of CF in each bounded subset [28] (i.e., ${lim}_{n\to \infty }{sup}_{\left|\right|\gamma \left|\right|<t}|\varphi \left(\gamma \right)-{\varphi }_n\left(\gamma \right)|=0$ almost surely for any fixed $t>0$) [28]. By the triangle inequality, this implies the following:

Proposition 1.

For a fixed $t>0$, ${lim}_{n\to \infty }\left|\right|{\Delta }_n-\Delta {\left|\right|}_{L_{\infty }}^t=0$, almost surely.

Theorem 2

([29]). If $t_n\to \infty$ and ${\displaystyle \frac{log{t}_n}{n}}\to 0$, as $n\to \infty$, then ${lim}_{n\to \infty }{sup}_{\left|\right|\gamma \left|\right|<t_n}|\xi \left(\gamma \right)-{\xi }_n\left(\gamma \right)|=0$ almost surely for any $\mathrm{CF}$$\xi \left(\gamma \right)$ and corresponding $\mathrm{ECF}$${\xi }_n\left(\gamma \right)$.

• This implies the convergence of the empirical estimator Equation (12):

Proposition 2.

If $t_n\to \infty$ and ${\displaystyle \frac{log{t}_n}{n}}\to 0$, as $n\to \infty$, then ${lim}_{n\to \infty }\left|\right|{\Delta }_n{\left|\right|}_{L_{\infty }}^{t_n}=\mathrm{UFDM}(X,Y)$, almost surely.

Proof. 

See Appendix A.1.    □

Note that ECF does not converge to CF [28,29] uniformly in the entire space. Therefore, to ensure the convergence of the empirical estimator of UFDM, we need to bound the norm by slowly growing balls as in Theorem 2. The finite–sample analysis of the convergence of empirical UFDM Equation (12) to its truncated population counterpart (${\mathrm{UFDM}}^t(X,Y)={\left|\right|\Delta \left|\right|}_{L_{\infty }}^t$) yields the following concentration inequality.

Theorem 3.

Let us assume that ${\mathbb{E}\left|\right|X\left|\right|}^2<\infty$, ${\mathbb{E}\left|\right|Y\left|\right|}^2<\infty$. Let us define $d=d_X+d_Y$, $Z={(X^T,Y^T)}^T$, and $W=\left|\right|X\left|\right|+\left|\right|Y\left|\right|+\left|\right|Z\left|\right|$. Then there exists a constant C, such that for every fixed $\epsilon >\frac{1}{n}$, $t>0$:

$$Pr\left(|{\mathrm{UFDM}}_n^t(X^n,Y^n)-{\mathrm{UFDM}}^t(X,Y)|>\epsilon \right)\le 2{\left(\frac{Ct}{\epsilon }\right)}^{d}exp\left(-\frac{n}{18}{\left(\frac{\epsilon }{2}-\frac{1}{n}\right)}^2\right)+{\displaystyle \frac{{\sigma }^2}{n{L}^2}},$$

where $L=\mathbb{E}W$, and ${\sigma }^2=\mathbb{E}{(W-L)}^2$.

Proof. 

See Appendix A.2.    □

3.3. Estimator Computation

In practice, UFDM can be estimated iteratively using Algorithm 1. Since it depends on initial parameters $\alpha$ and $\beta$, the complementary Algorithm 2 is designed for their data-driven initialisation. According to our experience with UFDM applications, Algorithm 2 is very important, since without it we often encountered stability issues, and initially had to rely on various heuristics, such as parameter normalisation to the unit sphere. In our opinion, this is because ${\Delta }_n$ is a highly nonlinear optimisation surface (especially in larger dimensions), which complicates the finding of the corresponding maxima.

Algorithm 1 UFDM estimation

Require: Number of iterations N, batch size $n_b$, initial $\alpha \in {\mathbb{R}}^{d_X},\beta \in {\mathbb{R}}^{d_Y}$.     for $iteration=1$ to N do           Sample batch $(X^{n_b},Y^{n_b})={(x_i,y_i)}_{i=1}^{n_b}$.           Standardise $(X^{n_b},Y^{n_b})$ to zero mean and unit variance.           $\alpha ,\beta \leftarrow \mathrm{AdamW}([\alpha ,\beta ],-|{\Delta }_{n_b}(\alpha ,\beta )\left|\right)$.     end for     return $\Delta (\alpha ,\beta )$, $\alpha$, $\beta$

Algorithm 2 SVD warm-up

Require: Batch size $n_b$.     Sample batch $(X^{n_b},Y^{n_b})={(x_i,y_i)}_{i=1}^{n_b}$.     Compute cross-covariance $C={\left(X^{n_b}\right)}^{\top }{Y}^{n_b}/n_b$.     Decompose: $[U,\Sigma ,V^H]=\mathrm{SVD}\left(C\right)$.     $\alpha \leftarrow U_{:,1},\beta \leftarrow V_{1,:}^{H,\top }$.     return $\alpha$, $\beta$

The computational complexity of Algorithm 2 consists of cross-covariance computation and finding its SVD a complexity of $O\left(n_b{d}_X{d}_Y+d_X{d}_{Y}min(d_X,d_Y)\right)$. Having initialisation of $\alpha$ and $\beta$, the complexity of Algorithm 1 is $O\left(N{n}_b(d_X+d_Y)\right)$. Hence, the total computational complexity of the sequential application of Algorithm 2 and Algorithm 1 is $O\left(n_b{d}_X{d}_Y+d_X{d}_{Y}min(d_X,d_Y)+N{n}_b(d_X+d_Y)\right)$. Finally, having the optimal ${\alpha }^{*}$ and ${\beta }^{*}$ computed by Algorithm 1, the evaluation of empirical $\mathrm{UFDM}$ has computational complexity linear in sample size.

4. Experiments

For UFDM, we used SVD warm-up (Algorithm 2) for parameter initialisation and fixed truncation parameter t to $25.0$. For kernel measures, HSIC and MEF, we used Gaussian kernels for both X and Y, with a bandwidth selected using median heuristics [30]. For MEF measure $\alpha$ was set to $1.01$, as in [21].

4.1. Permutation Tests

• Permutation tests with UFDM. We compared $\mathrm{UFDM}$, DCOR, HSIC, and MEF in permutation-based statistical independence testing ($H_0:X\perp Y$ versus the alternative $H_1:X\cancel{\perp }Y$) using a set of multivariate distributions. We investigated scenarios with a sample size of $n=750$ and data dimensions $d\in \{5,15,25\}$ ($d_X=d_Y=d$). To ensure valid finite-sample calibration, permutation p-values were computed with the Phipson–Smyth correction [31].
• Hyperparameters. We used 500 permutations per p-value. The number of iterations in UFDM estimation Algorithm 1 was set to 100. The batch size equaled the sample size ($n=750$). We used a learning rate of $0.025$. Due to the high computation time (permutation tests took $\approx 6.3$ days on five machines with Intel i7 CPU, 16GB of RAM, and Nvidia GeForce RTX 2060 12 GB GPU), we relied on 500 p-values for each test in the $H_0$ scenario and on 100 p-values for each test in the $H_1$ scenario.
• Distributions analysed. In the $H_0$ case, X was sampled from multivariate uniform, Gaussian, and Student $t\left(3\right)$ distributions (corresponding to no-tail, light-tail, and heavy-tail scenarios, respectively), and Y was independently sampled from the same set of distributions. Afterwards, we examined the uniformity of the p-values obtained from permutation tests using different statistical measures, through QQ-plots and Kolmogorov–Smirnov (KS) tests.

In the $H_1$ case, X and Y were related through statistical dependencies described in Table A2. These dependencies include structured dependence patterns, where X was sampled from the same set of distributions (multivariate uniform, Gaussian, and Student $t\left(3\right)$), and Y was generated as $Y=f\left(X\right)+0.1ϵ$, with $ϵ$ denoting additive Gaussian noise independent of X. We also examined more complex dependencies (Table A2), where the relationship between X and Y was modeled using copulas, bimodal, circular, and other nonlinear patterns. Using this setup, we evaluated the empirical power of the permutation tests based on the same collection of statistical measures.

• Results for $H_0$. As shown in Figure 1, UFDM, DCOR, HSIC, and MEF exhibited approximately uniform permutation p-values across all distribution pairs and dimensions, with empirical false rejection rates (FRR) remaining close to the nominal $0.05$ level. Isolated low KS p-values below $0.05$ occurred in only two cases: one for MEF in the Gaussian/Gaussian pair at dimension 5 (p-value of $0.01$) and one for UFDM in the Gaussian/Student-t pair at dimension 5 (p-value of $0.03$), suggesting minor sampling variability rather than systematic deviations from uniformity. These results show that UFDM remained comparably stable to DCOR, HSIC and MEF, in terms of type-I error control under $H_0$.
  

  Figure 1. Empirical QQ-plots of p-values under $H_0$. The dashed vertical line corresponds to the nominal significance level $0.05$. The empirical FRR and its Wilson confidence interval, p-values of KS test are reported in the legend.
• Results for $H_1$. The empirical power and its $0.95$-Wilson confidence intervals (CIs) are presented in Table 1 and Table 2. These results show that, in most cases, the empirical power of UFDM, DCOR, HSIC, and MEF was approximately equal to $1.00$. However, Table 2 also reveals that for the sparse Circular and Interleaved Moons patterns ($d\ge 15$), MEF exhibited a noticeable decrease in empirical power. We conjecture that this reduction may stem from MEF’s comparatively higher sensitivity to kernel bandwidth selection in these specific, geometrically structured patterns. On the other hand, UFDM’s robustness in these settings may also be explained by its invariance to augmentation with independent noise (Theorem 1, Property 9), which helps to preserve the detectability of sparse geometric dependencies embedded within high-dimensional noise coordinates.
  Table 1. Empirical power and Wilson CIs for the dependent data (structured dependence patterns) at $\alpha =0.05$.

  Distribution of Y	UFDM	DCOR	HSIC	MEF
  Linear (1.0)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	$\mathit{d}=5$	$X\sim U{[0,1]}^d$
  Linear (0.3)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Logarithmic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Quadratic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Polynomial	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  LRSO (0.05)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Heteroscedastic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Linear (1.0)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	$d=15$
  Linear (0.3)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Logarithmic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Quadratic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Polynomial	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  LRSO (0.05)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Heteroscedastic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Linear (1.0)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	$d=25$
  Linear (0.3)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Logarithmic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Quadratic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Polynomial	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  LRSO (0.05)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Heteroscedastic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Linear (1.0)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	$d=5$	$X\sim \mathcal{N}(0,I_d)$
  Linear (0.3)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Logarithmic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Quadratic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Polynomial	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  LRSO (0.05)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Heteroscedastic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Linear (1.0)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	$d=15$
  Linear (0.3)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Logarithmic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Quadratic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Polynomial	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  LRSO (0.05)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Heteroscedastic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Linear (1.0)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	$d=25$
  Linear (0.3)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Logarithmic	0.97 [0.92, 0.99]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Quadratic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Polynomial	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  LRSO (0.05)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Heteroscedastic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Linear (1.0)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	$d=5$	$X\sim \mathrm{Student}’\mathrm{s}t\left(3\right)$
  Linear (0.3)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Logarithmic	0.98 [0.93, 0.99]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Quadratic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Polynomial	0.96 [0.90, 0.98]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  LRSO (0.05)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Heteroscedastic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Linear (1.0)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	$d=15$
  Linear (0.3)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Logarithmic	0.98 [0.93, 0.99]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Quadratic	0.99 [0.95, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Polynomial	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	0.99 [0.95, 1.00]
  LRSO (0.05)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Heteroscedastic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Linear (1.0)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	$d=25$
  Linear (0.3)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Logarithmic	0.97 [0.92, 0.99]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Quadratic	0.98 [0.93, 0.99]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Polynomial	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  LRSO (0.05)	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Heteroscedastic	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]

  Table 2. Empirical power with 95% Wilson confidence intervals for dependent data (complex dependence patterns) at $\alpha =0.05$.

  Pattern	UFDM	DCOR	HSIC	MEF
  Mixture Bimodal Marginal	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	$\mathit{d}=5$
  Mixture Bimodal	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Circular	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Gaussian Copula	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Clayton Copula	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Interleaved Moons	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Mixture Bimodal Marginal	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	$d=15$
  Mixture Bimodal	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Circular	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	0.87 [0.79, 0.92]
  Gaussian Copula	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Clayton Copula	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Interleaved Moons	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	0.49 [0.39, 0.59]
  Mixture Bimodal Marginal	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	$d=25$
  Mixture Bimodal	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Circular	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	0.52 [0.42, 0.62]
  Gaussian Copula	0.98 [0.93, 0.99]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Clayton Copula	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]	1.00 [0.96, 1.00]
  Interleaved Moons	1.00 [0.96, 1.00]	0.97 [0.92, 0.99]	0.96 [0.90, 0.98]	0.27 [0.19, 0.36]

• Ablation experiment. The necessity of the SVD warm-up (Algorithm 2) is empirically demonstrated in Table A1, where the p-values obtained without SVD warm-up systematically fail to reveal dependence in many nonlinear patterns.
• Remark on the stability of the estimator. Since the UFDM objective is non-convex, different random initialisations may potentially lead to distinct local optima. To assess the impact of this issue, we investigated the numerical stability of the UFDM estimator. We computed the mean and standard deviation of the statistic across 50 independent runs for each distribution pattern and dimension (Table 1 and Table 2), as well as for the corresponding permuted patterns in which dependence is destroyed, as reported in Table 3. The obtained results align with the permutation test findings. While a slight upward shift is observed under independent (permuted) data, the proposed estimator retained consistent separation between dependent and independent settings and exhibited stable behaviour across random restarts.
  Table 3. UFDM statistic (mean ± std) under true dependence/permuted independence.

  Dependence Pattern	$\mathit{d}=5$	$\mathit{d}=15$	$\mathit{d}=25$
  Linear (strong)	$0.191\pm 0.035/0.023\pm 0.010$	$0.208\pm 0.012/0.045\pm 0.009$	$0.231\pm 0.011/0.083\pm 0.015$	$X\sim U{[0,1]}^d$
  Linear (weak)	$0.123\pm 0.035/0.023\pm 0.009$	$0.143\pm 0.014/0.044\pm 0.011$	$0.166\pm 0.012/0.081\pm 0.015$
  Logarithmic	$0.172\pm 0.048/0.023\pm 0.009$	$0.189\pm 0.022/0.042\pm 0.008$	$0.195\pm 0.014/0.087\pm 0.015$
  Quadratic	$0.199\pm 0.048/0.025\pm 0.013$	$0.200\pm 0.020/0.045\pm 0.009$	$0.212\pm 0.017/0.082\pm 0.017$
  Polynomial	$0.185\pm 0.047/0.023\pm 0.010$	$0.195\pm 0.026/0.047\pm 0.013$	$0.208\pm 0.015/0.083\pm 0.016$
  Contaminated sine	$0.059\pm 0.009/0.006\pm 0.002$	$0.080\pm 0.008/0.009\pm 0.004$	$0.116\pm 0.007/0.015\pm 0.006$
  Conditional variance	$0.102\pm 0.024/0.023\pm 0.010$	$0.142\pm 0.016/0.043\pm 0.010$	$0.173\pm 0.015/0.080\pm 0.016$
  Linear (strong)	$0.240\pm 0.013/0.042\pm 0.011$	$0.239\pm 0.011/0.077\pm 0.014$	$0.250\pm 0.010/0.101\pm 0.009$	$X\sim \mathcal{N}(0,I_d)$
  Linear (weak)	$0.230\pm 0.011/0.044\pm 0.014$	$0.235\pm 0.013/0.077\pm 0.013$	$0.244\pm 0.012/0.104\pm 0.013$
  Logarithmic	$0.254\pm 0.031/0.034\pm 0.010$	$0.184\pm 0.031/0.051\pm 0.013$	$0.136\pm 0.023/0.079\pm 0.014$
  Quadratic	$0.212\pm 0.035/0.028\pm 0.013$	$0.176\pm 0.025/0.049\pm 0.010$	$0.146\pm 0.022/0.080\pm 0.014$
  Polynomial	$0.190\pm 0.041/0.027\pm 0.009$	$0.176\pm 0.027/0.048\pm 0.011$	$0.174\pm 0.020/0.073\pm 0.009$
  Contaminated sine	$0.059\pm 0.009/0.006\pm 0.002$	$0.082\pm 0.010/0.011\pm 0.005$	$0.114\pm 0.009/0.014\pm 0.005$
  Conditional variance	$0.184\pm 0.014/0.038\pm 0.012$	$0.208\pm 0.013/0.077\pm 0.012$	$0.218\pm 0.012/0.102\pm 0.013$
  Linear (strong)	$0.173\pm 0.016/0.031\pm 0.013$	$0.181\pm 0.017/0.053\pm 0.012$	$0.207\pm 0.013/0.080\pm 0.013$	$X\sim$ Student’s $t\left(3\right)$
  Linear (weak)	$0.165\pm 0.018/0.030\pm 0.012$	$0.182\pm 0.018/0.059\pm 0.014$	$0.205\pm 0.013/0.082\pm 0.012$
  Logarithmic	$0.150\pm 0.041/0.024\pm 0.011$	$0.096\pm 0.019/0.041\pm 0.011$	$0.121\pm 0.026/0.064\pm 0.014$
  Quadratic	$0.082\pm 0.037/0.014\pm 0.007$	$0.078\pm 0.020/0.029\pm 0.010$	$0.097\pm 0.022/0.048\pm 0.012$
  Polynomial	$0.037\pm 0.022/0.009\pm 0.004$	$0.050\pm 0.023/0.016\pm 0.008$	$0.085\pm 0.020/0.033\pm 0.012$
  Contaminated sine	$0.057\pm 0.008/0.006\pm 0.002$	$0.078\pm 0.009/0.011\pm 0.004$	$0.115\pm 0.008/0.014\pm 0.004$
  Conditional variance	$0.124\pm 0.018/0.027\pm 0.009$	$0.158\pm 0.014/0.054\pm 0.011$	$0.180\pm 0.011/0.079\pm 0.013$
  Mixture bimodal marginal	$0.496\pm 0.007/0.048\pm 0.011$	$0.500\pm 0.008/0.083\pm 0.012$	$0.500\pm 0.008/0.101\pm 0.012$	Complex patterns
  Mixture bimodal	$0.883\pm 0.006/0.036\pm 0.013$	$0.935\pm 0.006/0.047\pm 0.016$	$0.972\pm 0.005/0.058\pm 0.016$
  Circular	$0.277\pm 0.023/0.048\pm 0.010$	$0.259\pm 0.022/0.089\pm 0.012$	$0.231\pm 0.032/0.114\pm 0.011$
  Gaussian copula	$0.241\pm 0.011/0.038\pm 0.016$	$0.248\pm 0.013/0.049\pm 0.013$	$0.254\pm 0.010/0.056\pm 0.013$
  Clayton copula	$0.284\pm 0.013/0.038\pm 0.013$	$0.287\pm 0.013/0.047\pm 0.014$	$0.290\pm 0.015/0.060\pm 0.014$
  Interleaved moons	$0.418\pm 0.017/0.020\pm 0.008$	$0.384\pm 0.024/0.052\pm 0.013$	$0.339\pm 0.041/0.095\pm 0.014$

4.2. Supervised Feature Extraction

Feature construction is often a key initial step in machine learning with tabular data. These methods can be roughly classified into feature selection and feature extraction. Feature selection identifies a subset of relevant inputs, either incrementally (e.g., via univariate filters) or through other strategies, and feature extraction transforms inputs into lower-dimensional, informative representations. In our experiments, we used the latter approach because of its computational effectiveness. The total computational time for these experiments was $\approx 94.3$ h on single Intel i7 CPU, 16GB of RAM, and Nvidia GeForce RTX 2060 12 GB GPU machine.

Let ${(x_i,y_i)}_{i=1}^n$ be a classification dataset consisting of n pairs of $d_X$-dimensional inputs $x_i$, and $d_Y$-dimensional one-hot encoded outputs $y_i$. In our experiments, we used a collection of OpenML classification datasets [32], which cover different domains, input and output dimensionalities. We randomly split the data into training, validation, and test sets using the proportions $(0.5,0.1,0.4)$, respectively. We followed the dependence maximisation scheme (e.g., [3,33]) by seeking

$$W^{*}=arg\underset{W}{max}\mathrm{DEP}(Wx,y)-\lambda \mathrm{tr}\left({(W^{T}W-I)}^T(W^{T}W-I)\right),$$

(13)

where $\mathrm{DEP}\in \{\mathrm{UFDM},\mathrm{DCOR},\mathrm{HSIC},\mathrm{MEF}\}$. To evaluate the obtained features $f\left(x\right)=W^{*}x$, we used logistic regression’s [34] accuracy, measured on the test set. For each baseline method, we selected the dimensions of the features that correspond to the maximal validation accuracy of the investigated method, checking all dimensions starting from 1 with a step of $10\%$ of $d_X$. Similarly, we selected $\lambda \in \{0.1,1.0,10.0\}$. The feature extraction loss Equation (13) was optimised via Algorithm 1 for 100 epochs, with the learning rate set to $0.025$, as in permutation testing experiments (Section 4.1).

• Baselines. We compared the following baselines: unmodified inputs (denoted as RAW); and Equation (13) scheme with dependence measures: UFDM, DCOR, MEF, and HSIC. We also included the neighbourhood component analysis (NCA) [35] baseline, which is specially tailored for classification.
• Evaluation metrics. Let us denote $a_{r,p}(b,b^{\prime }|d)=1$, if for r runs on the dataset d the average test set accuracy of baseline b is statistically significantly higher than that of $b^{\prime }$ with p-value threshold p. For statistical significance assessment, we used Wilcoxon’s signed-rank test [36]. We computed the win ranking (WR) and loss ranking (LR) as

$$\mathrm{WR}\left(b\right)=\sum _d\sum _{b^{\prime }\ne b}{a}_{25,0.05}(b,b^{\prime }|d)\mathrm{and}\mathrm{LR}\left(b\right)=\sum _d\sum _{b^{\prime }\ne b}{a}_{25,0.05}(b^{\prime },b|d).$$

(14)

• Based on these metrics, Table 4 includes full information on how many cases each baseline method statistically significantly outperformed the other method.
  Table 4. Pairwise wins matrix: entry $(i,j)$ is the number of cases where the method in row i outperformed the method in column j (Wilcoxon’s signed-rank test, 25 runs, p-value threshold $0.05$).

  	UFDM	DCOR	MEF	HSIC	NCA
  UFDM	0	6	4	5	5
  DCOR	2	0	3	4	3
  MEF	4	8	0	9	7
  HSIC	2	4	2	0	3
  NCA	5	7	6	8	0

• Results. Using 18 datasets, we conducted 80 feature efficiency evaluations (excluding the RAW baseline) and 160 feature efficiency comparisons, of which 97 (∼60%) were statistically different. The results of the feature extraction experiments are presented in Table 4 and Table 5. They reveal that, although MEF showed best WR, UFDM also performed comparable to other measures: it statistically significantly outperformed them in $6+4+5+5=20$ cases (listed in Table 6), and was outperformed in $2+4+2+5=13$ cases (Table 4).
  Table 5. Classification accuracy comparison. n denotes dataset size, $d_X$ is input dimensionality, and $n_c$ is the number of classes. Best-performing method that is also statistically significant when compared with all other methods (Wilcoxon’s signed-rank test, 25 runs, p-value threshold $0.05$) is indicated in bold (otherwise, best-performing method is underlined).

  Dataset	$(\mathit{n},{\mathit{d}}_{\mathit{X}},{\mathit{n}}_{\mathit{c}})$	RAW	UFDM	DCOR	MEF	HSIC	NCA
  Australian	(690, 14, 2)	0.710	0.853	0.846	0.850	0.824	0.844
  Collins	(500, 22, 2)	0.840	0.926	0.906	0.941	0.927	0.949
  Heart-statlog	(270, 13, 2)	0.621	0.824	0.823	0.826	0.816	0.817
  Mfeat-factors	(2000, 216, 10)	0.783	0.968	0.970	0.968	0.968	0.969
  Mfeat-pixel	(2000, 240, 10)	0.946	0.956	0.948	0.957	0.951	0.959
  Mfeat-zernike	(2000, 47, 10)	0.741	0.812	0.810	0.814	0.811	0.804
  Micro-mass	(360, 1300, 10)	0.874	0.925	0.919	0.931	0.923	0.882
  Optdigits	(5620, 64, 10)	0.949	0.964	0.961	0.960	0.957	0.963
  Parkinsons	(195, 22, 2)	0.756	0.827	0.828	0.850	0.836	0.837
  Scene	(2407, 299, 2)	0.886	0.987	0.988	0.953	0.988	0.962
  Segment	(2310, 19, 7)	0.760	0.912	0.911	0.943	0.936	0.941
  Sonar	(208, 60, 2)	0.685	0.745	0.733	0.757	0.734	0.770
  Spectf	(349, 44, 2)	0.729	0.737	0.739	0.738	0.739	0.750
  USPS	(9298, 256, 10)	0.924	0.944	0.941	0.934	0.936	0.940
  Wdbc	(569, 30, 2)	0.699	0.948	0.951	0.938	0.900	0.968
  Wine	(178, 13, 3)	0.552	0.945	0.917	0.954	0.947	0.936
  $\mathrm{WR}\left(b\right)$			20	12	28	11	26
  $\mathrm{LR}\left(b\right)$			13	25	15	26	18

  Table 6. Twenty cases (Measures Outperformed) where UFDM outperformed the other baselines.

  Dataset	n	${\mathit{d}}_{\mathit{X}}$	Measures Outperformed
  Australian	690	14	DCOR, HSIC, NCA
  Collins	500	22	DCOR
  Micro-mass	360	1300	NCA
  Mfeat-pixel	2000	240	DCOR, HSIC
  Mfeat-zernike	2000	47	NCA
  Optdigits	5620	64	DCOR, MEF, HSIC
  Scene	2407	299	MEF, NCA
  USPS	9298	256	DCOR, MEF, HSIC, NCA
  Wdbc	569	30	MEF, HSIC
  Wine	178	13	DCOR

In addition to pairwise statistical comparisons using Wilcoxon’s test, we also conducted statistical analysis to clarify whether some method is globally better or worse over multiple datasets using the methodology described in [37]. In this analysis, the Friedman/Iman–Davenport test ($\alpha =0.05$) showed a global significant difference between the five methods. The Nemenyi post hoc test ($\alpha =0.05$, critical difference $1.884$) revealed that RAW was significantly outperformed by the other methods; however, it also showed the absence of a global best-performing method.

5. Conclusions

• Results. We proposed and analysed an IPM-based statistical dependence measure, $\mathrm{UFDM}$, defined as the $L^{\infty }$ norm of the difference between the joint and product-marginal characteristic functions. $\mathrm{UFDM}$ applies to pairs of random vectors of possibly different dimensions and can be integrated into modern machine learning pipelines. In contrast to global measures (e.g., DCOR, HSIC, MEF), which aggregate information across the entire frequency domain, $\mathrm{UFDM}$ identifies spectrally localised dependencies by highlighting frequencies where the discrepancy is maximised, thereby offering potentially interpretable insights into the structure of dependence. We theoretically established key properties of $\mathrm{UFDM}$, such as invariance under linear transformations and augmentation with independent noise, monotonicity under dimension reduction, and vanishing under independence. We also showed that UFDM’s objective aligns with the vectorial representation of CFs. In addition, we investigated the consistency of the empirical estimator and derived a finite-sample concentration bound. For practical estimation, we proposed a gradient-based estimation algorithm with SVD warm-up, and this warm-up was found to be essential for stable convergence.

We evaluated $\mathrm{UFDM}$ on simulated and real data in permutation-based independence testing and supervised feature extraction. The permutation test experiments ($n=750$, $d\in \{5,15,25\}$) indicated that in this regime $\mathrm{UFDM}$ performed comparably to established baseline measures, exhibiting similar empirical power and calibration across diverse dependence structures. Notably, $\mathrm{UFDM}$ maintained high power on the Circular and Interleaved Moons datasets, where some other measures displayed reduced sensitivity under these geometrically structured dependencies. These findings suggest that $\mathrm{UFDM}$ provides a complementary addition to the family of widely used dependence measures (DCOR, HSIC, and MEF).

Further experiments with real data demonstrated that, in dependence-based supervised feature extraction, $\mathrm{UFDM}$ often performed on par with the well-established alternatives (HSIC, DCOR, MEF) and with NCA, which is specifically designed for classification. Across 16 datasets and 160 pairwise comparisons, $\mathrm{UFDM}$ statistically significantly outperformed other baselines in 20 cases and was outperformed in 13. To facilitate reproducibility, we provide an open-source repository.

• Limitations. Computing UFDM requires maximising a highly nonlinear objective, which makes the estimator sensitive to initialisation and optimisation settings. Although the proposed SVD warm-up substantially improves numerical stability, estimation may still become more challenging as dimensionality d increases or sample size n decreases. From the perspective of the effective $(n,d)$, our empirical evaluation covers two different tasks. First, in independence testing with synthetic data and $n=750$ and $d\in \{5,15,25\}$, UFDM maintained effectiveness across diverse dependence structures. Our preliminary experiments with $n=375,d\in \{5,15,25\}$, and $n=750,d=50$ indicate a reduction in power for several dependency patterns, whereas DCOR, HSIC, and MEF remained comparatively stable. Nonetheless, UFDM preserved its performance for sparse geometrically structured dependencies (e.g., Interleaved Moons), where alternative measures often show more pronounced loss of sensitivity. Due to the high computational cost of UFDM permutation tests, we omitted systematic exploration of these regimes, leaving it to future work. On the other hand, in supervised feature extraction on real datasets, we examined substantially broader $(n,d)$ ranges, including high-dimensional settings such as USPS$(n=9298,d=256)$, Micro-Mass$(n=360,d=1300)$, and Scene$(n=2407,d=299)$. UFDM outperformed one or more baselines on several such datasets (Table 6), suggesting that it may be effective in some larger-dimensional machine learning tasks.
• Future work and potential applications. Identifying the limit distribution of the empirical $\mathrm{UFDM}$ could enable faster alternatives to permutation-based statistical tests, which would also facilitate the systematic analysis of previously mentioned $(n,d)$ settings. However, since the empirical $\mathrm{UFDM}$ is not a U- or V-statistic like HSIC or distance correlation, this would require a non-trivial analysis of the extrema of empirical processes. Possible extensions of UFDM include multivariate generalisations [23] and weighted or normalised variants to enhance empirical stability. From an application perspective, $\mathrm{UFDM}$ may prove useful in causality, regularisation, representation learning, and other areas of modern machine learning where statistical dependence serves as an optimisation criterion.

Appendix A.1. Proofs

In the proofs, we interchangeably abbreviate ${\varphi }_X\left(\alpha \right)$ with $\varphi \left(\alpha \right)$, ${\varphi }_Y\left(\beta \right)$ with $\varphi \left(\beta \right)$, and ${\varphi }_{X,Y}(\alpha ,\beta )$ with $\varphi \left(\gamma \right)$, where $\gamma ={({\alpha }^T,{\beta }^T)}^T$.

Proof of Theorem 1.

Property 1. By Cauchy–Schwarz inequality.

$$\begin{array}{c}{|\Delta (\alpha ,\beta )|}^2={\left|{\mathbb{E}}_{X,Y}(e^{i{\alpha }^{T}X}-{\varphi }_X\left(\alpha \right))(e^{i{\beta }^{T}Y}-{\varphi }_Y\left(\beta \right))\right|}^2\le \hfill \\ \hfill \le {\mathbb{E}}_X|(e^{i{\alpha }^{T}X}-{\varphi }_X\left(\alpha \right)){|}^2{\mathbb{E}}_Y{\left|(e^{i{\beta }^{T}Y}-{\varphi }_Y\left(\beta \right))\right|}^2.\end{array}$$

(A1)

Recall that for complex numbers z and $z^{\prime }$ we have $|z-z^{\prime }{|}^2={\left|z\right|}^2-z\overline{z^{\prime }}-\overline{z}{z}^{\prime }+{\left|z^{\prime }\right|}^2$, where $\overline{z}$ is complex conjugate of z. Therefore by plugging $z=e^{i{\alpha }^{T}X}$ and $z^{\prime }={\varphi }_X\left(\alpha \right)$ from the definition of CF we obtain

$$\begin{array}{c}\hfill {\mathbb{E}}_X|e^{i{\alpha }^{T}X}-{\varphi }_X{\left(\alpha \right)|}^2=1-{\varphi }_X\left(\alpha \right)\overline{{\varphi }_X\left(\alpha \right)}-\overline{{\varphi }_X\left(\alpha \right)}{\varphi }_X\left(\alpha \right)+|{\varphi }_X{\left(\alpha \right)|}^2=1-{\left|{\varphi }_X\left(\alpha \right)\right|}^2,\end{array}$$

and similarly ${\mathbb{E}}_Y|(e^{i{\beta }^{T}Y}-{\varphi }_Y\left(\beta \right)){|}^2=1-{\left|{\varphi }_Y\left(\beta \right)\right|}^2$. Since the absolute value of CF is bounded by 1, we have that  Equation (A1) is also bounded by 1.

Property 2.

$$\begin{array}{c}\mathrm{UFDM}(X,Y)=\underset{\alpha ,\beta }{sup}|{\mathbb{E}}_{X,Y}{e}^{i({\alpha }^{T}X+{\beta }_{T}Y)}-{\mathbb{E}}_X{e}^{i{\alpha }^{T}X}{\mathbb{E}}_Y{e}^{i{\beta }^{T}Y}|\hfill \\ \hfill =\underset{\beta ,\alpha }{sup}|{\mathbb{E}}_{Y,X}{e}^{i({\beta }^{T}Y+{\alpha }^{T}X)}-{\mathbb{E}}_Y{e}^{i{\beta }^{T}Y}{\mathbb{E}}_X{e}^{i{\alpha }^{T}X}|=\mathrm{UFDM}(Y,X).\end{array}$$

Property 3. Let us assume that $X\perp Y$. Then ${\varphi }_{X,Y}(\alpha ,\beta )={\mathbb{E}}_{X,Y}{e}^{i({\alpha }^{T}X+{\beta }^{T}Y)}={\mathbb{E}}_X{\mathbb{E}}_Y{e}^{i({\alpha }^{T}X+{\beta }^{T}Y)}={\varphi }_X\left(\alpha \right){\varphi }_Y\left(\beta \right)$. Therefore, $\mathrm{UFDM}(X,Y)=0$. On the other hand, if $\mathrm{UFDM}(X,Y)=0$ then ${\varphi }_{X,Y}(\alpha ,\beta )={\varphi }_X\left(\alpha \right){\varphi }_Y\left(\beta \right)$ for all $\alpha \in {\mathbb{R}}^{d_X}$,$\beta \in {\mathbb{R}}^{d_Y}$. Let $\tilde{X}$ and $\tilde{Y}$ be two independent random vectors, having the same distributions as X and Y, respectively. Therefore ${\varphi }_{X,Y}(\alpha ,\beta )={\varphi }_X\left(\alpha \right){\varphi }_Y\left(\beta \right)={\varphi }_{\tilde{X}}\left(\alpha \right){\varphi }_{\tilde{Y}}\left(\beta \right)={\varphi }_{\tilde{X},\tilde{Y}}(\alpha ,\beta )$. The uniqueness of CF [16] implies that distributions of $(X,Y)$ and $(\tilde{X},\tilde{Y})$ coincide, from what directly follows that $X\perp Y$.

Property 4. Let ${\Sigma }_{X,Y}$ be cross-covariance matrix of X and Y. Since X and Y are Gaussian, we have ${\varphi }_X\left(\alpha \right)=e^{-{\displaystyle \frac{1}{2}}{\alpha }^T{\Sigma }_X\alpha }$, ${\varphi }_Y\left(\beta \right)=e^{-{\displaystyle \frac{1}{2}}{\beta }^T{\Sigma }_Y\beta }$, ${\varphi }_{X,Y}(\alpha ,\beta )=e^{-{\displaystyle \frac{1}{2}}({\alpha }^T{\Sigma }_X\alpha +{\beta }^T{\Sigma }_Y\beta +2{\alpha }^T{\Sigma }_{X,Y}\beta )}$. Therefore, by Equation (11)

$$\mathrm{UFDM}(X,Y)=\underset{\alpha ,\beta }{sup}{e}^{-{\displaystyle \frac{1}{2}}({\alpha }^T{\Sigma }_X\alpha +{\beta }^T{\Sigma }_Y\beta )}|e^{-{\alpha }^T{\Sigma }_{X,Y}\beta }-1|.$$

(A2)

Property 5. Since ${\varphi }_{AX+a,BY+b}(\alpha ,\beta )=e^{i{\alpha }^{T}a+i{\beta }^{T}b}{\varphi }_{X,Y}(A^T\alpha ,B^T\beta )$, and ${\varphi }_{AX+a}\left(\alpha \right)=e^{i{\alpha }^{T}a}{\varphi }_X\left(A^T\alpha \right)$, ${\varphi }_{BY+b}\left(\beta \right)=e^{i{\beta }^{T}b}{\varphi }_Y\left(B^T\beta \right)$, we have

$$\begin{array}{c}\mathrm{UFDM}(AX+a,BY+b)=\underset{\alpha ,\beta }{sup}|{\varphi }_{AX+a,BY+b}(\alpha ,\beta )-{\varphi }_{Ax+a}\left(\alpha \right){\varphi }_{BY+b}\left(\beta \right)|=\hfill \\ \hfill =\underset{\alpha ,\beta }{sup}|e^{i{\alpha }^{T}a+i{\beta }^{T}b}\left|\right|\Delta (A^T\alpha ,B^T\beta )|=\underset{\alpha ,\beta }{sup}|\Delta (A^T\alpha ,B^T\beta )|.\end{array}$$

Since both A and B are full-rank matrices, and $A\in {\mathbb{R}}^{d_X\times d_X}$, $B\in {\mathbb{R}}^{d_Y\times d_Y}$, the maximization of the last equation is equivalent to the maximization of $|\Delta (\alpha ,\beta \left)\right|$, which by definition is $\mathrm{UFDM}(X,Y)$.

Property 6. If $A^{\prime }\in {\mathbb{R}}^{d_{X^{\prime }}\times d_X}$, $B^{\prime }\in {\mathbb{R}}^{d_{Y^{\prime }}\times d_Y}$, $a^{\prime }\in {\mathbb{R}}^{d_{X^{\prime }}}$, $b^{\prime }\in {\mathbb{R}}^{d_{Y^{\prime }}}$ are parameters of linear dimension reduction, where $d_{X^{\prime }}<d_X$, and $d_{Y^{\prime }}<d_Y$, we have

$$\mathrm{UFDM}(A^{\prime }X+a^{\prime },B^{\prime }Y+b^{\prime })\le \mathrm{UFDM}(AX+a,BY+b),$$

(A3)

for any $A,B,a,b$ of the same dimensions (defined as in Property 5), because maximisation of LHS is conducted in smaller space than that of RHS. By Property 5, it follows that $\mathrm{UFDM}(AX+a,BY+b)=\mathrm{UFDM}(X,Y)$.

Property 7. The independence of X and $\mathcal{E}$ implies that

$$\begin{array}{c}\hfill \mathrm{UFDM}(X,f\left(X\right)+\lambda \mathcal{E})=\underset{\alpha ,\beta }{sup}|\mathbb{E}{e}^{i({\alpha }^{T}X+{\beta }^{T}f\left(X\right))}{\varphi }_{\mathcal{E}}\left(\lambda \beta \right)-{\varphi }_X\left(\alpha \right){\varphi }_{f\left(X\right)}\left(\beta \right){\varphi }_{\mathcal{E}}\left(\lambda \beta \right)|,\end{array}$$

which converges to 0, since by multivariate Riemann–Lebesgue lemma [39] the common term $|{\varphi }_{\mathcal{E}}\left(\lambda \beta \right)|\to 0$, when $\lambda \to \infty$. The multivariate Riemann–Lebesgue lemma can be applied since $\mathcal{E}$ has a density.

Property 8. Recall that the total variation distance between joint probability measure $P_{X,Y}$ and product measure $P_X{P}_Y$ is given by

$$\mathrm{TV}(P_{X,Y},P_X{P}_Y)={\displaystyle \frac{1}{2}}\int \left|p_{X,Y}(x,y)-p_X\left(x\right)p_Y\left(y\right)\right|dxdy,$$

where $p_{X,Y}(x,y)$ is joint density, and $p_X\left(x\right)$, $p_Y\left(y\right)$ are marginal ones. Recall that Pinsker’s inequality for total variation states that $\mathrm{TV}(P_{X,Y},P_X{P}_Y)\le \sqrt{{\displaystyle \frac{1}{2}}\mathrm{MI}(X,Y)}$, where $\mathrm{MI}(X,Y)$ is mutual information between X and Y. Therefore,

$$\begin{array}{c}\left|\Delta (\alpha ,\beta )\right|=\left|\int e^{i({\alpha }^{T}x+{\beta }^{T}y)}(p_{X,Y}(x,y)-p_X\left(x\right)p_Y\left(y\right))dxdy\right|\hfill \\ \hfill \le \int \left|p_{X,Y}(x,y)-p_X\left(x\right)p_Y\left(y\right)\right|dxdy=2\mathrm{TV}(P_{X,Y},P_X{P}_Y).\end{array}$$

By taking the supremum we have $\mathrm{UFDM}(X,Y)\le min\{1,2\mathrm{TV}(P_{X,Y},P_X{P}_Y)\}\le min\{1,\sqrt{2\mathrm{MI}(X,Y)}\}$ by Property 1 and Pinsker’s inequality.

Property 9. Independence condition $Z\perp {(X^{\top },Y^{\top })}^{\top }$ gives

$${\Delta }_{{(X^{\top },Z^{\top })}^{\top },Y}({\alpha }_X,{\alpha }_Z,\beta )={\phi }_Z\left({\alpha }_Z\right){\Delta }_{X,Y}({\alpha }_X,\beta ).$$

Since $|{\phi }_Z\left({\alpha }_Z\right)|\le 1$ and $|{\phi }_Z\left(0\right)|=1$, we have ${sup}_{{\alpha }_X,{\alpha }_Z,\beta }\left|{\Delta }_{{(X^{\top },Z^{\top })}^{\top },Y}\right|={sup}_{{\alpha }_X,\beta }\left|{\Delta }_{X,Y}({\alpha }_X,\beta )\right|.$ Therefore, $\mathrm{UFDM}({(X^{\top },Z^{\top })}^{\top },Y)=\mathrm{UFDM}(X,Y)$.    □

Proof of Proposition 2.

Let $ϵ>0$. Since ECF is CF, and a product of two CFs also is CF, by Theorem 2 and triangle inequality, we can find natural number $n_0$ such that $\forall n>n_0$: $\left|\right|\Delta -{\Delta }_n{\left|\right|}_{L_{\infty }}^{t_n}={sup}_{\left|\right|\gamma \left|\right|<t_n}|\Delta \left(\gamma \right)-{\Delta }_n\left(\gamma \right)|={sup}_{\left|\right|\gamma \left|\right|<t_n}|\varphi \left(\gamma \right)-\psi \left(\gamma \right)-{\varphi }_n\left(\gamma \right)+{\psi }_n\left(\gamma \right)|={sup}_{\left|\right|\gamma \left|\right|<t_n}|\varphi \left(\gamma \right)-{\varphi }_n\left(\gamma \right)+{\psi }_n\left(\gamma \right)-\psi \left(\gamma \right)|\le {sup}_{\left|\right|\gamma \left|\right|<t_n}|\varphi \left(\gamma \right)-{\varphi }_n\left(\gamma \right)|+{sup}_{\left|\right|\gamma \left|\right|<t_n}|\psi \left(\gamma \right)-{\psi }_n\left(\gamma \right)|\le ϵ$, almost surely. From the inverse triangle inequality for norms we have ${\left|\right||\Delta ||}_{L_{\infty }}^{t_n}-\left|\right|{\Delta }_n{\left|\right|}_{L_{\infty }}^{t_n}|\le ||\Delta -{\Delta }_n{\left|\right|}_{L_{\infty }}^{t_n}\le ϵ$, almost surely. On the other hand, along with the definition of $\mathrm{UFDM}(X,Y)={lim}_{n\to \infty }{\left|\right|\Delta \left(\gamma \right)\left|\right|}_{L_{\infty }}^{t_n}$, this implies that $|\mathrm{UFDM}(X,Y)-||{\Delta }_n{\left|\right|}_{L_{\infty }}^{t_n}{|\le |\mathrm{UFDM}(X,Y)-\left|\right|\Delta \left|\right|}_{L_{\infty }}^{t_n}{|+|\left|\right|\Delta \left|\right|}_{L_{\infty }}^{t_n}-\left|\right|{\Delta }_n{\left|\right|}_{L_{\infty }}^{t_n}|$ will be arbitrarily small almost surely, when n is sufficiently large.    □

Appendix A.2. Proof of Theorem 3

Proof of Theorem 3.

Recall that $Z={(X^T,Y^T)}^T$, $\gamma ={({\alpha }^T,{\beta }^T)}^T\in {\mathbb{R}}^d$ with $d=d_X+d_Y$ and

$$\Delta \left(\gamma \right)=\varphi \left(\gamma \right)-\psi \left(\gamma \right),{\Delta }_n\left(\gamma \right)={\varphi }_n\left(\gamma \right)-{\psi }_n\left(\gamma \right).$$

Step 1. Lipschitz continuity. First, we will prove that $\Delta \left(\gamma \right)$ and ${\Delta }_n\left(\gamma \right)$ are Lipschitz continuous. For the population version, consider

$$|\Delta \left(\gamma \right)-\Delta \left({\gamma }^{\prime }\right)|\le |\varphi \left(\gamma \right)-\varphi \left({\gamma }^{\prime }\right)|+|\psi \left(\gamma \right)-\psi \left({\gamma }^{\prime }\right)|.$$

Since $\varphi \left(\gamma \right)=\mathbb{E}exp\left(i{\gamma }^{T}Z\right)$, by inequality $|e^{ia}-e^{ib}|\le |a-b|$, $a,b\in \mathbb{R}$

$$|\varphi \left(\gamma \right)-\varphi \left({\gamma }^{\prime }\right)|\le \mathbb{E}|exp\left(i{\gamma }^{T}Z\right)-exp\left(i{\gamma }^{\prime T}Z\right)|\le \mathbb{E}|{(\gamma -{\gamma }^{\prime })}^{T}Z|\le \parallel \gamma -{\gamma }^{\prime }\parallel \mathbb{E}\parallel Z\parallel .$$

Similarly,

$$\begin{array}{cc}\hfill \left|\psi \left(\gamma \right)-\psi \left({\gamma }^{\prime }\right)\right|=\left|\varphi \left(\alpha \right)\varphi \left(\beta \right)-\varphi \left({\alpha }^{\prime }\right)\varphi \left({\beta }^{\prime }\right)\right|=\left|\varphi \left(\alpha \right)\right|\left|\varphi \left(\beta \right)-\varphi \left({\beta }^{\prime }\right)\right|+\left|\varphi \left({\beta }^{\prime }\right)\right|\left|(\varphi \left(\alpha \right)-\varphi \left({\alpha }^{\prime }\right)\right|& \\ \hfill \le |\varphi \left(\alpha \right)-\varphi \left({\alpha }^{\prime }\right)|+|\varphi \left(\beta \right)-\varphi \left({\beta }^{\prime }\right)|,\end{array}$$

since $\left|\varphi \right(\alpha \left)\right|\le 1$, $\left|\varphi \right(\beta \left)\right|\le 1$. Therefore,

$$|\varphi \left(\alpha \right)-\varphi \left({\alpha }^{\prime }\right)|\le \mathbb{E}\parallel X\parallel \parallel \alpha -{\alpha }^{\prime }\parallel ,|\varphi \left(\beta \right)-\varphi \left({\beta }^{\prime }\right)|\le \mathbb{E}\parallel Y\parallel \parallel \beta -{\beta }^{\prime }\parallel .$$

Thus,

$$|\psi \left(\gamma \right)-\psi \left({\gamma }^{\prime }\right)|\le (\mathbb{E}\parallel X\parallel +\mathbb{E}\parallel Y\parallel )\parallel \gamma -{\gamma }^{\prime }\parallel ,$$

so $\Delta \left(\gamma \right)$ is Lipschitz with constant $L=\mathbb{E}\parallel Z\parallel +\mathbb{E}\parallel X\parallel +\mathbb{E}\parallel Y\parallel <\infty$. For the empirical version,

$$|{\Delta }_n\left(\gamma \right)-{\Delta }_n\left({\gamma }^{\prime }\right)|\le |{\varphi }_n\left(\gamma \right)-{\varphi }_n\left({\gamma }^{\prime }\right)|+|{\psi }_n\left(\gamma \right)-{\psi }_n\left({\gamma }^{\prime }\right)|,$$

where

$$|{\varphi }_n\left(\gamma \right)-{\varphi }_n\left({\gamma }^{\prime }\right)|\le {\displaystyle \frac{1}{n}}\sum _{j=1}^n|{\gamma }^T{Z}_j-{\gamma }^{\prime T}{Z}_j|\le \left({\displaystyle \frac{1}{n}}\sum _{j=1}^n\parallel Z_j\parallel \right)\parallel \gamma -{\gamma }^{\prime }\parallel ,$$

and

$$|{\psi }_n\left(\gamma \right)-{\psi }_n\left({\gamma }^{\prime }\right)|\le |{\varphi }_n\left(\alpha \right)-{\varphi }_n\left({\alpha }^{\prime }\right)|+|{\varphi }_n\left(\beta \right)-{\varphi }_n\left({\beta }^{\prime }\right)|\le {\displaystyle \frac{1}{n}}\left(\sum _{j=1}^n\parallel X_j\parallel +\sum _{j=1}^n\parallel Y_j\parallel \right)\parallel \gamma -{\gamma }^{\prime }\parallel .$$

Define $L_n={\displaystyle \frac{1}{n}}{\sum }_{j=1}^n(\parallel Z_j\parallel +\parallel X_j\parallel +\parallel Y_j\parallel )$, so ${\Delta }_n\left(\gamma \right)$ is Lipschitz with random constant $L_n$. Recall that $\mathbb{E}{L}_n=L$, $\mathbb{E}{(L_n-L)}^2={\sigma }^2/n$ are finite because of bounded second moment assumption. $L_n$ concentrates around L, and by Cantelli’s inequality, we have

$$Pr(L_n\ge 2L)=Pr(L_n-L\ge L)\le {\displaystyle \frac{1}{1+n{(L/\sigma )}^2}}\le {\displaystyle \frac{{\sigma }^2}{n{L}^2}}.$$

(A4)

Step 2. Construct a $\delta$-net and bound the deviation on the $\delta$-net. For $B_t=\{\gamma :\parallel \gamma \parallel <t\}$, construct a $\delta$-net $\{{\gamma }_1,\dots ,{\gamma }_{N(t,\delta )}\}$ such that every $\gamma \in B_t$ is within $\delta$ of some ${\gamma }_k$. The cardinality satisfies $N(t,\delta )\le {(3t/\delta )}^d$ [40].

For fixed ${\gamma }_k$, bound $|{\Delta }_n\left({\gamma }_k\right)-\Delta \left({\gamma }_k\right)|$. Changing one $Z_j$ to $Z_j^{\prime }$ alters ${\varphi }_n\left({\gamma }_k\right)$ by at most $2/n$, ${\varphi }_n\left({\alpha }_k\right)$ and ${\varphi }_n\left({\beta }_k\right)$ by at most $2/n$ each, and ${\psi }_n\left({\gamma }_k\right)$ by at most $4/n$. Thus, $|{\Delta }_n\left({\gamma }_k\right)-{\Delta }_n^{\prime }\left({\gamma }_k\right)|\le 6/n$. By McDiarmid’s inequality,

$$Pr\left(\right|{\Delta }_n\left({\gamma }_k\right)-\mathbb{E}{\Delta }_n\left({\gamma }_k\right)|>u)\le 2exp\left(-{\displaystyle \frac{n{u}^2}{18}}\right).$$

Compute the bias: $\mathbb{E}{\varphi }_n\left({\gamma }_k\right)=\varphi \left({\gamma }_k\right)$, and $\mathbb{E}{\psi }_n\left({\gamma }_k\right)={\displaystyle \frac{1}{n}}\varphi \left({\gamma }_k\right)+\left(1-{\displaystyle \frac{1}{n}}\right)\psi \left({\gamma }_k\right),$ so

$$\mathbb{E}{\Delta }_n\left({\gamma }_k\right)=\left(1-{\displaystyle \frac{1}{n}}\right)\Delta \left({\gamma }_k\right),|\mathbb{E}{\Delta }_n\left({\gamma }_k\right)-\Delta \left({\gamma }_k\right)|\le {\displaystyle \frac{1}{n}}.$$

Thus,

$$Pr\left(\right|{\Delta }_n\left({\gamma }_k\right)-\Delta \left({\gamma }_k\right)|>\epsilon )\le 2exp\left(-{\displaystyle \frac{n}{18}}{\left(\epsilon -{\displaystyle \frac{1}{n}}\right)}^2\right),\epsilon >{\displaystyle \frac{1}{n}}.$$

Step 3. Extend to the entire frequency ball. For any $\gamma \in B_t$, choose ${\gamma }_k$ with $\parallel \gamma -{\gamma }_k\parallel \le \delta$. Then we have

$$\begin{array}{c}\hfill |{\Delta }_n\left(\gamma \right)-\Delta \left(\gamma \right)|\le |{\Delta }_n\left(\gamma \right)-{\Delta }_n\left({\gamma }_k\right)|+|{\Delta }_n\left({\gamma }_k\right)-\Delta \left({\gamma }_k\right)|+|\Delta \left({\gamma }_k\right)-\Delta \left(\gamma \right)|\end{array}$$

(A5)

$$\begin{array}{c}\hfill \le L_n\delta +|{\Delta }_n\left({\gamma }_k\right)-\Delta \left({\gamma }_k\right)|+L\delta .\end{array}$$

(A6)

Thus, ${sup}_{\gamma \in B_t}|{\Delta }_n\left(\gamma \right)-\Delta \left(\gamma \right)|\le (L_n+L)\delta +{max}_k|{\Delta }_n\left({\gamma }_k\right)-\Delta \left({\gamma }_k\right)|$. Then by union bound

$$\begin{array}{cc}\hfill Pr\left(\underset{\gamma \in B_t}{sup}|{\Delta }_n\left(\gamma \right)-\Delta \left(\gamma \right)|>\epsilon \right)& \le Pr\left((L_n+L)\delta >{\displaystyle \frac{\epsilon }{2}}\right)\hfill \\ \hfill & +Pr\left(\underset{k}{max}|{\Delta }_n\left({\gamma }_k\right)-\Delta \left({\gamma }_k\right)|>{\displaystyle \frac{\epsilon }{2}}\right).\hfill \end{array}$$

(A7)

Recall that in Equation (A4) we showed that $Pr(L_n>2L)\le {\displaystyle \frac{{\sigma }^2}{n{L}^2}}$. Choosing $\delta ={\displaystyle \frac{\epsilon }{6L}}$ implies

$$Pr((L_n+L)\delta >{\displaystyle \frac{\epsilon }{2}})=Pr(L_n>2L)\le {\displaystyle \frac{{\sigma }^2}{n{L}^2}}.$$

(A8)

For the max term, by the union bound,

$$Pr(\underset{k}{max}|{\Delta }_n\left({\gamma }_k\right)-\Delta \left({\gamma }_k\right)|>{\displaystyle \frac{\epsilon }{2}})\le 2N(t,\delta )exp\left(-{\displaystyle \frac{n}{18}}{\left({\displaystyle \frac{\epsilon }{2}}-{\displaystyle \frac{1}{n}}\right)}^2\right),$$

(A9)

where $N(t,\delta )\le {(3t/\delta )}^d={\left({\displaystyle \frac{18tL}{\epsilon }}\right)}^d$.

• Step 4: Final bound. Plugging Equation (A8) and Equation (A9) into Equation (A7) we have

$$Pr(\underset{\gamma \in B_t}{sup}|{\Delta }_n\left(\gamma \right)-\Delta \left(\gamma \right)|>\epsilon )\le 2{\left({\displaystyle \frac{Ct}{\epsilon }}\right)}^{d}exp\left(-{\displaystyle \frac{n}{18}}{\left({\displaystyle \frac{\epsilon }{2}}-{\displaystyle \frac{1}{n}}\right)}^2\right)+{\displaystyle \frac{{\sigma }^2}{n{L}^2}}.$$

Finally, the stated bound follows from the inverse triangle inequality for norms.   □

Appendix A.3. Ablation Experiment on SVD Warm-Up

Table A1. p-value means and standard deviations of the analysed dependence patterns in permutation tests for UFDM without SVD warm-up (Algorithm 2), uniformly initialising parameters $\alpha$ and $\beta$ from $[-1,1]$ interval. Here $X\sim \mathcal{N}(0,I_d)$.

Distribution of Y	$\mathit{d}=5$	$\mathit{d}=15$	$\mathit{d}=25$
Linear (1.0)	0.002 ± 0.000	0.002 ± 0.000	0.002 ± 0.000
Linear (0.3)	0.002 ± 0.000	0.002 ± 0.000	0.002 ± 0.000
Logarithmic	0.035 ± 0.097	0.192 ± 0.251	0.387 ± 0.297
Quadratic	0.023 ± 0.061	0.298 ± 0.291	0.285 ± 0.145
Polynomial	0.002 ± 0.000	0.062 ± 0.134	0.056 ± 0.078
LRSO (0.05)	0.002 ± 0.001	0.041 ± 0.066	0.026 ± 0.040
Heteroscedastic	0.004 ± 0.006	0.002 ± 0.001	0.003∗ ± 0.003

Table A1. p-value means and standard deviations of the analysed dependence patterns in permutation tests for UFDM without SVD warm-up (Algorithm 2), uniformly initialising parameters $\alpha$ and $\beta$ from $[-1,1]$ interval. Here $X\sim \mathcal{N}(0,I_d)$.

Distribution of Y	$\mathit{d}=5$	$\mathit{d}=15$	$\mathit{d}=25$
Linear (1.0)	0.002 ± 0.000	0.002 ± 0.000	0.002 ± 0.000
Linear (0.3)	0.002 ± 0.000	0.002 ± 0.000	0.002 ± 0.000
Logarithmic	0.035 ± 0.097	0.192 ± 0.251	0.387 ± 0.297
Quadratic	0.023 ± 0.061	0.298 ± 0.291	0.285 ± 0.145
Polynomial	0.002 ± 0.000	0.062 ± 0.134	0.056 ± 0.078
LRSO (0.05)	0.002 ± 0.001	0.041 ± 0.066	0.026 ± 0.040
Heteroscedastic	0.004 ± 0.006	0.002 ± 0.001	0.003∗ ± 0.003

Appendix A.4. Dependency Patterns

Table A2. Dependence structures. $Lin[a,b]$ denotes uniform linear spacing over given interval $[a,b]$, $a<b$, $X\perp \mathcal{E}\sim \mathcal{N}(0,I)$, and d is dimension. Fixed parameters $k=6$, $\rho =0.85$, $\theta =5.0$. By ⊙ we denote element-wise product.

Type	Formula
Structured dependence patterns ($X\sim \{\mathcal{N}(0,I_d),U{[0,1]}^d,\mathrm{Student}{t}_3(0,I_d)\}$)
Linear(p)	$Y=pWX+0.1\mathcal{E}$, $p\in \mathbb{R}$
Logarithmic	$Y=log(1.0+WX\odot WX)+0.1\mathcal{E}$
Quadratic	$Y=WX\odot WX+0.1\mathcal{E}$
Cubic	$Y=0.5(WX\odot WX\odot WX)-WX\odot WX+0.1\mathcal{E}$
LRSO(p)	$\begin{array}{c}{X}_0\sim P_X,Y_0=sin\left(k\left(w^T{X}_0\right)\right){\mathbf{1}}_d+0.1\mathcal{E}(\mathrm{proportion}1-p)\hfill \\ X_1\perp Y_1\sim \mathcal{N}(0,{25}^2{I}_d)(\mathrm{proportion}p),\hfill \\ (X,Y)=\mathrm{random}\text{-}\mathrm{shuffle}(X_0\cup X_1,Y_0\cup Y_1)\hfill \end{array}$
Heteroscedastic	$Y=(1.0+{\mathcal{E}}_1)WX+0.1\mathcal{E}$, ${\mathcal{E}}_1\sim \mathcal{N}(0,I)$
Complex dependence patterns
Bimodal	$\begin{array}{c}S\sim \mathrm{Uniform}\left(\right\{-1,1\left\}\right)\hfill \\ {\mu }_X=2{\mathbf{1}}_{d_X},{\mu }_Y=2{\mathbf{1}}_{d_Y}\hfill \\ X\sim \mathcal{N}(S{\mu }_X,I_{d_X})\hfill \\ Y\sim \mathcal{N}(S{\mu }_Y,I_{d_Y})\hfill \end{array}$
Sparse bimodal	$X\sim 0.5\mathcal{N}(\mu ,I_d)+0.5\mathcal{N}(-\mu ,I_d)$,   $\mu =(2,0,\dots ,0)$
Sparse circular	$\begin{array}{c}T\sim Lin[0,2\pi ],R\sim \mathcal{N}(1,0.2^2)\hfill \\ X=(RcosT,RsinT,\eta ),\eta \sim \mathcal{N}(0,I_{d-2})\hfill \\ Y=(Rcos(T+\delta ),Rsin(T+\delta ),\zeta )+0.1\mathcal{E},\delta \sim \mathcal{N}(0,1),\zeta \sim \mathcal{N}(0,I_{d-2})\hfill \end{array}$
Gaussian copula	Marginals $\sim \mathcal{N}(0,\rho {\mathbf{1}}_{d\times d}+(1-\rho )I_d)$.
Clayton copula	$\mathrm{Parameter}\theta \mathrm{and}\mathrm{standard}\mathrm{normal}\mathrm{marginals}\mathrm{for}\mathrm{each}\mathrm{component}.$
Interleaved Moons	$\begin{array}{c}(X_0,L_X)=\mathrm{make}\text{\_}\mathrm{moos}\left(\right),(Y_0,L_Y)=\mathrm{make}\text{\_}\mathrm{moos}\left(\right)\hfill \\ \mathrm{For}\mathrm{each}\mathrm{sample}\mathrm{i}:\hfill \\ X_i^{(1,2)}={\left(X_0\right)}_i\hfill \\ Y_i^{(1,2)}\sim \mathrm{Uniform}\{{\left(Y_0\right)}_j\mid {\left(L_Y\right)}_j\ne {\left(L_X\right)}_i\}\hfill \\ X_i^{(3:d)},Y_i^{(3:d)}\sim \mathcal{N}(0,I_{d-2})\hfill \end{array}$

We used sklearn.datasets.make_moons.

Table A2. Dependence structures. $Lin[a,b]$ denotes uniform linear spacing over given interval $[a,b]$, $a<b$, $X\perp \mathcal{E}\sim \mathcal{N}(0,I)$, and d is dimension. Fixed parameters $k=6$, $\rho =0.85$, $\theta =5.0$. By ⊙ we denote element-wise product.

Type	Formula
Structured dependence patterns ($X\sim \{\mathcal{N}(0,I_d),U{[0,1]}^d,\mathrm{Student}{t}_3(0,I_d)\}$)
Linear(p)	$Y=pWX+0.1\mathcal{E}$, $p\in \mathbb{R}$
Logarithmic	$Y=log(1.0+WX\odot WX)+0.1\mathcal{E}$
Quadratic	$Y=WX\odot WX+0.1\mathcal{E}$
Cubic	$Y=0.5(WX\odot WX\odot WX)-WX\odot WX+0.1\mathcal{E}$
LRSO(p)	$\begin{array}{c}{X}_0\sim P_X,Y_0=sin\left(k\left(w^T{X}_0\right)\right){\mathbf{1}}_d+0.1\mathcal{E}(\mathrm{proportion}1-p)\hfill \\ X_1\perp Y_1\sim \mathcal{N}(0,{25}^2{I}_d)(\mathrm{proportion}p),\hfill \\ (X,Y)=\mathrm{random}\text{-}\mathrm{shuffle}(X_0\cup X_1,Y_0\cup Y_1)\hfill \end{array}$
Heteroscedastic	$Y=(1.0+{\mathcal{E}}_1)WX+0.1\mathcal{E}$, ${\mathcal{E}}_1\sim \mathcal{N}(0,I)$
Complex dependence patterns
Bimodal	$\begin{array}{c}S\sim \mathrm{Uniform}\left(\right\{-1,1\left\}\right)\hfill \\ {\mu }_X=2{\mathbf{1}}_{d_X},{\mu }_Y=2{\mathbf{1}}_{d_Y}\hfill \\ X\sim \mathcal{N}(S{\mu }_X,I_{d_X})\hfill \\ Y\sim \mathcal{N}(S{\mu }_Y,I_{d_Y})\hfill \end{array}$
Sparse bimodal	$X\sim 0.5\mathcal{N}(\mu ,I_d)+0.5\mathcal{N}(-\mu ,I_d)$,   $\mu =(2,0,\dots ,0)$
Sparse circular	$\begin{array}{c}T\sim Lin[0,2\pi ],R\sim \mathcal{N}(1,0.2^2)\hfill \\ X=(RcosT,RsinT,\eta ),\eta \sim \mathcal{N}(0,I_{d-2})\hfill \\ Y=(Rcos(T+\delta ),Rsin(T+\delta ),\zeta )+0.1\mathcal{E},\delta \sim \mathcal{N}(0,1),\zeta \sim \mathcal{N}(0,I_{d-2})\hfill \end{array}$
Gaussian copula	Marginals $\sim \mathcal{N}(0,\rho {\mathbf{1}}_{d\times d}+(1-\rho )I_d)$.
Clayton copula	$\mathrm{Parameter}\theta \mathrm{and}\mathrm{standard}\mathrm{normal}\mathrm{marginals}\mathrm{for}\mathrm{each}\mathrm{component}.$
Interleaved Moons	$\begin{array}{c}(X_0,L_X)=\mathrm{make}\text{\_}\mathrm{moos}\left(\right),(Y_0,L_Y)=\mathrm{make}\text{\_}\mathrm{moos}\left(\right)\hfill \\ \mathrm{For}\mathrm{each}\mathrm{sample}\mathrm{i}:\hfill \\ X_i^{(1,2)}={\left(X_0\right)}_i\hfill \\ Y_i^{(1,2)}\sim \mathrm{Uniform}\{{\left(Y_0\right)}_j\mid {\left(L_Y\right)}_j\ne {\left(L_X\right)}_i\}\hfill \\ X_i^{(3:d)},Y_i^{(3:d)}\sim \mathcal{N}(0,I_{d-2})\hfill \end{array}$

We used sklearn.datasets.make_moons.