The Gromov–Wasserstein Distance: A Brief Overview

Abstract

We recall the construction of the Gromov–Wasserstein distance and concentrate on quantitative aspects of the definition.

1. Introduction

Modeling datasets as metric spaces seems to be natural for some applications and concepts revolving around the Gromov–Hausdorff distance—a notion of distance between compact metric spaces—provide a useful language for expressing properties of data and shape analysis methods. In many situations, however, this is not enough, and one must incorporate other sources of information into the model, with “weights” attached to each point being one of them. This gives rise to the idea of representing data as metric measure spaces, which are metric spaces endowed with a probability measure. In terms of a distance, the Gromov–Hausdorff metric is replaced with the Gromov–Wasserstein metric.

1.1. Notation and Background Concepts

The book by Burago, et al. [1] is a valuable source for many concepts in metric geometry. We refer the reader to that book for any concepts not explicitly defined in these notes.

We let $\mathcal{M}$ denote the collection of all compact metric spaces and by ${\mathcal{M}}^{\mathrm{iso}}$ the collection of all isometry classes of $\mathcal{M}$. Recall that for a given metric space $(X,d_X)\in \mathcal{M}$, its diameter is defined as $\mathbf{diam}\left(X\right):={max}_{x,x^{\prime }\in X}{d}_X(x,x^{\prime }).$ Similarly, the radius of X is defined as $\mathbf{rad}\left(X\right):={min}_{x\in X}{max}_{x^{\prime }\in X}{d}_X(x,x^{\prime }).$

For a fixed metric space $(Z,d_Z)$, we let $d_{\mathcal{H}}^Z$ denote the Hausdorff distance between (closed) subsets of Z.

We will often refer to a metric space $(X,d_X)$ by only X, but the notation for the underlying metric will be implicitly understood to be $d_X$. Recall that a map $\phi :X\to Y$ between metric spaces $(X,d_X)$ and $(Y,d_Y)$ is an isometric embedding if $d_Y(\phi \left(x\right),\phi \left(x^{\prime }\right))=d_X(x,x^{\prime })$ for all $x,x^{\prime }\in X$. The map φ is an isometry if it is a surjective isometric embedding.

Recall that given measurable spaces $(X,{\Sigma }_X)$ and $(Y,{\Sigma }_Y)$, a measure μ on $(X,{\Sigma }_X)$ and a measurable map $f:X\to Y$, the push-forward measure $f_{\#}\mu$ on $(Y,{\Sigma }_Y)$ acts according to $f_{\#}\mu \left(B\right)=\mu \left(f^{-1}\left(B\right)\right)$ for any $B\in {\Sigma }_Y$.

A metric measure space (mm-space for short) is a triple $(X,d_X,{\mu }_X)$ where $(X,d_X)$ is a compact metric space and ${\mu }_X$ is a Borel probability measure with full support: $\mathrm{supp}\left({\mu }_X\right)=X$. We denote by ${\mathcal{M}}^w$ the collection of all mm-spaces. An isomorphism between $X,Y\in {\mathcal{M}}^w$ is any isometry $\Psi :X\to Y$, such that ${\Psi }_{\#}{\mu }_X={\mu }_Y.$

2. The Gromov–Hausdorff Distance

One says that a subset $R\subset X\times Y$ is a correspondence between sets X and Y whenever ${\pi }_1\left(R\right)=X$ and ${\pi }_2\left(R\right)=Y,$ where ${\pi }_1:X\times Y\to X$ and ${\pi }_2:X\times Y\to Y$ are the canonical projections. Let $\mathcal{R}(X,Y)$ denote the set of all correspondences between X and Y.

The Gromov–Hausdorff (GH) distance between compact metric spaces $(X,d_X)$ and $(Y,d_Y)$ is defined as:

$$d_{\mathcal{G}\mathcal{H}}(X,Y):=\frac{1}{2}\underset{R}{inf}\underset{(x,y),(x^{\prime },y^{\prime })\in R}{sup}|d_X(x,x^{\prime })-d_Y(y,y^{\prime })|$$

(1)

where R ranges over $\mathcal{R}(X,Y).$

Example 1. 

The GH distance between any compact metric space X and the space with exactly one point is equal to $\frac{1}{2}\mathbf{diam}\left(X\right).$

It turns out that $(\mathcal{M},d_{\mathcal{G}\mathcal{H}})$ is a nice space in that is has many compact subclasses.

Theorem 1. 

([1]) Let $N:[0,+\infty )\to \mathbb{N}$ be a bounded function and $D>0$. Let $\mathcal{F}(N,D)\subset \mathcal{M}$ be any family of compact metric spaces, such that $\mathbf{diam}\left(X\right)\le D$ for all $X\in \mathcal{F}(N,D)$, such that for any $\epsilon >0$, any $X\in \mathcal{F}(N,\epsilon )$ admits an ε-net with at most $N\left(\epsilon \right)$ elements. Then, $\mathcal{F}(N,D)$ is pre-compact in the Gromov–Hausdorff topology.

Example 2. 

An important example of families, such as the above, is given by those closed n-dimensional Riemannian manifolds $(X,g_X)\in \mathcal{M}(n,\kappa ,D)$ with the diameter bounded by $D>0$ and the Ricci curvature bounded below by κ.

Theorem 2 

([2]). The space $({\mathcal{M}}^{\mathrm{iso}},d_{\mathcal{G}\mathcal{H}})$ is complete.

It then follows from the two theorems above that classes $\mathcal{F}(N,D)$, such as above, are totally bounded for the Gromov–Hausdorff distance. This means that such classes are easy to organize in the sense of clustering or databases.

In many practical applications, one would like to take into account “weights” attached to points in a dataset. For example, the two metric spaces with the weights below are isometric, but not isomorphic in the sense that no isometry respects the weights:

The idea is that weights represent how much we trust a given “measurement” in practical applications. This leads to considering a more general collection of datasets and, in turn, an adapted notion of equality and a compatible metric over them. This naturally leads to regarding datasets as mm-spaces and then finding a notion of distance on ${\mathcal{M}}^w$ compatible with isomorphism of mm-spaces.

3. A Metric on ${\mathcal{M}}^w$

Let $(X,d_X,{\mu }_X)$ and $(Y,d_Y,{\mu }_Y)$ be two given mm-spaces. In our path to defining a distance between mm-spaces, we emulate the construction of the Gromov–Hausdorff distance and start by identifying a notion of correspondence between mm-spaces.

A probability measure μ over $X\times Y$ is called a coupling between ${\mu }_X$ and ${\mu }_Y$ if ${\left({\pi }_1\right)}_{\#}\mu ={\mu }_X$ and ${\left({\pi }_2\right)}_{\#}\mu ={\mu }_Y.$ We denote by $\mathcal{U}({\mu }_X,{\mu }_Y)$ the collection of all couplings between ${\mu }_X$ and ${\mu }_Y$.

Example 3. 

When $Y=\left\{p\right\}$, ${\mu }_Y={\delta }_p$, and thus, there is a unique coupling between X and Y: $\mathcal{U}({\mu }_X,{\mu }_Y)=\{{\mu }_X\otimes {\delta }_p\}.$

Example 4. 

Consider for example the spaces with two points each that we depicted above. In that case, ${\mu }_X$ can be identified with the vector $(\frac{1}{2},\frac{1}{2})$ and ${\mu }_Y$ with the vector $(\frac{3}{4},\frac{1}{4}).$ In this case, one sees that the matrix:

$$\left[\begin{array}{cc}\frac{1}{4}& \frac{1}{2}\\ \frac{1}{4}& 0\end{array}\right]$$

induces a valid coupling.

Now, given $p\ge 1$, consider the function

$$(x,y,x^{\prime },y^{\prime })\mapsto |d_X(x,x^{\prime })-d_Y(y,y^{\prime }){|}^p$$

and pick any $\mu \in \mathcal{U}({\mu }_X,{\mu }_Y).$ One then integrates this function against the measure $\mu \otimes \mu$ and infimizes over the choice of $\mu \in \mathcal{U}({\mu }_X,{\mu }_Y)$ to define the Gromov–Wasserstein distance of order p [3]:

$$d_{\mathcal{G}\mathcal{W},p}(X,Y):=\frac{1}{2}\underset{\mu }{inf}{\left(\int \int |d_X(x,x^{\prime })-d_Y(y,y^{\prime }){|}^p\mu (dx\times dy)\mu (d{x}^{\prime }\times d{y}^{\prime })\right)}^{1/p}$$

Remark 1. 

This is an $L^p$ analogue of Equation (1).

Theorem 3 

([3]). The Gromov–Wasserstein distance of order $p\ge 1$ defines a proper distance on the collection of isomorphism classes of mm-spaces.

By standard compactness arguments, one can prove that the infimum above is always attained [3]. Let ${\mathcal{U}}_p^{\mathrm{opt}}(X,Y)$ denote the set of all the couplings in $\mathcal{U}({\mu }_X,{\mu }_Y)$ that achieve the minimum. The structure of the former set depends not only on ${\mu }_X$ and ${\mu }_Y$, but also on $d_X$, $d_Y$ and p.

Example 5. 

Consider the mm-space with exactly one point: $(\{*\},\left(0\right),{\delta }_{*})$. Then,

$$d_{\mathcal{G}\mathcal{W},p}(X,\{*\})=\frac{1}{2}{\left(\int \int {\left(d_X(x,x^{\prime })\right)}^p{\mu }_X\left(dx\right){\mu }_X\left(d{x}^{\prime }\right)\right)}^{1/p}$$

and we define ${\mathbf{diam}}_p\left(X\right)$—the p-statistical diameter of X—as twice the right-hand side. Notice that ${lim}_{p\to \infty }{\mathbf{diam}}_p\left(X\right)$ is equal to the usual diameter of X (as a metric space).

Question 1. 

To what extent are we able to replicate the nice properties of $(\mathcal{M},d_{\mathcal{G}\mathcal{H}})$ in the context of $({\mathcal{M}}^w,d_{\mathcal{G}\mathcal{W},p})$? In particular, it is of interest to investigate whether this new space of datasets is complete and whether one can easily identify rich pre-compact classes.

3.1. Pre-Compactness

Theorem 4 

([3]). For a function non-decreasing $\rho :[0,\infty )\to [0,1]$, such that $\rho \left(\epsilon \right)>0$ for $\epsilon >0$ and $D>0$, let ${\mathcal{F}}^w(\rho ,D)\subset {\mathcal{M}}^w$ denote the set of all mm-spaces X, such that $\mathbf{diam}\left(X\right)\le D$ and ${inf}_x{\mu }_X\left(B_{\epsilon }\left(x\right)\right)\ge \rho \left(\epsilon \right)$ for all $\epsilon >0$. Then, ${\mathcal{F}}^w(\rho ,D)$ is pre-compact for the Gromov–Wasserstein topology, for any $p\ge 1$.

Remark 2. 

Recall Example 2, where closed n-dimensional Riemannian manifolds were regarded as metric spaces. One can, all the same, embed closed Riemannian manifolds into ${\mathcal{M}}^w$ via $(X,g_X)\mapsto (X,d_X,{\mu }_X)$, where $d_X$ is the geodesic distance induced by the metric tensor $g_X$ and ${\mu }_X$ stands for the normalized volume measure on X. It is well known [4] that for $\epsilon >0$ small, ${\mu }_X\left(B_{\epsilon }\left(x\right)\right)=\frac{c_n}{\mathrm{vol}\left(X\right)}{\epsilon }^n\left(1-\frac{s_X\left(x\right)}{6(n+1)}{\epsilon }^2+O\left({\epsilon }^4\right)\right)$, where $s_X\left(x\right)$ is the scalar curvature of X at x, and $\mathrm{vol}\left(X\right)$ is the total volume of X. Thus, a lower bound on ${\mu }_X\left(B_{\epsilon }\left(x\right)\right)$ plays the role of a proxy for an upper bound on curvature.

3.2. Completeness

The Space ${\mathcal{M}}^w$ with any p-Gromov–Wasserstein distance is not complete. Indeed, consider the following family of mm-spaces: ${\Delta }_n\in {\mathcal{M}}^w$, where ${\Delta }_n$ consists of $n\in \mathbb{N}$ points at distance one from each other, and all with weights $1/n$.

Claim 1. 

For all $n,m\ge 1$, $d_{\mathcal{G}\mathcal{W},p}({\Delta }_n,{\Delta }_m)\le \frac{1}{2}\left(n^{-1/p}+m^{-1/p}\right)$.

The claim will follow from the following claim and triangle inequality for $d_{\mathcal{G}\mathcal{W},p}$:

Claim 2. 

For all $n,m\ge 1$, $d_{\mathcal{G}\mathcal{W},p}({\Delta }_n,{\Delta }_{n·m})\le \frac{1}{2}{n}^{-1/p}.$

In order to verify the claim, we denote by $\{x_1,x_2,\dots ,x_n\}$ the points of ${\Delta }_n$ and label the points in ${\Delta }_{n·m}$ by $\{y_{11},\dots ,y_{1m},y_{21},\dots ,y_{2m},\dots \dots ,y_{n1},\dots ,y_{nm}\}.$ Consider the following coupling between ${\mu }_n$ and ${\mu }_{n·m}$, the reference measures on ${\Delta }_n$ and ${\Delta }_{n·m}$:

$$\mu (x_i,y_{kj}):=\frac{1}{n·m}{\delta }_{ik},\mathrm{for}\mathrm{all}i,k\in \{1,\dots ,n\}\mathrm{and}j\in \{1,\dots ,m\}$$

It is clear that this defines a valid coupling between ${\mu }_n$ and ${\mu }_{n·m}.$

Now, note that

$$\begin{array}{cc}\hfill J\left(\mu \right):=& \sum _{i,i^{\prime }}\sum _{(k,j),(k^{\prime },j^{\prime })}{|d_{{\Delta }_n}(x_i,x_{i^{\prime }})-d_{{\Delta }_{n·m}}(y_{kj},y_{k^{\prime }{j}^{\prime }})|}^p\mu (x_i,y_{kj})\mu (x_{i^{\prime }},y_{k^{\prime }{j}^{\prime }})\hfill \\ \hfill =& \frac{1}{{(n·m)}^2}\sum _{i,i^{\prime }}\sum _{j,j^{\prime }}{|d_{{\Delta }_n}(x_i,x_{i^{\prime }})-d_{{\Delta }_{n·m}}(y_{ij},y_{i^{\prime }{j}^{\prime }})|}^p\hfill \\ \hfill =& \frac{1}{{(n·m)}^2}\sum _i\sum _{j,j^{\prime }}{|1-{\delta }_{j{j}^{\prime }}|}^p\hfill \\ \hfill =& \frac{m-1}{n·m}\hfill \\ \hfill \le & n^{-1}\hfill \end{array}$$

Now, by definition, $d_{\mathcal{G}\mathcal{W},p}({\Delta }_m,{\Delta }_{n·m})\le \frac{1}{2}{\left(J\left(\mu \right)\right)}^{1/p}$, so the claim follows.

Claim 1 indicates that ${\left\{{\Delta }_n\right\}}_{n\in \mathbb{N}}$ constitutes a Cauchy sequence in ${\mathcal{M}}^w$. However, a potential limit object for this sequence will have countably infinitely many points at distance one from each other. This space is not compact, thus $d_{\mathcal{G}\mathcal{W},p}$ is not a complete metric.

3.3. Other Properties: Geodesics and Alexandrov Curvature

Very recently, Sturm [5] pointed out that ${\mathcal{M}}^w$ is a geodesic space when endowed with any $d_{\mathcal{G}\mathcal{W},p}$, $p\ge 1$. This means that given any two spaces $X_0,X_1$ in ${\mathcal{M}}^w$, one can find a curve $[0,1]\ni t\mapsto X_t\in {\mathcal{M}}^w$, such that $d_{\mathcal{G}\mathcal{W},p}(X_t,X_s)=|t-s|d_{\mathcal{G}\mathcal{W},p}(X_0,X_1),s,t\in [0,1].$

Proposition 1 

([5]). For each $p\ge 1$, the space $({\mathcal{M}}^w,d_{\mathcal{G}\mathcal{W},p})$ is geodesic. Furthermore, for $p>1$, the following curves on ${\mathcal{M}}^w$ define geodesics between $(X_0,d_0,{\mu }_0)$ and $(X_1,d_1,{\mu }_1)$ in ${\mathcal{M}}^w$:

$$[0,1]\ni t\mapsto (X_0\times X_1,d_t,\mu )$$

where $d_t\left((x_0,x_1),(x_0^{\prime },x_1^{\prime })\right):=(1-t)d_0(x_0,x_0^{\prime })+t{d}_1(x_1,x_1^{\prime })$ for $(x_0,x_1),(x_0^{\prime },x_1^{\prime })\in X_0\times X_1$ and $\mu \in {\mathcal{U}}_p^{\mathrm{opt}}(X,Y).$ Furthermore, for $p>1$, all geodesics are of this form.

Sturm further proved that the completion $\overline{{\mathcal{M}}^w}$ of the space ${\mathcal{M}}^w$ with metric $d_{\mathcal{G}\mathcal{W},2}$ satisfies:

Theorem 5 

([5]). The metric space $\left(\overline{{\mathcal{M}}^w},d_{\mathcal{G}\mathcal{W},2}\right)$ is an Alexandrov space of curvature $\ge 0$.

Amongst the consequences of this property is the fact that one can conceive of gradient flows on the space of all mm-spaces [5].

3.4. The Metric $d_{\mathcal{G}\mathcal{W},p}$ in Applications

Applications of the notion of Gromov-Wasserstein distance arise in shape and data analysis. In shape analysis, the main application is shape matching under invariances. Many easily computable lower bounds for the GW distance have been discussed in [3,6]. All of them lead to solving linear programming optimization problems (for which there are polynomial time algorithms) or can be computed via elicit formulas. As an example, consider the following invariant of an mm-space $(X,d_X,{\mu }_X)$:

$$H_X:[0,\infty )\to [0,1],t\mapsto {\mu }_X\otimes {\mu }_X\left(\{(x,x^{\prime })|d_X(x,x^{\prime })\le t\}\right)$$

This invariant simply encodes the distribution of pairwise distances on the dataset X, and it is defined by analogy with the so-called shape distributions that are well known in computer graphics [7]. Then, one has:

Proposition 2 

([3,6]). Let $X,Y\in {\mathcal{M}}^w$ be any two mm-spaces and $p\ge 1$. Then,

$$d_{\mathcal{G}\mathcal{W},p}(X,Y)\ge \frac{1}{2}{\int }_0^{\infty }|H_X\left(t\right)-H_Y\left(t\right)|dt$$

Remark 3. 

This invariant is also related to the work of Boutin and Kemper [8] and Brinkman and Olver [9].

Other lower bounds which can be computed in time polynomial in the number of points in the underlying mm-spaces have been reported in [3]. As a primary example, one has that the local shape distributions of shapes provide a lower bound which is strictly stronger than the ones in the Proposition above. In more detail, consider for a given mm-space $(X,d_X,{\mu }_X)$ the invariant:

$$h_X:X\times [0,\infty )\to [0,1],(x,t)\mapsto {\mu }_X\left(\overline{B_t\left(x\right)}\right).$$

Then, for mm-spaces X and Y consider the cost function $c_{X,Y}:X\times Y\to {\mathbb{R}}^{+}$ given by:

$$c_{X,Y}(x,y):={\int }_0^{\infty }|h_X(x,t)-h_Y(y,t)|dt.$$

One then has:

Proposition 3 

([3,6]). Let $X,Y\in {\mathcal{M}}^w$ be any two mm-spaces and $p\ge 1$. Then,

$$d_{\mathcal{G}\mathcal{W},p}(X,Y)\ge \frac{1}{2}\underset{\mu }{inf}\int \int c_{X,Y}(x,y)\mu (dx\times dy),$$

where μ ranges in $\mathcal{U}({\mu }_X,{\mu }_Y).$

Remark 4. 

Solving for the infimum above leads to a mass transportation problem for which there exists efficient linear programming techniques.

Remark 5. 

It is possible to define a notion of spectral Gromov-Wasserstein distance which operates at the level of compact Riemannian manifolds without boundaries, and is based con the comparison of heat-kernels. This notion permits inter-relating many pre-existing shape matching methods and suggests some others [12].

4. Discussion and Outlook

The Gromov–Hausdorff distance offers a useful language for expressing different tasks in shape and data analysis. Its origins are in the work of Gromov on synthetic geometry. For finite metric spaces, the Gromov–Hausdorff distance leads to solving NP-hard combinatorial optimization problems. Related to construction is Gromov–Wasserstein distances that operate on metric measure spaces [3,10]. In contrast to the Gromov–Hausdorff distance, the computation of Gromov–Wasserstein distances leads to solving quadratic optimization problems on continuous variables. The space of all metric measures spaces endowed with a certain variant of the Gromov–Wasserstein distance [3] enjoys nice theoretical properties [5]. It seems of interest to develop provably correct approximations to these distances when restricted to some suitable subclasses of finite metric spaces. Other aspects of the Gromov–Wasserstein distance are discussed in [3,5,10,11,12].