# Matching the LBO Eigenspace of Non-Rigid Shapes via High Order Statistics

^{*}

## Abstract

**:**

## 1. Introduction

- Eigenfunctions are defined up to a sign.
- The order of the eigenfunctions, especially those representing higher frequencies, is not repeatable across shapes.
- The eigenvalues of the Laplace–Beltrami operator may have a multiplicity greater than one, with several eigenfunctions corresponding to each such eigenvalue.
- It is generally impossible to expect that an eigenfunction with a large eigenvalue of one shape will correspond to any eigenfunction of another shape.
- Intrinsic symmetries introduce self-ambiguity, adding complexity to the sign estimation challenge.

#### 1.1. Related Work

#### 1.2. Background

#### 1.2.1. Laplace–Beltrami Eigendecomposition

#### 1.2.2. Diffusion Maps

#### 1.2.3. Multivariate Distribution Comparison

## 2. Eigenfunction Matching

#### 2.1. Problem Formulation

- The respective signs of the eigenfunctions $\mathbf{s}:{s}_{i}\in \{+1,-1\}$.
- The permutation vector π of the eigenfunctions: $\pi :\{1,2,...,N\}\mapsto \{1,2,...,N\}$.

#### 2.2. Matching Cost Function

#### 2.2.1. Overview

- ⋄
- Pointwise signatures as side information: We mix in stable compatible signatures, like the heat kernel signature (HKS), employed in ${C}^{S}$, ${C}_{\nabla}^{P}$, ${C}_{\nabla}^{P,S}$.
- ⋄
- Raw moments over segments: We blindly (i.e. , without correspondence) segment the shapes into parts in a compatible way and integrate over these segments separately; employed in ${C}_{\nabla}^{P}$, ${C}_{\nabla}^{P,S}$.

#### 2.2.2. Cost Function Terms

- $C(\mathbf{s},\mathit{\pi})={\displaystyle \sum _{i,j,k}}{({\mu}_{i,j,k}^{X}-{s}_{i}{s}_{j}{s}_{k}{\mu}_{\pi \left(i\right),\pi \left(j\right),\pi \left(k\right)}^{Y})}^{2}$
- ${\mu}_{i,j,k}=E\left[{\varphi}_{i}{\varphi}_{j}{\varphi}_{k}\right],\phantom{\rule{1.em}{0ex}}i,j,k\in \{1,2,...,N\}$

- ${C}_{\nabla}^{P}(\mathbf{s},\mathit{\pi})={\displaystyle \sum _{i,j,k,p}}{({\xi}_{i,j,k,p}^{X}-{s}_{i}{s}_{j}{s}_{k}{\xi}_{\pi \left(i\right),\pi \left(j\right),\pi \left(k\right),p}^{Y})}^{2}$
- ${\xi}_{i,j,k,p}=E[{\nu}_{i,j}{\varphi}_{k}{w}_{p}\left(\right|{\varphi}_{k}\left|\right)],$
- $i,j,k\in \{1,2,...,N\},\phantom{\rule{1.em}{0ex}}p\in \{1..P\}$
- ${\nu}_{i,j}=({\nabla}_{G}{\varphi}_{i}\times {\nabla}_{G}{\varphi}_{j})\xb7\mathbf{n}$

- ${C}^{S}(\mathbf{s},\mathit{\pi})=N{\displaystyle \sum _{i,q}}{({\mu}_{i,q}^{X,S}-{s}_{i}{\mu}_{\pi \left(i\right),q}^{Y,S})}^{2}$
- ${\mu}_{i,q}^{S}=E\left[{\varphi}_{i}{\psi}_{q}\right],\phantom{\rule{1.em}{0ex}}i\in \{1,2,...,N\},\phantom{\rule{1.em}{0ex}}q\in \{1,2,...Q\}$

- ${C}_{\nabla}^{P,S}(\mathbf{s},\mathit{\pi})={\displaystyle \sum _{i,q,k,p}}{({\xi}_{i,q,k,p}^{X,S}-{s}_{i}{s}_{k}{\xi}_{\pi \left(i\right),q,\pi \left(k\right),p}^{Y,S})}^{2}$
- ${\xi}_{i,q,k,p}^{S}=E[{\nu}_{i,q}^{S}{\varphi}_{k}{w}_{p}\left(\right|{\varphi}_{k}\left|\right)],$
- $i,k\in \{1..N\},p\in \{1..P\},q\in \{1..Q\}$
- ${\nu}_{i,q}^{S}=({\nabla}_{G}{\varphi}_{i}\times {\nabla}_{G}{\psi}_{q})\xb7\mathbf{n}$

- ${\varphi}_{i}$ are the eigenfunctions of the Laplace–Beltrami operator $-{\Delta}_{G}{\varphi}_{i}={\lambda}_{i}{\varphi}_{i}$.
- ${w}_{p}:{\mathbb{R}}_{\ge 0}\mapsto [0,1]$ are nonlinear weighting functions.
- ${\psi}_{q}:M\mapsto \mathbb{R}$ are the components of an external point signature.
- ${\nabla}_{G}$ is the gradient induced by the metric tensor G.
- $E\left[z\right]={\int}_{M}zd{a}_{M}$, where $d{a}_{M}$ is the area element of the manifold M.
- $\mathbf{n}$ is the normal to the surface.
- × is the cross-product in ${\mathbb{R}}^{3}$, and · is the inner product in ${\mathbb{R}}^{3}$.
- The weighting parameter α determines the relative weight of the gradient cost functions.

- N: the number of eigenfunctions to be matched.
- ${\left\{{w}_{p}\right\}}_{p=1}^{P}$: the P nonlinear weighting functions.
- ${\left\{{\psi}_{q}\right\}}_{q=1}^{Q}$: the external point signature of size Q.
- α: the relative weight of the gradient cost functions.

#### 2.2.3. Resolving Sign Ambiguities and Permutations

#### 2.2.4. Resolving Antisymmetric Eigenfunctions

- The gradient $\nabla f$ of an antisymmetric eigenfunction f is not antisymmetric.
- The gradient is a linear operator. Consequently $\nabla (-f)=-\nabla f,\phantom{\rule{1.em}{0ex}}\forall f$.

#### 2.2.5. Raw Moments over Segments

#### 2.2.6. Pointwise Signatures As Side Information

#### 2.3. Solving the Minimization Problem

- Step 1: An initialization of ${\mathbf{s}}^{\mathbf{0}}$ is determined by ${s}_{i}=sign\left({\mu}_{i,i,i}^{X}{\mu}_{i,i,i}^{Y}\right)$ and ${\mathit{\pi}}^{\mathbf{0}}=[0,1,...N]$.
- Step 2: The permutation vector $\widehat{\pi}$ is found by minimizing $C(\mathbf{s},\mathit{\pi})+{C}^{S}(\mathbf{s},\mathit{\pi})$. We make an educated guess for the possible permutations, limiting the search for two permutation profiles:
- ⋄
- two consecutive eigenfunction switching (with possible sign change), i.e., $[{\pi}_{i},{\pi}_{j},{\pi}_{k},{\pi}_{l}]=[j,i,l,k],j=i+1,l=k+1$
- ⋄
- triplet permutation (with possible sign change), i.e., $[{\pi}_{i},{\pi}_{j},{\pi}_{k}]=[j,k,i]$ or $[k,i,j],\phantom{\rule{3.33333pt}{0ex}}j=k+1,i=j+1$;

- Step 3: The sign sequence is resolved again by minimizing $C(\mathbf{s},\mathit{\pi})+{C}^{S}(\mathbf{s},\mathit{\pi})$. In this step, all possible quadruple sign changes are checked, setting the permutation vector found in Step 2. If the cost function was decreased in Step 2 or Step 3, then return to Step 2. While finding the optimal sign sequence and permutation vector, we keep a list of all possible good sign sequences for the next step.
- Step 4: The optimal sign sequence $\widehat{\mathbf{s}}$ is found by comparing the entire cost function $C(\mathbf{s},\mathit{\pi})+{C}^{S}(\mathbf{s},\mathit{\pi})+\alpha ({C}_{\nabla}^{P}(\mathbf{s},\mathit{\pi})+{C}_{\nabla}^{P,S}(\mathbf{s},\mathit{\pi}))$ for each sign sequence in the list created in Step 3.

## 3. Results

- The matched low order eigenfunctions that represent the global structure of the shapes.
- The heat kernel signature (HKS) derivative (Equation (A11)) that, being a bandpass filter, expresses more local features.

- The global point signature (GPS) kernel (Equation (A12)), propagating the correspondence of each feature point.
- The heat kernel signature (HKS) derivative (Equation (A11)), as before.

**Figure 1.**The eigenfunction matching of two nearly isometric shapes. Hot and cold colors represent positive and negative values, respectively. (

**Top**) The first pose of a dog; (

**center**) the second pose of a dog; (

**bottom**) the second pose of a dog after the matching algorithm.

**Figure 3.**Eigenfunction matching of two nearly isometric shapes. Hot and cold colors represent positive and negative values, respectively. (

**Top**) The first pose of a human; (

**center**) the second pose of a human; (

**bottom**) the second pose of a human after the matching algorithm.

**Figure 5.**Eigenfunction matching of two nearly isometric shapes. Hot and cold colors represent positive and negative values, respectively. (

**Top**) The first pose of a horse; (

**center**) the second pose of a horse; (

**bottom**) the second pose of a horse after the matching algorithm.

- Matched eigenfunctions: the method proposed in this paper.
- Blended: the method proposed by Kim et al. [24] that uses a weighted combination of isometric maps.
- Best conformal: the least-distortive conformal map roughly describes what is the best performance achieved by a single conformal map without blending.
- Möbius voting: the method proposed by Lipman et al. [25] counts votes on the conformal Möbius transformations.
- Heat kernel matching (HKM) with one correspondence: This method is based on matching features in a space of a heat kernel for a given source point, as described in [26]. A full map is constructed from a single correspondence, which is obtained by searching a correspondence that gives the most similar heat kernel maps. We use the results shown in [24].

**Figure 7.**Evaluation of the proposed correspondence framework applied to shapes from the TOSCA database, using the protocol of [24].

**Figure 8.**Evaluation of the proposed correspondence framework applied to shapes from the SCAPE database, using the protocol of [24].

## 4. Conclusions

## Acknowledgments

## Author Contributions

## Appendix

## A1. Discretization

#### A1.1. Laplace–Beltrami Eigendecomposition

#### A1.2. Gradient

## A2. Application Specific Parameters

- We matched the first $N=8$ eigenfunctions from one shape to the first 10 eigenfunctions of the other shape.
- Soft thresholding was used to define $P=2$ nonlinear weighting functions ${w}_{p}$:$$\begin{array}{cc}& {w}_{0}\left(z\right)=\left\{\begin{array}{cc}0,\hfill & \mathrm{if}\left|z\right|<\mathrm{TH}\hfill \\ 1,\hfill & \mathrm{if}\left|z\right|>2\mathrm{TH}\hfill \\ \left(\right|z|-\mathrm{TH})/\mathrm{TH}\hfill & \mathrm{otherwise}\phantom{\rule{1.em}{0ex}}\phantom{\rule{1.em}{0ex}}\hfill \end{array}\right.\hfill \\ & {w}_{1}\left(z\right)=1-{w}_{0}\left(z\right)\hfill \end{array}$$
- For generating the external pointwise signature ${\psi}_{q}$, the heat kernel signature (HKS) was used [21]. In the approximation of the heat kernel signature ${\text{HKS}}_{t}\left(x\right)={\displaystyle \sum _{i=1}^{h}}{e}^{-{\lambda}_{i}t}{\varphi}_{i}^{2}\left(x\right)$, we used $h=120$ eigenfunctions. We used a bandpass filter form of the HKS by taking the derivative of the heat kernel signature. The HKS derivative was logarithmically sampled $Q=6$ times at $t={t}_{q},\phantom{\rule{1.em}{0ex}}q=1,2,...Q$, with ${t}_{1}=\frac{1}{50{\lambda}_{1}}$ and ${t}_{Q}=\frac{1}{{\lambda}_{1}}$. ${\psi}_{q}$ were normalized according to the inner product over the manifold.$$\begin{array}{cc}& {\psi}_{q}\left(x\right)=\frac{{\tilde{\psi}}_{q}\left(x\right)}{\sqrt{{\int}_{M}{\tilde{\psi}}_{q}^{2}\left(\tilde{x}\right)da\left(\tilde{x}\right)}}\hfill \\ & {\tilde{\psi}}_{q}\left(x\right)=\frac{\partial}{\partial t}{\text{HKS}}_{t}\left(x\right)\phantom{\rule{1.em}{0ex}}\mathrm{sampled}\mathrm{at}\phantom{\rule{1.em}{0ex}}t={t}_{q}\hfill \\ & \frac{\partial}{\partial t}{\text{HKS}}_{t}\left(x\right)={\displaystyle \sum _{i=1}^{h}}-{\lambda}_{i}{e}^{-{\lambda}_{i}t}{\varphi}_{i}^{2}\left(x\right)\hfill \end{array}$$
- For propagating the correspondence of each feature point p, the global pointwise signature (GPS) kernel was used [6].$$\text{GPS}(x,p)={\displaystyle \sum _{i=1}^{h}}\frac{1}{{\lambda}_{i}}{\varphi}_{i}\left(x\right){\varphi}_{i}\left(p\right)$$
- The relative weight parameter α was set by balancing the influence of the terms of the cost function.$$\alpha =\frac{{\displaystyle \sum _{i,j,k}}{\left({\mu}_{i,j,k}^{X}\right)}^{2}+N{\displaystyle \sum _{i,q}}{\left({\mu}_{i,q}^{X,S}\right)}^{2}}{{\displaystyle \sum _{i,j,k,p}}{\left({\xi}_{i,j,k,p}^{X}\right)}^{2}+{\displaystyle \sum _{i,q,k,p}}{\left({\xi}_{i,q,k,p}^{X,S}\right)}^{2}}$$

## Conflicts of Interest

## References

- Elad, A.; Kimmel, R. Pattern Analysis and Machine Intelligence. IEEE Trans. Bend. Invariant Signat. Surf.
**2003**, 25, 1285–1295. [Google Scholar] - Kimmel, R.; Sethian, J.A. Computing geodesic paths on manifolds. Proc. Natl. Acad. Sci. USA
**1998**, 95, 8431–8435. [Google Scholar] [CrossRef] [PubMed] - Bérard, P.; Besson, G.; Gallot, S. Embedding Riemannian manifolds by their heat kernel. Geom. Funct. Analy. GAFA
**1994**, 4, 373–398. [Google Scholar] [CrossRef] - Hausdorff, F. Set Theory; American Mathematical Society: Providence, RI, USA, 1957; Volume 119. [Google Scholar]
- Coifman, R.R.; Lafon, S.; Lee, A.B.; Maggioni, M.; Nadler, B.; Warner, F.; Zucker, S.W. Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps. Proc. Natl. Acad. Sci. USA
**2005**, 102, 7426–7431. [Google Scholar] [CrossRef] [PubMed] - Rustamov, R.M. Laplace–Beltrami eigenfunctions for deformation invariant shape representation. In Proceedings of the fifth Eurographics symposium on Geometry processing, Barcelona, Spain, 4–6 July 2007.
- Shapiro, L.S.; Michael, B.J. Feature-based correspondence: An eigenvector approach. Image Vis. Comput.
**1992**, 10, 283–288. [Google Scholar] [CrossRef] - Jain, V.; Zhang, H.; van Kaick, O. Non-rigid spectral correspondence of triangle meshes. Int. J. Shape Model.
**2007**, 13, 101–124. [Google Scholar] [CrossRef] - Umeyama, S. An eigendecomposition approach to weighted graph matching problems. IEEE Trans. Pattern Anal. Mach. Intell.
**1988**, 10, 695–703. [Google Scholar] [CrossRef] - Mateus, D.; Cuzzolin, F.; Horaud, R.; Boyer, E. Articulated shape matching using locally linear embedding and orthogonal alignment. In Proceedings of the IEEE 11th International Conference (ICCV 2007), Rio de Janeiro, Brazil, 14–20 October 2007.
- Mateus, D.; Horaud, R.; Knossow, D.; Cuzzolin, F.; Boyer, E. Articulated shape matching using laplacian eigenfunctions and unsupervised point registration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2008), Anchorage, AK, USA, 23–28 June 2008.
- Knossow, D.; Sharma, A.; Mateus, D.; Horaud, R. Inexact matching of large and sparse graphs using laplacian eigenvectors. Gr. Based Represent. Pattern Recognit.
**2009**, 5534, 144–153. [Google Scholar] - Dubrovina, A.; Kimmel, R. Approximately isometric shape correspondence by matching pointwise spectral features and global geodesic structures. Adv. Adapt. Data Anal.
**2011**, 3, 203–228. [Google Scholar] [CrossRef] - Kovnatsky, A.; Bronstein, A.M.; Bronstein, M.M.; Glashoff, K.; Kimmel, R. Coupled quasi-harmonic bases. Comput. Gr. Forum
**2013**, 32. [Google Scholar] [CrossRef] - Pokrass, J.; Bronstein, A.M.; Bronstein, M.M.; Sprechmann, P.; Sapiro, G. Sparse Modeling of Intrinsic Correspondences. Eurogr. Comput. Gr. Forum
**2013**, 32. [Google Scholar] [CrossRef] - Taubin, G. A signal processing approach to fair surface design. In Proceedings of the 22nd annual conference on Computer graphics and interactive techniques. (ACM), Los Angeles, CA, USA, 6–11 August 1995; pp. 351–358.
- Lévy, B. Laplace-Beltrami Eigenfunctions Towards an Algorithm That Understands Geometry. In Proceedings of the IEEE International Conference on Shape Modeling and Applications 2006, Washington, DC, USA, 14–16 June 2006.
- Kullback, S.; Leibler, R.A. On information and sufficiency. Ann. Math. Stat.
**1951**, 22, 79–86. [Google Scholar] [CrossRef] - Lin, J. Divergence measures based on the Shannon entropy. IEEE Trans. Inf. Theory
**1991**, 37, 145–151. [Google Scholar] [CrossRef] - Raviv, D.; Bronstein, A.M.; Bronstein, M.M.; Kimmel, R. Symmetries of non-rigid shapes. In IEEE 11th International Conference on Computer Vision (ICCV 2007), Rio de Janeiro, Brazil, 14–20 October 2007.
- Sun, J.; Ovsjanikov, M.; Guibas, L. A Concise and Provably Informative Multi-Scale Signature Based on Heat Diffusion. Comput. Gr. Forum.
**2009**, 28, 1383–1392. [Google Scholar] [CrossRef] - Bronstein, A.M.; Bronstein, M.; Kimmel, R. Numerical Geometry of Non-Rigid Shapes; Springer: Berlin, Germany, 2008. [Google Scholar]
- Anguelov, D.; Srinivasan, P.; Pang, H.C.; Koller, D.; Thrun, S.; Davis, J. The correlated correspondence algorithm for unsupervised registration of nonrigid surfaces. Adv. Neural Inf. Process. Syst.
**2005**, 17, 33–40. [Google Scholar] - Kim, V.G.; Lipman, Y.; Funkhouser, T. Blended intrinsic maps. ACM Trans. Gr.
**2011**, 30. [Google Scholar] [CrossRef] - Lipman, Y.; Funkhouser, T. Möbius voting for surface correspondence. ACM Trans. Gr.
**2009**, 28. [Google Scholar] [CrossRef] - Ovsjanikov, M.; Mérigot, Q.; Mémoli, F.; Guibas, L. One point isometric matching with the heat kernel. Comput. Gr. Forum
**2010**, 29, 1555–1564. [Google Scholar] [CrossRef] - Ovsjanikov, M.; Ben-Chen, M.; Solomon, J.; Butscher, A.; Guibas, L. Functional maps: A flexible representation of maps between shapes. ACM Trans. Gr.
**2012**, 31. [Google Scholar] [CrossRef] - Yang, C.; Medioni, G. Object modelling by registration of multiple range images. Image Vis. Comput.
**1992**, 10, 145–155. [Google Scholar] - Besl, P.J.; McKay, N.D. Method for registration of 3-D shapes. In Robotics-DL Tentative; Ihe International Society for Optics and Photonics: Bellingham, WA, USA, 1992. [Google Scholar]
- Aflalo, Y.; Raviv, D.; Kimmel, R. Scale Invariant Geometry for Non-Rigid Shapes; Technical Report for Society for Industrial and Applied Mathematics: Philadelphia, PA, USA, 2011. [Google Scholar]
- Ulrich, P.; Strasse, D.J.; Konrad, P. Computing Discrete Minimal Surfaces and Their Conjugates. Exp. Math.
**1993**, 2, 15–36. [Google Scholar] - Meyer, M.; Desbrun, M.; Schröder, P.; Barr, A.H. Discrete differential-geometry operators for triangulated 2-manifolds. Vis. Math.
**2002**, 3, 34–57. [Google Scholar]

© 2014 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

## Share and Cite

**MDPI and ACS Style**

Shtern, A.; Kimmel, R.
Matching the LBO Eigenspace of Non-Rigid Shapes via High Order Statistics. *Axioms* **2014**, *3*, 300-319.
https://doi.org/10.3390/axioms3030300

**AMA Style**

Shtern A, Kimmel R.
Matching the LBO Eigenspace of Non-Rigid Shapes via High Order Statistics. *Axioms*. 2014; 3(3):300-319.
https://doi.org/10.3390/axioms3030300

**Chicago/Turabian Style**

Shtern, Alon, and Ron Kimmel.
2014. "Matching the LBO Eigenspace of Non-Rigid Shapes via High Order Statistics" *Axioms* 3, no. 3: 300-319.
https://doi.org/10.3390/axioms3030300