Three-dimensional models are ubiquitous data in the form of 3D surface meshes, point clouds, volumetric data, etc. in a wide variety of domains such as material and mechanical engineering [1
], genetics [2
], molecular biology [3
], entomology [4
], and dentistry [5
], to name a few. Processing such large datasets (e.g., shape retrieval, matching, or recognition) is computationally expensive and memory intensive. For example, to query against a large database of 3D models to find the closest match for a 3D model of interest, one needs to develop an appropriate similarity measure as well as an efficient algorithm for search and retrieval. Shape descriptors assist with the example problem by providing discriminating feature vectors for shape retrieval [7
] and play a fundamental role when dealing with shape analysis problems such as shape matching [9
] and classification [11
In general, there are two types of shape descriptors: local descriptors, also called point signatures, and global descriptors, referred to as shape fingerprints. A local shape descriptor computes a feature vector for every point of a 3D model. On the other hand, a global shape descriptor represents the whole 3D shape model in the form of a low-dimension vector. A descriptor that is informative and concise captures as much information as possible from the 3D shape including the geometric and topological features. Such a vector drastically lowers the shape analysis burdens in terms of both computational intensity and memory.
While many successful non-spectral shape descriptors have been proposed in the literature, spectral descriptors have proved to be beneficial in many applications [12
]. The spectral methods take advantage of eigenvalues and eigenvectors computed from the eigen-decomposition of the Laplace–Beltrami (LB) operator applied on the surface of 3D shapes. These methods have found successful applications in graph processing [14
], computational biology [15
], and point-to-point correspondence [16
]. Several comprehensive surveys [17
] have been conducted on studying and classifying data-driven 3D shape descriptors to which we refer interested readers for more information. Our objective in this paper is to develop a concise and informative global, spectral shape descriptor.
One of the first spectral descriptors introduced to the computer graphics community is Shape-DNA, developed by Reuter et al. in 2006 [20
]. Shape-DNA attracted a great deal of attention for its unique isometric and rotation invariant features [20
]. Since then, several local as well as global shape descriptors have been introduced in accordance with Shape-DNA such as Heat Kernel Signature (HKS) [21
], Wave Kernel Signature (WKS) [22
], and Global Point Signature (GPS) [23
]. The common ground between these methods is the discretization approach used to solve the Laplacian eigenvalue problem, which uses a cotangent weighting scheme
along with area normalization.
Although there are many advantages of using variations of the cotangent scheme, there are several limitations. First, by their nature, they are limited to triangulated meshes. Second, they do not perform well when dealing with degenerate and non-uniform sampled meshes [24
]. In addition, their convergence error depends on factors such as the linearity of the function on the surface [25
]. One possible approach to address these limitations is through the use of manifold learning, which is investigated in the current contribution.
Nonlinear dimensionality reduction techniques, known as manifold learning
, assume the existence of a low-dimensional space, in which, a high-dimensional manifold can be represented without much loss of information [26
]. Similar to global descriptors, manifold learning methods attempt to learn the geometry of a manifold in order to extract a low dimensional vector of features that is informative and discriminative. However, unlike shape descriptors, the number of dimensions of a space does not confine manifold learning methods. To the best of our knowledge, the application of manifold learning, an active research topic in statistics and machine learning, has not been investigated in the computer graphics community for extracting global shape descriptors. This motivates the primary aim of this research, which is to explore the effectiveness of a manifold learning method, more specifically Laplacian Eigenmap
], in representing a 3D model with a low-dimensional vector. Our work introduces a novel Laplacian Eigenmap-based global shape descriptor and provides a straightforward normalization method that significantly outperforms existing spectral approaches.
In our first contribution
, inspired by the idea of Laplacian Eigenmaps [27
], we learned the manifold of a 3D model and then, analogous to the approach taken by Shape-DNA, used the spectrum of the embedded manifold to build the global shape descriptor. This approach has two main advantages. First, it relies on the adjacency of the nodes, disregarding the fine details of the mesh structure. Therefore, it can be used for degenerate or non-uniform sampled meshes. Second, as manifold learning does not rely on the mesh structure and is not limited to a specific type of meshes, e.g., triangulated meshes, it can be applied easily to any other mesh types such as quadrilateral meshes.
In our second contribution
, we presented a simple and straightforward normalization technique (motivated by the work in [20
]) to obtain a scale-invariant global shape descriptor that is more robust to noise. To this end, we propose to subtract the first non-zero eigenvalue from the shape descriptor after taking the logarithm of the spectrum. One advantage of our approach over the idea of Bronstein et al. [28
] is that we avoid taking the direct derivative; this advantage is significant since the differential operator amplifies the noise. Taking the logarithm additionally helps to suppress the effect of the noise that is present in higher order elements of the spectrum.
The remainder of this paper is organized as follows. In Section 2
, we briefly overview spectral shape analysis and manifold learning. Then, in Section 3
, we introduce the proposed shape descriptor along with some technical background. In Section 4
, the performance of the proposed method, as well as the robustness of the algorithm are examined and compared with multiple well-known shape descriptors by performing several qualitative and quantitative experiments using widely used 3D model datasets. Section 5
discusses the results in more detail and draws conclusions.
In this study, motivated by the unique properties of Laplacian Eigenmap (i.e., locality preservation, structural equivalence, and dimensionality reduction) and inspired by the existing spectral-based shape descriptors, we investigated the application of manifold learning in deriving a shape fingerprint in order to address the limitations tied to popular cotangent-based shape descriptors. We proposed a global descriptor (LESI) with an easy-to-compute and efficient normalization technique that facilitates applications such as shape classification and retrieval. Our method applies fewer restrictions on the class of meshes as well as improves the quality of tessellations. Analogous to other spectral descriptors, LESI uses the spectrum of the LB operator, which is independent of the shape location, informative (contains a considerable amount of geometrical and topological information), and above all isometric invariant. We compared the discriminating power of LESI with three prominent descriptors from the literature, namely Shape-DNA, cShape-DNA, and GPS, and found it to be superior.
In the first set of experiments illustrated in Figure 3
and Figure 5
, our method substantially outperforms the others. The superiority of LESI is more significant when the McGill dataset is used (Table 2
and Figure 6
). This dataset includes wide variations in mesh structure and scales, causing the failure of the other methods to generate acceptable results. However, LESI, due to utilizing a different method of discretization to form the LB operator, focuses on the vicinity rather than the quality of the triangulation. Therefore, our technique, unlike other methods, is not affected by the low quality of polygon meshes.
The second set of experiments evaluated the reliability of our method in the presence of noise, scale variations, as well as different sampling rates. LESI shows impressive robustness against the first two sets of perturbation. Despite the negative impact of down sampling in LESI descriptor, it continues to show better performance when compared to cShape-DNA and GPS. It should be noted that the result could also be improved by increasing the size of the output vector.
In addition to the discriminating power of the descriptor, degenerate and non-uniform meshes may also cause failure of an algorithm to converge. The cotangent weight-based algorithms were not able to compute the descriptors for two shapes from the McGill dataset. GPS also failed to compute descriptors for six models of the down sampled TOSCA dataset. However, our technique converges at all times despite the quality of the polygon mesh structure.
Moreover, LESI, unlike cotangent weight-based techniques, is not confined to the triangulated meshes as it disregards the mesh geometry [58
]. LESI inherits this property from the capability of manifold learning techniques in coping with high dimensional data. The discretization of the LB operator using cotangent weights on the quadrilateral meshes is not as straightforward as on triangular meshes. To compute the LB operator on a quadrilateral mesh, all rectangles need to be divided into triangles. It could be done easily; however, as for each quad there are two possible triangulations, thus the result is not unique.
The time needed to compute a descriptor, for all spectral-based descriptors discussed in this research, can be divided into two parts: (1) constructing the
matrices; and (2) solving the generalized eigenvalue problem. The computation time depends on n
(graph size) and d
(descriptor length). The computation cost increases quadratically with n
as does the size of matrices. Running the experiments on a desktop with system specifications mentioned on Section 4
, the average total time is 1.18 and 2.29 s with Shape-DNA and LESI algorithms as well as 0.41 and 0.90 s with Shape-DNA and LESI algorithms using TOSCA and McGill datasets, respectively. However, the Shape-DNA code for computing the first part is accelerated by using C MEX files, which makes it incomparable with our method. Therefore, we timed solving the generalized eigenvalue problem, reported as follows: on average it takes 1 and 0.97 s with Shape-DNA and LESI algorithms using TOSCA dataset (p
-value = 0.64, not statistically significant), as well as 0.35 and 0.40 s with Shape-DNA and LESI algorithms using McGill dataset (p
-value < 0.05, statistically significant). We compared our method with Shape-DNA as it is the basis of cotangent-based descriptors. Other cotangent-based methods have one or more extra steps, which add fraction of seconds. The time difference between these methods are negligible when descriptors are computed using a system with enough RAM, which is easily accessible in today’s world. Therefore, with respect to the computational cost, neither of these methods has advantage over the other.
In the original Laplacian Eigenmaps, the high dimension data require a considerable amount of processing as the list of all connections need to be computed for the dataset. In fact, for each point in the high dimension space, a given number of nearest neighbors need to be extracted which could be challenging and unmanageable. However, applying this technique to the 3D meshes, we skip this step as the neighbors are already defined and given in the mesh structure.
This work benefits from the Laplacian Eigenmap technique in a space in which the vicinities are given. LESI takes advantage of simple Laplacian computation, to form the LB operator, which provides concise and informative shape descriptors. Experimental results prove that LESI is more effective compared with the other powerful descriptors.
One limitation of LESI is related to the original Laplacian Eigenmap algorithm introduced by Belkin and Niyogi, in which the generalized eigenvalue problem was solved without specifying a boundary condition. It is not clear how the algorithm can handle manifolds with boundaries. Therefore, we limited our experiments to 3D shapes with closed manifolds. Not necessarily a limitation, but a matter of concern, is how the algorithm works in differentiating different models of men (David and Michael). While parameter t provides one degree of freedom for the algorithm, it is data-dependent and needs to be assigned carefully using cross-validation or the validation set approach. It accepts a wide range of values and no unique value is required.
Although we investigated only the application of Laplacian Eigenmap in introducing a shape descriptor, there are some other spectral-based manifold learning methods, such as Isomap, LLE, and Diffusion map, which have not been examined. Another possible future work can be investigating the possibility of extending the current contribution to other data modalities, e.g., classification of 2D images/sketches. One can construct a graph from a single 2D image by extracting features from all or sampled pixels. Such an approach benefits from the fact that there is no limit to the number of extracted features when using Laplacian Eigenmap.
In the end, even though the focus of this research has been on spectral shape descriptors, we would like to briefly discuss where our method can stand in comparison with learning-based methods. Recently, we have witnessed that deep learning methods have superseded traditional methods in many applications. Several deep learning techniques, (e.g., [59
]) have been introduced in this area. The process of training a neural network is very time consuming and requires high performance computing units, i.e., a GPU. However, they are capable of learning great deal of structural variations among objects within the same category. To achieve the goal of learning by seeing, they require a large and well-annotated sample set. In some applications, such as medical image analysis, obtaining a dataset with characteristics mentioned above is very challenging. In those cases, deep learning techniques may result in over-fitting. One approach to mitigate this issue is integrating independent feature sets, e.g., spectra of LB operator, with features extracted by learning methods. Recent research has shown that, in the case of small datasets, while spectral and deep-based methods may demonstrate similar behavior in terms of accuracy, their combination improves the total accuracy. In addition, as they can be computed in parallel, they do not constrain runtime execution. This brief comparison requires more investigation both experimentally and theoretically.