Skip to Content
  • Article
  • Open Access

30 June 2013

A Sketch-based 3D Garment Model Retrieval Algorithm

,
,
and
1
National Engineering Research Center of Digital Life, State-Province Joint Laboratory of Digital Home Interactive Applications, School of Information Science & Technology, Sun Yat-sen University, Guangzhou 510006, China
2
Shenzhen Digital Home Key Technology Engineering Laboratory, Research Institute of Sun Yat-sen University in Shenzhen, Shenzhen 518057, China
3
Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
*
Authors to whom correspondence should be addressed.

Abstract

With the rapid increase in the number of available 3D garment models, the ability to accurately and effectively search for 3D garment model is crucial for many applications such as industrial design and engineering, and in areas such as manufacturing. In this paper, a sketch-based 3D garment model retrieval algorithm is reported. Users can draw their sketch images according to their interest sensations. Then they can retrieve the 3D garment from the 3D garment database. Each garment in database is used to render an associated image to represent it. Then local features are extracted based on Gabor filtering for each image and the input sketch. The bag of features model is used for vectorizing each image and the sketch. Then the cosine similarities of each garment with the sketch are computed.Evaluations are performed to show the retrieval performance ofthe proposed algorithm.

1. Introduction

Virtual 3D garment plays an important role in computer aided garment design, character-based games environment and virtual reality. With the fast development of the advanced modeling [1,2,3] and visualization [4,5] techniques of virtual garment, 3D garments have been a more and more popular type of digital media. The number of available 3D garment models in different types increases steadily. The ability to accurately and effectively search for 3D model is crucial for many applications such as industrial design, engineering, and manufacturing area.
The research to retrieve 3D models based on their content has led to the development of several approaches to compute the similarity between two 3D models in recent years [6]. Several algorithms like shape histogram [7], shape distribution [8], moment [9], light field [10], spherical harmonics [11] have been reported. Following these approaches, users can search for 3D models by supplying an example query object. Depending on the task, 3D model retrieval using queryby-example is not the most intuitive approach. It assumes that users already have a well defined 3D query model, which is similar to what they are searching for. Another approach to search 3D models is based on two dimensional, user-drawn sketches. Among numerous interaction methodologies, humans can sketch important features of objects they are interested in very quickly [12]. Although a sketch is composed of only a few lines, it is a coarse but detailed picture including key features. In order to compute the similarity between a 3D model and a user sketch image, the main method is to project and render the model from several viewpoints and the best-matching similarity between the sketch image and the projected images are computed.
In this paper, a novel approach for sketch based garment model retrieval by computing viewbased descriptors using suggestive contours is proposed. In real life, garment designers use design drawings to illustrate the effect of garments. Following this habit, user sketch is used as input to find similar garment model. The goal of this paper is to score the 3D garment models in the database, and then recommend the highest ones to the user. It is particularly suited for 3D garment model retrieval, as the human visual system perceives 3D models as two dimensional projections.

3. Sketch-based 3D Garment Model Retrieval

The processing of sketch-based modeling retrieval is shown in Figure 1. First, 3D garment models were collected and saved into the database. The models are used to render the front view images with their suggestive contours and vectorizing the images with their features. Second, when user inputs a sketch, similar feature extracting and vectorizing work will be performed on it. At last, the retrieval algorithm computes the similarity between the user sketch vector and the vectors of 3D garment models in the database. As result, the garment model with the highest score is the best match of the model database. The main algorithm includes four parts: 3D garment model image generation, image feature representation, retrieval engine, and retrieving images.
Figure 1. Processing of sketch-based garment model retrieval.

3.1. 3D Garment Image Generation

In order to retrieve the 3D model based on sketches, an associated image is rendered to represent a 3D model, and an image-based retrieval model is built for the garment images. The algorithm proposed in paper is relied on the assumption that input sketch of a query is the front view drawing for the desired garment. The assumption is rational because people mainly care about the front view of a garment, and especially in the computer aid design area professional designers are used to drawing the front view of garments to show their ideas. Based on the assumption, the front image of a garment is the best view to be rendered. This is particular different with other 3D models retrieval methods because the dataset of this paper are garment models facing the front. Three categories of line rendering algorithms [19,20,21] can be used. The rendering image should be good representative for the garment, so suggestive contours [21] are chosen because it retains more detail features than the other two. The camera is fixed before rendering to center the 3D model on the render window.

3.2. Image Feature Representation

After rendering, garment image set I = {ik} is available. A comparison of each image ik with the user sketch is sufficient for the retrieval, but the efficiency is low. To resolve this issue, the representation for each image ik is conducted. There are many mature solutions for the representation. In general there are two types: 1) Local descriptors, which computes the distribution of the value of interest, such as Scale-invariant feature transform (SIFT) [22] and Speeded Up Robust Features (SURF) [23]; 2) A sparse set of basis elements, which performs a change of basis, such as Wavelet basis [24] and Curve let basis [25]. As analyzed in Eitz et al. [16], the GALIF feature proposed by them is more practical than other leading representation algorithms for contour line image retrieval. Since the target of this paper is also detecting features of rendering lines from models, the GALIF descriptor is chosen. Then the parameters are optimized similar to Eitz et al. The result was obtained based on the dataset: vocabulary size: 100, tiles: 4, line width: 0.1, λ:0.3, ω0:0.13, local feature size: 0.2, line-type: suggestive contours (For the meanings of parameters, the description of Eitz et al. [16] are referred to the readers). See Figure 2 (left), images are filtered by Gabor filters with four orientations {0, π/4, π/2, 3∗π/4}v and then local features are sampled by 4*4 tiles on the response images.
Figure 2. Illustrations for GALIF feature extraction (left) and image vectorization (right).

3.3. Retrieval Engine

The bag of features (BoF) model [26] is used to build the retrieval engine. This model capitalizes on the histogram of features to compare the similarity between images (Figure 2 (right)). First, the features are classified into clusters. The clustering method used in this paper is based on a Gaussian Mixture Model. With all features, the following objective function is maximized:
E l = ∑ i = 1 n log ( ∑ j = 1 k l ω j · N ( f | μ j , Σ j ) ) − P 2 log ( n )
where n is the total number of features, kl is the number of mixture Gaussians, represents a Gaussian with mean and covariance, is the corresponding coefficient of j Gaussian and P is the number of parameters in the model. The second term is used for penalizing model complexity, based on the Bayesian Information Criterion. Then E can be maximized as follow. First kl = 2, k-means clustering [27] method is performedon feature set F. The Expectation Maximization (EM) is taken to estimate, and, using the k-means clustering centers as initial value. In order to prevent over fitting, the covariance matrix is restricted to be diagonal. Then the estimation is iterated until El satisfies the Eq. (2) by increasing kl = 2, 3, 4, ….
El2 ≤ El1, l2 ∈ {l|l1 ≤ l ≤ l1 + 4}
Since the second term of Eq. (1) increases faster than the first term, the object function will get a highest value. Using parameters at this state, the best Gaussian Mixture Model with kbest Gaussians is built for feature space.
For each feature ft F, it is classified into the cluster j that has the maximal value of ωj · N(f|µj,σj). Then the occurrence number of features of different clusters in each image ik are calculated to generate a kbest dimension vector v(k) V for ik. Instead of simply using the appearing times of cluster elements appearing in an image, the tf-idf weights for histogram is computed. The tf-idf model is widely used in the information retrieval. It takes care of both the frequency of the feature in an image and the importance of the feature among images. The following tf-idf function is used to compute the histogram of each ik:
v ( k , s ) = ( 1 + log ( c ( k , s ) / ∑ t = 1 m c ( k , t ) ) ) log ( 1 + n s N )
where s = 1, 2, …, kbest, v(k, s) is the sth dimension of v(k), c(k, s) is the occurrence number of features of ith clusters in ik, m is the total number of images, ns is the number of image with c(k, s) > 0 and N is the total number of images. Furthermore, each vector is normalized by constraining.

3.4. Retrieving Images

Now a histogram set V = {vx} of tf-idf weights for garment images is generated. For user query sketch, its histogram vq is also calculated with Eq. (3). The retrieving algorithm begins with rank generation, computing similarity scores for garment models. The cosine similarity between histograms of user sketch and each garment image is computed. Using the following function:
similarity ( v q , v j ) = v q · v j ‖ v q ‖ ‖ v j ‖
Then the garment images are sorted according to their cosine similarity with user sketch and recommend ten with highest scores to user.

4. Results

In this section, several experiments were conducted to evaluate the effectiveness of the proposed method. When the user draws a stroke, the features of 2D image are extracted. The retrieval algorithm generates a ranked list of the best matching models and displays them to the user for reference. See Figure 3, the standard precision/recall is used to evaluate the retrieval performance. The combinations of different techniques are tested on the dataset. The test dataset of experiments are 96 garment models divided into six clusters: shirt, coat, dress, trouser, short and bra. These models are collected from the internet or generated by physical simulation from 2D patterns, or from publications of previous researches, such as [28]. As result, the combination of techniques described in the previous sections outperforms others. Eight examples of the retrieved 3D models from various user drawn sketches are shown in Figure 4. The images at the left column are sketches as input for queries and the corresponding top three retrieval results are shown at the same row order by similarity decreases from left to right. From the retrieval results, it is found that the users can draw their sketch images according to their interest sensations. Then they can retrieve t 3D garments from the 3D garment database. It is a very easy and flexible method that permits humans to sketch important features of objects that they are interested in very quickly.
Figure 3. Precision/recall plot for the retrieval performance evaluation.
Figure 4. Retrieval results with simple sketches as inputs.

5. Conclusion

In this paper, a new approach for sketch-based 3D garment model retrieval is presented. Based on the proposed approach, users can retrieve the similar garment models as intended by simple sketches. The technique is useful for garment related areas, especially computer aided design and virtual reality.

Acknowledgments

This research is supported by the National Natural Science Foundation of China (61073131, 61272192), NSFC-Guangdong Joint Fund (No. U1135003, U0935004), the National Key Technology R&D Program (No. 2011BAH27B01, 2011BHA16B08), the Industry Academy-research Project of Guangdong (No. 2011A091000032).

References

  1. Jiang, J.F.; Zhong, Y.Q.; Wang, S.Y. 3D mesh deformation technology for generation of virtual garments. In Textile Bioengineering and Informatics Symposium Proceedings; Li, Y., Liu, Y.F., Luo, X.N., Li, J.S., Eds.; 2011; Vols 1-3, pp. 25–30. [Google Scholar]
  2. Robson, C.; Maharik, R.; Sheffer, A.; Carr, N. Context-aware garment modeling from sketches. Comput. Graph. 2011, 35, 604–613. [Google Scholar] [CrossRef] [Scilit]
  3. Kong, H.Y.; Gu, B.F.; Gu, P.Y.; et al. Research of crotch curve line rules based on 3D point cloud data. In Textile Bioengineering and Informatics Symposium Proceedings; Li, Y., Takatera, M., Kajiwara, K., Li, J.S., Eds.; 2012; Vols 1 and 2, pp. 948–952. [Google Scholar]
  4. Lin, G.; Luo, X.; Li, C.; et al. The skim of balance theory of 3D garment simulation. Appl. Math. Comput. 2011, 218, 492–501. [Google Scholar] [CrossRef] [Scilit]
  5. Sun, C.; Zheng, R.; Li, J.; et al. Image perception of female breast beauty and its relation to 3D anthropometric measurements. In Textile Bioengineering and Informatics Symposium Proceedings; Li, Y., Liu, Y.F., Luo, X.N., Li, J.S., et al., Eds.; 2011; Vols 1-3, pp. 1383–1391. [Google Scholar]
  6. Tangelder, J.W.; Veltkamp, R.C. A survey of content based 3D shape retrieval methods. Multimed. Tools Appl. 2008, 39, 441–471. [Google Scholar] [CrossRef] [Scilit]
  7. Ankerst, M.; Kastenm, G. 3D shape histograms for similarity search and classification in spatial databases. In Proceedings of the 6th International Symposium on Advances in Spatial Databases; 1999, Springer-Verlag, 1999; pp. 207–226. [Google Scholar]
  8. Osada, R.; Funkhouser, T.; Chazelle, B.; Dobkin, D. Matching 3D models with shape distributions. Shape Modeling and Applications, SMI 2001 International Conference on 2001; pp. 154–166.
  9. Elad, M.; Tal, A.; Ar, S. Content based retrieval of VRML objects: an iterative and interactive approach. In Proceedings of the sixth Eurographics workshop on Multimedia 2001; Springer-Verlag New York, Inc.: Manchester, UK, 2002; pp. 107–118. [Google Scholar]
  10. Chen, D.Y.; Tian, X.P.; Shen, Y.T.; Ouhyoung, M. On visual similarity based 3D model retrieval. Comput. Graph. Forum 2003, 22, 223–232. [Google Scholar] [CrossRef] [Scilit]
  11. Kazhdan, M.; Funkhouser, T.; Rusinkiewicz, S. Rotation invariant spherical harmonic representation of 3D shape descriptors. In Proceedings of the 2003 Eurographics/ACM SIGGRAPH symposium on Geometry processing; Eurographics Association: Aachen, Germany, 2003; pp. 156–164. [Google Scholar]
  12. Yoon, S.M.; Scherer, M.; Schreck, T.; Kuijper, A. Sketch-based 3D model retrieval using diffusion tensor fields of suggestive contours. In Proceedings of the international conference on Multimedia; ACM: Firenze, Italy, 2010; pp. 193–200. [Google Scholar]
  13. Loffler, J. Content-based retrieval of 3D models in distributed Web databases by visual shape information. Information Visualization, 2000. Proceedings. IEEE International Conference on 2000; pp. 82–87.
  14. Funkhouser, T.; Min, P.; Kazhdan, M.; et al. A search engine for 3D models. ACM Trans. Graph. 2003, 22, 83–105. [Google Scholar] [CrossRef] [Scilit]
  15. Daras, P.; Axenopoulos, A. A 3D shape retrieval framework supporting multimodal queries. Int. J. Comput. Vis. 2010, 89, 229–247. [Google Scholar] [CrossRef] [Scilit]
  16. Eitz, M.; Richter, R.; Boubekeur, T.; et al. Sketch-based shape retrieval. ACM Trans. Graph. 2012, 31, 1–10. [Google Scholar] [CrossRef] [Scilit]
  17. Shin, H.; Igarashi, T. Magic canvas: Interactive design of a 3-D scene prototype from freehand sketches. In Proceedings of Graphics Interface 2007; ACM: Montreal, Canada, 2007; pp. 63–70. [Google Scholar]
  18. Lee, J.; Funkhouser, T. Sketch-based search and composition of 3D models. In Proceedings of the Fifth Eurographics conference on Sketch-Based Interfaces and Modeling; Eurographics Association: Annecy, France, 2008; pp. 97–104. [Google Scholar]
  19. Koenderink, J.J. What does the occluding contour tell us about solid shape? Perception 1984, 13, 321–330. [Google Scholar] [CrossRef] [Scilit]
  20. Canny, J. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 1986, 8, 679–698. [Google Scholar] [CrossRef] [Scilit]
  21. DeCarlo, D.; Finkelstein, A.; Rusinkiewicz, S.; Santella, A. Suggestive contours for conveying shape. ACM Trans. Graph. 2003, 22, 848–855. [Google Scholar] [CrossRef] [Scilit]
  22. Lowe, D.G. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 2004, 60, 91–110. [Google Scholar] [CrossRef] [Scilit]
  23. Bay, H.; Tuytelaars, T.; Gool, L. SURF: Speeded up robust features. In Computer Vision – ECCV 2006; Leonardis, A., Bischof, H., Pinz, A., Eds.; Springer: Berlin Heidelberg, 2006; pp. 404–417. [Google Scholar]
  24. Ricker, N. Wavelet contraction, wavelet expansion, and the control of seismic resolution. Geophysics 1953, 18, 769–792. [Google Scholar] [CrossRef] [Scilit]
  25. Candes, E.J.D.; David, L. Curvelets: A surprisingly effective nonadaptive representation for objects with edges; 2000. [Google Scholar]
  26. Sivic, J.; Zisserman, A. Video Google: a text retrieval approach to object matching in videos. Comput. Vis. Proceedings. Ninth IEEE Int. Conf. 2003, 2, 1470–1477. [Google Scholar] [CrossRef] [Scilit]
  27. MacQueen, J.B. Some methods for classification and analysis of multivariate observations. In Proc. of the fifth Berkeley Symposium on Mathematical Statistics and Probability.; Cam, L.M.L., Neyman, J., Eds.; University of California Press, 1967; pp. 281–297. [Google Scholar]
  28. Fisher, M.; Hanrahan, P. Context-based search for 3D models. In ACM SIGGRAPH Asia 2010 papers; ACM: Seoul, South Korea, 2010; pp. 1–10. [Google Scholar]

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.