# A Novel Sketch-Based Three-Dimensional Shape Retrieval Method Using Multi-View Convolutional Neural Network

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

- We propose to study the feature representations for 3D patterns and sketches, which ignore the predicament of the most outstanding view selection;
- Two original Siamese convolutional neural networks were used for dealing with the overfitting issue and to explore similar points successfully in and across the domains.
- Experiments using two large data sets show that the strategy proposed is greatly better compared with several prior art strategies.

## 2. Related Work

## 3. Framework

#### 3.1. Feature Extraction

#### Sketch

#### 3.2. Siamese Network Architecture

#### 3.3. Cross-Domain Matching Using Siamese Network

_{c}as well as the discrimination term ${L}_{d}$, which minimize/maximize the inter-class variations as well as the intra-class variations in all of the domains and ensure the distribution-consistency in diversified domains.

_{d}+ (1 − α)L

_{c}

## 4. Experiment

#### 4.1. Dataset

#### 4.2. Evaluation Metrics

#### 4.3. Experimental Settings

_{p}view pairs in the similar pairs (same class) and k

_{n}view samples from dissimilar pairs (other categories) for each training sketch. In general, training success rates for dissimilar pairs were ten times higher compared with the pairs with similarities. Here, we used k

_{p}= 2, k

_{n}= 10. The random pairing was performed by us for all of the training epochs.

**k**value of the maxout layer. In order to get the specific value of

**k**, we conducted a series of eight trials. In these trials, we set k = 2, 3, 4, 5, 6, 7, 8. Each trial of the test performed the train for 20 times. The experimental resources have been mentioned before. Diversified trials possess diversified k values while other parameters were consistent in the diversified trials. The experiment outcomes are presented within Figure 5. It was obvious that the predictive ability was significantly better when k was set to 3. Therefore, k was set to 3 in this paper. Later on, the network pattern’s constructionwas completed.

#### 4.4. Retrieval on SHREC 2013 Dataset

#### 4.5. Retrieval on SHREC 2014 Dataset

## 5. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Zeng, H.; Liu, Y.; Liu, J.; Fu, D. Non-rigid 3d model retrieval based on quadruplet convolutional neural networks. IEEE Access
**2018**, 6, 76087–76097. [Google Scholar] [CrossRef] - Lei, H.; Luo, G.; Li, Y.; Liu, J.; Ye, J. Sketch-based 3d model retrieval using attributes. Int. J. Grid High Perform. Comput. (IJGHPC)
**2018**, 10, 60–75. [Google Scholar] [CrossRef] - Li, Y.; Lei, H.; Lin, S.; Luo, G. A new sketch-based 3d model retrieval method by using composite features. Multimed. Tools Appl.
**2018**, 77, 2921–2944. [Google Scholar] [CrossRef] - Bo, L.; Lu, Y.; Godil, A.; Schreck, T.; Bustos, B.; Ferreira, A.; Furuya, T.; Fonseca, M.J.; Johan, H.; Matsuda, T. A comparison of methods for sketch-based 3d shape retrieval. Comput. Vis. Image Underst.
**2014**, 119, 57–80. [Google Scholar] - Eitz, M.; Richter, R.; Boubekeur, T.; Hildebrand, K.; Alexa, M. Sketch-based shape retrieval. ACM Trans. Graph.
**2012**, 31, 1–10. [Google Scholar] [CrossRef] - Dai, G.; Xie, J.; Fang, Y. Deep correlated holistic metric learning for sketch-based 3d shape retrieval. IEEE Trans. Image Process.
**2018**, 27, 3374–3386. [Google Scholar] [CrossRef] [PubMed] - Sang, M.Y.; Yoon, G.J.; Schreck, T. User-drawn sketch-based 3d object retrievalusing sparse coding. Multimed. Tools Appl.
**2015**, 74, 4707–4722. [Google Scholar] - Wang, F.; Li, Y. Sketch-based 3d shape retrieval using convolutional neural net. Comput. Sci.
**2015**, 1875–1883. [Google Scholar] - Funkhouser, T.; Min, P.; Kazhdan, M.; Chen, J.; Halderman, A.; Dobkin, D.; Jacobs, D. A search engine for 3D models. ACM Trans. Graph.
**2003**, 22, 83–105. [Google Scholar] [CrossRef] - Funkhouser, T.; Shilane, P. Partial matching of 3D shapes with priority-driven search. In Proceedings of the fourth Eurographics symposium on Geometry processing, Cagliari, Italy, 26–28 June 2006. [Google Scholar]
- Eitz, M.; Hildebrand, K.; Boubekeur, T.; Alexa, M. Sketch-based 3d shape retrieval. ACM Trans. Graph.
**2010**. [Google Scholar] [CrossRef] - Saavedra, J.M.; Bustos, B.; Schreck, T.; Yoon, S.; Scherer, M. Sketch-Based 3d Model Retrieval Using Keyshapes for Global and Local Representation. In Eurographics Workshop on 3D Object Retrieval; The Eurographics Association: Goslar, Germany, 2012. [Google Scholar]
- Eitz, M.; Hildebrand, K.; Boubekeur, T.; Alexa, M. An evaluation of descriptors for large-scale image retrieval from sketched feature lines. Comput. Graphi.
**2010**, 34, 482–498. [Google Scholar] [CrossRef] - Eitz, M.; Hildebrand, K.; Boubekeur, T.; Alexa, M. Sketch-based image retrieval: Benchmark and bag-of-features descriptors. IEEE Trans. Vis. Comput. Graphi.
**2010**, 17, 1624–1636. [Google Scholar] [CrossRef] [PubMed] - Li, B.; Lu, Y.; Godil, A.; Schreck, T.; Aono, M.; Johan, H.; Saavedra, J.M.; Tashiro, S. Shrec1’3 track: Large scale sketch-based 3d shape retrieval. In Proceedings of the Eurographics Workshop on 3d Object Retrieval, Girona, Spain, 11 May 2013. [Google Scholar]
- Li, B.; Lu, Y.; Li, C.; Godil, A.; Schreck, T.; Aono, M.; Burtscher, M.; Fu, H.; Furuya, T.; Johan, H. In Shrec’14 track: Extended large scale sketch-based 3d shape retrieval. In Proceedings of the Eurographics Workshop on 3d Object Retrieval, Strasbourg, France, 6 April 2014. [Google Scholar]
- Eitz, M.; Tu, B.; Hays, J.; Alexa, M.; Tu, B. How do humans sketch objects? ACM. Trans. Graphi.
**2012**, 31, 1–10. [Google Scholar] [CrossRef] - Shilane, P.; Min, P.; Kazhdan, M.; Funkhouser, T. The princeton shape benchmark. In Proceedings of the Shape Modeling Applications, Genova, Italy, 7–9 June 2004. [Google Scholar]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst.
**2012**, 25, 1097–1105. [Google Scholar] [CrossRef] - Dai, G.; Xie, J.; Zhu, F.; Fang, Y. Deep correlated metric learning for sketch-based 3d shape retrieval. In Proceedings of the AAAI, San Francisco, CA, USA, 4–6 February 2017; pp. 4002–4008. [Google Scholar]
- Gruber, L.Z.; Haruvi, A.; Basri, R.; Irani, M. Perceptual dominance in brief presentations of mixed images: Human perception versus deep neural networks. Front. Comput. Neurosci.
**2018**, 12, 57. [Google Scholar] [CrossRef] [PubMed] - Li, B.; Johan, H. Sketch-based 3d model retrieval by incorporating 2d-3d alignment. Multimed. Tools Appl.
**2013**, 65, 363–385. [Google Scholar] [CrossRef] - Tang, P.; Wang, C.; Wang, X.; Liu, W.; Zeng, W.; Wang, J. Object detection in videos by high quality object linking. IEEE Trans. Pattern Anal. Mach. Intell.
**2019**, 1. [Google Scholar] [CrossRef] [PubMed] - Lecun, Y.; Boser, B.; Denker, J.S.; Henderson, D.; Howard, R.E.; Hubbard, W.; Jackel, L.D. Backpropagation applied to handwritten zip code recognition. Neural Comput.
**2014**, 1, 541–551. [Google Scholar] [CrossRef] - Biswas, A.; Chandrakasan, A.P. CONV-SRAM: An Energy-Efficient SRAM With In-Memory Dot-Product Computation for Low-Power Convolutional Neural Networks. IEEE J. Solid State Circuits
**2019**, 54, 217–230. [Google Scholar] [CrossRef]

**Figure 2.**A sample of the rendered silhouette (

**left**), occluding contours (

**center**), and suggestive contours (

**right**).

**Figure 7.**Examples of the outcomes of the queries by adopting our strategy. In terms of all the sketches (top left of a cell), the top 19 outcomes are shown. The “ship” proved partial matching (i.e., sailboats were retrieved). The “church” shows some unsuccessful cases (i.e., chairs were retrieved), with the objects that were ranked for the highest position not matching the demanded object. It should be noted that the rest the sketches were matched perfectly.

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Mao, D.; Hao, Z.
A Novel Sketch-Based Three-Dimensional Shape Retrieval Method Using Multi-View Convolutional Neural Network. *Symmetry* **2019**, *11*, 703.
https://doi.org/10.3390/sym11050703

**AMA Style**

Mao D, Hao Z.
A Novel Sketch-Based Three-Dimensional Shape Retrieval Method Using Multi-View Convolutional Neural Network. *Symmetry*. 2019; 11(5):703.
https://doi.org/10.3390/sym11050703

**Chicago/Turabian Style**

Mao, Dianhui, and Zhihao Hao.
2019. "A Novel Sketch-Based Three-Dimensional Shape Retrieval Method Using Multi-View Convolutional Neural Network" *Symmetry* 11, no. 5: 703.
https://doi.org/10.3390/sym11050703