A Joint Training Model for Face Sketch Synthesis
Abstract
:Featured Application
Abstract
1. Introduction
- (1)
- To consider the training sketches during the reconstruction weight representation process, a joint training model is proposed to integrate the training photo and sketch information.
- (2)
- We design a modified locality constraint that modulates the reconstruction weight through the distance between the high-pass filtered images of test patches and the sampled training sketch patches.
- (3)
- The proposed method yields high quality sketches with more detail information and less noise over the wide range of datasets, promoting the accuracy of the sketch-based suspect identification.
2. Technical Backgrounds
2.1. The LLE Method
2.2. The RSLCR Method
3. Joint Training Model for Face Sketch Synthesis
3.1. Joint Training Model
3.2. Face Sketch Synthesis
4. Evaluation Experiments
4.1. Datasets
4.2. Experimental Setting
4.3. Synthesizesd Sketch Results Comparison
4.4. Face Sketch Recognition
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Wang, N.; Zhang, S.; Gao, X.; Li, J.; Song, B.; Li, Z. Unified framework for face sketch synthesis. Signal Process. 2017, 130, 1–11. [Google Scholar] [CrossRef]
- Li, J.; Yu, X.; Peng, C.; Wang, N. Adaptive representation-based face sketch-photo synthesis. Neurocomputing 2017, 269, 152–159. [Google Scholar] [CrossRef]
- Wan, W.; Lee, H.J. FaceNet Based Face Sketch Recognition. In Proceedings of the 2017 International Conference on Computational Science and Computational Intelligence, Las Vegas, NV, USA, 14–16 December 2017; pp. 432–436. [Google Scholar]
- Zhang, Y.; Wang, N.; Zhang, S.; Li, J.; Gao, X. Fast face sketch synthesis via KD-tree search. In Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherland, 8–16 October 2016; Springer: Cham, Switzerland, 2016; pp. 64–77. [Google Scholar]
- Wang, N.; Zhu, M.; Li, J.; Song, B.; Li, Z. Data-driven vs. model-driven: Fast face sketch synthesis. Neurocomputing 2017, 257, 214–221. [Google Scholar] [CrossRef]
- Tang, X.; Wang, X. Face sketch recognition. IEEE Trans. Circuits Syst. Video Technol. 2004, 14, 1–7. [Google Scholar] [CrossRef]
- Liu, Q.; Tang, X.; Jin, H.; Lu, H.; Ma, S. A nonlinear approach for face sketch synthesis and recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, 20–25 June 2005; pp. 1005–1010. [Google Scholar]
- Song, Y.; Bao, L.; Yang, Q.; Yang, M. Real-time exemplar-based face sketch synthesis. In Proceedings of the European Conference on Computer Vision, Zurich, Switzerland, 5–12 September 2014; Springer: Cham, Switzerland, 2014; pp. 800–813. [Google Scholar]
- Wang, X.; Tang, X. Face photo-sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2009, 31, 1955–1967. [Google Scholar] [CrossRef] [PubMed]
- Zhou, H.; Kuang, Z.; Wong, K. Markov weight fields for face sketch synthesis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Rhode Island, USA, 18–20 June 2012; pp. 1091–1097. [Google Scholar]
- Gao, X.; Wang, N.; Tao, D.; Li, X. Face sketch–photo synthesis and retrieval using sparse representation. IEEE Trans. Circuits Syst. Video Technol. 2012, 22, 1213–1226. [Google Scholar] [CrossRef]
- Peng, C.; Gao, X.; Wang, N.; Tao, D.; Li, X.; Li, J. Multiple representations-based face sketch–photo synthesis. IEEE Trans. Neural Netw. Learn. Syst. 2016, 27, 2201–2215. [Google Scholar] [CrossRef] [PubMed]
- Wang, N.; Gao, X.; Li, J. Random sampling for fast face sketch synthesis. Pattern Recognit. 2018, 76, 215–227. [Google Scholar] [CrossRef] [Green Version]
- Wang, N.; Gao, X.; Sun, L.; Li, J. Bayesian face sketch synthesis. IEEE Trans. Image Process. 2017, 26, 1264–1274. [Google Scholar] [CrossRef] [PubMed]
- Zhang, L.; Lin, L.; Wu, X.; Ding, S.; Zhang, L. End-to-end photo-sketch generation via fully convolutional representation learning. In Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, Shanghai, China, 23–26 June 2015; pp. 627–634. [Google Scholar]
- Zhang, D.; Lin, L.; Chen, T.; Wu, X.; Tan, W.; Izquierdo, E. Content-adaptive sketch portrait generation by decompositional representation learning. IEEE Trans. Image Process. 2017, 26, 328–339. [Google Scholar] [CrossRef] [PubMed]
- Goodfellow, I.J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative adversarial nets. In Proceedings of the International Conference on Neural Information Processing Systems, Montreal, QC, Canada, 8–13 December 2014; MIT: Cambridge, MA, USA; pp. 2672–2680. [Google Scholar]
- Jiang, J.; Yu, Y.; Wang, Z.; Liu, X.; Ma, J. Graph-Regularized Locality-Constrained Joint Dictionary and Residual Learning for Face Sketch Synthesis. IEEE Trans. Image Process. 2019, 28, 628–641. [Google Scholar] [CrossRef] [PubMed]
- Wang, N.; Gao, X.; Sun, L.; Li, J. Anchored neighborhood index for face sketch synthesis. IEEE Trans. Circuits Syst. Video Technol. 2018, 28, 2154–2163. [Google Scholar] [CrossRef]
- Wang, J.; Yang, J.; Yu, K.; Lv, F.; Huang, T.; Gong, Y. Locality-constrained linear coding for image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 13–18 June 2010; pp. 3360–3367. [Google Scholar]
- Zhang, W.; Wang, X.; Tang, X. Coupled information-theoretic encoding for face photo-sketch recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA, 21–25 June 2011; pp. 513–520. [Google Scholar]
- Tang, X.; Wang, X. Face photo recognition using sketch. In Proceedings of the IEEE International Conference on Image Processing, New York, NY, USA, 22–25 September 2002; pp. 257–260. [Google Scholar]
- Martinez, A.; Benavente, R. The AR Face Database; Technical Report; CVC: Barcelona, Spain, 1998. [Google Scholar]
- Messer, K.; Matas, J.; Kittler, J.; Luettin, J.; Maitre, G. XM2VTSDB: The extended M2VTS database. In Proceedings of the International Conference on Audio- and Video-Based Biometric Person Authentication, Washington, DC, USA, 22–24 March 1999; pp. 965–966. [Google Scholar]
- Phillips, P.; Moon, H.; Rauss, P.; Rizvi, S. The FERET evaluation methodology for face recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 2000, 22, 1090–1104. [Google Scholar] [CrossRef]
- Chen, C.; Tan, X.; Wong, K.K. Face sketch synthesis with style transfer using pyramid column feature. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe, NV, USA, 12–15 March 2018; pp. 485–493. [Google Scholar]
- Schroff, F.; Kalenichenko, D.; Philbin, J. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 8–12 June 2015; pp. 815–823. [Google Scholar]
Methods | LLE | MRF | MWF | RSLCR | Proposed |
---|---|---|---|---|---|
CUHK | 536.34 | 8.60 | 16.10 | 18.79 | 20.80 |
AR | 496.47 | 8.40 | 15.33 | 19.10 | 19.75 |
XM2VTS | 642.50 | 10.40 | 18.80 | 18.14 | 20.78 |
CUFSF | 1591.95 | 24.25 | 45.20 | 17.66 | 20.17 |
Methods | CUFS | CUFSF | ||||
---|---|---|---|---|---|---|
rank-1 | rank-5 | rank-10 | rank-1 | rank-5 | rank-10 | |
LLE | 38.7 | 63.6 | 72.8 | 11.3 | 26.3 | 36.2 |
MRF | 39.6 | 65.1 | 76.3 | 7.2 | 18.0 | 25.3 |
MWF | 47.3 | 68.0 | 77.5 | 10.6 | 23.6 | 31.5 |
RSLCR | 51.7 | 74.2 | 83.4 | 14.3 | 32.4 | 43.1 |
FCN | 51.4 | 76.3 | 84.0 | 11.4 | 26.4 | 35.2 |
GAN | 52.6 | 78.7 | 84.9 | 9.2 | 24.9 | 35.6 |
Proposed | 65.4 | 85.2 | 90.5 | 21.2 | 43.6 | 52.1 |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wan, W.; Lee, H.J. A Joint Training Model for Face Sketch Synthesis. Appl. Sci. 2019, 9, 1731. https://doi.org/10.3390/app9091731
Wan W, Lee HJ. A Joint Training Model for Face Sketch Synthesis. Applied Sciences. 2019; 9(9):1731. https://doi.org/10.3390/app9091731
Chicago/Turabian StyleWan, Weiguo, and Hyo Jong Lee. 2019. "A Joint Training Model for Face Sketch Synthesis" Applied Sciences 9, no. 9: 1731. https://doi.org/10.3390/app9091731
APA StyleWan, W., & Lee, H. J. (2019). A Joint Training Model for Face Sketch Synthesis. Applied Sciences, 9(9), 1731. https://doi.org/10.3390/app9091731