Body-Based Gender Recognition Using Images from Visible and Thermal Cameras
Abstract
:1. Introduction
- This is the first study using both the visible light and thermal images of the full human body for gender recognition.
- Based on the detected boxes of body in both visible and thermal images, the features for gender recognition are extracted from the visible light and thermal image. This is accomplished by using the histogram of oriented gradient (HoG) method with principal component analysis (PCA) in order to reduce the feature dimension, processing time, and the effects of noise on the extracted features.
- Gender classification using the features from visible light and thermal images are made by using two different SVMs classifiers.
- A score level fusion is performed to combine the two classification scores by the two SVMs using another SVM classifier in order to recognize the gender of human.
Category | Method | Strength | Weakness |
---|---|---|---|
Fingerprint-based gender recognition | Using fingerprint image for gender recognition [6,10,11]. | High accuracy | Requires the cooperation of users. The accuracy is affected by quality and the resolution of fingerprint image. |
Face-based gender recognition | Using human face for gender recognition [7,8,9,10,11]. | High accuracy | Requires the cooperation of users. It’s very hard to recognize gender for very young people. |
Body-based gender recognition | Using only visible images of human body for gender recognition [3,12], with 3D shape model [13,14]. | Does not require the cooperation of users. | Recognition accuracy is strongly affected by illumination conditions, body poses, the random appearance of image texture on body region such as clothes, accessories etc. Recognition accuracy is lower than using face-based gender recognition approach. |
Combining the visible and thermal images of human body using score level fusion using SVM for gender recognition (Proposed Method) | Does not require the cooperation of users. Enhances the recognition result compared to the systems that use only visible images for gender recognition. Reduces the affects by illumination condition, the body poses, the random appearance of image texture on boy region such as clothes, accessories etc. | Requires longer processing time than singly visible images. Recognition accuracy is still lower than using face-based gender recognition approach. |
2. Proposed Method
2.1. Overview of the Proposed Method
2.2. Image Acquisition and Human Detection from Image Sequences
2.3. Feature Extraction Methods for Gender Recognition
2.3.1. Histogram of Oriented Gradient
2.3.2. Multi-Level Local Binary Pattern
2.3.3. Feature Combination and Gender Recognition Using SVM
3. Experimental Results
3.1. Description of the Database, Performance Measurement and Experimental Setup
Database | Visible Database | Thermal Database | (Visible + Thermal) Database | |
---|---|---|---|---|
Number of persons | 103 (66 males/37 females) | 103 (66 males/37 females) | 103 (66 males/37 females) | |
Number of Images | 2926 | 2926 | 5852 |
Database | Training Sub-Database | Testing Sub-Database | The Entire Database |
---|---|---|---|
Number of persons | 83 (53 males/30 females) | 20 (13 males/7 females) | 103 (66 males/37 females) |
3.2. Gender Recognition Using Our Proposed Method
Methods | Linear Kernel | RBF Kernel | |||||
---|---|---|---|---|---|---|---|
EER | FAR | GAR | EER | FAR | GAR | ||
Using only visible images for recognition | Using HoG feature | 19.639 | 10.000 | 58.099 | 16.540 | 10.000 | 62.758 |
15.000 | 69.920 | 15.000 | 79.878 | ||||
19.642 | 80.364 | 16.542 | 83.461 | ||||
20.000 | 80.440 | 20.000 | 88.051 | ||||
25.000 | 85.697 | 25.000 | 91.716 | ||||
Using MLBP feature | 27.105 | 20.000 | 62.790 | 25.088 | 15.000 | 57.506 | |
25.000 | 69.676 | 20.000 | 66.359 | ||||
27.107 | 72.897 | 25.094 | 74.918 | ||||
30.000 | 77.793 | 30.000 | 80.778 | ||||
35.000 | 81.359 | 35.000 | 86.088 | ||||
Using only thermal images for recognition | Using HoG feature | 23.459 | 15.000 | 60.002 | 19.583 | 10.000 | 57.027 |
20.000 | 67.192 | 15.000 | 65.791 | ||||
23.518 | 76.600 | 19.605 | 80.439 | ||||
25.000 | 79.049 | 20.000 | 80.669 | ||||
30.000 | 83.499 | 25.000 | 89.815 | ||||
Using MLBP feature | 24.002 | 15.000 | 61.901 | 20.572 | 10.000 | 55.929 | |
20.000 | 68.824 | 15.000 | 68.506 | ||||
24.019 | 76.014 | 20.580 | 79.436 | ||||
25.000 | 76.771 | 25.000 | 85.236 | ||||
30.000 | 81.354 | 30.000 | 90.097 |
Method | Linear Kernel | RBF Kernel | ||||
---|---|---|---|---|---|---|
EER | FAR | GAR | EER | FAR | GAR | |
Using HoG feature | 19.553 | 10.000 | 61.445 | 15.946 | 10.000 | 73.050 |
15.000 | 70.169 | 15.000 | 80.534 | |||
19.605 | 80.498 | 16.192 | 84.299 | |||
20.000 | 80.790 | 20.000 | 89.341 | |||
25.000 | 84.764 | 25.000 | 92.250 | |||
Using MLBP feature | 21.022 | 15.000 | 63.366 | 18.126 | 10.000 | 60.993 |
20.000 | 76.482 | 15.000 | 73.669 | |||
21.043 | 78.999 | 18.155 | 81.902 | |||
25.000 | 83.096 | 20.000 | 84.426 | |||
30.000 | 87.075 | 25.000 | 89.898 |
Feature Extraction Method | The 1st SVM Kernel | The 2nd SVM Kernel | Recognition Accuracy | ||
---|---|---|---|---|---|
EER | FAR | GAR | |||
Using HoG feature | Linear | Linear | 17.891 | 15.000 | 75.063 |
17.892 | 82.111 | ||||
20.000 | 84.353 | ||||
25.000 | 88.862 | ||||
RBF | 17.628 | 15.000 | 75.143 | ||
17.667 | 82.411 | ||||
20.000 | 84.696 | ||||
25.000 | 89.035 | ||||
RBF | Linear | 15.158 | 10.000 | 65.884 | |
15.000 | 83.988 | ||||
15.254 | 84.937 | ||||
20.000 | 93.900 | ||||
RBF | 14.672 | 10.000 | 71.395 | ||
14.754 | 85.410 | ||||
15.000 | 85.410 | ||||
20.000 | 94.422 | ||||
Using MLP feature | Linear | Linear | 20.835 | 15.000 | 71.802 |
20.000 | 76.199 | ||||
21.068 | 79.397 | ||||
25.000 | 83.019 | ||||
RBF | 20.755 | 15.000 | 63.043 | ||
20.000 | 76.298 | ||||
21.068 | 79.559 | ||||
25.000 | 84.419 | ||||
RBF | Linear | 17.857 | 10.000 | 59.592 | |
15.000 | 75.281 | ||||
17.892 | 82.178 | ||||
20.000 | 85.796 | ||||
RBF | 17.642 | 10.000 | 57.442 | ||
15.000 | 75.895 | ||||
17.667 | 82.383 | ||||
20.000 | 86.253 |
Visible Light Image (Conventional Method Using Single Visible Light Images) | Thermal Image (Conventional Method Using Single Thermal Images) | (Visible Light + Thermal) Images | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Feature Level Fusion | Score Level Fusion (Proposed Method) | ||||||||||
EER | FAR | GAR | EER | FAR | GAR | EER | FAR | GAR | EER | FAR | GAR |
16.540 | 10.000 | 62.758 | 19.583 | 10.000 | 57.027 | 15.946 | 10.000 | 73.050 | 14.672 | 5.000 | 62.693 |
15.000 | 79.878 | 15.000 | 65.791 | 15.000 | 80.534 | 10.000 | 71.395 | ||||
16.542 | 83.461 | 19.605 | 80.439 | 16.192 | 84.299 | 14.754 | 85.410 | ||||
20.000 | 88.051 | 20.000 | 80.669 | 20.000 | 89.341 | 15.000 | 85.410 | ||||
25.000 | 91.716 | 25.000 | 89.815 | 25.000 | 92.250 | 20.000 | 94.422 |
Human Body Detection | Feature Extraction Using HoG (Two Images) | Feature Dimension Reduction (PCA) (Two Images) | The 1st SVM Layer | The 2nd SVM Layer | Total |
---|---|---|---|---|---|
23.1300 | 1.6335 | 2.7548 | 0.0700 | 0.0065 | 27.5948 |
Body’s Part | Head Part | Main Body Part | Leg Part | ||||||
---|---|---|---|---|---|---|---|---|---|
EER | FAR | GAR | EER | FAR | GAR | EER | FAR | GAR | |
Using Visible Image | 18.238 | 10.000 | 65.670 | 25.788 | 20.000 | 64.472 | 25.175 | 20.000 | 66.787 |
15.000 | 75.548 | 25.000 | 73.242 | 25.000 | 74.298 | ||||
18.242 | 81.767 | 25.794 | 74.217 | 25.256 | 74.906 | ||||
20.000 | 83.931 | 30.000 | 79.404 | 30.000 | 79.806 | ||||
25.000 | 88.913 | 35.000 | 85.630 | 35.000 | 84.513 | ||||
Using Thermal Image | 22.779 | 15.000 | 66.376 | 26.845 | 20.000 | 62.066 | 27.414 | 20.000 | 61.674 |
20.000 | 71.566 | 25.000 | 69.666 | 25.000 | 69.402 | ||||
22.856 | 77.298 | 26.982 | 73.291 | 27.419 | 72.592 | ||||
25.000 | 82.200 | 30.000 | 77.203 | 30.000 | 75.261 | ||||
30.000 | 87.841 | 35.000 | 83.998 | 35.000 | 78.912 | ||||
Our Proposed Method | 15.620 | 10.000 | 70.668 | 20.386 | 15.000 | 65.923 | 22.591 | 15.000 | 67.983 |
15.000 | 83.506 | 20.000 | 76.988 | 20.000 | 73.893 | ||||
15.679 | 84.440 | 20.580 | 79.808 | 22.631 | 77.448 | ||||
20.000 | 88.997 | 25.000 | 84.182 | 25.000 | 79.688 | ||||
25.000 | 92.181 | 30.000 | 88.564 | 30.000 | 84.863 |
4. Conclusions
Acknowledgments
Author Contributions
Conflicts of Interest
References
- Ng, C.B.; Tay, Y.H.; Goi, B.M. Recognizing human gender in computer vision: A survey. Lect. Notes Comput. Sci. 2012, 7458, 335–346. [Google Scholar]
- Lu, L.; Xu, Z.; Shi, P. Gender Classification of Facial Images Based on Multiple Facial Regions. In Proceedings the World Congress on Computer Science and Information Engineering, Los Angeles, CA, USA, 31 March–2 April 2009; Volume 6, pp. 48–52.
- Guo, G.; Mu, G.; Fu, Y. Gender from body: A biologically-inspired approach with manifold learning. Lect. Notes Comput. Sci. 2009, 5996, 236–245. [Google Scholar]
- Ichino, M.; Komatsu, N.; Wang, J.-G.; Yun, Y.W. Speaker Gender Recognition Using Score Level Fusion by Adaboost. In Proceedings of the 11th International Conference on Control Automation Robotics and Vision, Singapore, 7–10 December 2010; pp. 648–653.
- Sloka, S.; Sridharan, S. Automatic Gender Identification Optimised for Language Independence. In Proceedings of the IEEE Annual Conference on Speech and Image Technologies for Computing and Telecommunications, Brisbane, Australia, 4 December 1997; Volume 1, pp. 145–148.
- Arun, K.S.; Rarath, K.S. A Machine Learning Approach for Fingerprint Based Gender Identification. In Proceedings of IEEE Conference on Recent Advances in Intelligent Computational Systems, Trivandrum, India, 22–24 September 2011; pp. 163–167.
- Andreu, Y.; Garcia-Sevilla, P.; Mollineda, R.A. Face gender classification: A statistical study when neutral and distorted faces are combined for training and testing purposes. Image Vis. Comput. 2014, 32, 27–36. [Google Scholar] [CrossRef]
- Bekios-Calfa, J.; Buenaposada, J.M.; Baumela, L. Robust gender recognition by exploiting facial attributes dependencies. Pattern Recognit. Lett. 2014, 36, 228–234. [Google Scholar] [CrossRef]
- Hadid, A.; Pietikainen, M. Combining appearance and motion for face and gender recognition from videos. Pattern Recognit. 2009, 42, 2818–2827. [Google Scholar] [CrossRef]
- Li, X.; Zhao, X.; Fu, Y.; Liu, Y. Bimodal Gender Recognition from Face and Fingerprint. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 13–18 June 2010; pp. 2590–2597.
- Xu, X.-Y.; Yu, B.; Wang, Z.; Yin, Z. A Multimodal Gender Recognition Based on Bayesian Hierarchical Model. In Proceedings of the 5th International Conference on Intelligent Human-Machine Systems and Cybernetics, Hangzhou, China, 26–27 August 2013; pp. 410–413.
- Cao, L.; Dikmen, M.; Fu, Y.; Huang, T.S. Gender Recognition from Body. In Proceedings of the 16th ACM International Conference on Multimedia, New York, NY, USA, 2008; pp. 725–728.
- Tang, J.; Liu, X.; Cheng, H.; Robinette, K.M. Gender Recognition Using 3-D Body Shapes. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 2011, 41, 898–908. [Google Scholar] [CrossRef]
- Tang, J.; Liu, X.; Cheng, H.; Robinette, K.M. Gender Recognition with Limited Feature Points from 3-D Human Body Shapes. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, Seoul, Korea, 14–17 October 2012; pp. 2481–2484.
- Lee, J.H.; Choi, J.-S.; Jeon, E.S.; Kim, Y.G.; Le, T.T.; Shin, K.Y.; Lee, H.C.; Park, K.R. Robust pedestrian detection by combining visible and thermal infrared cameras. Sensors 2015, 15, 10580–10615. [Google Scholar] [CrossRef] [PubMed]
- Nguyen, D.T.; Cho, S.R.; Pham, D.T.; Park, K.R. Human age estimation method robust to camera sensor and/or face movement. Sensors 2015, 15, 21898–21930. [Google Scholar] [CrossRef] [PubMed]
- C600 Webcam Camera. Available online: https://support.logitech.com/en_us/product/5869 (accessed on 29 December 2015).
- Tau2 Thermal Imaging Camera. Available: http://www.flir.com/cores/display/?id=54717 (accessed on 29 December 2015).
- Dalal, N.; Triggs, B. Histogram of Oriented Gradients for Human Detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, 25 June 2005; Volume 1, pp. 886–893.
- Nguyen, D.T.; Park, Y.H.; Shin, K.Y.; Kwon, S.Y.; Lee, H.C.; Park, K.R. Fake finger-vein image detection based on fourier and wavelet transforms. Digit. Signal. Prog. 2013, 23, 1401–1413. [Google Scholar] [CrossRef]
- Lee, W.O.; Kim, Y.G.; Hong, H.G.; Park, K.R. Face recognition system for set-top box-based intelligent TV. Sensors 2014, 11, 21726–21749. [Google Scholar] [CrossRef] [PubMed]
- Nguyen, D.T.; Cho, S.R.; Park, K.R. Human age estimation based on multi-level local binary pattern and regression method. Lect. Notes Electr. Eng. 2014, 309, 433–438. [Google Scholar]
- Torrisi, A.; Farinella, G.M.; Puglisi, G.; Battiato, S. Selecting Discriminative CLBP Patterns for Age Estimation. In Proceedings of the IEEE International Conference on Multimedia & Expo Workshops, Turin, Italy, 29 June–3 July 2015; pp. 1–6.
- Santarcangelo, V.; Farinella, G.M.; Battiato, S. Gender Recognition: Methods, Dataset and Results. In Proceedings of the IEEE International Conference on Multimedia & Expo Workshops, Turin, Italy, 29 June–3 July 2015; pp. 1–6.
- Farinella, G.M.; Farioli, G.; Battiato, S.; Leonardi, S.; Gallo, G. Face re-identification for digital signage applications. Lect. Notes. Compt. Sci. 2014, 8811, 40–52. [Google Scholar]
- OpenCV Library. Available online: http://opencv.org/ (accessed on 29 December 2015).
© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons by Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Nguyen, D.T.; Park, K.R. Body-Based Gender Recognition Using Images from Visible and Thermal Cameras. Sensors 2016, 16, 156. https://doi.org/10.3390/s16020156
Nguyen DT, Park KR. Body-Based Gender Recognition Using Images from Visible and Thermal Cameras. Sensors. 2016; 16(2):156. https://doi.org/10.3390/s16020156
Chicago/Turabian StyleNguyen, Dat Tien, and Kang Ryoung Park. 2016. "Body-Based Gender Recognition Using Images from Visible and Thermal Cameras" Sensors 16, no. 2: 156. https://doi.org/10.3390/s16020156
APA StyleNguyen, D. T., & Park, K. R. (2016). Body-Based Gender Recognition Using Images from Visible and Thermal Cameras. Sensors, 16(2), 156. https://doi.org/10.3390/s16020156