sensors-logo

Journal Browser

Journal Browser

Human Detection, Identification, and Recognition of Gesture and Behavior based on Thermal Camera

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Physical Sensors".

Deadline for manuscript submissions: closed (30 September 2017) | Viewed by 84371

Special Issue Editors


E-Mail Website
Guest Editor
Division of Electronics and Electrical Engineering, Dongguk University, Seoul, Korea
Interests: Deep Learning, Digital Image Processing, Computer Vision, Biometrics, Computer Graphics

E-Mail Website
Guest Editor
Department of Electrical and Electronic Engineering, Yonsei University, Seoul 03722 Republic of Korea
Interests: human detection and recognition; gesture recognition; face recognition; HEVC
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

A thermal camera image can reveal a human body based on body temperature, which is detectable in the range of medium-wavelength infrared (MWIR) at 3–8 µm and long-wavelength infrared (LWIR) at 8–15 µm. Nevertheless, human detection, identification, and recognition of gestures and behaviors, based on thermal cameras, are challenging issues. This is because thermal cameras often produce an image of low signal-to-noise ratio (SNR) and a halo effect near the human. In addition, the distinction between the human and the background in the thermal image can be reduced when the background temperature is similar to that of the human (notably during the day). Therefore, all these factors can increase errors in human detection and identification, and the recognition of gestures and behaviors using a thermal camera.

The objective of this Special Issue is to invite high-quality, state-of-the-art research papers that deal with challenging issues in human detection, identification, and the recognition of gestures and behaviors using thermal cameras. We solicit the original papers of unpublished and completed research that are not currently under review by any other conference/magazine/journal. Topics of interests include, but are not limited to:

•    human detection using thermal camera
•    human identification using thermal camera
•    gesture recognition using thermal camera
•    behavior recognition using thermal camera
•    human detection, identification, and the recognition of gestures and behaviors using multimodal cameras (thermal camera with visible light camera, or with near-infrared (NIR) camera, etc.)
•    emotion recognition using thermal camera or multimodal cameras
•    applications and case studies

Prof. Kang Ryoung Park
Prof. Sangyoun Lee
Prof. Euntai Kim
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • human detection
  • human identification
  • gesture recognition
  • behavior recognition
  • emotion recognition
  • thermal camera
  • multimodal cameras

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue polices can be found here.

Published Papers (8 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

Jump to: Review

11928 KiB  
Article
Adapting Local Features for Face Detection in Thermal Image
by Chao Ma, Ngo Thanh Trung, Hideaki Uchiyama, Hajime Nagahara, Atsushi Shimada and Rin-ichiro Taniguchi
Sensors 2017, 17(12), 2741; https://doi.org/10.3390/s17122741 - 27 Nov 2017
Cited by 28 | Viewed by 7309
Abstract
A thermal camera captures the temperature distribution of a scene as a thermal image. In thermal images, facial appearances of different people under different lighting conditions are similar. This is because facial temperature distribution is generally constant and not affected by lighting condition. [...] Read more.
A thermal camera captures the temperature distribution of a scene as a thermal image. In thermal images, facial appearances of different people under different lighting conditions are similar. This is because facial temperature distribution is generally constant and not affected by lighting condition. This similarity in face appearances is advantageous for face detection. To detect faces in thermal images, cascade classifiers with Haar-like features are generally used. However, there are few studies exploring the local features for face detection in thermal images. In this paper, we introduce two approaches relying on local features for face detection in thermal images. First, we create new feature types by extending Multi-Block LBP. We consider a margin around the reference and the generally constant distribution of facial temperature. In this way, we make the features more robust to image noise and more effective for face detection in thermal images. Second, we propose an AdaBoost-based training method to get cascade classifiers with multiple types of local features. These feature types have different advantages. In this way we enhance the description power of local features. We did a hold-out validation experiment and a field experiment. In the hold-out validation experiment, we captured a dataset from 20 participants, comprising 14 males and 6 females. For each participant, we captured 420 images with 10 variations in camera distance, 21 poses, and 2 appearances (participant with/without glasses). We compared the performance of cascade classifiers trained by different sets of the features. The experiment results showed that the proposed approaches effectively improve the performance of face detection in thermal images. In the field experiment, we compared the face detection performance in realistic scenes using thermal and RGB images, and gave discussion based on the results. Full article
Show Figures

Figure 1

20611 KiB  
Article
Remote Marker-Based Tracking for UAV Landing Using Visible-Light Camera Sensor
by Phong Ha Nguyen, Ki Wan Kim, Young Won Lee and Kang Ryoung Park
Sensors 2017, 17(9), 1987; https://doi.org/10.3390/s17091987 - 30 Aug 2017
Cited by 59 | Viewed by 13142
Abstract
Unmanned aerial vehicles (UAVs), which are commonly known as drones, have proved to be useful not only on the battlefields where manned flight is considered too risky or difficult, but also in everyday life purposes such as surveillance, monitoring, rescue, unmanned cargo, aerial [...] Read more.
Unmanned aerial vehicles (UAVs), which are commonly known as drones, have proved to be useful not only on the battlefields where manned flight is considered too risky or difficult, but also in everyday life purposes such as surveillance, monitoring, rescue, unmanned cargo, aerial video, and photography. More advanced drones make use of global positioning system (GPS) receivers during the navigation and control loop which allows for smart GPS features of drone navigation. However, there are problems if the drones operate in heterogeneous areas with no GPS signal, so it is important to perform research into the development of UAVs with autonomous navigation and landing guidance using computer vision. In this research, we determined how to safely land a drone in the absence of GPS signals using our remote maker-based tracking algorithm based on the visible light camera sensor. The proposed method uses a unique marker designed as a tracking target during landing procedures. Experimental results show that our method significantly outperforms state-of-the-art object trackers in terms of both accuracy and processing time, and we perform test on an embedded system in various environments. Full article
Show Figures

Figure 1

16326 KiB  
Article
Efficient Pedestrian Detection at Nighttime Using a Thermal Camera
by Jeonghyun Baek, Sungjun Hong, Jisu Kim and Euntai Kim
Sensors 2017, 17(8), 1850; https://doi.org/10.3390/s17081850 - 10 Aug 2017
Cited by 67 | Viewed by 7129
Abstract
Most of the commercial nighttime pedestrian detection (PD) methods reported previously utilized the histogram of oriented gradient (HOG) or the local binary pattern (LBP) as the feature and the support vector machine (SVM) as the classifier using thermal camera images. In this paper, [...] Read more.
Most of the commercial nighttime pedestrian detection (PD) methods reported previously utilized the histogram of oriented gradient (HOG) or the local binary pattern (LBP) as the feature and the support vector machine (SVM) as the classifier using thermal camera images. In this paper, we propose a new feature called the thermal-position-intensity-histogram of oriented gradient (TPIHOG or T π HOG) and developed a new combination of the T π HOG and the additive kernel SVM (AKSVM) for efficient nighttime pedestrian detection. The proposed T π HOG includes detailed information on gradient location; therefore, it has more distinctive power than the HOG. The AKSVM performs better than the linear SVM in terms of detection performance, while it is much faster than other kernel SVMs. The combined T π HOG-AKSVM showed effective nighttime PD performance with fast computational time. The proposed method was experimentally tested with the KAIST pedestrian dataset and showed better performance compared with other conventional methods. Full article
Show Figures

Figure 1

6820 KiB  
Article
Thermal Infrared Pedestrian Image Segmentation Using Level Set Method
by Yulong Qiao, Ziwei Wei and Yan Zhao
Sensors 2017, 17(8), 1811; https://doi.org/10.3390/s17081811 - 6 Aug 2017
Cited by 29 | Viewed by 5773
Abstract
The edge-based active contour model has been one of the most influential models in image segmentation, in which the level set method is usually used to minimize the active contour energy function and then find the desired contour. However, for infrared thermal pedestrian [...] Read more.
The edge-based active contour model has been one of the most influential models in image segmentation, in which the level set method is usually used to minimize the active contour energy function and then find the desired contour. However, for infrared thermal pedestrian images, the traditional level set-based method that utilizes the gradient information as edge indicator function fails to provide the satisfactory boundary of the target. That is due to the poorly defined boundaries and the intensity inhomogeneity. Therefore, we propose a novel level set-based thermal infrared image segmentation method that is able to deal with the above problems. Specifically, we firstly explore the one-bit transform convolution kernel and define a soft mark, from which the target boundary is enhanced. Then we propose a weight function to adaptively adjust the intensity of the infrared image so as to reduce the intensity inhomogeneity. In the level set formulation, those processes can adaptively adjust the edge indicator function, from which the evolving curve will stop at the target boundary. We conduct the experiments on benchmark infrared pedestrian images and compare our introduced method with the state-of-the-art approaches to demonstrate the excellent performance of the proposed method. Full article
Show Figures

Figure 1

6431 KiB  
Article
Person Recognition System Based on a Combination of Body Images from Visible Light and Thermal Cameras
by Dat Tien Nguyen, Hyung Gil Hong, Ki Wan Kim and Kang Ryoung Park
Sensors 2017, 17(3), 605; https://doi.org/10.3390/s17030605 - 16 Mar 2017
Cited by 483 | Viewed by 13217
Abstract
The human body contains identity information that can be used for the person recognition (verification/recognition) problem. In this paper, we propose a person recognition method using the information extracted from body images. Our research is novel in the following three ways compared to [...] Read more.
The human body contains identity information that can be used for the person recognition (verification/recognition) problem. In this paper, we propose a person recognition method using the information extracted from body images. Our research is novel in the following three ways compared to previous studies. First, we use the images of human body for recognizing individuals. To overcome the limitations of previous studies on body-based person recognition that use only visible light images for recognition, we use human body images captured by two different kinds of camera, including a visible light camera and a thermal camera. The use of two different kinds of body image helps us to reduce the effects of noise, background, and variation in the appearance of a human body. Second, we apply a state-of-the art method, called convolutional neural network (CNN) among various available methods, for image features extraction in order to overcome the limitations of traditional hand-designed image feature extraction methods. Finally, with the extracted image features from body images, the recognition task is performed by measuring the distance between the input and enrolled samples. The experimental results show that the proposed method is efficient for enhancing recognition accuracy compared to systems that use only visible light or thermal images of the human body. Full article
Show Figures

Figure 1

2218 KiB  
Article
Supportive Noninvasive Tool for the Diagnosis of Breast Cancer Using a Thermographic Camera as Sensor
by Marco Antonio Garduño-Ramón, Sofia Giovanna Vega-Mancilla, Luis Alberto Morales-Henández and Roque Alfredo Osornio-Rios
Sensors 2017, 17(3), 497; https://doi.org/10.3390/s17030497 - 3 Mar 2017
Cited by 47 | Viewed by 8594
Abstract
Breast cancer is the leading disease in incidence and mortality among women in developing countries. The opportune diagnosis of this disease strengthens the survival index. Mammography application is limited by age and periodicity. Temperature is a physical magnitude that can be measured by [...] Read more.
Breast cancer is the leading disease in incidence and mortality among women in developing countries. The opportune diagnosis of this disease strengthens the survival index. Mammography application is limited by age and periodicity. Temperature is a physical magnitude that can be measured by using multiple sensing techniques. IR (infrared) thermography using commercial cameras is gaining relevance in industrial and medical applications because it is a non-invasive and non-intrusive technology. Asymmetrical temperature in certain human body zones is associated with cancer. In this paper, an IR thermographic sensor is applied for breast cancer detection. This work includes an automatic breast segmentation methodology, to spot the hottest regions in thermograms using the morphological watershed operator to help the experts locate the tumor. A protocol for thermogram acquisition considering the required time to achieve a thermal stabilization is also proposed. Breast thermograms are evaluated as thermal matrices, instead of gray scale or false color images, increasing the certainty of the provided diagnosis. The proposed tool was validated using the Database for Mastology Research and tested in a voluntary group of 454 women of different ages and cancer stages with good results, leading to the possibility of being used as a supportive tool to detect breast cancer and angiogenesis cases. Full article
Show Figures

Figure 1

12991 KiB  
Article
Tracking and Classification of In-Air Hand Gesture Based on Thermal Guided Joint Filter
by Seongwan Kim, Yuseok Ban and Sangyoun Lee
Sensors 2017, 17(1), 166; https://doi.org/10.3390/s17010166 - 17 Jan 2017
Cited by 17 | Viewed by 6273
Abstract
The research on hand gestures has attracted many image processing-related studies, as it intuitively conveys the intention of a human as it pertains to motional meaning. Various sensors have been used to exploit the advantages of different modalities for the extraction of important [...] Read more.
The research on hand gestures has attracted many image processing-related studies, as it intuitively conveys the intention of a human as it pertains to motional meaning. Various sensors have been used to exploit the advantages of different modalities for the extraction of important information conveyed by the hand gesture of a user. Although many works have focused on learning the benefits of thermal information from thermal cameras, most have focused on face recognition or human body detection, rather than hand gesture recognition. Additionally, the majority of the works that take advantage of multiple modalities (e.g., the combination of a thermal sensor and a visual sensor), usually adopting simple fusion approaches between the two modalities. As both thermal sensors and visual sensors have their own shortcomings and strengths, we propose a novel joint filter-based hand gesture recognition method to simultaneously exploit the strengths and compensate the shortcomings of each. Our study is motivated by the investigation of the mutual supplementation between thermal and visual information in low feature level for the consistent representation of a hand in the presence of varying lighting conditions. Accordingly, our proposed method leverages the thermal sensor’s stability against luminance and the visual sensors textural detail, while complementing the low resolution and halo effect of thermal sensors and the weakness against illumination of visual sensors. A conventional region tracking method and a deep convolutional neural network have been leveraged to track the trajectory of a hand gesture and to recognize the hand gesture, respectively. Our experimental results show stability in recognizing a hand gesture against varying lighting conditions based on the contribution of the joint kernels of spatial adjacency and thermal range similarity. Full article
Show Figures

Figure 1

Review

Jump to: Research

11798 KiB  
Review
A Survey on Banknote Recognition Methods by Various Sensors
by Ji Woo Lee, Hyung Gil Hong, Ki Wan Kim and Kang Ryoung Park
Sensors 2017, 17(2), 313; https://doi.org/10.3390/s17020313 - 8 Feb 2017
Cited by 54 | Viewed by 17257
Abstract
Despite a decrease in the use of currency due to the recent growth in the use of electronic financial transactions, real money transactions remain very important in the global market. While performing transactions with real money, touching and counting notes by hand, is [...] Read more.
Despite a decrease in the use of currency due to the recent growth in the use of electronic financial transactions, real money transactions remain very important in the global market. While performing transactions with real money, touching and counting notes by hand, is still a common practice in daily life, various types of automated machines, such as ATMs and banknote counters, are essential for large-scale and safe transactions. This paper presents studies that have been conducted in four major areas of research (banknote recognition, counterfeit banknote detection, serial number recognition, and fitness classification) in the accurate banknote recognition field by various sensors in such automated machines, and describes the advantages and drawbacks of the methods presented in those studies. While to a limited extent some surveys have been presented in previous studies in the areas of banknote recognition or counterfeit banknote recognition, this paper is the first of its kind to review all four areas. Techniques used in each of the four areas recognize banknote information (denomination, serial number, authenticity, and physical condition) based on image or sensor data, and are actually applied to banknote processing machines across the world. This study also describes the technological challenges faced by such banknote recognition techniques and presents future directions of research to overcome them. Full article
Show Figures

Figure 1

Back to TopTop