sensors-logo

Journal Browser

Journal Browser

Special Issue "Human Detection, Identification, and Recognition of Gesture and Behavior based on Thermal Camera"

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Physical Sensors".

Deadline for manuscript submissions: closed (30 September 2017).

Special Issue Editors

Prof. Dr. Kang Ryoung Ryoung Park
E-Mail Website
Guest Editor
Division of Electronics and Electrical Engineering, Dongguk University, Seoul, Korea
Interests: Deep Learning, Digital Image Processing, Computer Vision, Biometrics, Computer Graphics
Prof. Dr. Sangyoun Lee
E-Mail Website
Guest Editor
School of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul 120-749, Korea
Interests: human detection and recognition; gesture recognition; face recognition; HEVC
Special Issues and Collections in MDPI journals
Prof. Dr. Euntai Kim
E-Mail Website1 Website2
Guest Editor
School of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul 120-749, Korea
Interests: pedestrian and vehicle detection; recognition; vision for advanced driver assistance systems (ADAS); robot vision
Special Issues and Collections in MDPI journals

Special Issue Information

Dear Colleagues,

A thermal camera image can reveal a human body based on body temperature, which is detectable in the range of medium-wavelength infrared (MWIR) at 3–8 µm and long-wavelength infrared (LWIR) at 8–15 µm. Nevertheless, human detection, identification, and recognition of gestures and behaviors, based on thermal cameras, are challenging issues. This is because thermal cameras often produce an image of low signal-to-noise ratio (SNR) and a halo effect near the human. In addition, the distinction between the human and the background in the thermal image can be reduced when the background temperature is similar to that of the human (notably during the day). Therefore, all these factors can increase errors in human detection and identification, and the recognition of gestures and behaviors using a thermal camera.

The objective of this Special Issue is to invite high-quality, state-of-the-art research papers that deal with challenging issues in human detection, identification, and the recognition of gestures and behaviors using thermal cameras. We solicit the original papers of unpublished and completed research that are not currently under review by any other conference/magazine/journal. Topics of interests include, but are not limited to:

•    human detection using thermal camera
•    human identification using thermal camera
•    gesture recognition using thermal camera
•    behavior recognition using thermal camera
•    human detection, identification, and the recognition of gestures and behaviors using multimodal cameras (thermal camera with visible light camera, or with near-infrared (NIR) camera, etc.)
•    emotion recognition using thermal camera or multimodal cameras
•    applications and case studies

Prof. Kang Ryoung Park
Prof. Sangyoun Lee
Prof. Euntai Kim
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All papers will be peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2200 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • human detection
  • human identification
  • gesture recognition
  • behavior recognition
  • emotion recognition
  • thermal camera
  • multimodal cameras

Published Papers (8 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

Jump to: Review

Article
Adapting Local Features for Face Detection in Thermal Image
Sensors 2017, 17(12), 2741; https://doi.org/10.3390/s17122741 - 27 Nov 2017
Cited by 15 | Viewed by 3542
Abstract
A thermal camera captures the temperature distribution of a scene as a thermal image. In thermal images, facial appearances of different people under different lighting conditions are similar. This is because facial temperature distribution is generally constant and not affected by lighting condition. [...] Read more.
A thermal camera captures the temperature distribution of a scene as a thermal image. In thermal images, facial appearances of different people under different lighting conditions are similar. This is because facial temperature distribution is generally constant and not affected by lighting condition. This similarity in face appearances is advantageous for face detection. To detect faces in thermal images, cascade classifiers with Haar-like features are generally used. However, there are few studies exploring the local features for face detection in thermal images. In this paper, we introduce two approaches relying on local features for face detection in thermal images. First, we create new feature types by extending Multi-Block LBP. We consider a margin around the reference and the generally constant distribution of facial temperature. In this way, we make the features more robust to image noise and more effective for face detection in thermal images. Second, we propose an AdaBoost-based training method to get cascade classifiers with multiple types of local features. These feature types have different advantages. In this way we enhance the description power of local features. We did a hold-out validation experiment and a field experiment. In the hold-out validation experiment, we captured a dataset from 20 participants, comprising 14 males and 6 females. For each participant, we captured 420 images with 10 variations in camera distance, 21 poses, and 2 appearances (participant with/without glasses). We compared the performance of cascade classifiers trained by different sets of the features. The experiment results showed that the proposed approaches effectively improve the performance of face detection in thermal images. In the field experiment, we compared the face detection performance in realistic scenes using thermal and RGB images, and gave discussion based on the results. Full article
Show Figures

Figure 1

Article
Remote Marker-Based Tracking for UAV Landing Using Visible-Light Camera Sensor
Sensors 2017, 17(9), 1987; https://doi.org/10.3390/s17091987 - 30 Aug 2017
Cited by 25 | Viewed by 4645
Abstract
Unmanned aerial vehicles (UAVs), which are commonly known as drones, have proved to be useful not only on the battlefields where manned flight is considered too risky or difficult, but also in everyday life purposes such as surveillance, monitoring, rescue, unmanned cargo, aerial [...] Read more.
Unmanned aerial vehicles (UAVs), which are commonly known as drones, have proved to be useful not only on the battlefields where manned flight is considered too risky or difficult, but also in everyday life purposes such as surveillance, monitoring, rescue, unmanned cargo, aerial video, and photography. More advanced drones make use of global positioning system (GPS) receivers during the navigation and control loop which allows for smart GPS features of drone navigation. However, there are problems if the drones operate in heterogeneous areas with no GPS signal, so it is important to perform research into the development of UAVs with autonomous navigation and landing guidance using computer vision. In this research, we determined how to safely land a drone in the absence of GPS signals using our remote maker-based tracking algorithm based on the visible light camera sensor. The proposed method uses a unique marker designed as a tracking target during landing procedures. Experimental results show that our method significantly outperforms state-of-the-art object trackers in terms of both accuracy and processing time, and we perform test on an embedded system in various environments. Full article
Show Figures

Figure 1

Article
Efficient Pedestrian Detection at Nighttime Using a Thermal Camera
Sensors 2017, 17(8), 1850; https://doi.org/10.3390/s17081850 - 10 Aug 2017
Cited by 22 | Viewed by 2950
Abstract
Most of the commercial nighttime pedestrian detection (PD) methods reported previously utilized the histogram of oriented gradient (HOG) or the local binary pattern (LBP) as the feature and the support vector machine (SVM) as the classifier using thermal camera images. In this paper, [...] Read more.
Most of the commercial nighttime pedestrian detection (PD) methods reported previously utilized the histogram of oriented gradient (HOG) or the local binary pattern (LBP) as the feature and the support vector machine (SVM) as the classifier using thermal camera images. In this paper, we propose a new feature called the thermal-position-intensity-histogram of oriented gradient (TPIHOG or T π HOG) and developed a new combination of the T π HOG and the additive kernel SVM (AKSVM) for efficient nighttime pedestrian detection. The proposed T π HOG includes detailed information on gradient location; therefore, it has more distinctive power than the HOG. The AKSVM performs better than the linear SVM in terms of detection performance, while it is much faster than other kernel SVMs. The combined T π HOG-AKSVM showed effective nighttime PD performance with fast computational time. The proposed method was experimentally tested with the KAIST pedestrian dataset and showed better performance compared with other conventional methods. Full article
Show Figures

Figure 1

Article
Thermal Infrared Pedestrian Image Segmentation Using Level Set Method
Sensors 2017, 17(8), 1811; https://doi.org/10.3390/s17081811 - 06 Aug 2017
Cited by 19 | Viewed by 2641
Abstract
The edge-based active contour model has been one of the most influential models in image segmentation, in which the level set method is usually used to minimize the active contour energy function and then find the desired contour. However, for infrared thermal pedestrian [...] Read more.
The edge-based active contour model has been one of the most influential models in image segmentation, in which the level set method is usually used to minimize the active contour energy function and then find the desired contour. However, for infrared thermal pedestrian images, the traditional level set-based method that utilizes the gradient information as edge indicator function fails to provide the satisfactory boundary of the target. That is due to the poorly defined boundaries and the intensity inhomogeneity. Therefore, we propose a novel level set-based thermal infrared image segmentation method that is able to deal with the above problems. Specifically, we firstly explore the one-bit transform convolution kernel and define a soft mark, from which the target boundary is enhanced. Then we propose a weight function to adaptively adjust the intensity of the infrared image so as to reduce the intensity inhomogeneity. In the level set formulation, those processes can adaptively adjust the edge indicator function, from which the evolving curve will stop at the target boundary. We conduct the experiments on benchmark infrared pedestrian images and compare our introduced method with the state-of-the-art approaches to demonstrate the excellent performance of the proposed method. Full article
Show Figures

Figure 1

Article
Person Recognition System Based on a Combination of Body Images from Visible Light and Thermal Cameras
Sensors 2017, 17(3), 605; https://doi.org/10.3390/s17030605 - 16 Mar 2017
Cited by 90 | Viewed by 3612
Abstract
The human body contains identity information that can be used for the person recognition (verification/recognition) problem. In this paper, we propose a person recognition method using the information extracted from body images. Our research is novel in the following three ways compared to [...] Read more.
The human body contains identity information that can be used for the person recognition (verification/recognition) problem. In this paper, we propose a person recognition method using the information extracted from body images. Our research is novel in the following three ways compared to previous studies. First, we use the images of human body for recognizing individuals. To overcome the limitations of previous studies on body-based person recognition that use only visible light images for recognition, we use human body images captured by two different kinds of camera, including a visible light camera and a thermal camera. The use of two different kinds of body image helps us to reduce the effects of noise, background, and variation in the appearance of a human body. Second, we apply a state-of-the art method, called convolutional neural network (CNN) among various available methods, for image features extraction in order to overcome the limitations of traditional hand-designed image feature extraction methods. Finally, with the extracted image features from body images, the recognition task is performed by measuring the distance between the input and enrolled samples. The experimental results show that the proposed method is efficient for enhancing recognition accuracy compared to systems that use only visible light or thermal images of the human body. Full article
Show Figures

Figure 1

Article
Supportive Noninvasive Tool for the Diagnosis of Breast Cancer Using a Thermographic Camera as Sensor
Sensors 2017, 17(3), 497; https://doi.org/10.3390/s17030497 - 03 Mar 2017
Cited by 22 | Viewed by 4009
Abstract
Breast cancer is the leading disease in incidence and mortality among women in developing countries. The opportune diagnosis of this disease strengthens the survival index. Mammography application is limited by age and periodicity. Temperature is a physical magnitude that can be measured by [...] Read more.
Breast cancer is the leading disease in incidence and mortality among women in developing countries. The opportune diagnosis of this disease strengthens the survival index. Mammography application is limited by age and periodicity. Temperature is a physical magnitude that can be measured by using multiple sensing techniques. IR (infrared) thermography using commercial cameras is gaining relevance in industrial and medical applications because it is a non-invasive and non-intrusive technology. Asymmetrical temperature in certain human body zones is associated with cancer. In this paper, an IR thermographic sensor is applied for breast cancer detection. This work includes an automatic breast segmentation methodology, to spot the hottest regions in thermograms using the morphological watershed operator to help the experts locate the tumor. A protocol for thermogram acquisition considering the required time to achieve a thermal stabilization is also proposed. Breast thermograms are evaluated as thermal matrices, instead of gray scale or false color images, increasing the certainty of the provided diagnosis. The proposed tool was validated using the Database for Mastology Research and tested in a voluntary group of 454 women of different ages and cancer stages with good results, leading to the possibility of being used as a supportive tool to detect breast cancer and angiogenesis cases. Full article
Show Figures

Figure 1

Article
Tracking and Classification of In-Air Hand Gesture Based on Thermal Guided Joint Filter
Sensors 2017, 17(1), 166; https://doi.org/10.3390/s17010166 - 17 Jan 2017
Cited by 7 | Viewed by 2918
Abstract
The research on hand gestures has attracted many image processing-related studies, as it intuitively conveys the intention of a human as it pertains to motional meaning. Various sensors have been used to exploit the advantages of different modalities for the extraction of important [...] Read more.
The research on hand gestures has attracted many image processing-related studies, as it intuitively conveys the intention of a human as it pertains to motional meaning. Various sensors have been used to exploit the advantages of different modalities for the extraction of important information conveyed by the hand gesture of a user. Although many works have focused on learning the benefits of thermal information from thermal cameras, most have focused on face recognition or human body detection, rather than hand gesture recognition. Additionally, the majority of the works that take advantage of multiple modalities (e.g., the combination of a thermal sensor and a visual sensor), usually adopting simple fusion approaches between the two modalities. As both thermal sensors and visual sensors have their own shortcomings and strengths, we propose a novel joint filter-based hand gesture recognition method to simultaneously exploit the strengths and compensate the shortcomings of each. Our study is motivated by the investigation of the mutual supplementation between thermal and visual information in low feature level for the consistent representation of a hand in the presence of varying lighting conditions. Accordingly, our proposed method leverages the thermal sensor’s stability against luminance and the visual sensors textural detail, while complementing the low resolution and halo effect of thermal sensors and the weakness against illumination of visual sensors. A conventional region tracking method and a deep convolutional neural network have been leveraged to track the trajectory of a hand gesture and to recognize the hand gesture, respectively. Our experimental results show stability in recognizing a hand gesture against varying lighting conditions based on the contribution of the joint kernels of spatial adjacency and thermal range similarity. Full article
Show Figures

Figure 1

Review

Jump to: Research

Review
A Survey on Banknote Recognition Methods by Various Sensors
Sensors 2017, 17(2), 313; https://doi.org/10.3390/s17020313 - 08 Feb 2017
Cited by 21 | Viewed by 3501
Abstract
Despite a decrease in the use of currency due to the recent growth in the use of electronic financial transactions, real money transactions remain very important in the global market. While performing transactions with real money, touching and counting notes by hand, is [...] Read more.
Despite a decrease in the use of currency due to the recent growth in the use of electronic financial transactions, real money transactions remain very important in the global market. While performing transactions with real money, touching and counting notes by hand, is still a common practice in daily life, various types of automated machines, such as ATMs and banknote counters, are essential for large-scale and safe transactions. This paper presents studies that have been conducted in four major areas of research (banknote recognition, counterfeit banknote detection, serial number recognition, and fitness classification) in the accurate banknote recognition field by various sensors in such automated machines, and describes the advantages and drawbacks of the methods presented in those studies. While to a limited extent some surveys have been presented in previous studies in the areas of banknote recognition or counterfeit banknote recognition, this paper is the first of its kind to review all four areas. Techniques used in each of the four areas recognize banknote information (denomination, serial number, authenticity, and physical condition) based on image or sensor data, and are actually applied to banknote processing machines across the world. This study also describes the technological challenges faced by such banknote recognition techniques and presents future directions of research to overcome them. Full article
Show Figures

Figure 1

Back to TopTop