Submit to Sensors Review for Sensors Propose a Special Issue

Journal Browser

► Journal Browser

Visual Sensing and Sensor Fusion for Machine Intelligence

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Intelligent Sensors".

Deadline for manuscript submissions: closed (10 September 2023) | Viewed by 12345

Share This Special Issue

Special Issue Editors

Prof. Dr. Min Young Kim

E-Mail Website
Guest Editor

School of Electronics Engineering, IT College, Kyungpook National University, Daegu 41566, Republic of Korea
Interests: visual sensor; sensor fusion; intelligent sensor system; multi-modal sensor system

Dr. Byeong Hak Kim

E-Mail
Guest Editor

Korea Institute of Industrial Technology, Cheonan 31056, Republic of Korea
Interests: machine learning; object detection/tracking; image enhancement; noise reduction; 3D vision; laser radar; electro optical sensors; thermal imaging systems

Special Issue Information

Dear Colleagues,

Advanced machines and robots require high redundancy and robust sensor systems under variations of the environments that surround them. For this purpose, current intelligent sensor systems adopt multi-modal sensor integration, sensor networks, and machine learning based sensor fusion. In this Special Issue, we will focus on the new, state-of-the-art sensor fusion of the advanced machines and robots to combine new sensors and their intelligence.

Prof. Dr. Min Young Kim
Dr. Byeong Hak Kim
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

sensor fusion
image processing
object detection
object tracking
visual intelligence
machine learning

Published Papers (6 papers)

Download All Papers

Research

43 pages, 10776 KiB

Open AccessArticle

Enhancing Human–Robot Collaboration through a Multi-Module Interaction Framework with Sensor Fusion: Object Recognition, Verbal Communication, User of Interest Detection, Gesture and Gaze Recognition

by Shuvo Kumar Paul, Mircea Nicolescu and Monica Nicolescu

Sensors 2023, 23(13), 5798; https://doi.org/10.3390/s23135798 - 21 Jun 2023

Viewed by 2194

Abstract

With the increasing presence of robots in our daily lives, it is crucial to design interaction interfaces that are natural, easy to use and meaningful for robotic tasks. This is important not only to enhance the user experience but also to increase the task reliability by providing supplementary information. Motivated by this, we propose a multi-modal framework consisting of multiple independent modules. These modules take advantage of multiple sensors (e.g., image, sound, depth) and can be used separately or in combination for effective human–robot collaborative interaction. We identified and implemented four key components of an effective human robot collaborative setting, which included determining object location and pose, extracting intricate information from verbal instructions, resolving user(s) of interest (UOI), and gesture recognition and gaze estimation to facilitate the natural and intuitive interactions. The system uses a feature–detector–descriptor approach for object recognition and a homography-based technique for planar pose estimation and a deep multi-task learning model to extract intricate task parameters from verbal communication. The user of interest (UOI) is detected by estimating the facing state and active speakers. The framework also includes gesture detection and gaze estimation modules, which are combined with a verbal instruction component to form structured commands for robotic entities. Experiments were conducted to assess the performance of these interaction interfaces, and the results demonstrated the effectiveness of the approach. Full article

(This article belongs to the Special Issue Visual Sensing and Sensor Fusion for Machine Intelligence)

► Show Figures

Figure 1

21 pages, 3551 KiB

Open AccessArticle

Fusion Object Detection and Action Recognition to Predict Violent Action

by Nelson R. P. Rodrigues, Nuno M. C. da Costa, César Melo, Ali Abbasi, Jaime C. Fonseca, Paulo Cardoso and João Borges

Sensors 2023, 23(12), 5610; https://doi.org/10.3390/s23125610 - 15 Jun 2023

Viewed by 1710

Abstract

In the context of Shared Autonomous Vehicles, the need to monitor the environment inside the car will be crucial. This article focuses on the application of deep learning algorithms to present a fusion monitoring solution which was three different algorithms: a violent action detection system, which recognizes violent behaviors between passengers, a violent object detection system, and a lost items detection system. Public datasets were used for object detection algorithms (COCO and TAO) to train state-of-the-art algorithms such as YOLOv5. For violent action detection, the MoLa InCar dataset was used to train on state-of-the-art algorithms such as I3D, R(2+1)D, SlowFast, TSN, and TSM. Finally, an embedded automotive solution was used to demonstrate that both methods are running in real-time. Full article

(This article belongs to the Special Issue Visual Sensing and Sensor Fusion for Machine Intelligence)

► Show Figures

Figure 1

18 pages, 7705 KiB

Open AccessArticle

Robust H-K Curvature Map Matching for Patient-to-CT Registration in Neurosurgical Navigation Systems

by Ki Hoon Kwon and Min Young Kim

Sensors 2023, 23(10), 4903; https://doi.org/10.3390/s23104903 - 19 May 2023

Viewed by 1257

Abstract

Image-to-patient registration is a coordinate system matching process between real patients and medical images to actively utilize medical images such as computed tomography (CT) during surgery. This paper mainly deals with a markerless method utilizing scan data of patients and 3D data from CT images. The 3D surface data of the patient are registered to CT data using computer-based optimization methods such as iterative closest point (ICP) algorithms. However, if a proper initial location is not set up, the conventional ICP algorithm has the disadvantages that it takes a long converging time and also suffers from the local minimum problem during the process. We propose an automatic and robust 3D data registration method that can accurately find a proper initial location for the ICP algorithm using curvature matching. The proposed method finds and extracts the matching area for 3D registration by converting 3D CT data and 3D scan data to 2D curvature images and by performing curvature matching between them. Curvature features have characteristics that are robust to translation, rotation, and even some deformation. The proposed image-to-patient registration is implemented with the precise 3D registration of the extracted partial 3D CT data and the patient’s scan data using the ICP algorithm. Full article

(This article belongs to the Special Issue Visual Sensing and Sensor Fusion for Machine Intelligence)

► Show Figures

Figure 1

16 pages, 4503 KiB

Open AccessArticle

Handwritten Multi-Scale Chinese Character Detector with Blended Region Attention Features and Light-Weighted Learning

by Manar Alnaasan and Sungho Kim

Sensors 2023, 23(4), 2305; https://doi.org/10.3390/s23042305 - 18 Feb 2023

Viewed by 1476

Abstract

Character-level detection in historical manuscripts is one of the challenging and valuable tasks in the computer vision field, related directly and effectively to the recognition task. Most of the existing techniques, though promising, seem not powerful and insufficiently accurate to locate characters precisely. In this paper, we present a novel algorithm called free-candidate multiscale Chinese character detection FC-MSCCD, which is based on lateral and fusion connections between multiple feature layers, to successfully predict Chinese characters of different sizes more accurately in old documents. Moreover, cheap training is exploited using cheaper parameters by incorporating a free-candidate detection technique. A bottom-up architecture with connections and concatenations between various dimension feature maps is employed to attain high-quality information that satisfies the positioning criteria of characters, and the implementation of a proposal-free algorithm presents a computation-friendly model. Owing to a lack of handwritten Chinese character datasets from old documents, experiments on newly collected benchmark train and validate FC-MSCCD to show that the proposed detection approach outperforms roughly all other SOTA detection algorithms Full article

(This article belongs to the Special Issue Visual Sensing and Sensor Fusion for Machine Intelligence)

► Show Figures

Figure 1

24 pages, 1583 KiB

Open AccessArticle

ROMI: A Real-Time Optical Digit Recognition Embedded System for Monitoring Patients in Intensive Care Units

by Sanghoon Jeon, Byuk Sung Ko and Sang Hyuk Son

Sensors 2023, 23(2), 638; https://doi.org/10.3390/s23020638 - 5 Jan 2023

Cited by 1 | Viewed by 2405

Abstract

With advances in the Internet of Things, patients in intensive care units are constantly monitored to expedite emergencies. Due to the COVID-19 pandemic, non-face-to-face monitoring has been required for the safety of patients and medical staff. A control center monitors the vital signs of patients in ICUs. However, some medical devices, such as ventilators and infusion pumps, operate in a standalone fashion without communication capabilities, requiring medical staff to check them manually. One promising solution is to use a robotic system with a camera. We propose a real-time optical digit recognition embedded system called ROMI. ROMI is a mobile robot that monitors patients by recognizing digits displayed on LCD screens of medical devices in real time. ROMI consists of three main functions for recognizing digits: digit localization, digit classification, and digit annotation. We developed ROMI by using Matlab Simulink, and the maximum digit recognition performance was 0.989 mAP on alexnet. The developed system was deployed on NVIDIA GPU embedded platforms: Jetson Nano, Jetson Xavier NX, and Jetson AGX Xavier. We also created a benchmark by evaluating the runtime performance by considering ten pre-trained CNN models and three NVIDIA GPU platforms. We expect that ROMI will support medical staff with non-face-to-face monitoring in ICUs, enabling more effective and prompt patient care. Full article

(This article belongs to the Special Issue Visual Sensing and Sensor Fusion for Machine Intelligence)

► Show Figures

Figure 1

17 pages, 8107 KiB

Open AccessArticle

Accurate Ship Detection Using Electro-Optical Image-Based Satellite on Enhanced Feature and Land Awareness

by Sang-Heon Lee, Hae-Gwang Park, Ki-Hoon Kwon, Byeong-Hak Kim, Min Young Kim and Seung-Hyun Jeong

Sensors 2022, 22(23), 9491; https://doi.org/10.3390/s22239491 - 5 Dec 2022

Cited by 4 | Viewed by 2625

Abstract

This paper proposes an algorithm that improves ship detection accuracy using preprocessing and post-processing. To achieve this, high-resolution electro-optical satellite images with a wide range of shape and texture information were considered. The developed algorithms display the problem of unreliable detection of ships owing to clouds, large waves, weather influences, and shadows from large terrains. False detections in land areas with image information similar to that of ships are observed frequently. Therefore, this study involves three algorithms: global feature enhancement pre-processing (GFEP), multiclass ship detector (MSD), and false detected ship exclusion by sea land segmentation image (FDSESI). First, GFEP enhances the image contrast of high-resolution electro-optical satellite images. Second, the MSD extracts many primary ship candidates. Third, falsely detected ships in the land region are excluded using the mask image that divides the sea and land. A series of experiments was performed using the proposed method on a database of 1984 images. The database includes five ship classes. Therefore, a method focused on improving the accuracy of various ships is proposed. The results show a mean average precision (mAP) improvement from 50.55% to 63.39% compared with other deep learning-based detection algorithms. Full article

(This article belongs to the Special Issue Visual Sensing and Sensor Fusion for Machine Intelligence)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Visual Sensing and Sensor Fusion for Machine Intelligence

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Published Papers (6 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI