16 pages, 4911 KiB  
Article
An FPGA-Embedded Brain-Computer Interface System to Support Individual Autonomy in Locked-In Individuals
by Arrigo Palumbo, Nicola Ielpo and Barbara Calabrese *
Department of Medical and Surgical Sciences, Magna Graecia University, Viale Europa, 88100 Catanzaro, Italy
Sensors 2022, 22(1), 318; https://doi.org/10.3390/s22010318 - 1 Jan 2022
Cited by 7 | Viewed by 4103
Abstract
Brain-computer interfaces (BCI) can detect specific EEG patterns and translate them into control signals for external devices by providing people suffering from severe motor disabilities with an alternative/additional channel to communicate and interact with the outer world. Many EEG-based BCIs rely on the [...] Read more.
Brain-computer interfaces (BCI) can detect specific EEG patterns and translate them into control signals for external devices by providing people suffering from severe motor disabilities with an alternative/additional channel to communicate and interact with the outer world. Many EEG-based BCIs rely on the P300 event-related potentials, mainly because they require training times for the user relatively short and provide higher selection speed. This paper proposes a P300-based portable embedded BCI system realized through an embedded hardware platform based on FPGA (field-programmable gate array), ensuring flexibility, reliability, and high-performance features. The system acquires EEG data during user visual stimulation and processes them in a real-time way to correctly detect and recognize the EEG features. The BCI system is designed to allow to user to perform communication and domotic controls. Full article
(This article belongs to the Special Issue Integrated Circuits and Technologies for Real-Time Sensing)
Show Figures

Figure 1

12 pages, 407 KiB  
Communication
Fault Identification in Electric Servo Actuators of Robot Manipulators Described by Nonstationary Nonlinear Dynamic Models Using Sliding Mode Observers
by Alexander Zuev 1,*, Alexey N. Zhirabok 1,2, Vladimir Filaretov 3,4 and Alexander Protsenko 1
1 Laboratory of Intelligent Information Systems for Marine Robots, Institute of Marine Technology Problems, 690091 Vladivostok, Russia
2 Department of Automation and Robotics, Far Eastern Federal University, 690091 Vladivostok, Russia
3 Robotics Laboratory, Institute of Automation and Control Processes, 690041 Vladivostok, Russia
4 Department of Informatics and Control in Technical Systems, Sevastopol State University, 299053 Sevastopol, Russia
Sensors 2022, 22(1), 317; https://doi.org/10.3390/s22010317 - 1 Jan 2022
Cited by 3 | Viewed by 1938
Abstract
The problem of fault identification in electric servo actuators of robot manipulators described by nonstationary nonlinear dynamic models under disturbances is considered. To solve the problem, sliding mode observers are used. The suggested approach is based on the reduced order model of the [...] Read more.
The problem of fault identification in electric servo actuators of robot manipulators described by nonstationary nonlinear dynamic models under disturbances is considered. To solve the problem, sliding mode observers are used. The suggested approach is based on the reduced order model of the original system having different sensitivity to faults and disturbances. This model is realized in canonical form that enables relaxing the limitation imposed on the original system. Theoretical results are illustrated by practical example. Full article
(This article belongs to the Special Issue Smart Sensor-Based Robot Control and Calibration)
Show Figures

Figure 1

12 pages, 29171 KiB  
Article
The Application of Microelectromechanical Systems (MEMS) Accelerometers to the Assessment of Blast Threat to Armored Vehicle Crew
by Sławomir Kciuk 1, Edyta Krzystała 1,*, Arkadiusz Mężyk 1 and Paweł Szmidt 2
1 Department of Theoretical and Applied Mechanics, Faculty of Mechanical Engineering, Silesian University of Technology, 44-100 Gliwice, Poland
2 DUAL-PROJEKT Szmidt Paweł, 44-141 Gliwice, Poland
Sensors 2022, 22(1), 316; https://doi.org/10.3390/s22010316 - 31 Dec 2021
Cited by 6 | Viewed by 2644
Abstract
This paper describes the development and application of an autonomous register and measurement system (ARMS), and the application of microelectromechanical systems (MEMS) accelerometers to the assessment of blast threat to armored vehicle crews. Taking measurements with reference to an explosion is one of [...] Read more.
This paper describes the development and application of an autonomous register and measurement system (ARMS), and the application of microelectromechanical systems (MEMS) accelerometers to the assessment of blast threat to armored vehicle crews. Taking measurements with reference to an explosion is one of the principal issues in the protection of crews of special vehicles. The proposed ARMS reduces research costs and contributes to the development of an autonomous, wireless test stand, applicable in various research areas and industry. The ARMS performs data acquisition with simultaneous measurement in multiple channels. The maximum sampling rate is 100 kHz and the sensor range is ±500 g. This solution is an alternative to cable systems, which have a high energy demand. The functionality of the developed autonomous measuring system is demonstrated experimentally. The paper concludes with a field study of the proposed system and the application of MEMS accelerometers via a mine blast test of a military vehicle at level 4 of STANAG 4569. Full article
(This article belongs to the Special Issue Intelligent Mechatronic Systems—Materials, Sensors and Interfaces)
Show Figures

Figure 1

13 pages, 2943 KiB  
Article
Free Space Detection Algorithm Using Object Tracking for Autonomous Vehicles
by Yeongwon Lee and Byungyong You *
Department of Mechanical Engineering, Kyungil University, Gyeongsan 38428, Korea
Sensors 2022, 22(1), 315; https://doi.org/10.3390/s22010315 - 31 Dec 2021
Cited by 5 | Viewed by 6887
Abstract
In this paper, we propose a new free space detection algorithm for autonomous vehicle driving. Previous free space detection algorithms often use only the location information of every frame, without information on the speed of the obstacle. In this case, there is a [...] Read more.
In this paper, we propose a new free space detection algorithm for autonomous vehicle driving. Previous free space detection algorithms often use only the location information of every frame, without information on the speed of the obstacle. In this case, there is a possibility of creating an inefficient path because the behavior of the obstacle cannot be predicted. In order to compensate for the shortcomings of the previous algorithm, the proposed algorithm uses the speed information of the obstacle. Through object tracking, the dynamic behavior of obstacles around the vehicle is identified and predicted, and free space is detected based on this. In the free space, it is possible to classify an area in which driving is possible and an area in which it is not possible, and a route is created according to the classification result. By comparing and evaluating the path generated by the previous algorithm and the path generated by the proposed algorithm, it is confirmed that the proposed algorithm is more efficient in generating the vehicle driving path. Full article
(This article belongs to the Special Issue Advances in Sensor Related Technologies for Autonomous Driving)
Show Figures

Figure 1

23 pages, 9807 KiB  
Article
True 3D Nanometrology: 3D-Probing with a Cantilever-Based Sensor
by Jan Thiesler 1,*, Thomas Ahbe 1, Rainer Tutsch 2 and Gaoliang Dai 1,*
1 Physikalisch-Technische Bundesanstalt (PTB), Bundesallee 100, 38116 Braunschweig, Germany
2 IPROM, Technische Universität Braunschweig, Schleinitzstraße 20, 38106 Braunschweig, Germany
Sensors 2022, 22(1), 314; https://doi.org/10.3390/s22010314 - 31 Dec 2021
Cited by 6 | Viewed by 2865
Abstract
State of the art three-dimensional atomic force microscopes (3D-AFM) cannot measure three spatial dimensions separately from each other. A 3D-AFM-head with true 3D-probing capabilities is presented in this paper. It detects the so-called 3D-Nanoprobes CD-tip displacement with a differential interferometer and an optical [...] Read more.
State of the art three-dimensional atomic force microscopes (3D-AFM) cannot measure three spatial dimensions separately from each other. A 3D-AFM-head with true 3D-probing capabilities is presented in this paper. It detects the so-called 3D-Nanoprobes CD-tip displacement with a differential interferometer and an optical lever. The 3D-Nanoprobe was specifically developed for tactile 3D-probing and is applied for critical dimension (CD) measurements. A calibrated 3D-Nanoprobe shows a selectivity ratio of 50:1 on average for each of the spatial directions x, y, and z. Typical stiffness values are kx = 1.722 ± 0.083 N/m, ky = 1.511 ± 0.034 N/m, and kz = 1.64 ± 0.16 N/m resulting in a quasi-isotropic ratio of the stiffness of 1.1:0.9:1.0 in x:y:z, respectively. The probing repeatability of the developed true 3D-AFM shows a standard deviation of 0.18 nm, 0.31 nm, and 0.83 nm for x, y, and z, respectively. Two CD-line samples type IVPS100-PTB, which were perpendicularly mounted to each other, were used to test the performance of the developed true 3D-AFM: repeatability, long-term stability, pitch, and line edge roughness and linewidth roughness (LER/LWR), showing promising results. Full article
(This article belongs to the Special Issue Cantilever Sensors for Industrial Applications)
Show Figures

Figure 1

13 pages, 3232 KiB  
Article
Generalized Frequency Division Multiplexing-Based Low-Power Underwater Acoustic Image Transceiver
by Chin-Feng Lin 1,*, Cheng-Fong Wu 1, Ching-Lung Hsieh 1, Shun-Hsyung Chang 2,*, Ivan A. Parinov 3 and Sergey Shevtsov 4
1 Department of Electrical Engineering, National Taiwan Ocean University, Keelung 20224, Taiwan
2 Department of Microelectronics Engineering, National Kaohsuing University of Science and Technology, Kaohsuing 81157, Taiwan
3 I. I. Vorovich Mathematics, Mechanics, and Computer Science Institute, Southern Federal University, 344090 Rostov-on-Don, Russia
4 Head of Aircraft Systems and Technologies Lab at the South Center of Russian Academy of Science, 344006 Rostov-on-Don, Russia
Sensors 2022, 22(1), 313; https://doi.org/10.3390/s22010313 - 31 Dec 2021
Cited by 2 | Viewed by 2363
Abstract
In this paper, a low-power underwater acoustic (UWA) image transceiver based on generalized frequency division multiplexing (GFDM) modulation for underwater communication is proposed. The proposed transceiver integrates a low-density parity-check code error protection scheme, adaptive 4-quadrature amplitude modulation (QAM) and 16-QAM strategies, GFDM [...] Read more.
In this paper, a low-power underwater acoustic (UWA) image transceiver based on generalized frequency division multiplexing (GFDM) modulation for underwater communication is proposed. The proposed transceiver integrates a low-density parity-check code error protection scheme, adaptive 4-quadrature amplitude modulation (QAM) and 16-QAM strategies, GFDM modulation, and a power assignment mechanism in an UWA image communication environment. The transmission bit error rates (BERs), the peak signal-to-noise ratios (PSNRs) of the received underwater images, and the power-saving ratio (PSR) of the proposed transceiver obtained using 4-QAM and 16-QAM, with perfect channel estimation, and channel estimation errors (CEEs) of 5%, 10%, and 20% were simulated. The PSNR of the received underwater image is 44.46 dB when using 4-QAM with a CEE of 10%. In contrast, PSNR is 48.79 dB when using 16-QAM with a CEE of 10%. When BER is 10−4, the received UW images have high PSNR values and high resolutions, indicating that the proposed transceiver is suitable for underwater image sensor signal transmission. Full article
(This article belongs to the Special Issue Mathematical Modelling and Analysis in Sensors Networks)
Show Figures

Figure 1

23 pages, 71061 KiB  
Article
An IoT-Enabled Information System for Smart Navigation in Museums
by Muhammad Nawaz Khan 1, Haseeb Ur Rahman 1, Mohammad Faisal 1, Faheem Khan 2,* and Shabir Ahmad 2
1 Network System & Security Research Group, Department of Computer Science & IT, University of Malakand, Chakdara 18800, Pakistan
2 Department of Computer Engineering, Gachon University, Seongnam 13120, Korea
Sensors 2022, 22(1), 312; https://doi.org/10.3390/s22010312 - 31 Dec 2021
Cited by 13 | Viewed by 5075
Abstract
The Internet of Things (IoT) is a new paradigm that connects objects to provide seamless communication and contextual information to anyone, anywhere, at any time (AAA). These Internet-of-Things-enabled automated objects interact with visitors to present a variety of information during museum navigation and [...] Read more.
The Internet of Things (IoT) is a new paradigm that connects objects to provide seamless communication and contextual information to anyone, anywhere, at any time (AAA). These Internet-of-Things-enabled automated objects interact with visitors to present a variety of information during museum navigation and exploration. In this article, a smart navigation and information system (SNIS) prototype for museum navigation and exploration is developed, which delivers an interactive and more exciting museum exploration experience based on the visitor’s personal presence. The objects inside a museum share the information that assist and navigate the visitors about the different sections and objects of the museum. The system was deployed inside Chakdara Museum and experimented with 381 users to achieve the results. For results, different users marked the proposed system in terms of parameters such as interesting, reality, ease of use, satisfaction, usefulness, and user friendly. Of these 381 users, 201 marked the system as most interesting, 138 marked most realistic, 121 marked it as easy-in-use, 219 marked it useful, and 210 marked it as user friendly. These statistics prove the efficiency of SNIS and its usefulness in smart cultural heritage, including smart museums, exhibitions and cultural sites. Full article
Show Figures

Figure 1

19 pages, 3733 KiB  
Article
Electrochemical Properties of Phytosynthesized Gold Nanoparticles for Electrosensing
by Natalia Yu. Stozhko 1,*, Maria A. Bukharinova 2, Ekaterina I. Khamzina 1,2 and Aleksey V. Tarasov 2
1 Department of Physics and Chemistry, Ural State University of Economics, 8 Marta St., 62, 620144 Yekaterinburg, Russia
2 Scientific and Innovation Center of Sensor Technologies, Ural State University of Economics, 8 Marta St., 62, 620144 Yekaterinburg, Russia
Sensors 2022, 22(1), 311; https://doi.org/10.3390/s22010311 - 31 Dec 2021
Cited by 13 | Viewed by 3140
Abstract
Gold nanoparticles are widely used in electrosensing. The current trend is to phytosynthesize gold nanoparticles (phyto-AuNPs) on the basis of the “green” chemistry approach. Phyto-AuNPs are biologically and catalytically active, stable and biocompatible, which opens up broad perspectives in a variety of applications, [...] Read more.
Gold nanoparticles are widely used in electrosensing. The current trend is to phytosynthesize gold nanoparticles (phyto-AuNPs) on the basis of the “green” chemistry approach. Phyto-AuNPs are biologically and catalytically active, stable and biocompatible, which opens up broad perspectives in a variety of applications, including tactile, wearable (bio)sensors. However, the electrochemistry of phytosynthesized nanoparticles is not sufficiently studied. This work offers a comprehensive study of the electrochemical activity of phyto-AuNPs depending on the synthesis conditions. It was found that with an increase in the aliquot of the plant extract, its antioxidant activity (AOA) and pH, the electrochemical activity of phyto-AuNPs grows, which is reflected in the peak potential decrease and an increase in the peak current of phyto-AuNPs electrooxidation. It has been shown that AOA is an important parameter for obtaining phyto-AuNPs with desired properties. Electrodes modified with phyto-AuNPs have demonstrated better analytical characteristics than electrodes with citrate AuNPs in detecting uric and ascorbic acids under model conditions. The data about the phyto-AuNPs’ electrochemistry may be useful for creating highly effective epidermal sensors with good biocompatibility. Full article
(This article belongs to the Special Issue Micro- and Nanostructures for Sensing Applications)
Show Figures

Figure 1

22 pages, 29837 KiB  
Article
The Design and Development of a Ship Trajectory Data Management and Analysis System Based on AIS
by Chengxu Feng 1, Bing Fu 1, Yasong Luo 1,* and Houpu Li 2
1 College of Weapon Engineering, Naval University of Engineering, Wuhan 430033, China
2 College of Electrical Engineering, Naval University of Engineering, Wuhan 430033, China
Sensors 2022, 22(1), 310; https://doi.org/10.3390/s22010310 - 31 Dec 2021
Cited by 13 | Viewed by 3541
Abstract
To address the data storage, management, analysis, and mining of ship targets, the object-oriented method was employed to design the overall structure and functional modules of a ship trajectory data management and analysis system (STDMAS). This paper elaborates the detailed design and technical [...] Read more.
To address the data storage, management, analysis, and mining of ship targets, the object-oriented method was employed to design the overall structure and functional modules of a ship trajectory data management and analysis system (STDMAS). This paper elaborates the detailed design and technical information of the system’s logical structure, module composition, physical deployment, and main functional modules such as database management, trajectory analysis, trajectory mining, and situation analysis. A ship identification method based on the motion features was put forward. With the method, ship trajectory was first partitioned into sub-trajectories in various behavioral patterns, and effective motion features were then extracted. Machine learning algorithms were utilized for training and testing to identify many types of ships. STDMAS implements such functions as database management, trajectory analysis, historical situation review, and ship identification and outlier detection based on trajectory classification. STDMAS can satisfy the practical needs for the data management, analysis, and mining of maritime targets because it is easy to apply, maintain, and expand. Full article
(This article belongs to the Special Issue Recent Advances in Underwater Signal Processing)
Show Figures

Figure 1

41 pages, 915 KiB  
Review
Application of Reinforcement Learning and Deep Learning in Multiple-Input and Multiple-Output (MIMO) Systems
by Muddasar Naeem *, Giuseppe De Pietro and Antonio Coronato
Institute of High Performance Computing and Networking, National Research Council of Italy, 80131 Napoli, Italy
Sensors 2022, 22(1), 309; https://doi.org/10.3390/s22010309 - 31 Dec 2021
Cited by 33 | Viewed by 8472
Abstract
The current wireless communication infrastructure has to face exponential development in mobile traffic size, which demands high data rate, reliability, and low latency. MIMO systems and their variants (i.e., Multi-User MIMO and Massive MIMO) are the most promising 5G wireless communication systems technology [...] Read more.
The current wireless communication infrastructure has to face exponential development in mobile traffic size, which demands high data rate, reliability, and low latency. MIMO systems and their variants (i.e., Multi-User MIMO and Massive MIMO) are the most promising 5G wireless communication systems technology due to their high system throughput and data rate. However, the most significant challenges in MIMO communication are substantial problems in exploiting the multiple-antenna and computational complexity. The recent success of RL and DL introduces novel and powerful tools that mitigate issues in MIMO communication systems. This article focuses on RL and DL techniques for MIMO systems by presenting a comprehensive review on the integration between the two areas. We first briefly provide the necessary background to RL, DL, and MIMO. Second, potential RL and DL applications for different MIMO issues, such as detection, classification, and compression; channel estimation; positioning, sensing, and localization; CSI acquisition and feedback, security, and robustness; mmWave communication and resource allocation, are presented. Full article
(This article belongs to the Special Issue Ambient Intelligence in Healthcare)
Show Figures

Figure 1

38 pages, 5934 KiB  
Article
Clustering Users to Determine the Most Suitable Gamification Elements
by Alejandro Blanco-M, Ruth S. Contreras-Espinosa * and Jordi Solé-Casals *
Data and Signal Processing Group, University of Vic-Central University of Catalonia, c/de la Laura 13, 08500 Vic, Catalonia, Spain
Sensors 2022, 22(1), 308; https://doi.org/10.3390/s22010308 - 31 Dec 2021
Cited by 4 | Viewed by 3165
Abstract
The use of gamification elements has extended from being a complement for a product to being integrated into multiple public services to motivate the user. The first drawback for service designers is choosing which gamification elements are appropriate for the intended audience, in [...] Read more.
The use of gamification elements has extended from being a complement for a product to being integrated into multiple public services to motivate the user. The first drawback for service designers is choosing which gamification elements are appropriate for the intended audience, in addition to the possible incompatibilities between gamification elements. This work proposes a clustering technique that enables mapping different user profiles in relation to their preferred gamification elements. Additionally, by mapping the best cluster for each gamification element, it is possible to determine the preferred game genre. The article answered the following research questions: What is the relationship between the genre of the game and the element of gamification? Different user groups (profiles) for each gamification element? Results indicate that there are cases where the users are divided between those who agree or disagree. However, other elements present a great heterogeneity in the number of groups and the levels of agreement. Full article
(This article belongs to the Special Issue Pervasive Mobile-Based Games, AR/VR and Sensors)
Show Figures

Figure 1

19 pages, 9172 KiB  
Article
Fatigue Crack Evaluation with the Guided Wave–Convolutional Neural Network Ensemble and Differential Wavelet Spectrogram
by Jian Chen, Wenyang Wu, Yuanqiang Ren and Shenfang Yuan *
Research Center of Structural Health Monitoring and Prognosis, State Key Laboratory of Mechanics and Control of Mechanical Structures, Nanjing University of Aeronautics and Astronautics, No. 29 Yudao Street, Nanjing 210016, China
Sensors 2022, 22(1), 307; https://doi.org/10.3390/s22010307 - 31 Dec 2021
Cited by 19 | Viewed by 3288
Abstract
On-line fatigue crack evaluation is crucial for ensuring the structural safety and reducing the maintenance costs of safety-critical systems. Among structural health monitoring (SHM), guided wave (GW)-based SHM has been deemed as one of the most promising techniques. However, the traditional damage index-based [...] Read more.
On-line fatigue crack evaluation is crucial for ensuring the structural safety and reducing the maintenance costs of safety-critical systems. Among structural health monitoring (SHM), guided wave (GW)-based SHM has been deemed as one of the most promising techniques. However, the traditional damage index-based method and machine learning methods require manual processing and selection of GW features, which depend highly on expert knowledge and are easily affected by complicated uncertainties. Therefore, this paper proposes a fatigue crack evaluation framework with the GW–convolutional neural network (CNN) ensemble and differential wavelet spectrogram. The differential time–frequency spectrogram between the baseline signal and the monitoring signal is processed as the CNN input with the complex Gaussian wavelet transform. Then, an ensemble of CNNs is trained to jointly determine the crack length. Real fatigue tests on complex lap joint structures were carried out to validate the proposed method, in which several structures were tested preliminarily for collecting the training dataset and a new structure was adopted for testing. The root mean square error of the training dataset is 1.4 mm. Besides, the root mean square error of the evaluated crack length in the testing lap joint structure was 1.7 mm, showing the effectiveness of the proposed method. Full article
(This article belongs to the Special Issue Feature Papers in Fault Diagnosis & Sensors Section 2022)
Show Figures

Figure 1

27 pages, 5295 KiB  
Article
Damage Detection and Localization under Variable Environmental Conditions Using Compressed and Reconstructed Bayesian Virtual Sensor Data
by Jyrki Kullaa
Department of Automotive and Mechanical Engineering, Metropolia University of Applied Sciences, Leiritie 1, 01600 Vantaa, Finland
Sensors 2022, 22(1), 306; https://doi.org/10.3390/s22010306 - 31 Dec 2021
Cited by 15 | Viewed by 2590
Abstract
Structural health monitoring (SHM) with a dense sensor network and repeated vibration measurements produces lots of data that have to be stored. If the sensor network is redundant, data compression is possible by storing the signals of selected Bayesian virtual sensors only, from [...] Read more.
Structural health monitoring (SHM) with a dense sensor network and repeated vibration measurements produces lots of data that have to be stored. If the sensor network is redundant, data compression is possible by storing the signals of selected Bayesian virtual sensors only, from which the omitted signals can be reconstructed with higher accuracy than the actual measurement. The selection of the virtual sensors for storage is done individually for each measurement based on the reconstruction accuracy. Data compression and reconstruction for SHM is the main novelty of this paper. The stored and reconstructed signals are used for damage detection and localization in the time domain using spatial or spatiotemporal correlation. Whitening transformation is applied to the training data to take the environmental or operational influences into account. The first principal component of the residuals is used to localize damage and also to design the extreme value statistics control chart for damage detection. The proposed method was studied with a numerical model of a frame structure with a dense accelerometer or strain sensor network. Only five acceleration or three strain signals out of the total 59 signals were stored. The stored and reconstructed data outperformed the raw measurement data in damage detection and localization. Full article
(This article belongs to the Special Issue Model-Free Structural Health Monitoring Approaches)
Show Figures

Figure 1

33 pages, 14100 KiB  
Article
Sensor Data Fusion for a Mobile Robot Using Neural Networks
by Andres J. Barreto-Cubero 1, Alfonso Gómez-Espinosa 1,*, Jesús Arturo Escobedo Cabello 1, Enrique Cuan-Urquizo 1 and Sergio R. Cruz-Ramírez 2
1 Tecnologico de Monterrey, Escuela de Ingenieria y Ciencias, Av. Epigmenio González 500, Fracc. San Pablo, Querétaro 76130, Mexico
2 Tecnologico de Monterrey, Escuela de Ingenieria y Ciencias, Av. Eugenio Garza 10 Sada 300, Lomas del Tecnológico, San Luis Potosí 78211, Mexico
Sensors 2022, 22(1), 305; https://doi.org/10.3390/s22010305 - 31 Dec 2021
Cited by 29 | Viewed by 7501
Abstract
Mobile robots must be capable to obtain an accurate map of their surroundings to move within it. To detect different materials that might be undetectable to one sensor but not others it is necessary to construct at least a two-sensor fusion scheme. With [...] Read more.
Mobile robots must be capable to obtain an accurate map of their surroundings to move within it. To detect different materials that might be undetectable to one sensor but not others it is necessary to construct at least a two-sensor fusion scheme. With this, it is possible to generate a 2D occupancy map in which glass obstacles are identified. An artificial neural network is used to fuse data from a tri-sensor (RealSense Stereo camera, 2D 360° LiDAR, and Ultrasonic Sensors) setup capable of detecting glass and other materials typically found in indoor environments that may or may not be visible to traditional 2D LiDAR sensors, hence the expression improved LiDAR. A preprocessing scheme is implemented to filter all the outliers, project a 3D pointcloud to a 2D plane and adjust distance data. With a Neural Network as a data fusion algorithm, we integrate all the information into a single, more accurate distance-to-obstacle reading to finally generate a 2D Occupancy Grid Map (OGM) that considers all sensors information. The Robotis Turtlebot3 Waffle Pi robot is used as the experimental platform to conduct experiments given the different fusion strategies. Test results show that with such a fusion algorithm, it is possible to detect glass and other obstacles with an estimated root-mean-square error (RMSE) of 3 cm with multiple fusion strategies. Full article
(This article belongs to the Section Sensors and Robotics)
Show Figures

Graphical abstract

18 pages, 3589 KiB  
Article
A Novel Infrared and Visible Image Fusion Approach Based on Adversarial Neural Network
by Xianglong Chen 1, Haipeng Wang 1, Yaohui Liang 1, Ying Meng 1,2,* and Shifeng Wang 1,3
1 School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China
2 Key Laboratory of Optoelectronic Measurement, Optical Information Transmission Technology of Ministry of Education, School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China
3 Zhongshan Institute, Changchun University of Science and Technology, Changchun 130022, China
Sensors 2022, 22(1), 304; https://doi.org/10.3390/s22010304 - 31 Dec 2021
Cited by 5 | Viewed by 2201
Abstract
The presence of fake pictures affects the reliability of visible face images under specific circumstances. This paper presents a novel adversarial neural network designed named as the FTSGAN for infrared and visible image fusion and we utilize FTSGAN model to fuse the face [...] Read more.
The presence of fake pictures affects the reliability of visible face images under specific circumstances. This paper presents a novel adversarial neural network designed named as the FTSGAN for infrared and visible image fusion and we utilize FTSGAN model to fuse the face image features of infrared and visible image to improve the effect of face recognition. In FTSGAN model design, the Frobenius norm (F), total variation norm (TV), and structural similarity index measure (SSIM) are employed. The F and TV are used to limit the gray level and the gradient of the image, while the SSIM is used to limit the image structure. The FTSGAN fuses infrared and visible face images that contains bio-information for heterogeneous face recognition tasks. Experiments based on the FTSGAN using hundreds of face images demonstrate its excellent performance. The principal component analysis (PCA) and linear discrimination analysis (LDA) are involved in face recognition. The face recognition performance after fusion improved by 1.9% compared to that before fusion, and the final face recognition rate was 94.4%. This proposed method has better quality, faster rate, and is more robust than the methods that only use visible images for face recognition. Full article
(This article belongs to the Section Intelligent Sensors)
Show Figures

Figure 1