sensors-logo

Journal Browser

Journal Browser

Computer Vision Based Smart Sensing

A topical collection in Sensors (ISSN 1424-8220). This collection belongs to the section "Sensing and Imaging".

Viewed by 27993

Editor

Universidad Politécnica de Madrid, 28040 Madrid, Spain
Interests: visual communications; digital television and computer vision (applied to C-ITS transport, Cultural Heritage preservation, health, etc)

Topical Collection Information

Dear Colleagues,

Sensing of visual information with video cameras has become an almost ubiquitous element of our environment. Internet connectivity and the massive use of the Internet of Things (IoT) has contributed to this in a very important way. The number of video cameras deployed with the capacity to capture information both in the visible range and outside the visible range (IR, UV, or other bands) has grown very significantly in recent years, with figures ranging from 50 to 150 surveillance cameras per 1000 inhabitants in the most important cities in Asia, Europe, and America. Moreover, this is only with regard to fixed surveillance cameras (on streets, roads, and buildings). If, in addition, mobile and wearable devices incorporating cameras are taken into account, the possibilities for capturing visual information are almost infinite.

To be able to process such a large volume of information, it is necessary to develop computer vision applications for the different areas in which they are usually applied: security, health, multimodal transport, accessibility for people with disabilities of any kind, cultural heritage preservation, etc. The capacity for smart sensing with cameras has grown significantly with the application of deep learning strategies to images and video sequences. These applications usually have a high computing cost, which is encouraging the emergence of smart processing strategies: on the edge near the point of capture, distributed in the cloud, etc. In order to make these applications robust, it is usually necessary to provide them with certain self-calibration capabilities, resilience, or even the ability to fuse visual information with other devices, such as radars or LiDAR.

This Special Issue aims to address the open research challenges and unsolved problems related to computer-vision-based smart sensing applications in different domains, making use of IoT-connected, camera-acquired information (inside the visible range or outside it) in an isolated way or by fusing them with other image-based devices, such as LiDAR or radar, in a monocular configuration or a multicamera one.

Dr. Jose Manuel Menéndez García
Collection Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the collection website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • smart computer vision/IR/UV-based sensing
  • edge processing/distributed processing in the IoT
  • camera fusion with other devices
  • single-camera/multicamera/multidevice resilience
  • application of smart vision in different domains

Published Papers (12 papers)

2023

Jump to: 2022, 2021

19 pages, 5431 KiB  
Article
End-to-End Bubble Size Distribution Detection Technique in Dense Bubbly Flows Based on You Only Look Once Architecture
by Mengchi Chen, Cheng Zhang, Wen Yang, Suyi Zhang and Wenjun Huang
Sensors 2023, 23(14), 6582; https://doi.org/10.3390/s23146582 - 21 Jul 2023
Viewed by 805
Abstract
Accurate measurements of the bubble size distribution (BSD) are crucial for investigating gas–liquid mass transfer mechanisms and describing the characteristics of chemical production. However, measuring the BSD in high-density bubbly flows remains challenging due to limited image algorithms and high data densities. Therefore, [...] Read more.
Accurate measurements of the bubble size distribution (BSD) are crucial for investigating gas–liquid mass transfer mechanisms and describing the characteristics of chemical production. However, measuring the BSD in high-density bubbly flows remains challenging due to limited image algorithms and high data densities. Therefore, an end-to-end BSD detection method in dense bubbly flows based on deep learning is proposed in this paper. The bubble detector locates the positions of dense bubbles utilizing objection detection networks and simultaneously performs ellipse parameter fitting to measure the size of the bubbles. Different You Only Look Once (YOLO) architectures are compared, and YOLOv7 is selected as the backbone network. The complete intersection over union calculation method is modified by the circumferential horizontal rectangle of bubbles, and the loss function is optimized by adding L2 constraints of ellipse size parameters. The experimental results show that the proposed technique surpasses existing methods in terms of precision, recall, and mean square error, achieving values of 0.9871, 0.8725, and 3.8299, respectively. The proposed technique demonstrates high efficiency and accuracy when measuring BSDs in high-density bubbly flows and has the potential for practical applications. Full article
Show Figures

Figure 1

2022

Jump to: 2023, 2021

16 pages, 2699 KiB  
Article
Pedestrian Trajectory Prediction for Real-Time Autonomous Systems via Context-Augmented Transformer Networks
by Khaled Saleh
Sensors 2022, 22(19), 7495; https://doi.org/10.3390/s22197495 - 02 Oct 2022
Cited by 4 | Viewed by 2025
Abstract
Forecasting the trajectory of pedestrians in shared urban traffic environments from non-invasive sensor modalities is still considered one of the challenging problems facing the development of autonomous vehicles (AVs). In the literature, this problem is often tackled using recurrent neural networks (RNNs). Despite [...] Read more.
Forecasting the trajectory of pedestrians in shared urban traffic environments from non-invasive sensor modalities is still considered one of the challenging problems facing the development of autonomous vehicles (AVs). In the literature, this problem is often tackled using recurrent neural networks (RNNs). Despite the powerful capabilities of RNNs in capturing the temporal dependency in the pedestrians’ motion trajectories, they were argued to be challenged when dealing with longer sequential data. Additionally, whilst the accommodation for contextual information (such as scene semantics and agents interactions) was shown to be effective for robust trajectory prediction, they can also impact the overall real-time performance of prediction system. Thus, in this work, we are introducing a framework based on the transformer networks that were demonstrated recently to be more efficient and outperformed RNNs in many sequential-based tasks. We relied on a fusion of sensor modalities, namely the past positional information, agent interactions information and scene physical semantics information as an input to our framework in order to not only provide a robust trajectory prediction of pedestrians, but also achieve real-time performance for multi-pedestrians’ trajectory prediction. We have evaluated our framework on three real-life datasets of pedestrians in shared urban traffic environments and it has outperformed the compared baseline approaches in both short-term and long-term prediction horizons. For the short-term prediction horizon, our approach has achieved lower scores according to the average displacement error and the root-mean squared error (ADE/RMSE) of predictions over the state-of-the art (SOTA) approach by more than 11 cm and 23 cm, respectively. While for the long-term prediction horizon, our approach has achieved lower ADE and FDE over the SOTA approach by more than 62 cm and 165 cm, respectively. Additionally, our approach has achieved superior real time performance by scoring only 0.025 s (i.e., it can provide 40 individual trajectory predictions per second). Full article
Show Figures

Figure 1

17 pages, 6390 KiB  
Article
Night Vision Anti-Halation Method Based on Infrared and Visible Video Fusion
by Quanmin Guo, Hanlei Wang and Jianhua Yang
Sensors 2022, 22(19), 7494; https://doi.org/10.3390/s22197494 - 02 Oct 2022
Viewed by 1291
Abstract
In order to address the discontinuity caused by the direct application of the infrared and visible image fusion anti-halation method to a video, an efficient night vision anti-halation method based on video fusion is proposed. The designed frame selection based on inter-frame difference [...] Read more.
In order to address the discontinuity caused by the direct application of the infrared and visible image fusion anti-halation method to a video, an efficient night vision anti-halation method based on video fusion is proposed. The designed frame selection based on inter-frame difference determines the optimal cosine angle threshold by analyzing the relation of cosine angle threshold with nonlinear correlation information entropy and de-frame rate. The proposed time-mark-based adaptive motion compensation constructs the same number of interpolation frames as the redundant frames by taking the retained frame number as a time stamp. At the same time, considering the motion vector of two adjacent retained frames as the benchmark, the adaptive weights are constructed according to the interframe differences between the interpolated frame and the last retained frame, then the motion vector of the interpolated frame is estimated. The experimental results show that the proposed frame selection strategy ensures the maximum safe frame removal under the premise of continuous video content at different vehicle speeds in various halation scenes. The frame numbers and playing duration of the fused video are consistent with that of the original video, and the content of the interpolated frame is highly synchronized with that of the corresponding original frames. The average FPS of video fusion in this work is about six times that in the frame-by-frame fusion, which effectively improves the anti-halation processing efficiency of video fusion. Full article
Show Figures

Figure 1

16 pages, 5848 KiB  
Article
Open Source Assessment of Deep Learning Visual Object Detection
by Sergio Paniego, Vinay Sharma and José María Cañas
Sensors 2022, 22(12), 4575; https://doi.org/10.3390/s22124575 - 17 Jun 2022
Cited by 2 | Viewed by 1547
Abstract
This paper introduces Detection Metrics, an open-source scientific software for the assessment of deep learning neural network models for visual object detection. This software provides objective performance metrics such as mean average precision and mean inference time. The most relevant international object detection [...] Read more.
This paper introduces Detection Metrics, an open-source scientific software for the assessment of deep learning neural network models for visual object detection. This software provides objective performance metrics such as mean average precision and mean inference time. The most relevant international object detection datasets are supported along with the most widely used deep learning frameworks. Different network models, even those built from different frameworks, can be fairly compared in this way. This is very useful when developing deep learning applications or research. A set of tools is provided to manage and work with different datasets and models, including visualization and conversion into several common formats. Detection Metrics may also be used in automatic batch processing for large experimental tests, saving researchers time, and new domain-specific datasets can be easily created from videos or webcams. It is open-source, can be audited, extended, and adapted to particular requirements. It has been experimentally validated. The performance of the most relevant state-of-the-art neural models for object detection has been experimentally compared. In addition, it has been used in several research projects, guiding in selecting the most suitable network model architectures and training procedures. The performance of the different models and training alternatives can be easily measured, even on large datasets. Full article
Show Figures

Figure 1

14 pages, 6728 KiB  
Communication
A Study on the Correlation between Change in the Geometrical Dimension of a Free-Falling Molten Glass Gob and Its Viscosity
by Mazhar Hussain, Mattias O’Nils, Jan Lundgren and Irida Shallari
Sensors 2022, 22(2), 661; https://doi.org/10.3390/s22020661 - 15 Jan 2022
Viewed by 1676
Abstract
To produce flawless glass containers, continuous monitoring of the glass gob is required. It is essential to ensure production of molten glass gobs with the right shape, temperature, viscosity and weight. At present, manual monitoring is common practice in the glass container industry, [...] Read more.
To produce flawless glass containers, continuous monitoring of the glass gob is required. It is essential to ensure production of molten glass gobs with the right shape, temperature, viscosity and weight. At present, manual monitoring is common practice in the glass container industry, which heavily depends on previous experience, operator knowledge and trial and error. This results in inconsistent measurements and consequently loss of production. In this article, a multi-camera based setup is used as a non-invasive real-time monitoring system. We have shown that under certain conditions, such as keeping the glass composition constant, it is possible to do in-line measurement of viscosity using sensor fusion to correlate the rate of geometrical change in the gob and its temperature. The correlation models presented in this article show that there is a strong correlation, i.e., 0.65, between our measurements and the projected viscosity. Full article
Show Figures

Figure 1

2021

Jump to: 2023, 2022

15 pages, 1916 KiB  
Article
Computer Vision in Analyzing the Propagation of a Gas–Gunpowder Jet
by Irina G. Palchikova, Igor V. Latyshov, Evgenii S. Smirnov, Vasilii A. Vasiliev, Alexander V. Kondakov and Irina A. Budaeva
Sensors 2022, 22(1), 6; https://doi.org/10.3390/s22010006 - 21 Dec 2021
Cited by 1 | Viewed by 2023
Abstract
A method of mathematically processing the digital images of targets is developed. The theoretical and mathematical justification and the experimental validation of the possibility of estimating the amount of gunshot residue (GSR) and determining the GSR distribution over the target on the basis [...] Read more.
A method of mathematically processing the digital images of targets is developed. The theoretical and mathematical justification and the experimental validation of the possibility of estimating the amount of gunshot residue (GSR) and determining the GSR distribution over the target on the basis of its digital image is provided. The analysis of the optical density in selected concentric rings in the images reveals the radial dependence of soot distribution in the cross section of a gas–gunpowder jet. The analysis of the optical density in selected sectors of the circle reveals the angular dependence of the soot distribution in the gas–gunpowder jet cross section. It is shown that the integral optical density averaged over a selected area in the target image characterizes the mass of GSP deposited on it. It is possible to quantify the differences in the radial and angular distributions of the thickness of the GSR layer on various targets obtained both with the help of weapons of different types at the same distances and with the help of weapons of the same type at different distances, by calculating the distribution of optical density on their digital images. Full article
Show Figures

Figure 1

19 pages, 1808 KiB  
Article
Relation-Based Deep Attention Network with Hybrid Memory for One-Shot Person Re-Identification
by Runxuan Si, Jing Zhao, Yuhua Tang and Shaowu Yang
Sensors 2021, 21(15), 5113; https://doi.org/10.3390/s21155113 - 28 Jul 2021
Cited by 4 | Viewed by 1554
Abstract
One-shot person Re-identification, which owns one labeled sample among numerous unlabeled data for each identity, is proposed to tackle the problem of the shortage of labeled data. Considering the scenarios without sufficient labeled data, it is very challenging to keep abreast of the [...] Read more.
One-shot person Re-identification, which owns one labeled sample among numerous unlabeled data for each identity, is proposed to tackle the problem of the shortage of labeled data. Considering the scenarios without sufficient labeled data, it is very challenging to keep abreast of the performance of the supervised task in which sufficient labeled samples are available. In this paper, we propose a relation-based attention network with hybrid memory, which can make full use of the global information to pay attention to the identity features for model training with the relation-based attention network. Importantly, our specially designed network architecture effectively reduces the interference of environmental noise. Moreover, we propose a hybrid memory to train the one-shot data and unlabeled data in a unified framework, which notably contributes to the performance of person Re-identification. In particular, our designed one-shot feature update mode effectively alleviates the problem of overfitting, which is caused by the lack of supervised information during the training process. Compared with state-of-the-art unsupervised and one-shot algorithms for person Re-identification, our method achieves considerable improvements of 6.7%, 4.6%, and 11.5% on Market-1501, DukeMTMC-reID, and MSMT17 datasets, respectively, and becomes the new state-of-the-art method for one-shot person Re-identification. Full article
Show Figures

Figure 1

31 pages, 22124 KiB  
Article
Multi-Camera Vessel-Speed Enforcement by Enhancing Detection and Re-Identification Techniques
by Matthijs H. Zwemer, Herman G. J. Groot, Rob Wijnhoven, Egor Bondarev and Peter H. N. de With
Sensors 2021, 21(14), 4659; https://doi.org/10.3390/s21144659 - 07 Jul 2021
Cited by 3 | Viewed by 2802
Abstract
This paper presents a camera-based vessel-speed enforcement system based on two cameras. The proposed system detects and tracks vessels per camera view and employs a re-identification (re-ID) function for linking vessels between the two cameras based on multiple bounding-box images per vessel. Newly [...] Read more.
This paper presents a camera-based vessel-speed enforcement system based on two cameras. The proposed system detects and tracks vessels per camera view and employs a re-identification (re-ID) function for linking vessels between the two cameras based on multiple bounding-box images per vessel. Newly detected vessels in one camera (query) are compared to the gallery set of all vessels detected by the other camera. To train and evaluate the proposed detection and re-ID system, a new Vessel-reID dataset is introduced. This extensive dataset has captured a total of 2474 different vessels covered in multiple images, resulting in a total of 136,888 vessel bounding-box images. Multiple CNN detector architectures are evaluated in-depth. The SSD512 detector performs best with respect to its speed (85.0% Recall@95Precision at 20.1 frames per second). For the re-ID of vessels, a large portion of the total trajectory can be covered by the successful detections of the SSD model. The re-ID experiments start with a baseline single-image evaluation obtaining a score of 55.9% Rank-1 (49.7% mAP) for the existing TriNet network, while the available MGN model obtains 68.9% Rank-1 (62.6% mAP). The performance significantly increases with 5.6% Rank-1 (5.7% mAP) for MGN by applying matching with multiple images from a single vessel. When emphasizing more fine details by selecting only the largest bounding-box images, another 2.0% Rank-1 (1.4% mAP) is added. Application-specific optimizations such as travel-time selection and applying a cross-camera matching constraint further enhance the results, leading to a final 88.9% Rank-1 and 83.5% mAP performance. Full article
Show Figures

Figure 1

13 pages, 3345 KiB  
Article
Pose Estimation of Omnidirectional Camera with Improved EPnP Algorithm
by Xuanrui Gong, Yaowen Lv, Xiping Xu, Yuxuan Wang and Mengdi Li
Sensors 2021, 21(12), 4008; https://doi.org/10.3390/s21124008 - 10 Jun 2021
Cited by 6 | Viewed by 2781
Abstract
The omnidirectional camera, having the advantage of broadening the field of view, realizes 360° imaging in the horizontal direction. Due to light reflection from the mirror surface, the collinearity relation is altered and the imaged scene has severe nonlinear distortions. This makes it [...] Read more.
The omnidirectional camera, having the advantage of broadening the field of view, realizes 360° imaging in the horizontal direction. Due to light reflection from the mirror surface, the collinearity relation is altered and the imaged scene has severe nonlinear distortions. This makes it more difficult to estimate the pose of the omnidirectional camera. To solve this problem, we derive the mapping from omnidirectional camera to traditional camera and propose an omnidirectional camera linear imaging model. Based on the linear imaging model, we improve the EPnP algorithm to calculate the omnidirectional camera pose. To validate the proposed solution, we conducted simulations and physical experiments. Results show that the algorithm has a good performance in resisting noise. Full article
Show Figures

Figure 1

27 pages, 6298 KiB  
Article
Haptic Glove TV Device for People with Visual Impairment
by Diego Villamarín and José Manuel Menéndez
Sensors 2021, 21(7), 2325; https://doi.org/10.3390/s21072325 - 26 Mar 2021
Cited by 4 | Viewed by 3862
Abstract
Immersive video is changing the way we enjoy TV. It is no longer just about receiving sequential images with audio, but also playing with other human senses through smells, vibrations of movement, 3D audio, feeling water, wind, heat, and other emotions that can [...] Read more.
Immersive video is changing the way we enjoy TV. It is no longer just about receiving sequential images with audio, but also playing with other human senses through smells, vibrations of movement, 3D audio, feeling water, wind, heat, and other emotions that can be experienced through all human senses. This work aims to validate the usefulness of an immersive and interactive solution for people with severe visual impairment by developing a haptic glove that allows receiving signals and generating vibrations in hand, informing about what happens in a scene. The study case presented here shows how the haptic device can take the information about the ball’s location in the playing field, synchronized with the video reception, and deliver it to the user in the form of vibrations during the re-transmission of a soccer match. In this way, we take visually impaired people to live a new sensory experience, allowing digital and social inclusion and accessibility to audiovisual technologies that they could not enjoy before. This work shows the methodology used for the design, implementation, and results evaluation. Usability tests were carried out with fifteen visually impaired people who used the haptic device to attend a soccer match synchronized with the glove’s vibrations. Full article
Show Figures

Figure 1

15 pages, 3811 KiB  
Article
A Robust Road Vanishing Point Detection Adapted to the Real-world Driving Scenes
by Cuong Nguyen Khac, Yeongyu Choi, Ju H. Park and Ho-Youl Jung
Sensors 2021, 21(6), 2133; https://doi.org/10.3390/s21062133 - 18 Mar 2021
Cited by 7 | Viewed by 2981
Abstract
Vanishing point (VP) provides extremely useful information related to roads in driving scenes for advanced driver assistance systems (ADAS) and autonomous vehicles. Existing VP detection methods for driving scenes still have not achieved sufficiently high accuracy and robustness to apply for real-world driving [...] Read more.
Vanishing point (VP) provides extremely useful information related to roads in driving scenes for advanced driver assistance systems (ADAS) and autonomous vehicles. Existing VP detection methods for driving scenes still have not achieved sufficiently high accuracy and robustness to apply for real-world driving scenes. This paper proposes a robust motion-based road VP detection method to compensate for the deficiencies. For such purposes, three main processing steps often used in the existing road VP detection methods are carefully examined. Based on the analysis, stable motion detection, stationary point-based motion vector selection, and angle-based RANSAC (RANdom SAmple Consensus) voting are proposed. A ground-truth driving dataset including various objects and illuminations is used to verify the robustness and real-time capability of the proposed method. The experimental results show that the proposed method outperforms the existing motion-based and edge-based road VP detection methods for various illumination conditioned driving scenes. Full article
Show Figures

Figure 1

19 pages, 2361 KiB  
Article
Specular Detection on Glossy Surface Using Geometric Characteristics of Specularity in Top-View Images
by Seunghyun Kim, Moonsoo Ra and Whoi-Yul Kim
Sensors 2021, 21(6), 2079; https://doi.org/10.3390/s21062079 - 16 Mar 2021
Cited by 4 | Viewed by 3004
Abstract
In an autonomous driving assistance system (ADAS), top views effectively represent objects around the vehicle on a 2D plane. Top-view images are therefore widely used to detect lines in ADAS applications such as lane-keeping assistance and parking assistance. Because line detection is a [...] Read more.
In an autonomous driving assistance system (ADAS), top views effectively represent objects around the vehicle on a 2D plane. Top-view images are therefore widely used to detect lines in ADAS applications such as lane-keeping assistance and parking assistance. Because line detection is a crucial step for these applications, the false positive detection of lines can lead to failure of the system. Specular reflections from a glossy surface are often the cause of false positives, and since certain specular patterns resemble actual lines in the top-view image, their presence induces false positive lines. Incorrect positions of the lines or parking stalls can thus be obtained. To alleviate this problem, we propose two methods to estimate specular pixels in the top-view image. The methods use a geometric property of the specular region: the shape of the specular region is stretched long in the direction of the camera as the distance between the camera and the light source becomes distant, resulting in a straight line. This property can be used to distinguish the specular region in images. One estimates the pixel-wise probability of the specularity using gradient vectors obtained from an edge detector and the other estimates specularity using the line equation of each line segment obtained by line detection. To evaluate the performance of the proposed method, we added our methods as a pre-processing step to existing parking stall detection methods and investigated changes in their performance. The proposed methods improved line detection performance by accurately estimating specular components in the top-view images. Full article
Show Figures

Figure 1

Back to TopTop