Next Article in Journal
Physics-Based Device Models and Progress Review for Active Piezoelectric Semiconductor Devices
Next Article in Special Issue
High-Efficiency Microsatellite-Using Super-Resolution Algorithm Based on the Multi-Modality Super-CMOS Sensor
Previous Article in Journal
A Novel Statistical Method for Scene Classification Based on Multi-Object Categorization and Logistic Regression
Previous Article in Special Issue
Image Deblurring Using Multi-Stream Bottom-Top-Bottom Attention Network and Global Information-Based Fusion and Reconstruction Network
Open AccessArticle

Implicit and Explicit Regularization for Optical Flow Estimation

1
The Visual Computing Lab, Information Technologies Institute, Centre for Research and Technology Hellas, 57001 Thessaloniki, Greece
2
Universidad Politécnica de Madrid, 28040 Madrid, Spain
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Sensors 2020, 20(14), 3855; https://doi.org/10.3390/s20143855
Received: 2 June 2020 / Revised: 2 July 2020 / Accepted: 7 July 2020 / Published: 10 July 2020
(This article belongs to the Special Issue Data, Signal and Image Processing and Applications in Sensors)
In this paper, two novel and practical regularizing methods are proposed to improve existing neural network architectures for monocular optical flow estimation. The proposed methods aim to alleviate deficiencies of current methods, such as flow leakage across objects and motion consistency within rigid objects, by exploiting contextual information. More specifically, the first regularization method utilizes semantic information during the training process to explicitly regularize the produced optical flow field. The novelty of this method lies in the use of semantic segmentation masks to teach the network to implicitly identify the semantic edges of an object and better reason on the local motion flow. A novel loss function is introduced that takes into account the objects’ boundaries as derived from the semantic segmentation mask to selectively penalize motion inconsistency within an object. The method is architecture agnostic and can be integrated into any neural network without modifying or adding complexity at inference. The second regularization method adds spatial awareness to the input data of the network in order to improve training stability and efficiency. The coordinates of each pixel are used as an additional feature, breaking the invariance properties of the neural network architecture. The additional features are shown to implicitly regularize the optical flow estimation enforcing a consistent flow, while improving both the performance and the convergence time. Finally, the combination of both regularization methods further improves the performance of existing cutting edge architectures in a complementary way, both quantitatively and qualitatively, on popular flow estimation benchmark datasets. View Full-Text
Keywords: optical flow; regularization; semantic segmentation; motion consistency; coordconv optical flow; regularization; semantic segmentation; motion consistency; coordconv
Show Figures

Figure 1

MDPI and ACS Style

Karageorgos, K.; Dimou, A.; Alvarez, F.; Daras, P. Implicit and Explicit Regularization for Optical Flow Estimation. Sensors 2020, 20, 3855.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop