Next Article in Journal
Real-Time Automatic Calculation of Euro Coins and Banknotes in a Cash Drawer
Previous Article in Journal
Non-Contact Temperature Control System Applicable to Polymerase Chain Reaction on a Lab-on-a-Disc
Open AccessArticle

Grid Based Spherical CNN for Object Detection from Panoramic Images

School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China
*
Author to whom correspondence should be addressed.
Sensors 2019, 19(11), 2622; https://doi.org/10.3390/s19112622
Received: 29 April 2019 / Revised: 24 May 2019 / Accepted: 6 June 2019 / Published: 9 June 2019
(This article belongs to the Section Optical Sensors)
Recently proposed spherical convolutional neural networks (SCNNs) have shown advantages over conventional planar CNNs on classifying spherical images. However, two factors hamper their application in an objection detection task. First, a convolution in S2 (a two-dimensional sphere in three-dimensional space) or SO(3) (three-dimensional special orthogonal group) space results in the loss of an object’s location. Second, overlarge bandwidth is required to preserve a small object’s information on a sphere because the S2/SO(3) convolution must be performed on the whole sphere, instead of a local image patch. In this study, we propose a novel grid-based spherical CNN (G-SCNN) for detecting objects from spherical images. According to input bandwidth, a sphere image is transformed to a conformal grid map to be the input of the S2/SO3 convolution, and an object’s bounding box is scaled to cover an adequate area of the grid map. This solves the second problem. For the first problem, we utilize a planar region proposal network (RPN) with a data augmentation strategy that increases rotation invariance. We have also created a dataset including 600 street view panoramic images captured from a vehicle-borne panoramic camera. The dataset contains 5636 objects of interest annotated with class and bounding box and is named as WHU (Wuhan University) panoramic dataset. Results on the dataset proved our grid-based method is extremely better than the original SCNN in detecting objects from spherical images, and it outperformed several mainstream object detection networks, such as Faster R-CNN and SSD. View Full-Text
Keywords: spherical convolution; panoramic image; object detection; rotation invariance spherical convolution; panoramic image; object detection; rotation invariance
Show Figures

Figure 1

MDPI and ACS Style

Yu, D.; Ji, S. Grid Based Spherical CNN for Object Detection from Panoramic Images. Sensors 2019, 19, 2622.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop