Next Article in Journal
Online Streaming Features Selection via Markov Blanket
Next Article in Special Issue
Spatial-Temporal Epidemiology of COVID-19 Using a Geographically and Temporally Weighted Regression Model
Previous Article in Journal
Exploiting the Pascal Distribution Series and Gegenbauer Polynomials to Construct and Study a New Subclass of Analytic Bi-Univalent Functions
Previous Article in Special Issue
Data-Driven Analytics Leveraging Artificial Intelligence in the Era of COVID-19: An Insightful Review of Recent Developments
 
 
Article

Intelligent System for Estimation of the Spatial Position of Apples Based on YOLOv3 and Real Sense Depth Camera D415

1
Department of Data Analysis and Machine Learning, Financial University under the Government of the Russian Federation, 125167 Moscow, Russia
2
Laboratory of Robotics, Internet of Things and Embedded Systems, Financial University under the Government of the Russian Federation, 125167 Moscow, Russia
3
Department of Mathematical Methods in Economics and Management, State University of Management, 109542 Moscow, Russia
4
Institute of Environmental Sciences, Warsaw University of Life Sciences, 02-787 Warsaw, Poland
*
Author to whom correspondence should be addressed.
Academic Editor: Pecchinenda Anna
Symmetry 2022, 14(1), 148; https://doi.org/10.3390/sym14010148
Received: 1 December 2021 / Revised: 21 December 2021 / Accepted: 5 January 2022 / Published: 13 January 2022
(This article belongs to the Special Issue Mathematical Modelling in Science and Engineering)
Despite the great possibilities of modern neural network architectures concerning the problems of object detection and recognition, the output of such models is the local (pixel) coordinates of objects bounding boxes in the image and their predicted classes. However, in several practical tasks, it is necessary to obtain more complete information about the object from the image. In particular, for robotic apple picking, it is necessary to clearly understand where and how much to move the grabber. To determine the real position of the apple relative to the source of image registration, it is proposed to use the Intel Real Sense depth camera and aggregate information from its depth and brightness channels. The apples detection is carried out using the YOLOv3 architecture; then, based on the distance to the object and its localization in the image, the relative distances are calculated for all coordinates. In this case, to determine the coordinates of apples, a transition to a symmetric coordinate system takes place by means of simple linear transformations. Estimating the position in a symmetric coordinate system allows estimating not only the magnitude of the shift but also the location of the object relative to the camera. The proposed approach makes it possible to obtain position estimates with high accuracy. The approximate root mean square error is 7–12 mm, depending on the range and axis. As for precision and recall metrics, the first is 100% and the second is 90%. View Full-Text
Keywords: pattern recognition; stereovision; object detection; YOLOv3; Intel Real Sense; coordinate estimation; data aggregation; agriculture; horticulture; apple picking pattern recognition; stereovision; object detection; YOLOv3; Intel Real Sense; coordinate estimation; data aggregation; agriculture; horticulture; apple picking
Show Figures

Figure 1

MDPI and ACS Style

Andriyanov, N.; Khasanshin, I.; Utkin, D.; Gataullin, T.; Ignar, S.; Shumaev, V.; Soloviev, V. Intelligent System for Estimation of the Spatial Position of Apples Based on YOLOv3 and Real Sense Depth Camera D415. Symmetry 2022, 14, 148. https://doi.org/10.3390/sym14010148

AMA Style

Andriyanov N, Khasanshin I, Utkin D, Gataullin T, Ignar S, Shumaev V, Soloviev V. Intelligent System for Estimation of the Spatial Position of Apples Based on YOLOv3 and Real Sense Depth Camera D415. Symmetry. 2022; 14(1):148. https://doi.org/10.3390/sym14010148

Chicago/Turabian Style

Andriyanov, Nikita, Ilshat Khasanshin, Daniil Utkin, Timur Gataullin, Stefan Ignar, Vyacheslav Shumaev, and Vladimir Soloviev. 2022. "Intelligent System for Estimation of the Spatial Position of Apples Based on YOLOv3 and Real Sense Depth Camera D415" Symmetry 14, no. 1: 148. https://doi.org/10.3390/sym14010148

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop