Next Article in Journal
Laser Safety Calculations for Imaging Sensors
Next Article in Special Issue
Multi-View Fusion-Based 3D Object Detection for Robot Indoor Scene Perception
Previous Article in Journal
A Sensor for Broken Wire Detection of Steel Wire Ropes Based on the Magnetic Concentrating Principle
Previous Article in Special Issue
Dual-Resolution Dual-Path Convolutional Neural Networks for Fast Object Detection
Article

Intelligent 3D Perception System for Semantic Description and Dynamic Interaction

1
Graduate School of Electrical Engineering and Computer Science (CPGEI), Federal University of Technology-Paraná (UTFPR), Avenida 7 de Setembro 3165, Curitiba 80230-901, Brazil
2
CENPES, Rio de Janeiro 21941-915, Brazil
*
Author to whom correspondence should be addressed.
Sensors 2019, 19(17), 3764; https://doi.org/10.3390/s19173764
Received: 30 June 2019 / Revised: 7 August 2019 / Accepted: 13 August 2019 / Published: 30 August 2019
(This article belongs to the Special Issue Semantic Representations for Behavior Analysis in Robotic System)
This work proposes a novel semantic perception system based on computer vision and machine learning techniques. The main goal is to identify objects in the environment and extract their characteristics, allowing a dynamic interaction with the environment. The system is composed of a GPU processing source and a 3D vision sensor that provides RGB image and PointCloud data. The perception system is structured in three steps: Lexical Analysis, Syntax Analysis and finally an Analysis of Anticipation. The Lexical Analysis detects the actual position of the objects (or tokens) in the environment, through the combination of RGB image and PointCloud, surveying their characteristics. All information extracted from the tokens will be used to retrieve relevant features such as object velocity, acceleration and direction during the Syntax Analysis step. The anticipation step predicts future behaviors for these dynamic objects, promoting an interaction with them in terms of collisions, pull, and push actions. As a result, the proposed perception source can assign relevant information to mobile robots, not only about distances as traditional sensors, but about other environment characteristics and object behaviors. This novel perception source introduces a new class of skills to mobile robots. Experimental results obtained with a real robot are presented, showing the proposed perception source efficacy and potential. View Full-Text
Keywords: perception; PointCloud; prediction and object recognition perception; PointCloud; prediction and object recognition
Show Figures

Figure 1

MDPI and ACS Style

Teixeira, M.A.S.; Nogueira, R.d.C.M.; Dalmedico, N.; Santos, H.B.; Arruda, L.V.R.d.; Neves-Jr, F.; Pipa, D.R.; Ramos, J.E.; Oliveira, A.S.d. Intelligent 3D Perception System for Semantic Description and Dynamic Interaction. Sensors 2019, 19, 3764. https://doi.org/10.3390/s19173764

AMA Style

Teixeira MAS, Nogueira RdCM, Dalmedico N, Santos HB, Arruda LVRd, Neves-Jr F, Pipa DR, Ramos JE, Oliveira ASd. Intelligent 3D Perception System for Semantic Description and Dynamic Interaction. Sensors. 2019; 19(17):3764. https://doi.org/10.3390/s19173764

Chicago/Turabian Style

Teixeira, Marco A.S., Rafael d.C.M. Nogueira, Nicolas Dalmedico, Higor B. Santos, Lucia V.R.d. Arruda, Flavio Neves-Jr, Daniel R. Pipa, Julio E. Ramos, and Andre S.d. Oliveira 2019. "Intelligent 3D Perception System for Semantic Description and Dynamic Interaction" Sensors 19, no. 17: 3764. https://doi.org/10.3390/s19173764

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop