Next Article in Journal
OD-XAI: Explainable AI-Based Semantic Object Detection for Autonomous Vehicles
Previous Article in Journal
Prediction of Shallow Failure on a Slope Using Volumetric Water Content Gradient Characteristics
Previous Article in Special Issue
Virtual Reality as a Reflection Technique for Public Speaking Training
 
 
Article

Object Pose Detection to Enable 3D Interaction from 2D Equirectangular Images in Mixed Reality Educational Settings

1
Department of Industrial Engineering, University of Trento, Via Sommarive 9, 38123 Trento, Italy
2
Institute of Computer Graphics and Vision, Graz University of Technology, Rechbauerstraße 12, 8010 Graz, Austria
3
Resources Innovation Center Leoben, Montanuniversität Leoben, Franz Josef Strasse 18, 8700 Leoben, Austria
*
Author to whom correspondence should be addressed.
Academic Editor: Jorge Martin-Gutierrez
Appl. Sci. 2022, 12(11), 5309; https://doi.org/10.3390/app12115309
Received: 20 April 2022 / Revised: 21 May 2022 / Accepted: 23 May 2022 / Published: 24 May 2022
(This article belongs to the Special Issue Human Empowerment through Mixed Reality)
In this paper, we address the challenge of estimating the 6DoF pose of objects in 2D equirectangular images. This solution allows the transition to the objects’ 3D model from their current pose. In particular, it finds application in the educational use of 360° videos, where it enhances the learning experience of students by making it more engaging and immersive due to the possible interaction with 3D virtual models. We developed a general approach usable for any object and shape. The only requirement is to have an accurate CAD model, even without textures of the item, whose pose must be estimated. The developed pipeline has two main steps: vehicle segmentation from the image background and estimation of the vehicle pose. To accomplish the first task, we used deep learning methods, while for the second, we developed a 360° camera simulator in Unity to generate synthetic equirectangular images used for comparison. We conducted our tests using a miniature truck model whose CAD was at our disposal. The developed algorithm was tested using a metrological analysis applied to real data. The results showed a mean difference of 1.5° with a standard deviation of 1° from the ground truth data for rotations, and 1.4 cm with a standard deviation of 1.5 cm for translations over a research range of ±20° and ±20 cm, respectively. View Full-Text
Keywords: image processing; 6DoF pose estimation; mixed reality; human empowerment; educational setting image processing; 6DoF pose estimation; mixed reality; human empowerment; educational setting
Show Figures

Figure 1

MDPI and ACS Style

Zanetti, M.; Luchetti, A.; Maheshwari, S.; Kalkofen, D.; Ortega, M.L.; De Cecco, M. Object Pose Detection to Enable 3D Interaction from 2D Equirectangular Images in Mixed Reality Educational Settings. Appl. Sci. 2022, 12, 5309. https://doi.org/10.3390/app12115309

AMA Style

Zanetti M, Luchetti A, Maheshwari S, Kalkofen D, Ortega ML, De Cecco M. Object Pose Detection to Enable 3D Interaction from 2D Equirectangular Images in Mixed Reality Educational Settings. Applied Sciences. 2022; 12(11):5309. https://doi.org/10.3390/app12115309

Chicago/Turabian Style

Zanetti, Matteo, Alessandro Luchetti, Sharad Maheshwari, Denis Kalkofen, Manuel Labrador Ortega, and Mariolino De Cecco. 2022. "Object Pose Detection to Enable 3D Interaction from 2D Equirectangular Images in Mixed Reality Educational Settings" Applied Sciences 12, no. 11: 5309. https://doi.org/10.3390/app12115309

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop