Real-Time 3D Mapping in Isolated Industrial Terrain with Use of Mobile Robotic Vehicle

Abstract

Simultaneous localization and mapping (SLAM) is a dual process responsible for the ability of a robotic vehicle to build a map of its surroundings and estimate its position on that map. This paper presents the novel concept of creating a 3D map based on the adaptive Monte-Carlo location (AMCL) and the extended Kalman filter (EKF). This approach is intended for inspection or rescue operations in a closed or isolated area where there is a risk to humans. The proposed solution uses particle filters together with data from on-board sensors to estimate the local position of the robot. Its global position is determined through the Rao–Blackwellized technique. The developed system was implemented on a wheeled mobile robot equipped with a sensing system consisting of a laser scanner (LIDAR) and an inertial measurement unit (IMU), and was tested in the real conditions of an underground mine. One of the contributions of this work is to propose a low-complexity and low-cost solution to real-time 3D-map creation. The conducted experimental trials confirmed that the performance of the three-dimensional mapping was characterized by high accuracy and usefulness for recognition and inspection tasks in an unknown industrial environment.

1. Introduction

Mobile robotics have seen an increase in interest in the last few decades, especially due to their ability to be deployed in a hazardous environment without endangering humans. Currently, it is common to use mobile robotic vehicles to accomplish missions such as environmental recognition, the inspection of urbanized and industrial terrains, and search and rescue operations. In the military field, they are employed in tasks such as reconnaissance missions, surveillance, intelligence gathering, hazardous-site exploration, and more. The robot can explore an area of interest, teleoperated from a remote area or autonomously. Knowing the obstacles in the vicinity of the robot is essential for the successful completion of a mission in a working area, avoiding any collisions that could make the robot unusable.

If the map is given a priori, the robot can self-localize by matching some features of the environment observed at the given moment to features of the same type existing in the known map. A feature suitable for self-localization is a static, salient object (or part of an object) in the environment, which can be described with respect to some other co-ordinate frame. On the other hand, the predefined map can be perceptually incompatible, i.e., it may not properly reflect all features in the environment perceived with the given sensing modality. Therefore, the robot should be able to build its own model of the environment.

In recent decades, many designs for mobile robotic vehicles created to explore a partially or fully unknown environment have been developed and demonstrated in both academia and industry [1,2,3,4,5,6,7]. The simultaneous localization and mapping (SLAM), currently one of the main research topics in robotics, appears to be the attractive solution to this problem [8,9,10,11,12,13,14,15,16]. The objective of SLAM is to concurrently build a map of investigated environment and allow the moving vehicle to localize itself within this environment using its on-board sensors.

At present, the most widely used SLAM frameworks are based on laser sensors, because these kinds of sensors can detect the distances of the obstacles around the robot with high accuracy. However, 2D-mapping methods based on laser scanners still have some problems. The obstacles can only be detected when they are in the same height with the sensors. Therefore, 3D-mapping methods based on multiple-laser sensors or 3D-distance sensors have been developed because these methods can reflect the shapes of an area and objects in the environment and allow the robot to plan its path with collision avoidance.

This paper focuses on a map creation for the recognition and inspection purposes of an isolated industrial environment carried out by a mobile wheeled robotic vehicle. The map is generated based on data sets obtained from a laser scanner (LIDAR) and an inertial measurement unit (IMU) installed on its board. We propose a robust real-time mapping concept based on probabilistic SLAM formulation, including both extended Kalman filters and particle filters, to build a 3D map of the surrounding area. The effectiveness of the developed solution has been confirmed by mapping experiments in the underground environment of the mine for different operational conditions.

The remainder of this paper is organized as follows. In Section 2, a brief description of the SLAM problem is provided. Details of the implementation of the proposed approach are presented in Section 3. Section 4 provides the results of the experimental studies carried out in an experimental underground mine. Finally, the conclusions and future works are summarized in Section 5.

2. SLAM Mapping Overview

Mapping approaches are generally divided into a 2D and 3D representation of the environment. Each type can be used for different tasks, e.g., navigation or workspace-state representation purposes. A planar map is good enough for navigation tasks. However, from the inspection point of view, a three-dimensional picture of the surroundings is definitely more useful, but it impacts on the calculation loading of the SLAM process and requires more advanced sensory systems.

There are many ways to describe the SLAM problem represented in the literature [17,18,19,20,21,22,23,24]. Currently, the most common way is to define SLAM as a probability density function, which can be described by the following generic form [25,26,27]:

$$p(X_{0:t},m|Z_{1:t},U_{1:t})$$

(1)

where:

• $X_{0:\mathrm{t}}=\left\{X_0, X_{1,}\dots ,X_t\right\}$—a sequence of the mobile vehicle’s poses (passed path) in a sampling time interval <0, t>;
• $U_{1:\mathrm{t}}=\left\{U_1{U}_2\dots U_t\right\}$—a sequence of control signals;
• $Z_{1:\mathrm{t}}=\left\{Z_1{Z}_2\dots Z_t\right\}$—a sequence of relative observations.

The observations $Z_{1:t}$ are made by sensors, which measure the distance to the closest objects in all directions from the poses $X_{1:t}$ at the same time instant.

Such a form of expression (1), where wanted values are reconstructed for all previous states, is called “Full SLAM”. The opposite approach, where only a recent position is estimated, called “Online SLAM”, can be calculated by recursive integration [23]. It is possible to apply Bayes and Markov rules to (1) and define the probability density function as a recursive process of predictions and corrections of the robot’s localization in the map m, which depends on motion (kinematic constrains and controls) and observation models [19]:

$$p(X_{0:t},m|Z_{1:t},U_{1:t})=\alpha · p(Z_t|X_{0:t},m,Z_{1:t-1},U_{1:t} · p(X_{0:t},m|Z_{1:t-1},U_{1:t})$$

(2)

where α is a normalization constant coefficient.

The recursive SLAM definition (2) is very convenient, since estimators such as the extended Kalman filter (EKF) [12,28] and particle filters [29,30,31] began to be applied to solve the estimation problem. The Kalman filter has the ability to use data from different sources and then efficiently fuse them into precise estimated values. It is one of its more valuable features. On the other hand, the biggest disadvantage of the EKF is its dependency on previous states that might cause huge average error decreases in the case of an unexpected measurement disaster. The use of EKF for the SLAM problem was first proposed in the work in reference [32], after which, many researchers further improved this method [12,33,34].

The particle filter [18,35,36] is a recursive filter using a Monte-Carlo algorithm to estimate the pose and movement path of the moving object. They are good and fast in global localization but are limited in their use of data from only one domain. Their wide usage for SLAM came from the brilliant idea of the Rao–Blackwellization method, which was introduced into regular map-building algorithms [37]. Although the particle filter algorithm can be used as an effective method to solve the SLAM problem [9,13,18,21,38], the main problem is that a large number of samples are required to approximate the posterior probability density of the system. The more complex the environment that the robot meets, the more samples that are needed to describe the posterior probability, and the higher the complexity of the algorithm.

Regarding the map construction of SLAM, there are three commonly used maps [39], i.e., feature-based maps, topological maps, and grid maps. The last one, also called an occupancy map, is the most common way for robots to describe the model of the environment. It divides the workspace into a series of grids, where each cell in the grid corresponds to binary random variables that describe the occupancy probability [8,20,40,41,42]. Hence, the map m is given by a product occupancy probability of each cell mi, which is associated with a certain point in three-dimensional Cartesian space [23,34]:

$$p\left(m\right)={\prod }_{i}p\left(m_{x_i,y_i,z_i}\right)$$

From the SLAM perspective, Equation (3) can be formulated as a probabilistic problem of the recursive pose estimation from given observations. There are many different SLAM techniques based on occupancy grids for map building in 2D or 3D space [10,22]. One of possible representations of the 3D map as an occupancy grid is octrees, which instead of the 2D quadtrees representation presented directly in references [8,42], uses an octree hierarchical data structure, where each volume-named node has eight child connections with inner nodes [25]. Currently, in the scientific literature, a rich number of occupancy grids-based SLAM techniques can be found [18,20,26,43]. Nevertheless, in any case, the best SLAM algorithm for a particular environment depends on hardware restrictions, the size of the map to be built, and the optimization criterion of the processing time.

The algorithms depend heavily on the sensors with which the robot perceives its surrounding. The selection and installation of sensors determine the specific form of observation results, and also affect the difficulty of SLAM problems. In case of mobile robots performing in indoor environments, exploration cameras or LIDARs are mostly used. Some approaches use a fusion of laser and visual data [34]. As mentioned before, LIDARs provide data that are more accurate, robust, less noisy, and sensitive to changes in the lighting. For these reasons, it is a very reliable data acquisition mechanism for industrial site mapping [44]. However, due to the limited number of dimensions, the laser data are not sufficient to estimate the robot’s pose on uneven ground with six degrees of freedom (DOF) [45]. Therefore, most algorithms need auxiliary data from other sensors, such as an inertial measurement unit (IMU) or odometer. In recent years, along with the rapid development of artificial intelligence (AI), learning methods are also used to solve the SLAM problem [46,47,48].

3. Hybrid Mapping Concept for Inspection Purposes

The map-building approach applied in this paper has its basis in a robot’s pose estimation, which is presented in references [42,49], called the AMCL-EKF, which connects two separate paradigms—Kalman and particle filters. The pose is estimated by the extended Kalman filter by using measurements data from LIDAR and IMU sensors and adaptive Monte-Carlo localization (AMCL).

In the case of a planar motion, the local pose is described by a state vector ${\mathit{X}}^{\left(2\right)}$ consisting of three components:

$${\mathit{X}}^{\left(2\right)}={\left[\begin{array}{ccc}x& y& \psi \end{array}\right]}^T$$

(3)

where x, y are Cartesian positions and ψ is a yaw angle.

However, from the inspection point of view, tridimensional maps showing environment features in their natural scale are more preferred. Due to the fact that 3D mapping requires information about the global location of the robotic vehicle with six degrees of freedom (DOF), it is necessary to recreate missing coordinates. It may be performed by taking the needed coordinates from the local 3D pose that the EKF performs [49,50] (see Figure 1). Then, the state vector ${\mathit{X}}^{\left(3\right)}$ can be written in the following form:

$${\mathit{X}}^{\left(3\right)}={\left\{\begin{array}{cc}{\mathit{X}}^{\left(2\right)}& \begin{array}{ccc}z& \phi & \theta \end{array}\end{array}\right\}}^T$$

(4)

where z is the Cartesian position and φ, θ are roll and pitch angles. The global localization can be solved by a Rao–Blackwellized 2D SLAM algorithm [18].

As far as ${\mathit{X}}^{\left(3\right)}$ is delivered, it allows for the creation of a three-dimensional map. In this way, the 2D SLAM algorithm is used continuously to efficiently build the 2D map, and at the same time the 3D SLAM algorithm (the delivered precise ${\mathit{X}}^{\left(3\right)}$ and recent cloud of points data allows to create the 3D map (see Figure 2)). This task can be performed both online as a separate task or offline postprocess based on previously collected sensor data.

The efficiency and accuracy of the developed method, called “hybrid 3D SLAM”, for locating the robot in unknown terrains, have been confirmed in several simulation experiments presented in our previous works [19,50,51], where the 3D-mapping algorithm was implemented in accordance with the Robot Operation System framework, and was verified with using the V-REP simulation environment. The carried-out investigations affirmed the legitimacy of the proposed approach to 3D-map building for inspection tasks.

4. Environment Description and Experimental Results

4.1. Robotic Vehicle and Environment

The SLAM strategy presented in the previous section was implemented in the Dr Robot Jaguar 4 × 4 mobile vehicle constructed by the Canadian company Dr Robot Inc. (see Figure 3). The robot is equipped with the LIDAR (Velodyne VPL-16) and the IMU (Pololu MinIMU-9) sensors.

The robot is a four-wheeled robotic platform designed for tough indoor and outdoor operation, and has the following operational parameters: mass 20 kg, max speed 11 km/h, and a ground clearance 88 mm. The Velodyne VLP-16 LIDAR uses an array of 16 infra-red lasers paired to detect and measure distances to objects. The array provides 300,000 data points per second in real-time. The IMU sensor Pololu MinIMU-9 is a compact board that combines a 3-axis gyroscope, a 3-axis accelerometer, and a 3-axis magnetometer for measuring robot data parameters.

Investigations were conducted in a real industrial environment—the experimental underground mine being a part of the AGH University of Science and Technology infrastructure. It was assumed that the robotic vehicle had to inspect the site in two operational cases, i.e., before and after the mining disaster. This task required the robot to move on both smooth and rough surfaces that were sometimes blocked by degraded infrastructure. Figure 4 shows two fragments of real operational scenarios.

4.2. Experiments

Carried out trials were divided into two stages, i.e., creating the map of the mine area for normal working conditions and after shutdown due to an accident. In accordance with the methodology described in Section 3, the 2D maps were first built using the AMCL-EKF technique, and then the 3D maps were estimated by applying the Rao–Blackwellized SLAM approach.

The results of the map creation for the terrain before accident are presented in Figure 5. Both 2D and 3D maps precisely recreate the real state of the mine’s corridors with all observable main features. The 2D map (Figure 5a) clearly shows free spaces and therefore can be fully used for navigation tasks, whereas the 3D map (Figure 5b) contains all features of the passed corridors and therefore is much more useful for a precise and safe inspection of the underground environment. It is especially important when some parts of the terrain have become inaccessible.

Such a situation is presented in Figure 6, where a photographic image of the corridor and destroyed infrastructure is shown. Due to the disaster, the main road was partially blocked, and the terrain became rough.

The results of the map creation are depicted in Figure 7. The depicted maps show the examined area from the same perspective in which it was captured by the camera. The comparison of the scenes from the photo and the generated maps shows a high compliance of the presented mine’s environment. The general differences between 2D and 3D maps, shown in Figure 7a,b, respectively, are similar to those described for the previous case. It can be seen that the 3D map is accurate enough to assess the real state of the environment.

The performed investigations showed a good compliance of the obtained 2D map with the geodetic map of the laboratory ground mine. The final product, which is a 3D-space map, properly reflects the current of the working environment.

However, its quality and practical usability is limited by specific problems related to the limitations connected with real working conditions. Although the AMCL provides a global location based on reference points, the quality of the pose estimation process strongly depends on errors concerned with the robot’s slippage or skidding. A problem with LIDAR can be long spaces with a limited number of potential reference points, such as a tunnel with single surrounding features.

5. Conclusions

The method for the concurrent construction of a 3D map of the industrial terrain realized with use of a mobile wheeled robot is presented and discussed in the paper. Since the three-dimensional map shows the complete picture of the explored terrain, it is more convenient for use in environmental inspection tasks.

The main contribution of this work is to develop a real-time method for creating a 3D map in both structured and unstructured industrial environments. We propose to solve this problem by using LIDAR to preprocess the scanning and IMU to generate odometries.

After mapping the environment, we created a grid map based on the 2D SLAM, the AMCL, and the EKF. The robot’s global position was determined by means of the Rao–Blackwellized technique. This approach significantly reduced computational costs in comparison with the Full 3D SLAM.

The efficiency, accuracy, and robustness of the developed hybrid 3D SLAM approach for 3D-map creation was validated by a number of real experiments conducted in the different scenarios of the underground mine. All obtained results confirmed the correctness of the developed solution for the creation of the 3D map in the indoor environment. The findings also demonstrated that the worked-out algorithm can be run in conditions similar to real ones by using a mobile robotic vehicle equipped with modest hardware.

Further works will be devoted to examining the applicability of the developed solution to create a 3D map in other types of real environments, such as underwater or aerial.