Tomographic Feature-Based Map Merging for Multi-Robot Systems

Abstract

Multi-robot systems require collective map information on surrounding environments to efficiently cooperate with one another on assigned tasks. This paper addresses the problem of grid map merging to obtain the collective map information in multi-robot systems with unknown initial poses. If inter-robot measurements are not available, the only way to merge the maps is to find and match the overlapping area between maps. This paper proposes a tomographic feature-based map merging method, which can be successfully conducted with relatively small overlapping areas. The first part of the proposed method is to estimate a map transformation matrix using the Radon transform which can extract tomographically salient features from individual grid maps. The second part is to determine the search space using Gaussian mixture models based on the estimated map transformation matrix. The final part is to optimize an objective function modeled from tomographic information within the determined search space. Evaluation results with various pairs of individual maps produced by simulations and experiments showed that the proposed method can merge the individual maps more accurately than other map merging methods.

1. Introduction

Multi-robot systems have received attention in recent years because they have many advantages over single-robot systems such as time-efficiency and cost reduction [1]. Even though multiple robots can complete not only a single task faster but also multiple tasks simultaneously, there are many challenging problems to be resolved to realize the multi-robot systems. One of the challenging problems is to share the knowledge of their surrounding environments because the shared knowledge is the fundamental information for task allocation, path planning, and collision avoidance. Generally, in multi-robot systems, the knowledge of the surrounding environments of a single robot is represented by a grid map that consists of occupied grid points indicating obstacles and unoccupied grids indicating empty places. Then, the shared knowledge can be obtained by merging the individual grid maps of the robots. Note that the performance of map merging affects the performance of the overall multi-robot system. For example, when two robots perform their own tasks, a robot can obtain the information on unexplored areas from the merged map including the individual map of the other robot. But, if the merged map is inaccurate, the former robot cannot efficiently perform its own task because the path planned in the inaccurately merged map is not efficient.

If the relative initial poses among multiple robots are known to one another, the problem of map merging can be easily solved because the map transformation matrices among the individual maps can be derived from the relative initial poses among the robots. However, if the relative initial poses among the robots are unknown to one another, the problem of map merging becomes more challenging due to the lack of initial clues to map the transformation matrix. The researches on map merging with unknown relative initial poses can be divided into two categories, which are direct map merging and indirect map merging.

The direct map merging is to directly acquire the map transformation matrix by obtaining the inter-robot measurements which consist of relative distance and orientation between robots, which can be performed under a rendezvous. Konolige et al. [2] proposed a hypothesis-and-verification based map merging algorithm. The first step, called hypothesis, sees robots try to meet each other with inter-robot measurements. If they succeed in meeting each other at the estimated location, the hypothesis is accepted, which means that their maps are merged. Zhou et al. [3] proposed a geometry and formulations to acquire the map transformation matrix using inter-robot measurements obtained from omnidirectional vision sensors. Tungadi et al. [4] proposed a two-step map merging system which is the combination of place recognition and laser scan matching. Kim et al. [5] presented an extension to iSAM (incremental smoothing and mapping) to facilitate online multi-robot mapping based on multiple pose graphs. Their method was conducted by a probabilistic approach to solving the full multi-robot nonlinear SLAM optimization problem when robots encountered each other. Li et al. [6] proposed a vehicle-to-vehicle relative pose estimation method using an objective function based on occupancy likelihood and provide some concrete procedures designed in the spirit of a genetic algorithm to optimize the defined objective function. Dinnissen et al. [7] proposed a reinforcement learning-based map merging method using the current status of the mapping particle filters and the current status of the environment when robots meet each other upon rendezvous, which can decide when is the best to merge. Garcia-Cruz et al. [8] proposed a method to reduce the processing time for obstacle or robot detection, which can be used to prevent missing the chances to obtain inter-robot measurements. Lindner et al. [9] proposed a new approach of estimating this residual error by use of a friction model to improve their laser scan system, which can be used to obtain inter-robot or robot-to-object measurements. In our previous work [10], we proposed a probabilistic map merging framework for multi-robot Rao-Blackwellized particle filter-based simultaneous localization and mapping (RBPF-SLAM). The most appropriate map merging bases were obtained by Gaussian processes, and map merging was performed by the inter-robot measurements. However, since the inter-robot measurements included inevitable errors caused by imperfect sensors, the performance of the direct map merging depends on the system configuration to acquire the inter-robot measurements. In another of our previous works [11], we proposed a grid map merging technique based on one-way observations, which reduced the conditions on rendezvous points. However, it needed supplementary methods to improve the accuracy of map merging such as curvature-based map matching and particle swarm optimization.

The indirect map merging acquires the map transformation matrix by finding and matching the overlapping areas of the individual maps of robots, which is called map matching. Generally, map matching techniques define an objective function which represents how much the individual maps are matched. Then, they find iteratively the map transformation matrix as the optimal solution of the objective function. Zhou et al. [3] used the nearest neighbor test as a map matching algorithm to improve the accuracy of direct map merging based on inter-robot measurements. Birk et al. [12] proposed a good index to indicate the similarity between individual maps for map merging and applied it to the random walk stochastic search algorithm. Leon et al. [13] mentioned that their main problem was to obtain very precise inter-robot detection for map merging. They solved the problem by laser scan matching techniques. Wang et al. [14] also used the well-known scan matching technique, ICP (iterative closest points), to more accurately merge individual maps. Saeedi et al. [15,16] proposed a combined approach to map merging, which includes image preprocessing, neural networks, cross-correlation, approximating the relative transformation matrix, tuning of the transformation through the Radon image transform and similarity index. Howard [17] proposed the concept of the virtual robot traveling backward in time to build a complete map. After direct map merging with the inter-robot measurements, the virtual robots traveled backward in time and updated the past information iteratively to improve the accuracy of the merged map. However, his work was developed in the framework of iterative processes which could cause much computation time. Carpin [18] tackled the iterative process of the conventional map matching algorithms using spectral information on robot maps. His method reduced computation time compared with the previous grid map matching techniques in a deterministic and non-iterative manner. In another previous work [19], we proposed a variant of spectra-based map merging algorithm using virtual supporting lines, which was suitable for merging not grid maps but feature maps.

This paper proposes a tomographic feature-based map merging method for multi-robot systems with unknown initial poses, which is categorized as indirect map merging. The first part of the proposed method is to estimate a map transformation matrix using the Radon transform which can extract tomographically salient features from individual grid maps. The second part of the proposed method is to determine the search space using Gaussian mixture models based on the estimated map transformation matrix. The final part of the proposed method is to optimize an objective function modeled from tomographic information within the determined search space. No predetermined rendezvous between robots and no common landmark between maps and no a priori information on overlapping regions between maps are required in the proposed map merging method. The remainder of this paper is organized as follows. In Section 2, the formulation of the map transformation and the problem of the overlapping areas in grid map merging are described. In Section 3, the proposed grid map merging method is presented in detail. Section 4 shows experimental results with public grid map data and a real multi-robot system. Finally, in Section 5, we give conclusions.

2. Problem Description

2.1. Map Merging in Multi-Robot Systems

When multi-robot systems with unknown initial poses are utilized to explore unknown areas, each robot has built the individual map of the explored areas by the simultaneous localization and mapping (SLAM) technique with range or vision sensors. If a laser scan sensor is used as the range sensor, the individual map is generally represented by a grid map that consists of occupied, unoccupied, and unknown grids. To efficiently obtain a collective map for the surrounding environments, the robots should share and merge accurately their own individual maps. The map merging technique is the key to reducing the cost for exploration, which is the main advantage of multi-robot systems because the individual maps of the different robots represent the different parts of the given environment. When the map merging technique is implemented, the issue of the missing data may be raised. The missing data can be considered as two categories: systematic missing data and environmental missing data. First, the systematic missing data may occur from communication systems between robots, which depends on the real-time performance of the robot platforms. In this work, I assumed that the systematic missing data does not occur to focus on improving the accuracy of a map merging algorithm. This assumption can be relaxed by analyzing the timing diagram of communications between individual robots and optimizing the number of messages and the period of transmissions. The detailed description for solving the problem of real-time communications is presented in my previous works [20,21]. The problem of the systematic missing data may be considered in my future work. Second, the environmental missing data may occur from the occlusion between individual robots, which means that a structure or an object may be differently observed due to the different trajectories of the robots and the limitations of sensor ranges and line-of-sight. In other words, for the same structure, one robot can observe the large part of the structure, but the other robot can observe the small part structure. In the worst-case, the other robot can observe nothing for the same structure. However, it is hard that the information on the occlusion is unknown for individual robots in real multi-robot systems, which is the most challenging point of map merging algorithms. If the information on the occlusion is known for individual robots, the level of difficulty of map merging becomes low because it can be avoided that matching missing data from one robot with missing data from another robot as you commented. However, this is not a reasonable assumption because the information on the occlusion is generally unknown for individual robots in real multi-robot systems. Consequently, the difficulties caused by the environmental missing data should be overcome by algorithmic abilities.

Grid map merging can be formulated as follows. A 2D grid map $M$ is assumed as a matrix with $n_r$ rows and $n_c$ columns, which can be regarded as a $n_r\times n_c$ binary image. Each grid contains map information on the location represented by the grid in the global coordinate system. For convenient map transformation, $M$ is represented as a matrix with three rows and $N_{occ}$ columns. $N_{occ}$ is the number of occupied grids in the original $M$. The first and second rows represent x-coordinate and y-coordinate of the occupied grids, respectively. The last row is filled with 1 so that the computation with a map transformation matrix (MTM) is performed conveniently. Given two grid maps, $M_1$ and $M_2$, the MTM $T$ which translates ${\mathsf{\Delta }}_x$ in the direction of x-coordinate and ${\mathsf{\Delta }}_y$ in the direction of y-coordinate, and rotates ${\mathsf{\Delta }}_{\theta }$ in a counter-clockwise is defined as follows:

$$T\left({\mathsf{\Delta }}_x,{\mathsf{\Delta }}_y,{\mathsf{\Delta }}_{\theta }\right)=\left[\begin{array}{lll}{\cos \mathsf{\Delta }}_{\theta }\hfill & -{\sin \mathsf{\Delta }}_{\theta }\hfill & {\mathsf{\Delta }}_x\hfill \\ {\sin \mathsf{\Delta }}_{\theta }\hfill & {\cos \mathsf{\Delta }}_{\theta }\hfill & {\mathsf{\Delta }}_y\hfill \\ 0\hfill & 0\hfill & 1\hfill \end{array}\right],$$

(1)

where ${\overline{M}}_2=T{M}_2$. Here, ${\overline{M}}_2$ is a new map rotated and translated from $M_2$, which is represented in the same coordinate system with $M_1$.

2.2. The Accuracy of Map Merging

To fully take advantage of multi-robot systems, the map merging should be accurately conducted to provide accurate collective information on the surrounding environments for each robot. For example, if a robot refers to the merged map including non-exploring areas, the robot will decide its own collision-free and efficient path to complete the assigned tasks through the merged map. If the merged map is not accurate, the robot may be faced with unexpected collisions or task failure. When robots use the measurements of relative robot poses for map merging, inevitable errors caused by the uncertainties of sensors and communications should be considered. Besides, the measurements of relative robot poses may not be available due to the limitations of extreme environments and unexpected damages on sensors. Therefore, it is hard to solve the problem of map merging solely with direct observation for relative poses among the robots.

This paper focuses on the indirect map merging technique which obtains the MTM between the individual maps by finding and matching the overlapping areas among them, which can be successfully conducted without any inter-robot measurements. If the corridors are perfectly orthogonal or symmetric, map merging cannot be accurately conducted because there are multiple maximum correlations. In that case, map merging needs additional methods as visual features and indoor localization systems. This work has been conducted with an assumption that the explored environments are not orthogonal or symmetric corridors. Although the conventional indirect map merging techniques have their own advantages, they are commonly faced with the problem of local maxima or minima. There are several hybrid works to avoid the local maxima or minima using one-way observation [11] or common landmarks [1], however, they do not meet the conditions in this work. This paper proposes a new map merging method using tomographic features, which can avoid the local maxima or minima more robustly with no inter-robot measurements and no a priori information on overlapping regions.

3. Proposed Method

This section describes the proposed map merging method using tomographic features without any initial relative poses among individual robots. The proposed method consists of three parts which are the estimation of a coarse MTM using the Radon transform, the search space determination using the multivariate normal distribution, and the acquisition of a fine MTM based on tomographically derived score function.

3.1. Estimation of the MTM Using Tomographic Features

The Radon transform [22] is an integral transform which consists of the integral of a function over straight lines. The Radon transform data is often called a $sinogram$ because the Radon transform of a Dirac delta function is a distribution supported on the graph of a sine wave. Let $f\left(x,y\right)$ be a continuous function vanishing outside some large disc in $R^2$ as shown in Figure 1a. The straight line $L$ is parametrized by:

$$\left(x\left(t\right),y\left(t\right)\right)=\left(\left(t\sin \alpha +s\cos \alpha \right),\left(-t\cos \alpha +s\sin \alpha \right)\right),$$

(2)

where $t$ is a parameter for the parametric form of $L$, and $s$ is the distance of $L$ from the origin, and $\alpha$ is the angle of the normal vector to $L$ with the $x$ axis. The Radon transform, $R{T}_f$, is a function defined on the space of straight lines $L$ in $R^2$ by the line integral along each such line:

$$\begin{array}{ll}R{T}_f\left(s,\alpha \right)& ={\displaystyle {\int }_{-\infty }^{\infty }f\left(x\left(t\right),y\left(t\right)\right)\mathrm{d}t}\\ & ={\displaystyle {\int }_{-\infty }^{\infty }f\left(\left(t\text{sin}\alpha +s\text{cos}\alpha \right),\left(-t\text{cos}\alpha +s\text{sin}\alpha \right)\right)\mathrm{d}t}\\ & \equiv S_f.\end{array}$$

(3)

The visualization of $S_f$ called $sinogram$ is shown in Figure 1b, which is 3D data with intensity along $\alpha$-$s$ axes. The tomographic features represented by the sinogram can be used to find the salient parts for grid map merging.

3.1.1. Estimation of a Rotation Angle

In the sinogram of a grid map, tomographic features are represented by the variation of column-wise levels according to $\alpha$, which can give the clue to the shape of the grid map. Especially, ${\alpha }_{max}$ at the location of the maximum level indicates that the most elongated direction of the grid map is ${\alpha }_{max}-{90}^{°}$. For example, if the maximum intensity of $S_f$ in Figure 1b appears at $\left(3,{106}^{°}\right)$, the most elongated direction of $f\left(x,y\right)$ is ${16}^{°}$. Therefore, the rotation angle between two grid maps can be estimated by comparing the locations indicating the maximum levels in their sinograms.

Figure 2 shows two individual grid maps, $M_1$ and $M_2$, which are generated from the whole map of ACES3 [23]. The sizes of $M_1$ and $M_2$ are respectively $r_1\times c_1$ and $r_2\times c_2$. Their sinograms are respectively generated for $0^{°}\le \alpha \le {180}^{°}$ as follows:

$$S_{M_1}=R{T}_{M_1}\left(s,\alpha \right), -{\rho }_1\le s\le {\rho }_1,$$

(4)

$$S_{M_2}=R{T}_{M_2}\left(s,\alpha \right), -{\rho }_2\le s\le {\rho }_2,$$

(5)

where ${\rho }_n$ is the maximum $s$-value of $S_{M_n}$ which are computed as follows:

$${\rho }_n=\lceil \sqrt{{\left(r_n-r_n^{\ast }\right)}^2+{\left(c_n-c_n^{\ast }\right)}^2}\rceil +1,$$

(6)

where the center of $M_1$ is $\left(r_1^{\ast },c_1^{\ast }\right)=\left(\lfloor \frac{r_1+1}{2}\rfloor ,\lfloor \frac{c_1+1}{2}\rfloor \right)$, and the center of $M_2$ is $\left(r_2^{\ast },c_2^{\ast }\right)=\left(\lfloor \frac{r_2+1}{2}\rfloor ,\lfloor \frac{c_2+1}{2}\rfloor \right)$. Here, $\lceil \cdot \rceil$ denotes a ceiling function, and $\lfloor \cdot \rfloor$ denotes a floor function.

Let $\left(s_{1,max},{\alpha }_{1,max}\right)$ and $\left(s_{2,max},{\alpha }_{2,max}\right)$ be the locations of the maximum levels in $S_{M_1}$ and $S_{M_2}$. Then, the rotation angle ${\mathsf{\Delta }}_{\theta }$ between $M_1$ and $M_2$ can be estimated as follows:

$${\mathsf{\Delta }}_{\theta }={\alpha }_{2,max}-{\alpha }_{1,max}.$$

(7)

For example, the given maps and their sinograms as shown in Figure 3, since ${\alpha }_{2,max}={106}^{°}$ and ${\alpha }_{1,max}={91}^{°}$, the rotation angle ${\mathsf{\Delta }}_{\theta }={15}^{°}$. Note that in this work since the sinograms were extracted for the range of $0\le \alpha \le {180}^{°}$ as shown in Figure 3, the problem of Moebius strip symmetry of sinogram can be avoided. Consequently, the rotation angle is acquired from $-{180}^{°}\le {\mathsf{\Delta }}_{\theta }\le {180}^{°}$.

3.1.2. Estimation of $X$-$Y$ Translations

The tomographic features represented in sinograms also provide the clue to translations between two grid maps. Based on the estimated rotation angle ${\mathsf{\Delta }}_{\theta }$ at the previous step, ${\tilde{M}}_2$ is obtained by rotating $M_2$ of ${\mathsf{\Delta }}_{\theta }$ as follows:

$${\tilde{M}}_2^a=T\left(0,0,{\mathsf{\Delta }}_{\theta }\right)M_2^a,$$

(8)

where ${\tilde{M}}_2^a$ and $M_2^a$ are the alternative forms of ${\tilde{M}}_2$ and $M_2$, respectively. The alternative form is another version of the original grid map, which is a $3\times N_{occ}$ matrix for the convenient matrix calculation, where $N_{occ}$ is the number of occupied grid points. The first and second rows represent the x and y coordinates of occupied grid points, respectively. The last row is filled with 1 for the convenient matrix multiplication.

The size of ${\tilde{M}}_2$ is ${\tilde{r}}_2\times {\tilde{c}}_2$. For $0^{°}\le \alpha \le {180}^{°}$, the sinogram of ${\tilde{M}}_2$ is generated as follows:

$$S_{{\tilde{M}}_2}=R{T}_{{\tilde{M}}_2}\left(s,\alpha \right), -{\tilde{\rho }}_2\le s\le {\tilde{\rho }}_2,$$

(9)

where ${\tilde{\rho }}_2$ is the maximum $s$-value of $S_{{\tilde{M}}_2}$, which is computed as follows:

$${\tilde{\rho }}_2=\lceil \sqrt{{\left({\tilde{r}}_2-{\tilde{r}}_2^{\ast }\right)}^2+{\left({\tilde{c}}_2-{\tilde{c}}_2^{\ast }\right)}^2}\rceil +1,$$

(10)

The estimation of each translation can be obtained by finding the $s$-value which maximizes the cross-correlation between two $partial sinograms$ corresponding to a certain $\alpha$-value. Each partial sinogram indicates the distribution of grids along the $\alpha$ direction, which means that it is the distribution of $s$ corresponding to the $\alpha$-value. Given $S_{M_n}$ with $2{\rho }_n+1$ rows and ${\varphi }_n$ columns generated from the $n$-th map $M_n$ with $r_n$ rows and $c_n$ columns, the partial sinogram corresponding to the ${\alpha }^{\prime }$-value is defined as follows:

$$P{S}_{M_n}^{{\alpha }^{\prime }}\left(t\right)=S_{M_n}\left(t-{\rho }_n-1,{\alpha }^{\prime }\right), 0<t\le 2{\rho }_n+1,$$

(11)

where ${\rho }_n$ is the maximum $s$-value of $S_{M_n}$, which can be computed by (6).

Especially, the partial sinogram of $s$ for $x$ and $y$ axes may be simply computed with ${\alpha }^{\prime }={180}^{°}$ and ${\alpha }^{\prime }={90}^{°}$ such as $P{S}_{M_n}^{{180}^{°}}\left(s\right)$ and $P{S}_{M_n}^{{90}^{°}}\left(s\right)$. However, since the sizes of the maps are generally different as shown in $M_2$ and ${\tilde{M}}_2$, the partial sinograms for $x$ and $y$ axes are redefined to obtain the cross correlation function between them. In other words, they should be represented with the same dimension, which is defined as the maximum size of the maps $M_1$ and ${\tilde{M}}_2$ as follows: $r_m=\max \left(r_1,{\tilde{r}}_2\right)$ and $c_m=\max \left(c_1,{\tilde{c}}_2\right)$. Then, the partial sinograms of $M_1$ for $x$ and $y$ axes are respectively redefined by

$$P{S}_{M_1}^X\left(i\right)=(\begin{array}{ll}P{S}_{M_1}^{{180}^{°}}\left(i+{\rho }_1^{\ast }-c_m^{\ast }\right)\hfill & 1\le i\le 2c_m^{\ast }+1\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array},$$

(12)

$$P{S}_{M_1}^Y\left(j\right)=(\begin{array}{ll}P{S}_{M_1}^{{90}^{°}}\left(j+{\rho }_1^{\ast }-r_m^{\ast }\right)\hfill & 1\le j\le 2r_m^{\ast }+1\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array},$$

(13)

where ${\rho }_1^{\ast }=\lceil \frac{2{\rho }_1+1}{2}\rceil$, $c_m^{\ast }=\lceil \frac{2c_m+1}{2}\rceil$, and $r_m^{\ast }=\lceil \frac{2r_m+1}{2}\rceil$.

The partial sinograms of ${\tilde{M}}_2$ for $x$ and $y$ axes are similarly redefined. Then, the $X$ and $Y$ translation amounts between $M_1$ and ${\tilde{M}}_2$ can be estimated by finding the arguments which maximize the discrete cross correlation between their partial sinograms. The discrete cross correlation between them for $x$-axis is computed as follows:

$$CC{X}_{M_1{\tilde{M}}_2}\left(\tau \right)=(\begin{array}{ll}{\displaystyle {\sum }_{k=-\infty }^{\infty }X{S}_{M_1}\left(K+\tau \right)X{S}_{{\tilde{M}}_2}\left(k\right)}\hfill & 1\le \tau \le c_m\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array},$$

(14)

where $c_m$ is the maximum column size of the map matrices $M_1$ and ${\tilde{M}}_2$. Within the maximum range, the discrete cross-correlation is computed. Otherwise, it is zero. Figure 4 shows the visualization of $P{S}_{M_1}^X$, $P{S}_{{\tilde{M}}_2}^X$, and $CC{X}_{M_1{\tilde{M}}_2}$, respectively. Similarly, the cross correlation between them for $y$-axis is computed as follows:

$$CC{Y}_{M_1{\tilde{M}}_2}\left(\upsilon \right)=(\begin{array}{ll}{\displaystyle {\sum }_{l=-\infty }^{\infty }Y{S}_{M_1}\left(l+\upsilon \right)Y{S}_{{\tilde{M}}_2}\left(l\right)}\hfill & 1\le \upsilon \le r_m\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array},$$

(15)

where $r_m$ is the maximum row size of the map matrices $M_1$ and ${\tilde{M}}_2$. Within the maximum range, the discrete cross-correlation is computed. Otherwise, it is zero. Figure 5 shows the visualization of $P{S}_{M_1}^Y$, $P{S}_{{\tilde{M}}_2}^Y$, and $CC{Y}_{M_1{\tilde{M}}_2}$, respectively.

Finally, the $X$-translation and the $Y$-translation amounts between the maps are respectively obtained by selecting the argument which maximizes $CC{X}_{M_1{\tilde{M}}_2}$ and $CC{Y}_{M_1{\tilde{M}}_2}$ as follows:

$${\mathsf{\Delta }}_x=\begin{array}{l}\underset{\mathsf{\tau }}{\mathrm{argmax}} CC{X}_{M_1{\tilde{M}}_2}\left(\tau \right)\hfill \end{array},$$

(16)

$${\mathsf{\Delta }}_y=\begin{array}{l}\underset{\upsilon }{\mathrm{argmax}} CC{Y}_{M_1{\tilde{M}}_2}\left(\upsilon \right)\hfill \end{array}.$$

(17)

Then, since the elements of the MTM between $M_1$ and $M_2$ are completely computed, $M_2$ can be merged into $M_1$ by rotation and translations.

3.2. Search Space Determination with Gaussian Mixture Models

The MTM estimated by matching the tomographic features of the individual maps may be inaccurate due to inevitable transformation errors and local maxima. The more accurate MTM can be obtained by optimization algorithms, but it requires too much computation time because the initially given search space is too large. In the proposed method, the search space can be efficiently determined using Gaussian mixture models (GMMs). The search range for the rotation angle is determined using a univariate Gaussian mixture model (UGMM), and the search space for the translations is determined using a multivariate Gaussian mixture model (MGMM).

First, the numbers of the reference points in $S_{M_1}$ and $S_{M_2}$ to determine the means and standard deviations of the univariate random variable for the rotation angle are respectively determined with the concept of entropy as follows:

$$N_{{\theta }_1}=\lceil -{\displaystyle \sum p_1{\log }_2{p}_1}\rceil ,$$

(18)

$$N_{{\theta }_2}=\lceil -{\displaystyle \sum p_2{\log }_2{p}_2}\rceil ,$$

(19)

where $p_1$ and $p_2$ are respectively the histograms of $S_{M_1}$ and $S_{M_2}$. Then, the reference points for each sinogram are extracted by $N_{{\theta }_1}$ and $N_{{\theta }_2}$ in the descending order of the intensities of $S_{M_1}$ and $S_{M_2}$ as follows: ${\Theta }_1={\left\{{\alpha }_{1,n}\right\}}_{n=1,\cdots ,N_{UGMM}}$ and ${\Theta }_2={\left\{{\alpha }_{2,n}\right\}}_{n=1,\cdots ,N_{UGMM}}$ where $N_{UGMM}=\min \left(N_{{\theta }_1},N_{{\theta }_2}\right)$, which is the number of the univariate random variables for the rotation angle. Then, means of the univariate random variable for the rotation angle are calculated as follows:

$${\Omega }_{\theta }=\left\{{\alpha }_{2,n}-{\alpha }_{1,n} | {\alpha }_{1,n}\in {\Theta }_1,{\alpha }_{2,n}\in {\Theta }_2\right\}.$$

(20)

The above equation can be rewritten for the simplicity as follows: ${{\Omega }_{\theta }}^{\prime }={\left\{{\beta }_i\right|{\beta }_i\in {\Omega }_{\theta }\}}_{i=1,\cdots ,N_{UGMM}}$. The standard deviations of the univariate random variable is calculated as follows:

$${\Sigma }_{\theta }={\left\{{\sigma }_{\theta ,i}|{\sigma }_{\theta ,i}=\sqrt{\left({\displaystyle \sum p_1{\log }_2{p}_1}\right)\left({\displaystyle \sum p_2{\log }_2{p}_2}\right)}\right\}}_{i=1,\cdots ,N_{UGMM}}$$

(21)

Next, the means of the multivariate random variable for $X$ and $Y$ translations are respectively calculated using the sets of peak locations, ${\Lambda }_X$ and ${\Lambda }_Y$ which are sorted in the descending order of the correlation values of $CC{X}_{M_1{\tilde{M}}_2}$ and $CC{Y}_{M_1{\tilde{M}}_2}$ as follows:

$${\Omega }_X={\{{\tau }_n | {\tau }_n\in {\Lambda }_X\}}_{n=1,\cdots ,N_{MGMM}},$$

(22)

$${\Omega }_Y={\{{\nu }_m | {\nu }_m\in {\Lambda }_Y\}}_{m=1,\cdots ,N_{MGMM}},$$

(23)

where $N_{MGMM}=\max (\lceil {\displaystyle -\sum p_x{\log }_2{p}_x}\rceil ,\lceil {\displaystyle -\sum p_y{\log }_2{p}_y}\rceil )$. Here, $p_x$ and $p_y$ are the histograms of $CC{X}_{M_1{\tilde{M}}_2}$ and $CC{Y}_{M_1{\tilde{M}}_2}$, respectively. The standard deviations for the multivariate random variables are respectively calculated using the obtained observation points and the coarse MTM as follows:

$${\Sigma }_X={\left\{{\sigma }_{x,i}|{\sigma }_{x,i}=\sqrt{\left({\displaystyle \sum p_x{\log }_2{p}_x}\right)\left({\displaystyle \sum p_y{\log }_2{p}_y}\right)}\right\}}_{i=1,\cdots ,N_{MGMM}},$$

(24)

$${\Sigma }_Y={\left\{{\sigma }_{y,i}|{\sigma }_{y,i}=\sqrt{\left({\displaystyle \sum p_x{\log }_2{p}_x}\right)\left({\displaystyle \sum p_y{\log }_2{p}_y}\right)}\right\}}_{i=1,\cdots ,N_{MGMM}}.$$

(25)

Now, if there is no dependency among the variances, the covariance matrices of the multivariate random variables for $X$ and $Y$ translations are calculated using ${\Sigma }_X$ and ${\Sigma }_Y$ as follows:

$${\Sigma }_{XY}={\left\{\left[\begin{array}{ll}{\sigma }_{x,i}\hfill & 0\hfill \\ 0\hfill & {\sigma }_{y,i}\hfill \end{array}\right]| {\sigma }_{x,i}\in {\Sigma }_X, {\sigma }_{y,i}\in {\Sigma }_Y\right\}}_{i=1,\cdots ,N_{MGMM}}$$

(26)

3.3. Optimization for the More Accurate MTM

The estimation results of the MTM with tomographic features presented in Section 3.1 may contain slight mismatches due to inevitable sensing errors. If the multi-robot system requires a more accurate MTM, the accuracy of the MTM can be improved by modeling the objective function and optimizing it. In this paper, we propose a new objective function based on tomographic information. Also, this paper presents how the Monte-Carlo optimization (MCO) algorithm can be applied to map merging with the proposed objective function. The overall optimization algorithm is summarized in

Table 1. Optimization to acquire the more accurate MTM (map transformation matrix).

Algorithm:	Monte-Carlo optimization for the more accurate MTM
Input:	Given individual maps: $M_1$ and $M_2$
	The mean set and variance set of the UGMM: ${{\Omega }_{\theta }}^{\prime }$ and ${\Sigma }_{\theta }$
	The mean vector set and covariance set of the MGMM: ${\Omega }_X$, ${\Omega }_Y$, and ${\Sigma }_{XY}$
Output:	The best MTM in $T_{best}$
1:	Determine the number of samples $N_{s,\theta }$ and $N_{s,xy}$, where $N_s=N_{s,\theta }{N}_{s,xy}$
2:	Initialize the objective function ${\rm Y}$
3:	Sample one-dimensional candidates ${\widehat{T}}_{\theta }$ for the rotation angle in Eq. (27)
4:	Sample two-dimensional candidates ${\widehat{T}}_{xy}$ for the translation amounts in Eq. (28)
5:	Obtain the set of three-dimensional combined candidates, $\Gamma$ by combining ${\widehat{T}}_{\theta }$ and ${\widehat{T}}_{xy}$
6:	Define the normal distribution with ${\mu }_{MND}$ and ${\Sigma }_{MND}$
7:	for$n$= 1 to$N_S$do
8:	Pick the $n$-the candiate ${\widehat{T}}_n$ from $\Gamma$
9:	Obtain ${\widehat{T}}_n{M}_2$ by transforming $M_2$ using ${\widehat{T}}_n$
10:	Calculate the $n$-th value of the objective function ${{\rm Y}}_n\left(M_1,{\widehat{T}}_n{M}_2,\right)$ in Eq. (29)
11:	end for
12:	Take$\widehat{T_n}$ indicating the maximum of the objective function in Eq. (30)
13:	return$T_{best}$

.

The sampling process in the MCO is divided into two categories such as the rotation angle sampling from the UGMM and the translation amounts sampling from the MGMM. One-dimensional candidates for the rotation angle and two-dimensional candidates for the translation amounts are respectively sampled from the UGMM and the MGMM as follows:

$${\widehat{T}}_{\theta }\sim {\mathcal{N}}_1\left({{\Omega }_{\theta }}^{\prime },{\Sigma }_{\theta }\right),$$

(27)

$${\widehat{T}}_{xy}\sim {\mathcal{N}}_2\left(\left[\begin{array}{l}{\Omega }_X\hfill \\ {\Omega }_Y\hfill \end{array}\right],\left[\begin{array}{ll}{\Sigma }_X\hfill & 0\hfill \\ 0\hfill & {\Sigma }_Y\hfill \end{array}\right]\right),$$

(28)

where the numbers of components in ${\widehat{T}}_{\theta }$ and ${\widehat{T}}_{xy}$ are respectively represented as $N_{s,\theta }$ and $N_{s,xy}$, which are user-defined parameters. Then, the set of three-dimensional combined candidates, $\Gamma =\left\{{\widehat{T}}_1,\cdots ,{\widehat{T}}_{N_s}\right\}$, are obtained by connecting each candidate in ${\widehat{T}}_{\theta }$ with each candidate in ${\widehat{T}}_{xy}$, where $N_s=N_{s,\theta }{N}_{s,xy}$.

The sampled configuration is evaluated by the objective function which indicates how the tomographically salient features in the individual maps are matched as follows:

$${\text{Υ}}_n\left(M_1,{\widehat{T}}_n{M}_2\right)={\displaystyle {\sum }_{k=1}^{K_d}\left[{\widehat{PS}}_{M_1}^{{\alpha }^{\prime }}\left(k\right)-{\widehat{PS}}_{{\widehat{T}}_n{M}_2}^{{\alpha }^{\prime }}\left(k\right)\right]},$$

(29)

where $K_d$ is the length of saliency check points. ${\widehat{PS}}_{M_1}^{{\alpha }^{\prime }}$ and ${\widehat{PS}}_{{\widehat{T}}_n{M}_2}^{{\alpha }^{\prime }}$ are respectively sorted forms in the descending order of $P{S}_{M_1}^{{\alpha }^{\prime }}$ and $P{S}_{{\widehat{T}}_n{M}_2}^{{\alpha }^{\prime }}$, where ${\alpha }^{\prime }$ is generally set to ${90}^{°}$ or ${180}^{°}$. After the evaluation of all sampled configurations is over, the best MTM for map merging is obtained by:

$$T_{best}=\underset{{\widehat{T}}_n\in \Gamma }{\mathrm{argmax}} {\mathsf{{\rm Y}}}_n\left(M_1,{\widehat{T}}_n{M}_2\right).$$

(30)

4. Evaluation Results

4.1. Simulation Results

To test and evaluate the performance of the proposed method, the individual maps were generated from the whole map produced by one of the public datasets [23], which is also used to describe the proposed method in the previous sections. The result of merging the individual maps using the proposed method is shown in Figure 6. The proposed method does not always outperform the other map merging methods. This is because the accuracy of indirect map merging without any initial correspondences methods depends on not only the size of overlapping areas but also the shape of overlapping areas. A map merging method works well in a certain case with overlapping areas, but it does not work well in other cases with the same amount of overlapping areas because its shape is hard for the method. In other words, even though there are enough overlapping areas, the method may not work well due to the shape of the overlapping areas. Therefore, the accuracy of map merging methods should be evaluated by average and standard deviation from the various cases of overlapping areas as many as possible. Therefore, for the quantitative evaluation of the proposed map merging method, one hundred different pairs of individual maps were randomly produced from the whole map and rearranged according to the average ratio of overlapping areas to individual maps as shown in Figure 7. The maximum average ratio and the minimum average ratio were 0.8532 and 0.1929, respectively. To evaluate the accuracy of a result MTM, $T_{result}$, obtained by the proposed method or other methods, a matching index $\mathsf{\Psi }$ was defined as follows:

$$\mathsf{\Psi }\left(M_1,T_{result}{M}_2\right)=\frac{{\sum }_{x=a_1}^{a_2}{\sum }_{y=b_1}^{b_2}{M}_1\left(x,y\right)\cdot \left[T_{result}{M}_2\left(x,y\right)\right]}{N_{overlap}}$$

(31)

where $N_{overlap}$ is the number of occupied grids in overlapping areas, and the sizes of $M_1$ and $M_2$ are commonly as $a_1\le x\le a_2$ and $b_1\le y\le b_2$.

The performance of the proposed method was compared with spectra-based map merging (SMM) [18] because it is a well-known grid map merging method with unknown initial correspondences. Also, the proposed method was compared with a local feature descriptor matching method such as SURF (speeded up robust features) [24], Harris corner detection (HCD) [25], and maximum eigenvalue-based corner detection (MEV) [26]. Also, the proposed method was compared with the intensity-based image registration method (REG) because it is similar to this work in the context that each individual map can be regarded as a binary image. The comparison results with the different pairs of individual maps are shown in Figure 8. In some cases, the accuracy of the proposed method was similar to others or lower than others, but the differences were not significant. The proposed method showed consistently good accuracy and averagely better performance than others. For the statistical comparison and analysis, the averages and standard deviations of the matching indices with the proposed method and other methods are summarized in

Table 2. Comparison of the averages and standard deviations of matching indices with different methods in simulations and experimental data.

Type		Proposed.	SMM	SURF	HCD	MEV	REG
Sim.	Avg.	0.9457	0.5970	0.7244	0.3209	0.3487	0.2351
	Std. Dev.	0.0334	0.4442	0.2553	0.4122	0.4140	0.3292
Exp.	Avg.	0.9393	0.6665	0.0543	0.6507	0.0596	0.0756
	Std. Dev.	0.0552	0.1557	0.0483	0.2668	0.0658	0.0232

, which indicates that the proposed method can work consistently better than the other methods.

4.2. Experimental Results

The performance of the proposed method was also tested by individual maps produced by a real multi-robot system in indoor environments. The multi-robot system was composed of three mobile robots with laser scan sensors and a wireless router. The robot was Pioneer3-DX, and the laser scan sensor was Hokuyo UTM-30LX. The indoor environment was the third floor of the Automation and Systems Research Institute at Seoul National University. Its size is about 48 m × 17 m. Two individual maps were produced by the multiple robots using our multi-robot SLAM framework as shown in Figure 9, and the sinograms of them are shown in Figure 10. The individual maps were successfully merged by the proposed method as shown in Figure 11.

To evaluate the accuracy of the proposed algorithm according to the ratio of overlapping areas to individual maps, the matching indices should be measured with various cases in the experimental environment. Thus, the translation amounts and the rotation angles were randomly selected and combined into various pairs of individual maps. Then, they were rearranged according to the average ratio of overlapping areas to individual maps, which are represented by the blue dotted line as shown in Figure 12. Differently from the simulation results, the accuracy of map merging was significantly affected by the number of overlapping areas due to the errors in individual maps. Therefore, the matching indices were analyzed according to the average ratio of overlapping areas to individual maps, which are represented by the red solid line as shown in Figure 12. In the microscopic context, it was difficult to find any patterns for the matching index between the individual maps since there were many abrupt changing points in the matching index. But, in the macroscopic context, the matching index gradually decreases according to the average ratio. Even though the matching index decreased as the average ratio decreased, the proposed method showed relatively high matching indices for all the cases of individual maps.

For more comparative analysis, the matching indices of the proposed method with the different pairs of individual maps were compared with those of the SMM, SURF, HCD, MEV, and REG, which are shown in Figure 13. For the statistical comparison and analysis, the averages and standard deviations of the matching indices with the proposed method and other methods are summarized in Table 2. The SMM showed relatively high matching indices, but its accuracy abruptly decreased for a low average ratio of overlapping areas. The HCD also showed relatively high matching indices, which was much different from the simulation results. However, its abrupt decrease in the accuracy of map merging was larger than the SMM, and its average matching index was also smaller than not only the proposed method but also the SMM. The SURF showed low matching indices even though it showed relatively high matching indices for map merging in simulations. The MEV and REG did not show acceptable performance. These varying results indicate that the accuracy of the descriptor-based map matching methods highly depends on the shape and quality of individual maps. In other words, although a descriptor-based map matching method was good for map merging in a certain environment, its accuracy could not be guaranteed for map merging in different environments. Consequently, the matching indices of the proposed method were higher than those of other methods, which verifies that the proposed method is more accurate than the others.

The computational times were quantitatively compared using averages and standard deviations from as shown in

Table 3. Comparison of the averages and standard deviations of the computational times with different methods in simulations and experimental data (unit: sec).

Type		Proposed.	Proposed. w/o MCO	SMM	SURF	HCD	MEV	REG
Sim.	Avg.	7.6219	3.3047	3.1527	5.3756	4.7475	4.8870	17.3969
	Std. Dev.	1.1282	0.3013	0.4235	0.3948	0.4463	0.3995	1.2543
Exp.	Avg.	0.3766	0.1781	0.1531	0.2198	0.4297	0.3017	0.6509
	Std. Dev.	0.0432	0.0223	0.1263	0.0480	0.0907	0.0613	0.1208

. Since the sizes of individual maps in simulations were larger than those in experiments, the computational times in simulations were overall smaller than those in experiments. The average of the computational times of the proposed method was larger than the others except for REG. This is because the Monte-Carlo optimization included in the proposed method to improve the accuracy of map merging requires much computational time due to its iterative property. Note that the computational times decreased by about 56.64% when Monte-Carlo optimization is excluded from the proposed method, which was similar to the others as shown in Table 3. If the total computational time of the proposed method is larger than the update period of the collective map of the multi-robot system, one can consider acceleration methods based on graphics processing unit (GPU) [27] or field-programmable gate array (FPGA) [20], which will be researched more thoroughly in my future work.

5. Conclusions

The grid map merging without any information on initial robot poses or inter-robot measurements is one of the challenging problems in multi-robot systems. Especially, if the overlapping area between maps is insufficient, it is too difficult to accurately merge the given individual maps. This paper proposed a new map merging method using tomographic features, which can be conducted well with relatively small overlapping area. The evaluation results showed that the proposed method can successfully merge the given individual maps with relatively small overlapping areas, and its accuracy was higher than other map merging methods.