Accurate Mobile Urban Mapping via Digital Map-Based SLAM
 ^†

Abstract

This paper presents accurate urban map generation using digital map-based Simultaneous Localization and Mapping (SLAM). Throughout this work, our main objective is generating a 3D and lane map aiming for sub-meter accuracy. In conventional mapping approaches, achieving extremely high accuracy was performed by either (i) exploiting costly airborne sensors or (ii) surveying with a static mapping system in a stationary platform. Mobile scanning systems recently have gathered popularity but are mostly limited by the availability of the Global Positioning System (GPS). We focus on the fact that the availability of GPS and urban structures are both sporadic but complementary. By modeling both GPS and digital map data as measurements and integrating them with other sensor measurements, we leverage SLAM for an accurate mobile mapping system. Our proposed algorithm generates an efficient graph SLAM and achieves a framework running in real-time and targeting sub-meter accuracy with a mobile platform. Integrated with the SLAM framework, we implement a motion-adaptive model for the Inverse Perspective Mapping (IPM). Using motion estimation derived from SLAM, the experimental results show that the proposed approaches provide stable bird’s-eye view images, even with significant motion during the drive. Our real-time map generation framework is validated via a long-distance urban test and evaluated at randomly sampled points using Real-Time Kinematic (RTK)-GPS.

1. Introduction

The recent development of autonomous vehicles encompasses many key issues in robotics problems, including perception, planning, control, localization and mapping. Among these problems, this paper focuses on solutions for an accurate urban map generation, specifically targeting 3D and lane maps for autonomous cars. Having an accurate map is an important issue for self-driving cars [1,2,3] and many other areas such as virtual reality [4], visualization [5], recognition [6], localization and navigation [7,8].

In conventional mapping approaches, aerial sensing has been heavily investigated. In these approaches, the fusion of aerial images with aerial Light Detection and Ranging (LiDAR) and/or radar is usually applied for an accurate digital map at the cm-level, including urban structures. This line of studies focuses on undistorting building shapes from aerial sensors, which is mostly too costly (ranging from $0.5 M to $1.4 M). To obtain cost-effective solutions for mapping, recent advances have appeared in mobile mapping systems. Using a car-like system with sensors mounted, 3D maps are often generated from a set of point clouds, using LiDAR sensors to actively acquire accurate and dense 3D point clouds of building façades or surfaces of objects. Mobile scanning is widely used for modeling in architecture [4] and agriculture [9], as well as for urban and regional planning [10]. Many studies combine a highly accurate GPS, a high precision Inertial Measurement Unit (IMU) and the wheel/visual odometry of the vehicle to compute fully timestamped trajectories. The absence of the GPS in constricted spaces is an exceptionally limiting factor for mobile scanning. It is more critical, in a complex urban environment, and thus it is difficult to calculate accurate positions using a GPS sensor due to blackouts or multipath problems. Generally, using a variety of sensors has became a popular strategy to complement the drawbacks of a single sensor.

We propose a SLAM-based approach to localize the mobile sensor system mounted on a car-like platform and generate an accurate urban map consisting of a large amount of 3D points. For accurate localization, we use publicly available digital maps together with cameras, Light Detection and Ranging (LiDAR)s, and inertial sensors. Using this accurate localization, motion compensated Inverse Perspective Mapping (IPM) is applied for accurate lane-map generation. Given accurate localization, projections of front-looking cameras follow the construction of the lane map. For the lane map description, we propose an adaptive IPM algorithm to obtain accurate bird’s-eye view images from the sequential images of forward-looking cameras. These images are often distorted by the motion of the vehicle; even a small motion can cause a substantial effect on bird’s-eye view images.

To use a digital map in Simultaneous Localization and Mapping (SLAM), we incorporate a shape file to extract structural and elevation information. Although digital maps with 1:1000 and 1:5000 scales offer sub-meter global average accuracy, only a subset of map data (e.g., buildings and road boundaries which are globally static) are available for vehicle localization. In this paper, we introduce a measurement model for both GPS and structural information, and leverage the digital map data as observations to actively fuse digital maps in SLAM when GPS signals are sporadic.

The contribution of the proposed method can be summarized as follows.

• Digital map-based SLAM: Digital map information was incorporated with a building model, road network and elevation model. This paper introduces a measurement model to add global measurements for full 3D SLAM. Instead of addressing this problem as localization to a prior map, our modeling handles the structural information in the digital map as sporadic spatial measurements.
• Development of online mobile mapping system with sub-meter accuracy: Typical LiDAR-based mapping system known to be slow, suffering from association problems. Despite a large number of nodes and a long trajectory, the entire method runs in real-time using only 40% of the total mission time. Not only the computational time, our algorithm provides memory-efficient implementation. Wall information obtained from point cloud turns into an urban signature, which then can effectively be used in the matching phase. Fully integrated single step implementation allows us to build a 3D map over 9 km while driving.
• Thorough analysis for resulting 3D and lane maps accuracy: We perform a thorough analysis on the resulting maps to evaluate the effects of digital maps. The addition of elevation and buildings from digital maps enables accurate 3D urban mapping. We performed quantitative evaluation on random sample points. Using RTK-GPS with 10-mm-accuracy, we analyzed the accuracy of the points from buildings and lane maps.

2. Related Works

Conventional approaches in digital map generation rely on aerial sensors, including high-resolution aerial images, Light Detection and Ranging (LiDAR)s, and radars. Toward accurate building detection, point cloud data from aerial LiDAR has been widely used for a Digital Building Model (DBM) [11,12,13,14]. One line of study used only aerial imagery to extract building information [15,16], while merging the high-resolution aerial imagery and aerial LiDAR [17] has been introduced.

The aforementioned approaches use costly aerial sensors and have limitations in generating maps inside tunnels and multi-layered roads. For a cost-effective mobile mapping system, urban mapping systems similar to ours have been introduced in the literature [18,19,20,21]. Blanco et al. [18] presented a collection of outdoor datasets using a large and heterogeneous set of sensors comprising color cameras, several laser scanners, precise GPSs, and an IMU. Their posterior research [19] introduced a dataset gathered entirely in urban scenarios using a car equipped with one stereo camera and five laser scanners, among other sensors. Elseberg et al. [20] used a commercial data logging system and collected data with an experimental platform constructed by the RIEGL scanner [22]. Bok et al. [21] used a mobile mapping sensor system mounted on a ground vehicle. A vertical 2D LiDAR scans structures and the reconstruction is done by accumulating scanned data. All of the conventional approaches cover a small region, assuming a 2D environment without taking altitude information into account.

For these mobile mapping systems, cameras are the most popular sensors in detecting lanes. For visual road-mark detection, IPM is often used in the vision-based perception of roads and lanes. IPM produces bird’s eye-view images that remove the perspective effect by using information about camera parameters and the relationship between the camera and the ground. The results of IPM can provide post-processing algorithms with more efficient information, such as lane perception, mapping, localization, and pattern recognition. Many researchers have studied IPM in many applications, such as distance detection [23], production of bird’s eye-view images of a spacious area using a mosaic method [24], provision of appropriate bird’s eye-view images for parking assistance [25], and lane-level map generation [26].

The challenge for mobile mapping systems is to achieve a consistent map. Researchers investigated the SLAM algorithm for accurate vehicle localization [27] and 3D mapping [28] using perceptual sensors mounted on a vehicle. Using a digital map has also been recently investigated by many researchers [29,30,31]. Schindler et al. [29] first proposed lane map generation and representation methods using RTK GPS-based localization and showed high-precision digital map-based self-localization in [30]. They showed an efficient Monte Carlo localization approach using a map model based on smooth arc splines. Floros et al. [31] proposed an approach for global vehicle pose estimation that combines visual odometry with map information from OpenStreetMaps [32] to provide accurate estimates of the vehicle’s pose. Similarly, Pink and Stiller [33] described landmark (lane-based) generation from aerial images with image classification geometric representation. This method used aerial images for prior maps and focused on the way to match orthographic lane images to a global lane map from aerial images. Guo et al. [26] presented a system for lane-level map generation from local IPM images and the global OpenStreetMap (OSM) database. They locally constructed an orthographic lane map, matched it with a segmented aerial map, and finally generated a global lane graph of real-world roads.

As an accurate localization aspect, some previous works [34,35,36] used a lane map as a local vehicle position estimation by matching pre-built local lane maps from IPM images and current incoming raw images. Napier and Newman [34] proposed a vehicle localization method based on synthesized local orthographic lane maps. They generated local orthographic images from a first run and localized the vehicle by Mutual Information (MI) between live-stream images and synthesized images. Schreiber et al. [35] constructed prior lane-based maps with extended sensors such as GPS/INS and matched road markings and curbs for lane-level localization. Rose et al. [36] used lane maps for lateral position estimations of a vehicle. These studies only applied localization or pose estimation algorithms; no cases have generated 3D maps using the SLAM algorithm with a digital map. In this paper, we are interested in using publicly available digital maps in a SLAM framework to leverage for accurate lane map and 3D map generation.

A similar approach to ours was reported in [37] which incorporated bundle adjustment using monocular vision. Although extracting buildings from digital maps and using them as constraints were introduced and conceptually similar to ours, their implementation relied mostly on vision, covering a relatively small urban area. Unlike their approach, proposed algorithm focuses on using LiDAR accomplishing real-time performance with a single-step mapping algorithm that can build an urban map while driving.

3. System Overview

For accurate urban mapping, we developed a sensor suite called Urban Mapping System (UMS), which is mountable to a car-like platform. As shown in Figure 1a, the platform used for mapping was equipped with three Light Detection and Ranging (LiDAR) sensors, four cameras, a Global Positioning System (GPS), an Inertial Measurement Unit (IMU), an altimeter, and wheel encoders. The detailed specifications are summarized in

Table 1. Specifications of Urban Mapping System (UMS).

Item	Specification
Dimensions	1.67 m × 1.36 m × 0.31 m (L × W × H )
Dry weight	35.8 kg
LiDAR	SICK LMS291,200 (35 Hz)
Imaging sensor	Point Grey Flea3, 1380 × 1024 pixel, 12-bit CCD (30 Hz)
GPS	HUACE B20 (1 Hz)
IMU sensor	Xsens MTi (100 Hz)
Altimeter	WITHROBOT myPressure (1 Hz)
Wheel encoder	Autonics E68S, rotary encoder type (100 Hz)
Processor	Intel(R) Core(TM) i7-3790 [email protected] GHz
Battery	Delkor 80 Ah, 12 V , lead–acid type

. As in Figure 1, coordinates of each sensor were represented with respect to the vehicle center coordinates (red, green and blue arrows).

Using a geometrical relationship between the vehicle center and LiDAR, the position of the camera is computed using extrinsic calibration results of LiDAR and cameras. For more details, refer to [38] the system configuration.

In the proposed configuration, LiDAR sensors are facing each side of the vehicle, while the sweeping direction is orthogonal to the ground. This configuration allows us to capture surrounding data line-by-line. Motion induces accumulation of lines, and a 3D point cloud is obtained by stacking a sequence of point cloud lines. Researchers using similar configurations recommended a vehicle speed of approximately 15–30 km/h [39], which we also adopted in the experiments.

The objective of the proposed platform is to build an accurate 3D and lane map given various sensors and digital maps. The process of generating urban lane maps with 3D point clouds involves four main stepsas shown in Figure 2: (1) sensor data logging; (2) generating a node and the factor; (3) optimizing using an Incremental Smoothing and Mapping (iSAM) SLAM back-end; and (4) generating the final lane map based on the optimized trajectory from the SLAM results and IPM images. Measurements from odometry, digital map information, elevation, and GPS are all used.

4. Pose-Graph Simultaneous Localization and Mapping (SLAM)

4.1. State Definition

We estimate the vehicle’s full 6-degree of freedom (DOF) pose, $\mathbf{x}={[x,y,z,\varphi ,\theta ,\psi ]}^{\top }$. The augmented state representation is expressed as follows for n keyframes,

$$X={\left[\begin{array}{c}{\mathbf{x}}_1^{\top },\cdots ,{\mathbf{x}}_i^{\top },\cdots ,{\mathbf{x}}_n^{\top }\end{array}\right]}^{\top }$$

(1)

while each of the pose samples (${\mathbf{x}}_i$) correspond to the time instance $t_i$ of a keyframe. For the SLAM back-end, we use iSAM [40] to find an optimized solution of the trajectory, from which we build the urban map. Factors from each sensor measurement are collected as in Figure 3. The attitude measurements and altitude are given as absolute factors per node, with all nodes sequentially connected by odometry measurements. Loop-closing factors are mainly generated by walls that we generate from LiDAR when we add building measurement from digital maps. GPS availability is quite sporadic in urban area, and measurements are intermittently applied to a node using Dynamic Covariance Scaling (DCS).

4.2. Odometry Modeling

The system has two rotary encoders, as shown in Figure 1a, which are mounted to record wheel revolution counts and the resulting rotation angles ($C_l$ and $C_r$) of each wheel. We define vehicle pose in 2D space, $\mathrm{x}={[x,y,\theta ]}^{\top }$ for odometry measurements. We calculate the relative pose difference as a function of left/right wheel diameter ($D_l,D_r$), left and right wheel rotation angle ($C_l$ and $C_r$), wheel diameter ($D_l,D_r$), and wheel base ($W_b$).

$$\begin{array}{ccc}\hfill {\mathrm{z}}_{odo}& =& \left[\begin{array}{c}\Delta x\\ \Delta y\\ \Delta \theta \end{array}\right]=\left[\begin{array}{c}{L}_{avg}\cos (\Delta \theta )\\ L_{avg}\sin (\Delta \theta )\\ L_{diff}{W}_b^{-1}\end{array}\right]\hfill \end{array}$$

(2)

Here, $L_{avg}$ is the average and $L_{diff}$ is the difference of two travel distances from the left and right wheel.

4.3. Altitude and IMU Modeling

We also provide z-directional measurements from the altitude sensor. When mapping hilly terrain, this leverages the localization performance by providing both absolute and relative measurements on height. We used theWITHROBOT MyPressure (WITHROBOT, Seoul, Korea) altitude sensor [41], which has accuracy within 1 cm. Attitude information was obtained via IMU. A MTI Xsens [42] was mounted into the system to provide roll, pitch, and yaw in 100 Hz. Altitude and attitude information is used as absolute measurement factors and fed into the SLAM back-end.

The proposed implementation produces loop-closure on height term based on wall information Section 5.3. When a loop-closure happens in wall-to-wall matching (i.e., when a wall is revisited), we apply a relative constraint on two associated nodes so as for them to be the same height.

4.4. GPS Modeling

GPS becomes extremely unreliable in highly complex urban environments. Signals are lost or deteriorate in urban canyons. Despite these limitations, GPS still provides valuable information to ground vehicles. For our GPS measurement error modeling, we mainly used the GPS Pseudorange Noise Statistics (GPGST), which include the standard deviation of longitude/latitude errors. The error measurement is the most sensitive, depending on the complexity of the environment, showing high error values in complex urban area.

Outlier handling in SLAM determines the robustness of the entire framework, since a single wrong measurement may critically deteriorate the results. To tackle this issue, Sünderhauf and Protzel [43] introduced Switchable Constraints (SC) and switching variables ($S_{ij}\in [0,1]$) for the robust SLAM back-end. This research area was further developed by Agarwal et al. [44], who solve for a closed form solution to determine these switching variables $S_{ij}=\min .\left(1,\frac{2\mathrm{\Phi }}{\mathrm{\Phi }+{\chi }_{ij}^2}\right)$ As can be seen, the choice depends on ${\chi }_{ij}^2$, which represents the error for each loop closing constraint. By considering the induced error by a factor, the associated covariance is dynamically scaled. This substantially improves the results when a certain measurement may be unreliable during the mission, such as GPS. In this implementation, we adopted DCS to cope with intermittent and unreliable GPS signals.

5. Digital Map-Based SLAM

We introduce using a digital map with Light Detection and Ranging (LiDAR) sensor to correct the navigation error even under unreliable Global Positioning System (GPS). We first create walls from a 3D point cloud generated by LiDAR. Using this point cloud as a prior reference, the building edges are automatically detected to correct navigation error. In this section, we introduce the generation of walls, matching with a digital map, and matching between walls for loop-closure.

5.1. Digital Map

We leverage publicly available digital maps for SLAM application. Specifically, we focus on vectorized digital maps widely used in a geospatial information system. Studies that automatically generate the digital map have been continued in recent years [11,12,13,14,15,16,17,45,46,47,48] but it has mostly relied on aerial sensors. For the digital map data format, we chose to use a shapefile format developed by ESRI [49]. The shapefile stores non-topological geometry and attribute information for the spatial features as a set of vector coordinates. Vector features including points, lines, and polygons are described in the shapefile format. The format also represents area features via closed loop and double-digitized polygons. Each attribute record has a one-to-one relationship with the associated shape record. Among many types of records, we focus on polygons that contain information about structural buildings.

As illustrated in Figure 4a, a polygon consists of one or points that form a closed, non-self-intersecting loop. A polygon may contain more closed curves. A closed curve is a connected sequence of four or more multiple outer closed curves. The order of vertices or orientation for a closed curve indicates which side of the closed curve is the interior of the polygon. Recording contents of this polygon data type are shown in

Table 2. Description on polygon record contents. The fields for a polygon type are box, numParts, numPoints, parts, and points. Box means the bounding box for the polygon stored in the order Xmin, Ymin, Xmax, Ymax. The number of closed curves in the polygon is described by NumParts. NumPoints is the total number of points for all closed curves. parts means an array of length numParts. For each closed curve, the index of its first point stored in the points array. Points are array of length numPoints. The points for each closed curve in the polygon are stored end to end.

Position	Field	Value	Type	Number	Byte Order
Byte 0	Shape Type	5	Integer	1	Little
Byte 4	Box	Box	Double	4	Little
Byte 36	NumParts	NumParts	Integer	1	Little
Byte 40	NumPoints	NumPoints	Integer	1	Little
Byte 44	Parts	Parts	Integer	NumParts	Little
Byte X†	Points	Points	Point	NumPoints	Little
†X = 44 + 4 × NumParts

. Figure 5b presents an example of a building extracted from polygon data in a shapefile. In this paper, shapefiles from National Geographic Informaion Institue (NGII) are used for building extraction.

We also incorporate DEM for elevation information. For certain topographies, regions contain substantial elevation change. In general, the height of the terrain is represented by the contours. As presented in Figure 4d, points are connected by contour lines indicating the same height. The absolute height value of the contour is described in the attribute information of the shape-file. When the trajectory of the vehicle passes a contour, SLAM algorithm optimize position using absolute measurement of Z-direction from this absolute height.

5.2. Wall Segmentation

Given building information parsed from the digital maps, we now extract walls from the mobile sensor system. Building façades and other structures’ surfaces are major features of the urban area. As digital maps are widely available these days, we propose a matching algorithm that uses a digital map and an accumulated local point cloud. In general, use of the point-to-point Iterated Closest Point (ICP) algorithm is popular but it is time-consuming and gives rise to issues when point clouds are not sufficiently dense.

We define a point cloud $P=\{p_1,p_2,p_3,...\}$ as consisting of LiDAR points, while each point indicates a 3D position $p_k=[x_k,y_k,z_k]$ in the global coordinate frame. From the LiDAR 3D point cloud in the sensor frame, ($P_s$) is obtained by converting each beam ranging from $i=0$ to 360. The angular resolution of SICK LMS291 is $N_{res}=0.5^{\circ }$, covering a ${180}^{\circ }$ field of view for each size. Sensor frame point cloud $P_s$ is then converted to vehicle frame point cloud $P_v$ using a coordinate transformation.

After the transformation, we obtain a vertical line of the point cloud as described in Figure 5a. As the vehicle proceeds, these lines are accumulated and form a local point cloud set. Given a single scan line of LiDAR and odometry information from the wheel encoder and IMU, the set of the line forms a local point cloud-based on the estimated pose. During the local point cloud forming, we evaluate the wall criteria to see if the set can be segmented to a wall. When a wall is registered, the algorithm creates a node in the SLAM graph as a keyframe. Dropping the node clears the accumulated local point cloud and starts a new accumulation of incoming scan lines.

We propose a fast wall segmentation algorithm-based on RANSAC when segmenting a wall from a point cloud (summarized in Figure 6). For the incoming line of the point cloud, we fit a plane using a 5-point RANSAC. Our wall initialization criteria involve checking the wall width (initialization criteria in Figure 6). If the width of the created wall reaches our threshold (minimum 3 m), the algorithm initializes a wall and stores normal vector and error statistics for the wall. Once the initial wall is established, a newly incoming scan line is evaluated to be merged to the initial wall (merging criteria in Figure 6).

5.3. Wall-to-wall Loop Closing

For loop-closure, we use building wall patches by introducing a wall-to-wall measurement model. This method provides an explicit relationship between the walls in the factor graph and produces efficient maps using a data-rich 3D point cloud.

We parameterize the wall ${\mathbf{ß}}_i={[n_i^x,n_i^y,n_i^z,p_i^x,p_i^y,p_i^z]}^{\top }$ using normal vector ${\mathbf{n}}_i={[n_i^x,n_i^y,n_i^z]}^{\top }$ and the center point of the wall ${\mathbf{p}}_i={[p_i^x,p_i^y,p_i^z]}^{\top }$. Following wall coordinate transform operators $⊞$ and $\boxminus$ introduced by Paul Ozog [50] we derive our wall-to-wall measurement model. The $⊞$ and $\boxminus$ operations change the base coordinate frame that a plane is described in, which can be written as ${\mathbf{ß}}_{jk}={\mathbf{x}}_{ij}\boxminus {\mathbf{ß}}_{ik}$ and ${\mathbf{x}}_{ij}⊞{\mathbf{ß}}_{jk}=\ominus {\mathbf{x}}_{ij}\boxminus {\mathbf{ß}}_{jk}$.

For two walls to be recognized as the same wall, they need to have the same normal vectors and the distance between a point in one plane to another plane should be zero. We compute them by evaluating the dot-product of two normals ($e_n$) and the distance from a center point on one wall to another wall ($e_d$).

$${\mathrm{z}}_{wall-wall}={\left[\begin{array}{c}{e}_n,e_d\end{array}\right]}^{\top }$$

(3)

Let a wall k seen at frame i to be ${\mathbf{ß}}_{ik}$ and the same wall revisited at frame j to be ${\mathbf{ß}}_{jk}$. To compare normal vectors, ${\mathbf{ß}}_{ik}$ needs to be transformed to frame j using $\boxminus$ operation. For simple notation we denote this as ${\mathbf{ß}}_{jk}^{\prime }=x_{ij}\boxminus {\pi }_{ik}$ with associated normal ${\mathbf{n}}_{jk}^{\prime }$ and the center point ${\mathbf{p}}_{jk}^{\prime }$. Then the error between two normal vectors can be written as $e_n=1-{({\mathbf{n}}_{jk}^{\prime })}^{\top }{\mathbf{n}}_{jk}$. Similarly, the plane to point distance ($e_d$) is computed by measuring distance from ${\mathbf{ß}}_{jk}^{\prime }$ to a wall center point in j frame (${\mathbf{p}}_{jk}$).

It should be noted that we intentionally excluded the center point’s coincident constraints. This is because our point cloud density strongly depends on the vehicle’s motion as we accumulate line scans. When a vehicle takes turns the density may increase or decrease, and the computed center point may be varying with respect to the accumulated point cloud density.

5.4. Wall-Based Digital Map Localization

With odometry having a moderate accuracy, structural information from buildings can be used as a feature to correct the accumulated navigation error. However, buildings usually deteriorate the GPS signal reception. For a complex urban area, therefore, GPS and buildings can be exploited complementary. The digital map contains a variety of local information and there are vectorized building data among them.

An extracted wall from a point cloud represented by ${\mathbf{ß}}_i$ consists of normal vector ${\mathbf{n}}_i={[n_i^x,n_i^y,n_i^z]}^{\top }$ and the center point of the wall ${\mathbf{p}}_i={[p_i^x,p_i^y,p_i^z]}^{\top }$.

Similarly, we can calculate wall information that consists of normal vector and a center position from digital map ${\mathbf{d}}_i$ in global coordinates. Using $⊞$ operation and current node pose, we write extracted wall information from point cloud as ${\mathbf{ß}}_{ik}={\mathbf{x}}_{ij}⊞{\mathbf{ß}}_{jk}$.

We obtain the measurement which consists of the dot-product of two normals ($e_n$) and the distance from a center point on one wall to another wall ($e_d$) in the same manner as in the Section 5.3.

$${\mathrm{z}}_{wall-digitalmap}={\left[\begin{array}{c}{e}_n,e_d\end{array}\right]}^{\top }$$

(4)

5.5. Elevation-based Full 3D Mapping

For full 3D mapping, we use DEM that consists of contour lines by introducing a node-to-contour measurement model. When the vehicle traverses a contour, the algorithm detects an intersection between vehicle motion vector and a line segment from the contour.

Let the vehicle pose be $\mathbf{x}={[x,y,z,\varphi ,\theta ,\psi ]}^{\top }$. The DEM contour line, $\mathbf{c}$, consists of vertices ${\mathbf{c}}_i={[c_i^x,c_i^y,c_i^z]}^{\top }$. Then we can extract two line segments, vehicle motion induced line (${\mathbf{l}}_{\mathbf{x}}=[x_{i-1},y_{i-1},z_{i-1},x_i,y_i,z_i]$) that connects vehicle current node and previous node and contour line segment (${\mathbf{l}}_{\mathbf{c}}=[c_{k-1}^x,c_{k-1}^y,c_{k-1}^z,c_k^x,c_k^y,c_k^z]$) from a nearby contour. Using these two segments, we calculate the intersection point of these two line segments to determine the case when vehicle traverses contour line. Given vehicle pose that meets the DEM, our proposed method produces absolute measurement, which is described in the attribute information of the shape-file (Figure 7c).

6. Motion-Compensated Adaptive Lane Map Generation

Once we achieved an accurate localization by using a digital map-based SLAM, we apply IPM to back project road images onto the SLAM-induced map. This section briefly presents the basic IPM model [51] by using the physical parameters of a camera before illustrating an adaptive IPM model. IPM is a mathematical technique that relates to coordinate systems with different perspectives. More detailed derivation and explanation can be found in [52]. For lane map generation, we relate undisturbed bird’s-eye view images and forward-facing distorted images (Figure 8).

6.1. Basic Inverse Perspective Mapping (IPM) Model

The objective is to map pixel points $(u,v)$ to the world coordinate points $(X,Y,Z)$. The notation $\left(\widehat{·}\right)$ indicates a vectored version of the variables. We first define a unit vector $\widehat{X}$ to set the camera’s viewing direction. Being orthogonal to $\widehat{X}$, we define another unit vector, $\widehat{Y}$, that is orthogonal to the camera-viewing direction on the ground. The IPM is to find the relation between the world coordinate ($\widehat{X},\widehat{Y},\widehat{Z}$) and image coordinate ($\widehat{u},\widehat{v}$) in order to map image pixels to the world coordinate points. Note that two types of coordinates on an image are set as ($\widehat{u},\widehat{v}$) and ($\widehat{r}$, $\widehat{c}$), depending on the unit. The relation between an image point in a pixel space ($\widehat{u}$, $\widehat{v}$) and the same point in a meter space ($\widehat{r}$, $\widehat{c}$) is defined as follows

$$\begin{array}{ccc}\hfill \mathbf{u}\left(\mathbf{c}\right)=\frac{n+1}{2}+\mathbf{K}\mathbf{c}& ⟷& \mathbf{c}\left(\mathbf{u}\right)=\frac{1}{\mathbf{K}}(\mathbf{u}-\frac{n+1}{2})\hfill \end{array}$$

(5)

$$\begin{array}{ccc}\hfill \mathbf{v}\left(\mathbf{r}\right)=\frac{m+1}{2}-\mathbf{K}\mathbf{r}& ⟷& \mathbf{r}\left(\mathbf{v}\right)=\frac{1}{\mathbf{K}}(\frac{m+1}{2}-\mathbf{v})\hfill \end{array}$$

(6)

with a scale factor between pixel and meter (px / m) $\mathbf{K}$, the image width $\mathbf{m}$, and the images height $\mathbf{n}$. The location of the camera’s optical center ($\mathbf{P}$) is $[0,0,h]$ in the world coordinate system. The unit vector of the optical axis $\widehat{o}$ is orthogonal to the image plane.

Using the top and side view in Figure 9 we can derive Equations (7) and (8) for a fixed camera case.

$$\begin{array}{ccc}\hfill \mathbf{X}\left(\mathbf{v}\right)& =& h\frac{\tan \left({\theta }_0\right)\left(1-2\frac{\mathbf{v}-\mathbf{1}}{\mathbf{m}-\mathbf{1}}\right)\tan \left({\alpha }_r\right)-1}{\tan \left({\theta }_0\right)+\left(1-2\frac{\mathbf{v}-1}{m-1}\right)\tan \left({\alpha }_r\right)}\hfill \end{array}$$

(7)

$$\begin{array}{ccc}\hfill \mathbf{Y}(\mathbf{u},\mathbf{v})& =& \left(1-2\frac{\mathbf{u}-1}{n-1}\right)\tan \left({\alpha }_c\right)\mathbf{X}\left(\mathbf{v}\right)\hfill \end{array}$$

(8)

The location of $\mathbf{Y}(\mathbf{u},\mathbf{v})$ in the world coordinate is dependent on $(\mathbf{u},\mathbf{v})$ because $\mathbf{Y}(\mathbf{u},\mathbf{v})$ includes $\mathbf{X}\left(\mathbf{v}\right)$.

6.2. Adaptive IPM Model

When images are obtained from a moving vehicle, it is difficult for them to be transformed to the accurate bird’s-eye view images because of the motion of the vehicle, especially its pitch direction. To resolve this problem, oscillatory pitch angle (${\theta }_p$) of the camera is also added to original pitch angle (${\theta }_0$) in this model. This oscillatory pitch angle can be obtained either from sensors or via visual odometry. In this work, we exploited IMU sensor for oscillatory pitch angle.

$$\begin{array}{ccc}\hfill \mathbf{X}(\mathbf{v},{\theta }_p)& =& \mathbf{h}\frac{\tan ({\theta }_0+{\theta }_p)\left(1-2\frac{\mathbf{v}-1}{m-1}\right)\tan \left({\alpha }_r\right)-1}{\tan ({\theta }_0+{\theta }_p)+\left(1-2\frac{\mathbf{v}-1}{m-1}\right)\tan \left({\alpha }_r\right)}\hfill \end{array}$$

(9)

$$\begin{array}{ccc}\hfill \mathbf{Y}(\mathbf{u},\mathbf{v},{\theta }_p)& =& \left(1-2\frac{\mathbf{u}-1}{n-1}\right)\tan \left({\alpha }_c\right)\mathbf{X}(\mathbf{v},{\theta }_p)\hfill \end{array}$$

(10)

Finally, the adaptive Inverse Perspective Mapping (IPM) modeling Equations (9) and (10) can be derived by adding ${\theta }_p$. $\mathbf{X}(v,{\theta }_p)$ is dependent on the pitch angle of the camera (${\theta }_p$) and $\mathbf{Y}(u,v,{\theta }_p)$ is also dependent on it, which means that bird’s-eye view images are properly compensated against the pitch angle.

6.3. Consistent Lane Map Processing

As described in Section 5, the algorithm drops nodes based upon the wall extraction events. Depending on the circumstances, structure occurrence may be sparse and thus produce a large gap between nodes. When the distance between two consecutive nodes are usually a couple of meters, inter-node inconsistency for measurements becomes substantial. This is because the SLAM algorithm only corrects navigation error for the nodes. Since only nodes are optimized through SLAM update, uncorrected measurements needs post-processing for self-consistent maps. This discrepancy statistics is shown in

Table 3. Discrepancy prior to the compensation (per unit meter).

	Min	Max	Average
x (m)	$7.98\times {10}^{-4}$	$9.37\times {10}^{-2}$	$1.99\times {10}^{-2}$
y (m)	$3.64\times {10}^{-4}$	$1.49\times {10}^0$	$4.13\times {10}^{-2}$
yaw (${}^{\circ }$)	$2.08\times {10}^{-3}$	$2.47\times {10}^1$	$1.58\times {10}^0$

that describes pose error prior to compensation. The error is computed as the distance and angle between the node position that is generated by the proposed method and is the expected pose by using the vehicle encoder. The cause of this discrepancy is encoder error from the wheel slip and modeling error, and this large discrepancy clearly indicates the need for the compensation. Smooth vehicle poses corrected with SLAM data are obtained after this compensation process.

The effect of compensation is presented in Figure 10. In our algorithm, valid SLAM nodes are sparse, and thus are not compact enough to create a lane map via IPM. Interpolation is required for a dense map by updating the navigation correction between the nodes using vehicle encoder data. Figure 10a shows interpolated positions between nodes without any compensation. There are lots of errors between expected positions using encoder data and nodes because all nodes are corrected when wall extraction events occur. To overcome this problem, we apply a compensation rule, as described below.

$$\begin{array}{ccc}\hfill E_k& =& \frac{{\mathbf{p}}_{\mathbf{e}}-{\mathbf{p}}_{\mathbf{n}}}{n}k\hfill \end{array}$$

(11)

As shown in Figure 11, we compute the compensation value $E_k$ per section between nodes. For a section with two nodes on each end, we compared the Dead-Reckoned (DR) position $p_e$ and the SLAM-corrected node position $p_n$. When there are n number of images in the section we apply compensation value $E_k$ for the position associated with $k^{\mathrm{th}}$ image. The result of this interpolation is shown in Figure 10c,d. Note that the compensated projection demonstrates a more smooth and consistent lane map.

7. Experiments and Results

To validate the proposed method in the real world, we conduct an experiment covering a campus area with 9.32 km path length. The dataset is the route around a campus environment including four loop-closures by passing the center building repeatedly. The campus area is usually composed of low-rise buildings and wide roads that offer sufficient but sporadic GPS signals. As can be seen in

Table 4. Summary of Digital-Based SLAM results.

Path Length	Logging Time	Computation Time	GPS Node	Wall Node	Digital Map Node	Total Nodes	No. of Point
9322.35m	1952.06s	792.04s (40.57%)	482 (41.4%)	249 (20.09%)	136 (11.67%)	1165	23,017,120

, we have a 41.4% of GPS signal reception.

The overview of the entire campus area is as in Figure 12 depicted in the top-down and perspective view with a scale marked. In total 9.32 km of path is covered in 32 min. Overall the campus shows an altitude difference of 29.5 m between the northern and southern part. The final SLAM graph consists of 1165 nodes, and the average distance between the nodes is 8.06 m.

7.1. SLAM Results

We first evaluate the SLAM results. Compared to Dead-Reckoned (DR), SLAM provides a more consistent map as shown in Figure 13b. The position is a value converted to a standard reference point (38${}^{\circ }$00’00”N/129${}^{\circ }$00’00”E), which follows the method of transverse Mercator coordinates. The algorithm was capable of making four times of loop closure successfully (Figure 13c) using a wall-to-wall loop-closing measurements. The node uncertainty propagation plot shows the SLAM effectively bounds the uncertainty (Figure 13a). We plot both the entire covariance determinant ($\sqrt[6]{\mathrm{\Sigma }}$ in m·rad) and the positional covariance determinant ($\sqrt[6]{{\mathrm{\Sigma }}_{xyz}}$ in m). DR shows approximately 10% of the positional uncertainty, while SLAM results are mutually bounded by wall-to-wall loop closure, GPS and digital map correction algorithm. In Table 4, the computation time with respect to the total mission time is summarized. The entire method runs in real time using only 40% of the total mission time. This is due to the fact that the low-rise building allowed more GPS signal reception , because the number of the GPS and the wall node was the critical factor in terms of process time.

Figure 14 visualizes the nodes by the sensor types for the entire trajectory. As the environment is with low level buildings, GPS signals (blue) are available for many nodes (41.4%). However, the signal is sporadic and vulnerable to the surrounding structures. When the GPS signal becomes unreliable we use complementarity available structural information and correct against the digital map (green). DEM is also effective when altitude change is substantial within a region.

7.2. Qualitative Urban Mapping Results

For further validation of the proposed method, we back-project 3D point clouds and images using the accurate localization and mapping results. By doing so, we can generate a 3D urban map with lanes potentially for autonomous car driving. By confirming the consistency in the back-projected dataset, we validate the accuracy of the proposed method and application to urban map generation.

7.2.1. 3D Point Cloud Map

Collected LiDAR points are locally accurate with respect to the vehicle pose under exact extrinsic calibration. However, the entire map consistency may not be guaranteed if the estimated trajectory is with errors. Using the SLAM refined trajectory, we use point cloud information in order to create a 3D map of the urban environment. Figure 15 presents a qualitative result on two sample buildings created from the back-projection. As can be seen in the sample buildings of Figure 15, the proposed method is capable of detecting even highly complex walls The accuracy of the 3D modeling is significantly affected by the trajectory estimation accuracy. The SLAM based method enables accurate estimation even under a complex motion with many turns (Figure 15c).

7.2.2. Lane Map via IPM

We also generate a lane map for urban mapping. Using the Simultaneous Localization and Mapping (SLAM) results and the previously introduced adaptive Inverse Perspective Mapping (IPM) model, we produce precise bird’s eye view images compensating for vehicle motion. To generate dense and precise lane map, additional vehicle positions are required because the average distance between nodes of SLAM result is 8.06 m. Additional vehicle position and heading angle are interpolated by using vehicle encoder data from nodes that generated by SLAM. Figure 16 shows the entire campus lane map with four representative samples (shown in a zoomed view). Note the consistency of the generated map by the IPM images back-projection.

7.3. Accuracy Analysis

In the previous section, we have demonstrated consistency and qualitative evaluation. We also performed quantitative evaluation on the mapping results. Using RTK-GPS with 10 mm-accuracy, we analyzed the accuracy of the points from both sample buildings and lane maps. We use GRX2 by Topcon [56] in the RTK mode to measure ground truth point sampling. As shown in Figure 17, we measure sample points from both buildings and lanes.

We analyze the effect of each measurement model for the entire map generation. Our evaluation process is described in Figure 18. Four building corners measured by RTK-GPS are connected to complete the boundary of a building (red square in Figure 18a. By measuring a perpendicular distance from LiDAR-extracted wall to the ground truth, we compute the Root Mean Square Error (RMSE) mapping error for 3D structures. The error analysis is summarized in

Table 5. Positional error measurement between the 3D building point cloud and RTK-GPS measured building corners. The map error is computed from average of RMSE between the ground truth wall generated from RTK measured sample points and LiDAR 3D points in the mapped wall. Note that this error is over 9.32 km of travel distance.

Sample	No. of 3D Map Points	Average RMSE
		GPS Only [m]	Digital Map-Based [m]
		Set 1	3352	0.437	0.190
Set 2	1967	1.010	0.193
Set 3	227	2.070	0.347
Set 4	3314	1.407	0.136

. GPS-based method shows about a meter accuracy due to the sensor accuracy and unreliable availability. Compared to GPS-based mapping, the digital map-based SLAM shows substantial improvement over the conventional approach.

Similarly, we examine accuracy on a lane map from a set of sample points on road markings. In total, 16 points from five different sets are measured. Among them, Figure 19 presents validation process for a sample set with six sample points. Six RTK-GPS positions are obtained and used as ground truth (Figure 19a). In this analysis, we compare the proposed lane map with aerial image and digital map. Overlaying ground truth on aerial images (Figure 19c) reveals substantial discrepancy due to distortion without compensation from other sensors. In digital maps, road boundaries are accurate but road marking and lanes are missing. Compared to digital and aerial maps, the proposed method (Figure 19b) demonstrates substantial accuracy improvement.

Table 6. Error analysis for the proposed method on road marking. Each dataset has two to six sample points respectively. For set 2, no road marking in the aerial map and excluded in the comparison. The proposed method’s error is also written as a ratio to the aerial image for clear comparison.

Sample	No. of Lane Points	Average RMSE
		Aerial Image [m]	Proposed Method [m]
		set 1	2	7.180	1.000 (13.927%)
set 2	2	-	1.055 ( - %)
set 3	2	8.948	1.699 (18.989%)
set 4	6	8.044	0.622 (7.727%)
set 5	2	11.261	0.610 (5.418%)

lists accuracy analysis for the proposed method for all five test sets. As in the 3D map analysis, we compare the RTK-GPS measured ground truth with aerial image and proposed method. Unlike the 3D structural map error, the SLAM error is not the only error source for this road marking error. When manually selecting a pixel from distorted and stitched road images, other factors related to image processing influences (e.g., image resolution, image quality, image distortion, manual pixel selection accuracy, and interpolated motion error). Lane parameterization and online conversion will surely improve overall lane results from fine refinement. In this work, however, we mainly aim to validate the SLAM result accuracy by examining error from structural and lane mapping results.

8. Conclusions

This paper proposes a seamless way to incorporate publically available digital maps and sensors mounted on a mobile platform. For accurate urban mapping, we focus on 3D structural and lane mapping. The proposed method tackles the accurate map building problem by integrating global measurements with local sensor measurements via Simultaneous Localization and Mapping (SLAM). The results validate the accuracy of the created map. As the ground truth is not available we have conducted accuracy analysis by measuring a set of sample points using RTK-GPS. The proposed method has proven itself capable of balancing unreliable GPS and digital map information to create an accurate map.