1. Introduction
Highly updated road maps are crucial in applications, such as intelligent transportation, autonomous driving, and disaster emergency response. High-resolution remote-sensing imagery with wide coverage and fast update speeds has been the main data source to update road maps collected historically. Most of the current research focuses on extracting vector road maps from remote-sensing images [
1,
2,
3].
Given that there are already high-quality road maps, such as the Open Street Map, covering the world, the construction of vector road maps has gradually transitioned from road extraction from scratch to updating changed roads [
4,
5]. However, even with high-quality road maps as the basis, updating a vector road map is still a labor-intensive task [
4,
6,
7]. Hence, there is an urgent need to develop automatic updating methods for vector road maps based on high-resolution remote-sensing images. Road centerline extraction methods from remote-sensing images are used to update vector road maps.
With the increasing availability of high-resolution remote-sensing images, many road centerline extraction methods based on high-resolution images have been proposed in the past decades [
8,
9,
10]. These methods can be divided into methods based on road segmentation, methods based on direct graph extraction, and multi-task methods that extract both the road surface and the centerline. Road segmentation-based methods first segment the road surface and then obtain the road centerline by thinning the road surface [
11,
12,
13,
14,
15,
16,
17].
However, the road surface segmentation itself has many difficulties, and the thinning process is prone to producing centerline disconnections and burrs. To improve the topological connectivity of road centerlines, methods based on direct graph extraction were proposed to directly infer road maps from different viewpoints [
18,
19]. In addition, some multi-task cascade networks are proposed to extract road surface and road centerline simultaneously [
20,
21,
22].
The current methods obtain high accuracy in road centerline extraction. However, complex post-processing should be conducted on the discontinuous and burred road centerline extraction results to add missed roads, connect broken sections, and remove false roads when using these results to update a historical road map. Therefore, it is necessary to perform research on road map updating based on remote-sensing images directly.
Road map updating involves verifying unchanged roads, extracting newly built roads, and removing disappeared roads. Taking the public MUNO21 dataset [
4] as a starting point, there has been some research on updating roads based on change detection in bi-temporal remote-sensing images. For example, Bastani et al. [
23] proposed a two-stage road update framework based on bi-temporal imagery. Zhou et al. [
5] proposed the UGRoadUpd framework for guiding road updates by unchanged roads. However, obtaining a historical remote-sensing image that matches the collection time of the historical road map is difficult. Therefore, updating a historical road map with only a current image remains an issue.
To solve the above-mentioned problem, a novel road vector map updating (VecRoadUpd) framework is proposed based on the observation that road changes are highly correlated with road-intersection changes. Road-intersection changes involve changes to the locations and the branches of the intersections. To accurately discover the change in road intersections, a CNN-based intersection-detection network (CINet) is used to extract intersections from current images and historical vector road maps.
A metric named the Threshold of Partial Intersection-of-Union () is introduced to measure whether there are newly built or disappeared road junctions. Based on the discovery of changed intersections, a road branch detection network (RoadBranchNet) and a spatial analysis strategy are combined to detect intersection branches in current images and old road maps. A threshold of angle () is used to assess whether the direction of road branches changed for a homonymic road intersection. Based on the discovery of direction-changed road branches, directed road tracing is designed to update road maps accurately.
The remainder of this paper is organized as follows:
Section 2 introduces the related road extraction and road map update methods.
Section 3 introduces an overview of the proposed road change detection and update framework.
Section 4 presents the experimental results and analysis.
Section 5 demonstrates the ablation analyses.
Section 6 shows and explains the failure cases. Our conclusions are presented in
Section 7.
3. Methodology
The vector road map updating (VecRoadUpd) framework proposed in this paper updates road maps by detecting changed road intersections and tracing roads directionally. Different from the existing road map updating methods, VecRoadUpd captures possible road changes by discovering road-intersection changes. Taking the location and direction of changed road branches, VecRoadUpd updates road maps accurately through directed road tracing. The workflow of the VecRoadUpd framework is shown in
Figure 1.
It can be seen from
Figure 1 that the proposed VecRoadUpd framework takes a current remote-sensing image and an old road vector map as input and directly outputs an updated road vector map. The VecRoadUpd framework includes road intersection change detection, road branch change detection, and directed road tracing. The intersection change detection process finds newly built and disappeared intersections. Newly built intersections mean there are newly built roads added, and disappeared intersections mean there are disappeared roads.
Newly built and disappeared roads can also be built and removed from existing road intersections, so VecRoadUpd extracts the road branches of intersections by using a road branch detection network (RoadBranchNet) in the second stage. In this way, VecRoadUpd can detect changes in the old road map while also providing changed road branch directions for tracing newly built roads and removing disappeared roads. Based on the changed road branch directions, VecRoadUpd updates road maps through directed road tracing in the last stage.
3.1. Road Intersection Change Detection
Road intersection change detection is used to find newly built and disappeared intersections. The idea of extracting the new and old temporal road intersections first and change detection later is used to discover the changed road intersections. To extract the new and old temporal road intersections, a CNN-based intersection-detection network (CINet) is applied to extract road intersections from a current remote-sensing image and a historical road vector map. The CINet uses CSPDarkNet53 [
58] as the backbone, FPN [
59] as the feature fusion neck, and a decoupled head [
60] commonly used in one-stage object-detection networks as the head.
The architecture of CINet is shown in
Figure A1. A hybrid loss function consisting of an object confidence loss (
), a classification loss (
), and a target box regression loss (
) is used to train CINet to learn features, such as the shape and texture of intersections. Among them,
and
are calculated using binary cross-entropy loss (
), and
is calculated using Complete IoU (CIoU) loss [
61]. A well-trained CINet is used to extract road intersections from current remote-sensing images and historical road vector maps.
where
represents the true value and
represents the predicted value.
Then, an intersection change analysis rule is presented to find newly built and disappeared road junctions based on the road intersection extraction results. A novel indicator named partial intersection over union (PIoU) is developed to judge whether two boxes belong to the same intersection. The PIoU is calculated by the formula PIoU = max (area (
)/area (
A), area (
)/area (
B)). Based on PIoU, the road intersection change analysis rule is introduced to detect newly built and disappeared intersections as shown in
Figure 2.
The road intersection change analysis rule takes new and old temporal intersections extracted from a current image and a historical road map as inputs and outputs the road intersection change detection result. In this rule, overlay analysis is first performed to obtain the overlap area between the old and new intersections, and then the PIoU is calculated. If the calculated PIoU value is less than the threshold of PIoU (), the intersection is considered a changed intersection and, otherwise, is considered unchanged.
It is worth noting that the value of
is adjustable and is a variable that affects the intersection change detection and road map update. Therefore, the detailed experiments about how
influences the road update accuracy were conducted in
Section 5.1. Through intersection change detection, changes in the historical road map are initially detected. Furthermore, the prior information required for road branch change detection is also obtained: the position of the intersection and the number (
N) of road branches connected to the intersection.
3.2. Road Branch Change Detection
The intersection change detection in
Section 3.1 obtains the location of the changed intersection and the number (
N) of road branches connected to the intersection without obtaining specific changed road branches. Therefore, to obtain more specific changed road branches, the road branch change detection process is designed in
Section 3.2. The road branch change detection process consists of two steps, road branch extraction and change discovery, based on the changed intersections obtained in
Section 3.1.
3.2.1. Road Branch Detection
Road branch detection aims to extract road branch directions from current remote-sensing images and historical vector road maps. Due to the different data types of images and vector road maps, different road branch detection methods are used in this paper. The flowchart of the two road branch detection methods is shown in
Figure 3.
Figure 3 illustrates the detailed process of extracting road branches from a current image and a historical vector road map. As seen in
Figure 3a, a road branch detection network (RoadBranchNet) is used to extract road branch directions from the current remote-sensing image. The RoadBranchNet is inspired by the CNN-based decision module designed in RoadTracer. However, the decision module in RoadTracer decodes only one road branch direction from the angle output (
) and cannot obtain multiple road branch directions at intersections.
Therefore, we modified the output layer of the decision module and decoded
using local maximum analysis. The structure of RoadBranchNet is shown in
Figure 3a. The process of decoding
is shown in
Figure 3c. The
is a 1 × 64 vector, where “64” means that the 2
radian centered on the intersection is divided into 64 equal parts, and each part represents a branch direction. Each value in the
represents the probability that the intersection has branches in each direction.
To obtain road branch directions, the local maximum analysis is conducted on these 64 probability values. Furthermore, the local maximums are sorted by the number (N) of road branches connected to the intersection. Then, the directions represented by the larger N probability local maximums are the branch directions of the intersection. In this way, the road branches for different intersections are extracted from a current remote-sensing image.
As shown in
Figure 3b, a road branch detection process based on the spatial analysis strategy is used to extract road branch directions from a historical vector road map. First, the spatial analysis strategy of the road intersection box and vector road map is performed to obtain the cross points. Then, the road intersection center point is connected with the corresponding cross points to obtain the road branch directions. In this way, the road branches for different intersections are extracted from a historical vector road map.
3.2.2. Road Branch Change Detection
A branch change detection process based on the intersection branch detection results is presented in this section to find newly built, disappeared, and unchanged road branches in old road maps from current images. The main idea is to obtain changed and unchanged road branches by comparing the differences between the directions of corresponding old and new road intersection branches detected from old road maps and new remote-sensing images. The workflow is shown in
Figure 4.
The road branch change detection is performed by comparing the angles between the road branches detected from a current image and a vector road map. As shown in
Figure 4, the road branch direction is calculated with the intersection center point as the origin and the horizontal left direction as the positive direction. Then, the absolute difference between road branches detected from a current image and the vector road map is calculated. If the difference is less than or equal to
, the two branches are considered to be the same branch; otherwise, the newly added or disappeared branches are obtained.
is set to
/8, and the influence of the value setting of
on the road map update is analyzed in
Section 5.2.
3.3. Directed Road Tracing
A directed road-tracing strategy inspired by RoadTracer was applied to extract newly built roads and verify changed roads in the proposed road map updating process. In RoadTracer, however, the tracing starting points need to be given manually, which reduces the degree of automation of the algorithm. Although Wei et al. [
47] used multiple starting point tracing (MspTracer) to improve the automation of RoadTracer, MspTracer cannot extract a complete road map because the starting points of MspTracer rely on incomplete road segmentation results. Different from the above-mentioned road tracing methods, our directed road-tracing strategy takes the intersections extracted from images and vector road maps as the starting points and uses changed road branch directions as the initial tracing directions to extract and verify the changed roads.
As shown in
Figure 5, directed road tracing takes the road branches, the old road map, and the current image as input and outputs the updated road map.
It can be seen from
Figure 5 that the directed road tracing is divided into two processing steps based on the class of direction-changed branches. For new branches, directed tracing aims to extract the newly built roads that do not exist in old road maps but exist in current images. First, a starting point stack and an initial direction stack are generated from intersection center points and new branch directions, and 256 × 256 windows are generated with the image as the base map and the starting points as the center. These windows are then fed into the CNN decision module, and the output of this module decides whether to continue tracing. If continuing tracing, it repeats the above steps; otherwise, it returns to the current starting point or direction from the stack and starts tracing from the next starting point or direction.
We consider the computational cost of tracing duplicate roads. We stipulate that, if the intersection center points are tracked, the tracing will stop, and the current starting point will be popped from the stack to start tracing from the next starting point until the stack is empty. For disappeared branches, directed tracing aims to remove and validate vanishing roads that exist in old road maps but not in current images. The processing steps for disappeared branches are similar to those for new branches with the main difference being how the CNN-based decision module output is handled. For new branches, when the angle output () is greater than or equal to the tracing threshold (T) and the action output () is “”, the algorithm will add the tracked edges to the current road map and generate a next point to continue tracing; otherwise, it will stop tracing.
For disappeared branches, when is smaller than T and is “”, the algorithm will remove the edges that have been verified as disappeared from the historical road map and generate a next point to continue the verification; otherwise, the verification will be stopped. In this way, when both the starting points stack and the initial directions stack are empty, the updated road map is obtained.
7. Conclusions
In this paper, a vector road map updating framework (VecRoadUpd) was proposed for updating historical vector road maps based on changed road intersections instead of roads extracted from scratch as in the existing road updating methods.
The VecRoadUpd framework takes current images and historical road maps as the input and outputs updated road maps. The VecRoadUpd framework discovers and updates historical vector road maps using the change detection first and update later strategy. First, VecRoadUpd extracts intersections from current images and historical road maps using a CNN-based intersection-detection network (CINet). Then, VecRoadUpd identifies changed intersections based on the road intersection change-detection rule. Based on the discovery of changed intersections, VecRoadUpd detects intersection branches from current images and old road maps using the road branch detection network (RoadBranchNet).
Then, VecRoadUpd identifies direction-changed road branches based on the road branch change-detection rule. After road map change discovery, a CNN-based directed tracing algorithm was introduced to extract and verify the changed roads for accurate road map updating. The algorithm starts tracing from the center points of changed intersections, and the tracing directions are restricted by direction-changed road branches to accurately extract and verify the changed roads. Finally, updated road maps were obtained.
Extensive experiments on MUNO21, a large road map update dataset containing 21 cities and 1294 different scenarios, confirmed the effectiveness of the proposed VecRoadUpd in the road map update task and also showed that road intersections play an important role in road map change discovery. However, VecRoadUpd did not update historical vector road maps accurately in areas with severe tree occlusion. The problem of tree occlusion has always been a difficulty in road extraction in complex scenes. Therefore, we will continue to investigate how to accurately update road maps in complex road scenarios in the future.