Accurate and up-to-date road network information is extremely critical for various urban applications, such as navigation and infrastructure maintenance [1
]. The advent of modern remote sensing has enabled the extraction of information from very high-resolution (VHR) and highly detailed optical images of roads to update urban road networks [4
]. High spatial resolution enriches feature details but complicates object extraction [6
]. Although considerable effort has been devoted to road-feature extraction from VHR images, a completely practical road-feature extraction technology remains unrealistic.
A considerable number of articles have been published on road-feature extraction from remote sensing images. Generally, state-of-the-art methods for road-feature extraction from VHR images fall into two categories: Automatic and semiautomatic methods. Automatic approaches require no prior information and can be executed by a series of image-processing algorithms, such as mathematical morphology [11
], active snake model [13
], dynamic programming [14
], neural networks [15
], probabilistic graphical models [18
], filtering-based methods [19
], and object-oriented methods [20
]. In general, however, the unsatisfactory performance of the automatic method in road-feature extraction from images presenting complex natural road scenarios (e.g., image noise and tree and shadow occlusion) restricts its practical applications [21
]. The limitation of automatic methods has encouraged the proliferation of studies on semiautomatic methods. In contrast to automatic methods, semiautomatic methods require user input or other prior information to achieve robust and stable results.
Two technical ideas are present for semi-automatic road extraction; the first involves treating the extraction as a problem of image segmentation (divided into road and non-road) and then obtaining the final result by post-processing [22
]. This method is easily affected by vegetation occlusion or large shadows, which leads to low recognition rates. In addition, due to the introduction of a post-processing algorithm, other features are easily misjudged as roads.
The other idea involves treating the extraction as a network optimization problem. The road network is obtained by the connection of road seed points, and the final result is acquired with the use of graph theory or dynamic programming techniques [27
]. The local features of the road (such as extensibility, edge characteristics, and topological structure of the road network, etc.) are fully considered in this method, and a reliable initial road seed point is obtained through human–computer interaction. Therefore, the accuracy of road extraction results is relatively high. However, the extraction effect of shaded and occluded roads is poor because of the different methods of connecting road seed points. In addition, the number of seed points needed for U- or S-shaped roads is more than that for linear roads, thereby requiring considerable manual work.
According to this analysis, the method based on image segmentation is more efficient but less accurate than that based on road seed points, which is less efficient but more precise. Inspired by Reference [33
], we propose to treat road seed point connection as a shortest-path problem to improve the efficiency of road extraction on the basis of seed points. The fast marching method was recently developed for connecting road seed points [33
]; it is a particular case of level set methods, which were developed by Osher and Sethian [34
] as an efficient computational numerical algorithm for tracking and modeling the motion of a physical wave interface (front). This method has been applied to different research fields, including computer graphics, medical imaging, computational fluid dynamics, image processing, and computation of trajectories [33
]. In Reference [33
], this method showed high stability and general advantages and suitability for processing low-/medium-resolution remote sensing images. However, it is difficult to extract unbiased road centerline information from VHR remote sensing images by using the fast marching method alone.
In VHR remote sensing images, “noise” is produced by the improvement of resolution, which leads to inconspicuous useful edge information. Complex image backgrounds also produce a large number of finely divided edges, which are difficult to process and thus result in the difficulty of road edge extraction. Extracting straight roads and planar roads is challenging due to the existence of the same objects with different spectra and different objects with the same spectrum, which make the extraction of roads effectively by using the road spectral feature alone a difficult task. Thus, this study presents a semiautomatic edge-constraint fast marching (ECFM) method to extract road centerlines from VHR images. Edge information, road spectral feature, and the road centerline probability map are utilized and an edge-constraint-based weighted fusion model is introduced to assist the fast marching method. The proposed method enables accurate and unbiased road centerline extraction and shows high generalization capability in processing complex road scenarios, such as S-shaped, U-shaped, and shaded roads. The contributions of the method are as follows.
Edge information of remote sensing imagery has been studied extensively and widely used in the extraction and tracking of linear objects, such as roads and rivers, in medium-/low-resolution remote sensing imagery. The present study indicates that the synergy of edge information, road centerline probability map, and road spectral feature can overcome the shortcomings of the bias of the road centerline extracted by the fast marching method, which uses spectral feature only. Moreover, our method is robust to road extraction in shaded areas.
Another contribution of this study is that the proposed method needs only a few road seed points when extracting an S-shaped or U-shaped road. This characteristic leads to the efficiency of the widespread practical application of road centerline extraction from remote sensing images.
The remainder of this paper is organized as follows. In Section 2
, related state-of-the-art methods are reviewed. Section 3
provides an introduction to the proposed method. Experimental results are given in Section 4
and discussed in Section 5
. The conclusion is presented in Section 6
2. Related Work
Many approaches have been proposed in the last decades for extracting road segmentation from aerial and satellite images. Low-level features can be extracted and heuristic rules (such as connectivity and shape) can be defined in numerous ways to classify structures similar to roads. A geometric stochastic road model based on road width, length, curvature, and pixel intensity was applied in Reference [37
]. Hinz and Baumgartner [38
] used road models and their contexts, including their knowledge of radiation, geometry, and topology. The disadvantage of these rule-based heuristic models is that obtaining the optimal set of rules and parameters is difficult because of the wide variety of roads. Therefore, these methods can work only in areas where the features used (such as image edges) occur mainly on roads (e.g., rural areas).
Most approaches consider road extraction a binary segmentation problem. The path trajectory point [22
] and the angle-based texture feature [23
] of a particular pixel can be defined to quantify road probability on the basis of shape. Das et al. [24
] adopted the spectral and local linear features of multispectral road images. By combining the probabilistic support vector machine (PSVM) method, dominant singular value method, local gradient function, and vertical central axis transformation method to classify the region, the authors detected the road edge, linked the broken road, and eliminated the non-road area. The advantages of this method were verified by experiments on many road images. In Reference [25
], the image was initially segmented by fused multiscale collaborative representation and graph cuts, and the initial contour of the road was then obtained by filtering the road shape. Finally, the road centerline was obtained through tensor voting. In Reference [26
], the image was first divided into road and non-road through SVM soft classification; then, the probability of each pixel belonging to the road was obtained simultaneously; the final road was acquired through the graph cut method. However, these methods work well in multispectral images only and can detect only the main roads in urban areas. Thus, extracting roads from areas with dense buildings or other areas which are similar to road grayscale is challenging.
Another semiautomatic road extraction method regards road extraction as the connection and tracking problem of road seed points. Hu et al. [27
] proposed a segmented parabolic model to delineate road centerline networks. The method first uses seed points to generate parabolic segments and then applies least-squares template matching to calculate parameters for precise parabola extraction. Miao et al. [28
] proposed a kernel density estimation method combined with the geodesic method to decrease the number of seed points required for road extraction. Zhou et al. [29
] used particle filtering to track road segments between seed points. However, particle filtering is limited by its incapability to effectively deal with road branches. To extend the generalization capability of particle filtering to complex scenarios, Movaghati et al. [30
] integrated particle filtering with extended Kalman filtering. Lv et al. [31
] proposed a multifeature sparsity-based model that can utilize multifeature complementation to extract roads from high-resolution imagery. Dal Poz et al. [32
] proposed a semiautomatic method to extract urban/suburban roads from stereoscopic satellite images. This method uses seed points to construct the road model in the object space. Optimal road segments between seed points are then generated through dynamic programming. Road extraction based on seed points can achieve high precision, but the efficiency is low. The main reason is that the input of road seed points needs human intervention. A large number of required seed points will affect the efficiency of road extraction.
4. Experimental Study
An experimental study was performed with eight VHR remote sensing images to validate the effectiveness and adaptability of the proposed method in road extraction. A discussion of the experimental study is presented in this section, which is divided into three subsections. The first subsection provides a description of the study. The second subsection presents a discussion of the four experimental set-ups. The detailed parameter settings applied in the experimental set-ups are also given in this subsection. Finally, the results of the four experiments are provided in the last subsection.
To assess the effectiveness and adaptability of the presented method, experiments were conducted with eight VHR remote sensing images. The images are described below.
The first image is shown in the first row of Figure 3
. It is an aerial image with a spatial resolution of 0.3 m/pixel and a spatial size of 400 pixels × 400 pixels. It was downloaded from Computer Vision Lab [45
The second image has a spatial resolution of 0.6 m/pixel and a spatial size of 512 pixels × 512 pixels. It was collected by the QuickBird satellite and was downloaded from VPLab [46
]. The image is shown in the second row of Figure 3
The third and fourth remote sensing images have spatial sizes of 400 pixels × 400 pixels and are shown in Figure 4
. The images were downloaded from Computer Vision Lab [45
]. They have a spatial resolution of 0.6 m/pixel and show an area that is mainly covered by vegetation, roads, and buildings.
The fifth image is shown in Figure 5
and has a spatial size of 3500 pixels × 3500 pixels and a spatial resolution of 1 m/pixel. It was collected by the IKONOS satellite and shows an area of Hobart, Australia. This image includes different types of noises, such as vehicle occlusion, sharp roadway curves, and building shadows.
The sixth image, which is shown in Figure 7, was collected by the QuickBird satellite. The image shows an area in Hong Kong. It has a spatial resolution of 0.6 m PAN band and a size of 1200 pixels × 1600 pixels. It includes various road conditions, such as road material changes, vehicle occlusion, and overhanging trees.
The seventh image has a spatial size of 3000 pixels × 3000 pixels and a spatial resolution of 2 m/pixel, as shown in Figure 8. This image was collected by the WorldView-2 satellite and shows an area of Shenzhen, China, covering a variety of roads with different materials. The image also includes several types of noise, such as zebra crossings, traffic-marking lines, and toll stations.
The eighth image, as shown in Figure 10, is a grayscale image with a spatial size of 725 pixels × 1018 pixels and a spatial resolution of 1 m/pixel. This image was collected by the IKONOS satellite and shows an area of Hobart, Australia, depicting several road conditions, such as overhanging trees, vehicle occlusion, and roads with large curvatures.
Road extraction from these datasets is challenging because of their very high spatial resolution of 1 m or higher. In addition, as seen from each image, roads, buildings, vehicles, and shade may be conflated with one another. Hence, uncertainties may be encountered during road centerline extraction from these datasets.
4.2. Experimental Setup and Parameter Setting
The accuracy and efficiency of the proposed ECFM road extraction method was investigated through the following six experimental setups with the eight VHR remote sensing images shown above.
The first experiment was designed to test the effect of the edge constraint in the proposed approach. Two VHR remote sensing images were used in the experiment, as shown in Figure 3
. Two road seed points were marked by the user, and the road centerline was extracted through our proposed method with edge constraint and through a method without edge constraint. The parameters of the proposed method were T = 0.2, α
= 0.9, β
= 0.7, and λ
The second experiment aimed to assess the performance of the proposed approach in extracting the centerlines of U-shaped roads. Two VHR remote sensing images showing U-shaped roads were adopted in the experiment, as depicted in Figure 4
. The images have a resolution of 0.6 m. To ensure fair comparison, we compared the proposed ECFM method with (1) Hu et al.’s method [27
] and (2) Miao et al.’s method [28
] because these two methods rely on user-selected seed points. We used the endpoints at both ends of the U-shaped road as the seed points for road extraction. If the two seed points failed to provide the correct road extraction results, we added some intermediate points to ensure the integrity of the road extraction results. The optimal parameters of each experiment were identified through the trial-and-error method. The parameters of these approaches were as follows: (1) In Hu’s method, the window size of the step-edge template was set at h = 5; (2) In Miao’s method, the threshold parameter was set at T = 0.002; (3) In the proposed method, the parameters were set as T = 0.2, α
= 0.9, β
= 0.9, and λ
The third and fourth experiments were designed to investigate the accuracy and efficiency of the proposed ECFM method. This experiment employed satellite images with high spatial resolution and had two objectives. First, similar to the first experiment, it aimed to test the efficiency of the proposed method. Second, it aimed to verify the robustness of our proposed method for the centerline extraction of shadowed roads. We compared the proposed ECFM method with (1) Hu et al.’s method [27
] and (2) Miao et al.’s method [28
]. The parameter details of each approach are as follows: (1) In Hu’s method, the window size of the step-edge template was set at h = 5; (2) In Miao’s method, the threshold parameter was set at T = 0.002; (3) In the proposed method, the parameters were varied in accordance with the shading condition of the road. When the road was not shaded, the parameters were set as T = 0.2, α = 0.9, β = 0.9, and λ = 0.4. By contrast, when the road was shaded, the parameters were set as T = 0.2, α = 0.5, β = 0.5, and λ = 0.05.
The experiments were designed as follows:
For all methods, as few seed points are selected as possible to improve the efficiency of road extraction while ensuring integrity.
For an occluded road area, road seed points that are not occluded by shadows or automobiles are selected as much as possible to ensure the accuracy of road extraction.
The fifth and sixth experiments aimed to test the road extraction efficiency and accuracy of different methods under the same seed points. The fifth experiment used a Worldview-2 color image, and the sixth experiment used an IKONOS grayscale image. This design had two purposes. The first was to verify the efficiency and accuracy of different methods under the condition of using the same seed points, and the other was to verify the robustness of the methods proposed in this work on images with different color modes (color images and grayscale images). Seed points for these two groups of experiments were obtained by artificial marking. To ensure fairness, road extraction should be conducted according to the collection sequence of artificial seed points when different methods are adopted. (1) Hu et al.’s method [27
] and (2) Miao et al.’s method [28
] were used here for comparison. The parameters used in these experiments were the same as those applied in the third and fourth experiments.
4.3. Results and Quantitative Evaluation
Four accuracy measures [27
] were used to evaluate the performance of the presented method. These measures included: (1) Completeness = TP/(TP + FN); (2) Correctness = TP/(TP + FP); (3) Quality = TP/(TP + FP + FN), where TP, FN, and FP represent true positive, false negative, and false positive, respectively; (4) Seed-point number. The ground truth was produced through the hand-drawing method, and the buffer width was set to four pixels.
4.3.1. Test of the Edge Constraint
Two remote sensing images were selected to test the edge constraint effect on road centerline extraction. The results are presented in Figure 3
. The method using edge constraint provided better results than those provided by the method without edge constraint. The results obtained through the method without edge constraint easily deviated from the true road centerline, whereas those obtained through the proposed method with edge constraint could preserve the road centerline. The proposed method using edge constraint is more accurate than other methods because of the two following advantages: First, edge-energy computation and distance transformation can provide the ridgeline of the road segment, as shown in Figure 2
g. Second, the fast marching method can trace the road centerline along the ridgeline. The visual comparison of the results, as presented in Figure 3
, illustrates the advantages of the proposed method in road centerline extraction.
4.3.2. Experiment on Centerline Extraction from U-Shaped Roads
The results of the three methods are compared in Figure 4
. This figure shows that all the three methods extracted the expected road centerlines. Compared with that of Hu’s method, the performance of Miao’s method and the proposed ECFM method improved with the number of road seed points. The proposed ECFM method, however, provided better results for both images than Hu’s and Miao’s methods. Table 1
shows the quantitative evaluation results of the three methods. Among the three tested methods, the presented method achieved the highest quality values for the two cases. These values coincided with the extraction results presented in Figure 4
. Although Hu’s method accurately extracted centerlines, it consumed more road seed points than the other two methods because it requires intermediate road seed points when extracting centerlines from S- or U-shaped road segments. By contrast, the proposed method extracts centerlines from S- or U-shaped roads with only two road seed points.
4.3.3. Experiment on An IKONOS Image
shows that the proposed ECFM method extracted most of the road segments and provided satisfactory results. A visual comparison between the extraction results is shown in Figure 6
a–d. This figure shows that the proposed method performed better than the other methods. Table 2
shows the quantitative results of the three methods. The results shown in Table 2
indicate that the three methods successfully extracted a relatively complete road centerline with relatively high extraction quality. Nevertheless, the efficiency of the proposed ECFM method is superior to that of Hu’s and Miao’s methods. For example, the proposed method used the fewest seed points among all three tested methods. Given that the solution of Hu’s method for parabola parameters is heavily dependent on the radiometric features of dual edges, this method will encounter problems when extracting features from images with unclear edges. Specifically, Hu’s method will not provide the desired result if the road boundary is unclear. Miao’s method exploits the geodesic method to connect road seed points. Its performance, however, is affected by road occlusions. The presented method achieved the highest quality values among all tested methods, indicating that it achieves the best balance between road extraction quality and seed-point consumption. Although Hu’s method can extract relatively complete centerlines, its quality values are lower than those of the presented method because the result obtained through Hu’s method is biased to the ground truth, whereas that obtained through the presented method is considerably closer to the ground truth.
4.3.4. Experiment on A QuickBird Image
shows that Miao’s method cannot efficiently manage abrupt changes, such as road junctions and sudden material changes or conflations, in images. This limitation is attributed to the method’s requirement for an intermediate step to measure initial road centerline probability, which is computed on the basis of seed-point information, from the binary road image. Miao’s method could not extract the expected road centerline if road segments between seed points were occluded by shadows or by a vehicle. By contrast, the proposed method utilizes edge energy and curvature to reduce the effect of shadows and vehicles on the road. The performance of Hu’s method was comparable with that of the proposed method. However, the road seed-point consumption of the proposed method was superior to that of Hu’s method. Table 2
shows the quantitative evaluation results of three methods. Although the proposed method used fewer seed points than the other two methods, it obtained higher completeness, correctness, and quality values. These values coincided with the extraction results presented in Figure 7
. The experimental results illustrate that the proposed method is robust to noise and has considerable potential applications in road extraction from VHR remote sensing images.
4.3.5. Experiment on A WorldView-2 Image
shows that the proposed ECFM method can be used to reliably and accurately extract roads in a wide range of high-resolution remote sensing images. Figure 9
shows the local comparison of roads extracted by different methods. Overall, all three methods can achieve satisfactory results. The comparison in Figure 9
a shows that ECFM and Hu’s methods have good anti-noise performance when encountering toll stations, and compared with Miao’s method, the road centerline extracted is closer to the center. This difference is due to the fact that Miao’s method considers only the spectral features of roads while our and Hu’s methods not only consider the spectral features but also combine the edge features. Figure 9
b shows that in road sections where road materials change greatly, all three methods can extract the road centerline accurately. Nevertheless, comparison indicates that the road centerline extracted by the ECFM method is smooth, and the technique can maintain high accuracy in sections with large road curvatures. Figure 9
c shows the differences among three methods of extracting roads near road intersections. According to the figure, the road centerlines extracted by ECFM and Hu’s methods are relatively smooth. The road centerline extracted by Miao’s method is easily influenced by vehicles on the road, so the extraction results in the vehicle-intensive area are not smooth enough. Figure 9
d shows the results of different methods in the case of shadow occlusion. Comparison shows that Hu’s extraction result is relatively smooth because the technique adopts the piecewise parabolic model, which can obtain a relatively smooth curve. However, according to the figure, the road centerline acquired by this method can easily shift. Miao’s method is influenced by shadows and cars, which lead to the unsmooth extraction results. The ECFM method proposed in this paper has achieved a relatively balanced performance, and it is better than the compared techniques in terms of road smoothness and accuracy. The statistical results in Table 3
are also consistent with those in Figure 9
. Table 3
shows that the ECFM method performs well in terms of completeness, correctness, and quality under the condition of using the same number and location of road seed points.
4.3.6. Experiment on An IKONOS Grayscale Image
shows the results of three different methods for extracting the road centerline from an IKONOS grayscale remote sensing image. As can be seen from the figure, all roads can be extracted completely by the three methods. The road centerline extracted by Hu’s method is the smoothest, but the limitation of the piecewise parabolic model it uses causes the extracted results in areas with large changes in road curvature to tend to deviate from the road center. Miao’s method and the ECFM method can avoid this problem. Compared with Miao’s technique (which considers only the spectral features of roads), the ECFM method (which fuses the edge features and spectral features, thereby potentially overcoming the influence of spectral changes placed by shadows on road extraction results to a certain extent) shows better performance on shadow and vegetation occlusion. As can be seen from the statistical results in Table 3
, the extraction completeness of all three methods is high when the same number and location of road seed points are used. However, our method achieves the best performance in terms of extracting correctness indicators. Similarly, our method demonstrates the best quality.
This study presents a semiautomatic approach that uses road seed points to extract road centerlines from VHR remote sensing images. An edge-constraint-based weighted fusion model was introduced to overcome the influence of road occlusion and noise on road extraction. Finally, an edge-constraint fast marching method was proposed to improve the accuracy and quality of the road extraction results.
Six experiments were conducted on eight VHR remote sensing images that are related to different road conditions, including vehicle occlusion, sharp roadway curves, and building shadows. The advantages of the proposed method are as follows: (1) favorable road extraction accuracy and efficiency and (2) robustness to extracting road centerlines from VHR remote sensing images. Overall, the presented method is a superior and practical solution to road extraction from VHR optical remote sensing images.
In future work, the performance of the proposed method on additional types of remote sensing images, such as unmanned aerial vehicle images with very high spatial resolution, will be extensively investigated. The application of the proposed method to roads constructed from different materials and the automatic selection of road seed points are interesting future research directions.