Review Reports - Object-Based High-Rise Building Detection Using Morphological Building Index and Digital Map

Round 1

Reviewer 1 Report

Overall, it is a well-written manuscript. The methodology, results and discussions were described in great detail. However, I have some critical concerns:

1/ Why it has to be only detection of high-rise buildings? Agree that many high-rise buildings have complex shapes, but on optical images, they look better discriminated from the surroundings than the low-rise old houses?

2/ Sufficient background information on the need for building detection, but why the morphological-based method was chosen?

3/ Specification of KOMPSAT and WorldView 3 images used in this study (spatial resolution and spectral bands) and please be more specific in describing the used Digital Map (building polygon?). I suggest some details should be given in section "Method and material". If you planned to develop a generic approach, which can be applied in every optical satellite image, then the sensor description can be given in Results. Satellite images were terrain-corrected?

4/ Since you use both satellite images and digital maps, what if the digital map is outdated? and if it's updated, do we still need to detect from satellite images?

5/ Multi-resolution segmentation use eCognition tool? the morphological-based method can be also used to do this job? why not?

6/ It is unclear how the product of multi-resolution segmentation is combined with MBI. Segmentation is scale-dependent, any issue if under- or over-segmentation?

7/ Sources of reference data, any conflict with the digital map used in the detection? Do the reference data present only high-rise or buildings overall?

8/ By design, what would be the final product, high-rise buildings with the correct shape? and also correct footprint, to be well aligned with the digital map, not the roof. If that's the case, the accuracy assessment needs to be an object-based assessment.

9/ There was no difference in compactness and shape of the 3 sites, why? and how you could come up with selected parameters. Likewise, the Otsu threshold is the same for 3 sites?

10/ Discussion and Conclusion sessions are quite overlapped. Please consider including Discussion with Results.

Author Response

Thank you for your very insightful and valuable comments and recommendation. They helped us greatly to improve the paper. In the revised manuscript, we have made further modifications according to the reviewer's suggestion. The detailed corrections are highlighted in red. An item-by-item response to the reviewers' comments is provided as follows.

Reviewer comments: Why it has to be only detection of high-rise buildings? Agree that many high-rise buildings have complex shapes, but on optical images, they look better discriminated from the surroundings than the low-rise old houses?

Response: High-rise buildings can be said to be symbols of modern cities. In the process of conducting urban analysis, we discovered several problems caused by relief displacement of high-rise buildings, and this study was conducted to solve them. I added a detailed description of it.

“This geometric difference between relief displacement and nadir angle is sensitive and difficult to apply when applying not only building detection but also many unsupervised change detection techniques because even the same building recognizes the building as different buildings [28]. In this way, buildings with relief displacement in urban monitoring such as change detection act as an obstacle to accurate analysis. For accurate urban analysis, HRBs with relief displacement should be classified through separate processes, and even though they are the same buildings, it is necessary to detect building objects detected as different objects due to different relief displacement directions.”

Reviewer comments: Sufficient background information on the need for building detection, but why the morphological-based method was chosen?

Response: We added an explanation for this.

“Therefore, in this study, object-based HRB detection was performed in high-resolution image using morphological building index (MBI) that can automatically represent the existence of buildings without training data or supervision in high-resolution images and digital map.”

Reviewer comments: Specification of KOMPSAT and WorldView 3 images used in this study (spatial resolution and spectral bands) and please be more specific in describing the used Digital Map (building polygon?). I suggest some details should be given in section "Method and material". If you planned to develop a generic approach, which can be applied in every optical satellite image, then the sensor description can be given in Results. Satellite images were terrain-corrected?

Response: For accurate terrain correction, multi-temporal data for the same area is required. In this study, only short-term data were used for one area, so terrain correction was not performed separately.

Also, as you said, we added detailed specifications for each satellite sensor.

“The digital map of Korea was produced and managed by the National Geographic Information Service under the Ministry of Land, Infrastructure and Transport. The 1:5,000 scale digital map used in this study is renewed every two years, and in the case of large buildings or roads, it is modified every two weeks. In the building polygon layer provided by the digital map, only HRBs according to the Korean Building Act (i.e., the architecture with more than 30 floors or more than 120 meters high) were classified and used in this study. The HRBs defined in the digital map were overlaid with red polygons on the satellite images and expressed visually”

Reviewer comments: Since you use both satellite images and digital maps, what if the digital map is outdated? and if it's updated, do we still need to detect from satellite images?

Response: First of all, when detecting high-rise buildings by the proposed method in this study, detection of building objects with missing digital maps may not be performed properly. These limitations are additionally described in this paper.

“However, there is still a disadvantage that if a building object is omitted from the building polygon layer in the digital map during the final HRB detection process, it will affect the final result, so it is necessary to solve this problem.”

And the purpose of the study is to detect buildings with relief displacement obtained from remote sensing using digital maps generated based on the footprint of buildings, so the more we use the latest digital maps for buildings, the better the results will be.

Reviewer comments: Multi-resolution segmentation use eCognition tool? the morphological-based method can be also used to do this job? why not?

Response: In this study, object-based high-rise building detection was performed. Of course, MBI is an excellent building detection index, but it has the disadvantage of misdetection of shadows and roads. In addition, in detecting high-rise buildings, which is the purpose of the study, MBI also detects low-rise buildings, not objects of interest. To reduce this, we removed low-rise buildings and shadow objects, not high-rise buildings by using multi-resolution segmentation image that focus on high-rise objects.

We added a detailed explanation of this to the paper.

Reviewer comments: It is unclear how the product of multi-resolution segmentation is combined with MBI. Segmentation is scale-dependent, any issue if under- or over-segmentation?

Response: In the process of creating segmentation images, segment images were created with the aim of accurately creating segments for high-rise building rather than efficient segment generation on all objects on the ground, so the frequency of large over- or under- segments was low.

“The leftmost figure in Figure 2 is a building detection image generated based on the Otsu threshold in the MBI, where 1 is a pixel classified as a building, and 0 is a non-building pixel. This binary image is overlaid with the multi-resolution segmentation image, which is the middle, in Figure 2. The multi-resolution segmentation image has a value based on an object (blue line), not a pixel value (black line). As shown in the rightmost figure of Figure 2, which is the result of Majority voting, if the building pixel of an object in the segment image is more than half of the object area, the object is judged as a building object. Otherwise, the object is judged as a non-building object.”

Reviewer comments: Sources of reference data, any conflict with the digital map used in the detection? Do the reference data present only high-rise or buildings overall?

By design, what would be the final product, high-rise buildings with the correct shape? and also correct footprint, to be well aligned with the digital map, not the roof. If that's the case, the accuracy assessment needs to be an object-based assessment.

Response:

“To evaluate the accuracy of the proposed method, after classifying the HRBs polygon layer in the digital map, reference data for HRBs were generated based on this data. The reference data were generated for objects, which are HRBs, in satellite images, not in the form of footprint as in digital map, which includes both sides and roofs of the building.”

The amount of data between building objects and non-building objects in the study area is unbalanced. Therefore, we decided that it was right to proceed with the accuracy evaluation using the confusion matrix and the Kappa coefficient rather than other accuracy evaluation methods.

“The amount of data between building objects and non-building objects in the study area used in this study is imbalanced.”

Reviewer comments: There was no difference in compactness and shape of the 3 sites, why? and how you could come up with selected parameters. Likewise, the Otsu threshold is the same for 3 sites?

Response: Although they are different research areas, all research areas focused on high-rise buildings. Since objects other than high-rise buildings were not objects of interest, it was not difficult to segment high-rise objects properly even if the scale parameters related to the size of the object were set to the same default value.

“The focus was on HRB objects in generating segmentation images.

…

Setting all the same default values could lead to better results in segmenting HRB objects than setting different Shape and Compactness parameters for each study area.”

In the process of detecting buildings through MBI, the Otsu thresholds at each site were different and added.

“Thereafter, the Otsu threshold value in the MBI of each image was calculated, and the Otsu thresholds of each site were 0.1961, 0.4431 and 0.4549 in order. When pixels with values greater than or equal to the threshold value were classified as building pixels.”

Reviewer comments: Discussion and Conclusion sessions are quite overlapped. Please consider including Discussion with Results.

Response: As you advised, I combined the two chapters into one.

Author Response File: Author Response.docx

Reviewer 2 Report

The manuscript proposes high-rise building detection method from high resolution satellite image using object-based approach based on digital map. The paper is well written and organized. It also has fair coverage of related works where appropriate, alongside rigorous explanations of the supporting math for the algorithms.

In terms of enhancements I would suggest:
- The main contribution need to be emphasize and reformulate. From the reader point of view, why the method is proposed is not so clear.
- Please state clearly what kind of definition or criteria used by authors to categorize building as LRB or HRB according to Article 2 (1) 19 of the Building Act of the Republic of Korea. It will improve the understanding if there is a physical criteria, for example a building can be categorized as HRB if its height > 50m or has more than 3 level, etc.
- In the Method section, it would be better to add some description on the complexity analysis of the proposed method.
- It would be better add a discussion why are authors not try to use and compare with the current hot topic deep learning or other object-based segmentation.
- Figure 2 should be improved in terms of symbology (what is pixel 0 and 1 mean?) so that the reader will understand better the majority voting process. The difference of blue and bold blue grids is also not so clear.
- I suggest to reformulate sentence line 286-289 page 10. It is too long and too many information which may lead to confusion.
- Please provide MBI symbology (color index) in Figure 7 to inform the reader the meaning of the grey value means.
- Is there any specific reason to put equation number to some of equations in page 9? Otherwise, provide the number for each equation.

Author Response

Reviewer comments: The main contribution need to be emphasize and reformulate. From the reader point of view, why the method is proposed is not so clear.

Response: As you advised, we added the need for a proposal technique.

Reviewer comments: Please state clearly what kind of definition or criteria used by authors to categorize building as LRB or HRB according to Article 2 (1) 19 of the Building Act of the Republic of Korea. It will improve the understanding if there is a physical criteria, for example a building can be categorized as HRB if its height > 50m or has more than 3 level, etc.

Response: We added an explanation for this.

“In the building polygon layer provided by the digital map, only HRBs according to the Korean Building Act (i.e., the architecture with more than 30 floors or more than 120 meters high) were classified and used in this study.”

Reviewer comments: In the Method section, it would be better to add some description on the complexity analysis of the proposed method.

Figure 2 should be improved in terms of symbology (what is pixel 0 and 1 mean?) so that the reader will understand better the majority voting process. The difference of blue and bold blue grids is also not so clear.

Response: As you advised, I added a detailed description of the proposed technique.

“High-rise buildings (HRBs) as modern and visually unique land use continue to increase due to urbanization. Therefore, large-scale monitoring of HRB is very important for urban planning and environmental protection. This paper performed object-based HRB detection using high-resolution satellite image and digital map. Three study areas acquired from KOMPSAT-3A, KOMPSAT-3 and WorldView-3, and object-based HRB detection was performed using the direction according to relief displacement by satellite image. Object-based multi-resolution segmentation images were generated focusing on HRB in each satellite image, and then combined with pixel-based building detection results obtained from MBI through Majority voting to derive object-based building detection results. After that, to remove objects misdetected by HRB, the direction between HRB in the polygon layer of the digital map HRB and the HRB in the object-based building detection result was calculated. It was confirmed that the direction between the two calculated using the centroid coordinates of each building object converged with the azimuth angle of the satellite image, and results outside the error range were removed from the object-based HRB results. The HRBs in satellite images were defined as reference datas and the performance of the results obtained through the proposed method was analyzed. In addition, to evaluate the efficiency of the proposed technique, it was confirmed that the proposed method provides relatively good performance compared to the results of object-based HRB detection using shadows.”

“During this process, the central coordinates of detected HRBs in the satellite image and digital map were extracted and the direction between the two was calculated; then, the final object-based HRB detection result was derived by comparing it to the satellite's azimuth angle when the satellite image was taken. After detecting the same HRB in the digital map and satellite images, if the two buildings were the same building, the direction between the two converged with the direction of undulating displacement by the satellite's azimuth angle, otherwise removed from the detection results (Figure 1).”

. The leftmost figure in Figure 2 is a building detection image generated based on the Otsu threshold in the MBI, where 1 is a pixel classified as a building, and 0 is a non-building pixel. This binary image is overlaid with the multi-resolution segmentation image, which is the middle, in Figure 2. The multi-resolution segmentation image has a value based on an object (blue line), not a pixel value (black line). As shown in the rightmost figure of Figure 2, which is the result of Majority voting, if the building pixel of an object in the segment image is more than half of the object area, the object is judged as a building object. Otherwise, the object is judged as a non-building object.”

Reviewer comments It would be better add a discussion why are authors not try to use and compare with the current hot topic deep learning or other object-based segmentation.

Response: We added an explanation for this.

“Recently, deep learning based on artificial neural networks has been implemented, and various studies have been conducted to extract buildings from satellite images, thermal infrared images, and LiDAR data and recognize 3D building models [29]. Deep learning may be effective in detecting and restoring buildings using LiDAR data or thermal infrared images, but most of the analysis of images containing buildings with relief displacement uses fusion images or cross-sharpening [30,31]. Furthermore, HRB detection is a key study area, as studies show that small and complex HRB geometries can lead to increased spread of infectious diseases [32,33]. Therefore, this study attempted to propose a method of preferentially detecting HRBs with severe relief displacement.”

Reviewer comments: I suggest to reformulate sentence line 286-289 page 10. It is too long and too many information which may lead to confusion.

Response: As you advised, we have revised our explanation to be delivered more clearly.

“The focus was on HRB objects in generating segmentation images. When applying multi-resolution segmentation, the Scale parameters were selected by considering the size of the building objects located in the study area. And Shape and Compactness parameters, they were set to 0.1, 0.5, respectively, which are the default values when dividing multi-resolution segmentation (Table 3). Setting all the same default values could lead to better results in segmenting HRB objects than setting different Shape and Compactness parameters for each study area.”

Reviewer comments: Please provide MBI symbology (color index) in Figure 7 to inform the reader the meaning of the grey value means.

Response: We added an index for the MBI.

Reviewer comments: Is there any specific reason to put equation number to some of equations in page 9? Otherwise, provide the number for each equation.

Response: Thank you for your advice. Each number was provided for each equation.

Author Response File: Author Response.docx

Reviewer 3 Report

This manuscript presents a study on HRB detection by MBI with digital map. The authors performed object-based HRB detection using high-resolution satellite images and digital maps. The subject is timely for HRB detection, but it suffers from unclear or missing method details. And nothing else is given in the article to support the accuracy of the " HRB detection results by MBI with digital map", which is questionable.

Given these limitations and questions, I would recommend the manuscript with rejection.

MAJOR COMMENTS:

Method should be concise but informative. Much of the content is unnecessary and more details should be given on the detection of HRB using digital maps based on MBI.
The authors should make sure that the parts are connected to each other, and the current manuscript is somewhat scattered in terms of content.
The results in Table 4、Table 5、Table 6 are expected to give some content to support it.

MINOR COMMENTS：

The abstract section may contain a more obvious description of the innovation points. I only found a description of the contribution in lines 12-13, which I thought was inadequate.
The sentence in lines 11-12, “However, it remains challenging to routinely monitor large-scale HRBs due to their complex 3D structure and seasonal dynamic image capabilities. ”. I can’t get the meaning of this sentence. The description of ''complex 3D structure'' and ''seasonal dynamic'' do not appear in other parts of the article.
Equation (1) is suggested to be modified according to the format in reference [35].
The size of the font in formula (6) needs to be adjusted.
Suggest to add the description of Figure 2, More details can be added on how to obtain "Object-based building detection majority voting" from the overlay of "Pixel-based building detection by MBI" and "Multi-resolution segmentation image".
The size of the font in formula (12) needs to be adjusted.
In Figures 6,7, if there is no particular intention, keeping all three images of the same size is recommended.
In the conclusion, it would be better to link to the results of the “Evaluation criteria and Results” part, e.g. the F1 score.

Author Response

Reviewer comments: Method should be concise but informative. Much of the content is unnecessary and more details should be given on the detection of HRB using digital maps based on MBI.

The authors should make sure that the parts are connected to each other, and the current manuscript is somewhat scattered in terms of content.

The abstract section may contain a more obvious description of the innovation points. I only found a description of the contribution in lines 12-13, which I thought was inadequate.

The sentence in lines 11-12, “However, it remains challenging to routinely monitor large-scale HRBs due to their complex 3D structure and seasonal dynamic image capabilities. ”. I can’t get the meaning of this sentence. The description of ''complex 3D structure'' and ''seasonal dynamic'' do not appear in other parts of the article.

Response: As you advised, we added the need for a proposal method.

“. The leftmost figure in Figure 2 is a building detection image generated based on the Otsu threshold in the MBI, where 1 is a pixel classified as a building, and 0 is a non-building pixel. This binary image is overlaid with the multi-resolution segmentation image, which is the middle, in Figure 2. The multi-resolution segmentation image has a value based on an object (blue line), not a pixel value (black line). As shown in the rightmost figure of Figure 2, which is the result of Majority voting, if the building pixel of an object in the segment image is more than half of the object area, the object is judged as a building object. Otherwise, the object is judged as a non-building object.”

As you advised, we revised the introduction overall.

“High-rise buildings (HRBs) as modern and visually unique land use continue to increase due to urbanization. Therefore, large-scale monitoring of HRB is very important for urban planning and environmental protection. This paper performed object-based HRB detection using high-resolution satellite image and digital map. Three study areas acquired from KOMPSAT-3A, KOMPSAT-3 and WorldView-3, and object-based HRB detection was performed using the direction according to relief displacement by satellite image. Object-based multi-resolution segmentation images were generated focusing on HRB in each satellite image, and then combined with pixel-based building detection results obtained from MBI through Majority voting to derive object-based building detection results. After that, to remove objects misdetected by HRB, the direction between HRB in the polygon layer of the digital map HRB and the HRB in the object-based building detection result was calculated. It was confirmed that the direction between the two calculated using the centroid coordinates of each building object converged with the azimuth angle of the satellite image, and results outside the error range were removed from the object-based HRB results. The HRBs in satellite images were defined as reference data and the performance of the results obtained through the proposed method was analyzed. In addition, to evaluate the efficiency of the proposed technique, it was confirmed that the proposed method provides relatively good performance compared to the results of object-based HRB detection using shadows.”

Reviewer comments: The results in Table 4、Table 5、Table 6 are expected to give some content to support it.

In the conclusion, it would be better to link to the results of the “Evaluation criteria and Results” part, e.g. the F1 score.

Response: We revised the study results and the conclusion part overall and added explanations for the experimental results in the process.

Reviewer comments: Equation (1) is suggested to be modified according to the format in reference [35].

The size of the font in formula (6) needs to be adjusted.

The size of the font in formula (12) needs to be adjusted.

Response: We revised the size and font of the equation as you advised.

Reviewer comments: Suggest to add the description of Figure 2, More details can be added on how to obtain "Object-based building detection majority voting" from the overlay of "Pixel-based building detection by MBI" and "Multi-resolution segmentation image".

Response: We added an explanation for this.

Reviewer comments: In Figures 6,7, if there is no particular intention, keeping all three images of the same size is recommended.

Response: As you advised, we revised the overall picture size. Please check.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

Dear Authors,

Many thanks for your responses. I agree that relief displacement of HRB on optical images is the critical issue to be addressed. Also, take note of the limitation of the presenting methods due to the mismatch between image and map data.

Happy New Year 2022.

Author Response

Thank you very much for your careful review. Reflecting the reviewer's opinion as much as possible, we finally revise the paper and submit it again.

I wish you good luck and peace in 2022 and all your wishes come true.

Reviewer 2 Report

The manuscript is improved. I think it is ready for further step for publication.

Author Response

Thank you very much for your careful review. Reflecting the reviewer's opinion as much as possible, we finally revise the paper and submit it again.

I wish you good luck and peace in 2022 and all your wishes come true.

Reviewer 3 Report

Please check the paper carefully again.

Author Response

Thank you very much for your careful review. Reflecting the reviewer's opinion as much as possible, we finally revise the paper and submit it again.

I wish you good luck and peace in 2022 and all your wishes come true.