Optimising Floor Plan Extraction: Applying DBSCAN and K-Means in Point Cloud Analysis of Valencia Cathedral

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The paper is entitled: ‘Automating the Geometric Documentation of Historical Buildings: Applying Machine Learning to Point Cloud Analysis of the Cathedral of Valencia’, but there are no traces of machine learning models in the paper. The methodology used by the researchers is mainly the application of two well-known automatic clustering methods DBSCAN and K-Means (Voronoy-based). Therefore, the title of the paper must be changed and all sentences on AI-ML-DL must be deleted, improving the description of the clustering algorithms adopted. Some sections need to be replaced as in the following pdf. Please specify how many times it cost to manually draw the same plan and compare it with the semi-automatic process presented in the document. For these reasons I will reject this title and suggest the authors resubmit the application with a new title and the requested changes. I know there is a lot of work behind this kind of experiments, I hope my review enhances this great work on the 3D laser scanned replica of the Chatedral of Valencia.

Comments for author File: Comments.pdf

Comments on the Quality of English Language

English needs minor revisions.

Author Response

Comments 1: The paper is entitled: ‘Automating the Geometric Documentation of Historical Buildings: Applying Machine Learning to Point Cloud Analysis of the Cathedral of Valencia’, but there are no traces of machine learning models in the paper. The methodology used by the researchers is mainly the application of two well-known automatic clustering methods DBSCAN and K-Means (Voronoy-based). Therefore, the title of the paper must be changed and all sentences on AI-ML-DL must be deleted, improving the description of the clustering algorithms adopted. Some sections need to be replaced as in the following pdf. Please specify how many times it cost to manually draw the same plan and compare it with the semi-automatic process presented in the document. For these reasons I will reject this title and suggest the authors resubmit the application with a new title and the requested changes. I know there is a lot of work behind this kind of experiments, I hope my review enhances this great work on the 3D laser scanned replica of the Chatedral of Valencia.

Response 1: We agree with this comment. Therefore, we have made the following changes: As noted, the article originally had a title and certain approaches that were not applicable, so the title has been modified, and sections that were not relevant have been removed. A "Background and Related Works" section has been added to provide context for the research and highlight the differences compared to previous studies. Additionally, a comparison has been included between the time it takes to manually draw the floor plan and the time required using the automated process. Finally, the observations from the attached file have been addressed.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The topic covered in this contribution certainly presents interesting developments in the segmentation of 3D data obtained from reality, applying ML techniques and using DL algorithms, to obtain consequent extraction and production (vectorization) of contour lines that represent the plans of a historical building, such as the case study presented: the cathedral of Valencia.

Although the methodological approach and the procedure developed are correct, some points of attention and possible further development of the defined methodology are suggested.

Historical buildings (in particular a Gothic cathedral) present architectural characteristics (base details of columns, moldings, etc.) in the definition of the masonry equipment, with dimensional differences/deviations on the X,Y plane even of tens of centimeters, which make the sample of data extracted and analyzed from the point cloud included between the pavement plane and the level at + 2.00 meters (see Sections 2.2 and 2.3) characterized by a notable dispersion, which could be confused with noise in the acquired data, losing instead important information for the correct and accurate reproduction of the shape of the plan that is intended to be represented.

A solution to this problem could be to use as a cutting section of the point cloud (see Sec 2.3, lines 193-202) - jointly with the point cloud of the entire building (the one used in the case study) - a scan with a higher density in a horizontal band such as to intersect only the surface of the smooth walls, at the level at which the plan section is to be determined.

As regards the evaluation of the results, in addition to the useful comparison made between those obtained with the two algorithms (K-means and DBSCAN), it would be useful to compare them with the plan restitution of a survey carried out with traditional tools and techniques and produced not in a semi- or automatic way, but 'hand-made'. This comparison would allow us to qualitatively and quantitatively evaluate the automatic 'vectorization' results. Due to the complexity and size of the Cathedral, this analysis and comparison could be conducted - as a first analysis - on a portion/part of the case study, with similar and comparable architectural characteristics, but more limited in terms of size.

Among the possible future developments, it would be interesting to have some prediction or estimate of what the limits, critical issues, and possible applications could be in the case of a vertical section of a building.

Author Response

Comments 1: The topic covered in this contribution certainly presents interesting developments in the segmentation of 3D data obtained from reality, applying ML techniques and using DL algorithms, to obtain consequent extraction and production (vectorization) of contour lines that represent the plans of a historical building, such as the case study presented: the cathedral of Valencia.

Although the methodological approach and the procedure developed are correct, some points of attention and possible further development of the defined methodology are suggested.

Historical buildings (in particular a Gothic cathedral) present architectural characteristics (base details of columns, moldings, etc.) in the definition of the masonry equipment, with dimensional differences/deviations on the X,Y plane even of tens of centimeters, which make the sample of data extracted and analyzed from the point cloud included between the pavement plane and the level at + 2.00 meters (see Sections 2.2 and 2.3) characterized by a notable dispersion, which could be confused with noise in the acquired data, losing instead important information for the correct and accurate reproduction of the shape of the plan that is intended to be represented.

Response 1: According to this comment, we have highlighted this issue in the limitations section at the end of the article. During the experimentation, we sectioned the Cathedral in the XZ plane at a height of 2.79 m above the pavement. The pavement of the Cathedral is at Z = 14.36, and we performed the cut at Z = 17.15, which is at a height with fewer noise points. As mentioned, it is essential to have a very clean point cloud to avoid errors in point joining. However, if we excessively apply noise filtering, such as SOR, we risk losing density in the point cloud and, consequently, precision in the generated plan.

Comments 2: A solution to this problem could be to use as a cutting section of the point cloud (see Sec 2.3, lines 193-202) - jointly with the point cloud of the entire building (the one used in the case study) - a scan with a higher density in a horizontal band such as to intersect only the surface of the smooth walls, at the level at which the plan section is to be determined.

Response 2: In relation to this comment, you are correct. The issue with using a denser point cloud is that it can lead to errors in point joining, as the lines may intersect with one another. The ideal scenario would be to use a higher density of points in more complex areas where greater detail is needed, while reducing the number of points in more orthogonal areas where less information is required. This would allow for process optimization without compromising the accuracy of the final result.

Comments 3: As regards the evaluation of the results, in addition to the useful comparison made between those obtained with the two algorithms (K-means and DBSCAN), it would be useful to compare them with the plan restitution of a survey carried out with traditional tools and techniques and produced not in a semi- or automatic way, but 'hand-made'. This comparison would allow us to qualitatively and quantitatively evaluate the automatic 'vectorization' results. Due to the complexity and size of the Cathedral, this analysis and comparison could be conducted - as a first analysis - on a portion/part of the case study, with similar and comparable architectural characteristics, but more limited in terms of size.

Response 3: In the results, a comparison was made between this process and a manual method to quantitatively assess the execution times of both approaches.

Comments 4: Among the possible future developments, it would be interesting to have some prediction or estimate of what the limits, critical issues, and possible applications could be in the case of a vertical section of a building.

Response 4: The aim of this method is to apply it to as many plans as possible of a building to automatically generate 3D BIM models through the union of polygons.

Reviewer 3 Report

Comments and Suggestions for Authors

The manuscript compares the DBSCAN and k-means algorithms for automated extraction of floor plans from point cloud data.

** General comments **
1) The manuscript is missing a related works section. The lines 52-74 hint in this direction, but it would strengthen the manuscript if there was a dedicated 'related works' section. Such a section will also aid in the justification of the research and highlighting the novelty of the research.
l. 52 'Several researchers' which researchers? what did they do? add appropriate references.
l. 63 'remains limited' this statement needs justification. Mention the relevant existing literature, and show that is actually limited. Same holds for l. 73 'limitations of existing'
l. 66 'this research stands out' likewise, justify this statement by providing a good overview of existing literature.
l. 68 'unlike previous studies' which studies are referred to? briefly describe them and add references.

2) The manuscript is missing validation.
Section 4, discussion of results is very subjective. A comparison with a ground truth is missing. I understand that making a floor plan of the entire cathedral by hand is too much, therefore I would suggest to make a floor plan of the chapel of the holy chalice manually using the traditional way of working. This manual floor plan can act as ground truth. This way an objective comparison can be made between the two automated algorithms.
l. 276 'more accurate' this is very vague. It needs to be quantified.
l. 279 'improved in precision' quantify this statement.
l. 256 'greater accuracy' how is this accuracy defined? Be specific.

Suggestion: When creating the floor plan manually, also measure the time it requires to make. This time can then be used to justify a statement such as in l. 286 'highly efficient'.


** Specific comments **
title suggestion: ...: machine learning based point cloud analysis of ...
l. 11 The word 'integrating' can be removed
l. 15 'floors and converting points into 2D section lines' -> floor plans
l. 17 'reducing point distances to optimise file management' -> culling points to reduce file size
l. 18 'automated floor analysis' -> automated floor plan extraction
l. 27 'incorporated' -> adopted
l. 30 'potential issues' give a few examples of these issues
l. 33 a large volume -> large volumes
l. 42 'alleviate' strange choice of word. 'filter it' -> be more specific with what is filtered and how
l. 46 'years has' - > years have
l. 46 facilitated -> automated
l. 83 and l. 84 omit 'beautiful' and 'splendid' these words are better suited for a touristic information pamphlet.
l. 87 'as a unit' -> 'it as a unit'
l. 92 'sought, promising for' -> 'explored, with the goal of '
l. 98 'to evaluate' -> 'to generate'
l. 107 add a bit of introductory text between 'Materials and methods' and 'data acquisition'
l. 112 'massive' can be omitted
l. 119 'stations' not very clear. Do you mean 'scans' ?
l. 119 'general resolution of 1/8' what is meant with this statement?. Does it have a unit?
l. 125 'georeferenced' mention which CRS is used
l. 137 'minimise the point spacing' Point spacing is not minimised. Suggestion: 'adjust the point spacing'
Table 1: Add a row corresponding to the values where NO filtering was applied, i.e. the original data
Figure 1: I think it is nice to add a small zoomed in picture of the point cloud (e.g. window detail). There is some empty space in the lower right corner.
l. 154 'with information on colours' -> with colour information
l. 154 'average calculation' it is unclear what is meant with this.
Figure 2 and Figure 3, I think they can be combined. Figure 3 is a nice one to keep.
Figure 2: an odd number of bins (13) was used for a 10m span for the histogram, I would suggest to use 10 bins for a 10m span. Now the range given in e.g. l. 170 looks very strange
l. 174 'homogenised' not an appropriate word to use. suggestion: Finally, the points are projected on the Z -plane (Figure 4). Also see line 200. Or instead of projected something like 'flattened' ?
l. 206 'kd-tree algorithm'. As far as I know a kd-tree is a data structure. I am not sure what is visualised here in Figure 6b. Please elaborate.
l. 240/241 a range of cluster sizes is tested. I would suggest to look into a more objective method, very classic is the 'elbow' method. But perhaps there are better methods out there.
l. 256 'radius (e) value' -> 'radius(e) value of the DBSCAN algorithm'

Comments on the Quality of English Language

Regarding the writing style: there is sometimes redundant information is adjacent sentences.

Author Response

Comments 1: The manuscript is missing a related works section. The lines 52-74 hint in this direction, but it would strengthen the manuscript if there was a dedicated 'related works' section. Such a section will also aid in the justification of the research and highlighting the novelty of the research.
l. 52 'Several researchers' which researchers? what did they do? add appropriate references.
l. 63 'remains limited' this statement needs justification. Mention the relevant existing literature, and show that is actually limited. Same holds for l. 73 'limitations of existing'
l. 66 'this research stands out' likewise, justify this statement by providing a good overview of existing literature.
l. 68 'unlike previous studies' which studies are referred to? briefly describe them and add references.

Response 1: Thank you for your valuable feedback regarding the manuscript. I appreciate your suggestion to include a dedicated ‘Background and Related Works ' section, as it will enhance the context of the research and emphasize its novelty. I will incorporate this section to provide a comprehensive overview of existing literature, addressing the contributions of relevant researchers and clarifying how their work relates to our study.

Comments 2: The manuscript is missing validation.
Section 4, discussion of results is very subjective. A comparison with a ground truth is missing. I understand that making a floor plan of the entire cathedral by hand is too much, therefore I would suggest to make a floor plan of the chapel of the holy chalice manually using the traditional way of working. This manual floor plan can act as ground truth. This way an objective comparison can be made between the two automated algorithms.
l. 276 'more accurate' this is very vague. It needs to be quantified.
l. 279 'improved in precision' quantify this statement.
l. 256 'greater accuracy' how is this accuracy defined? Be specific.

Response 1: A methodology has been introduced to determine the most optimal way to utilize the DBSCAN and K-means algorithms. The results include an analysis of the margin of error between both methodologies, focusing on the fidelity between the generated lines and the original points.

Each specific comment has been taken into account, and the necessary revisions have been made. We sincerely appreciate your review efforts.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The new document has improved based on the work of the authors and the input of the reviewers. The overall assessment from my point of view has improved. No other problems were found and no further requirements are necessary.

Author Response

ok

Reviewer 3 Report

Comments and Suggestions for Authors

The manuscript has improved significantly. Well done.

Some minor points:
The title mentions 'optimising', but does not stat *what* it optimises. Accuracy? Or time? Line 105 states that it optimises the time.

L41: omit the '(SfM)' SfM is an application of photogrammetry. They are not synonymous. Also the term might not be known by everyone in the target audience.

L107: to unlabelled -> on unlabelled

Table 1. I hade some reservations about the '<2' it is a bit confusing. I presume it referes to the original data, hence I would suggest to replace it with a dash '-' and mention in the text that the first row corresponds to the original data.

Section 2.3.1 on line 227 and section 2.3.2 on line 255 are not consistent. I think they should be 3.3.1 and 3.3.2

Just a suggestion for line 292: ..... creating the 2D floor plan manually using .... floor plan, we connected the points by hand ....
This way it become more clear that it is a manual process.

L305: 'average error' What kind of error is this? Is it an absolute error or RMSE or something else. Mention explicitly which error is meant here.

L318: 'optimised as possible' make explicit what is optimised.

Suggestion for line 339: 'data analysis' -> 'clustering'

In the conclusion mention the results from lines 294-295, where you reduce the time from 4.5 ours to 50 seconds. This is an important piece of information.

Comments on the Quality of English Language

None.

Author Response

We would like to express our gratitude for the corrections made. We have addressed all the requested observations to enhance clarity and understanding.

Comments 1: The title mentions 'optimising', but does not state what it optimises. Accuracy? Or time? Line 105 states that it optimises the time.
Response 1: The workflow aims to optimise both time and accuracy. Therefore, revisions have been made to lines 105-106 to clarify this point.

Comments 2: Table 1. I had some reservations about the '<2'; it is a bit confusing. I presume it refers to the original data, hence I would suggest replacing it with a dash '-' and mentioning in the text that the first row corresponds to the original data.
Response 2: This has been modified as suggested.

Comments 3: L305: 'average error' What kind of error is this? Is it an absolute error or RMSE or something else? Mention explicitly which error is meant here.
Response 3: The clarification has been incorporated in line 308 to specify the type of error.

Comments 4: L318: 'optimised as possible' make explicit what is optimised.
Response 4: The necessary modifications have been incorporated in line 320-321 to specify what is being optimised.

Comments 5: In the conclusion, mention the results from lines 294-295, where you reduce the time from 4.5 hours to 50 seconds. This is an important piece of information.
Response 5: This information has been incorporated into line 341 of the conclusión.