A Unified Framework for Asphalt Pavement Distress Evaluations Based on an Extreme Gradient Boosting Approach
Round 1
Reviewer 1 Report (Previous Reviewer 4)
Comments and Suggestions for AuthorsVery interesting study. Few minor comments prior publication.
- I do not completely understand the scheme of Figure 1a. What is the purpose of the accuracy analysis using 3 diferent techniques? Is this evaluated on the training data or on the test data?
- Figure 4. Is the colorbar fully representative?
- Page 15 line 378: Change Shap to SHAP.
- Conclusions or where does it fit. Any recommendations for real applications?
Author Response
Thank you so much for your time and consideration on our manuscript. We appreciate your comments, and we reply each and every comments in the attached word file, please find the attached file.
Author Response File: 
Author Response.pdf
Reviewer 2 Report (Previous Reviewer 2)
Comments and Suggestions for AuthorsThe manuscript has a title „A Unified Framework for Asphalt Pavement Distress Evaluation 2 Based on Extreme Gradient Boosting Approach“. The main aim of this study is to propose a system that will analyze the pavement performance, specifically rutting behavior analysis and cracks damage by using single XGboost algorithm.
It would be appropriate to expand the conclusions.
In chapter 4.3. you refer to figures 10 – 13. Figure 12 is not available in the manuscript.
It would be appropriate to describe figure 5, figure 11 and figure 13 more in the manuscript.
Author Response
Thank you so much for your time and consideration on our manuscript. We appreciate your comments, and we reply each and every comments in the attached word file, please find the attached file.
Author Response File: Author Response.pdf
Reviewer 3 Report (Previous Reviewer 1)
Comments and Suggestions for AuthorsThe document offers an advanced rutting deformation and crack damage prediction system based on the XGBoost model. The system relies on essential parameters such as traffic load, material characteristics, climatic conditions, and imaging information to foresee rutting behavior and classify the types of cracks accordingly. The XGBoost model proves to be highly efficient, having an R² measure of 0.9 in rutting behavior prediction, accompanied by impressive measures of precision, recall, accuracy, and F1-score in classifying cracks.
- The reliability and accuracy of the XGBoost model are highly reliant upon the nature and availability of the data used. If the training data used to train the model are limited or biased, the model will likely generate wrong predictions and lower the replicability of the findings.
- While this research does assess the XGBoost model against other artificial intelligence techniques to some degree, the comparison algorithms chosen and the measures used to assess them can truly affect how the efficacy of the model is judged. Comparison using a larger selection of models and differing datasets can create a more complex understanding of the specific strengths and limitations surrounding the XGBoost model.
- The study achieves high R² and accuracy (around 0.9); however, it does not address the possibility that the model will suffer from overfitting due to the model's complexity.
- The study does not provide details about the dataset size, geographic distribution, or potential biases in the data collection process.
- The study identifies key variables (e.g., Truck Volume, ESAL, Temperature, Resilient Modulus, etc.) through SHAP analysis, but it does not clarify whether any feature selection techniques were used before training the model.
- The study provides SHAP analysis, which is useful for model interpretability, but it does not discuss how the model could be practically deployed in real-world pavement maintenance systems.
- The study does not analyze how sensitive the model predictions are to variations in input features.
- The study does not cover the computational efficiency of XGBoost compared to other models, an important factor to be considered upon the system's integration within large-scale networks of pavements.
- The study mentions environmental factors, but it does not explicitly discuss how climate change or extreme weather conditions may impact pavement deterioration.
Author Response
Thank you so much for your time and consideration on our manuscript. We appreciate your comments, and we reply each and every comments in the attached word file, please find the attached file.
Author Response File: Author Response.pdf
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsWhile it introduces a promising framework of intelligent forecasting of rutting deformation and cracks damage in flexible pavements using an Xtreme Gradient Boosting-XGBoost-algorithm, its following disadvantages and weak points can be considered:
1. Besides, the possible bias and overfitting problems were not considered, which may inflate the performance metrics of the models. Without considering these issues, the predicting results from the model might not be as reliable or generalizable.
2. In fact, discussions of the paper just focus on the demonstration that the performance of the XGBoost model is superior compared to the considered alternative algorithms. It could be very important for detailed comparison and profound insight into strengths and weaknesses related to each of these algorithms.
3. Although it refers to the SHAP analysis for interpretation, the paper did not consider the limitation of low interpretability for an XGBoost model because of its black box. Thus, going deeply into the factors which drive the predictions will not be presented.
4. All the possible limitations concerning the performance of the XGboost model in the task of rutting deformation and crack damage prediction were not sufficiently discussed. Moreover, the knowledge about the potential constraints and in which scenarios a model may result in poor outcomes should represent one part of the practical use for the model.
5. While the study shows that the proposed system for pavement distress prediction works effectively, it does not account for how practically feasible the system will be in real-time pavement maintenance. Several factors include data collection, integration, and maintenance requirements that have to be met for successful deployment.
6. The study has exclusively focused on flexible pavements. The result and model derived from this cannot be replicated for other categories of pavements, say rigid pavements, or other infrastructure with the same approximation.
7. The important parameters (e.g., Truck Volume, ESAL, AC Thickness, Temperature Condition, and Resilient Modulus) considered here may not be practically available every time in field conditions, hence limiting the real-world applicability of the model.
8. Though the study has mentioned that XGBoost outperformed "alternative algorithms," types of alternative algorithms are not mentioned, nor does the text specify how those were implemented or if there was optimization of their hyperparameters so that the comparison can be fair enough.
9. Some of the critical parameters such as moisture content and MRI have been identified to have less importance in this particular study. This might go against general conventional engineering belief and practices where these parameters generally show influence in pavement performance analysis.
Comments on the Quality of English LanguageThe English could be improved to more clearly express the research.
Author Response
we uploaded the whole comments response of respected reviewer in the attached pdf file
Author Response File: Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsThe article is on the topic A Unified Framework for Asphalt Pavement Distress Evaluation Based on Xtreme Gradient Boosting Approach. This manuscript presents a framework of intelligent prediction system by utilizing Xtreme Gradient Boosting (XGboost) to predict two relevant functional indices: rutting deformation and cracks damage.
The manucsript deals with the prediction of rutting and cracks on the road surface. In the introduction, the issue should be made more concrete, when it would be appropriate to state, for example, at what temperatures these phenomena usually occur (ruts - from what temperature the road deformation occurs = rutting, the same applies to failure by cracks, the assumption of the number of freeze-thaw cycles before failure occurs). There are also certainly some basic road surfaces and their composition. This is not stated here, only the dynamic modulus and modulus of elasticity are mentioned.
It might be appropriate to state some distribution of the surface type depending on the typical range of the dynamic modulus, the modulus of elasticity. The basic percentage composition of individual components for typical dynamic modulus and modulus of elasticity of roads. Some basic acquaintance with the investigated surfaces in the given monitored area.
How can these phenomena be prevented? By changing the raw material composition of the road? Which road surface components can influence the formation of cracks or ruts?
In the winter months, when chemical road gritting occurs, the road surface is more disturbed. Is the chemical treatment of the road surface and its effect on the formation of cracks taken into account?
Different types of damage occur in different seasons. In the summer months, ruts is mainly caused by heavy vehicles such as trucks and exposure to high temperatures. This suggests that this is also influenced by the duration of the temperatures at which the given type of damage occurs. The longer the road is exposed to high/low temperatures, the more pronounced the changes, whether rutting or cracks. Also, with some temperature dispersion, significant changes/damage do not occur. Is the influence of the duration/repetition of temperatures affecting the deformation of the road taken into account? The formation and width of cracks and their subsequent spread are also related to this.
Rutting can also be influenced by other layers below the top asphalt layer, where the composition of the individual layers can be different. The analysis takes this into account. What were the usual road compositions? What layers occur most frequently here. In what type of road structure do the largest ruts appear on the road surface?
It would be better to describe Fig. 5. Figure 5 contains 3 regression models marked a), b) and c), where based on the value of R2 – the coefficient of determination, it can be assumed that there is a very good agreement (the coefficient of determination ranges from approximately 0.85-0.9). It would also be appropriate to describe the x and y axes on the graphs in Figure 5. It is not entirely clear from what types of variable values ​​the regression analysis was performed. Only the predicted and actual values ​​are given here. What are these values? Species?
It would be appropriate to state in the conclusions:
- Under what conditions does the greatest influence occur on the damage to the road surface – the failure of cracking - Specific values ​​of temperatures, thicknesses, modulus of elasticity, dynamic modulus, etc., when the worst results occur in terms of assessing crack formation.
- under what conditions does the greatest influence occur on the damage to the road surface – the failure of rutting. Specific values ​​of temperatures, thicknesses, modulus of elasticity, dynamic modulus, etc., when the worst results occur in terms of assessing ruts.
- Determination of situations in which damage/failure did not occur at all or, for example, to a very small extent. Again, some values ​​of the monitored quantities. At what temperature types of road surface etc.
The conclusions could also include a suitable solution for road rehabilitation/repair for a given type of damage.
In what time period were the parameters temperature, traffic load/frequency monitored. Was the development and width of cracks on the road monitored in a given time period, or the depth of ruts? What values ​​were measured?
What was the monitored thickness of the road surface. Some kind of graph showing the number/amount of occurrence of the given road surface. Graphically describe all input parameters such as, for example, the dynamic modulus and the modulus of elasticity.
It would also be appropriate to evaluate basic statistical quantities.
Perhaps this would require some kind of graphical evaluation of the development of temperatures in a given time period or a graphical evaluation of the frequency of traffic in the monitored period. For example, graphically in the form of a histogram?
Comments on the Quality of English LanguageEnglish can be improved.
Author Response
We uploaded the response of respected reviewer in the the attached pdf file. Thank you.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsThank you to the authors for submitting this paper. In this paper, the authors use XGBoost to predict rutting from LTPP data and cracking from a single image dataset. However, both tasks have been done before; additionally, there are major issues with the paper. Please find below my comments:
1. Lines 46-47 - Which "mechanistic-empirical" methods are used for inspection?
2. Figure 1 is not readable.
3. Second Figure 1 - why is this a "heatmap"? Just show how many missing values in each category.
4. Lines 241-243 - The data cleaning process needs much more explanation.
5. Several studies have shown that using images from the same dataset to test and train will lead to artificially good results but the model falls apart when used on new data. Is this not the case for the data herein only trained on a single Kaggle dataset?
6. Lines 272-273 - It is not obvious that all variables are positively associated with each other because they are not.
7. Lines 273-275 - There is not a "moderate" relationship between dynamic modulus and GESAL or truck volume at 0.22, this is not a strong relationship at all.
8. There is no explanation of what the point of the correlation matrix actually is or how it was used to select variables since both of the redundant variables were used anyway.
9. The authors are not clear which type of structures from LTPP they are actually considering.
10. Figure 5 - was all data for training and there is no testing data?
11. Figure 8 - What is the explanation for several truck volumes affecting the rutting in the opposite of the expected way?
12. Figure 10 - The accuracy looks terrible in this image- how can it be over 80%?
13. Overall, the research has two different directions. The rutting prediction is more for forecasting and the cracking is more for inspection. Why would these two be put together in the same paper?
Comments on the Quality of English LanguageThere are several English grammar mistakes which must be corrected.
Author Response
Please find the attached pdf file , which has detailed reviews of each comments of respected reviewers.
Author Response File: Author Response.pdf
Reviewer 4 Report
Comments and Suggestions for AuthorsThis is a sound and well organized study which is worth to be considered for publication. However, the manuscript requires some important corrections and clarifications prior publication. The main issue is that it is not clear from the analysis if the ML algorithms have been trained and tested to different samples of the existing dataset and if a cross validation has been performed. The authors should either clarify or add the missing part of the analysis. Additionally:
- Figure 1: what is the message of this figure? What is the colormap representing?
- Figure 5: I do not understand the legends of the figure. The predicted vs actual values are defined by the two axes. What are then the two different markers representing? Are these the data on the entire dataset, training dataset, test dataset?
- Figure 10: Please see the comment above. Additionally, what are the black markers? Are the high values of the accuracy representative of the scatter that the predicted values are demonstrating?
Author Response
Please find the attached pdf file of the respected reviewer comments response.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsAccept in present form
Comments on the Quality of English LanguageThe English could be improved to more clearly express the research.
Author Response
Thank you once again for your support and for contributing to the improvement of our study. Regarding English improvement we make some correction in the updated version of manuscript , and also response of each changes in the word file.
Please find the attached file.
Thank you.
Author Response File: Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsThank you for the previous answers.
You state that the data is drawn from a database. Are the specific values ​​that were used for the given analysis known (open access data)? Which is also evident from Figure 5. It is therefore possible to use these values/data and perform an analysis on these data (it is not necessarily necessary to conduct new experiments, but it is also possible to use already measured data from the said database). It is possible to indicate the average, standard deviation, variance, coefficient of variation and other statistical data based on statistical analysis. Display the monitored parameters that enter the analysis using a graph called Histograms, where the frequency of some values ​​in the given data set will also be evident. A histogram is a graphical evaluation, where the monitored variable is located on the x-axis (for example, temperature, modulus of elasticity, etc.) and the frequency (relative or absolute) is on the y-axis. This is a column chart.
The article would be more readable and understandable if the article also consists of the analysis of the input data/data set itself. Although it is not experimental data performed by you, the data can be used and evaluated more.
For Figure 6, the description of the figure needs to be expanded, for example
Figure 6. Comparative analysis of actual and expected values ​​of the proposed models: a) xxxxxx, b)xxxxxx, c)xxxxxx, d)xxxxxxx.
Can the parameters and their specific dispersion of values ​​be evaluated based on the data and image analysis - at which a given type of failure occurs most often and when the failure occurs to a limited extent? For example, at which combinations of given values ​​(modulus of elasticity, thickness of the road surface, temperature, load, etc.) does failure most often occur (i.e. failure due to cracks most often occurs at a temperature range of -10 to -15, thickness of the road surface XX mm, modulus of elasticity xxx MPa. These conclusions would also be very interesting if it is possible to determine based on the given study.
What is the usual material composition of the top layer of the road surface - the recipe of the top layer of the road surface - in the monitored area. For roads, the composition is usually different than for sidewalks/cycle paths, when in the case of the sidewalk it is assumed that it will not be exposed to as much stress as the roads (car load).
Comments on the Quality of English LanguageEnglish can be improved.
Author Response
Thank you once again for your support and for contributing to the improvement of our study. Please find the response of each comments in the attached word file. The comments are valuable to us, we tried our best to improved our manuscript with these comments
Author Response File: Author Response.pdf
Reviewer 4 Report
Comments and Suggestions for AuthorsNo further comments.
Author Response
We would like to express our sincere gratitude for the thoughtful evaluation and positive feedback on our manuscript. We greatly appreciate the time and effort dedicated to reviewing our work. Your constructive assessment is invaluable to us, and we are encouraged by your recognition of the quality of our research. Thank you once again for your support and for contributing to the improvement of our study.
