PDS-UAV: A Deep Learning-Based Pothole Detection System Using Unmanned Aerial Vehicle Images

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Your paper tackles a highly relevant and innovative issue by leveraging advanced technology to address the real-world problem of pothole detection. I really appreciate how you've approached this topic. However, I believe there are a few areas where you could deepen the impact of your work:

1. Literature Review Enhancements: It's great that you've covered various pothole detection methods. Yet, I think you could strengthen your paper by directly comparing these methods with your PDS-UAV system. Highlighting what sets your system apart from existing solutions could really underscore the advancements you're bringing to the field.

2. Addressing System Limitations: You've touched on future enhancements, which is fantastic. However, delving into the current system's limitations could enrich your discussion. Issues like dependency on weather conditions, UAV battery life, and the necessity for high-resolution imaging are critical and acknowledging these challenges would not only provide a more balanced perspective but also point out fruitful directions for future research.

3. Impact on Stakeholders: I'd love to see more about how different stakeholders, such as city planners, road maintenance teams, and daily commuters, might benefit from the PDS-UAV system. Discussing how this system could integrate with city traffic management and emergency response services would make the practical applications of your work even clearer.

4. Practical Implications in the Conclusion: While you've outlined the theoretical benefits of the system, expanding on the practical implications in your conclusion could really drive the message home. A more quantitative discussion on how the system could reduce road maintenance costs and enhance traffic safety would give your conclusions more weight.

By addressing these points, I believe your paper could make an even more significant contribution to the fields of urban infrastructure and smart city solutions. Keep up the fantastic work; you're onto something really impactful here!

Comments on the Quality of English Language

Minor editing of English language required

Author Response

We would like to thank the reviewer for the valuable comments and the time and effort made to review the manuscript. The comments have significantly improved the quality and presentation of the manuscript.

Reviewer 1 Comments

Your paper tackles a highly relevant and innovative issue by leveraging advanced technology to address the real-world problem of pothole detection. I really appreciate how you've approached this topic. However, I believe there are a few areas where you could deepen the impact of your work:

1. Comment: Literature Review Enhancements: It's great that you've covered various pothole detection methods. Yet, I think you could strengthen your paper by directly comparing these methods with your PDS-UAV system. Highlighting what sets your system apart from existing solutions could really underscore the advancements you're bringing to the field.

Response: The following paragraph has been added to the end of the related work section, on page 5, to highlight the differences between the existing solutions and the proposed system PDS-UAV:

“In summary, the reviewed pothole detection approaches utilize machine learning or deep learning for pothole detection from either UAV images or smartphones and in-vehicle cameras. The proposed system, PDS-UAV, advances pothole detection using a State-of-the-art deep learning model, particularly YOLOv8, for accurate identification from UAV images. Yolov8 architecture includes multiple convolutional layers that automatically identify relevant features such as edges, textures, and shapes associated with potholes. It is known for its speed and accuracy, crucial for timely maintenance and safety precautions. Using cross-city testing, the ability of the proposed system model to be generalized across domains is demonstrated. This highlights the model's high accuracy on datasets from different regions, setting it apart from prior approaches that typically focused on localized datasets. Additionally, the proposed system includes a web-based application for monitoring, reporting, and viewing detected potholes, helping to streamline the process for road maintenance employees and road users. Thus, PDS-UAV not only enhances detection accuracy, but also offers significant operational benefits by automating data collection and analysis, reducing the need for manual inspections, and increasing overall road safety for drivers and passengers.

2. Comment: Addressing System Limitations: You've touched on future enhancements, which is fantastic. However, delving into the current system's limitations could enrich your discussion. Issues like dependency on weather conditions, UAV battery life, and the necessity for high-resolution imaging are critical and acknowledging these challenges would not only provide a more balanced perspective but also point out fruitful directions for future research.

Response: Thank you for your comment. The following paragraph has been updated and improved in Section 7 (Discussion) to discuss the mentioned limitations as highlighted on page 28:

“Despite the promising results of the proposed PDS-UAV system, several limitations can be highlighted. First, the performance of the YOLOv8 model relies heavily on the quality and diversity of the training dataset. In addition, potholes can vary depending on regions, road surfaces, lighting conditions, and weather, impacting the model's detection accuracy. Second, While there are many benefits to using UAVs for data collection, there are also shortcomings. Deploying UAVs for pothole detection faces operational limitations such as battery life, which could result in restricted flight length (usually 30-40 minutes). In addition, the effects of bad weather conditions, such as heavy rain, strong winds, or low visibility, can limit UAV operations. These UAV-related variables may impact the dependability and consistency of data gathering across long periods or wide geographic areas. Third, the use of UAVs in urban contexts is restricted by stringent regulatory frameworks designed to ensure safety and privacy, making it time-consuming to get appropriate approvals and manage the danger of accidents with other airborne objects or structures. Addressing these limitations through future research and development will be crucial for enhancing the robustness, scalability, and practical applicability of the PDS-UAV system in diverse urban environments.”

1. Comment: Impact on Stakeholders: I'd love to see more about how different stakeholders, such as city planners, road maintenance teams, and daily commuters, might benefit from the PDS-UAV system. Discussing how this system could integrate with city traffic management and emergency response services would make the practical applications of your work even clearer.

Response: Thank you for your comment. The following paragraph was added to the discussion section to highlight the impact of the system on stakeholders as highlighted on page 28:

“Automating pothole identification using UAV images has a significant impact on infrastructure management, transportation efficiency, and road safety, benefiting various stakeholders, including road users, road maintenance employees, and city planners. For road users, such as drivers, bikers, and daily commuters, the technology offers safer routes by providing information on pothole locations, allowing them to avoid hazardous areas and reducing the risk of accidents and vehicle damage. Maintenance employees gain from enhanced safety due to reduced exposure to traffic and the elimination of manual inspections, which also cuts labor costs and allows for more efficient repairs. The system’s continuous monitoring capabilities ensure proactive maintenance, preventing costly road damage and enabling timely interventions. For city planners, the PDS-UAV system provides valuable data to prioritize road repair projects and allocate resources efficiently. Its integration with traffic management and emergency services enhances urban mobility and safety by maintaining critical routes in good condition. Furthermore, the precise data supports informed decision-making and smart city initiatives, contributing to better infrastructure management and environmental sustainability. Utilizing an advanced detection algorithm, YOLOv8, enhances the system’s reliability and accuracy, making it an essential tool for improving road maintenance and ensuring safer and more efficient transportation networks.”

4. Comment: Practical Implications in the Conclusion: While you've outlined the theoretical benefits of the system, expanding on the practical implications in your conclusion could really drive the message home. A more quantitative discussion on how the system could reduce road maintenance costs and enhance traffic safety would give your conclusions more weight.

Response: Thank you for your comment. The following sentences were added to illustrate the practical implications of the system in the conclusion section as highlighted on page 29:

“In practice, the proposed PDS-UAV system can significantly reduce road maintenance costs by automating detection and monitoring processes, thereby decreasing the need for frequent and risky manual inspections. Furthermore, the system's ability to deliver timely information to drivers about pothole locations improves traffic safety, potentially reducing accidents caused by hazardous road conditions.”

By addressing these points, I believe your paper could make an even more significant contribution to the fields of urban infrastructure and smart city solutions. Keep up the fantastic work; you're onto something really impactful here!

 

Reviewer 2 Report

Comments and Suggestions for Authors

A Pothole Detection System utilizing Unmanned Aerial Vehicles, 5 called PDS-UAV, is developed. The authors claim that the system aids in automatically detecting potholes using deep 6 learning techniques and managing their status and repairs. In addition, it allows road users to view 7 an overlay of the detected potholes on the maps based on their selected route, enabling them to 8 avoid the potholes and increase their safety on the roads. Two data collection methods were used, an 9 interview and a questionnaire, to gather data from the target system users. For the pothole detection, a  deep learning model using YOLOv8 was developed, which achieved an overall performance of 95%, 12 98%, and 92% on F1 score, precision, and recall, respectively.

Concerns:

The comparisions with other  pothole detection algorithms are absent.

The testing are carried out by using NOT standard & open data set. This impairs the persuasion.

Comments on the Quality of English Language

None

Author Response

We would like to thank the reviewer for the valuable comments and the time and effort made to review the manuscript. The comments have significantly improved the quality and presentation of the manuscript.

Reviewer 2 Comments

A Pothole Detection System utilizing Unmanned Aerial Vehicles, 5 called PDS-UAV, is developed. The authors claim that the system aids in automatically detecting potholes using deep 6 learning techniques and managing their status and repairs. In addition, it allows road users to view 7 an overlay of the detected potholes on the maps based on their selected route, enabling them to 8 avoid the potholes and increase their safety on the roads. Two data collection methods were used, an interview and a questionnaire, to gather data from the target system users. For the pothole detection, a  deep learning model using YOLOv8 was developed, which achieved an overall performance of 95%, 12 98%, and 92% on F1 score, precision, and recall, respectively.

Concerns:

Comment (1): The comparisons with other pothole detection algorithms are absent.

Response (1): The following paragraph has been added to the end of the related work section, on page 5, to highlight the differences between the existing solutions and the proposed system PDS-UAV:

“In summary, the reviewed pothole detection approaches utilize machine learning or deep learning for pothole detection from either UAV images or smartphones and in-vehicle cameras. The proposed system, PDS-UAV, advances pothole detection using a State-of-the-art deep learning model, particularly YOLOv8, for accurate identification from UAV images. Yolov8 architecture includes multiple convolutional layers that automatically identify relevant features such as edges, textures, and shapes associated with potholes. It is known for its speed and accuracy, crucial for timely maintenance and safety precautions. Using cross-city testing, the ability of the proposed system model to be generalized across domains is demonstrated. This highlights the model's high accuracy on datasets from different regions, setting it apart from prior approaches that typically focused on localized datasets. Additionally, the proposed system includes a web-based application for monitoring, reporting, and viewing detected potholes, helping to streamline the process for road maintenance employees and road users. Thus, PDS-UAV not only enhances detection accuracy, but also offers significant operational benefits by automating data collection and analysis, reducing the need for manual inspections, and increasing overall road safety for drivers and passengers.

Comment (2): The testing are carried out by using NOT standard & open data set. This impairs the persuasion.

Response (2): Thank you for bringing up this topic for discussion. While testing AI models on standard and open datasets is a common practice, it is not a strict requirement. Non-standard datasets that closely align with the intended deployment scenario can provide more relevant and accurate testing. For certain applications, such as pothole detection, standard datasets may not fully represent the specific domain, which could differ from one city to the other. Cross-city testing, where the model is trained on a dataset from one city and tested on data from another city, is defined and employed in a few studies in the literature as a method for cross-domain generalization between different cities. [25,26,27]. In our case, the developed model was trained on a public dataset of pothole images collected from Spain. However, pothole characteristics can vary significantly depending on geography, road type, construction practices, surface materials, lighting conditions, and weather patterns. Non-standard datasets can capture these unique characteristics that may not be present in standard datasets. This ensures that the model is tested on data that closely matches the real use case of the application and the specific conditions of the area where the pothole detection model will be deployed. Additionally, testing the model on a non-standard dataset helps ensure that the model performs well across diverse conditions and is not biased toward specific environments or regions. This approach enhances the model’s generalization across different environments, leading to more robust and accurate real-world performance.   It is worth mentioning that the testing dataset used in this paper is part of an ongoing study to collect a larger dataset, which will be made publicly available in the future.

The following information has been added to the manuscript on pages 18 and 19 to illustrate the discussed points and provide references:

“Cross-domain generalization [25], specifically cross-city testing [26,27], is utilized to assess how well the developed deep learning model trained on the Spain dataset [16] performs on detecting potholes from unseen images collected from Jeddah, Saudi Arabia.”

References

25. Wang, J.; Lan, C.; Liu, C.; Ouyang, Y.; Qin, T.; Lu, W.; Chen, Y.; Zeng, W.; Yu, P.S. Generalizing
to Unseen Domains: A Survey on Domain Generalization. IEEE Transactions on Knowledge and
Data Engineering 2023, 35, 8052–8072. https://doi.org/10.1109/TKDE.2022.3178128.
26. Saremi, F.; Abdelzaher, T. Combining Map-Based Inference and Crowd-Sensing for Detecting
Traffic Regulators. In Proceedings of the 2015 IEEE 12th International Conference on Mobile Ad
Hoc and Sensor Systems, 2015, pp. 145–153. https://doi.org/10.1109/MASS.2015.18.
27. Zourlidou, S.; Sester, M. Traffic Regulator Detection and Identification from Crowdsourced
Data—A Systematic Literature Review. ISPRS International Journal of Geo-Information 2019, 8.
https://doi.org/10.3390/ijgi8110491. 

Reviewer 3 Report

Comments and Suggestions for Authors

To improve the quality of the paper, the reviewer gives the following comments.

(1) In Figure 1, it is unreasonable that E does not have the inputs, i.e., arrows (from A and B for example).

(2) In Figure 3, the process “Modify pothole status” cannot go to the end.

(3) In Section 5.4, the feature extraction algorithm is not presented.

(4) In Section 5.5, the YOLO algorithm used in this paper is not described.

(5) Section 5.7 does not discuss how to compute F1 score, precision, and recall.

(6) Although Section 6.3 provides many time results, Section 6.3 does not describe the hardware, software, and network environments deployed PDS-UAV.

Comments on the Quality of English Language

The writing quality of the paper should be improved. The title “PDS-UAV: A Deep Learning based Pothole Detection System using Unmanned Aerial Vehicle Images” should be “PDS-UAV: A Deep Learning Based Pothole Detection System Using Unmanned Aerial Vehicle Images”. Other editing errors include “compare the their performance” in lines 166-167, “cosidering employees” in line 235, “Figure 3 and Figure 4 illustrates” in line 387, “wither on the map or a a list” in line 406, “confirm that the application functions correctly, with each test case passing as expected” in lines 601-602, and “the user change status of the” in line 622, etc.

Author Response

We would like to thank the reviewer for the valuable comments and the time and effort made to review the manuscript. The comments have significantly improved the quality and presentation of the manuscript.

Reviewer 3 Comments:

Comments and Suggestions for Authors

To improve the quality of the paper, the reviewer gives the following comments.

Comment (1): In Figure 1, it is unreasonable that E does not have the inputs, i.e., arrows (from A and B for example).

Response (1): Thank you for your comment. Section E in Figure 1 should indeed be connected to the other system components. The road user interface retrieves the information about detected potholes stored in the database according to the user’s route. Therefore, part E in Figure 1 is updated by connecting it to the database. The updated figure is presented on page 7.

Comment (2): In Figure 3, the process “Modify pothole status” cannot go to the end.

Response (2): We appreciate your comment and for pointing out the missing end in Figure 3. The diagram has been updated to include the missing arrow and ending for the 'Modify pothole status' activity. The updated figure is presented on page 12.

Comment (3): In Section 5.4, the feature extraction algorithm is not presented.

Response (3): Thank you for your comment. The information presented in Section 5.4 has been updated to clarify the feature extraction algorithm as highlighted on pages 19 and 20:

“Feature extraction involves identifying the specific qualities that define an object. YOLO typically uses a CNN-based algorithm for feature extraction. The specific CNN architecture can vary depending on the version, but generally, it involves using a series of convolutional layers to extract hierarchical features from the input image. In our case, the most important features for identifying a pothole are its edges and texture. Edge detection involves comparing each pixel with its surrounding pixels to determine color variations. In pothole detection, a pixel is classified as an edge if the color of that pixel differs from neighboring pixels' colors.”

Comment (4): In Section 5.5, the YOLO algorithm used in this paper is not described.

Response (4): Thank you for your comment. More details about the YOLO model have been added to Section 5.5. Additionally, the YOLO description and the figure illustrating the steps involved in object detection using YOLO have been moved from the beginning of Section 5 to Section 5.5 for better readability and clarity. The following paragraphs have been added to the manuscript, as highlighted on pages 20 and 21:

“The proposed model for this system is a CNN-based model, specifically utilizing YOLO, which was proposed by Joseph Redmon et al. in May 2016 [30]. YOLO is known for its speed and minimal memory consumption in object detection. It is an object detection framework for identifying various objects within images or videos. During training, the YOLO model requires objects to be marked by bounding boxes and labeled with their corresponding class for detection. The YOLO architecture consists of three main components: the backbone, the neck, and the head, which can vary between different versions of YOLO. The backbone is a pre-trained CNN responsible for extracting useful features from the input image. The neck is another component that connects the backbone to the head. It merges the feature maps using path aggregation blocks like the Feature Pyramid Network (FPN) and then passes them onto the head. The head is responsible for classifying objects and predicting bounding boxes based on the features provided by the backbone and neck.

The detection process is illustrated using a series of steps as shown in Figure 20. The detection process begins by taking the input image and passing it through the feature extraction phase, which enhances detection efficiency by reducing data redundancy. The CNN layers are responsible for detecting potholes by computing the probability of an image containing a pothole based on the learned patterns from the training dataset. Each layer contributes to the classification decision. The final decision of the deep learning algorithm confirms the detection. Ultimately, the output classifies an image as either containing a pothole or not.”

 

Comment (5):  Section 5.7 does not discuss how to compute F1 score, precision, and recall.

Response (5): Thank you for your comment. We have added the definitions, and the equation used for computing F1 score, precision, and recall. Changes have been made to the manuscript as highlighted on pages 21 and 22.

Comment (6):  Although Section 6.3 provides many time results, Section 6.3 does not describe the hardware, software, and network environments deployed PDS-UAV.

Response (6): Thank you for your comment. The following paragraph has been added to the manuscript on page 26 to illustrate the usability testing environment:

“Different devices were utilized for the usability testing. The RMEs website was tested on a Windows 11 desktop with an HD display and a high-speed wired internet connection. Google Chrome was used to access the RMEs’ website. The RUs website was evaluated on an Android device with GPS and a 4G internet connection. All testing sessions were conducted in a controlled physical environment, maintaining consistent room, lighting, and seating conditions. Data on users' task performance and feedback were collected during the usability sessions.”

Comment (7):  Comments on the Quality of English Language

The writing quality of the paper should be improved. The title “PDS-UAV: A Deep Learning based Pothole Detection System using Unmanned Aerial Vehicle Images” should be “PDS-UAV: A Deep Learning Based Pothole Detection System Using Unmanned Aerial Vehicle Images”. Other editing errors include “compare the their performance” in lines 166-167, “cosidering employees” in line 235, “Figure 3 and Figure 4 illustrates” in line 387, “wither on the map or a a list” in line 406, “confirm that the application functions correctly, with each test case passing as expected” in lines 601-602, and “the user change status of the” in line 622, etc.

Response (7): Thank you for your comment. The manuscript has been revised to improve the English language. In addition, the following updates and corrections were made to the manuscript as requested:

• The title of the manuscript has been updated as suggested to be “PDS-UAV: A Deep Learning Based Pothole Detection System Using Unmanned Aerial Vehicle Images”.
• The sentence in lines 165-166 (previously in lines 166-167) has been updated as follows: “The running times of all four algorithms on the same PC were recorded to compare their performance.”
• The sentence in lines 250-251 (previously in line 235) has been rewritten as follows: “In our scenario, we focus on employees at the Jeddah City Municipality.”
• In line 401 (previously in line 387), the verb has been corrected as follows: “Figure 3 and Figure 4 illustrate”
• In lines 419-420 (previously in line 406), the sentence has been rewritten as follows: “The RMEs log in and retrieve pothole information from the database, which can be viewed either on the map or in list view.”
• The sentence in lines 644-646 (previously in lines 601-602) has been updated as follows: “The results of the integration testing, detailed in Table 6, confirm that the application’s functions are working correctly, with all test cases producing the expected outcomes.”
• The sentence in lines 673-675 (Previously in line 622) has been updated as follows:” The user selects the edit option from the potholes table at the bottom of the home page and updates the pothole status to "repaired" in the update pothole status page.”

Reviewer 4 Report

Comments and Suggestions for Authors

The authors present a comprehensive report about the development work of a system using a deep learning model to detect potholes from UAV images. 

This work is a good fit for the scope of this Journal.

The paper is well structured.

The methodology is also correctly described.

The results achieved are solid, and the discussion follows those results.

A minor highlighted issue is related to the focus on engineering.  

 

Comments on the Quality of English Language

This paper is well written and affordable to read.

 

Author Response

We would like to thank the reviewer for the valuable comments and the time and effort made to review the manuscript. The comments have significantly improved the quality and presentation of the manuscript.

Reviewer Comments:

The authors present a comprehensive report about the development work of a system using a deep learning model to detect potholes from UAV images. 

This work is a good fit for the scope of this Journal.

The paper is well structured.

The methodology is also correctly described.

The results achieved are solid, and the discussion follows those results.

Comment (1): A minor highlighted issue is related to the focus on engineering.

Response (1): Thank you for your observation regarding the focus on engineering in our paper. We have revised the relevant sections, which are the methodology and all its subsections including the implementation details. Changes made to the manuscript are highlighted in the submitted revised manuscript. 

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

All is OK

Author Response

We would like to thank the reviewer for the valuable comments and the time and effort made to review the manuscript. The comments have significantly improved the quality and presentation of the manuscript.

Reviewer Comments and Suggestions: All is OK.

Response: Thank you.

 

Reviewer 3 Report

Comments and Suggestions for Authors

No further comment.

Comments on the Quality of English Language

English language still needs to be polished. The errors sentences include “5.5. Building the deep learning Model” in line 559, “The Target times were” in line 692, and “Second, While there” in line 774, etc.

Author Response

Reviewer 3 Comments on the Quality of English Language:

We would like to thank the reviewer for the valuable comments and the time and effort made to review the manuscript. The comments have significantly improved the quality and presentation of the manuscript.

Reviewer 3 Comments on the Quality of English Language:

English language still needs to be polished. The errors sentences include “5.5. Building the deep learning Model” in line 559, “The Target times were” in line 692, and “Second, While there” in line 774, etc.

Response: Thank you for your comment. All the errors in the mentioned sentences have been corrected. The changes made are highlighted in pages 20, 26, and 28. In addition, the whole paper was reviewed to improve the quality of the English language and correct similar punctuation and capitalization errors.