Assuring Safe and Efficient Operation of UAV Using Explainable Machine Learning
Abstract
:1. Introduction
- This paper proposes a hybrid explainable machine-learning model, in order to support UTM demand and capacity-management services. Within this model, a set of functions are enabled, encompassing trajectory allocation, flight planning, and capacity optimization. This integrated approach produces an optimal solution, which minimizes operational costs while maintaining traffic density under urban-airspace thresholds. The suggested model has been validated using simulated scenarios of UTM operations (e.g., drone delivery applications). These simulations consider uncertainties arising from weather conditions, static and dynamic obstacles, and emergency operations, especially in urban environments.
- This paper proposes, in addition, a data-analytics framework to characterize traffic flow patterns for UTM airspace evaluated on the example of analysis of simulated historical data. The methodology focuses on two main components that intervene in a DCM process, namely, the prediction of congestion figures for each trajectory, and the accurate estimation of airspace capacity. Specifically, we identified five congestion levels, and a clustering algorithm-based mechanism was developed to determine available urban airspace for Urban Air Mobility (UAM) operations, based on the UTM traffic-flow analysis.
- In terms of the explainability of the decision-support system, this study proposes a transparency-based methodology with a fusion of both Black-Box and explainable White-Box models for our UTM recommendation systems. The Black-Box models are not transparent, due to a lack of clarity associated with their internal configuration. By contrast, White-Box models manifest observable and understandable behaviors. We have introduced metrics-based scoring to illustrate the overall explainability of our hybrid model, based on the transparency of the individual components. In light of these metrics, we have confirmed that our proposed advisory system is approximately 70% explainable.
2. Literature Review and Background
2.1. The Challenges of Certifying AI
2.2. Methods of Interpretability
2.2.1. Explainable AI
- Trustworthiness: An ML model cannot realistically be deployed without a basis of trust. Otherwise, users may simply ignore model output. As noted above, EASA thus regards explicability as central to trustworthiness, and the latter is one of the key objectives of their AI roadmap [56].
- Causality: An additional objective of “explainability” is to facilitate the finding of causation between data variables. For models that assess UAS systemic health, for instance, explainability may reveal that a given component tends to fail after a certain load time.
- Transferability: Explainability can also help to clarify model constraints and limitations. Models learn to solve particular problems during training, but an understanding of boundaries is required to ascertain how, or if, the model may be applied to other problems. If a model has been trained to detect obstacles in daylight, for instance, it should not be used at night, at least without suitable modification.
- Accessibility: Explainable models will reassure non-expert users, who may feel intimidated by algorithms that, at first glance, appear inexplicable.
2.2.2. Transparent Models
2.2.3. Post-Hoc Explainability
3. Proposed Advisory System Framework
3.1. Overall Framework
- UAV trajectory-data generation is undertaken using Particle Swarm Optimization (PSO) simulation, for different environmental scenarios, on an hourly basis. This provides optimal paths from a UAV service start point to the relevant delivery point [105]. For this study, we acquired data for three hours (9:00 am to 12:00 pm). Moreoverconditionsc and dynamic structural changes of the airspace, and adverse, extreme weather conditions, are also considered in this research (see Section 3.2, below).
- The pre-processing of acquired data is expanded by up-sampling, in order to increase the resolution of UAV trajectories. This generates better air traffic flow and congestion analysis.
- An LSTM-based congestion-prediction model has been utilized to obtain predicted congested values for each trajectory. A detailed explanation of the congestion-prediction model is furnished in Section 3.3, below. The predicted congestion values are normalized between 0–100%, in order to threshold the congestion levels.
- In Table 1 we defined five congestion levels, both for better explainability to UTM authorities and to assist further analysis:
- Since the congested levels are distributed over the entire Bedfordshire UTM airspace (64 km × 64 km), we have identified the congested zones or sub-regions for each of the five congestion levels. This can be conducted by running and tuning the DBSCAN clustering algorithm, iteratively, for each congestion level. The optimal tuning is conducted by adjusting the parameters “eps” and minimum points (“minPts”) for DBSCAN. The parameter tuning is required for better trajectory cluster-grouping formation; moreover, it also helps in defining better congestion-area polygons.
- The area polygons are created around these congested clusters or groups, both to estimate the covered area per cluster and to locate the centroid position (x, y) around which a cluster is formed. The covered area around these clusters is built by forming an irregular polygon (using the boundary points), and by measuring the area using the MATLAB poly-shape function. The count of UAV trajectory points for these congested zones is also measured.
- The traffic flow for each congestion cluster is calculated using the ratio between UAV trajectory counts and the area encapsulated by that cluster.
- The capacity of each congested cluster is then measured by defining a safe traffic-flow threshold. This is derived from the notion of safe separation distance. In our work, a safe lateral separation distance of 100 m is applied, while the vertical distance is not considered in this study. This, in turn, indicates 10 UAVs per km, which implies about 100 UAVs per km2 within each cluster. The available capacity for each cluster is calculated by taking the difference between the current traffic flow and traffic-flow threshold (100 UAV trajectories/km2).
- The rule-based decision tree is then designed and implemented for each of the five congestion levels (lowest-highest) using three inputs: (1) available airspace capacity (capacity per cluster), which is computed via our congestion analysis; (2) the number of new incoming UAV trajectory points that happen to traverse the congested regions (lowest to highest) and finally, (3) the mission priorities of incoming UAVs, which are required for optimal recommendations. The output of the advisory system is the updated capacity, either allowing the UAV mission within a particular congestion cluster, or disallowing (for safety reasons) the usage of a particular congested airspace. This is followed by a recommendation to use specific, available airspace.
3.2. Description of Data
3.3. Congestion-Prediction Model
3.4. Demand and Capacity Management
4. Results and Discussion
4.1. UTM Congested Subzones: Identification and Area Distribution
4.1.1. Congestion-Level Identification Using DBSCAN
4.1.2. Airspace-Congestion Distribution
4.1.3. Traffic-Flow Distribution and Airspace-Capacity Identification
4.2. Analysis and Design of Explainability for the DCM Advisory System
4.2.1. Rule-Based Explanation for DCM Decisions
4.2.2. Post-Hoc Local Explanation: Visual Explanation
4.2.3. Post-Hoc Local Explanation: Explanation by Example
4.3. Advisory-System Efficiencies for Capacity and Safety
4.4. Comparison of the Proposed Model with Other Approaches
5. Conclusions and Future Work
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Kim, Y.; Jo, J.; Shaw, M. A Lightweight Communication Architecture for Small UAS Traffic Management (sUTM). In Proceedings of the 2015 Integrated Communication, Navigation and Surveillance Conference (ICNS), Herdon, VA, USA, 21–23 April 2015. [Google Scholar] [CrossRef]
- Kopardekar, P.; Rios, J.; Prevot, T.; Johnson, M.; Jung, J.; Robinson, J.E. Unmanned Aircraft System Traffic Management (UTM) Concept of Operations. In Proceedings of the AIAA Aviation and Aeronautics Forum (Aviation 2016), 2016 (ARC-E-DAA-TN32838), Washington, DC, USA, 13–17 June 2016; pp. 1–16. [Google Scholar]
- Mueller, E.; Kopardekar, P.; Goodrich, K. Enabling Airspace Integration for High-Density On-Demand Mobility Operations. In Proceedings of the 17th AIAA Aviation Technology, Integration, and Operations Conference, Denver, CO, USA, 5–9 June 2017; pp. 1–24. [Google Scholar] [CrossRef]
- Kistan, T.; Gardi, A.; Sabatini, R.; Ramasamy, S.; Batuwangala, E. An evolutionary outlook of air traffic flow management techniques. Prog. Aerosp. Sci. 2017, 88, 15–42. [Google Scholar] [CrossRef]
- Ali, B.S. Management for Drones Flying in the City. In Proceedings of the 22nd Air Transport Research Society (Atrs) World Conference Atcoex, Seoul, Republic of Korea, 2–5 July 2018; Available online: http://eprints.um.edu.my/18968/1/Traffic_Management_for_Drones_Flying_in_the_City.pdf (accessed on 4 March 2023).
- Liu, Y.; Zhang, X.; Wang, Z.; Gao, Z.; Liu, C. Ground Risk Assessment of UAV Operations Based on Horizontal Distance Estimation under Uncertain Conditions. Math. Probl. Eng. 2021, 2021, 3384870. [Google Scholar] [CrossRef]
- Primatesta, S.; Rizzo, A.; la Cour-Harbo, A. Ground Risk Map for Unmanned Aircraft in Urban Environments. J. Intell. Robot. Syst. Theory Appl. 2020, 97, 489–509. [Google Scholar] [CrossRef]
- McCarthy, T.; Pforte, L.; Burke, R. Fundamental elements of an urban UTM. Aerospace 2020, 7, 85. [Google Scholar] [CrossRef]
- International Civil Aviation Organization. Doc. 4444—Procedures for Air Navigation Services—Air Traffic Management; International Civil Aviation Organization (ICAO): Montreal, QC, Canada, 2016; ISBN 978-92-9258-081-0. [Google Scholar]
- Bertsimas, D.; Patterson, S.S. The air traffic flow management problem with enroute capacities. Oper. Res. 1998, 46, 406–422. [Google Scholar] [CrossRef]
- Crespo, A.M.F.; Weigang, L.; de Barros, A.G. Reinforcement learning agents to tactical air traffic flow management. Int. J. Aviat. Manag. 2012, 1, 145. [Google Scholar] [CrossRef]
- Gardi, A.; Sabatini, R.; Ramasamy, S. Multi-objective optimisation of aircraft flight trajectories in the ATM and avionics context. Prog. Aerosp. Sci. 2016, 83, 1–36. [Google Scholar] [CrossRef]
- Pongsakornsathien, N.; Bijjahalli, S.; Gardi, A.; Symons, A.; Xi, Y.; Sabatini, R.; Kistan, T. A performance-based airspace model for unmanned aircraft systems traffic management. Aerospace 2020, 7, 154. [Google Scholar] [CrossRef]
- Bauranov, A.; Rakas, J. Designing airspace for urban air mobility: A review of concepts and approaches. Prog. Aerosp. Sci. 2021, 125, 100726. [Google Scholar] [CrossRef]
- Balakrishnan, K.; Polastre, J.; Mooberry, J.; Golding, R.; Sachs, P. Blueprint for the Sky: The Roadmap for the Safe Integration of Autonomous Aircraft; Airbus_UTM_Blueprint; Airbus UTM: San Francisco, CA, USA, 2018. [Google Scholar]
- Chan, W.N.; Barmore, B.E.; Kibler, J.; Lee, P.; Connor, N.O.; Palopo, K.; Thipphavong, D.; Zelinski, S. Overview of NASA’s Air Traffic Management—Exploration (ATM-X) Project. In Proceedings of the AIAA Aviation Forum 2018, No. ARC-E-DAA-TN57276. Atlanta, GA, USA, 25–29 June 2018. [Google Scholar]
- Engineer, P. ATM-X UAM Subproject. 2020. Available online: https://ntrs.nasa.gov/citations/20210000102 (accessed on 3 April 2023).
- Nawaz, H.; Ali, H.M.; Laghari, A.A. UAV Communication Networks Issues: A Review. Arch. Comput. Methods Eng. 2021, 28, 1349–1369. [Google Scholar] [CrossRef]
- Degas, A.; Islam, M.R.; Hurter, C.; Barua, S.; Rahman, H.; Poudel, M.; Ruscio, D.; Ahmed, M.U.; Begum, S.; Rahman, M.A.; et al. A Survey on Artificial Intelligence (AI) and eXplainable AI in Air Traffic Management: Current Trends and Development with Future Research Trajectory. Appl. Sci. 2022, 12, 1295. [Google Scholar] [CrossRef]
- Kistan, T.; Gardi, A.; Sabatini, R. Machine learning and cognitive ergonomics in air traffic management: Recent developments and considerations for certification. Aerospace 2018, 5, 103. [Google Scholar] [CrossRef]
- Borst, C.; Bijsterbosch, V.A.; van Paassen, M.M.; Mulder, M. Ecological interface design: Supporting fault diagnosis of automated advice in a supervisory air traffic control task. Cogn. Technol. Work 2017, 19, 545–560. [Google Scholar] [CrossRef]
- Borghini, G.; Aricò, P.; DI Flumeri, G.; Cartocci, G.; Colosimo, A.; Bonelli, S.; Golfetti, A.; Imbert, J.P.; Granger, G.; Benhacene, R.; et al. EEG-Based Cognitive Control Behaviour Assessment: An Ecological study with Professional Air Traffic Controllers. Sci. Rep. 2017, 7, 547. [Google Scholar] [CrossRef] [PubMed]
- Sheh, R.; Monteath, I. Defining Explainable AI for Requirements Analysis. KI-Kunstl. Intelligenz 2018, 32, 261–266. [Google Scholar] [CrossRef]
- Mueller, S.T.; Hoffman, R.R.; Clancey, W.; Emrey, A.; Klein, G. Explanation in Human-AI Systems: A Literature Meta-Review. synopsis of key ideas and publications, and bibliography for explainable AI. arXiv 2019, arXiv:1902.01876. [Google Scholar]
- Marcos, R.; García-cantú, O.; Herranz, R. A Machine Learning Approach to Air Traffic Route Choice Modelling. arXiv 2018, arXiv:1802.06588. [Google Scholar]
- Choi, S.; Kim, Y.J.; Briceno, S.; Mavris, D. Prediction of Weather-Induced Airline Delays Based on Machine Learning Algorithms. In Proceedings of the 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC), Sacramento, CA, USA, 25–29 September 2016. [Google Scholar]
- Carvalho, L.; Sternberg, A.; Maia Gonçalves, L.; Beatriz Cruz, A.; Soares, J.A.; Brandão, D.; Carvalho, D.; Ogasawara, E. On the relevance of data science for flight delay research: A systematic review. Transp. Rev. 2021, 41, 499–528. [Google Scholar] [CrossRef]
- Lee, H. Taxi-out Time Prediction for Departures at Charlotte Airport. In Proceedings of the 16th AIAA Aviation Technology, Integration, and Operations Conference, Washington, DC, USA, 13–17 June 2016. [Google Scholar]
- Ayhan, S.; Samet, H. Aircraft Trajectory Prediction Made Easy with Predictive Analytics. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; pp. 21–30. [Google Scholar] [CrossRef]
- Oehling, J.; Barry, D.J. Using machine learning methods in airline flight data monitoring to generate new operational safety knowledge from existing data. Saf. Sci. 2019, 114, 89–104. [Google Scholar] [CrossRef]
- Büsing, C.; Kadatz, D.; Cleophas, C. Capacity uncertainty in airline revenue management: Models, algorithms, and computations. Transp. Sci. 2019, 53, 383–400. [Google Scholar] [CrossRef]
- Chung, S.H.; Ma, H.L.; Chan, H.K. Cascading Delay Risk of Airline Workforce Deployments with Crew Pairing and Schedule Optimization. Risk Anal. 2017, 37, 1443–1458. [Google Scholar] [CrossRef] [PubMed]
- Liu, Y.; Liu, Y.; Hansen, M.; Pozdnukhov, A.; Zhang, D. Using machine learning to analyze air traffic management actions: Ground delay program case study. Transp. Res. Part E Logist. Transp. Rev. 2019, 131, 80–95. [Google Scholar] [CrossRef]
- Yanying, Y.; Mo, H.; Haifeng, L. Classification Prediction Analysis of Flight Cancellation Based on Spark. Procedia Comput. Sci. 2020, 162, 480–486. [Google Scholar] [CrossRef]
- Tian, Y.; Ye, B.; Wan, L.; Yang, M.; Xing, D. Restricted airspace unit identification using density-based spatial clustering of applications with noise. Sustainability 2019, 11, 5962. [Google Scholar] [CrossRef]
- Pouyanfar, S.; Sadiq, S.; Yan, Y.; Tian, H.; Tao, Y.; Reyes, M.P.; Iyengar, S.S. A survey on deep learning: Algorithms, techniques, and applications. ACM Comput. Surv. 2018, 51, 1–36. [Google Scholar] [CrossRef]
- Sternberg, A.; Carvalho, D.; Murta, L.; Soares, J.; Ogasawara, E. An analysis of Brazilian flight delays based on frequent patterns. Transp. Res. Part E Logist. Transp. Rev. 2016, 95, 282–298. [Google Scholar] [CrossRef]
- Cheng, J.; Rong, C.; Ye, H.; Zheng, X. Risk Management Using Big real Time Data. In Proceedings of the 2015 IEEE 7th International Conference on Cloud Computing Technology and Science (CloudCom), Vancouver, BC, Canada, 30 November–3 December 2015; 2016; pp. 542–547. [Google Scholar] [CrossRef]
- Arnaldo Scarpel, R.; Pelicioni, L.C. A data analytics approach for anticipating congested days at the São Paulo International Airport. J. Air Transp. Manag. 2018, 72, 1–10. [Google Scholar] [CrossRef]
- Cheng, J. Estimation of Flight Delay Using Weighted Spline Combined with ARIMA Model. proceedings of the 7th IEEE/International Conference on Advanced Infocomm Technology, Fuzhou, China, 14–16 November 2014; 2015; pp. 8–20. [Google Scholar] [CrossRef]
- Alligier, R.; Gianazza, D.; Durand, N. Machine Learning and Mass Estimation Methods for Ground-Based Aircraft Climb Prediction. IEEE Trans. Intell. Transp. Syst. 2015, 16, 3138–3149. [Google Scholar] [CrossRef]
- Takeichi, N.; Kaida, R.; Shimomura, A.; Yamauchi, T. Prediction of delay due to air traffic control by machine learning. AIAA Model. Simul. Technol. Conf. 2017, 8. [Google Scholar] [CrossRef]
- Torens, C.; Jünger, F.; Schirmer, S.; Schopferer, S.; Maienschein, T.; Dauer, J.C. Machine Learning Verification and Safety for Unmanned Aircraft—A Literature Study. In Proceedings of the AIAA Scitech 2022 Forum, San Diego, CA, USA, 3–7 January 2022. [Google Scholar] [CrossRef]
- FAA Concept of Operations v2.0. Enabling Civ. Low-altitude Airsp. Unmanned Aircr. Syst. Oper. 2020. Available online: https://utm.arc.nasa.gov/index.shtml (accessed on 5 March 2023).
- Krittanawong, C.; Zhang, H.J.; Wang, Z.; Aydar, M.; Kitai, T. Artificial Intelligence in Precision Cardiovascular Medicine. J. Am. Coll. Cardiol. 2017, 69, 2657–2664. [Google Scholar] [CrossRef]
- Keneni, B.M.; Kaur, D.; Al Bataineh, A.; Devabhaktuni, V.K.; Javaid, A.Y.; Zaientz, J.D.; Marinier, R.P. Evolving Rule-Based Explainable Artificial Intelligence for Unmanned Aerial Vehicles. IEEE Access 2019, 7, 17001–17016. [Google Scholar] [CrossRef]
- Xie, Y.; Pongsakornsathien, N.; Gardi, A.; Sabatini, R. Explanation of Machine-Learning Solutions in Air-Traffic Management. Aerospace 2021, 8, 224. [Google Scholar] [CrossRef]
- Faria, J.M. Machine learning safety: An overview. In Proceedings of the 26th Safety-Critical Systems Symposium, York, UK, 6–8 February 2018; pp. 6–8. [Google Scholar]
- Tambon, F.; Laberge, G.; An, L.; Nikanjam, A.; Mindom, P.S.N.; Pequignot, Y.; Khomh, F.; Antoniol, G.; Merlo, E.; Laviolette, F. How to certify machine learning based safety-critical systems? A systematic literature review. Autom. Softw. Eng. 2022, 29, 38. [Google Scholar] [CrossRef]
- Amodei, D.; Olah, C.; Steinhardt, J.; Christiano, P.; Schulman, J.; Mané, D. Concrete Problems in AI Safety. arXiv 2016, arXiv:1606.06565. [Google Scholar]
- Serban, A.C. Designing Safety Critical Software Systems to Manage Inherent Uncertainty. In Proceedings of the 2019 IEEE International Conference on Software Architecture Companion (ICSA-C), Hamburg, Germany, 25–26 March 2019; pp. 246–249. [Google Scholar] [CrossRef]
- Hains, G.; Jakobsson, A.; Khmelevsky, Y. Towards Formal Methods and Software Engineering for Deep Learning: Security, Safety and Productivity for dl Systems Development. In Proceedings of the 2018 Annual IEEE International Systems Conference (SysCon), Vancouver, BC, Canada, 23–26 April 2018; pp. 1–5. [Google Scholar] [CrossRef]
- Bharadwaj; Prakash, K.B.; Kanagachidambaresan, G.R. Pattern Recognition and Machine Learning; Springer: New York, NY, USA, 2021; ISBN 9780387310732. [Google Scholar]
- Rodríguez-Dapena, P. Software safety certification: A multidomain problem. IEEE Softw. 1999, 16, 31–38. [Google Scholar] [CrossRef]
- Youn, W.K.; Hong, S.B.; Oh, K.R.; Ahn, O.S. Software certification of safety-critical avionic systems: DO-178C and its impacts. IEEE Aerosp. Electron. Syst. Mag. 2015, 30, 4–13. [Google Scholar] [CrossRef]
- Artificial, E.; Roadmap, I.; Artificial, E.; Roadmap, I. A Human-Centric Approach to AI in Aviation; European Aviation Safety Agency: Cologne, Germany, 2020; Volume 1. [Google Scholar]
- Gabreau, C.; Pesquet-Popescu, B.; Kaakai, F.; Lefevre, B. AI for Future Skies: On-going standardization activities to build the next certification/approval framework for airborne and ground aeronautical products. CEUR Workshop Proc. 2021, 2916. [Google Scholar]
- Jean-Marc, C.; Xavier, C.; Henriquel; Soudain, S.G.; Vaubourg, S.; van der Brugge, H.; van Dijk, L.; Perret-Gentil, J.M.K.C.; Whittington, P.I. Concepts of Design Assurance for Neural Networks (CoDANN) II Public Extract. 2021. Available online: https://avsi.aero/wp-content/uploads/2020/06/AFE-87-Final-Report.pdf (accessed on 8 April 2023).
- Aerospace Vehicle Systems Institut. AVSI Final Report AFE 87—Machine Learning. Document ID: 87-REP-01. Available online: https://avsi.aero/projects/current-projects/cert-of-ml-systems/afe-87-machine-learning/ (accessed on 8 April 2023).
- Kumeno, F. Sofware engneering challenges for machine learning applications: A literature review. Intell. Decis. Technol. 2020, 13, 463–476. [Google Scholar] [CrossRef]
- Pereira, A.; Thomas, C. Challenges of Machine Learning Applied to Safety-Critical Cyber-Physical Systems. Mach. Learn. Knowl. Extr. 2020, 2, 31. [Google Scholar] [CrossRef]
- Ashmore, R.; Calinescu, R.; Paterson, C. Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges. ACM Comput. Surv. 2021, 54, 1–39. [Google Scholar] [CrossRef]
- Liu, W.; Wang, Z.; Liu, X.; Zeng, N.; Liu, Y.; Alsaadi, F.E. A Survey of Deep Neural Network Architectures and Their Applications. Neurocomputing 2017, 234, 11–26. Available online: https://wwwsciencedirectcom/science/article/pii/S0925231216315533 (accessed on 8 April 2023). [CrossRef]
- Padakandla, S. A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments. ACM Comput. Surv. 2021, 54, 1–25. [Google Scholar] [CrossRef]
- Arulkumaran, K.; Deisenroth, M.P.; Brundage, M.; Bharath, A.A. Deep reinforcement learning: A brief survey. IEEE Signal Process. Mag. 2017, 34, 26–38. [Google Scholar] [CrossRef]
- Lu, J.; Behbood, V.; Hao, P.; Zuo, H.; Xue, S.; Zhang, G. Transfer learning using computational intelligence: A survey. Knowledge-Based Syst. 2015, 80, 14–23. [Google Scholar] [CrossRef]
- Weiss, K.; Khoshgoftaar, T.M.; Wang, D.D. A Survey of Transfer Learning; Springer International Publishing: Cham, Switzerland, 2016; Volume 3, ISBN 4053701600. [Google Scholar]
- Gomes, H.M.; Barddal, J.P.; Enembreck, A.F.; Bifet, A. A survey on ensemble learning for data stream classification. ACM Comput. Surv. 2017, 50, 1–23. [Google Scholar] [CrossRef]
- Krawczyk, B.; Minku, L.L.; Gama, J.; Stefanowski, J.; Woźniak, M. Ensemble learning for data stream analysis: A survey. Inf. Fusion 2017, 37, 132–156. [Google Scholar] [CrossRef]
- Katz, G.; Huang, D.A.; Ibeling, D.; Julian, K.; Lazarus, C.; Lim, R.; Shah, P.; Thakoor, S.; Wu, H.; Zeljić, A.; et al. The Marabou Framework for Verification and Analysis of Deep Neural Networks. In Computer Aided Verification: 31st International Conference, CAV 2019, New York City, NY, USA, 15–18 July 2019; Springer International Publishing: Cham, Switzerland, 2019; pp. 443–452. [Google Scholar] [CrossRef]
- Hawkins, R.; Paterson, C.; Picardi, C.; Jia, Y.; Calinescu, R.; Habli, I. Guidance on the Assurance of Machine Learning in Autonomous Systems (AMLAS). arXiv 2021, arXiv:2102.01564. [Google Scholar]
- Molnar, C. Interpretable Machine Learning A Guide for Making Black Box Models Explainable. Available online: https://christophm.github.io/interpretable-ml-book/ (accessed on 22 March 2023).
- Marcus, G. Deep Learning: A Critical Appraisal. arXiv 2018, arXiv:1801.00631. [Google Scholar]
- Preece, A.; Harborne, D.; Braines, D.; Tomsett, R.; Chakraborty, S. Stakeholders in Explainable AI. arXiv 2018, arXiv:1810.00184. [Google Scholar]
- Gunning, D. DARPA’s Explainable Artificial Intelligence Program. AI Mag. 2019, 40, 44–58. [Google Scholar] [CrossRef]
- Tjoa, E.; Guan, C. A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI. IEEE Trans. Neural Networks Learn. Syst. 2021, 32, 4793–4813. [Google Scholar] [CrossRef]
- Zhu, J.; Liapis, A.; Risi, S.; Bidarra, R.; Youngblood, G.M. Explainable AI for Designers: A Human-Centered Perspective on Mixed-Initiative Co-Creation. In Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games (CIG), Maastricht, the Netherlands, 14–17 August 2018; pp. 1–8. [Google Scholar] [CrossRef]
- Dosilovic, F.K.; Brcic, M.; Hlupic, N. Explainable Artificial Intelligence: A Survey. In Proceedings of the 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics, Opatija, Croatia, 21–25 May 2018; pp. 210–215. [Google Scholar]
- Barredo Arrieta, A.; Díaz-Rodríguez, N.; Del Ser, J.; Bennetot, A.; Tabik, S.; Barbado, A.; Garcia, S.; Gil-Lopez, S.; Molina, D.; Benjamins, R.; et al. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion 2020, 58, 82–115. [Google Scholar] [CrossRef]
- Lipton, Z.C. The Mythos of Model Interpretability: In Machine Learning, the Concept of Interpretability Is Both Important and Slippery. Queue 2018, 16, 31–57. [Google Scholar] [CrossRef]
- Kucklick, J.-P. Towards a Model- and Data-Focused Taxonomy of XAI Systems. In Wirtschaftsinformatik 2022 Proceedings. 2022. Available online: https://aisel.aisnet.org/wi2022/business_analytics/business_analytics/2 (accessed on 5 March 2023).
- Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 2019, 1, 206–215. [Google Scholar] [CrossRef] [PubMed]
- Lechner, M.; Hasani, R.; Amini, A.; Henzinger, T.A.; Rus, D.; Grosu, R. Neural circuit policies enabling auditable autonomy. Nat. Mach. Intell. 2020, 2, 642–652. [Google Scholar] [CrossRef]
- Ribeiro, M.T.; Singh, S.; Guestrin, C. “Why Should I Trust You?” Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; 2016; pp. 97–101. [Google Scholar] [CrossRef]
- Lundberg, S.M.; Lee, S.-I. A Unified Approach to Interpreting Model Predictions. Adv. Neural Inf. Process. Syst. 2017, 30. [Google Scholar]
- Strumbelj, E.; Kononenko, I. An Efficient Explanation of Individual Classifications using Game Theory. J. Mach. Learn. Res. 2010, 11, 1–18. [Google Scholar]
- Lundberg, S.M.; Erion, G.; Chen, H.; Degrave, A.; Prutkin, J.M.; Nair, B.; Katz, R.; Himmelfarb, J.; Bansal, N.; Lee, S. From local explanations to global understanding with explainable AI for trees. Nat. Mach. Intell. 2020, 2, 56–67. [Google Scholar] [CrossRef]
- Lei, T.; Barzilay, R.; Jaakkola, T. Rationalizing neural predictions. arXiv 2016, arXiv:1606.04155. [Google Scholar]
- Kim, B.; Wattenberg, M.; Gilmer, J.; Cai, C.; Wexler, J.; Viegas, F.; Sayres, R. Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV). In Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, 10–15 July 2018; Volume 6, pp. 4186–4195. [Google Scholar]
- Lamy, J.B.; Sekar, B.; Guezennec, G.; Bouaud, J.; Séroussi, B. Explainable artificial intelligence for breast cancer: A visual case-based reasoning approach. Artif. Intell. Med. 2019, 94, 42–53. [Google Scholar] [CrossRef]
- Hoffman, R.R.; Mueller, S.T.; Klein, G.; Litman, J. Metrics for Explainable AI: Challenges and Prospects. arXiv 2018, arXiv:1812.04608. [Google Scholar]
- Mohseni, S.; Zarei, N.; Ragan, E.D. A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems. ACM Trans. Interact. Intell. Syst. 2021, 11, 1–45. [Google Scholar] [CrossRef]
- Doshi-Velez, F.; Kim, B. Towards A Rigorous Science of Interpretable Machine Learning. arXiv 2017, arXiv:1702.08608. [Google Scholar]
- Majumdar, A.; Ochieng, W.Y.; Bentham, J.; Richards, M. En-route sector capacity estimation methodologies: An international survey. J. Air Transp. Manag. 2005, 11, 375–387. [Google Scholar] [CrossRef]
- Klein, A.; Cook, L.; Wood, B.; Simenauer, D. Airspace Capacity Estimation Using Flows and Weather-Impacted Traffic Index the Task of Translating Weather Information into TFM Constraints Scope of Current Research. In Proceedings of the 2008 Integrated Communications, Navigation and Surveillance Conference, Bethesda, MD, USA, 5–7 May 2008. [Google Scholar]
- Sunil, E.; Hoekstra, J.; Ellerbroek, J.; Bussink, F.; Delahaye, D.; Nieuwenhuisen, D.; Sunil, E.; Hoekstra, J.; Ellerbroek, J.; Bussink, F.; et al. How Do Layered Airspace Design Parameters Affect Airspace Capacity and Safety? In Proceedings of the 7th International Conference on Research in Air Transportation, Philadelphia, PA, USA, 20–24 June 2016; ICRAT: Philadelphia, PA, USA, 2016; pp. 1–8. [Google Scholar]
- Welch, J.D. En Route Sector Capacity Model Final Report. 2015. Available online: https://archive.ll.mit.edu/mission/aviation/publications/publication-files/atc-reports/Welch_2015_ATC-426(accessed on 8 April 2023).
- Prandini, M.; Piroddi, L.; Puechmorel, S.; Brazdilova, S.L. Toward Air Traffic Complexity Assessment in New Generation Air Traffic Management Systems. IEEE Trans. Intell. Transp. Syst. 2011, 12, 809–818. [Google Scholar] [CrossRef]
- Banerjee, S. Mathematical Modeling Models, Analysis and Applications, 2nd ed.; Chapman and Hall/CRC: New York, NY, USA, 2021. [Google Scholar]
- Kopardekar, P.H.; Schwartz, A.; Magyarits, S.; Rhodes, J. Airspace complexity measurement: An air traffic control simulation analysis. Int. J. Ind. Eng. 2009, 16, 61–70. [Google Scholar]
- Kopardekar, P. Dynamic Density—A Review of Proposed Variables. In Overall Conclusions and Recommendations; FAA WJHTC Internal Document; Federal Aviation Administration: Washington, DC, USA, 2000; pp. 1–13. Available online: http://tinyurl.com/ahbgfrr (accessed on 18 February 2023).
- Ho, F.; Geraldes, R.; Gonçalves, A.; Rigault, B.; Sportich, B.; Kubo, D.; Cavazza, M.; Prendinger, H. Decentralized Multi-Agent Path Finding for UAV Traffic Management. IEEE Trans. Intell. Transp. Syst. 2020, 23, 997–1008. [Google Scholar] [CrossRef]
- Wang, Z.; Pan, W.; Li, H.; Wang, X.; Zuo, Q. Review of Deep Reinforcement Learning Approaches for Conflict Resolution in Air Traffic Control. Aerospace 2022, 9, 294. [Google Scholar] [CrossRef]
- Brown, T.B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. Language Models are Few-Shot Learners. CoRR 2020, 33, 1877–1901. Available online: https://arxivorg/abs/200514165 (accessed on 18 February 2023).
- Wang, D.; Tan, D.; Liu, L. Particle swarm optimization algorithm: An overview. Soft Comput. 2018, 22, 387–408. [Google Scholar] [CrossRef]
- Bijjahalli, S.; Sabatini, R.; Gardi, A. Advances in intelligent and autonomous navigation systems for small UAS. Prog. Aerosp. Sci. 2020, 115, 100617. [Google Scholar] [CrossRef]
- Radzki, G.; Golinska-Dawson, P.; Bocewicz, G.; Banaszak, Z. Modelling Robust Delivery Scenarios for a Fleet of Unmanned Aerial Vehicles in Disaster Relief Missions. J. Intell. Robot. Syst. Theory Appl. 2021, 103, 63. [Google Scholar] [CrossRef]
- García, A.; Delahaye, D.; Soler, M. Air Traffic Complexity Map based on Linear Dynamical Systems. 2020. Available online: https://hal-enac.archives-ouvertes.fr/hal-02512103 (accessed on 2 March 2023).
- Ribeiro, M.; Ellerbroek, J.; Hoekstra, J. Analysis of conflict resolution methods for manned and unmanned aviation using fast-time simulations. 9th SESAR Innov. Days 2019. Available online: https://www.sesarju.eu/sites/default/files/documents/sid/2019/papers/SIDs_2019_paper_69.pdf (accessed on 2 March 2023).
- Alharbi, A.; Petrunin, I.; Panagiotakopoulos, D. Deep Learning Architecture for UAV Traffic-Density Prediction. Drones 2023, 7, 78. [Google Scholar] [CrossRef]
- Tang, Y.; Xu, Y.; Inalhan, G. An Integrated Approach for On-Demand Dynamic Capacity Management Service in U-Space. IEEE Trans. Aerosp. Electron. Syst. 2022, 58, 4180–4195. [Google Scholar] [CrossRef]
- Capitán, C.; Pérez-León, H.; Capitán, J.; Castaño, Á.; Ollero, A. Unmanned Aerial Traffic Management System Architecture for U-Space In-Flight Services. Appl. Sci. 2021, 11, 3995. [Google Scholar] [CrossRef]
- Escalonilla, P.S.; Janisch, D.; Forster, C.; Büddefeld, M.; Teomitzi, H.E. Towards a continuous Demand and Capacity. Balancing Process for U-Space.
- Thipphavong, D.P.; Apaza, R.D.; Barmore, B.E.; Battiste, V.; Belcastro, C.M.; Burian, B.K.; Dao, Q.V.; Feary, M.S.; Go, S.; Goodrich, K.H.; et al. Urban Air Mobility Airspace Integration Concepts and Considerations. In Proceedings of the 2018 Aviation Technology, Integration, and Operations Conference, Atlanta, Georgia, 25–29 June 2018; p. 3676. [Google Scholar] [CrossRef]
- Vascik, P.; Hansman, R.; Dunn, N. Analysis of Urban Air Mobility Operational Constraints. J. Air Transp. 2018, 26, 1–14. [Google Scholar] [CrossRef]
- Chin, C.; Gopalakrishnan, K.; Balakrishnan, H.; Egorov, M.; Evans, A. Tradeoffs between Efficiency and Fairness in Unmanned Aircraft Systems Traffic Management. In Proceedings of the 9th International Conference on Research in Air Transportation, Virtual Event, 15 September 2020; The European Organisation for the Safety of Air Navigation (EUROCONTROL): Brussels, Belgium, 2020; pp. 1–8. [Google Scholar]
- Quinlan, J.R. Simplifying decision trees. Int. J. Hum. Comput. Stud. 1999, 51, 497–510. [Google Scholar] [CrossRef]
- Gokceoglu, C.; Nefeslioglu, H.A.; Sezer, E.; Bozkir, A.S.; Duman, T.Y. Assessment of landslide susceptibility by decision trees in the metropolitan area of Istanbul, Turkey. Math. Probl. Eng. 2010, 2010, 901095. [Google Scholar] [CrossRef]
- Schubert, E.; Sander, J.; Ester, M.; Kriegel, H.P.; Xu, X. DBSCAN revisited, revisited: Why and how you should (still) use DBSCAN. ACM Trans. Database Syst. 2017, 42, 1–21. [Google Scholar] [CrossRef]
- Allen, J.C. Sample size calculation for two independent groups: A useful rule of thumb. Proc. Singapore Healthc. 2011, 20, 138–140. [Google Scholar] [CrossRef]
- Alharbi, A.; Petrunin, I.; Panagiotakopoulos, D. Modeling and Characterization of Traffic Flow Patterns and Identification of Airspace Density for UTM application. IEEE Access 2022, 10, 130110–130134. [Google Scholar] [CrossRef]
- Grau, I.; Sengupta, D.; Matilde, M.; Lorenzo, G.; Nowe, A. Grey-Box Model: An Ensemble Approach for Addressing Semi-Supervised Classification Problems. In Proceedings of the 25th Belgian-Dutch Conference on Machine Learning, Kortrijk, Belgium, 12–13 September 2016; pp. 1–3. [Google Scholar]
- Robnik-Šikonja, M.; Kononenko, I. Explaining classifications for individual instances. IEEE Trans. Knowl. Data Eng. 2008, 20, 589–600. [Google Scholar] [CrossRef]
- Pintelas, E.; Livieris, I.E.; Pintelas, P. A Grey-Box ensemble model exploiting Black-Box accuracy and White-Box intrinsic interpretability. Algorithms 2020, 13, 17. [Google Scholar] [CrossRef]
- Wang, Q.; Shen, Y.P.; Chen, Y.W. Rule extraction from support vector machines. J. Natl. Univ. Def. Technol. 2006, 28, 106–110. [Google Scholar] [CrossRef]
- Vilone, G.; Longo, L. Classification of Explainable Artificial Intelligence Methods through Their Output Formats. Mach. Learn. Knowl. Extr. 2021, 3, 32. [Google Scholar] [CrossRef]
- Strobelt, H.; Gehrmann, S.; Pfister, H.; Rush, A.M. LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks. IEEE Trans. Vis. Comput. Graph. 2018, 24, 667–676. [Google Scholar] [CrossRef] [PubMed]
- Wongsuphasawat, K.; Smilkov, D.; Wexler, J.; Wilson, J.; Mané, D.; Fritz, D.; Krishnan, D.; Viégas, F.B.; Wattenberg, M. Visualizing Dataflow Graphs of Deep Learning Models in TensorFlow. IEEE Trans. Vis. Comput. Graph. 2018, 24, 1–12. [Google Scholar] [CrossRef]
- Xie, Y.; Gardi, A.; Sabatini, R. Hybrid AI-Based Demand-Capacity Balancing for UAS Traffic Management and Urban Air Mobility. In Proceedings of the AIAA Aviation 2021 Forum, Virtual Event, 2–6 August 2021; pp. 8–10. [Google Scholar] [CrossRef]
Level Number | % Congestion Range | Congestion Definition |
---|---|---|
Level-1 | 00–20% | Lowest |
Level-2 | 20–40% | Lower |
Level-3 | 40–60% | Medium |
Level-4 | 60–80% | Higher |
Level-5 | 80–100% | Highest |
Scenario | Congestion Levels | DBSCAN Parameters | Number of Congestion Clusters Detected | |
---|---|---|---|---|
eps | minPts | |||
3 | Lowest | 0.20 | 5 | 20 |
Lower | 0.20 | 5 | 24 | |
Medium | 0.20 | 5 | 19 | |
Higher | 0.12 | 5 | 4 | |
Highest | 0.40 | 5 | 3 |
Scenario 1 | |||
Congestion level | Maximum area (km2) | Mean Area (km2) | STD (km2) |
Lowest | 264.1664 | 17.8119 | 51.7698 |
Lower | 248.8680 | 23.3337 | 63.8795 |
Medium | 172.3195 | 12.3405 | 42.7612 |
Higher | 135.0970 | 20.9247 | 50.4984 |
Highest | 13.3878 | 7.3998 | 5.2072 |
Scenario 2 | |||
Congestion level | Maximum area (km2) | Mean Area (km2) | STD (km2) |
Lowest | 375.5019 | 22.0865 | 69.6584 |
Lower | 122.3201 | 13.4467 | 31.0048 |
Medium | 159.9153 | 25.2928 | 50.5788 |
Higher | 47.0391 | 10.6722 | 16.4358 |
Highest | 80.3852 | 42.2738 | 53.8976 |
Scenario 3 | |||
Congestion level | Maximum area (km2) | Mean Area (km2) | STD (km2) |
Lowest | 549.8083 | 47.8039 | 135.2778 |
Lower | 123.2273 | 13.2575 | 30.3633 |
Medium | 60.6678 | 10.6818 | 18.2609 |
Higher | 308.1550 | 10.6722 | 16.4358 |
Highest | 31.6595 | 12.7255 | 16.5392 |
Congestion Level | Cumulative Area | ||
---|---|---|---|
Scenario 1 | Scenario 2 | Scenario 3 | |
Highest | 498.7 | 750.9 | 956.1 |
Lowest | 22.2 | 84.55 | 38.1 |
Congestion Level | Capacity Ratio | ||
---|---|---|---|
Scenario 1 9–10 am | Scenario 2 10–11 am | Scenario 3 11–12 pm | |
Lowest | 6/4 = 1.50 | 8/1 = 8.0 | |
Lower | 2/3 = 0.66 | 4/6 = 0.66 | 2/8 = 0.25 |
Medium | 1/4 = 0.25 | 1/3 = 0.33 | 0/9 = 0.0 |
Higher | 0/2 = 0.0 | 1/5 = 0.20 | 0/1 = 0.0 |
Highest | 0/3 = 0.0 | 0/2 = 0.0 | 0/3 = 0.0 |
Methodology | Scores |
---|---|
Black Box | 0 |
Gray Box | 0.5 |
White Box | 1 |
Advisory DCM Components | Methodology | Transparency Type | Score |
---|---|---|---|
Congestion prediction | Deep learning (LSTM) | Black Box | 0 |
Congestion-level assignment | Simple rule based | White Box | 1 |
Congestion-subzone identification | Unsupervised clustering ML algorithm (BDSCAN) | Grey Box | 0.5 |
Airspace-capacity estimation | Rule based | White Box | 1 |
DCM decision | Decision tree | White Box | 1 |
Total explainability percentage (%) | 70% |
Congestion Level | Advisory System Efficiencies (%) | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Scenario 1 | Scenario 2 | Scenario 3 | ||||||||||
CG | SG | ECG | ESG | CG | SG | ECG | ESG | CG | SG | ECG | ESG | |
Lowest | 7 | 21 | 25 | 75 | 8 | 26 | 24 | 76 | 7 | 13 | 35 | 65 |
Lower | 2 | 15 | 12 | 88 | 4 | 24 | 14 | 86 | 3 | 21 | 13 | 87 |
Medium | 1 | 15 | 6 | 94 | 1 | 11 | 8 | 92 | 0 | 19 | 0 | 100 |
Higher | 0 | 7 | 0 | 100 | 1 | 11 | 8 | 92 | 0 | 4 | 0 | 100 |
Highest | 0 | 3 | 0 | 100 | 0 | 2 | 0 | 100 | 0 | 3 | 0 | 100 |
Metrics/Parameters | Ref [111] | Ref [129] | Proposed Model | |||
---|---|---|---|---|---|---|
Remark | Weight | Remark | Weight | Remark | Weight | |
Congestion Prediction Usage | No | 0 | No | 0 | Yes | 1 |
Dynamic Weather Considerations | No | 0 | No | 0 | Yes | 1 |
Airspace Structure Consideration | Yes | 1 | No | 0 | Yes | 1 |
Mission Priorities Consideration | No | 0 | No | 0 | Yes | 1 |
Path Planning Optimization | A * | 1 | GA | 1 | PSO | 1 |
Mission Scenarios | Yes | 1 | No | Yes | Real mission | 1 |
Conflict Resolution | Yes | 1 | Yes | 1 | No | 0 |
Physical Airspace Consideration | 20 × 20 km2 | 1 | 90 × 90 m2 | 1 | 64 × 64 km2 | 1 |
Traffic Flow Measurements | No | 0 | GA | 1 | DBSCAN | 1 |
Contingency landing | Yes | 1 | No | 0 | No | 0 |
Simulate Demand | Yes | 1 | Yes | 1 | Yes | 1 |
XAI of Decision Support | No | 0 | No | 0 | Post-hoc | 1 |
Visual Aid for Decision Support | No | 0 | No | 0 | 3D Graphs | 1 |
Capacity Efficiency % | Demand/Capacity Ratio 64% | 0.64 | Capacity Overload/DCB Ratio 55% | 0.55 | Capacity Gain Ratio 35% | 0.35 |
Total Counts | 7.64 | 5.55 | 11.35 | |||
Counts % | 55% | 40% | 81% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Alharbi, A.; Petrunin, I.; Panagiotakopoulos, D. Assuring Safe and Efficient Operation of UAV Using Explainable Machine Learning. Drones 2023, 7, 327. https://doi.org/10.3390/drones7050327
Alharbi A, Petrunin I, Panagiotakopoulos D. Assuring Safe and Efficient Operation of UAV Using Explainable Machine Learning. Drones. 2023; 7(5):327. https://doi.org/10.3390/drones7050327
Chicago/Turabian StyleAlharbi, Abdulrahman, Ivan Petrunin, and Dimitrios Panagiotakopoulos. 2023. "Assuring Safe and Efficient Operation of UAV Using Explainable Machine Learning" Drones 7, no. 5: 327. https://doi.org/10.3390/drones7050327
APA StyleAlharbi, A., Petrunin, I., & Panagiotakopoulos, D. (2023). Assuring Safe and Efficient Operation of UAV Using Explainable Machine Learning. Drones, 7(5), 327. https://doi.org/10.3390/drones7050327