Unlocking the Potential of Explainable Artificial Intelligence in Remote Sensing Big Data

1. Introduction

In the ever-evolving landscape of artificial intelligence and big data, the concept of explainable artificial intelligence (XAI) [1] has emerged as a crucial factor in ensuring transparency, accountability, and trustworthiness. In this Special Issue, we delve into the significance of XAI in the realm of remote sensing big data and its implications for various applications, emphasizing its role in bridging the gap between advanced algorithms and human understanding.

As the volume of data generated continues to surge, ranging from initial image processing to understanding high-level data and knowledge discovery [2], the need for advanced AI systems to interpret, analyze, and make decisions from these data has become increasingly apparent. However, these AI systems often operate as “black boxes”, making it challenging for humans to comprehend their decision-making processes. At its core, XAI addresses this issue by making AI models or systems more transparent and interpretable. As discussed in some papers [3,4], it provides insights into why a particular decision was made, enabling humans to trust and act upon AI recommendations. By achieving this, XAI paves the way for a broader acceptance and integration of AI in decision-making processes across various domains.

Since DARPA [1] came up with the concept of XAI, algorithms have come a long way and made remarkable progress. Typical models and algorithms in XAI research mainly include logic rules, decision attribution, and internal structure representation of deep learning models. Logic rules for XAI mainly refer to rule-extractor methods (such as regional tree [5], DeepRED-rule [6]), knowledge graph methods (such as knowledge-aware path recurrent networks [7] and knowledge embedding [8]), etc. Decision attribution for XAI is based on assessing the influence of the input features on the output to determine their importance in decision making, and mainly includes methods based on disturbance [9,10], backpropagation [11], and agent models [12]. Internal structural representations [13] of XAI models aim to understand the role and structure of the data flowing through these networks, including the interpretation of hidden representations of hidden layers, the behavior of individual neurons, etc. In the field of remote sensing, much research on XAI is related to attribution research, such as in [14,15,16], which may be because this field is more concerned about the causes of the information extraction effect.

This Special Issue presented by MDPI’s Remote Sensing journal delves into the intersection of XAI and big data, highlighting the latest research and developments in this critical field. Articles in this issue explore a multitude of topics, including XAI techniques tailored to the analysis of large datasets, the applications of XAI in remote sensing, and discussions on the mechanism that exerts an AI effect in big data contexts.

2. An Overview of Published Articles

This Special Issue mainly explores the advantages and interpretability of current deep-learning-based data-processing algorithms from the aspects of RS image segmentation, RS image classification, RS image enhancement, etc.

The articles by Hasanpour (contribution 6), Xia (contribution 9), and Fang (contribution 10) are all based on semantic segmentation. In contribution 6, the effectiveness of attention mechanisms (AMs) for improving building segmentation in remote sensing big data using convolutional neural network (CNN) backbones is explored, showing that AMs significantly improve quantitative metrics and their visualization aligns with the metrics. In contribution 9, based on the idea of feature-driven classification, the landslide extraction model of a fully convolutional spectral–topographic fusion network (FSTF-Net) based on a deep convolutional neural network of multi-source data fusion is proposed, which takes into account the topographic factors (slope and aspect) and the normalized difference vegetation index (NDVI) as the multi-source data inputs by which to train the model. The model has high reliability and robustness. In contribution 10, the authors introduce a novel approach called conditional co-training (CCT) for unsupervised remote sensing image segmentation in coastal areas, addressing the challenge of limited pre-labeled training data. It offers enhanced explainability and transparency compared to traditional supervised methods and demonstrates a strong performance and efficiency on real-world coastal remote sensing datasets.

The articles by Papadopoulou (contribution 1), Chen (contribution 2), Guo (contribution 4), and Feng (contribution 7) relate to the classification problem. In contribution 1, temporal Convolutional Neural Networks (CNNs) and Recurrent and 2D Convolutional Neural Networks (R-CNNs) are compared to the well-established Random Forest (RF) machine learning algorithm. This comparison assesses the uncertainty of the classification results using an entropy measure and the spatial distribution of the classification errors. In contribution 2, the authors introduce an interactive and open-source Python tool to enhance land cover mapping and monitoring using Google Earth Engine, emphasizing the importance of explainable AI (XAI) for improving model performance and enabling globally accessible and free applications of remote sensing technologies. In contribution 4, to improve the ship classification accuracy and training efficiency, the authors propose a CNN-based ship classification method by creating a new convolutional module, namely the Inception-Residual Controller (IRC) module. A convolutional neural network was built based on the IRC module to extract image features and establish a ship classification model, which is 3% more accurate than the traditional network model. In contribution 7, the authors introduce a bidirectional flow decision tree (BFDT) module that is combined with Convolutional Neural Networks (CNNs) to create a reliable and interpretable framework for remote sensing image scene classification, addressing the lack of explainability and trustworthiness in deep-learning-based approaches, achieving good generalization, and providing correctable and trustworthy results.

The articles by Wang (contribution 3) and Alizadeh (contribution 5) can be interpreted as expanded applications of AI in inverse problems. In contribution 3, the authors present an unsupervised feature extraction method for reducing the dimensionality of hyperspectral images similar to [11], combining endmember extraction and spectral band clustering. In contribution 5, a novel GAN-based image generation model, CFM-GAN, is proposed to improve the image quality of crucial components of high-voltage transmission lines captured by UAVs, achieving high-resolution images with rich semantic details and outperforming existing models.

In addition, contribution 8 is an expanded application of AI for disease prediction. It presents a tree-based interpretable machine learning model that adopts the bagging concept for robust COVID-19 case/death predictions.

3. Conclusions

In conclusion, we believe that explainable AI (XAI) and remote sensing big data are highly complementary in the development of global Earth observation technology. XAI provides a means to understand and interpret the decision-making processes of AI models, while remote sensing data provide rich and diverse sources of information about the Earth’s environment and human activities. Although much of the research in remote sensing is about decision attribution of deep learning, we believe that the research on decision logic and intrinsic structure explanations of deep learning will become more important in the future. Especially in the era of remote sensing big data, important future trends revolve around combining XAI with geoscience knowledge graphs, developing deep learning capable of reasoning, and finally realizing the goal of trusted artificial intelligence in remote sensing. This study will unlock the potential of XAI in remote sensing big data.