Special Issue: Deep Learning in Remote Sensing Applications (Editorial)

Satellite remote sensing applications are data-intensive, and monitoring activities are the most common use of remote sensing data. This means that remote sensing is a data-intensive context in which the time series of data are widely available. Data science and, in particular, machine learning methods and algorithms are natural candidates for improving the performance of existing remote sensing applications and suggesting new applications. Sensors capture images across the electromagnetic spectrum, ranging from ultraviolet to infrared images. These images are taken in a variety of modes, from single-band to hyperspectral images. Statistical and signal processing methods, such as moving averages, linear regression, and principal component analysis, have been used to analyze these images since the early days of remote sensing. However, recently, new techniques originating from the field of data science, such as neural networks, support vector machines, and decision trees, have received a great deal of attention as image processing tools. This Special Issue comprises a collection of eleven papers on algorithmic innovations, data analysis, and hardware architectures involving machine learning methods and remote sensing applications. Several remote sensing applications, including agricultural production, monitoring of mudslides, solid waste monitoring, sea ice image segmentation, and building extraction are considered, exploring applications using images taken from satellites and UAVs (Unmanned Aerial Vehicles). Several methods are used to study these images, with neural networks and, in particular, convolutional neural networks being the most prevalent tools. In fact, the YOLO (You Only Look Once) algorithm, which is mentioned in several of the Special Issue papers, is based on convolutional neural networks.

This Special Issue consists of eleven high-quality papers which explore the use of artificial intelligence tools to interpret remote sensed data. Please see below for a brief commentary on each paper.

Whenchen Chen et al.’s paper, “Domain—Invariant Few-Shot Contrastive Learning for Hyperspectral Image Classification”, is concerned with Hyperspectral Image (HSI) classification. Labeled samples of hyperspectral data usually have limited availability due to the cost and practical difficulties involved. Consequentially, in this context, the use of standard deep learning methods is challenging due to the lack of training data [1,2,3].

Naturally, the idea of using synthetic data to train deep learning tools has arisen. The aforementioned paper suggests using Few-Shot Contrastive Learning (FSCL), a generative AI-based technique developed based on the well-known Generative Adversarial Networks (GANs). The FSCL technique is adapted to the context of remote sensing with the help of traditional image analysis tools, such as domain adaptation and feature extraction. The methodology developed has been tested in numerical experiments, with very promising results.

Sarentuya Bao et al. present “Enhancing a You Only Look One-Plated Detector via Auxiliary Textual Coding for Multi-Scale Rotating Remote Sensing Object in Transportation Monitoring Applications”, in which they outline a new detection method for remotely sensed multiscale objects with a rotating status, possibly in the presence of occlusions. Their proposed methodology is based on YOLO (You Only Look Once) and or a Swin Transformer. YOLO is a celebrated real-time object detection algorithm, founded on single-shot data and CNN (Convolutional Neural Network). It is characterized by the fact that the processing of the image and the detection of the objects contained in it are performed simultaneously. Swin Transformers have been introduced in natural language processing and are used here for the purpose of feature extraction. Previous use of Swin Transformer in the context of image analysis has been described in [4]. Extensive experiments on publicly available datasets have achieved excellent results.

Lin Gao et al. wrote “MSACN: A Cloud Extraction Method from Satellite Image Using Multiscale Soft Attention Convolutional Neural Network”.

In satellite remote sensing, the presence of clouds in the atmosphere may occlude access to ground truth information. This is certainly the case when optical sensors are used. In light of this, the detection and extraction of cloud in images is an important goal. The paper presents MSACN, a sophisticated software tool which combines a CNN and spatial information to obtain very good results in a series of numerical experiments on publicly available data. Note that the results obtained with MSACN are compared with those obtained with alternative methods and that this comparison confirms the quality of the work presented. For a comprehensive review of this kind of technique, see [5].

Shihao Ma et al., the authors of “Application of Enhanced YOLOX for Debris Flow Detection in Remote Sensing Images”, explore the detection of mudslides from remotely sensed data using deep learning. In particular, several versions of YOLO exist and are used in many different circumstances [6]. The paper primarily takes the form of an experimental study based on the processing of real data. A fair comparison of the results obtained is possible only after carefully reprocessing the data. The YOLOX version of the YOLO algorithm emerges as the recommended choice between the algorithms considered.

Yang Liu et al. present “A Practical Deep Learning Architecture for Large-Area Solid Wastes Monitoring Based on UAV Imagery”. The ongoing development of urban areas results in a great deal of solid waste production, and using UAV (Unmanned Aerial Vehicle) images to monitor this waste is an interesting possibility that is worthy of investigation [7].

The paper combines deep learning with traditional image processing tools, such as image fusion, to pursue its goals. The study area considered for testing the method developed in this work is a specific location in the Nanhu District of Jiaxing City in Zhejiang Province, China.

However, the results obtained are not expected to be associated with this specific study area. In fact, it should be possible to obtain similar results everywhere when suitable data are available.

Leibo Yu et al. wrote “A Rotating Object Detector with Convolutional Dynamic Adaptive Matching”. The usual sliding along a coordinate direction during CNN processing is a source of trouble when rotating object detection is considered. Note that aerial targets can be difficult to detect [8] and that rotating targets may be relevant in both military and civil situations. The paper attempts to eliminate these difficulties by implementing a kind of “dynamic convolution” combined with a spatial feature detection tool. The resulting detector is stabilized with several ad hoc artifacts. The developed tool is calibrated and tested on publicly available data. These experiments show the quality of the results obtained and the balance between accuracy and computational complexity achieved.

Xiaoyong Zhang et al. present “An Enhanced Dual Stream Network Using Multi Source Remote Sensing Imagery for Water Body Segmentation”, in which the problem of accurate surface water mapping is considered [9]. It is easy to imagine practical situations in which this issue is relevant. When many different shapes and scales of water surfaces are present in the same remotely sensed image, automatically extracting them from the image can be challenging. The paper combines a Swin Transformer with a CNN to produce a tool that can resolve this issue. After its implementation, the tool is tested on real data relating to water surface variations recorded during a period of several years in Mongolia, with good results. Note that there is no alternative to the use of remote sensing in these kinds of studies.

Bing Liu et al. wrote “Enhanced Atrous Extractor and Self-Dynamic Gate Network for Superpixel Segmentation”. A superpixel is a group of pixels with similar properties. Superpixel segmentation can substantially speed up the processing of remotely sensed images, reducing the computational cost of the processing [10]. The variety of possible situations present in the images render standard machine learning tools ineffective in such scenarios. To overcome this, the authors develop a dynamic method that adapts to the processed images. The proposed solution is implemented and tested in numerical experiments on real data, and state-of-the-art results are obtained.

Keliang Liu et al. present “MFFNet: A Building Extraction Network for Multi Source High Resolution Remote Sensing Data”. The extraction of buildings from remote sensing images is a well-known problem [11]. Its solution is relevant in many contexts, such as the monitoring of unauthorized constructions in urban areas. The presence of multiscale spatial information in the images to be processed degrades the performance of standard CNN-based methods. This paper presents MFFNet, a solution that implements a “pyramid mechanism” to efficiently handle the multiscale characteristics of the buildings present in the images. It is tested on several publicly available datasets, with very good results.

Yongjian Li et al. wrote “Improved Sea Ice Image Segmentation Using U²-Net and Dataset Augmentation”. Sea ice segmentation of remote sensing images has been studied since the early days of remote sensing engineering [12]. In recent years, interest in this problem has grown due to ongoing attempts to monitor the effects of global warning. This paper presents U²-Net, an innovative solution that combines multilayer convolutional networks with data augmentation and data expansion techniques. The proposed solution is tested on real images in different circumstances. Its robustness and accuracy are shown.

Finally, Qing Tian et al. present “GLFFNet: A Global and Local Features Fusion Network with Biencoder for Remote Sensing Image Segmentation”. The problem of resolving remote sensing images detecting the presence of small objects is considered [13]. The authors present GLFFNet, a tool based on deep learning and image fusion. Several experiments on publicly available datasets are discussed. High-quality results are obtained. The results justify the conceptual and computational complexity of the processing implemented in GLFFNet.

The Editor is grateful to and wishes to thank all the authors that have contributed to this Special Issue. The great innovations in the field of artificial intelligence and the dramatic climate change-related challenges associated with remote sensing will ensure that ‘artificial intelligence and remote sensing’ remains a highly active research field for the foreseeable future.

It is possible that a new Special Issue on this subject will be published in the next few years.

To the authors and readers of this Special Issue, the Editor says, ‘See you then’.