Automated High-Precision Recognition of Solar Filaments Based on an Improved U²-Net

Abstract

Solar filaments are a significant solar activity phenomenon, typically observed in full-disk solar observations in the H-alpha band. They are closely associated with the magnetic fields of solar active regions, solar flare eruptions, and coronal mass ejections. With the increasing volume of observational data, the automated high-precision recognition of solar filaments using deep learning is crucial. In this study, we processed full-disk H-alpha solar images captured by the Chinese H-alpha Solar Explorer in 2023 to generate labels for solar filaments. The preprocessing steps included limb-darkening removal, grayscale transformation, K-means clustering, particle erosion, multiple closing operations, and hole filling. The dataset containing solar filament labels is constructed for deep learning. We developed the Attention U²-Net neural network for deep learning on the solar dataset by introducing an attention mechanism into U²-Net. In the results, Attention U²-Net achieved an average Accuracy of 0.9987, an average Precision of 0.8221, an average Recall of 0.8469, an average IoU of 0.7139, and an average F1-score of 0.8323 on the solar filament test set, showing significant improvements compared to other U-net variants.

1. Introduction

Solar filaments are a typical solar activity phenomenon in the solar atmosphere. These filaments consist of plasma on the solar surface with temperatures ranging from 6000K to 8000K, approximately 100 times cooler than the surrounding corona. The composition of filaments is similar to that of the chromosphere, and they form ring-like structures that extend into the corona. When observed against the space background at the solar limb, they are referred to as prominences [1]. Solar filaments sometimes align with the polarity inversion lines of the solar photospheric magnetic field. Filaments located in active regions are often unstable and prone to eruption. Eruptive filaments are closely associated with solar flares and coronal mass ejections (CMEs) [2], and are considered manifestations of the same physical process of magnetic energy release at different times and heights in the solar atmosphere [3]. The eruption of solar filaments is not only regarded as the core of CMEs but also as central to the research of CMEs [4]. CMEs can cause damage to satellites and other spacecraft, disrupt communication, navigation, and remote sensing on Earth, and damage power transmission equipment, leading to large-scale power outages. Furthermore, CMEs may jeopardize the normal operation of space stations and pose a threat to the safety of astronauts [5].

Solar filaments are often observed in full-disk H-alpha images. Many solar telescopes around the world can observe the H-alpha wavelength, such as those in the global high-resolution H-alpha network, which includes the Big Bear Solar Observatory in California, Kanzelhöhe Solar Observatory in Austria, the Catania Astrophysical Observatory (CAO) in Italy, Meudon and Pic du Midi Observatories in France, and the Huairou Solar Observing Station (HSOS) and the Yunnan Astronomical Observatory (YNAO) in China [6]. Other notable telescopes include the Daniel K. Inouye Solar Telescope [7] and the planned European Solar Telescope [8]. To obtain accurate details of solar filaments in the H-alpha band, solar telescopes have been launched into heliocentric orbit. The Chinese H-alpha Solar Explorer (CHASE), launched in 2021, carries the H-alpha Imaging Spectrograph, which provides solar filament images with clear details in the H-alpha wavelength [9,10].

With the dramatically increasing volume of solar observational data, achieving automated high-precision recognition and segmentation of solar filaments, while preserving as much detail as possible, is crucial. This allows for the investigation of the formation, deformation, and eruption of solar filaments. Understanding the dynamics and evolution of solar filaments and their relationship with the magnetic fields in solar active regions, as well as predicting space weather phenomena such as solar flares and coronal mass ejections, are of paramount importance [1].

Artificial intelligence technologies, represented by convolutional neural networks [11], have broad applications in image recognition and segmentation, including in fields such as biomedical imaging [12], satellite remote sensing [13], and astronomical image processing [14]. In solar physics, deep learning-based image segmentation techniques are frequently applied to the identification of solar filaments, sunspots, and solar plages [15]. Zhu [16] proposed a solar filament segmentation method based on the U-net neural network. Liu et al. [17] implemented solar filament segmentation using a U-net improved with generative adversarial networks. Xin et al. [18] introduced attention mechanisms, deep supervision networks, and Inception modules into V-Net to achieve solar filament segmentation. Guo et al. [19] proposed a deep learning model based on CondInst for the detection and classification of solar filaments, facilitating the classification of isolated and non-isolated filaments. Zheng et al. [20] utilized U-net for solar filament segmentation and employed the CSRT algorithm to track their evolution.

2. Data Preprocessing and Dataset Construction

The CHASE telescope’s H-alpha Imaging Spectrograph captures full-disk solar images, storing the data in FITS format. Each dataset includes high-quality solar grayscale images obtained at 118 wavelength nodes within the 655.97 nm to 656.59 nm range. This range is centered around the H-alpha wavelength node (656.28 nm), where the detailed features of solar filaments can be clearly observed. The following data preprocessing steps apply to all 118 solar grayscale images within each dataset.

2.1. Limb-Darkening Removal

Solar filaments are plasma structures with lower temperatures than the corona. When located at the solar limb, they are referred to as prominences and appear brighter against the space background. The Sun also exhibits limb darkening, a phenomenon where the brightness decreases from the center of the solar disk to the edge. The darker regions at the edge may resemble the morphology of solar filaments, which can hinder the precise identification of filament details using K-means clustering. Therefore, to minimize the occurrence of false negatives during deep learning training, it is necessary to fit the solar disk’s edge using a second-order cosine function. This approach helps retain most of the brighter regions of the solar surface, excluding the edge. Finally, a solar image of size 2048 ∗ 2048 is obtained, with the sun center located at the center of the image and the full solar disk radius of 950.

2.2. Grayscale Transformation and K-Means Clustering

The grayscale values of the solar images are stretched to the range of 0–255. Since K-means clustering is sensitive to regions with concentrated pixel distributions, stretching the grayscale values of the solar H-alpha monochromatic images after limb-darkening removal enhances the contrast between solar filaments and the solar surface. This makes it easier for K-means clustering to identify filaments accurately and efficiently.

K-means clustering [21] is an unsupervised deep learning method primarily used to partition a dataset into K pre-specified clusters. The goal of K-means clustering is to maximize the similarity of samples within each cluster while maximizing the difference between samples from different clusters. This method effectively distinguishes solar filaments, sunspots, and solar plages, and is widely used in solar physics. After limb-darkening removal and grayscale stretching, K-means clustering analysis is performed on the solar images at all 118 wavelength nodes, with the number of clusters K set to 50. A lower K value would be insufficient to differentiate solar filament signals from noise. The cluster that best matches the detailed features of solar filaments is manually selected from the K clusters obtained through K-means clustering.

2.3. Particle Erosion, Multi-Closing Operations, and Hole Filling

The initial segmented images of solar filaments obtained from K-means clustering may contain small background particles, as well as holes and fragments within the filaments. These small particles may originate from incomplete differentiation of tiny chromospheric fibers, while the holes and fragments may result from the imperfect recognition of detailed features of solar filaments by K-means clustering. Using OpenCV, an image processing library, particles with a pixel area smaller than 60 are firstly eroded from the initial segmented images. Secondly, multi-closing operations are applied to connect fragmented pieces of solar filaments. Finally, the contours are filled to eliminate holes within the solar filaments. This process yields the final labeled images of solar filaments, effectively minimizing noise while preserving the filament signals as much as possible.

2.4. Dataset Construction

In the study, we utilized 150 full-disk solar images captured in the H-alpha wavelength band by the Chinese H-alpha Solar Explorer’s imaging spectrometer in 2023, downloaded from the Solar Science Data Center of Nanjing University [22]. The images underwent preprocessing steps, including limb-darkening removal, grayscale transformation, K-means clustering, particle erosion, multiple closing operations, and hole filling, to generate corresponding labeled images. Figure 1 illustrates the specific steps of data preprocessing, while Figure 2 illustrates the results of the solar full-disk H-alpha images before and after preprocessing. All images were uniformly resized to 512 × 512 pixels to ensure consistency. Randomly, a total of 560 selected images were assigned to the training set, 22 images to the validation set, and the other 18 images were used as a test set to evaluate the efficiency of the neural network in deep learning. Due to the lack of symmetry in convolutional neural networks when processing images, the dataset was augmented through automatic rotations, horizontal flips, and vertical flips in the training set.

3. Deep Learning

3.1. Neural Networks

3.1.1. U-Net Family

U-Net is a neural network architecture proposed by Ronneberger et al. [12] for semantic segmentation in deep learning, originally applied in biomedical image segmentation. Inspired by an encoder–decoder structure, U-Net exhibits a symmetrical U-shaped appearance, hence its name. The network consists of two main parts: the encoder and the decoder. The encoder consists of a series of convolutional and pooling layers designed to gradually reduce the spatial dimensions of the input image and extract features. With each convolutional operation, the number of channels increases progressively, while pooling layers reduce the spatial resolution of the feature maps while retaining important features. The symmetric decoder consists of a series of upsampling and convolutional layers. Upsampling layers are used to restore the spatial resolution of the feature maps to the original input image size.

A distinctive feature of U-Net is its use of skip connections, which connect corresponding features from the encoder to the decoder. This mechanism helps address issues of information loss and spatial inconsistency in segmentation tasks, enabling the network to learn richer features and improving overall performance. The architecture of U-Net is illustrated in Figure 3.

Ozan et al. [23] proposed Attention U-Net based on the U-Net architecture, incorporating the attention mechanism [24] into the upsampling path for medical image segmentation. This enhancement improves the neural network’s capability of feature selection, thereby enhancing segmentation accuracy, training efficiency, and model generalization. The architecture of Attention U-Net is illustrated in Figure 4. Building on the U-Net architecture, Qin [25] proposed U²-Net, which utilizes a two-level nested U-Net structure. The classical U²-Net employs residual U-blocks (RSU) to replace the standard convolutional blocks used as encoders or decoders in U-Net. Each RSU block consists of an encoder and decoder with a structure similar to U-Net, combining different receptive field sizes. Each RSU block can be viewed as a “small U-Net”, incorporating residual connections to merge local and multi-scale features through summation. The height of the RSU blocks ranges from 7 to 4, decreasing with the height of the U²-Net encoder or decoder. This design captures more contextual information at different scales, significantly increasing the network’s depth and enhancing training and generalization efficiency. Consequently, U²-Net is widely applicable to various image segmentation tasks in solar observation environments. The architecture of the residual U-block is shown in Figure 5. The architecture of U²-Net is illustrated in Figure 6.

Li et al. [26] proposed AM-SegNet, a model designed for segmentation and quantification of high-resolution X-ray images. AM-SegNet combines a lightweight convolutional block with a custom attention mechanism to perform image segmentation and feature quantification on X-ray imaging data collected from various synchrotron experiments. AM-SegNet outperforms other state-of-the-art segmentation models in terms of accuracy, speed, and robustness.

3.1.2. Attention Gate

Attention mechanisms are widely used in machine translation and computer vision [23]. In the field of semantic segmentation, the attention mechanism is incorporated into U-Net in the form of attention gates, creating Attention U-Net. Attention gates receive a set of low-level features (typically high-resolution feature maps) from the encoder path and a set of high-level features (typically low-resolution feature maps) from the decoder path. Through convolution operations and ReLU activation functions, the input feature maps are mapped to a common space.

Specifically, a 1 × 1 convolution is applied to the low-level features to reduce the number of channels, and a 1 × 1 convolution is also applied to the high-level features to map both sets of features to the same dimension. The mapped features are then added together to obtain a new feature map. This combined feature map is processed using the ReLU activation function to set all negative values to zero, adding nonlinearity. Subsequently, a 1 × 1 convolution and a sigmoid activation function are applied to compute the attention weights for each pixel location. Finally, these attention weights are multiplied with the original low-level feature map to produce the weighted feature map.

Attention gates can enhance the response to target regions in semantic segmentation, suppress irrelevant background noise, and dynamically adjust based on the input image’s features. This enhances the neural network’s robustness when handling complex backgrounds or small target images. The attention mechanism is shown in Figure 7.

3.1.3. Attention U²-Net

Building upon U²-Net and Attention U-Net, this work adopts the U-Net encoder–decoder architecture as the backbone network. Attention gates are introduced into the upsampling stages of the RSU blocks, replacing the standard convolutional blocks of U-Net with RSU blocks enhanced by attention gates. Additionally, attention gates are incorporated into the upsampling stages of the backbone network, resulting in the creation of Attention U²-Net. This architecture not only captures more contextual information at different scales but also enhances the neural network’s robustness in handling complex backgrounds or small object images. The architecture of Attention U²-Net is illustrated in Figure 8.

Sigmoid is defined in Equation (1):

$$f\left(x\right)={\displaystyle \frac{1}{(1+e^{-x})}}$$

(1)

ReLU is defined in Equation (2):

$$f\left(x\right)=\left\{\begin{array}{cc}0,\hfill & x<0\hfill \\ x,\hfill & x\ge 0\hfill \end{array}\right.$$

(2)

3.2. Experimental Setup and Training Parameters

The neural network training was conducted on a Windows 10 operating system using Python 3.11. The software environment included the parallel computing architecture CUDA 12.1, the deep learning library PyTorch 2.1, and the GPU acceleration library cuDNN 9.0.10. The hardware environment comprised an Intel Xeon Platinum 8173M CPU with 128 GB RAM and an NVIDIA Quadro RTX4000 GPU with 8 GB VRAM. During training, the learning rate was set to 0.0001, the batch size was set to 2, and the number of iterations was set to 150. At the beginning of each training session, the loss functions for both the training set and validation set are recorded and compared after each iteration. Training is terminated when the loss functions for both sets gradually decrease and converge, in order to prevent the model from overfitting on the test set.

3.3. Optimization Function and Loss Function

After selecting U²-Net as the neural network for image segmentation in this study, it was essential to determine the most effective optimizer and loss function to facilitate network convergence. The Adam optimizer provided by PyTorch was chosen in the study [27]. Adam is an adaptive learning rate optimization algorithm that adjusts the learning rate dynamically using estimates of the first and second moments of the gradients. The loss function selected for this study was the binary cross entropy loss (BCELoss) function.

The Adam optimizer is defined in Equation (3):

$$\begin{array}{cc}\hfill & g={\nabla }_{{\theta }_{k-1}}L\left(\theta \right)\hfill \\ \hfill & m_k={\beta }_1{m}_{k-1}+(1-{\beta }_1)g\hfill \\ \hfill & v_k={\beta }_2{v}_{k-1}+(1-{\beta }_2)g\odot g\hfill \\ \hfill & {\widehat{m}}_k={\displaystyle \frac{m_k}{1-{\beta }_1^k}}\hfill \\ \hfill & {\widehat{v}}_k={\displaystyle \frac{v_k}{1-{\beta }_2^k}}\hfill \\ \hfill & {\theta }_k={\theta }_{k-1}-{\displaystyle \frac{\eta }{\sqrt{{\widehat{v}}_k}+ϵ}}{\widehat{m}}_k\hfill \end{array}$$

(3)

The BCELoss function is defined in Equation (4):

$$H_p\left(q\right)=-{\displaystyle \frac{1}{N}}\sum _{i=1}^N{y}_i·log\left(p\left(y_i\right)\right)+(1-y_i)·log(1-p\left(y_i\right))$$

(4)

3.4. Evaluation Metrics

This study evaluates and compares the performance of four neural networks: U-Net, Attention U-Net, U²-Net, and Attention U²-Net. The distinctions among these networks are detailed in

Table 1. Four neural networks.

Net	Non Attention Gate	With Attention Gate
Classic U-Net	U-Net	Attention U-Net
RSU Nested U-Net	U2-Net	Attention U2-Net

. Precision, Recall, IoU [28], F1 score, and Accuracy are used to evaluate the performance of the models. TN, TP, FP, and FN represent true negative, true positive, false positive, and false negative measurements, respectively. The calculation methods for each evaluation metric are shown in the

Table 2. Five evaluation metrics.

Evaluation Metrics	Formula
IoU	${\displaystyle \frac{TP}{TP+FN+FP}}$
Precision	${\displaystyle \frac{TP}{TP+FP}}$
Recall	${\displaystyle \frac{TP}{TP+FN}}$
F1	${\displaystyle \frac{2\ast Precision\ast Recall}{Precision+Recall}}$
Accuracy	${\displaystyle \frac{TP+TN}{TP+FP+FN+TN}}$

.

3.5. Result

A qualitative comparison of the segmentation results of solar H-alpha grayscale images U-Net, Attention U-Net, U²-Net, and Attention U²-Net is found in Figure 9. The average values of evaluation metrics Precision, Recall, IoU, and F1 for the segmentation of solar filaments on the test set by the four neural networks, showing a quantitative comparison of the segmentation results are provided in

Table 3. Average metrics of each neural network on the test set.

Net	Precision	Recall	IoU	F1	Accuracy
U-Net	0.8041	0.8198	0.6790	0.8077	0.9985
Attention U-Net	0.8072	0.8336	0.6883	0.8143	0.9985
U2-Net	0.8137	0.8270	0.6924	0.8171	0.9986
Attention U2-Net (ours)	0.8221	0.8469	0.7139	0.8323	0.9987

. Figure 10 shows the Precision–Recall curve of four networks. Figure 11 and Figure 12 show the loss function curves for the validation sets over each iteration during the learning process of the four neural networks.

4. Discussion

Based on the comparison of evaluation metrics for U-Net, Attention U-Net, U²-Net, and Attention U²-Net on the test set, Attention U²-Net achieved an average Accuracy of 0.9987, an average Precision of 0.8221, an average Recall of 0.8469, an average IoU of 0.7139, and an average F1 score of 0.8323. Except for the Accuracy value, the other four metrics, which are used to evaluate the inference accuracy of the neural network, showed significant improvement compared to the other neural networks. The results demonstrate that all U-Net series networks achieve high-precision segmentation of different solar filaments. However, when it comes to multi-scale features, Attention U²-Net exhibits superior segmentation accuracy compared to U-Net, Attention U-Net, and U²-Net. This improvement is evident in both the complex contour details of larger filaments and the fine-grained features of smaller filaments. Moreover, under the same dataset and with random data augmentation, Attention U²-Net demonstrates a faster iteration speed. These advantages can likely be attributed to the combined effect of the residual U-Block and attention gate.

5. Conclusions

In this study, we utilize full-disk H-alpha solar images from the CHASE solar telescope for the automatic and high-precision recognition of solar filaments based on deep learning. The preprocessing steps include limb-darkening removal, grayscale transformation, K-means clustering, grain erosion, multiple closing operations, and hole filling, resulting in binarized label images of solar filaments. By incorporating the attention mechanism into U²-Net, we developed Attention U²-Net. Compared to other methods, Attention U²-Net significantly improves the recognition accuracy of full-disk H-alpha solar filaments over other U-Net variants. The high-precision automatic recognition of the detailed features of solar filaments is crucial for understanding the dynamics and evolution of solar filaments, their connection with solar active region magnetic fields, and predicting solar flares and coronal mass ejections.