Automatic Classification of Spectra with IEF-SCNN

Abstract

Telescopes such as the Large Sky Area Multi-Object Spectroscopic Telescope and the Sloan Digital Sky Survey have produced an extensive collection of spectra, challenging the feasibility of manual classification in terms of accuracy and efficiency. To overcome these limitations, machine learning techniques are increasingly being utilized for automated spectral classification. However, these approaches primarily treat spectra as frequency domain signals, and lack robustness in low signal-to-noise ratio (S/N) scenarios and for small datasets of rare celestial objects. Moreover, they frequently neglect nuanced expert astronomical understanding. In this study, we draw inspiration from the human spectral discrimination process and propose a new model called the Image-EFficientNetV2-Spectrum Convolutional Neural Network (IEF-SCNN). IEF-SCNN combines spectral images using EfficientNetV2 with one-dimensional (1D) spectra through a 1DCNN. This integration effectively incorporates astronomical expertise into the classification process. Specifically, we plot the spectrum as an image and then classify it in a way that incorporates an attention mechanism. This attention mechanism mimics human observation of images for classification, selectively emphasizing relevant information while ignoring irrelevant details. Experimental data demonstrate that IEF-SCNN outperforms existing models in terms of the F1-score and accuracy metrics, particularly for low S/N (<6) data. Using progressive learning and an attention mechanism, the model trained on 12,000 M-class stars with an S/N below 6 achieved an accuracy of 87.38% on a 4000-sample test set. This surpasses traditional models (support vector machine with 83.15% accuracy, random forest with 65.40%, and artificial neural network with 84.40%) and the 1D stellar spectral CNN (85.65% accuracy). This research offers a foundation for the development of innovative methods for the automated identification of specific celestial objects, and can promote the creation of user-friendly software for astronomers who may not have computational expertise.

1. Introduction

In recent decades, astronomical survey telescopes such as the Sloan Digital Sky Survey (SDSS) [1] and the Large Sky Area Multi-Object Spectroscopic Telescope (LAMOST) [2] have amassed a substantial volume of spectral data. This invaluable spectral information has made significant contributions to the advancement of astronomical research. Nonetheless, the task of categorizing this vast array of spectra is formidable and presents a substantial challenge.

The MK classification system [3] is a widely utilized method for categorizing stars based on their temperature and brightness attributes. Initially, this classification relied on human experts who applied their experience and visual analysis. However, manual classification is susceptible to subjective errors and is notably inefficient. Consequently, template matching methods [4,5] were developed as an alternative approach. However, template matching may face performance challenges when handling intricate spectra.

The exponential growth in data samples coupled with advancements in computer hardware has spurred the development of numerous machine learning and deep learning-based methods that can address the challenges involved in spectral classification. Machine learning techniques have found widespread application in this context. For instance, in [6], an artificial neural network (ANN) was employed for spectral classification. In [7], the authors utilized principal component analysis to reduce dimensionality before employing ANN for classification. In cases with a low signal-to-noise ratio (S/N), as described in [8], an ANN was applied to process spectral data. Furthermore, ref. [9] introduced the MKCLASS program, which classifies spectra based on their deviation from the MK standard spectra. The random forest algorithm, as detailed in [10], was employed for stellar spectral classification and feature evaluation. In [11], the authors investigated the application of probabilistic neural networks, support vector machines, and K-means in spectral classification. Finally, ref. [12] proposed an entropy-based approach for classifying unbalanced spectral data. For an in-depth exploration of how machine learning is employed in the classification of astronomical spectra, ref. [13] provides a comprehensive review.

Deep learning techniques have gained traction in spectral classification. For instance, ref. [14] introduced deep neural networks for the prediction of star parameters, while [15] proposed an automated method for classifying stellar spectra in the optical range using convolutional neural networks (CNNs). In [16], the authors introduced a nine-layer one-dimensional stellar spectral CNN (1D SSCNN) architecture for the classification of F, G, and K stellar spectra. In [17], the authors improved CNN performance by incorporating a residual structure and an attention mechanism.

Nevertheless, prior studies have typically involved the extraction of features from raw spectral data (or direct input of raw data) and subsequent construction of classification models using processed data. These approaches primarily rely on statistical knowledge, and do not effectively incorporate the expertise of astronomers.

In this study, we draw inspiration from the process of human spectral discrimination and propose a novel approach to categorizing stellar spectra by treating them as images within the field of computer vision. We suggest that human experts classify spectra by visually examining these spectral “images”, focusing on salient features while disregarding less relevant ones. In our research, we emulate this process by plotting spectra into image representations and incorporating an attention mechanism. Recent advancements in image classification models, including MobileNetV2 [18], ViT [19], and EfficientNetV2 [20], have been noteworthy; in particular, EfficientNetV2 has demonstrated remarkable performance and efficiency, achieving top-1 accuracy on ImageNet ILSVRC2012 and surpassing ViT while training significantly more quickly with equivalent computing resources. Therefore, in light of the substantial volume of astronomical data, we opted to utilize EfficientNetV2 for processing spectral images. Furthermore, we explored a hybrid approach by combining 1D spectral data with the image representations plotted from the spectra. Motivated by this concept, we have devised a model built upon the foundations of EfficientNetV2 and a 1D SSCNN that is able to handle both images and 1D spectral data. In the case of EfficientNetV2, we tailored the attention mechanism and progressive learning components to better suit the characteristics of astronomical data. Simultaneously, for the 1D SSCNN component, we integrated a convolutional structure and employed an attention mechanism to enhance compatibility with EfficientNetV2.

The subsequent sections of this paper are structured as follows. Section 2 introduces the data and describes the preprocessing steps applied to the spectral data obtained from the SDSS database. Section 3 provides an overview of our methodology. Section 4 presents a comparative analysis of the experimental results obtained with our model in relation to other existing models. Section 5 comprises a series of discussions regarding the implications of the experimental findings. Finally, Section 6 concludes the paper and identifies potential areas for further research.

2. Data and Preprocessing

The SDSS represents a notably ambitious and successful survey project that provides access to spectra and deep multicolor images of celestial objects. Spectral data from SDSS DR16 are openly available to the public through the Casjobs server [21],1 enabling users to download data accompanied by relevant labels. In this particular study, we acquired stellar data including five classes: A, F, G, K, and M, as well as five subclasses within the M-type stars spanning the range from 0 to 4. These two sets of classifications correspond to two distinct categorization tasks. The initial step in data preprocessing entails the normalization of the data. To accomplish this, the flux values must be transformed into the range of 0 to 1 following the procedure outlined below:

$$S=\frac{f-f_{Min}}{f_{Max}-f_{Min}}$$

(1)

where f represents the flux and $f_{Min}$ and $f_{Max}$ denote the minimum and maximum flux values, respectively.

To maintain a consistent input format, we confined the wavelength range in the normalized data to span from 4000 Å to 9000 Å, resulting in a data size of 1 × 3522. As our model incorporates images, it is crucial to visualize the preprocessed data in the form of images as well. In the domain of deep learning, the resolution of an image can exert a significant influence on training results. In theory, higher-resolution images can encapsulate more information and potentially yield superior training results. Nevertheless, larger images may introduce challenges such as overfitting and protracted training times. Considering that EfficientNetV2 employs a resolution of 380 × 380 [20], we maintained this resolution in order to ensure consistency and avoid compromising performance. During the training phase, we employed a resolution of 300 × 300 to expedite the training process, with a transition to 380 × 380 during testing, aligning with the specifications outlined in the original EfficientNetV2 papers.

In addition to image resolution, the line width of the drawn spectrum plays a crucial role in influencing the experimental results. The term “line width” pertains to the thickness of the lines, and is measured in pixels. We utilized the matplotlib plot function2 to plot these spectral images; line width serves as one of the optional parameters within this function. While a larger line width facilitate improved information acquisition, it may sacrifice finer details; on the other hand, a narrower line width capture more complex details at the expense of slower convergence during training. Considering that EfficientNetV2 requires input data with three channels corresponding to RGB colors, and our spectral images inherently possess only one color, we addressed this disparity by generating three distinct images, each with a different line width, to populate the three color channels. This approach is elucidated in Figure 1, where the model simultaneously processes three distinct images of a single spectrum.

Figure 1 illustrates that the conventional model accepts the normalized 1D data obtained by extracting the spectral flux and resizing it to a consistent dimension as input. In this study, the model endeavors to enhance its classification capacity by incorporating the spectral image generated through spectral plotting alongside the 1D spectral data.

3. Method

In this study, we introduce an innovative model referred to as IEF-S CNN (Image EfficientNetV2-Spectrum) that seamlessly integrates spectral images and 1D spectral data to facilitate the classification process. The network architecture of EfficientNetV2 is harnessed to extract salient features from the image component. Furthermore, specialized attention and progressive learning modules are enhanced to cater specifically to the characteristics of astronomical spectral data. To capture features from the 1D spectral component, we devised a structured 1D convolutional neural network (1DCNN). Ultimately, the features extracted from both the images and the 1D spectral data are synergistically amalgamated to serve the classification objective.

3.1. EfficientNetV2

EfficientNet [22], proposed by Google in 2019, represents an advanced network architecture that has demonstrated remarkable performance in image classification. The success of EfficientNet can be attributed to two pivotal factors. First, Google harnessed its formidable computing power to conduct a thorough neural network search, culminating in the discovery of an enhanced network structure. Second, while traditional CNN models typically rely on augmenting the size of the convolutional kernel, the depth of the network, and the input resolution individually to improve performance, EfficientNet adopts a holistic approach by optimizing all three aspects concurrently, leading to superior results.

In this study, we employed EfficientNetV2, an improved iteration of EfficientNet [20], which introduces two notable enhancements. First, it accelerates the training speed of shallower networks. Second, it introduces a progressive learning technique that dynamically adjusts the regularization method (e.g., dropout) based on the training image size. This approach expedites the training process and elevates accuracy. In comparison to its predecessor, EfficientNetV2 achieves an eleven-fold increase in training speed while reducing the number of parameters to 1/6.8. EfficientNetV2 comprises a combination of MBConv [18] and FusedMBConv blocks. The fundamental element within MBConv is the mobile inverted bottleneck, which incorporates an attention mechanism. It comprises two standard convolutional layers, namely, a depthwise convolutional layer and a squeeze–excitation (SE) [23] attention mechanism. FusedMBConv addresses the inefficiency of the depthwise convolution layer in MBConv by replacing the previous normal convolution and depthwise convolution layers with a single normal convolution, facilitating hardware acceleration.

EfficientNetV2 models of various sizes have been developed and tailored to different tasks. In this specific study, recognizing the relatively lower complexity of spectral images compared to natural images, we chose to utilize the smallest model from the EfficientNetV2 family, EfficientNetV2-S.

3.2. Optimization of the Attention Mechanism

The behavior exhibited by experts who focus on specific features of the spectrum aligns with the concept of the attention mechanism in deep learning. The squeeze–excitation (SE) attention mechanism is employed by EfficientNetV2. However, recognizing the critical role played by the attention mechanism, we opted to replace the SE attention mechanism with two alternative attention mechanisms for the purpose of comparative experiments: efficient channel attention (ECA) [24] and convolutional block attention module (CBAM) [25].

The SE attention mechanism centers its attention on the interplay between different channels. Following a global pooling operation and two fully connected layers, the model discerns the significance of various channels. Nevertheless, the SE attention mechanism compresses the input feature map, leading to a reduction in dimensionality, which can impede the model’s capacity to learn dependencies between channels. In contrast, the ECA attention mechanism bypasses dimensional reduction and adaptively captures local cross-channel interactions through the use of 1D convolution, facilitating the extraction of interchannel dependencies. The size of the 1D convolution kernel, denoted as k, represents a hyperparameter that is varied according to the number of channels, denoted as C. To address this variability, ECA proposes an adaptive approach to determine the optimal size for the 1D convolution kernel.

The calculation of the convolution kernel k is depicted in the following equation, with $\gamma$ set to 2 and b set to 1. It is noteworthy that the ECA attention mechanism is conceptually and operationally simpler than the SE attention mechanism, exerting a negligible effect on network processing speed.

$$\begin{array}{c}\hfill k=\psi \left(C\right)={\left|\frac{{log}_2\left(C\right)}{\gamma }+\frac{b}{\gamma }\right|}_{odd}\end{array}$$

(2)

In comparison to the SE and ECA attention mechanisms, CBAM is a module that amalgamates both spatial and channel information. In theory, employing CBAM may yield superior experimental results, although it necessitates a greater computational effort. In this study, considering their significance in our research, we included all three attention mechanisms in the experimental phase to facilitate a thorough comparison and evaluation.

3.3. 1DCNN

The structure of the 1DCNN designed in this study is visually presented in Figure 2. It includes four convolutional units along with the ECA (efficient channel attention) mechanism. Additionally, there are flattened and fully connected layers that produce 1 × 1280 features. The 1DCNN effectively extracts features in 1280 dimensions by progressively expanding the number of channels using these four convolutional units. These extracted features undergo compression within the network through a maximum pooling layer. Subsequently, the flattened and fully connected layers convert these compressed features into 1280-dimensional representations. To enhance the feature extraction process, two ECA attention mechanism modules were integrated into the convolutional process.

Each convolutional unit within the 1DCNN consists of four essential components: a convolutional layer, batch normalization [26], rectified linear unit (ReLU) activation function, and dropout layer. The convolutional layer is tasked with feature extraction; in this study, we employed a 1 × 16 convolutional kernel. Batch normalization was incorporated to expedite training and facilitate network convergence, and can additionally help to control gradient issues such as explosion and vanishing gradients while mitigating overfitting.

Following batch normalization, the ReLU activation function is applied. It promotes efficient training via gradient descent and backpropagation. This choice of activation function effectively mitigates problems related to gradient explosion and vanishing gradients. The ReLU activation function is mathematically represented by the following equation, and its calculation process is straightforward, simplifying the overall computational process:

$$f\left(x\right)=max(0,x).$$

(3)

3.4. Overall Model Structure and Training Process

The overall architecture of the model, depicted in Figure 2, can be divided into two components: EfficientNetV2 and 1DCNN. Both components extract features in the form of 1 × 1280 dimensions, which are subsequently concatenated to form 1 × 2560 features. These combined features pass through a fully connected layer, followed by a softmax activation function, resulting in a probability distribution for each category.

All preprocessed spectra, having previously undergone normalization, are represented as 1 × 3522 dimensional data. These normalized spectra serve as the input for the entire model, and training occurs in two parallel aspects:

1. The input data undergo transformation into three 380 × 380 images, each with varying line widths. These three images are then amalgamated into a single image through the RGB channels, collectively serving as input to the EfficientNetV2 backbone. The output of this process yields a 1 × 1280 feature matrix denoted as ${\chi }_{\mathrm{image}}$.

2. The data are directly used as the input of the 1DCNN, and the output is a 1 × 1280 feature matrix denoted as ${\chi }_{\mathrm{spec}}$.

The outputs from both parts are concatenated to form ${\chi }_{\mathrm{both}}$. The classification results are obtained after passing ${\chi }_{\mathrm{both}}$ through a fully connected layer.

The training of our model is divided into two steps. First, in the initial stage of training, which uses two thirds of the training epoch, the loss for the output results of $y_{\mathrm{image}}^{\mathrm{pred}}$ and $y_{\mathrm{spec}}^{\mathrm{pred}}$ is calculated using the cross-entropy loss [27]. The loss is defined in Equations (4) and (5), where C represents the number of classes and $y^{\mathrm{true}}$ is the true label of the data. Second, in the later stage of training, using one third of the training epoch, the cosine similarity ${\mathrm{Loss}}_{\cos }$ between ${\chi }_{\mathrm{image}}$ and ${\chi }_{\mathrm{spec}}$ is calculated in order to facilitate the fusion of the features from both parts (${\chi }_{\mathrm{image}}$ and ${\chi }_{\mathrm{spec}}$). These two losses are defined in Equations (6) and (7).

$${\mathrm{Loss}}_{\mathrm{image}}\left(y_{\mathrm{image}}^{\mathrm{pred}},y^{\mathrm{true}}\right)=-log\frac{exp\left(y_{\mathrm{image}}^{\mathrm{pred}},y^{\mathrm{true}}\right)}{{\sum }_{c=1}^{C}exp\left(y_{\mathrm{image},c}^{\mathrm{pred}}\right)}$$

(4)

$${\mathrm{Loss}}_{\mathrm{spec}}\left(y_{\mathrm{spec}}^{\mathrm{pred}},y^{\mathrm{true}}\right)=-log\frac{exp\left(y_{\mathrm{spec}}^{\mathrm{pred}},y^{\mathrm{true}}\right)}{{\sum }_{c=1}^{C}exp\left(y_{\mathrm{spec},c}^{\mathrm{pred}}\right)}$$

(5)

$${\mathrm{Loss}}_{\mathrm{both}}\left(y_{\mathrm{both}}^{\mathrm{pred}},y^{\mathrm{true}}\right)=-log\frac{exp\left(y_{\mathrm{both}}^{\mathrm{pred}},y^{\mathrm{true}}\right)}{{\sum }_{c=1}^{C}exp\left(y_{both,c}^{\mathrm{pred}}\right)}$$

(6)

$${\mathrm{Loss}}_{\cos }=\frac{{\sum }_{i=1}^n\left({\chi }_{\mathrm{image}}^i\times {\chi }_{\mathrm{spec}}^i\right)}{\sqrt{{\sum }_{i=1}^n{\left({\chi }_{\mathrm{image}}^i\right)}^2}\times \sqrt{{\sum }_{i=1}^n{\left({\chi }_{\mathrm{spec}}^i\right)}^2}}$$

(7)

$$\mathrm{Loss}=\alpha \times {\mathrm{Loss}}_{\mathrm{both}}+\beta \times {\mathrm{Loss}}_{\mathrm{image}}+\gamma \times {\mathrm{Loss}}_{\mathrm{spec}}+\delta \times {\mathrm{Loss}}_{\cos }$$

(8)

In summary, the comprehensive loss calculation for our model is depicted in Equation (8). Following numerous experiments, we determined the hyperparameters $\alpha$, $\beta$, $\gamma$, and $\delta$, which we respectively set as {0, 0.2, 0.8, 0} during the initial training phase and {0.2, 0, 0.4, 0.2} during the later stages of training.

3.5. Optimization of Progressive Learning

Progressive learning is a methodology that entails commencing training with low-resolution images during the early stages and gradually increasing the resolution of training images over time. The primary objective of progressive learning is to enhance training speed, although it often comes at the cost of decreased accuracy. To mitigate this decline in accuracy, EfficientNetV2 incorporates adaptive tuning regularization techniques such as dropout to simultaneously improve both training speed and accuracy. Specifically, adaptive tuning regularization is applied when dealing with smaller training image sizes, whereas stronger data augmentation is employed when working with larger training image sizes.

In this study, we introduce an additional dimension termed the S/N of the spectrum to augment this process. As illustrated in Figure 3, higher S/N data are initially utilized to facilitate the fitting process during training, while lower S/N data are introduced later to capture more complex and complex features.

4. Results

4.1. Dataset and Experimental Environment

Our experiments included two datasets. As outlined in

Table 1. Description of dataset-A and dataset-B.

Dataset	Type	No. of Instances	Training (%)	Validation (%)	Test (%)
dataset-A	A	5000	60	20	20
	F	5000	60	20	20
	G	5000	60	20	20
	K	5000	60	20	20
	M	5000	60	20	20
dataset-B	M0	4000	60	20	20
	M1	4000	60	20	20
	M2	4000	60	20	20
	M3	4000	60	20	20
	M4	4000	60	20	20

, Dataset-A comprised 25,000 data entries, including A, F, G, M, and K stars. All spectra within this dataset exhibited S/N exceeding 5. Dataset-A was specifically employed for the stellar classification task. In contrast, dataset-B comprised 20,000 data entries dedicated exclusively to M stars, including subclasses 0–4. The S/N distribution for dataset-B is visualized in Figure 4, demonstrating that all spectra in this dataset possess S/N values less than 6. Dataset-B was utilized for the subclass classification of M stars. Across both datasets, 60% of the data were allocated for training, 20% for validation, and 20% for testing.

The classification performance of each model was evaluated using two primary metrics: accuracy and F1-score. Accuracy represents the ratio of correct predictions, including both true positives and true negatives, to the total number of predictions. Precision quantifies the proportion of true positive samples among all samples classified as positive, while recall represents the ratio of correctly classified positive samples to all true positive samples. The F1-score, a comprehensive performance metric, combines precision and recall. These metrics are mathematically defined as follows:

$$\mathrm{Accuracy}=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}$$

(9)

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$

(10)

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

(11)

$$\mathrm{F}1\text{-}\mathrm{score}=\frac{2\times (\mathrm{Precision}\times \mathrm{Recall})}{\mathrm{Precision}+\mathrm{Recall}}$$

(12)

In the above equations, the terms are defined as follows: TP (true positive) refers to the count of positive samples predicted correctly, TN (true negative) signifies the count of negative samples predicted correctly, FP (false positive) represents the count of negative samples inaccurately predicted as positive, and FN (false negative) denotes the count of positive samples inaccurately predicted as negative. These metrics collectively provide a comprehensive assessment of the model’s classification performance.

All programs in this study were implemented in Python and executed on a desktop computer equipped with a 2.80 Ghz Intel(R) Core(TM) i9-10900F processor, 32 GB of RAM, a 64-bit Windows operating system, and an RTX 2080s GPU for computation.

4.2. Model Optimization

We embarked on a series of experiments designed to compare our model with other established techniques, including SVM [28], random forest [29], ANN [6], and 1D SSCNN [16]. Each experiment leveraged the two datasets outlined in Table 1. For all the models under evaluation (SVM, random forest, ANN, and 1D SSCNN) we engaged in diligent parameter tuning using both datasets. To achieve this, we conducted a comprehensive grid search across all parameters, utilizing the accuracy as the guiding metric to pinpoint the optimal parameter configurations. The experimental results of the models under various parameters can be found in Appendix A.

The model parameters of SVM include C and $\gamma$. In our study, we opted for the radial basis function (rbf) kernel function for SVM to achieve superior performance. The parameter ranges for each model were as follows:

1.

C: 0.1, 1, 10, 100, 1000, 10,000

2.

$\gamma$: 1, 0.1, 0.01, 0.001, 0.0001

For SVM, a total of 25 parameter combinations were considered. Each experiment involved five-fold cross-validation; the optimal model parameters for dataset-A were {‘C’: 10, ‘$\gamma$’: 0.01, ‘kernel’: ‘rbf’}, while the optimal parameters for dataset-B were {‘C’: 1000, ‘$\gamma$’: 0.0001, ‘kernel’: ‘rbf’}.

For the random forest model, the parameters included the number of trees in the forest, max_depth, min_samples_split, and min_samples_leaf. The parameter ranges were as follows:

1.

max_depth: 20, 30, 40

2.

min_samples_leaf: 1, 2, 5

3.

min_samples_split: 2, 5, 10

4.

n_estimators: 300, 400, 500

There were a total of 81 parameter combinations. Each experiment involved five-fold cross-validation; the optimal model parameters for dataset-A were {‘max_depth’: 30, ‘min_samples_leaf’: 2, ‘min_samples_split’: 2, ‘n_estimators’: 300}, while the optimal model parameters for dataset-B were {‘max_depth’: 20, ‘min_samples_leaf’: 1, ‘min_samples_split’: 2, ‘n_estimators’: 400}.

For ANN, the model parameters included the activation function, $\alpha$, hidden_layer_sizes, learning_rate, and solver. The parameter ranges were as follows:

1.

activation function: relu, tanh, logistic

2.

$\alpha$: 0.0001, 0.05

3.

hidden_layer_sizes: (100,), (200,), (300,)

4.

learning_rate: constant, adaptive

5.

solver: adam, sgd

A total of 72 parameter combinations were considered. Each experiment was subjected to five-fold cross-validation; the optimal model parameters for dataset-A were {‘activation function’: ‘relu’, ‘$\alpha$’: 0.0001, ‘hidden_layer_sizes’: (100,), ‘learning_rate’: ‘adaptive’, ‘solver’: ‘adam’}, while the optimal model parameters for dataset-B were {‘activation function’: ‘relu’, ‘$\alpha$’: 0.0001, ‘hidden_layer_sizes’: (100,), ‘learning_rate’: ‘adaptive’, ‘solver’: ‘adam’}.

The model parameters of the 1D SSCNN included the learning rate and dropout rate; the range of each model parameter was as follows:

1.

learning_rate: 0.01, 0.005, 0.001, 0.0005, 0.0001

2.

dropout_rate: 0, 0.1, 0.3, 0.5

There were 20 parameter combinations in total. The optimal parameters for dataset-A were {learning_rate: 0.0005, dropout_rate: 0.3}, while the optimal parameters for dataset-B were {learning_rate: 0.0005, dropout_rate: 0.1}.

Finally, regarding our proposed model, the initial learning rate was set to 0.01, the cosine annealing algorithm [30] was employed to adjust it to 0.001 during training, the dropout rate was chosen as 0.2, and the stochastic gradient descent method [31] was utilized.

4.3. Experimental Results

Table 2. Classification result for dataset-A. The text in bold indicates the highest performance according to the metrics (Training Time, Accuracy, and F1-Score).

Algorithm	Training Time	Accuracy (%)	F1-Score (%)
			A	F	G	K	M
SVM	19  s	91.78	95.761	78.781	86.810	97.451	99.299
Random forest	1 min 12 s	90.10	93.147	77.472	86.838	94.998	97.295
ANN	1 min 14 s	93.62	95.470	86.004	91.576	96.604	98.323
1D SSCNN	2 h 23 min 36 s	93.04	96.445	83.070	88.388	97.520	99.450
Our model (only image)	10 h 18 min 20 s	93.86	95.090	86.446	91.263	97.558	98.901
Our model (only spectra)	2 h 11 min 2 s	94.26	96.263	87.107	91.361	97.600	98.855
Our model	10 h 34 min 52 s	94.62	96.293	87.691	91.913	97.908	99.201
Our model (without preprocessing)	10 h 31 min 43 s	94.20	96.348	87.192	90.927	97.410	99.053

presents the experimental results for dataset-A, including the accuracy and F1-score for each class. Our model exhibits superior accuracy compared to the other tested models. Furthermore, we conducted a comparison within our model to examine the performance when using only image data or only spectral data. The results indicate that combining both images and spectra leads to improved experimental results. We additionally compared the results with and without data preprocessing, revealing that all metrics achieved with preprocessing outperform those without preprocessing. This underscores the importance of preprocessing in our experiment. However, in light of the high S/N of dataset-A and the relatively insignificant differences among all models, we place greater emphasis on the results for dataset-B.

For dataset-B, we specifically conducted subclass classification for M-class stars to further assess our model’s performance. Unlike dataset-A, the majority of training data in dataset-B feature low S/N, providing an opportunity to exhibit our model’s effectiveness in challenging conditions. We employed progressive training, as described in Section 3.5, to maximize the utilization of these data. The experimental results in

Table 3. Classification results for dataset-B. The text in bold indicates the highest performance according to the metrics (Training Time, Accuracy, and F1-Score).

Algorithm	Training Time	Accuracy -(%)	F1-Score -(%)
			M0	M1	M2	M3	M4
SVM	18  s	83.15	85.941	79.156	79.332	82.658	88.761
Random forest	2 min 54 s	65.40	71.647	55.659	60.414	61.974	76.097
ANN	2 min 8 s	84.40	88.732	82.157	81.707	82.440	87.344
1D SSCNN	1 h 44 min 22 s	86.13	87.555	82.447	84.197	84.072	89.449
Our model (only image)	8 h 33 min 24 s	84.98	87.010	80.419	82.473	84.867	90.120
Our model (only spectra)	1 h 49 min 22 s	86.08	89.097	83.696	85.327	84.328	88.138
Our model	8 h 37 min 21 s	86.75	89.169	83.744	85.587	85.178	90.087
Our model (progressive learning)	7 h 22 min 32 s	87.38	89.497	84.045	85.732	86.582	90.932
Our model (without preprocessing)	8 h 55 min 2 s	85.30	88.239	82.491	84.000	83.963	87.796

consistently demonstrate our model’s superior performance across all metrics. Notably, the model’s performance improves and the training time decreases with the application of progressive learning.

Finally, we replaced the attention mechanism in the EfficientNetV2 network structure with CBAM [25] to assess the effect of this modification. This comparison experiment was conducted on dataset-B, with only the attention mechanism being changed while keeping all other parameters constant. The results in

Table 4. Classification with attention mechanism for dataset-B. The text in bold indicates the highest performance according to the metrics (Training Time, Accuracy, and F1-Score).

Algorithm	Training Time	Accuracy (%)	F1-Score (%)
			M0	M1	M2	M3	M4
Our model (SE)	5 h 47 min 21 s	86.65	89.969	85.133	85.965	83.833	88.177
Our model (ECA)	5 h 2 min 43 s	85.18	88.384	82.571	83.505	83.544	87.635
Our model (CBAM)	7 h 22 min 32 s	87.38	89.497	84.045	85.732	86.582	90.932

show that the experimental performance remains nearly unchanged when using ECA [24] instead of SE [23], while the training time is reduced. On the other hand, the experimental results with CBAM exhibit improvement along with a significant increase in training time, aligning with our earlier speculations in Section 3.2.

5. Discussion

Table 2 illustrates the excellent performance of all models on dataset-A, which is characterized by high S/N. Data with high S/N tend to yield good results even with simpler methods, making it challenging for models to exhibit substantial improvements. Consequently, our primary focus shifted to dataset-B, where most of the data have low S/N.

The random forest method performs well on dataset-A and poorly on dataset-B. This discrepancy may be attributed to random forest’s unsuitability for data with low S/N or subtle differences. However, because we did not specifically analyze the performance of the random forest model, we do not explore this matter further here. The random forest method was excluded from the subsequent analyses of model performance.

To emphasize our model’s advantages on low S/N data, we divided the test set of dataset-B into two subsets based on the S/N ratio. One subset contained data with an S/N ratio less than 2, while the other contained data with an S/N ratio greater than 2. We calculated the accuracy for each subset, as shown in Figure 5. Our model, especially when trained with the progressive training method, exhibits a significant advantage on the low S/N subset. In contrast, all models show minimal differences on the high S/N subset. These findings further validate our model’s superiority in handling low S/N data.

Additionally, we generated confusion matrices for each model using the test set of dataset-B, which are depicted in Figure 6 and Figure 7. Our model does not exhibit substantial improvements on M0 and M1, corresponding to relatively high average S/N ratios. However, it demonstrates significant enhancements on M2, M3, and M4, which represent low S/N scenarios.

Furthermore, we analyzed the contribution of each component of our model by plotting confusion matrices for each component with and without the use of progressive learning, as shown in Figure 8. On M4, which has the lowest average S/N ratio, the model with both image and spectral components outperforms the models with only one of these components. This suggests that our model can effectively combine the strengths of both image and spectral data, resulting in improved overall accuracy. Additionally, our model with progressive learning exhibits enhanced accuracy in most categories, particularly M4, thanks to the progressive learning technique.

In summary, the main contributions of this study are as follows:

1. We propose a novel approach to spectral classification using spectral images utilizing EfficientNetV2 and optimizing it for astronomical data, which yields promising results.

2. We design a model that effectively combines spectral images and 1D spectral data, which is particularly beneficial for low S/N data; this is particularly significant in light of the prevalence of the low S/N ratios prevalent in astronomical data.

6. Conclusions

In this study, we present a novel approach to spectral classification that departs from traditional methods. Our model integrates both spectral images and 1D spectral data, leveraging the features extracted from both for classification. For the image component, we employ EfficientNetV2 with specific optimizations for astronomical data. To process the raw spectral data, we introduce a new 1DCNN architecture.

We evaluated our model using spectra from SDSS DR16 and conducted experiments on two datasets: one for stellar spectral classification, and another for M-class stellar subclass classification. Our model achieved remarkable results, with an accuracy of 94.62% on the stellar spectral classification test set (dataset-A) and 87.38% on the M-class stellar subclass classification test set (dataset-B). These results outperformed the other tested models, particularly in the challenging scenario of M-class subclass classification with low S/N data.

Our model’s strength lies in its ability to handle data with lower S/N levels; this is a crucial aspect in astronomy, where a significant portion of the data exhibit low S/N levels.