A Prior Knowledge-Enhanced Deep Learning Framework for Improved Thermospheric Mass Density Prediction

Abstract

Accurate thermospheric mass density (TMD) prediction is critical for applications in solar-terrestrial physics, spacecraft safety, and remote sensing systems. While existing deep learning (DL)-based TMD models are predominantly data-driven, their performance remains constrained by observational data limitations. This study proposes ResNet-MSIS, a novel hybrid framework that integrates prior knowledge from the empirical NRLMSIS-2.1 model into a residual network (ResNet) architecture. The incorporation of NRLMSIS-2.1 enhanced the performance of ResNet-MSIS, yielding a lower root mean squared error (RMSE) of 0.2657 × ${10}^{-12}$ kg/m³ in TMD prediction compared with 0.2750 × ${10}^{-12}$ kg/m³ from ResNet, along with faster convergence during training and better generalization on Gravity Recovery and Climate Experiment (GRACE-A) data, which was trained and validated on the CHAllenging Minisatellite Payload (CHAMP) TMD data (2000–2009, altitude of 305–505 km, avg. 376 km) under quiet geomagnetic conditions (Kp ≤ 3). The DL model was subsequently tested on the remaining CHAMP-derived TMD observations, and the results demonstrated that ResNet-MSIS outperformed both ResNet and NRLMSIS-2.1 on the test dataset. The model’s robustness was further demonstrated on GRACE-A data (2002–2009, altitude of 450–540 km, avg. 482 km) under magnetically quiet conditions, with the RMSE decreasing from 0.3352 × ${10}^{-12}$ kg/m³ to 0.2959 × ${10}^{-12}$ kg/m³, indicating improved high-altitude prediction accuracy. Additionally, ResNet-MSIS effectively captured the horizontal TMD variations, including equatorial mass density anomaly (EMA) and midnight density maximum (MDM) structures, confirming its ability to learn complex spatiotemporal patterns. This work underscores the value of merging data-driven methods with domain-specific prior knowledge, offering a promising pathway for advancing TMD modeling in space weather and atmospheric research.

1. Introduction

On 4 February 2022, SpaceX experienced the loss of 38 out of the 49 Starlink satellites it had recently launched due to the satellites being deployed at very low altitudes of 200 km before being boosted by thrusters, combined with a minor geomagnetic storm that caused intense atmospheric drag [1]. Hapgood et al. [2] provide detailed discussions on the factors leading to this event. This scenario, along with the ongoing progression toward the solar maximum of solar cycle 25, highlights the need for the accurate forecasting, modeling, and understanding of how increased solar activity affects thermospheric mass density (TMD) [3]. Indeed, atmospheric drag constitutes the predominant perturbation force and the greatest source of uncertainty for celestial bodies orbiting the Earth at altitudes beneath 1000 km [4]. Thus, attaining an exact assessment of atmospheric drag is of paramount importance to various space-related endeavors, encompassing collision prevention strategies, reentry trajectory predictions, estimates of orbital lifespan, propagation of orbits, and object tracking [5]. As a critical component of Earth’s thermosphere environment, TMD exhibits intricate temporal and spatial fluctuations along with responsive attributes, predominantly steered by solar ultraviolet radiation, solar wind, magnetospheric energetic particles, and wave phenomena arising from the lower atmospheric strata. Given its pivotal role as the main contributor to atmospheric drag, TMD exerts a substantial impact on the orbit determination and operational safety of satellites and spacecraft in low Earth orbit (LEO), particularly during extreme space weather scenarios [6].

Extensive research efforts have been devoted over the past decades to improving the accuracy of TMD prediction. Empirical models have been widely developed and applied in both research and industry, including the mass spectrometer and incoherent scatter radar model (MSIS) class models [7], the Jacchia class models [8], and the drag temperature model (DTM) class models [9]. In recent years, noteworthy progress has been made in the MSIS class models, particularly the NRLMSIS-2.1 model, which incorporates new empirical data on nitric oxide [10]. Moreover, significant advances have been made with new observational data and space weather indices [6,11,12]. Satellite laser ranging (SLR) measurements to spherical geodetic satellites [13] have also been shown to be valuable for TMD determination. However, despite continuous refinements and updates with new observational data and space weather indices, empirical models, including NRLMSISE-00 [7], still exhibit notable limitations. For instance, the average error of NRLMSISE-00 typically ranges from 15% to 20%, with even higher errors observed during geomagnetic storms [14]. Notably, model density prediction errors can be particularly high during the recovery phases of extreme geomagnetic storms [15]. These challenges originate from the complex interplay of various factors influencing the thermosphere, which poses significant difficulties for empirical models in accurately capturing its variations under diverse space weather conditions [16]. Moreover, the complex physical mechanisms associated with space weather events can induce substantial changes in TMD, which are often characterized by a variety of space environment indices. Consequently, achieving accurate TMD predictions remains a formidable challenge in this highly complex and multidimensional environment [17,18].

Deep learning (DL), a data-driven nonlinear fitting technique, excels at uncovering complex patterns within high-dimensional data [19]. When applied to TMD modeling, DL offers the potential to extract valuable insights from observations and enhance the understanding of underlying physical characteristics. Over the past decade, DL has demonstrated outstanding performance in solar-terrestrial physics applications [20,21,22,23]. Studies in computer science also indicate that increasing the complexity of network architectures often improves model accuracy [24]. However, challenges such as limited interpretability and a tendency toward overfitting persist in DL applications [25,26]. In TMD modeling and prediction, models with relatively shallow architectures have already outperformed traditional empirical and theoretical approaches [27,28,29,30], indicating the promising potential of DL in this field.

Most TMD models based on DL are data-driven, with performance heavily dependent on observations. Incorporating prior information from empirical model predictions, calculated using corresponding CHAllenging Minisatellite Payload (CHAMP) observation [31] data, can enhance DL models by improving the prediction accuracy and interpretability. This study leveraged NRLMSIS-2.1 predictions as prior knowledge to enhance a Res-Net-based TMD model (ResNet-MSIS). The DL models were trained on CHAMP data combined with NRLMSIS-2.1 predictions, as described in Section 2. Their performance is evaluated in Section 3 on both the training and test datasets, with an emphasis on capturing specific horizontal TMD variations, namely the equatorial mass density anomaly (EMA) [32] and the midnight density maximum (MDM) [33]. Conclusions are made in Section 4.

2. Data and Methods

2.1. Data Description

Deep learning (DL) relies on data-driven modeling, and thus requires a large volume of observational data for effective training. The TMD dataset from CHAMP observations, spanning 2000–2009 under magnetically quiet conditions (Kp ≤ 3), and covering the altitude range of 305–505 km (average altitude: 376 km), provides approximately 22 million TMD measurements with a temporal resolution of 10 s. This extensive dataset, along with corresponding MSIS predictions obtained during the same conditions, formed the foundation for training the DL model. This study incorporated predictions from NRLMSIS-2.1 (referred to as MSIS in the rest of the paper), which were calculated based on the corresponding CHAMP dataset parameters, as prior knowledge to augment a DL model for improved TMD prediction. CHAMP observations have been prominently featured in studies exploring TMD variability and modeling [34,35]. The Gravity Recovery and Climate Experiment (GRACE-A) satellite [31] provided TMD observations in the altitude range of 450–540 km (average altitude: 482 km) during the period from 2002 to 2009, under magnetically quiet conditions, with a temporal resolution of 10 s. The observations from both satellites, along with the empirical models, enabled us to objectively evaluate the DL model’s generalization ability.

In this study, local solar time (LST), day of year (DoY), geographic latitude (Lat), and geographic altitude (Alt) were used as key indicators for the DL model. These indicators collectively account for the effects of the Earth’s rotation, seasonal changes, geographic location, and altitude on the state of the thermosphere. LST has been used in thermospheric studies and is particularly relevant for capturing local time-dependent phenomena such as EMA [29]. To more comprehensively reflect the impact of solar activity on the thermosphere, daily and 81-day moving average values of solar radiance at the wavelength of 10.7 cm (F10.7 and F10.7a) were selected as indicators to represent the solar activity intensity [36].

Traditionally, indices such as Ap and Kp are used in TMD models to indicate the energy injection process during disturbed periods [7,8,37]. Ap is a 24-h planetary geomagnetic index that quantifies geomagnetic activity, while Kp is an index that measures the level of geomagnetic disturbance on a global scale, both of which are indicative of space weather activity. However, these indices have a limited temporal resolution of three hours. Furthermore, these indices fail to adequately describe the Joule heating and particle heating processes at high latitudes as they are derived from low- and mid-latitude observations [18]. Therefore, the Auroral Electrojet (AE) and Symmetric Hourly (SYM-H) indices were used as indicators of energy injection in the DL model. These indices, with higher temporal resolutions (1-min for both SYM-H and AE), are more compatible with high-time resolution satellite measurements, which can assist the DL model in capturing the features of TMD variation. The SYM-H index is used by the DL model to provide a guide for the global TMD variation baseline during disturbance times by measuring the intensity of the ring current [38]. It also captures perturbations from magnetopause (Chapman–Ferraro) currents, especially during compressions of the magnetosphere caused by interplanetary shocks [39]. Meanwhile, the AE index, indicating the intensity of the auroral electrojet variation, primarily focuses on energy injection in the high-latitude region [40,41]. With the AE and SYM-H indices as inputs, the DL model is able to better represent the response process of TMD to disturbance events at various latitudes. Considering the delayed effect of the space environment indices will greatly improve the estimation capacity of the DL model [42]. Therefore, the influence of the historical information (daily mean, current value, and the mean values of the previous 0–3, 3–6, 6–9, 9–12, 12–33, and 33–57 h, respectively) of SYM-H and AE was also considered. Data standardization helped balance the impact of different parameters on the loss function. Trigonometric functions were employed to make the time and space inputs periodic. Specific processing strategies for the parameters are detailed in

Table 1. Input parameters used in the neural network model.

Parameter Name	Parameter Value
Input 1–2: Local Solar Time (LST), in hours (h)	$\mathrm{s}\mathrm{i}\mathrm{n}\left({\displaystyle \frac{2\mathsf{\pi }}{24}}\mathrm{L}\mathrm{S}\mathrm{T}\right),\mathrm{c}\mathrm{o}\mathrm{s}\left({\displaystyle \frac{2\mathsf{\pi }}{24}}\mathrm{L}\mathrm{S}\mathrm{T}\right)$
Input 3–4: Day of Year (DoY)	$\mathrm{s}\mathrm{i}\mathrm{n}\left({\displaystyle \frac{2\mathsf{\pi }}{365.25}}\mathrm{D}\mathrm{o}\mathrm{Y}\right),\mathrm{c}\mathrm{o}\mathrm{s}\left({\displaystyle \frac{2\mathsf{\pi }}{365.25}}\mathrm{D}\mathrm{o}\mathrm{Y}\right)$
Input 5–6: Latitude (Lat), in degrees (°)	$\mathrm{s}\mathrm{i}\mathrm{n}\left({\displaystyle \frac{2\mathsf{\pi }}{180}}\mathrm{L}\mathrm{a}\mathrm{t}\right),\mathrm{c}\mathrm{o}\mathrm{s}\left({\displaystyle \frac{2\mathsf{\pi }}{180}}\mathrm{L}\mathrm{a}\mathrm{t}\right)$
Input 7: Altitude (Alt), in kilometers (km)	$(\mathrm{A}\mathrm{l}\mathrm{t}-250)/300$
Input 8–9: F10.7 and F10.7a, in solar flux units (sfu)	${\log }_{10}\left(\mathrm{F}10.7\right)/3,{\log }_{10}\left(\mathrm{F}10.7\mathrm{a}\right)/3$
Input 10–16: SYM-H (with the historical information), in nanoteslas (nT)	$\mathrm{SYM}\text{-}\mathrm{H}/500$
Input 17–23: AE (with the historical information), in nanoteslas (nT)	${\log }_{10}\left(\mathrm{A}\mathrm{E}\right)/5$
$\mathrm{Input} 24\text{:} \mathrm{NRLMSIS}\text{-}2.1 ({\mathrm{T}\mathrm{M}\mathrm{D}}_{\mathrm{M}\mathrm{S}\mathrm{I}\mathrm{S}}$), in kg/m3	${\log }_{10}\left({\mathrm{T}\mathrm{M}\mathrm{D}}_{\mathrm{M}\mathrm{S}\mathrm{I}\mathrm{S}}\right)$
Out: Thermospheric Mass Density (TMD), in kg/m3	${\log }_{10}\left(\mathrm{T}\mathrm{M}\mathrm{D}\right)$

.

2.2. Deep Learning Model

The MSIS series models are among the most widely used series for atmospheric temperature and density predictions. The latest model in this series is NRLMSIS 2.1, which builds upon the successful NRLMSISE-00 model. In this study, the TMD data output from the NRLMSIS-2.1 model, calculated using the same input parameters as those from the CHAMP dataset (covering the period 2000–2009 and the altitude range of 305–505 km), were incorporated as prior knowledge into the DL model. The ResNet-MSIS and ResNet models share a similar architecture, with the key distinction that the ResNet-MSIS model incorporates the TMD values from the NRLMSIS-2.1 model as prior information to enhance the prediction process. Additionally, both models take into account historical information such as the daily mean, current value, and the mean values of the SYM-H and AE indices over the previous 0–3, 3–6, 6–9, 9–12, 12–33, and 33–57 h. These historical features were included to capture the delayed impact of geomagnetic activity on thermospheric density. The specific DL model metrics used are shown in

Table 2. The training metrics used for the ResNet and ResNet-MSIS models.

Model	Basic Environmental Parameters	Geomagnetic Indices	MSIS Density Added
ResNet	LST, DoY, Lat, Alt, F10.7, F10.7a	SYM-H, AE (with the historical information)	No
ResNet-MSIS	LST, DoY, Lat, Alt, F10.7, F10.7a	SYM-H, AE (with the historical information)	Yes

.

The performance of DL models is closely linked to the number of stacked layers. As the depth increases, the number of internal parameters grows, enhancing the model’s ability to fit the training dataset [24]. However, deeper models often encounter the vanishing/exploding gradient problem, which hinders convergence [43]. Additionally, the deep model may lead to higher training errors [44]. To address these problems, the deep residual learning framework incorporates “shortcut connections”, as shown in Figure 1, to mitigate these issues [44]. Models using residual learning are referred as residual networks (ResNet) [45], originally proposed by He et al. (2016), who introduced the concept of residual blocks, where layers learn a residual mapping with reference to the input of each block. This design facilitates the preservation of identity mappings—where the input is passed directly to the output without modification—and enables the effective training of very deep networks. The residual design significantly alleviates degradation problems and has profoundly influenced the development of deep neural architectures. ResNet has demonstrated superior performance in TMD prediction compared with the standard multilayer perceptron (MLP) model [46], a traditional neural network model consisting of multiple layers of perceptrons, which is commonly used for regression tasks in machine learning. This study employed the ResNet framework to construct the DL model for TMD (see Figure 1).

Figure 1 shows the architecture of ResNet-MSIS and ResNet, where MSIS outputs are incorporated as prior information in ResNet-MSIS. The first and second hidden layers consisted of two ResNet blocks, aimed at constructing a deeper network structure to capture more complex patterns in the data. The output dimensions of the first and second hidden layers were 128 and 256, respectively. These ResNet blocks utilize shortcut connections, which facilitate the flow of gradients during backpropagation and improve the training efficiency. The shortcut connections consist of dense layers, where each neuron is connected to every neuron in the previous layer, and a dropout layer, which randomly deactivates a fraction of neurons during training to reduce overfitting. Following the ResNet blocks, dense layers with an output dimensionality of 32 were employed to process the learned features before connecting to the output layer, ensuring accurate predictions.

Dataset construction is a critical step in DL. Regarding the time interval sampling, considering that the CHAMP satellite requires 131 days [47] to collect data for all local times on ascending and descending orbit arcs, a longer time interval is needed to obtain a dataset with uniform spatial and temporal characteristics. However, solar radiation, geomagnetic disturbances, and atmospheric waves have profound impacts on TMD at different time scales [48,49], leading to significant feature variations on daily and longer time scales. Considering these environmental factors, fixed time-interval sampling may lead to the overemphasis or underemphasis of specific features within the dataset. To address this, random sampling from all CHAMP observations was employed to ensure a broader representation of the spatial and temporal conditions in the training dataset, achieving a more uniform distribution across different datasets. In this study, approximately 22 million data points were available, with 70% allocated for training, 15% for testing, and 15% for validation. This division ensured that the model was trained on a large, diverse dataset, allowing for effective evaluation on data that the model had not encountered during training, which helped assess its generalization ability, reducing the risk of overfitting. The randomly sampled datasets allow for the assessment of different model architectures in terms of their ability to extract meaningful observational information.

2.3. DL Model Performance Evaluation

Given that the logarithm of the ratio of empirical density to the thermospheric density calculated using accelerometers on the CHAMP satellite is typically Gaussian [50], the base-10 logarithm of the thermospheric density was taken, followed by mean-variance normalization. This process maintains the relative magnitudes of the data, thereby helping the model better learn the characteristics while preserving the relative proportions. The normalization process can be expressed as follows:

$${\stackrel{-}{\rho }}_{\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}}={\displaystyle \frac{{\log }_{10}\left(\rho \right)-\mu }{\sigma }}$$

(1)

where $\rho$ is the thermospheric density, and ${\stackrel{-}{\rho }}_{\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}}$ is the normalized thermospheric density. This formula uses the mean ($\mu$) and standard deviation ($\sigma$) values of ${\log }_{10}\left(\rho \right)$ to normalize the log-transformed values.

For the DL model training, the mean squared error was used as the loss function. The loss function process can be expressed as follows:

$$\mathrm{l}\mathrm{o}\mathrm{s}\mathrm{s}={\displaystyle \frac{1}{\mathrm{n}}}\sum _{\mathrm{i}=1}^{\mathrm{n}}{\left({\rho }_{\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}}-{\stackrel{-}{\rho }}_{\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}}\right)}^2$$

(2)

Here, ${\rho }_{\mathrm{norm}}$ is the neural network output, while ${\stackrel{-}{\rho }}_{\mathrm{norm}}$ is the normalized thermospheric density.

To assess the performance of the DL model, three metrics were used: the root mean squared error (RMSE), the mean squared error (MSE), and the Pearson correlation coefficient (Corr). RMSE is used to represent the deviation of the model density from the “true” density, as shown in Equation (3), MSE is used to quantify the average squared differences between the predicted and true values, as shown in Equation (5), and Corr measures the linear dependence between two series, as shown in Equation (4).

$$\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{{\displaystyle \frac{1}{\mathrm{n}}}\sum _{\mathrm{i}=1}^{\mathrm{n}}{\left({\rho }_{\mathrm{r}\mathrm{e}\mathrm{f}}-{\stackrel{-}{\rho }}_{\mathrm{r}\mathrm{e}\mathrm{f}}\right)}^2}$$

(3)

$$\mathrm{C}\mathrm{o}\mathrm{r}\mathrm{r}={\displaystyle \frac{{\sum }_{\mathrm{i}=1}^{\mathrm{n}}\left({\rho }_{\mathrm{r}\mathrm{e}\mathrm{f}}-{\stackrel{-}{\rho }}_{\mathrm{r}\mathrm{e}\mathrm{f}}\right)\left({\rho }_{\mathrm{r}\mathrm{e}\mathrm{f}}-{\stackrel{-}{\rho }}_{\mathrm{r}\mathrm{e}\mathrm{f}}\right)}{\sqrt{{\left({\rho }_{\mathrm{r}\mathrm{e}\mathrm{f}}-{\stackrel{-}{\rho }}_{\mathrm{r}\mathrm{e}\mathrm{f}}\right)}^2\sum _{\mathrm{i}=1}^{\mathrm{n}}{\left({\rho }_{\mathrm{r}\mathrm{e}\mathrm{f}}-{\stackrel{-}{\rho }}_{\mathrm{r}\mathrm{e}\mathrm{f}}\right)}^2}}}$$

(4)

$$\mathrm{M}\mathrm{S}\mathrm{E}={\displaystyle \frac{1}{\mathrm{n}}}\sum _{\mathrm{i}=1}^{\mathrm{n}}{\left({\rho }_{\mathrm{r}\mathrm{e}\mathrm{f}}-{\stackrel{-}{\rho }}_{\mathrm{r}\mathrm{e}\mathrm{f}}\right)}^2$$

(5)

Here, ${\rho }_{\mathrm{ref}}$ is the predicted value of the model, and ${\rho }_{\mathrm{ref}}$ is the corresponding thermospheric density from the CHAMP dataset.

3. Results and Discussions

The deep learning (DL) models were developed using CHAMP data to identify relationships among various spatial and temporal input parameters and produce the global TMD distributions at specific times. This section evaluates the performance of these models using the CHAMP test dataset, focusing on their ability to predict TMD and capture horizontal variations. Next, a randomly sampled GRACE-A dataset was used to examine the generalization ability of the DL models in predicting TMD at higher altitudes, corresponding to the GRACE-A orbital altitude of 450–540 km (average altitude: 482 km), which was higher than the altitude range used for training. Additionally, the models were assessed under varying solar conditions to comprehensively analyze their performance. Finally, the DL models’ capability to accurately represent two classic TMD horizontal variations (EMA and MDM) was examined.

3.1. Convergence and Feature Extraction Performance During Training

The performance of the DL models during training was evaluated using the mean squared error (MSE) as the primary metric to quantify the difference between the predicted and actual values. The Adam optimizer [51] was employed for its efficiency with large datasets. To facilitate stable training, the process was divided into five groups of 40 epochs each. The learning rate for the Adam optimizer was adjusted sequentially for these groups as ${10}^{-7}$, ${10}^{-6}$ ${10}^{-5}$, ${10}^{-6}$, and ${10}^{-7}$, respectively. A batch size of 4096 was used, allowing each parameter update to incorporate 4096 samples, thereby ensuring efficient and robust learning.

Figure 2 illustrates the MSE of the DL models (ResNet-MSIS and ResNet) for the training set (left) and validation set (right) as a function of the number of epochs. Solid lines represent the training loss of the ResNet-MSIS model, while dashed lines represent the ResNet model. Different activation functions are indicated by colors. The activation functions used in the models include ELU, ReLU, and Tanh. ELU [52] introduces non-linearity by allowing negative values, improving the learning of complex patterns. ReLU [53], a widely used activation function, outputs the input directly if it is positive, and zero otherwise, which helps mitigate the vanishing gradient problem. Tanh [54], on the other hand, maps input values to the range [−1, 1], which can help center the data and improve training efficiency. During the early training phase, particularly within the first 40 epochs, the ResNet-MSIS models exhibited significantly faster convergence across all activation functions compared with the ResNet models. This result suggests that incorporating the NRLMSIS-2.1 (MSIS) predictions effectively accelerated convergence. By the end of the training cycle, both models achieved a good fit to the data.

Figure 3 shows the prediction results of ResNet-MSIS (left), ResNet (middle), and MSIS (right) on the CHAMP training dataset. Each plot shows the relationship between the TMD predictions (y-axis) and CHAMP TMD (x-axis), annotated with the root mean squared error (RMSE) and correlation coefficient (Corr). For the CHAMP training set, both ResNet-MSIS and ResNet almost achieved a correlation coefficient exceeding 0.98, while MSIS achieved 0.9158. The average RMSE values for the models were as follows: ResNet-MSIS (0.2657 × ${10}^{-12}$ kg/m³), ResNet (0.2749 × ${10}^{-12}$ kg/m³), and MSIS (0.7539 × ${10}^{-12}$ kg/m³). Notably, ResNet-MSIS consistently outperformed ResNet and MSIS under all activation functions, achieving a lower average RMSE and higher average Corr. This demonstrates that ResNet-MSIS more accurately captured and preserved the features of the CHAMP TMD compared with ResNet and MSIS.

3.2. Evaluation of DL Models on Test Data

Figure 4 shows the prediction results of ResNet-MSIS (left), ResNet (middle), and MSIS (right) on the CHAMP test dataset. The average RMSE values for the models on the CHAMP test dataset were as follows: ResNet-MSIS (0.2657 × ${10}^{-12}$ kg/m³), ResNet (0.2750 × ${10}^{-12}$ kg/m³), and MSIS (0.7535 × ${10}^{-12}$ kg/m³). The DL models showed similar RMSE values on the test set as observed during training, suggesting effective generalization and reasonable handling of the unseen data. ResNet-MSIS consistently performed better than both ResNet and MSIS across all activation functions, achieving a lower average RMSE and a higher Corr. Specifically, ResNet-MSIS performed better than ResNet and MSIS in TMD prediction.

Figure 5 and Figure 6 display the density plot of the CHAMP test dataset for ResNet-MSIS, ResNet, and MSIS under low solar activity (F10.7 < 100 sfu) and high solar activity (F10.7 > 150 sfu). The average RMSE values for ResNet-MSIS and ResNet were consistently lower during low solar activity (0.2408 × ${10}^{-12}$ kg/m³ and 0.2454 × ${10}^{-12}$ kg/m³, respectively) compared with high solar activity (0.3440 × ${10}^{-12}$ kg/m³ and 0.3637 × ${10}^{-12}$ kg/m³). In contrast, MSIS exhibited a higher RMSE during low solar activity (0.8577 × ${10}^{-12}$ kg/m³) than during high solar activity (0.6517 × ${10}^{-12}$ kg/m³). Additionally, across varying solar activity levels, ResNet-MSIS consistently achieved the lowest RMSE and highest Corr, suggesting improved performance in TMD prediction relative to ResNet under both solar conditions. In particular, for ResNet-MSIS, the model demonstrated an average RMSE of 0.2408 × ${10}^{-12}$ kg/m³ during low solar activity compared with 0.3440 × ${10}^{-12}$ kg/m³ during high solar activity. The performance of ResNet-MSIS in TMD prediction was better under low solar activity than under high solar activity. This trend was consistent across all activation functions used, suggesting that ResNet-MSIS exhibits improved generalization in TMD prediction during low solar activity. DL models using the ReLU activation function exhibited smaller prediction errors compared with those using ELU and Tanh. Therefore, ReLU was selected as the activation function for subsequent work. Additionally, the activation function primarily influenced the convergence speed and stability during model training. Future efforts will focus on refining the network architecture to further enhance model performance.

3.3. Generalization of DL Models for TMD Prediction at Different Altitudes

To further evaluate its generalization capability, approximately 3.3 million GRACE-A data points—matching the size of the CHAMP test dataset—were randomly selected. These GRACE-A test data, spanning from 2002 to 2009 and covering altitudes of 450–540 km, were utilized for simulations under varying conditions to assess the impact of incorporating NRLMSIS-2.1 TMD on model generalization. The corresponding parameters of the GRACE-A dataset were used as inputs to the DL model to predict TMD. The average altitude of the CHAMP dataset was approximately 376 km, while the average altitude of the GRACE dataset was around 482 km.

Figure 7 compares the predictions of ResNet-MSIS_relu, ResNet_relu, and NRLMSIS-2.1. Each subplot illustrates the relationship between the observed (x-axis) and predicted (y-axis) TMD values, annotated with the root mean squared error (RMSE) and correlation coefficient (Corr). The high Corr values confirmed that the DL models generalized effectively, providing a robust description of TMD at the GRACE-A orbital altitude. To evaluate the performance under differing solar activity levels, the GRACE-A dataset was partitioned into high solar activity periods (F10.7 > 150; Figure 8) and low solar activity periods (F10.7 < 100; Figure 9).

Notably, the CHAMP test dataset exhibited tighter clustering and more uniform alignment along the best-fit lines (dashed lines) compared with the GRACE-A data. This discrepancy arises because DL, as a data-driven approach, inherently suffers performance degradation at higher altitudes due to limited training samples. Furthermore, TMD follows an exponential decay profile with altitude, and the thermospheric physical characteristics vary significantly across altitude regimes. Consequently, TMD features learned at lower altitudes cannot be directly extrapolated to higher altitudes.

A potential strategy to improve cross-altitude prediction is to integrate training data from two satellites, enabling the DL model to learn broader TMD variability. However, this approach requires the careful mitigation of systematic biases between satellite measurements. Additionally, the dataset may encompass diverse time periods, altitudes, and spatial environmental conditions to ensure comprehensive feature learning. Balancing these factors while maintaining a rigorous division between the training and test sets remains challenging.

In this study, we incorporated the prior knowledge from NRLMSIS-2.1 into the DL framework. As shown in Figure 7, the ResNet-MSIS_relu predictions clustered more tightly and aligned more uniformly with the best-fit lines compared with ResNet_relu. Similarly, during both high (Figure 8) and low (Figure 9) solar activity periods, ResNet-MSIS_relu achieved superior prediction accuracy. These results indicate that prior knowledge enhances the DL model’s extrapolation capability and altitude-generalization performance, even with limited altitude coverage in the training data. However, it is important to note that the MSIS model performed better than ResNet-MSIS and ResNet. As clearly shown in Figure 9, the MSIS model aligned more uniformly with the best-fit lines compared with ResNet-MSIS and ResNet, indicating that NRLMSIS-2.1 still outperformed the DL model on GRACE-A data.

Additionally,

Table 3. Performance of the ResNet-MSIS_relu and ResNet_relu models on the GRACE-A test set.

Model	Condition	Slope (k Value)	Corr	$\mathbf{RMSE} ({10}^{-12}$ kg/m3)
ResNet-MSIS_relu	Overall	0.9415	0.9078	0.2959
	High solar (F10.7 > 150)	0.7713	0.8444	0.3218
	Low solar (F10.7 < 100)	0.5901	0.7408	0.2994
ResNet_relu	Overall	0.7456	0.8802	0.3352
	High solar (F10.7 > 150)	0.6937	0.8000	0.3885
	Low solar (F10.7 < 100)	0.4190	0.8269	0.3240
MSIS	Overall	1.0187	0.9621	0.0891
	High solar (F10.7 > 150)	0.9508	0.9012	0.2095
	Low solar (F10.7 < 100)	0.8991	0.9063	0.0605

summarizes the slope (k), Corr, and RMSE values between the predictions and GRACE-A TMD for ResNet-MSIS, ResNet, and MSIS, further highlighting the advantages of incorporating prior knowledge. These results suggest that prior knowledge enables the DL model to learn higher-altitude TMD features from NRLMSIS-2.1, improving the predictions beyond the training altitude range. Therefore, future work could explore two avenues: (1) integrating prior knowledge as a feature input into the DL model, and (2) adopting a physics-informed neural network (PINN) framework [55]. Given TMD’s exponential altitude decay, explicitly incorporating this physical constraint into the loss function could guide the model to better adapt to altitude-dependent variations, particularly at higher altitudes. Using TMD data from various satellites at different altitudes and levels of solar and geomagnetic activity for model training would further improve model performance. At the same time, addressing the system errors associated with different satellite datasets and ensuring the model’s robustness across various altitudes is an important issue to consider for future work.

3.4. Evaluating DL Models for Capturing Horizontal Variations in TMD

This section evaluates the capability of DL models to capture horizontal variations in TMD. The TMD data from CHAMP revealed a two-cell EMA structure in the thermosphere, characterized by TMD peaks at mid-latitudes and a trough near the equator [56]. EMA is a globally significant feature of the thermosphere, represented by two TMD peaks on the polar sides of the equatorial ionization anomaly (EIA). Factors such as ion drag, plasma-neutral heating, and the non-migrating diurnal tide with wavenumber 3 (DE3) play key roles in shaping this phenomenon [32].

Figure 10a–f illustrates the TMD distributions predicted by the ResNet-MSIS, ResNet, and MSIS models at an altitude of 400 km during the vernal (March and April) and autumnal equinoxes (August and September). Both the ResNet-MSIS and ResNet models successfully reproduced the two-cell EMA structure at 14 h LT for both periods, while the MSIS model failed to accurately represent this structure. Furthermore, Figure 10a,d reveals the presence of a local maximum in equatorial TMD around midnight, marked by a circle in each figure, during periods of both high and low solar activity, respectively. These results highlight the capability of the ResNet-MSIS model to represent the equatorial mass density anomaly (EMA) and the midnight density maximum (MDM) structures. Moreover, Figure 10a–c reveals that under high solar activity conditions, the MSIS model predicted the highest TMD values, while the ResNet model predicted the lowest, with ResNet-MSIS providing values in between, indicating its potential to balance these two approaches. Consequently, incorporating prior information may contribute to a more balanced representation of TMD variations in high solar activity conditions. In addition, a comparison between Figure 10a,b revealed a more pronounced two-cell EMA structure in Figure 10b, indicating that the incorporation of prior information from MSIS may smooth out horizontal variations in TMD. This effect is likely due to the inherent smoothness of the MSIS model. When used as prior input, the MSIS output may suppress fine-scale features. Nevertheless, the ResNet-MSIS model still captured key structures such as EMA and MDM under both high and low solar activity conditions. These results suggest that while prior information may improve model generalization, its integration requires careful design to preserve critical physical structures in the predictions.

To facilitate a comprehensive comparison of the models’ performance in predicting horizontal variations of TMD, CHAMP datasets were used for detailed validation. Following the data screening criteria established by H. Liu et al. [57], CHAMP data from March–April and August–September 2002, collected during magnetically quiet conditions (ap < 22) between 14:00 and 18:00 local time, were employed to investigate the EMA structure. These CHAMP observations were normalized to an altitude of 400 km using the NRLMSIS-2.1 (MSIS) model. The normalization to 400 km was performed to eliminate altitude-dependent variations in TMD, ensuring consistency in the reference altitude for predictions and enabling fair comparison across different datasets. The ResNet-MSIS, ResNet, and MSIS models leveraged observational timestamps along with relevant indicator data to predict TMD at 400 km. To generate a two-dimensional (2D) TMD map, both the observed and model-predicted TMD values were averaged within spatial grids of 5° latitude × 10° longitude in geographic coordinates. Figure 11a–d displays the CHAMP observations and the predictions from the three models: ResNet-MSIS, ResNet, and MSIS. The grid-averaged F10.7 in Figure 11e shows the regional variations of F10.7 and the potential correlation with TMD at different locations. This averaged F10.7 index was computed by the mean value of the F10.7 data along the orbit within spatial grids of 5° latitude × 10° longitude.

In particular, Figure 11d shows that the MSIS model had an RMSE of 2.2860 × ${10}^{-12}$ kg/m³, with the corresponding prediction error distribution presented in Figure 11h. Moreover, the two-cell structure of EMA was strongly influenced by the solar radiation conditions [32,37]. This was particularly evident from the correlation between the elevated F10.7 values and a more pronounced two-cell EMA structure, as shown in Figure 11e. These results indicate that regions experiencing stronger solar activity exhibit a more distinct EMA structure, emphasizing the critical role of solar radiation in shaping TMD distributions. In addition, Figure 11f–g illustrates the prediction errors of the DL models relative to CHAMP observations. ResNet-MSIS achieved the highest prediction accuracy, as reflected by its lower RMSE. Additionally, ResNet-MSIS exhibited a higher Corr (0.9772) compared with both ResNet (0.9712) and MSIS (0.7762) and demonstrated superior spatial distribution characteristics. These results indicate that incorporating MSIS predictions as prior knowledge generally enhances the model’s ability to capture the TMD horizontal variations more effectively. This improvement was particularly evident when the model was evaluated under conditions similar to those in the training dataset, including comparable geomagnetic activity levels and altitude range, as such conditions allow the model to better leverage the learned patterns and prior information.

4. Conclusions

In this study, a ResNet model enhanced with prior knowledge from the empirical model NRLMSIS-2.1, termed ResNet-MSIS, was developed. The CHAMP TMD dataset (2000–2009, altitude of 305–505 km, avg. 376 km) under quiet magnetic conditions (Kp ≤ 3) was divided into training (70%), validation (15%), and test (15%) sets. The performance of ResNet-MSIS was intensively evaluated on both the CHAMP test dataset (2000–2009, altitude of 305–505 km, avg. 376 km) and the GRACE-A data (2002–2009, altitude of 450–540 km, avg. 482 km) across different levels of solar activity (low and high). The evaluation focused on the deep learning (DL) model’s predictive performance on TMD at different altitudes and its ability to capture horizontal TMD variations, specifically the EMA and MDM structures. The results showed that incorporating NRLMSIS-2.1 predictions as prior knowledge accelerated model convergence, and during training, ResNet-MSIS achieved an RMSE of 0.2657 × ${10}^{-12}$ kg/m³, outperforming both ResNet (0.2749 × ${10}^{-12}$ kg/m³) and MSIS (0.7539 × ${10}^{-12}$ kg/m³) in capturing and preserving the observational features more effectively. On the CHAMP test dataset, compared with these models, ResNet-MSIS demonstrated stronger generalization in TMD prediction (0.2657 vs. 0.2750 for ResNet, and 0.7535 for MSIS; unit: ${10}^{-12}$ kg/m³), especially excelling during periods of low solar activity (0.2408 vs. 0.2454 for ResNet, and 0.8577 for MSIS; unit: ${10}^{-12}$ kg/m³). Although the DL models performed less accurately than the MSIS model at the GRACE-A orbital altitude of 450–540 km (avg. 482 km), they still maintained a reasonable level of consistency with the observations (Corr > 0.88), and the incorporation of prior knowledge helped reduce the RMSE from 0.3352 to 0.2959 (unit: ${10}^{-12}$ kg/m³). The ResNet-MSIS model captured the equatorial mass density anomaly (EMA) and the midnight density maximum (MDM) structures, verifying that the DL model identified these structures and effectively learnt the variation patterns within them. Overall, ResNet-MSIS achieved the best TMD prediction performance on the CHAMP test dataset (altitude of 305–505 km, avg. 376 km). Although incorporating prior knowledge improved the DL model’s generalization at the GRACE-A orbital altitude of 450–540 km (avg. 482 km), NRLMSIS-2.1 still outperformed both ResNet-MSIS and ResNet. Accordingly, identifying more effective methods to incorporate prior knowledge for improving performance beyond the training altitude range warrants further investigation.

This work highlights the potential of integrating prior knowledge into DL frameworks, improving both the performance and interpretability. However, while prior knowledge enhances the model’s generalization, NRLMSIS-2.1 still outperformed the DL model on the GRACE-A dataset, indicating opportunities for further refinement in high-altitude TMD prediction. To address this, future research could explore two strategies: (1) integrating prior knowledge as a feature input into DL models, and (2) adopting physics-informed neural networks (PINNs) [51]. PINNs can directly embed physical principles, such as TMD’s exponential decay with altitude, into the loss function, guiding the model to learn altitude-dependent variations and improve predictions beyond the training range. TMD data can be used from various satellites at different altitudes and levels of solar and geomagnetic activity for model training. Subsequent studies will investigate these approaches to advance DL-based TMD modeling across diverse altitude regimes.