Tracking Carbon Dioxide with Lagrangian Transport Simulations: Case Study of Canadian Forest Fires in May 2021

Abstract

The large amounts of greenhouse gases, such as carbon dioxide, produced by severe forest fires not only seriously affect the ecosystems in the area where the fires occur but also cause a greenhouse effect that has a profound impact on the natural environment in other parts of the world. Numerical simulations of greenhouse gas transport processes are often affected by uncertainties in the location and timing of the emission sources and local meteorological conditions, and it is difficult to obtain accurate and credible predictions by combining remote sensing satellite data with given meteorological forecasts or reanalyses. To study the regional transport processes and impacts of greenhouse gases produced by sudden large-scale forest fires, this study applies the Lagrangian particle dispersion model Massive-Parallel Trajectory Calculations (MPTRAC) to conduct forward simulations of the CO₂ transport process of greenhouse gases emitted from forest fires in the central region of Saskatchewan, Canada, during the period of 17 May to 25 May 2021. The simulation results are validated with the Orbiting Carbon Observatory-2 Goddard Earth Observing System (OCO-2 GEOS) Level 3 daily gridded CO₂ product over the study area. In order to leverage the high computational costs of the numerical simulations of the model, we implement the forward simulations on the Tianhe-2 supercomputer platform and the JUWELS HPC system, which greatly improves the computational efficiency through parallel computation and makes near-real-time predictions of atmospheric transport processes feasible.

1. Introduction

Greenhouse gases (GHGs) such as carbon dioxide (CO₂), methane (CH₄), and nitrous oxide (N₂O), among others, affect the Earth’s energy balance and contribute to the greenhouse effect, which is one of the main factors contributing to climate change [1,2]. The international community is highly concerned about the emission of GHGs, especially CO₂. Normally, the concentration of CO₂ in the atmosphere is at a certain background level, but large-scale human activities and sudden natural events may lead to a sharp increase in the level of CO₂ in the atmosphere [3]. The impact of forest fires as a natural disaster cannot be ignored [4], and the subsequent transport of CO₂ usually crosses multiple countries and regions. By studying CO₂ transport pathways and sharing data and information, we can better understand the distribution of CO₂ in the atmosphere, thus laying the foundation for further backtracking and modelling of the dispersion of atmospheric pollutants or GHGs from unknown sources, and contributing positively to the monitoring and management of CO₂ emission issues.

Large-scale trajectory simulations play a crucial role in understanding and predicting the dispersion of trace gases or other atmospheric constituents. One key aspect of large-scale trajectory simulations is their significance in environmental studies. By accurately simulating the transport of trace gases, researchers can gain a better understanding of how pollutants disperse in the atmosphere. This knowledge is essential for assessing the impact of industrial emissions, forest fires, or other sources of pollution on the environment and human health. It allows scientists and policymakers to develop effective strategies for mitigating pollution and improving air quality. Furthermore, large-scale trajectory simulations are crucial in assessing the potential risks and consequences of accidental releases or hazardous events. Accurate simulations enable decision-makers to make informed choices and minimize the potential harm to the affected population and the environment. In addition, large-scale trajectory simulations are essential in the field of atmospheric science and climate research. Scientists can study the transport of greenhouse gases, such as CO₂ and CH₄, which play a significant role in global climate change. Understanding how these trace gases disperse and interact with the atmosphere allows researchers to improve climate models and enhance predictions regarding weather phenomena, climate patterns, and long-term climate trends.

The main types of chemistry transport models used to study transport and dispersion of CO₂ are the Lagrangian particle dispersion model [5], the Eulerian model [6], and the hybrid model [7]. The Lagrangian particle dispersion model describes air flow by tracking the position and velocity of individual air parcels or particles. Eulerian models describe air flow based on the advection and diffusion of trace gases on a regular grid. Compared to the Eulerian model, the Lagrangian model is based on a microscopic perspective, assuming that each air parcel or particle has its own unique velocity and pressure, allowing it to capture small-scale features and reduce numerical diffusion. The independent motion of the particles allows us to compute and update particle trajectories independently. At the same time, it facilitates the allocation of particles to different processing units for parallel computation of their positions, velocities, and other properties. This property gives the Lagrangian particle diffusion model excellent parallel performance.

The Lagrangian particle dispersion model (LPDM) has been widely used in recent years for the forward and inverse modelling of atmospheric pollutants and trace gas emissions [8,9,10] due to its scalability, numerical efficiency, capability of retrieving source–receptor relationships, and ability to implement stochastic processes representing transport uncertainties [11]. A modified version of the Stochastic Time-Inverted Lagrangian Transport (STILT) model, known as “X-STILT” [12], has been developed to simulate urban CO₂ signals using NASA’s OCO-2 satellite data. X-STILT incorporates satellite profiles and provides comprehensive uncertainty estimates of urban CO₂ enhancements, successfully reproducing observed CO₂ enhancements and quantifying the contributions of transport and emission uncertainties to the overall signal. This model, X-STILT, is expected to serve as a valuable tool for interpreting column measurements, estimating urban emissions, and enhancing our understanding of urban CO₂ quantification.

Currently, the LPDM is widely used in atmospheric research and environmental studies to understand the transport, dispersion, and deposition of atmospheric constituents on regional and global scales. Commonly used LPDMs include the Flexible Particle Dispersion model (FLEXPART) [13,14], the Hybrid Single-Particle Lagrangian Integrated Trajectory model (HYSPLIT) [15,16], the three-dimensional unsteady Lagrangian diffusion model California Puff (Calpuff) [17,18], and Massive-Parallel Trajectory Calculations model (MPTRAC) [19,20] among others. Different models differ in their numerical methods to solve the advection and diffusion problem, as characterized by the complex flow field environment during particle transport and the complexity of the diffusion processes. Compared to other Lagrangian models, MPTRAC is a newly developed LDPM specifically designed for supercomputer platforms. Based on an MPI-OpenMP-OpenACC hybrid parallelization scheme, it provides high computational efficiency, is suitable for large-scale simulations, and has been applied in several volcanic SO₂ dispersion case studies [21].

In this paper, we applied the Lagrangian particle dispersion model MPTRAC [19] mainly on the Tianhe-2 supercomputer. Our aim was to use MPTRAC to test new field applications other than volcanic eruption simulation and to develop a numerical modelling approach for CO₂ transport processes. We focused on a forest fire in the region near Saskatchewan, Canada. The modeling case study covered the period from 17 May to 25 May 2021. We conducted a qualitative comparison of our model’s simulation results with a satellite-based data product of daily CO₂ distributions provided by the National Aeronautics and Space Administration (NASA). The objective was to demonstrate the validity of the long-range transport simulations of CO₂ from forest fires provided by MPTRAC. This is the first time that the MPTRAC model has been used to simulate the long-range transport of CO₂ following forest fires.

2. Data and Methods

2.1. The MPTRAC Lagrangian Transport Model

The Massive-Parallel Trajectory Calculations (MPTRAC) (version2.6, manufactured by the Jülich Supercomputing Centre, Jülich, Germany)model [19] is a Lagrangian particle dispersion model and an open-source software package developed by the Jülich Supercomputing Center in Germany. It can be utilized to simulate and analyze the transport of trace gases and aerosols in the free troposphere and stratosphere. It is suitable for studying fine-scale structures, filamentary transport, and mixing processes in the atmosphere. The MPTRAC model implements an MPI-OpenMP-OpenACC hybrid parallelization scheme, making it highly suitable for large-scale simulations on high-performance computing systems.

Given the wind and velocity field $v(x,t)$, the MPTRAC model uses the trajectory equation in Equation (1) to calculate the position $x\left(t\right)$ of an air parcel at time $t$,

$$\frac{dx}{dt}=v(x,t).$$

(1)

To obtain a reasonably accurate and computationally efficient solution to the trajectory equation [22], the model uses the explicit midpoint method [23] to calculate the air parcel trajectory,

$$x\left(t+∆t\right)=x\left(t\right)+∆t·v\{x\left(t\right)+\frac{∆t}{2}v\left[x\left(t\right),t\right],t+\frac{∆t}{2}\},$$

(2)

where x(t) represents the spatial position of a particle at time t, v(t) represents the velocity of the particle at time t, and $∆t$ is the time step of the model.

Following the approach of the FLEXPART model [13], MPTRAC simulates the diffusion of particles in the troposphere and stratosphere by adding random perturbations to the positions $x$ of the air parcels at each time step $∆t$. The random perturbations are characterized by a horizontal diffusivity of 50 m² s⁻¹ in the troposphere and a vertical diffusivity of 0.1 m² s⁻¹ in the stratosphere. A smooth transition between tropospheric and stratospheric diffusivities is achieved by linearly interpolating the horizontal and vertical diffusivities over a logarithmic pressure altitude range of ±1 km around the tropopause. In addition to the fixed diffusivities, a parameterization for subgrid-scale wind fluctuations based on the Langevin equation is applied.

In our experiments, the “air parcels” were the “CO₂ particles” emitted by the forest fire. Each trajectory corresponded to a single CO₂ parcel. The transport of CO₂ was represented by an ensemble of trajectories, each carrying a specific amount of CO₂. Large quantities of CO₂ particles released by forest fires can be obtained by calculations using the Lagrangian model.

2.2. The OCO-2 GEOS Level 3 Data Product

Forest fires release large amounts of CO₂ into the atmosphere, causing strong anomalies that can be observed by satellite measurements. We used NASA’s existing OCO-2 GEOS Level 3 data product to help analyze the simulation results. The OCO-2 GEOS Level 3 daily, 0.5° × 0.625° assimilated CO₂ V10r data product [24] provides assimilated gridded daily CO₂ data from OCO-2 satellite measurements and the GEOS Earth system model. The OCO-2 mission provides the highest-quality space-based XCO₂ retrievals to date [25]. However, the instrument data are characterized by large gaps in coverage due to OCO-2’s narrow 10 km ground track and an inability to see through clouds and dense aerosols. The global gridded Level 3 dataset is produced using a data assimilation technique commonly referred to as state estimation in the geophysical literature. Data assimilation integrates simulations and observations and adjusts the state of atmospheric constituents, such as CO₂, to reflect observed values. This process effectively fills observational gaps in time and space where data are unavailable, utilizing information from prior observations and short transport simulations conducted by GEOS. Compared to other methods, data assimilation has the advantage of providing estimates based on collective scientific understanding, particularly of the Earth’s carbon cycle and atmospheric transport.

OCO-2 GEOS (Goddard Earth Observing System) Level 3 data are produced by ingesting OCO-2 Level 2 retrievals every 6 h with GEOS CoDAS, a modeling and data assimilation system maintained by NASA’s Global Modeling and Assimilation Office (GMAO). GEOS CoDAS uses a high-performance computing implementation of the Gridpoint Statistical Interpolation (GSI) approach to solve the state estimation problem. GSI finds the analyzed state that minimizes the three-dimensional variational (3D-Var) cost function formulation of the state estimation problem. Detailed information of the OCO-2 GEOS Level 3 data is shown in

Table 1. Characteristics of the OCO2_GEOS_L3CO2_DAY data product [24].

Property	Description
Shortname	OCO2_GEOS_L3CO2_DAY
Longname	OCO-2 GEOS Level 3 daily, 0.5° × 0.625° assimilated CO2 V10r
DOI	10.5067/Y9M4NM9MPCGH
Version	10r
Format	netCDF
Spatial Coverage	−180.0, −90.0, 180.0, 90.0
Temporal Coverage	1 January 2015 to 1 March 2022
File Size	57 MB per file
Spatial Resolution	0.5° × 0.625°
Temporal Resolution	1 day
Data Dimensions	longitude = 576, latitude = 361, time = 1

.

3. CO₂ Trajectory Modeling of Forest Fires in Canada

3.1. Problem Description and Selection of Case Study

Although Canada has mostly a subarctic climate [26], severe droughts in recent years have forced more forest fires [27]. According to the statistics from the Canadian National Fire Database (CNFDB), the area affected by forest fires reached a small peak in 2021, and the occurrence of forest fires is mainly concentrated between May and August of that year. In our numerical experiment, to study the long-range transport of CO₂ emissions from severe forest fires, we selected the largest affected area by a forest fire as a case study for simulation. The selected forest fire case occurred in mid-May 2021 in the central region of Saskatchewan (SK), Canada. The fire, numbered 21PA-CLOVERDALE, occurred on 17 May, at a longitude of 105°67′ W and a latitude of 53°24′ N. It covered an area of 5470 hectares and was completely extinguished on 10 June 2021. After investigation, it was determined that the fire was human-induced. It was reported on the day of ignition, and the relevant data were last updated in the CNFDB on 11 April 2022.

3.2. Preparation of Data and Operating Environment

The data used in our case study are divided into two parts: one is the meteorological data that provide the horizontal wind and vertical velocity fields to calculate the trajectories with MPTRAC, and the other is the OCO-2 GEOS data product of CO₂ concentrations around the forest fire area in Canada. In this case study, the meteorological data were obtained from the ERA5 reanalysis provided by the European Centre for Medium-Range Weather Forecasts (ECMWF) [28], a dataset with a temporal resolution of 1 h, a horizontal resolution of 0.3° × 0.3°, and a vertical resolution of 137 model levels from the surface up to 0.01 hPa. The CO₂ concentration data used in the experiment were obtained from the assimilated data product of CO₂ concentrations from the OCO-2 satellite and the GEOS model [24] provided by NASA. The temporal resolution of the data is 24 h, matching the output time interval of the simulation, and the spatial resolution is 0.5° × 0.625°. These two datasets used here have undergone years of research and validation, are widely used in fields such as meteorology, climatology, and environmental research, and were chosen to ensure reliability and accuracy.

Most of the transport simulations were conducted on the Tianhe-2 high-performance computing system operated by the National Supercomputing Center in Guangzhou, China. In the Tianhe-2H cluster, the regular compute nodes consist of two Intel Xeon E5-2692 v2 12-core multi-core central processing units (CPUs) with a clock frequency of 2.2 GHz. Each compute node has 64 GByte of memory shared between the two CPUs. The operating system of the Tianhe-2H cluster is Red Hat Enterprise Linux Server release 7.3 (Maipo) (manufactured by Red Hat Inc., Raleigh, North Carolina, United States). Currently, the Tianhe-2H system is configured with GNU and Intel compilation environments, supporting the development of programs in the C, C++, Fortran77, and Fortran90 programming languages. Additionally, the Tianhe-2H system supports two parallel programming models: OpenMP and MPI. OpenMP is a shared memory programming model that enables parallelism within a single compute node, with the maximum number of threads not exceeding the number of physical cores of the regular compute nodes (set at 24). In contrast, MPI is a distributed memory parallel programming model enabling computational tasks to run across multiple nodes, with the maximum number of processes being constrained by the total number of available compute nodes under the user’s account. The OpenMP parallelization, which operates on shared memory, is typically supported directly by the compilers. Both the GNU and Intel compiler have implemented support for this standard. In this case, each individual simulation was launched with the full set of CPU threads with OpenMP parallelization to achieve the best runtime. The MPI parallelization, which operates on distributed memory, is enabled by the specific MPI environment and software stack. Here, MPI was utilized to enable sensitivity tests via ensemble simulations.

3.3. Baseline Experiment

This study applied the MPTRAC Lagrangian transport model to conduct forward simulations of the transport trajectories of CO₂ released by the Canadian forest fire. The parameters and data that need to be set are mainly the source term, i.e., the total mass of CO₂ being released as well as the number and initial positions and time of the particles, the time range simulated in the experiment, the meteorological data that drive the simulations, and the types of outputs required for further analysis.

The study region covered a longitude range from 120° W to 0° and a latitude range from 10° to 70° N. The model output was selected as grid output with a grid size of 240 × 120 boxes. Each grid box covered approximately 55 km × 55 km. As described in Section 3.1, the fire was located at (105°67′ W, 53°24′ N), and we set the initial position of the particle source term with a Gaussian distribution with full width at half-maximum of 200 km centered on these coordinates. In addition, according to observations provided by the CALIPSO [29] satellite mission, aerosol emissions in the atmosphere after the fire were found to be concentrated at altitudes between 8 and 12 km, as shown in Figure 1. Therefore, we tentatively chose 10 km as the initial altitude of the CO₂ emissions. In this experiment, we qualitatively assigned a total amount of CO₂ of $4\times {10}^{10}$ kg released by the fire. The model setup is summarized in

Table 2. MPTRAC model set up for the Canadian forest fire case study.

Parameter	Value
Simulation area	120°∼0° W, 10°∼70° N
Particle source location	Gaussian centered at (105°67′ W, 53°24′ N)
Particle total mass	$4\times {10}^{10}$ kg
Total number of particles	$1\times {10}^6$
Initial height of particles	10 km
Simulation time range	17 May 2021, 00:00 UTC to 25 May 2021, 00:00 UTC
Meteorological data	ECMWF ERA5 reanalysis
Output type	gridded output
Output time resolution	6 h
Output grid spacing	240 × 120 grid boxes

.

We simulated the transport pathways of CO₂ from the forest fire from 17 May, 00:00 UTC, to 25 May, 00:00 UTC. We compared the MPTRAC simulation results of the horizontal CO₂ transport with the NASA OCO-2 Level 3 data product. For the NASA OCO-2 CO₂ data, we calculated the CO₂ anomalies caused by the fire and other sources by subtracting the daily mean value within the observation area to exclude the variability of the background CO₂ concentration for a better comparison. The simulated and observed horizontal dispersion of CO₂ from the fire is shown in Figure 2.

Inspection of the MPTRAC simulation results shows that a significant amount of CO₂ emitted from the fire formed a distinct transport pathway. On May 18th, the main area of CO₂ distribution was between 45°~55° N and 110°~80° W, forming a transport belt roughly oriented to the southeast. On 22 May and 23, the distribution of CO₂ transport showed a narrow elongated area extending eastward. At the same time, a circular transport belt was formed in the area of 75°~40° W and 25°~50° N. As time progresses, CO₂ continues to be transported eastward, while also spreading over a larger area. On May 24th, the distribution of CO₂ transport can be clearly divided into two parts: one part formed a southeastward trajectory in the region of 90°∼60° W, 40°∼55° N, and the other part dispersed over a large area in the region of 60°∼0°, 30°∼60° N. Inspecting the OCO-2 data product, it was found that on 21 May and 22 May, a long and narrow eastward transport belt also formed at the source location, consistent with the MPTRAC simulation results. On 23 May and 24 May, the distribution of CO₂ near the source in the observational data also matched the simulation results. At the same time, during the eastward transport, there was also a phenomenon of large-scale diffusion in the observations.

However, we also noticed that the distribution of CO₂ in the observational data shows peak anomalies in the region of 120°~60° W, 20°∼40° N, which are not reflected in the simulation results. At the same time, comparing the two sets of maps also reveals differences in CO₂ concentration. For these inconsistencies between the simulation results and the observation, we considered several reasons. First, on 18 May and 19 May, the OCO-2 satellite data observations do not show clear CO₂ signals around the fire site. Considering cloud index data from the Atmospheric Infrared Sounder (AIRS) [30] on these days, we assumed that significant fractions of clouds and aerosols in the atmosphere led to the blocking of observations near the location of the fire. The AIRS cloud index data within the simulation area are shown in Figure 3 for reference. Second, regarding the CO₂ anomalies at lower altitudes that are not present in the simulation results, we assumed that the satellite observations also included CO₂ emissions from fires or other sources of CO₂ in other locations that were not considered in the simulations, which caused interference when comparing the results.

3.4. Sensitivity Test on CO₂ Release Height

To show the sensitivity of the simulation results to the particle release height, we performed a series of simulations with air parcels released at different altitudes centered at 3, 7, 10, and 13 km, which cover the altitude ranges of aerosol observations provided by the CALIPSO [29] satellite project. As in the baseline experiment, we disabled the convection parametrization of MPTRAC to avoid any influence of vertical mixing on the CO₂ transport process. In Figure 4, we compare the CO₂ plumes released at different heights on the day of 23 May. The comparison indicates that there is a correlation between the horizontal transport paths of CO₂ and the release height of the particles. The plume moves faster towards the east coast and the Atlantic at higher altitudes, which is related to stronger westerly winds in the upper troposphere.

3.5. Comparison of Simulation Results on Different HPC Systems

To evaluate possible differences in the simulation results of the MPTRAC model on different HPC systems, we conducted simulations on the Tianhe-2 supercomputer platform at the National Supercomputer Center in Guangzhou in China and the JUWELS HPC system [31] at the Jülich Supercomputing Centre in Germany. After excluding blank grid boxes outside the particle diffusion area to avoid potential division by zero issues, we used Equation (3) to calculate the average differences between each grid box in the simulation results of these two HPC systems,

$$Difference=\frac{1}{n}\sum \frac{x_{JUWELS} – x_{Tianhe-2}}{\mathrm{m}\mathrm{a}\mathrm{x}\{x_{JUWELS},{ x}_{Tianhe-2}\}},$$

(3)

where $x_{Tianhe-2}$ and $x_{JUWELS}$ represent the simulation results of Tianhe-2 and JUWELS at each data point within the simulation range, respectively, and n represents the total number of non-empty grid boxes on the grid. The spatial distribution of the differences between the simulation results is shown in Figure 5. The main plumes simulated on two HPC systems demonstrate good consistency with each other, with only some differences in small diffusion features. The results on the Tianhe-2 platform show more diffusion features than on JUWELS. This can be attributed to different random numbers being generated.

4. Summary and Outlook

In this study, we used the Lagrangian particle dispersion model MPTRAC to conduct forward simulations of CO₂ transport pathways. We focused on a case study of forest fire emissions in Central Saskatchewan, Canada, for the period from 17th May to 25th May 2021. We presented a baseline simulation, for which we qualitatively analyzed the long-range transport and dispersion of CO₂. We incorporated the NASA OCO-2 Level 3 data product to facilitate the analysis of the simulation results. We noted that both the measurements and simulation showed a largely eastward spread of CO₂ from the fire and that there was a degree of overlap in the major plume components.

Based on the baseline simulation, a sensitivity test to the height of CO₂ release was conducted by adjusting the corresponding set-up parameters of the model. The test showed that the rate of plume dispersion to the east accelerated as the height of the CO₂ release increased. This finding highlights the importance of carefully assessing the release height when predicting the long-range transport of CO₂.

Moreover, we compared the simulation results obtained from the MPTRAC model on two different high-performance computing platforms: the Tianhe-2 supercomputing platform in China and the JUWELS supercomputer at the Jülich Supercomputing Centre in Germany. The evaluation showed a high level of agreement between the major plumes simulated on both HPC systems, with only minor differences observed in diffusion features. Overall, this study illustrates the feasibility of using the MPTRAC model to simulate and predict the long-range transport of CO₂ in different high-performance computing environments.

We set up a preliminary simulation to explore the feasibility of using MPTRAC to simulate the CO₂ anomalies caused by forest fires, laying the groundwork for further development of our research. In the future, we will try to improve the simulation with some source estimation techniques. Accurate simulation results can compensate for the shortcomings of satellite data and track the dispersion trajectories of CO₂ emissions from a single fire. The forward modelling of CO₂ dispersion lays the foundation for subsequent inversion to solve the source term identification problem, which would help to monitor and manage CO₂ emissions.