The Application of a Convolutional Neural Network for the Detection of Contrails in Satellite Imagery

Abstract

This study presents a novel approach for the detection of contrails in satellite imagery using a convolutional neural network (CNN). Contrails are important to monitor because their contribution to climate change is uncertain and complex. Contrails are found to have a net warming effect because the clouds prevent terrestrial (longwave) radiation from escaping the atmosphere. Globally, this warming effect is greater than the cooling effect the clouds have in the reduction of solar (shortwave) radiation reaching the surface during the daytime. The detection of contrails in satellite imagery is challenging due to their similarity to natural clouds. In this study, a certain type of CNN, U-Net, is used to perform image segmentation in satellite imagery to detect contrails. U-Net can accurately detect contrails with an overall probability of detection of 0.51, a false alarm ratio of 0.46 and a F1 score of 0.52. These results demonstrate the effectiveness of using a U-Net for the detection of contrails in satellite imagery and could be applied to large-scale monitoring of contrail formation to measure their impact on climate change.

1. Introduction

Contrails, also known as aviation-induced cirrus (AIC) are ice clouds that form when atmospheric conditions are near saturation and aircraft emissions inject enough water vapor to form clouds that would otherwise not have existed. The resulting white streaks can persist in the atmosphere for just a few minutes or for hours. The theory for forecasting conditions distinguishing temporary from persistent contrails is straightforward as it only involves knowing pressure, temperature, humidity, and engine efficiency. In practice, while pressure, temperature, and engine efficiency are relatively easy to obtain with sufficient accuracy, humidity in the upper troposphere is not [1,2]. It is important that science improves in observing and forecasting the existence and location (including altitude) of persistent contrails, as the aviation industry is considering procedures to mitigate the impact with changes in flight levels. If the science is incorrect, this could cause the unnecessary use of fuel and therefore extra CO₂ emissions. Moreover, if we do not have sufficient information, operational procedures will not be able to reroute aircraft to avoid regions that have conditions that are favorable for the formation of persistent contrails that cause the largest warming impact. It has been suggested that 5% of anthropogenic radiative forcing can be attributed to aviation [3]. Of all of the emissions from aviation, contrails contribute between 36–81% of radiative forcing [4], and the largest uncertainty in terms of radiative forcing is associated with contrails [5]. It is the persistent contrails that have an impact on the planetary albedo and the earth’s radiation budget. Studies have found that just 12% of flights contribute on the order of 80% of energy forcing [3]. Therefore, diverting aircraft away from where conditions are favorable for the formation of contrails can reduce this radiative forcing [6]. One such study showed that during the height of the COVID-19 pandemic, air traffic declined by 72%, which corresponded with a drop in contrail radiative forcing by 0.7 W/m² [7]. Over just the US, air travel reduced by 36%, which resulted in a reduction of contrails by 22% [8].

The detection of contrails in satellite imagery is crucial to understanding their spatial and temporal extent and their impact on climate. Previous studies on contrail detection have found that contrail formation is overestimated by a factor of 2.0 to 3.5 and that the lifespan of contrails is overestimated by a factor of 17–45% [4]. Earlier work by Mannstein et al. 1999 developed an AVHRR approach for contrail detection [9] that was later adapted for SEVIRI [10]. This paper explores the application of a Convolutional Neural Network (CNN) with a specific U-shaped architecture, U-Net [11], to perform image segmentation on satellite imagery for the detection of contrails. Although the physical and spectral characteristics are quite different, the long and linear shape of contrails is similar to that of fractures in sea ice known as sea ice leads. The approach to detecting contrails presented here is not a new technology, but rather, the detection model used for the detection of sea ice leads that was developed by Hoffman et al. 2019 and 2022 [12,13] is reused—but in a new application. The architecture of the U-Net is the same; the detection model simply relies on training from a different dataset—contrail imagery rather than sea ice imagery. A similar approach has been used to detect ship tracks in satellite imagery [14]. Several other studies have demonstrated AI-based approaches to detecting contrails in satellite imagery, including Zhang et al. [15], Kulik [16], and Siddiqui et al. [17]. Recent work by Ng et al. [18] also uses a CNN to detect contrails in Geostationary Operational Environmental Satellites (GOES) Advanced Baseline Imager (ABI) imagery. The primary advantage demonstrated in this study is the value added by height retrieval by aligning contrail detections with the GOES ABI Cloud Top Height (ACHA) product [19].

2. Materials and Methods

The first step in the formation of a contrail detection method is to utilize a dataset of contrail images that can be used to train, test, and validate a detection method [18]. The detection architecture is based on the same AI architecture that was used to detect quasi-linear sea ice lead features as described by Hoffman et al. in 2021 [12] and updated in 2022 [13]. Both the contrail and sea ice lead detection methods uses a particular kind of convolutional neural network, U-Net, to perform image segmentation, which was first described by Ronneberger et al. in 2015 [11]. For sea ice lead detection, a single thermal imagery channel was used. The adaptation for contrail detection is to use brightness temperature difference imagery between the 11 μm and 12 μm channels. To process GOES ABI, the imagery is divided into several small regions (256 × 256 pixel samples) that are stitched together to cover the entire full disk. Contrails can be detected near processing region boundaries, and contrails that span a processing region boundary can be detected in both regions when the contrails are sufficiently long in each processing region (as long as the contrail segment in each sub-region maintains the spatial characteristics of a contrail). However, when a contrail spans a processing region boundary, the contrail segment on one side of the processing region boundary can be too short for contrail detection. To mitigate any detection bias near processing grid boundaries, the processing allows for a random amount of overlap for neighboring grids. Over a time series of imagery, processing region boundaries rarely overlap and therefore detection artifacts are avoided.

For this project, we initially used a previously published dataset of Landsat-based contrail detections developed by McCloskey et al. in 2021 [20]. The dataset was later replaced by the newer Ng et al. 2023 dataset with GOES-16 imagery [18]. The dataset contains a set of contrails identified by hand analysis, which we use as a truth mask. Because of the differences in sensitivity between ice and water clouds between 11 μm and 12 μm [21], we use brightness temperature differences to identify cirrus clouds. For the training, testing, and validation set, the GOES ABI 11 μm and 12 μm channels are Band 14 and Band 15. The original Ng et al. dataset [18] contains several thousand cases that can be segregated for training, testing, and validation. For our study, we have used a subset of 10,000 cases. To avoid overfitting, care was taken to avoid cross-contamination of the training, testing, and validation datasets. From the 10,000 cases, a ratio of 70%/20%/10% was used for training, testing, and validation. The Ng et al. dataset [18] contains a time series of images for contrail masks; however, for operational simplicity, our analysis only uses the central time period in each contrail time series. Example validation imagery is shown for three randomly selected cases in Figure 1: the RGB imagery (red = 8 μm, green = 11 μm, blue = 12 μm) is shown for reference in the left column, and the second column is the 12 μm minus 11 μm brightness temperature difference imagery that is used in the detection model. The third column is the Ng et al. [18] mask used as truth, and the final column is the U-Net detection model image segmentation solution. In general, the model detections are largely consistent with the truth masks. In top row, there appear to be some contrails oriented in the vertical direction that are errors of omission in the U-Net. However, in the middle and bottom rows, there may be some errors of omission in the hand analysis; some of the U-Net detections resemble contrails in the satellite imagery but escaped detection in the hand analysis mask. Overall, the U-Net does appear to positively identify the contrails that are apparent in the imagery.

3. Results

One way to illustrate detection performance is in the form of a receiver operating characteristic (ROC) curve which is provided in Figure 2; generally, curves that are near the upper left corner are good, and a detection model near the black line is indicative of low skill. Another way to assess skill is to measure the area under the curve (AUC-PR), and the results we achieved have an AUC-PR of 73.9, which compares favorably to the AUC-PR of 72.7 achieved by Ng et al. [18].

The image segmentation results form an image of probabilities from 0 to 1, encoded as bytes ranging from 0 to 255. To make deterministic evaluations, a threshold to distinguish between contrail or clear must be chosen. For our application, we selected a value of 65 as the detection threshold that achieves the maximum F1 score. At a detection threshold of 65 (or 0.254 if encoded 0 to 1), the probability of detection is 0.508 and the false alarm rate is 0.460. The F1 Score, also known as the Dice coefficient, is 0.524 and may be the best measure of success for the detection model, as it captures a balance of omission and commission error. A lower detection threshold could have been chosen to achieve a higher rate of true positives but at the expense of a higher false positive detection rate. However, the image segmentation results tend to be largely bimodal with the majority of the solutions being near zero or near 250; relatively few solutions are near the value of the detection threshold. Despite some imperfections in the validation dataset and a rather conservative detection threshold, the detection metrics of the technique are overall very good, as shown in

Table 1. Contingency table and contrail detection performance statistics including percent correct (PC), probability of detection (POD), false alarm ratio (FAR), critical success index (CSI), Hanssen–Kiuper skill score (KSS), and F1 Score.

		Validation Mask: Contrail			Validation Mask: Clear
Product: Contrail		A (True Positive, “hit”)			B (False Positive, “false alarm”)
Product: Clear		C (False Negative, “miss”)			D (True Negative, “correct negative”)
	PC $\frac{A+D}{A+B+C+D}$	POD $\frac{A}{A+C}$	FAR $\frac{B}{A+B}$	CSI $\frac{A}{A+B+C}$	KSS $\frac{AD-BD}{\left(A+C\right)\left(B+D\right)}$	F1 Score $\frac{A}{A+0.5\left(B+C\right)}$
	0.995	0.508	0.460	0.355	0.076	0.524

.

An advantage of using thermal IR bands on ABI is that the technique can be applied day or night. The main advantage of using the GOES ABI is that it provides nearly complete coverage over the Western hemisphere every 10 min at a nominal 2 km resolution. An example image of contrail detection over the Gulf of Mexico in GOES imagery is shown in Figure 3. In this example, contrail detections are overlaid on an image of the 11 µm brightness temperature image from 0420 UTC (or 10:20 PM local time) on 10 March 2023. The contrails are color-coded by the GOES ABI Cloud Top Height (ACHA) product [19]. At that time, the native ACHA product had a 4 km nominal resolution, so the product is interpolated to match the 2 km resolution of the native ABI brightness temperature imagery. Later in March of 2023, a 2 km resolution ABI Cloud Top Height product became operational [19] to match the 2 km resolution of the imagery so that interpolation will not be needed in future work. Additionally, the ACHA height is given in km; for aviation applications, it is converted to feet, and for each contrail, the height reported is the highest height retrieval within each contrail object.

4. Discussion

Another GOES contrail detection case is presented in Figure 4. On 14 March 2023 at 2100 UTC, true color imagery is presented in Panel A and the corresponding 11 μm brightness temperature imagery is presented in Panel B. It is interesting to note that the contrails are barely distinguishable in the true color image against the snow-covered background; however, the contrails are readily apparent in the thermal IR. The GOES Cloud Mask and Cloud Top Pressure products are shown in Panels C and D, respectively. The contrail detection product is shown in Panel E, where it is interesting to note that the western and northern contrails are detected with a maximum height of between 20–25 thousand feet (orange), but the eastern and southern Wisconsin contrails are not detected by the GOES cloud mask (black). Finally, as validation, a photograph is shown looking northward from southern Wisconsin, taken at 4:05 P.M. local time (2105 UTC). The primary advantage of the detection method is that it uses techniques previously demonstrated to be effective in satellite remote sensing applications [12,13], and that it can be applied to this new application without changing the model architecture. The technique also is also relatively simple to run in an operational sense because the detections are based on a single brightness temperature difference image rather than more complex multi-spectral time series of images [18]. One weakness of using a simple brightness temperature difference image for contrail detection is that there are non-cloud features that can have similar spatial characteristics to contrails and also have a brightness temperature difference between 11 μm and 12 μm. Rivers and coastlines can cause apparent false detections; as an example, in Figure 4, the Mississippi River along the Illinois/Iowa border shows a contrail detection not associated with any cloud mask detections (black in Figure 4E). This was a larger issue in early iterations of the detection method that used Landsat based training imagery [20]; however, this has become less of an issue after training with a more extensive training dataset [18]. Because ACHA height retrievals are limited to cloudy regions identified by the ABI Cloud Mask [19], non-cloud false contrail detections can be filtered out by using the ACHA product. However, this does not remove all false contrail detections because there are cases where natural clouds are misclassified as contrails. One other point to make is that contrail detections are often longer and wider features than are detected in the cloud mask or cloud height products. This is because the cloud mask and height product detection techniques are based primarily on spectral differencing tests. In contrast, the U-Net used for the contrail detection model is based on a combination of spatial and spectral characteristics. Features that have low patches of low spectral contrast may escape continuous detection in the cloud mask but may be identified as a continuous contrail due to the spatial characteristics identified by the U-Net.

5. Conclusions

The premise that the same AI-based detection method that was used for the identification of sea ice leads in satellite imagery [12,13] can also be used for the detection of contrails in Geostationary satellite imagery has proven to be quite successful. Future iterations may focus on a multi-channel approach, using a time series of imagery and/or using more customized neural networks that have been used in other contrail detection studies [8,15,16,18,20]. It will likely be possible to achieve better detection metrics with more advanced techniques; however, there will likely be tradeoffs in terms developmental time, operational complexity, and/or processing loads. As a proof-of-concept, the single brightness temperature difference does very well at identifying ice clouds. However, the single temperature difference image is prone to false positives due to non-cloud features such as rivers and coastlines. Adding more cloud testing by using imagery from more satellite channels will be beneficial for cloud detection and therefore contrail detection. Visible bands on ABI, for example, would add skill in cloud detection, and the near-IR bands have sensitivity to water vapor and can help differentiate between water and ice cloud. These bands would however be limited to daytime only and would therefore not help detect contrails at night, when they have the largest warming effect.

Future work will also bring in information on atmospheric conditions. For example, Zhang et al. [15] identify regions where the atmosphere is nearly saturated with respect to ice at altitude. We would be able to combine forecasts where contrail detection may be likely to occur with observations. Further, by assimilating this with maps of flight paths or knowledge of where commercial air traffic is prevalent, we could gain greater confidence in the detection of contrails when they are detected within regions where conditions are known to be favorable for the formation of contrails.