Evaluation of Five Global Top-of-Atmosphere Outgoing Longwave Radiation Products

Abstract

Five global monthly top-of-atmosphere (TOA) outgoing longwave radiation (OLR) products are evaluated in this study, including the products derived from the High-Resolution Infrared Radiation Sounder (HIRS), Clouds and the Earth’s Radiant Energy System (CERES), Advanced Very High Resolution Radiometer (AVHRR), the CM SAF cLoud, Albedo and surface RAdiation dataset from AVHRR data (CLARA), and the Global Energy and Water Cycle EXchanges (GEWEX) project. Results show that overall there is good consistency among these five products. Larger differences are found between GEWEX and CERES (HIRS) after (before) 2000 (RMSE ~ 5 W/m²), particularly in the tropical regions. In terms of global mean values, GEWEX shows large differences with the other products from the year 1992 to 2002, and CLARA shows large differences from the year 1979 to 1981, which are more obvious in the global ocean values. Large discrepancies among these products exist at low latitudinal bands, particularly before the year 2000. Australia and Asia (mid–low latitude part) are two typical regions in which larger differences are found.

1. Introduction

Top-of-atmosphere (TOA) outgoing longwave radiation (OLR), defined as the radiation emitted from the Earth’s surface and atmosphere to space, plays a crucial role in the Earth’s energy budget [1]. It has been widely used in climate research due to its ability to indicate the complicated atmospheric conditions and evaluate climate drivers and feedbacks [2]. Additionally, as a key component of the Earth’s energy budget at the top-of-atmosphere (TOA), accurate estimation of OLR is essential to the study of the Earth’s energy imbalance [3]. Remote sensing provides the only way to obtain global and regional OLR. Typically, OLR products are derived from radiative transfer simulations using the traditional two-step method (i.e., converting radiance to flux and then flux to OLR), and direct retrieval methods have also been used to obtain the TOA OLR. Using these methods, numerous satellite TOA OLR products have been derived since the 1980s [4,5,6,7,8,9,10,11].

Currently, these TOA OLR products are being widely used in various aspects. For instance, Rao et al. [12] used the TOA OLR product derived from Advanced Very High Resolution Radiometer (AVHRR) data to build a machine learning model for the estimation of daily average surface air temperature over the Tibetan Plateau. Compared to the AVHRR OLR, another globally gridded climate data record (CDR) for OLR using the High-Resolution Infrared Radiation Sounder (HIRS) sensor has the advantage of resolving the large diurnal cycle of OLR and thus has been used for identifying tropical subseasonal variabilities [13]. Spectrally resolved OLR products are also available from the Atmospheric Infrared Sounder (AIRS) and the Infrared Atmospheric Sounding Interferometer (IASI) sounders [14,15], and fourteen years of spectral fluxes derived from collocated AIRS and Clouds and the Earth’s Radiant Energy System (CERES) observations have been used to examine the trends of zonal mean spectral OLR and greenhouse efficiency in the Arctic [16].

To ensure that these products can be more effectively used, it is necessary to evaluate them. Since there are no in situ measurements at the TOA level, the evaluation of the TOA radiation products can only be performed via intercomparisons. For example, Zhan et al. (2018) intercompared CERES and the Multi-angle Imaging SpectroRadiometer (MISR) collocated instantaneous TOA albedos for overcast ocean and snow/ice scenes based on three years of summertime observations and found significant agreement between the CERES and MISR within the Arctic [17]. Zhan et al. (2019) intercompared five satellite TOA albedo products over land and found that the differences among these products in the high-latitude regions were relatively larger, which is different from the results of Zhan et al. (2018). Larger differences were also found during the years before 2000 [18]. Susskind et al. (2012) examined spatial anomaly time series of OLR as determined using observations from CERES and AIRS and found excellent agreement of the two OLR datasets in almost every detail [19]. Wang et al. (2021) presented the results of detailed validation tests with NOAA-20 Cross Track Infrared Sounder (CrIS) OLR at a global scale compared with CERES and Suomi National Polar-orbiting Partnership (SNPP) CrIS OLR products. Results show that the NOAA-20 CrIS OLR products agreed very well with the other two OLR products at all timescales [20]. Taylor et al. (2022) compared CERES TOA radiative fluxes with Arctic Radiation-IceBridge Sea and Ice Experiment (ARISE) airborne measurements, and results indicated excellent agreement in the longwave flux and good agreement in the shortwave flux [21]. From these studies, one can see that the existing intercomparisons are limited to no more than three TOA radiation products, although some studies exploited the OLR datasets from different instruments to evaluate the change in the OLR over time [20]. The different use of input cloud products also contributes to the biases among different products [22], which have their own advantages and disadvantages (e.g., the accuracy of CERES data is high, but their spatial resolutions are very coarse), and no study has systematically compared five TOA OLR products at the same time. In this study, five TOA OLR products are comprehensively evaluated to show their differences.

The organization of the remainder of this paper is as follows. Section 2 introduces the data and method used in this study. Section 3 shows the results and the corresponding analyses. Conclusions are drawn in the final section.

2. Data and Method

Table 1. Details of the five TOA OLR products used in this study.

Data Source	Time Span	Spatial Resolution	Temporal Resolution	Reference
CERES	2000–2021	1°	Hourly/daily/monthly	[7]
HIRS	1979–2020	2.5°	Daily/monthly	[8]
GEWEX	1988–2009	1°	3-hourly/daily/monthly	[23]
AVHRR	1981–2018	0.05°	Daily/monthly	[1]
CLARA	1979–2023	0.25°	Daily/monthly	[24]

shows the details of the five all-sky TOA OLR products evaluated in this study. As the temporal resolution of these products is different, we focused on the intercomparisons at monthly scales. Each dataset is briefly described in the following section.

2.1. CERES OLR

CERES is a broadband instrument measuring shortwave reflected radiation (0.3–5 μm), longwave thermal radiation (8–12 μm), and broadband radiation from 0.3 to 200 μm. The sensor has been onboard Terra, Aqua, and Suomi National Polar-orbiting Partnership (Suomi NPP). The Level-2 Single Scanner Footprint (SSF) provides the 20 km (at nadir) instantaneous TOA OLR, whereas the Level-3 Synoptic products (i.e., the SYN1deg data) incorporating the geostationary satellite data have much coarser resolutions (1 degree) [7]. The CERES TOA radiation products have been widely recognized as the most accurate TOA broadband flux retrievals. Additionally, CERES Energy Balanced and Filled (EBAF) TOA flux products, which conduct adjustments to TOA radiation to ensure that global mean net TOA flux is consistent with the in situ values. In this paper, monthly CERES EBAF OLR data are used as the reference dataset.

2.2. HIRS OLR

The National Oceanic and Atmospheric Administration (NOAA) Climate Data Record (CDR) program provides a long-term record of global OLR derived from the HIRS instruments. The HIRS OLR was estimated by multispectral regression models proposed by [25]. This new technique holds the promise of eliminating large systematic errors, and thus the Root Mean Square Error (RMSE) is much smaller than the estimated fluxes from AVHRR data. NOAA has been generating HIRS OLR operationally since 1998, and a new radiance calibration procedure was applied to achieve better accuracy. Global monthly diurnal models were constructed that are consistent with the HIRS OLR retrievals to reduce errors. Therefore, the HIRS OLR presents a comparable stability as that in the Earth Radiation Budget Satellite (ERBS) nonscanner OLR measurements [8].

2.3. GEWEX OLR

The Global Energy and Water Cycle EXchanges (GEWEX) project began more than thirty years ago within the framework of the World Climate Research Programme (WCRP). This project can help us gain a better understanding of the Earth’s energy and water cycles. One of the focuses of the project is the quantification of Earth’s energy imbalance (EEI), which is defined as the imbalance between incoming and outgoing radiation at the TOA [26]. Various datasets are obtained from satellites and in situ observation networks around the globe, and the surface radiation budget (SRB) is the one containing surface and TOA shortwave (SW) and longwave (LW) radiative fluxes. The dataset is produced on a 1° × 1° global grid using satellite-derived cloud parameters and ozone fields, reanalysis meteorology, and a few other ancillary datasets.

2.4. AVHRR OLR

Due to their long time span and relatively high spatial resolution, AVHRR data have been widely used to generate radiation products [27,28]. Recently, Zhou et al. (submitted) developed a new method to generate the OLR-AVHRR with a spatial resolution of 0.05°. Specifically, a look-up table (LUT) of regression coefficients that quantitatively links the AVHRR TOA radiance to the instantaneous broadband OLR was built by a radiative transfer model (MODTRAN 5). Based on the instantaneous retrieval, the daily mean OLR was calculated as the quantitative relationship between the daily mean and the instantaneous OLR was obtained using CERES 3-hourly and daily OLR data.

2.5. CLARA OLR

CLARA-A3 is also an AVHRR-based climate data record on clouds, radiation, and surface albedo, and here we use CLARA OLR to differentiate it from AVHRR OLR in Section 2.4. The retrieval of CLARA OLR consists of two procedures. Firstly, the instantaneous OLR is derived from the AVHRR observations in channels 4 and 5 using a large database of collocated AVHRR-CERES observations. Secondly, daily and monthly OLR are obtained from the instantaneous AVHRR observations using the OLR from ERA5 reanalysis data or simple linear regressions. As they remap the GAC orbit grid to a nested 0.25° lat–lon grid, the spatial resolution of the final product is also 0.25° [24].

2.6. Data Processing

As the five TOA OLR products are of different spatial resolutions, we aggregated them to 2.5° for better intercomparisons. The evaluation was conducted at different spatial extents, including global, latitudinal, and regional scales. The latitudinal average was obtained for every 30° latitudinal band and the regional average was obtained based on different regions. Climatology (the monthly average values over 10–20 years) was used to report the systematic issues of different products, following ideas from [29]. Time-series anomalies of the products with a common base period of 2006–2009 were obtained by subtracting the corresponding average values from the absolute values.

3. Results Analysis

Figure 1 shows the average TOA OLR differences between CERES and the other four products in January and July from 2001 to 2009 (overlapping time among the products), and

Table 2. The RMSE and bias between the four TOA OLR products and CERES in January and July from 2001 to 2009 (unit: W/m²).

	GEWEX		HIRS		AVHRR		CLARA
	RMSE	Bias	RMSE	Bias	RMSE	Bias	RMSE	Bias
January	5.31	−1.58	2.87	−2.08	3.41	−2.40	3.04	−2.44
July	5.04	−1.29	2.43	−1.16	3.31	−2.19	3.28	−2.57

presents the corresponding RMSE (the RMSE was calculated from all the pixels) and bias values. The largest differences (the RMSEs were 5.31 W/m² and 5.04 W/m² in January and July, respectively) were found between GEWEX and CERES, as Figure 1a,b show. Obvious positive biases were found in the tropical regions in both January and July, and notable negative biases can be seen to the east of Atlantic Ocean in July. The positive bias in the tropics was attributed to the effect of clouds added to the atmosphere [30]. As for the differences between HIRS and CERES, in January, there were negative biases in the South Pole, and small positive biases were found in other parts. In July, there were more obvious positive biases, particularly in the north of Africa. As for the differences between AVHRR and CERES, as Figure 1e,f show, overall they were small, although notable biases can also be found (e.g., positive biases in the Tibetan Plateau in January). Figure 1g,h present the differences between CLARA and CERES. From the two sub-figures, one can see that the differences were slightly smaller (the RMSEs were 3.04 W/m² and 3.28 W/m² in January and July, respectively) than in Figure 1e,f, and similar patterns can be seen, as both of them were derived from AVHRR data. Additionally, from Table 2 one can see that overall all four TOA OLR products showed negative biases when compared to the CERES data.

Figure 2 presents similar results using the average values from 1988 to 1999, which is also the overlapping time among the products. However, as the time span of CERES did not cover the years before 2000, only four products were intercompared in this part (i.e., HIRS, GEWEX, AVHRR, and CLARA), and HIRS data were taken as reference data.

Table 3. The RMSE and bias between the two TOA OLR products and HIRS in January and July from 1988 to 1999 (unit: W/m²).

	GEWEX		AVHRR		CLARA
	RMSE	Bias	RMSE	Bias	RMSE	Bias
January	4.49	1.06	1.75	−0.24	2.35	−0.95
July	4.76	0.78	2.25	−0.86	3.22	−1.89

presents the corresponding RMSE and bias values. The larger differences (the RMSEs were 4.49 W/m² and 4.76 W/m² in January and July, respectively) were found between GEWEX and HIRS, as Figure 2a,b show. Obvious positive biases were also found in the tropical regions in both January and July. As for the differences between AVHRR and HIRS, they were much smaller (the RMSEs were 1.75 W/m² and 2.25 W/m² in January and July, respectively). However, notable negative biases were found near 70°E in July. As for the differences between CLARA and HIRS, they were also much smaller but relatively larger than the differences between AVHRR and HIRS, with RMSEs of 2.35 W/m² and 3.22 W/m² in January and July, respectively.

These TOA OLR products are widely used in analyzing the long-term changes of the Earth’s energy budget. Thus, it is necessary to evaluate them in time series, and Figure 3 shows the time series of the average TOA OLR anomalies of the five products at global scale. Figure 3a shows the global mean TOA OLR anomalies of these products, and overall there was good consistency among these five products. GEWEX showed overestimations from the year 1992 to 2002, and CLARA showed obvious underestimations from the year 1979 to 1981 compared to HIRS and AVHRR, which may be attributed to the periods with degraded temporal coverage [23]. As the properties of land are quite different from those of the ocean, the separate results are presented in Figure 3b and Figure 3c, respectively. From Figure 3b, one can see that these five TOA OLR products matched quite well with each other, although some small differences can be seen in certain years. In Figure 3c, however, much larger discrepancies are shown. For instance, GEWEX OLR exhibited obvious overestimations compared to the other products. Additionally, CLARA also showed underestimations before 2000 compared to HIRS and AVHRR.

Figure 4 shows HIRS, CERES, GEWEX, AVHRR, and CLARA monthly mean TOA OLR anomalies of different latitudinal bands. Overall, the five TOA OLR products matched each other very well at 60–90°N, 30–60°N, and 60–90°S. However, there were large discrepancies among these products at the other latitudinal bands. For example, at 0–30°S (Figure 4d), the range of GEWEX OLR values was much smaller than that of the other products, indicating the relatively low accuracy at this latitudinal band; this is also obvious in Figure 4c. In Figure 4e, overestimations can be found in GEWEX and slight underestimations can be seen in CLARA before the year 2000 at this latitudinal band (30–60°S).

To fully understand the performance of the five TOA OLR products in different regions, we divided the world area into 12 parts (2 additional ANT and GRL parts added to [18]). Then, similar to Figure 4, we also calculated the HIRS, CERES, GEWEX, AVHRR, and CLARA monthly mean TOA OLR anomalies in each region, as shown in Figure 5. From the 12 sub-figures, one can see that the five TOA OLR products matched each other very well in EUR, NA2, AS2, MCT, ANT, and GRL, as shown in Figure 5b,d,h,i,k,l. However, there were also large discrepancies among these products in other regions. For instance, the differences between GEWEX OLR and the other products were much larger in AS1 and AUS, as shown in Figure 5g,j.

4. Conclusions

We examine the differences in five TOA OLR datasets (i.e., HIRS, CERES, GEWEX, AVHRR, and CLARA) in this study. By comparing CERES and the other four products in 2001–2009, larger differences between GEWEX and CERES are found in the tropical regions, with an RMSE of ~5 W/m², whereas the differences of the other three products are much smaller, with an RMSE of ~3 W/m². By comparing HIRS and the other three products (AVHRR, GEWEX, and CLARA) in 1988–1999, larger differences between GEWEX and HIRS are also found in the tropical regions, with an RMSE of ~5 W/m², whereas the differences between HIRS and AVHRR or CLARA are much smaller, with an RMSE of ~2 W/m².

Time-series evaluation results show that overall there is good consistency among these five products. The five TOA OLR products match each other very well at 60–90°N, 30–60°N, and 60–90°S, whereas there are large discrepancies among these products at the other latitudinal bands. To demonstrate how these five products perform regionally, we divide the world area into twelve parts. Results show that there are larger discrepancies among the five products in Australia and Asia (mid–low latitude part). Overall, the consistency among the five TOA OLR products is relatively lower in the mid–low latitude regions, where TOA albedo products show good consistency. As for the high-latitude regions, however, there is much better consistency among the five TOA OLR products.