Using the NASA EOS A-Train to Probe the Performance of the NOAA PATMOS-x Cloud Fraction CDR

Abstract

An important component of the AVHRR PATMOS-x climate date record (CDR)—or any satellite cloud climatology—is the performance of its cloud detection scheme and the subsequent quality of its cloud fraction CDR. PATMOS-x employs the NOAA Enterprise Cloud Mask for this, which is based on a naïve Bayesian approach. The goal of this paper is to generate analysis of the PATMOS-x cloud fraction CDR to facilitate its use in climate studies. Performance of PATMOS-x cloud detection is compared to that of the well-established MYD35 and CALIPSO products from the EOS A-Train. Results show the AVHRR PATMOS-x CDR compares well against CALIPSO with most regions showing proportional correct values of 0.90 without any spatial filtering and 0.95 when a spatial filter is applied. Values are similar for the NASA MODIS MYD35 mask. A direct comparison of PATMOS-x and MYD35 from 2003 to 2014 also shows agreement over most regions in terms of mean cloud amount, inter-annual variability, and linear trends. Regional and seasonal differences are discussed. The analysis demonstrates that PATMOS-x cloud amount uncertainty could effectively screen regions where PATMOS-x differs from MYD35.

1. Introduction

The Pathfinder Atmospheres Extended (PATMOS-x) is a NOAA project focused mainly on the generation of cloud and related satellite-derived climate data records (CDRs). According to the National Research Council [1], a CDR is a time series of measurements of sufficient length, consistency, and continuity to determine climate variability and change. After decades of studies, clouds still constitute one of the largest uncertainties in projecting future climate change [2]. Therefore, it is essential to establish a cloud CDR for assessing current climate and better understanding the roles of clouds in the earth–atmosphere system. PATMOS-x aims to be a dataset that can contribute to reducing this uncertainty. The Advanced Very High Resolution (AVHRR) version of PATMOS-x is hosted by the NOAA National Centers for Environmental Information (NCEI) [3]. The first generation AVHRR/1 flown onboard TIROS-N was launched in 1978 and later on other NOAA satellites. It had four bands centered at 0.63, 0.83, 3.7, and 11 µm. The second generation AVHRR/2 had an additional channel centered at 12 µm, and the latest generation AVHRR/3 has added one more channel centered at 1.6 µm. The AVHRR/3 is onboard both NOAA’s and EUMETSAT’s Metop satellites. The AVHRR PATMOS-x spans 1978 to the present and is updated daily. Other versions of PATMOS-x applied to other sensors are also being developed and are being served by the Climate Data Portal (CDP) at the University of Wisconsin-Madison/Cooperative Institute for Meteorological Satellite Studies (CIMSS). While PATMOS-x was the only AVHRR climatology participating in the Global Energy and Water Experiment (GEWEX) Cloud Climatology Assessment [4], other AVHRR climatologies now exist or are under development. One such data set is that from the European Organization for the Exploitation of Meteorological Satellites (EUMETSAT) Climate Monitoring Satellite Application Facility (CM-SAF) called Clouds, Albedo, and Radiation dataset from AVHRR data (CLARA-A1) [5].

This paper focuses on the cloud fraction CDR of PATMOS-x. The cloud detection components in PATMOS-x include the pixel-level cloud probability from the naïve Bayesian cloud detection scheme and the derived four-level cloud mask [6]. It is important to point out there are many methods used to detect clouds and many have been applied to the AVHRR [7,8,9], and the results shown here may differ from other AVHRR cloud climatologies. In addition, PATMOS-x includes a cloud fraction derived over a 3 × 3 pixel array with a corresponding measure of uncertainty. The cloud fraction CDR is often applied directly in the study of multi-decadal climate change. For example, the recent GEWEX cloud climatology [4] included PATMOS-x results in its cloud fraction comparisons, and cloud fraction patterns have been used to track the evolution of deep convection in the Tropics [10]. The PATMOS-x cloud fraction CDR has also been used in studies of Dust [11], regional cloudiness variations [12,13,14] , and the global diurnal cycles of cloudiness [15].

In addition to being necessary for the creation of the cloud fraction CDR, the process of cloud detection is important because PATMOS-x has been designed to serve as a basis for subsequent CDR generation. For example, the NCEI AVHRR Aerosol CDR uses PATMOS-x observations and cloud detection to generate its CDR [16], as does the NCEI HIRS Cloud CDR [17]. The NCEI Advanced Microwave Sounding Unit (AMSU-B) Fundamental CDR also uses PATMOS-x results to characterize its scan asymmetry corrections. This analysis may provide guidance to those and other users dependent on PATMOS-x cloud detection and cloud fractions for generation of their own CDRs.

The goal of this paper is to provide the remote sensing community an assessment of the performance of the PATMOS-x cloud fraction CDR compared to two well-regarded cloud fraction standards provided by NASA EOS A-Train. The A-Train constellation includes a group of satellites tasked with observing several aspects of the earth–atmosphere system. These satellites closely follow one another along the same orbital track, allowing for co-location with high temporal and spatial consistency. The first A-Train cloud fraction source is the NASA Aqua/MODIS Science Team cloud mask (MYD35) [18,19]. The MYD35 algorithm has been developed for over 20 years and has been applied to many applications and validated extensively [4,20,21]. The second is derived from the cloud layer products from the Cloud–Aerosol Lidar with Orthogonal Polarization (CALIOP) instrument flown on the NASA Cloud–Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) mission [22]. CALIOP is a space- borne lidar and provides direct detection of cloud within the narrow lidar beam, allowing it to be free from many assumptions about surface or atmospheric properties. It is important to note that these comparisons are only relevant for AVHRR data that flew in orbits similar to those of the EOS A-Train. These results also cannot be used to predict the performance for early AVHRR sensors, which suffered from high amounts of noise and other radiometric issues.

While the bulk of this paper deals with the performance of the AVHRR, PATMOS-x also supports measurements from the MODIS and Visible and Infrared Imaging Radiometer Suite (VIIRS) polar-orbiting sensors. In the last section we use MODIS—which has all of the AVHRR channels and almost all of the VIIRS channels—to simulate cloud detection using the available spectral information from each sensor. This allows for the isolation of the impact of additional spectral channels offered by MODIS and VIIRS relative to AVHRR.

Table 1. Spectral channels used in the PATMOS-x cloud mask and the presence on each sensor used in this study. DNB: Day Night Band.

Sensor	Nominal Wavelengths (µm) of Channels Used in PATMOS-x Cloud Mask
	0.65	0.86	1.38	1.6	3.75	6.7	8.5	11	12	DNB
AVHRR/3a	✓	✓		✓ (day)	✓ (night)			✓	✓
AVHRR/3b	✓	✓			✓			✓	✓
MODIS	✓	✓	✓	✓	✓	✓	✓	✓	✓
VIIRS	✓	✓	✓	✓	✓		✓	✓	✓	✓

provides the spectral channels used by the NOAA Enterprise mask in PATMOS-x on each sensor. Unless stated, the AVHRR results always refer to the AVHRR/3b channel set. Lastly, this paper will quantify the sensitivity of cloud detection to the assumed prior cloud fraction used in the naïve Bayesian algorithm.

2. Materials and Methods

As stated above, this paper compares the PATMOS-x cloud detection relative to the MODIS MYD35 C6 cloud mask and to the CALIPSO/CALIOP cloud layer product version 3. There are four input data sets used here. The first is from the CALIPSO/CALIOP instrument. The second is the MODIS Level-1b data, and the third are the NASA MODIS cloud detection results. The fourth are the actual NOAA-19 AVHRR results, which are included to help provide context for the MODIS-derived AVHRR results. The main output from this study is the analysis of the PATMOS-x results generated from the MODIS Level-1b compared to the NASA MODIS and CALIPSO/CALIOP cloud detection results. An intermediate step in the CALIPSO/CALIOP analysis is co-location with MODIS Level-1b. This CALIPSO/CALIOP analysis was also applied to the NOAA-19 AVHRR data. All of these data and the co-location process are described in this section. The CALIPSO/CALIOP analysis was done for all data from 2013. The multi-year comparisons were carried out on every third day from 2003 to 2014. The thinning of every third day was done to fit within hardware and storage constraints. This temporal thinning is assumed to not impact the global and long-term averages.

2.1. CALIPSO/CALIOP Data

The NASA CALIPSO/CALIOP cloud detection products are generated at several different spatial resolutions. For this study, the Level-2 1-km cloud layer products [22] are used. The 1-km CALIOP products have a comparable spatial resolution to the MODIS retrievals and span the entire Troposphere. CALIOP products at coarser resolution are also available, but these products are much more sensitive to the presence of upper Tropospheric Cloud than the MODIS retrievals. In our experience, the 1-km resolution products appear to be best suited for this analysis. Clouds absent from the 1-km product are likely to be undetectable by the MODIS observations. A CALIOP result is classified as cloudy if the number of detected cloud layers exceeds zero.

2.2. MODIS Data

For convenience, PATMOS-x is run on the 5-km 5-min sub-sampled AQUA/MODIS Level1b data (MYD02SSH C6), which is a standard product available from NASA LAADSWEB [23] Note that TERRA/MODIS is not included here as it is a morning satellite and not part of the A-Train. The MODIS sub-sampled data are derived from the Level 1B 1-km data by extracting information from every fifth pixel, starting at the third one, both along and across track. Essentially, this means the center of every 5 × 5 pixels. All results shown here are generated on MYD02SSH data, including those simulated for AVHRR and VIIRS. This is possible because MODIS provides all of the channels used for cloud detection for AVHRR and VIIRS [24]. In this paper, the MYD35 results were provided by MYDATML2 data, which is also sub-sampled at 5-km. The same pixels present in the MYD02SSH data are present in MYDATML2 data. Any and all MODIS data used in this study are the 5-km sub-sampled data provided by MYD02SSH and MYDATML2 C6. Given its convenient size, quality, and length of record, MYD02SSH is a critical climate data product and the performance of cloud fraction CDRs generated from it warrant study.

2.3. PATMOS-x Data

PATMOS-x uses algorithms from NOAA Enterprise Operational Suite. The term “Enterprise” is now used at NOAA because of NOAA’s plan to apply the same algorithm codes to multiple sensors in an effort to provide more consistent services to its customers. Therefore, the same PATMOS-x cloud detection algorithm operates on all sensors. This capability is exploited here to provide insight into the PATMOS-x cloud fraction CDR by using MYD02SSH data. PATMOS-x can restrict the use of channels and therefore can simulate the results from VIIRS and AVHRR when processing MODIS data. PATMOS-x was also modified to read the MYD35 cloud mask. This setup allows us to directly compare the PATMOS-x mask to the MYD35 mask without any assumption on co-location and avoidance of viewing geometry differences [25]. While this assumption ignores the spatial resolution and radiometric differences of the AVHRR and MYD02SSH, it allows for an isolation of the impact of the spectral channels. Throughout this paper, the authentic PATMOS-x AVHRR data generated from NOAA-19 AVHRR will also be used in these comparisons to provide a check on the reality of the MODIS-derived AVHRR results.

The primary product analyzed here is the cloud fraction. PATMOS-x uses a pixel’s cloud fraction computed over the surrounding 3 × 3 pixel array. The cloud mask for each pixel is converted to binary mask, and the mean of binary mask over the 3 × 3 pixel array is the cloud fraction. Therefore, the pixel-level cloud amounts are multiplied by 1/9. For the MYD35 cloud amounts, the exact same procedure is followed. The four-level MYD35 mask is converted to a binary mask and averaged over the surrounding 3 × 3 pixel array.

PATMOS-x was run and generated the standard Level-2b data, which are global fields with a spatial resolution of 0.1° [3]. The Level-2b data are generated from each satellite node (ascending and descending). For AQUA/MODIS, the ascending node is primarily daytime and the descending node is primarily nighttime. The Level-2b data are then averaged in space and time to give Level-3 data with a spatial resolution of 2.5° and averaged temporally over each meteorological season. The Level-3 data are the basis of the MODIS comparisons.

2.4. Colocation of MODIS and CALIPSO/CALIOP

One of the primary analyses here uses co-locations in space and time of the PATMOS-x and the CALIPSO/CALIOP cloud fraction results. Because all data were generated from the same MODIS input, the co-location was done once. A nearest neighbor approach was used to select the closest MODIS pixel for each CALIPSO point. No optical depth or cloud height filters were used. The CALIPSO pixels were classified as cloudy if the number of cloud layers was greater than zero. The CALIPSO cloud fraction was computed as the mean of all the binary CALIPSO mask values used in the comparison. A parallax correction to CALIOP cloud layer results was performed using the CALIOP cloud height and viewing geometry of MODIS.

Figure 1 shows a schematic illustration of this process. The image on the left shows the MYD02SSH/CALIPSO co-location and the image on the right shows the AVHRR/CALIPSO co-location. The larger grids represent the native 1-km pixel rids from both sensors. As these figures illustrate, MYD02SSH represents a direct sampling of every fifth pixel from MYD021KM. The AVHRR Global Area Coverage (GAC) sampling differs with each GAC being comprised of the mean of four 1-km AVHRR pixels with the spacing shown in Figure 1. The CALIPSO results are generated at a spatial scale that matches the dimension of the area represented by a 3 × 3 pixel array of either MYD02SSH or AVHRR/GAC data. As stated above, this is how the PATMOS-x Cloud Fraction CDR is generated. For MYD02SSH, the CALIPSO data is averaged over the 11 CALIPSO pixels centered on the co-located CALIPSO pixel. For AVHRR/GAC, the CALIPSO data is averaged over the seven CALIPSO pixels centered on the co-located CALIPSO pixel. The line of rectangles represents the 1-km CALIOP cloud layer product. In Figure 1, the orange pixels denote the co-located CALIPSO pixels and the yellow pixels denote those used in the CALIPSO cloud fraction computation. For the AVHRR co-locations, data time differences greater than 5 min were excluded. No time filter is needed for the MYD02SSH and CALIPSO comparisons.

3. Results and Discussion

The purpose of this effort is to provide current and future users of the PATMOS-x CDR knowledge of characteristics of the cloud fraction CDR behavior. The techniques described above allow a direct comparison of the PATMOS-x results to those from MYD35 and CALIPSO. This section will first show a comparison of PATMOS-x to MYD35 and analyze the regional differences in key quantities. Then, we use CALIPSO results to provide a quantitative analysis of both PATMOS-x and MYD35. The PATMOS-x cloud fraction uncertainty estimates will also be verified. Lastly, an analysis is performed to show how the spectral content of other sensors impacts the PATMOS-x cloud fraction.

3.1. Comparison of MODIS PATMOS-x to NASA MODIS MYD35

As stated above, the NASA AQUA/MODIS sensor has all the channels provided by the NOAA AVHRR. Applying the PATMOS-x cloud detection to MODIS observations allows a direct comparison to the NASA MODIS cloud detection products (MYD35). In this section, we compare the performance by season, averaged over the period 2003–2014. This discussion will point out the relative differences between MYD35 and PATMOS-x, and the following section will provide more quantitative measures of the MYD35 and PATMOS-x compared to CALIPSO/CALIOP. Again, all PATMOS-x results are generated using the corresponding AVHRR channels on MODIS, run through the PATMOS-x algorithm. All data has been mapped to an equal angle grid with a resolution of 2.5°. Figure 2 shows the comparison of MYD35 and PATMOS-x for winter season, including day and night data. There are six panels in Figure 2 and they are designed to give a concise summary of the comparison. Panel (a) shows the mean global cloud amount for PATMOS-x. As described in [3], the PATMOS-x cloud fraction is computed over the 3 × 3 pixel array. Each pixel in the 3 × 3 array is cloudy if the naïve Bayesian cloud probability for that pixel exceeds 0.5; Panel (b) shows the difference (PATMOS-x–MYD35) cloud amount map; Panel (c) shows the mean cloud detection uncertainty as provided by the PATMOS-x naïve Bayesian algorithm; Panel (d) shows the anomaly correlation of the MYD35 and PATMOS-x time-series. Values of 1.0 indicate grid-cells where the year-to-year variations between MYD35 and PATMOS-x are similar. Inspection of the mean differences in Panel (b) and the anomaly correlations in Panel (d) allow one to separate differences that are systematic biases and those that are not; Panel (e) shows the linear trend applied to the PATMOS-x seasonal time-series. It is not corrected from ENSO or other known atmospheric oscillations; Panel (f) shows a scatterplot of the linear trend values from each grid-cell for MYD35 and PATMOS-x. In summary, these panels are meant to provide a convenient and efficient summary of the performance of PATMOS-x relative to MYD35 on a mean, inter-annual, and decadal basis.

As Figure 2 shows, there is generally more cloud over ocean in the MYD35 than PATMOS-x. Comparison of Panel (b) with (a) shows this occurring in areas of low cloud amount (<0.5). Comparison with Panel (d) shows that these differences occur in regions with high values of anomaly correlations, and this would indicate that these differences are systematic biases (occur each year). There are also significant cloud fraction differences in winter in Siberia and Canada. Unlike the oceanic differences, these differences occur in regions with lower values of anomaly correlation and high values of uncertainty. Therefore, these differences between PATMOS-x and MYD35 point to true differences in the cloud detection performance. For example, MYD35 C6 does appear to have an issue of falsely detecting cloud over snow-covered land during the day [26], and the comparisons shown here are consistent with that. Also, the differences might be contributed from different snow/ice ancillary maps used by MYD35 and PATMOS-x, but this question lies beyond the scope of this paper. While the likely cause of this issue is known, the analysis against CALIPSO in the next section will also provide insight into differences that are not already diagnosed. Lastly, in the bottom two panels are an analysis of the linear trends constructed for the time-series in each grid-cell. Panel (e) shows the linear trend from PATMOS-x. The linear trend from MYD35 was visually very similar. To make this point, Panel (f) shows a scatterplot of trends from MYD35 and PATMOS-x; Panel (f) shows that trends agree in magnitude and sign for most of the globe. This agreement is significant, since one of the most important uses of the PATMOS-x CDR is for multi-decadal climate analysis. For these studies, the stability of multi-decadal variation (for which the linear trend is a surrogate) is more important than the absolute values of the cloud fraction. As discussed later, PATMOS-x uses a prior cloud probability in its formulation, and this raises concern that the PATMOS-x cloud fraction trends may be influenced by this prior value. MYD35 has no such prior cloud probability constraint. The agreement in the trends between PATMOS-x and MYD35 is a reassuring sign that the PATMOS-x trends are indeed valid, and not overly controlled by the choice of prior cloud probability.

Figure 3 shows the same analysis presented in Figure 2 applied to summer seasons. Comparisons for spring and fall were also made, but they provided no unique information and are therefore not shown. In summer, the Northern Land Masses are nearly free of snow, which greatly improves the cloud detection skill. This is confirmed by the decreased values of uncertainty in Panel (c) over the Northern Land Masses. The oceanic differences observed in winter remain in the summer. Over land, the cloud fraction differences are the largest over desert regions, and these differences are larger in summer than winter. The anomaly correlation values for the grid-cells with these differences are high, which again points to the differences being systematic and not influencing the year-to-year or decadal variations. Another difference with the winter results is the change in Antarctic results. In summer, the Antarctic region is mainly free of solar illumination, and the skill in cloud detection drops [27]. For most of Antarctica, PATMOS-x shows much more cloud than MYD35. This region, though, is characterized by high uncertainties, and therefore disagreements are expected. The trends shown in Panels (e) and (f) show the same high level of agreement seen in summer for most regions. While not shown, the grid-cells with trend differences occur in regions with low anomaly correlations and in the predicted high uncertainties.

3.2. Comparison of MODIS PATMOS-x to NASA CALIPSO CALIOP

The CALIPSO/CALIOP sensor in the EOS A-Train provides a direct measure of the presence of cloud and has been used extensively to validate other cloud detection techniques. CALIOP is a lidar and provides nearly direct detection of cloud and other atmospheric scatterers with little dependence on solar illumination or surface characteristics. In this section, the 1 km cloud layer product from CALIOP is used to provide a direct estimate of cloud fraction to better assess the PATMOS-x cloud fraction. Figure 1 and the associated text describe the physical meanings of these two cloud fractions. While there will certainly be differences due to the spatial scales of the PATMOS-x and CALIPSO cloud fractions, there is no better direct and instantaneous comparison from a space-borne sensor at this time.

It is important to note that both the PATMOS-x and MYD35 results have been tuned to optimize their performance. As described in [2], the PATMOS-x naïve Bayesian approach was derived from the same type of CALIPSO/CALIOP observations used here but for the years 2007 and 2009. MYD35 has also used CALIPSO/CALIOP co-locations to help augment the manual derivation of cloud detection thresholds. In both of these algorithms, the threshold values (MYD35) or curves (PATMOS-x) are determined over large areas and long time periods. Therefore, we are confident that the results for this one year are representative for other years. As stated earlier, CALIPSO/CALIOP represents the best global validation source and using it as a reference in cloud fraction comparisons is still relevant, even though both PATMOS-x and MYD35 are tuned to CALIPSO/CALIOP.

The results in this section include the values of cloud fraction and the values probability correct (PC). Using the standard cross-comparison matrix shown in

Table 2. The comparison matrix used to define the PC metrics used in this analysis.

	PATMOS-x or MYD35
CALIPSO	Clear	Cloudy
Clear	a	b
Cloudy	c	d

, PC = (a + d)/(a + b + c + d).

As Figure 1 shows, a comparison of the PATMOS-x cloud fraction from MYD02SSH and AVHRR/GAC is susceptible to spatial sampling issues. PC is computed by converting the cloud fractions to a binary clear or cloudy mask and comparing the agreement of these values using the formulism in Table 2. When computing PC, two spatial filters were applied. The first one attempts to remove all spatial sampling differences by excluding points where the PATMOS-x and CALIPSO (P/C) values were not completely clear (cloud fraction = 0%) or completely cloudy (cloud fraction = 100%). These results are labelled as the 0/100 filter in the following tables. This filter demands homogeneity over 10-km for MYD02SSH and roughly 7-km for AVHRR/GAC. The other filter makes no attempt to remove spatial issues. In the second filter, the binary mask is clear if the cloud fraction is less than 50% and cloudy if the binary mask is greater than or equal to 50%. These results are labelled as to 50/50 filter. In terms of data loss, the 50/50 Filter excludes nothing, but the 0/100 Filter excludes roughly a third of the data. We expect truth to lie between the results of these two filters. While the 0/100 results are optimistic, they should allow for a differentiation of the true failings of the spectral cloud tests from the ambiguity caused by spatial sampling differences. When showing MYD35 and CALIPSO (M/C) results, the same two filters were applied. Comparisons of PATMOS-x relative to MYD35 (P/M) are also shown for both filter settings. Because these data are from the same pixels, there are no spatial sampling issues in the P/M results.

The following comparisons were computed for every day of the year 2013. The results are separated by day and night. While the CALIPSO cloud layer algorithm is not dependent on solar illumination, the same cannot be said for the PATMOS-x and MYD35 cloud detection algorithms. Solar illumination is a major factor in the selection of spectral tests in both the MYD35 and PATMOS-x approaches. The CALIOP instrument does have a day/night difference in performance due to the noise from solar contamination, but this is not accounted for here. The results are stratified by surface type, and the seven surface types used in the PATMOS-x naïve Bayesian Training are employed here [6]. The Antarctic surface type includes Greenland. The distinction between ocean and other water surfaces is taken from the land-sea dataset used in the MODIS C6 [28] processing. The water surface type includes all inland waters and some coastal waters.

Table 3. Variation in CALIPSO-derived verification metrics. CALIPSO 1 km Cloud Layer products from 2013 were used for this analysis. In the table headers, CAL refers to CALIPSO, PM refers to the AVHRR PATMOS-x scheme run on MYD02SSH and MYD refers MYD35. P/C refers to PATMOS-x compared to CALIPSO. M/C refers to MYD35 compared to CALIPSO. P/M refers to PATMOS-x compared to MYD35. A/C refers to NOAA-19/AVHRR/GAC compared to CALIPSO. The regions are those used in the PATMOS-x naïve Bayesian training. These results are for daytime observations for all seasons.

Region	Cloud Fractions			PC 0/100 Filter				PC 50/50 Filter
	CAL	PM	MYD	A/C	P/C	M/C	P/M	A/C	P/C	M/C	P/M
Global	66	63	64	98	99	99	99	89	91	90	95
Ocean	70	67	70	99	100	100	100	89	91	91	97
Water	67	65	65	99	100	99	100	91	93	93	97
Land	60	54	55	98	99	99	99	86	88	88	95
Snow	73	69	75	96	96	98	97	88	88	89	86
Arctic	78	72	72	91	95	96	99	82	90	90	91
Antarctic	79	78	74	98	99	98	98	94	91	89	90
Desert	36	31	28	99	99	99	100	93	90	89	94

presents the results for the daytime analysis of 2013 and shows several characteristics worth noting. First, the P/C and M/C PC values agree within 2% for all surface types and are all above 95% with some values approaching 100% when the 0/100 filter is applied. The A/C and P/C cloud fractions generally agree within 1%. The 50/50 results are uniformly lower than the 0/100 results by as much as 13%. This is expected because the 50/50 results include all of the partly cloudy situations. Again, the P/C and M/C results are very consistent. The A/C values are generally 2% lower than the P/C results, except for the Arctic where the A/C value is 8% lower. The cause of this difference may be due to the inferior radiometric performance of AVHRR in cold regions compared to MODIS or the difference in the angular sampling in the AVHRR and MODIS CALIPSO co-locations. It is problematic that the A/C results beat the P/C results by 3% in the Antarctic. The cloud fractions in Table 3 are computed using all results without any filter. The PATMOS-x and MYD35 results agree within 3% for most surfaces, with the largest exception being snow-covered land, where the MYD35 cloud fraction is 6% higher. The CALIPSO cloud fractions are generally 5% higher than PATMOS-x or MYD35 with the exception of snow-covered land, where MYD35 exceeds CALIPSO by 2%.

The nighttime 2013 comparisons to CALIPSO are shown in

Table 4. Variation in CALIPSO-derived verification metrics. CALIPSO 1 km Cloud Layer products from 2013 were used for this analysis. The regions are those used in the PATMOS-x naïve Bayesian training. These results are for nighttime observations for all seasons.

Region	Cloud Fraction			PC 0/100 Filter				PC 50/50 Filter
	CAL	PAT	MYD	A/C	P/C	M/C	P/M	A/C	P/C	M/C	P/M
Global	72	66	69	95	96	97	96	84	88	88	89
Ocean	76	71	75	98	99	99	99	85	90	90	93
Water	77	72	77	98	98	99	99	90	91	92	92
Land	61	52	62	90	97	97	96	81	91	91	89
Snow	77	60	67	86	90	91	90	76	79	81	78
Arctic	77	66	67	89	95	90	92	78	85	81	84
Antarctic	76	71	57	82	73	85	66	72	67	74	59
Desert	23	17	24	96	97	97	96	89	93	90	90

. In general, cloud detection at night is more uncertain than during the day, and this is reflected in the PC values in Table 3 compared to Table 4. The global PC values dropped by 2% to 6%. The relative agreement between P/C and M/C values remain (with some exceptions) for both spatial filters The A/C shows the same pattern of Arctic degradation and Antarctic improvement relative to P/C, as seen during the day. At night, the snow PC values for A/C are lower than P/C, and this might be due to AVHRR performance at cold temperatures. The M/C values in Antarctic are much higher (7%–10%) compared to P/C. This might be explained by the additional spectral tests in the MYD35 algorithm that are designed for high-latitude nighttime cloud detection. The nighttime cloud fractions show more disagreement for the frozen surfaces than during the day. For example, the PATMOS-x and MYD35 Arctic values are 10% lower than CALIPSO, and MYD35 nighttime Antarctic cloud fraction is 19% lower than CALIPSO and 14% lower than PATMOS-x. PATMOS-x nighttime land fractions appear to be 10% lower than both CALIPSO and MYD35. However, the PC values for P/C are similar to M/C for this surface type. An overriding conclusion from Table 3 is that the additional spectral information used in MYD35 does not dramatically alter the mean performance relative to the AVHRR spectral information. Undoubtably, the spectral information improves the performance, but this analysis indicates this improvement occurs in a relatively small amount of the data.

The results in Table 3 and Table 4 differ from those shown in GEWEX Cloud Climatology Assessment Report. In that report, the global cloud amounts given from CALIPSO was 73% and the PATMOS-x (AVHRR) and MODIS (MYD35) cloud amounts were 68% and 69%, respectively. There are several potential reasons for this. First, the data in GEWEX are not corrected for latitude, and therefore sample the higher latitudes more than lower latitudes due to the sampling characteristics of sub-synchronous satellites flying in the EOS-A-train. Also, the GEWEX CALIPSO data was generated from the 5-km resolution CALIPSO/CALIOP Cloud Layer products. The 5-km product is more sensitive, and would detect more cloud than the 1-km product used here. These numbers are not meant to serve as absolute reference values. They are simply a metric for the comparison of the three data sources in the context of this analysis.

The PATMOS-x to MYD35 (P/M) comparisons are also included in Table 3 and Table 4. Since PATMOS-x was generated on MYD02SSH, the comparisons are of exactly the same pixels and no spatial sampling difference exists. For the daytime land surface types, the 50/50 P/M values are 7% higher than the P/C or P/M values. One could imagine that daytime land with surface heating-driven convection would present the most small-scale cloudiness (and associated spatial sampling issues). Small scale cloud is also ubitiquous over the open oceans in both the day and night, and these surfaces also show larger PC values for P/M than for P/C or M/C. These differences could potentially be used to estimate the degradation in the PC values relative to CALIPSO due to spatial sampling differences in the 50/50 PC results in Table 3 and Table 4. While this indicates the 50/50 PC values maybe underestimated, this correction was not applied.

3.3. Verification of the PATMOS-x Cloud Fraction Uncertainty

One of the main advantages of the probabilistic approach used in PATMOS-x is the uncertainty measure it provides. For each pixel, the naïve Bayesian approach gives a cloud probability of the presence of cloud. PATMOS-x uses a cloud probability threshold of 0.5 for the separation of the binary clear and cloudy distinctions. The uncertainty for a cloudy pixel is assumed to be 1.0 minus cloud probability, and the uncertainty for clear pixels is the cloud probability value. In PATMOS-x, these are averaged over the same 3 × 3 pixel array used to compute cloud fraction. The cloud fraction uncertainty values are included in the PATMOS-x data to provide users additional information to further screen or to apply error bars to their analysis. The 0/100 filter PC values in the previous section also provide another measure of uncertainty which is relative to CALIPSO results. The CALIPSO uncertainty is computed as 100 minus PC, where the PC values are given in Table 3 and Table 4. A natural question is how these two uncertainties relate. Panel (a) of Figure 4 shows a scatterplot of the PATMOS-x cloud fraction uncertainty on the x-axis and the CALIPSO uncertainty on the y-axis. Each point is the mean over 1 of the 7 surface types shown in Table 3 and Table 4. As Panel (a) of Figure 4 shows, the variation is linear but the slope is not unity. The CALIPSO cloud fraction uncertainty is roughly 1.6 times the value predicted by PATMOS-x. It is important to remember that the PATMOS-x cloud detection was tuned to CALIPSO [2]. It is not obvious (to the authors) what constraints the tuning places on this relationship; Panel (b) shows the same analysis except the y-axis shows the 100-PC values computed relative to MYD35 instead of CALIPSO. The slope in Panel (b) is similar to that in Panel (a). This indicates that the slope is robust. The important point of this analysis is that users of PATMOS-x who would like a more realistic value of cloud fraction uncertainty can simply multiply the values in the PATMOS-x CDR by 1.6.

3.4. Sensitivity of PATMOS-x Cloud Fraction to Prior Cloud Amount Assumptions

One additional source of uncertainty in the PATMOS-x cloud fraction CDR is the prior cloud probability value in the naïve Bayesian formulation. The prior cloud probability values are simply the assumed mean cloud fraction for each of the surface types. The actual values used are given in [2] and are close to the mean of the day and night CALIPSO values in Table 3. In the case of no information, the naïve Bayesian cloud detection will return the prior cloud probability values. In the case where none of the cloud detection tests are definitely clear or cloudy, the prior cloud probability can influence the final posterior cloud probability. In this section, we run the PATMOS-x cloud detection scheme in the AVHRR/3b configuration on MYD02SSH, but we increase the prior cloud probabilities by 10% and these results are labeled PATMOSx_10.

Figure 5 shows the impact on the PATMOS-x cloud fraction by changing the prior cloud probability by 10%. The format and contents are identical to that used in Figure 2 and Figure 3, except for Panel (e). Panel (b) shows the difference in cloud fraction due to the 10% change in prior cloud probability. As was the case in Figure 2 and Figure 3, cloud amount differences are seen in oceanic regions with low cloud fraction (<0.5). The increase in the prior cloud probability by 10% can cause increases of 10% in the cloud fraction in these regions. Other noticeable changes are the increases in cloud amount in the Polar Regions. These regions have higher uncertainties, and would therefore show more sensitivity to the prior cloud probability assumption. In all, a 10% change in the prior cloud probability increases the global cloud amount by 6%. The high values of the anomaly correction in Panel (d) indicate that the changes in cloud fraction are systematic. The bottom two panels show the impact on the linear trends. Panel (e) shows a difference map and Panel (d) shows a scatterplot. The trend difference shows the largest differences occur in the areas of high uncertainty, which is consistent with the pattern of the cloud fraction differences. There are trend differences with a pattern echoing the trend patterns seen in Figure 2 and Figure 3. However, the magnitude of these differences are much less than the absolute values seen in Figure 2 and Figure 3.

While this analysis certainly did reveal a non-negligible sensitivity of the PATMOS-x results to the assumed prior cloud probability, the assumed error in the prior probability of 10% is likely too large. If the results of Table 3 and Table 4 are used, the difference in the global cloud fraction between MYD35, PATMOS-x, and CALIPSO is less than 6%, with the difference being less for many surface types. Therefore, the expected impact due to realistic errors in the mean prior probabilities is less than shown in Figure 5. However, the difference in a particular small region’s mean prior cloud probability to that of its surface type may be larger, and the change in the prior cloud probability across surface type boundaries may introduce artifacts in the spatial distribution of cloud fraction. No obvious artifacts are seen in Figure 5, but this remains an issue to be investigated further. Most importantly, the linear trend shown in Panel (e) shows no significant sensitivity to the prior cloud probability perturbation.

3.5. Sensitivity Based on Spectral Content

As stated above, the PATMOS-x cloud detection algorithm is designed to adjust to the channels available. PATMOS-x can currently process 10 different spectral bands and use them in 16 different tests. The tests involving the AVHRR channels are described in [6], and the additional tests are described in [29]. In this section, we again use the MYD02SSH data as a test-bed to explore the impact of these channels on the PATMOS-x cloud detection performance. The DNB refers to the day-night band, which is visible-near infrared nighttime channel on VIIRS. Note that the DNB tests are excluded since MODIS does not have an analogous channel.

Figure 6 shows comparisons of the global cloud fraction for all seasons from 2003 to 2014. The upper panel in Figure 6 shows the mean PATMOS-x cloud fraction using the MODIS channels in Table 4. This result is not from MYD35 data but is generated by PATMSO-x using MYD02SSH data. In describing these figures, it is worth noting the global pattern of cloud fraction uncertainty shown in Figure 2 and Figure 3. The upper right panel in Figure 6 shows the difference in MODIS and VIIRS cloud fractions. The significant differences occur in the high latitudes, which coincide with the regions of higher cloud detection uncertainty. VIIRS cloud fractions are higher in these regions. A cause of this behavior is the skill provided by the 6.7 µm water vapor channel in detecting clear polar region [30]. Without this channel, the very cold surfaces at high latitudes are sometimes classified as cloud. The other notable difference is the Tibetan Plateau and the Andes.

The lower left panel in Figure 6 shows the difference between the AVHRR in the channel 3b configuration compared to MODIS. The striking feature of this image is the general increase in cloud over the ocean reported by AVHRR/3b relative to MODIS. Comparison to the top left panel shows that this occurs primarily in oceanic regions with cloud fractions <0.5. These regions are also dominated by low cloud. The 6.7 and 1.38 µm tests predominately act to detect high cloud. In the naïve Bayesian logic, each test contributes to the final answer. The presence of tests that are not sensitive to low cloud may reduce the final sensitivity of the cloud mask to low clouds. One can therefore hypothesize that the absence of these high-cloud sensitive tests in the AVHRR/3b mask may increase the sensitivity to low cloud. The switch AVHRR/3b configuration also decreases the cloud detection off of the coast of Antarctica. The other features in the AVHRR/3b comparison are similar to the VIIRS comparison. The lower right panel in Figure 6 shows the difference in global cloud fraction from the AVHRR/3a and AVHRR/3b detection results. This panel shows that the switch of the AVHRR channel configuration has little impact on the cloud detection over most of the globe. AVHRR/3b shows slightly more cloud in some regions (i.e., The Sahel) where the surface characteristics of the 1.6 and 3.75 µm channels are most uncertain and variable.

Table 5. Variation in Proportional Correct (PC) values (%) in comparison to CALIOP for the PATMOS-x cloud detection scheme run on MYD02SSH in 2013 for the sensor/channel configurations shown in Table 4. The last row shows the same analysis applied to NOAA-19/AVHRR/GAC for 2013.

Sensor	Proportional Correct (%) for P/C Using 50/50 Filter
	Global	Ocean	Water	Land	Snow	Arctic	Antarctica	Desert
AVHRR/3a	88	91	91	88	80	80	75	91
AVHRR/3b	89	91	93	89	82	83	77	91
MODIS	87	89	92	86	79	79	81	91
VIIRS	87	90	93	87	76	76	80	93
NOAA-19-AVHRR/3b	86	87	91	84	81	75	73	91

shows the sensitivity of the CALIPSO PC metrics as a function of the different spectral contents for each sensor. The results in Table 5 do confirm the belief that more spectral information improves the global performance of the PATMOS-x cloud fraction. However, the variation in the PC values is less than 3% for non-snow/ice covered surface types. Only in the Antarctic does PATMOS-x run with MODIS channels have the highest PC value. For ocean, MODIS is in fact the worst performing, but the variation in PC is small. This does raise the suspicion that the naïve Bayesian formulation used in PATMOS-x may not be optimally utilizing the additional spectral signatures offered by MODIS. This decrease in performance in the ocean surface type is consistent with the decrease in oceanic cloud amount in Figure 6c. Except for the desert surface type, the NOAA-19/AVHRR/GAC PC values are less than those seen for the MYD02SSH/AVHRR. The differences are biggest for Arctic (8%). While radiometric differences and the angular differences in co-locations exist, it is not clear if these can explain all of these differences.

4. Conclusions

The goals of this paper were to use the CALIPSO and MYD35 products from the NASA EOS A-Train to probe the performance of NOAA Enterprise cloud mask as used in the PATMOS-x Cloud CDR. This information should help users of the PATMOS-x CDR understand the limits of the cloud detection products. The MYD35 was chosen as a reference data set, since it has been established as a climate quality cloud mask and used in many studies. CALIPSO was chosen since it provides the most direct measure of cloud available from a satellite platform. This analysis was done entirely on MODIS observations, which provided the additional capability of exploring the performance of the PATMOS-x cloud mask applied to other sensors.

• For regions where the reported PATMOS-x cloud fraction uncertainty is less than 5%, the PATMOS-x and MYD35 cloud fraction annual anomaly correlations are high and the linear trends over 2003 to 2014 agree well.
• Relative to CALIPSO, PATMOS-x and MYD35 global cloud fractions generally agree within 2%, except for snow-covered land and nighttime Arctic and nighttime Antarctic surface types.
• Comparisons of the reported PATMOS-x cloud fraction uncertainty to direct estimates of error relative to CALIPSO or MYD35 reveal that the PATMOS-x cloud fraction uncertainties are 1.6 times too small but show a linear relationship.
• Being a naïve Bayesian technique, the PATMOS-x cloud fraction is dependent on the assumed surface-type-dependent climatological cloud fraction. Regions with low cloud and small cloud fractions over the ocean showed the most sensitivity to the climatological clouds, as did regions where the cloud fraction uncertainty was high (>10%).
• The cloud fraction trends from the naïve Bayesian PATMOS-x approach agreed well with those from the non-Bayesian MYD35 approach over most regions. This supports the idea that naïve Bayesian cloud detection approaches are suitable for multi-decadal satellite climate research.
• The PATMOS-x AVHRR results show little sensitivity to the spectral switch from AVHRR Ch3a to Ch3b. The PATMOS-x AVHRR cloud fractions are higher than those from MODIS and VIIRS in oceanic regions with low cloud amounts. The PC values also show little impact, except for the Antarctica region where the additional MODIS spectral information adds skill.
• In general, the PATMOS-x MYD02SSH/AVHRR results agree better with CALIPSO than the PATMOS-x AVHRR generated from NOAA-19/AVHRR/GAC data. These differences were small for most surface types. The larger differences occurred over snow and ice-covered surfaces where the radiometric differences between the AVHRR and the MODIS sensors are the largest.

In summary, this paper should serve as a relevant reference for the performance of the cloud detection used to make the AVHRR PATMOS-x CDR hosted by NCEI. All evidence shown points to comparability of the quality of the PATMOS-x cloud detection compared to MYD35 in terms of mean values, inter-annual variability and decadal linear trends. The agreement between AVHRR/PATMOS-x and MODIS over most regions and seasons will hopefully encourage the MODIS community to extend their studies back into the 1980s and 1990s with the PATMOS-x CDR.