Suomi NPP VIIRS Reflective Solar Bands Operational Calibration Reprocessing

Radiometric calibration coefficients for the VIIRS (Visible Infrared Imaging Radiometer Suite) reflective solar bands have been reprocessed from the beginning of the Suomi NPP (National Polar-orbiting Partnership) mission until present. An automated calibration procedure, implemented in the NOAA (National Oceanic and Atmospheric Administration) JPSS (Joint Polar Satellite System) operational data production system, was applied to reprocess onboard solar calibration data and solar diffuser degradation measurements. The latest processing parameters from the operational system were used to include corrected solar vectors, optimized directional dependence of attenuation screens transmittance and solar diffuser reflectance, updated prelaunch calibration coefficients without an offset term, and optimized Robust Holt-Winters filter parameters. The parameters were consistently used to generate a complete set of the radiometric calibration coefficients for the entire duration of the Suomi NPP mission. The reprocessing has demonstrated that the automated calibration procedure can be successfully applied to all solar measurements acquired from the beginning of the mission until the full deployment of the automated procedure in the operational processing system. The reprocessed calibration coefficients can be further used to reprocess VIIRS SDR (Sensor Data Record) and other data products. The reprocessing has also demonstrated how the automated calibration procedure can be used during activation of the VIIRS instruments on the future JPSS satellites.


Introduction
VIIRS (Visible Infrared Imaging Radiometer Suite) instrument has been operating onboard the Suomi NPP (National Polar-orbiting Partnership) satellite since November 2011 [1].Suomi NPP VIIRS is the first instrument in the series that is being deployed on the JPSS (Joint Polar Satellite System) spacecraft by NOAA (National Oceanic and Atmospheric Administration) and NASA (National Aeronautics and Space Administration).Although VIIRS is an integrated system with common telescope and electronics, it can also be seen as a suite of three scanning radiometers with distinct performance characteristics and calibration methods: ‚ TEB (Thermal Emissive Band) measurements (in seven channels) cover the spectral range from 3.5 to 12.5 µm and are calibrated using onboard blackbody and space-view data collected during every scan.
‚ RSB (Reflective Solar Band) measurements consist of 14 channels covering the range from 400 to 2280 nm: RSB calibration uses onboard solar diffuser data collected once per orbit and space-view data from every scan.
‚ DNB (Day/Night Band) covers the broad spectral range of 500-900 nm with high dynamic range measurements that are only partially calibrated by the solar diffuser data: complete DNB calibration requires special Earth observations conducted once per month (an alternative method that uses only the onboard calibrator data has been proposed [2], but its implementation in operational software is not fully tested yet).
Only RSB radiometric calibration is a subject of this paper.While DNB is a reflective band as well, its calibration methodology differs so much that it is beyond scope of the present work.
Since it was found early in the Suomi NPP mission that sensitivity of the VIIRS instrument in several spectral bands decreases with time much faster than expected [3], accuracy of the radiometric calibration has been maintained by updating processing coefficients with the weekly frequency (approximately every 100 orbits).The radiometric calibration procedure together with processing parameters has been improved several times throughout the lifetime of the mission.Initially, only values of the calibration coefficients were updated every week.Later, changes of the coefficients during each week were predicted as well.To further reduce calibration uncertainty between the updates, an automated procedure has been implemented that calculates the RSB radiometric calibration coefficients after every orbit.Rausch et al. [4] developed the automated calibration computer code, called RSBAutoCal, and delivered it to the JPSS program for implementation in the operational processing software.The RSBAutoCal code has enabled us to conduct long-term testing and reprocessing of the RSB calibration.Partial, preliminary results from the reprocessing have been presented previously [5], but this is the first description and discussion of the complete results that include all reprocessed spectral bands and cover the entire duration of the Suomi NPP mission.
This paper describes a long-term testing of the automated calibration procedure based on reprocessing VIIRS solar calibration data starting from the beginning through near four years of the Suomi NPP mission.The presented work had two goals: 1.
Validation: To demonstrate that the automated calibration procedure can be successfully applied to all solar diffuser measurements acquired from the beginning of the mission until the full operational deployment of the automated procedure.

2.
Reprocessing: To generate a complete set of the calibration coefficients while consistently using the latest processing parameters that are optimized based on instrument knowledge gained during the entire mission so far.
The reprocessed radiometric calibration coefficients derived by the automated procedure are compared with those previously applied in the SDR (Sensor Data Record) production at the NOAA operational IDPS (Interface Data Processing Segment) system.
IDPS generates VIIRS data products in near real time with latency currently limited to no more than few hours.The processing coefficients in IDPS are not based on interpolation between past calibration measurements, but rather an extrapolation of the calibration coefficients into the future is always required.Reprocessing that uses this approach may be less accurate than those based solely on the interpolation [6,7], but it provides the SDR products that are most consistent with the latest operational output.

Calibration Equation
Radiometric calibration is applied for each VIIRS RSB Earth observation sample using the following quadratic equation [8], which is simplified here by omitting dependence on spectral band, detector, gain state, HAM (half-angle mirror) side, scan angle, and instrument temperature, but otherwise defines the main terms in the calculations: Radiance L is derived from the Earth measurements after a dark signal (Space view) subtraction (dn is the difference between the Earth and Space views digital counts).The c i coefficients have been obtained from prelaunch tests and are functions of instrument temperature [9].RVS (response versus scan-angle) corrects for dependence on the Earth viewing angle along the scan.RVS has been derived from the prelaunch test data acquired by the sensor manufacturer [10].The F factors provide scaling between the prelaunch testing and on orbit performance.They are calculated from the onboard calibrator data as a ratio of radiance predicted from the solar spectrum to on-orbit measurements of radiance reflected from the solar diffuser: A Space view signal is also subtracted from the solar diffuser measurements (dn SD ).Angle of incidence (AOI) of the solar illumination on the diffuser is calculated for each scan from spacecraft's coordinates and attitude.Transmittance of the solar diffuser attenuation screen (τ SDS ) is tabulated for different directions of the sunlight, as is the bidirectional reflectance of the diffuser (BRDF SD ).Solar spectrum (Φ sun ) is integrated over the relative spectral response (RSR) of each band and is corrected for the changes in the Sun-Earth distance (d).Although screen transmittance and diffuser reflectance do not change much within the spectral response range of each band, they are still included in the integration.
Since VIIRS solar calibration is based on measurements of light reflected from the onboard solar diffuser, knowledge of the diffuser reflectance is fundamental for calibration accuracy.Changes in the solar diffuser's reflectance with time are measured by a separate, onboard instrument: the SDSM (Solar Diffuser Stability Monitor).SDSM measurements are used to derive for each band an H factor: a ratio of the current BRDF SD to its value at the beginning of the mission [8,11].The H factors are used to calculate the BRDF SD values used in Equation (2).In this way, the H factors are directly used to scale the calibration coefficients for the visible and near-infrared bands (M1-M7 and I1-I2).Degradation of the solar diffuser reflectance is considered negligible for the short-wave infrared bands (I3 and M8-M11) [12].
While the solar calibration is conducted once per orbit, stability of the diffuser is monitored less frequently.Only during the initial on-orbit checkout were the SDSM measurements conducted on every orbit.Later, SDSM was operated once per day.The frequency has been reduced to three times per week since May 2014.Linear extrapolation is used to calculate the H factor values for each orbit from the less frequent SDSM measurements.

Data Processing
VIIRS SDR products are generated by the IDPS system that implements the above calibration equation.The IDPS processing codes are also included in the ADL (Algorithm Development Library) software that can be executed outside of the IDPS system to support additional development of the algorithms (https://jpss.ssec.wisc.edu/).To conduct the long-term testing of the automated calibration procedure, the VIIRS RSB radiometric calibration coefficients were calculated using the ADL software that includes version Mx8.10 of the IDPS code. Figure 1 shows a flowchart of the conducted ADL processing.The reprocessing was started with the VIIRS RDRs (Raw Data Records) to mitigate earlier processing errors such as an incorrect solar vector rotation or omission of some SDSM data.Only granules that contain the solar calibration data were processed for each orbit.The datasets were obtained from the NOAA CLASS (Comprehensive Large Array-data Stewardship System) archive (http://www.class.noaa.gov/)or from the JPSS GRAVITE (Government Resources for Algorithm Verification, Independent Testing and Evaluation) system (https://gravite.jpss.noaa.gov/).When multiple RDR sets existed in the archives for the same data granule, only the most complete one or the most recent one (when equal in size) was selected for processing.OBC IP (Onboard Calibrator Intermediate Product) files, which have been generated by IDPS since 21 November 2014 (orbit 15,893), did not require reprocessing and were directly obtained from the above data archives.In IDPS, the program that generates the SDR and OBC IP files (ProSdrViirsController) is executed in parallel with the automated calibration program (RSBAutoCal).After a new OBC IP file is created by ProSdrViirsController, RSBAutoCal is invoked to process that file, together with two OBC IP files from the preceding and subsequent granules.When a new solar calibration event is detected in the OBC IP data, RSBAutoCal generates an updated VIIRS-RSBAUTOCAL-HISTORY-AUX file.Before producing SDR files in the automated calibration mode, ProSdrViirsController selects the VIIRS-RSBAUTOCAL-HISTORY-AUX file that contains the F factors for the orbit being processed.In our calibration reprocessing with ADL, these two processes were separated.All OBC IP files with solar calibration data were produced first using ProSdrViirsController, and then they were processed using RSBAutoCal to generate the VIIRS-RSBAUTOCAL-HISTORY-AUX files.To apply the reprocessed calibration coefficients in SDR production using ADL, the VIIRS-RSBAUTOCAL-HISTORY-AUX files should be provided, together with the RDR files required for a given SDR granule, as input to ProSdrViirsController executed in the automated calibration mode.
The calculations were based on the latest processing parameters from the operational system that incorporate the following major changes:

‚
Corrected solar vectors: After a processing code update, orientation of the Sun-satellite vectors has changed by as much as 0.2 ˝ [13].
‚ Improved solar attenuation screens transmittance and solar diffuser bidirectional reflectance tables: angular dependence was optimized based on reanalysis of the on-orbit yaw maneuver and the routine onboard calibrator measurements acquired during the first three years of the Suomi NPP mission [14].
‚ Updated prelaunch calibration coefficients: To improve consistency between bands with similar spectral response, offset terms were set to zero while only the linear and quadratic terms remained [9].

‚
Optimized RHW (Robust Holt-Winters) filter parameters: Smoothing of the calibration coefficients time series was improved by damping oscillations while maintaining sensitivity to trend changes [11].
The same set of the processing parameters was consistently used in the entire reprocessing of the VIIRS RSB calibration coefficients.Table 1 list versions of the LUT (Lookup Table ) files applied during the calibration reprocessing.Effectivity start dates of these LUT files have been often extended to the beginning of the Suomi NPP mission.Only the spectral response LUT was changed during the reprocessed time period (as indicated in Table 1 by the effectivity dates), similarly to the changes that occurred in the operational SDR production.The calibration history file, VIIRS-RSBAUTOCAL-HISTORY-AUX, which was used to initialize the calculations, did set the H and F factor values to one and their trends to zero at time of the Suomi NPP launch.

Solar Diffuser Degradation
Two graphs included in Figure 2 show time series of the H factors calculated during the current reprocessing with and without applying the RHW filter.While only the filtered (smoothed and extrapolated) H values are used in the reprocessed F factor calculations, the unfiltered values provide additional insights into uncertainties of the SDSM measurements.The H factor time series show that the measured solar diffuser reflectance appears to decrease exponentially from the beginning of the mission until February 2014.Since then, the pace of the diffuser degradation changes from time to time.Although the irregular trend changes in Figure 2 seem more pronounced for the shortest wavelength bands M1-M3, they are in fact similar for all reflective bands when compared with the degradation range for each band.The cause of these unexpected trend changes is still unknown, and it is not clear if these are real changes in the diffuser reflectance or artifacts in the SDSM data, either from the measurements themselves or from their analysis.Nevertheless, the H factor trend changes emphasize the need for using an automated calibration procedure that promptly responds to the observed variability.
Application of the RHW filter removes some artifacts from the SDSM data such as the incompletely corrected measurements acquired during the yaw maneuver in February 2012.Effectiveness of the filter occasionally appears less certain (Figure 3).RHW filter parameters have been optimized in a tradeoff between smoothness of the time series and sensitivity to temporal changes, but the choice of the RHW smoothing parameters is only briefly discussed in the literature [15].A recent analysis has shown that the smoothing parameters for the measured values should be at least several times larger than the smoothing parameters for the measurement's trends [16].Such RHW parameters have been used to filter both the H factors and the F factors in this study and in the operational processing of the VIIRS data [11].

Radiometric Gain Changes
Figure 4 shows a comparison of the reprocessed F factors with those previously used in the operational SDR production for one of the spectral bands that is not significantly affected by the VIIRS telescope throughput degradation: band M3 [17].The F factors calculated by the automated procedure not only extend to the beginning of the Earth observations from VIIRS, but also appear more stable throughout the duration of the mission.Variability visible in the time series of the reprocessed F factors is on the order of only 1% and reflects uncertainty of the calibration.The differences between the reprocessed coefficients and the original operational ones do not generally exceed ˘2%, in agreement with the radiometric calibration uncertainty requirements.During the first year of the mission, the changes in the operational F factors shown on the graph originate from improvements in characterization of the VIIRS radiometric response, with the most significant differences before May 2012.Since February 2014, the operational F factor variability for band M3 is mostly affected by the observed H factor trend changes.The automated calibration procedure appears to respond better to these changes.Small seasonal variability that can be seen in the reprocessed F factors coincides with changes in direction of solar illumination that make SDSM measurements more difficult during the months of October through December each year.Future improvements of the VIIRS SDR processing parameters are expected to further reduce this variability.
Figure 5 shows the F factor comparison for band M7 that is strongly affected by the rotating telescope mirror degradation [3].Even with the F factor changes dominated by the throughput decrease, one can notice that the reprocessed calibration coefficients are free of the spurious variability that affected the earlier operational F factors (changes in December 2012 are only due to an investigation of the degradation anomaly).Several phases of the operational calibration improvements can be detected in the differences between the operational and reprocessed F factors.The largest differences existed until October 2012 when angular dependence of the attenuation screen transmittance and the solar diffuser reflectance were determined more accurately from the Suomi NPP orbital yaw maneuver [18].The next change occurred in April 2014 when the offset terms in the calibration equation (the c 0 coefficients) were set to zero and the prelaunch c 1 and c 2 coefficients were recalculated from the prelaunch test data [9].The differences diminished in 2015 when calculations of the operational F factors became similar to the automated procedure with the RHW filter [11].This illustrates the tradeoff between the operational procedure and the automated one: by applying the automated calibration, at the cost of small variability observed in 2015, larger biases such as those from the first half of 2014 can be avoided.
Although some improvements of the calibration trending can be expected just from the consistent use of the same processing parameters for the whole mission period, Figures 4 and 5 show that after the reprocessing the solar diffuser degradation is better corrected for band M3, and the telescope degradation that is diminishing with time is better corrected after every orbit for band M7.Similar improvements in the reprocessed F factors can be seen for the other visible and near-infrared bands as well as for the short-wave infrared bands.The respective graphs are shown below and in the figures included in the Appendix.Since the F factors provide scaling between the effective radiometric gains determined from prelaunch testing and the gains measured on orbit during solar calibration events, values of the F factors can be either smaller (as for bands M2-M4) or larger than one (as for M1) depending on whether the prelaunch c coefficients overestimated or underestimated the actual radiometric response on orbit.The gains may have changed during launch or as a result of differences between the conditions during the thermal vacuum testing on the ground and in the real Space environment on orbit.The gain changes can also originate in instrument degradation occurring on orbit, such as in the case of bands M5-M11.The apparent radiometric response changes may also indicate uncertainties of the prelaunch testing created by such factors as the differences between irradiance spectra of the laboratory calibration lamps and the Sun.The SWIR (short-wave infrared) bands (I3 and M8-M11) also show an F factor discontinuity in March 2012.This reflects an actual gain change that occurred after the Suomi NPP spacecraft was temporarily in safe mode during which the spacecraft was oriented toward the Sun.After this gain change, the reprocessed F factors are updated sooner by the automated calibration procedure than they were during the original operational processing.This creates the spikes in the differences between the reprocessed and operational F factors for the SWIR bands, and it also shows once more that the automated calibration procedure improves upon the operational processing.
A downward spike in the reprocessed SWIR-band F factors occurs in February 2012 during an orbital pitch maneuver of the Suomi NPP spacecraft.During the pitch maneuver, SWIR detectors temperature briefly increased above the nominal level, and the SWIR bands gains changed.Solar calibration data acquired during the pitch maneuver were excluded from the operational F factor calculations, because Earth observations during the maneuver were mostly invalid, but these solar data were processed by the automated procedure and are included on the presented graphs for completeness.
During the early on-orbit checkout of the Suomi NPP satellite, the VIIRS SWIR bands were activated later than the other reflective bands: the SWIR detectors were cooled down to an operational temperature after the RSB data acquisition has already started.The SWIR measurements acquired before the detectors reached the nominal temperature were dominated by random noise and therefore were processed by initially applying the automated calibration procedure without filtering.The RHW filter was applied only after the SWIR detector temperature has stabilized.Residual variability can still be seen at the beginning of the F factor time series for the SWIR bands, but magnitude of the transient changes is small and they diminish quickly.This approach demonstrates how the automated calibration procedure can be used during activation of the VIIRS instruments on the future JPSS satellites.

Conclusions
Reprocessing of the VIIRS RSB calibration coefficients using a new automated procedure has increased confidence in this approach by demonstrating that it can be applied to all solar calibration data previously acquired during the entire Suomi NPP mission.With optimized RHW filter parameters, the automated calculations of the F factors remained stable for the entire reprocessed time period.While the automated procedure improved upon calibration used in the operational processing of the VIIRS data, the differences between the reprocessed and operational F factors in general have not exceeded the VIIRS SDR radiometric calibration uncertainty and remain in the 1%-2% range, except for bands M1-M3 before May 2012.The improved calibration coefficients can be further used to reprocess VIIRS SDR and other data products.Since the SDR radiance and reflectance products are directly proportional to the F factors (Equation 1), the relative improvements between the original, operational products and the reprocessed ones will be the same as the differences between the F factors.Because the F factor differences appear cross-correlated for some band pairs, the differences for data products derived from SDR using reflective band ratios may be smaller.Such will be the case for NDVI (Normalized Difference Vegetation Index) EDR (Environmental Data Record) that is produced from bands I1 and I2 [19].The improved calibration coefficients may especially benefit direct broadcast ground stations when reprocessing of previously acquired VIIRS datasets is needed.The reprocessed calibration history files are available though the VIIRS SDR home page at the NOAA National Calibration Center website (http://ncc.nesdis.noaa.gov/VIIRS/).

Figure 2 .
Figure 2. Degradation of the Suomi NPP VIIRS solar diffuser reflectance (the H factor) for bands M1-M7 as measured by the SDSM and analyzed by the automated calibration procedure with (a) and without (b) the time series filtering.

Figure 3 .
Figure 3.Comparison of the band M1 H factor values derived from the actual SDSM measurements without filtering and the values smoothed and interpolated with the RHW filter.

Figure 4 .
Figure 4. (a) VIIRS band M3 (488 nm, dual gain) radiometric calibration scaling coefficients (the F factors) derived by the automated calibration procedure; (b) the F factors used in the operational production of the VIIRS SDR; and (c) relative difference between the operational F factors and the reprocessed ones (used here as a baseline).The half-angle mirror sides and the gain states are indicated by different symbols (dots and circles), while the F factors are shown for each detector with a different color from the red-yellow (for high/single gain) and blue-green (for low gain) palettes.

Figure 5 .
Figure 5. (a) VIIRS band M7 (865 nm, dual gain) radiometric calibration scaling coefficients (the F factors) derived by the automated calibration procedure; (b) the F factors used in the operational production of the VIIRS SDR; and (c) relative difference between the operational F factors and the reprocessed ones (used here as a baseline).Symbol legend is the same as in Figure 4.

Figure 6 ,
Figure 6, for band M6, shows F factor values that are distinctly different for the two HAM sides.The difference compensates for the reciprocal difference that exists in the prelaunch c coefficients for the two HAM sides: the radiometric gains for band M6 are very similar for both HAM sides.This exemplifies that the F factors are a good estimate of the calibration coefficients (or the inverse of the F factors are a good estimate of the sensor radiometric gains) only when the c coefficients are virtually the same.If values of the c coefficients are different or have significantly changed, the product of the F factor and the c 1 coefficient is a better estimate, but it is still an approximation because of the non-linear form of the calibration equation.This is also illustrated in Figure 7 for band I3, which had the largest calibration change after the offset terms in the calibration equation were set to zero.The operational F factors have abruptly changed in April 2014 because the c coefficients were changed.The reprocessed F factors do not display such a change because the same set of the c coefficients was used throughout the entire reprocessing.

Figure 6 .
Figure 6.(a) VIIRS band M6 (746 nm, single gain) radiometric calibration scaling coefficients (the F factors) derived by the automated calibration procedure; (b) the F factors used in the operational production of the VIIRS SDR; and (c) relative difference between the operational F factors and the reprocessed ones (used here as a baseline).Symbol legend is the same as in Figure 4.

Figure 7 .
Figure 7. (a) VIIRS band I3 (1610 nm, single gain) radiometric calibration scaling coefficients (the F factors) derived by the automated calibration procedure; (b) the F factors used in the operational production of the VIIRS SDR; and (c) relative difference between the operational F factors and the reprocessed ones (used here as a baseline).Symbol legend is the same as in Figure 4.

Figure A3 .
Figure A3.(a) VIIRS band M4 (555 nm, dual gain) radiometric calibration scaling coefficients (the F factors) derived by the automated calibration procedure; (b) the F factors used in the operational production of the VIIRS SDR; and (c) relative difference between the operational F factors and the reprocessed ones (used here as a baseline).Symbol legend is the same as in Figure 4.

Figure A4 .
Figure A4.(a) VIIRS band M5 (672 nm, dual gain) radiometric calibration scaling coefficients (the F factors) derived by the automated calibration procedure; (b) the F factors used in the operational production of the VIIRS SDR; and (c) relative difference between the operational F factors and the reprocessed ones (used here as a baseline).Symbol legend is the same as in Figure 4.

Figure A5 .
Figure A5.(a) VIIRS band M8 (1240 nm, single gain) radiometric calibration scaling coefficients (the F factors) derived by the automated calibration procedure; (b) the F factors used in the operational production of the VIIRS SDR; and (c) relative difference between the operational F factors and the reprocessed ones (used here as a baseline).Symbol legend is the same as in Figure 4.

Figure A6 .
Figure A6.(a) VIIRS band M9 (1378 nm, single gain) radiometric calibration scaling coefficients (the F factors) derived by the automated calibration procedure; (b) the F factors used in the operational production of the VIIRS SDR; and (c) relative difference between the operational F factors and the reprocessed ones (used here as a baseline).Symbol legend is the same as in Figure 4.

Figure A7 .
Figure A7.(a) VIIRS band M10 (1610 nm, single gain) radiometric calibration scaling coefficients (the F factors) derived by the automated calibration procedure; (b) the F factors used in the operational production of the VIIRS SDR; and (c) relative difference between the operational F factors and the reprocessed ones (used here as a baseline).Symbol legend is the same as in Figure 4.

Figure A8 .
Figure A8.(a) VIIRS band M11 (2250 nm, single gain) radiometric calibration scaling coefficients (the F factors) derived by the automated calibration procedure; (b) the F factors used in the operational production of the VIIRS SDR; and (c) relative difference between the operational F factors and the reprocessed ones (used here as a baseline).Symbol legend is the same as in Figure 4.

Figure A9 .
Figure A9.(a) VIIRS band I1 (640 nm, single gain) radiometric calibration scaling coefficients (the F factors) derived by the automated calibration procedure; (b) the F factors used in the operational production of the VIIRS SDR; and (c) relative difference between the operational F factors and the reprocessed ones (used here as a baseline).Symbol legend is the same as in Figure 4.

Figure A10 .
Figure A10.(a) VIIRS band I2 (865 nm, single gain) radiometric calibration scaling coefficients (the F factors) derived by the automated calibration procedure; (b) the F factors used in the operational production of the VIIRS SDR; and (c) relative difference between the operational F factors and the reprocessed ones (used here as a baseline).Symbol legend is the same as in Figure 4.

Table 1 .
Lookup tables applied in the VIIRS RSB calibration reprocessing.