An Overview of the Joint Polar Satellite System (jpss) Science Data Product Calibration and Validation

The Joint Polar Satellite System (JPSS) will launch its first JPSS-1 satellite in early 2017. The JPSS-1 and follow-on satellites will carry aboard an array of instruments including the Visible Infrared Imaging Radiometer Suite (VIIRS), the Cross-track Infrared Sounder (CrIS), the Advanced Technology Microwave Sounder (ATMS), and the Ozone Mapping and Profiler Suite (OMPS). These instruments are similar to the instruments currently operating on the Suomi National Polar-orbiting Partnership (S-NPP) satellite. In preparation for the JPSS-1 launch, the JPSS program at the Center for Satellite Applications and Research (JSTAR) Calibration/Validation (Cal/Val) teams, have laid out the Cal/Val plans to oversee JPSS-1 science products' algorithm development efforts, verification and characterization of these algorithms during the pre-launch period, calibration and validation of the products during post-launch, and long-term science maintenance (LTSM). In addition, the team has developed the necessary schedules, deliverables and infrastructure for routing JPSS-1 science product algorithms for operational implementation. This paper presents an overview of these efforts. In addition, this paper will provide insight into the processes of both adapting S-NPP science products for JPSS-1 and performing upgrades for enterprise solutions, and will discuss Cal/Val processes and quality assurance procedures.


Introduction
The Joint Polar Satellite System (JPSS) is the National Oceanic and Atmospheric Administration's (NOAA) operational program that provides continuity of global environmental data from multiple polar-orbiting satellites for operational remote sensing of weather, climate and other environmental applications.The Suomi National Polar-orbiting Partnership (S-NPP) satellite launched in October 2011 was the first satellite designed to bridge into the future JPSS constellation.The S-NPP satellite carried aboard the following five instruments: Visible Infrared Imaging Radiometer Suite (VIIRS), Cross-track Infrared Sounder (CrIS), Advanced Technology Microwave Sounder (ATMS), Ozone Mapping and Profiler Suite (OMPS) and Clouds and the Earth's Radiant Energy System (CERES).Details of these instruments, channel characteristics, calibration, and the science data products derivable from these instruments are discussed in a compendium of scientific papers published as a special issue in the Journal of Geophysical Research [1].The JPSS-1 (planned for launch in early 2017) and follow-on satellites will carry aboard similar instruments that are currently operating on-board the S-NPP with additional improvements [2].
The NOAA Center for Satellite Applications and Research (STAR) leads the efforts to develop, test, validate and refine the science algorithms to process S-NPP and JPSS instruments data from VIIRS, CrIS, ATMS, and OMPS into user required data products such as Temperature Data Records (TDRs), Sensor Data Records (SDRs), and Environmental Data Records (EDRs) (here after referred to as xDRs collectively).Raw Data Records (RDRs) refer to the raw data generated by sensors on the satellites.Science algorithms that perform calibration and geo-location are applied on the RDRs to produce SDRs and TDRs.EDRs are geophysical parameters that are derived by applying retrieval algorithms on the SDRs.EDR products provide global measurements of such quantities as sea surface temperature, ocean color, ozone and trace gases, aerosols, clouds, temperature and moisture profiles, wind speeds, land surface properties, and snow and ice cover, etc.The data products produced from the S-NPP/JPSS satellites provide continuity of critical observations for accurate weather forecasting, reliable severe storm outlooks, and ocean ecosystem dynamics [3].Examples of ocean applications include detecting and predicting harmful algal blooms [4], tracking runoff plumes and sediment [5], monitoring water quality [6], and hazards detection (e.g., oil spill detection, [7]).
JPSS provides data products to the primary NOAA User Community, which includes all centers of NOAA and worldwide weather agencies.The National Weather Service (NWS) and the European Center for Medium Range Weather Forecasting (ECMWF) assimilate JPSS products into their Numerical Weather Prediction (NWP) models.JPSS also partners with the users from National Aeronautics and Space Administration (NASA), Department of Commerce (DOC), Department of Agriculture, and Environmental Protection Agency (EPA) to utilize JPSS data products to support their research and operations.Direct readout users receive live S-NPP/JPSS data using direct downlink capabilities and the Community Satellite Processing Package (CSPP) allows creation of many SDR and imagery products in realtime [8].S-NPP/JPSS data products are also accessible to public users worldwide through the NOAA Comprehensive Large Array-data Stewardship System (CLASS, [9]).

Science Data Products and Key Performance Parameters (KPPs)
Figure 1 provides a list of xDR products that are operationally produced from the current S-NPP suite of instruments.The S-NPP SDRs and associated EDR products (Figure 1) generated from the VIIRS, CrIS, ATMS, and OMPS instruments, and the Cal/Val efforts validating the products, are published in many journal articles [1,[10][11][12][13][14].The VIIRS SDRs are used to produce more than 20 EDR products (e.g., imagery, aerosol optical thickness (AOT), sea surface temperature, and many others as shown in Figure 1) and provide critical data for environmental assessments, forecasts and warnings [10].The CrIS/ATMS SDR products are assimilated into NWP models worldwide and have been shown to provide considerable impact in reducing medium range forecast errors [15].A variety of CrIS/ATMS EDR sounding products derived from the NOAA Unique CrIS/ATMS Processing System (NUCAPS) are currently ingested into the Advanced Weather Interactive Processing System (AWIPS-2, [16]) for their utility by many Weather Forecasting Offices (WFOs) nationwide for analyzing atmospheric instabilities, potential outbreaks of severe weather, and now-casting applications [17].The OMPS Nadir Mapper and Nadir Profiler measurements are used to create global ozone maps and UV index forecasts [18].
The JPSS-1 and subsequent series of satellites will continue to produce these base-line products, along with upgrades planned as part of science improvements, and new and additional products derivable as a direct result of those improvements.Based on the utility of these science data products by customers and the user community, these products are prioritized as Key Performance Parameters (KPPs), and priority 2, 3, and 4 products.The requirements and priorities of the JPSS products are defined in the Level 1 Requirement Document (L1RD, [19]) and L1RD Supplement (L1RDS, [20]).The designations of the xDR priorities also dictate the priorities of the Cal/Val efforts towards evaluation of threshold or objective attributes.The S-NPP xDR product algorithms are in operations through either the Interface Data Processing Segment (IDPS) implemented by Raytheon or through S-NPP Data Exploration (NDE) at Environmental Satellite Processing Center (ESPC).Operationalization of the JPSS-1 xDR products (either in IDPS or through NDE/ESPC) will follow the S-NPP operational protocols and time-to-time directives from the NOAA JPSS Program office (NJO) on the implementation strategies for the S-NPP and JPSS data products.Most of the S-NPP products are operationally available to all of the user agencies worldwide through the NOAA CLASS.In preparation for the JPSS-1 launch, the JPSS STAR (JSTAR) Calibration/Validation (Cal/Val) teams have laid out the Cal/Val plans to oversee JPSS-1 science products algorithm development efforts, verification and characterization of these algorithms.The team also has developed schedules for deliverables and infrastructure for routing JPSS-1 xDR algorithm(s), improvements and updates in compliance with the Algorithm Change Process (ACP, [21]).
Operationalization of the JPSS-1 xDR products (either in IDPS or through NDE/ESPC) will follow the S-NPP operational protocols and time-to-time directives from the NOAA JPSS Program office (NJO) on the implementation strategies for the S-NPP and JPSS data products.Most of the S-NPP products are operationally available to all of the user agencies worldwide through the NOAA CLASS.In preparation for the JPSS-1 launch, the JPSS STAR (JSTAR) Calibration/Validation (Cal/Val) teams have laid out the Cal/Val plans to oversee JPSS-1 science products algorithm development efforts, verification and characterization of these algorithms.The team also has developed schedules for deliverables and infrastructure for routing JPSS-1 xDR algorithm(s), improvements and updates in compliance with the Algorithm Change Process (ACP, [21]).The JSTAR Cal/Val teams consist of four SDR teams to generate high quality ATMS, CrIS, VIIRS and OMPS SDRs.In addition, twelve EDR teams cover all of the JPSS EDR products related to the Atmosphere, Land and Oceans.All of these SDR/EDR teams coordinate with the JSTAR program management, other government agencies and industry partners to define various Cal/Val activities and execute them, thereby meeting schedules and deliverables.The SDR/EDR teams lay out different tasks to be accomplished during the pre-launch characterization, post-launch validation, and subsequent reactive maintenance and sustainment activities to realize and maintain high quality SDR and EDR products.Details of various Cal/Val tasks for each xDR product, as well as the roles and responsibilities of team members, are discussed in the respective Cal/Val plan documents [22].The JSTAR Cal/Val teams consist of four SDR teams to generate high quality ATMS, CrIS, VIIRS and OMPS SDRs.In addition, twelve EDR teams cover all of the JPSS EDR products related to the Atmosphere, Land and Oceans.All of these SDR/EDR teams coordinate with the JSTAR program management, other government agencies and industry partners to define various Cal/Val activities and execute them, thereby meeting schedules and deliverables.The SDR/EDR teams lay out different tasks to be accomplished during the pre-launch characterization, post-launch validation, and subsequent reactive maintenance and sustainment activities to realize and maintain high quality SDR and EDR products.Details of various Cal/Val tasks for each xDR product, as well as the roles and responsibilities of team members, are discussed in the respective Cal/Val plan documents [22].The Cal/Val teams envision improvements to the existing products, investigate feasibility of new products development, and test algorithm updates accordingly in their offline systems.The teams coordinate with the STAR Algorithm Integration Team (AIT) in delivering Algorithm Change Packages (ACPs) for IDPS [21] and NDE [23] operations.The JSTAR management, in association with the Algorithm Management Program (AMP) team members, and the JPSS Ground Segment engineers, ensures that the Cal/Val plans, processing and analysis approach are robust for realization and in maintaining data throughput and integrity.2. During pre-launch, the emphasis is on characterizing the product performance utilizing the data sets available for the specific instrument (e.g., Thermal Vacuum Chamber (TVAC) test data sets; proxy or synthetic data sets generated from previous missions) and leveraging on the expertise gained from the previous missions.The Cal/Val teams also develop tools during the pre-launch to exercise post-launch activities.During the post-launch EOC phase, the emphasis is on sensor characterization and calibration, and quick-look analysis of the products derived from real satellite observations.This is followed by post-launch ICV with emphasis on characterizing the product performance (Accuracy, Precision and Uncertainty, APU) for different scenarios defined in the L1RDS using a variety of correlative data sets.The Cal/Val teams envision improvements to the existing products, investigate feasibility of new products development, and test algorithm updates accordingly in their offline systems.The teams coordinate with the STAR Algorithm Integration Team (AIT) in delivering Algorithm Change Packages (ACPs) for IDPS [21] and NDE [23] operations.The JSTAR management, in association with the Algorithm Management Program (AMP) team members, and the JPSS Ground Segment engineers, ensures that the Cal/Val plans, processing and analysis approach are robust for realization and in maintaining data throughput and integrity.

Pre-Launch to Post-Launch and Cal/Val Maturity
Declaring SDR/EDR product maturity is the result of a specific review of artifacts that document that the products meet a series of criteria defined for each maturity stage.The Cal/Val plan documents describe in detail the methodologies on how xDR products specified in the JPSS Program L1RD and L1RDS will be evaluated and validated for each of the maturity status.The planned Cal/Val activities during the (a) Pre-Launch; (b) Early Orbit Check-out (EOC); (c) Intensive Cal/Val (ICV), and (d) Long-term Monitoring (LTM) phases are also included in the Cal/Val plans.This is schematically depicted in Figure 2.During pre-launch, the emphasis is on characterizing the product performance utilizing the data sets available for the specific instrument (e.g., Thermal Vacuum Chamber (TVAC) test data sets; proxy or synthetic data sets generated from previous missions) and leveraging on the expertise gained from the previous missions.The Cal/Val teams also develop tools during the pre-launch to exercise post-launch activities.During the post-launch EOC phase, the emphasis is on sensor characterization and calibration, and quick-look analysis of the products derived from real satellite observations.This is followed by post-launch ICV with emphasis on characterizing the product performance (Accuracy, Precision and Uncertainty, APU) for different scenarios defined in the L1RDS using a variety of correlative data sets.The Cal/Val teams identify any algorithm issues and make updates necessary to meet the required criteria as defined in product maturity definitions as shown in Table 1.The critical path during the post-launch Cal/Val phase is to ensure that the operational xDRs meet the beta and provisional maturity requirements and are ready for validated maturity, and to transition the products towards LTM.The JPSS Cal/Val teams have followed this process for S-NPP and currently most of the S-NPP products have reached validated maturity and are rolling towards the LTM phase.For JPSS-1, all of the Cal/Val teams have described in detail their Cal/Val plans and schedules starting from pre-launch to post-launch in their respective Cal/Val plan documents [22].

JPSS-1Launch
We Are Here for JPSS-1 Beta Provisional Validated We Are Here for SNPP The Cal/Val teams identify any algorithm issues and make updates necessary to meet the required criteria as defined in product maturity definitions as shown in Table 1.The critical path during the post-launch Cal/Val phase is to ensure that the operational xDRs meet the beta and provisional maturity requirements and are ready for validated maturity, and to transition the products towards LTM.The JPSS Cal/Val teams have followed this process for S-NPP and currently most of the S-NPP products have reached validated maturity and are rolling towards the LTM phase.For JPSS-1, all of the Cal/Val teams have described in detail their Cal/Val plans and schedules starting from pre-launch to post-launch in their respective Cal/Val plan documents [22].

‚ Beta
Product is minimally validated, and may still contain significant identified and unidentified errors.Information/data from validation efforts can be used to make initial qualitative or very limited quantitative assessments regarding product fitness-for-purpose.Documentation of product performance and identified product performance anomalies, including recommended remediation strategies, exists.

‚ Provisional
Product performance has been demonstrated through analysis of a large, but still limited (i.e., not necessarily globally or seasonally representative) number of independent measurements obtained from selected locations, time periods, or field campaign efforts.Product analyses are sufficient for qualitative, and limited quantitative, determination of product fitness-for-purpose.Documentation of product performance, testing involving product fixes, identified product performance anomalies, including recommended remediation strategies, exists.Product is recommended for potential operational use (user decision) and in scientific publications after consulting product status documents.

‚ Validated
Product performance has been demonstrated over a large and wide range of representative conditions (i.e., global, seasonal).
Comprehensive documentation of product performance exists that includes all known product anomalies and their recommended remediation strategies for a full range of retrieval conditions and severity level.Product analyses are sufficient for full qualitative and quantitative determination of product fitness-for-purpose.
Product is ready for operational use based on documented validation findings and user feedback.Product validation, quality assurance, and algorithm stewardship continue through the lifetime of the instrument.

Cal/Val Review Process
The JSTAR team facilitates the Cal/Val maturity review process for all of the xDR products.Each product's Cal/Val maturity status is determined by evaluating the algorithm performance with the truth data sets and comparing the performance with the requirements.The review panel is comprised of the JPSS Program and Project Scientists, NOAA customers (National Weather Service, NWS; National Ocean Service, NOS; National Marine Fisheries Services, NMFS; Office of Oceanic and Atmospheric Research, OAR) and external users, Low-earth Orbiting Working Group (LORWG) advisors, product development managers (NDE and IDPS), and the JSTAR program manager.Following the Cal/Val maturity science reviews presented by the Cal/Val teams, the review board assesses the product performance and makes an overall assessment of each algorithm and approves the maturity status according to the criteria for Beta, Provisional, and Validated Maturity (Table 1).Based on the assessment, the review panel may also recommend further actions to the respective Cal/Val teams to achieve the desired product maturity status.

Quality Assurance
To ensure efficient research-to-operations transitions for science product algorithm updates, the JSTAR Program has implemented Quality Assurance (QA) procedures.The QA procedures include coordination activities between the algorithm teams on Cal/Val activities, supporting the algorithm change process (ACP), identifying potential shortfalls (risks) and mitigation, Configuration Management (CM) for documentation, schedules and milestones, and Earned Value Management (EVM).The J-STAR QA also facilitates technical reviews and coordination among STAR Cal/Val teams and the AIT to bring consistency to algorithm development and the delivery processes for operations.As part of the quality assurance and users engagement, JSTAR also developed a website [24] that provides detailed information on the JPSS instruments, science data products and documents, product maturity status, long-term validation and science monitoring of SDR/EDR products.

S-NPP Science Data Products Cal/Val Maturity Status
The Algorithm Theoretical Basis Documents (ATBDs) provide detailed descriptions of the science algorithms, and the Cal/Val documents associated with each product provide details of the Cal/Val efforts implemented during various phases of the S-NPP Cal/Val process.All of these documents are accessible from the JSTAR document website [22].Most of the S-NPP data products have been validated with truth data sets and the product performances have been verified with the L1RDS requirements for progression from Beta, Provisional, and Validated Maturity stages at the times specified in Table 2.These SDR/EDR products and the product utility "readme" files are available through the NOAA CLASS and the JSTAR website [25] to global users for end-user applications.The review panel assessment reports on the Cal/Val maturity status presentations on each product are available on the STAR website [24] to assess the utility of the data product for various applications.The JPSS STAR Annual Science Team Meetings [26,27] have numerous presentations on the utility of the data products for many end-user applications.As of this publication, all of the SDRs and most of the EDRs had been declared to have reached the validated maturity.CrIS and ATMS radiances have been operationally assimilated in the National Weather Prediction (NWP) Centers and provided continuity of essential atmospheric sounding information for weather forecasting.Figure 3 demonstrated that CrIS and ATMS are among the top two contributors for reducing forecast errors.Another example of EDR impacts on weather forecasting is the VIIRS Imagery EDR, one of the Key Performance Parameter (KPP) products [14], which reached the validated maturity in January 2014.Both Non-Near Constant Contrast (NCC) and NCC Imagery attained this level simultaneously.Non-NCC imagery has been well validated and ahead of NCC Imagery for months/years.NCC Imagery recently made significant advances in full coverage and stray light suppression (an SDR issue).The VIIRS imagery products have been widely used and proved to be critical for weather forecasts and environmental monitoring applications.Figure 4 is a VIIRS RGB composite imagery of channels M4, M7, and M11, showing the eruption of Bárðarbunga in Iceland, as it appeared at 13:42   Another example of EDR impacts on weather forecasting is the VIIRS Imagery EDR, one of the Key Performance Parameter (KPP) products [14], which reached the validated maturity in January 2014.Both Non-Near Constant Contrast (NCC) and NCC Imagery attained this level simultaneously.Non-NCC imagery has been well validated and ahead of NCC Imagery for months/years.NCC Imagery recently made significant advances in full coverage and stray light suppression (an SDR issue).The VIIRS imagery products have been widely used and proved to be critical for weather forecasts and environmental monitoring applications.Figure 4 is a VIIRS RGB composite imagery of channels M4, M7, and M11, showing the eruption of Bárðarbunga in Iceland, as it appeared at 13:42 Another example of EDR impacts on weather forecasting is the VIIRS Imagery EDR, one of the Key Performance Parameter (KPP) products [14], which reached the validated maturity in January 2014.Both Non-Near Constant Contrast (NCC) and NCC Imagery attained this level simultaneously.Non-NCC imagery has been well validated and ahead of NCC Imagery for months/years.NCC Imagery recently made significant advances in full coverage and stray light suppression (an SDR issue).The VIIRS imagery products have been widely used and proved to be critical for weather forecasts and environmental monitoring applications.Figure 4 is a VIIRS RGB composite imagery of channels M4, M7, and M11, showing the eruption of Bárðarbunga in Iceland, as it appeared at 13:42 UTC 17 November 2014.The VIIRS Imagery EDR User's guide is now available on the JSTAR website [22].The Users' Guide is a required part of the Validation Maturity process and it has been released for many S-NPP SDR and EDR products that reached the validated maturity.

Product Performance Monitoring
Product performance monitoring is an integral element of Cal/Val and essential for anomaly detection and mitigation, and science maintenance of the long-term stability of the data products.The JSTAR teams have developed the Intensive Cal/Val System (ICVS, [28]) to monitor in near real-time the S-NPP spacecraft and onboard instruments' health status, performance, and SDR product data quality.The ICVS aids the SDR/EDR Cal/Val teams with corrective actions and in making occasional algorithm upgrades for reactive maintenance and long-term sustainment.The ICVS has demonstrated its tremendous value in monitoring satellite instrument health for JPSS as well as other national/international operational weather satellites.It has become a powerful tool that is being widely used by operational and research users for timely monitoring of the sensors' performance.It also ensures that qualities of the satellite observations are intercomparable and are tied to international standards for weather, climate, ocean and other environmental applications.Leveraging on the experience of ICVS, an EDR product long-term science monitoring system (LTSM, [29]) is also in the developmental phase to integrate science product monitoring and routine validations and to aid the operational users and product scientists in monitoring the EDR product quality in near real time, both routinely as well as on a long-term basis.Figure 5 shows an example of the CrIS radiance standard deviation per scan as monitored by the ICVS system.Figure 6a UTC 17 November 2014.The VIIRS Imagery EDR User's guide is now available on the JSTAR website [22].The Users' Guide is a required part of the Validation Maturity process and it has been released for many S-NPP SDR and EDR products that reached the validated maturity.

Product Performance Monitoring
Product performance monitoring is an integral element of Cal/Val and essential for anomaly detection and mitigation, and science maintenance of the long-term stability of the data products.The JSTAR teams have developed the Intensive Cal/Val System (ICVS, [28]) to monitor in near real-time the S-NPP spacecraft and onboard instruments' health status, performance, and SDR product data quality.The ICVS aids the SDR/EDR Cal/Val teams with corrective actions and in making occasional algorithm upgrades for reactive maintenance and long-term sustainment.The ICVS has demonstrated its tremendous value in monitoring satellite instrument health for JPSS as well as other national/international operational weather satellites.It has become a powerful tool that is being widely used by operational and research users for timely monitoring of the sensors' performance.It also ensures that qualities of the satellite observations are intercomparable and are tied to international standards for weather, climate, ocean and other environmental applications.Leveraging on the experience of ICVS, an EDR product long-term science monitoring system (LTSM, [29]) is also in the developmental phase to integrate science product monitoring and routine validations and to aid the operational users and product scientists in monitoring the EDR product quality in near real time, both routinely as well as on a long-term basis.Figure 5 shows an example of the CrIS radiance standard deviation per scan as monitored by the ICVS system.Figure 6a

Enterprise Algorithm Developments and Other Future Improvements
Based on feedback from science panels and end-users acquired during JSTAR Annual Science Meetings, the NJO has directed STAR to work on NOAA enterprise systems for JPSS Priority 3 and 4 products.Enterprise solutions use the same scientific methodology and software base to create the same classification of product from differing input data (e.g., satellite, in situ, ancillary) [30].This architecture provides a wide variety of benefits alleviating the challenges of ever increasing satellite and ancillary data volumes, cost effectiveness, and user demands for enhanced product quality and consistency.Enterprise solutions provide optimal and reliable data products across multiple satellite platforms through consistent algorithms, blended products, and longevity in conjunction with legacy and next generation systems (POES to JPSS, GOES to GOES-R, and NASA EOS to the Decadal Survey Mission).Enterprise solutions offer cost effectiveness by maximizing code reuse, better maintenance and quality control, facilitated integration of observations from multiple data streams, coordinated programs for inter-comparisons with other instruments, and risk identification and management.JSTAR teams are currently working towards developing and implementing the Enterprise algorithms for both S-NPP and JPSS-1.The Cal/Val process described earlier in this article will be applied to the S-NPP/JPSS enterprise algorithms.

Enterprise Algorithm Developments and Other Future Improvements
Based on feedback from science panels and end-users acquired during JSTAR Annual Science Meetings, the NJO has directed STAR to work on NOAA enterprise systems for JPSS Priority 3 and 4 products.Enterprise solutions use the same scientific methodology and software base to create the same classification of product from differing input data (e.g., satellite, in situ, ancillary) [30].This architecture provides a wide variety of benefits alleviating the challenges of ever increasing satellite and ancillary data volumes, cost effectiveness, and user demands for enhanced product quality and consistency.Enterprise solutions provide optimal and reliable data products across multiple satellite platforms through consistent algorithms, blended products, and longevity in conjunction with legacy and next generation systems (POES to JPSS, GOES to GOES-R, and NASA EOS to the Decadal Survey Mission).Enterprise solutions offer cost effectiveness by maximizing code reuse, better maintenance and quality control, facilitated integration of observations from multiple data streams, coordinated programs for inter-comparisons with other instruments, and risk identification and management.
JSTAR teams are currently working towards developing and implementing the Enterprise algorithms for both S-NPP and JPSS-1.The Cal/Val process described earlier in this article will be applied to the S-NPP/JPSS enterprise algorithms.
To further enhance the utilization of S-NPP/JPSS data products and to bridge the gaps between data products and users' applications, JSTAR teams are also working on setting up the environments to process/reprocess SDR/EDR products with the most advanced algorithms in a demonstration testbed mode.The testbed approach would provide a mechanism to evaluate alternative algorithms requested by the users and speed up the transition from research to operation.

Plans and Preparations for JPSS-1
The JPSS-1 and S-NPP xDR algorithms may be similar for those instruments that have remained essentially the same between the two platforms (e.g., ATMS, VIIRS), while the algorithms could be different for instruments with updated characteristics (e.g., OMPS and CrIS).In addition to pre-launch configuration files and Processing Coefficient Table (PCT) updates, the JPSS-1 SDR algorithms also have to accommodate upstream modifications and mitigations due to instrument waivers.Furthermore, based on the experience gained through the S-NPP science product development and end-user utility/feedback, the Cal/Val teams have identified many new products and associated algorithm refinements and improvements.The JPSS-1 science data product Cal/Val thus starts with the pre-launch utilization of a variety of data sets, including: (1) test data sets provided by various instrument vendors; (2) proxy data sets generated by the SDR and EDR Cal/Val teams for functional testing; and (3) data sets specifically required to evaluate whether the performance of SDR and EDR algorithms (Cal/Val) will meet the requirements slated by the L1RDS.These algorithms and the upgrades tested off-line have to be optimized for Research to Operations (R2O).The process thus calls for a coordinated activity among JPSS teams starting with the Flight team, the STAR xDR Cal/Val teams, the STAR AIT, and the JPSS Ground Segment team.All of these teams have defined their roles and responsibilities, and the Cal/Val teams have updated the ATBDs and laid-out JPSS-1 Cal/Val process plans spanning from pre-launch characterization to post-launch validations (documents accessible through STAR JPSS website, [22].Based on specific algorithm readiness levels, the JSTAR teams established a schedule of anticipated dates for the algorithms to achieve Beta, Provisional and Validated statuses.Figure 7 shows the projected Cal/Val timeline for the JPSS-1 xDR products.Figure 8a-c  To further enhance the utilization of S-NPP/JPSS data products and to bridge the gaps between data products and users' applications, JSTAR teams are also working on setting up the environments to process/reprocess SDR/EDR products with the most advanced algorithms in a demonstration testbed mode.The testbed approach would provide a mechanism to evaluate alternative algorithms requested by the users and speed up the transition from research to operation.

Plans and Preparations for JPSS-1
The JPSS-1 and S-NPP xDR algorithms may be similar for those instruments that have remained essentially the same between the two platforms (e.g., ATMS, VIIRS), while the algorithms could be different for instruments with updated characteristics (e.g., OMPS and CrIS).In addition to pre-launch configuration files and Processing Coefficient Table (PCT) updates, the JPSS-1 SDR algorithms also have to accommodate upstream modifications and mitigations due to instrument waivers.Furthermore, based on the experience gained through the S-NPP science product development and end-user utility/feedback, the Cal/Val teams have identified many new products and associated algorithm refinements and improvements.The JPSS-1 science data product Cal/Val thus starts with the pre-launch utilization of a variety of data sets, including: (1) test data sets provided by various instrument vendors; (2) proxy data sets generated by the SDR and EDR Cal/Val teams for functional testing; and (3) data sets specifically required to evaluate whether the performance of SDR and EDR algorithms (Cal/Val) will meet the requirements slated by the L1RDS.These algorithms and the upgrades tested off-line have to be optimized for Research to Operations (R2O).The process thus calls for a coordinated activity among JPSS teams starting with the Flight team, the STAR xDR Cal/Val teams, the STAR AIT, and the JPSS Ground Segment team.All of these teams have defined their roles and responsibilities, and the Cal/Val teams have updated the ATBDs and laid-out JPSS-1 Cal/Val process plans spanning from pre-launch characterization to post-launch validations (documents accessible through STAR JPSS website, [22].Based on specific algorithm readiness levels, the JSTAR teams established a schedule of anticipated dates for the algorithms to achieve Beta, Provisional and Validated statuses.Figure 7 shows the projected Cal/Val timeline for the JPSS-1 xDR products.Figure 8(a-c) shows a comparison of the projected JPSS-1 timelines to that of S-NPP in achieving Beta, Provisional and Validated maturity status.These projections clearly indicate that the S-NPP Cal/Val experience aids in expediting the JPSS-1 Cal/Val maturity process.The Cal/Val activities for JPSS-1 are expected to be much more accelerated than those for S-NPP, and JPSS-1 data products will be provided to decision makers/users with a much improved latency.

Summary and Conclusions
Most of the operational S-NPP data products have reached the Validated maturity level.The S-NPP data products are easily accessible via NOAA operations, direct readout, and NOAA CLASS.The Cal/Val activities for JPSS-1 are expected to be much more accelerated than those for S-NPP, and JPSS-1 data products will be provided to decision makers/users with a much improved latency.

Summary and Conclusions
Most of the operational S-NPP data products have reached the Validated maturity level.The S-NPP data products are easily accessible via NOAA operations, direct readout, and NOAA CLASS.The scientific maturity of these products is well documented and the Cal/Val artifacts are available on the JSTAR website.S-NPP Product evaluation and updates are continuing and are currently in long-term monitoring and reactive maintenance phase.Replacement and upgrade of current S-NPP algorithms with NOAA enterprise algorithms are ongoing.
With improved knowledge of the pre-launch characterization of the J1 instruments and by leveraging the S-NPP Cal/Val experience, the JSTAR teams work with JPSS Algorithm Management Project (AMP) and JPSS Program Science, and are ready to accelerate the Cal/Val activities for JPSS-1 sensor and data products.Detailed Cal/Val activities and schedules are described in the JPSS-1 Cal/Val plans, which have been delivered to the JPSS program and are also available to the public on the JSTAR webpage.The Cal/Val activities for JPSS-1 are expected to be much more accelerated than those for S-NPP, and JPSS-1 data products will be provided to decision makers/users with a much-improved latency.

3 .
JSTAR Cal/Val Teams Two primary goals of the JSTAR Cal/Val Program are: (1) providing robust, affordable, and flexible state-of-the-art scientific solutions to meet JPSS requirements and (2) assuring operational viability of the data products meeting JPSS mission objectives.To achieve these goals, the JSTAR Cal/Val Program formed the Cal/Val teams consisting of highly distinguished scientists and engineers from an array of government agencies (NOAA, NASA; Department of Defense, DOD), NOAA's Cooperative Institutes (University partners), and industry partners.The Cal/Val team members have first-hand knowledge of algorithms developed for a variety of satellite systems (NOAA Polar orbiting Environmental Satellites, POES; Geostationary Operational Environmental Satellites, GOES; Defense Meteorological Satellite Program, DMSP; Earth Observing Systems, EOS; Meteorological Operational satellites (MetOp) program; and GOES-R) to define robust, affordable, and flexible state-of-art scientific solutions meeting JPSS requirements.

3 .
JSTAR Cal/Val Teams Two primary goals of the JSTAR Cal/Val Program are: (1) providing robust, affordable, and flexible state-of-the-art scientific solutions to meet JPSS requirements and (2) assuring operational viability of the data products meeting JPSS mission objectives.To achieve these goals, the JSTAR Cal/Val Program formed the Cal/Val teams consisting of highly distinguished scientists and engineers from an array of government agencies (NOAA, NASA; Department of Defense, DOD), NOAA's Cooperative Institutes (University partners), and industry partners.The Cal/Val team members have first-hand knowledge of algorithms developed for a variety of satellite systems (NOAA Polar orbiting Environmental Satellites, POES; Geostationary Operational Environmental Satellites, GOES; Defense Meteorological Satellite Program, DMSP; Earth Observing Systems, EOS; Meteorological Operational satellites (MetOp) program; and GOES-R) to define robust, affordable, and flexible state-of-art scientific solutions meeting JPSS requirements.

4. 1 .
Pre-Launch to Post-Launch and Cal/Val Maturity Declaring SDR/EDR product maturity is the result of a specific review of artifacts that document that the products meet a series of criteria defined for each maturity stage.The Cal/Val plan documents describe in detail the methodologies on how xDR products specified in the JPSS Program L1RD and L1RDS will be evaluated and validated for each of the maturity status.The planned Cal/Val activities during the (a) Pre-Launch; (b) Early Orbit Check-out (EOC); (c) Intensive Cal/Val (ICV), and (d) Long-term Monitoring (LTM) phases are also included in the Cal/Val plans.This is schematically depicted in Figure

Figure 2 .
Figure 2. Components of Cal/Val Process from Pre-Launch to Post-Launch.

Figure 2 .
Figure 2. Components of Cal/Val Process from Pre-Launch to Post-Launch.

Figure 3 .
Figure 3. Satellite observations contribution to reducing forecast errors (plot courtesy of Carla Cardinali and Sean Healy, ECMWF).

Figure 4 .
Figure 4.The multi-spectral capabilities of VIIRS are useful for investigating the volcano: VIIRS RGB composite of channels M4, M7, and M11 (red is 2.15 um, green is 0.86 um, and the blue is 0.55 um reflectance values).This combination makes vegetation appear green, ice appears dark cyan, clouds appear a light cyan color and the hot spot from the volcano appears red.The image is showing the eruption of Bárðarbunga in Iceland, as it appeared at 13:42 UTC 17 November 2014 (Image courtesy of C. Seaman, Cooperative Institute for Research in the Atmosphere, CIRA, Colorado State University).

Figure 3 .
Figure 3. Satellite observations contribution to reducing forecast errors (plot courtesy of Carla Cardinali and Sean Healy, ECMWF).

Figure 3 .
Figure 3. Satellite observations contribution to reducing forecast errors (plot courtesy of Carla Cardinali and Sean Healy, ECMWF).

Figure 4 .
Figure 4.The multi-spectral capabilities of VIIRS are useful for investigating the volcano: VIIRS RGB composite of channels M4, M7, and M11 (red is 2.15 um, green is 0.86 um, and the blue is 0.55 um reflectance values).This combination makes vegetation appear green, ice appears dark cyan, clouds appear a light cyan color and the hot spot from the volcano appears red.The image is showing the eruption of Bárðarbunga in Iceland, as it appeared at 13:42 UTC 17 November 2014 (Image courtesy of C. Seaman, Cooperative Institute for Research in the Atmosphere, CIRA, Colorado State University).

Figure 4 .
Figure 4.The multi-spectral capabilities of VIIRS are useful for investigating the volcano: VIIRS RGB composite of channels M4, M7, and M11 (red is 2.15 um, green is 0.86 um, and the blue is 0.55 um reflectance values).This combination makes vegetation appear green, ice appears dark cyan, clouds appear a light cyan color and the hot spot from the volcano appears red.The image is showing the eruption of Bárðarbunga in Iceland, as it appeared at 13:42 UTC 17 November 2014 (Image courtesy of C. Seaman, Cooperative Institute for Research in the Atmosphere, CIRA, Colorado State University).
,b shows the S-NPP VIIRS and Aqua Moderate-Resolution Imaging Spectroradiometer (MODIS) global Land Surface Temperature (LST) maps as depicted by the EDR product monitoring system.Remote Sens. 2016, 8, 139 8 of 13 ,b shows the S-NPP VIIRS and Aqua Moderate-Resolution Imaging Spectroradiometer (MODIS) global Land Surface Temperature (LST) maps as depicted by the EDR product monitoring system.

Figure 5 .
Figure 5. CrIS radiance standard deviation per scan as monitored by the ICVS system.Figure 5. CrIS radiance standard deviation per scan as monitored by the ICVS system.

Figure 5 .
Figure 5. CrIS radiance standard deviation per scan as monitored by the ICVS system.Figure 5. CrIS radiance standard deviation per scan as monitored by the ICVS system.

Figure 6 .
Figure 6.(a) S-NPP VIIRS and (b) Aqua MODIS global Land Surface Temperature (LST) maps as depicted by the EDR product monitoring system.The VIIRS instrument's wider swath and the cloud flag criteria used by the LST EDR product results in much better spatial coverage.

Figure 6 .
Figure 6.(a) S-NPP VIIRS and (b) Aqua MODIS global Land Surface Temperature (LST) maps as depicted by the EDR product monitoring system.The VIIRS instrument's wider swath and the cloud flag criteria used by the LST EDR product results in much better spatial coverage.
shows a comparison of the projected JPSS-1 timelines to that of S-NPP in achieving Beta, Provisional and Validated maturity status.These projections clearly indicate that the S-NPP Cal/Val experience aids in expediting the JPSS-1 Cal/Val maturity process.Remote Sens. 2016, 8, 139 10 of 13

Figure 8 .
Figure 8. (a-c) Projected JPSS-1 Timelines (green bars) and comparison with the S-NPP timelines (red bars) in achieving (a) beta (top); (b) provisional (middle); and (c) validated (bottom) maturity status.The Cal/Val activities for JPSS-1 are expected to be much more accelerated than those for S-NPP, and JPSS-1 data products will be provided to decision makers/users with a much improved latency.

Figure 8 .
Figure 8. (a-c) Projected JPSS-1 Timelines (green bars) and comparison with the S-NPP timelines (red bars) in achieving (a) beta (top); (b) provisional (middle); and (c) validated (bottom) maturity status.The Cal/Val activities for JPSS-1 are expected to be much more accelerated than those for S-NPP, and JPSS-1 data products will be provided to decision makers/users with a much improved latency.
* Within the Cloud Properties Algorithms, all products except Cloud Cover Layer and Nighttime Cloud Optical Properties have been validated.** Enterprise Algorithms are being implemented.