# Hyperspectral IASI L1C Data Compression

^{*}

## Abstract

**:**

## 1. Introduction

## 2. IASI Instrument

#### 2.1. Space Program of IASI Instrument

#### 2.2. IASI Instrument Details

#### 2.3. IASI Processing Chain

#### 2.3.1. On-Board Processing Chain

#### 2.3.2. On-Ground Processing Chain

#### 2.4. Data Dissemination

## 3. Data Compression

#### 3.1. Data Coding System Pipeline

#### 3.2. Characteristics of the Coding Techniques

#### 3.3. Setting and Parameter Configuration

#### 3.4. Spectral Transforms

#### 3.5. Divide-and-Conquer Strategy for KLT/RKLT

#### 3.5.1. Computational Cost

#### 3.5.2. Execution Time

#### 3.5.3. Transform Coding Performance

## 4. Experimental Results

#### 4.1. Data Collection and Software

#### 4.2. Lossless Compression Results

- Coding performance for IASI-A and IASI-B products is nearly the same. Lossless compression of IASI-B products is, on average, only 0.75% better than for IASI-A. This negligible difference happens for all IASI-A and IASI-B products and for all compression schemes.
- IASI L1C data present high spectral redundancy. M-CALIC, CCSDS-123.0 and HEVC, which originally exploit the spectral redundancy, achieve better outcomes than JPEG-LS, JPEG2000 or CCSDS-122.0, which do not exploit this redundancy. For the latter techniques, taking advantage of this redundancy through a spectral transform yields significantly better compression performance, bridging the gap with the former techniques.
- Compression techniques that already exploit the spectral redundancy by themselves also benefit from applying a spectral transform. When paired with a spectral transform, M-CALIC, CCSDS-123.0, and HEVC usually achieve better coding performance too (except for IWT + M-CALIC and RPOT + CCSDS-123.0). This effect is specially significant in the case of HEVC, where up to 11.11% can be improved, but also for M-CALIC, where gains are close to 9%. Gains for CCSDS-123.0, which was the coding technique providing the best performance, are less meaningful.
- Multilevel Clustering RKLT or RWA yield the best coding performance. Multilevel Clustering RKLT brings the largest improvements, closely followed by RWA. As compared to original CCSDS-123.0, which is the coding technique providing the best performance when no spectral transform is applied, the improvements for Multilevel Clustering RKLT and for RWA when combined with M-CALIC are, respectively, of 4.7% and 2.4%.
- Compression ratios over 2.5:1 (bit-rates close to 6.3 bpppc) can be achieved for lossless compression of IASI L1C products. The best results are obtained by Multilevel Clustering RKLT + M-CALIC, which achieves, on average, a compression ratio of 2.54:1 for IASI-A products and 2.56 for IASI-B products.

#### 4.3. Near-Lossless Compression Results

- As expected, compression ratio increases as PAE increases.
- Competitive compression performance is achieved even by allowing small errors. Large savings over 17% and 30% with respect to lossless compression are already achieved for such small PAE as 1 and 3.
- M-CALIC yields higher compression ratio than JPEG-LS. M-CALIC uses an arithmetic coder, while JPEG-LS uses Golomb codes, for which bit-rates below 1 bpppc are not achievable.

#### 4.4. Lossy Compression Results

- Exploiting the spectral redundancy is essential to achieve competitive performance. Applying a spectral transform always outperforms the scheme that does not exploit the spectral redundancy. Performance difference is more apparent as the compression ratio decreases, growing from 5 to over 15 dB.
- Multilevel Clustering KLT yields the best coding performance. As happened for lossless compression, also in the case of lossy compression, Multilevel Clustering KLT furnishes the highest results, followed by POT and DWT. At high compression ratios (higher than 20:1), POT yields almost equivalent performance, mostly because of the larger size of the side-information needed by Multilevel Clustering KLT.
- JPEG 2000 outperforms CCSDS-122.0. JPEG 2000 is a more complex coding technique that is able to produce more competitive results.
- Plain 2D CCSDS-122.0 yields low performance at high compression ratios. This standard starts achieving good results for compression ratios lower than 100:1.

#### 4.5. Comparison between Near-Lossless and Lossy Compression

- Near-lossless outperforms lossy compression in terms of PAE. Near-lossless compression introduces lower maximum errors in the data than lossy compression.
- Lossy compression outperforms near-lossless compression in terms of SNR Energy. Lossy compression yields larger results, especially at large compression ratios.

#### 4.6. Compression and Decompression Runtimes

#### 4.7. Analysis of the Reconstructed Radiances

#### 4.8. Discussion

## 5. Concluding Remarks

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Prunet, P.; Thépaut, J.N.; Cassé, V. The information content of clear sky IASI radiances and their potential for numerical weather prediction. Q. J. R. Meteorol. Soc.
**1998**, 124, 211–241. [Google Scholar] [CrossRef] - Hilton, F.; Armante, R.; August, T.; Barnet, C.; Bouchard, A.; Camy-Peyret, C.; Capelle, V.; Clarisse, L.; Clerbaux, C.; Coheur, P.F.; et al. Hyperspectral Earth observation from IASI: Five years of accomplishments. Bull. Am. Meteorol. Soc.
**2012**, 93, 347–370. [Google Scholar] [CrossRef] - Amato, U.; Cuomo, V.; Serio, C. Assessing the impact of radiometric noise on IASI performances. Remote Sens.
**1995**, 16, 2927–2938. [Google Scholar] [CrossRef] - Pougatchev, N.; August, T.; Calbet, X.; Hultberg, T.; Oduleye, O.; Schlüssel, P.; Stiller, B.; Germain, K.S.; Bingham, G. IASI temperature and water vapor retrievals-error assessment and validation. Atmos. Chem. Phys.
**2009**, 9, 6453–6458. [Google Scholar] [CrossRef] - George, M.; Clerbaux, C.; Hurtmans, D.; Turquety, S.; Coheur, P.F.; Pommier, M.; Hadji-Lazaro, J.; Edwards, D.P.; Worden, H.; Luo, M.; et al. Carbon monoxide distributions from the IASI/METOP mission: Evaluation with other space-borne remote sensors. Atmos. Chem. Phys.
**2009**, 9, 8317–8330. [Google Scholar] [CrossRef] - Clarisse, L.; Coheur, P.F.; Prata, A.J.; Hurtmans, D.; Razavi, A.; Phulpin, T.; Hadji-Lazaro, J.; Clerbaux, C. Tracking and quantifying volcanic SO
_{2}with IASI, the September 2007 eruption at Jebel at Tair. Atmos. Chem. Phys.**2008**, 8, 7723–7734. [Google Scholar] [CrossRef] - Clerbaux, C.; Boynard, A.; Clarisse, L.; George, M.; Hadji-Lazaro, J.; Herbin, H.; Hurtmans, D.; Pommier, M.; Razavi, A.; Turquety, S.; et al. Monitoring of atmospheric composition using the thermal infrared IASI/ MetOp sounder. Atmos. Chem. Phys.
**2009**, 9, 6041–6054. [Google Scholar] [CrossRef] - Wespes, C.; Hurtmans, D.; Clerbaux, C.; Santee, M.L.; Martin, R.V.; Coheur, P.F. Global distributions of nitric acid from IASI/MetOp measurements. Atmos. Chem. Phys.
**2009**, 9, 7949–7962. [Google Scholar] [CrossRef] - Pommier, M.; Law, K.S.; Clerbaux, C.; Turquety, S.; Hurtmans, D.; Hadji-Lazaro, J.; Coheur, P.F.; Schlager, H.; Ancellet, G.; Paris, J.D.; et al. IASI carbon monoxide validation over the Arctic during POLARCAT spring and summer campaigns. Atmos. Chem. Phys.
**2010**, 10, 10655–10678. [Google Scholar] [CrossRef] - Grieco, G.; Masiello, G.; Matricardi, M.; Serio, C. Partially scanned interferogram methodology applied to IASI for the retrieval of CO, CO
_{2}, CH_{4}and N_{2}O. Opt. Express**2013**, 21, 24753–24769. [Google Scholar] [CrossRef] - Liuzzi, G.; Masiello, G.; Serio, C.; Venafra, S.; Camy-Peyret, C. Physical inversion of the full IASI spectra: Assessment of atmospheric parameters retrievals, consistency of spectroscopy and forward modelling. J. Quant. Spectrosc. Radiat. Transf.
**2016**, 182, 128–157. [Google Scholar] [CrossRef] - Coheur, P.F.; Clarisse, L.; Turquety, S.; Hurtmans, D.; Clerbaux, C. IASI measurements of reactive trace species in biomass burning plumes. Atmos. Chem. Phys.
**2009**, 9, 5655–5667. [Google Scholar] [CrossRef] - Turquety, S.; Hurtmans, D.; Hadji-Lazaro, J.; Coheur, P.F.; Clerbaux, C.; Josset, D.; Tsamalis, C. Tracking the emission and transport of pollution from wildfires using the IASI CO retrievals: Analysis of the summer 2007 Greek fires. Atmos. Chem. Phys.
**2009**, 9, 4897–4913. [Google Scholar] [CrossRef] - CNES. Dossier de Définition des Algorithmes IASI; REF. IA-DF-0000-2006-CNE; CNES: Toulouse, France, 2009. [Google Scholar]
- Masiello, G.; Serio, C. Dimensionality-reduction approach to the thermal radiative transfer equation inverse problem. Geophys. Res. Lett.
**2004**, 31. [Google Scholar] [CrossRef] - Hultberg, T. IASI Principal Component Compression (IASI PCC) FAQ. Available online: http://www.eumetsat.int/website/wcm/idc/idcplg?IdcService=GETFILEdDocName=pdfipccfaqRevisionSelectionMethod=LatestReleasedRendition=Web (accessed on 14 June 2017).
- Atkinson, N.C.; Hilton, F.I.; Illingworth, S.M.; Eyre, J.R.; Hultberg, T. Potential for the use of reconstructed IASI radiances in the detection of atmospheric trace gases. Atmos. Meas. Tech.
**2010**, 3, 991–1003. [Google Scholar] [CrossRef] - Camps-Valls, G.; Muñoz-Marí, J.; Gómez-Chova, L.; Guanter, L.; Calbet, X. Nonlinear statistical retrieval of atmospheric profiles from MetOp-IASI and MTG-IRS infrared sounding data. IEEE Trans. Geosci. Remote Sens.
**2012**, 50, 1759–1769. [Google Scholar] [CrossRef] - Masiello, G.; Serio, C.; Antonelli, P. Inversion for atmospheric thermodynamical parameters of IASI data in the principal components space. Q. J. R. Meteorol. Soc.
**2012**, 138, 103–117. [Google Scholar] [CrossRef] - Serio, C.; Masiello, G.; Liuzzi, G. Demonstration of random projections applied to the retrieval problem of geophysical parameters from hyper-spectral infrared observations. Appl. Opt.
**2016**, 55, 6576–6587. [Google Scholar] [CrossRef] - Motta, G.; Rizzo, F.; Storer, J.A. Hyperspectral Data Compression; Springer Science & Business Media: New York, NY, USA, 2006. [Google Scholar]
- Huang, B. Satellite Data Compression; Springer Science & Business Media: New York, NY, USA, 2011. [Google Scholar]
- Mercier, G.; Mouchot, M.; Cazuguel, G. Joint classification and compression of hyperspectral images. IEEE Int. Remote Sens. Symp.
**1999**, 4, 2035–2037. [Google Scholar] - Blanes, I.; Serra-Sagristà, J. Quality evaluation of progressive lossy-to-lossless remote-sensing image coding. In Proceedings of the 16th IEEE International Conference in Image Processing (ICIP), Cairo, Egypt, 7–9 November 2009; pp. 3709–3712. [Google Scholar]
- García-Vílchez, F.; Muñoz-Marí, J.; Zortea, M.; Blanes, I.; González-Ruiz, V.; Camps-Valls, G.; Plaza, A.; Serra-Sagristà, J. On the impact of lossy compression on hyperspectral image classification and unmixing. IEEE Geosci. Remote Sens. Lett.
**2011**, 8, 253–257. [Google Scholar] [CrossRef] - García-Sobrino, J.; Blanes, I.; Laparra, V.; Camps-Valls, G.; Serra-Sagristà, J. Impact of Near-Lossless Compression of IASI L1C data on Statistical Retrieval of Atmospheric Profiles. In Proceedings of the On-Board Payload Data Compression Workshop (OBPDC), Venice, Italy, 23–24 October 2014. [Google Scholar]
- García-Sobrino, J.; Serra-Sagristà, J.; Laparra, V.; Calbet, X.; Camps-Valls, G. Statistical Atmospheric Parameter Retrieval Largely Benefits from Spatial-Spectral Image Compression. IEEE Trans. Geosci. Remote Sens.
**2017**, 55, 2213–2224. [Google Scholar] [CrossRef] - Qian, S.E.; Bergeron, M.; Cunningham, I.; Gagnon, L.; Hollinger, A. Near lossless data compression onboard a hyperspectral satellite. IEEE Trans. Aerosp. Electron. Syst.
**2006**, 42, 851–866. [Google Scholar] [CrossRef] - ISO/IEC. JPEG-LS Lossless and Near-Lossless Compression for Continuous-Tone Still Images; ITU: Geneva, Switzerland, 1999. [Google Scholar]
- JPEG-Committee. Standard JPEG2000, Document ISO/IEC 15444. Available online: http://www.jpeg.org/jpeg2000/workplan.html (accessed on 14 June 2017).
- Magli, E.; Olmo, G.; Quacchio, E. Optimized onboard lossless and near-Lossless compression of hyperspectral data using CALIC. IEEE Geosci. Remote Sens. Lett.
**2004**, 1, 21–25. [Google Scholar] [CrossRef] - Consultative Committee for Space Data Systems (CCSDS). Image Data Compression CCSDS 122.0-B-1; Blue Book, CCSDS, 2005. Available online: https://public.ccsds.org/Pubs/122x0b1c3.pdf (accessed on 14 June 2017).
- Consultative Committee for Space Data Systems (CCSDS). Lossless Multispectral & Hyperspectral Image Compression CCSDS 123.0-B-1; Blue Book, CCSDS, 2012. Available online: https://public.ccsds.org/Pubs/123x0b1ec1.pdf (accessed on 14 June 2017).
- ISO/IEC. High Efficiency Coding and Media Delivery in Heterogeneous Environments—Part 2: High Efficiency Video Coding, 2013. Available online: http://www.iso.org/iso/cataloguedetail.htm?csnumber=67660 (accessed on 14 June 2017).
- Chang, L.; Cheng, C.M.; Chen, T.C. An efficient adaptive KLT for multispectral image compression. Proceedings of 4th IEEE Southwest Symposium on Image Analysis and Interpretation, Austin, TX, USA, 2–4 April 2000. [Google Scholar]
- Salomon, D. Data Compression: The Complete Reference; Springer Science & Business Media: New York, NY, USA, 2004. [Google Scholar]
- Blanes, I.; Serra-Sagristà, J. Pairwise orthogonal transform for spectral image coding. IEEE Trans. Geosci. Remote Sens.
**2011**, 49, 961–972. [Google Scholar] [CrossRef] - Amrani, N.; Serra-Sagristà, J.; Laparra, V.; Marcellin, M.W.; Malo, J. Regression Wavelet Analysis for Lossless Coding of Remote-Sensing Data. IEEE Trans. Geosci. Remote Sens.
**2016**, 54, 5616–5627. [Google Scholar] [CrossRef] - EUMETSAT. IASI Mission. Available online: http://www.eumetsat.int/website/home/Satellites/CurrentSatellites/Metop/MetopDesign/IASI/index.html (accessed on 14 June 2017).
- The World Data Center for Remote Sensing of the Atmosphere (WDC-RSAT). IASI (Infrared Atmospheric Sounding Interferometer). Available online: http://andromeda.caf.dlr.de/sensors/iasi (accessed on 14 June 2017).
- EUMETSAT. MetOp Mission. Available online: http://www.eumetsat.int/website/home/Satellites/CurrentSatellites/Metop/index.html (accessed on 14 June 2017).
- EUMETSAT. EUMETSAT News. Available online: http://www.eumetsat.int/website/home/News/DAT3304789.html (accessed on 14 June 2017).
- Step, A.M. The EUMETSAT Polar System. ESA Bull.
**2006**, 127, 19. [Google Scholar] - Klaes, K.D.; Cohen, M.; Buhler, Y.; Schlëssel, P.; Munro, R.; Engeln, A.; Clérigh, E.; Bonekamp, H.; Ackermann, J.; Schmetz, J.; et al. An introduction to the EUMETSAT polar system. Bull. Am. Meteorol. Soc.
**2007**, 88, 1085–1096. [Google Scholar] [CrossRef] - Chalon, G.; Cayla, F.; Diebel, D. IASI: An advanced sounder for operational meteorology. In Proceedings of the 52nd International Astronautical Congress (IAF), Toulouse, France, 1–5 October 2001. [Google Scholar]
- CNES. Spécification Technique de Besoin du Logiciel Opérationnel IASI; REF. IA-SB-2100-9462-CNE; CNES: Toulouse, France, 2006.
- EUMETSAT. IASI Measurement and Verification Data; REF. IA-ID-1000-6477-AER; EUMETSAT: Darmstadt, Germany, 2010. [Google Scholar]
- Hébert, P.; Blumstein, D.; Buil, C.; Carlier, T.; Chalon, G.; Astruc, P.; Clauss, A.; Siméoni, D.; Tournier, B. IASI instrument: Technical description and measured performances. In Proceedings of the 5th International Conference on Space Optics, Toulouse, France, 30 March–2 April 2014; Volume 554, pp. 49–56. [Google Scholar]
- Simeoni, D.; Astruc, P.; Miras, D.; Alis, C.; Andreis, O.; Scheidel, D.; Degrelle, C.; Nicol, P.; Bailly, B.; Guiard, P.; et al. Design and development of IASI instrument. In Proceedings of the SPIE 49th Annual Optical Science and Technology Meeting, Denver, Colorado, 2–6 August 2004; pp. 208–219. [Google Scholar]
- EUMETSAT. IASI Level 1: Product Guide; REF. EUM/OPS-EPS/MAN/04/0032; EUMETSAT: Darmstadt, Germany, 2012. [Google Scholar]
- Tournier, B.; Blumstein, D.; Cayla, F.; Chalon, G. IASI level 0 and 1 processing algorithms description. In Proceedings of the 12th International TOVS Study Conference (ITSC-XII), Lorne, Victoria, Australia, 27 February–5 March 2002. [Google Scholar]
- EUMETSAT. IASI Level 2: Product Guide; REF. EUM/OPS-EPS/MAN/04/0033; EUMETSAT: Darmstadt, Germany, 2012. [Google Scholar]
- ESA. IASI Data Processing Chain. Available online: http://www.esa.int/OurActivities/ObservingtheEarth/TheLivingPlanetProgramme/Meteorologicalmissions/MetOp/Dataprocessingchain (accessed on 14 June 2017).
- EUMETCast. EUMETCast Website. Available online: http://www.eumetsat.int/website/home/Data/DataDelivery/EUMETCast/index.html (accessed on 14 June 2017).
- PODAAC. PODAAC Website. Available online: https://podaac.jpl.nasa.gov/ (accessed on 14 June 2017).
- CEDA. CEDA Website. Available online: http://catalogue.ceda.ac.uk/ (accessed on 14 June 2017).
- EUMETSAT. Central Operations Report for the Period January to June 2016. EUM/OPS/REP/16/866335, v1A, 2016. Available online: http://www.eumetsat.int/website/home/Data/ServiceStatus/CentralOperationsReports/index.html (accessed on 14 June 2017).
- CEDA Support.
**2017**. Private correspondence. - EUMETCast. IASI Regional Data Service Level 1 Website. Available online: http://navigator.eumetsat.int/discovery/Start/DirectSearch/Extended.do?f(r0)=EO:EUM:DAT:METOP:EARS-IASI (accessed on 14 June 2017).
- EUMETSAT. IASI PCA-Based Compression Package. Available online: https://nwpsaf.eu/site/software/iasi-pca/ (accessed on 14 June 2017).
- Hultberg, T.; August, T.; Atkinson, N.C.; Smith, F. IASI PC compression—Searching for signal in the residuals. In Proceedings of the ECMWF/EUMETSAT NWP-SAF Workshop on Efficient Representation of Hyper-Spectral Infrared Satellite Observations, Exeter, UK, 5–7 November 2013. [Google Scholar]
- Hilton, F.; Collard, A.D. Recommendations for the Use of Principal Component-Compressed Observations from Infrared Hyperspectral Sounders; Met Office Forecasting R&D Technical Report; Met Office: Exeter, UK, 2009; Volume 536. [Google Scholar]
- Atkinson, N.; Ponsard, C.; Hultberg, T. AAPP Enhancements for the EARS-IASI Service. Available online: https://www.eumetsat.int/website/wcm/idc/idcplg?IdcService=GETFILEdDocName=PDFCONFP55S839ATKINSONPRevisionSelectionMethod=LatestReleasedRendition=Webusg=AFQjCNHm0O2USellr5iHpsff7Y0l17EBRQsig2=Ad6FE7ZHHZUpxV03AJZQTA (accessed on 14 June 2017).
- Antonelli, P.; Revercomb, H.E.; Sromovsky, L.A.; Smith, W.L.; Knuteson, R.O.; Tobin, D.C.; Garcia, R.K.; Howell, H.B.; Huang, H.L.; Best, F.A. A principal component noise filter for high spectral resolution infrared measurements. J. Geophys. Res. Atmos.
**2004**, 109, D23102. [Google Scholar] [CrossRef] - Hultberg, T. IASI Principal Components—Experiences at EUMETSAT, 2011. Available online: ftp://193.17.11.194/pub/EPS/out/Hultberg/IasiPCC/EUMETSATPCCPresentationatECMWFpdf.pdf (accessed on 14 June 2017).
- Golomb, S.W. Run-length encodings. IEEE Trans. Inf. Theory
**1966**, 12, 399–401. [Google Scholar] [CrossRef] - Witten, I.H.; Neal, R.M.; Cleary, J.G. Arithmetic coding for data compression. Commun. ACM
**1987**, 30, 520–540. [Google Scholar] [CrossRef] - Consultative Committee for Space Data Systems (CCSDS). Spectral Pre-Processing Transform for Multispectral & Hyperspectral Image Compression, 2017. Available online: http://cwe.ccsds.org/fm/Lists/Projects/AllOpenChartersWithDraftProjects.aspx (accessed on 14 June 2017).
- Blanes, I.; Serra-Sagristà, J. Cost and Scalability Improvements to the Karhunen-Loêve Transform for Remote-Sensing Image Coding. IEEE Trans. Geosci. Remote Sens.
**2010**, 48, 2854–2863. [Google Scholar] [CrossRef] - EUMETSAT. IASI Level 1 Product Formats and Dissemination. EUM/OPS-EPS/MAN/04/0032, 2012. Available online: http://oiswww.eumetsat.org/WEBOPS/eps-pg/IASI-L1/IASIL1-PG-6ProdFormDis.htm (accessed on 14 June 2017).
- Clunie, D.A. JPEG-LS Software. Available online: http://www.dclunie.com/jpegls.html (accessed on 14 June 2017).
- Taubman, D.S. Kakadu Software. Available online: http://www.kakadusoftware.com/ (accessed on 14 June 2017).
- Magli, E. M-CALIC Software. Available online: http://www1.tlc.polito.it/oldsite/sas-ipl/download.php (accessed on 14 June 2017).
- GICI-UAB. TER Software. Available online: http://gici.uab.cat/GiciWebPage/downloads.php#ter (accessed on 14 June 2017).
- GICI-UAB. EMPORDA Software. Available online: http://gici.uab.cat/GiciWebPage/downloads.php#emporda (accessed on 14 June 2017).
- Fraunhofer-HHI. HEVC Software. Available online: https://hevc.hhi.fraunhofer.de/svn/svnHEVCSoftware/tags/ (accessed on 14 June 2017).
- GICI-UAB. Spectral Transform Software. Available online: http://gici.uab.cat/GiciWebPage/downloads.php#spectral (accessed on 14 June 2017).
- GICI-UAB. Pairwise Orthogonal Transform (POT) Software. Available online: http://gici.uab.cat/GiciWebPage/downloads.php#pot (accessed on 14 June 2017).
- GICI-UAB. Regression Wavelet Analysis (RWA) Transform Software. Available online: http://gici.uab.cat/GiciWebPage/downloads.php#RWA (accessed on 14 June 2017).
- Magli, E. Multiband lossless compression of hyperspectral images. IEEE Trans. Geosci. Remote Sens.
**2009**, 47, 1168–1178. [Google Scholar] [CrossRef] - Blanes, I.; Serra-Sagristà, J.; Marcellin, M.W.; Bartrina-Rapesta, J. Divide-and-Conquer Strategies for Hyperspectral Image Processing: A Review of Their Benefits and Advantages. IEEE Signal Process. Mag.
**2012**, 29, 71–81. [Google Scholar] [CrossRef]

**Figure 1.**EPS program elements. The space component comprises the MetOp-A, MetOp-B, and MetOp-C satellites, while the ground component includes reception and operating stations.

**Figure 2.**Modus operandi of IASI instrument. The instrument scans the Earth’s surface at regular intervals producing 30 FORs per line. Each FOR consists of 4 IFOVs, each of which represents a full spectrum.

**Figure 3.**FOR and IFOV details. A single FOR consists of 4 IFOVs. Each IFOV spreads 12 km of the Earth’s surface and is separated from its neighboring IFOVs by 12.5 km. Each FOR corresponds to, approximately, 50 km of the Earth’s surface.

**Figure 6.**Data compression systems are usually composed of three main stages: pre-processing, coding, and post-processing. The coding stage may, in turn, comprise three steps: either transform or prediction, quantization, and encoding. Only the encoding process is displayed; decoding proceeds in reverse order.

**Figure 7.**Cost comparison in FLOPs for the different spectral transforms used in the experiments applied to an IASI L1C orbit with 8461 spectral channels and a spatial resolution of $765\times 30\times 4$ (number of scan lines × number of FORs per line × number of IFOVs per FOR).

**Figure 8.**Structure of plain KLT/RKLT, Classical Clustering KLT/RKLT, and Multilevel Clustering KLT/RKLT. This example decorrelates 15 spectral channels. Each arrow denotes a channel and each coloured rectangle represents the computation of a KLT/RKLT transform. In the case of Classical Clustering KLT/RKLT, three clusters are employed. In the case of Multilevel Clustering KLT/RKLT, 3 levels of Multilevel Clustering are applied. (

**a**) Plain KLT/RKLT; (

**b**) Classical Clustering KLT/RKLT; (

**c**) Multilevel Clustering KLT/RKLT.

**Figure 9.**Cost comparison in FLOPs for different cluster sizes of Multilevel Clustering RKLT applied to an orbit with ${2}^{13}$ spectral channels and a spatial resolution of $765\times 30\times 4$ (number of scan lines × number of FORs per line × number of IFOVs per FOR).

**Figure 10.**Runtime comparison in minutes for different cluster sizes of Multilevel Clustering RKLT applied to an IASI L1C orbit with ${2}^{13}$ spectral channels and a spatial resolution of $765\times 30\times 4$ (number of scan lines × number of FORs per line × number of IFOVs per FOR). The dissemination granularity of the data is 3 min for Level 1c [70].

**Figure 11.**Rate-distortion performance of near-lossless compression of IASI L1C products. Results report SNR Energy (in dB, higher is better) vs. PAE. (

**a**) IASI-A; (

**b**) IASI-B.

**Figure 12.**Rate-distortion performance of lossy compression of IASI L1C products. Results report SNR Energy (in dB, higher is better) vs. compression ratio. Results for different spectral transforms are plotted in the columns. In each plot, curves for JPEG 2000 and CCSDS-122.0 performance are displayed. Ranges are the same in all the plots to ease the comparison. Top row: IASI-A products; Bottom row: IASI-B products. POT and Multilevel Clustering KLT are not able to reach such high compression ratios (over 1000:1) as DWT because side-information needs to be transmitted besides the compressed data.

**Figure 13.**Performance comparison between near-lossless (M-CALIC) and lossy compression (Multilevel Clustering KLT + JPEG 2000). Top row: PAE (lower is better); Bottom row: SNR Energy (in dB, higher is better).

**Figure 14.**Noise covariance matrix of the original radiances and noise covariance matrix of the reconstructed radiances after Principal Component Compression when 200 and 150 PCS are employed.

**Figure 15.**Normalized radiance residuals statistics. The average of the normalized radiance residuals is shown in blue, standard deviation in red, and maximum and minimum values in green.

**Figure 16.**Covariance matrix of the original radiances and covariance matrix of the reconstructed radiances.

**Figure 17.**Differences between the covariance matrix of the original radiances and the covariance matrix of the reconstructed radiances.

**Table 1.**Main characteristics of IASI instrument [50].

Characteristics of IASI instrument | |
---|---|

Orbit | Polar sun-synchronous |

Time for one orbit | 101 min |

Global Earth coverage | 2 times per day |

Repeat cycle | 29 days (412 orbits) |

Altitude | ∼819 km |

Scan type | Step and stare |

Interferograms | 30 per scan line |

151 ms per interferogram | |

taken in equally spaced time intervals every 8/37 s | |

FOR | 30 per line |

50 km (3.33°) at nadir position | |

4 simultaneous IFOVs of 12 km | |

Full swath width | ∼2200 km (±48.3°) |

Data production | 120 spectra every 8 s |

∼1,300,000 observations per day | |

Data acquisition rate | 45 Mbps |

Data transmission rate | 1.5 Mbps |

Spectral range | Band-1: 645–1240 cm${}^{-1}$ |

Band-2: 1200–2040 cm${}^{-1}$ | |

Band-3 :1960–2760 cm${}^{-1}$ | |

Spectral sampling | 0.25 cm${}^{-1}$ (0.5 cm${}^{-1}$ apodized) |

**Table 2.**Technical characteristics of the considered compression techniques (year, compression paradigm, reference, pre-processing, and post-processing).

JPEG-LS | JPEG 2000 | M-CALIC | CCSDS-122.0 | CCSDS-123.0 | HEVC | |
---|---|---|---|---|---|---|

Year | 1999 | 2000 | 2004 | 2005 | 2012 | 2013 |

Compression Paradigm | Lossless and near-lossless | Lossless and lossy | Lossless and near-lossless | Lossless and lossy | Lossless | Lossless and lossy |

Prediction-based | Transform-based | Prediction- based | Transform-based | Prediction-based | Prediction- and Transform-based | |

Reference | [29] | [30] | [31] | [32] | [33] | [34] |

PRE-PROCESSING | ||||||

✗ | Possibility of multi-channel transform, tile partitioning, and level-shift for unsigned data | ✗ | ✗ | ✗ | Possibility of tiles. channels are partitioned into Coding Tree Units (CTUs). | |

POST-PROCESSING | ||||||

✗ | Bit-stream organization (bit-allocation, data ordering, error resilience, and file format) | ✗ | ✗ | ✗ | Deblock Filtering (DBF) and Sample-Adaptive Offset (SAO). Both stages are optional. |

JPEG-LS | JPEG 2000 | M-CALIC | CCSDS-122.0 | CCSDS-123.0 | HEVC | |
---|---|---|---|---|---|---|

CODING | ||||||

Spatial transform | ✗ | Wavelet transform (up to 32 levels of IWT 5/3 or DWT 9/7) | ✗ | Wavelet transform (3 levels of 9/7 Integer DWT or 9/7 Float DWT) | ✗ | Discrete cosine transform (DCT) and discrete sine transform (DST) |

Prediction | Intra: using 3 neighbor samples | ✗ | Inter: using 2 channels for spectral prediction | ✗ | • Intra: using 1 or 4 neighbor samples • Inter: up to 15 channels for spectral prediction | • Intra: using adjacent blocks as reference, 33 directional plus 2 special modes supported. • Inter: up to 15 frames |

Quanti- zation | Uniform scalar quantization | • Uniform scalar deadzone quantization (Part-1 of standard) • Variable scalar deadzone quantization, and Trellis coded quantization (Part-2 of standard) | Uniform scalar quantization | Uniform scalar quantization | ✗ | Uniform scalar quantization |

Bitplane coding | ✗ | Each bitplane is encoded with three coding passes: (1) significance propagation pass, (2) magnitude refinement pass, and (3) clean-up pass. For the first bitplane only clean-up pass is used | ✗ | First, the first bits of the quantized DC coefficients are encoded. Then, the remaining DC coefficients bit planes are encoded along with the bit planes of AC coefficients using several refinement passes | ✗ | ✗ |

Entropy coder | Golomb Coder and Run Length Coder | MQ Arithmetic Coder. Contextual binary arithmetic coder. Contexts are defined using the 8 adjacent neighbors | Contextual Arithmetic Coder using up to 1024 contexts | Variable Length Coder and Fixed Length Coder | Golomb Coder | Arithmetic Coder (CABAC with 154 contexts) and Variable Length Coder (CAVLC) |

**Table 4.**For each coding technique, the configuration used in the experiments is reported. Default option is employed for the parameters not specified in the table.

Coding Technique | Paradigm | Setting and Mode | Spatial Transform | Spectral Transform |
---|---|---|---|---|

JPEG-LS | Lossless | Plane-interleaved mode | — | • Multilevel Clustering RKLT (200 clusters in first level and multilevel mode) • IWT 5/3 (5 levels) • RPOT • Maximum RWA (Exogenous variant) |

Near-lossless | Plane-interleaved mode | — | — | |

JPEG 2000 | Lossless | Code-blocks of $64\times 64$ size and 1 quality layer | IWT 5/3 (5 levels) | • Multilevel Clustering RKLT (200 clusters in first level and multilevel mode) • IWT 5/3 (5 levels) • RPOT • Maximum RWA (Exogenous variant) |

Lossy | Code-blocks of $64\times 64$ size and 1 quality layer | DWT 9/7 (5 levels) | • Multilevel Clustering KLT (200 clusters in first level and multilevel mode) • DWT 9/7 (5 levels) • POT | |

M-CALIC | Lossless | Default | — | • Multilevel Clustering RKLT (200 clusters in first level and multilevel mode) • IWT 5/3 (5 levels) • RPOT • Maximum RWA (Exogenous variant) |

Near-lossless | Default | — | — | |

CCSDS-122.0 | Lossless | Default | Default | • Multilevel Clustering RKLT (200 clusters in first level and multilevel mode) • IWT 5/3 (5 levels) • RPOT • Maximum RWA (Exogenous variant) |

Lossy | Default | Default | • Multilevel Clustering KLT (200 clusters in first level and multilevel mode) • DWT 9/7 (5 levels) • POT | |

CCSDS-123.0 | Lossless | Default | — | • Multilevel Clustering RKLT (200 clusters in first level and multilevel mode) • IWT 5/3 (5 levels) • RPOT • Maximum RWA (Exogenous variant) |

HEVC | Lossless | Intra and inter prediction | Default | • Multilevel Clustering RKLT (200 clusters in first level and multilevel mode) • IWT 5/3 (5 levels) • RPOT • Maximum RWA (Exogenous variant) |

**Table 5.**Computational cost in FLOPs for IWT, RPOT, RWA Maximum, RWA Exogenous, RKLT, and Multilevel Clustering RKLT. z is the number of spectral channels, m is the number of spatial samples per channel, y is the number of rows, l is the number of wavelet decomposition levels, k is the number of detail channels employed in the prediction level i [38], s is the number of spectral channels per cluster ($s\ll z$), and C is the total number of clusters.

Transform | FLOPs |
---|---|

IWT | $2\times 14(1-\frac{1}{{2}^{l}})mz$ |

RPOT | $16mz+26zy-12m-28y+11mz+5zy-10m-5y$ |

RWA Maximum | $\begin{array}{c}\hfill 8(1-\frac{1}{{2}^{l}})mz+({\sum}_{i=1}^{l}(2m-1){({k}_{i}+1)}^{2}+{({k}_{i}+1)}^{3}+(\frac{z}{{2}^{i}})({k}_{i}+1)\left[(2m-1)+(2{k}_{i}+1)\right])+\hfill \\ \hfill +(2{\sum}_{i=1}^{l}(2{k}_{i}-1)m\frac{z}{{2}^{i}})+2m(z-1)\hfill \end{array}$ |

RWA Exogenous | $8(1-\frac{1}{{2}^{l}})mz+(2{\sum}_{i=1}^{l}(2{k}_{i}-1)m\frac{z}{{2}^{i}})+2m(z-1)$ |

RKLT | $m(4{z}^{2}+3z+1)+\frac{32}{3}{z}^{3}+\frac{1}{2}{z}^{2}-\frac{37}{6}z+5+m(3{z}^{2}+z-3)$ |

Multilevel Clustering RKLT | ${\sum}_{c\in C}m(4{s}^{2}+3s+1)+\frac{32}{3}{s}^{3}+\frac{1}{2}{s}^{2}-\frac{37}{6}s+5+m(3{s}^{2}+s-3)$ |

**Table 6.**Computational cost (in FLOPs) and transform performance (entropy) for different cluster sizes of Multilevel Clustering RKLT. Transform performance results are not provided when ${2}^{0}$, ${2}^{1}$, ${2}^{2}$, and ${2}^{3}$ clusters are defined in the first level. For these cases, applying the spectral transform would require several days due to the high computational cost, which results impractical in a real scenario.

Number of Clusters Defined in the First Level | Cluster Size | Total Number of Clusters | FLOPs | Entropy |
---|---|---|---|---|

${2}^{0}$ | ${2}^{13}$ | 1 | $4.90\times {10}^{13}$ | - |

${2}^{1}$ | ${2}^{12}$ | 3 | $3.45\times {10}^{13}$ | - |

${2}^{2}$ | ${2}^{11}$ | 7 | $1.95\times {10}^{13}$ | - |

${2}^{3}$ | ${2}^{10}$ | 15 | $1.03\times {10}^{13}$ | - |

${2}^{4}$ | ${2}^{9}$ | 31 | $5.27\times {10}^{12}$ | $5.20$ |

${2}^{5}$ | ${2}^{8}$ | 63 | $2.67\times {10}^{12}$ | $5.14$ |

${2}^{6}$ | ${2}^{7}$ | 127 | $1.35\times {10}^{12}$ | $5.14$ |

${2}^{7}$ | ${2}^{6}$ | 255 | $6.78\times {10}^{11}$ | $5.13$ |

${2}^{8}$ | ${2}^{5}$ | 511 | $3.42\times {10}^{11}$ | $5.13$ |

${2}^{9}$ | ${2}^{4}$ | 1023 | $1.74\times {10}^{11}$ | $5.17$ |

${2}^{10}$ | ${2}^{3}$ | 2047 | $8.98\times {10}^{10}$ | $5.25$ |

${2}^{11}$ | ${2}^{2}$ | 4095 | $4.74\times {10}^{10}$ | $5.48$ |

${2}^{12}$ | ${2}^{1}$ | 8191 | $2.56\times {10}^{10}$ | $5.94$ |

**Table 7.**IASI L1C products used in the experiments. Sizes and averaged zero-order entropies per instrument are provided (48 orbits per instrument). M is the number of spectral channels, Ns is the number of scan lines, N-FORs is the number of FORs per line, and N-IFOVs is the number of IFOVs per FOR.

Instrument | Size (M × Ns × N-FORs × N-IFOVs) | Average Entropy |
---|---|---|

IASI-A Products | 8461 × (630-787) × 30 × 4 | 12.84 |

IASI-B Products | 8461 × (742-788) × 30 × 4 | 12.83 |

Average | 8461 × (761) × 30 × 4 | 12.83 |

**Table 8.**Lossless compression of IASI L1C products. Results are reported in compression ratio (higher is better). Percent savings (higher is better) with respect to original technique are provided within brackets.

IASI-A—Lossless Compression Ratio & Percent Savings | ||||||

Tra. | No Transform | IWT | RPOT | RWA | Multilevel Clustering RKLT | |

Tech | ||||||

JPEG-LS | 1.78:1 | 2.26:1 (21.24%) | 2.26:1 (21.24%) | 2.44:1 (27.05%) | 2.46:1 (27.64%) | |

JPEG 2000 | 1.73:1 | 2.24:1 (22.77%) | 2.24:1 (22.77%) | 2.43:1 (28.81%) | 2.47:1 (29.96%) | |

M-CALIC | 2.32:1 | 2.32:1 (0.00%) | 2.34:1 (0.85%) | 2.48:1 (6.45%) | 2.54:1 (8.66%) | |

CCSDS-122.0 | 1.68:1 | 2.13:1 (21.13%) | 2.13:1 (21.13%) | 2.29:1 (26.64%) | 2.33:1 (27.90%) | |

CCSDS-123.0 | 2.42:1 | 2.42:1 (0.00%) | 2.39:1 (−1.24%) | 2.46:1 (1.63%) | 2.47:1 (2.02%) | |

HEVC | 2.23:1 | 2.29:1 (2.62%) | 2.28:1 (2.19%) | 2.45:1 (8.98) | 2.50:1 (10.80%) | |

IASI-B—Lossless Compression Ratio & Percent Savings | ||||||

Tra. | No Transform | IWT | RPOT | RWA | Multilevel Clustering RKLT | |

Tech | ||||||

JPEG-LS | 1.79:1 | 2.28:1 (21.49%) | 2.27:1 (21.15%) | 2.45:1 (26.94%) | 2.48:1 (27.82%) | |

JPEG 2000 | 1.74:1 | 2.25:1 (22.67%) | 2.25:1 (22.67%) | 2.44:1 (28.69%) | 2.49:1 (30.12%) | |

M-CALIC | 2.34:1 | 2.33:1 (−0.43%) | 2.35:1 (0.43%) | 2.50:1 (6.40%) | 2.56:1 (8.59%) | |

CCSDS-122.0 | 1.69:1 | 2.14:1 (21.03%) | 2.14:1 (21.03%) | 2.30:1 (26.52%) | 2.34:1 (27.78%) | |

CCSDS-123.0 | 2.44:1 | 2.44:1 (0.00%) | 2.40:1 (−1.64%) | 2.48:1 (1.61%) | 2.48:1 (1.61%) | |

HEVC | 2.24:1 | 2.30:1 (2.61%) | 2.29:1 (2.18%) | 2.47:1 (9.31%) | 2.52:1 (11.11%) |

**Table 9.**Near-lossless compression of IASI L1C products. Results are reported in compression ratio (higher is better). Results for lossless compression (PAE = 0) are included. Percent savings (higher is better) with respect to lossless compression are provided within brackets.

IASI-A | IASI-B | |||
---|---|---|---|---|

PAE | JPEG-LS | M-CALIC | JPEG-LS | M-CALIC |

0 | 1.78 | 2.32 | 1.79 | 2.34 |

1 | 2.17 (17.97%) | 3.02 (23.18%) | 2.18 (17.89%) | 3.05 (23.28%) |

3 | 2.60 (31.54%) | 3.90 (40.51%) | 2.61 (31.42%) | 3.95 (40.76%) |

7 | 3.15 (43.49%) | 5.21 (55.47%) | 3.18 (43.71%) | 5.28 (55.68%) |

15 | 3.93 (54.71%) | 7.34 (68.39%) | 3.98 (55.03%) | 7.48 (68.72%) |

31 | 5.11 (65.17%) | 11.11 (79.18%) | 5.18 (65.44%) | 11.35 (79.38%) |

63 | 6.99 (74.54%) | 18.39 (87.38%) | 7.08 (74.72%) | 18.82 (87.57%) |

127 | 10.00 (82.20%) | 33.33 (93.03%) | 10.19 (82.43%) | 34.04 (93.13%) |

255 | 15.09 (88.20%) | 61.54 (96.23%) | 15.38 (88.36%) | 64.00 (96.34%) |

**Table 10.**Compression and decompression runtimes for the coding schemes that produce the best performance for lossless, near-lossless, and lossy compression. The PAE employed for near-lossless compression is 1. The target bit-rate used for lossy compression is 2 bpppc. All times are expressed in minutes.

Runtimes (in Minutes) | Lossless | Near-Lossless | Lossy |
---|---|---|---|

Compression | 81.7 | 15 | 13.4 |

Decompression | 41.4 | 11.3 | 6.2 |

PCC | M-CALIC | Multilevel Clustering KLT + JPEG 2000 | ||
---|---|---|---|---|

Compression ratio | PC scores | PAE | Target bit-rate | |

Experiment 1 | 9:1 | 200 | 19 | 1.78 |

Experiment 2 | 12:1 | 150 | 29 | 1.33 |

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

García-Sobrino, J.; Serra-Sagristà, J.; Bartrina-Rapesta, J. Hyperspectral IASI L1C Data Compression. *Sensors* **2017**, *17*, 1404.
https://doi.org/10.3390/s17061404

**AMA Style**

García-Sobrino J, Serra-Sagristà J, Bartrina-Rapesta J. Hyperspectral IASI L1C Data Compression. *Sensors*. 2017; 17(6):1404.
https://doi.org/10.3390/s17061404

**Chicago/Turabian Style**

García-Sobrino, Joaquín, Joan Serra-Sagristà, and Joan Bartrina-Rapesta. 2017. "Hyperspectral IASI L1C Data Compression" *Sensors* 17, no. 6: 1404.
https://doi.org/10.3390/s17061404