Designing a Compressive Sensing Demonstrator of an Earth Observation Payload in the Visible and Medium Infrared: Instrumental Concept and Main Features ^†

Abstract

Increased spatial resolution and revisit time of payloads operating in the infrared spectral region can offer unprecedented advantages to Earth Observation. This, however, poses several technological challenges, such as large array detector availability and data bandwidth. In this paper, we present a super-resolved demonstrator—based on a compressive sensing architecture—which is being developed to address enhanced performance in terms of at-ground spatial resolution, on-board data processing and encryption functionalities for Earth Observation payloads. The demonstrator’s architecture is here presented, together with its working principle, main features and the approach used for image reconstruction.

1. Introduction

Earth Observation (EO) data are playing a key role in addressing societal and environmental challenges; yet the development of novel, improved application products often requires high-resolution imagery, which implies the generation and handling of a large amounts of data and increased requirements on the computation and memory resources available on the onboard processing units. In addition, EO payloads that operate in the medium infrared (MIR) are further limited by technological challenges, especially for the availability of large focal plane detectors. In this context, a Compressed Sensing (CS) paradigm offers several advantages: it enables the design of a novel class of computational imaging instruments based on the use of a Spatial Light Modulator (SLM) and a single element detector; at the same time, it addresses the handling of large amounts of data by providing native compression and encryption, and enabling onboard data analysis at little computational cost.

In this paper, we introduce the design of a novel CS demonstrator of an EO payload that uses the CS paradigm and SLM technology to improve performances in terms of at-ground spatial resolution, on-board data processing and encryption functionalities. Here we present the demonstrator’s architecture, together with its working principle, main features and the approach used for image reconstruction.

2. Compressive Sensing and Relevant Instrumental Concept

CS is a signal processing technique that allows one to acquire a signal in a very compact fashion and reconstruct it by finding suitable solutions to an underdetermined linear system [1].

An underdetermined system of linear equations has more unknowns than equations and generally has an infinite number of solutions (i.e., possible reconstructed images). In order to choose a solution to the system, extra constraints or conditions, as appropriate, must be imposed. In the CS case, the constraint of “sparsity” is added in order to have only solutions with a small number of nonzero coefficients in a suitable signal representation (e.g., a linear transform such as the discrete cosine transform, also employed in the JPEG image compression standard).

In other words, in a classical approach to sampling, N samples are acquired and then, during compression, only K << N significant coefficients are retained. On the other hand, CS is aimed at the acquisition of only the K significant measurements by leveraging on the concept of sparsity. In this respect, CS represents a way of merging the acquisition and compression into a single step.

The architecture of a basic CS instrument (single pixel camera) consists of the following components:

• Collection optics, which is used to collect the light coming from the target;
• Spatial Light Modulator (SLM), which physically implements the scalar product between the coding mask applied to the SLM and the image focused by the collection optics on the SLM plane;
• Condenser, which implements the integration;
• Single-element detector, which is used to acquire the signal.

If the SLM has N pixels, it is possible to reliably reconstruct the original image by acquiring only K measurements, each one corresponding to a different modulation mask coded on the SLM, and by using suitable algorithms at the ground segment.

Following the first prototype implemented at Rice University [2], other CS instruments have been developed by relying on this architecture, although only a few of them addressed space applications [3,4,5].

3. The SURPRISE Demonstrator

The demonstrator here presented, which is under development in the frame of the EU H2020 SURPRISE project [6], is conceived as a whiskbroom spectral imager featuring 10 channels in the visible-near infrared (VIS-NIR) and 2 channels in the MIR at 3.3 μm and 4.0 μm, respectively. The demonstrator will be able to acquire super-resolved images of a generic target placed in the laboratory by using the implementation of a CS architecture. This uses an SLM as a core element to implement a CS architecture inspired to the single pixel camera concept described in the previous section. Figure 1 shows a simplified diagram of the SURPRISE demonstrator’s optical layout.

The scene is scanned along two axes by means of a suitable scanning system. The dimensions of the target (that is the portion of the scene observed by the fore-optics) are 30 mm × 30 mm. The fore-optics provides the image of the target on the image plane field stop at which the SLM is placed. In Figure 1, for the sake of clarity, the SLM is shown working in transmission. The image captured in the demonstrator’s Instantaneous Field Of View (IFOV) corresponds to a ‘macropixel’ on the SLM, which is made of several micromirrors (or binned micromirrors). The macro-pixel is typically modulated at a higher spatial resolution (super-resolution) by a series of coding masks applied to the SLM micromirrors. The demonstrator was designed to ensure a super-resolution factor of at least 4 × 4; this means that the number of the pixels of the reconstructed images will be increased by a factor at least 4 × 4. The maximum super-resolution factor that can be achieved is 32 × 32. Spectral splitting is applied after the SLM-based coding stage by means of dichroic mirrors and is followed by the spatial integration stage implemented by the condensers. The signal is further spectrally filtered (or dispersed by the spectrometer, for the VIS-NIR channels) and acquired in the spectral bands of interest. Finally, the image is reconstructed by applying suitable reconstruction algorithms to the series of (integrated) measurements obtained while applying different coding masks to the SLM. The demonstrator will be completed with an additional block constituted by a high-resolution camera (not shown in Figure 1), which will be used for alignment and validation purposes.

The overall architecture of the SURPRISE demonstrator will include a master unit that guarantees proper synchronization among the different sensors and actuators and provides data handling capabilities. At the beginning of the sequence of CS measurements, the target scanning system must be positioned so that a given portion of the scene (target) is seen by the instrument’s IFOV; secondly, a modulation mask must be set on the SLM; thirdly, acquisition by each detector can be triggered. Once the integration time has run for all the detectors, another mask must be set on the SLM and a new acquisition by the detectors can be triggered. These operations are repeated until the required number of CS measurements is reached. By leveraging on image sparsity, the image can be reconstructed by acquiring a total number of CS measurements smaller than the number of pixels of the reconstructed image, typically with a Compression Ratio (CR) of 50%. At the end of the sequence, the scanning system makes another portion of the scene available to the instrument’s IFOV and the sequence of operations can be run again until all the scene is scanned. The timely sequence of operations for a measurement is under the control of a master unit. This unit manages the time at which an operation is achieved by one of the subsystems. In addition, it collects the data and provides inputs to the SLM, defining the position of the micromirrors on the modulation mask.

4. Image Reconstruction

Most existing reconstruction techniques exploit image sparsity in a given domain (wavelet, discrete cosine transform, gradient domain, and so on) and attempt to perform the reconstruction with different approaches: greedy algorithms, iterative thresholding algorithms, convex relaxation algorithms or non-convex relaxation algorithms (see references in [7]). However, even for moderate-sized images, the reconstruction can still be very slow due to the complexity of the problem. In practice, traditional algorithms are very complex, and performing an exhaustive search of the sparsest solution is not feasible. Several other approaches are possible, such as iterative and greedy algorithms. However, these methods do not achieve excellent results for real images because they are based on rather simple models; moreover, their complexity is typically large.

Deep learning instead can learn a suitable reconstruction method that outperforms traditional techniques. “Deep” refers to the number of cascaded operations between the input (the sensed image data) and the output (the reconstructed image). These types of algorithms are able to learn nonlinear models very well and are increasingly used in any field. In particular, deep learning algorithms are widely used in image processing and they can be used to deal with the CS problem. In the SURPRISE demonstrator, ISTA NET frameworks [8] were used in preliminary experiments for the reconstruction of natural target datasets and simulated hyperspectral images. The comparison of the results obtained with traditional methods at different CRs have highlighted the excellent performance of the deep learning methods. The quality of the reconstructed image has been evaluated using the PSNR, which is a common metric adopted to evaluate the reconstruction quality.

Besides image reconstruction from a series of CS measurements, the demonstrator will also address the feasibility of native encryption and information extraction directly from the acquired CS measurements, enabling onboard data analysis at little computational costs.

5. Conclusions

A super-resolved CS demonstrator of an EO payload has been conceived and designed to ensure a super-resolution factor of at least 4x4, thanks to the use of an SLM as a core modulation element of its architecture. The demonstrator operates in whiskbroom acquisition mode and has 10 channels in the VIS-NIR and 2 channels in the MIR, while its implementation is in progress by using mainly Commercial Off The Shelf (COTS) components, all of them with space-grade counterparts available or under qualification. The CS approach adopted in the design of the demonstrator enables joint sensing and compression, and it paves the way to native encryption and on-board data processing at little computational cost. Experiments conducted on natural target dataset and simulated hyperspectral images have shown that deep learning methods can provide improved accuracy in image reconstruction over traditional methods.