Exploring the Potential of Compressive Sensing and Super-Resolution for Space Applications in the MIR-TIR ^†

Abstract

Earth Observation applications could take advantage from the availability of high spatial resolution data in the MIR-TIR. This, however, poses technological challenges, especially for the availability of large focal plane. Here we discuss two approaches, the compressive sensing and the super-resolution, that could provide a basis for the development of innovative EO instruments working in the MIR-TIR and able to enjoy an augmented resolution without increasing the number of detection elements in the matrix detector. Main pros and cons are analyzed, focusing on simulations done for an instrument combining both approaches to achieve super-resolved images for studying hot events.

1. Introduction

The spectral region of the medium and thermal infrared (MIR-TIR) is regarded as a major source of information for several Earth Observation (EO) applications, from agriculture, food safety and coastal areas, to meteorological and climatic studies and monitoring of natural and man-made hazards. Most of these applications could take great advantage from the availability of very high spatial resolution (VHR) data. On the other hand, the construction of a payload able to provide VHR data raises several issues, especially in this spectral range where detector matrices are very expensive or not available, depending on the required specifications. In addition, the VHR requirements imply also high sensor throughput and the storage and transmission of the acquired data can be challenging, so that a compression board is often needed. In this respect, the use of a Spatial Light Modulator (SLM) can open the way to innovative instrumental concepts based on either Compressive Sensing (CS) single pixel camera or a super-resolution approach.

CS is an innovative signal acquisition technique that benefits from the feature of many natural signals being highly correlated. The main idea of CS is that, with prior knowledge about the signal’s sparsity, the signal can be reconstructed using fewer samples than those dictated by the Nyquist-Shannon theorem. A standard signal compression strategy consists in sampling the data and then compressing them to decrease the final data volume. CS instead aims at reducing the volume of the acquired signal samples by acquiring only a set of spatially integrated measurements of the scene, modulated by a suitable spatial pattern, typically implemented by means of an SLM. Finally, the signal is reconstructed by using suitable algorithms identifying the sparsest signal that matches the available measurements.

While CS can be regarded as a smart approach to merge data acquisition and compression into a single step, its use for the acquisition of images whose resolution—after reconstruction—is increased up to that of the coding pattern is referred to as the super-resolution approach. In general, a super-resolution imaging system features a resolution enhanced with respect to its native one. Many techniques can be used to implement a super-resolution approach: amongst these, Computational Imaging relies on the use of an SLM to code the incoming signal, so that CS can be considered as a limit case of Computational Imaging, where the number of measurements is lower than the number of pixels of the high resolution image to be reconstructed.

In this paper we analyse the main pros and cons of MIR-TIR instruments based on the concepts of CS and super-resolution for space applications, with particular reference to EO applications.

2. CS-Based Approach

The typical architecture of a CS-based instrument is the single-pixel camera configuration (Figure 1): the image produced by the collection optics is modulated at the image plane by an SLM used as a coding mask; the signal transmitted through the SLM is focused by an optical condenser on a single pixel detector. A set of measurements is acquired—one measurement for each modulation pattern applied via the SLM—and the image is reconstructed by means of suitable CS reconstruction algorithms [1].

The main advantages provided from a single-pixel camera configuration relies in the use of single element detector instead of expensive 2D-array detectors. Typically, ratios up to of 1 × 1:512 × 512 can be obtained, to be traded off with integration time needed for the specific application and required computational burden for reconstructing the image.

A pros and cons analysis of CS-based instrumental concepts for different application domains—Planetary Exploration, Space Science and EO—was carried out in the frame of two ESA-funded projects [2,3]. In general, beside the use of a single element detector instead of a 2D-array detector, the use of CS technique can yield a decrease in terms of payload budgets, although to a limited extent. Power consumption and detector throughput can mainly benefit from a CS approach. However, the efficiency of SLM at those wavelengths could be limited due to the dimensions of commercial micro-mirrors, which are comparable with the wavelengths in the considered spectral range. An additional issue that stands specifically for EO applications is posed by the constraints on SLM speed, detector frame rate and integration time that have to match the satellite speed, unless a geostationary platform is chosen [2].

3. Super-Resolution Approach

SLM technology and CS approach can be exploited to yield a significant improvement of the performance of EO super-spectral instruments in the MIR-TIR, especially in terms of their spatial resolution. At-ground spatial resolution of such instruments can in fact be enhanced—without increasing the number of pixels of the detector—by adopting a super-resolution approach. In order to guarantee that the highest spatial frequency present in the image to be acquired will be captured, the dimensions of the single elements of the SLM must be equal to or larger in size than the diffraction-limited spot blur formed by the foreoptics at the image plane.

Increasing the number of detector’s pixels can be very expensive, especially in spectral ranges like MIR. The use of a high number of pixels also yields a more complex optical design. It requires the use either of very large Focal Plane Arrays (FPAs), which are noisy and make the design of the optics difficult or of pixels of smaller dimensions, which raises diffraction-related issues, particularly in the IR. A consequence of these technical issues is that, at present, there is a gap in the production of high spatial resolution—yet with frequent revisit time—IR data. Geostationary platforms seem the best candidates to host payloads exploiting the super-resolution approach: integration time for the acquisition is not strictly fixed by orbital parameters, as it happens for polar orbiting platforms. CS approach can be thus merged with super-resolution, lowering data link requirements. Using minimal on-sensor processing, CS can provide a compressed data stream natively, without requiring an additional compression board. Thus, it has a twofold advantage of reducing bandwidth requirements and simplifying the design of on-board processing unit.

Figure 2 shows the schematic of a super-resolved instrumental concept. The main blocks of the optical section of the instrument are: (1) Acquisition and focusing optics; (2) SLM, on which the acquired image is focused and which encodes it by superimposing a binary spatial mask. This mask is characterized by a high number of pixels, corresponding to the number of pixels of the final image; (3) Concentrator, which constitutes the optical system that projects the image modulated by the SLM on the detector; (4) Matrix detector (with a low number of pixels with respect to the final data), on which (through deposition or masking) the filters for the selection of the different spectral bands are deposited. The matrix structure allows the parallelization of the acquisition process, so a fixed number of pixels on the SLM are focused on a specific (macro) pixel of the detector. Each macro-pixel of the detector operates as a single pixel camera.

For a preliminary assessment of the performance of this instrumental concept, simulations were carried out by calculating the radiance emitted by a black body at different temperatures and the corresponding number of photons per µm (Figure 3a and Figure 3b, respectively). The expected number of photons during a single acquisition made with a super-resolved instrument is reported in Figure 3c and in Figure 3d (logarithmic scale). This preliminary evaluation shows that the signal detection limit is expected above 333 K and that the maximum signal occurs for hot events at temperatures between 500 K and 1000 K. Calculations were made assuming a detector error level of about 1000 electrons per pixel and a quantum efficiency of 50%.

Table 1. Main parameters considered for data simulation.

Parameter	Value
Orbit altitude	700 km
Entrance pupil diameter	100 mm
Integration time	2 ms
Single channel Band width	100 nm
Super-resolution ratio	16
Observation geometry	nadir

shows the main parameters used for the simulation of the data in a temperature range from 500 K to 1100 K.

4. Discussion and Conclusions

In general, CS approach can provide novel solutions for space applications to reduce sensor throughput and payload budgets, especially when transmission and storage of all acquired data are demanding. Although CS- based single-pixel camera concept can be an interesting alternative to the use of array detectors, the super-resolution, possibly combined with CS, can provide better opportunities for EO applications in the MIR-TIR, especially in terms of VHR data. This approach, however, is expected to be a valid alternative only for studying events at high temperatures, where it can additionally provide mitigation of saturation and blooming effects.