Reconstruction Methods in Oceanographic Satellite Data Observation—A Survey

Abstract

Oceanographic parameters, such as sea surface temperature, surface chlorophyll-a concentration, sea surface ice concentration, sea surface height, etc., are listed as Essential Climate Variables. Therefore, there is a crucial need for persistent and accurate measurements on a global scale. While in situ methods tend to be accurate and continuous, these qualities are difficult to scale spatially, leaving a significant portion of Earth’s oceans and seas unmonitored. To tackle this, various remote sensing techniques have been developed. One of the more prominent ways to measure the aforementioned parameters is via satellite spacecraft-mounted remote sensors. This way, spatial coverage is considerably increased while retaining significant accuracy and resolution. Unfortunately, due to the nature of electromagnetic signals, the atmosphere itself and its content (such as clouds, rain, etc.) frequently obstruct the signals, preventing the satellite-mounted sensors from measuring, resulting in gaps—missing data—in satellite recordings. One way to deal with these gaps is via various reconstruction methods developed through the past two decades. However, there seems to be a lack of review papers on reconstruction methods for satellite-derived oceanographic variables. To rectify the lack, this paper surveyed more than 130 articles dealing with the issue of data reconstruction. Articles were chosen according to two criteria: (a) the article has to feature satellite-derived oceanographic data (b) gaps in satellite data have to be reconstructed. As an additional result of the survey, a novel categorising system based on the type of input data and the usage of time series in reconstruction efforts is proposed.

1. Introduction

Oceanographic variables represent a significant subsection of Essential Climate Variables (ECV) [1,2,3]. Similar to all other ECVs, if oceanographic variables are to be used for analysis and forecasting purposes, persistent measuring and monitoring is required, both locally and globally. Generally, measuring methods can be divided into two distinct categories: in situ methods and remote sensing methods [4]. Depending on the specific measurement needs, one or the other method provides a better fit. For example, in situ methods, despite being difficult to perform accurately, are vastly superior compared to remote sensing methods when dealing with localised measurements, allowing for maximum accuracy and significantly higher spatial and temporal resolution [4]. However, scaling the in situ measurement coverage both spatially and temporally, in most cases, is exponentially difficult in regard to financial costs and logistical planning. This issue becomes even more prominent when one’s aim is to achieve global coverage. This issue can be rectified by implementing remote sensing methods. Remote sensing encompasses measuring techniques based on analysing the electromagnetic signal retrieved from a target location [5]. Remote sensors are particularly useful when mounted on satellites [4,6], allowing for vast spatial coverage with relatively high temporal resolution.

As mentioned, the crux of remote sensing is the analysis of the detected electromagnetic signal. Depending on the target variable, a certain range of the electromagnetic spectrum is considered [4,6]. Furthermore, depending on the mentioned range, the measurement is carried out either actively or passively. Active measurement implies emission of the signal and detection of the subsequent reflected signal. Examples of active measurement include sea surface height and sea roughness. When measuring the two, a signal from the microwave range of the spectrum is emitted. The travel time and strength of the backscattered are measured and used as a starting point for the derivation of the aforementioned variables [4,6].

Not all oceanographic variables can be detected using the active measurement. Some of them rely on passive measurements: meaning the detection of naturally emitted and/or reflected signals (sunlight) from the sea. This infers that different parts of the electromagnetic spectrum are considered: visible light for ocean colour (and subsequently the concentration of chlorophyll-a suspended particulate matter, organic and inorganic carbon, coloured dissolved matter, etc.) [7,8] and infrared for sea surface temperature [9,10]. This, however, poses a problem, especially when considering satellite-based remote sensing because not all parts of the electromagnetic spectrum contain the same penetrative properties [11]. While the Earth’s atmosphere is generally transparent for some wavelengths (such as microwaves), other parts of the spectrum can be significantly attenuated [11]. Due to non-penetrative nature of visible and infrared light cloud cover significantly diminished the area from which signals can be retrieved [12,13,14,15]. Other factors, such as unfavourable sun glint and high aerosol loading [16,17,18], further diminish spatial coverage. Even successful measurements are subject to removal in efforts of quality control [19,20,21,22,23]. All these negative influences result in gaps in satellite data, which hinders its usefulness.

Reducing the occurrences of gaps in satellite data are the next natural step. While improvements to the sensors [24,25] might reduce these gaps in future measurements, it will not help remove the gaps in already obtained data. Therefore, in the past two decades, significant efforts have been poured into various data-driven reconstruction techniques. By analysing existing available data, these techniques aim to infer the values of missing data in an effort to create complete, gap-free datasets. This paper provides a review of these methods when applied to oceanographic data. Among the data, sea surface temperature (SST) and surface chlorophyll-a concentration (${\mathrm{chl}}_{\mathrm{a}}$) are significantly more prominent than the rest. Because some reconstruction approaches may simultaneously utilise data at various processing stages, next, Section 2 introduces the handling and derivation of satellite data, from raw data to final product. Section 3 briefly summarises the most popular gap-filling techniques used in satellite oceanography. Finally, Section 4 showcases examples of those techniques being applied in literature throughout the years.

2. Satellite Data

This section will briefly guide the reader through the nuances of what satellite data actually represents, how it is gathered and processed—from raw electromagnetic signals to mapped geophysical variables. In order not to digress too far from the main idea of the paper, only ${\mathrm{chl}}_{\mathrm{a}}$ and SST products will be explored and explained.

2.1. Levels of Data

Obtaining measurements via satellites is an expensive and complex process—from the initial measurements, through signal conversion to the final data acquisition. While the exact nuances of this process may differ from one organisation to the other [26,27], the backbone procedure is mostly the same. Specifically, this subsection will shed light on the process implemented by NASA [26]. Depending on the process steps applied to the satellite data, the data are separated into specific stages called levels.

The first stage, level 0, refers to the initial content and format of data retrieved from the satellite after downloading the files to data cloud; this includes anything and everything that the satellite generates. These data have not been altered in any way, shape or form. At this level, data are not usable for any scientific purposes, but the data are always saved as it can be used for recreating any subsequent data level in case of data loss. Additionally, the data are stored in specific packets, referred to as the Consultative Committee for Space Data Systems (CCSDS) packets [28].

Level 1 is usually subdivided into two sub-levels. Level 1A combines the CCSDS packets and restructures the data in units reflecting the way the sensor measures: for example, if the sensor in question is a scanner, then the unit of storage/structure is the set of pixels detected during that scan. Apart from measurement data, satellite navigation and other telemetry data are acquired. The final step of level 1A separates data into timed chunks (e.g., 5-min chunks), as a means of data size management [28]. Level 1B calibrates and geolocates level 1A data. Level 1A data comes in the form of integer counts. Calibration converts these counts into physical units, such as radiance/reflectance and brightness temperature. During calibration, instrument corrections are applied—nonlinear signal responses, temperature effects, sensor degradation and other instrument effects are all taken into account when considering instrument correction [4]. After the correction has been applied, navigation data of the spacecraft are combined with the geometry model of the instrument to determine the origin location of each measurement. This way, the data are geolocated [28]. Generally speaking, level 1 data are not an overly popular target for gap-filling [29,30,31,32].

Up to this point, satellite data can be viewed as “engineering data”—information on the spacecraft position and viewing angle, effects of instrument imperfections on data and so on. Level 2 data are the first instance of data being formatted into “scientific data”. Level 2 data are the output of various algorithms, some of which will be explored in Section 2.2 and Section 2.3, which take level 1B data as input. These algorithms may require ancillary data—data regarding different parameters, such as atmospheric water vapour, surface pressure, ozone layer thickness, etc., which are retrieved from sources outside the satellite in question itself. Level 2 data also contain quality flags—various parameters put in place to ensure data are within uncertainty standards set by the scientific community [28]. In the context of gap-filling, level 2 data are utilised for local/regional instances [33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57].

After level 2 data have been obtained, all instances of it from a single day can be combined spatially to create composites, creating level 3 data. While these composites usually contain a day’s worth of information, composition can be carried out along the time axis, combining progressively longer time periods (8 days, month, season, year and finally entire duration of the satellite mission) [28]. Naturally, since level 3 data do not have spatial constrains, unlike level 2 data, it should be obvious that it is the most popular choice when dealing with data reconstruction [58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158].

The final step in data processing is the optional level 4. Level 4 data are optional in a sense that level 3 data are already well-suited and are used as-is by the scientific community. This level represents either a new variable type of data that has been derived from previously obtained satellite data (e.g., ocean primary productivity data are derived from ${\mathrm{chl}}_{\mathrm{a}}$ data [26]) or satellite data that have been augmented in some way [26]. This augmentation can be achieved via an increase in spatial resolution [155,159,160,161], or, more popularly as will be showcased in Section 4, via missing data reconstruction. Some of the reviewed articles utilised level 4 data for gap-filling proof-of-concept purposes [162,163,164].

2.2. Surface Chlorophyll—A Concentration

Radiance and reflectance were mentioned in previous the subsection. In order to better grasp the physical characteristics of these variables, they are best explained through the design behind their respective sensors. ${\mathrm{chl}}_{\mathrm{a}}$ is derived from ocean colour—visible light being reflected from the sea towards the satellite. Measuring the intensity of this light can be carried out via a Gershun tube radiometer [165]. The design of the radiometer is displayed in Figure 1. On one end of the tube is a hole of a fixed size which allows light to enter the device. Entering the hole, the light passes through the collection tube, which has internal light baffles. By building the hole and the collection tube to be of specific proportions, along with the baffles, incoming light is filtered according to the incoming angle, denoted $\alpha$. This angle is usually $5^{\circ }$ to ${10}^{\circ }$. If any light enters from an angle $\widehat{\xi }$ greater than $\alpha$, the light is blocked and does not affect the measurement. Knowing $\alpha$, the viewing solid angle of the detector is given by $\Delta \mathsf{\Omega }=2\pi (1-cos\alpha )$ [165]. Passing through the collection tube, light is filtered according to wavelength $\lambda$. This filter allows for a certain bandwidth $\Delta \lambda$ to pass through. In most cases today, the bandwidth is around 10 nm, while hyperspectral instruments may shrink this to just a few nanometers [24,25,165]. Before hitting the photo-sensitive detector located at the end of the tube, light is diffused over an area $\Delta A$. Knowing the properties of the light that is landing on the detector, radiance L is defined as [165]:

$$L\left(t,\widehat{\xi },\lambda \right)=\frac{\Delta Q}{\Delta t\Delta A\Delta \mathsf{\Omega }\Delta \lambda }\left(\mathrm{W}{\mathrm{m}}^{-2}{\mathrm{sr}}^{-1}{\mathrm{nm}}^{-1}\right).$$

(1)

Essentially radiance is the power of the light radiation at a specific wavelength per unit solid angle per unit area. Remove the collection tube dependence on the viewing angle and, instead of radiance, irradiance E is measured [165]:

$$E\left(t,\lambda \right)=\frac{\Delta Q}{\Delta t\Delta A\Delta \lambda }\left(\mathrm{W}{\mathrm{m}}^{-2}{\mathrm{nm}}^{-1}\right).$$

(2)

Depending onthe orientation of the device, the irradiance measured is usually either downwelling $E_d$ or upwelling$E_u$ [165].

Figure 1. Cross-section of the Gershun tube radiometer.

Unfortunately, property of both radiance and irradiance is sensitive to receiving radiation due to sea surface perturbations and, as such, they usually are not implemented individually. However, since they exhibit a certain degree of correlation, their ratio attenuates this sensitivity enough to be usable for scientific purposes [165]. This ratio can be defined in multiple ways [165], but the most used one in ocean colour sciences is the so-called remote-sensing reflectance $R_{\mathrm{rs}}$ [165]:

$$R_{\mathrm{rs}}\left(\theta ,\varphi ,\lambda \right)=\frac{L_{\mathrm{w},\mathrm{air}}\left(\theta ,\varphi ,\lambda \right)}{E_{\mathrm{d},\mathrm{air}}\left(\theta ,\varphi ,\lambda \right)}\left({\mathrm{sr}}^{-1}\right),$$

(3)

where the subscript $\mathrm{air}$ denotes evaluation just above the sea surface, and subscript $\mathrm{w}$ denotes water-leaving (or upwelling) radiance. Angles $\theta$ and $\varphi$ denote the upward direction of the light exiting the water [165]. Combining this information with the level division provided in the previous subsection, it is prudent to note how raw radiance and raw irradiance are practically the only level 1 data available. Every variable derived henceforth, including water-leaving radiance $L_{\mathrm{w}}$ and normalised water-leaving radiance $nL$, is considered to be at least level 2 data as some algorithms (e.g., the atmospheric correction) need be applied [26,165].

With radiances and reflectances explained, ${\mathrm{chl}}_{\mathrm{a}}$ algorithms become much more comprehensible. Depending on the data provider, a variety of algorithms may be used, on both global and regional scales [156,166,167,168,169,170,171,172,173,174]. Most of these algorithms follow one of the two general formula shapes, depending on if they estimate ${\mathrm{chl}}_{\mathrm{a}}$ in oligotrophic [168] or eutrophic [166] waters. For example, NASA [26] offers a global weighted blend of both algorithms. The oligotrophic algorithm [168] is given as:

$$\mathrm{CI}=R_{\mathrm{rs}}\left({\lambda }_{\mathrm{green}}\right)-\left[R_{\mathrm{rs}}\left({\lambda }_{\mathrm{blue}}\right)+\frac{{\lambda }_{\mathrm{green}}-{\lambda }_{\mathrm{blue}}}{{\lambda }_{\mathrm{red}}-{\lambda }_{\mathrm{blue}}}\left(R_{\mathrm{rs}}\left({\lambda }_{\mathrm{red}}\right)-R_{\mathrm{rs}}\left({\lambda }_{\mathrm{blue}}\right)\right)\right],$$

(4)

where $\mathrm{CI}$ is the so-called colour-index, and ${\lambda }_{\mathrm{red}}$, ${\lambda }_{\mathrm{green}}$ and ${\lambda }_{\mathrm{blue}}$ are the sensor appropriate wavelengths. $\mathrm{CI}$ is then used to estimate ${\mathrm{chl}}_{\mathrm{a}}$:

$${log}_{10}\left({\mathrm{chl}}_{\mathrm{a}}\right)=a_0+a_1\mathrm{CI}.$$

(5)

For higher concentrations, the eutrophic (or the standard) algorithm is used [166]:

$${log}_{10}\left({\mathrm{chl}}_{\mathrm{a}}\right)=a_0+\sum _{i=1}^4{a}_i{\left({log}_{10}\left(\frac{R_{\mathrm{rs}}^{\mathrm{MAX}}}{R_{\mathrm{rs}}\left({\lambda }_{\mathrm{green}}\right)}\right)\right)}^i.$$

(6)

Here, $R_{\mathrm{rs}}^{\mathrm{MAX}}$ denotes the maximum value of $R_{\mathrm{rs}}$ between several different wavelengths [166]. Exact values of $a_i$ depend on the scale and region where the algorithm is applied [166,170,171,172,174].

${\mathrm{chl}}_{\mathrm{a}}$ has been targeted for gap-filling via various methods since the early 2000s [84,131,137], but the popularity soared in the past several years [30,31,33,34,35,36,37,38,39,40,41,42,43,44,45,46,50,51,52,53,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,85,86,87,88,89,90,91,92,93,132,133,134,135,136,138,148,155,156,157].

2.3. Sea Surface Temperature

Measurement devices developed for SST measurements are similar in design to the ones developed for ${\mathrm{chl}}_{\mathrm{a}}$ [4,6,175]. The only difference between the two is, whereas the first measures visible light and derives radiance (and related variables), SST measuring devices measure other parts of the electromagnetic spectrum and derive brightness temperature $B_{\lambda }$. Measuring SST using remote sensing methods is based on Planck’s law: every body emits a flux of energy, $B_{\lambda }$, proportional to the body’s temperature, T, at a given wavelength $\lambda$. The flux is given by:

$$B_{\lambda }\left(\mathrm{T}\right)=\frac{2h{c}^2{\lambda }^{-5}}{e^{hc/\left(\lambda k\mathrm{T}\right)-1}},$$

(7)

where h is the Planck’s constant, c is the speed of light, and k is Boltzmann’s constant. Naturally, the spectrum of the flux is continuous regarding $\lambda$; however, this does not mean that measuring of sea’s temperature can be achieved at any given wavelength. Generally, three conditions are considered when determining the target wavelength(s). Firstly, the sea should emit a detectable amount of energy given the usual temperature. Next, since the sensors are mounted on satellite flying above the atmosphere, the signal reaching the sensor should be as unaffected by the atmosphere as possible. Finally, measuring the flux at the given wavelength should be technologically possible, not to mention practical [4,175]. All things considered, SST can be measured in the infrared and microwave part of the spectrum [4,175,176]. Both wavelength choices have their advantages and disadvantages: while infrared measurements are more accurate and achieve higher resolution, microwave measurements are more robust in terms of measuring through cloud cover [177,178]. Generally, infrared measurements are more prominent [179,180,181,182,183,184,185,186]. However, the entire infrared spectrum not is used for SST measurement. Typically, infrared sensors are designed to measure between 3–5 $\mathsf{\mu }\mathrm{m}$ and 8–12 $\mathsf{\mu }\mathrm{m}$ [4,187]. These spectra are usually called channels [4]. Assuming sensors are well-calibrated, the next step in SST retrieval is removal of effects of the atmosphere and its contents have on the signal. Simplified, an assumption can be made that the difference between the energy fluxes detected between two wavelengths is related to the temperatures difference between them [4,175,187]:

$${\mathrm{SST}}_{ij}-{\mathrm{T}}_i=f({\mathrm{T}}_i-{\mathrm{T}}_j),$$

(8)

where subscripts i and j refer to the sensor channels. Considering that the atmospheric effects are small in these particular channels, SST can be estimated by a simple linear function [4,180,182,184,185,187,188]:

$${\mathrm{SST}}_{ij}=a_0+a_i{\mathrm{T}}_i+a_j{\mathrm{T}}_j,$$

(9)

where $a_x$ are coefficients derived by regression analysis or in situ comparisons [4,189]. Expanding upon this linear estimation, higher accuracy can be achieved if sensor viewing angle is considered [4,183,184,186,187]:

$${\mathrm{SST}}_{ij}=b_0+b_1{\mathrm{T}}_i+b_2({\mathrm{T}}_i-{\mathrm{T}}_j){\mathrm{SST}}_r+b_3({\mathrm{T}}_i-{\mathrm{T}}_j)(sec\theta -1),$$

(10)

where $b_x$ are the new coefficients, $SS{T}_r$ is the basic linear estimation of SST, and $\theta$ is the sensor zenith angle. Further improvements of the algorithms are mostly derived from the presented ideas. Depending on the specific sensors and product suite, different algorithms may be utilised. For example, NASA’s ocean colour suite [190] offers a variety of SST products obtained by different algorithms [183,186].

In comparison to ${\mathrm{chl}}_{\mathrm{a}}$, SST received slightly less reconstruction attention [43,44,46,47,48,49,52,54,55,56,87,88,89,90,91,94,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,140,141,142,143,144,145,146,147,150,151,152,153,154,162,191,192,193,194]. However, examining the timeline of the development of reconstruction methods, one can notice that, in some cases, SST is the first variable to be targeted with novel methods, sort of as a proof-of-concept. This should not necessarily be surprising, as SST is slightly more trivial to derive than ${\mathrm{chl}}_{\mathrm{a}}$, is significantly less volatile than ${\mathrm{chl}}_{\mathrm{a}}$, and microwave SST is significantly more impervious to atmospherically-induced gaps.

2.4. Other Satellite Derived Oceanographic Variables

While ${\mathrm{chl}}_{\mathrm{a}}$ and SST are the most popular oceanographic variables, other types of variables are also available and have also been targeted for reconstruction. These include, but are not limited to: sea surface height/sea level anomaly (SSH/SLA) [100,101,102,103,104,139,158] sea surface salinity (SSS) [29,33,54], total suspended matter/suspended particulate matter concentration (TSM/SPM) [45,52,130,148,195], photosynthetically active radiation (PAR) [86,87], diffuse attenuation coefficient ($K_{\mathrm{d}}$) [30,40,148,149], phytoplankton and particulate concentration ($a_{\mathrm{ph}}$, $b_{\mathrm{bp}}$) [53,83,136] and so on [31,32,41,42,96,99,129,163,164].

3. Reconstruction Methods

Before delving into the details of various reconstruction methods, it is prudent to define several categories of reconstruction approaches to better make sense of similarities and dissimilarities between the approaches. Our findings show that it is possible and practical to separate the methods (and the subsequent application of the respective methods in literature) regarding two criteria: type of data used and time-instances of data used. Type of data used refers to whether or not reconstruction of target data has been achieved using additional, relevant, non-target type of data. The additional, non-target type data are referred to as proxy data. Proxy data may improve reconstruction accuracy if there is significant correlation between the target and proxy data. Mathematically, let x be the target data. Then, in a single time-instance of satellite data (a single satellite image), let $r_{\mathrm{a}}$ represent areas of the image containing available data and $r_{\mathrm{m}}$ areas of the image with missing data, so that:

$$r=r_{\mathrm{a}}\oplus r_{\mathrm{m}}$$

(11)

represents the entire area of the image, where ⊕ denotes the operation of spatial composition. Let f be some reconstruction function that, for a given input, outputs the value of the target data in missing areas. Finally, let y be a proxy data. Then, two approaches may be defined:

$$x\left(r_{\mathrm{m}}\right)=f\left(x\left(r_{\mathrm{a}}\right)\right),$$

(12)

and

$$x\left(r_{\mathrm{m}}\right)=f\left(x\left(r_{\mathrm{a}}\right),y(r_{\mathrm{m}}\vee r)\right)$$

(13)

or

$$x\left(r_{\mathrm{m}}\right)=f\left(y(r_{\mathrm{m}}\vee r)\right),$$

(14)

where ∨ denotes the logic operation “or”. Equation (12) represents the non-proxy approach, while both Equations (13) and (14) represent the proxy approach. These approaches may also be called the univariate and multivariate approaches. An example of this distinction between the two is illustrated in Figure 2. For clarity, these examples are limited to one proxy data, but no such limit need exist in reality.

Figure 2. Examples of univariate and multivariate reconstruction approaches. In both cases, ${\mathrm{chl}}_{\mathrm{a}}$ is the target data x. Missing data in this case were caused by rough sea surface. The top row exemplifies the non-proxy approach. The bottom row exemplifies the proxy approach, where SST has been taken as proxy data y. Both examples imply available data have been composited with reconstructed missing data to obtain complete data, $x\left(r\right)=x\left(r_{\mathrm{a}}\right)\oplus x\left(r_{\mathrm{m}}\right)$.

Similarly, data can be reconstructed with or without drawing resources from time series. Let $t_{\mathrm{n}}$ denote the targeted image time-instance. The single time-instance approach is virtually identical to the univariate approach from Equation (12):

$$x(r_{\mathrm{m}},t_{\mathrm{n}})=f\left(x(r_{\mathrm{a}},t_{\mathrm{n}})\right).$$

(15)

However, the approach with multiple time-instances is then given by:

$$\begin{array}{cc}\hfill x(r_{\mathrm{m}},t_{\mathrm{n}})=& f(x(r_{\mathrm{a}},t_{\mathrm{n}}-{\mathrm{q}}_1),x(r_{\mathrm{a}},t_{\mathrm{n}}-({\mathrm{q}}_1-1)),\hfill \\ & \dots ,x(r_{\mathrm{a}},t_{\mathrm{n}}),\dots ,x(r_{\mathrm{a}},t_{\mathrm{n}}+({\mathrm{q}}_2-1)),x(r_{\mathrm{a}},t_{\mathrm{n}}+{\mathrm{q}}_2)),\hfill \end{array}$$

(16)

where $q_1$ and $q_2$ denote the number of considered time-instances prior and after the targeted time-instance. In practical terms, since most satellite data have a temporal resolution of a single day, time-instances usually refer to day; if $t_{\mathrm{n}}$ is 15 July 2014, then $t_{\mathrm{n}-1}$ denotes 14 July 2014, $t_{\mathrm{n}+1}$ denotes 16 July 2014 and so forth. Distinction between the two latter approaches is depicted in Figure 3. If the reconstruction is completed using information from a time series, the reconstruction is simply referred to as multitemporal, but if it draws resources from only one time-instance, it is referred to as unitemporal.

Figure 3. Examples of unitemporal (top row) and multitemporal (bottom row) approaches being used for ${\mathrm{chl}}_{\mathrm{a}}$ reconstruction. The targeted time-instance is $t_{\mathrm{n}}$. Both examples imply available data have been composited with reconstructed missing data to obtain complete data, $x(r,t_{\mathrm{n}})=x(r_{\mathrm{a}},t_{\mathrm{n}})\oplus x(r_{\mathrm{m}},t_{\mathrm{n}})$.

It is important to note how the two criteria are not mutually exclusive: a method may be categorised as any one of the four possible combinations: e.g., simple spatial interpolation of a single ${\mathrm{chl}}_{\mathrm{a}}$ swath would be placed in the Univariate/Unitemporal category, while spatio-temporal interpolation of combined ${\mathrm{chl}}_{\mathrm{a}}$ and SST data would be played in the Multivariate/Multitemporal category. It is also important to point out that these categories are not always rigid. As will be seen in the following subsections, one method might be originally developed for univariate reconstruction and subsequently minimally modified to allow for multivariate reconstruction. Therefore, rather than putting emphasis on sorting the methods themselves into these categories, it is more sensible to focus on sorting the actual reconstruction approaches, meaning the combination of method and input data. To put this categorisation system in place, Table 1 has been added.

Table 1. Reconstruction approaches sorted according to the novel variate-temporal categorisation system.

	Unitemporal	Multitemporal
Univariate	[35,38,41,42,48,51,59,60,61,68, 74,75,76,79,80,84,97,98,99,105,108, 113,114,115,116,117,122,124,126,129,137,138,141,148, 152,153,156,158,163,192]	[30,31,32,34,37,39,40,43,44,45,47,50,51,52,54, 55,57,58,59,63,64,66,68,69,71,72,73,74,76,77,78,79, 81,82,83,85,86,87,91,93,100,101,104,105, 107,108,109,110,111,112,113,117,119,120,121,125,127,128,130,133, 135,139,140,142,143,144,147,148, 150,151,152,154,155,158,162,163,164,193,194,195,121]
Multivariate	[29,36,46,49,53,56,61,62,65,67,94,106, 118,123,131,132,136,145]	[29,64,67,70,88,89,90,92,93,95,96,102,146, 157,195]

The remainder of this section briefly presents the main idea behind the most popular reconstruction methods presented in literature. Naturally, explaining every detail and modification of every existing variation would detract too much from the main idea of this paper. Thus, should the reader wish to inquire more about a particular reconstruction method, they should refer directly to source material.

3.1. EOF/DINEOF

Empirical orthogonal functions (EOF) have been hypothesised to be useful in data interpolation [151]. EOF are calculated using singular value decomposition (SVD) [196]. A matrix X containing observations so that $X_{ij}$ represents the value of the field $f(r,t)$ at location $r_i$ at the moment $t_j$, i.e.:

$$X_{ij}=f(r_i,t_j),$$

(17)

can be written as a succession of n column vectors $x_j$ size of m at moment $t_j$:

$$X=(x_1,x_2,\dots ,x_n).$$

(18)

This matrix may represent a set of satellite images, among other types of data [151]. Then, applying SVD, the u and v eigenvectors (sizes of m and n) are obtained:

$$\begin{array}{cc}\hfill Xv& =\rho u\hfill \\ \hfill X^{*}u& =\rho v,\hfill \end{array}$$

(19)

where $\rho$ is the singular value, and $X^{*}$ is the adjoint of X. The decomposition is equivalent to:

$$\begin{array}{cc}\hfill X{X}^{*}u& ={\rho }^{2}u\hfill \\ \hfill X^{*}Xv& ={\rho }^{2}v.\hfill \end{array}$$

(20)

By decomposing this way, u and v can be thought of as the eigenvectors of time-averaged covariance matrix $X{X}^{*}$ and the spatially-averaged covariance matrix $X^{*}X$ [151]. Since the eigenvectors are normalised, initial matrix X can be decomposed into:

$$X=UD{V}^{*}=\sum _{k=1}^q{\rho }_k{u}_k{v}_k^{*},$$

(21)

where U and V are the matrices composed of columns of $u_i$ and $v_i$, and D is the diagonal matrix so that $D_{ij}={\rho }_i{\Delta }_{ij}$, where ${\Delta }_{ij}$ is the Kronecker symbol. SVD allows for accurate approximation of X by using just the first k dominant eigenvectors instead of the complete set of q eigenvectors [197]. SVD is only applicable on complete matrices—datasets that have no missing values. This, as stated before, is not a given in the domain of satellite oceanography. Since EOF aims to describe the signal, combinations of dominant EOF might be used to estimate missing values [151]. In order to apply EOF for this purpose, a certain setup is needed. Firstly, let I denote the set of missing data, containing $n_0$ missing data. $X_t$ is the reconstructed dataset, $X_o$ is the observed dataset with missing data, and $X_a$ is the dataset with missing data filled in. $X_o$ is obtained by setting the missing values $(i,j)$ of I to 0. After that, SVD is applied to $X_o$, so that [151]:

$$UD{V}^{*}=X_o.$$

(22)

Then, using the first N dominant eigenvectors, interpolated values of $X_a$ can be obtained [151]:

$${\left(X_a\right)}_{ij}={\left(U_N{D}_N{V}_N^{*}\right)}_{ij}=\sum _{k=1}^N{\rho }_k{\left(u_k\right)}_i{\left(v_k^{*}\right)}_j,(i,j)\in I.$$

(23)

$U_N$ and $V_N^{*}$ consist of N first spatial and temporal EOF. Now, $X_a$ can be written as:

$$X_a=X_0+\Delta X,$$

(24)

where $\Delta X$ is zero for all non-missing data points. This procedure can be repeated again:

$$\begin{array}{cc}\hfill & UD{V}^{*}=X_a\hfill \\ \hfill \left(X_{a1}\right)={\left(U_N{D}_N{V}_N^{*}\right)}_{i,j}& =\sum _{k=1}^N{\rho }_k{\left(u_k\right)}_i{\left(v_k^{*}\right)}_j,(i,j)\in I,\hfill \end{array}$$

(25)

where $X_{a1}$ denotes filled data that have been subjected to mentioned estimation twice. Theoretically, this procedure can be repeated indefinitely. In practice, this procedure goes on until convergence of estimation error based on a cross-validation technique [193]. This method is more popularly known as Data INterpolating Empirical Orthogonal Functions (DINEOF) [193].

DINEOF, its variations and other EOF-based reconstruction methods are arguably the most popular reconstruction methods: its development sparked a boom in data reconstruction, and it has served in many instances as benchmark for reconstruction accuracy comparison [29,30,31,33,34,37,39,40,43,44,45,50,51,52,55,58,59,63,66,69,70,71,72,73,74,77,78,79,81,82,85,87,88,89,90,91,92,93,95,101,102,104,105,107,119,120,125,127,128,130,133,135,142,143,144,148,150,151,157,162,191,193,194,195].

3.2. Support Vector Regression

One of the older methods utilised for dealing with regression problems is Support Vector Regression (SVR) [198]. Support Vectors, as the name suggest, represent data points which serve as a basis for decisions implemented for the rest of the data: from classification to regression, etc. [198,199]. To explain this approach, let $x_n$ be a set of independent variables obtained during N measurements and let $y_n$ be a set of corresponding dependent (or target) variables. SVR then primarily aims to determine the function f which combines them. For a linear problem, this function is expressed as:

$$y_i=f\left(x_i\right),i\in 1,\dots ,N,$$

(26)

so that the error margin between the actual measurements and estimations is $ϵ$. Explicitly, let f be:

$$f\left(x\right)=a\odot x+b,$$

(27)

so that a, x and b represent vectors, and ⊙ represents scalar product. The secondary aim of SVR is to minimise the slope a. This can be achieved through minimising the norm $\parallel a^2\parallel =a\odot a$, so that the optimal hyperplane between the data can be achieved [200]. With these two conditions in place, SVR is essentially an optimisation problem:

• $min\frac{1}{2}\parallel a^2\parallel$ s. t.;
• $|y_i-(a\odot x_i+b)|\le ϵ.$

The formulation assumes both conditions can be fulfilled simultaneously, which may or may not be possible for a given problem [198]. To rectify this, slack variables ${\xi }_i$ and ${\xi }_i^{*}$ are introduced, practically making the error margin $ϵ$ flexible [201]. Visualisation of $ϵ$ and $\xi$ is presented in Figure 4. Optimisation problem now becomes:

• $min\frac{1}{2}\parallel a^2\parallel +C{\sum }_{i=1}^l({\xi }_i+{\xi }_i^{*})$ s. t.;
• $y_i-(a\odot x_i)-b\le ϵ+{\xi }_i;$
• $(a\odot x_i)+b-y_i\le ϵ+{\xi }_i^{*}$ and
• ${\xi }_i,{\xi }_i^{*}\ge 0$.

This is the backbone of SVR [201]. SVR can be applied to nonlinear problems; delinearisation is achieved via Lagrange multipliers [202] and by introducing kernels [198,203,204].

Figure 4. Error margin $ϵ$ in regard to a linear problem. The dotted box represents the slack variable $\xi$, allowing data outside the grey error margin to influence the attributes of function f.

SVR has seldom been used in the context of satellite oceanographic data reconstruction [56,74,84], but it is worth noting in the context of machine-learning-based reconstructions.

3.3. Kriging/Optimal Interpolation

Let X represent a random geophysical variable and its realisation $x_1$ at a random location $s_1$. The space A contains N such realisations $X\left(s_1\right),X\left(s_2\right),\dots ,X\left(s_N\right)$. Furthermore, let $X\left(s_m\right)$ represent the value at a location within A that is not available, effectively a missing data point. Even though these variables may be random, there is some underlying geophysical correlation between them [205,206,207]. At this point, kriging (also known as Optimal interpolation OI) functions if two assumptions can be made [205,206,207]: stationarity and that the correlation between the two variables depends only on the distance between them and not on their actual location. Stationarity implies that any two subspaces of A contain the same statistics (i.e., mean and standard deviation). If those two assumptions are reasonable, then kriging can be written as [205,206,207]:

$$X\left(s_m\right)=\sum _{i=1}^N{w}_{i}X\left(s_i\right),$$

(28)

where $w_i$ is a weight corresponding to variable $X\left(s_i\right)$. To determine the $w_i$, a function called variogram ($\gamma$) is needed. One such variogram is depicted in Figure 5. Formally, variogram can be written as [205,206]:

$$\gamma (s_i,s_j)=\frac{1}{2}{(x_i-x_j)}^2.$$

(29)

Effectively, variogram indicates that the closer two points $s_i$ and $s_j$ are, the difference between their respective variables should be lower and vice versa. Using the variogram, weights $w_i$ can be determined by solving the matrix equation [207]:

$$Tw=R,$$

(30)

or its transpose:

$$w=T^{-1}R,$$

(31)

where w is the vector comprised of $w_i$, T is the matrix comprised of $\gamma (s_i,s_j)$ and R is the matrix consisting of $\gamma (s_m,s_i)$ [207]. Naturally, the estimation may not only be carried out spatially but also temporally [205,206]. Interpolation-based methods are amongst the oldest methods to be utilised in gap-filling in satellite oceanography [100,109] and have remained somewhat popular throughout the years [38,47,54,57,60,64,67,73,76,83,86,99,108,110,111,112,113,117,120,121,132,140,152,153,154,157,158,162,163,164,191,193].

Figure 5. A theoretical variogram. The further two points are, their correlation becomes less and less significant up until a final cutoff value.

3.4. K-Nearest Neighbours

K-nearest neighbours (KNN) is a machine learning regression-solving method [208]. Regression takes form based on the sum of the K closest available data points. The sum can take many different forms, but it is most commonly expressed as a weighted function or as uniform [208]. In case of a weighted sum, increased distance equates to diminished influence. The distance is usually Euclidean, but if KNN is used in a geophysical domain, the distance may be the actual geographical distance [208]. Just like SVR, KNN is not overly popular [67,74,163,164], but as a part of machine-learning reconstruction, it is also noteworthy.

3.5. Random Forest/Decision Tree

Random Forest (RF) stems from the Decision Tree (DT) algorithm. Although primarily used for classification problems, RF exhibits a capacity for regression problems as well [209]. Starting with DT, let ${\overline{x}}_i=\left(x_i^{(1}\right),x_i^{\left(2\right)},\dots ,x_i^{\left(m\right)}$ be a vector or a single point in data space. The vector components $x_i^{\left(j\right)}$ are then its features. DT functions by diving the data space into arbitrary classes according to various conditions based on the data features. With sufficiently enough conditions (so-called nodes), DT effectively divides the data space into enough classes so that each and every outcome of the nodes (so-called leaf) corresponds to an individual data point [209,210]. A simplified DT is depicted in Figure 6. RF is essentially made up of numerous DTs. Decisions made this way are the result of majority of decisions obtained from individual DTs. This improves stability and accuracy [209]. RF solves regression problems by first modelling a forest which accurately connects independent and target variables. Once the RF has been sufficiently trained, RF is provided with available independent variables and is tasked with determining the missing dependent variables [211].

Figure 6. A simple DT for sorting the numbers A, B and C, assuming no two are equal. Since the DT is well defined, every leaf corresponds to possible sorting order.

RF-based (including DT) reconstructions have appeared consistently during the past several years [56,62,67,74,75,106,157,163].

3.6. Self-Organising Maps

Self-Organising Maps (SOM) is an unsupervised learning method typically used in classification problems [212]. However, it may be utilised for data reconstruction purposes [64]. SOM operates on neuron mesh, of a predetermined shape and structure [212]. The idea of SOM is as follows: let M denote a data manifold containing vectors v. This manifold can be described via the aforementioned mesh by moving the neurons (also known as Best Matching Units $BMU$), so that the distance between the individual BMUs and the vectors v $d(v,BM{U}_i)={|v-BM{U}_i|}^2$ is minimal [212]. Depending on the mesh, there are N $BMUs$ available. Should the data vectors v be distributed by the probability distribution $P\left(v\right)$ within the data manifold M, the reconstruction error L is given by:

$$L=\int P\left(v\right)d(v,BM{U}_i)dv.$$

(32)

Effectively, SOM functions by moving the BMUs through data space, reshaping the mesh to fit the manifold as closely as possible. However, since the BMUs are located on a “fixed” mesh, forming a strong correlation between individual $BMU$s, so that moving one $BMU$ affects the neighbouring $BMU$s [212]. The mesh often comes in a form of two-dimensional lattice [212]. Utilising this concept, SOM can be trained to describe typical oceanographic (or any other type of data) data manifolds by learning on complete datasets. Similarly to RF, once the model has been sufficiently trained, SOM can be used to predict the missing values based on available data [64,96,157], but unlike RF, SOM’s popularity is more akin to the popularity of SVR and KNN.

3.7. Neural Networks

Neural networks (NN) are a subsection of machine learning methods with wide applicability. Traditionally, NN are introduced by examining the perceptron model [213]. Perceptron is a function which intakes input variable x and outputs the output variable $f\left(x\right)$. One of the simpler transformations is:

$$f\left(x\right)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}a\odot x+b>0\hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(33)

where a is the weight, and b is bias [213]. After transformation, the output $f\left(x\right)$ is compared to the desired output value y. Depending on this comparison, the weight is updated, and the process repeats itself until the error between $f\left(x\right)$ and y is acceptable [213]. By combining these transformative functions in succession, deep neural networks are constructed [214,215]. The aforementioned functions now act as layers and can be grouped into the input layer, the output layer and the convoluted layer. The way the layers are ordered and organised is customisable, allowing for creation of various architectures of NN, such as Feed-forward NN [216] and Long Short-Term Memory [217]. Among these architectures are also the fairly new Generative Adversarial Networks (GAN) [218] and AutoEncoders (AE) [219].

3.7.1. Generative Adversarial Networks

GAN actually consists of two separate networks: a generating network, or the generator (G), and a discriminating network, or the discriminator (D). The aim of the generator is to generate a distribution $p_g$ to be as similar as possible to the distribution $p_x$ of real data x. For a given input z from distribution $p_z$, the generator generates $G\left(z\right)$ as output. Discriminator, on the other hand, has to determine whether $G\left(z\right)$ originated from $p_x$ or $p_z$. This decision comes in the form of scalar $D\left(G\right(z\left)\right)$. Therefore, the discriminator aims to become as accurate as possible when discriminating between “real” and "generated" data, while the generator aims to generate data as similar as possible to “real” data [218]. Mathematically, this is a min-max optimisation of the so-called loss function $V(G,D)$ [218]:

$$\underset{G}{min}\underset{D}{max}V(D,G)={\mathbb{E}}_{x\sim p_x}[logD\left(x\right)]+{\mathbb{E}}_{z\sim p_z}[log(1-D\left(G\left(z\right)\right))].$$

(34)

3.7.2. AutoEncoders

AE function on the principle of information bottleneck [219,220]. AE consists of three layers: the encoder, the bottleneck and the decoder. The idea behind AE is based on dimensionality reduction: reducing the number of features required for data description. The encoder e is tasked with determining the reduced dimensionality, so that [219,220]:

$$\begin{array}{c}x\in R^n\\ x\to e\left(x\right),e\left(x\right)\in R^m,\mathrm{s}.\mathrm{t}.(m<n).\end{array}$$

(35)

This way, point data x from n-dimensional data space $R^n$ is transcribed into m-dimensional latent data space $R^m$. Effectively, the data are being encoded. Decoding of the data is achieved by the decoder d [219,220]:

$$\begin{array}{c}e\left(x\right)\in R^m\\ e\left(x\right)\to d\left(e\left(x\right)\right),d\left(e\left(x\right)\right)\in R^n,\end{array}$$

(36)

which is nothing more than returning the data back into the original n-dimensional data space $R^n$. Finally, the bottleneck is tasked with further reducing the dimension m, so that the final dimension of the latent data space is lower than needed for direct reconstruction. AE aims to reduce the difference [219,220]:

$$loss=\left|\right|x-{d\left(e\left(x\right)\right)\left|\right|}^2$$

(37)

utilising the noise variable $ϵ$:

$$loss=\left|\right|x-{d\left(e(x+ϵ)\right)\left|\right|}^2.$$

(38)

In terms of data reconstruction, $ϵ$ takes shape of missing data. AutoEncoder is the basis of a popular reconstruction method called DINCAE: Data INterpolating Convolutional AutoEncoder [94,105]. Similarly to DINEOF, DINCAE effectively reduces dimensionality in the input data, but whereas DINEOF relied on EOF decomposition, DINCAE relies on the convolutional layers of the encoder and the bottleneck [105].

NN-based reconstructions have seen a recent rise in popularity [49,56,59,61,65,70,74,94,105,106,115,120,121,122,123,124,134,137,139,147,163,192].

3.8. Data Merge and Other Reconstruction Methods

Apart from the listed methods, there are several other methods useful for reconstruction. Out of these, the most prominent is data merging—merging data from various sources [35,41,42,46,48,51,68,79,97,98,108,114,117,131,148,149,152,158]. Technically, this approach is better referred to as gap-filling, but for the purposes of this paper, data merging was still taken into consideration. The remainder of these methods refers to various numerical models, filters, composites, spatial scaling, etc. [29,32,36,53,80,103,116,118,126,129,136,138,141,145,155,156,157,192].

4. Summary Review of Reconstruction Methods in Literature

Presented in this section are more than 130 articles surveyed in preparation of this paper. During research, papers were selected by two criteria. The first criterion is that the paper has to feature any oceanographical satellite data. As mentioned earlier, these data mostly refer to ${\mathrm{chl}}_{\mathrm{a}}$ and SST, but other types of oceanographic data are also considered. The second criterion is that the gaps in the obtained satellite data have to be filled using any viable method. This criterion includes articles that propose novel gap-filling methods—i.e., development articles, but also any article that fills gaps in the initially retrieved data before further analysing and processing the data in ways that do not necessarily have to deal with the issue of missing data reconstruction—i.e., application articles. Naturally, out of all the surveyed articles, more are classified as application articles than as development articles. The statistics of survey covering the source journals are presented in Appendix A. Considering the number of articles reviewed, it is impractical to thoroughly display all nuances of each and every one of them. Therefore, articles were surveyed in the most concise manner possible, extracting parameters relevant to the majority of the articles. Due to the sheer size of the findings, the resulting Table A1 is placed in Appendix B.

To illustrate the popularity of reconstruction methods throughout the years from Table A1, Figure 7a,b have been added. Methods have been grouped as defined in Section 2. While EOF-based methods are the most popular ones, during the past several years, NN-based (and machine-learning-based in general) methods started to be utilised more consistently and gained certain popularity.

Figure 7. Visualisation of the popularity of the methods. While (a) showcases the aforementioned trend in popularity, (b) displays the overall popularity of methods, with EOF-based methods being the most popular.

Finally, to provide another useful point of view, surveyed articles have been cross-referenced against the three groups of data (${\mathrm{chl}}_{\mathrm{a}}$, SST and other) and the nine groups of methods (EOF-based, SVR, interpolation-based, KNN, RF, SOM, NN, data merge and other). This cross-referencing is displayed in Table 2.

Table 2. Surveyed articles cross-referenced against the target data and utilised reconstruction method.

	${\mathbf{chl}}_{\mathbf{a}}$	SST	Other
EOF	[30,34,37,39,43,44,45, 50,51,52,58,59,63,66, 69,70,71,72,73,74,77,78,79,81,82,85, 87,88,89,90,91,92,93,95,133,135,148,157]	[43,44,52,55, 87,88,89,90,91,92,93, 95,105,107,119,120,125,127, 128,142,143,144,150,151, 162,191,193,194,195]	[29,30,31,33,40,45, 50,52,85,87,93,101, 102,104,130,148,195]
SVR	[74,84]	[56]	[84]
Interp.	[38,47,57,60,64,67,68, 73,76,83,86,132,157]	[47,54,108,109,110,111,112, 113,117,120,121,140,152,153, 154,162,191,193,194]	[54,83,86,99,100,158, 163,164]
KNN	[67,74]	n/a	[163,164]
RF	[62,67,74,75,157]	[56,106]	[163]
SOM	[64,157]	n/a	[96]
NN	[59,61,65,70,74,134, 137]	[49,56,94,105,106,115, 120,121,122,123,124,147,192]	[94,137,139]
Data merge	[35,46,51,68,79,131,148, 149]	[46,48,108,113,114, 117,152]	[41,42,97,98,148,149, 158]
other	[36,53,76,80,92,136,138, 155,156,157]	[92,116,118,126,138, 141,145,146,162,192]	[29,32,53,100,103,129, 136,138]

5. Discussion and Conclusions

Presented in this paper are more than 130 articles dealing with reconstruction of missing satellite oceanographic data. While this number of articles does not represent the entire effort of the scientific community, it does provide a representative insight into gap-filling methods. The survey provided several results. Firstly, ${\mathrm{chl}}_{\mathrm{a}}$ seems to be the lead targeted type of variable in papers considered, with SST following. Other variables are of significantly lesser interest. From a data-level point of view, level 3 data are the most prominent, with individual swaths of data receiving some attention. Reconstruction at level 1 is significantly hindered as atmospheric correction plays a vital role in satellite observations. Regarding method popularity, DINEOF is undoubtedly the most popular reconstruction method: it is clear that reconstruction efforts in general started garnering ever-increasing attention only after the advent of DINEOF. While DINEOF has proven to be the gold standard, as demonstrated by the abundance in the univariate-time-series box of the novel approach categorisation system, the nearer future might witness a shift from EOF-based to machine-learning-based reconstructions. With the ever-growing and diverging sea of reconstruction methods, the proposed categorisation system might become an invaluable navigation tool amidst the chaos.