The On-Orbit Non-Uniformity Correction Method with Modulated Internal Calibration Sources for Infrared Remote Sensing Systems

Abstract

The scanning infrared focal plane array (IRFPA) suffers from stripe-like non-uniformity due to the usage of many detectors, especially when working with a large time scale. Typical calibration systems tend to block the sensor aperture and expose the detectors to an on-board blackbody calibration source. They may also point at deep space. Full aperture calibration sources of this type tend to be large and expensive. To address these problems, a dynamic non-uniformity correction (NUC) method is proposed based on a modulated internal calibration device. By employing the on-board calibration device to generate a dynamic scene and fully integrating the system characteristics of the scanning IRFPA into the scene-based non-uniformity correction (SBNUC) algorithm, on-orbit high dynamic range NUC is achieved without blocking the field of view. Here we simulate an internal calibration system alternative, where a dynamic calibration signal is superimposed on the normal imagery, thus requiring no mechanisms and a smaller size. This method using this type of calibrator shows that when the sensor is pointing at deep space for calibration, it provides an effective non-uniformity correction of the imagery. After performing the proposed method, the NU of the two evaluation images was reduced from the initial 12.99% and 8.72% to less than 2%. Compared to other on-board NUC methods that require an extended reference blackbody source, this proposed approach has the advantages of miniaturization, a short calibration time, and strong adaptability.

1. Introduction

The infrared focal plane array (IRFPA) has wide-ranging applications in the areas of remote sensing, security monitoring, electronic surveillance, and scientific research, among other fields. The linear time-delay integral IRFPA (TDI-IRFPA), a typical configuration, is suitable for space-based remote sensing applications. This array can perform on-orbit search imaging at a fixed angular velocity and can meet the needs of not only required observation frequency, scanning range, resolution, and the capacity for high-dynamic-range (HDR) imaging, but also other requirements of the infrared remote sensing system. The TDI-IRFPA has orientated a widely implemented infrared sensor in orbital spaceflight. The Spatial Infrared Imaging Telescope III (SPIRIT III) carried by the Midcourse Space Experiment (MSX) uses five 192 × 8 focal plane arrays and covers the observation area [1,2]. The Space Based Infrared Warning System-High Orbit (SBIRS-HIGH) system uses a more complex focal plane design. The system uses techniques including multiple operating bands, time-delayed integration, oversampling, large dynamic measurement ranges, and interaction with staring sensors [3]. However, infrared sensors have a common and predominant error source known as non-uniformity (NU) or fixed-pattern noise (FPN) [4]. Variability during manufacturing in combination with thermal and electronic variation during operations triggers a non-uniform pixel response involving a pattern that appears across the focal plane [5]. Characterization of a sensor’s radiometric domain is key to understanding how well the sensor responds on an absolute and/or relative scale [6]. This NU also changes slowly with factors such as the operating time, ambient temperature, shock, and space radiation, particularly in the HDR sensor of the infrared remote sensing system, in which there are significant nonlinear response properties, thus making its NU more challenging to correct. Therefore, the IRFPA non-uniformity correction (NUC) is an important processing step for infrared remote sensing. In addition, the sensors must be periodically re-calibrated on-orbit to obtain high-quality imaging [7].

The non-uniformity correction (i.e., relative radiometric calibration) can be classified into two major categories: calibration-based non-uniformity correction (CBNUC) and scene-based non-uniformity correction (SBNUC) [8]. A typical CBNUC method uses a uniform on-board source (e.g., a blackbody) to block the input aperture or switch the sensor’s optical path to the blackbody [9]. The uniform radiance of the blackbody is radiated to the IRFPA, the gain and offset parameters of the linear model are obtained by changing the temperature of the source, and then the NUC parameters of the IRFPA are obtained. The typical calibration models from the literature are as follows: the one-point method, the two-point method [10], the multi-point method [11,12] and the staircase method [13]. This type of CBNUC method has a prominent effect and minimal algorithmic complexity, but is challenging to adapt to an application system that cannot block the input aperture or switch the optical path. In particular, the high-earth-orbit infrared remote sensing system operating in the narrower non-atmospheric window band can achieve a dynamic range of up to 3–4 orders of magnitude. Furthermore, due to the generally larger aperture, it is impractical to calibrate the entire optical system with a blackbody of the same size as the primary lens given the limited space and weight restrictions. Coupled with power consumption and other restrictions, the CBNUC method based on the extended source (i.e., external calibration) does not apply here. For the internal calibration scheme, it is sometimes challenging to effectively separate the calibration optical paths from the image capturing optical paths in the on-orbit optical system, and the internal calibration sources (ICS) alone results in NU and other unfavorable effects.

The scene-based non-uniformity correction is a class of algorithms designed to compute the NU pattern from infrared video data without the use of a reference source and to remove it [14]. SBNUC algorithms can realize adaptive dynamic NUC using an image sequence and by relying on the motion between frames. The scene-based non-uniformity correction methods are the principal direction of NUC research at present, existing algorithms such as constant-statistical algorithms [15,16,17,18], least-mean-square algorithms [19], neural network algorithms [20] and registration-based algorithms [5,21]. Since SBNUC algorithms estimate the true scene irradiance based on thousands of image frames or the frame-to-frame global motion, which typically have high requirements regarding the dynamic scene and even the local motion characteristics between scenes. During the calibration procedures, the scene of a high-earth-orbit infrared remote sensing system is a deep-space background, and it may be impossible for the system to find a dynamic scene or the scene’s motion characteristics that meet the SBNUC algorithm’s assumption. In addition, there are relatively brighter stars in the scene, and it is challenging for the system to find a uniformly distributed scene. These defects all pose severe challenges for the implementation of HDR SBNUC methods.

To meet the mission’s/sensor’s long-term repeatability requirements, the calibration sources are application specific and included as part of the sensor design process. In a Schmidt telescope program, an internal calibration unit (ICU) for infrared sensors was designed [22,23,24]. The internal calibration unit makes full use of the hole in the main mirror within the central obscuration of the Schmidt telescope and forms an internal calibration reference source. Radiation from nichrome ribbons is projected through the optical assembly. The optical assembly is designed to operate synergistically with the telescope’s optics to produce a uniform and stable illumination at the focal plane. Due to the ICU not being an external source, it first serves to convey the calibration from the limited focal plane region viewed by a point source to the remainder of the focal plane array (FPA) and then achieves the HDR NU correction. This approach constitutes a compact design of an ICS, however without blocking the entrance pupil, the scene imaging is concurrently in the calibration optical path during the calibration process. It is difficult to implement the CBNUC methods because of this interference. Therefore, the calibration process must rely on the idea of the SBNUC. Here, we propose an on-orbit NUC method for the HDR infrared remote sensing system based on the modulated internal calibration source and space scenes (i.e., the MICS-NUC method). To construct a dynamic HDR scene to satisfy the SBNUC algorithm’s assumptions, we superimpose the dynamic radiance of the ICS onto the space scene. Then, an SBNUC method can be realized for the on-orbit HDR infrared remote sensing system. This paper describes the MICS-NUC method and the simulation of the method.

2. The Infrared Remote Sensing system and Internal Calibration Device

The high-earth-orbit infrared remote sensing system is an infrared remote sensing system operating in the narrower non-atmospheric window band. Many differences exist between the typical earth observation remote sensing system and the high-earth-orbit infrared remote sensing system, such as the type of the detected target, the operating band, and the structure of the optical system, among other factors. Thus, these applications have many special features in on-orbit calibration.

Figure 1 shows a proposed radiometric calibration scheme based on the ICS. The on-board ICS is designed to adjust and control the different grades of radiance. The calibration optics are designed to operate synergistically with the telescope’s optics to produce a uniform and HDR radiance at the IRFPA. In addition, the illumination is combined with the relatively small dynamic space scene radiance acquired through the telescope’s optical path, and these are both projected onto the IRFPA during the calibration process. Thus, a large dynamic range of radiant energy is obtained on the IRFPA during the on-orbit radiometric calibration, which provides the basic radiation for realizing the NUC for each grade of radiance.

Our internal calibration device design employed a dual-band linear IRFPA. The number of pixels is 2 × 4000 × 12 for 4000-pixel push-broom imaging. The calibration projection area of the IRFPA is 144 mm × 8.6 mm at a central wavelength of short-wave infrared 2.8 μm and medium-wave infrared 4.3 μm. During the calibration procedures, the irradiances at the image plane are from 1.08 × 10⁻⁴ to 8.2 × 10⁻² W/m² in the short-wave infrared bands, and 1.88 × 10⁻³ to 9.2 × 10⁻² W/m² in the medium-wave infrared bands. The range of the irradiance is approximately 2 orders of magnitude. The internal calibration source uses a nichrome Helioworks-EF8530 as the calibration black body (Figure 2) [25]. The internal calibration source can operate in the pulsed (modulated) or steady-state mode and consists of three radiating elements. The radiation of each radiating element passes through the calibration optical path to form a substantially uniform illumination on the IRFPA surface. However, approximately 1.5% of the NU remains [26]. The internal calibration source meets requirements associated with temperature stability, repeatability, high precision, HDR, radiance distribution uniformity and other performance indicators and has a short heating and cooling time, small size, low weight, low power consumption and many other advantages.

The on-board relative radiometric calibration of the infrared remote sensing system achieves the correction of the IRFPA’s response NU to ensure that high-quality infrared images are obtained around several radiation intensity intervals.

3. Non-Uniformity Correction Method with Modulated Internal Calibration Sources

By considering the characteristics of the infrared remote sensing system and its radiometric calibration, we proposed an on-orbit NUC method for the HDR infrared remote sensing system based on the MICS-NUC method. The modulated illumination (e.g., the dynamic output) provided by the ICS in each grade and the acquired calibration starry scene on-board enable us to obtain a large dynamic sequence image during the on-orbit radiometric calibration. The process filters the image outliers (e.g., relatively bright stars) that violate the statistics assumption due to the unblocking large primary lens. The system can meet the requirements of the SBNUC algorithms in image processing and ensure the effective implementation of the MICS-NUC method. For the sensor’s NU and the illumination’s residual NU in the ICS, the MICS algorithm achieves simultaneous correction by calculating the gain and offset parameters, thus enabling a self-adaptive on-board NUC of the IRFPA.

Figure 3 shows the processing flow chart of the MICS-NUC method. In the following parts, we illustrate the main implementation process of the algorithm.

3.1. Space Scenes Superposition with the Modulated Internal Calibration Source Illumination

In the on-orbit radiometric calibration process, the infrared image obtained by the IRFPA has the following characteristics:

• it contains a small number of bright stars,
• it contains several stars of moderate brightness, and
• it contains a large number of dimmer stars that are distributed almost randomly and that fill in the field of view, which together with the deep-space background form a weak undulating background.

At this point, although the space scene required for the adaptive NUC can be obtained, high-brightness stars are not conducive to HDR calibration, and their influence must be reduced. However, the remaining space scene’s dynamic range can change slightly in the process, and it is challenging to directly obtain different levels of radiation. Thus, we cannot achieve HDR radiometric calibration. Therefore, while obtaining the dynamic space scene, it is also necessary to supplement the corresponding radiation from the on-board calibration device and extend the dynamic range.

Using a modulated ICS with various levels of radiation (even one superposed on the space scene), we can obtain an HDR calibration image during the scan of the IRFPA. This image contains three parts:

• Illuminate the on-board calibration device on the IRFPA so that the radiation uniformity can be supplemented to provide an equivalent radiation output.
• Determine the segmented points and baseline of the radiation: According to the system’s high dynamic range and detection accuracy requirements, the IRFPA’s dynamic range (R_min, R_max) is divided into K intervals (R_k−1, R_k) (k = 1, 2, …, K, R₀ = R_min, R_K = R_max). Use the on-board calibration device to provide the reference radiance corresponding to the segmentation interval and obtain the NUC parameters for each segmentation interval. When the IPFPA is in the image capturing process, according to the input radiance from the scene, select the corresponding NUC parameters at each segmentation interval, and complete the corresponding NUC according to the calibration model.
• On-board calibration device modulation output: Within the defined segment, turn on the internal calibration device, set the ICS blackbody to operate in pulse mode, and modulate the reference blackbody in a periodic way (low-to-high or high-to-low changes in the dynamic range of the IRFPA). The device scans across the IRFPA to obtain the periodic changes in the equivalent radiance output (grayscale) on the IRFPA. Finally, a superimposed image of the modulation calibration device and the starry space scene is obtained within the segmentation interval.

This approach shortens the modulation period and facilitates temperature control of the on-board calibration source, thereby improving the calibration accuracy. In addition, this approach reduces the image sequence required by the MICS-NUC algorithm (i.e., it can obtain calibration parameters from a single frame of the space scene) and decreases the time required for radiometric calibration.

3.2. Filter the Image Outliers

The image to be processed by the IRFPA is a radiation-modulated starry image. According to the characteristics of the system (which do not block the entrance pupil), NUC must be performed while the space scene is radiated into the camera. Due to the limitations of the IRFPA’s dynamic range, some bright stars in the deep-space scene do not meet the requirements of the MICS-NUC algorithm and directly affect the estimation of the corrected true value. This fraction of the image points must be filtered out. In addition, the image also contains influencing factors, such as sensor random noise, on-board calibration device illumination NU, and modulation greyscale variation. Therefore, before the calibration parameters are obtained, the filtering method must retain the superimposed modulated radiation in the image while removing the high-irradiance stars and middle-irradiance stars that do not meet the statistical assumptions (such as where the digital number, DN, exceeds the range of the pre-set greyscale segments and other image abnormalities).

The basic idea of the algorithm is as follows. Each point in the image is traversed, a neighborhood S_xy is set for each point, and then the mean and standard deviation of all pixels in the neighborhood are used as local parameters to determine whether the point conforms to the statistical rule. The evaluation method compares the local parameters with the set reasonable thresholds. The point at which the local parameter is less than the threshold is the abnormal point.

The specific steps for using a local threshold-based filtering processing method are as follows. For the scanning of the TDI-IRFPA system, set the way of obtaining an image as the horizontal scanning mode. The neighborhood S_xy is set as the 1 × d area adjacent to the center point, where d = 2r + 1 and r is the number of pixels on both sides of the center point. When the center point is at the border of the image, the outer neighbors are ignored. Let $f(x,y)$ be the modulated space scene acquired by the scanning of the IRFPA, where x = 1, 2, …, M and y = 1, 2, …, N. M is the total number of rows of the scanned image, and N is the total number of columns of the scanned images. Let m_xy and σ_xy denote local parameters. The corresponding local parameter calculation formula at the center point $(x,y)$ is

$$m_{xy}=\frac{1}{d}{\displaystyle \sum _{s=-r}^{r}f(x,y+s)},$$

(1)

$${\sigma }_{xy}={\left(\frac{1}{d}{\displaystyle \sum _{s=-r}^r{\left[f(x,y+s)-m_{xy}\right]}^2}\right)}^{\frac{1}{2}}.$$

(2)

Considering the following predicate, an image $g(x,y)$ with outliers can be obtained by determining the point-by-point local parameters of the image and the threshold,

$$g(x,y)=\{\begin{array}{cc}1,\hfill & \left|f(x,y)-m_{xy}\right|<aAND{\sigma }_{xy}<b\\ f(x,y),\hfill & \mathrm{otherwise}\end{array},$$

(3)

where a and b are the non-negative threshold parameters determined by experiments. In the subsequent process of calculation, the radiance value of the marked point is ignored.

3.3. Determination of Non-Uniformity Correction Parameters in Each Segmentation Interval

Before performing NUC on the IRFPA, it is necessary to analyze the IRFPA’s NU. The internal calibration device is assembled in the infrared remote sensing system. In addition, this configuration also provides the NU of the illumination distribution through the calibration optical system. The NU appears as a low-spatial-frequency FPN with a Gaussian distribution and must be corrected in the NUC algorithm during on-orbit flight. Furthermore, the IRFPA may generate new FPNs according to changes in the operating environment or status, and it is necessary to perform a periodic dynamic NUC. The SBNUC method can meet the requirements. The internal calibration device provides a feasible method for the application of the SBNUC algorithm in the on-orbit radiometric calibration system.

We proposed to obtain the NUC parameters based on the local constant-statistical NUC (LCS-NUC) algorithm. After the scene has been sufficiently random for a certain time, the NU due to the readout electronics is corrected by treating every row of pixels as one channel and normalizing the channel outputs so that each channel produces pixels with the same mean and standard deviation as the median value of the local channels statistics [27,28]. During the IRFPA scan imaging, the modulation output of the internal calibration device and the scene’s radiance change are generated by the same internal standard source. Therefore, after filtering out the image outliers, the mean and standard deviation of the IRFPA naturally meet the requirements of the algorithm for dynamic changes.

According to the IRFPA linear response model, for the linear IRFPA obtained by scanning the size of M × N images, a total of M channels need to be corrected, where N is the number of columns in a scanned image. Assume that the linear IRFPA NUC model is

$$Y(i,j)=G_i\cdot X(i,j)+O_i(i=1,2,\dots ,\mathrm{M},j=1,2,\dots ,\mathrm{N}),$$

(4)

where $G_i$ and $O_i$ are the gain and offset parameters of the i channel, respectively, $X(i,j)$ is original output image for the IRFPA, and $Y(i,j)$ is the output image after NUC for the IRFPA.

First, find the mean and standard deviation for each channel. The statistics for each channel have similar outputs for adjacent channels. The mean and standard deviation of each row of pixels in the image are expressed as

$$\mu (i)=\frac{1}{\mathrm{N}}{\displaystyle \sum _{j=1}^{\mathrm{N}}X(i,j)},\sigma (i)={\left(\frac{1}{\mathrm{N}}{\displaystyle \sum _{j=1}^{\mathrm{N}}{\left[X(i,j)-\mu (i)\right]}^2}\right)}^{\frac{1}{2}}(i=1,2,\dots ,\mathrm{M},j=1,2,\dots ,\mathrm{N}),$$

(5)

where μ(i) and σ(i) are the mean and standard deviation, respectively, during N columns of scanning the IRFPA for each channel.

Perform median filtering on μ(i) and σ(i) as follows:

$$\begin{array}{c}\overline{\mu }(i)=Median\underset{L}{\underbrace{\left[\mu (i-\frac{L-1}{2}),\dots ,\mu (i-1),\mu (i),\mu (i+1),\dots ,\mu (i+\frac{L-1}{2})\right]}},\\ \overline{\sigma }(i)=Median\underset{L}{\underbrace{\left[\sigma (i-\frac{L-1}{2}),\dots ,\sigma (i-1),\sigma (i),\sigma (i+1),\dots ,\sigma (i+\frac{L-1}{2})\right]}},\end{array}$$

(6)

where L (an odd number) is the length of the one-dimensional median filter window and $\overline{\mu }(\overline{\sigma })$ is the median filtered output of μ(σ) by the sliding window. The appropriate L is determined by comparison with the performance index beforehand [28]. Thus, we apply the following correction to obtain the estimate of Y_i:

$$\widehat{Y}(i,j)=\left[\frac{X(i,j)-\mu (i)}{\sigma (i)}\right]\overline{\sigma }(i)+\overline{\mu }(i),$$

(7)

The effective gain ${\widehat{G}}_i$ and offset ${\widehat{O}}_i$ parameter estimates from Equation (7) are given by

$${\widehat{G}}_i=\frac{\sigma (i)}{\overline{\sigma }(i)},{\widehat{O}}_i=\mu (i)-\frac{\sigma (i)}{\overline{\sigma }(i)}\cdot \overline{\mu }(i),$$

(8)

The traditional constant-statistical algorithm is mainly aimed at continuous multi-frame sequences. A single-frame image output is obtained by scanning the linear IRFPA in which each row of pixels is equivalent to the effect of starting the IRFPA consecutive multi-frame sequences, and thus the correction parameters can be obtained within a single frame.

In the process of the IRFPA imaging, the internal calibration device is turned off for imaging and detection. NUC is performed using Equation (4) according to the determined calibration parameters of each interval segment to achieve on-orbit HDR calibration.

4. Verification Test and Results

To verify the processing effectiveness of the MICS-NUC algorithm, the implementation of this algorithm is analyzed and evaluated by simulating the on-board NUC method of the infrared remote sensing system. The verification takes the following steps:

• the system superimposes various influencing factors in the imaging scene based on the star-space scene and generates simulated images for the NUC;
• the system implements the MICS-NUC algorithm on the generated simulated images; and
• we analyze and discuss the NUC results.

4.1. Acquisition of Simulated Images

The simulated NU image is obtained by adding the influencing factors onto the IRFPA during the radiometric calibration process while pointing to a certain area of the space scene. Influencing factors include the ICS illumination NU of a distribution, the sensor FPN, and the sensor random noise.

4.1.1. Space Scenes

Assuming that the infrared remote sensing system points to the calibration space during the radiometric calibration process and that a small amount of star irradiance exists in the acquired calibration region, we select the actual space scene of the Wide-field Infrared Survey Explorer mission (WISE) in the W2 band (4.36 μm) (NASA/IPAC Infrared Science Archive). The coordinates of the celestial sphere where the image is located are 61.1362, −55.2189 J2000 (Figure 4a,d), 63.5123, −54.5296 J2000 (Figure 4b,e), and 72.9068, −40.0313 J2000 (Figure 4c,f).

The upper row and lower row figures in Figure 4 are the results of the linear and logarithmic image compression processing, respectively, of the original space scene. The original space scene is a 32-bit flexible image transport system (FITS) format image with a resolution of 436 × 436.

4.1.2. Influence Factors of Radiometric Calibration

Before the simulation process, each influencing factor is added to the original space scene (the following simulated images are all equivalent 14-bit greyscale images). For the $M$ channels of IRFPA, the output image $X_k(M)$ of each influencing factor is superimposed on the space scene $P_k(M)$ below K intervals.

$$X_k(M)=[a_1(M)+a_2(M)+1]P_k(M)+[o_2(M)+o_3(M)],$$

(9)

where $a_1(M)$ is the illumination NU of the ICS, $a_2(M)$ is the gain NU of the sensor, $o_2(M)$ is the offset NU of the sensor and $o_3(M)$ is the random noise of the sensor.

Assume that $a_1(M)$ follows a Gaussian distribution:

$$a_1(M)=\exp \left(-\frac{(x-\mu )}{2{\sigma }^2}\right),$$

(10)

Set the position parameter as μ = 128 and the scale parameter as σ = 410. As a result, the illumination from the ICS causes the radiance at the edge of the IRFPA to be approximately 0.89 times that at the center of the IRFPA.

When the ICS is in the steady-mode, take the ICS equivalent radiance output at 5000 (for example). The ICS radiance distribution is obtained after the scanning of the IRFPA (see Figure 5a). Assume that the sensor gain’s NU $a_2(M)$ follows a Gaussian random distribution with a mean of 1 and a standard deviation of 0.06. The sensor’s offset NU $o_2(M)$ follows a Gaussian random distribution with a mean of 655 and a standard deviation of 370. The sensor’s random noise $o_3(M)$ conforms to a Gaussian random distribution with a mean of 0 and a standard deviation of 10. According to Equation (9), Figure 5b shows the data obtained by superimposing various influential factors and a steady-mode ICS equivalent output at 5000 using a scanning IRFPA.

Figure 6 shows the image obtained by superimposing various influential factors and a steady-mode ICS equivalent output at 1000, 3000 and 5000. In the figure, the original space scenes are taken from Figure 4.

4.2. Process and Effects

4.2.1. Segment and Superposed with Modulated Radiation

During the radiometric calibration, the scanning IRFPA can obtain images with the space scene superimposed with a modulated-mode ICS output. These images can conform to the assumption of the MICS-NUC algorithm. First, identify the segmentation point of the equivalent ICS output. Then, set the modulated-mode output of the ICS illumination in the circuit hardware design, such as the sine function modulation output. This output is easier to control assuming that the equivalent radiation intensity A is

$$\mathrm{A}={\mathrm{A}}_{base}+{\mathrm{A}}_{step}\left(1+\sin (j\cdot \frac{2\pi }{T})\right)(j=1,2,\cdots \mathrm{N}),$$

(11)

where A_base is the baseline reference amplitude, which is set according to the segmentation interval. A_step, the amplitude of the ICS radiation, is combined with the modulation period T so that the output image satisfies the statistical assumption of the MICS-NUC algorithm.

Set A_step as 800 in the modulation image to complete 2 periods within 436 pixels (N columns). Taking the ICS reference amplitude at 5000 as an example, the obtained modulated data are shown in Figure 7a. As a comparison to the steady-mode ICS in Figure 5b, Figure 7b shows the data obtained by superimposing various influential factors and a modulated-mode ICS equivalent output at 5000 using a scanning IRFPA.

Figure 8 shows the image obtained by the scanning IRFPA after the actual space scene of the corresponding greyscale segment is superimposed on the ICS modulation output (the ICS equivalent reference output A_base is 1000, 3000, and 5000).

4.2.2. Performing Filter Pre-Treatment on the Image

Set the local parameters (the length of the neighborhood d as 9 pixels, a = 30, and b = 100), and perform the filtering processing according to Equation (3). As shown in Figure 9, the outlier in the simulated image Figure 8 is marked. The algorithm can effectively mark most of the stars or nebulas (the white part is an outlier).

4.2.3. Processing of the Algorithm

This non-uniformity correction algorithm can be used on a single-frame image of the scanning IRFPA output, and the calibration parameters can be obtained in a single-frame. The NUC parameters (i.e., gain and offset parameters) can be obtained after the simulated image is processed by the MICS-NUC algorithm.

The space scene is a non-uniform scene. For evaluating and analyzing the calibration results of the MICS-NUC algorithm, we also simulated a series of test images with the ideal uniform output at the IRFPA. The fixed-pattern noise distribution is the same as that of the simulated images. The test image has the same NUC operation; thus, the change of the test image NU also shows the effect of the MICS-NUC algorithm process. Figure 10 shows two test images with the same FPN distribution as the aforementioned simulation images. The equivalent greyscale output of Figure 10a is 2500, and the NU is 12.99%. In Figure 10b, the equivalent greyscale baseline output is 5000, and the NU is 8.72%.

Note that the non-uniformity (NU) is defined as follows:

$$\begin{array}{c}NU=\frac{1}{V_{avg}}\sqrt{\frac{1}{M\times N-(D+H)}{\displaystyle \sum _{i=1}^M{\displaystyle \sum _{j=1}^N(V_{ij}-V_{avg}{)}^2}}},\\ V_{avg}=\frac{1}{M\times N-(D+H)}{\displaystyle \sum _{i=1}^M{\displaystyle \sum _{j=1}^N{V}_{ij}}},\end{array}$$

(12)

where M and N are the number of rows and columns of the IRFPA, respectively; D is the number of dead pixels in the IRFPA, H is the number of overheated pixels in the IRFPA, and V_ij is the corresponding pixel out of row i and column j on the IRFPA. V_avg is the average of all effective pixels’ output on the IRFPA. When calculating the sum and NU of the outputs, the values of the invalid pixels are not included.

The length of the filter kernel function in the MICS-NUC algorithm is determined by comparing the correction effects in advance. Using the actual space scene superimposed with the modulated ICS output (the ICS equivalent greyscale reference output was 1000, 3000, and 5000), the MICS-NUC algorithm was performed. Figure 11 shows the NUC results of the calibration parameters obtained for the test image 10a using the MICS-NUC algorithm with various filter kernel lengths (L = 8 to 100 pixels). By comparing the NU after the NUC, when the ICS equivalent greyscale reference output is 3000 and L = 35 to 45, the NU is smaller, and the NUC at corresponding greyscale segment can be realized. Therefore, we use L = 35 as the length of the filtering kernel function of the MICS-NUC algorithm.

4.3. Non-Uniformity Correction Results

Using the MICS-NUC algorithm, the ICS steady-mode and the ICS modulated-mode outputs were superimposed on the original space scene, and the NUC was applied to the test image of Figure 10 with the same NUC algorithm.

In the ICS steady-mode, first, the original space scene (Figure 4a–c) is superimposed with steady-mode ICS equivalent outputs at 1000, 3000, and 5000, respectively. Secondly, apply the MICS-NUC algorithm to this processed star map to obtain the corresponding three groups of NUC parameters. Finally, the NUC process is performed on the test image based on these correction parameters to obtain a corrected image. The corrected images of Figure 10a with these correction parameters are shown in Figure 12, and the images corrected with the same correction parameters of Figure 10b are shown in Figure 14.

Considering that the ICS radiation is in the modulated-mode, only change the superimposed ICS radiation pattern into the modulation output at 1000, 3000, and 5000, respectively. After the same MICS-NUC algorithm processing, the images corrected with these correction parameters of Figure 10a are shown in Figure 13 and the images corrected with these correction parameters of Figure 10b are shown in Figure 15. As a comparison between Figure 12 and Figure 13, as well as Figure 14 and Figure 15, after the MICS-NUC process, the images achieve better uniformity and the image quality is greatly improved.

5. Discussion

The non-uniformity evaluation results following NUC processing are summarized and shown in

Table 1. Non-uniformity correction results of the evaluation images.

Evaluation Image with Different Digital Number ( DN )	Actual Space Scene Superposed with Internal Calibration Sources (ICS) Output	Equivalent Reference Radiance
		DN = 1000	DN = 3000	DN = 5000
Figure 10a; equivalent radiance DN = 2500; NU = 12.99%	Steady-mode ICS	6.55%	5.05%	19.37%
	Modulated-mode ICS	1.27%	1.06%	1.99%
Figure 10b; equivalent radiance DN = 5000; NU = 8.72%	Steady-mode ICS	12.83%	5.12%	1.90%
	Modulated-mode ICS	0.91%	0.85%	0.79%

. As indicated in Figure 12, Figure 13, Figure 14 and Figure 15, when the superposed ICS output is in steady-mode, the scene cannot be fully randomized, and the requirements of the SBNUC algorithm cannot be satisfied, making the NUC effect not prominent. However, when the original space scene is superimposed on the ICS modulated-mode output, the FPN is effectively corrected. Furthermore, for different evaluation images with different equivalent radiance, different ICS equivalent reference outputs will provide different results in each mode. For instance, in the space scene represented by Figure 10a, the equivalent radiance DN is 2500. For both modes, an equivalent reference output with DN of 3000 provides the best result. The initial NU of 12.99% is reduced to 5.05% and 1.06% for steady-mode and modulated-mode respectively, as shown in Figure 12b and Figure 13b. Similarly, the space scene represented in Figure 10b has an equivalent radiance DN of 5000. An equivalent reference output with DN of 5000 provides the best result. The initial NU of 8.72% is reduced to 1.90% and 0.79% in the steady-mode and modulated-mode respectively.

The NUC results in Table 1 prove that the MICS-NUC method is more accurate in estimating the NUC gain and offset parameters following the modulated ICS radiation. In addition, it shows that the segment correction of the MICS-NUC algorithm in Section 4.2.1 is effectual since the NUC parameters obtained in the selected greyscale have a better correction effect than those from other greyscale segments.

To evaluate the MICS-NUC processing effect on the actual space scene, a WISE W2 band (4.36 μm) image (located at 323.841, 1.48141 J2000) is superimposed on the FPN to generate an original image before correction (see Figure 16a). By the MICS-NUC algorithm implementation (the ICS equivalent output is set to 3000, and the ICS is in modulated-mode), calibration parameters are obtained and subsequently the corrected image is obtained (see Figure 16b). As can be seen from Figure 16, the stripe-like and irregular non-uniform noise in the original image is corrected. Note that intensity transformation is implemented to display the corrected image.

Note that during the MICS-NUC process, only one frame is employed for one group of parameters. The computational load is unquestionably small and the process is straightforward. Furthermore, this approach has various advantages of being highly accurate, highly efficient and self-adaptive. To our best knowledge, the MICS-NUC method, for the first time, fully integrates the system characteristics of a scanning IRFPA into the SBNUC algorithm, providing an effective solution for the NUC of large-aperture high dynamic range on-orbit remote sensing systems. The MICS-NUC algorithm allows the system to periodically implement on-orbit dynamic corrections and update the radiometric calibration parameters, which ensures on-orbit high-quality imaging in a large time scale.

The proposed NUC method in this paper is designed for the remote sensing system operating in the narrower non-atmospheric window band. This kind of scene-based NUC algorithm is also suitable for typical earth imaging thermal remote sensing systems. For a scene that is constantly changing, the correction result should be better. However, the application of different types of scene-based NUC algorithms is conditional to the characteristics of the scene. At present, most scene-based NUC algorithms have been applied to thermal imaging systems with small array sensors. Since most of the scene-based NUC algorithms are highly complex and require real-time image processing performance, the application of these algorithms to space-based platforms with large array detectors is still in the experimental stage.

Regarding the calibration accuracy, there are several ways to improve this, such as further improvement of the internal calibration sources control and optimization of the calibration optics and the NUC algorithm. A comprehensive discussion on the calibration accuracy is beyond the scope of this paper.

6. Conclusions

To solve the problem of infrared remote sensing NUC without blocking the field of view, we have developed a MICS-NUC method. The method implements the algorithm’s steps, including the segmentation, space scene and modulated ICS image acquisition method, the variable threshold function of filtering outliers, the local constant-statistical NUC method and other algorithm implementation steps, to achieve IRFPA on-orbit HDR dynamic self-adaptive NUC. This approach can avoid several problems, such as scene sequence images dissatisfying the algorithm’s assumption under special conditions on the platform and large-aperture telephoto systems that cannot be effectively blocked. The verification with the simulated image shows that the NU of the IRFPA is greatly reduced. Compared to other on-board NUC methods requiring referenced blackbodies (CBNUC methods), this algorithm has the advantages relating to its self-adaptiveness, system miniaturization, and cost savings. We believe the implementation of this calibration method will meet the new high-tech requirements of remote sensing systems in meteorology, defense and plenty of other fields.