Hyperspectral Ghost Image Residual Correction Method Based on PSF Degradation Model

Highlights

• We innovatively construct a three concentric Gaussian PSF degradation model to ac-curately represent the non-ideal energy distribution of the ghosts of hyperspectral images, enabling high-fidelity simulation and correction while preserving spectral calibration accuracy.
• A novel simulated annealing algorithm is proposed to iteratively correct the ghosts across multiple spectral bands, achieving significant ghost energy suppression with accurate spectral precision.

Abstract

In hyperspectral images, ghost image residuals exceeding a certain threshold not only reduce the recognition accuracy of the imaging detection system but also decrease the target identification rate. Ghost image residuals affect both the recognition accuracy of the detection system and the accuracy of spectral calibration, thereby influencing qualitative and quantitative inversion. Conventional ghost image residual correction methods can significantly affect both the relative and absolute calibration accuracy of hyperspectral images. To minimize the impact on spectral calibration accuracy during ghost image residual correction, we propose a ghost image degradation model and an iterative optimization algorithm. In the proposed approach, a ghost image residual degradation model is constructed based on the point spread function (PSF) of ghost image residuals and their energy distribution characteristics. Using the proportion of ghost image residuals and the accuracy of hyperspectral image calibration as constraints, we iteratively optimized typical regional target ghost image residuals across different spectral channels, achieving automated correction of ghost image residuals in various spectral bands. The experimental results show that the energy proportion of ghost image residuals at different wavelengths decreased from 4.6% to 0.3%, the variations in spectral curves before and after correction were less than 0.8%, and the change in absolute radiometric calibration accuracy was below 0.06%.

1. Introduction

Recently, imaging spectrometers have been widely used in various fields, including remote sensing, biomedicine, chemical detection, mineral exploration, environmental monitoring, missile interception, explosion analysis, and combustion diagnostics [1,2,3,4,5]. Ghost image residuals widely exist in optical systems, mainly caused by factors such as light reflection, refraction, and scattering. In other words, abnormal path light rays converge on the CCD target surface, affecting the recognition accuracy and decreasing the target recognition rate of detection imaging systems when ghost image residuals exceed a certain threshold [6,7,8]. In hyperspectral images, ghost image residuals affect not only the recognition accuracy of the detection system but also the spectral calibration accuracy, thereby affecting qualitative and quantitative inversion of material properties [9,10,11,12]. Therefore, it is highly significant to correct ghost image residuals of hyperspectral images. The Jet Propulsion Laboratory in the United States simulated stray light and ghost image residual energy distributions by testing its EMIT system [13,14,15], reducing its energy to 1–5 DN values, and achieving the residual signal of approximately 0.0005 μW·nm·cm⁻²·Sr⁻¹, which is three orders of magnitude lower than the water reflectance signal. Shanghai Institute of Technical Physics, Chinese Academy of Sciences, conducted stray light analysis on the Medium Resolution Imaging Spectroradiometer (MERSI) carried on Fengyun-3D, modeling different surfaces using the bidirectional scatter distribution function (BSDF) for ray tracing on the end-to-end link light. We propose a simulated ghost image degradation model combined with an iterative optimization algorithm. The model was constructed by leveraging the point spread function (PSF) of ghost image residuals and their energy distribution characteristics. Using the proportion of ghost image residuals and hyperspectral calibration accuracy as constraints, the proposed method iteratively optimizes the regional target residuals across different hyperspectral channels, enabling automatic correction of ghost image residuals across spectral bands. This ghost image residual correction model offers significant value for correcting dispersion-type hyperspectral ghost image residuals.

2. Methods

The core goal of ghost image residual correction in hyperspectral images is to reduce the proportion of ghost image residuals while ensuring the hyperspectral image features remain unchanged, thereby diminishing the impact on spectral calibration accuracy during the ghost image residual correction in hyperspectral images. The schematic diagram of the ghost image residual correction in hyperspectral images is shown in Figure 1.

Based on the schematic diagram, we propose a workflow of ghost image residual correction in hyperspectral images, as illustrated in Figure 2. The key steps are ghost image residual enhancement, ghost image residual registration, and ghost image residual removal. The goal of enhancement and registration is to ensure spatial registration between target images and ghost image residuals, while iterative optimization aims to reduce the proportion of ghost image residuals to meet the requirements of image utilization.

The ghost image residual registration in hyperspectral images primarily aims to align the spatial dimensions of target images with those of ghost image residuals. However, due to the significantly low energy of ghost image residuals in hyperspectral images compared to that of the sky background, their contrast ratio is closest to that of the sky background, posing substantial challenges to structural registration. To address this difficulty, signal enhancement is executed before ghost image residual registration. The ghost image feature enhancement is primarily achieved through the use of a different contrast-limited adaptive histogram equalization (CLAHE) algorithm in varying scales of local regions [16,17,18] to improve the registration accuracy of hyperspectral images. The improved CLAHE algorithm is presented in Equation (1).

$$I_{out}(x,y)={\displaystyle \sum _{k=1}^k{w}_k(x,y)\cdot I_k^{CLAHE}}(x,y)$$

(1)

Here, $I_k^{CLAHE}(x,y)$ denotes the CLAHE-enhanced result of coordinates (x, y) at scale k, and w_k(x,y) represents the corresponding weight of the enhancement at scale k. With the enhancement of the ghost image residual features, some noise in the sky background image are also synchronously enhanced. To suppress the noise and achieve the registration between ghost image residuals and target images, the SURF registration was performed on hyperspectral images of the same channel in different-scale local regions to accurately achieve registration between ghost image residuals and target images [19,20].

The energy distribution registration of the target image with ghost image residual primarily relies on the fitting of the ghost image residual point spread function (PSF) [21,22,23]. Based on the PSF energy distribution of ghost image residuals in a single channel provided by optical simulation analyses, the ideally fitted two-dimensional Gaussian function is used as the optimization process of ghost image residual degradation. Since the energy distribution of actual ghost image residuals after a single reflection on the CCD does not conform to the ideal Bessel function, three concentric Gaussian distribution functions are selected to simulate the ghost image residual energy distribution, and the equation is listed below:

$${\mathrm{PSF}}_{2\mathrm{D}}(x,y)={\displaystyle \sum _{k=1}^3{A}_k\cdot \exp }\left\{\left[{\frac{(x-x_0)}{2{\sigma }_{xk}^2}}^2+{\frac{(y-y_0)}{2{\sigma }_{yk}^2}}^2\right]\right\}$$

(2)

where PSF_2D(x,y) represents the two-dimensional ghost image residual energy distribution function; A_k denotes the coefficients of Gaussian distribution functions; x₀ and y₀ represent the central positions along the x-axis and y-axis, respectively; and σ_xk and σ_yk correspond to the standard deviations along the x and y directions, respectively. When x = x₀, Equation (2) is equivalent to the one-dimensional PSF energy distribution function in the y direction, and the coefficients Aₖ should meet the normalization condition ${\sum }_{i=1}^3{A}_k=1$.

In order to minimize the sum of the squared errors between the simulated ghost image residual energy distribution and the simulation data, the parameter estimation equation is as follows:

$$L(\theta )={\displaystyle \sum _i{(\mathrm{PSF}}_{\mathrm{x}}}(x_i;\theta )-{\mathrm{PSF}}_{\mathrm{x}}^{sim}(x_i){)}^2+{\displaystyle \sum _j{(\mathrm{PSF}}_{\mathrm{y}}}(y_i;\theta )-{\mathrm{PSF}}_{\mathrm{y}}^{sim}(y_i){)}^2$$

(3)

where $PS{F}_x$ is the one-dimensional simulated energy distribution in the x direction. When y ≠ y₀, ${\mathrm{PSF}}_x^{sim}$ is the one-dimensional PSF data in the x-direction; Similarly, when y = y₀, ${\mathrm{PSF}}_x^{sim}$ is the one-dimensional PSF data in the y-direction. Equation (3) is used to evaluate the degree between the fitted results and data, so the goal of the optimization process is to obtain the minimum value of L(θ). θ is defined below:

$$\theta =\left\{A_1,A_2,A_3,{\sigma }_{x1},{\sigma }_{x2},{\sigma }_{x3},{\sigma }_{y1},{\sigma }_{y1},{\sigma }_{y3}\right\}$$

(4)

The control optimization results require adding boundary conditions A_k > 0, σ_xk > 0, σ_yk > 0, as well as controlling the primary and secondary orders of three concentric Gaussian distributions and adding constraint conditions σ_x1 < σ_x2 < σ_x3. The final PSF_2D(x,y) obtained is the peak normalized distribution function. At this point, the double integral of PSF_2D(x,y) is not 1 and cannot be used as a degradation function or for image convolution. Therefore, the following conditions should also be satisfied:

$${\displaystyle \iint H_{\lambda }}(x,y)dxdy={\displaystyle \iint \gamma PS{F}_{2D}}(x,y)dxdy=1$$

(5)

where H_λ is the final degradation function and γ is the scaling factor of PSF_2D (x,y). The final correction result can be obtained by calculating the degradation function in all bands and convolving it with real images, and then using the ghost image iteration algorithm to perform spatial dimension operations on the image. The pseudo code of the ghost image iteration algorithm is shown in Algorithm 1.

Algorithm 1. Pseudo Code for Ghost Image Residual Correction Iteration Algorithm in Hyperspectral Images

Ghost Image Removal via Iterative Optimization

Description: Minimizing ghost image representation in images based on simulated annealing algorithm

Input: rate of temperature drop γ, Image with Ghost I0

Output: Image I of Removed Ghost

Initialization: set I = I0

set Iterative Counter k = 0

set Original Temperature T0

set Minimum Temperature Tmin

Iteration:

$\mathrm{while} T_k>T_{min}$

$I_{new}=f\left(a_{k-1},I_{k-1},H_{\lambda }\right)$

$E_{new}=g\left(I_{new}\right)$

$I_k=I_{new}\hspace{1em}if\hspace{1em}{E}_{new}-E_{k-1}-1<0$

$I_k=I_{k-1}\hspace{1em}if\hspace{1em}{E}_{new}-E_{k-1}-1>0 \&\exp \left(-{\displaystyle \frac{E_{new}-E_{k-1}}{T_{k-1}}}\right)>ϵ$

$k=k+1$

$a_k=f\left(a_{k-1},{\mathsf{\delta }}_k\right)$

$T_k=\mathsf{\gamma }\cdot T_{k-1}$

end while

$\mathrm{Stopping} \mathrm{Criterion}: T_k<T_{\min }$

$\mathrm{Return}: I_k$

According to the simulated degradation algorithm [24,25], ghost image residuals and target images are iterated. Firstly, the degradation function H_λ and the ghost image residual energy coefficient α_k of hyperspectral images are used to eliminate the ghost image continuously in the previous iteration output, as shown in Equation (6).

I_new = I_(k) − α_(k) · (H_λ × I_(k))

(6)

$${\alpha }_{(k+1)}={\alpha }_{(k)}+{\delta }_{(k)},{\delta }_{(k)}~U(-{\epsilon }_{(k)},+{\epsilon }_{(k)})$$

(7)

$${\epsilon }_{(k)}={\epsilon }_{(0)}\cdot {(1-{\displaystyle \frac{k}{k\max }})}^p,p > 1$$

(8)

In the equation, I_(k) is the output image of the kth iteration, and I₍₀₎ is the original image.

The optimization of the ghost image residual correction in hyperspectral images is achieved through the convolution of H_λ and I_(k) and the linear constraint of α_(k). Degradation from the input image to the ghost image is implemented. The ghost image energy coefficient α_k is jointly updated by the ghost image energy coefficient α_k−1 from the previous iteration and the random error term δ_k−1. The random error term δ_k−1 is a sample in the uniformly distributed set U(−ε^(k), ε^(k)), and the set interval is determined by the iteration round k, as shown in Equations (7) and (8), where p is the convergence speed of the random error term, and the larger the p, the faster the convergence speed of the random error term interval.

For I_new that undergoes ghost removal one time, the evaluation function E_(k) is used to evaluate its DN value dispersion in the sky region. The general equation for the evaluation function is shown in Equation (9):

$$E_{(k)}={\displaystyle \sum _{i=1}^N{\mathrm{STD}}_{x,y\in \Omega i}}[I_{ghost(k)}(x,y)-u_{sky}]$$

(9)

where Ι_ghost(k) (x,y) represents the ghost image region in the kth step image processing and μ_sky represents the mean of the sky region without ghost images. Comparing the calculation results of the evaluation function before and after ghost image removal, I_new after ghost image removal is evaluated, as shown in Equation (10).

$$P_{accept}=\left\{\begin{array}{l}1,\begin{array}{c}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}if{E}_{new}\end{array}\ge E_{(k)}\\ \exp (-{\displaystyle \frac{E_{new}-E_{(k)}}{T_{(k)}}}),if{E}_{new}\ge E_{(k)}\end{array}\right.$$

(10)

When the evaluation function E_new of I_new is smaller than that of the previous iteration, it indicates that the ghost image removal effect is significant this time, and the result is the output I_new of this iteration. On the contrary, the probability is accepted, and I_new and I_k are output when accepted and rejected, respectively. In addition to the above iterative steps, the temperature coefficient T_(k) is used to constrain the overall iterative process of the algorithm. The equation of the temperature coefficient calculation and its relationship with the maximum iteration step k_max are shown in Equations (11) and (12).

$$T_{(k)}=T_{(0)}\cdot {\gamma }^k,(0 < \gamma < 1)$$

(11)

$$k_{max}=\left[{\displaystyle \frac{\lg (T_{min}/T_0)}{\lg (\gamma )}}\right]$$

(12)

In the equations, T₀ is the initial temperature, γ is the cooling rate, and T_min is the lowest temperature. When T_(k) < T_min, the iteration stops and outputs I_(k).

The ghost image residual degradation algorithm mainly iteratively removes the ghost image residual elements to gradually reduce the impact of ghost image residuals on the target image details.

3. Results

The ghost image residual hyperspectral data cube has a wavelength range of 430–850 nm, a total of 72 spectral channels, and an image spatial pixel resolution of 2000 pixels. The number of image splicing frames is 2500. The ghost image residual hyperspectral data cube is shown in Figure 3.

To analyze the impacts of ghost image residuals on targets and image quality, Building A, Building B, Tower C, and Vegetation D were selected as typical targets, and the spectral curve changes for the targets and the spectral curve of ghost image residuals were used as objects for evaluating ghost image residual correction performance.

When ghost image residual images are registered with target images, the extraction of image feature points is particularly important. However, the proportion of ghost image residual energy in the sky background image is very small, and it is impossible to extract sufficient feature points for registration, so the accuracy requirements of image registration will not be met. The multi-scale CLAHE algorithm was used to enhance the target zone (Zone B) and ghost image residual zone (Zone A) of the hyperspectral image. The enhanced images are shown in Figure 4.

Zone A is the ghost image residual zone of the hyperspectral image, while Zone B is the hyperspectral target image zone. Figure 4a,b are enhanced effect diagrams of the ghost image residual zone, and the ghost image residual in the hyperspectral image, respectively. Comparing Figure 4a,b reveals that the enhanced ghost image residual image has clear and visible contour details, as well as prominent image features. However, at the same time, the noise in the sky area is also enhanced, limiting the accuracy of image matching to some degree. In order to solve this problem, four registration algorithms with good anti-noise performance, namely, RIFT, AKAZE, SIFT, and SURF, were selected for use in comparative experiments in the matching process between ghost image residual images and target images.

In order to address the impact of noise in the sky region, four image registration algorithms with good anti-noise performance, namely, RIFT [26], AKAZE, SIFT [27], and SURF [28], were selected for registration experiments before and after ghost image residual enhancement. The experimental results are shown in Figure 5, Figure 6, Figure 7, Figure 8, demonstrating the feature point extraction and matching performance of different algorithms with wavelengths of 450 nm, 550 nm, 650 nm, and 850 nm, respectively.

The RIFT algorithm was used to compare the feature point extraction and matching performance regarding single-wavelength ghost image residual, Zone A, and target image, Zone B, with wavelengths of 450 nm, 550 nm, 650 nm, and 850 nm before and after enhancement, as shown in Figure 5. As shown in the figure, the feature point matching effect remains unchanged, and the accuracy is extremely low before and after ghost image residual enhancement, and the image registration effect is poor.

The AKAZE algorithm was used to compare the feature point extraction and matching performance of the single-wavelength ghost image residual, Zone A, and target image, Zone B, with wavelengths of 450 nm, 550 nm, 650 nm, and 850 nm before and after enhancement, as shown in Figure 6. Obviously, the number of extracted feature points slightly increased after ghost image enhancement, and concurrently, the feature point matching performance improved to some extent. However, there are still some feature points with low matching accuracy, and the image registration effect is still poor.

The SIFT algorithm was used to compare the feature point extraction and matching performance of the single-wavelength ghost image residual, Zone A, and target image, Zone B, with wavelengths of 450 nm, 550 nm, 650 nm, and 850 nm before and after enhancement, as shown in Figure 7. As shown in the figure, after ghost image residual enhancement, the number of feature points increases sharply, and the matching accuracy of feature points significantly improves. Compared with the RIFT and AKAZE algorithms, the effect is significantly better.

The SURF algorithm was used to compare the feature point extraction and matching performance of the single-wavelength ghost image residual, Zone A, and target image, Zone B, with wavelengths of 450 nm, 550 nm, 650 nm, and 850 nm before and after enhancement, as shown in Figure 8. As shown in the figure, after ghost image residual enhancement, the number of extracted feature points increased by 2–3 times. At the same time, there are almost no incorrectly matched feature points in each band image. This algorithm performs best among the four algorithms.

The offsets of RIFT, AKAZE, SIFT, and SURF registration algorithms in the same target image region are shown in

Table 1. Image Offset after Registration using Different Registration Algorithms.

	Region A	Region B	Region C	Region D
AKAZE	5.1864	9.1462	5.7414	7.1483
SIFT	2.8284	8.5440	4.4721	5.9413
SURF	1.2361	1.6158	1.7623	1.9819

. The SURF image registration offset is less than two pixels, meeting the image registration accuracy requirements.

The spatial offset details of the target image and the ghost image residual image processed via the SURF algorithm optimization are shown in Figure 9 [29,30,31,32,33]. As shown in the figure, there is only a spatial dimension offset between the target image and the ghost image residual, and the offset is less than two pixels.

Based on the results of the above comparative experiments, the SURF algorithm has the best image registration accuracy, and the image offset is less than two pixels after registration, meeting the requirements of image registration accuracy.

The registered ghost image residual image was subjected to two-dimensional PSF distribution fitting to obtain a degraded image of the ghost image residual. The degradation result is shown in Figure 10.

The blurred edges of the building after hyperspectral ghost image residual enhancement are shown in Figure 10a. The target image was fitted with the PSF distribution of ghost image residual, and the fitting result is shown in Figure 10b. As shown in the figure, the degraded target image edge has an energy distribution similar to that of the ghost image residual edge [34,35]. The experimental results indicate that the method of fitting the two-dimensional PSF distribution of the ghost image residual was correct. Through this method, the continuous weakening of the ghost image residual features is used as the constraint, and the target image and the ghost image residual image are repeatedly fitted. The iterative optimization results of the ghost image residual 3D data cube are shown in Figure 11a. As shown in Figure 11c, the ghost image residuals in Zone A are completely eliminated in the sky background. Simultaneously, the optimization and correction of ghost image residuals in a single-channel sky background were analyzed. The ghost image residual images with center wavelengths of 450 nm, 550 nm, 650 nm, and 850 nm after single channel correction were extracted. These ghost image residual images after correction are shown in Figure 12. As shown in the figure, after ghost image residual correction, on the premise of preserving the characteristics of the sky image, the ghost image residual spatial features and spatial contours in the sky background are basically eliminated, achieving the goal of ghost image residual correction.

3.1. Standard Deviation and Energy Proportion of the Images of the Ghost Image Residual Before and After Correction

The standard deviation, as a core indicator for measuring the degree of data dispersion, can be used to evaluate the visibility of ghost image features in the sky region. The larger the standard deviation, the more obvious the ghost image residual features. Figure 13 displays the standard deviation of radiance values of the ghost image zones corresponding to typical targets before and after the removal of the ghost image residuals. As shown in the figure, the standard deviation of ghost image residual in the typical target area decreased by about 75% before and after correction. The proportion of ghost image residual energy is indicated in Figure 14. The energy proportion before and after ghost image residual correction at different wavelengths shows that the proportion of ghost image residual energy decreased from 4.6% to 0.3% after ghost image residual correction.

3.2. The Influence of Ghost Image Residuals Before and After Correction on Absolute Radiometric Calibration

The correction of ghost image residuals has a certain impact on the spectral features of three-dimensional cube images. In order to further analyze the influence of ghost image residual correction on spectral features, the spectral radiance of typical target ghost image residuals given in Buildings A and B, Tower C, and Vegetation D before and after correction was analyzed. The results are shown in Figure 15 and Figure 16. As shown in the figures, the corrected ghost image residual radiance is very small, and the radiance regions of different wavelengths are stable.

To analyze the influence of ghost image residuals on the spectral curves of typical targets before and after correction, we plotted the spectral radiance curves of a typical target ghost image residual before and after correction, as shown in Figure 17. As revealed in the figure, the spectral curve trends of four typical targets, A–D, did not change. Their spectral radiance ranges from 450 nm to 600 nm, and the amplitude of spectral radiance varies greatly, with the maximum change in amplitude being about 0.005 W·cm⁻²·Sr⁻¹, accounting for 0.8% of the typical target radiance. The variation range is within the standard deviation range of the target area grayscale value and can be almost ignored.

The accuracy of spectral instrument radiance is primarily depends on the radiometric calibration accuracy [36,37,38,39]. Absolute radiometric calibration accuracy correlates with absolute calibration coefficients. Analyzing the absolute calibration coefficients before and after ghost image residual correction, the influence of absolute calibration accuracy is reflected, with the absolute calibration coefficients as shown in Figure 18. Before and after ghost image residual correction, the absolute calibration coefficient curves are basically consistent. At a wavelength of 733.6 nm, the curve difference before and after correction is the largest. The amplitude difference accounts for 0.06% of the typical target radiance, and such an effect can be ignored. Finally, comparing and verifying the accuracy of absolute radiometric calibration through the calculation of ASD data with different reflectivity based on absolute calibration coefficients, we obtained the ASD prediction curves before and after ghost image residual correction under different reflectivity, as shown in Figure 19. Notably, at a wavelength of 721.7 nm, the degree of curve separation is the highest. At this moment, the radiance difference before and after ghost image residual correction is 0.0002 W·cm⁻²·sr⁻¹, representing 0.25% of total radiance. This finding confirms that the absolute radiometric calibration coefficients before and after ghost image residual correction are almost completely unaffected.

As shown in the results and analysis, in cases where the impact on the target spectral curve and absolute radiometric calibration coefficient is smaller, the ghost image residual correction algorithm can achieve ghost image correction, thereby achieving its goal.

4. Conclusions

The rich spectral information of hyperspectral data makes any algorithm highly sensitive [40,41,42]. We designed a targeted ghost image residual processing method based on the analysis of ghost image residual characteristics and typical target features. The method achieved effective ghost image residual correction while preserving the spectral features and calibration coefficient integrity of the image. The experimental results show that the energy of most ghost images decreased from 4.5% to approximately 0.3%, and the spectral characteristics of ghost image residuals were effectively suppressed. Variations in the target spectral curve were less than 0.8%, indicating that the target spectral features remained unchanged, while the impact on the absolute calibration coefficient did not exceed 0.06%. These results validate that the proposed algorithm effectively eliminates ghost effects under known one-dimensional PSF distributions while preserving image information, providing a theoretical basis for hyperspectral ghost image correction.