A Remote Sensing Image Destriping Model Based on Low-Rank and Directional Sparse Constraint

: Stripe noise is a common condition that has a considerable impact on the quality of the images. Therefore, stripe noise removal (destriping) is a tremendously important step in image processing. Since the existing destriping models cause different degrees of ripple effects, in this paper a new model, based on total variation (TV) regularization, global low rank and directional sparsity constraints, is proposed for the removal of vertical stripes. TV regularization is used to preserve details, and the global low rank and directional sparsity are used to constrain stripe noise. The directional and structural characteristics of stripe noise are fully utilized to achieve a better removal effect. Moreover, we designed an alternating minimization scheme to obtain the optimal solution. Simulation and actual experimental data show that the proposed model has strong robustness and is superior to existing competitive destriping models, both subjectively and objectively.


Introduction
The non-uniform photoresponse of image detectors causes stripe noise with distinct directional and structural features. It will reduce the subjective quality of images and limit their subsequent application in many fields. Therefore, the purpose of our research is to estimate potential prior components to separate the clear image from the degraded image.
In the past few decades, many researchers have carried out related work, which can be roughly divided into two categories: one relies on radiometric calibration and the other is based on image processing. The former establishes a mathematical model between spectral radiation and the response of the image sensor with radiation sources of varying degree generated by the integrating sphere. The latter analyzes the causes of stripe noise and establishes a degradation model to achieve destriping. Since there are many limitations of the method based on calibration, this paper adopts an idea, based on image processing, for removing stripe noise. At present, there are three kinds of destriping methods based on image processing: methods based on filtering, methods based on statistical theory and methods based on optimization.
The first method filters the degraded image in the transform domain by designing different filters [1][2][3][4][5][6][7][8]. In [3], wavelet analysis was used to remove stripe noise from satellite imagery. In [4], an FIR filter was proposed to filter the image in the frequency domain. In addition, Münch et al. [6] proposed a combination filter that uses wavelet decomposition to improve filtering accuracy to separate stripes. This method is simple in operation and fast in processing, but it can not remove non-periodic stripes completely.
The second method usually considers using the statistical characteristics of the sensors to remove stripe noise [9][10][11][12][13][14][15][16]. Histogram matching and moment matching are two typical methods. The former is usually matched with a histogram of the reference signal to remove the stripe noise. The latter generally assumes that each image sensor has the same standard deviation and mean value, then selects the ideal reference data, using moment matching to restore the image. In [9], histogram matching was used. Wegener et al. [10] introduced a process of calculating homogeneous regions before histogram matching. The author also used moment matching in [12]. In [14], local-least-squares fitting was considered for combination with histogram matching to restore the image. Limited by previous assumptions, the destriping effect of this method shows great variation , which indicates that the model has poor reliability and robustness.
In recent years, lots of models based on optimization [17][18][19][20][21][22][23][24][25][26][27] have been proposed that regard the destriping issue as an ill-posed inverse problem. To find the optimal solution, constructing a proper regularized model for the underlying prior information of the image is necessary. Therefore, this method focuses on finding potential prior information and corresponding regularization terms. In [17], the Huber-Markov variation model was proposed, firstly. In [18], the author proposed a complex single-term total variation model (UTV), which used stripes' structure and direction characteristics to preserve image details. Chang et al. [22] adopted the idea of image decomposition, proposing the low-rank single image destriping (LRSID) model to estimate two priors simultaneously. Liu et al. [23] separated stripe noise from degraded images by considering global sparsity and local variational (GSLV) properties. In [24], the author used a regularized model that combines the total variation and global sparse (TVGS) constraint. In [27], a destriping model based on hybrid total variation and nonconvex low-rank (HTVLR) regularization was proposed to reduce the staircase effect caused by the TV model.
In general, the mentioned models can remove stripe noise in most cases, but they still have some drawbacks when dealing with different remote sensing images. For instance, in the low-rank constrained model proposed in [22], the structural and directional characteristics of the stripes are not fully utilized. In [23], the author only focuses on the stripe noise components in the degraded image, ignoring the properties of the underlying image information, which will destroy the smoothness of the restored image. The TVGS model proposed in [24] lacks a constraint term perpendicular to the stripe direction and may result in ripple effects. In [27], the HTVLR model could reduce the staircase effect caused by the TV model but could not maintain well its destriping performance when dealing with different stripes.
Focusing on the problems in the above methods, we apply image decomposition and propose a destriping model based on total variation and the low-rank direction sparse constraint. The TV model and low rank are taken to constrain the image prior and the stripenoise prior globally. Different directional sparse constraints are adopted along and cross the stripe direction after taking full advantage of the structural and directional characteristics of the stripe noise. l 1 norm and l 0 norm are respectively taken to constrain the gradient matrix perpendicular to and along the stripe direction. Since the proposed model should estimate two factors simultaneously, an alternating minimization scheme is taken to find the optimal solution effectively. The specific framework of the solution is shown in Figure 1. Simulation and actual experimental data indicate that the proposed model shows better destriping performance compared with the five typical models. The main research work and innovative content of this article are summarized as follows: (a) Under the destriping model of image decomposition, a sparsity constraint, perpendicular to the stripes, is added to reduce the ripple effects of the output image. (b) After thoroughly analyzing the potential properties of stripe noise, we propose a regularization model combining low-rank and directional sparsity, enhancing the robustness of the stripe noise-removal model. (c) An alternate minimization scheme to the model is designed to estimate both potential priors in degraded images. The subsequent contents are arranged as follows: the image-degradation model and the destriping model are introduced in Section 2. In Section 3, an alternating minimization algorithm is designed. The Section 4 verifies the destriping performance of the proposed model through many related experiments. In Section 5, the parameter value determination and future research are discussed. Section 6 is the conclusion of this paper.

Degradation Model and Proposed Model
Stripe noise, in remote sensing images, usually contains additive and multiplicative noise components [17]. Since multiplicative noise can be converted into additive noise through a logarithmic operation [15], stripe noise is usually treated as additive noise. Therefore, this type of image degradation model can be summarized as where o(x, y), i(x, y) and s(x, y) represent the original noisy image, clear image and stripenoise image. For convenience in the subsequent work, the formula (1) can be rewritten as follows: where O, I and S represent the matrix forms of o(x, y), i(x, y) and s(x, y), respectively. Both clear image, I, and stripe noise image, S, are the data we want to obtain from the degraded image, O, and the regularization can be considered to solve this typical ill-posed problem.
Taking into account the image decomposition model in [22], the constrained model for destriping can be expressed as: where 1 2 O − I − S 2 F denotes the item representing the closeness between the degraded image and the sum of the clear image and the stripe noise. R(I) and R(S) are regularization terms, representing the information of the image prior and the stripe-noise prior. λ and γ are positive penalty parameters that are used to balance the constraint model. To obtain a better separation effect, it is necessary to select the appropriate regularization term and method.

The Regularization Term and Regularization Method of the Real Image
The most extensively used regularization methods in image processing are Tikhonovlike regularization [28] and TV-based regularization [29]. This paper adopts TV-based regularization to constrain the image prior due to its better performance in preserving image details.
For a two-dimensional image, TV constraint model can be expressed as: we take the vertical and the horizontal direction as the y and x direction respectively in this paper; then, the regularization constraint of the clear image can be expressed as [30]: where D x and D y represent the first derivative operator in the corresponding direction.

The Regularization Term and Regularization Method of the Stripe Noise Image
Singular value decomposition and eigenvalue decomposition can both be used to extract the matrix's features. The difference is that eigenvalue decomposition only works with square matrices, but singular value decomposition works with any matrix. We can divide the original matrix into the product of three matrices using singular value decomposition. The second matrix is diagonal and its diagonal elements are the matrix's singular values. We use the SVD function to perform singular value decomposition on the stripe image and plot its singular values in columns (see Figure 2). It can be found that the singular value quickly drops to 0 in the first few columns, which indicates that the stripe-noise prior can be regarded as a low-rank matrix [22]. In addition, the stripe noise image can also be viewed as a matrix with lots of zero elements. However, considering that the sparsity characteristic will disappear when stripes are too dense, we use the kernel norm to constrain the global low rank of the stripe noise. Therefore, this regularization term can be formulated as: For stripe noise images, we assume that the direction of the stripes is the same as the y direction. The gradient matrix along the stripe direction is an obvious sparse matrix due to the sameness of intensity of each column, so this regularization term can be formulated as: In addition, a constraint along the x-direction is needed to minimize the first derivative along the horizontal direction to ensure the continuity and smoothness of the clear images. According to formula (2), this constraint is added to the stripe-noise prior, and then this regularization term can be formulated as: Based on the above analysis, the regularization term of the stripe-noise prior can be summarized as: Finally, the destriping model in this paper can be summarized as: arg min where λ 1 , λ 2 , γ 1 , γ 2 and γ 3 represent regularization parameters used to adjust the weight of each item to balance the model.

ADMM Optimization
The alternating direction multiplier method (ADMM) is usually considered to estimate the optimal value of this type of optimization problem. Therefore, we can decompose the above problem into two optimization sub-problems: the sub-problem of solving stripenoise prior, S, and the sub-problem of solving image prior, I .

Image Prior Optimization Process
First, we fixed the stripe-noise prior, S, solving the image prior, I. The optimization model of I can be expressed as: For convenience in the subsequent work, two auxiliary variables, M = D x I and N = D y I, are introduced to transform the above equation into the following form: Subject to M = D x I , N = D y I Next, according to [31,32], the augmented Lagrangian equation, formula (12) can be expressed as: where L 1 , L 2 and β respectively represent the Lagrange multipliers and the positive penalty parameter. The problem of formula (13) can be considered to be divided into the following three sub-problems: (1) The M sub-problem can be summarized as arg min Soft threshold shrinkage is an effective way to solve this type of optimization problem [33]. Therefore, we can obtain the solution as follows: where: (2) The N sub-problem can be summarized as arg min Same as the M sub-problem, we can get the solution as follows: (3) The I sub-problem can be described aŝ This equation is a typical quadratic optimization problem from which an optimal solution can be obtained. By differentiating the above equation, the formula (19) is converted to: The two-dimensional Fourier transform is an effective method to solve the above problem [34]. Therefore, we update the image prior, I, as follows: F and F −1 represent the fast Fourier transform and the inverse fast Fourier transform. Finally, we make the following update to the Lagrange multipliers L 1 and L 2 :

Stripe-Noise Prior Optimization Process
Second, we fix the image prior, I, solving the stripe-noise prior, S. The optimization model of S can be expressed as: Similarly, three auxiliary variables, , are introduced to transform the above equation into the following constrained optimization problem: where: L 3 , L 4 , L 5 and µ are Lagrange multipliers and a positive penalty parameter. Similar to the formula (11), the problem of formula (25) can be considered to be divided into the following four sub-problems: (1) The W sub-problem can be summarized as arg min Singular value soft threshold shrinkage can be used to solve this type of optimization problem [35]: where: (2) The H sub-problem can be summarized as arg min This sub-problem can be solved by hard threshold shrinkage [36,37]: where: (3) The K sub-problem can be summarized as arg min Soft threshold shrinkage can be used to solve this type of optimization problem: (4) The S sub-problem can be summarized as arg min Similar to the I sub-problem, the solution of this problem is: where: Finally, we will make the following update to the Lagrange multipliers L 3 , L 4 and L 5 : The solution process of the model can be summarized in Algorithm 1:

Simulation and the Actual Destriping Experiment
In order to accurately evaluate the destriping performance of the proposed model, we carried out the simulation experiments and actual destriping experiments at the same time, assessing the destriping results both subjectively and objectively. Different indexes are chosen to evaluate the results, considering the differences between the simulation and actual destriping experiments.
Furthermore, five typical destriping methods, SLD [15], LRSID [22], GSLV [23], TVGS [24] and HTVLR [27], are selected as references to evaluate the proposed model. All methods have adjusted parameters to make the destriping effect suitable for comparison, except for LRSID, whose source code is published by the author on his website homepage. To compare the destriping effect intuitively, we make special remarks for the obvious differences.

Simulation Experiment
During the simulation experiments, we selected two typical image evaluation indicators, peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) [38], to objectively evaluate the processing results. These indexes are as follows, whereû and u are, respectively, the restored and the undegraded image, while n is the number of pixels.
where m_û and m_u denote the mean value of the two images, δ xy represents the covariance, J 1 and J 2 are constants that can be calculated in the following way: L represents the dynamic range of a pixel, and k 1 = 0.01, k 2 = 0.03. There are two parts of the simulation experiments: simulation experiments under periodic and non-periodic stripe noise. The degree of image degradation is determined by r and I. Here, r denotes the proportion of the degraded region, and I represents the intensity of the added stripe noise . During the simulation experiments, we select noise ratios of 0.3, 0.5, 0.7 and 0.9, and intensities of 30, 50, 70 and 90. We treat these two parameters as an array for the convenience of expression. For example, (0.3, 50) represents the stripe ratio of 0.3 and the intensity of 50.
For periodic stripe noise, MODIS image band 32 and one typical region of the hyperspectral image of Washington DC Mall are selected to carry out the destriping experiment. The former is available from https://ladsweb.nascom.nasa.gov/, (accessed on 5 September 2021) and the latter can be downloaded from https://engineering.purdue.edu/~biehl/ MultiSpec/ (accessed on 5 September 2021). Since the existing destriping methods all have a good removal effect in this case, we have only selected the results with noticeable differences for comparison. The partial destriping results of MODIS data are shown in Figure 3. The complete simulation data and results can be obtained from Tables 1 and 2, and the best performance indexes are shown in bold.   According to the data in Table 1, SLD performs better when dealing with low-ratio and -intensity stripes; the proposed model shows a better destriping effect for the high-ratio and -intensity stripes. However, it can be found in Figure 3 that some residual stripes remain in the images restored by SLD, which shows a different result from Table 1. Furthermore, the results of the hyperspectral image show that the PSNR of SLD decreased rapidly with the noise ratio and intensity of (0.7, 90) and (0.9, 90), which indicates that it lost the original destriping effect. In terms of structural similarity (SSIM), the proposed model always performed best.
MODIS image band 20 and two typical regions of a hyperspectral image of the Washington DC Mall were chosen to carry out the destriping experiment with non-periodic stripes. The partial destriping results are shown in Figures 4 and 5, with rectangular boxes in the images marking regions with obvious differences. Tables 3 and 4 show PSNR and SSIM, respectively, with the best-performing indices highlighted in bold.     The objective evaluation indexes in Tables 3 and 4 show that the TVGS performs better in some cases, but the proposed model shows stronger robustness over different noise intensities and ratios. HTVLR shows a good destriping effect at low intensity, but, as the noise gradually intensity increased, it was difficult for HTVLR to maintain good destriping performance. Subjectively, residual stripes and gray-scale loss are observed in the images restored by SLD, LRSID and HTVLR. TVGS and GSLV can remove the most noticeable stripes, but ripple effects influence the smoothness of the images. According to Figure 5, it can be found that the structure of the stripes is clear and there is no obvious image information. Additionally, the stripe image separated by the proposed model is much more similar to the added stripe when we focus on the region marked by the red rectangle.
During the experiments, it was found that the existing methods showed worse removal effects when dealing with the stripes of high intensity and ratio, while the proposed model still maintained excellent performance. In this paper, related simulation experiments were carried out with MODIS data. We conducted experiments on a degraded image with a noise ratio of 0.9 and a noise intensity of 80, and the destriping results are shown in Figure 6. It shows that there are lots of residual stripes in the images restored by other methods. Additionally, we compared the column mean value of the restored images and the undegraded image; the results are shown in Figure 7. The curve in blue is the column mean value of the undegraded image, while the curve in orange represents that of restored images in Figure 6. We can find that the curve restored by the proposed model is generally consistent with the original curve.  Remote sensing images usually contain other random noise types that may affect the removal of stripe noise, so we conducted a simple simulation experiment on this situation. We added stripe noise with a ratio of 0.5 and an intensity of 50 to an image containing Gaussian noise, Poisson noise, salt-and-pepper noise and speckle noise to carry out the destriping experiment. The results are shown in Figure 8, from which we can find that the proposed model could still remove the stripes, but the removal effect was be affected by other random noise types. There is a certain degree of rippling effect in the processed results of the images containing Gaussian noise, Poisson noise and speckle noise, and there are some residual stripes in the image containing speckle noise. We also tested the destriping performance on non-remote sensing images using data from the SIDD dataset, which can be found at https://paperswithcode.com/dataset/sidd, (accessed on 15 November 2021). Figure 9 shows these destriping results, which indicate that the stripes are properly separated and there are no residual stripes in the restored images. The proposed model also works well with non-remote sensing data.

Actual Destriping Experiment
During the actual destriping experiments, MODIS and our data were selected for verification of the destriping performance and applied effects of the proposed model. The decrease of the standard deviation and the photo response non-uniformity (PRNU) of the image's uniform region were selected to evaluate the processing results objectively. The PRNU is as follows: where σ and m_u represent the standard deviation and the mean value of the image u. Figure 10 displays our data, with the uniform region utilized to calculate the PRNU, itself denoted by the rectangular box in the bottom-right corner. Table 5 depicts the PRNU. The proposed model is found to have the best performance, and the image's PRNU decreases by 4.2%.  Additionally, we can get an intuitive comparison of MODIS data from Figure 11, and the red ellipses are used to mark the residual stripes in the images. Figure 12 is the enlarged processing effect of the rectangular region. Figure 13 shows the destriping results of our data, wherein ellipses are used to mark regions with a very poor removal effect. Figure 14 shows the partial enlargement of the rectangular region in the upper-right corner of Figure 10. The standard deviations of the images are shown in Table 6.    Considering the influence of excessive smoothing, it is necessary to integrate subjective and objective factors to evaluate models. Table 6 demonstrates that the LRSID performs better on MODIS02. However, it is clear from the view of the destriping results in Figure 12 that SLD and LRSID do not completely remove stripes. Besides, the results in Figures 11 and 12 show that the proposed model removes stripe noise more thoroughly, and the restored image is clearer, though there are still a few residual stripes in the regions marked by the red ellipse. From the results of processing our data, TVGS and the proposed model performed better; however, the proposed model obtained a clearer image with no ripple effects or residual stripes.

Result Discussion
According to the results of all experiments, the proposed model can remove the stripe noise in most cases, but there are still some issues worth discussing. As can be seen from Tables 1 and 3, the PSNR of the proposed model is always lower than that of other models when dealing with stripe noise of low intensity and ratio; in some cases, the gap is still large. It means that it cannot fully constrain the image prior and the stripe-noise prior in such case. The proposed model has several parameters that influence the destriping performance in various conditions, by determining the weight of each constraint item. When dealing with low-ratio and -intensity stripe noise, the low-rank feature can successfully constrain the stripe noise components; however, when dealing with high-noise-ratio and -intensity stripe noise, the sparsity feature constrains the stripe noise more effectively. We utilized the same parameters to achieve the destriping effect in different conditions during the experiments, to ensure the model's reliability and practicability. This limited the weight of each constraint item, making it difficult to reach optimal constraints in some cases. Where stripes can be detected before removal, we can choose a more reasonable destriping model to achieve a better removal effect. Additionally, there is another problem in Figure 7. In the first and last few columns, the column mean values of the image restored by the proposed model are slightly different from those of the undegraded image. This could be have been caused by inappropriate border treatment, which could be improved in follow-up research.

Limitation
Although the proposed model achieves a superior destriping effect, it still has some limitations. The current research has mainly focused on removing stripe noise from a single remote sensing image, and the model is unable to perform destriping on remote sensing images with different channels, such as multispectral images. Furthermore, when there are some small fragment cases, the low-rank characteristics of the stripe components will be destroyed, significantly weakening the model's stripe noise-removal effect. Therefore, when the image contains a lot of random noise, the model's destriping effect may be considerably diminished.

Conclusions
The majority of image processing problems are ill-posed inverse problems that can be addressed with a suitable regularization model. The optimum result can be obtained by adding appropriate regularization terms to the underlying priors.
In this paper, under the premise of completely retaining the image information, we fully considered stripe noise's potential low-rank and sparse properties and proposed a stable and effective destriping model. Constrained by these properties, the model can preserve image details while dealing with stripe noise. Combining the subjective and objective experimental results, the proposed model obtained better destriping performance than the other five existing typical models. Furthermore, the proposed model could still stably remove the stripe noise when other methods lost their effect. It shows an excellent stripe noise removal effect for different degrees of degraded images with strong robustness, and is, thus, worthy of promotion.
The proposed model shows strong competitiveness in both subjective and objective evaluation. However, it still has some problems, such as many solving processes, a large amount of calculation, and a long processing time, which will be continuously improved in follow-up research . In addition, random noise will pose a new challenge to the removal of stripe noise; whichever noise is preferentially processed will affect other types of noise. Therefore, related research will also be carried out in follow-up to remove all of types of noise simultaneously.

Acknowledgments:
The authors would like to thank the anonymous reviewers for their valuable comments.

Conflicts of Interest:
The authors declare no conflict of interest.

Abbreviations
The following abbreviations are used in this manuscript: