A New Sparse Collaborative Low-Rank Prior Knowledge Representation for Thick Cloud Removal in Remote Sensing Images

Abstract

Efficiently removing clouds from remote sensing imagery presents a significant challenge, yet it is crucial for a variety of applications. This paper introduces a novel sparse function, named the tri-fiber-wise sparse function, meticulously engineered for the targeted tasks of cloud detection and removal. This function is adept at capturing cloud characteristics across three dimensions, leveraging the sparsity of mode-1, -2, and -3 fibers simultaneously to achieve precise cloud detection. By incorporating the concept of tensor multi-rank, which describes the global correlation, we have developed a tri-fiber-wise sparse-based model that excels in both detecting and eliminating clouds from images. Furthermore, to ensure that the cloud-free information accurately matches the corresponding areas in the observed data, we have enhanced our model with an extended box-constraint strategy. The experiments showcase the notable success of the proposed method in cloud removal. This highlights its potential and utility in enhancing the accuracy of remote sensing imagery.

1. Introduction

Remote sensing technology has been widely employed across various fields, including for unmixing [1,2], fusion [3,4,5,6], and classification [7]. However, these images often suffer from inevitable cloud contamination, resulting in significant information loss and constraining the further analysis of remote sensing data [8,9]. Consequently, advancing cloud removal techniques is critical for enhancing the practical utility of remote sensing images.

Mathematically, multitemporal images contaminated by clouds can be represented by a tensor $\mathcal{O}\in {\mathbb{R}}^{a\times b\times d\times t}$ (a and b denote the spatial dimension, d represents the spectral dimension, and t is the time dimension); the clean image component is represented by $\mathcal{U}$, the cloud component is denoted by $\mathcal{C}$, and the model noise is signified by $\mathcal{E}$. Then, the degradation process is formulated as follows:

$$\begin{array}{c}\hfill \mathcal{O}=\mathcal{U}+\mathcal{C}+\mathcal{E}.\end{array}$$

(1)

Numerous cloud removal techniques have been developed by researchers [10,11]. These methods are generally classified into two main approaches depending on whether a cloud mask is available, offering distinct strategies for addressing the challenge of cloud removal.

The first approach is called the nonblind method. It uses a given cloud mask as prior knowledge to reconstruct information obscured by clouds. Traditional methods for addressing this problem are spatial-based methods, which only utilize the interrelations between pixels across the spatial dimension [12,13,14,15]. To more effectively harness the correlations across spectral bands, researchers [16,17] have developed spectral-based methods aimed at improving the reconstruction of missing data. However, the above methods always fail to produce promising reconstruction results if the remote sensing imagery is obscured by thick clouds. Methods that take advantage of multiple images have been developed, which are classified as either multitemporal [18,19,20,21,22] or multisource [23,24] methods. Wang et al. [20] proposed a method for scene reconstruction that employs a robust matrix completion technique through temporal contiguity. Then, they presented an efficient algorithm based on the augmented Lagrangian method (ALM) with inexact proximal gradients (IPGs) to address optimization problems. Zhang et al. [21] proposed a cloud and shadow removal technique based on the learning of spatial–temporal patches. Li et al. [23] developed an innovative cloud removal methodology employing a CMD network that incorporates optical and SAR imagery. Gao et al. [24] utilized a generative adversarial network to fuse optical and SAR images in order to reconstruct information obscured by clouds. To effectively leverage image prior knowledge, hybrid methods have been proposed to exploit two or all aspects of an image’s spatial, spectral, and temporal features. For instance, Chen et al. [25] proposed a Spatially and Temporally Weighted Regression (STWR) method that fully leverages cloud-free information from input Landsat scenes. Melgani [26] introduced contextual multiple linear and nonlinear prediction models. These models assume that the image’s spectral and spatial characteristics remain relatively consistent across the image sequence. While nonblind methods are effective in cloud removal, their heavy reliance on cloud masks somewhat limits their applicability in comprehensively addressing cloud removal challenges.

The second approach, known as the blind method, removes clouds without providing a cloud mask. Wen et al. [27] utilized a technique known as Robust Principal Component Analysis (RPCA) to initially identify cloud cover, followed by reconstructing the missing information using discriminant RPCA to eliminate thick clouds. Chen et al. [28] proposed TVLRSDC, which means total-variation plus low-rank sparsity decomposition. A deep residual neural network model created by Meraner et al. [29] focused on the effective removal of clouds from multispectral satellite images from Sentinel-2. By integrating SAR and optical data through a fusion process, the synergistic characteristics of both imaging systems were exploited to provide guidance for image reconstruction. Wang et al. [30] developed an unsupervised domain factorization network aimed at eliminating thick clouds from multitemporal remote sensing images. These blind removal methods always perform cloud detection and removal separately and independently, which usually changes information for cloud-free regions.

In order to remove clouds better, it is necessary to study prior knowledge about the clouds. It is widely recognized that cloud components can be effectively characterized by sparse functions. Recently, the adoption of element-wise sparse functions, such as the $l_1$-norm, has become prevalent [27,28,31] owing to their concise form and convexity. However, an element-wise sparse function ignores the correlations across the spectrum. Therefore, investigating an appropriate sparse function to characterize the cloud properties becomes crucial. Recently, Ji et al. [32] introduced a model that combines box-constrained low-rank and group sparse techniques, with the specific purpose of detecting and removing clouds. This approach defines cloud properties using group sparsity in the spectral dimension. Nonetheless, it falls short of fully exploring the characteristics of the cloud component.

To further refine the sparsity representation of the cloud component, we introduce the tri-fiber-wise sparse (TriSps) function. This function utilizes the sparsity of mode-1, -2, and -3 fibers to capture more cloud information. Our development of TriSps is inspired by the inherent structural sparsity of cloud components. Specifically, when each tube is considered as a whole, the fibers exhibit sparse characteristics, termed fiber-wise sparsity. In Figure 1a, an image affected by cloud cover is illustrated, and Figure 1b shows the corresponding cloud imagery. In Figure 1c–e, histograms of the $l_2$-norms of mode-1, -2, and -3 fibers extracted from the cloud component are depicted. It is evident from Figure 1c–e that the majority of $l_2$-norm values for mode-1, -2, and -3 fibers are zero, indicating significant sparsity. Additionally, we incorporate the global prior of image components, characterized by multi-rank. Leveraging the proposed sparse function and global low rank, we devised a novel cloud removal model and take advantage of a proximal alternating minimization (PAM)-based approach [33] to efficiently solve the proposed model.

The contributions of this paper are threefold:

• To leverage the inherent structural sparsity of cloud components, we propose the TriSps function specifically for cloud removal purposes. Unlike element-wise and tube-wise sparsity, the TriSps function is designed to capture the properties of clouds in three-dimensional directions more effectively.
• Building upon the TriSps function, we propose a novel cloud removal model that simultaneously estimates both the image and cloud aspects.
• We devised an effective algorithm based on PAM to tackle our method. Experiments with synthetic and real datasets highlight the proposed method’s prowess in cloud removal, outperforming other advanced methods currently available in the field.

This paper unfolds as follows. Section 2 introduces fundamental notations and definitions essential for understanding. In Section 3, we introduce a tri-fiber-wise sparse function to characterize the cloud component’s properties. Furthermore, we have devised a method for detecting and removing clouds. In Section 4, we validate the effectiveness of our method through synthetic and real experiments. Lastly, Section 5 provides some conclusions.

2. Notations and Preliminaries

We present key notations and definitions [34] that are fundamental in our study. the primary notations employed in this paper are outlined in

Table 1. Notation description.

Notations	Interpretations
a, $\mathit{a}$, $\mathit{A}$, $\mathcal{A}$	scalar, vector, matrix, tensor
$\mathrm{Tr}\left(\mathit{A}\right)$	trace of $\mathit{A}\in {\mathbb{R}}^{n\times n}$, with $\mathrm{Tr}\left(\mathit{A}\right)={\sum }_{i=1}^n{a}_{ii}$
${∥\mathcal{A}∥}_F$	${∥\mathcal{A}∥}_F:=\sqrt{{\sum }_{h,w,d,t}{\left|a_{h,w,d,t}\right|}^2}$
${∥\mathit{A}∥}_{*}$	${∥\mathit{A}∥}_{*}:=\mathrm{Tr}\left(\sqrt{{\mathit{A}}^T\mathit{A}}\right)$
${\mathcal{A}}_{\left(k\right)}$	unfolding of $\mathcal{A}$ in its kth mode.
${\mathcal{A}}_{\left(k\right)}^i$	ith mode-k fiber of $\mathcal{A}$
${\mathit{A}}_r$	rth frontal slice of $\mathcal{A}$
$\mathcal{A}=\mathcal{X}{\times }_3\mathit{Q}$	${\mathcal{A}}_{\left(3\right)}^i=\mathit{Q}{\mathcal{X}}_{\left(3\right)}^i$
$\mathcal{T}(·)$	reshaping of fourth-order tensor into third-order tensor
$\mathcal{D}(·)$	difference operator in fourth mode

for clarity and reference.

Furthermore, some definitions are presented below.

Definition 1

(Tensor mode-n product [35]). Given a tensor $\mathcal{A}\in {\mathbb{R}}^{R_1\times R_2\times \cdots \times R_N}$ and a matrix $\mathit{U}\in {\mathbb{R}}^{J\times R_N}$, the mode-n product of $\mathcal{A}$ by $\mathit{U}$ is defined as

$$\mathcal{A}{\times }_n\mathit{U}={\mathit{Fold}}_n\left(\mathit{U}{\mathit{A}}_{\left(n\right)}\right).$$

(2)

In this context, ${\mathit{A}}_{\left(n\right)}\in {\mathbb{R}}^{R_N\times {\prod }_{d\ne n}{R}_d}$ stands for the matricization of tensor $\mathcal{A}$ in the nth mode, while ${\mathit{Fold}}_n(·)$ represents the operator that, in the nth mode, reshapes the matrix back into a tensor form.

Definition 2

(TNN [36]). The tensor nuclear norm of tensor $\mathcal{A}\in {\mathbb{R}}^{R_1\times R_2\times R_3}$ is characterized as follows:

$${∥\mathcal{A}∥}_{\mathit{TNN}}=\sum _{k=1}^{I_3}{∥{\mathit{Z}}_k∥}_{*}.$$

(3)

Here, ${\mathit{Z}}_k$ represents the kth frontal slice obtained from the Fourier-transformed tensor $\mathcal{Z}=\mathcal{A}{\times }_3{\mathit{Q}}_{R_3}$.

Based on the TNN, the tensor completion model can be written as follows:

$$\underset{\mathcal{A}}{min}{\parallel \mathcal{A}\parallel }_{\mathit{TNN}},s.t.{\mathcal{A}}_{\Omega }={\mathcal{B}}_{\Omega },$$

(4)

where ${\mathcal{B}}_{\Omega }$ signifies the tensor obtained by copying entries from $\mathcal{A}$ corresponding to the index set Ω while setting all other entries to zero.

3. The Proposed Method

In this section, we present our proposed sparse function, TriSps. Utilizing the TriSps function as a foundation, we introduce a cloud removal model that combines tensor multi-rank. Then, we devise a PAM algorithm to solve the proposed model.

3.1. The Tri-Fiber-Wise Sparse Function

Existing sparse functions, such as element-wise and tube-wise sparsity, fail to fully exploit the correlations across the spectrum when it comes to cloud properties in remotely sensed images. To overcome this deficiency, we introduce a novel sparse function designed to efficiently capture cloud characteristics across three dimensions. the presence of clouds in contaminated remotely sensed images is relatively sparse compared to the entire image, as shown in Figure 1b. This observation implies that the cloud component exhibits fiber-wise sparsity, aligning with the fundamental nature of clouds. Figure 1 shows the sparsity within a cloud-contaminated image, where Figure 1c–e particularly highlight that most of the $l_2$-norms of mode-n$(n=1,2,3)$ fibers derived from the cloud component are zero. in other words, the mode-1, -2, and -3 fibers of the cloud component exhibit clear fiber-wise sparsity. This realization allows us to simultaneously consider the sparsity across mode-1, -2, and -3 fibers rather than focusing on a single fiber’s sparsity. Leveraging this insight, we introduce the tri-fiber-wise sparse (TriSps) function. This novel function adeptly captures the sparse structures of mode-1, -2, and -3 fibers simultaneously. Utilizing the TriSps function, we can more thoroughly characterize cloud properties and significantly enhance the accuracy of cloud detection in remotely sensed imagery. Mathematically, the TriSps function of $\mathcal{C}$ is defined as

$${\mathcal{J}}_{\lambda }\left(\mathcal{C}\right)=\sum _{n=1}^3{\lambda }_n{\mathcal{S}}_n\left(\mathcal{C}\right),$$

(5)

with

$${\mathcal{S}}_n\left(\mathcal{C}\right)=\sum _i{∥{\mathcal{C}}_{\left(n\right)}^i∥}_2,$$

(6)

where ${\lambda }_n(n=1,2,3)$ denotes the positive weights, and ${\mathcal{C}}_{\left(n\right)}^i$ denotes the ith column of mode-n unfolding ${\mathcal{C}}_{\left(n\right)}$, i.e., the ith mode-n fiber of $\mathcal{C}$.

Remark 1.

1. Equation (6) characterizes the sparsity of mode-n $(n=1,2,3)$ fibers, since it is the approximation of the following $l_0$-quasi-norm of $\mathcal{C}$ with respect to mode-n fibers:

$${∥\mathcal{C}∥}_{f_n^0}=\#\left\{i|{\mathcal{C}}_{\left(n\right)}^i\ne 0\right\},$$

(7)

which is to indicate the number of mode-n fibers that are non-zero. Therefore, the proposed TriSps function in Equation (5) can sufficiently take advantage of the sparsity property of the cloud component.

2. the proposed TriSps function reduces to the sparse function proposed by Ji et al. [32] if the weights are ${\lambda }_1={\lambda }_2=0$ and ${\lambda }_3=1$, which means that the function only considers the sparsity along the tubes. Different from the sparse function in [32], our proposed TriSps function is more general and can capture the properties of clouds in three-dimensional directions.

3.2. Proposed Model

The multitemporal images have high correlations among mode-1, -2, and -3 fibers. The tensor rank function [37,38] serves as an effective tool for characterizing the global correlation of image components $\mathcal{U}$, adeptly capturing the inherent low-rank characteristics of images. in our study, we employ a multi-rank regularization function for $\mathcal{U}$. We transform $\mathcal{U}$ into a third tensor and establish the regularization function as described below.

$${∥\mathcal{U}∥}_{\mathrm{TNN}}=\sum _{r=1}^{dt}{∥{\mathit{X}}_r∥}_{*},$$

(8)

where ${\mathit{X}}_r$ represents the rth frontal slice of $\mathcal{X}$. That is, $\mathcal{X}=\mathcal{T}\left(\mathcal{U}\right){\times }_3{\mathit{Q}}^T$, where $\mathcal{T}\left(\mathcal{U}\right)$ restructures $\mathcal{U}$ into a third-order tensor, and $\mathit{Q}$ satisfies ${\mathit{Q}}^T\mathit{Q}=\mathit{I}$ ($\mathit{I}$ denotes the identity matrix).

Additionally, we observe that the images exhibit similarity during adjacent temporal periods, which can be described by

$${∥\mathcal{D}\left(\mathcal{U}\right)∥}_F^2={∥\mathcal{D}{\mathcal{U}}_{\left(4\right)}∥}_F^2.$$

(9)

Here, $\mathcal{D}(·)$ represents a difference operator, $\mathcal{D}$ represents a difference matrix, and ${\mathcal{U}}_{\left(4\right)}$ is the unfolding of $\mathcal{U}$ in its fourth mode.

Using the proposed TriSps function and the prior knowledge of the image component, we propose the following cloud removal model:

$$\begin{array}{cc}\hfill & \underset{\genfrac{}{}{0pt}{}{\mathcal{C},\mathcal{X},{\mathit{Q}}^T\mathit{Q}=\mathit{I},}{\mathcal{U}=f_{bc}(\mathcal{O},\mathcal{C},\tilde{\mathcal{U}})}}{min}{\mathcal{J}}_{\lambda }\left(\mathcal{C}\right)+\sum _{r=1}^{dt}{∥{\mathit{X}}_r∥}_{*}+\frac{\gamma }{2}{∥\mathcal{D}{\mathcal{U}}_{\left(4\right)}∥}_F^2,\hfill \\ \hfill & s.t.,\mathcal{T}\left(\mathcal{U}\right)=\mathcal{X}{\times }_3\mathit{Q},\mathcal{O}=\mathcal{U}+\mathcal{C}.\hfill \end{array}$$

(10)

The first term in the objective function represents the prior knowledge derived from the cloud component $\mathcal{C}$, while the last two terms encapsulate the prior knowledge from the image component $\mathcal{U}$. $\lambda$ and $\gamma$ denote positive regularization parameters. $\mathcal{U}=f_{bc}(\mathcal{O},\mathcal{C},\tilde{\mathcal{U}})$ represents a box constraint designed to preserve cloud-free details within the image component, ensuring it remains consistent with the observed data. the adoption of the box-constraint strategy is motivated by the need to maintain the integrity of the cloud-free portions in $\mathcal{U}$. Without this constraint, these portions may differ from those in the observed data, leading to a decrease in the quality of the reconstruction. Thus, they need the assistance of some strategies to improve the reconstruction quality of the cloud-free part. in this paper, we use the following box-constraint strategy and extend it to the proposed model. the box-constraint function is determined by $\mathcal{U}=f_{bc}(\mathcal{O},\mathcal{C},\tilde{\mathcal{U}})$, whose ith mode-3 fiber is

$${\mathcal{U}}_{\left(3\right)}^i=\left\{\begin{array}{c}{\mathcal{O}}_{\left(3\right)}^i,\mathrm{if}\mathrm{avg}\left({\mathcal{C}}_{\left(3\right)}^i\right)<ϵ,\hfill \\ {\tilde{\mathcal{U}}}_{\left(3\right)}^i,\mathrm{otherwise},\hfill \end{array}\right.$$

(11)

where $ϵ\ge 0$ is a given thresholding value, and $\mathrm{avg}\left(\mathit{x}\right)$ denotes the average value of vector $\mathit{x}$.

3.3. Optimization Algorithm

This subsection outlines the development of the PAM algorithm [33], crafted to address our proposed model. To facilitate this, we integrate auxiliary variables $\mathcal{M}=\mathcal{X}$, $\mathcal{N}=\mathcal{C}$, $\mathcal{S}=\mathcal{C}$ and reformulate the model (10) as

$$\begin{array}{cc}\hfill & \underset{\genfrac{}{}{0pt}{}{\mathcal{C},\mathcal{X},\mathcal{N},{\mathit{Q}}^T\mathit{Q}=\mathit{I},}{\mathcal{S},\mathcal{M},\mathcal{U}=f_{bc}(\mathcal{O},\mathcal{C},\tilde{\mathcal{U}})}}{min}\sum _{r=1}^{dt}{∥{\mathit{M}}_r∥}_{*}+{\lambda }_1\sum _i{∥{\mathcal{N}}_{\left(1\right)}^i∥}_2+{\lambda }_2\sum _i{∥{\mathcal{S}}_{\left(2\right)}^i∥}_2+{\lambda }_3\sum _i{∥{\mathcal{C}}_{\left(3\right)}^i∥}_2+\frac{\gamma }{2}{∥\mathcal{D}{\mathcal{U}}_{\left(4\right)}∥}_F^2,\hfill \\ \hfill & s.t.,\mathcal{T}\left(\mathcal{U}\right)=\mathcal{X}{\times }_3\mathit{Q},\mathcal{O}=\mathcal{U}+\mathcal{C},\mathcal{M}=\mathcal{X},\mathcal{N}=\mathcal{C},\mathcal{S}=\mathcal{C},\hfill \end{array}$$

(12)

where ${\mathit{M}}_r$ denotes the frontal slice of $\mathcal{M}$ corresponding to index r.

The aforementioned constrained problem can be reformulated as follows:

$$\begin{array}{cc}\hfill & \underset{\genfrac{}{}{0pt}{}{\mathcal{C},\mathcal{X},\mathcal{N},{\mathit{Q}}^T\mathit{Q}=\mathit{I},}{\mathcal{S},\mathcal{M},\mathcal{U}=f_{bc}(\mathcal{O},\mathcal{C},\tilde{\mathcal{U}})}}{min}\sum _{r=1}^{dt}{∥{\mathit{M}}_r∥}_{*}+{\lambda }_1\sum _i{∥{\mathcal{N}}_{\left(1\right)}^i∥}_2+{\lambda }_2\sum _i{∥{\mathcal{S}}_{\left(2\right)}^i∥}_2+{\lambda }_3\sum _i{∥{\mathcal{C}}_{\left(3\right)}^i∥}_2\hfill \\ \hfill & +\frac{\gamma }{2}{∥\mathcal{D}{\mathcal{U}}_{\left(4\right)}∥}_F^2+\frac{{\eta }_1}{2}{∥\mathcal{T}\left(\mathcal{U}\right)-\mathcal{X}{\times }_3\mathit{Q}∥}_F^2+\frac{{\eta }_2}{2}{∥\mathcal{M}-\mathcal{X}∥}_F^2\hfill \\ \hfill & +\frac{{\eta }_3}{2}{∥\mathcal{O}-\mathcal{U}-\mathcal{C}∥}_F^2+\frac{{\eta }_4}{2}{∥\mathcal{N}-\mathcal{C}∥}_F^2+\frac{{\eta }_5}{2}{∥\mathcal{S}-\mathcal{C}∥}_F^2,\hfill \end{array}$$

where ${\eta }_i>0(i=1,\dots ,5)$ represents penalty parameters.

The aforementioned objective function refers to $g(\mathcal{U},\mathcal{C},\mathcal{X},\mathcal{M},\mathit{Q},\mathcal{N},\mathcal{S})$. Within the PAM-based algorithm framework, we iteratively update individual variables in an alternating fashion:

$$\left\{\begin{array}{c}{\mathcal{U}}^{s+1}=\underset{\mathcal{U}=f_{bc}(\mathcal{O},{\mathcal{C}}^s,\tilde{\mathcal{U}})}{\arg \min }g(\mathcal{U},{\mathcal{C}}^s,{\mathcal{X}}^s,{\mathcal{M}}^s,{\mathit{Q}}^s,{\mathcal{N}}^s,{\mathcal{S}}^s)+\frac{\mu }{2}{∥\mathcal{U}-{\mathcal{U}}^s∥}_F^2,\hfill \\ {\mathcal{C}}^{s+1}=\underset{\mathcal{C}}{\arg \min }\hspace{0.17em}g({\mathcal{U}}^{s+1},\mathcal{C},{\mathcal{X}}^s,{\mathcal{M}}^s,{\mathit{Q}}^s,{\mathcal{N}}^s,{\mathcal{S}}^s)+\frac{\mu }{2}{∥\mathcal{C}-{\mathcal{C}}^s∥}_F^2,\hfill \\ {\mathcal{X}}^{s+1}=\underset{\mathcal{X}}{\arg \min }\hspace{0.17em}g({\mathcal{U}}^{s+1},{\mathcal{C}}^{s+1},\mathcal{X},{\mathcal{M}}^s,{\mathit{Q}}^s,{\mathcal{N}}^s,{\mathcal{S}}^s)+\frac{\mu }{2}{∥\mathcal{X}-{\mathcal{X}}^s∥}_F^2,\hfill \\ {\mathcal{M}}^{s+1}=\underset{\mathcal{M}}{\arg \min }\hspace{0.17em}g({\mathcal{U}}^{s+1},{\mathcal{C}}^{s+1},{\mathcal{X}}^{s+1},\mathcal{M},{\mathit{Q}}^s,{\mathcal{N}}^s,{\mathcal{S}}^s)+\frac{\mu }{2}{∥\mathcal{M}-{\mathcal{M}}^s∥}_F^2,\hfill \\ {\mathit{Q}}^{s+1}=\underset{{\mathit{Q}}^T\mathit{Q}=\mathit{I}}{\arg \min }\hspace{0.17em}g({\mathcal{U}}^{s+1},{\mathcal{C}}^{s+1},{\mathcal{X}}^{s+1},{\mathcal{M}}^{s+1},\mathit{Q},{\mathcal{N}}^s,{\mathcal{S}}^s)+\frac{\mu }{2}{∥\mathit{Q}-{\mathit{Q}}^s∥}_F^2,\hfill \\ {\mathcal{N}}^{s+1}=\underset{\mathcal{N}}{\arg \min }\hspace{0.17em}g({\mathcal{U}}^{s+1},{\mathcal{C}}^{s+1},{\mathcal{X}}^{s+1},{\mathcal{M}}^{s+1},{\mathit{Q}}^{s+1},\mathcal{N},{\mathcal{S}}^s)+\frac{\mu }{2}{∥\mathcal{N}-{\mathcal{N}}^s∥}_F^2,\hfill \\ {\mathcal{S}}^{s+1}=\underset{\mathcal{S}}{\arg \min }\hspace{0.17em}g({\mathcal{U}}^{s+1},{\mathcal{C}}^{s+1},{\mathcal{X}}^{s+1},{\mathcal{M}}^{s+1},{\mathit{Q}}^{s+1},{\mathcal{N}}^{s+1},\mathcal{S})+\frac{\mu }{2}{∥\mathcal{S}-{\mathcal{S}}^s∥}_F^2.\hfill \end{array}\right.$$

(13)

Here, the superscript notation ^s signifies the outcome obtained after the sth iteration. $\mu$ represents a proximal parameter. Subsequently, each variable can be updated according to the following procedure.

• Updating the $\mathcal{U}$-subproblem

The $\mathcal{U}$-subproblem is

$$\begin{array}{c}\hfill \underset{\mathcal{U}=f_{bc}(\mathcal{O},\mathcal{C},\tilde{\mathcal{U}})}{min}\frac{{\eta }_1}{2}{∥\mathcal{T}\left(\mathcal{U}\right)-{\mathcal{X}}^s{\times }_3{\mathit{Q}}^s∥}_F^2+\frac{\gamma }{2}{∥\mathcal{D}{\mathcal{U}}_{\left(4\right)}∥}_F^2+\frac{{\eta }_3}{2}{∥\mathcal{O}-\mathcal{U}-{\mathcal{C}}^s∥}_F^2+\frac{\mu }{2}{∥\mathcal{U}-{\mathcal{U}}^s∥}_F^2.\end{array}$$

$\mathcal{T}(·)$, as a reversible reshape operator, allows for the rephrasing of the aforementioned equation.

$$\begin{array}{c}\underset{\mathcal{U}=f_{bc}(\mathcal{O},\mathcal{C},\tilde{\mathcal{U}})}{min}\frac{{\eta }_1}{2}{∥{\mathit{U}}_{\left(4\right)}-{\mathcal{T}}^{-1}{\left({\mathcal{X}}^s{\times }_3{\mathit{Q}}^s\right)}_{\left(4\right)}∥}_F^2+\frac{\gamma }{2}{∥\mathcal{D}{\mathit{U}}_{\left(4\right)}∥}_F^2\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+\frac{{\eta }_3}{2}{∥{\mathit{O}}_{\left(4\right)}-{\mathit{U}}_{\left(4\right)}-{\mathit{C}}_{\left(4\right)}^s∥}_F^2+\frac{\mu }{2}{∥{\mathit{U}}_{\left(4\right)}-{\mathit{U}}_{\left(4\right)}^s∥}_F^2.\hfill \end{array}$$

(14)

Here, the third tensor is reformatted to its initial four-dimensional form using the inverse operator ${\mathcal{T}}^{-1}(·)$, which is applied to transform the tensor from ${\mathbb{R}}^{a\times b\times dt}\to {\mathbb{R}}^{a\times b\times d\times t}$. Clearly, $\mathcal{U}$ has the following solution:

$$\begin{array}{cc}\hfill & {\mathit{U}}_{\left(4\right)}^{s+1}={\left(({\eta }_1+{\eta }_3+\mu )\mathit{I}+\gamma {\mathcal{D}}^T\mathcal{D}\right)}^{-1}\left({\eta }_1{\mathcal{T}}^{-1}{\left({\mathcal{X}}^s{\times }_3{\mathit{Q}}^s\right)}_{\left(4\right)}+{\eta }_3\left({\mathit{O}}_{\left(4\right)}-{\mathit{C}}_{\left(4\right)}^s\right)+\mu {\mathit{U}}_{\left(4\right)}^s\right).\hfill \end{array}$$

In this formulation, $\mathit{I}\in {\mathbb{R}}^{t\times t}$ represents the identity matrix. Incorporating the box constraint, we obtain the image component, as detailed below:

$${\tilde{\mathcal{U}}}^{s+1}={\mathrm{Fold}}_4\left({\mathit{U}}_{\left(4\right)}^{s+1}\right).$$

(15)

In this context, the matrix is reformatted into a tensor by the operator ${\mathrm{Fold}}_{\left(4\right)}(·)$. the image component $\mathcal{U}$ is computed by the box-constraint function $\mathcal{U}=f_{bc}(\mathcal{O},\mathcal{C},\tilde{\mathcal{U}})$.

• Updating the $\mathcal{X}$-subproblem

The $\mathcal{X}$-subproblem is written as

$$\begin{array}{c}\hfill \underset{\mathcal{X}}{min}\frac{{\eta }_1}{2}{∥\mathcal{T}\left({\mathcal{U}}^{s+1}\right)-\mathcal{X}{\times }_3{\mathit{Q}}^s∥}_F^2+\frac{{\eta }_2}{2}{∥{\mathcal{M}}^s-\mathcal{X}∥}_F^2+\frac{\mu }{2}{∥\mathcal{X}-{\mathcal{X}}^s∥}_F^2.\end{array}$$

(16)

Equation (16) yields a closed-form solution, which is given by

$${\mathcal{X}}^{s+1}=\frac{{\eta }_1\mathcal{T}\left({\mathcal{U}}^{s+1}\right){\times }_3{\left({\mathit{Q}}^s\right)}^T+{\eta }_2{\mathcal{M}}^s+\mu {\mathcal{X}}^s}{{\eta }_1+{\eta }_2+\mu }.$$

(17)

• Updating the $\mathcal{C}$-subproblem

The subproblem $\mathcal{C}$ is outlined below.

$$\begin{array}{c}\hfill \underset{\mathcal{C}}{min}{\lambda }_3\sum _i{∥{\mathcal{C}}_{\left(3\right)}^i∥}_2+\frac{{\eta }_3}{2}{∥\mathcal{O}-\mathcal{U}-\mathcal{C}∥}_F^2+\frac{{\eta }_4}{2}{∥\mathcal{N}-\mathcal{C}∥}_F^2+\frac{{\eta }_5}{2}{∥\mathcal{S}-\mathcal{C}∥}_F^2+\frac{\mu }{2}{∥\mathcal{C}-{\mathcal{C}}^s∥}_F^2.\end{array}$$

By integrating the last four parts, the presented problem can be equivalently rewritten as follows:

$$\begin{array}{c}\hfill \underset{\mathcal{C}}{min}{\lambda }_3\sum _i{∥{\mathcal{C}}_{\left(3\right)}^i∥}_2+\frac{\delta +\mu }{2}{∥\mathcal{C}-\mathcal{Q}∥}_F^2,\end{array}$$

with $\mathcal{Q}=\frac{{\eta }_3(\mathcal{O}-\mathcal{U})+{\eta }_4\mathcal{N}+{\eta }_5\mathcal{S}+\mu {\mathcal{C}}^s}{\delta +\mu }$ and $\delta ={\sum }_{i=3}^5{\eta }_i$. Subsequently, the $\mathcal{C}$-subproblem can be dissected into the following constituent subproblems:

$$\begin{array}{c}\hfill \underset{\mathcal{C}}{min}\sum _i\left({\lambda }_3{∥{\mathcal{C}}_{\left(3\right)}^i∥}_2+\frac{\gamma +\mu }{2}{∥{\mathcal{C}}_{\left(3\right)}^i-{\mathcal{Q}}_{\left(3\right)}^i∥}_F^2\right),\end{array}$$

where ${\mathcal{Q}}_{\left(3\right)}^i$ is the tube of $\mathcal{Q}$.

Then, the solution of the tube ${\mathcal{C}}_{\left(3\right)}^i$ is given by

$${\left({\mathcal{C}}^{s+1}\right)}_{\left(3\right)}^i=\max ({∥{\mathcal{Q}}_{\left(3\right)}^i∥}_2-\frac{{\lambda }_3}{\gamma +\mu },0)\frac{{\mathcal{Q}}_{\left(3\right)}^i}{{∥{\mathcal{Q}}_{\left(3\right)}^i∥}_2},$$

(18)

where we define $\frac{0}{0}=1$.

• Updating the $\mathcal{M}$-subproblem

The $\mathcal{M}$-subproblem is

$$\begin{array}{c}\hfill \underset{\mathcal{M}}{min}\sum _{r=1}^{dt}{∥{\mathit{M}}_r∥}_{*}+\frac{{\eta }_2}{2}{∥\mathcal{M}-{\mathcal{X}}^{s+1}∥}_F^2+\frac{\mu }{2}{∥\mathcal{M}-{\mathcal{M}}^s∥}_F^2.\end{array}$$

By including the two final terms, the aforementioned issue can be transformed into the following equivalent form:

$$\begin{array}{c}\hfill \underset{\mathcal{M}}{min}\sum _{r=1}^{dt}\left({∥{\mathit{M}}_r∥}_{*}+\frac{{\eta }_2+\mu }{2}{∥{\mathit{M}}_r-{\mathit{R}}_r∥}_F^2\right),\end{array}$$

where ${\mathcal{R}}_r$ denotes the frontal slice of $\mathcal{R}=\frac{{\eta }_2{\mathcal{X}}^{s+1}+\mu {\mathcal{M}}^s}{{\eta }_2+\mu }$ corresponding to index r. As a result, the $\mathcal{M}$-subproblem can be effectively addressed through the following $dt$ subproblems:

$$\underset{M_r}{min}{∥{\mathit{M}}_r∥}_{*}+\frac{{\eta }_2+\mu }{2}{∥{\mathit{M}}_r-{\mathit{R}}_r∥}_F^2,r=1,\dots ,dt.$$

(19)

Each subproblem’s (19) solution can be achieved through the implementation of a singular value thresholding (SVT) operator, specifically

$${\mathit{M}}_r^{s+1}=\mathrm{SVT}({\mathit{R}}_r,\frac{1}{{\eta }_2+\mu })=\mathit{U}diag(max({\sigma }_i-\frac{1}{{\eta }_2+\mu },0)){\mathit{V}}^T.$$

(20)

In this context, ${\sigma }_i$ represents the ith singular value on $\mathbf{\Sigma }$. the matrix ${\mathit{R}}_r$ undergoes singular value decomposition to yield the matrices $\mathit{U}$, $\mathbf{\Sigma }$, and ${\mathit{V}}^T$.

• Updating the $\mathcal{N}$-subproblem

The $\mathcal{N}$-subproblem is

$$\begin{array}{c}\hfill \underset{\mathcal{N}}{min}{\lambda }_1\sum _i{∥{\mathcal{N}}_{\left(1\right)}^i∥}_2+\frac{{\eta }_4}{2}{∥\mathcal{N}-\mathcal{C}∥}_F^2+\frac{\mu }{2}{∥\mathcal{N}-{\mathcal{N}}^s∥}_F^2.\end{array}$$

By including the two final terms, the aforementioned issue can be written as

$$\underset{\mathcal{N}}{min}{\lambda }_1\sum _i{∥{\mathcal{N}}_{\left(1\right)}^i∥}_2+\frac{{\eta }_4+\mu }{2}{∥\mathcal{N}-\mathcal{F}∥}_F^2,$$

where $\mathcal{F}=\frac{{\eta }_4\mathcal{C}+\mu {\mathcal{N}}^s}{{\eta }_4+\mu }$. Next, the aforementioned subproblem can be divided into the following subproblems:

$$\underset{\mathcal{N}}{min}\sum _i\left({\lambda }_1{∥{\mathcal{N}}_{\left(1\right)}^i∥}_2+\frac{{\eta }_4+\mu }{2}{∥{\mathcal{N}}_{\left(1\right)}^i-{\mathcal{F}}_{\left(1\right)}^i∥}_F^2\right).$$

Then, the solution of the tube of ${\mathcal{N}}_{\left(1\right)}^i$ can be obtained:

$${\left({\mathcal{N}}^{s+1}\right)}_{\left(1\right)}^i=\max \left({∥{\mathcal{F}}_{\left(1\right)}^i∥}_2-\frac{{\lambda }_1}{{\eta }_4+\mu },0\right)\frac{{\mathcal{F}}_{\left(1\right)}^i}{{∥{\mathcal{F}}_{\left(1\right)}^i∥}_2}.$$

(21)

• Updating the $\mathcal{S}$-subproblem

The $\mathcal{S}$-subproblem is

$$\begin{array}{c}\hfill \underset{\mathcal{S}}{min}{\lambda }_2\sum _i{∥{\mathcal{S}}_{\left(2\right)}^i∥}_2+\frac{{\eta }_5}{2}{∥\mathcal{S}-\mathcal{C}∥}_F^2+\frac{\mu }{2}{∥\mathcal{S}-{\mathcal{S}}^s∥}_F^2.\end{array}$$

Similarly to solving the $\mathcal{N}$-subproblem, we incorporate the last two terms:

$$\underset{\mathcal{S}}{min}{\lambda }_2\sum _i{∥{\mathcal{S}}_{\left(2\right)}^i∥}_2+\frac{{\eta }_5+\mu }{2}{∥\mathcal{S}-\mathcal{B}∥}_F^2,$$

where $\mathcal{B}=\frac{{\eta }_5\mathcal{C}+\mu {\mathcal{S}}^s}{{\eta }_5+\mu }$. The aforementioned subproblem can be divided into the following subproblems:

$$\underset{\mathcal{S}}{min}\sum _i\left({\lambda }_2{∥{\mathcal{S}}_{\left(2\right)}^i∥}_2+\frac{{\eta }_5}{2}{∥{\mathcal{S}}_{\left(2\right)}^i-{\left({\mathcal{C}}^{s+1}\right)}_{\left(2\right)}^i∥}_F^2+\frac{\mu }{2}{∥{\mathcal{S}}_{\left(2\right)}^i-{\left({\mathcal{S}}^s\right)}_{\left(2\right)}^i∥}_F^2\right).$$

Then, the solution of the tube of ${\mathcal{S}}_{\left(2\right)}^i$ can be obtained:

$${\left({\mathcal{S}}^{s+1}\right)}_{\left(2\right)}^i=\max \left({∥{\mathcal{B}}_{\left(2\right)}^i∥}_2-\frac{{\lambda }_2}{{\eta }_5+\mu },0\right)\frac{{\mathcal{B}}_{\left(2\right)}^i}{{∥{\mathcal{B}}_{\left(2\right)}^i∥}_2}.$$

(22)

• Updating the $\mathit{Q}$-subproblem

The $\mathit{Q}$-subproblem is

$$\begin{array}{c}\hfill \underset{{\mathit{Q}}^T\mathit{Q}=\mathit{I}}{min}\frac{{\eta }_1}{2}{∥\mathcal{T}\left({\mathcal{U}}^{s+1}\right)-{\mathcal{X}}^{s+1}{\times }_3\mathit{Q}∥}_F^2+\frac{\mu }{2}{∥\mathit{Q}-{\mathit{Q}}^s∥}_F^2.\end{array}$$

The problem can be addressed by solving the following formulation:

$$\begin{array}{c}\underset{{\mathit{Q}}^T\mathit{Q}=\mathit{I}}{min}\frac{{\eta }_1}{2}{∥\mathcal{T}\left({\mathcal{U}}^{s+1}\right)-{\mathcal{X}}^{s+1}{\times }_3\mathit{Q}∥}_F^2+\frac{\mu }{2}{∥\mathit{Q}-{\mathit{Q}}^s∥}_F^2\hfill \\ =\underset{{\mathit{Q}}^T\mathit{Q}=\mathit{I}}{min}\frac{{\eta }_1}{2}{∥{\left(\mathcal{T}\left({\mathcal{U}}^{s+1}\right)\right)}_{\left(3\right)}-\mathit{Q}{\mathit{X}}_{\left(3\right)}^{s+1}∥}_F^2+\frac{\mu }{2}{∥\mathit{Q}-{\mathit{Q}}^s∥}_F^2\hfill \\ =\underset{{\mathit{Q}}^T\mathit{Q}=\mathit{I}}{min}\frac{{\eta }_1}{2}\mathrm{Tr}\left\{{\left[{\left(\mathcal{T}\left({\mathcal{U}}^{s+1}\right)\right)}_{\left(3\right)}-\mathit{Q}{\mathit{X}}_{\left(3\right)}^{s+1}\right]}^T\left[{\left(\mathcal{T}\left({\mathcal{U}}^{s+1}\right)\right)}_{\left(3\right)}-\mathit{Q}{\mathit{X}}_{\left(3\right)}^{s+1}\right]\right\}\hfill \\ +\frac{\mu }{2}\mathrm{Tr}\left[{(\mathit{Q}-{\mathit{Q}}^s)}^T(\mathit{Q}-{\mathit{Q}}^s)\right]\hfill \\ =\underset{{\mathit{Q}}^T\mathit{Q}=\mathit{I}}{max}\mathrm{Tr}\left({\eta }_1{\left(\mathcal{T}\left({\mathcal{U}}^{s+1}\right)\right)}_{\left(3\right)}{\left({\mathit{X}}_{\left(3\right)}^{s+1}\right)}^T+\mu {\mathit{Q}}^s\right){\mathit{Q}}^T.\hfill \end{array}$$

Here, $\mathrm{Tr}(·)$ signifies the trace of a matrix.

This subproblem yields a closed-form solution, which is given by

$${\mathit{Q}}^{s+1}=\widehat{\mathit{U}}{\widehat{\mathit{V}}}^T.$$

(23)

where $\widehat{\mathit{U}}\widehat{\mathbf{\Sigma }}{\widehat{\mathit{V}}}^T=\mathrm{SVD}\left({\eta }_1{\left(\mathcal{T}\left({\mathcal{U}}^{s+1}\right)\right)}_{\left(3\right)}{\left({\mathit{X}}_{\left(3\right)}^{s+1}\right)}^T+\mu {\mathit{Q}}^s\right)$.

We outline the algorithm for cloud detection and removal in Algorithm 1.

Algorithm 1 Tri-fiber-wise sparse collaborative low-rank prior knowledge algorithm.

Input: Multitemporal images contaminated by clouds $\mathcal{O}$ and the parameters ${\lambda }_1,{\lambda }_2,{\lambda }_3$, ${\eta }_i(i=1,\dots ,5)$, and $\gamma$.

1: for $s=1$ to $maxiter$ do 2:       Update ${\mathcal{U}}^{s+1}$ by (15); 3:       Update ${\mathcal{X}}^{s+1}$ via (17); 4:       Update ${\mathcal{C}}^{s+1}$ via (18); 5:       Update ${\mathcal{N}}^{s+1}$ via (21); 6:       Update ${\mathcal{S}}^{s+1}$ via (22); 7:       for $i=1$ to $dt$ do 8:             Update ${\mathit{M}}_r^{s+1}$ via (20); 9:       end for 10:     Update ${\mathit{Q}}^{s+1}$ via (23); 11:     Check for the convergence. If satisfied, then stop. 12: end for

Output: Image component $\mathcal{U}$.

4. Experiments

In this section, the advantages of our method in cloud detection and removal are investigated. We evaluated the effectiveness of the sparse function against the TNN [37], ALM-IPG [20], TVLRSDC [28], and BC-SLRpGS [32] methods. The algorithm was stopped when

$$\frac{{∥{\mathcal{U}}^{s+1}-{\mathcal{U}}^s∥}_F}{{∥{\mathcal{U}}^s∥}_F}<tol\mathrm{and}\frac{{∥{\mathcal{C}}^{s+1}-{\mathcal{C}}^s∥}_F}{{∥{\mathcal{C}}^s∥}_F}<tol,$$

or the number of iterations exceeded $maxiter=2000$. The initial values were set to ${\mathcal{U}}^0=\mathbf{0}$, ${\mathcal{C}}^0=\mathbf{0}$, ${\mathcal{M}}^0=\mathbf{0}$, ${\mathcal{S}}^0=\mathbf{0}$, ${\mathcal{X}}^0=\mathbf{0}$, ${\mathcal{N}}^0=\mathbf{0}$, ${\mathcal{Q}}^0=\mathbf{0}$, and $\mu =0.01$. We set $tol={10}^{-5}$ and $maxiter=2000$. The experiments were run on Windows 10 and MATLAB (R2017b). This computer has an Intel Core i7-9700K CPU @ 3.60 GHz with 16 GB of RAM.

4.1. Synthetic Experiments

Initially, we employed multitemporal remote sensing images to showcase the efficiency of our proposed technique in restoring information hidden by cloud cover. We generated four simulated datasets—Munich, Picardie, France, and Beijing—by extracting data from the Sentinel-2 (https://earthexplorer.usgs.gov) and Landsat-8 (https://theia.cnes.fr/atdistrib/rocket/#/home) (accessed on 21 April 2024) collections. The details of these datasets are comprehensively outlined in

Table 2. Multitemporal remote sensing images for synthetic experiments.

Data	Image Size	Spectral	Temporal	GSD	Source
Munich	512 × 512	3	4	30 m	Landsat-8
Picardie	500 × 500	6	3	20 m	Sentinel-2
Beijing	256 × 256	6	4	20 m	Sentinel-2
France	400 × 400	7	3	30 m	Landsat-8

and visually represented in Figure 2, Figure 3, Figure 4 and Figure 5. We refer to the ground sampling distance as GSD. Various types of cloud masks were applied to these multitemporal images to augment the complexity of the task.

To rigorously assess the effectiveness of the proposed method, we employ three quantitative metrics, mean PSNR, mean CC [39], and mean SSIM [40], as established benchmarks for this type of analysis. Superior performance is indicated by higher values across these metrics. In

Table 3. Quantitative metrics for simulated data. Highest values are emphasized in bold.

Dataset	Index	Method
		Observed	TNN	ALM-IPG	TVLRSDC	BC-SLRpGS	Proposed
Munich	PSNR	4.29	26.66	23.26	26.23	27.81	29.6
	SSIM	0.4769	0.8344	0.8462	0.8385	0.8506	0.8496
	CC	0.154	0.8897	0.841	0.8545	0.9013	0.9184
Picardie	PSNR	4.56	42.89	43.49	37.33	43.35	44.06
	SSIM	0.543	0.987	0.9924	0.9493	0.9899	0.9918
	CC	0.0904	0.9397	0.9509	0.7558	0.9421	0.9514
Beijing	PSNR	5.84	36.71	38.34	36.14	38.88	39.58
	SSIM	0.6182	0.9379	0.9608	0.949	0.9622	0.9638
	CC	0.0196	0.9384	0.9646	0.9287	0.9689	0.9729
France	PSNR	6.28	28.31	28.32	27.09	29.56	30.14
	SSIM	0.6224	0.7947	0.8001	0.8049	0.8488	0.8536
	CC	0.2398	0.9603	0.9582	0.9194	0.9661	0.9697

, we provide a comprehensive quantitative comparison of our method against established methods like TNN, ALM-IPG, TVLRSDC, and BC-SLRpGS. The highest values of PSNR, CC, and SSIM are indicated in bold for clear distinction. Our method consistently surpasses the comparative methods in PSNR across all datasets. Regarding SSIM, our method displays a slight disadvantage compared to the BC-SLRpGS method in the Munich dataset and to the ALM-IPG method in the Picardie dataset, highlighting areas for further improvement.

To provide a more intuitive comparison of the methods’ performance, we detail the cloud removal results in Figure 6, Figure 7, Figure 8 and Figure 9. These figures include zoomed-in patches and corresponding residual images for a more nuanced comparison. Our method and BC-SLRpGS successfully reconstructed most cloud information in the Munich dataset, whereas other methods introduced distortions in image details. In the Picardie dataset, the TVLRSDC method was unable to adequately reconstruct cloud regions, in contrast to TNN, ALM-IPG, BC-SLRpGS, and our proposed method, which yielded satisfactory results. Notably, the TNN and ALM-IPG methods necessitate a predefined cloud mask, potentially incorporating additional information into the reconstruction process. Our method produced darker residual images, as seen at the bottom of Figure 6 and Figure 7, signifying a closer similarity to cloud-free images and, thus, more effective cloud removal.

The exceptional performance of our method can be attributed to our effective utilization of the sparse structure of mode-1, -2, and -3 fibers. For the Beijing dataset, our method, along with ALM-IPG and BC-SLRpGS, achieved promising outcomes, outperforming TNN and TVLRSDC methods, which were visually inferior. The zoomed-in views in Figure 8 further substantiate our method’s clarity and precision in reconstructing more detailed and clearer images compared to other methods. For the France dataset, as illustrated in Figure 9, all tested methods failed to achieve promising results, primarily due to the dataset’s intricate details that defy reconstruction through global correlations. Nonetheless, our method yielded notably clearer outcomes compared to the others and demonstrated superior accuracy in cloud mask detection relative to BC-SLRpGS.

4.2. Real Experiments

Two real datasets, namely, the Eura and Morocco data, as outlined in

Table 4. Multitemporal remote sensing images for real experiments.

Data	Image Size	Spectral	Temporal	GSD	Source
Eure	400 × 400	4	4	10 m	Sentinel-2
Morocco	600 × 600	4	4	10 m	Sentinel-2

and visualized in Figure 10 and Figure 11, were used to assess the efficacy of our method. The reconstructed results for the Eura and Morocco datasets using various methods are depicted in Figure 12 and Figure 13, respectively. From Figure 12, it is evident that the TNN, ALM-IPG, and TVLRSDC methods are unsuccessful in effectively removing the clouds and reconstructing cloud information. Conversely, both BC-SLRpGS and the proposed method exhibit similar performance in this aspect. From the zoomed-in figures presented in Figure 12, it is evident that our method delivers smoother results and captures a greater amount of detail in comparison to the BC-SLRpGS method. As shown by the visual comparisons in Figure 13, our method demonstrates enhanced performance compared to the other methods. From temporal node 1 in Figure 13, the proposed method effectively eliminates the cloud, while the result of TVLRSDC still exhibits some discrete clouds. Furthermore, as displayed in the zoomed-in figure in Figure 13, our method produces a clearer reconstruction result than the other compared methods. From temporal node 2 in Figure 13, we find that the proposed method removes the cloud areas well and can preserve the cloud-free areas, whereas the TNN and ALM-IPG methods change the cloud-free information after the reconstruction process. In conclusion, the proposed method outperforms other methods in terms of both quantitative metrics and visual quality.

5. Conclusions

We have introduced a novel tri-fiber-wise sparse function to characterize the cloud component. By leveraging sparse prior information, our method excels at detecting clouds and effectively reconstructing contaminated values, providing robust results. Moreover, we described the global prior of the image component by tensor multi-rank. Utilizing the introduced novel sparse and low-rank functions, we have developed a cloud removal model that incorporates a box-constrained strategy. This model not only effectively detects clouds but also simultaneously estimates missing information. The experiments demonstrate the superior effectiveness of the proposed method to other cloud removal techniques.