A Multi-Objective Gray Consistency Correction Method for Mosaicking Regional SAR Intensity Images with Brightness Anomalies

Abstract

In the process of mosaicking regional synthetic aperture radar (SAR) intensity images, multiple images with significant brightness anomalies can cause a considerable number of pixels to exceed the grayscale quantization range. Applying traditional color harmonization methods increases this issue, causing a loss of brightness information. We propose a multi-objective gray consistency correction method designed explicitly for mosaicking regional SAR intensity images with brightness anomalies to address this. We constructed a two-objective optimization model to ensure regional image gray consistency and mitigate brightness information loss. The truncation values of brightness anomaly images were selected as decision variables, maximizing the overall gray consistency of overlapping image pairs and minimizing the number of pixels with grayscale values that were out of bounds as the objective functions. To synchronously solve the truncation values of brightness anomaly images and linear stretch parameters of all images, a hybrid framework that combines the non-dominated sorting genetic algorithm II (NSGA-II) with the quadratic programming (QP) algorithm was proposed. Two large-area experimental results show that the proposed method achieves a balanced optimization between gray consistency and brightness information loss for regional SAR intensity image mosaicking. Compared with the traditional method, our method reduces brightness information loss by 99.552–99.647% and 99.973–99.969%, respectively, while maintaining better peak signal-to-noise ratio performance.

1. Introduction

Synthetic aperture radar (SAR) enables all-weather, all-day Earth observations and is widely used in environmental monitoring [1], land use classification [2], and disaster management [3]. Owing to the limited imaging width of SAR satellites, large-scale remote sensing mapping often requires stitching multiple scene images into regional images through image mosaicking [4,5]. Gray consistency correction is a critical step in regional SAR image mosaicking, and radiometric normalization is currently a widely used correction method [6,7]. Ensuring radiometric consistency among images is essential, enabling surface reflectance or normalized values under varying conditions to be standardized on a standard scale [8].

Seasonal variations, radar signal attenuation, and other factors can degrade the radiometric quality of images, leading to significant discrepancies in SAR images even after calibration [9,10]. As shown in Figure 1a, the brightness anomaly SAR intensity image is darker overall. As shown in Figure 1b, most grayscale values in the brightness anomaly image are relatively small, with a low gray mean and standard deviation of the image. Although the traditional color harmonization method can resolve color consistency issues in images with brightness anomalies, it may also lead to the loss of brightness information. For instance, the existing color harmonization method determines the stretch parameters of all images by solving a quadratic programming optimization problem [6], which can enhance the gray consistency of SAR intensity images. However, it may also result in grayscale values exceeding the quantization range after image stretching, as shown in the red markers in Figure 2. The pixel values exceeding the quantization range in the first, third, and fourth images were 559, 14,376, and 10,622, respectively. Typically, out-of-bounds pixel values are directly set to 0 or the maximum grayscale value, leading to the loss of brightness information.

There are few publicly reported methods for the gray consistency of SAR intensity images with brightness anomalies. Most studies have focused on the gray consistency of images without brightness anomalies, preserving image details while optimizing gray consistency [11], but they have ignored the problem of brightness information loss.

Most current radiometric normalization methods for SAR intensity images have been inspired by optical image processing techniques [5]. Radiometric normalization approaches for multiple images can be categorized into global, local, or combination models [7]. The global strategy is a mainstream method [12]. The global model employs a single function to correct the overall color consistency of intensity images, while the local model uses multiple local functions to eliminate intensity discrepancies in overlapping regions. The combined model leverages both approaches’ advantages, effectively reducing local and global color discrepancies.

Global models often assume that the radiometric relationships between image pairs can be fitted using a linear or nonlinear function [5]. The least-squares method and linear regression model using pixel pairs in overlapping regions are typically used to process images that require high registration accuracy and slight radiation differences [11]. The linear model assumes that the intensity discrepancies between different images are characterized by a linear relationship [6]. To describe the linear relationship between images, researchers have constructed linear models based on pseudo-invariant features (PIFs) [13,14,15]. Nonlinear models, such as histogram matching, align the histograms of two images to achieve similar brightness and contrast characteristics [16,17,18,19,20].

The basic principle of a local model is to calibrate local radiometric differences based on regional characteristics [7]. To eliminate the color differences that exist in local areas and preserve the image gradient information, Li proposed a color-correction model based on a set of local grid linear models [21]. Tai addressed the problem of local color transfer between two images via probabilistic segmentation using a new expectation maximization (EM) algorithm [22]. Based on global and local models, some scholars proposed combination models. Yu presented an auto-adapting global-to-local color balancing method in which global optimization transforms the color difference elimination problem into least-squares optimization, and local optimization eliminates the color differences in the overlapping areas of the target images with the gamma transformation algorithm [23]. To address radiometric normalization (RN) in multiple remote sensing (MRS) images, Zhang proposed a block adjustment-based RN method that considers global and local radiometric differences [24].

Inspired by previous studies [5], this paper focuses on a multi-objective gray consistency correction for mosaicking regional SAR intensity images with brightness anomalies. The motivation of this study is to maintain the overall brightness consistency of image pairs in the overlapping regions and reduce the number of out-of-bound pixels. They are used as two objective functions of the multi-objective optimization model, and trade-offs are made between them according to user preferences. The difficulty in controlling information loss arises from simply using the maximum and minimum values of anomalous images as boundaries, as they do not effectively describe the goal of maintaining the number of out-of-bounds pixels. This overly strict constraint limits the feasible domain, and solving for two conflicting optimization objectives becomes challenging. Our method introduces a flexible inequality constraint for a balanced optimization approach. The optimization model can better manage the trade-off between gray consistency and information preservation. The challenge of multi-objective optimization is effectively solved by exploring the boundary of constraint relaxation. The main contributions can be summarized as follows:

(1)

A multi-objective optimization model for SAR image gray consistency correction is proposed to tackle regional SAR intensity images with brightness anomalies, which simultaneously optimizes gray consistency and brightness information loss;

(2)

The maximum truncation values of brightness anomaly images in the inequality constraints are formulated as decision variables, optimizing two objective functions through the adaptive adjustment of the constraint relaxation degrees;

(3)

A hybrid solution framework combining the NSGA-II and QP algorithm is proposed, effectively addressing the complex optimization problem involving multiple constraints and multi-type objective functions.

The remaining sections of this paper are organized as follows: Section 2 describes the framework of the proposed method, and introduces a detailed description of the multi-objective optimization model and the solution algorithm framework, respectively. Section 3 and Section 4 describes the results of the different algorithms and discusses the experimental results. Finally, Section 5 presents the conclusions.

2. Materials and Methods

The overall workflow of the proposed method is illustrated in Figure 3, which comprises three main steps: (1) Data Preprocessing: To obtain the input parameters for the optimization model, the grayscale values of all images, the grayscale values in the overlapping regions of image pairs, and the histograms of the brightness anomaly images were collected. (2) Optimization Modeling: Maximizing the overall gray consistency of overlapping image pairs and minimizing brightness information loss were formulated as two objective functions. The maximum brightness truncation values of the anomalous images were selected as decision variables. The preservation of the overall brightness and contrast of the images before and after processing was considered as an equality constraint. The grayscale values of the images after stretching were kept within the quantization range as inequality constraints. This model formulated the specific problem of gray consistency as a mathematical problem. (3) Solution Algorithm: A hybrid solution framework integrating the NSGA-II [25,26] and QP [27] algorithms was developed to solve the complex optimization model. This framework enabled the simultaneous determination of truncation values for brightness anomaly images and linear stretch parameters for all images. Each solution in the final Pareto optimal front solution set represented a unique combination of stretch parameters for all images and corresponding truncation values for brightness anomaly images.

2.1. Multi-Objective Optimization Model

Equality constraints were set in the traditional color harmonization model [6], ensuring that the overall brightness and contrast of the image remained consistent after stretching. However, when the dataset contains images with abnormal brightness, relying solely on equality constraints may lead to the loss of brightness information. To overcome this technical challenge, we introduced inequality constraints to control the number of pixels with grayscale values out of bounds. Considering the extreme distribution of pixel values in the abnormal images, the truncated values in the inequality constraint were selected as decision variables. This allows for adaptive ‘relaxation’ of the constraints, achieving a balanced optimization between gray consistency and brightness information loss.

2.1.1. Linear Stretch Model

A gray consistency model was defined for the $n$ original images. In this study, we consider a general linear transformation model by applying linear stretching to the $n$ images, which is [15]

$$Y^{\prime }=aY+b$$

(1)

where $a$ and $b$ are the linear stretch parameters of the image and the variables to be solved; $Y$ represents all valid pixel values before linear stretching; and $Y^{\prime }$ represents all valid pixel values after linear stretching.

2.1.2. Objective Functions

This study considers two objective functions. Maximizing the overall gray consistency of overlapping areas after stretching is defined as Objective Function 1 (Obj 1). The formula is as follows:

$$minimize\sum _{i=1}^{n-1}\sum _{j=i+1}^n{S}_{ij}{\left({\mu }_{ij}-{\mu }_{ji}\right)}^2+\sum _{i=1}^{n-1}\sum _{j=i+1}^n{S}_{ij}{\left({\sigma }_{ij}-{\sigma }_{ji}\right)}^2$$

(2)

$${\mu }_{ij}=a_i{\mu }_i+b_i$$

(3)

$${\mu }_{ji}=a_j{\mu }_j+b_j$$

(4)

$${\sigma }_{ij}=a_i{\sigma }_i$$

(5)

$${\sigma }_{ji}=a_j{\sigma }_j$$

(6)

where $i$ and $j$ represent image pairs with overlapping regions; $n$ is the total number of images; ${\mu }_{ij}$ and ${\mu }_{ji}$ are the gray means in the overlapping part of the images $i$ and $j$ after stretching; ${\sigma }_{ij}$ and ${\sigma }_{ji}$ are the standard deviations in the overlapping part of the images $i$ and $j$ after stretching; $a_i$ and $b_i$ are the linear stretching parameters of image $i$; $a_j$ and $b_j$ are the linear stretching parameters of image $j$; ${\mu }_i$ is the gray mean of the overlapping region of the image $i$ before stretching; ${\mu }_j$ is the gray mean of the overlapping region of the image $j$ before stretching; ${\sigma }_i$ is the standard deviation of the overlapping region of the image $i$ before stretching; ${\sigma }_j$ is the standard deviation of the overlapping region of the image $j$ before stretching; $S_{ij}$ is equal to $S_{ji}$, which denotes the number of pixels in the overlapping part of the images $i$ and $j$. Figure 4 shows the overlapping image pairs.

Let $x$ be the parameter vector of the $n$ images: $x=({a_1,\dots a}_n,b_1\dots b_n)$. The first objective function can be expressed as a quadratic function of the parameter vector $x$. The formula is as follows:

$$minimize{\displaystyle \frac{1}{2}}{x}^{T}Qx$$

(7)

$\mathrm{Q}$ is a $2\mathrm{n}\times 2n$ matrix, which is equal to

$$\mathrm{Q}=\left\{\begin{array}{cc}\left[\begin{array}{ccc}{H}_{11}^{\mu }+H_{11}^{\sigma }& \dots & H_{1n}^{\mu }+H_{1n}^{\sigma }\\ ⋮& \ddots & ⋮\\ H_{n1}^{\mu }+H_{n1}^{\sigma }& \dots & H_{nn}^{\mu }+H_{nn}^{\sigma }\end{array}\right]& \left[\begin{array}{ccc}{K}_{11}& \cdots & K_{1n}\\ ⋮& \ddots & ⋮\\ K_{n1}& \cdots & K_{nn}\end{array}\right]\\ \left[\begin{array}{ccc}{K}_{11}& \cdots & K_{1n}\\ ⋮& \ddots & ⋮\\ K_{n1}& \cdots & K_{nn}\end{array}\right]& \left[\begin{array}{ccc}{L}_{11}& \cdots & L_{1n}\\ ⋮& \ddots & ⋮\\ L_{n1}& \cdots & L_{nn}\end{array}\right]\end{array}\right\}$$

(8)

$$H_{ij}^{\mu }=-S_{ij}{\mu }_{ij}{\mu }_{ji},\hspace{1em}if\hspace{1em}i\ne j\hspace{1em}and\hspace{1em}{H}_{ii}^{\mu }=\sum _{j=1,j\ne i}^n{S}_{ij}{\mu }_{ij}^2$$

(9)

$$H_{ij}^{\sigma }=-S_{ij}{\sigma }_{ij}{\sigma }_{ji},\hspace{1em}if\hspace{1em}i\ne j\hspace{1em}and\hspace{1em}{H}_{ii}^{\sigma }=\sum _{j=1,j\ne i}^n{S}_{ij}{\sigma }_{ij}^2$$

(10)

$$K_{ij}=-S_{ij}{\mu }_{ij},\hspace{1em}if\hspace{1em}i\ne j\hspace{1em}and\hspace{1em}{K}_{ii}=\sum _{j=1,j\ne i}^n{S}_{ij}{\mu }_{ij}$$

(11)

$$L_{ij}=-S_{ij},\hspace{1em}if\hspace{1em}i\ne j\hspace{1em}and\hspace{1em}{L}_{ii}=\sum _{j=1,j\ne i}^n{S}_{ij}$$

(12)

Minimizing the number of pixels with grayscale values that are out of bounds is objective function 2 (Obj 2). The formula is as follows:

$$A=\left\{Y^iϵN^{\mathrm{*}}|a_i{Y}^i+b_i>2^m-1,a_i{Y}^i+b_i<0\right\}$$

(13)

$$minimize\sum _{i=1}^{n}num\left(A\right)$$

(14)

where $Y^i$ denotes all pixel values of the image $i$; $N^{\mathrm{*}}$ denotes a positive integer, $a_i$ and $b_i$ are the linear stretch parameters of image $i$; $m$ is the number of quantization bits of the original image, the values 0 and $2^m-1$ correspond to the minimum and maximum pixel values derived from the quantization range of the image pixel values, $num$ denotes the number of elements in set A.

2.1.3. Constraints

Equality constraints were established to ensure consistency in the overall brightness and contrast before and after the linear stretching of images. The formula is as follows:

$$\sum _{i=1}^n{s}_i{\mu }_i=\sum _{i=1}^n{s}_i{(\mu }_i{a}_i+b_i)$$

(15)

$$\sum _{i=1}^n{s}_i{\sigma }_i=\sum _{i=1}^n{s}_i{(\sigma }_i{a}_i)$$

(16)

where $s_i$ denotes the number of pixels in the image $i$; ${\mu }_i$ is the gray mean of the image $i$; ${\sigma }_i$ is the standard deviation of the image $i$.

The equality constraints can be expressed as follows:

$$A_{eq}x=b_{eq}$$

(17)

$A_{eq}$ is a $2\times 2n$ matrix and $b_{eq}$ is a vector, which are equal to

$$A_{eq}=\left[\begin{array}{cccccc}{S}_1{\mu }_1& \cdots & S_n{\mu }_n& S_1& \cdots & S_n\\ S_1{\sigma }_1& \cdots & S_n{\sigma }_n& 0& \cdots & 0\end{array}\right]$$

(18)

$$b_{eq}=\left(\begin{array}{c}{\displaystyle \sum _{i=1}^n}{s}_i{\mu }_i\\ {\displaystyle \sum _{i=1}^n}{s}_i{\sigma }_i\end{array}\right)$$

(19)

Out-of-bounds pixels result in the loss of brightness information. Therefore, it is essential to constrain the pixel values within the quantification range of the image. To prevent out-of-bound pixels after stretching, the following inequality constraints are formulated:

$$a_i{y}_{min}^i+b_i\ge 1$$

(20)

$$a_i{y}_{max}^i+b_i\le 2^m-1$$

(21)

where $y_{min}^i$ and $y_{max}^i$ denote the minimum and maximum pixel values of the original image $i$, respectively.

The inequality constraints can be defined as follows:

$$\mathrm{A}x\le \mathrm{b}$$

(22)

where $\mathrm{A}$ is a $2n\times 2n$ matrix and $\mathrm{b}$ is a vector, which are equal to

$$\mathrm{A}=\left(\begin{array}{cc}\left[\begin{array}{ccc}{x}_{min}^1& 0& 0\\ 0& \ddots & 0\\ 0& 0& x_{min}^n\end{array}\right]& {\left[\begin{array}{ccc}-1& 0& 0\\ 0& \ddots & 0\\ 0& 0& -1\end{array}\right]}_{n\times n}\\ \left[\begin{array}{ccc}{x}_{max}^1& 0& 0\\ 0& \ddots & 0\\ 0& 0& x_{max}^n\end{array}\right]& {\left[\begin{array}{ccc}1& 0& 0\\ 0& \ddots & 0\\ 0& 0& 1\end{array}\right]}_{n\times n}\end{array}\right)$$

(23)

$$\mathrm{b}=\left(\begin{array}{c}{\left[\begin{array}{c}-1\\ ⋮\\ -1\end{array}\right]}_{n\times 1}\\ {\left[\begin{array}{c}{2}^m-1\\ ⋮\\ 2^m-1\end{array}\right]}_{n\times 1}\end{array}\right)$$

(24)

2.1.4. Decision Variables

To control the brightness information loss in the gray correction processing, we introduce the truncation values of the brightness anomaly images as the decision variables to relax the inequality constraints in Equation (21). It is as follows:

$$y_{max}^r\left(r\le t\le n\right)$$

(25)

where $y_{max}^r$ denotes the truncation value parameter for the $r$-th abnormal image, $t$ represents the total number of abnormal images.

2.2. Solving Algorithm

This study addresses a multi-objective optimization problem involving two types of objective functions. The first type is a continuous quadratic function, which, together with the constraints, constitutes the QP model. The second type is a discrete nonlinear statistical function. Based on these characteristics of the optimization model, we propose a solution algorithm. Based on the NSGA-II algorithm framework, a set of truncation values for the brightness anomaly images is generated. Once the truncation values are determined, the Obj 1 and linear constraints can be formulated as a QP model. The complexity of solving this model increases significantly with the number of decision variables. The barrier function interior-point method [28,29] offers global convergence and numerical stability, making it highly effective for handling large-scale and high-dimensional QP problems. The barrier function interior-point method solves for the linear stretch parameters and Obj 1 values. As the solution approaches the constraint boundaries, the barrier function value increases rapidly, maintaining the optimization process within a feasible region. The Obj 2 value is determined by statistics based on the linear stretch parameters. Pareto optimization [30] considers the combined effects of multiple objective functions. It can handle optimization problems with conflicting objectives by providing a set of solutions known as the Pareto front, which consists of numerous “non-dominated” solutions. The combination of metaheuristic and deterministic algorithms, through the iterative process of the NSGA-II algorithm, generates a set of Pareto optimal solutions, achieving the integrated solution of truncation values and stretching parameters.

Figure 5 shows the flowchart of the solution algorithm, which can be decomposed into nine steps.

Step 1: Data pre-processing: The following data should be obtained, including the total number of all images $n$, the total number of abnormal images $t$, the maximum pixel value $y_{max}^i$ of each image, the minimum pixel value $y_{min}^i$ of each image, the gray mean ${\mu }_i$ of each image, the standard deviation ${\sigma }_i$ of each image, the number of pixels $s_i$ of each image, the gray mean ${\mu }_{ij}$ and ${\mu }_{ji}$ of the overlapping area between images $i$ and $j$, the standard deviations ${\sigma }_{ij}$ and ${\sigma }_{ji}$ of the overlapping area between images $i$ and $j$, and the number of pixels $S_{ij}$ of the overlapping area between images $i$ and $j$. To determine the range of values of the decision variable $y_{max}^r$, it is necessary to collect the histogram for all abnormal images.

Step 2: The initial population is generated.

Step 3: The fitness of all individuals in the initial population is calculated.

(1) Calculation of Obj 1: The decision variables (truncation values of the brightness anomaly images) are substituted into the inequality constraints in Equation (21). Equations (7), (17), and (22) are combined to form the QP model. This model is solved using the barrier function interior point method to obtain the stretching parameters $a_i$ and $b_i$ for all images. The value of Obj 1 is calculated using Equation (2).

(2) Calculation of Obj 2: The linear stretch parameters $a_i$ and $b_i$ of all images are substituted into Equation (14) to calculate the total number of pixels that exceeded the quantization range of the images after stretching.

Step 4: Non-dominated sorting, selection, crossover, and mutation operations are performed on the population to form a new offspring population.

Step 5: The parent and offspring populations are merged.

Step 6: Fast non-dominated sorting and crowding distance calculations are applied to the merged population. Individuals are selected from the highest-ranking sorting layers to constitute a new parent population.

Step 7: A new offspring population is produced using selection, crossover, and mutation operations. The new parent population is combined with the latest offspring population to form an updated population.

Step 8: Repeat Steps 6 to 7. The iterations are stopped when the number exceeds the maximum limit.

Step 9: Output the final Pareto-front solutions and terminate the algorithm. Each solution represents a set of stretching parameters for all images and the truncation values of the brightness anomaly images.

3. Results

3.1. Experimental Datasets and Parameter Settings

Two experiments were performed using the HaiSi-1 single-polarization SAR images. The SAR intensity images used in this study are derived from Single Look Complex (SLC) data, which contains the complex information of the images. These images are calculated from the real and imaginary parts of the complex data. Figure 6 illustrates the coverage of the datasets (China, Fujian Province). The latitude range of Fujian Province is approximately 23.5°N to 28.3°N, and the longitude range is approximately 115.8°E to 120.7°E.

Table 1. Basic information on the two datasets.

	Total Number of Images	Number of Abnormal Images	Imaging Time Range	Range of Gray Mean
Dataset 1	122	39	20,210,427–20,220,608	3.525–5068.596
Dataset 2	737	235	20,210,424–20,220,701	1.521–5929.694

lists the specifics of the images.

All algorithms in this study were developed based on MATLAB R2022b. All experiments were conducted on a Windows 10 Professional 64-bit operating system, using an Intel(R) Core(TM) i7-8700K CPU with a clock speed of 3.70 GHz. The system was equipped with 32 GB of RAM, with 31.8 GB available.

Figure 7 shows the statistics of the two datasets. The large blue box in the figure is the amplification of the small blue box.

Table 2. Parameter settings for different datasets.

	Parameter	Dataset 1	Dataset 2
NSGA-II	Population quantity	100	300
	Maximum number of iterations	200	200
	Mutation probability	0.1	0.1
	Crossover probability	0.8	0.8
QP	Constraint Tolerance	$1\times {10}^{-8}$	$1\times {10}^{-8}$
	Optimality Tolerance	$1\times {10}^{-8}$	$1\times {10}^{-8}$
	Step Tolerance	$1\times {10}^{-12}$	$1\times {10}^{-12}$

lists the parameters of the two datasets used in this study. The population size is different due to the different number of decision variables.

As shown in

Table 3. Components of different methods.

Method	Model						Algorithm
	Obj 1	Obj 2	Equality Constraints	Inequality Constraints	Decision Variables	Solved Variables
QP_EC	$\surd$	--	$\surd$	--	The linear stretch parameters of all images.	The linear stretch parameters of all images.	QP algorithm
QP_BC	$\surd$	--	$\surd$	$\surd$	The linear stretch parameters of all images.	The linear stretch parameters of all images.	QP algorithm
Our	$\surd$	$\surd$	$\surd$	$\surd$	The truncation values of brightness anomaly images.	(1) The linear stretch parameters of all images. (2) The truncation values of brightness anomaly images.	The hybrid algorithm framework

, the first comparison method in this study was the QP_Equality_Constraint (QP_EC) method, which incorporated only equality constraints and focused on the gray consistency of images after stretching. The second method, QP_Both_Constraint (QP_BC), was an extension of the QP_EC method that added inequality constraints. The inequality constraints of the QP_BC method were used to control cases in which the pixel values exceeded the quantization range of the image pixel values after stretching.

This study used the peak signal-to-noise ratio (PSNR) of the overlapping parts to evaluate color consistency [6]. The number of pixels with grayscale values that were out of bounds and the visual quality of the mosaic image were also used as evaluation metrics.

3.2. Experimental Analysis of 122 Images

Six different output mosaics are produced. As shown in Figure 8c, the QP_BC method resulted in insufficient continuity between the images. In contrast, the mosaic results in Figure 8b,d–f display better gray consistency, visually reducing the brightness differences between the images. Figure 9 illustrates the distribution of the Pareto front solutions of the 122 images. The three solutions shown in Figure 8 are in the blue box of Figure 9.

Table 4. Performance results of various methods on Dataset 1.

	Original Image	QP_EC	QP_BC	Our Results
Gray consistency	--	$1.27\times {10}^{14}$	$1.67\times {10}^{16}$	$1.98\times {10}^{14}$	$2.29\times {10}^{14}$	$3.48\times {10}^{14}$
Number of out-of-bounds pixels	--	11,695,234	29	52,401	47,490	41,229
PSNR (dB)	25.691	30.200	27.831	30.307	30.286	30.229

presents the quantitative analysis outcomes of the different methods for the 122 images. The QP_EC method achieved superior performance in gray consistency but also resulted in the most loss of brightness information. In contrast, the QP_BC method resulted in the least brightness information loss. However, it performed the worst in terms of gray consistency and PSNR.

The three solutions of the proposed method reduced the count of out-of-bounds grayscale pixels to 1/223, 1/246, and 1/284, respectively, compared with the QP_EC method. The PSNR in the three solutions of the proposed method was improved by 0.107 dB, 0.086 dB, and 0.029 dB compared with that of the QP_EC method and 2.476 dB, 2.455 dB, and 2.398 dB compared with that of the QP_BC method. The gray consistency was also improved for the three solutions obtained using the proposed method.

3.3. Experimental Analysis of 737 Images

Six different output mosaics are produced. As shown in Figure 10c, the gray consistency obtained using the QP_BC method was discontinuous. Figure 10b,d–f demonstrates better gray consistency in the mosaic results, visually reducing the gray discrepancies between the images. Figure 11 illustrates the distribution of the Pareto-front solutions for the 737 images. The three solutions shown in Figure 10 are in the blue box of Figure 11.

Table 5. Performance results of various methods on Dataset 2.

	Original Image	QP_EC	QP_BC	Our Results
Gray consistency	--	$5.79\times {10}^{14}$	$3.92\times {10}^{16}$	$8.92\times {10}^{14}$	$8.66\times {10}^{14}$	$7.08\times {10}^{14}$
Number of out-of-bounds pixels	--	818,936,260	289	219,281	226,724	256,705
PSNR	26.051	30.152	28.647	30.213	30.237	30.155

presents the quantitative analysis outcomes of the different methods for the 737 images. The QP_EC method achieved the best performance in gray consistency but also exhibited the highest amount of brightness information loss. The QP_BC method resulted in the least brightness information loss, but it performed the worst in terms of gray consistency and PSNR.

The number of pixels with grayscale values that were out of bounds in the three solutions of the proposed method was 3735, 3612, and 3190 times less than that in the QP_EC method. The PSNR in the three solutions of the proposed method improved by 0.061 dB, 0.085 dB, and 0.003 dB compared with that of the QP_EC method. The PSNR in the three solutions of the proposed method improved by 1.566 dB, 1.59 dB, and 1.508 dB compared to that of the QP_BC method. In addition, the gray consistency of the three solutions obtained using the proposed method was greatly improved.

4. Discussion

Table 6. Constraint violation degree of different methods on Dataset 1.

		QP_EC	QP_BC	Our Data
Equality constraints	$\mathrm{E}\mathrm{q}{1}_{\mathrm{D}\mathrm{C}\mathrm{V}}$	0	$1.74\times {10}^5$	$2.83\times {10}^4$	$5.34\times {10}^4$	$1.63\times {10}^5$
	$\mathrm{E}\mathrm{q}{2}_{\mathrm{D}\mathrm{C}\mathrm{V}}$	0.0039	0.002	0	0.0039	0.002
Inequality constraints	$\mathrm{I}\mathrm{n}\mathrm{e}\mathrm{q}{1}_{\mathrm{D}\mathrm{C}\mathrm{V}}$	$1.3\times {10}^4$	0	0	0	0
	$\mathrm{I}\mathrm{n}\mathrm{e}\mathrm{q}{2}_{\mathrm{D}\mathrm{C}\mathrm{V}}$	$2.9\times {10}^7$	0.00512	$2.8\times {10}^7$	$2.7\times {10}^7$	$2.6\times {10}^7$
Number of out-of-bounds pixels	--	11,695,234	29	52,401	47,490	41,229

DCV: Degree of constraint violation.

and

Table 7. Constraint violation degree of different methods on Dataset 2.

		QP_EC	QP_BC	Our Data
Equality constraints	$\mathrm{E}\mathrm{q}{1}_{\mathrm{D}\mathrm{C}\mathrm{V}}$	0.0313	$5.80\times {10}^5$	$1.21\times {10}^6$	$1.07\times {10}^6$	$3.92\times {10}^5$
	$\mathrm{E}\mathrm{q}{2}_{\mathrm{D}\mathrm{C}\mathrm{V}}$	0.0156	0.0781	0.0469	0.0469	0.0781
Inequality constraints	$\mathrm{I}\mathrm{n}\mathrm{e}\mathrm{q}{1}_{\mathrm{D}\mathrm{C}\mathrm{V}}$	$2.7\times {10}^{15}$	0	0	0	0
	$\mathrm{I}\mathrm{n}\mathrm{e}\mathrm{q}{2}_{\mathrm{D}\mathrm{C}\mathrm{V}}$	$1.1\times {10}^8$	0.07	$1.1\times {10}^8$	$1.1\times {10}^8$	$1.2\times {10}^8$
Number of out-of-bounds pixels	--	818,936,260	289	219,281	226,724	256,705

DCV: Degree of constraint violation.

provide a quantitative comparison of the two datasets, which describe the degree of constraint violation for different methods and the number of pixels with grayscale values that were out of bounds. The degree of violation for the two equality constraints is denoted as Eq1_DCV and Eq2_DCV, while the degree of violation for the inequality constraints is represented by Ineq1_DCV and Ineq2_DCV, as shown in Equations (26)–(29). Based on the data presented in both tables, it can be concluded that the QP_EC method exhibits the highest degree of inequality constraint violation and the largest number of pixels with grayscale values that were out of bounds, as it lacks inequality constraints and fails to address the issue of pixel values exceeding the quantization range after stretching. According to the data in Table 4 and Table 5, the gray consistency index of the QP_EC method is in the same order as the method proposed in this paper.

The QP_BC method introduces inequality constraints, significantly reducing the number of pixels with grayscale values that were out of bounds. The degree of equality constraint violation is higher than that of the QP_EC method. Figure 8c and Figure 10c show that the continuity between images after stretching is insufficient. According to the data in Table 4 and Table 5, the gray consistency index of the QP_BC method is significantly higher than that of the QP_EC method.

The total constraint violation degree of the proposed method is smaller than that of the QP_EC method and larger than that of the QP_BC method. The number of pixels with grayscale values that were out of bounds is smaller than that of the QP_EC method and larger than that of the QP_BC method. The data in Table 4 and Table 5 show that the gray consistency index is worse than the QP_EC method but better than the QP_BC method. In this section, we present the analysis of our data based on comparison with the QP_EC and QP_BC methods. Due to the addition of inequality constraints, the degree of equality constraint violation is significantly increased, and the degree of inequality constraint violation is reduced considerably (QP_EC). The degree of equality constraint violation is similar. Owing to the introduction of truncation values, the inequality constraints are relaxed, and the degree of inequality constraint violation is significantly increased (QP_BC). In the first dataset, the proposed algorithm has a low degree of equality constraint violation and a high degree of inequality constraint violation. In the second dataset, the proposed algorithm has a slightly larger degree of equality constraint violation and a higher degree of inequality constraint violation. The QP_BC method adds equality constraints and inequality constraints. Although it can control the number of pixels with grayscale values that were out of bounds, it will significantly decrease gray consistency between images after stretching. Based on the QP_BC method, the proposed method introduces the truncation values to achieve the appropriate relaxation of the solution space and realize the balanced optimization of the two objective functions. The proposed method’s brightness information loss and gray consistency are far superior to those of the QP_EC and QP_BC methods.

$$\mathrm{E}\mathrm{q}{1}_{\mathrm{D}\mathrm{C}\mathrm{V}}=\sum _{i=1}^n\left|s_i{[\mu }_i-{(\mu }_i{a}_i+b_i\right)\left]\right|$$

(26)

$$\mathrm{E}\mathrm{q}{2}_{\mathrm{D}\mathrm{C}\mathrm{V}}=\sum _{i=1}^n\left|s_i{\sigma }_i\left(1-a_i\right)\right|$$

(27)

$$\mathrm{I}\mathrm{n}\mathrm{e}\mathrm{q}{1}_{\mathrm{D}\mathrm{C}\mathrm{V}}=\sum _{i=1}^n\left|a_i{y}_{min}^i+b_i-1\right|$$

(28)

$$\mathrm{I}\mathrm{n}\mathrm{e}\mathrm{q}{2}_{\mathrm{D}\mathrm{C}\mathrm{V}}=\sum _{i=1}^n\left|a_i{y}_{max}^i+b_i-2^m+1\right|$$

(29)

5. Conclusions

When regional SAR intensity images exhibit brightness anomalies, balancing gray consistency and brightness information loss becomes a significant challenge in gray consistency processing. Traditional color harmonization methods have struggled to avoid losing brightness information while improving the color consistency of the overlapping regions. We proposed a multi-objective optimization model to deal with this issue, utilizing the truncation values of brightness anomaly images as decision variables. The optimization objectives were to maximize gray consistency in overlapping image pairs and to minimize brightness information loss. The overall brightness and contrast of the images were maintained before and after correction (equality constraints), while the pixel values of the corrected images conformed to the specified range for image pixel value quantization (inequality constraints). Both objective functions were optimized by optimizing the relaxation of the inequality constraints. In terms of the solution algorithm, when faced with multiple constraints, a continuous objective function, and a discrete objective function, this study proposed an algorithm that combined the global search capability of the NSGA-II metaheuristic with the precise solution capability of the QP algorithm. Within the NSGA-II framework, the barrier function interior-point method was used to solve Obj 1 and its corresponding linear stretch parameters. Based on the linear stretch parameters, Obj 2 was computed. The proposed method provided an integrated solution for the truncation values of brightness anomaly images and linear stretch parameters.

Two datasets were used to evaluate the performance and applicability of the proposed method, and comparisons were made with other methods. The experimental results demonstrated that the proposed method effectively maintains gray consistency while significantly reducing brightness information loss. Given the assumption that the overall brightness and contrast of the entire image dataset should be preserved, adaptive handling of equality constraints based on the proportion of abnormal images will be a key focus of future research. In addition, we plan to utilize learning-based methods to achieve the mosaicking of large-scale SAR images with anomaly images.