Optimized Contrast Enhancement for Infrared Images Based on Global and Local Histogram Speciﬁcation

.


Introduction
Recent years, infrared imaging systems have been extensively applied in military and civilian areas such as night version, video surveillance, driver assistance, fire detection, and disease diagnosis [1][2][3].In these areas, infrared imaging has broad application prospects due to its ability to reflect the thermal radiation distribution of the scene, which is impossible for visual imaging.However, compared with visual images, there are many inherent drawbacks in infrared images.Among these drawbacks, the low contrast and low resolution are the key factors that reduce the visual quality of infrared images [4].Therefore, contrast enhancement is essential for improving the visual quality of infrared images.
Contrast enhancement methods have been studied for decades, and they can be categorized into two families depending on the area adopted for defining the enhancement method such as global contrast enhancement (GCE) and local contrast enhancement (LCE) [5].The GCE methods aim to enhance the overall contrast by finding a mapping function and considering the characteristics of the entire image.On the contrary, LCE methods try to enhance the local contrast of each area by using different mapping functions, which are correlated to the characteristics of the local areas.
As for the GCE methods, histogram equalization (HE) [6] is one of the most well-known methods.Due to the low computational complexity and effective contrast enhancement capability, HE has been widely applied to improve the visual quality of images.However, the enhanced results of HE may cause serious over-enhancement artifacts when there are large, homogeneous regions in infrared images [7].To deal with this problem, some improved HE methods have been investigated.Among these methods, the plateau histogram equalization (PHE) [8,9], double plateaus histogram equalization (DPHE) [10], and adaptive double plateaus histogram equalization (ADPHE) [11] try to introduce one or two proper thresholds to avoid over-enhancement.Besides these plateau-based methods, some other methods such as the brightness preserving bi-histogram equalization (BBHE) [12], dualistic sub-image histogram equalization (DSIHE) [13], recursive mean-separate histogram equalization (RMSHE) [14], adaptive histogram segmentation (AHS) [15], and adaptive histogram partition (AHP) [16] address the over-enhancement by adaptively dividing the histogram into two or more partitions.Other methods such as the histogram modification framework (HMF) [17] and histogram specification (HS) [18] also gives better performance for avoiding over-enhancement.In summary, the GCE methods aim to make a trade-off between avoiding over-enhancement and enhancing the contrast as much as possible [15].Although the GCE methods naturally improve the contrast with low computational complexity and good light order preservation, the single mapping function ignores the local contrast characteristics of the image, which limits the performance of GCE methods.
The LCE methods try to enhance the local contrast by taking the characteristics of local areas into consideration.One effective approach is to divide the input image into multiple sub-blocks and then enhance the contrast of each sub-block respectively.For instance, this includes the contrast limited adaptively histogram equalization (CLAHE) [19] and partially overlapped sub-block histogram equalization (POSHE) [20].The CLAHE method divides the input image into multiple non-overlapped sub-blocks and the histogram of each sub-block is clipped and redistributed to limit the contrast.The bilinear interpolation is also introduced to prevent the blocking artifacts.The POSHE method prevents the blocking artifacts by dividing the input image into a partially overlapped sub-block and the enhanced results of the pixels in overlapped areas are computed based on the correlated sub-blocks.By improving the CLAHE method with a new redistribution mechanism and local contrast enhancement, the balanced CLAHE and contrast enhancement (BCCE) [21] performs better than CLAHE in terms of visual quality improvement.The adaptive trilateral contrast enhancement (ATCE) [22] utilizes the double plateau histogram equalization for the local contrast enhancement and then manipulates the sharpness and intensity based on the extracted feature of the input image.Nevertheless, these methods may still suffer from blocking artifacts and over-enhancement in homogeneous regions.In order to take the content of each sub-block into consideration, the mechanisms for calculating the mapping functions of the sub-blocks can vary along with the local content.Thus, the adjacent-blocks-based modification for local histogram equalization (ABMHE) [23] and local gradient-grayscale statistical feature (LGGSF) [24] categorize the sub-blocks into two or three categories.For each category, the particular mechanism is applied to calculate the mapping functions of the sub-blocks.Besides these block-based methods, other LCE methods called unsharp masking (UM) based methods [25][26][27][28][29] decompose the input image into a base layer and a detail layer by a filter.These two layers are enhanced separately and then composited to get the final enhanced image.Some other methods introduce the retinex theory [30,31] and the wavelet transform [32,33] to realize the contrast enhancement of infrared images.The LCE methods have better performance for improving the local contrast, but they may cause unnatural and over-enhanced results in some local areas.
As described above, the GCE and LCE methods have complementary characteristics.It is necessary to study a new contrast enhancement method that combines the metrics of both methods [5,34].These methods are designed for the visual color images while there are no color components and the contrast is poor in the infrared images.Therefore, they are not suitable for the infrared images.In this paper, based on the 2D histogram, an optimized contrast enhancement method combining the merits of global and local methods is proposed to improve the visual quality of infrared images.First, the 2D histogram, which contains both global and local gray level distribution characteristics of the input image, is computed according to the definition in Reference [35].Then, based on the histogram specification, the global and local contrast enhancement are conducted.The global mapping function is obtained by applying a histogram specification to the clipped and redistributed 2D histogram.The local mapping functions are also computed by applying histogram specification to each row of the 2D histogram and then updated based on the global mapping function.The GCE and LCE results are computed based on the global mapping function and local mapping functions.Lastly, an objective function is conducted with global and local constrained conditions and the optimized enhancement result is obtained through the optimization approach.Thus, the final enhanced image is possible to make a compromise between the merits of global and local enhancements.The main contributions of our paper are: (1) The 2D histogram is utilized to realize both global and local contrast enhancement.(2) The global enhancement result is obtained by applying the histogram specification to the clipped 2D histogram.(3) The local mapping functions are computed by applying the histogram specification to each row of the 2D histogram.The local enhancement result is obtained based on the local content information and the corresponding mapping functions.(4) The desired image is obtained by finding a solution for the quadratic optimization function, which contains both global and local constrained conditions.For the local content adaptation, the edge information of the input image is also taken into consideration.
The remainder of this paper is organized as follows.In Section 2, we first make a brief review of the 2D histogram.In Section 3, we describe the proposed method in detail.Section 4 gives the experimental results and Section 5 concludes the paper.

Review of the 2D Histogram
The concept of the 2D histogram was first proposed in Reference [35] to improve the contrast of visual images.In that paper, the 2D histogram h(i, j) was defined to indicate the joint distribution of two spatially adjacent pixels with grey levels i and j, respectively.For a given input image X of size M × N pixels and dynamic range [x d , x u ], i.e., X = x(m, n The main objective of the contrast enhancement methods is to generate an enhanced image Y of the same size as X, i.e., Y = y(m, n)|1 ≤ m ≤ M, 1 ≤ n ≤ N, y(m, n) ∈ N , which has better visual quality than X and y(m, n) ∈ [y d , y u ].The dynamic range of Y can be stretched or compressed [35].For an 8-bit image, the dynamic range of the enhanced image is [0, 255] to take full use of the entire dynamic range.For the input image, assume that the dynamic range is [0, L − 1], and L = 256 for the 8-bit image, which is the number of distinct gray-levels of the image.The 2D histogram can be expressed as the equation below.
where h(i, j) is the number of the occurrences of gray level j in the neighborhood of gray level i and h(i, j) is computed as the equation below.
where r is an integer introduced to determine the square (2r + 1) × (2r + 1) neighborhood around each pixel.The binary function φ i,j (u, v) is used to identify the occurrence of the gray levels i and j at the spatial square neighborhood and is defined as the equation below.
In order to apply the 2D histogram to the calculation of global mapping function, the 2D histogram is redefined in Reference [36] as follows.
In this scenario, the initial 2D histogram is weighted by the absolute-valued differences i − j + 1 .Therefore, higher weight is assigned to the 2D histogram when the difference between i and j is large.The 2D histogram based global contrast enhancement method gets better performance for improving the visual quality of visual images.However, due to the low contrast of infrared images, the global contrast enhancement is not enough for the visual quality improvement of infrared images.

Proposed Method
In this section, the proposed method is described in detail, which includes histogram specification-based global and local contrast enhancement and the optimized enhancement result combining both global and local enhanced results.The 2D histogram is utilized in our proposed method to achieve both global and local contrast enhancement.Figure 1 shows the overall block diagram of the proposed method.In this scenario, the initial 2D histogram is weighted by the absolute-valued differences ( ) − +1 i j .
Therefore, higher weight is assigned to the 2D histogram when the difference between i and j is large.The 2D histogram based global contrast enhancement method gets better performance for improving the visual quality of visual images.However, due to the low contrast of infrared images, the global contrast enhancement is not enough for the visual quality improvement of infrared images.

Proposed Method
In this section, the proposed method is described in detail, which includes histogram specification-based global and local contrast enhancement and the optimized enhancement result combining both global and local enhanced results.The 2D histogram is utilized in our proposed method to achieve both global and local contrast enhancement.Figure 1 shows the overall block diagram of the proposed method.Usually, the data width of the raw images captured by an infrared imaging system is 14-bit and the histogram of the raw image is sparse.This means that most of the gray levels take on no pixel.

Local histogram
Therefore, we pre-process the raw images by removing the gray levels that take no pixel and linearly mapping the remaining ordered gray levels to   −    Usually, the data width of the raw images captured by an infrared imaging system is 14-bit and the histogram of the raw image is sparse.This means that most of the gray levels take on no pixel.Therefore, we pre-process the raw images by removing the gray levels that take no pixel and linearly mapping the remaining ordered gray levels to 0, 2 8 − 1 .In order to obtain both global and local enhancement results based on the 2D histogram, the 2D histogram should contain both global and local gray level distribution characteristics of the input image.Thus, in our proposed method, the 2D histogram is calculated based on the definition in Equation (2).
Take an infrared image for reference.Figure 2 shows the reference infrared image and its normalized 2D histogram using a 5 × 5 neighborhood.There are more dark regions than bright regions.Therefore, the 2D histogram has larger values located at lower gray values.There are also some homogeneous regions in the image.The neighbors of each pixel in these homogeneous regions take on very similar gray levels, which result in higher peaks at the diagonal or near-diagonal element of the 2D histogram.As can be observed in Figure 2b, the 2D histogram is diagonal symmetry.This is due to the mutual relation between each pixel and its neighbors.Take an infrared image for reference.Figure 2 shows the reference infrared image and its normalized 2D histogram using a × 5 5 neighborhood.There are more dark regions than bright regions.Therefore, the 2D histogram has larger values located at lower gray values.There are also some homogeneous regions in the image.The neighbors of each pixel in these homogeneous regions take on very similar gray levels, which result in higher peaks at the diagonal or near-diagonal element of the 2D histogram.As can be observed in Figure 2(b), the 2D histogram is diagonal symmetry.This is due to the mutual relation between each pixel and its neighbors.

Global Enhancement
Global enhancement methods aim to obtain an optimized mapping function to map the input image to an enhanced one for better visual quality.In order to obtain the mapping function, the histogram specification is utilized in this case.First of all, to improve the global contrast and avoid over-enhancement, the 2D histogram is clipped by an adaptively adjusted clip point [37] as shown below.
where the clip is computed by the formula below.

MN r clip H L
In this case, max H is the maximum value of the 2D histogram while β is the clipping factor and is set to be 0.2 for the experiments in this paper.The exceeded pixels are then equally redistributed and the updated 2D histogram Based on the modified 2D histogram, the corresponding cumulative distribution function is written as the equation below.

Global Enhancement
Global enhancement methods aim to obtain an optimized mapping function to map the input image to an enhanced one for better visual quality.In order to obtain the mapping function, the histogram specification is utilized in this case.First of all, to improve the global contrast and avoid over-enhancement, the 2D histogram is clipped by an adaptively adjusted clip point [37] as shown below.
where the clip is computed by the formula below.
In this case, H max is the maximum value of the 2D histogram while β is the clipping factor and is set to be 0.2 for the experiments in this paper.The exceeded pixels are then equally redistributed and the updated 2D histogram Based on the modified 2D histogram, the corresponding cumulative distribution function is written as the equation below.
Then, as for the desired 2D histogram, it is supposed to satisfy the following uniformly distributed probability density function [36].
Similarly, the corresponding cumulative distribution function of the desired 2D histogram is computed as the equation below.
Combining Equations ( 7) and ( 9), according to the definition of the histogram specification [38], the mapping function aims to map the gray level i of the 2D histogram to the gray level i of the desired 2D histogram and satisfy the equation c G (i) = ĉG (i ).Therefore, the mapping function is computed according to the equation below.
Using Equation (10), the gray levels of the input image X are transformed to the corresponding gray levels of the enhanced output image.Thus, the global enhancement result Y G is computed by the formula below.
The resultant global mapping function and the enhanced image are shown in Figure 3a,b.As observed in Figure 3b, the global enhanced result gets a natural appearance and improves the contrast without any artifacts.However, there is no improvement in terms of local contrast.Therefore, the enhancement performance is limited.Then, as for the desired 2D histogram, it is supposed to satisfy the following uniformly distributed probability density function [36].
Similarly, the corresponding cumulative distribution function of the desired 2D histogram is computed as the equation below.
Combining Equations ( 6) and ( 8), according to the definition of the histogram specification [38], the mapping function aims to map the gray level i of the 2D histogram to the gray level ' i of the desired 2D histogram and satisfy the equation c i c i .Therefore, the mapping function is computed according to the equation below.
Using Equation ( 9), the gray levels of the input image X are transformed to the corresponding gray levels of the enhanced output image.Thus, the global enhancement result G Y is computed by the formula below.

Gray level
Gray level (b) (a) The resultant global mapping function and the enhanced image are shown in Figure 3(a) and Figure 3(b).As observed in Figure 3(b), the global enhanced result gets a natural appearance and improves the contrast without any artifacts.However, there is no improvement in terms of local contrast.Therefore, the enhancement performance is limited.

Local Enhancement
Local enhancement methods aim to enhance the contrast of each local area by using different mapping functions, which are computed based on the characteristics of the local areas.In this section, the local mapping function i T is defined to be the mapping function of the neighbors of pixels whose

Local Enhancement
Local enhancement methods aim to enhance the contrast of each local area by using different mapping functions, which are computed based on the characteristics of the local areas.In this section, the local mapping function T i is defined to be the mapping function of the neighbors of pixels whose gray level is i.The histogram specification algorithm is utilized again to compute the local mapping function T i .As for the local histogram, let H i = h i ( j) = h(i, j)/ L−1 j=0 h(i, j) j = 0, 1, • • • , L − 1 be the normalized one row of the 2D histogram H with index i, which indicates the gray level distribution of the neighbors of each pixel whose gray level is i in the input image.Then, the corresponding cumulative distribution function is computed by the formula below.
As for the desired local histogram, similar to the desired 2D histogram, we assume that the desired local histogram satisfies the following Gaussian distribution probability density function [18].
where ĥi In addition, σ is the weighted average value of the standard deviation of each row in the 2D histogram.The desired local histogram is also normalized by ĥi ( j ) = ĥi ( j )/ L−1 j =0 ĥi ( j ).Then, we can compute the corresponding cumulative distribution function by using the formula below.
Once again, based on the histogram specification, the original local mapping function T i is computed according to the equation below.
An indisputable fact is that each local area is just a small part of the whole image.Therefore, not only the local areas but also the whole image have to be taken into consideration when enhancing the local contrast.This inspires us to update the original local mapping based on the global mapping function.When computing the optimal local mapping function, both local and global mapping functions have to be taken into account.To satisfy this condition, the preferred option is to find an optimal mapping function T i , which is closer to T G and ensure the small residual T i − T i .The optimal local mapping function T i can be treated as the solution to a bi-criteria optimization problem.If the square of the Euclidean norm is used for the problem, the optimal local mapping function T i is computed by using the equation below.
where T is just an intermediate variable while α is a regularization parameter and varies over [0, ∞).The solution to this quadratic optimization problem is shown below.
Therefore, the optimal mapping function T i turns out to be a weighted average of T G and T i .Simply by varying α, T i takes on the optimal trade-off value between the two objectives.When α = 0, T i is equal to the global mapping function T G .Therefore, the local enhancement result tends to be the same as the global enhancement result.However, as α goes to infinity, T i converges to the original local mapping function T i .Thus, the local enhancement result usually results in over enhancement of the local contrast.
Since the local mapping function T i aims to enhance the contrast of the neighbors of the pixels whose gray level is i, each pixel in the input image is conversely one element of the neighbors of the pixels that belong to its neighborhood.Therefore, the local enhancement result of each pixel should be the average of the enhanced results using the mapping function of its neighbors.The local enhancement result where W r = w r (k, l) and w r (k, l) is defined as the element of the Gaussian weight matrix.The expression of w r (k, l) is shown below.
As shown in Figure 4, when α = 0, the local enhancement result is the same as the global enhancement result in Figure 3a.Thus, the local contrast is not enhanced.This is because the local mapping functions turn to coincide with the global mapping function.As the value of α gradually increases, the local mapping functions become different from the global mapping function, which results in a more enhanced local contrast.We empirically set α = 1 in the later experiments.Compared with the global enhancement result, local enhancement results look unnatural and the noise in homogeneous regions also emerges.Since the local mapping function i T aims to enhance the contrast of the neighbors of the pixels whose gray level is i , each pixel in the input image is conversely one element of the neighbors of the pixels that belong to its neighborhood.Therefore, the local enhancement result of each pixel should be the average of the enhanced results using the mapping function of its neighbors.The local enhancement result n N of the input image is computed by the equation below.
where Compared with the global enhancement result, local enhancement results look unnatural and the noise in homogeneous regions also emerges.

Optimized Enhancement
As described above, the global enhancement result improves the contrast of the input image without considering the characteristics of the input area.Thus, the contrast enhancement performance is limited.On the contrary, the local enhancement result shows much better performance on the local contrast improvement, but it looks unnatural and makes the noise emerge

Optimized Enhancement
As described above, the global enhancement result improves the contrast of the input image without considering the characteristics of the input area.Thus, the contrast enhancement performance is limited.On the contrary, the local enhancement result shows much better performance on the local contrast improvement, but it looks unnatural and makes the noise emerge in homogeneous regions.Since the performance of global enhancement and local enhancement have complementary characteristics, the optimized enhancement is expected to combine the merits of both global and local enhancements.To achieve this goal, the optimized enhancement result Y has to make a trade-off between two objectives: (1) the overall brightness variation should follow that of the global enhancement result and (2) the local contrast should follow that of the local enhancement result [5].These two objectives are expressed as: 1) : where Y is the desired image, Y is the set of average values of sub-blocks in Y , Y L is the set of average values of sub-blocks in Y L , c is a constant and is defined as the contrast parameter, E G and E L are the lexicographically ordered sets of difference values in the condition of global brightness and local contrast similarity, and the operator ./indicates that the corresponding elements of the two sets are divided separately.The goal is to find a solution that can simultaneously minimize E G and E L .Moreover, notice that the variation ranges of E G and E L are not of the same magnitude.The expression of the second objective is rewritten by the equation below. where In this scenario, the operator .* indicates that the corresponding elements of the two sets are multiplied separately.Therefore, the goal should be changed to find a solution that can simultaneously minimize E G and E L .This is a bi-criteria optimization problem, and, in this paper, the square of the Frobenius norm is used to compute the solution below.
where λ is a regularization parameter and varies over [0, ∞).The optimized enhancement result obtained by λ = 0 is equal to the global enhancement result.Nevertheless, as λ increases to infinity, it turns to resemble the local enhancement result.Therefore, a different optimized enhancement result is obtained according to λ and c.Considering the region characteristics of the input image, larger λ and c are needed for the detailed regions to get local contrast enhanced, but smaller λ and c are more suitable for the homogeneous regions.Thus, λ and c should be a region with adaptive variables and should be represented as the component of 287 294

295
where  is a regularization parameter and varies over )   0, .The optimized enhancement result

296
obtained by  = 0 is equal to the global enhancement result.Nevertheless, as  increases to infinity, 297 it turns to resemble the local enhancement result.Therefore, a different optimized enhancement

298
result is obtained according to  and c .Considering the region characteristics of the input image, 299 larger  and c are needed for the detailed regions to get local contrast enhanced, but smaller  300 and c are more suitable for the homogeneous regions.Thus,  and c should be a region with 301 adaptive variables and should be represented as the component of is unknown and the desired image  Y satisfies the objective functions in Equation (18).We assume 304 that the average value of the desired image is approximately equal to that of the global enhancement 20) is rewritten as the formula below. 306

307
The optimal solution of Equation ( 21) is shown below. 308

309
where 1 is the  MN one matrix.As shown in Equation ( 22), the first term indicates that the of a constant.Moreover, notice that the value of Y is unknown and the desired image Y satisfies the objective functions in Equation (19).We assume that the average value of the desired image is approximately equal to that of the global enhancement image (Y ≈ Y G ). Then Equation ( 21) is rewritten as the formula below.
eneous regions.Since the performance of global enhancement and local enhancement have entary characteristics, the optimized enhancement is expected to combine the merits of both d local enhancements.To achieve this goal, the optimized enhancement result Y has to ade-off between two objectives: 1) the overall brightness variation should follow that of the hancement result and 2) the local contrast should follow that of the local enhancement result two objectives are expressed as: indicates that the corresponding elements of the two sets are d separately.Therefore, the goal should be changed to find a solution that can eously minimize . This is a bi-criteria optimization problem, and, in this paper, e of the Frobenius norm is used to compute the solution below.
is a regularization parameter and varies over instead of a constant.Moreover, notice that the value of  Y wn and the desired image  Y satisfies the objective functions in Equation (18).We assume verage value of the desired image is approximately equal to that of the global enhancement . Then Equation ( 20) is rewritten as the formula below.
al solution of Equation ( 21) is shown below.

( ). ( . / ) . . / ( )
is the  MN one matrix.As shown in Equation ( 22), the first term indicates that the d enhancement result resembles the global enhancement result, and the second term adds hted local contrast of the local enhancement result to the average value of the global The optimal solution of Equation ( 22) is shown below.
egions.Since the performance of global enhancement and local enhancement have aracteristics, the optimized enhancement is expected to combine the merits of both nhancements.To achieve this goal, the optimized enhancement result Y has to etween two objectives: 1) the overall brightness variation should follow that of the nt result and 2) the local contrast should follow that of the local enhancement result ectives are expressed as: the operator  .
indicates that the corresponding elements of the two sets are ately.Therefore, the goal should be changed to find a solution that can inimize . This is a bi-criteria optimization problem, and, in this paper, robenius norm is used to compute the solution below.
ularization parameter and varies over n N instead of a constant.Moreover, notice that the value of  Y he desired image  Y satisfies the objective functions in Equation (18).We assume alue of the desired image is approximately equal to that of the global enhancement Then Equation ( 20) is rewritten as the formula below.
ion of Equation ( 21) is shown below.

( ). ( . / ) . . / ( )
 MN one matrix.As shown in Equation ( 22), the first term indicates that the ement result resembles the global enhancement result, and the second term adds al contrast of the local enhancement result to the average value of the global  287 294

295
where  is a regularization parameter and varies over is unknown and the desired image  Y satisfies the objective functions in Equation (18).We assume 304 that the average value of the desired image is approximately equal to that of the global enhancement . Then Equation ( 20) is rewritten as the formula below. 306

307
The optimal solution of Equation ( 21) is shown below. 308

309
where 1 is the  MN one matrix.As shown in Equation ( 22), the first term indicates that the 287 294

295
where  is a regularization parameter and varies over is unknown and the desired image  Y satisfies the objective functions in Equation (18).We assume 304 that the average value of the desired image is approximately equal to that of the global enhancement . Then Equation ( 20) is rewritten as the formula below. 306

307
The optimal solution of Equation ( 21) is shown below. 308

309
where 1 is the  MN one matrix.As shown in Equation ( 22), the first term indicates that the at the corresponding elements of the two sets are should be changed to find a solution that can a bi-criteria optimization problem, and, in this paper, pute the solution below.
of a constant.Moreover, notice that the value of  Y s the objective functions in Equation (18).We assume proximately equal to that of the global enhancement en as the formula below.
). ( ./ ) . ./ ( ) n in Equation ( 22), the first term indicates that the and C. In this paper, C and , x FOR PEER REVIEW 9 of 21 regions.Since the performance of global enhancement and local enhancement have haracteristics, the optimized enhancement is expected to combine the merits of both enhancements.To achieve this goal, the optimized enhancement result Y has to between two objectives: 1) the overall brightness variation should follow that of the ent result and 2) the local contrast should follow that of the local enhancement result jectives are expressed as: the operator  .
indicates that the corresponding elements of the two sets are rately.Therefore, the goal should be changed to find a solution that can inimize . This is a bi-criteria optimization problem, and, in this paper, Frobenius norm is used to compute the solution below.
n N instead of a constant.Moreover, notice that the value of  Y the desired image  Y satisfies the objective functions in Equation (18).We assume alue of the desired image is approximately equal to that of the global enhancement Then Equation ( 20) is rewritten as the formula below.
tion of Equation ( 21) is shown below.
 MN one matrix.As shown in Equation ( 22), the first term indicates that the are adaptively determined based on the edge information of the input image.The edge information ∆ is first computed by using the Sobel operator and then is linearly mapped to the same dynamic range of the output image.The value of each element in 294

295
where  is a regularization parameter and varies over instead of a constant.Moreover, notice that the value of 303 is unknown and the desired image  Y satisfies the objective functions in Equation (18).We assu 304 that the average value of the desired image is approximately equal to that of the global enhancem . Then Equation ( 20) is rewritten as the formula below.

306
307 The optimal solution of Equation ( 21) is shown below.

308
and C are defined to be positively correlated with the corresponding value of edge information and are computed by the equation below.
s a regularization parameter and varies over )   0, .The optimized enhancement result  = 0 is equal to the global enhancement result.Nevertheless, as  increases to infinity, resemble the local enhancement result.Therefore, a different optimized enhancement tained according to  and c .Considering the region characteristics of the input image, nd c are needed for the detailed regions to get local contrast enhanced, but smaller  more suitable for the homogeneous regions.Thus,  and c should be a region with riables and should be represented as the component of instead of a constant.Moreover, notice that the value of  Y and the desired image  Y satisfies the objective functions in Equation( 18).We assume rage value of the desired image is approximately equal to that of the global enhancement  ) G Y . Then Equation ( 20) is rewritten as the formula below.
where λ 0 and c 0 are user-defined parameters and are empirically set as λ 0 = 0.1 and c 0 = 2 in the later experiments.
Figure 5 shows the map of edge information ∆, the optimized enhancement result Y, and the map of edge information of Y.All these edge information maps are computed by applying the Sobel operator to the corresponding images and the values are non-negative.It can be clearly seen from the two edge information maps so that the edge information in Figure 5c looks much clearer than that in Figure 5a.Besides this, the pixels containing edge information have the same coordinates in these two edge information maps.This indicates that the enhancement result of our proposed method can preserve and enhance the edge information of the original image.In addition to that, for the detailed regions, the edge information in Figure 5c is more abundant than that in Figure 5a.In addition, for the homogeneous regions, the edge information in Figure 5c is similar to that in Figure 5a.This indicates that the proposed method is able to adaptively enhance the contrast in detailed regions and keep the contrast in homogeneous regions unchanged.For a better understanding of our proposed method (Algorithm 1), a brief pseudo code of the proposed method is given as follows.

333
preserve and enhance the edge information of the original image.In addition to that, for the detailed 334 regions, the edge information in Figure 5(c) is more abundant than that in Figure 5(a).In addition,

335
for the homogeneous regions, the edge information in Figure 5(c) is similar to that in Figure 5(a).This

336
indicates that the proposed method is able to adaptively enhance the contrast in detailed regions and 337 keep the contrast in homogeneous regions unchanged.For a better understanding of our proposed 338 method, a brief pseudo code of the proposed method is given as follows.
339 340 H and the desired uniformly distributed 2D histogram Ĥ using Equation ( 10) ndicates that the corresponding elements of the two sets are the goal should be changed to find a solution that can  L E .This is a bi-criteria optimization problem, and, in this paper, used to compute the solution below.
eter and varies over  instead of a constant.Moreover, notice that the value of  Y  Y satisfies the objective functions in Equation (18).We assume image is approximately equal to that of the global enhancement ) is rewritten as the formula below.
and C based on Equation ( 24), λ 0 and c 0 are the introduced parameters, ∆ is computed by applying the Sobel operator to X 11.Solve the bi-criteria optimization function in Equation ( 22), obtain the optimized contrast enhancement result Y, 9 of 21 rmance of global enhancement and local enhancement have ized enhancement is expected to combine the merits of both eve this goal, the optimized enhancement result Y has to : 1) the overall brightness variation should follow that of the l contrast should follow that of the local enhancement result s: ates that the corresponding elements of the two sets are goal should be changed to find a solution that can his is a bi-criteria optimization problem, and, in this paper, to compute the solution below.
and varies over nstead of a constant.Moreover, notice that the value of  Y atisfies the objective functions in Equation (18).We assume ge is approximately equal to that of the global enhancement rewritten as the formula below.
, and C are the regularization parameter set and the contrast parameter set 12. Return Y

Experimental Results and Discussion
In this section, experiments are conducted to evaluate the performance of the proposed method and the results were compared with those of three LCE methods: BCCE, ABMHE, and LGGSF.All these methods are applied to enhance the contrast of six different infrared images, which are captured by a cooled mid-wave infrared camera.The resolution of these images are 320 × 256 and the pre-processed 8-bit images are shown in Figure 6 as a reference.All these experiments are performed on a PC with Intel(R) Core(TM) i7 CPU(3.40GHz) and 16.00 GB RAM using MATLAB on a Windows 10 operation system.
10. Calculate  and C based on Equation (24), λ 0 and 0 c are the introduced parameters, Δ is computed by applying the Sobel operator to X 11.Solve the bi-criteria optimization function in Equation ( 22), obtain the optimized contrast enhancement result Y ,  , and C are the regularization parameter set and the contrast parameter set 12. Return Y

341
In this section, experiments are conducted to evaluate the performance of the proposed method 342 and the results were compared with those of three LCE methods: BCCE, ABMHE, and LGGSF.All

Parameter Setting
There are five parameters introduced in our proposed method.For these parameters, the values we chose are given in the above section.In this paper, we conduct a further discussion on the value set.The parameter r determines the size of the neighbor region when calculating the 2D histogram.A larger value of r means more neighbor pixels are taken into account and also corresponds to the higher computational complexity.Thus, we set r = 2 in our proposed method.β is introduced to reduce the over-enhancement of the global enhancement result in Section 3.1.The value of β is recommended to be set within the range of [0.1, 1].In our experiments, for the images with a resolution of 320 × 256, we set β = 0.2.For other images with higher resolution, a larger β value is recommended.As for the parameter α, the effect of different α values on the local enhancement results in Section 3.2 can be seen in Figure 4.A larger value of α means more local contrast is enhanced.In the meantime, the noise in the homogeneous regions is also enhanced.Thus, we set α = 1 in our proposed method to make a trade-off between the local contrast enhancement and the noise amplification.The parameters λ 0 and c 0 are introduced to determine the regularization parameter set   294

295
where  is a regularization parameter and varies over )   0, .The optimized enhancement result 296 obtained by  = 0 is equal to the global enhancement result.Nevertheless, as  increases to infinity, 297 it turns to resemble the local enhancement result.Therefore, a different optimized enhancement 298 result is obtained according to  and c .Considering the region characteristics of the input image, 299 larger  and c are needed for the detailed regions to get local contrast enhanced, but smaller  300 and c are more suitable for the homogeneous regions.Thus,  and c should be a region with 301 adaptive variables and should be represented as the component of n N instead of a constant.Moreover, notice that the value of  Y 303 is unknown and the desired image  Y satisfies the objective functions in Equation (18).We assume 304 that the average value of the desired image is approximately equal to that of the global enhancement 20) is rewritten as the formula below.

306
307 The optimal solution of Equation ( 21) is shown below. 308 309 and the contrast parameter set C based on the edge information ∆.As discussed before, when the value of   294

295
where  is a regularization parameter and varies over

303
is unknown and the desired image  Y satisfies the objective functions in Equation (18).We assume 304 that the average value of the desired image is approximately equal to that of the global enhancement . Then Equation ( 20) is rewritten as the formula below. 306 307 The optimal solution of Equation ( 21) is shown below.

308
approaches zero, the optimized enhancement result tends to keep the brightness variation of the global enhancement result.On the contrary, if the value of indicates that the corresponding elements of the two sets are tely.Therefore, the goal should be changed to find a solution that can imize . This is a bi-criteria optimization problem, and, in this paper, obenius norm is used to compute the solution below.
n N instead of a constant.Moreover, notice that the value of  Y e desired image  Y satisfies the objective functions in Equation( 18).We assume lue of the desired image is approximately equal to that of the global enhancement hen Equation ( 20) is rewritten as the formula below.
increases to infinity, the optimized enhancement result turns to keep the local contrast of the local enhancement result.Considering the noise suppression, we set λ 0 = 0.1 in our experiments.If more local contrast enhancement is required, λ 0 can be set to a larger value.
From the definition of C in Equation (24), it is clear that the values of the components in C are within the range of [0, c 0 ].Thus, the value of c 0 represents the maximum contrast enhancement ratio.In our experiments, we empirically set c 0 = 2 to obtain the proper contrast enhancement.

Qualitative Evaluation
The enhancement results of the reference images in Figure 6 using BCCE, ABMHE, LGGSF, and the proposed method are shown in Figures 7-12.The numbers of horizontal and vertical divisions of the image for BCCE and LGGSF are set to be k 1 = k 2 = 7.The clip limit parameter and local contrast enhancement factor for BCCE are set as β = 0.01 and α = 1.2.The local contrast enhancement factor for LGGSF is set as α = 8.For the ABMHE method, the block size is set to be 64 × 64.The horizontal and vertical steps are set to be one-quarter of the block size.
As shown in these figures, the intermediate global and local enhancement results are given together with the optimized result.For each test image, the final enhancement result of our proposed method is the optimized result of the intermediate enhancement results.It can be seen that the global enhancement results improve the visual quality with no artifacts, but the local contrast is not well enhanced.In the local enhancement results, the enhanced local contrast is enhanced but looks unnatural and the noise in homogeneous regions also emerges.The optimized results adaptively combine the merits of both global and local enhancement results.Therefore, the results improve the visual quality with an enhanced local contrast and no artifacts.
As shown in Figure 7, we can clearly find that the edge information of the buildings in the result of the proposed method and BCCE looks clearer than the others.The result of ABMHE causes over-enhancement in the trees of the image.However, in the result of LGGSF, more details of the trees and mountain are lost.In the region marked by the red rectangle, BCCE amplifies the noise and ABMHE causes blocking artifacts.
In Figure 8, as shown in the region marked by the red rectangle, the result of LGGSF shows over-enhancement artifacts and that of ABMHE lost some details in the darker area.It can be observed that there is brightness distortion in the enhancement results of BCCE, ABMHE, and LGGSF.Once again, BCCE and the proposed method produces clearer visual quality than ABMHE and LGGSF.
In Figure 9, over-enhancement artifacts appear in the homogeneous regions of the results of BCCE and ABMHE.The noise is also amplified in the result of BCCE.We can also observe that the tower crane marked by the red rectangle looks fuzzy in the results of BCCE, ABMHE, and LGGSF.The result of the proposed method looks more natural and clearer than that of the others.
In Figure 10, the result of BCCE appears to have severe brightness distortion in the region marked by the red rectangle and other homogeneous regions.This distortion also appears in the results of ABMHE and LGGSF.In addition, LGGSF also causes artifacts in some homogeneous regions.The enhancement result of the proposed method preserves the naturalness of the original image and gives better local contrast enhancement with brightness distortion.
In Figure 11, it is clear that the people in the region marked by the red rectangle of the proposed method are clearer than that of the BCCE, ABMHE, and LGGSF.Additionally, the result of the proposed method produces more legible edge information than the other methods.
In Figure 12, there are no large homogeneous regions in the scene.Thus, the brightness distortion is not apparent in the results of BCCE, ABMHE, and LGGSF.However, the enhancement results of ABMHE and LGGSF still look unnatural at the regions of the trees.We can see more details lost in that area.BCCE performs best in terms of local contrast enhancement.The proposed method is slightly inferior to BCCE.As observed in the region marked by the red rectangle, the objects in the results of ABMHE and LGGSF look fuzzy.BCCE and the proposed method give better visual quality.In Figure 11, it is clear that the people in the region marked by the red rectangle of the proposed method are clearer than that of the BCCE, ABMHE, and LGGSF.Additionally, the result of the proposed method produces more legible edge information than the other methods.
In Figure 12, there are no large homogeneous regions in the scene.Thus, the brightness distortion is not apparent in the results of BCCE, ABMHE, and LGGSF.However, the enhancement results of ABMHE and LGGSF still look unnatural at the regions of the trees.We can see more details lost in that area.BCCE performs best in terms of local contrast enhancement.The proposed method is slightly inferior to BCCE.As observed in the region marked by the red rectangle, the objects in the results of ABMHE and LGGSF look fuzzy.BCCE and the proposed method give better visual quality.

Quantitative Evaluation
For further evaluation, some objective assessment metrics are employed to numerically evaluate the performance of the proposed method.The metrics used in this paper are the measure of enhancement by entropy (EMEE) [39], structural similarity (SS) [40], no-reference structural sharpness (NRSS) [41], and the lightness order error (LOE) [38].
EMEE is defined based on the concept of entropy, which calculates the average ratio of maximum to minimum intensities in each block in decibels.The definition of EMEE is shown below.

Quantitative Evaluation
For further evaluation, some objective assessment metrics are employed to numerically evaluate the performance of the proposed method.The metrics used in this paper are the measure of enhancement by entropy (EMEE) [39], structural similarity (SS) [40], no-reference structural sharpness (NRSS) [41], and the lightness order error (LOE) [38].
EMEE is defined based on the concept of entropy, which calculates the average ratio of maximum to minimum intensities in each block in decibels.The definition of EMEE is shown below.
where k 1 and k 2 are the numbers of blocks the given image I is divided in the horizontal and vertical direction, I max k,l and I min k,l are, respectively, the maximum and minimum intensity of the block (k, l), α is an additional parameter, and c is a small constant to avoid dividing by zero.Generally, the larger EMEE indicates better contrast of the given image.In this paper, we set α = 0.2, c = 0.0001, and 8 × 8 for the block size.
SS is based on the assumption that natural images are highly structured and is defined as the equation below.
where I o and I e are, respectively, the original image and enhanced image, σ I o ,I e is the covariance of I o , and I e , σ I o , and σ I e are the corresponding standard deviation, c is a small constant to avoid dividing by zero, and σ I o ,I e is estimated by the equation below.
where N is the number of pixels in the images and µ I o and µ I e are, respectively, the mean intensity of I o and I e .It is clear that SS indicates the structural similarity of the enhanced image when compared to the original image.The larger SS means the enhanced image is more close to the original image in terms of structure.NRSS is defined based on the well-known SSIM (structural similarity index measurement) [42] to indicate the structural sharpness of the given image, which is defined by the equation below.
where B G I i is one of the K overlapped sub-blocks with higher variations in G I , G I is the gradient image of the given image I and is calculated by applying Sobel operator to the given image, and F(•) is the 2D Gaussian-blurred function.Large NRSS means that the given image looks clear and is rich in details.In our experiments, the block size is set to be 32 × 32 and the step is set to half the block size, and K is set to be half of the total number of the overlapped sub-blocks.
LOE is defined based on the lightness order error between the original image and the enhanced image.The definition of LOE is shown below.
where U(x, y) is the unit step function, ⊕ is the exclusive or operator, and I D o and I D e are, respectively, the down-sampled version of I o and I e to reduce the computational complexity.The ratio between the size of the original image and that of the down-sampled image is set as r = 16 in our experiments.These four methods are also applied to another 10 infrared images shown in Figure 13 for the quantitative evaluation.The values of the objective assessment metrics are given in Table 2.In Table 2, it can be seen that the proposed method gives lower EMEE values for the images that contain more homogeneous regions or fewer detail regions, which is the same as those EMEE values in Table 1.The NRSS values of the proposed method are close to those of BCCE and higher than those   In Table 2, it can be seen that the proposed method gives lower EMEE values for the images that contain more homogeneous regions or fewer detail regions, which is the same as those EMEE values in Table 1.The NRSS values of the proposed method are close to those of BCCE and higher than those of ABMHE and LGGSF.Comparison of SS and LOE values shows that the proposed method always gives the highest SS values and lowest LOE values.This means the proposed method has better performance than the other methods for preserving the naturalness and structure of the original images.All these quantitative evaluation values demonstrate that the proposed method has better performance on adaptive contrast enhancement, structural similarity, visual quality improvement, and naturalness preservation.
The average computation time for the four methods to process the test images in Figures 6 and 13 are given in Table 3.As shown in the table, the average computation time of BCCE is lower than the other methods.For LGGSF, the sub-block classification and labeling lead to more time consumption than BCCE.In our proposed method, the utilization of the histogram specification and the local contrast enhancement part require more computation time than LGGSF and BCCE.In ABMHE, the overlapped sub-blocks and the sub-block search operations make it the most time-consuming method among the four methods.

Conclusions
In this paper, we proposed an optimized contrast enhancement method for the visual quality improvement of infrared images.The proposed method produces enhanced images by making an optimal compromise between the global and local enhancement results that consider the edge information of the original image.In the proposed method, based on the 2D histogram of the original image, the global and local enhancement results are obtained by applying histogram specification globally and locally.Then, an objective function is constructed with global brightness and local contrast constrained conditions.The desired image is obtained by finding an analytical solution for the quadratic optimization problem.The regularization parameter set elements of the two sets are divided separately.The goal is to find a solution that can simultaneously minimize G E and L E .Moreover, notice that the variation ranges of G E and L E are not of the same magnitude.The expression of the second objective is rewritten by the equation below.
where  is a regularization parameter and varies over )   0, .The optimized enhancement result obtained by  = 0 is equal to the global enhancement result.Nevertheless, as  increases to infinity, it turns to resemble the local enhancement result.Therefore, a different optimized enhancement result is obtained according to  and c .Considering the region characteristics of the input image, larger  and c are needed for the detailed regions to get local contrast enhanced, but smaller  and c are more suitable for the homogeneous regions.Thus,  and c should be a region with adaptive variables and should be represented as the component of n N instead of a constant.Moreover, notice that the value of  Y is unknown and the desired image  Y satisfies the objective functions in Equation( 18).We assume that the average value of the desired image is approximately equal to that of the global enhancement image   () G YY .Then Equation ( 20) is rewritten as the formula below.
The optimal solution of Equation ( 21) is shown below.


where 1 is the  MN one matrix.As shown in Equation ( 22), the first term indicates that the optimized enhancement result resembles the global enhancement result, and the second term adds the weighted local contrast of the local enhancement result to the average value of the global and contrast parameter set C for the optimization function are adaptively determined by the edge information of the original image.Experiments are conducted for the qualitative and quantitative assessment of the proposed method with three block-based methods for comparison.Experimental results demonstrate that the proposed method produces better visual quality improvement in terms of structural similarity, naturalness preservation, and local contrast enhancement.

Figure 1 .
Figure 1.The overall block diagram of the proposed method.

8 0,2 1 .
In order to obtain both global and local enhancement results based on the 2D histogram, the 2D histogram should contain both global and local gray level distribution characteristics of the input image.Thus, in our proposed method, the 2D histogram is calculated based on the definition in Equation(2).

Figure 1 .
Figure 1.The overall block diagram of the proposed method.

Figure 2 .
Figure 2. (a) The input infrared image and (b) its 2D histogram using a × 5 5 neighborhood.The values of ( , ) h i j are shown in a logarithmic scale.

Figure 2 .
Figure 2. (a) The input infrared image and (b) its 2D histogram using a 5 × 5 neighborhood.The values of h(i, j) are shown in a logarithmic scale.

Figure 3 .
Figure 3. (a) This is the global mapping function and (b) is the global enhancement result of the input image in Figure 2(a).

Figure 3 .
Figure 3. (a) This is the global mapping function and (b) is the global enhancement result of the input image in Figure 2a.

Figure 4 .
Figure 4.The local enhancement result of the input image in Figure 2(a) with different values of α (first row) and the corresponding local mapping functions 50 T , 100 T , and 150 T with the global

Figure 4 .
Figure 4.The local enhancement result of the input image in Figure 2a with different values of α (first row) and the corresponding local mapping functions T 50 , T 100 , and T 150 with the global mapping function as a reference (second row).

310
optimized enhancement result resembles the global enhancement result, and the second term adds 311 the weighted local contrast of the local enhancement result to the average value of the global

)
  0, .The optimized enhancement result by  = 0 is equal to the global enhancement result.Nevertheless, as  increases to infinity, o resemble the local enhancement result.Therefore, a different optimized enhancement btained according to  and c .Considering the region characteristics of the input image, and c are needed for the detailed regions to get local contrast enhanced, but smaller  re more suitable for the homogeneous regions.Thus,  and c should be a region with variables and should be represented as the component of enhancement result 296 obtained by  = 0 is equal to the global enhancement result.Nevertheless, as  increases to infinity, 297 it turns to resemble the local enhancement result.Therefore, a different optimized enhancement 298 result is obtained according to  and c .Considering the region characteristics of the input image, 299 larger  and c are needed for the detailed regions to get local contrast enhanced, but smaller  300 and c are more suitable for the homogeneous regions.Thus,  and c should be a region with 301 adaptive variables and should be represented as the component of

290
In this scenario, the operator  .indicates that the corresponding elements of the two sets are 291 multiplied separately.Therefore, the goal should be changed to find a solution that can a bi-criteria optimization problem, and, in this paper, 293 the square of the Frobenius norm is used to compute the solution below.
enhancement result 296 obtained by  = 0 is equal to the global enhancement result.Nevertheless, as  increases to infinity, 297 it turns to resemble the local enhancement result.Therefore, a different optimized enhancement 298 result is obtained according to  and c .Considering the region characteristics of the input image, 299 larger  and c are needed for the detailed regions to get local contrast enhanced, but smaller  300 and c are more suitable for the homogeneous regions.Thus,  and c should be a region with 301 adaptive variables and should be represented as the component of

310) ( 23 ) where 1
optimized enhancement result resembles the global enhancement result, and the second term adds 311 the weighted local contrast of the local enhancement result to the average value of the global 312 is the M × N one matrix.As shown in Equation (23), the first term indicates that the optimized enhancement result resembles the global enhancement result, and the second term adds the weighted local contrast of the local enhancement result to the average value of the global enhancement result.Therefore, based on the proposed method, the brightness variation of the global enhancement result and the local contrast of the local enhancement result are merged into the optimized enhancement result.It is clear that the characteristics of the optimized enhancement result are related to the values of 9 of 21 e of global enhancement and local enhancement have nhancement is expected to combine the merits of both s goal, the optimized enhancement result Y has to overall brightness variation should follow that of the ast should follow that of the local enhancement result to the global enhancement result.Nevertheless, as  increases to infinity, ble the local enhancement result.Therefore, a different optimized enhancement according to  and c .Considering the region characteristics of the input image, are needed for the detailed regions to get local contrast enhanced, but smaller  suitable for the homogeneous regions.Thus,  and c should be a region with s and should be represented as the component of  = 0 is equal to the global enhancement result.Nevertheless, as  increases to infi 297 it turns to resemble the local enhancement result.Therefore, a different optimized enhancem 298 result is obtained according to  and c .Considering the region characteristics of the input im 299 larger  and c are needed for the detailed regions to get local contrast enhanced, but smalle 300 and c are more suitable for the homogeneous regions.Thus,  and c should be a region w 301 adaptive variables and should be represented as the component of Remote Sens. 2019, 11, x FOR PEER REVIEW 10 of 21 enhancement result.Therefore, based on the proposed method, the brightness variation of the global 313 enhancement result and the local contrast of the local enhancement result are merged into the 314 optimized enhancement result.It is clear that the characteristics of the optimized enhancement result 315 are related to the values of  and C .In this paper, C and  are adaptively determined based on 316 the edge information of the input image.The edge information Δ is first computed by using the 317 Sobel operator and then is linearly mapped to the same dynamic range of the output image.The value 318 of each element in  and C are defined to be positively correlated with the corresponding value 319 of edge information and are computed by the equation below.

cFigure 5 .Figure 5
Figure 5. (a) is the map of the edge information Δ , (b) is the optimized enhancement result Y , and

Figure 5 .Algorithm 1 :
Figure 5. (a) is the map of the edge information ∆, (b) is the optimized enhancement result Y, and (c) is the map of the edge information of Y. Algorithm 1: Optimized Contrast Enhancement Method.Input: original infrared image X Output: enhanced image Y 1. Pre-process X to be the 8-bit image 2. Calculate the 2D histogram H of image X based on Equation (2), the parameter r determines the size of the neighbor region 3. Obtain the clipped 2D histogram H * by clipping H and redistributing the exceeded pixels, normalize H * , the clip point is determined by the parameter β 4. Calculate the global mapping function T G based on the cumulative distribution functions of H * and the desired uniformly distributed 2D histogram Ĥ using Equation (10) 5. Map the image X to the global enhancement result Y G using Equation (11) 6. Construct the desired local histogram Ĥi based on the Gaussian distribution probability density function using Equation (13) 7. Calculate the original local mapping function T i based on the cumulative distribution functions of H i and the desired local 2D histogram Ĥi using Equation (15) 8. Solve the bi-criteria optimization function in Equation (16), obtain the optimal local mapping function T i , α is introduced to be a regularization parameter 9. Map the image X to the local enhancement result Y L using Equation (18) 10.Calculate is the set of average values of sub-blocks in  Y , L Y is the set of , c is a constant and is defined as the contrast parameter, G E ordered sets of difference values in the condition of global arity, and the operator ./indicates that the corresponding separately.The goal is to find a solution that can simultaneously tice that the variation ranges of G E and L E are not of the same cond objective is rewritten by the equation below.

)
  0, .The optimized enhancement result bal enhancement result.Nevertheless, as  increases to infinity, cement result.Therefore, a different optimized enhancement d c .Considering the region characteristics of the input image, detailed regions to get local contrast enhanced, but smaller  mogeneous regions.Thus,  and c should be a region with resented as the component of

e
set of average values of sub-blocks in  Y , L Y is the set of is a constant and is defined as the contrast parameter, G E red sets of difference values in the condition of global, and the operator ./indicates that the corresponding rately.The goal is to find a solution that can simultaneously that the variation ranges of enhancement result nhancement result.Nevertheless, as  increases to infinity, ent result.Therefore, a different optimized enhancement .Considering the region characteristics of the input image, ailed regions to get local contrast enhanced, but smaller  eneous regions.Thus,  and c should be a region with nted as the component of

343
these methods are applied to enhance the contrast of six different infrared images, which are captured 344 by a cooled mid-wave infrared camera.The resolution of these images are × 320 256 and the pre-345 processed 8-bit images are shown in Figure 6 as a reference.All these experiments are performed on 346 a PC with Intel(R) Core(TM) i7 CPU(3.40GHz) and 16.00GB RAM using MATLAB on a Windows 10 347 operation system.
Remote Sens. 2019, 11, x FOR PEER REVIEW 9 of 21 in homogeneous regions.Since the performance of global enhancement and local enhancement have 274 complementary characteristics, the optimized enhancement is expected to combine the merits of both 275 global and local enhancements.To achieve this goal, the optimized enhancement result Y has to 276 make a trade-off between two objectives: 1) the overall brightness variation should follow that of the 277 global enhancement result and 2) the local contrast should follow that of the local enhancement result 278 [5].These two objectives are expressed as: 280 where  Y is the desired image,  Y is the set of average values of sub-blocks in  Y , L Y is the set of 281 average values of sub-blocks in L Y , c is a constant and is defined as the contrast parameter, G E 282 and L E are the lexicographically ordered sets of difference values in the condition of global 283 brightness and local contrast similarity, and the operator ./indicates that the corresponding 284 elements of the two sets are divided separately.The goal is to find a solution that can simultaneously 285 minimize G E and L E .Moreover, notice that the variation ranges of G E and L E are not of the same 286 magnitude.The expression of the second objective is rewritten by the equation below.

290
In this scenario, the operator . indicates that the corresponding elements of the two sets are 291 multiplied separately.Therefore, the goal should be changed to find a solution that can 292 simultaneously minimize G E and  L E .This is a bi-criteria optimization problem, and, in this paper,293the square of the Frobenius norm is used to compute the solution below.
Remote Sens. 2019, 11, x FOR PEER REVIEW 9 of 21 in homogeneous regions.Since the performance of global enhancement and local enhancement have 274 complementary characteristics, the optimized enhancement is expected to combine the merits of both 275 global and local enhancements.To achieve this goal, the optimized enhancement result Y has to 276 make a trade-off between two objectives: 1) the overall brightness variation should follow that of the 277 global enhancement result and 2) the local contrast should follow that of the local enhancement result 278 [5].These two objectives are expressed as: 280 where  Y is the desired image,  Y is the set of average values of sub-blocks in  Y , L Y is the set of 281 average values of sub-blocks in L Y , c is a constant and is defined as the contrast parameter, ordered sets of difference values in the condition of global 283 brightness and local contrast similarity, and the operator ./indicates that the corresponding 284 elements of the two sets are divided separately.The goal is to find a solution that can simultaneously expression of the second objective is rewritten by the equation below.

290
In this scenario, the operator  .indicates that the corresponding elements of the two sets are 291 multiplied separately.Therefore, the goal should be changed to find a solution that can a bi-criteria optimization problem, and, in this paper,293the square of the Frobenius norm is used to compute the solution below.
 = 0 is equal to the global enhancement result.Nevertheless, as  increases to infinity, 297 it turns to resemble the local enhancement result.Therefore, a different optimized enhancement 298 result is obtained according to  and c .Considering the region characteristics of the input image, 299 larger  and c are needed for the detailed regions to get local contrast enhanced, but smaller  300 and c are more suitable for the homogeneous regions.Thus,  and c should be a region with 301 adaptive variables and should be represented as the component of instead of a constant.Moreover, notice that the value of  Y the performance of global enhancement and local enhancement have racteristics, the optimized enhancement is expected to combine the merits of both hancements.To achieve this goal, the optimized enhancement result Y has to tween two objectives: 1) the overall brightness variation should follow that of the t result and 2) the local contrast should follow that of the local enhancement result ctives are expressed as:  Y is the set of average values of sub-blocks in  Y , constant and is defined as the contrast parameter, G E exicographically ordered sets of difference values in the condition of global al contrast similarity, and the operator ./indicates that the corresponding sets are divided separately.The goal is to find a solution that can simultaneously L E .Moreover, notice that the variation ranges of G E and L E are not of the same ression of the second objective is rewritten by the equation below.
enhancement result is equal to the global enhancement result.Nevertheless, as  increases to infinity, le the local enhancement result.Therefore, a different optimized enhancement ccording to  and c .Considering the region characteristics of the input image, re needed for the detailed regions to get local contrast enhanced, but smaller  itable for the homogeneous regions.Thus,  and c should be a region with and should be represented as the component of
performs best on naturalness preservation than the other three methods.The higher LOE values of BCCE, ABMHE, and LGGSF are caused by the block-based principle and the neglect of global characteristics.All values of these metrics demonstrate that the proposed method outperforms the other methods in terms of structural similarity, local contrast enhancement, and naturalness preservation.

Figure 13 .
Figure 13.Ten infrared images for quantitative evaluation.
of ABMHE and LGGSF.Comparison of SS and LOE values shows that the proposed method always gives the highest SS values and lowest LOE values.This means the proposed method has better performance than the other methods for preserving the naturalness and structure of the original images.All these quantitative evaluation values demonstrate that the proposed method has better performance on adaptive contrast enhancement, structural similarity, visual quality improvement, and naturalness preservation.

Figure 13 .
Figure 13.Ten infrared images for quantitative evaluation.
In this scenario, the operator . indicates that the corresponding elements of the two sets are multiplied separately.Therefore, the goal should be changed to find a solution that can simultaneously minimize G E and  L E .This is a bi-criteria optimization problem, and, in this paper, the square of the Frobenius norm is used to compute the solution below.
Y is the desired image,  Y is the set of average values of sub-blocks in  Y , L Y is the set of 281 average values of sub-blocks in L Y , c is a constant and is defined as the contrast parameter, G E 282 and L E are the lexicographically ordered sets of difference values in the condition of global 283 brightness and local contrast similarity, and the operator ./indicates that the corresponding 284 elements of the two sets are divided separately.The goal is to find a solution that can simultaneously 285 minimize G E and L E .Moreover, notice that the variation ranges of G E and L E are not of the same 286 magnitude.The expression of the second objective is rewritten by the equation below.
280where  the set of average values of sub-blocks in  Y , is a constant and is defined as the contrast parameter, G Eand L E are not of the same xpression of the second objective is rewritten by the equation below.
. * Since the performance of global enhancement and local enhancement have 274 complementary characteristics, the optimized enhancement is expected to combine the merits of both 275 global and local enhancements.To achieve this goal, the optimized enhancement result Y has to 276 make a trade-off between two objectives: 1) the overall brightness variation should follow that of the 277 global enhancement result and 2) the local contrast should follow that of the local enhancement result 278 [5].These two objectives are expressed as: ) LL E E Y 290 In this scenario, the operator  .indicatesthat the corresponding elements of the two sets are 291 multiplied separately.Therefore, the goal should be changed to find a solution that can 292 simultaneously minimize G E and  L E. This is a bi-criteria optimization problem, and, in this paper,293the square of the Frobenius norm is used to compute the solution below.
Remote Sens. 2019, 11, x FOR PEER REVIEW 9 of 21 in homogeneous regions.Since the performance of global enhancement and local enhancement have 274 complementary characteristics, the optimized enhancement is expected to combine the merits of both 275 global and local enhancements.To achieve this goal, the optimized enhancement result Y has to 276 make a trade-off between two objectives: 1) the overall brightness variation should follow that of the 277 global enhancement result and 2) the local contrast should follow that of the local enhancement result 278 [5].These two objectives are expressed as: 310 optimized enhancement result resembles the global enhancement result, and the second term adds 311 the weighted local contrast of the local enhancement result to the average value of the global 312 ./(1+ magnitude.The expression of the second objective is rewritten by the equation below.

Table 2 .
The values of EMEE, SS, NRSS, and LOE of the enhanced results obtained by applying the four methods to the 10 infrared images.

Table 3 .
The average computation time in seconds of the four methods on the test images.