Retinex Jointed Multiscale CLAHE Model for HDR Image Tone Compression

Abstract

Tone-mapping algorithms aim to compress a wide dynamic range image into a narrower dynamic range image suitable for display on imaging devices. A representative tone-mapping algorithm, Retinex theory, reflects color constancy based on the human visual system and performs dynamic range compression. However, it may induce halo artifacts in some areas or degrade chroma and detail. Thus, this paper proposes a Retinex jointed multiscale contrast limited adaptive histogram equalization method. The proposed algorithm reduces localized halo artifacts and detail loss while maintaining the tone-compression effect via high-scale Retinex processing. A performance comparison of the experimental results between the proposed and existing methods confirms that the proposed method effectively reduces the existing problems and displays better image quality.

1. Introduction

The human visual system (HVS) has a color constancy that aims to perceive the color of an object consistently, even if the lighting conditions change [1]. Retinex theory, a representative tone-mapping algorithm, is designed to mimic how the human eye and brain recognize and process light and color. Retinex separates images into illumination and reflectance components, enhancing the overall brightness, luminance contrast, and detail information in images. Retinex is used to adjust luminance under low-light conditions and reduce noise [2,3]. According to the hyperparameter σ of the Gaussian filter used to extract the illumination layer, the details and noise of the output image exhibit a trade-off. The multiscale Retinex (MSR) method combines single-scale Retinex (SSR) with multiple scales to address this problem [4]. The MSR method improves noise reduction and enhances details. However, it has limitations regarding noise removal and can induce a halo effect, distorting colors along the boundaries of high-contrast objects or figures.

In addition to the Retinex technique, various tone-mapping processes have been developed for compressing images with wide dynamic ranges. For example, Reinhard et al. proposed correcting the image through a simple sigmoid transfer curve to reproduce wide dynamic range images in various display environments [5]. The sigmoid transfer curve is defined by an experimental parameter, which could make it challenging to establish the optimal level of parameters for each image. Due to the significant differences in dynamic range between the input and output caused by tone mapping, some information may be lost or artifacts can occur. Therefore, several methods effectively preserve detailed image information by processing the basic and detailed components separately [6,7]. A prominent method for base–detail separation is the bilateral filter, which smooths input images while preserving the edge components. Durand and Dorsey developed an accelerated bilateral filter, the fast bilateral filter, to expedite filtering [8]. Moreover, Kwon et al. proposed edge-adaptive layer blurring to estimate and compensate for halo areas to reduce these artifacts caused by localized tone mapping [9]. Lee et al. proposed a tone-mapping technique through visual achromatic responses to reduce artifacts caused by differences between the compressed base and detail layers [10]. The extended version of the color appearance model (CAM) based on the HVS, iCAM06, is one of the techniques to reproduce wide dynamic range images. The iCAM06 reflects characteristics of the HVS, such as color adaptation and tone compression under diverse lighting conditions [11]. In iCAM06, feature-based fusion is applied to preserve image edge structures and details. However, there may be a detail reduction problem due to feature-based fusion with fixed-edge stopping functions. Hence, Kwon et al. proposed using the characteristics of the contrast sensitivity function as a base-detail separation method and detail compensation technique to preserve iCAM06 edge components [12].

To implement HDR image, multiple LDR images with various exposures under the same conditions are required. However, acquiring multiple images under precisely identical conditions in the real world can be challenging. Therefore, a technology is required to acquire an image with wide dynamic range through tone compression from one LDR image.

The existing method of removing surround through multiple blurring uses multiple Gaussian filters, so the output image is dependent on the σ value. A larger σ value leads to decreased detail, while a smaller one can result in halo and noise issues. Learning-based methods often suffer from low tone compression effects and cause some image distortions, such as an overall white balance shift in the image. As such, existing methods have limitations in expressing a single LDR image as an HDR image.

In this paper, we aim to represent a single LDR image as a superior HDR image in various aspects. This paper proposes effective tone compression and halo artifact reduction using Retinex theory and multiscale contrast limited adaptive histogram equalization (CLAHE). High-scale SSR is advantageous for global tone compression, and CLAHE is suitable for local tone compression. Therefore, by combining the two theories, we derive an image with excellent overall tone compression.

The level of image improvement of CLAHE varies depending on the TileSize and ClipLimit parameters. The proposed method enhances global tone-mapping performance based on high-σ value Retinex while integrating low levels of TileSize and ClipLimit in CLAHE to reduce halo artifacts and noise. Additionally, to increase sharpness and enhance detail, high levels of TileSize and ClipLimit are coordinated. The proposed method applies a multiscale CLAHE to the optimal global scale Retinex image to enhance global and local toning performance. Retinex images with reduced details due to trade-offs according to the high sigma value of the Gaussian filter were compensated for using multiscale CLAHE. The multiscale CLAHE technique compensates for reduced details using localized histogram equalizations based on two hyperparameters: TileSize and ClipLimit. Finally, the resultant image is generated through the color compensation step with the color channel of the original image. Thus, the proposed method enhances the image quality via effective tone mapping, detail representation, noise reduction, and color expression. In summary, Retinex jointed multiscale CLAHE improves the tone-compression effect across the entire image by combining the two technologies, which are complementary each other in terms of global and local.

2. Related Work

2.1. Retinex-Based Tone Processing

The human eye adjusts the visual system to perceive the inherent color of objects when color distortion occurs due to lighting conditions. However, for cameras, the dynamic region is narrower than the human eye and does not effectively compensate for color distortion. Retinex theory, which mimics the HVS, can compensate for the camera’s vulnerability to lighting and improve the visual quality of images. The SSR value is computed as follows [13]:

$$R_{ssr,i}\left(x,y\right)=\log I_i\left(x,y\right)-\mathrm{l}\mathrm{o}\mathrm{g}[G\left(x,y\right)\ast I_i\left(x,y\right)],$$

(1)

$$G\left(x,y\right)=Kexp[-(x^2+y^2)/{\sigma }^2],$$

(2)

$$K=1/{\sum }_x{\sum }_y\mathrm{e}\mathrm{x}\mathrm{p}[-(x^2+y^2)/{\sigma }^2],$$

(3)

where $I$ denotes the input image and $i$ represents the separated channel. $R_{ssr,i}\left(x,y\right)$ represents the result of SSR for the channel, whereas $G(x,y)$ represents the Gaussian function. In addition, the $\ast$ symbol is the convolution operator and $G\left(x,y\right)\ast I_i\left(x,y\right)$ denotes the blurred image, which is the illumination component of the original image, where $\sigma$ is the standard deviation of the Gaussian function, which is a hyperparameter adjustable by the user. In addition, $K$ is the normalization factor, set to 1 for the maximum value of the Gaussian filter.

Moreover, $\sigma$ is a parameter that determines the degree of blurring of the illumination component, and its value affects the trade-off between noise and detail. A higher $\sigma$ value improves noise reduction but may decrease the detail and increase the computational time. In contrast, a lower $\sigma$ value enhances the detail and reduces computational time but may lead to more noise, resulting in unnatural-looking images. Figure 1 illustrates the trade-off between the components for various $\sigma$ sizes.

Figure 1. Comparisons of the single-scale Retinex method with various standard deviation (σ) values.

Through Figure 1, it is observed that as the σ value decreases, halo artifacts intensify, and noise within the image increases significantly. The halo artifacts typically induce black rings around objects or cause internal colors of objects to blur into white, resulting in unnatural images. On the contrary, details such as cloud patterns or surface textures of objects can be better expressed. As the σ value increases, the image is strongly blurred by the Gaussian filter, so halo artifacts and noise decrease. The global toning performance also improves, resulting in improved visibility. However, strong blurring also damages the details of the image. It can be observed that cloud patterns or wall surfaces become fainter. Additionally, a higher σ value for the Gaussian filter leads to an increase in computational complexity during the convolution process with the original image.

The MSR method is an improved technique that uses this trade-off relationship to obtain natural results. Further, MSR can be obtained as a weighted sum of several SSR images. The following equation represents MSR:

$$R_{msr}\left(x,y\right)={\sum }_{n=1}^N{w}_n(log{I}_i\left(x,y\right)-\mathrm{l}\mathrm{o}\mathrm{g}[G_n\left(x,y\right)\ast I_i(x,y)]),$$

(4)

where $w_n$ and $G_n$ represent the weight and Gaussian filter in the nth SSR, respectively. An improved image can be obtained by connecting the benefits of noise reduction at high scales and detail enhancement at low scales through intermediate-scale images. As a result, $R_{msr}\left(x,y\right)$ represents the resulting image synthesized with multiple scale values.

An overall tone-mapped image can be obtained using the MSR method. However, in this process, the original colors of objects are somewhat distorted, resulting in unintended colors in the output. Therefore, MSR color restoration (MSRCR) aims to restore the original colors to acquire natural results from the results of MSR [4]. The MSRCR technique is obtained by multiplying the original MSR results by the color restoration function (CRF), given in Equation (5):

$${I^{\prime }}_i\left(x,y\right)=\frac{I_i(x,y)}{\sum _{j=0}^S{I}_j(x,y)},$$

(5)

where $I_i$ represents the ith channel of the input image and $S$ denotes the number of color channels in the input image. In addition, ${I^{\prime }}_i$ denotes the ratio of the total color component to the ith channel color component. In the commonly used RGB color space, $S$ typically has a value of 3. The CRF applied to the ith channel is calculated as follows:

$${CRF}_i\left(x,y\right)=\beta \mathrm{l}\mathrm{o}\mathrm{g}\left[\alpha {I^{\prime }}_i\left(x,y\right)\right],$$

(6)

where $\beta$ denotes the gain constant and $\alpha$ indicates the nonlinearity control constant.

Finally, the MSRCR image is obtained by multiplying the CRF obtained from MSR, represented in Equation (7):

$$R_{msrcr,i}\left(x,y\right)={CRF}_i\left(x,y\right)\ast R_{msr,i}(x,y).$$

(7)

While these multiscale syntheses of the SSR images and the color compensation can compensate for some faults of the original SSR images, some problems remain, such as distortion-induced halo artifacts and noise problems, highlighting prominent problems depending on the image characteristics.

2.2. Contrast Limited Adaptive Histogram Equalization

Histogram equalization is an image enhancement method that increases the contrast of an image using the cumulative distribution function (CDF) of pixel values in the image [14]. This method is widely used as one of the most straightforward image enhancement methods; however, it suffers from the problem of some regions of the image becoming saturated as a result of attempting to distribute the histogram of pixel values evenly across the entire image. The CLAHE method improves the limitations of histogram equalization by locally applying histogram equalization to multiple tiles of size M × M instead of the entire image [15]. This approach also prevents the excessive amplification of specific pixel values by limiting the maximum number of bins in the histogram using a hyperparameter, ClipLimit β. The control constant β is calculated as follows:

$$\beta =\frac{M^2}{L}(1+\frac{\alpha }{100}\left(S_{max}-1\right)),$$

(8)

where $M$ represents the size $M$ × $M$ of the parameter TileSize; therefore, $M^2$ denotes the number of pixels in each tile. In addition, $L$ represents the maximum brightness value of the image pixels, which is 256 for an 8-bit image. Moreover, $\propto$is the clip factor, which takes an arbitrary value between 0 and 100. The value of $S_{max}$is the maximum allowable slope, and when the clip factor α is set to 100, ClipLimit becomes a constant determined by $\left(\frac{M^2}{L}\right)\ast S_{max}$.

Moreover, CLAHE exhibits a trade-off while enhancing the image, such as performing detail representation, tone compression, and reduction of noise and halo artifacts, depending on the parameters ClipLimit and TileSize. Therefore, in the case of a single CLAHE performed by fixing the parameters, an overall image quality improvement is not expected. Additionally, optimal parameter values are not explicitly specified and are determined differently for each image based on the experience of the user. Consequently, there are limitations in achieving optimal quality enhancement for each image, and even setting incorrect parameter values may result in unnatural-looking images, as depicted in Figure 2. Figure 2a represents the original image, and Figure 2b reveals only slight improvement due to relatively low parameter settings. Figure 2c exhibits strong halo artifacts due to the high TileSize settings. In addition, Figure 2d demonstrates improved detail representation due to the high ClipLimit settings, but significant noise is observed.

Figure 2. Problems with incorrect parameter values for a single-scale CLAHE: (a) Original image, (b) low-enhanced image with low parameter values, (c) increased halo artifacts due to the high TileSize, and (d) increased noise due to the high ClipLimit.

3. Proposed Method

Figure 3 illustrates the tone-mapping algorithm proposed in this paper, which employs Retinex processing and multiscale CLAHE. The proposed method aims to address image distortion problems, such as halo artifacts and noise, which are not adequately resolved by Retinex theory and other tone-mapping algorithms by introducing multiscale CLAHE based on high-scale Retinex. The proposed method also aims to enhance tone compression and detail representation capabilities. The proposed method consists of SSR, multiscale CLAHE, and color correction, as illustrated in Figure 3.

Figure 3. Flow chart for the proposed method.

As shown in Figure 3, the algorithm proceeds in the order of SSR, multiscale CLAHE, and color correction. First, to reduce halo artifacts and noise, high-scale SSR is performed. In this process, a high-σ value Gaussian filter is applied to heavily blur the surround layer, effectively removing halo artifacts and obtaining a reflection layer with superior global toning. Next, the multiscale CLAHE described in the blue box is performed. CLAHE is used to compensate for visibility, such as details lost by strong SSR. However, single-parameter CLAHE can introduce noise while compensating for details, thereby compromising overall image quality. Therefore, this paper synthesizes multiple CLAHE layers with different parameters to suppress noise induction while compensating for details, thus enhancing overall image quality. The images from each scale are weighted and synthesized. Finally, the algorithm undergoes a process to naturally restore some distorted colors, as indicated in the green box of Figure 3. Lastly, as shown in the fourth box, the synthesized luminance channel and the color-restored a and b channels are combined, and the output image is provided by converting them into RGB channels. In summary, the main progress of the algorithm is as follows:

• To effectively address the halo artifact and noise issues that arise in conventional multi-blurring techniques, high-scale SSR is performed.
• To compensate for visibility while maintaining image quality, we perform CLAHE on multiple scales.
• To restore some distorted colors due to the previous procedure, the color correction method, MSRCR, is performed.

3.1. Decomposition of Reflectance and Illumination Layers

First, the input image in the RGB color space is converted to the Lab color space. Because the Lab color space separates the luminance channel and color channels, the luminance channel can be synthesized, reducing the color information distortion in the image. The SSR method is applied to the separated luminance channel to divide it into illumination and reflectance components. As depicted in Figure 1, a higher value of the standard deviation σ of the Gaussian function results in a more effective reduction of the halo artifacts and noise. Therefore, considering the compensation for details through CLAHE, even as image details decrease, σ is set to a high value to reduce noise significantly and perform high tone mapping. In this paper, considering that the computational time is more than double the standard deviation increases, σ is set appropriately to 250. The decomposition of the luminance channel into reflectance and illumination layers through the Gaussian function is performed as follows:

$$L_s\left(x,y\right)=G\left[L\right(x,y\left)\right],$$

(9)

$$L_R\left(x,y\right)=L\left(x,y\right)-G\left[L\left(x,y\right)\right],$$

(10)

where $G$ represents the Gaussian filter, $L_s$ denotes the illumination layer image obtained by applying a Gaussian filter to the luminance channel $L$ of the input image, and $L_R$ indicates the reflectance layer image obtained by removing the illumination layer component from the luminance image $L$.

3.2. Multiscale CLAHE

High-scale SSR processing enables acquiring images with no noise and effective tone compression. However, the brightness contrast and detail are decreased due to the trade-off. The image enhancement method CLAHE is applied to increase brightness contrast and compensate for the decreased detail to address this problem. The degree of image enhancement varies depending on the direction in which the hyperparameters of CLAHE (ClipLimit and TileSize) are set. Figure 4 depicts images with CLAHE applied with various parameters.

Figure 4. Comparison of image enhancement with various parameter values for ClipLimit and TileSize.

Generally, as the size of ClipLimit increases, the contrast and detail compensation also increase. However, when ClipLimit becomes too large, excessive compensation leads to increased noise and results in unnatural output. As for TileSize, increasing it reduces the overall noise and improves tone compression, but excessive values lead to severe halo artifacts and image degradation, resulting in unnatural output. Therefore, an image with superior brightness contrast, detail compensation, noise reduction, and tone compression can be obtained by composing n pairs (ClipLimit and TileSize), applying weights to the images obtained with each pair, and combining them. For optimal synthesis, the diagonal matrix images from the image array in Figure 3 are adopted as the images used for composition. High-scale CLAHE achieves excellent detail representation and tone compression, whereas low-scale CLAHE reduces noise and mitigates halo artifacts. Middle-scale CLAHE serves as a connector for combining them. Superior image quality can be achieved through this synthesis. The parameters of the diagonal matrix result in each pair of parameters being (n, 2ⁿ⁺¹). In this paper, n = 3; thus, three CLAHE layers were synthesized, and the parameter pairs (β_n, M_n × M_n) for each CLAHE layer were set to (1.0, 4 × 4), (2.0, 8 × 8), and (3.0, 16 × 16), respectively.

The reflectance layer image L_R of the luminance channel is divided into tiles of the corresponding size according to the TileSize parameter M_n × M_n to apply local histogram equalization. The representation of this process follows:

$$L_R=\left[\begin{array}{ccc}{L}_{R11}& \cdots & L_{R1j}\\ ⋮& \ddots & ⋮\\ L_{Ri1}& \cdots & L_{Rij}\end{array}\right](1\le i\le \frac{P}{M_n},1\le j\le \frac{Q}{M_n}),$$

(11)

where $L_{Rij}$represents the regions divided into tiles of size $M_n$ × $M_n$, and $P$and $Q$are the dimensions of the input image. The ij-th region of the image corresponding to each tile is as follows:

$$L_{Rij}=\left[\begin{array}{ccc}{T}_{11}& \cdots & T_{1M_N}\\ ⋮& \ddots & ⋮\\ T_{M_{n}1}& \cdots & T_{M_n{M}_n}\end{array}\right],$$

(12)

where $T_{M_n{M}_n}$ represents the pixel values of the ij-th tile of the input image.

Histogram equalization is applied to each tile by limiting the number of histogram bins based on the ClipLimit value. First, the probability mass function (PMF) is constructed from the pixel values of each tile to obtain the CDF, which serves as a means of applying histogram equalization, as follows:

$$h_{ij}\left(l\right)={\sum }_{x=1}^{M_n}{\sum }_{y=1}^{M_n}\delta \left(T_{xy}-l\right),l=0,1,2,\cdots ,L-1,$$

(13)

where $h_{ij}$represents the PMF of the ijth tile, $T_{xy}$ denotes the pixel value at the xy-th position within the ij-th tile, and $L$ represents the maximum pixel value. Additionally, $\delta$ is the impulse function that determines which brightness value the $T_{xy}$ corresponds to.

The PMF must be limited so that it does not have more than a certain number of pixels at specific brightness values to prevent excessive brightness in certain areas. Moreover, if the limit is exceeded, the excess pixels are redistributed evenly across the entire brightness range. This process is conducted according to the following equations:

$$N_{excess}={\sum }_{k=0}^{L-1}\mathrm{m}\mathrm{a}\mathrm{x}(\left(h_{ij}\left(k\right)-{\beta }_n\right),0),$$

(14)

$$N_{average}=\frac{N_{excess}}{L},$$

(15)

$$h_{clipped}\left(l\right)=h_{ij}\left(l\right)+N_{average},$$

(16)

where $N_{excess}$ represents the number of pixels exceeding the ClipLimit${\beta }_n$, $L$ denotes the maximum brightness value of the pixels (256 for an 8-bit image), and $N_{average}$ indicates the number of pixels to be evenly redistributed across the entire brightness range. In addition, $h_{clipped}$ represents the new PMF of the ijth tile after the redistribution.

After redistribution, the process is repeated by limiting the pixel count again until $N_{excess}$ reaches zero. Finally, $h_{clipped}$ takes the following form:

$$h_{clipped}\left(l\right)=\left\{\begin{array}{c}{ \beta }_n \left(h\left(l\right)\ge \beta \right)\\ h_{ij}\left(l\right)+N_{up} \left(h\left(l\right)<\beta \right)\end{array}\right..$$

(17)

The number of pixels exceeding the ClipLimit is constrained to ${\beta }_n$, whereas the number of pixels less than ${\beta }_n$ increases by the amount of $N_{up}$ which is the amount of the pixels for redistribution. The formula for obtaining the CDF for histogram equalization using the constrained PMF is as follows.

$$C_{ij}\left(l\right)=\frac{1}{M_n^2}{\sum }_{k=0}^l{h}_{clipped}\left(k\right),$$

(18)

where $C_{ij}$ represents the CDF to be multiplied by each pixel value of the ij-th tile. In this process, the CDF is normalized by dividing by the total number of pixels in the tile, $M_n^2$, to ensure that the slope of the CDF does not exceed 1 within the tile.

The CDF, $C_{ij}$, obtained from Equation (18), is applied to the ij-th tile. Then, histogram equalization is performed by multiplying by the maximum pixel intensity value, L − 1, to the tile. This process is conducted according to the following equation:

$${L^{\prime }}_{Rij}\left(x,y\right)=C_{ij}\left(L_{Rij}\left(x,y\right)\right)\times \left(L-1\right),$$

(19)

where ${L^{\prime }}_{Rij}$ represents the ij-th tile of the $L_R$ image, where CLAHE was applied, and the $\times$ symbol is the multiplication operator.

After performing local histogram equalization for each tile, block artifacts occur (Figure 5). Pixel values between tile boundaries do not smoothly transition, resulting in noticeable gaps. Bilinear interpolation is applied to each pixel based on the tile centers to address this problem. Before applying the technique, pixels are categorized into internal regions, boundary regions, and corner regions based on their positions and neighboring tiles (Figure 6). Different equations are applied to each region accordingly.

Figure 5. Blocking effect due to local histogram equalization.

Figure 6. Region classification based on pixel positions in the input image: (a) Inside region; (b) Boundary region; (c) Corner region.

First, the transformation function for pixels in the internal region is calculated as follows for the inside region:

$${L^{″}}_{Rij}\left(x,y\right)=\frac{s}{r+s}\left(\frac{q}{p+q}f\left({L^{\prime }}_{Ri-1j-1}\right)+\frac{p}{p+q}f\left({L^{\prime }}_{Rij-1}\right)\right)+\frac{r}{r+s}\left(\frac{q}{p+q}f\left({L^{\prime }}_{Ri-1j}\right)+\frac{q}{p+q}f\left({L^{\prime }}_{Rij}\right)\right)$$

(20)

where ${L^{\prime }}_{Rij}$, ${L^{\prime }}_{Ri-1j}$, ${L^{\prime }}_{Rij-1}$, and ${L^{\prime }}_{Ri-1j-1}$ are the center pixels of the four adjacent tiles of a desired pixel, in which each tile is subjected to contrast restriction histogram equalization. The values of p, q, s, and r represent distances between the current pixels from the centers of the four adjacent tiles. $f\left({L^{\prime }}_{Rij}\right)$, $f\left({L^{\prime }}_{Ri-1j}\right), f\left({L^{\prime }}_{Rij-1}\right),$and $f\left({L^{\prime }}_{Ri-1j-1}\right)$ represent the value of the center pixel of each tile, and ${L^{″}}_{Rij}\left(x,y\right)$ indicates the pixel value after the interpolation method is applied.

The transformation functions for the boundary and edge regions, respectively, are similarly expressed as follows:

$${L^{″}}_{Rij}\left(x,y\right)=\frac{s}{r+s}f\left({L^{\prime }}_{Rij-1}\right)+\frac{r}{r+s}f\left({L^{\prime }}_{Rij}\right),$$

(21)

$${L^{″}}_{Rij}\left(x,y\right)={L^{\prime }}_{Rij}\left(x,y\right).$$

(22)

Hence, each output image fully processed by CLAHE with each scale is represented as follows:

$$L_{mapped,n}=\left[\begin{array}{ccc}{L″}_{R11}& \cdots & {L″}_{R1j}\\ ⋮& \ddots & ⋮\\ {L″}_{Ri1}& \cdots & {L″}_{Rij}\end{array}\right](1\le i\le \frac{P}{M_n},1\le j\le \frac{Q}{M_n}).$$

(23)

The combined image of the obtained output images at each scale is represented as follows:

$$L_{fused}={\sum }_{n=1}^N{w}_n{L}_{mapped,n},$$

(24)

$\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e} L_{mapped}$ represents the reflection layer image $L_R$ with the high-scale SSR and single-scale CLAHE applied, where $w_n$ denotes the weight applied to each image. In addition, $N$indicates the number of images for fusion, which is set to $N$ = 3 in this paper. Finally, $L_{fused}$ denotes the reflection layer image with multiscale CLAHE applied.

3.3. Color Compensation

After the conversion to the Lab color space, the color information of the original image is separated into the a and b channels, whereas the luminance information is represented by the $L$ channel. Therefore, changes in the luminance channel $L$ due to tone adjustments do not affect the original color information. However, imbalances may occur between the modified luminance channel $L$ and the color channels a and b, leading to unnatural color representation. Additionally, when applying Retinex theory, some colors may distort during the tone-mapping process, differing from the intended colors. Retinex theory incorporates gray world assumption (GWA) theory [4]. GWA is a technique based on the assumption that the mean values of the R, G, and B channels of an image are equal. It adjusts the gain to make the mean values of the R, G, and B channels equal. While GWA performs well in images with a uniform range of colors, it can lead to color desaturation in images where certain colors dominate, resulting in a grayscale appearance. Similarly, Retinex theory that embeds GWA can cause the same problem. As seen in Figure 7, it can distort the original colors of plants and buildings, making them gray. Therefore, a process to restore the distorted grayscale images using the color information from the original image is necessary.

Figure 7. Color desaturation caused by Retinex and different L channel ratios with a and b channels: (a) an original input image, (b) an image without color correction, and (c) an image with color correction.

Color channels that match the modified luminance channel must be obtained by correcting the color information of the output image with the color information of the original image to address these problems. The color correction method uses the MSRCR technique, which performs color restoration based on multiscale Retinex theory. The algorithm for color correction is provided as follows:

$$I^{\prime }\left(x,y\right)=\frac{I(x,y)}{\sum _{j=0}^S{I}_j(x,y)},$$

(25)

$$CC\left(x,y\right)=\gamma \mathrm{l}\mathrm{o}\mathrm{g}\left(\alpha I^{\prime }\left(x,y\right)\right),$$

(26)

$${RGB}_{CC}\left(x,y\right)=CC(x,y)·{RGB}_{fused}(x,y),$$

(27)

$\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e} I$ represents the input image and $I_j$ represents each R, G, and B channel of the input image. The term $CC$ is the color correction function obtained for $I\prime$ through constants $\propto$ and $\gamma$. In this case, the experimentally determined appropriate values are $\propto$ = 125 and $\gamma$ = 46. The term ${RGB}_{fused}(x,y)$ represents the image without color correction. Finally, ${RGB}_{CC}\left(x,y\right)$ represents the result image with color correction.

The color correction function, obtained by applying the proposed method to the processed luminance channel $L_{fused}$ and the original a and b channels, is multiplied by the RGB fused image, which is the image transformed into the RGB color space. However, due to the high correlation between luminance and each color channel in the RGB color space, the luminance component could be distorted. Therefore, only the color information of the image after color correction is obtained, and the luminance information is synthesized using the luminance channel $L_{fused}$ of the proposed method. For this purpose, the ${RGB}_{CC}$ image is converted back to the Lab color space, and only the a and b color channels are extracted. Finally, the synthesized luminance channel $L_{fused}$ and color channels$a_{cc}$ and $b_{cc}$, processed by the color correction method, are combined, and the final synthesized image is obtained after converting it to the RGB color space.

4. Simulations

4.1. Experimental Results

In this paper, we tested our model with various images, including benchmark datasets, i.e., CS [16] and Meylan et al. HDR [17,18] datasets, and self-images [19]. The results of several tone-mapping algorithms and the proposed method were compared to evaluate the performance of the proposed method. Low-light images for comparing tone compression in bright and dark areas, outdoor images for comparing detail compensation, and a color booth image suitable for the simultaneous comparison of color reproduction and tone compression of input images were used for the performance evaluation. The experiments and evaluation were implemented on Windows 10 Pro 64bit and an Intel(R) Core(TM) i7-7700 CPU @ 3.60 GHz, using Python 3.11.4 and MATLAB R2023a. Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13 and Figure 14 depict the comparison results for images with a wide range of brightness, from dark to bright areas. For existing methods, such as Reinhard [5] and L1L0, the tone increases across the brightness range, resulting in saturation in bright areas and reduced visibility in dark areas. Although iCAM06 and the method by Kwon et al [12]. ensure visibility in dark areas, color distortions, such as saturation, occur in some areas, and an overall reddish tone appears in the image. In contrast, the proposed method effectively compresses tones while maintaining visibility in dark areas without saturation in bright areas. Furthermore, detailed areas, such as tree trunks, leaves, tiles, and tire surfaces, display strong contrasts according to each area, effectively representing detailed information.

Figure 8. Input and result images: (a) Original, (b) Reinhard (2012) [5], (c) L1L0, (d) iCAM06, (e) Kwon et al. [12], and (f) proposed.

Figure 9. Input and result images: (a) Original, (b) Reinhard (2012) [5], (c) L1L0, (d) iCAM06, (e) Kwon et al. [12], and (f) proposed.

Figure 10. Input and result images: (a) Original, (b) Reinhard (2012) [5], (c) L1L0, (d) iCAM06, (e) Kwon et al. [12], and (f) proposed.

Figure 11. Input and result images: (a) Original, (b) Reinhard (2012) [5], (c) L1L0, (d) iCAM06, (e) Kwon et al. [12], and (f) proposed.

Figure 12. Input and result images: (a) Original, (b) Reinhard (2012) [5], (c) L1L0, (d) iCAM06, (e) Kwon et al. [12], and (f) proposed.

Figure 13. Input and result images: (a) Original, (b) Reinhard (2012) [5], (c) L1L0, (d) iCAM06, (e) Kwon et al. [12], and (f) proposed.

Figure 14. Input and result images: (a) Original, (b) Reinhard (2012) [5], (c) L1L0, (d) iCAM06, (e) Kwon et al. [12], and (f) proposed.

Figure 15 and Figure 16 present the comparison results for the night-time images. The existing methods often suffer from oversaturation, leading to saturation in some areas or decreased color contrast, causing overall haziness or reduced detail. Especially in Figure 16, areas like the sky lack accurate color reproduction and appear hazy. In contrast, the proposed method represents colors without oversaturation, maintaining a clear color contrast and accurately expressing details in the walls, patterns, and vegetation.

Figure 15. Input and result images: (a) Original, (b) Reinhard (2012) [5], (c) L1L0, (d) iCAM06, (e) Kwon et al. [12], and (f) proposed.

Figure 16. Input and result images: (a) Original, (b) Reinhard (2012) [5], (c) L1L0, (d) iCAM06, (e) Kwon et al. [12], and (f) proposed.

Figure 17 illustrates the performance comparison of a color chart. The Reinhard and L1L0 methods effectively represent the primary colors of the color chart, but the achromatic colors exhibit saturation. In contrast, the iCAM06 and Kwon et al. methods suffer from oversaturation, resulting in an inaccurate color representation and excessive brightness in the image. In contrast, the proposed method accurately reproduces natural colors for both the primary and achromatic color charts while demonstrating great detailed representation in the buildings and floor tiles.

Figure 17. Input and result images: (a) Original, (b) Reinhard (2012) [5], (c) L1L0, (d) iCAM06, (e) Kwon et al. [12], and (f) proposed.

4.2. Objective Assessment

To conduct an objective evaluation, we performed six quantitative metrics: the local phase coherence–sharpness index (LPC_SI), cumulative probability blur detection (CPBD), maximum contrast with minimum artifacts (MCMA), tone-mapped image quality index (TMQI), natural image quality evaluator (NIQE), and perception-based image quality evaluator (PIQE) to compare image quality using 17 images. First, LPC_SI evaluates the sharpness of the image based on the weakening of the local phase coherence intensity through blur [20]. The CPBD evaluates the sharpness and blur level of the image based on a probabilistic model detecting blur at the edges from the changing contrast values [21]. In addition, MCMA is a metric similar to human perception, evaluating the visual quality enhancement and detail preservation by comparing the original and enhanced images [22]. Further, the TMQI is an evaluation metric for tone-mapped images, including a multiscale structural fidelity measure and a statistical naturalness measure [23]. The TMQI indicates the level of structural distortion and provides an overall quality score for the enhanced image compared to the original. These four metrics indicate better image quality and sharpness as their values increase. The PIQE is a no-reference evaluation method, estimating localized distortions in images and evaluating image quality through the local variance of distorted blocks [24]. Further, PIQE scores range from 0 to 100, with lower values indicating higher image quality. The NIQE provides a perceptual quality score for images based on a model computed from natural images compared to the image itself [25]. Similar to the PIQE, the NIQE scores range from 0 to 100, with lower values indicating higher image quality.

Figure 18 presents the results of the evaluation based on the scores for 17 comparison images and their averages. Table 1 represents the average values of the 17 experimental images in the specific numerical form.

Figure 18. Metric scores [5,12]: (a) local phase coherence-sharpness index (LPC_SI) score, (b) cumulative probability blur detection (CPBD) score, (c) maximum contrast with minimum artifact (MCMA) score, (d) tone-mapped image quality index (TMQI) score, (e) natural image quality evaluator (NIQE) score, and (f) perception-based image quality evaluator (PIQE) score.

Table 1. Comparison with metric scores.

	Reinhard (2012) [5]	L1L0	iCAM06	Kwon et al. [12]	Proposed
LPC_SI (↑)	$0.9121$	$0.9127$	$0.9178$	$0.9343$	${\mathbf{0.9344}}^{\mathbf{ }\mathbf{1}}$
CPBD (↑)	$0.5844$	$0.5846$	$0.5830$	$0.6570$	${\mathbf{0.6600}}^{\mathbf{ }\mathbf{1}}$
MCMA (↑)	$0.6391$	$0.6431$	$0.6236$	$0.6489$	${\mathbf{0.6881}}^{\mathbf{ }\mathbf{1}}$
TMQI (↑)	$0.8911$	$0.8816$	$0.8863$	$0.8830$	${\mathbf{0.9035}}^{\mathbf{ }\mathbf{1}}$
NIQE (↓)	$2.594$	$\mathbf{2.5796}$	$2.6572$	$3.2236$	${2.6838}^{ 4}$
PIQE (↓)	$36.1583$	$35.6512$	$35.0158$	$\mathbf{31.7953}$	${33.4686}^{ 2}$

$x^y$ indicates that $x$ is ranked as the $y$th value in the row; $x$ can refer to either a method or a specific evaluation metric. The bold value represents the best method in the corresponding evaluation metric. The “up arrow” indicates higher scores are preferable, while the “down arrow” signifies lower scores are preferable.

While the rankings of each image may differ based on the evaluation metrics, the proposed method demonstrates excellent results across all evaluation metrics on average. Notably, the CPBD, MCMA, and TMQI metrics display significant performance improvement compared to other synthesis methods. Overall, the proposed method consistently ranks highly for all evaluation metrics without significant variation, unlike other synthesis methods that may exhibit considerable discrepancies in performance evaluation across metrics. Thus, the proposed method is outstanding in image quality performance regarding detail representation, noise reduction, and tone-compression effects compared to the existing synthesis methods.

5. Conclusions

Conventional tone-mapping algorithms can introduce unnecessary noise and halo artifacts during the tone-compression process. This paper proposes a tone-mapping algorithm that combines Retinex theory with CLAHE in a multiscale synthesis approach to resolve these problems and achieve better image quality. The proposed method enhances tone-compression performance and reduces noise and halo artifacts by performing high-σ SSR on images with a wide dynamic range. In addition, multiscale CLAHE secures the visibility and detailed information expression by synthesizing each image with exceptional detail representation and noise reduction. Finally, the algorithm enhances color representation by employing saturation compensation based on the color ratio of the original image to the synthesized image. Compared to the existing tone-mapping algorithms, such as MSR, Reinhard, L1L0, iCAM06, and the method by Kwon et al., the proposed algorithm has excellent tone-compression performance and increases local contrast, resulting in prominent clarity characteristics. Additionally, the color of the original image is well represented by minimizing the color-channel distortion due to the change in the luminance channel. In addition to subjective evaluations, objective image quality metrics confirm the superiority of the proposed method for various aspects of image quality. Thus, the proposed method outperforms existing methods in image sharpness, visibility, and overall quality performance.

Additionally, we extend the application of existing theories by employing them at a multiscale level. CLAHE has primarily been used with a single scale of two parameters. Two theories, Retinex and CLAHE, rely on their own parameters. As such, the degree of image improvement has a trade-off relationship depending on the parameter values. Furthermore, both theories do not have separate optimal parameter values and the values required for each image are different. Due to these problems, conventional methods have limitations in their application and effectiveness. By employing conventional theory at a multiscale level and integrating two theories which are complementary to each other, this paper overcomes the limitations imposed by trade-offs and enhances the resulting image quality in various ways. Ultimately, the proposed method presents a more dynamic approach to using existing theories and combines them to reproduce superior HDR images from single LDR images compared to traditional techniques.