An Endoscope Image Enhancement Algorithm Based on Image Decomposition

Abstract

The visual quality of endoscopic images is a significant factor in early lesion inspection and surgical procedures. However, due to the interference of light sources, hardware, and other configurations, the endoscopic images collected clinically have uneven illumination, blurred details, and contrast. This paper proposed a new endoscopic image enhancement algorithm. The image decomposes into a detail layer and a base layer based on noise suppression. The blood vessel information is stretched by channel in the detail layer, and adaptive brightness correction is performed in the base layer. Finally, Fusion obtained a new endoscopic image. This paper compares the algorithm with six other algorithms in the laboratory dataset. The algorithm is in the leading position in all five objective evaluation metrics, further indicating that the algorithm is ahead of other algorithms in contrast, structural similarity, and peak signal-to-noise ratio. It can effectively highlight the blood vessel information in endoscopic images while avoiding the influence of noise and highlight points. The proposed algorithm can well solve the existing problems of endoscopic images.

1. Introduction

Medical endoscopy is of great significance in early lesion screening and improving the success rate of surgical operations. Whether it is the tracking detection of wireless capsule endoscopy [1] or the high-precision surgical navigation of AR (Augmented Reality) [2], it is closely related to the endoscopic image. The visual quality of endoscopic imaging is often affected by the intricacies of the internal structure of the human body, plus factors such as light source interference [3] and hardware limitations during endoscopic image acquisition, while the cost of access to the underlying image processing side of the hardware is vast [4], so we can improve the results of conventional endoscopic imaging.

Under normal circumstances, uneven illumination and low contrast are the most critical factors affecting the clinical diagnosis of endoscopy [5]. At the same time, further lesion inspection and polyp diagnosis are inseparable from high-quality endoscopic images. To improve image quality, early researchers made a series of improvements based on gamma correction [6] and a single-scale retinex [7] algorithm. Huang et al. [8] proposed weighted adaptive gamma correction (AGCWD), which adaptively modifies the function curve by normalizing the gamma function to achieve the effect of adaptive correction of luminance. Jobson et al. [9] proposed Multi-Scale Retinex (MSRCR) with a color recovery function to solve the phenomenon of color distortion and saturation loss arising in the enhanced images. However, it is difficult to maintain the brightness and color fidelity of endoscopic images alone when these methods are applied to endoscopic images.

Image enhancement algorithms based on multi-exposure fusion are also widely used in image processing [10]. Hayat et al. [11] proposed a multi-exposure fusion technology based on multi-resolution fusion, which effectively solved the problem of image artifacts. Ying et al. [12] proposed an accurate image contrast enhancement algorithm to solve the problem of insufficient contrast in some areas of the image and excessive contrast in some areas, to use illumination estimation technology to design a fusion weight matrix, and to synthesize multi-exposure images through the camera response model.

Histogram equalization (HE) [13] is commonly used for image contrast enhancement due to the ease and straightforwardness of implementation. Still, its application to endoscopic images can lead to noise amplification and over-enhancement problems. To solve the defects of histogram equalization, researchers proposed some algorithms based on histogram equalization improvement. Zuiderveld et al. [14] proposed restricted contrast adaptive histogram equalization (CLAHE) using threshold clipping histogram to prevent over enhancement. Chang et al. [15] proposed quadruple histogram equalization by dividing the image into four sub-images through the mean and variance of the image histogram. However, these methods cannot handle the luminance error and have certain limitations.

More and more algorithms for medical image enhancement have been developed in recent years. Al-Ameen et al. [16] proposed a new algorithm to improve the low contrast of CT images by adjusting the single-scale Retinex and adding a normalized Sigmoid function to improve the contrast of CT images. Palanisamy et al. [17] proposed a framework for enhancing color fundus images by improving luminance using gamma correction and singular value decomposition and local contrast using contrast-limited adaptive histogram equalization (CLAHE), which adequately preserves detail while improving visual perception. Wang et al. [4] proposed an endoscopic image brightness enhancement technique based on the inverse square law of illumination and retinex to solve problems such as overexposure and color errors arising from endoscopic image brightness enhancement. An initial luminance weighting based on the inverse square law of illuminance is designed. A saturation-based model is proposed to finalize the luminance weighting, effectively reducing image degradation caused by bright spots. These algorithms cannot solve the defects of endoscopic images in a multifaceted way.

To ensure the outstanding effect of the underlying blood vessel details without color distortion, this paper proposes a new endoscopic image enhancement framework. Global enhancement with noise suppression, brightness correction by adaptive bilateral gamma function for the base layer separated by weighted least squares based, the separated detail layer is subjected to sub-channel selective adaptive stretching with high highlight suppression to achieve the effect of detail enhancement. The main contributions of this paper are as follows:

1. This paper proposes a novel framework for endoscopic image enhancement to avoid the interference of high brightness and noise in endoscopic imaging by separating the noise layer and high brightness mask.

2. According to the characteristics of endoscopic images, an adaptive brightness correction function based on bilateral gamma is proposed, which enhances the brightness of light and dark areas while preventing excessive enhancement of high-light areas.

3. According to the color characteristics of endoscopic images, this paper proposes a detail layer sub-channel processing method, which uses different image weights for each channel for scaling. Then, the detail layer enhancement factor was designed according to the connection before and after the base layer enhancement. It achieves the effect of detail enhancement while highlighting the color features of endoscopic images.

2. Dataset Introduction

We collected 200 endoscopic images jointly with Hefei Deming Electronics Co., Ltd. (Hefei, China) and the First Affiliated Hospital of Anhui Medical University. Create a dataset named LEI_D, as shown in Figure 1. The LEI_D dataset is used to carry out real minimally invasive surgeries in the relevant departments of the hospital through the CMOS image sensor dedicated to endoscopy and the endoscopy system developed by the company. The video is stored in MP4 format on the endoscopy device side to the mobile hard disk without communication through network encoding to ensure the video’s original quality.

The LEI_D dataset is collected from different departments such as urology, thoracic surgery, oncology, cardiovascular surgery, cardiovascular surgery, otolaryngology, nephrology, obstetrics and gynecology, and hepatobiliary and pancreatic surgery. The collected endoscopic images cover different organ tissues and lesions in the human body, including oral cavity, nostril, bladder, liver, intestine, gallbladder, uterine fibroids, urinary tumors, pituitary adenomas, recurrent maxillary sinus tumors, gastric cancer, rectal cancer, etc. In addition, the dataset also covers some animal tissue images, such as chicken guts, intestines, bronchial tubes, etc.

3. Proposed Algorithm

The framework flow of the algorithm is shown in Figure 2. which is inspired by the different absorption characteristics of human tissues for different spectra and the multipath processing mechanism. First, to prevent the effect of noise on microvascular imaging, the input map was decomposed into structural and noise layers according to a modified total-variation method for endoscopic image design and enhancement of structural layers with noise suppression. Secondly, to improve the image brightness and highlight local vascular information, the structural layer is divided into base and detail layers by the weighted least squares method. The main brightness information exists in the base layer, and the detail information such as blood vessels exists in the detail layer. In terms of brightness, the proposed adaptive bilateral gamma function is used to correct the brightness of the base layer; In terms of details, since the gain of the green and blue channels are more beneficial to highlight the detailed information of blood vessels and the highlighted area will affect the channel stretching effect, the three channels are selectively stretched on the premise of removing the highlight information. Finally, the improved base, detail, and noise layers are fused to obtain a result map with a better visual effect.

3.1. Modified Total-Variation to Extract the Noise Layer

In endoscopic images, there is usually some noise [18] that affects the image’s visual quality, the enhancement of the image will lead to noise amplification, and the removal of noise may cause the loss of information in particularly small blood vessels, so this paper adopts the method of noise suppression. Before the endoscopic image is enhanced, the corresponding noise layer is extracted by applying the improved global noise estimation of the endoscopic image to the total-variation method.

The image edge structure has strong second-order differential properties [19], so the image is sensitive to the noise statistics of the Laplace mask. We use a kernel consisting of two Laplacian masks to participate in the convolution operation (Equation (2)). Conventional noise estimates may contain vascular details since endoscopic images differ from conventional images. We add a noise suppression factor k to the arithmetic mean-based image noise estimation [20] algorithm to control the level of extracted noise. The set global noise parameter ${\theta }^c$ is obtained from Equation (1).

$${\theta }^c=\left(\sqrt{\frac{\pi }{2}}\frac{1}{6\left(W-2\right)\left(H-2\right)}{\displaystyle \sum }_{\left(x,y\right)}\left|\left(I^c\ast N_s\right)\left(x,y\right)\right|\right)\ast \left(1/k\right)$$

(1)

$$N_S=\left(\begin{array}{ccc}1& -2& 1\\ -2& 4& -2\\ 1& -2& 1\end{array}\right)$$

(2)

where $\ast$ represents the convolution operator, W and H represent the width and height of the image, I^c is the input image, and k is 40 in this paper (the value analysis of k is explained in the experimental analysis).

In the total-variation structure texture method [21], the input image consists of a superposition of a structural layer and a noise layer; The model is shown in Equation (3). Combined with TV (total variation) regularization, the structural layer is obtained by minimizing the objective function. This objective function (4) consists of two components. The first is a different term adapted to the texture components. The second is a regularization term based on the total amount of variation used to limit the details of the image.

$$I^c\left(x,y\right)=I_{structure}^c\left(x,y\right)+I_{noise}^c\left(x,y\right),c\in \left\{R,G,B\right\}$$

(3)

$$min{\displaystyle \sum }_{\left(x,y\right)}{\left(I_{structure}^c\left(x,y\right)-I^c\left(x,y\right)\right)}^2+{\lambda }^c\left|\mathit{\nabla }{I}_{structure}^c\left(x,y\right)\right|$$

(4)

where $\mathit{\nabla }$ denotes the ladder segment operator and ${\lambda }^c$ is the parameter ${\theta }^c$ obtained by the set endoscopic global noise estimation.

3.2. Weighted Least Squares Decomposition of Images

Perform the weighted least squares (WLS) [22] method on the structural layer to obtain the base layer, and then subtract the base layer from the structural layer to obtain the detail layer. The extraction of detail layers based on weighted least squares can extract good detail information while maintaining the original graph architecture. Compared to the artifacts that tend to appear with bilateral filtering and the complexity of bootstrap filtering to guide image selection, the weighted least squares method is applied to the endoscopic enhancement to smooth as much as possible in the areas with small gradients and keep as much as possible in the edge parts with strong gradients. After processing, we get a base layer with background brightness information of the subject and an avascular layer with minute detail information.

Decomposition Model:

$$I_{structure}^c\left(x,y\right)=I_{base}^c\left(x,y\right)+I_{detail}^c\left(x,y\right),c\in \left\{R,G,B\right\}$$

(5)

WLS Model:

$$mi{n}_{I_{base}}{\displaystyle \sum }\left({\left(u_p-g_p\right)}^2+\lambda \left(a_{x,p}\left(I_{structure}\right){\left(\frac{\partial I_{base}}{\partial x}\right)}^2+a_{y,p}{\left(\frac{\partial I_{base}}{\partial I_{base}}\right)}^2\right)\right)$$

(6)

where p represents the position of the pixel point, and a_x and a_y control the degree of smoothing at different positions. The first term represents that the input and output images are as similar as possible. The second term is a common term that smoothes the output image by minimizing the partial derivative. λ is used as a regularization parameter to balance the two weights.

3.3. Adaptive Bilateral Gamma Correction Brightness

After observing a series of endoscopic images, it is found that there is a problem of uneven illumination in the endoscopic images, especially the dark area information cannot be well highlighted. To ensure that the brightness area is not over-enhanced and improve the brightness of the dark area without changing the color. In this paper, we use the bilateral gamma function to correct the luminance channel of the base layer and propose adaptive weighting of the image pixel positions. Figure 3 shows the flow chart of luminance correction.

Bilateral gamma functions:

$$V_1\left(l\right)=l_{max}{\left(\frac{l}{l_{max}}\right)}^{1/\gamma }$$

(7)

$$V_2\left(l\right)=l_{max}-l_{max}{\left(1-\frac{l}{l_{max}}\right)}^{1/\gamma }$$

(8)

$$V^{\prime }\left(l\right)=\alpha V_1\left(l\right)+\left(1-\alpha \right)V_2\left(l\right)$$

(9)

where V is the base layer luminance channel in HSV color space, $V^{\prime }$ is the corrected luminance channel. The internationally recommended $\gamma$ value is 2.5.

As shown in Figure 4, Above y = x is the graph of $V_1$ with $\gamma$, and below y = x is the graph of $V_2$ with $\gamma$. As the value of $\gamma$ increases, the function curve becomes steeper and steeper, indicating that the degree of change becomes more and more drastic. The images of V channels are corrected separately by bilateral gamma functions ($V_1 and V_2$) to get two images, V1 and V2. V1 is a brighter image by simple gamma correction and V2 is a darker image by negative gamma correction. So, when two images are fused, the brighter the region should have less weight, and the darker the region should have more weight. Combined with the curve change trend of the sigmoid function, this paper proposes adaptive weights α1.

$$\alpha 1=1-\frac{1}{\left(1+e^{\left(7\ast \left(0.5-V\right)\right)}\right)}$$

(10)

The obtained functions, weights, and stretching effects are plotted in Figure 5. It can be seen that the brightness of the image deviates considerably, and the brightness is enhanced, but the enhancement is much more than the expected guesses. Some areas of the image appear too bright and too dark, which can blur the frame of the subject of the base layer. Therefore, functions of the sigmoid type are not suitable for stretching here, and α1 will be improved. Using the arctan curve transformation trend in the function, this paper proposes adaptive weights α2.

$$\alpha 2=\phi -\left(\frac{1}{\pi }\right)\arctan \left(V\right)$$

(11)

By comparing the base layer obtained by the α2 function with the original base layer, it can be seen that this rendering has achieved a satisfactory result, and the brightness of the light and dark areas has been improved, while the brightness of the bright areas has not undergone excessive enhancement.

To reduce the algorithm’s complexity while optimizing its accuracy, the value of γ is obtained more precisely by the following steps.

Step1: m = 15 endoscopic images of different sites were selected (5 images each of bright, medium, and dark brightness) with γ ϵ (2.0–3.0).

Step2: Define a new comparison metric, which is a combination of two performance metrics (MSE (Mean Squared Error) [23] and SSIM (structural similarity index) [24]).

$$F=\sqrt{\frac{{\left(MSE/10\right)}^2+{\left(SSIM\right)}^2}{2}}$$

(12)

$$MSE=\frac{1}{M\times N}{\displaystyle \sum }_{i=1}^M{\displaystyle \sum }_{j=1}^N{\left(I_{base}^{\prime }-I_{base}\right)}^2$$

(13)

Step3: The images are subjected to adaptive bilateral gamma correction according to the above method, and the indicator value F of each $\gamma$ value is recorded when $\gamma$ varies at a spacing of 0.1, and the average indicator $F_{ave}$ corresponding to each $\gamma$ value is calculated (i corresponds to the image serial number).

$$F_{ave}={\displaystyle \sum }_{i=1}^{m}F\left(i\right)/m$$

(14)

Step4: The optimal parameter of γ is selected after comparison.

The optimal parameter of γ is 2.2 when applied to the endoscopic image dataset, obtained by computational comparison.

It can be obtained from the above design process of this module. The adaptive bilateral gamma correction function we designed performs a reasonable scaling transformation of each pixel value of the base layer image by assigning corresponding weights to each pixel point for bilateral fusion. This method avoids the phenomenon of over-enhancement and over-compression in the local image area during the brightness correction process so that the problem of uneven brightness in endoscopic images is effectively solved.

3.4. Highlight Detail Layer Information

To prevent the high bright spot area in the endoscopic image from contrast enhancement in the detail layer, we decided to bring up the high bright spot before the detail layer processing and restore the high bright spot area after the processing is completed. Before making a highlight mask, the original image is pre-processed for enhancement (16). The purpose of enhancement is to make the reflective areas more visible and reduce the associated interference factors. This paper adopts the theory [25] that the reflective pixel’s brightness Y (luminance) is greater than its color brightness y (chromatic luminance). Convert the image from RGB to CIE-XYZ space to get $Y$, and then find $y$ according to the formula. The area where $Y$ is greater than $y$ is extracted as the highlighted area.

$$y=\frac{Y}{X+Y+Z}$$

(15)

Pre-processing enhancement model.

$$\left(\begin{array}{c}{R}^{\prime }\\ G^{\prime }\\ B^{\prime }\end{array}\right)=\frac{\min \left(R,G,B\right)}{\max \left(R,G,B\right)}\left(\begin{array}{c}R\\ G\\ B\end{array}\right)$$

(16)

As shown in Figure 6, the extracted mask is the high brightness areas of the image. If the high brightness mask does not separate these areas, they will be overly enlarged in the detail layer, thus blurring the edges and thus affecting the visual quality of the image.

For the endoscopic images, separation of the three-channel images shows that the R-channel has minor distinct vascular features and the G and B-channel planes have visible margins and lesion borders. Blue light is most suitable for enhancing superficial mucosal structures and detecting minor mucosal changes. Greenlight is relatively more suitable for enhancing thick blood vessels in the middle layer of the mucosa [26]. Therefore, the blue component and the green component are more advantageous for extracting endoscopic image information. To ensure the consistency of the image structure while highlighting the vascular details, a stretch factor $w^c$ is set based on the connection between the enhanced base layer and the foundation layer. R channel remains unchanged, and the following stretching model enhances G and B channels.

$$I_{detail}^R{}^{\prime }\left(x,y\right)=I_{detail}^R\left(x,y\right)$$

(17)

$$I_{detail}^c{}^{\prime }\left(x,y\right)=I_{detail}^c\left(x,y\right)+w^c{I}_{detail}^c\left(x,y\right)\ast Z{\left(x,y\right)}^c,c\in \left\{g,b\right\}$$

(18)

$$w^c=1-{\left(norm\left(std\left(I_{base}^{\prime }{}^c\right)\right)-norm\left(std\left(I_{base}{}^c\right)\right)\right)}^{1/2},c\in \left\{g,b\right\}$$

(19)

where $norm$ is the normalization and $std$ is the variance of the image, $Z\left(x,y\right)$ is a weighted guided filter based on the improved canny operator.

Conventional filtering [27] in processing edge information will perform excessive smoothing operations for endoscopic images, i.e., the image’s texture and vascular detail information is lost. This paper proposes an improved weighted guided filter based on the canny operator. The main operation uses the edge (CWGIF) weight calculated by the canny operator to replace the local variance calculated by the window in the original weighted guided filter (WGIF).

Original edge weighting factor calculation formula:

$${\mathsf{\Gamma }}_G\left(p^{\prime }\right)=\frac{1}{N}{\displaystyle \sum }_{p=1}^N\frac{{\sigma }_{G,1}^2\left(p^{\prime }\right)+\epsilon }{{\sigma }_{G,1}^2\left(p\right)+\epsilon }$$

(20)

where ${\sigma }_{G,1}^2\left(p\right)$ is the variance of a 3 × 3 window with $p$ as the center of radius 1, $\epsilon$ is a constant taking the value of ${\left(0.001\mathrm{L}\right)}^2$.

Canny operator improves the calculation of edge weight calculation factor:

$${\mathsf{\Gamma }}_G\left(p\right)=\frac{1}{N}{\displaystyle \sum }_{p=1}^N\frac{C\left(p^{\prime }\right)+\epsilon }{C\left(p\right)+\epsilon }$$

(21)

where C(p) is the value of the canny operator detection of pixel p.

The weighted bootstrap filtering model based on the canny operator is as follows.

$$Z\left(i\right)=a_{k}G\left(i\right)+b_k$$

(22)

where $a_k$ and $b_k$ are the parameters to be solved, and $G\left(i\right)$ is the bootstrap image (corresponding to the G and B single-channel maps). According to the edge weights (Equation (22)), the cost function is as follows.

$$E\left(a_k,b_k\right)={\displaystyle \sum }_{iϵW_k}\left\{{(a_k{G}_i+b_k-p_i)}^2+\frac{\lambda }{{\Gamma }_G\left(i\right)}{a}_k{}^2\right\}$$

(23)

The parameters $a_k$ and $b_k$ can be calculated using the least-squares method on the Formula (24).

$$a_k=\frac{\frac{1}{\left|w\right|}{{\displaystyle \sum }}_{iϵw_k}{p}_i{G}_i-u_k\overline{p_k}}{{\sigma }_k^2+\frac{\lambda }{{\Gamma }_G\left(i\right)}}$$

(24)

$$b_{k=}\overline{p_k}-a_k{u}_k$$

(25)

where $u_k$ and ${\sigma }_k^2$ are the mean and variance are corresponding to the single-channel bootstrap image G within window $W_k$. $\left|w\right|$ is the total number of pixels within the window $W_k$, $\overline{p_k}$ is the grayscale mean of the input single-channel image $p$ within the window $W_k$.

As shown in Figure 7, both WGIF and CWFGIF smooth the gradient in the normal area (Standard tissue areas of the human body in the endoscope), but the blue line is closer to the red line in the areas with large gradients (Vascular and arterial texture areas in the endoscope). These indicate that CWGIF has better retention in this region.

4. Experimental Analysis

4.1. Experimental Environment

The experiment’s environment is as follows: the CPU is an 11th Gen Intel (R) Core (TM) i5-11400F @ 2.60 GHz 2.59 GHz, Windows 10 is the operating system, and Matlab R2019b is the test algorithm platform.

4.2. Parameter Setting

4.2.1. Noise Rejection Factor k

When a global noise estimation technique is applied to endoscopic images, vascular detail information can be wrongly treated as noise. Add a noise rejection factor k to maximize noise separation while protecting vascular detail information. Initially, we set k to ten. Based on the canny operator [28] and a combined examination of edge retention (EPI) [29] and peak signal-to-noise ratio (PSNR) [23]. Figure 8a shows the structure layer canny operator detection plots for image1 and image3 at k at 40 and 50, and Figure 8b represents the PSNR values of the structure layer and the EPI values of the structure layer relative to the original image (in the case of image1). As shown in Figure 8, the EPI and PSNR of the resulting structural layer and the original map gradually increase as k increases. Some noise and light patches are identified as pseudo-edges when k = 50. Multiple images were combined and analyzed, and the k value for the endoscopic images was set at 40.

4.2.2. Setting of Parameter φ in Brightness Correction

As shown in Figure 9, as the value of $\phi$ gradually increases, the image’s average brightness grows larger and larger, and the image’s histogram concentration value grows closer and closer to the high pixel value region. Setting the brightness enhancement to 0.6 makes it easier for human eyes to notice and judge.

4.3. Subjective and Objective Analysis

On a laboratory-built dataset (LEI D), we compare our algorithm to many existing image enhancement algorithms in this section. Algorithms for comparison include AGCWD, MSRCR, Al-Ameen et al., Palanisamy et al., Ying et al., and Wang et al. The increased effect images were evaluated objectively using five evaluation metrics: PCQI(patch-based contrast quality index) [30], SSIM(structural similarity index) [24], PSNR(Peak Signal-to-Noise Ratio) [23], C_II(contrast improvement index) [31], and Tenengrad gradient [32].

4.3.1. Subjective Analysis

Endoscopic images are mainly used by physicians to analyze and judge the images of blood vessels and organ tissues collected from patients to identify abnormal areas. As a result, the enhanced image should keep the image’s brightness, color, and naturalness while emphasizing the details of lesions and blood vessels to meet the typical observation range of the human eye. Consider the following photographs of different human tissues as examples.

AGCWD over-enhanced localized areas when enhancing endoscopic images. The high bright spot area in Figure 10a is over-enhanced, and the vascular features around the bright spot are nearly unnoticeable. These phenomena are better illustrated in Figure 11a.

MSRCR captures information at several scales before completing color recovery, which frequently results in color distortion in the recovered endoscopic images, as shown in Figure 10a,c. It is possible that the extraction scale is not exact enough, or that the color recovery technique produces bias, and that when used to endoscopic pictures, the algorithm produces judgment errors.

Al-Ameen et al.’s algorithm would have greatly increased contrast in some regions, but the image as a whole is excessively dark, impairing human eye visual observation.

Palanisamy et al.’s algorithm delivers good contrast results, which are considerably more reasonable than the prior algorithms, and accomplishes the basic goal of endoscopic image enhancement. On the negative side, magnifying Figure 10c reveals a tendency for the blood vessels in the endoscopic image to darken, affecting the physician’s assessment of the lesion.

Ying et al.’s algorithm provides a significant increase in brightness, and the enhanced image is clearly visible; however, there appears to be a decrease in contrast as the brightness is increased, and part of the subject architecture is blurred, resulting in the loss of some information in the enhanced endoscopic image. This defect is highlighted more clearly in Figure 11b.

Wang et al.’s algorithm improves brightness and contrast more effectively; nevertheless, the brightness of the light and dark areas is overly boosted in some photos, obscuring the information in the dark areas. As seen in the lower-left corner of Figure 10e.

We zoomed in on a localized section of the individual technique effect figure in the preceding figure to observe. The high brightness area of the map obtained by AGCWD is unduly magnified, as seen in the zoomed-in view of the local area in image 1, which hampers normal observation. The contrast intensity of the plot obtained by Palanisamy et al. was excessively high, making the trend of the blood vessel color change from red to black more evident. Wang et al.’s algorithm is brightened, but the accompanying detail information is not well synchronized, and some tiny blood vessel details are blurred.

Our algorithm improves the brightness, contrast, and naturalness of blood vessels in general. The image’s contrast is improved in the normal brightness area, while the intricacies of blood vessels in the dark area are highlighted, avoiding the artifacts, over-enhancing, and color distortion that can occur with traditional image enhancement.

4.3.2. Objective Analysis

This research uses five indexes to conduct an objective examination of the good and bad images.

PCQI index [30] (patch-based contrast quality index) was developed as an adaptable representation based on local block structure to forecast contrast variation. Each block calculates the average intensity, signal strength, and signal structure. Images with higher PCQI values have better contrast.

SSIM index [24] (structural similarity index) compares three sample and outcome variables (luminance, contrast, and structure) to determine how similar the improved image and the original image are.

$$SSIM\left(x,y\right)={\left[l\left(x,y\right)\right]}^{\alpha }\times {\left[c\left(x,y\right)\right]}^{\beta }\times {\left[s\left(x,y\right)\right]}^{\gamma }$$

(26)

where $l\left(x,y\right)$, $c\left(x,y\right)$, and $s\left(x,y\right)$ represent the image’s brightness, contrast, and structure, respectively.

The peak signal-to-noise ratio (PSNR [23]) indicator is used to assess noise performance. When reconstructing enhanced images, a greater PSNR suggests that the rebuilt enhanced images are of higher quality.

$$MSE=\frac{1}{mn}{\displaystyle \sum }_{i=0}^{m-1}{\displaystyle \sum }_{j=0}^{n-1}{\left[I\left(i,j\right)-K\left(i,j\right)\right]}^2$$

(27)

$$PSNR=10·lo{g}_{10}\left(\frac{MAX{I}^2}{MSE}\right)$$

(28)

where $I$ and $K$ represent the improved and original pictures, $MAXI$ represents the image’s maximum pixel value, and $MSE$ represents the image’s mean square error.

$C_{II}$ [31] is a contrast evaluation index for medical pictures that is calculated by dividing the average value of local contrast of images before and after processing by the ratio, with a larger ratio value indicating more contrast.

$$C_{II}=\frac{C_{processed}}{C_{original}}$$

(29)

$$C_{local}=\frac{max-min}{max+min}$$

(30)

where $max$ and $min$ are the maximum and minimum pixel intensity values in a window. $C_{original}$ and $C_{processed}$ are the average of the local contrast of the image before and after processing, respectively. The window size of the conventional image is set to 3 × 3 pixels. Here, considering the larger proportion of details such as blood vessels in endoscopic images, and referring to the window size set by other medical image calculations, the window size was set to 50 × 50 pixels for a more accurate evaluation of the endoscopic pictures [33].

The Tenengrad [26] gradient is utilized as an image sharpness evaluation index, and the Sobel operator is used to extract the gradient values of the picture’s horizontal and vertical directions, respectively. The image becomes sharper and more suited for human eye observation as the Tenengrad gradient value increases.

As shown in

Table 1. Comparison chart of five evaluation indexes.

Methods	Image	PCQI	SSIM	PSNR	C_II	Tenengrad
AGCWD	Im1	0.98	0.87	14.06	1.18	3.37
	Im2	0.88	0.96	18.94	1.03	1.06
	Im3	0.94	0.94	18.49	1.07	2.89
	Ave	0.91	0.93	17.72	1.08	2.44
MSRCR	Im1	0.63	0.72	7.83	0.71	2.37
	Im2	1.01	0.88	17.14	2.16	1.70
	Im3	0.68	0.83	11.06	0.91	2.29
	Ave	0.79	0.81	12.20	1.28	2.12
Al-Ameen	Im1	0.75	0.71	14.01	1.82	2.43
	Im2	0.81	0.75	12.73	2.06	1.20
	Im3	0.78	0.75	12.97	1.58	2.45
	Ave	0.80	0.73	13.34	1.74	2.03
Palanisamy	Im1	1.10	0.89	23.10	1.26	4.15
	Im2	1.10	0.90	20.93	1.56	1.70
	Im3	0.61	0.78	12.48	0.75	2.70
	Ave	0.99	0.88	21.97	1.27	2.55
Ying	Im1	0.90	0.93	17.23	0.83	2.87
	Im2	0.89	0.95	18.42	0.83	1.06
	Im3	0.94	0.97	22.03	0.90	2.65
	Ave	0.91	0.95	18.71	0.86	2.19
Wang	Im1	0.75	0.89	14.81	0.69	2.60
	Im2	0.75	0.94	18.10	0.72	0.95
	Im3	0.71	0.90	16.14	0.74	2.15
	Ave	0.74	0.91	16.78	0.72	1.93
Ours	Im1	1.01	0.96	27.65	1.43	3.02
	Im2	0.94	0.96	20.90	1.58	1.24
	Im3	1.01	0.96	22.84	1.46	3.11
	Ave	0.98	0.96	23.23	1.53	2.49

, we present both the metric results for the three sets of a, b, and c images in Figure 10 and the mean values of the 50 image metrics in the dataset. In the table, Im1, Im2, and Im3 correspond to the three groups of a, b, and c images in Figure 10, and “Ave” represents the average of the 50 image metrics.

In terms of contrast. The algorithm in this paper is ahead of the mean value of the algorithm proposed by Palanisamy by 0.01 (0.98 and 0.99) in PCQI. It is second only to the algorithm proposed by Al-Ameen in terms of C_II index. Taken combined, the algorithm in this paper improves contrast significantly.

The SSIM value of this algorithm is the best in terms of similarity, ahead of the other six evaluated algorithms, demonstrating that the image enhancement of this algorithm does not lose the picture’s significant topic information, which is critical for medical image processing.

In terms of picture quality, this algorithm’s PSNR values are higher than those of other algorithms, implying that the reconstructed images are of higher quality. In terms of Tenengrad gradient index, the approach given in this study is pretty near to both the algorithm proposed by Palanisamy et al. and the algorithm value of adaptive gamma correction, showing that the clarity aspect is also ensured.

When image contrast, image quality, and information preservation are all considered, as can be seen from Figure 12, our algorithm is excellent for processing endoscopic images.

5. Discussion

This work demonstrates a study on enhancing the visibility of endoscopic images to achieve accuracy in machine-assisted physician diagnosis [34]. During endoscopic imaging, organ substructures or surroundings are hidden in the obtained endoscopic images due to the complexity of the internal structures of the human body and the limitations of hardware devices for imaging. Based on the imaging characteristics of endoscopy, this work introduces the image decomposition architecture to address these defects.

5.1. Effectiveness

The architecture of image decomposition can obtain multiple image layers with different features. Further processing corresponding to each image layer can effectively highlight the contrast of blood vessels and tissues in the endoscope while maintaining the architecture of the image subject. From the experimental analysis comparing the enhancement effect plots of various algorithms, we can see that most of the enhancement algorithms cannot be well applied to endoscopic images. The reason is that they do not consider the imaging characteristics of endoscopic images and the importance of the detailed information of tiny blood vessels, and all perform image enhancement from the image as a whole. They [4,5,6,7,8,9,10,11,12,13,14] changed the three-channel pixel values, resulting in the blurring or loss of the underlying detail information of endoscopic images. The image decomposition model uses various forms of filtering to decompose the figure into structural, noise, and detail layers for separate processing, which effectively solves the problems of biased noise estimation, uneven brightness, and inconspicuous detail information in endoscopic images. The algorithm in this paper is not affected by external factors during the experiment. It can be applied to images of any tissue inside a human or animal, and of course, the framework idea of the algorithm can be used in any image enhancement.

5.2. Limitations

Although the algorithm outperforms other existing algorithms in solving existing problems for endoscopic images, it still has some potential limitations or open issues. First, the algorithm analysis is performed on an existing set of images. However, the images that physicians are exposed to in some extreme cases may be more complex, and individual extremely specific images could be collected for experiments in subsequent studies to ensure that the generalizability of the algorithm is maximized. Secondly, compared with other algorithms, it may be designed to set some parameters in different algorithms or set parameters in the evaluation index. We need to study these parameters further to obtain more accurate results. Finally, the algorithm can currently run well on the Matlab R2019b. In the next step, we consider multi-threaded programming and hardware device enhancement to achieve synchronization and setting in the physician’s surgical system to highlight the significance of practicality further.

6. Conclusions

This paper provides an image decomposition-based endoscopic image enhancement algorithm that effectively avoids the interference of high bright spots and noise in endoscopic pictures. Adaptive enhancements were applied to the brightness of the base layer pictures, as well as stretching of the detail layer vascular lesions and other information in sub-channels suited for endoscopic image characteristics. Traditional image enhancement difficulties such as color distortion, hazy disappearance of vascular features, and excessive local brightness augmentation are efficiently solved by this algorithm. The algorithm produces good visual results and assessment indications through subjective and objective examination. It was demonstrated that this algorithm outperforms numerous other conventional algorithms for processing endoscopic images. Due to the limitations of traditional algorithms in image processing, in future work, we will consider applying the image decomposition ideas used in this algorithm to neural networks to design a suitable image quality loss to optimize the network model.