Dermoscopy Images Enhancement via Multi-Scale Morphological Operations

: Skin dermoscopy images frequently lack contrast caused by varying light conditions. Indeed, often low contrast is seen in dermoscopy images of melanoma, causing the lesion to blend in with the surrounding skin. In addition, the low contrast prevents certain details from being seen in the image. Therefore, it is necessary to design an approach that can enhance the contrast and details of dermoscopic images. In this work, we propose a multi-scale morphological approach to reduce the impacts of lack of contrast and to enhance the quality of the images. By top-hat reconstruction, the local bright and dark features are extracted from the image. The local bright features are added and the dark features are subtracted from the image. In this way, images with higher contrast and detail are obtained. The proposed approach was applied to a database of 236 color images of benign and malignant melanocytic lesions. The results show that the multi-scale morphological approach by reconstruction is a competitive algorithm since it achieved a very satisfactory level of contrast enhancement and detail enhancement in dermoscopy images.


Introduction
Medical images are the visual representations of the interior of a body. These visual representations have facilitated some health care tasks such as diagnosing diseases. Despite technological advances in recent years, image acquisition, storage, or transmission still suffer from various types of degradation [1]. These factors can cause inefficient or inaccurate diagnoses, thus compromising the healing of patients.
Many techniques have been proposed to improve the contrast of medical images [2]. The traditional histogram equalization (HE) [3], one of the most popular techniques, was the first attempt to automatically improve contrast. HE distributes the gray levels within the image (each gray level has an equal chance to occur) to enhance contrast and brightness. Studies have shown that HE introduces saturation and over-enhancement in the images [4,5]. Several improved techniques have been proposed to maintain average image brightness, reducing saturation effects, thus avoiding unnatural image enhancement. Some of these techniques are: brightness preserving bi-histogram equalization (BBHE) [6], dual sub-image histogram equalization (DSIHE) [7], minimum mean brightness error bihistogram equalization (MMBEBHE) [8], and quadri-histogram equalization with limited contrast (QHELC) [9].
With the emergence of mathematical morphology (MM) based approaches in recent decades, new techniques have been developed for contrast enhancement [10][11][12][13]. Due to its ability to extract dark and light features from images using structuring elements of different shapes and sizes [14], the top-hat transformation has received a lot of attention. In [15], the top-hat transformation was used to correct the illumination of images with melanocytic lesions as a preprocessing for a subsequent feature extraction study. In [16], a method for segmentation of retinal blood vessels is presented. Vessel enhancement is performed using the contrast enhancement technique based on the top-hat transform.
Various authors have proposed to use a multi-scale approach, called multi-scale top-hat transformation (MTH). An advantage of MTH is that it allows to process the image content from the most global to the most detailed level. Several works propose to improve different types of medical images by integrating MTH in a morphological based image enhancement approach [10,[17][18][19][20][21][22][23][24]. Currently, in the field of computer vision, MTH-based algorithms are used as a preliminary step for other applications based on artificial intelligence. For example, in [25], a deep learning approach using convolutional neural networks was proposed to detect vessel regions in angiography images. In this work, the multi-scale top-hat transform for contrast enhancement (MSTH) algorithm [10] was used to preprocess the images by enhancing their contrast. In [26], a method for edge detection in images based on top-hat operators with multidirectional and multiscale structuring elements was proposed.
A multimodal medical image fusion scheme based on multiscale top-hat transform combined with morphological contrast operators is presented in [27].
In [28], an automatic coronary artery segmentation approach was proposed. In this work, in the preprocessing stage, the input image was processed with the MSTH algorithm for better segmentation of coronary arteries. In [29], MSTH was used as the first step of an algorithm for detecting bright lesions in retinal images. In [30] proposed Sine-Net, an automated tool based on a fully convolutional deep learning architecture for the segmentation of blood vessels in retinal images. The architecture obtained better segmentation results on three databases by using the combination of the MSTH and contrast-limited adaptive histogram equalization (CLAHE) [31] algorithms in the preprocessing.
In this work an improvement of MTH is proposed. The underlying idea is to replace the opening and the closing operation with morphological filters by reconstruction. This replacement is done because the morphological operators by reconstruction are very attractive by avoiding damaging the image contour, the edge and many other important information of the medical image.
For such purpose MTH integrates the concept of geodesic reconstruction [32]. By combining the advantages of morphological reconstruction with MTH ability to extract dark and bright characteristics, the resulting strategy is a multi-scale morphological approach capable of enhancing medical images. Experiments show that the resulting skin dermoscopy images have less distortion, greater detail accuracy, and better contrast than different image enhancement approaches.
We can summarize the contributions in this work as follows: (a) Propose a new MTH based strategy which incorporates the geodesic reconstruction concept in combination with a mathematical morphological approach; Design a novel contrast enhancement algorithm based on the proposed MTH approach.
The rest of the article is organized as follows. In Section 2, the basic concepts are introduced. Then, in Section 3, the proposed approach is described. The experiments and discussions of the results achieved are presented in Section 4. Finally, Section 5 presents the conclusions of the work.

Mathematical Morphology
In mathematical morphology (MM) the aim is to analyze and extract unknown structures contained in an image. For such purpose, it uses a structuring element of known shape and size, and the erosion and dilation operators [14]. By providing a wide range of filters represented by the combination of these two basic operators, MM offers efficient tools and represents a relatively simple and powerful tool in terms of image analysis.

Dilation and Erosion
Given an image I, the morphological dilation δ H (I) and erosion ε H (I) of I using the structuring element H at the pixel x with respect to the structuring element H of domain D H , are defined as follows [14]: (1)

Opening and Closing
Opening γ(I, mH) operator is the sequential combination of erosion ε mH (I) with mH (structuring element H of size m) followed by dilation δ mH (I). Closing, on the other hand, φ(I, mH) is the sequential combination of dilation δ mH (I) with mH followed by erosion ε mH (I). Both operators are defined as [14]: where m is the size of the structuring element and mH the reflection of mH. In the case of symmetrical structuring element, mH = mH.
Viewing an image as a two-dimensional surface in a three-dimensional space, applying opening (closing) has the consequence of removing peaks (or filling valleys) smaller than the structuring element.

Classical Top-Hat Transform
By taking the difference between the original image and its opening, some different peaks can be extracted. In a dual way, we can extract different valleys by making the difference between a closed image and the original one. Top-hat transform represents the mathematical formalism of this idea. During top-hat transform by opening (WTH) [14] is the rest of the original image I and its opening γ(I, mH), top-hat transform by closing (BTH) [14] is the subtraction between the morphological closing φ(I, mH) and the original image I, defined as follows: where m is the size of the structuring element.

Geodesic Transformation and Reconstruction
In geodesic transformations, two equally sized input images are used, denoted as marker and mask. The first image (marker) is modified by a morphological transformation and restricted below (geodesic dilation) or above (geodesic erosion) the second image (mask) [14].
Let J and I be the marker and mask images, respectively, with the same domain (D J = D I ). Geodesic dilation δ I (J) and erosion ε I (J) can be defined as [14]: where ∧ is the minimum between the pixels of J(x) and I(x) and ∨ is the maximum between the pixels of J(x) and I(x). If we perform k times the geodesic dilation or erosion of J with respect to I, we have to δ (k) . In practice, we can define geodesic reconstruction ρ I (J) and dual geodesic reconstruction ρ * I (J) as follows: Similar to the standard opening and closing, opening γ (m) ρ and closing φ (m) ρ by reconstruction of an image I can be defined as follows [14]: where I is the mask image, ε mH (I) and δ mH (I) are the markers image and m is the size of the structuring element.
In the full experiment, the structuring element has the shape of a disk, and m indicates the size of the radius.

Top-Hat by Reconstruction
The use of morphological operators by reconstruction has shown that, contrary to the standard ones, they remove details without modifying the structure of remaining objects. Another significant advantage is that geodesic reconstructions use an elementary isotropic structuring element and it is not necessary to specify sizes like in standard morphological operators. Analogously to the standard top-hat transform, it is possible to preserve or remove structures through geodesic reconstruction (dual geodesic reconstruction) that will have the role of opening (closing).
Although structures removed in the image I from the opening by reconstruction can be recovered with the white top-hat transform by reconstruction (RWTH), similarly, we can recover structures removed from the closing with the dark top-hat transformation by reconstruction (RBTH) as follows [14]:

Proposed Algorithm
The proposed algorithm, called Multi-scale Geodesic Reconstruction based Top-Hat transform (MGRTH), is presented in this section. Additionally, all operations are presented in detail, step by step.
Let I be the image, H be the structuring element and n be the number of iterations. The proposed algorithm is divided into five stages.
First stage: the bright structures at level i are extracted by RWTH m as follows: where WTH(ImH) is the mask obtained by Equation (3), RWTH (m) (I) is the marker obtained by Equation (7). Each RWTH m represents the m-level of bright structures of the original image I controlled by m = {1, 2, 3, . . . , n} which is the size of the structuring element, and the dark structures at level m are extracted by RBTH m as follows: where BTH(I, mH) is the mask obtained by Equation (3), RBTH (m) (I) is the marker obtained by Equation (7). Each RBTH m represents the m-level of dark structures of the original I controlled by m = {1, 2, 3, . . . , n} the size of the structuring element.
Second stage: the light residues SW m are extracted from the dark structures at levels m and m − 1 and the dark residues SB m are extracted from the light structures at levels m and m − 1 as follows: Third stage: The maximum bright scaled details are computed from the bright structures extracted at the first stage by RWTH m , and the maximum dark scaled details are computed from the dark structures extracted at the first stage by RBTH m as follows: Fourth stage: The maximum light residues are computed from the light residues extracted at the second stage by SW m , and the maximum dark residues are computed from the dark residues extracted at the second stage by SB m as follows [13]: Final stage: the enhanced image I E is performed per pixel as follows: where ω is a factor used to adjust the level of brightness or darkness added to the image.

Results and Discussions
This section describes the experiments conducted to quantify the relative performance of the proposed algorithm. The database used in these experiments contains 236 color images of benign and malignant melanocytic lesions and used in [33]. For the tests on the RGB images, first the RGB images were converted to HSV images. Then, the algorithms and evaluations were applied to the V-channel of the images. Finally, the HSV images were converted to RGB images.
All algorithms were implemented using the ImageJ [34] library, for algorithms based on MM an extra library called MorphoLibJ [35] was used.
The results were evaluated with the metrics: • Entropy (E) [13,21,36]: E is used to measure the details in the image. E is defined as, where I is melanoma image, k is intensity of the pixel in the image, P(k) is probability of occurrence of the k-value in the image. If b is number of bits of the image then L is equal to 2 b (b = 8 for grayscale images). An image is considered to have good detail when its entropy value is high; • Peak Signal-to-Noise Ratio (PSNR) [10,21,37]: PSNR measures how much distortion is added to the image in the contrast enhancement process. PSNR is defined as, where Mean Squared Error (MSE) is defined as, After the enhancement process, an image is considered to have low distortion if it has a high PSNR value; • Relative Enhancement in Contrast (REC) [36,38]: REC measures the contrast of the enhanced melanoma image. REC is defined as, where I is the melanoma image, I E is the melanoma image with contrast enhancement, C is the image contrast and C is calculated as follows, where M × N are the dimensions of the melanoma image and (x, y) are the spatial coordinates and µ is calculated as follows, After image processing, if the REC value is higher than 1, the enhanced image is considered to have enhanced contrast.
To test the performance of the proposed method, we have considered two different experiments: • In the first part (Section 4.1), we performed parameter tuning to find good parameter values ω and n of the proposed algorithm. For this purpose, a comparison of the results obtained with respect to the number of iterations and the contrast adjustment weight was performed. The objective of this experiment was to observe the performance of the proposal with respect to the E, PSNR, and REC metrics; • Then, in the second part (Section 4.2) the proposed algorithm was compared with algorithms based on the multiscale top-hat transform and algorithms based on histogram equalization.
In both experiments, apart from the numerical results, a visual assessment of the dermatologist is presented.

Parameters Tuning
In this subsection, the parameter settings used in the MGRTH algorithm in relation to the number of iterations n and the contrast adjustment weight ω are described. For this purpose, the Shannon entropy Equation (17) and the PSNR Equation (18) are used. In addition, the Equations (18) and (20) are used to visualize the the relation between the PSNR and REC metric. This is done because as contrast increases more noise tends to be added. Table 1 shows the parameters of the MGRTH algorithm to be adjusted. In Figure 2 it can be seen that: MGRTH with ω = 0.25 has longer increasing entropy than using larger weights and from n = 7, it already equals or exceeds average values obtained than using larger weights.
In Figure 3 it can be seen that also with ω = 0.25, there is a higher value of PSNR and in Figure 4 a higher ratio between REC and PSNR can be observed in all iterations evaluated.     Figure 5 shows that as the value of the ω increases, the brightness of the image also increases. This causes bright or dark artifacts to appear. Due to this and the results obtained in the previous subsection, the value ω = 0.25 is chosen for the next experiment. In Figure 6, we can see that as the iterations increase, distortions are introduced to the image. Compared to the original image (Figure 6a), the images in Figure 6c,d present a higher sharpness of the lesions and without much brightness. An image with too much brightness may cause dermoscopic assignments or translations of malignancy that do not correspond to the lesion. This can be considered as an artifact of the modified image. The sharpness seen in the images in Figure 6c,d is also apparent in healthy peripheral skin.

Visual Assessment by the Dermatologist
In the image in Figure 6b, the resulting image is visually similar to the original image. In the images of Figure 6e-g the brightness observed in the lesion is pronounced, and could induce errors of assessment by the dermatologist.

Comparison of the Proposed Algorithm with State-of-the-Art Algorithms
MGRTH was compared with histogram-based algorithms: HE, BBHE, MMBEBHE, and QHELC. These are good at improving the contrast and average brightness of medical images. It was also compared with competitive algorithms based on multi-scale MM: geodesic reconstruction multi-scale morphological contrast enhancement (GRMMCE) [23], and multi-scale morphological approach to local contrast enhancement by reconstruction (MMALCER) [39]. These are good at improving the local contrast of the images. Table 2 shows the parameters of the algorithms based on multi-scale MM. The parameters ω of the algorithms presented in [23,39] are used in the reference articles.  Figure 7, it can be observed that as n grows and starting from n = 4, MGRTH obtains higher image entropy with respect to the compared algorithms. This gives an indication that the proposed algorithm is good at improving image detail. In Figure 7, the entropy value of the original image I is also added.  Table 3 shows the average results obtained by the compared algorithms. For the algorithms based on the multiscale top-hat transform the value of n = 4 was considered. The best average results are highlighted in bold. The average results in Table 3 show that: • MGRTH has better average performance according to the E metric, indicating that the approach enhances the details of melanoma images; • Among the algorithms based on the multiscale top-hat transform, MGRTH is the second best performer for PSNR. This means that it introduces low distortion to the images; • According to the REC metric, all compared algorithms enhance images on average.
In the Wilcoxon signed rank test, the differences of the q-pairs of observations are calculated, and based on these differences in absolute value, order ranks are assigned. In Table 4 the number of positive ranks observed is presented, i.e., the number of times that the proposal has obtained higher values of the metric than the compared algorithm, in the same way the number of negative ranks is observed, i.e., the number of times that the proposal has obtained lower values of the metric than the compared algorithm. The Wilcoxon statistic constitutes the sum of ranks (positive or negative) and for the number of pairs q ≥ 20 can be considered to be approximately normally distributed [40]. In Table 4, the Z statistic of the standard normal distribution and the significance associated with the observed Wilcoxon statistic are presented.  Figures 8 and 9, the images enhanced with different state-of-the-art algorithms can be visualized. The images obtained by the multiscale MM based algorithms use an iteration number n = 4. It can be seen that MGRTH and MMALCER are the algorithms that preserve the most features and provide the best sharpness. They also avoid adding unnecessary brightness and improve the visualization of circumscribed skin. These algorithms are possibly the most applicable for dermoscopic image assessment and classification.

Conclusions
In this work, a contrast and detail enhancement algorithm was presented. The proposed algorithm is based on multi-scale morphological operations. The extraction of features from the medical images is performed by combining the operations of the classic top-hat with the top-hat by reconstruction. This combination of operations is used at multiple scales, which are finally added to the image in a strategic way to enhance the useful features of the image, such as details and edges.
The numerical and visual results show that MGRTH improves the contrast of melanoma images according to REC metrics and is superior to comparative algorithms in improving details images according to E metric. Among the multiscale top-hat transform-based algorithms compared, MGRTH is the second one that introduces low distortion in the process of detail improvement and contrast enhancement.
For future work this algorithm may be useful for preprocessing images before using deep learning applications for segmentation, detection or classification purposes.