Review of Recent Type-2 Fuzzy Image Processing Applications

Abstract

This paper presents a literature review of applications using type-2 fuzzy systems in the area of image processing. Over the last years, there has been a significant increase in research on higher-order forms of fuzzy logic; in particular, the use of interval type-2 fuzzy sets and general type-2 fuzzy sets. The idea of making use of higher orders, or types, of fuzzy logic is to capture and represent uncertainty that is more complex. This paper is focused on image processing systems, which includes image segmentation, image filtering, image classification and edge detection. Various applications are presented where general type-2 fuzzy sets, interval type-2 fuzzy sets, and interval-value fuzzy sets are used; some are compared with the traditional type-1 fuzzy sets and others methodologies that exist in the literature for these areas in image processing. In all accounts, it is shown that type-2 fuzzy sets outperform both traditional image processing techniques as well as techniques using type-1 fuzzy sets, and provide the ability to handle uncertainty when the image is corrupted by noise.

1. Introduction

The definition of information is completely related to the definition of uncertainty [1,2]. Uncertainty is an attribute of information [3], and when it is involved in any situation of problem solving, then as a consequence some deficiency is produced in the obtained results or there is missing information, as this may be imprecise, incomplete, vague, or not be completely reliable. Through fuzzy reasoning, it is possible to deal with a large part of that uncertainty, as fuzzy logic systems use type-1 fuzzy sets (T1 FS), which represent imprecision with numerical values in the range [0, 1]. When it is difficult to establish the exact value of an entity, e.g., measurement, it is more convenient to use type-1 fuzzy logic systems (T1 FLS) rather than traditional sets [3]. In addition, if the problem to be treated has a high degree of uncertainty or it has uncertainty that is more complex, it is convenient to use interval type-2 fuzzy logic systems (IT2 FS) [4], or an interval-valued fuzzy set [3]; however, there are also generalized type-2 fuzzy logic systems (GT2 FLS) which are capable of handling large amounts of uncertainty.

There have been a large amount of applications that use an IT2 FLS in clustering, time series prediction, pattern recognition, intelligent control, and others [4,5,6]. However, in this paper we will concentrate on applications of image processing which include topics such as image segmentation, image filtering, image classification and edge detection. Diverse methods of image processing have been developed over the last year; however, these methods are not always able to handle uncertain, or inaccurate, data in the best way. As an attempt to improve these methods, in recent years, researchers have focused on applying IT2 or GT2 FLS to be able to have a greater handling of uncertainty, either in the information or the parameters which are used in the algorithms, and, through experimentation, it has been shown that better results are obtained than with the original methods. Several research papers have been developed using FLS in image processing; therefore, in this review, we are including all adaptations where IT2 or GT2 FLS were used; e.g., in image segmentation, Zexuan Ji et al. (2014) [7], an interval-valued possibilistic FCM clustering algorithm was proposed. In image filtering (Ba, A., and Yüksel, M.E., 2008) [8], a filter based on IT2 FLS to restore images corrupted by impulsive noise was proposed. In edge detection (Patricia Melin et al., 2014) [9], a method for image processing based on GT2 FS was presented; and in image classification (Luís A. Lucas et al., 2008) [10], a fuzzy classifier based on GT2 FS was proposed.

We must mention that the selection of papers considered in this review has been performed by using the search engine available in the Scopus online system of Elsevier, where the papers can be searched for by author names or by subject. In this sense, we must mention that a search for papers on T2 FLS on applications of image processing was done by using the following keywords: (“type-2 fuzzy” or “fuzzy type-2”) and (“processing” or “edge detection” or “segmentation” or “filter” or “morphology” or “smoothing” or “image” or “classification”). This resulted in 35 research papers found in various journals which were published from the year 2000 to 2017.

The rest of the paper is structured as follows. Section 2 offers a brief overview of the basic concepts of type-2 fuzzy sets. Section 3 provides a review of type-2 fuzzy logic applied on image processing. Section 4 presents the general overview and future trend in the area. And finally, Section 5 presents the conclusions.

2. Type-2 Fuzzy Sets

Type-2 fuzzy sets are usually divided in the literature as IT2 and GT2 fuzzy sets; both have been successfully used in many real-world applications. Although GT2 FS were defined some years ago, their practical application has been limited due to their higher computational complexity, favoring the IT2 FS version. IT2 $\left(A\right)$ and GT2 FS $(\tilde{A})$ definitions are described in

Table 1. Interval type-2 (IT2) and generalized type-2 (GT2) fuzzy set (FS) representation.

Fuzzy Set Type	Representation
IT2 FS	$A=\left\{\left(\left(x,u\right),{\mu }_A\left(x,u\right)=1\right)|\forall x\in X,\forall u\in \left[0,1\right]\right\}$
GT2 FS	$\tilde{A}=\left\{\left(\left(x,u\right),{\mu }_{\tilde{A}}\left(x,u\right)\right)|\forall x\in X,\forall u\in \left[0,1\right]\right\}$ where $X$ is the universe for the primary variable of $\tilde{A}$, $x$. The 3D membership function is usually denoted by ${\mu }_{\tilde{A}}\left(x,u\right)$, where $xϵX$ and $\mathrm{u}ϵU\subseteq \left[0,1\right]$ and $0\le {\mu }_{\tilde{A}}\left(x,u\right)\le 1$.

.

The uncertainty of an IT2 FS $\left(A\right)$ is represented by the union of all the primary membership functions, which is called the footprint of uncertainty (FOU) of $\left(A\right)$, and the FOU of a GT2 FS $(\tilde{A})$ is represented by the two-dimensional support of ${\mu }_{\tilde{A}}\left(x,u\right)$. Both are expressed in

Table 2. Footprint of uncertainty (FOU) IT2 and GT2 FS representation.

Fuzzy Set Type	FOU Representation	Description
IT2 FS	$FOU\left(A\right)={{\displaystyle \cup }}_{\forall x\in X}[{\underset{\_}{\mu }}_A\left(x\right),{\overline{\mu }}_A\left(x\right)]$	Upper membership function (UMF) is associated with the upper bound of the FOU$\left(A\right)$ and is denoted by ${\overline{\mu }}_A\left(x\right),\forall x\in X$. The lower membership function (LMF) is associated with the lower bound of the FOU$\left(A\right)$ and is denoted by ${\underset{\_}{\mu }}_A\left(x\right)$.
GT2 FS	$FOU(\tilde{A})=\left\{\left(x,u\right)\in X\times \left[0,1\right]{\mu }_{\tilde{A}}\left(x,u\right)>0\right\}$	The FOU of $(\tilde{A})$ is the 2D support of ${\mu }_{\tilde{\tilde{A}}}\left(\mathrm{x},\mathrm{u}\right)$ and represents uncertainty in the primary membership function of a GT2 FS.

.

Several kinds of membership functions exist to represent IT2 and GT2 FS; e.g., triangular, Gaussian, trapezoidal, etc. An IT2 and a GT2 Gaussian membership function are illustrated in Figure 1 and Figure 2, respectively. As we can see in Figure 2, the GT2 membership function has a 3-dimensional form; this additional degree of freedom is powerful enough to cope with higher uncertainty levels, to capture more information, and provides many advantages over IT2 FS. However, at the same time the inference system for a GT2 FLS has more complexity and the computational cost is high due to the defuzzification process, which increases in dimensionality as discretization increases. There exist different representations for GT2 FSs; with these it is possible to decompose the 3-dimensional membership function (MF) of a GT2 FS by using different kinds of cuts. The most commonly used cuts are horizontal slices, vertical slices [11], and wavy slices [12]. In horizontal slice representations, we can find α-planes introduced by Jerry M. Mendel et al. [13], and zSlices proposed by Christian Wagner et al. [14].

A GT2 FS has more degrees of freedom (present in the 3D MF) than an IT2 FS, and an IT2 FS has more degrees of freedom than a T1 FS. Increasing the degrees of freedom offers the possibility that a GT2 FS or IT2 FS can outperform a T1 FS. GT2 and IT2 FS have more potential to cope with the uncertainty in any problem than a typical T1 FS. In image processing systems, this a helpful attribute to be considered, because, when an image is captured by any image acquisition hardware, there are diverse factors that could disturb the image (lighting, environment, and distance) and consequently add noise and uncertainty by varying the brightness, color, or changing the original information. This phenomenon could affect the results for a precise segmentation, filtering, edge detection or classification. Using T2 FS, we have the possibility to model the uncertainty present on an image in a better way.

3. Type-2 Fuzzy Logic Applied on Image Processing

3.1. T2 FS in Image Segmentation

Image segmentation is the process of finding meaningful segments in an image in order to improve post-object detection, scene detection, or context detection. It finds areas delimited by pixels which share common traits, such as textures and/or colors.

In this section, a representative review of all image segmentation techniques using IT2 FS or IV FS is presented, chosen from the group of journal publications found using the search query defined in the introduction of this paper.

In Amer Dawoud (2015) [15], a thresholding-based approach for image segmentation was proposed which focuses on improving the accuracy of binarization by using a combination of T2 fuzzy measures, graph cuts, and by integrating spatial information. This approach was tested with dermoscopic images which proved the effectiveness of the approach.

In Zexuan Ji et al. (2014) [7], an interval-valued possibilistic fuzzy c-means clustering algorithm was proposed which uses fuzzy memberships and possibilistic typicalities in order to model uncertainty from data, thus overcoming footprint of uncertainty selection, type-reduction and defuzzification, which tend to require complex solutions for T2 FLS. The effectiveness of this approach was verified with image segmentation datasets from brain magnetic resonances as well as natural images.

In Cunyong Qiu et al. (2014) [16], an enhanced IT2 FCM clustering algorithm was proposed which reduces the common shortfalls of uncertainty handling from the normal FCM; this is done by focusing on the cluster center initialization as well as optimizing the type-reduction. This approach was validated through image segmentation datasets.

In Lotfi Tlig et al. (2014) [17], an image segmentation approach was proposed which performs fuzzy clustering and feature extraction, where a new descriptor is created by combining texture sub-features from the grating cell operator responses of an optimized Gabor filter bank and from local binary pattern outputs. Performances were measured via various experiments with natural texture images.

In Dzung Dinh Nguyen et al. (2014) [18], a genetic IT2 FCM clustering algorithm was proposed, used specifically for image segmentation of Multicolour fluorescence in situ hybridization (M-FISH) images, where chromosome pixels are first segmented into two clusters, and afterwards these pixels serve as a mask for the remaining channels.

In Zhi Liu et al. (2014) [19], a multiple-feature and multiple-kernel support vector machine for scene image segmentation was proposed, where, via a local homogeneity model and Gabor filter, pixel wise intensity, gradient, and C1 MF features are extracted. Also, a feature validity-IT2 FCM clustering algorithm was proposed which improves on the segmentation stage. Experimentation was done using Berkeley Segmentation Dataset (BSDS) datasets and real natural scene images.

In Howard Lee and Yi-Ping Phoebe Chen (2014) [20], an optimum threshold image segmentation approach based on T2 FS was proposed, where this technique is applied to the preprocessed image obtained from a 3D color algorithm in order to improve the detection of variations in the image. This approach was directly compared to the Otsu algorithm when using skin cancer images and showed significant improvement in detection.

In Tae-Koo Kang et al. (2013) [21], an efficient stereovision-based motion compensation method for moving robots was proposed, wherein an ego motion compensation method with three stages is given, where the first stage is the segmentation phase in which extended T2 fuzzy information theory is used, and the second stage is a feature extraction module in which wavelet level-set transform is used. Experimentation was done with moving robots and it was shown that ego-motion errors were reduced.

In Cunyong Qiu et al. (2013) [22], a fuzzy segmentation algorithm for MRI data was proposed, where two fuzzifiers are utilized in an IT2 FCM along with a spatial constraint on membership functions. Experimentation showed that this approach has better results than traditional FCM when testing in both synthetic and MRI images.

In Miguel Pagola et al. (2013) [23], a technique for selecting IT2 FS for thresholding was proposed, where fuzzy sets are proposed which can be chosen in order to better cope with multiple images instead of one, and, with such fuzzy sets, entropy is minimized in order to binarize images. Experiments showed that proposed fuzzy sets performed better when dealing with the uncertainty of not knowing which fuzzy sets to use for specific images.

In Murugeswari Palanivelu and Manimegalai Duraisamy (2012) [24], a feature extraction and fuzzy clustering-based unsupervised color texture image segmentation method was proposed, where color segmentation is done using Harlick features from Integrated Color and Intensity Co-occurrence Matrix (ICICM), then α-cuts from a IT2 FCM are used in order to better cluster texture regions from an image. Experimental results showed a better performance with the proposed approach when compared to other state-of-the-art techniques.

In Murugeswari Palanivelu and Manimegalai Duraisamy (2012) [25], a color texture segmentation approach using Haralick features extracted from ICICM was proposed, where an extended IT2 FCM clustering algorithm is used for classifying different regions of texture images. Experimentation showed better results when compared to current image segmentation algorithms.

In Lotfi Tlig et al. (2012) [26], a texture descriptor which combines cell operator outputs derived from a Gabor filter bank and local binary features was proposed, where micro-features can be extracted by using an extended version of T2 FCM. Experimentation with various textured images showed superior segmentation accuracy with respect to quantitative and qualitative comparisons.

In M. Emin Yuksel and Murat Borlu (2009) [27], a thresholding-based segmentation algorithm was proposed, where it utilizes T2 FLS for the automatic thresholding determination for accurate segmentation of pigmented skin lesion images. Experimentation showed the successful handling of uncertainty in determining borders between lesion and skin.

In Jiao Shi et al. (2017) [28], an active contour model which employs IT2 FS for auroral image segmentation was proposed, where it is used to reduce the negative influence of inhomogeneity in aural oval images as it can robustly segment these images, even in the presence of high intensity variations. Experimentation was done with data collected by NASA polar satellites on Ultraviolet Imager (UVI) aural oval images, where results demonstrated the advantages of the proposed method with better segmentation accuracy.

In Mohammed Shehab et al. (2016) [29], an analysis of speed gains using Graphics Processing Unit (GPU) technology on FCM, T2 FCM and IT2 FCM is explored, where speedup gains of 6 to 20 times are achieved. Portions of highly parallel code are shown to help the image segmentation community.

In Wang Chunyan et al. (2016) [30], a supervised image segmentation algorithm was proposed, where T1 and T2 fuzzy models are used in order to improve the performance of the final model. Qualitative and quantitative analysis demonstrated that it had better accuracy than other common techniques when using both synthetic image datasets and panchromatic images.

In M. Zarinbal et al. (2015) [31], an IT2 fuzzy automated tumor detection system was proposed, where the image quality of MRI scans is enhanced via preprocessing, segmentation and feature extraction, such that multiple consecutive planes are processed to improve accuracy. Evaluation of this approach was done with MRI scans, where astrocytomas were differentiated in order to assess the quality of the proposed method.

In Jiao Shi et al. (2015) [32], an IT2 fuzzy approach for segmenting images with large quantities of uncertainty was proposed, where pixels within a narrow band region near the contour boundary are updated in order to reduce the computational load of T2 FS. Results obtained with synthetic and real images showed an effective and efficient performance independent of initial conditions.

In M.H. Fazel Zarandi et al. (2016) [33], a T2 fuzzy expert system for meniscal tear diagnosis using MRI images was proposed, where in the preprocessing stage an λ-enhancement algorithm is used, followed by an IT2 FCM clustering algorithm for segmentation, and finally a neural network is used in the classification stage. Experimentation showed that the proposed approach was superior in performance when compared to competing algorithms.

Summary

Published papers reviewed for the category of image segmentation share no common image databases for comparison with each other, which highly limits a direct comparison in order to ascertain which has the better performance in this category. Among these papers [7,16,17,23,24,25,26,32], they were all validated using databases of real images from varied sources and in combination, used several types of validation techniques, such as a visual appreciation of the end result, recognition rate, error rate, and number of produced segments. This leads to a highly-varied collection of experimental tests performed among these papers. Application-wise papers, where the use of T2 FLS was used for solving very specific problems where image segmentation was a portion of the solution, were done by two types of papers. First, by [19,21,28], such that [19] and [21] both are direct applications to a humanoid robot vision system, although both focused the use of IT2 FLS in different ways, as the first was used for object sample selection and the later was used for feature validity; and [28] is an application in auroral image segmentation. Secondly, the rest of the papers focused their algorithms to specifically solve medical imaging databases of different natures [18,20,22,27,29,31,33], using image segmentation.

Validation methods used throughout this category was also highly varied, yet the two most common types of validation were the qualitative assessment of results via visual appreciation and the error rate for segmentation. Although the error rate for segmentation was used via multiple terms (recognition rate, error rate, segmentation accuracy, classification rate, and prediction rate), with some slight differences in how these indices were obtained, they were all still used to measure the quality of the image segmentation performance.

It is also worth noting that, although most research papers used variations of the IT2 FCM clustering algorithm, some papers solely used IT2 FLSs as some type of expert system [20,23,27,28,31]. Some even used hybrid versions of IT2 FLS or IT2 FCM, such as Interval-Valued Possibilistic Fuzzy C-Means (IVPFCM) [7] and Extended Type-2 Fuzzy Information Theory (ET2FIT) [21].

To finalize this summary, it is quite clear that in the category of image segmentation there is a disparity in what image databases are used, leaving direct comparisons between papers impossible to achieve; the measures used are also not standardized. This leaves the question as to which image databases and measures could be used to get a common ground between experimentation and results to better discern the quality of new proposed techniques.

3.2. T2 FS in Image Filtering

Digital images are often distorted or contaminated by noise when taken by faulty sensors, or when transmitted by noisy channels. Filtering consists of a set of tools which suppresses or highlights information contained within an image at different scales, to highlight some elements of the image, or to hide anomalous values. Filtering refers to enabling or deleting certain frequency components and the main objectives are to smooth an image to reduce variations or intense changes between neighboring pixels, or eliminate noise, which allows removing or attenuating those pixels whose intensity levels are very different from their neighbors. In this section, a review of the applications of T2 FS image filtering which have been proposed is presented.

In Zhai Daoyuan, et al. (2012) [34] an approach for removing mixed Gaussian and impulse noise removal from images based on a non-singleton IT2 FLS was proposed; this also promotes the use of a quantum-behaved Particle Swarm Optimization (PSO) algorithm for determining the parameters of the FLS. This noise is considered to be one of the most essential topics in the domain of image restoration, and this approach was tested in black and white benchmark images corrupted with impulse noise. The authors make evident the efficiency of the approach over other techniques that use FLS.

In the proposal of Yuksel, M. E., and Basturk, A., (2008) [8] an impulse detector based in an IT2 FLS for restoring noisy color images was presented. This detector is based on a selective filtering behavior by using an additional image processing operator that decides whether the pixel studied needs to be filtered. This detector reduce the blurring effects and distortions of the impulse noise filter without affecting the filtering process. This approach was tested with benchmark color images corrupted with impulse noise, which proved the effectiveness of the method, and this can be used as a tool for enhancing the output achievement of a noise filter.

In Ba, A., and Yüksel, M. T. (2005) [35] a filter based on IT2 FLS to restore images corrupted by impulsive noise was presented. This filter has the ability to effectively preserve texture, edges, thin lines, and other important information within the image. Tests performed with corrupted images with different levels of impulsive noise show that, with the proposed filter, higher results were obtained than with other noise-control operators included in the literature.

In Own, C. M., et al. (2006) [36] an adaptive T2 fuzzy median filter to attenuate impulsive noise to preserve details of images was proposed. This filter takes advantage of the ability of T2 fuzzy uncertainty management systems in combination with median filters, T1 based in fuzzy media, and Arakawa filters. The advantages of this filter are that it reduces the memory required by the calculations and according to tests performed with benchmark images, the results demonstrate that it is a robust method in eliminating noise.

In Wang, S. T., et al. (2005) [37] a selective feedback fuzzy neural network based on IT2 FLS to eliminate Gaussian noise and preserve the characteristics of a damaged image is presented. The potential of the method was demonstrated when using two-dimensional Lena and Goldhill benchmark images, corrupted by several levels of Gaussian noise, where experimental results show that the filter works better than the average filter and the Wiener filters; however, this filter requires a high computational power.

Summary

Research papers reviewed in the filtering category use some benchmark images for their validation process, such as the baboon, Lena, house, papers, pentagon, boat, bride, and Goldhill; however, they do not all coincide in the publications, nor do they share the same characteristics as regards the type or percentage of noise applied, which makes it difficult to make a direct comparison.

In [8], the authors present an impulse detector based on three input and one output first-order Sugeno-type IT2 FLS, while in [34] a non-singleton IT2 FLS was proposed. In [35], a filter based on T2 FLS techniques was presented, and in [36] a filter based on an adaptive T2 fuzzy median was proposed. Finally, a selective feedback fuzzy neural network based on IT2 FLS was introduced in [35]. In the research papers [8,34,35,36,37], filtering accuracy is validated using visual appreciation, mean squared error (MSE), mean absolute error (MAE), the peak signal to noise ratio (PSNR), and normalized mean square error (NMSE).

In [8,35,36], images were corrupted by impulse noise ranging from 5% to 75%. Only in [8], tests are performed on color images using validation metrics MSE and MPE. In [35], validation was performed with MSE, while in [36], MAE and PSNR were used. In [34,37], images were corrupted with Gaussian noise ranging from 30% to 70%, 0.01% and 0.06% respectively. Additionally, [34] also was corrupted with 30% of impulse noise and was validated using visual appreciation and MSE, while in [37] MAE and NMSE was used.

Although some benchmark images were used in a few of the research papers reviewed, the type and percentage of noise as well as the metrics used are not unified, making it difficult to compare the methods reviewed. In this section, five journals from 2005 to 2012 were reviewed, in which the presented results determined that the methods based on T2 FS are better than T1 FS as well as other traditional methods, so it is considered that there is a long way to go in this area of image processing.

3.3. T2 FS in Edge Detection

The process of edge detection in a digital image consists of the identification of the pixels where brightness changes dramatically or has discontinuities with respect to their neighbor pixels. Edge detection is frequently the first step in recovering information from images and is useful for simplifying the analysis of images by drastically reducing the amount of data to be processed. Some traditional approaches exist to perform the edge detection process. The idea to implement fuzzy sets in the edge detection process is because the detection of subtle changes may be mixed up by noise, and this depends on the pixel threshold of change that defines an edge; therefore, the detection of these continuous edges is very difficult and fuzzy theory is useful to solve this problem.

This section presents a review of edge detection methodologies based on T2 fuzzy sets.

In Patricia Melin et al. (2010) [38] an edge detection method based on the morphological gradient technique and IT2 FS was proposed. The IT2 FLS was designed using Gaussian membership functions, and the parameters were calculated dynamically according with the gradient values found in each image; the fuzzy system consists of four inputs, one output, and three fuzzy rules which were proposals after the sever of test and based on the human expert in this kind of application. The type reduction was performed using the center of sets method. In the simulation results, a sample of the Olivetti Research Laboratory (ORL) databases of faces [39] and USC-Signal and Image Processing Institute (USC-SIPI) Image Database (Laboratories Cambridge) [40] in gray scale format was used. The edge detection based on IT2 FS is applied using different footprints of uncertainty (FOU) and the results are compared against the morphological gradient, Sobel, Canny and the edge detector based on T1 FS. The authors concluded that the edge detector method based on IT2 FS outperformed the edge detection process based on T1 FS and, of course, the traditional techniques.

In the proposal of Claudia I. Gonzalez et al. (2016) [41], an improved Sobel edge detection method based on GT2 FS was presented. According to this paper, the authors presented in the same work the approach based on IT2 FS and GT2 FS; both fuzzy systems were designed using two inputs and one output. In the inputs and outputs, they implemented Gaussian membership functions with uncertain mean. The structure of the rules is the standard Mamdani-type, and for the type-reducer process the Centroid method was used, and the GT2 FS was approximate with α-planes. The approach was tested with a synthetic images database, which was built plotting ten mathematical functions as original images and as ground truth reference for the ideal edges. The accuracy of the edge detection process was measured with the figure of merit of Pratt [42]. Simulation results were obtained with the Sobel operator; the edge detector was based on T1 FS, an IT2 FS and with a GT2 FS. For the IT2 and GT2, the FOU was selected arbitrarily and was varied in a range between 0 and 1. Based on the statistical test presented, the authors affirmed that the GT2 FS improved the results achieved by the IT2 FS, while IT2 FS was better than T1 FS, and the traditional Sobel method was the worst for images with Gaussian noise and without noise.

In Patricia Melin et al. (2014) [9] a new edge detection method for image processing based on GT2 FS was presented. In this approach, image gradients were obtained using the morphological gradient technique in a similar way to that used in the proposal presented in [38]; however, with the difference that the gradients were processed using GT2 FS. The GT2 FS was designed with four inputs which were granulated in three linguistic variables and one output granulated in two linguistic variables. For both, the author used Gaussian membership functions with uncertain mean. In the structure of GT2 fuzzy rules, the standard Mamdani-type was used, and the defuzzification process was performed using the heights and approximation methods; additionally, the GT2 FS was approximate using the α-planes theory. For testing, a sample of the USC-SIPI [38] database and one synthetic image were used. Tests were performed using images with Gaussian noise as well as without noise. The quality of the edge detection process was measured using the figure of merit of Pratt [42]. They presented a plethora of comparative studies: in the first, the FOU of GT2 membership functions was varied to improve the edge detection results; in the second, the defuzzification process was performed using heights and approximation method, where the heights method had better performance in images without noise, but in images with Gaussian noise, the approximation method improved the results; in the third, the number of α-planes was varied (5, 10, 50, 100, 150, 200, and 1000) with the aim of finding the α-planes necessary to approximate the output; for images without noise, the same quality edge detection was achieved when the different α-planes number was applied, and for images with a Gaussian noise level of 0.001 and 0.002, 50 and 100, α-planes number was necessary; finally, a comparative study was presented, applying the traditional morphological gradient technique, T1 FS, IT2 FS and GT2 FS. The edge detection based on GT2 achieved the best results, the second best method was the IT2 FS, followed by the T1 FS and, finally, the lowest values were obtained by the morphological gradient. The authors concluded that the GT2 FS allows for a better modeling of uncertainty because it gives more degrees of freedom in comparison to T1 and IT2 FS. Besides this, the traditional technique does not consider any parameters to prevent noise or regions with very low or high contrast.

In Olivia Mendoza et al. (2009) [43] an efficient edges detector using an IT2 FS was proposed. The fuzzy system is a Mamdani-type, which was designed with three inputs, one output and five fuzzy rules; the first and second input represent the horizontal and vertical gradients obtained with the Sobel operator after being applied over the input image, and the third input is obtained after applied a low-pass filter. The fuzzy edge detection approach was tested using the ORL database (a sample of 40th person) [39]. The results were illustrated by building frequency histograms of some images, and the results achieved by the IT2 FS were contrasted against the outputs reached by the Sobel operator and the T1 FS. To illustrate the advantages of the approach based on IT2 FS against the results reached by the Sobel operator and the T1 FS, the frequency histograms of some images and the binarization process was presented, where the authors affirmed that better performance was obtained when the edge detection based on IT2 FS was implemented over the results achieved by the Sobel operator and T1 FS, because IT2 FS found the background and edges pixels without losing image detail. The authors concluded that this kind of edge detection method can be helpful and improve different pattern recognition applications.

In Claudia I. Gonzalez et al. (2016) [44], the optimization of a fuzzy edge detector based on the traditional Sobel technique combined with IT2 FS was presented. This paper also included the optimization for an edge detector based on T1 FS in order to perform a comparative study against results obtained with IT2 FS. For the optimization of the fuzzy inference systems, cuckoo search (CS) [45] and genetic algorithms (GAs) [46] were applied with the idea of finding the optimal design of the antecedent parameters for an IT2 FS and to achieve better results than the non-optimized IT2 FS in edge detection applications for digital images. The principal parameters of CS and GA algorithms were obtained using simple Monte-Carlo simulations, and the objective functions were evaluated with the figure of merit of Pratt [42]. Both algorithms were used to optimize the antecedent parameters for Gaussian membership functions with uncertain mean. In the fuzzification process, a T2 singleton fuzzifier was used. The structure of the rules was standard Mamdani-type and the centroid method was implemented in the defuzzification process. All the tests were applied on synthetic images and using reference images, with the aim of measuring the quality of the detection process. In the results presented by the authors, the values achieved by the edge detection method based on IT2 FS after the optimization process using CS and GA were very similar; nevertheless, these optimizations improved the results achieved by the non-optimized IT2 FS. However, the values reached with the non-optimized IT2 FS were better than the T1 FS optimized by CS and GA and the non-optimized T1 FS. In general, a hybrid combination of soft computing techniques was proposed in this paper, which was useful in outperforming the edge detection process.

In Humberto Bustince et al. (2009) [47] two edge detectors as a practical application of interval-valued fuzzy sets (IVFS) were presented. In the proposal, interval-valued fuzzy entropy, t-representable interval-valued t-norms, and s-representable t-co-norms are used; therefore, the aim of this paper was to analyze concepts of IVFS theory and their application for edge detection in grayscale images, considering that each element of an IVFS is associated with not just a membership degree but also the length of its membership interval. As the interval length can be used to indicate the range of intensities associated with a pixel and its neighbors, IVFS is a convenient choice for edge detection problems to find pixels whose intensity (gray level) is very different from those of its neighbors. In simulation results, three edge detection methods were presented: two based on IVFS and the other one using Canny’s detector. These were applied on a sample of more than 1000 images, including the database from the computer vision home page (http://www-2.cs.cmu.edu/~ci/vision.html) and a sample of the USC-SIPI database [40]. The author concluded that their proposal achieved favorable results and was much better than those obtained by Canny’s detector, with the great advantage that the IVFS methods constructed partial edges before determining the final edge; also, the authors affirmed that the complexity of the algorithm using IVFS did not increase. Additionally, they explained that the performance of Canny’s detector is dependent on the parameters selected and their proposals (IVFS) are unsupervised algorithms, in which user decisions are not required.

In Jiao Shi et al. (2014) [48] an unsupervised change detection approach for synthetic aperture radar images based on a fuzzy active contour model and a genetic algorithm was presented. The aim was to enhance the active contour model (ACM) by employing IT2 FS to properly handle uncertainties in the difference images (DIs). This approach consists of two steps: firstly, the DIs are analyzed by the IT2 FS active contour model and different intermediate change detection masks are generated; secondly, a GA was employed to find the final change detection mask by evolving the realization of intermediate change detection masks. The proposed method was applied over three Synthetic Aperture Radar (SAR) image data sets: the Bern dataset, Ottawa dataset and Yellow River dataset, and the performance was validated using the percentage correct classification (PPC), Kappa statistic and overall error (OE) measures [49]. In order to illustrate the advantage of the proposed method against other existing approaches, a comparative analysis was presented against the other techniques. According with the results reported by the authors, for the tests applied over the Bern dataset, the proposed method exhibited the best OE, PCC and Kappa compared with the other approaches; with the Ottawa dataset, the highest PCC and Kappa and the lowest OE was achieved by the proposed method; and for the Yellow River dataset, the lowest value of OE and the highest values of PCC and Kappa were obtained by the proposed method. Finally, the authors concluded that the approach based on IT2 FS is a good technique to be applied over real SAR image datasets due to its better performance, as IT2 FSs have a good adaptability to model uncertainties which cannot be appropriately managed by traditional fuzzy sets or in others methodologies.

Summary

In this section, published papers in the category of edge detection were reviewed, using approaches based on IT2, GT2 and IV fuzzy sets. The results presented in these papers use different image databases and metrics, which is a limitation that inhibits a comparison between all methodologies. Therefore, it is impossible to determine the better approach; however, we can highlight the follows points.

All papers use real images and the benchmark images database for edge detection, with the exception of papers [41,44], which only use synthetic images to evaluate performance. In general, edge detection accuracy is validated using frequency histograms, visual appreciation, figure of merit of Pratt, percentage correct classification, Kappa statistic and overall error measures.

In papers [38,43,44], IT2 FS are combined with traditional techniques (Sobel, morphological gradient) to improve the edge detection process. In accordance with the published results, the edge detection based on IT2 FS improved the results of T1 FS as well as from traditional methods. This is because IT2 FS preserves more information about the edges and has the capability to handle the uncertainty to determine if a pixel is an edge or not.

In [9,41], the authors present an edge detection based on GT2 FS; results are compared against the IT2, T1 FS and the traditional techniques. The interesting part in this paper is when the image database is corrupted by noise. According to the results and the figure of merit of Pratt metric, the edge detection accuracy achieved by GT2 FS was better than IT2 FS; in the same manner IT2 FS was better than T1 FS and, of course, the traditional methods were worse (Sobel, and morphological gradients); this is because these last do not have additional parameters to handle the uncertainty or noise.

In [47], the authors present a good methodology for edge detection based on IV FS. The limitation of this is that it is compared only with the Canny edge detector and the results are evaluated using the visual appreciation, which means that it is difficult to conclude if the approach based on IV FS is better.

In summary, we can notice that GT2 FS represent an important advantage in this application area and represent a good option for image processing systems to better handle uncertainty, but it is important to evaluate the computing time for real-world applications, which is a problem in this kind of methodology (GT2 FS). Another problem in the edge detection process is how to evaluate the edge detection accuracy; for this reason, it is important to select an adequate metric. Besides this, some metrics needs the ground truth image (ideal edge) to be compared with edge detection results achieved by any proposed approach (T1, IT2, GT2, or IV FS); these databases are difficult to find and the alternative for some researchers is to use synthetic images.

3.4. T2 FS in Image Classification

A classifier can be defined as a system that assigns a class label to an object, based on the object description; i.e., the classifier can predict the class label. In image processing systems, the classification process consists of assigning a class label to each pixel, in which each pixel can be part of an object. Image classification is a hard task when each pixel is allocated in a specific class and, above all, when the image is corrupted by noise, it is difficult to identify the right class, providing conflict in the results. For this reason, FS has been successfully employed for combining data with uncertain and incomplete information in the classification procedure, so that pixels may have multiple or partial class membership.

This section presents a review of some important classification approaches which use T2 FS theory to solve the imprecision and uncertainty problem which is presented in the traditional classifier methods.

In Saugat Bhattacharyya et al. (2015) [50], an IT2 fuzzy approach for real-time EEG-based control of wrist and finger movement was proposed. The classification of motor imaginary signals for wrist rotation and grasping activity were performed using an IT2 FS classifier. The IT2 FS classifier was designed using Gaussian membership functions, and this was implemented in a 3-level hierarchical architecture. At level-1, the classifier decides whether a movement has been imagined or not. The movement classifier at level-2 distinguishes between wrist or finger movement. If it is a finger-related movement, the classifier at level-3 decides whether the operation is that of an opening or closing of the fist. The performance of the IT2 FS classifier, trained on EER based features over several days, is compared with the previous work of other researchers. The test was applied over an EEG data set; this data set was collected from eight subjects (four male, four female) in the age group of 25 ± 3 years over a period of three consecutive days as training data. The proposed classifier was trained in the offline mode with EEG observation of three days, and the testing was done for two more days. The train-set and test-set for each classification level was selected using 5-fold cross validation. The performance of the classifiers is measured based on classification accuracy, sensitivity, and specificity [51]. The results achieved by the IT2 FS classifier were compared with other classifiers; after that, the Friedman’s Test [52] was applied. The authors concluded that the proposed IT2 classification method provided superior performance compared with that of the other standard algorithms, with an accuracy of 86.45% in offline mode and 78.44% in online mode.

In Luís A. Lucas et al. (2008) [10], a fuzzy classifier based on GT2 FS for application in land cover classification was proposed. The aim was to minimize the effects of uncertainties in the usual fuzzy rule-based classifiers. The fuzzy system was designed as a GT2 singleton where antecedents and consequent were represented by Gaussian membership functions and Mamdani-type for the structure of the rules; the rule number was based on the same classes number; i.e., one rule for each class. The type reducer process was performed with the vertical slice centroid type reduction. The proposed method was compared with two other classifier algorithms: a maximum likelihood classifier and a T1 FS classifier. The experiments were applied in a land cover data set which consisted of three classes, with five samples for each one. The classifiers were trained using cross-validation with a total of 125 different tests. In accordance with the published average, the authors concluded that the GT2 FS classifier was better than the T1 FS classifier, but worse than the maximum likelihood classifier. GT fuzzy sets have the ability to handle greater uncertainty; therefore, the GT2 classifier could be applied in a more complex dataset to evidence its advantages against other classification algorithms.

In the proposal R.I. John et al. (2000) [53] a neuro-fuzzy clustering of radiographic tibia image data using T2 FS was presented. The T2 FS were combined with two neural network approaches for clustering: the adaptive resonance theory (ART) [54] and the MINMAX network [55]. In the simulation results the proposed method was compared with the T1 fuzzyART algorithm and the T1 FuzzyMINMAX [55]. Tests were applied on a tibia image data set with 203 images; in order to measure the accuracy of the different classifiers, the contingency coefficient C [56] and Kappa [49] coefficient were applied. The best classification rate was achieved with the T2 fuzzyART (60% accuracy); the worst were the T1 fuzzyART (54%) and T2 fuzzyMINMAX (54%).

Summary

In this section, some classifier methods based on T2 FS were analyzed. Regarding these research papers, we can summarize the following.

Papers reviewed in this section [10,50,53] use real images, where [50,53] are focused on medical applications. They use different metrics to evaluate the classification accuracy in which we can single out the Friedman test, contingency coefficient, and Kappa. In [10,50], the training set and testing set are obtained using the cross-validation technique. All papers show comparative results against T1 FS as well as other classifier techniques, where, in accordance with the classification rates, the approach based on T2 FS [50,53] is better than T1 FS approaches and other known methods. Based on these results, we can observe that T2 fuzzy techniques perform better when applied to classification applications, especially when digital images are incomplete or involve some noise, in which case it is difficult for the system to determine if a pixel is part or not of a class.

In accordance with the achieved results in [10], the maximum likelihood classifier was better than the classifier method based on GT2 FS. This may be because the dataset is simpler and the solution can be found with the traditional methods; therefore, it is necessary to apply this approach in more complex data sets or add some noise to illustrate the advantage of using GT2 FS classifiers. Another solution is to optimize the GT2 FLS to obtain optimal membership functions, parameters, and fuzzy rules to improve final performance.

In this section, only three papers were found which applied T2 fuzzy sets to classify the pixels in digital images; therefore, this represents a wide area of opportunity to further explore this kind of applications.

4. General Overview and Future Trend

Four main areas of image processing were reviewed in this paper: classification, edge detection, segmentation, and filtering, for which T2 FS or IVS were involved. First, a general analysis on the combined areas will be given, and afterwards, individual insights into research trends in each area will be put forward.

All mentioned areas in this review paper have a combined total number of publications of 35 journal papers, in which classification research has three publications, filter research has five publications, edge detection research has seven publications, and segmentation research has 20 publications. Shown in Figure 3 is a pie chart where these values, represented by percentages from the total, give a better view of publication area distribution. This graph clearly shows that the segmentation research is far more prominent and is more active, followed by edge detection research, then filter research, and finally, classification research trails behind with the lowest participation in research publications.

It is interesting, in relation to the previous statistic, that the number of citations is differently arranged, such that classification research papers have a combined number of citations of 73, filter research papers have 306 combined citations, edge detection research papers have 157 combined citations, and segmentation research papers have 88 combined citations. Shown in Figure 4 is a pie chart where these citation numbers are seen with percentages. These numbers become of interest regarding the fact that most citations belong to filter research papers, which has the second lowest amount of publications, and that segmentation research papers have the second lowest amount of citations. To better understand this phenomenon,

Table 3. Relation between number of combined citations and number of publications per research area.

Area	Number of Combined Citations	Number of Publications
Classification	73	3
Filter	306	5
Edge detection	157	7
Segmentation	88	20

shows the relation between number of publications and combined citations per area.

On the individual side, classification research papers, as shown in Figure 5, have the lowest interest, with only three publications in total, when T2 FLS are involved. Interestingly enough, these publications are very spaced out, each one being seven and eight years apart, as well as each one using different representations: T2 FuzzyART (2000), GT2 FS (2008), and IT2 FS (2015). As for active authors with repeated contributions, there are none; i.e., with each of the three existing papers, each author only appeared once. This leaves a very good area of opportunity, since it is very clear that research development in this category is next to none.

As for edge detection research papers when IT2 FLS or IVS are involved, Figure 6 shows the publication trend. Such research has only been carried out fairly recently, yet it still lacks support from the research community, as considerably few publications exist. However, it is positive to say that, in 2016, interest still exists, although it received no attention from 2001 to 2009. Among all research areas, this is the one for which there are most authors with recurring publications, as shown in

Table 4. Authors with recurring publication in the area of edge detection.

Edge Detection	Number of Recurring Publications
Melin P.	5
Mendoza O.	5
Castillo O.	4
Castro J.R.	3
Gonzalez C.I.	3

, where only recurring authors who have published more than once appear in the list. In this case, Melin P. and Mendoza O. are the two most prominent researchers in this category, such that both authors jointly appear in 5 of the 7 existing publications. Both of these authors are the only ones among all authors from reviewed papers with this many recurring publications.

For filtering research papers when IT2 FLS or IVS are involved, Figure 7 shows the publication trend, where low interest by the research community can be seen, such that it has been a couple of years since any publications have come to existence; five years, to be precise. Regarding author interest to create multiple contributions in this area, only one author exists with two recurring publications (Baştürk A.).

Finally, as segmentation research papers when IT2 FLS or IVS are involved, there is more interest from the general research community, as shown in Figure 8. The first research was published in 2009, and in 2014, its peak of interest was achieved; yet, in recent years, publications in this area has calmed down. However, this is the most active area in comparison with the other research areas also reviewed in this paper. In terms of recurring published papers by the same authors, it shows that authors have only worked in this area two times at most, as seen in

Table 5. Authors with recurring publication in the area of segmentation.

Segmentation	Number of Recurring Publications
Fnaiech F.	2
Han L.	2
Manimegalai D.	2
Murugeswari P.	2
Qiu C.	2
Sayadi M.	2
Shi J.	2
Tlig L.	2
Turksen I.B.	2
Xiao J.	2

; interest is not as high as with edge detection techniques.

It is worth noting that T2 FLS tools began to exist around the year 2000, when it began to become more computationally viable, marking the beginning of the included time period of selected papers. Around 5–10 years later, these tools became even more common; this is therefore the reason most research papers in these areas began to appear with more frequency.

All shown statistics, global and individual, show that some areas have more interest from researchers and others do not. However, this is considering that only research when IT2 FLS or IVS are involved were considered. Although segmentation research has generated far more interest in contrast with the other covered research areas, this does not mean the others are of no interest, shown when combined citations were presented in Table 4, where one of the least published areas, filter techniques, has almost the same amount of citations of all other areas combined. As to what and how all these statistics should be interpreted, this is left mostly to the reader. However, in contrast, the only certain fact is that these four research areas which involve IT2 FLS or IVS have managed to capture interest from the general research community.

5. Conclusions

In this paper, a representative and concise review of type-2 fuzzy logic applications in image segmentation, image filtering, image classification and edge detection is presented. Type-2 fuzzy logic is quite a recent area in fuzzy logic and computational intelligence, since type-2 fuzzy logic tools began to appear up until the early 2000s, which comes from an extension of the concepts of traditional type-1 fuzzy logic. The extension from type-1 to type-2 fuzzy sets enables a better modeling of uncertainty, which in turn helps to better manage uncertainty or noise in real-world systems. In this case, it has been a natural process to transition from type-1 to type-2 fuzzy logic, as this transition enables the better management of noise in image processing. We envision that, in the future, the number of total papers using type-2 fuzzy logic in image processing will increase, as even more complex real-world applications would require this kind of higher type models.

Reviewed papers have provided not only a collection of existing applications in image processing using type-2 fuzzy logic, but also a window as to how type-2 fuzzy logic is applied to solving this type of problems. They have also given an insight into when these research papers have been published by showing trends in a timeline, as well as the names of the authors who have maintained interest or have simply inquired into the potential use of type-2 fuzzy logic in image processing by only publishing once.