A Fast Segmentation Method for Fire Forest Images Based on Multiscale Transform and PCA

Forests provide various important things to human life. Fire is one of the main disasters in the world. Nowadays, the forest fire incidences endanger the ecosystem and destroy the native flora and fauna. This affects individual life, community and wildlife. Thus, it is essential to monitor and protect the forests and their assets. Nowadays, image processing outputs a lot of required information and measures for the implementation of advanced forest fire-fighting strategies. This work addresses a new color image segmentation method based on principal component analysis (PCA) and Gabor filter responses. Our method introduces a new superpixels extraction strategy that takes full account of two objectives: regional consistency and robustness to added noises. The novel approach is tested on various color images. Extensive experiments show that our method obviously outperforms existing segmentation variants on real and synthetic images of fire forest scenes, and also achieves outstanding performance on other popular benchmarked images (e.g., BSDS, MRSC). The merits of our proposed approach are that it is not sensitive to added noises and that the segmentation performance is higher with images of nonhomogeneous regions.


Introduction
The conventional detection systems of smoke and fire use sensors [1]. One of the major drawbacks, is that the systems do not issue the alarm unless the particles reach the sensors [2]. Recently, as an appropriate alternative to conventional techniques, vision-based fire and smoke detection methods have been adopted. Here, smoke and fire are regarded as a specific kind of texture. It is difficult to accurately detect the appearance of mentioned regions from images due to large variations of color intensities and texture. Although, many research works confirmed that texture features play a very important role in smoke and fire detection [3,4]. A wide recent work demonstrated that the multi-scale based techniques play an important role in smoke and texture classification [5,6]. Developed methods cover both areas; images and videos processing [4,7,8]. In this work, we aim to segment images into significant regions. This will be used to generate useful information for our project. In this paper, we propose a new segmentation approach based on Gabor filtering and Principal Component Analysis (PCA). The proposed method is based on the modification of the superpixels extraction methodology to increase the robustness to added noises and to improve the segmentation accuracy of the fire forests color images. For the extracted features clustering, we used the new version of the fuzzy classifier recently proposed in [9]. Our choice is done regarding the (1) A multiresolution image transformation based on 2-D Gabor filtering combined with a morphological gradient construction to generate a superpixel image with accurate boundaries. This proposition integrates a multiscale neighboring system to solve the problems of rotation, illumination, scale, and translation variance. This is very useful specially with images of high resolution. (2) We introduce a Principal Component Analysis (PCA) to reduce the number of extracted Gabor features. From obtained regions we compute a simple color histogram to reduce the number of different intensities (pixels) and achieve a fast clustering for color image segmentation.
In summary, image segmentation methods can be roughly classified into two categories: supervised and unsupervised. In this paper, we mainly discuss a fuzzy unsupervised framework. No features learning is involved. This task remains one of the most challenging research topics because there is no unified approach to achieve fast, robust, and accurate segmentation.
In this work, a detailed study of existing color image segmentation approaches was carried out to investigate the most common stages in segmentation's techniques. In Section 2, we discuss the motivation for using the different implemented techniques. Furthermore, we thoroughly described each phase and introduced ideas for improvements. Next, we describe the development of the proposed method. Section 4 presents an evaluation study of the proposed improvement using a set of synthetic and real color images from the well-known dataset (BSD 500 and MSRC). As a validation stage, the developed method is applied on fire forest images and compared to the standard and recent methods.

Motivation for Using Superpixels with Gabor Filtering
In color image segmentation, non-texture areas are relatively uniform, and it is easy to obtain the accurate boundaries. Color and spatial information are sufficient for the clustering task. In texture areas, Sensors 2020, 20, 6429 3 of 27 the boundaries are the combination of micro and macro adjacent regions. Here, texture edges cannot be incorporated only by characteristics of single pixels: intensities and spatial coordinates. Hence, to obtain these boundaries requires a combination of multi scale characteristics. Many researchers have verified that multi-resolution features are able to get the main outline for various texture regions of the image [12,13]. In the last decade, the Gabor filters, firstly proposed by Dennis Gabor in 1946 in 1-D and extended, in 1985, to 2-D by Daugman, have received much attention. Their wide usage in multiple fields can be taken as proof of their success: image analysis, compression and restoration, object tracking and movement estimation, face recognition, smoke detection, texture retrieval, contour extraction, or image segmentation [14][15][16][17].

Motivation for Using Color Images Histograms
For C-means oriented algorithms, the clustering task has to compute the distance between each pixel and centers of different clusters. This task leads to a high computational complexity especially with images of higher resolution. Moreover, it is difficult to extend this idea of FCM for color image segmentation. This is due to the number of different colors which is usually close to the number of pixels in a color image. Compared to a grayscale image, the c-means clustering algorithms require a longer execution time to segment its corresponding color image. Because the histogram level is far less than the whole image pixels, the use of histogram-based features reduces the computational complexity of the clustering procedure. In [9], an enhanced FCM method for grayscale images was proposed. It is called the Spatial Fast Fuzzy C-means clustering algorithm (SFFCM). Authors demonstrate that it is faster to implement FCM on histogram-gray-levels than on pixel's intensities. This novel extension of fuzzy clustering algorithm is used in our segmentation pipeline (see Figure 1).
Sensors 2020, 20, x FOR PEER REVIEW 3 of 28 researchers have verified that multi-resolution features are able to get the main outline for various texture regions of the image [12,13]. In the last decade, the Gabor filters, firstly proposed by Dennis Gabor in 1946 in 1-D and extended, in 1985, to 2-D by Daugman, have received much attention. Their wide usage in multiple fields can be taken as proof of their success: image analysis, compression and restoration, object tracking and movement estimation, face recognition, smoke detection, texture retrieval, contour extraction, or image segmentation [14][15][16][17].

Motivation for Using Color Images Histograms
For C-means oriented algorithms, the clustering task has to compute the distance between each pixel and centers of different clusters. This task leads to a high computational complexity especially with images of higher resolution. Moreover, it is difficult to extend this idea of FCM for color image segmentation. This is due to the number of different colors which is usually close to the number of pixels in a color image. Compared to a grayscale image, the c-means clustering algorithms require a longer execution time to segment its corresponding color image. Because the histogram level is far less than the whole image pixels, the use of histogram-based features reduces the computational complexity of the clustering procedure. In [9], an enhanced FCM method for grayscale images was proposed. It is called the Spatial Fast Fuzzy C-means clustering algorithm (SFFCM). Authors demonstrate that it is faster to implement FCM on histogram-gray-levels than on pixel's intensities. This novel extension of fuzzy clustering algorithm is used in our segmentation pipeline (see Figure 1).

Fire Forest Image Application
Recently, wildfires devasted millions of hectares over the world. The lack of information about the current state and the dynamic evolution of fire plays a central role in the accidents. Nowadays the demand increases for remote monitoring of this natural disaster [2,[18][19][20]. For that, artificial visual control is a new area that has gained interest. In literature, many techniques have been developed mainly for wildfire image processing [4,8,21,22]. In real applications, for smoke and fire, there is a different useful information: area, location, direction, etc. Because the forest environment suffers from many perception field drawbacks (uncontrollable and sudden changes in environmental conditions, calibration problems, non-rigid fire-model, etc.), this study involves many advanced computer vision techniques in 2D [13] and extends them to the 3D domain [23]. Our project is divided into different research interests: image segmentation, semantic fire and smoke detection, and flame direction estimation. In this work we developed a color image segmentation technique as a part of mentioned tasks. The goal of the proposed method is to improve the segmentation performance of wildfire noisy images and to reduce the clustering computational complexity.

Fire Forest Image Application
Recently, wildfires devasted millions of hectares over the world. The lack of information about the current state and the dynamic evolution of fire plays a central role in the accidents. Nowadays the demand increases for remote monitoring of this natural disaster [2,[18][19][20]. For that, artificial visual control is a new area that has gained interest. In literature, many techniques have been developed mainly for wildfire image processing [4,8,21,22]. In real applications, for smoke and fire, there is a different useful information: area, location, direction, etc. Because the forest environment suffers from many perception field drawbacks (uncontrollable and sudden changes in environmental conditions, calibration problems, non-rigid fire-model, etc.), this study involves many advanced computer vision techniques in 2D [13] and extends them to the 3D domain [23]. Our project is divided into different research interests: image segmentation, semantic fire and smoke detection, and flame direction estimation. In this work we developed a color image segmentation technique as a part of mentioned tasks. The goal of the proposed method is to improve the segmentation performance of wildfire noisy images and to reduce the clustering computational complexity.

Methodology
The developed method is based on two principal tasks: - The Pre-segmentation, also called the Superpixels Extraction, - The Clustering of firstly extracted superpixels The framework of our proposed algorithm is shown in Figure 1.

Superpixels Extraction: An Overview
Superpixels extraction, called also pre-segmentation, is the subdivision of the input image into a number of regions. Each region is a collection of pixels with homogenous characteristics. This procedure is always used for image classification and labeling. Compared to neighboring window-based methods, it is able to provide more representative local spatial information [9].
As given by [24], superpixel algorithms are classified into two principal categories: Graph-based methods: each pixel is considered as a node in a graph. Similarities between neighboring pixels are defined as edge weights. Superpixels extraction minimizes a cost function defined over the graph. This category includes a large variety of developed methods: Normalized Cuts (NC), Homogeneous Superpixels (HS), Superpixels via Pseudo-Boolean Optimization (PB), and Entropy Rate Superpixels (ERS) [25,26].
Clustering-based methods: all image pixels are iteratively grouped until satisfying some convergence criteria. As given by [27], the most popular techniques are Simple Linear Iterative Clustering named (SLIC), Watersheds Transform (WT), Quick Shift (QS), and Turbo Pixel (TP). More details and evaluation of 15 superpixel algorithms are given in [24]. All mentioned approaches are usually considered as over-segmentation algorithms to improve the final segmentation. Referring to [9,27], in our work, we use the implementation of WT for the superpixels extraction. In the last part of experiments (Section 5.2), the SLIC is also implemented.

Gabor Filters and Their Characteristics
Image filtering based on Gabor filters is a procedure widely used for the extraction of spatially localized spectral features. The frequency and orientation representation of Gabor filters are similar to human visual system, and they have been found vital features that can be used for image segmentation [16,28]. In our project, the processed images of fire forest combine many complexities due to the higher intensity's variation and the texture geometrical diversity. To cope with complex image regions, we use a bank of filters as a multi-scale features extractor.
The Gabor filter is obtained by a Gaussian kernel function modulated by multiplying a sinusoidal plane wave. As shown in Figure 2, combining a 2D sinusoid with a Gaussian function results in a 2D Gabor filter.

Methodology
The developed method is based on two principal tasks: -The Pre-segmentation, also called the Superpixels Extraction, - The Clustering of firstly extracted superpixels The framework of our proposed algorithm is shown in Figure 1.

Superpixels Extraction: An Overview
Superpixels extraction, called also pre-segmentation, is the subdivision of the input image into a number of regions. Each region is a collection of pixels with homogenous characteristics. This procedure is always used for image classification and labeling. Compared to neighboring windowbased methods, it is able to provide more representative local spatial information [9].
As given by [24], superpixel algorithms are classified into two principal categories: Graph-based methods: each pixel is considered as a node in a graph. Similarities between neighboring pixels are defined as edge weights. Superpixels extraction minimizes a cost function defined over the graph. This category includes a large variety of developed methods: Normalized Cuts (NC), Homogeneous Superpixels (HS), Superpixels via Pseudo-Boolean Optimization (PB), and Entropy Rate Superpixels (ERS) [25,26].
Clustering-based methods: all image pixels are iteratively grouped until satisfying some convergence criteria. As given by [27], the most popular techniques are Simple Linear Iterative Clustering named (SLIC), Watersheds Transform (WT), Quick Shift (QS), and Turbo Pixel (TP). More details and evaluation of 15 superpixel algorithms are given in [24]. All mentioned approaches are usually considered as over-segmentation algorithms to improve the final segmentation. Referring to [9,27], in our work, we use the implementation of WT for the superpixels extraction. In the last part of experiments (Section 5.2), the SLIC is also implemented.

Gabor Filters and Their Characteristics
Image filtering based on Gabor filters is a procedure widely used for the extraction of spatially localized spectral features. The frequency and orientation representation of Gabor filters are similar to human visual system, and they have been found vital features that can be used for image segmentation [16,28]. In our project, the processed images of fire forest combine many complexities due to the higher intensity's variation and the texture geometrical diversity. To cope with complex image regions, we use a bank of filters as a multi-scale features extractor.
The Gabor filter is obtained by a Gaussian kernel function modulated by multiplying a sinusoidal plane wave. As shown in Figure 2, combining a 2D sinusoid with a Gaussian function results in a 2D Gabor filter.  G(x, y) = I(x, y) ⊗ g(x, y) (1) x and y are the spatial coordinates of the plane. The Gabor kernel generating g(x, y) is defined as follows: As we have shown in [29], in the spatial domain, the 2-D Gabor function is formulated by: where x = x cos θ + y sin θ y = −x sin θ + y cos θ σ is the standard deviation of the Gaussian factor that determines the size of the receptive field. The parameter λ is the wavelength and F = 1/λ the spatial frequency of the cosine factor. They are, respectively, called the preferred wavelength and preferred spatial frequency of the Gabor function. The ratio σ/λ determines the spatial frequency bandwidth and the number of parallel excitatory and inhibitory stripe zones that can be observed in the receptive field (see Figure 3).
Sensors 2020, 20, x FOR PEER REVIEW 5 of 28 ( , ) ( , ) ( , ) G x y I x y g x y = ⊗ (1) x and y are the spatial coordinates of the plane.
The Gabor kernel generating ( , ) g x y is defined as follows: As we have shown in [29], in the spatial domain, the 2-D Gabor function is formulated by: σ is the standard deviation of the Gaussian factor that determines the size of the receptive field. The parameter λ is the wavelength and 1 F / = λ the spatial frequency of the cosine factor. They are, respectively, called the preferred wavelength and preferred spatial frequency of the Gabor function. The ratio / σ λ determines the spatial frequency bandwidth and the number of parallel excitatory and inhibitory stripe zones that can be observed in the receptive field (see Figure 3). γ is a constant, called the spatial aspect ratio, that determines the ellipticity of the receptive field.
θ represents the preferred orientation of the normal of the parallel stripes of a Gabor function, ϕ is the phase offset which defines the symmetry of Gabor filter.
As an example, with a different range of frequencies , the convolution generates a Gabor feature matrix given by: The set of 3 spatial frequencies and 8 equidistant orientations is applied. Each Gabor kernel size is proportional to the wavelength value. The replication padding is used to reduce boundary artifacts. γ is a constant, called the spatial aspect ratio, that determines the ellipticity of the receptive field. θ represents the preferred orientation of the normal of the parallel stripes of a Gabor function, ϕ is the phase offset which defines the symmetry of Gabor filter.
As an example, with a different range of frequencies f k = √ 2 k (k = {1, 2, 3}) and orientations θ l = l· π 8 (l = {0, 1, . . . , 7}), the convolution generates a Gabor feature matrix given by: The set of 3 spatial frequencies and 8 equidistant orientations is applied. Each Gabor kernel size is proportional to the wavelength value. The replication padding is used to reduce boundary artifacts. For each specific pair of frequency and orientation ( f k , θ l ), the feature image size is (M × N).
In our work, only the magnitude r x i , y j is considered. r x i , y j gives the intensity variations near the object boundaries (see Figure 4).
The Gabor features are processed with L2 normalization technique. The L2 norm is performed by: g is the normalized Gabor feature image.  The Gabor features are processed with L2 normalization technique. The L2 norm is performed by; g is the normalized Gabor feature image.

Gabor Feature Reduction Based on PCA
High dimension data are extremely complex to process due to the inconsistences in the features which increase the computation time [30,31]. In our work, we only focus on the variation of frequency and orientation parameters of Gabor filters. In Figure 5, we present a convolution results of a synthetic image of sinusoids of different orientations, frequencies and magnitudes by Gabor filters of different orientations and frequencies.

Gabor Feature Reduction Based on PCA
High dimension data are extremely complex to process due to the inconsistences in the features which increase the computation time [30,31]. In our work, we only focus on the variation of frequency and orientation parameters of Gabor filters. In Figure 5, we present a convolution results of a synthetic image of sinusoids of different orientations, frequencies and magnitudes by Gabor filters of different orientations and frequencies. The Gabor features are processed with L2 normalization technique. The L2 norm is performed by; g is the normalized Gabor feature image.

Gabor Feature Reduction Based on PCA
High dimension data are extremely complex to process due to the inconsistences in the features which increase the computation time [30,31]. In our work, we only focus on the variation of frequency and orientation parameters of Gabor filters. In Figure 5, we present a convolution results of a synthetic image of sinusoids of different orientations, frequencies and magnitudes by Gabor filters of different orientations and frequencies.  Both the mentioned parameters (frequencies and orientations) generate a large feature dimension (K × L). As mentioned above, a set of K = 3 different frequencies and L = 8 orientations are considered producing 24 features for each position of the filter. This is not performant because of the redundancy of features due to correlation of the overlapping filters. Moreover, as illustrated in Figure 5, by comparing the convolution results we notice a higher sensibility of filter parametrization. Many researchers propose the use of a small bank of filters [30][31][32]. In this work, the problem of redundancy was addressed. Because its performance compared to other dimensionality reducer methods [32], we have used the PCA retaining only the most representative response of 24 outputs. It will be considered as the input of the superpixesls extraction stage (see Figure 1).

Pre-segmentation Based on Gabor-WT
The WT produces a set of basins starting with a local minimal of a gradient image and searching lines between adjacent local minima that separate catchment watersheds. As given by [33], this is a relatively fast algorithm used for images with high resolution.
For noisy image segmentation, to fulfill both regional consistency and boundary keeping simultaneously, become more and more difficult. As shown by Figure 6, the MMGR-WT, introduced in [9], causes an over-segmentation or under-segmentation because it is sensitive to added noise.
Both the mentioned parameters (frequencies and orientations) generate a large feature dimension ( ) K L × . As mentioned above, a set of 3 K = different frequencies and 8 L = orientations are considered producing 24 features for each position of the filter. This is not performant because of the redundancy of features due to correlation of the overlapping filters. Moreover, as illustrated in Figure 5, by comparing the convolution results we notice a higher sensibility of filter parametrization. Many researchers propose the use of a small bank of filters [30][31][32]. In this work, the problem of redundancy was addressed. Because its performance compared to other dimensionality reducer methods [32], we have used the PCA retaining only the most representative response of 24 outputs. It will be considered as the input of the superpixesls extraction stage (see Figure 1).

Pre-segmentation Based on Gabor-WT
The WT produces a set of basins starting with a local minimal of a gradient image and searching lines between adjacent local minima that separate catchment watersheds. As given by [33], this is a relatively fast algorithm used for images with high resolution.
For noisy image segmentation, to fulfill both regional consistency and boundary keeping simultaneously, become more and more difficult. As shown by Figure 6, the MMGR-WT, introduced in [9], causes an over-segmentation or under-segmentation because it is sensitive to added noise. Moreover, these techniques greatly depend on the accurate extraction of region boundaries. The superpixels extraction performance of these methods deteriorates when the processed regions are textured or are of high varying intensities (see Figure 7). Moreover, these techniques greatly depend on the accurate extraction of region boundaries. The superpixels extraction performance of these methods deteriorates when the processed regions are textured or are of high varying intensities (see Figure 7). As a summary of all the superpixels extraction given by Figures 6 and 7, the MMGR-WT results exhibit major limits, namely the poor boundary keeping and superpixels consistency. This is clearly noticed with noisy images and ones of textured regions (grass, trees, sand, etc.). In our work, it should be noted that Fire Forest images suffer from all the general drawbacks (noise, higher textured regions, environmental conditions, etc.). In literature, many algorithms have been introduced to avoid such issues. Major methods tend to modify the gradient output of original image. In this paper, a 2-D Gabor filtering stage is used for the enhancement of the boundaries of regions for better superpixels extraction.

Overview
A clustering divides data objects into homogeneous groups and performs a high similarity within a cluster (called compactness). Data partitioning is made according to a membership degree, in the range (0,1), which is proportional to the distance between the data and each cluster center. The partitioning result depends on the final centroid location [29]. The fuzzy oriented methods are based on the mentioned aspects and have been successfully used. For many an application, the traditional FCM clustering algorithm, firstly introduced by Bezdek, has depicted a higher performance. It is widely used for image segmentation. As an unsupervised clustering method, FCM does not need any prior knowledge about the image.
where ij u and j v are given as follows; and m is the degree of fuzziness. As a summary of all the superpixels extraction given by Figures 6 and 7, the MMGR-WT results exhibit major limits, namely the poor boundary keeping and superpixels consistency. This is clearly noticed with noisy images and ones of textured regions (grass, trees, sand, etc.). In our work, it should be noted that Fire Forest images suffer from all the general drawbacks (noise, higher textured regions, environmental conditions, etc.). In literature, many algorithms have been introduced to avoid such issues. Major methods tend to modify the gradient output of original image. In this paper, a 2-D Gabor filtering stage is used for the enhancement of the boundaries of regions for better superpixels extraction.

Overview
A clustering divides data objects into homogeneous groups and performs a high similarity within a cluster (called compactness). Data partitioning is made according to a membership degree, in the range (0,1), which is proportional to the distance between the data and each cluster center. The partitioning result depends on the final centroid location [29]. The fuzzy oriented methods are based on the mentioned aspects and have been successfully used. For many an application, the traditional FCM clustering algorithm, firstly introduced by Bezdek, has depicted a higher performance. It is widely used for image segmentation. As an unsupervised clustering method, FCM does not need any prior knowledge about the image.
Let X = {x 1 , x 2 , . . . , x n } be a color image and nc be the number of clusters. Each i th image pixel belongs to the j th cluster with a fuzzy membership degree denoted by u ij according to its distance from the cluster center v j . FCM can yield a good segmentation result by minimizing the following objective function: where u ij and v j are given as follows: and m is the degree of fuzziness.
The FCM algorithm is summarized in Algorithm 1.

The Proposed Clustering Method
Further to the time-consuming, FCM describes an image in terms of fuzzy classes. It only depends on global features. As given by Figure 8, we developed a Gabor-PCA superpixels-based method to extract the most representative local spatial information. By this, the input data to be clustered include only the subsegment levels. In our work, the proposed segmentation method has three goals. The first is to reach a higher robustness to added noise with the multiscale processing based on Gabor Filters. The second goal is the improvement of the segmentation accuracy by incorporating local features. The third, is reducing the computational complexity and time consuming by minimizing the size of data to be clustered.
In this paper, the SFFCM algorithm, firstly proposed by [9], is adopted. Adding the spatial information, the problem of fuzzily partitioning into nc clusters becomes formulated as the minimization of the objective function given by: With Med i the mean value of color pixels within the corresponding region R i of i th superpixel image given by: where: ns is the number of superpixels, 1 ≤ i ≤ ns the color level, and S i the number of pixels with color x p in R i . The new objective function incorporates the histogram information by the level's frequencies given by S i . Thereby, each color pixel in original image is replaced by the mean value Med i of the region for which was assigned. The "Med-image" is called the pre-segmented image (see Figure 8). In this paper, the SFFCM algorithm, firstly proposed by [9], is adopted. Adding the spatial information, the problem of fuzzily partitioning into nc clusters becomes formulated as the minimization of the objective function given by:  New SFFCM objective function generates two novel formulation memberships (u ij ) and centroid functions (v j ) as follows: In Algorithm 2, we show the pseudo-code of the Spatial Fast Fuzzy C-means clustering method (SFFCM). Output: clustered data (pixel groups map) 3: Begin 4: Step 1. Compute the membership matrix U by using Equation (10) 5: Step 2. Update the cluster centers v (t+1) with Equation (11) 6: Step 3. Test if v (t+1) − v (t) < ε, execute step 4; otherwise, t = t + 1 and go to step 1 7: Step 4. Output the pixels group map 8: End

Evaluation Criteria
In the last decade, several metrics have been applied to evaluate the segmentation methods [38]. The major ones focus on segmentation accuracy, superpixels compactness, regularity, coherence, and efficiency. In [24], Wang et al. divided the set of metrics into three groups: segmentation quality evaluation, superpixels quality, and the efficiency measure based on runtime. In this work, two metrics categories are considered.

a. Segmentation accuracy
To test the clustering performance, we use two metrics given in [9]. The first measures the Equality Degree (ED) between Clustered Pixels (CP) and Ground truth Prediction (GP). The second measures the Segmentation Accuracy (SA) based on the sum of correctly classified pixels. Both metrics are, respectively, given by: where, CP k is the set of pixels assigned to k th cluster and GP k the set of pixels belonging to the same class k of Ground Truth (GT). nc denotes the number of clusters. CP k ∩ GP k : the comprised of the labeled pixels AND the ground truth of the k th cluster. CP k ∪ GP k : the comprised of all pixels found in either the prediction OR the ground truth of the k th cluster.

b. Sensitivity and Specificity
These measures are based on region overlapping. Here, two aspects are considered: the matching direction and the corresponding criteria. For the sensitivity measure, the matching direction is defined as a ground truth to segmentation result directional correspondence and vice versa for the recall measure. Sensitivity (SEN) and Specificity (SPE) are formulated as follows: Measures based on (SEN) and (SPE) are commonly used for the semantic segmentation. In our work, the mentioned metrics are applied to evaluate the clustering performance for supervised topics where the number of classes and region contents are known.
For real images, cluster frequencies are unbalanced. Mentioned metrics are not appropriate for evaluating because they are biased by the dominant classes. To avoid this, we have conducted the evaluation per-class. The obtained results are averaged over the total number of classes.
For multiclass, Sensitivity and Specificity are called Average True Positive Rate (Av_TPR) and Average True Negative Rate (Av_TNR) and given by:

Experimental Results
Experimental Setting For all the experiments discussed below, the particular parameters for the compared methods are summarized in Table 1. Only Gabor filtering parametrization is detailed in [29].

Method
Pre-segmentation Classification

Gabor-WT
Structured Element SE: a disk Radius: r = 5 m = 2, ε = 10 −3 In the first experiments, the tested images are synthetic with natural textures of Smoke, Fire, Sky, Sand, and Grass. For the second, we have tested on six real images from real scene of fire forest images. For the limited length of paper, in the last experiments, we only demonstrate the robustness and segmentation performance of our proposed method on a subset of twenty images from BSDS500 and MRSC datasets.

Results on Synthetic Images
At the first part of evaluation, we test the proposed method with the WT and the SFFCM algorithm on a set of six synthetic images shown by Figure 9a. For each class, (Fire, Smoke, Grass, Sand, Sky), the selected region is chosen from a random location in the original corresponding texture. All of the used synthetic images are with regions of regular boundaries. This is more suitable to manually generate the desired segmentation (see Figure 9b).
In this experiment, two types of noise are considered: Gaussian and Salt and Pepper. The robustness of each method is tested with four different densities of each kind of mentioned noises (10%, 20%, 30%, 40%). The quantitative segmentation results on the different blurred images achieved by using our developed method with WT and the SFFCM proposed by Lei 2019 [9]. Each experiment is repeated 10 times. All the obtained results are depicted in the boxplots in Figures 10 and 11, reporting both ED and SA metrics values given by Equations (12) and (13). The graphs of boxplots arranged similarly as the map of images (SI1-SI6) given by Figure 9 (e.g., the top left boxplot corresponds to the results on image (SI1)).
At the first part of evaluation, we test the proposed method with the WT and the SFFCM algorithm on a set of six synthetic images shown by Figure 9a. For each class, (Fire, Smoke, Grass, Sand, Sky), the selected region is chosen from a random location in the original corresponding texture. All of the used synthetic images are with regions of regular boundaries. This is more suitable to manually generate the desired segmentation (see Figure 9b). In this experiment, two types of noise are considered: Gaussian and Salt and Pepper. The robustness of each method is tested with four different densities of each kind of mentioned noises (10%, 20%, 30%, 40%). The quantitative segmentation results on the different blurred images achieved by using our developed method with WT and the SFFCM proposed by Lei 2019 [9]. Each experiment is repeated 10 times. All the obtained results are depicted in the boxplots in Figures 10 and 11, reporting both ED and SA metrics values given by Equations (12) and (13). The graphs of boxplots arranged similarly as the map of images (SI1-SI6) given by Figure 9 (e.g., the top left boxplot corresponds to the results on image (SI1)). In Figures 10 and 11, the lower and the upper bounds of each boxplot represent the first and third quartiles of the distribution, respectively. The mean values of used metrics (ED, SA) are represented by a black solid line and outliers are displayed as black diamonds. We observe that there is a greater variability of the SFFCM results compared to our proposed method. Moreover, the boxplots pertaining to the proposed method results present the lowest statistical dispersion in terms of box height and number of outliers, thus implying a lower standard deviation compared to the SFFCM method. Therefore, the use of the novel method allows for considerably robust and accurate segmentation results.

Results on Real Images
In addition to synthetic images, we shall evaluate the performance of our method on natural images. We apply the proposed method to the images from real fire forest sequences to examine the segmentation performance of our approach. The test images, given by Figure 12, are with different regions (fire, forest, smoke, cloud, grass, etc.). We shall assess the segmentation accuracy according to the visual inspection because no ground truths are available.
(b) In this experiment, two types of noise are considered: Gaussian and Salt and Pepper. The robustness of each method is tested with four different densities of each kind of mentioned noises (10%, 20%, 30%, 40%). The quantitative segmentation results on the different blurred images achieved by using our developed method with WT and the SFFCM proposed by Lei 2019 [9]. Each experiment is repeated 10 times. All the obtained results are depicted in the boxplots in Figures 10 and 11, reporting both ED and SA metrics values given by Equations (12) and (13). The graphs of boxplots arranged similarly as the map of images (SI1-SI6) given by Figure 9 (e.g., the top left boxplot corresponds to the results on image (SI1)).   In Figures 10 and 11, the lower and the upper bounds of each boxplot represent the first and third quartiles of the distribution, respectively. The mean values of used metrics (ED, SA) are represented by a black solid line and outliers are displayed as black diamonds. We observe that there is a greater variability of the SFFCM results compared to our proposed method. Moreover, the boxplots pertaining to the proposed method results present the lowest statistical dispersion in terms of box height and number of outliers, thus implying a lower standard deviation compared to the SFFCM method. Therefore, the use of the novel method allows for considerably robust and accurate segmentation results.

Results on Real Images
In addition to synthetic images, we shall evaluate the performance of our method on natural images. We apply the proposed method to the images from real fire forest sequences to examine the segmentation performance of our approach. The test images, given by Figure 12, are with different regions (fire, forest, smoke, cloud, grass, etc.). We shall assess the segmentation accuracy according to the visual inspection because no ground truths are available. In Figures 10 and 11, the lower and the upper bounds of each boxplot represent the first and third quartiles of the distribution, respectively. The mean values of used metrics (ED, SA) are represented by a black solid line and outliers are displayed as black diamonds. We observe that there is a greater variability of the SFFCM results compared to our proposed method. Moreover, the boxplots pertaining to the proposed method results present the lowest statistical dispersion in terms of box height and number of outliers, thus implying a lower standard deviation compared to the SFFCM method. Therefore, the use of the novel method allows for considerably robust and accurate segmentation results.

Results on Real Images
In addition to synthetic images, we shall evaluate the performance of our method on natural images. We apply the proposed method to the images from real fire forest sequences to examine the segmentation performance of our approach. The test images, given by Figure 12, are with different regions (fire, forest, smoke, cloud, grass, etc.). We shall assess the segmentation accuracy according to the visual inspection because no ground truths are available. The difficulty of real image segmentation can be attributed to two reasons. The first is that image segmentation is a multiple solution problem. The number of clusters differs from a person to another. The second is that an image is always complex because of added noise and regions nonuniformity. The difficulty of real image segmentation can be attributed to two reasons. The first is that image segmentation is a multiple solution problem. The number of clusters differs from a person to another. The second is that an image is always complex because of added noise and regions nonuniformity.
In this study, in order to address the first mentioned difficulty, we have shared the real test images with a group of 30 of our students in order to obtain their observations about the number of different observed clusters. The obtained statistics are summarized in Figure 13. For each image, only the first three decisions with higher percentages were considered. i.e., as given by Figure 13, 53.1% of persons have considered that the image "Ima 2" is of 4 clusters, 21.9% have considered that the mentioned image is only of 3 clusters, and 15.6% observed that "Ima 2" is with 5 clusters. In our experiments, for mentioned image, we conduct the segmentation with 4, 3, and 5 clusters. All the obtained results are illustrated by Figures 14-19. Figures 14-19 show the segmentation results of the real images depicted by Figure 12 and corrupted by the salt and pepper. In this experiment, we compare the SFFCM and our proposed method for two versions. The first by using the WT and for the second, we have introduced the SLIC pre-segmentation technique. By a visual inspection, for three compared methods, we notice that the region partition is satisfying. When the noise density is added, a lower performance is achieved. This is mainly due to the fact that high density of noise affects the texture structures, leading to the input image color degradation. Added noise affects the pre-segmentation performance and yields a lower classification performance. This clearly noticed with the SFFCM algorithm compared to the proposed method with WT and SLIC.
As given by Figures 14b, 15b, 16b, 17b, 18b and 19b, for different corrupted images, the obtained results with the proposed method using WT depicts that the different regions separation is more accurate than using the SLIC. For instance, we can see that the "fire" in Figure 14b and the "smoke" in Figures 15b, 16b and 17b are accurately segmented.
In summary, the segmentation results obtained by the proposed method, using WT or SLIC, are still more satisfying. This is due to the higher robustness of the multiresolution transform based on Gabor filters and the integration of the PCA in the pre-segmentation stage. Sensors 2020, 20, x FOR PEER REVIEW 18 of 28        Figures 14-19 show the segmentation results of the real images depicted by Figure 12 and corrupted by the salt and pepper. In this experiment, we compare the SFFCM and our proposed

Application on Other Natural Images
To assess the performance of the proposed method, we further tested it on natural images from the BSDS and MSRC datasets (see Table 2). The both mentioned datasets are the most popular benchmarks and they are widely used by researchers for color image segmentation [27,39,40]. The results reported are averaged after 10 experiments and illustrated by Figure 20. proposed method using WT depicts that the different regions separation is more accurate than using the SLIC. For instance, we can see that the "fire" in Figure 14b and the "smoke" in Figures 15b-17b are accurately segmented. In summary, the segmentation results obtained by the proposed method, using WT or SLIC, are still more satisfying. This is due to the higher robustness of the multiresolution transform based on Gabor filters and the integration of the PCA in the pre-segmentation stage.

Application on Other Natural Images
To assess the performance of the proposed method, we further tested it on natural images from the BSDS and MSRC datasets (see Table 2). The both mentioned datasets are the most popular benchmarks and they are widely used by researchers for color image segmentation [27,39,40]. The results reported are averaged after 10 experiments and illustrated by Figure 20.
The selected images contain nonhomogeneous regions within the same class, and thus, grouping the superpixel regions in these cases would be a difficult task because these image blocks, which belong to the same group, are easily identified into two different groups. For instance, we can see the nonuniform texture patterns of "Trees" in images (I9, I10, I11, I13, I16, I17, I19). Nevertheless, the proposed method with WT (G-WT) reaches the higher degrees of true positive and true negative rates. This superiority is noted with original images and becomes greater for the case of blurred ones (see Figure 20). This is because our superpixel approach is based on Gabor filtering which is effective for macro texture characterization. This is was proved in our previous work [29].
The obtained results show that the segmentation with MMGR-WT proposed by Lei et al. gives the best Sensitivity and Specificity only for images with homogeneous regions. It is still with lower performance for images with textured regions of higher intensities variations (e.g., cloud, trees, grass). As a summary of all the obtained results, it is clearly noticed that the proposed is more performant for our application on fire forest images. Where, for the major cases, the different regions are with a large texture variety and higher nonhomogeneous regions (e.g., smoke, fire, trees, grass, etc. Referring to the barplots shown in Figure 20, a higher segmentation performance with the SFFCM is recorded. This superiority, of Sensitivity and Specificity, is clearly shown with original images (I1, I2, I3, I4, I6, I8, I12). For this image's subset, it can be seen that different classes are with homogenous microtexture regions. By adding the salt and pepper noise, we can notice the degradation of the SFFCM segmentation accuracy compared to the proposed method. This robustness limitations of SFFCM was previously illustrated by the boxplots (see Figures 10 and 11). Furthermore, it is clearly shown by Figure 20.
The selected images contain nonhomogeneous regions within the same class, and thus, grouping the superpixel regions in these cases would be a difficult task because these image blocks, which belong to the same group, are easily identified into two different groups. For instance, we can see the nonuniform texture patterns of "Trees" in images (I9, I10, I11, I13, I16, I17, I19). Nevertheless, the proposed method with WT (G-WT) reaches the higher degrees of true positive and true negative rates. This superiority is noted with original images and becomes greater for the case of blurred ones (see Figure 20). This is because our superpixel approach is based on Gabor filtering which is effective for macro texture characterization. This is was proved in our previous work [29].
The obtained results show that the segmentation with MMGR-WT proposed by Lei et al. gives the best Sensitivity and Specificity only for images with homogeneous regions. It is still with lower performance for images with textured regions of higher intensities variations (e.g., cloud, trees, grass). As a summary of all the obtained results, it is clearly noticed that the proposed is more performant for our application on fire forest images. Where, for the major cases, the different regions are with a large texture variety and higher nonhomogeneous regions (e.g., smoke, fire, trees, grass, etc.).

Conclusions and Future Works
Segmentation is an important topic in the image processing community. In this study, we presented an end to end framework for application in fire forest image segmentation. The proposed approach is divided into two principle stages: the pre-segmentation and the fuzzy clustering. Our main contributions are in the pre-segmentation stage. First, we have applied a multiscale transformation based on Gabor filtering to improve the superpixel extractions. Second, for the variety  Table 2. Second row: test on corrupted images with 10% Salt and Pepper noise.

Conclusions and Future Works
Segmentation is an important topic in the image processing community. In this study, we presented an end to end framework for application in fire forest image segmentation. The proposed approach is divided into two principle stages: the pre-segmentation and the fuzzy clustering. Our main contributions are in the pre-segmentation stage. First, we have applied a multiscale transformation based on Gabor filtering to improve the superpixel extractions. Second, for the variety of outputs generated by the different pairs of frequencies and orientations (24 filters), we have introduced the PCA to fulfill the dimensionality reduction. The goal is to keep only the most relevant output to improve the regional consistency at the end of presegmentation stage. The clustering is processed by the fuzzy method recently proposed by Lei et al.
The comparison results discussed above show the efficiency of the novel approach. This is clearly shown with images of nonhomogeneous regions. The robustness of the proposed method is experimentally justified by all the above segmentation results on a set of blurred images with different kinds and intensities of noise.
It is worth noting that, generally, our proposed method gives promising image segmentation performance, but it suffers from some shortcomings. First, a few parameters in the algorithm need to be selected appropriately so as to achieve satisfactory results (e.g., Gabor filter frequencies and orientation). Second, the first stage of pre-segmentation (i.e., Gabor filtering and PCA features reduction) is computationally expensive compared to the SFFCM method. Thus, it would be a future work on a fast and effective method can be used with fire forest images. Moreover, the fire and smoke are identified based on the range of color intensities. To improve the automatic fire and smoke detection, a semantic segmentation will be performed by introducing the Deep Learning techniques.

Conflicts of Interest:
The authors declare no conflict of interest.