A Region-Based GeneSIS Segmentation Algorithm for the Classification of Remotely Sensed Images

Abstract

This paper proposes an object-based segmentation/classification scheme for remotely sensed images, based on a novel variant of the recently proposed Genetic Sequential Image Segmentation (GeneSIS) algorithm. GeneSIS segments the image in an iterative manner, whereby at each iteration a single object is extracted via a genetic-based object extraction algorithm. Contrary to the previous pixel-based GeneSIS where the candidate objects to be extracted were evaluated through the fuzzy content of their included pixels, in the newly developed region-based GeneSIS algorithm, a watershed-driven fine segmentation map is initially obtained from the original image, which serves as the basis for the forthcoming GeneSIS segmentation. Furthermore, in order to enhance the spatial search capabilities, we introduce a more descriptive encoding scheme in the object extraction algorithm, where the structural search modules are represented by polygonal shapes. Our objectives in the new framework are posed as follows: enhance the flexibility of the algorithm in extracting more flexible object shapes, assure high level classification accuracies, and reduce the execution time of the segmentation, while at the same time preserving all the inherent attributes of the GeneSIS approach. Finally, exploiting the inherent attribute of GeneSIS to produce multiple segmentations, we also propose two segmentation fusion schemes that operate on the ensemble of segmentations generated by GeneSIS. Our approaches are tested on an urban and two agricultural images. The results show that region-based GeneSIS has considerably lower computational demands compared to the pixel-based one. Furthermore, the suggested methods achieve higher classification accuracies and good segmentation maps compared to a series of existing algorithms.

1. Introduction

In recent years, the growing development and availability of satellite imagery with high spectral and spatial resolution (HSSR), poses new challenges in the field of land cover classification. An attractive method recently receiving considerable attention is the incorporation of spatial information to improve the classification results obtained by traditional pixel-based classifiers. One way to achieve this goal is to extract contextual information from fixed-window neighborhoods around pixels and incorporate it into their feature vector of spectral values. The drawback of this method is that it raises the issue of scale selection, due to the existence of structures of different sizes within the image. A more effective alternative for integrating spatial information is to perform image segmentation. Segmentation is the partitioning of the image into disjointed regions so that each region is connected and homogeneous with respect to some homogeneity criterion of interest.

Most of the existing image segmentation techniques can be distinguished into one of the following three categories [1]: clustering/feature thresholding, region growing, and edge detection. Clustering techniques operate in the spectral space, searching for significant modes in the pattern distribution [2,3]. The created clusters are then mapped back to the spatial domain to form the segmentation map. Important issues to be addressed with cluster methods are the determination of the proper number of clusters and the consideration of the spatial association of pixels, which is usually ignored. Region growing methods start usually from a pixel level and merge neighboring objects sequentially until a homogeneity criterion exceeds a user-defined threshold [4,5,6]. The selection of the termination conditions has always been a challenging task in these methods. Usually, a set of different scales are elaborated, giving rise to a hierarchy of coarser-to-finer segmentations. This multi-scale approach has been applied successfully in various remote sensing tasks [7,8,9,10,11]. Nevertheless, the problem of selecting the proper scale becomes of great importance, so that the map adequately represents all components of different classes. The approach relies on the hypothesis that ground objects of the same land-cover category exhibit similar spectral, textural, and scale characteristics throughout the image, an assumption that is not always true. Several methods now focus on the strategy of automatically finding the most suitable scales so as to avoid scale parameter selection [12,13].

Edge-based methods search for discontinuities in the image by examining the existence of local edges. The extracted edges finally enclose the created objects. The watershed transformation is the most commonly used method of this category and has been employed in various segmentation applications [14,15,16,17,18,19]. A significant limitation of watershed is its sensitivity to local variations, which typically results in severe over-segmentation of the image. For this reason, watershed is often incorporated into more sophisticated methods as a preliminary segmentation step. For instance, in [19] the initially created watershed objects are subsequently merged through graph partitioning techniques. To overcome the oversegmentation problem, markers are used in [15,16], while the authors of [17] consider a genetic algorithm to tune the segmentation parameters of the watershed algorithm.

Recently, the pixel-based Genetic Sequential Image Segmentation (GeneSIS) method has been suggested for the classification of remotely sensed images [20]. GeneSIS is a marker-driven iterative segmentation algorithm, whereby the global segmentation problem is broken down into a succession of simpler tasks, i.e., the extraction of a unique object at each iteration. GeneSIS exploits the searching capabilities of genetic algorithms (GAs) with the aim to locate spatially the proper objects to be extracted from the image. The method evaluates the fuzzy content of candidate regions, creating objects that exhibit an optimal balance locally between fuzzy coverage, consistency, and smoothness. GeneSIS exhibits a number of interesting properties such as reduced over-/under-segmentation, adaptive spatial search, and multi-scale search. Despite its high classification accuracies, two demerits of the pixel-based GeneSIS are the increased execution times required for the completion of segmentation and the fact that, under certain parameter settings, the created segments occasionally exhibit rough boundary shapes.

In this paper, we propose a region-based GeneSIS variant of our approach with enhanced capabilities. With regard to the pixel-based GeneSIS, the modifications and extensions introduced here are outlined as follows:

(1)

The original image is initially fine segmented via the watershed transform method with the goal of reducing the noise effect corrupting single pixels. The created watershed objects are now regarded as the structural elements, instead of the single pixels considered in our previous framework. The watershed segmentation map serves as the basis for the operation of GeneSIS algorithm, while the generated segments are considered as a collection of connected watersheds.

(2)

In view of the region-based representation, significant parts of the algorithm are properly redesigned, such as the marker selection and the fitness function calculation, making extensive usage of the region-adjacency graph (RAG) of the initial map. In addition, we develop a fuzzy integral-based decision fusion scheme for the labeling of watershed objects to the various classes.

(3)

At each iteration, GeneSIS evolves a population of structuring elements placed on the image, called the basic search frames (BSFs), which are continuously relocated over the generations, trying to find the best object for extraction. In a preliminary view of the region-based GeneSIS proposed herein [21], the BSFs were represented by rotating rectangles of varying size. In the new proposal, we adopt a more flexible polygonal representation of the BSFs, which enables GeneSIS to extract irregularly shaped ground structures. A special tuning operator is also designed to enhance the evolutionary search process.

(4)

Finally, exploiting the inherent property of GeneSIS to produce multiple segmentations, we propose two segmentation fusion approaches, namely, the fuzzy majority voting and a minimum spanning forest based scheme. The fusion methods combine the results of an ensemble of different segmentation maps to obtain a final single classification map.

The new algorithm, along with the two segmentation fusion schemes, is tested on two image datasets. The results show that region-based GeneSIS produces better and more robust results compared to the pixel-based GeneSIS, improving the average, the standard deviation, and the best overall classification accuracies. Moreover, the fusion methods achieve similar or higher accuracies compared to the best results obtained from an ensemble of segmentations by GeneSIS. Finally, a significant asset of the region-based GeneSIS is that, owing to the watershed representation and the RAG-based redesign, it has considerably lower computational demands with regard to pixel-based GeneSIS.

The rest of the paper is organized as follows. In Section 2, we provide a general description of the proposed scheme, whereas Section 3 focuses on the GA part of GeneSIS, the object extraction algorithm (OEA). The description of the two segmentation fusion methods follows in Section 4. Experimental results on the classification of two remotely sensed images are presented in Section 5, and the paper concludes in Section 6 with some final remarks.

2. General Configuration

The architecture of the proposed scheme is depicted in Figure 1. Initially, the watershed algorithm is applied in order to create a preliminary fine segmentation map. In addition, supervised pixel-wise classification is performed by applying the fuzzy output SVM (FO-SVM) [22]. As a result, fuzzy classification provides a set of fuzzy membership maps (FMMs), which contain the membership values of image pixels to every class. In the next stage, the fuzzy degrees of pixels contained in each watershed object are combined through the fuzzy integral fusion method with two goals: compute the fuzzy values of the watershed to the different classes, and then assign a specific class label to this object. The class-labeled connected components of watershed objects, along with their membership values, serve as inputs to the GeneSIS segmentation algorithm. The segmentation result produced by GeneSIS provides the final classification map.

Figure 1. Flowchart of the proposed scheme.

Figure 1. Flowchart of the proposed scheme.

2.1. Watershed Segmentation

Watershed transform is a morphological approach widely used in image segmentation. The image, considered as a topographic surface, is flooded from its minima and dams are built in order to prevent merging of water from different sources. Dams represent the watershed lines, enclosing the catchment basins. Watershed transformation is usually applied to a gradient image, so that catchment basins delineate the homogeneous regions of the image. Among the different approaches investigated in [14] to obtain gradient images from hyperspectral data, we use the Robust Color Morphological Gradient (RCMG) method [23]. To reduce the oversegmentation resulting from watershed, we perform an initial filtering using a 3 × 3 median filter to smooth the surface, while at the same time preserving the significant edges.

The watershed implementation presented in [24] is used in this work. As a result of the segmentation, a set $\mathcal{W}=\left\{W_i/i=1,\dots ,\text{Ω}\left(W\right)\right\}$ of watershed objects is obtained, along with the set of watershed pixels representing the edges between regions ($\text{Ω}\left(\bullet \right)$ denotes the crisp cardinality operator). The assignment of the watershed pixels to the neighboring objects is carried out as described in [14]. For each W_i, the standard vector median is computed:

$$x_{VM}^i=\underset{x\in W_i}{\arg \min }\left\{{\displaystyle \sum _{x_j\in W_i}{\Vert x-x_j\Vert }_1}\right\}$$

(1)

Every watershed pixel is then assigned to its neighboring object with the “closest median”, i.e., the object exhibiting the minimal distance between the vector median of this region and the watershed pixel vector. It should be noted that the watershed transform creates an over-segmented map, containing mostly small and compact regions that enclose a few pixels.

2.2. Pixelwise SVM Classification

Support vector machines (SVM) is a valuable classifier from machine learning that has recently attracted considerable interest in the analysis of remote sensing images. Further, it is well recognized that the availability of fuzzy degrees of pixels to the various classes provides a better description of the image context. In this work, we perform a pixel-based classification using the fuzzy output SVM approach [22]. Following the one-versus-all (OVA) decomposition strategy, we first construct an ensemble of M binary SVMs $\left\{f_1\left(x\right),\dots ,f_j\left(x\right),\dots ,f_M\left(x\right)\right\}$, where M is the number of classes and denotes the decision function of the jth classifier, trained independently to discriminate class j from the rest of the classes. Then, the method manipulates the SVM decision values, providing for each pixel $x\in ℐ$ a membership vector:

$$\mu \left(x\right)=\left[{\mu }_1\left(x\right),\mathrm{...},{\mu }_j\left(x\right),\mathrm{...},{\mu }_M\left(x\right)\right]$$

(2)

As a result, M fuzzy membership maps (FMMs) are created, each one corresponding to a particular class. These maps contain all the important information required for the different stages of our method. Hence, the fuzzy classification process can be regarded as an image transformation from spectral space to the space of membership values. Based on the above values, each pixel is assigned to a class label, following the max argument principle:

$$ℒ(x)=\underset{j=1,\mathrm{...},M}{\arg \max }\{{\mu }_j(x)\}$$

(3)

where $ℒ\left(\bullet \right)$ is the class label assignment function.

2.3. Assignment of Fuzzy Degrees to Watershed Objects

In the proposed classification scheme, we consider watershed objects as structural units, instead of pixels. Therefore, before proceeding to the image segmentation by GeneSIS, we need to determine the fuzzy content of each watershed object at the various class labels. To accomplish this task, we employ the decision fusion approach of fuzzy integral, which is defined with respect to a fuzzy measure, usually a g_λ-fuzzy measure. Fuzzy integral has been used in previous works to combine the results of multiple classifiers [25]. Here, all pixels contained in a watershed object are regarded as different and equivalent sources of fuzzy information. Then, the fuzzy degrees conveyed by pixels are combined via decision aggregation to produce the membership values of watershed objects.

Let us consider an arbitrary watershed object $W_i=\left\{x_j^i/ j=1,\dots ,\text{Ω}\left(W\right)\right\}$, where each pixel $x_j^i$ retains a vector $\mu \left(x_j^i\right)$ of fuzzy degrees. To compute the membership degree of Wi to label $k\in \left\{1,\dots ,M\right\}$, we proceed along the following steps:

(1)

The fuzzy densities $g_k^j$ represent the degree of importance of ${\mu }_k\left(x_j^i\right)$ toward the final evaluation. In our case, these densities are determined locally by considering a 3x3 neighborhood $N\left(x_j^i\right)$ of each pixel. Specifically, $g_k^j$ is defined as the fuzzy coverage of label k in $N\left(x_j^i\right)$ of the examined pixel:

$$g_k^j={\displaystyle \sum _{\begin{array}{l}x\in N(x_j^i)\\ ℒ(x)=L_k\end{array}}{\mu }_k(x)}$$

(4)

These densities are then normalized so that ${\displaystyle \sum }_{j=1}^{\text{Ω}\left(W_i\right)}{\displaystyle \sum }_{k=1}^M{g}_k^j=1$ .

(2)

Calculate the unique root $\lambda >-1$ of the equation:

$$\lambda +1={\displaystyle \prod _{j=1}^{\Omega (W_i)}\left(1+\lambda g_k^j\right)}$$

(5)

(3)

Sort the elements of $\left\{{\mu }_k\left(x_j^i\right)\right\}$ in descending order: ${\mu }_k\left(x_{j1}^i\right),\dots ,{\mu }_k\left(x_{j\text{Ω}\left(W_i\right)}^i\right)$ with ${\mu }_k\left(x_{j1}^i\right)$ denoting the highest membership value.

(4)

Sort the densities correspondingly, i.e., $g_k^{j1},\dots ,g_k^{j\text{Ω}\left(W_i\right)}$.

(5)

Set $g\left(1\right)=g_k^{j1}$ and calculate the rest of the fuzzy measures according to the following recursion:

$$g(l)=g_k^{jl}+g(l-1)+\lambda g_k^{jl}g(l-1),2\le l\le \Omega (W_i)$$

(6)

(6)

Finally, the membership value of W_i to label k is computed as:

$${\mu }_k(W_i)=\underset{l=1}{\overset{\Omega (W_i)}{\max }}\left\{\min \left\{{\mu }_k(x_{jl}^i),g(l)\right\}\right\}$$

(7)

2.4. Connected Component Labeling

Based on the above values, each watershed object is assigned to its dominant class label, following the max argument principle:

$$ℒ(W)=\underset{j=1,\mathrm{...},M}{\arg \max }\{{\mu }_j(W)\}$$

(8)

Adjacent watershed objects of the same label can now be connected to a single hyper object. This is achieved by applying a connected-component (CC) labeling algorithm. As a result, we obtain an initial segmentation map containing the set of CCs: $\mathcal{C}=\left\{C_j/j=1,\dots ,\text{Ω}\left(\mathcal{C}\right)\right\}$. Each CC shares the same label with its watershed objects: $ℒ\left(C_j\right)\in \left\{L_1,\mathrm{..},L_j,\mathrm{..},L_M\right\}$. Figure 2a illustrates the result of this stage, where three CCs containing watersheds of similar label are formed after CC labeling. The set $\mathcal{C}$ of labeled CCs obtained at the output of this stage might be considered as the final classification map of the image; therefore, it is examined as a separate classification approach in the simulation results. This map exhibits considerably lower over-segmentation compared to the pixel-wise SVM classification. Nevertheless, the result still suffers from adequate over-segmentation, mainly due to the appearance of incorrectly labeled small CCs lying in the interior of broader CCs of different label. The proposed segmentation algorithm aims at removing the above demerit, producing more homogeneous objects with reduced over-segmentation.

Figure 2. (a) An example of initial map of labeled CCs with illustration of marked watersheds, (b) Possible segmentation of the same area after GeneSIS.

Figure 2. (a) An example of initial map of labeled CCs with illustration of marked watersheds, (b) Possible segmentation of the same area after GeneSIS.

2.5. Marker Selection

After the formation of the initial CCs, we proceed to the marker selection step. Markers are confident regions of the image that should retain their label after the whole procedure. Contrary to [20], marking is now performed on watershed objects instead of pixels. The CCs to be marked are selected according to their size and their attributed fuzzy degrees. As a first step, we choose those CCs from $\mathcal{C}$ with area larger than a specified threshold ${\text{ Ω}}_{min}^C$, which approximately represents the area of the smallest region of interest we want to recognize. Next, the most reliable watershed components contained in the selected CCs should be determined. For this reason, for each of these watersheds, we consider the difference $\text{Δ}\mu \left(W\right)={\mu }_{dom}\left(W\right)-{\mu }_{comp}\left(W\right)$, where ${\mu }_{dom}\left(W\right)$ is the highest fuzzy degree in the dominant class, while ${\mu }_{comp}\left(W\right)$ denotes the highest degree associated with the most competing class. Since these two membership values are tight by FO-SVM to unity, $\text{Δ}\mu$ is an indication of the confidence of the examined watershed.

Watershed components, inside large CCs, with degree difference above a defined fuzziness threshold $\text{Δ}\mu \left(W\right)>\text{Δ}{\mu }_{th}$ are selected as markers. The value of $\text{Δ}{\mu }_{th}$ depends on the level of image uncertainty, described by the fuzzy classification output. Here, it is defined as the $median\left(\text{Δ}C\right)$, where $\text{Δ}C$ denotes the image map of differences between the dominant and the most competing class degrees for every watershed object. Highly mixed images need lower values, while those partitioned confidently take a higher one.

Concluding, highly reliable watersheds are marked, and hence they retain their label after segmentation by GeneSIS. An initial CC containing marked watersheds will be denoted in the sequel as $C_j^{\left(m\right)}$. On the other hand, mixed watersheds with small difference of degrees between the dominant and its most competing class remain unmarked, being considered as ambiguous objects. As a result, despite the original labeling obtained by the fuzzy integral stage, their label might change after GeneSIS. Figure 2a shows that some confident watersheds within the two connected components are marked (indicated with a colored circle), while the third component remained unmarked. The set of CCs is used by GeneSIS segmentation to estimate the size of components appearing in the uncovered area of the image and delineate the active areas of the population chromosomes to be extracted as objects.

2.6. Segmentation by GeneSIS

In this stage the GeneSIS algorithm is performed, adapted to operate on a region-based image representation obtained by watershed transform. Each object extracted by GeneSIS is now considered as an aggregation of connected watershed objects existing in the terrain. In this context, GeneSIS aims at fulfilling the following two objectives simultaneously. First, partition the image into larger, more homogeneous and well-shaped segments, including primarily highly confident watersheds of a specific label. Secondly, properly apportion the ambiguous (unmarked) watersheds to the neighboring segments according to spectral similarity, measured here in terms of fuzzy degrees to the various classes. This latter objective refers also to the small unmarked CCs of the initial map $\mathcal{C}$, the watersheds of which are necessarily shared with the adjacent segments. As an illustration, Figure 2b shows that two segments are finally created upon the initial map of three CCs, shown in Figure 2a.

Figure 3. Outline of GeneSIS procedure.

Figure 3. Outline of GeneSIS procedure.

An outline of the proposed segmentation algorithm is shown in Figure 3. In the following, we describe the different parts of GeneSIS. Notice that owing to the initial segmentation into watershed objects, the image is now represented by the respective RAG. Accordingly, we have considerably redesigned our algorithm to comply with this new framework. Specifically, many parts of GeneSIS make full exploitation of the node and adjacency information provided by RAG, especially those pertaining to the object extraction algorithm.

2.6.1. Iterative Object Extraction

After initialization, GeneSIS enters a sequential procedure of repeated extractions, where at each iteration t, a unique object $S_t$ is extracted. Due to the iterative nature of GeneSIS, the covered part of the image gradually increases after each iteration. Henceforth, the set of extracted segments up to iteration t will be denoted as $\mathcal{S}\left(t\right)$ with the initial condition $\mathcal{S}\left(0\right)=\varnothing$. On the other hand, the uncovered part of the image is constantly decreasing. So we need to define the set of uncovered initial CCs after iteration t, which is denoted as $RC\left(t\right)$ and is initialized to the initial CCs, $RC\left(0\right)=\mathcal{C}$.

2.6.2. Size Estimation of Uncovered Area

Given $RC\left(t-1\right)$ and prior to the object search at iteration t, we compute the mean $A_{avg}\left(t\right)$ and standard deviation $A_{std}\left(t\right)$ of the area of all spatial structures existing in the uncovered part of the image. These quantities give an approximate view of the distribution of the remaining structures’ area, thus providing an estimation of the spatial scale to be searched in the sequel. They will be used by the Object Extraction Algorithm (OEA), in order to adjust the region growing capabilities of the GA individuals and adapt the object search to the spatial characteristics of the currently uncovered area. In their calculation, we exclude insignificant CCs with area smaller than ${\text{Ω}}_{min}^C$. Finally, $A_{avg}\left(t\right)$ and $A_{std}\left(t\right)$ are updated after a fixed number of iterations (e.g., 20), in order to reduce computational demands.

2.6.3. Object Extraction Algorithm

The object extraction algorithm (OEA) is the fundamental part at each iteration, being implemented by a GA. Each individual in the population of GA represents a different object. The evolutionary process then tries to find the best possible object, by minimizing a specially designed fitness function. At the end of the GA, the elite individual contains the extracted object $S_t$, tagged along its own class label. A detailed description of OEA is provided in Section 3.

2.6.4. Adaptation of Covered and Uncovered Areas

After the extraction of $S_t$, the set of extracted segments is updated as:

$$\mathcal{S}(t)=\mathcal{S}(t-1)\cup S_t$$

(9)

At the same time, we need to update the remaining part of the image. So, the watershed components of $S_t$ are removed from the set $RC\left(t-1\right)$ and each $C_j\in RC\left(t-1\right)$ is rearranged as follows:

$$C_j\leftarrow C_j\setminus \left(C_j\cap S_t\right)$$

(10)

where $A\backslash B=\left\{x | x\in A,x\notin B\right\}$, in order to create the $RC\left(t\right)$. The iterative process terminates when a specified percentage p of the whole image has been covered (e.g., $p=90\%$).

2.6.5. Assignment of Remaining Parts

The remaining part is mainly composed of small regions of uncertain class, dispersed around the image. These regions are finally apportioned to the already extracted objects via M-HSEG [26], a marker-based region growing method. The already extracted objects are considered as markers, with the same label assigned to them after GeneSIS. In addition, the markers set contains the initially marked watersheds that have not been extracted after completion of GeneSIS segmentation. During the iterative region growing, mergers between markers of different labels are prevented. The merging process terminates when all unmarked watersheds are absorbed. The decision upon which pair of objects should be merged each time is made using a dissimilarity criterion. Since our algorithm operates on the fuzzy space of membership values instead of the spectral space, we employ the fuzzy region dissimilarity measure proposed in [19].

3. Object Extraction Algorithm

As mentioned earlier, OEA is a GA-based routine, each time searching for the best possible object to be extracted from the uncovered area of the image. Over the next subsections, we describe the main issues involved in GA, such as the individual’s encoding, the population initialization, the fitness function and the genetic operators used.

3.1. Chromosome Encoding

Each individual represents a candidate object for extraction and is associated with a so-called basic search frame (BSF). In previous versions of GeneSIS, the BSFs were represented by rotating rectangles of varying size and orientation. Here, we enhance the flexibility of the chromosomes by considering the more descriptive polygonal shape (Figure 4a). A simpler polygon representation was preferred, by applying the following constraint on the angles $A_i\widehat{C}{A}_{i+1}=2\pi /n, i=1,\dots ,n$, where n is the number of the polygon’s vertices $A_i$. Since the directions of the vertices are predetermined, the only parameters left for setting are the center of the polygon and the radians $r_i$ of the vertices. As a result, an individual of the population is encoded as a sequence of (n+2) real-coded genes:

$$O_k=\left(c_x^{(k)},c_y^{(k)},r_1^{(k)},R_2^{(k)},\mathrm{...},R_n(k)\right)$$

(11)

The above encoding enables a better representation of the polygon’s three basic properties, i.e., location, size, and shape. $\left(c_x^{\left(k\right)},c_y^{\left(k\right)}\right)$ is the polygon’s center, representing the location of the chromosome. The $r_1^{\left(k\right)}$ is the radius of the first vertex, which operates as a scale factor. Its value is restricted by an upper limit Rmax, which denotes the maximum allowable radius. The remaining vertices are coded in relation to the scale, through the ratio $R_i^{\left(k\right)}=r_i^{\left(k\right)}/r_1^{\left(k\right)}, k=1,\dots ,n$.

The vector $R^{\left(k\right)}=\left[1,R_2^{\left(k\right)},\dots ,R_n^{\left(k\right)}\right]$ determines the shape of the polygon, being independent of the scale. In order to prevent the formation of highly irregular polygons, these ratios are restricted to take values in the range [0.5, 2]. Any change in the first two genes leads to spatial relocation of the polygon, with its shape and size remaining intact. When changing the scale factor $r_1^{\left(k\right)}$, the polygon’s size is decreased or increased, while retaining its shape and location. Finally, changes in any of the rest of genes modify the polygon’s shape only, leaving its scale and location unaffected.

Figure 4. Polygon representation of BSFs. (a) Typical polygonal BSF with n = 8 vertices, and (b) illustration of watersheds inclusion by the previous BSF.

Figure 4. Polygon representation of BSFs. (a) Typical polygonal BSF with n = 8 vertices, and (b) illustration of watersheds inclusion by the previous BSF.

In the current version of GeneSIS (watershed representation), it is not straightforward to define which objects should be contained in the BSF. We choose to consider as internal those objects whose geometric centroid is included within the borders delineated by BSF. In that respect, a BSF can be viewed as a spatial loop placed somewhere over the image, which embraces a collection of adjacent watershed regions. The above procedure is facilitated by using the connectivity information provided by RAG. An illustrative example is presented in Figure 4b. The polygonal BSF of Figure 4a is now placed upon a set of watershed objects, whose centers are marked with blue color. The border line of the area composed by the internal objects is indicated in red. As can be seen, the blue-lined polygon (genotype) is different from the actual content of the chromosome (phenotype).

3.2. Population Initialization

Exploiting the information contained in $RC\left(t-1\right)$, the individuals of the initial population are placed at spatial regions covered by large and marked CCs. Particularly, in order to create $O_k$, we randomly select a marked component $C_k\in RC\left(t-1\right)$ with a probability proportional to its area. Next, we find its bounding box $BB\left(C_k\right)$ aligned orthogonally to the image axis. The center of the bounding box is chosen as the polygon’s center while the scale factor $r_1^{\left(k\right)}$ is set to half the vertical height of $BB\left(C_k\right)$. The ratios $R_i^{\left(k\right)}, k=2,\dots ,n$ are initialized randomly in the range [0.8, 1.2]. The above initialization assures that the evolutionary search will be focused mostly on large and uncovered areas.

3.3. Active Region Determination

When evaluating candidate solutions, we are particularly interested in obtaining an object the major part of which is homogeneous, i.e., it contains watersheds with high fuzzy degrees in the same class. Nevertheless, owing to the genetic evolution, an object may be located spatially in such a way that some watersheds included in the BSF are already extracted at previous invocations of the OEA, while some others are marked with a different label. To cope with this situation, an object $O_k$ is evaluated in terms of the so-called active area, denoted as $AR\left(O_k\right)$.

The determination of the active area is accomplished as follows. In the first step, we remove from $O_k$ watersheds extracted from previous calls of OEA, since our main objective is to segment currently uncovered regions of the image. Let us define the overlapping region between $O_k$ and the already extracted segments $S_t$:

$$OVE\left(O_k\right)=O_k\cap S(t-1)$$

(12)

The remaining area $O_k^{\prime }$, obtained by excluding $OVE\left(O_k\right)$ from $O_k$ is determined by:

$${O^{\prime }}_k=O_k\backslash OVE\left(O_k\right)$$

(13)

Next, we determine the dominant class label of the individual. This is decided on the basis of the fuzzy coverage of $O_k^{\prime }$ for the different classes:

$${\tilde{\Omega }}_j\left(O_k\right)={\displaystyle \sum _{\begin{array}{l}W\in O_k^{\prime }\\ ℒ(W)=L_j\end{array}}\Omega (W){\mu }_j(W)}$$

(14)

${\tilde{\Omega }}_j\left(O_k\right)$ indicates the fuzzy degree to which watersheds of class j exist in $O_k^{\prime }$. Finally, the dominant label of $O_k$ is derived via the max argument rule:

$$ℒ\left(O_k\right)=\underset{j=1,\mathrm{...},M}{\arg \max }\left\{{\tilde{\Omega }}_j\left(O_k\right)\right\}$$

(15)

Generally, the sub-area $O_k^{\prime }$ includes watersheds of the object’s class, as well as watersheds assigned to different labels. The former are regarded as positive examples (PEs), whereas the latter ones are considered as negative examples (NEs). The homogeneity property of a region dictates that $O_k^{\prime }$ should contain as many PEs as possible with strong fuzzy degrees, and a smaller portion of NEs, preferably with lower degrees to other labels. A special occasion of interest occurs when $O_k^{\prime }$ includes sections of NEs with marked watersheds inside. Let us define these sections as a set comprising the marked overlapping regions of $O_k^{\prime }$ with the uncovered CCs of different labels:

$$OVM\left(O_k\right)={\displaystyle \underset{\begin{array}{l}j=1\\ ℒ(C_j)\ne ℒ(O_k)\end{array}}{\overset{\Omega \left(RC(t-1)\right)}{\cup }}{\left(O_k^{\prime }\cap C_j\right)}^{(m)}}$$

(16)

In the following, $OVM\left(O_k\right)$ is excluded from $O_k^{\prime }$. This is explained by noticing that, based on the marker selection scheme, marked image parts are considered as large and confident regions. Thus, it seems reasonable to allow them to be absorbed by a different object at a subsequent invocation of OEA. Moreover, with this removal we avoid under-segmentation, since the object is prevented from expanding into regions that possibly have different labels. The active area of a candidate solution is now formulated as follows:

$$AR\left(O_k\right)=O_k\setminus \left(OVE(O_k)\cup OVM(O_k)\right)$$

(17)

Finally, an important requirement of our method is that the active area should be a connected component of watersheds. This constraint is imposed in order to avoid the extraction of spatially disjointed segments from a single call to the OEA. In cases where the active area is not connected, we find the component with the largest area $AR{C}_{max}^{\left(k\right)}$, and consider this component as the new active region, i.e., $AR\left(O_k\right)=AR{C}_{max}^{\left(k\right)}$.

After the previous readjustments, $AR\left(O_k\right)$ is a subset of $O_k$ and its location may differ significantly from the corresponding area of BSF. For this reason, a chromosome is repaired so as to include only the active region. Henceforth, we will consider that chromosomes have been repaired and that their active region is connected. The active area represents the useful region of an individual. Its fuzzy content is consistently employed in the computation of fitness function components, as discussed in the following.

Figure 5. Illustration of an object extraction (iteration t): (a) set of uncovered connected components RC(t – 1), (b) internal of the elite chromosome, (c) modified set of uncovered connected components RC(t).

Figure 5. Illustration of an object extraction (iteration t): (a) set of uncovered connected components RC(t – 1), (b) internal of the elite chromosome, (c) modified set of uncovered connected components RC(t).

An illustration of the chromosome evaluation process and clarification of the relevant notation is given in Figure 5 through an artificial example. Suppose we are in the t iteration of GeneSIS and the set $RC\left(t-1\right)$ of the uncovered connected components after the first (t-1) iterations is presented in Figure 5a, including three CCs from the initial map. The centers of the marked watersheds are depicted with colored circles in the various class colors, while the centers of unmarked watersheds are denoted with the symbol ⁽*⁾. Assume that the polygon appearing in the same figure represents the best chromosome attained after termination of genetic evolution. Following the rationale described in Section 3.1, the internal area of this chromosome is enclosed by the red polygonal line, demonstrated in more detail in Figure 5b. As a first step, the sections OVE (overlap with the previous extracted segments) are excluded. Next, the remaining part is used to determine the dominant class of the object. The resulting label is Class 2, as assumed from Figure 5b. In the following, we check if the chromosome is intersected with marked regions of other labels. Indeed, the section OVM is such a case and, as discussed previously, this undesirable region is excluded. The remaining part of the chromosome constitutes the active region (AR). This is the part of the chromosome that is evaluated by the fitness function and is finally extracted as segment $S_t$. After the extraction of $S_t$, the uncovered part of the image is rearranged and the new set $RC\left(t\right)$ of remaining CCs is shown in Figure 5c. Portions of the connected components of Class 2 and 3 were removed, while components of Class 1 remained intact.

3.4. Fitness Function

The determination of fitness function is of particular importance for the GA and hence the OEA. The suggested fitness function design aims at fulfilling three goals simultaneously: the extracted objects should be large, homogeneous (that is, they should not contain mixed regions of different labels), and smoothly shaped. The first two objectives are attained by means of the coverage and consistency criteria, while for the third one we devise a suitable smoothness criterion. All fitness components are computed in a fuzzy manner by manipulating the fuzzy degrees of watersheds to the various classes. Given the dominant class of $O_k$, we define the fuzzy coverage of the PEs and NEs, respectively, covered by the active area of $O_k$:

$${\tilde{\Omega }}_p\left(O_k\right)={\displaystyle \sum _{\begin{array}{l}W\in AR\left(O_k\right)\\ ℒ\left(W\right)=L_j,ℒ\left(W\right)=ℒ\left(O_k\right)\end{array}}\Omega (W){\mu }_j(W)}$$

(18)

$${\tilde{\Omega }}_n\left(O_k\right)={\displaystyle \sum _{\begin{array}{l}W\in AR\left(O_k\right)\\ ℒ\left(W\right)=L_j,ℒ\left(W\right)\ne ℒ\left(O_k\right)\end{array}}\Omega (W){\mu }_j(W)}$$

(19)

The coverage criterion promotes the extraction of large objects by maximizing the fuzzy coverage of PEs. The notion of a large object is strongly related to the size of existing components in the uncovered part of the image, and therefore differs along the various extractions of GeneSIS. In order to match the GA search to the currently available components size, we define a threshold value $A_{thr}\left(t\right)$ that is considered as an estimate of a large object’s area:

$$A_{thr}(t)=A_{avg}(t)+A_{std}(t)$$

(20)

The coverage fitness $f_{COV}\in \left[0, 1\right]$ is then defined by passing ${\tilde{\Omega }}_p\left(O_k\right)$ through the following monotonically increasing sigmoid function:

$$f_{COV}=\frac{1}{1+e^{-b\left({\tilde{\Omega }}_p\left(O_k\right)-A_{avg}(t)\right)}}$$

(21)

Parameter b controls the slope of the sigmoid; it is defined so that for the threshold value $A_{thr}\left(t\right)$ we obtain a large coverage value d (for example, $d=0.99$). Notice that objects with ${\tilde{\Omega }}_p\left(O_k\right)=A_{avg}\left(t\right)$ are assigned a fitness value $f_{COV}=0.5$, thereby being regarded as solutions of moderate quality. In addition, highly qualified solutions with $f_{COV}\cong 1.0$ are obtained for objects whose active areas fulfill the condition ${\tilde{\Omega }}_p\left(O_k\right)\ge A_{thr}\left(t\right)$. As a result, GA search is properly adapted to the scale of the uncovered area of the image, while at the same time promoting the extraction of large objects, thus avoiding oversegmentation.

Consistency serves as a measure of the region’s homogeneity, acting along an opposite direction to the coverage criterion. It prevents the continuous growth of an object and its expansion into highly mixed regions, thereby avoiding under-segmentation. Let ${\tilde{G}}_p\left(O_k\right)$ denote the cumulative degrees of NEs to the object’s label:

$${\tilde{G}}_p\left(O_k\right)={\displaystyle \sum _{\begin{array}{l}W\in AR\left(O_k\right)\\ ℒ\left(O_k\right)=L_j,ℒ\left(W\right)\ne ℒ\left(O_k\right)\end{array}}\Omega (W){\mu }_j(W)}$$

(22)

This term represents the penetration of the dominant class in NEs and is an indication of their ambiguity. NEs with higher values of ${\tilde{G}}_p$ are more consistent than those with low values of ${\tilde{G}}_p$. Finally, the consistency fitness $f_{CONS}\in \left[0, 1\right]$ is defined as follows:

$$f_{CONS}=\{\begin{array}{ll}0,\hfill & {\tilde{\Omega }}_p+{\tilde{G}}_p\le {\tilde{\Omega }}_n\hfill \\ \frac{\left({\tilde{\Omega }}_p+{\tilde{G}}_p\right)-{\tilde{\Omega }}_n}{\left({\tilde{\Omega }}_p+{\tilde{G}}_p\right)},\hfill & otherwise.\hfill \end{array}$$

(23)

A zero consistency value is assigned to those objects that cover more NEs than PEs. The fitness value then increases linearly to 1 when the number of NEs diminishes. Thereby, consistency encourages the formation of objects covering a large number of confident PEs and fewer NEs.

The third fitness component quantifies the smoothness of the object by evaluating the shape of its external borders. Objects with strongly irregular shape are penalized, to avoid the simultaneous extraction of spatially distant regions of the same label. Initially, we compute the following ratio:

$$\lambda =\frac{\Omega \left(C{H}_k\right)-\Omega \left(AR(O_k)\right)}{\Omega \left(AR(O_k)\right)}$$

(24)

where $C{H}_k$ is the convex hull of $AR\left(O_k\right)$, i.e., the smallest convex set that contains $AR\left(O_k\right)$. This quantity measures the matching degree of an object with a prototype convex shape. Objects with small λ are nearly convex ($\lambda =0$ for ideally convex) which is considered as the ideal shape of an object. To obtain normalized fitness values in the range [01], the smoothness fitness is defined as follows:

$$f_{SMO}=\frac{1}{1+a\lambda }$$

(25)

Parameter α controls the slope of the function. It is defined so that the smoothness fitness takes a large value (e.g., $f_{SMO}\left({\lambda }_{acc}\right)=0.9$) for an acceptable value ${\lambda }_{acc}$ (e.g., ${\lambda }_{acc}=0.1$). Objects with nearly convex shapes receive high $f_{SMO}$ values close to unity, while those with highly irregular shapes are penalized.

The overall fitness function is obtained by combining the above three criteria:

$$f=f_{COV}\cdot f_{CONS}\cdot f_{SMO}$$

(26)

During the initial iterations where the image is mostly uncovered, the OEA extracts large and pure objects, which fulfill both coverage and consistency criteria to a high degree. As the image is progressively segmented, the OEA spatially achieves an optimal balance between coverage (region growing) and consistency (homogeneity), while maintaining the shape of the object into acceptable limits.

3.5. Genetic Operators

We apply the BLX-α [27] crossover operator, suitable for real-coded GAs with a probability $p_c$. As regards mutation, each gene is chosen with a probability $p_m$ and assigned a random value within its domain. The mutation rate is defined as the inverse of the number of genes in the solution encoding (e.g., for $n=16$, we have $p_m\cong 0.05$). Tournament selection is used for selecting individuals to be recombined for the next generation, while elitism ensures that the fittest solution is retained during evolution. Starting from the initial population of polygons and through crossover and mutation operators, new polygons are created at each generation of the GA. Thus the search space is globally explored and via the survival of the fittest individuals, GA is led to a desirable solution. The algorithm terminates after a maximum number of iterations or when the fitness value of the best individual does not increase after a fixed number of generations.

Figure 6. Description of the elite tuning operator.

Figure 6. Description of the elite tuning operator.

In addition to the standard genetic operators, we also apply a specially designed RAG-based local tuning operator on the elite individual at each generation to improve its fitness (Figure 6). This operator is activated when, for a specific number of generations (e.g., 10), the fitness has been increased by less than 1%. Our objective with this tuning process is to assist the elite chromosome in improving its fitness value by expanding to its fruitful neighboring regions. This can be achieved mainly by considering the possible mergers with its adjacent neighbors of the same class (PEs). In this case, both the coverage and consistency criteria are increased and, therefore, it is most likely that the overall fitness will also increase. In addition, we examine the expansion to regions of NEs by considering the most ambiguous adjacent neighbor, i.e. the one with the smallest difference $\text{Δ}\mu$. In this case, the consistency criterion is decreased while the coverage one is increased. Hence, the balance between these two contradicting criteria will finally decide whether this merger is valuable or not. After the first generations, the population usually converges to a specific region, so this operator assists in quickly finding a better solution. In that respect, the local tuning operator serves as a means to boost the spatial search capabilities of OEA.

4. Segmentation Fusion

Owing to the stochastic nature of the object extraction mechanism, GeneSIS is able to produce multiple segmentations of the same image, emanating from different initializations of OEA. Exploiting this inherent property of GeneSIS, we propose two segmentation fusion schemes, where different segmentation maps are combined to decide the final class assignment for each watershed object. Thereby, we can eliminate the stochastic effect of our algorithm, since a single classification map is obtained after fusion. In addition, the effective combination of multiple segmentations can improve the classification accuracies compared to the ones provided by the individual segmentations participating in the ensemble of segmentations.

Initially, we have an ensemble of Q segmentations of the image $𝒮ℳ=\left\{S{M}^{\left(1\right)},\dots ,S{M}^{\left(Q\right)}\right\}$ after repeated runs of GeneSIS. These maps share all the properties underlying GeneSIS. Nevertheless, due to random initializations, they provide different classification results, especially for the ambiguous regions of the image. Each segmentation map $S{M}^{\left(l\right)}=\left\{V_q^{\left(l\right)}, q=1,\dots ,N_l\right\}$ is considered as the collection of objects $V_q^{\left(l\right)}$, which denote the union of connected components of all spatially adjacent watersheds of the same class label in this segmentation map.

4.1. Fuzzy Majority Voting

The configuration of the fuzzy majority voting (FMV) scheme is depicted in Figure 7a. Decision fusion in this approach is performed across the different segmentations of the ensemble on a per watershed basis, i.e., the primitive structural elements appearing in all segmentations. For each watershed, the final label assigned by FMV is determined by combining the certainty degrees of the extracted segments in the different segmentations, which enclose this watershed.

Specifically, for every $S{M}^{\left(l\right)}, l=1,\dots ,Q$, we compute the degree of certainty of $V_q^{\left(l\right)}$ to the various class labels $R_j\left(V_q^{\left(l\right)}\right)\in \left[0, 1\right]$, $j=1,\dots ,M$. This is achieved by considering the fuzzy degrees of the watershed objects:

$$R_j\left(V_q^{(\ell )}\right)=\frac{{\tilde{\Omega }}_j\left(V_q^{(\ell )}\right)}{{\displaystyle \sum _{r=1}^M{\tilde{\Omega }}_r\left(V_q^{(\ell )}\right)}}$$

(27)

${\tilde{\Omega }}_j\left(V_q^{\left(l\right)}\right)$ are computed by Equation (14) using the watersheds contained in $V_q^{\left(l\right)}$. $R_j\left(V_q^{\left(l\right)}\right)$ indicates the degree of certainty to which class j exists in $V_q^{\left(l\right)}$ (as delineated by GeneSIS), according to the information offered by SVM classifier. Highly confident objects take large certainty values in the dominant class and lower ones in the other classes. On the other hand, weakly segmented regions receive comparable certainties among the various classes. Consequently, the certainty degrees reflect the quality of a segmentation map, locally.

Figure 7. (a) Fuzzy Majority Voting Fusion; (b) MSF-based Fusion.

Figure 7. (a) Fuzzy Majority Voting Fusion; (b) MSF-based Fusion.

Next, we assume that for every $S{M}^{\left(l\right)}$, a particular watershed W in the image shares the same certainty degrees as the ones apportioned to the segment $V_q^{\left(l\right)}$ that contains it. FMV operates on a watershed basis, using the following rule:

$$ℱ(W)=\underset{j=1,\mathrm{...},M}{\arg \max }\left\{{\displaystyle \sum _{\ell =1}^Q{\displaystyle \sum _{q=1}^{N_{\ell }}IN{D}_q^{(\ell )}(W)\cdot R_j\left(V_q^{(\ell )}\right)}}\right\}$$

(28)

where

$$IN{D}_q^{(\ell )}(W)=\{\begin{array}{l}1ifW\in V_q^{(\ell )}\\ 0otherwise\end{array}$$

(29)

The class label of a watershed W is defined as the one exhibiting the largest cumulative certainty degree, across the different segmentations of the ensemble. Towards the final decision, each segmentation $S{M}^{\left(l\right)}$ votes for the various classes according to the certainties of the segments including W.

4.2. MSF-Based Fusion

The configuration of the second fusion scheme examined in this work is shown in Figure 7b. It relies on a region growing segmentation method, the minimum spanning forest (MSF) rooted from markers [28]. A similar approach has been previously applied in [29] for the pixel-based fusion of different classification maps, obtained by various segmentation algorithms.

This fusion strategy is a two-stage procedure, operating again on a watershed basis. In the first stage, the classification results obtained by the different segmentations in the ensemble are combined to select a set of confident region markers. Specifically, we select as markers those watersheds that are assigned to the same class by all segmentations $S{M}^{\left(l\right)}, l=1,\dots ,Q$, i.e. there is a full consensus between all segmentation maps on the assignment of these watershed class labels. These markers delineate the reliably classified regions of the image, each one receiving the corresponding class label.

The second stage entails the construction of a region-based MSF, rooted from the above markers. This step now undertakes the task of defining the final label assignments to the unmarked objects. Accordingly, markers start growing iteratively by absorbing at each time the most similar neighbor, according to a similarity criterion. This process continues until the entire image is covered. Upon completion of the expansion, the class of each marker is assigned to all watersheds grown from this marker.

5. Experimental Results

The proposed methodology is tested on the land cover classification of two agricultural areas (the Indiana and Koronia datasets), and an urban area (the Pavia image). The images are acquired by different sensors, with varying spatial resolution and number of bands. Because of the stochastic nature of GeneSIS, we performed 30 independent runs (segmentations) with different initializations, to obtain a robust assessment of our methodology. Table 1 shows the GeneSIS parameters commonly used in all experiments. The remaining parameters are defined below for each dataset individually. GeneSIS was coded in C++ and all experiments were conducted on an Intel Core i5-4670 at 3.4 GHz.

Table 1. Parameters used in GeneSIS.

Parameter	Value
Number of polygon nodes (n)	16
Maximum allowed radius (Rmax)	50
Image coverage for termination of GeneSIS (%)	90
Maximum number of generations (Nmax)	1000
Number of generations allowed without change (ng)	80
Population size (Np)	20
Tournament size	2
Crossover parameter α	0.5
Crossover probability (pc)	0.8
Mutation probability (pm)	0.05
Smoothing factor λacc	0.1

Table 1. Parameters used in GeneSIS.

Parameter	Value
Number of polygon nodes (n)	16
Maximum allowed radius (Rmax)	50
Image coverage for termination of GeneSIS (%)	90
Maximum number of generations (Nmax)	1000
Number of generations allowed without change (ng)	80
Population size (Np)	20
Tournament size	2
Crossover parameter α	0.5
Crossover probability (pc)	0.8
Mutation probability (pm)	0.05
Smoothing factor λacc	0.1

In the comparative analysis, we consider the pixel-based SVM classification and the results given by the initial segmentation map $\mathcal{C}$, after CC labeling of labeled watersheds (Section 2.4). We also consider the results from SVM classification after spatial postregularization (PR) to reduce the noise. The SVM map is filtered using an 8-neighborohood pixel mask and majority voting. Particularly, if more than five pixel neighbors have a class label different from the one of the considered pixel, then the pixel is reclassified to this label. The filtering is repeatedly applied until stability is reached. In addition, we test the results produced by other recently proposed segmentation-based methods from remote sensing. Specifically, we examine the CaHO[30], HSwC[31], and marker-based M-HSEG^op [26] methods. All these algorithms are extensions of HSEG [6], automatically providing a unique segmentation map from the hierarchy of multi-scale maps generated by HSEG. We choose $S_{wght}=0$ in order to avoid the merging of non-adjacent regions. In that case, HSEG is equivalent to the HSWO algorithm [4]. Finally, we consider the marker-based MSF [28], which operates on a set of labeled markers. For fair comparison, in all algorithms we used the same supervised SVM map obtained from a dataset of training instances. The markers set utilized in MSF and M-HSEG^op is the same as that employed in [20]. The dissimilarity criteria SAM, L1 used by the different methods for region merging are described in the aforementioned original works.

5.1. Indiana Image

The Indiana image is a vegetation area acquired by AVIRIS sensor over the Indian Pines site, Northern Indiana. The image has spatial dimensions of 145 × 145 pixels, a spectral range of 220 channels, and a spatial resolution of 20 m/pixel. Twenty water absorption bands have been removed [32], and the remaining 200 bands were used in the experiments. A three-band false-color composite and the reference sites are shown in Figure 8a,b, respectively. The 16 classes of interest existing in this image (mostly different types of crop) are described in Table 3. The training set is randomly selected from reference data, including 15 samples from the three smaller classes (alfalfa, grass/pasture-mowed, and oats), and 50 samples for the remaining classes. The remaining reference data comprised the test set.

Initially, watershed segmentation is performed as described in Section 2.1. The resulting map is shown in Figure 8c, where each watershed is represented by its mean spectral value on an arbitrarily chosen band (Band 120). As expected, the image is highly oversegmented, containing small, well-shaped, and compact watershed regions. This fine segmentation result forms the initial map of structural elements used as the basis for the GeneSIS operation. After the assignment of the watershed pixels to their neighboring objects, a segmentation map with 1109 initial watersheds is created.

Pixel-based classification is next performed by fuzzy output SVM using the entire space of 200 spectral bands. The RBF kernel was considered, while the optimal parameters were chosen by 5-fold cross validation: $C=512$ and $\gamma =2^{-9}$. After hardening of fuzzy degrees, we obtain the supervised classification map shown in Figure 8d. As can be seen, the majority of the fields are correctly classified. Nevertheless, there exists a strong confusion between the spectrally similar corn and soybean types, which produces many misclassifications within certain fields of the corresponding classes. Apparently, the absence of contextual information leads to a highly fragmented SVM map.

Next, we compute the membership degrees of watershed objects via decision fusion by fuzzy integral. After CC labeling we obtain map $\mathcal{C}$, shown in Figure 8e. As can be seen, although the salt and pepper effect is considerably reduced, adequate misclassifications between the spectrally mixed classes still remain. In the marker selection stage, we set ${\text{Ω}}_{min}^C=20$ as the size of structures to be marked, in order to enable GeneSIS to recognize the smallest reference field (oats). The global threshold of fuzziness $\text{Δ}{\mu }_{th}$ is set to a low value, specifically $\text{Δ}{\mu }_{th}=0.2$, due to the aforementioned spectral mixings. As a result, 714 watersheds are selected for marking.

In the following, we proceed to image segmentation by GeneSIS. A typical segmentation map obtained by GeneSIS is displayed in Figure 8f. We can notice that the extracted segments cover mostly the large and homogeneous areas of the image, achieving also a good match with the respective reference fields. It is also remarkable that GeneSIS is now able to cover a whole reference field with a single extraction, without splitting it in more segments. In addition, it should be stressed that the extracted objects appear with varying shapes and irregular boundaries. Particularly, their shapes are delineated from the boundaries of watershed objects included in the BSFs, while the polygonal representation of the chromosome facilitates the extraction of non-convex objects. These are some major differences to the pixel-based version of GeneSIS, where the delineated boundaries and the shape of the objects were strongly constrained by the rectangular shape of the BSF. Finally, an interesting property of the GeneSIS approach is that it is a marker-driven but scale-free segmentation algorithm. Specifically, for each local region, OEA automatically achieves the best compromise between coverage and consistency, according to its size and homogeneity, i.e., it adapts the segment to be extracted to the existing local scale. Hence, GeneSIS does not necessitate the prior determination of a scale parameter to control the segmentation results, contrary to other segmentation methods.

Figure 8. Indiana image: (a) three-band false color composite, (b) reference sites, (c) watershed segmentation map, (d) classification map by SVM, (e) initial segmentation via CC labeling, (f) segmentation map after GeneSIS (black areas denote the yet uncovered small regions of the image), (g) final classification map, (h) total agreement map after 30 runs, (i) classification map after FMV-fusion, and (j) classification map after MSF-based fusion.

Figure 8. Indiana image: (a) three-band false color composite, (b) reference sites, (c) watershed segmentation map, (d) classification map by SVM, (e) initial segmentation via CC labeling, (f) segmentation map after GeneSIS (black areas denote the yet uncovered small regions of the image), (g) final classification map, (h) total agreement map after 30 runs, (i) classification map after FMV-fusion, and (j) classification map after MSF-based fusion.

As a last step, the remaining components are merged to the previously extracted objects via region growing, thus obtaining the final map in Figure 8g. This map is clearly more homogeneous compared to the initial map of CCs (Figure 8e), since many of the previous misclassifications have been resolved. This can also be deduced by considering the number of connected components in the two maps. GeneSIS generated on average 70 CCs, considerably fewer than the 217 components appearing in the initial map of Figure 8e.

The maps resulting after the fusion of the 30 independent segmentations, using the proposed FMV and the MSF-based fusion methods, are depicted in Figure 8i,j, respectively. Further, Figure 8h shows the set of objects (colored areas) that are assigned to the same class by all segmentations, which are used as markers for the MSF-based fusion method. As can be noticed, these objects cover a large portion of the image, comprising mainly the large and homogeneous regions. Hence, the fusion methods operate mostly on the mixed and uncertain parts of the image (white areas in Figure 8h).

In Table 2 we compare the two GeneSIS-based approaches, i.e., the previous pixel-based GeneSIS [20] and the currently proposed region-based approach, in terms of overall accuracy (OA) and execution time. The new scheme attains better maximum and average overall accuracy, while at the same time indicating enhanced robustness, by reducing the variance of the produced accuracies. However, the main impact of the region-based representation is reflected in the execution time, since the new scheme is 62% faster than the previous one. Particularly, pixel-based GeneSIS requires 15.52 s while the region-based one needs 5.87 s, on average.

Table 2. Average OA, standard deviation of OA, maximum OA, and execution time for the two GeneSIS versions in Indiana Image.

	Average	Standard Deviation	Max	Time (s)
Pixel-based GeneSIS	93.61	0.39	94.51	15.2
Region-based GeneSIS	94.30	0.34	95.03	5.87

Table 2. Average OA, standard deviation of OA, maximum OA, and execution time for the two GeneSIS versions in Indiana Image.

	Average	Standard Deviation	Max	Time (s)
Pixel-based GeneSIS	93.61	0.39	94.51	15.2
Region-based GeneSIS	94.30	0.34	95.03	5.87

Table 3 hosts the classification results given by the fusion methods and the competing segmentation algorithms of the literature for the Indiana image. The results are evaluated by means of overall accuracy (OA), average accuracy (AA), kappa coefficient k, and class-specific accuracies. In regard to the above results, the following comments are in order. (1) First, the pixel-wise SVM classification offers by far the worst accuracy compared to the other segmentation-based classification approaches. This finding justifies the need to formulate meaningful objects to be classified, instead of handling single pixels. (2) Both pixel-based and especially the new region-based GeneSIS outperform the accuracy of the initial segmentation map $\mathcal{C}$ by 2%–3%. This improvement implies that GeneSIS further homogenizes the input map $\mathcal{C}$ by creating a smaller number of segments. On the other hand, it correctly assigns the ambiguous areas of the image, which leads to higher classification accuracies. (3) Both fusion methods achieve better results than an average GeneSIS run, since their OAs are higher than the average OA obtained from the ensemble of segmentations. Especially, the MSF-based fusion method performs slightly higher than the best accuracy attained from the ensemble of 30 segmentations by GeneSIS. This indicates that, through fusion, a significant number of the disagreements existing between the different segmentations is effectively resolved. (4) The spatial PR considerably improves the accuracy of the pixel-based SVM classification. However, both region-based GeneSIS and the two fusion schemes significantly outperform the filtered SVM results by over 5%. This is due to the fact that filtering refines the image components locally, while GeneSIS evaluates much larger areas to formulate the optimal segments. (5) Compared to the other segmentation algorithms, we observe that GeneSIS achieves higher classification performance even in terms of average OA. Moreover, using the MSF-based fusion, GeneSIS offers the highest AA, which shows its ability to sufficiently handle all the classes without underestimating any of them.

Table 3. Classification accuracies for the Indiana Image.

	SVM	FMV-Fusion	MSF-Fusion	SVM + PR	Initial Map C	CaHO	HSwC	M-HSEG°p	MSF
DC			SAM			SAM	SAM	SAM	L1
OA	76.22	94.51	95.08	89.99	91.71	93.48	92.56	93.18	93.27
AA	84.03	94.20	96.52	93.81	92.96	96.05	95.71	89.24	89.20
k	73.09	93.73	94.36	88.59	90.54	92.55	91.49	92.17	92.28
Alfalfa	87.18	92.31	92.31	100	92.31	89.74	89.74	89.74	89.74
Corn-notill	72.62	95.09	96.17	88.95	93.28	93.21	88.95	93.64	92.85
Corn-min	67.35	88.14	83.93	80.23	81.76	85.08	85.59	88.90	90.18
Corn	76.09	97.28	97.83	98.91	97.28	100	100	100	100
Grass/Pasture	92.39	96.20	96.20	95.30	96.20	96.42	96.20	96.20	94.18
Grass/Trees	95.41	99	97.99	99.43	98.71	99	99.14	96.84	100
Grass/pasture-mowed	100	100	100	100	100	100	100	100	100
Hay-windrowed	97.04	99.77	99.77	98.86	99.77	99.77	99.77	99.77	99.77
Oats	80	60	100	80	60	100	100	0	0
Soybeans-notill	77.12	99.24	95.10	93.03	96.41	98.80	99.02	82.03	82.24
Soybeans-min	58.35	88.30	93.05	80.73	82.13	90.28	88.75	94.38	94.25
Soybean-clean	84.40	97.16	97.16	95.39	96.81	95.21	95.74	96.28	96.45
Wheat	99.38	99.38	99.38	99.38	99.38	100	99.38	99.38	100
Woods	88.91	97.91	97.91	95.66	96.78	90.84	90.84	91	91
Bldg-Grass-Tree-Drives	72.73	99.70	99.70	95.15	98.79	98.48	98.18	99.70	98.79
Stone-steel towers	95.56	97.78	97.78	100	97.78	100	100	100	97.78

Table 3. Classification accuracies for the Indiana Image.

	SVM	FMV-Fusion	MSF-Fusion	SVM + PR	Initial Map C	CaHO	HSwC	M-HSEG°p	MSF
DC			SAM			SAM	SAM	SAM	L1
OA	76.22	94.51	95.08	89.99	91.71	93.48	92.56	93.18	93.27
AA	84.03	94.20	96.52	93.81	92.96	96.05	95.71	89.24	89.20
k	73.09	93.73	94.36	88.59	90.54	92.55	91.49	92.17	92.28
Alfalfa	87.18	92.31	92.31	100	92.31	89.74	89.74	89.74	89.74
Corn-notill	72.62	95.09	96.17	88.95	93.28	93.21	88.95	93.64	92.85
Corn-min	67.35	88.14	83.93	80.23	81.76	85.08	85.59	88.90	90.18
Corn	76.09	97.28	97.83	98.91	97.28	100	100	100	100
Grass/Pasture	92.39	96.20	96.20	95.30	96.20	96.42	96.20	96.20	94.18
Grass/Trees	95.41	99	97.99	99.43	98.71	99	99.14	96.84	100
Grass/pasture-mowed	100	100	100	100	100	100	100	100	100
Hay-windrowed	97.04	99.77	99.77	98.86	99.77	99.77	99.77	99.77	99.77
Oats	80	60	100	80	60	100	100	0	0
Soybeans-notill	77.12	99.24	95.10	93.03	96.41	98.80	99.02	82.03	82.24
Soybeans-min	58.35	88.30	93.05	80.73	82.13	90.28	88.75	94.38	94.25
Soybean-clean	84.40	97.16	97.16	95.39	96.81	95.21	95.74	96.28	96.45
Wheat	99.38	99.38	99.38	99.38	99.38	100	99.38	99.38	100
Woods	88.91	97.91	97.91	95.66	96.78	90.84	90.84	91	91
Bldg-Grass-Tree-Drives	72.73	99.70	99.70	95.15	98.79	98.48	98.18	99.70	98.79
Stone-steel towers	95.56	97.78	97.78	100	97.78	100	100	100	97.78

5.2. Koronia Image

Koronia image is an IKONOS bundle image acquired over a cultivated area around Lake Koronia, northern Greece. The image has four spectral channels (three visible and one near-infrared) with a spatial resolution of 4 m/pixel. Our experiments were conducted on a sub-image of $1000\times 1000$ pixels, extracted from the agricultural zone nearby the lake. Five classes of interest were identified: alfalfa, cereals, maize, orchards, and urban areas, with the first three being the major ones. The reference sites are collected after extensive field survey and photo-interpretation by the experts, in combination with high-resolution orthophotos. The training set was selected randomly from the reference data and comprises 300 samples for the first three classes and 150 for the remaining two. The rest of the reference data comprised the test set, as detailed in Table 5.

Pixel-wise SVM classification in this image is performed using an advanced space of 53 features overall, including the original four bands of the image, transformed spectral features (TSF), and textural features. Particularly, we consider the intensity (I) and hue (H) from the HIS color space, and the three data structures from Tasseled Cap transformation, suitable for vegetation representation. Furthermore, we examine 16 features from Gray-Level Co-occurrence Matrix (GLCM) and 28 wavelet features. Textural features are computed from fixed local windows around pixels of appropriate size. A detailed discussion on the derivation of the above features can be found in [33].

Figure 9. Koronia image: (a) three-band false color composite, (b) reference sites, (c) watershed segmentation map, (d) initial segmentation map after CC labeling, (e) segmentation map after GeneSIS (black areas denote the yet uncovered regions of the image), and (f) classification map after FMV-fusion.

Figure 9. Koronia image: (a) three-band false color composite, (b) reference sites, (c) watershed segmentation map, (d) initial segmentation map after CC labeling, (e) segmentation map after GeneSIS (black areas denote the yet uncovered regions of the image), and (f) classification map after FMV-fusion.

For clarity in presentation, the obtained results will be depicted on a portion of $450\times 450$ pixels of the whole study area. The three-band false color image and the reference data of this portion are shown in Figure 9a,b, respectively. The initially oversegmented map is shown in Figure 9c, where the watershed objects are depicted on Band 4. After the assignment of watershed pixels to their neighboring objects, the set of the initial structural elements comprises 43,565 watersheds. For the pixel-wise SVM classification, the optimal parameters C and γ were chosen through five-fold cross validation: $C=512$ and $\gamma =2^{-11}$. After derivation of the watershed fuzzy degrees, we obtain the map shown in Figure 9d. The visual assessment of the map shows that the majority of fields are correctly classified. However, within some large physical structures, i.e., crop fields, there exist small patches being classified erroneously. For instance, some components within certain maize fields are wrongly assigned to the alfalfa class. By observing the size of the fields existing in the study area, the marking threshold is set to ${\text{Ω}}_{min}^C=100$. In this case, the global threshold of fuzziness $\text{Δ}{\mu }_{th}$ is set to a higher value compared to the Indiana image, specifically $\text{Δ}{\mu }_{th}=0.6$, since the classification results are more precise here. As a result, 21,945 watersheds are selected for marking.

A typical segmentation after GeneSIS is shown in Figure 9e. The extracted objects follow the size and orientation of ground truth structures, especially avoiding oversegmentation of large ground components. GeneSIS generated on average 475.5 CCs, considerably smaller than the 4178 connected components existing in the initial map $\mathcal{C}$, shown in Figure 9d. Finally, the obtained map after FMV-fusion is depicted in Figure 9f.

In Table 4, we present the comparative results of the two GeneSIS variants. Similar conclusions to the Indiana image can also be drawn for this case study. For the region-based GeneSIS, we notice a small increase in average and maximum OA, while the standard deviation of the results is slightly decreased. As in the previous image, the critical asset of the region-based representation lies in the substantial reduction of the average execution time, in this case by a percentage of 57%. Table 5 summarizes the results from GeneSIS after fusion and the comparative methods. It can be seen that the proposed segmentation fusion methods exhibit similar or higher OA than the best one in the ensemble of segmentations. Both fusion methods outperform the competing methods in terms of OA and AA in the majority of the different classes. Finally, the three GeneSIS-based methods improve the results of the initial segmentation map $\mathcal{C}$ and the filtered SVM+PR results by a percentage of 2%.

Table 4. Average OA, standard deviation of OA, maximum OA, and execution time for the two GeneSIS versions in Koronia Image.

	Average	Standard Deviation	Max	Time (m)
Pixel-based GeneSIS	82.93	0.19	83.26	17.07
Region-based GeneSIS	83.53	0.12	83.74	7.40

Table 4. Average OA, standard deviation of OA, maximum OA, and execution time for the two GeneSIS versions in Koronia Image.

	Average	Standard Deviation	Max	Time (m)
Pixel-based GeneSIS	82.93	0.19	83.26	17.07
Region-based GeneSIS	83.53	0.12	83.74	7.40

Table 5. Classification accuracies for the Koronia image.

-	SVM	FMV-Fusion	MSF-Fusion	SVM + PR	Initial Map C	CaHO	HSwC	M-HSEG°p	MSF
DC			L1			SAM	SAM	SAM	L1
OA	77.37	83.87	83.59	81.48	81.28	82.30	82.49	80.52	81.44
AA	78.94	85.84	85.93	83.75	83.05	82.71	81.57	84.00	78.54
k	64.19	73.71	73.21	70.29	69.83	71.31	71.36	68.43	69.78
Alfalfa	64.78	70.01	69.40	69.49	67.18	68.58	66.09	63.95	66.47
Cereals	81.17	85.43	85.12	83.77	83.78	85.07	85.88	84.78	83.40
Maize	81.94	89.82	89.69	86.25	87.09	88.00	89.38	86.76	87.96
Orchards	80.61	93.40	94.42	88.24	92.14	93.40	91.82	92.93	65.09
Urban	86.19	90.53	91	90.99	85.03	78.47	74.69	91.58	89.80

Table 5. Classification accuracies for the Koronia image.

-	SVM	FMV-Fusion	MSF-Fusion	SVM + PR	Initial Map C	CaHO	HSwC	M-HSEG°p	MSF
DC			L1			SAM	SAM	SAM	L1
OA	77.37	83.87	83.59	81.48	81.28	82.30	82.49	80.52	81.44
AA	78.94	85.84	85.93	83.75	83.05	82.71	81.57	84.00	78.54
k	64.19	73.71	73.21	70.29	69.83	71.31	71.36	68.43	69.78
Alfalfa	64.78	70.01	69.40	69.49	67.18	68.58	66.09	63.95	66.47
Cereals	81.17	85.43	85.12	83.77	83.78	85.07	85.88	84.78	83.40
Maize	81.94	89.82	89.69	86.25	87.09	88.00	89.38	86.76	87.96
Orchards	80.61	93.40	94.42	88.24	92.14	93.40	91.82	92.93	65.09
Urban	86.19	90.53	91	90.99	85.03	78.47	74.69	91.58	89.80

5.3. University of Pavia Image

The University of Pavia image is a hyperspectral image acquired by the ROSIS-03 sensor over the University of Pavia, northern Italy. The spatial dimension of the image is $610\times 340$ and its spatial resolution is 1.3 m/pixel. The full spectral range of the initially recorded image contains 115 bands (ranging from 0.43 to 0.86 μm). The 12 most noisy channels were removed and the remaining 103 spectral bands were used in our experiments. The comparison of the two GeneSIS approaches are shown in Table 6. The nine classes of interest existing in the terrain are detailed in Table 7. For the exact number of training and test samples per class, the reader can refer to [20]. A three band true color composite and the reference sites are shown in Figure 10a,b, respectively.

Figure 10. University of Pavia image: (a) three-band false color composite, (b) reference sites, (c) watershed segmentation map, (d) initial segmentation map after CC labeling, (e) segmentation map after GeneSIS, and (f) classification map after FMV-fusion.

Figure 10. University of Pavia image: (a) three-band false color composite, (b) reference sites, (c) watershed segmentation map, (d) initial segmentation map after CC labeling, (e) segmentation map after GeneSIS, and (f) classification map after FMV-fusion.

Table 6. Average OA, standard deviation of OA, maximum OA, and execution time for the two GeneSIS versions in University of Pavia Image.

	Average	Standard Deviation	Max	Time (m)
Pixel-based GeneSIS	88.41	0.22	88.96	3.92
Region-based GeneSIS	89.86	0.65	90.95	1.07

Table 6. Average OA, standard deviation of OA, maximum OA, and execution time for the two GeneSIS versions in University of Pavia Image.

	Average	Standard Deviation	Max	Time (m)
Pixel-based GeneSIS	88.41	0.22	88.96	3.92
Region-based GeneSIS	89.86	0.65	90.95	1.07

Table 7. Classification accuracies for the University of Pavia image.

	SVM	FMV-Fusion	MSF-Fusion	SVM + PR	Initial Map C	CaHO	HSwC	M-HSEG°p	MSF
DC			SAM			SAM	SAM	SAM	SAM
OA	81	90.49	89.56	87.19	86.55	88.45	87.51	89.96	88.48
AA	88.15	94.95	95.01	92.73	93.41	94.45	93.21	95.39	93.30
k	75.74	87.59	86.43	83.46	82.70	85.07	83.87	86.97	85.11
Asphalt	76.51	94.91	93.78	89.58	90.40	93.51	90.75	97.73	97.26
Meadows	73.59	83.42	81.53	79.03	76.91	79.22	78.81	80.80	78.76
Gravel	71.35	85.51	89.20	75.37	81.10	86.39	85.40	92.29	89.48
Trees	98.70	96.46	96.09	99.52	98.21	98.73	98.52	96.91	95.54
Metal sheets	99.01	99.91	99.91	100	99.46	99.82	99.82	99.91	99.91
Bare soil	91.80	98.88	98.47	98.08	97.22	98.21	97.05	97.88	97.73
Bitumen	91.54	99.59	100	94.90	98.78	97.35	98.17	98.88	99.18
Bricks	91.14	99.26	99.11	98.19	98.72	99.20	98.81	99.79	99.85
Shadows	99.75	96.60	96.98	99.87	99.87	97.61	91.57	94.34	82.01

Table 7. Classification accuracies for the University of Pavia image.

	SVM	FMV-Fusion	MSF-Fusion	SVM + PR	Initial Map C	CaHO	HSwC	M-HSEG°p	MSF
DC			SAM			SAM	SAM	SAM	SAM
OA	81	90.49	89.56	87.19	86.55	88.45	87.51	89.96	88.48
AA	88.15	94.95	95.01	92.73	93.41	94.45	93.21	95.39	93.30
k	75.74	87.59	86.43	83.46	82.70	85.07	83.87	86.97	85.11
Asphalt	76.51	94.91	93.78	89.58	90.40	93.51	90.75	97.73	97.26
Meadows	73.59	83.42	81.53	79.03	76.91	79.22	78.81	80.80	78.76
Gravel	71.35	85.51	89.20	75.37	81.10	86.39	85.40	92.29	89.48
Trees	98.70	96.46	96.09	99.52	98.21	98.73	98.52	96.91	95.54
Metal sheets	99.01	99.91	99.91	100	99.46	99.82	99.82	99.91	99.91
Bare soil	91.80	98.88	98.47	98.08	97.22	98.21	97.05	97.88	97.73
Bitumen	91.54	99.59	100	94.90	98.78	97.35	98.17	98.88	99.18
Bricks	91.14	99.26	99.11	98.19	98.72	99.20	98.81	99.79	99.85
Shadows	99.75	96.60	96.98	99.87	99.87	97.61	91.57	94.34	82.01

The initially obtained watershed segmentation map is depicted upon Band 80 in Figure 10c. After the assignment of watershed pixels to their neighboring objects, the set of the initial structural elements comprises 9152 watersheds. The optimal parameters C and γ of the pixel-based SVM classification were chosen through five-fold cross validation: $C=8$ and $\gamma =2^{-5}$. The obtained SVM map is next combined with the watershed segmentation via the fuzzy integral approach, and the initial map $\mathcal{C}$ is obtained, as shown in Figure 10d. As can be seen, most of the class areas are correctly classified, except mainly from the large meadows region in the lower part of the image, which is confused with the trees and bare soil. This can be interpreted by observing Figure 10a, where it can easily be seen that this region is spectrally heterogeneous although it belongs to the same class. In order for GeneSIS to be able to recognize some small components of trees and shadows, the marking threshold parameter was set here to ${\text{Ω}}_{min}^C=20$. The global threshold of fuzziness $\text{Δ}{\mu }_{th}$ is set to a medium value $\text{Δ}{\mu }_{th}=0.4$, since the classification results are of moderate precision compared to the previous two paradigms. As a result, 4735 watersheds are selected for marking. Finally, in Figure 10e,f, we can see a typical segmentation after GeneSIS and the final map obtained after FMV fusion, respectively. Although most of the extracted segments follow the orientation and shape of the ground truth objects, there are still some misclassifications at the lower part of the image. This can be attributed to the erroneous marking of some initial objects, which were initially misclassified by the SVM.

The comparison of the two GeneSIS approaches, through Table 6, leads to conclusions similar to those in the previous case studies. The region-based GeneSIS exhibits higher average and maximum accuracies of about 1.5%–2%, although the standard deviation of the results is increased. The most obvious effect of the region-based representation lies again in the execution time, which is on average decreased by a percentage of 73%. Table 7 summarizes the results from GeneSIS after fusion and the comparative methods. It can be seen that the GeneSIS-based methods clearly outperform both the initial segmentation map $\mathcal{C}$ and the SVM+PR results. Noticeably, this latter method achieves high AAs for the small classes, while on the other hand it is unable to correctly classify the larger ones. Finally, the FMV-fusion method performs better than the competing methods in terms of OA and k, with the exception of M-HSEG^op, which is superior in terms of AA.

6. Discussions

The purpose of the region-based GeneSIS and the two decision fusion schemes presented in this paper is to improve the performance qualities of the previously developed pixel-based GeneSIS. The striking difference between the different algorithms lies in the decisions made on two critical issues, namely, the selection of the structural element in combination with the shape of the BSFs used by OEA. In our original proposal, we considered single pixels as the structural elements, while BSFs were represented by rectangles of varying size and orientation. As a consequence of these settings, the pixel-based GeneSIS suffers from increased computational cost and, potentially, rough description of the ambiguous areas of the image. The increased execution time is attributed to the fact that each pixel is repeatedly used for the evaluation of the fuzzy content of BSFs, i.e., the computation of the cover, the consistency, and smoothness fitness components. Hence, in view of the global search strategy followed by OEA, for larger image sizes the computational burden is considerably aggravated. Furthermore, the active areas are considered as connected components of pixels appearing within a BSF, and therefore the segments extracted by OEA are close to the rectangular shape of the BSFs. As a result, for lower values of the smoothness controlling parameter ${\lambda }_{acc}$, the segments covering the boundary regions between the different classes occasionally appear with irregular shapes, especially for more complicated landscapes. In the newly developed schemes, we provide a different design for the handling of the critical issues. On the one hand, the watershed objects generated by a fine segmentation of the initial map are regarded as the structural elements in the new framework, while on the other hand we incorporate a more constructive representation of the BSFs in the form of polygonal shapes. The above settings enhance the performance of the resulting algorithms, as summarized in the following.

6.1. Computational Cost Reduction

The initial map, which serves as the basis for the GeneSIS operation, comprises a reduced number of watershed structural units, each one carrying out their own fuzzy degrees for the various classes. For instance, the initial map of Koronia Image contains 43,565 watersheds, much smaller compared to the 10⁶ pixels of the original image. This results in a substantial saving of computational cost, reducing the execution times spent by the region-based GeneSIS by 56%–73% compared to the pixel-based variant for the images considered in our experiments. The required segmentation time may be of less attention for smaller images, but it is of particular importance when dealing with large size images. Generally, the execution time spent by region-based GeneSIS depends on the number of watersheds generated in the initial map, which in turn is related to the image size and content. Particularly, the Indiana image is a small image with 1109 watersheds requiring 5.87 s on average, while the Koronia image with 43,565 watersheds needs a much larger time of 7.40 m for the segmentation task.

6.2. Classification Accuracy

While accomplishing considerable algorithmic cost savings, the region-based GeneSIS achieves also better classification accuracies compared to the pixel-based algorithm. Specifically, it improves both the average OA and the best OA attained from an ensemble of different segmentations, by a factor of approximately 1%, for all images elaborated. In addition, the new algorithm is proved to be more robust, since it reduces to some extent the standard deviation of OAs with the exception of Pavia Image. It should be noticed that GeneSIS is an evolutionary segmentation algorithm, producing different segmentation results for different population initializations. The measures cited in Table 2, Table 4 and Table 6 show that a sample map obtained by a single GeneSIS trial statistically achieves the average OAs, while all trials appear with small deviations around these values.

In order to alleviate the stochastic effect of GeneSIS, we also introduce in this paper two segmentation fusion schemes (Section 4), both operating on the watershed basis. In the former configuration, an ensemble of different segmentations is aggregated through a fuzzy majority voting rule, while in the latter the segmentation ensemble is combined to create a map of reliable region markers, which is subsequently used by MSF to complete the final classification map. Given a segmentation ensemble by GeneSIS, the above fusion schemes provide a single classification map. The results in Table 3, Table 5 and Table 7 reveal that the fusion methods produce similar or higher classification accuracies compared to the best accuracies attained by GeneSIS in the segmentation ensemble.

The results offered by our methods are favorably compared with the ones given by the initial map $\mathcal{C}$ after CC labeling of watersheds. Specifically, the aforementioned Tables indicate that both GeneSIS and the fusion approaches outperform the accuracies of the initial maps by 2%–4%. This implies that GeneSIS reduces considerably the oversegmentation effect appearing in $\mathcal{C}$, thus generating larger and more homogenous segments. Finally, our methods outperform the pixel-based SVM classifications, without or after applying spatial filtering. According to the spatial mask being used, filtering homogenizes the objects locally, reassigning the erroneously classified pixels in the maps. Using filtering with 8-neighborhood of pixels actually produces similar results to the ones given by the initial map $\mathcal{C}$. As expected, the improvement over the SVM classifications alone is prominent, an observation advocating the usage of object-based classification. Nevertheless, the proposed approaches also outperform the filtered maps by 2%–5% for the different images examined. This is attributed to the fact that, contrary to the limited observation scale of filtering, GeneSIS evaluates broader areas of the image, namely, all watershed objects contained in the BSFs, and thus is able to formulate more compact segments fitting to the ground truth structures.

Table 8. Summary of pair-wise classification comparisons using the McNemar test. Our region-based GeneSIS and the two fusion schemes (rows) are compared to the pixel-based GeneSIS and six other methods (columns). One-sided tests are performed with 5% level of significance.

-	Methods	Pixel-Based GeneSIS	SVM + PR	Initial Map C	CaHO	HSwC	M-HSEG°p		MSF
Indiana	Region-based GeneSIS	2.24	17.46	16.10	6.48	9.57	6.86	6.61
	FMV-fusion	0.00	15.44	13.91	4.34	7.39	5.06	4.78
	MSF-fusion	2.58	17.04	13.76	6.89	9.85	7.77	7.41
Koronia	Region-based GeneSIS	27.03	69.25	88.89	49.54	42.77	86.27	64.41
	FMV-fusion	30.98	73.89	98.92	32.67	46.95	90.63	68.31
	MSF-fusion	22.63	63.03	74.45	24.61	41.51	85.37	62.70
Pavia	Region-based GeneSIS	15.61	26.59	32.37	18.58	24.86	7.86	17.91
	FMV-fusion	12.53	24.38	31.74	16.11	22.67	4.56	15.39
	MSF-fusion	4.96	17.84	23.01	8.66	15.63	−3.40	8.61

Table 8. Summary of pair-wise classification comparisons using the McNemar test. Our region-based GeneSIS and the two fusion schemes (rows) are compared to the pixel-based GeneSIS and six other methods (columns). One-sided tests are performed with 5% level of significance.

-	Methods	Pixel-Based GeneSIS	SVM + PR	Initial Map C	CaHO	HSwC	M-HSEG°p		MSF
Indiana	Region-based GeneSIS	2.24	17.46	16.10	6.48	9.57	6.86	6.61
	FMV-fusion	0.00	15.44	13.91	4.34	7.39	5.06	4.78
	MSF-fusion	2.58	17.04	13.76	6.89	9.85	7.77	7.41
Koronia	Region-based GeneSIS	27.03	69.25	88.89	49.54	42.77	86.27	64.41
	FMV-fusion	30.98	73.89	98.92	32.67	46.95	90.63	68.31
	MSF-fusion	22.63	63.03	74.45	24.61	41.51	85.37	62.70
Pavia	Region-based GeneSIS	15.61	26.59	32.37	18.58	24.86	7.86	17.91
	FMV-fusion	12.53	24.38	31.74	16.11	22.67	4.56	15.39
	MSF-fusion	4.96	17.84	23.01	8.66	15.63	−3.40	8.61

Finally, Table 8 summarizes the statistical significance of the results reported in Section 5. For this purpose, we use the directional (one sided) McNemar test with 5% level of significance. The table hosts the values of the z statistic, comparing in a pair-wise basis the three suggested methods against all competing algorithms. It can be seen that for all datasets and for all comparisons, the statistic takes much larger values compared to the critical value $z>z_{0.05}=1.65$, which verifies the superiority of our approaches. An exception appears for the Pavia Image, where M-HSEG method dominates the MSF-based fusion scheme.

6.3. Classification Map Quality

The properties of the segmentation/classification maps resulting from the new methods emanate from the following four factors. First, the preliminary watershed segmentation into small and well-shaped objects eliminates the corrupting noise usually pertaining to pixel-wise representations. Watersheds retain their own boundaries along with their fuzzy degrees assigned to them by Fuzzy integral aggregation (Section 2.3). Secondly, the polygonal representation of chromosomes allows the BSFs to take a great variety of flexible forms with different locations, scales, and shapes. Third, our choice to define a watershed as containing a BSF as soon as its geometric centroid falls within the area delineated by the BSF provides additional flexibility to the OEA since it decouples the polygon shape (genotype) from the shape of the actual solution of the chromosome (phenotype). The shapes of the extracted objects are now defined by the borders of the watersheds contained in a BSF, and hence, they do not necessarily comply with the BSF shape. The synergetic effect of the above tools renders region-based GeneSIS capable of generating arbitrary shaped and homogeneous segments with a good fit to the reference sites. This is demonstrated, for instance, by comparing the map created by GeneSIS (Figure 8g) to the reference map (Figure 8b) for the Indiana Image. The soybean no-till and soybean min-till segments in the middle of the image are, among others, indicative cases showing the creation of large and smooth segments of varying shapes and good matching. Notice that the creation of large segments covering the whole reference field by one OEA extraction is due to the effect of the fourth contributing factor, namely the incorporation of the tuning operator (Section 3.5), which adjusts the elite solutions by merging adjacent watersheds of the same class. Finally, observing the map of Figure 8j, it can be seen that the already good results attained by GeneSIS are improved when using the best performing MSF-based fusion scheme, further improving the degree of matching. Similar conclusions can also be drawn for the rest of the images examined. Generally, the level of accuracy is closely reflected in the quality of the resulting classification maps.

6.4. Parametric Robustness

The parameter set used by GeneSIS can be distinguished into four groups. The first group includes the GA parameters $\left(N_{max},n_g,N_p,a,p_c,p_m\right)$ used to control the population evolution for each invocation of OEA. The second group includes the parameters involved in the chromosome encoding, namely, the number of polygon nodes (n) and the max allowable radius ($R_{max}$). The third group contains the parameter ${\lambda }_{acc}$ used in the calculation of the smoothness component $f_{SMO}$. Finally, the fourth group includes parameters ${\text{Ω}}_{min}^C$ and $\text{Δ}{\mu }_{th}$ pertaining to the marker selection module. The majority of parameters receive common values for all test cases considered (Table 1), with the exception of ${\text{Ω}}_{min}^C$ and $\text{Δ}{\mu }_{th}$, which are adapted to each image individually.

The parameters of the first group take typical values from GA literature, taking into consideration the search requirements of the optimization task undertaken by OEA. Preliminary experimentation with different settings shows that they have negligible impact on the results. In the following we examine the influence of n and $R_{max}$ in the second group. Specifically, Table 9 shows the results for different values of polygon nodes, considering the smaller Indiana image and the larger Koronia image. For the evaluation of different choices, we also incorporate the corresponding average execution times required by GeneSIS. It can be seen that all accuracy measures remain practically intact for different number of nodes and for both images. Nevertheless, increasing n has a reasonably adverse effect on the computational cost. Specifically, larger values of nodes require considerably higher execution times, this being more prominent for the Koronia image. The proposed rule is that n should take a suitable value according to the image demands, compromising between the search flexibility and the computational resource savings. On the one hand, n should be adequately high so that OEA is able to locate irregularly shaped components in the terrain, thus providing acceptable classification accuracies. On the other hand, for larger image sizes, n should take a relatively moderate value when the segmentation times are an issue.

Table 9. Classification accuracies and execution times obtained by region-based GeneSIS for varying values of the number of polygon nodes. The experiments refer to the Indiana and Koronia images. The results corresponding to the selected parameter values are shown in bold.

-	Indiana				Koronia
Polygon Nodes ( n)	8	16	24	32	8	16	24	32
OAavg	93.88	94.30	94.95	94.82	83.31	83.53	83.55	83.65
OAbest	94.62	95.03	96.10	95.53	83.48	83.74	83.73	83.84
AAavg	93.75	94.13	94.25	94.22	85.32	85.57	85.64	85.68
AAbest	94.31	96.39	94.90	94.69	85.65	86.13	85.92	85.99
Time	5.71 (s)	5.87 (s)	5.70 (s)	5.69(s)	4.62 (m)	7.40 (m)	9.45 (m)	11.05 (m)

Table 9. Classification accuracies and execution times obtained by region-based GeneSIS for varying values of the number of polygon nodes. The experiments refer to the Indiana and Koronia images. The results corresponding to the selected parameter values are shown in bold.

-	Indiana				Koronia
Polygon Nodes ( n)	8	16	24	32	8	16	24	32
OAavg	93.88	94.30	94.95	94.82	83.31	83.53	83.55	83.65
OAbest	94.62	95.03	96.10	95.53	83.48	83.74	83.73	83.84
AAavg	93.75	94.13	94.25	94.22	85.32	85.57	85.64	85.68
AAbest	94.31	96.39	94.90	94.69	85.65	86.13	85.92	85.99
Time	5.71 (s)	5.87 (s)	5.70 (s)	5.69(s)	4.62 (m)	7.40 (m)	9.45 (m)	11.05 (m)

In Table 10, we can see the results for varying values of $R_{max}$ around the typical value ($R_{max}=50$) selected in the experiments for the Indiana image. The results are evaluated in terms of OAs and AAs, considering both the average and the best records in the ensemble of 30 segmentations. In this way, we are able to assess the behavior of GeneSIS more robustly. The results show that $R_{max}$ has insignificant influence on all accuracy measures. Table 10 also shows the results for the parameter ${\lambda }_{acc}$, which affects the smoothness of the extracted segments by evaluating the shape of their external borders. It can be noticed that different values of this parameter around the typical ${\lambda }_{acc}=0.1$ again have no significant influence on the obtained accuracies. Finally, the parameters of the fourth group are selected according to the image content. ${\text{Ω}}_{min}^C$ is taken as the smallest reference components to be recognized, while $\text{Δ}{\mu }_{th}$ is set as the median of the map, comprising the differences of fuzzy degrees in the dominant and the most competing classes, respectively. Concluding, the previous analysis indicates that, with the exception of number of nodes, the proposed region-based GeneSIS is mostly insensitive to parameter settings.

Table 10. Classification accuracies by region-based GeneSIS for varying values of R_max and λ_acc in the Indiana dataset. The results corresponding to the selected parameter values are shown in bold.

	λacc				Rmax
	0.1	0.3	0.5	0.7	30	40	50	60	70
OAavg	94.30	94.37	94.38	94.38	94.30	94.39	94.30	94.36	94.40
OAbest	95.05	95.02	95.70	95.11	95.05	95.21	95.03	94.91	95.50
AAavg	94.13	94.07	94.01	94.03	94.04	94.11	94.13	94.07	94.13
AAbest	96.39	94.56	94.73	94.40	94.32	94.63	96.39	94.26	94.69

Table 10. Classification accuracies by region-based GeneSIS for varying values of R_max and λ_acc in the Indiana dataset. The results corresponding to the selected parameter values are shown in bold.

	λacc				Rmax
	0.1	0.3	0.5	0.7	30	40	50	60	70
OAavg	94.30	94.37	94.38	94.38	94.30	94.39	94.30	94.36	94.40
OAbest	95.05	95.02	95.70	95.11	95.05	95.21	95.03	94.91	95.50
AAavg	94.13	94.07	94.01	94.03	94.04	94.11	94.13	94.07	94.13
AAbest	96.39	94.56	94.73	94.40	94.32	94.63	96.39	94.26	94.69

7. Conclusions

A novel version of the GeneSIS algorithm is presented in this paper, where the main segmentation is performed on an initial region-based map of the image acquired via watershed transform. The evolutionary part of GeneSIS is also enhanced by considering the more descriptive polygonal shape in the chromosomes’ encoding. As a final step, two fusion schemes are applied so as to overcome the stochasticity effect of our algorithm. The effectiveness of the proposed scheme is validated on the classification of three remote sensing images. Comparing to the pixel-based version, the execution time of region-based GeneSIS is considerably reduced in all test cases. At the same time, higher average accuracies are exhibited, indicating enhancement of the method’s robustness. Moreover, more arbitrarily shaped objects are obtained, since their shapes are now formed by the boundaries of the watershed objects included in the BSFs. The incorporation of the polygon in chromosomes’ representation also enables the extraction of more irregular and non-convex structures. Finally, both fusion methods attain accuracies similar to the best from the ensemble of different segmentations.