The proposed OMRF-AP model provided a new way to introduce the anisotropic interactions between different classes into the classic MRF model for remote sensing image segmentation, which could not only consider the spatial interactions but also the class interactions with the EVPI term. To experimentally evaluate this method, synthetic texture images and various HSR remote sensing images were tested in the following experiments. Two modules were discussed in this section. First, how to set the anisotropic penalty matrix (APM) and other parameters was discussed for the OMRF-AP model. Then, comparisons between the OMRF-AP and other MRF-based methods were demonstrated with different remote sensing images.
3.1. Parameter Settings of the OMRF-AP
In the proposed OMRF-AP model, a heuristic setting approach was designed to set the APM. Namely, there were
different
,
in the APM, and their initial values were set as the default value of Equation (13). Then, the result was evaluated by the confusion matrix [
29], and the term
would be reset if the misclassification between class
and
had the maximum value during all the misclassifications in the confusion matrix. This process would continue until the accuracy rate of each class is higher than a given threshold. For instance, a SOPT5 image, located in Pingshuo, China, was tested with the threshold as 0.9 in
Figure 4a. It included the agriculture field, vegetation, and urban area, which are denoted as class 1, 2, and 3, respectively. In this tested image, there were many vegetations in the urban area. Hence, these urban areas were wrongly recognized as the vegetation in the result with the default APM value, as shown in
Figure 4d. According to the confusion matrix in
Table 1, the accuracy rate of detecting an urban area was only 29.71% (<0.9), and the rate that urban areas misclassify as the vegetation was 51.21%, which had the maximum value during all the misclassifications.
Hence, according to the heuristic setting approach,
in the APM, i.e., the class interaction between urban area and vegetation, was set with a large value to correct these misclassifications. Namely, we set different
values from 1 to 1.1 with step 0.01, their kappa coefficient and overall accuracy (OA) [
29] are shown in
Figure 5a, and some results with different
values are shown in
Figure 4e–i. As we could observe, when the
value started to increase from 1, some misclassifications of the urban area were corrected, as shown in
Figure 4e–g, and the kappa and OA indexes of the OMRF-AP method also had the significant increase during the first interval (1, 1.019), indicating that the introduction of the APM could enhance the tradition OMRF and improve the segmentation accuracy. Then, the OMRF-AP method showed an optimal stationary performance during (1.02, 1.032), as shown in
Figure 4g,h. It means that the appropriate APM value was robust in this interval. Moreover, with a further increase of
, the accuracy of the OMRF-AP would gradually reduce.
Although misclassifications between vegetation and urban areas were corrected by setting
, the accuracy rate of detecting urban area was still less than 0.9, as shown in the confusion matrix of
Table 2. It was due to that some urban areas were misclassified as the agriculture field, as shown in
Figure 4h, which also had the maximum value during all the misclassifications in the current confusion matrix. Hence, according to the heuristic setting approach,
in the APM, the class interaction between the urban area and agriculture field, was further adjusted under the condition of
. Similar to
, different
values from 1 to 1.1 were also tested with step 0.01. From
Figure 5b, we could see that two quantitative indicators firstly increased in the interval (1,1.015), then had a robust optimal performance in the interval (1.015,1.036), and finally decreased in the interval (1.036,1.1). It showed the same trend as the curves of kappa and OA of
and further improved the accuracy of the OMRF-AP model. Some results with different
values are also demonstrated in
Figure 6.
With the setting
and
, we could see that the accuracy of each class, i.e., the rate in the diagonal of the confusion matrix, was more than the threshold 0.9, as shown in
Table 3. Hence, the heuristic setting approach could stop further exploring other
,
in the APM, and the final APM was
In addition to the APM value, there were parameters of the likelihood function
and the joint distribution
in the OMRF-AP model that need to be set. Namely, the Gaussian mixture model was employed to define the likelihood function
. Its probability distribution was denoted as
Here,
and
are the mean and variance of the Gaussian distribution for the class
,
,
denotes the mean value of a pixel in region
, and
is the dimension of the observed feature. The maximum likelihood estimation [
30] could be used to evaluate these parameters. That is
For the joint distribution
, the multilevel logistic model (MLL) [
31], as shown in Equation (7), was used to define the potential function of the energy function. Similar to the discussion in literature [
17,
19], the potential parameter
of the potential function was also quite robust to the OMRF-AP model by testing different values from 0 to 50 with step 0.5, as shown in
Figure 7a. The optimal interval of
was (0,20).
Because the OMRF-AP was an object-based method, how to set the minimum region areas (MRA) of each object was another important issue. In the proposed method, the mean shift (MS) algorithm [
32,
33] was employed to get the initial over-segmented regions as the objects, and different MRA values were also used to test the robustness of MS by taking values from 1 to 400. As shown in
Figure 7b, the OMRF-AP model was very robust to the degree of the MRA. Especially, it showed good performances when the MRA took value from 20 to 400, except for the slight fluctuations around 50.
In summary, although there were many parameters in the OMRF-AP model, some could be estimated according to the statistical method, such as parameters of Gaussian distribution; some could be set according to the developed heuristic setting approach, such as the APM value; and other parameters that need to be manually set were robust to the proposed method. They worked for the following experiments, as well.
3.3. Post-Processing with Pixel-Based MRF
As mentioned in the literature [
28,
36,
37,
38], the result of the object-based method usually had the rough and blurred boundaries between different objects. The proposed OMRF-AP was an object-based method, and had a similar phenomenon in the results, as shown in
Figure 14a–c,g–i. To correct these boundaries, a feasible way is to introduce the pixel-based method with detailed information as the post-processing [
28,
36,
37]. In this section, the classic pixel-based MRF method, ICM, was employed as the post-processing module for the results of the OMRF-AP in the above experiments. Results of OMRF-AP with post-processing (OMRF-APP) are demonstrated in
Figure 14 d–f,j–l.
As we could observe, the post-processing would not change the main part of previous results and just focus on the correction of boundaries. In fact, by comparing boundaries circled in red between OMRF-AP and OMRF-APP in
Figure 14, one could see that some rough boundaries were smoothed by the post-processing, especially boundaries in
Figure 14a,i. All the quantitative indicators could be slightly improved, as well, as shown in
Table 5. Hence, the post-processing could indeed optimize the result of the OMRF-AP method. Please note that the post-processing was not a necessary step of the OMRF-AP, and this optional step was used according to the need of the application.
Because this post-processing module was developed for object-based methods, it was further tested for previous experimental results of the other two object-based comparison methods, i.e., the IRGS and OMRF. Quantitative indicators of their post-processing results are also illustrated in
Table 5, which were denoted as the IRGS-P and OMRF-P, respectively. Similar to the OMRF-APP, the post-processing module could also slightly improve the original segmentation results of these two methods. To evaluate the efficiency of the post-processing module, the computational times of both original object-based methods and their post-processing modules are illustrated in
Table 6, where the computational time of each original object-based method was the value before the plus sign, and the computational time of each post-processing module was the value after the plus sign. For instance, in
Table 6, ‘12.36 + 0.93’ of IRGS for
Figure 8 means that the computational time of original IRGS was 12.36 seconds, and the post-processing time was 0.93 seconds. In general, the post-processing module would only increase a small amount of computational time.