In remote sensing image analysis, traditional per-pixel and emerging object-based approaches exist. In a per-pixel method, single pixels are assigned to different geographic classes based on their reflectance values from different spectral bands without employing additional and potentially useful spatial, geometrical or contextual features [1
]. The per-pixel approaches operates at the spatial scale of the pixel. This poses the problem of mixed pixels, where a pixel represents more than one type of image objects. The use of a per-pixel approach is diminishing with the increase in resolution of satellite images. One of the reasons is the “salt and pepper” effect where single pixels are misclassified in a group of pixels representing certain class [2
There has been a shift from per-pixel to object-based methods [4
]. Geographic Object-Based Image Analysis (GEOBIA) combines contiguous homogeneous pixels to segment geographical objects in remote sensing images. The grouping of a pixel is carried out using segmentation algorithms [6
]. After segmentation, feature values of the segmented image objects are extracted. Applying these feature values, rule sets are developed to classify image objects into classification categories. In contrast to per-pixel methods, object-based methods can be applied at multiple scales and make use of spatial, contextual, and textural features alongside spectral features for image object classification.
The GEOBIA approach relies on expert knowledge to perform image segmentation and classification [5
]. Such expert knowledge, if systematically organised, can guide future image analysis [8
]. However, GEOBIA methods currently lack a systematic method to conceptualise and formalise domain knowledge. With a high dependency on human experts and a lack of formal knowledge, GEOBIA processes are highly subjective and largely irreproducible [9
]. Image object extraction and classification can be biased by human subjectivity and vary depending on the users’ capabilities and experiences [8
]. With knowledge formalisation, the classification process is prescribed and automated with less human intervention and using consensus knowledge approved and developed by domain experts.
Knowledge representation languages can be used to formalise a domain expert’s knowledge and reduce the issues of subjectivity, automation and transferability. A knowledge representation approach can employ logic-based formalism or non-logic-based representation. In a logic-based approach, the representation language is usually a variant of first-order predicate calculus and reasoning is based on formal verification of logical consequences [10
], while, in a non-logical approach, the knowledge is represented using specialised data structures and reasoning is performed by applying specialised procedures on the structures [10
]. A semantic network represents a non-logical approach with a specialised network structure in which a graphical network of nodes and arcs is used for knowledge representation. The graphical representation shows the semantic relations between concepts, which can be used to create and share the knowledge of thematic experts [11
]. Ontology, as a logic-based approach, has well-defined and formal semantics to represent knowledge.
In our work, we have used ontology as a knowledge representation language that provides a representation vocabulary specialised to a certain domain or subject. The highly cited definition of ontology from Gruber [12
] states that an ontology is a “specification of a shared conceptualisation”. Ontology is a shared understanding of a domain agreed by experts and intended to make domain knowledge interoperable, reusable and sareable [13
]. The Web Ontology Language 2 (OWL 2) [14
] is a machine-readable knowledge representation language for authoring and sharing ontologies. The logical consistency of an ontology can be established using reasoning engines such as Pellet [15
], Fact++ [16
] and KAON2 [17
]. The Semantic Web Rules Language (SWRL) [18
] allows the creation of conditional rules supplementing the capability of the reasoner. SWRL are executed using the semantic reasoner, which discovers new entailments and incorporates them into an existing ontology.
In this work, we attempt to develop an ontological framework where domain knowledge is formalised to assist in rule-based classification. For this, the domain ontology is constructed in OWL language and the rules are written in SWRL. Using a semantic reasoner, the instances of segmented image objects are assigned to concepts representing domain entities to perform an image object classification.
In remote sensing, classification of image objects is highly dependent on the knowledge of domain experts. However, the use of that knowledge is limited when the knowledge is not formalised because the knowledge becomes incomprehensible and unsharable [19
]. There is a need for a knowledge organisation and representation method to handle the inefficient and excessive dependency on expert knowledge [20
]. Concisely, GEOBIA needs to adopt knowledge formalisation techniques that can reduce human involvement and bring transferability to image classification. The use of ontology for formalising expert knowledge has been explored in [19
]. In GEOBIA, the ontological framework helps in data discovery, automatic image interpretation, data interoperability, workflow management and data publication [9
]. Previous studies show that the ontological framework have been developed for land cover extraction [20
], ocean image classification [23
], and biodiversity monitoring [24
In this paper, we aim to answer the following research questions:
Can ontology be used to formalise domain knowledge in the manner required by GEOBIA for image classification, and
What methodological changes are required to apply the formalised knowledge as a spatial ontology in object-based image classification?
This paper is a methodological contribution in the field of geo-spatial image analysis with a focus on ontological implementation. The specific contributions of the formalised ontology in this study are:
transferability of knowledge with modularisation of highly transferable domain ontology and data dependent feature ontology;
extensibility of the knowledge base with a clear separation of knowledge construction and classification tasks;
minimisation of human intervention during image classification by developing prior rules with the consensus of experts;
use of inferencing capability in image classification with reasoner (Pellet); and
data interoperability with the use of W3C (World Wide Web Consortium) standard languages (OWL, SWRL).
The remainder of this paper is organised as follows. Section 2
outlines the case study on landslides, data and study area. In Section 3
, the methodology for ontology-based image classification is proposed. In Section 4
, the outcome of the experiment is presented and later discussed in Section 5
. In Section 6
, conclusion of this work is presented.
5.1. Ontological Approach
The adoption of an ontological approach in image classification considers construction of the knowledge base using ontology and defining rules as its basis for image object identification. These rules are developed using high-level information derived from domain experts and low-level feature value extracted from remote sensing data. The ontology-based methodology has both strengths and limitations, which are discussed below.
The use of a knowledge representation language such as OWL for constructing formalised domain knowledge makes it more shareable and extensible [21
]. The modular approach treats the knowledge building process as a separate module from the image analysis process. This benefits domain experts, who are able to augment new knowledge at any moment without interrupting the classification process. Furthermore, a bias-free knowledge developed through a consensus of domain experts will be used in image interpretation. Incorporating human expert knowledge as ontology decreases human intervention at the time of image classification process. This helps GEOBIA to evolve towards becoming a more automated process.
The ontology-based framework supports and improves spatial data interoperability. The use of open standard formats, such as OWL and SWRL, in this framework bring both syntactic and semantic interoperability. Hence, interoperability assists in transferability of knowledge, rules or results. In GEOBIA, the use of ontology also assists in data analysis, in addition to data discovery, data integration, and data publication [9
]. The knowledge base created in landslides can be shared across different disciplines. In this case study, the rule sets developed to identify landslides for the Okhimath area can be transferred and reused for different region datasets with slight modifications.
The reuse of only a certain part of an existing ontology is possible with modularisation, which reduces the overhead of loading the whole of an ontology when only a part of it is needed. Ontology modularisation also assists with tackling the limitation of degrading performance of reasoners with an increase in the size of an ontology. The inferencing capability performed automatic classification of image segments by providing the SWRL rules to the reasoner. One of the issue to be considered when using inference based classification is that it may take a significant time when complex SWRL rules are used.
The lack of spatial analysis capability in the ontology based classification module leads to a requirement for more interaction with image analysis tools. This can bring complexity to the overall process of classification. In the case study presented, we had to use eCognition software to calculate area and distance to class for newly classified image objects that are used for classifying other classes. The use of custom spatial built-in in SWRL rules can reduce such complexity. Remote sensing or domain experts involved in the image analysis process might have far less or no knowledge of ontology engineering. This methodology requires the users to possess an understanding of ontology and knowledge engineering.
The concept of modularity has been introduced in technological and organisation design for tackling complexity [50
]. Modularity refers to the subdivision of a system into smaller parts called modules. The key aspect of modular programming is that we can reuse the modules at different stages. In this work, modularisation can be seen at two stages—firstly, the modularity in framework allowing division of steps into independent modules; secondly, the modularisation of developed ontology and rules.
A modular approach has been adopted in GEOBIA frameworks [51
]. Modularity ensures a framework to become more customisable and expandable. For instance, with modularisation of segmentation and classification as independent modules, we have an ability to use two different tools for each module. This demonstrates the flexibility and extensibility of the proposed framework over proprietary software with functionalities limited within their software. Using a modular approach, Ref. [51
] introduced system extension capabilities in their framework to incorporate third party functionalities. This opens the possibility for researchers to integrate different tools or build new algorithms on top of the existing framework.
In GEOBIA, complexity increases with the iterative process and composite workflow [54
]. The modular approach follows the rule of divide and conquer to break down a complex task into a number of simpler tasks. This allows error assessment of output results at the end of each module, which becomes input to the next module in a work chain.
With modularity, the ontology construction process is isolated from image analysis tasks, allowing domain experts to create the knowledge base independently. A group of domain experts can work together collaboratively in the knowledge construction process. Thus, knowledge-based image analysis in GEOBIA becomes less subject to the expertise of a particular analyst. Ontology modularisation further assists in tackling transferability issues. The transferable domain ontology is developed separately from data dependent feature ontology. The data dependent ontology or rules needs adaptation to make them transferable.
5.3. Classification Results
We performed two rule-based classifications using the same set of rules: firstly, the ontological method proposed in this paper and later the non-ontological method from the published literature [28
]. In the experiment, the same segmentation technique was used for both cases resulting in the same number and shapes of segmented images. The classification result of the ontological method was found to be consistent with a non-ontological method in terms of classified object counts, thus benchmarking the performance of the proposed method.
This shows that an ontological approach contributes a complementary classification method in GEOBIA but with added benefits provided by knowledge formalisation. The advantages of the ontological method over non-ontological methods are data interoperability, knowledge transferability, semantic inferencing and more automation with less human intervention.
The numbers of classified objects and shapes did not match when compared with the reference landslide inventory based on ground truth. Thus, a confusion matrix was calculated to assess the accuracy of the image classification by comparing classified image segments with reference landslide inventory. An overall accuracy of 86.3% was achieved. The discrepancy between the classification result and reference data is dependent on the segmentation result and the threshold values for different features defined in the rules. This demands further study in improving image segmentation, which was not considered in this study as our work is primarily focussed on finding applicability of ontological methodology in GEOBIA.
The proposed ontological framework may suffer from computational inefficiency due to its dependency on the capabilities of the reasoner. With an increase in the number of instances, classes, relations and axiom, the reasoning time may significantly increase. To tackle this issue in future work, we will further explore the use of OWL 2 profiles that trade expressive power for the efficiency of reasoning.
The accuracy of the rule-based classification is influenced by the scale-level of image segmentation. Different objects are identified at different scale levels depending on their spatial and thematic characteristics. This means that, if the classification is carried out with the segmented image objects at an inappropriate scale-level, there will be inaccuracy in the classification result. However, this is an open issue in GEOBIA, which warrants further explorations and studies [55
With machine learning, there is a need for good training data to achieve better prediction. In the absence of adequate training data, the threshold value extracted might depart from the actual value.
This study proposes a framework for object-based image analysis using ontology and applies the framework to landslide detection. The framework requires construction of an ontology for a domain of interest. In the case study reported here, the ontology is based on knowledge provided in previously published work that used GEOBIA. The use of ontology allows inference on domain knowledge to bring semantic image analysis into GEOBIA. GEOBIA requires human intervention in the form of expert knowledge, for defining classification rules and the threshold values for attributes used in those rules. To tackle this intervention, we combined machine learning into an ontological framework for automatic extraction of threshold values used in the rule-based classification of GEOBIA. Modularisation of ontology is introduced to separate transferable domain ontology and non-transferable feature ontology. This study captures high level domain knowledge from experts and low-level knowledge from data using machine learning techniques. We formalised domain expert knowledge in the specific field of landslides and benchmarked by comparing with published work. The scope of this study is landslide detection, which has not previously been studied using an ontological framework.
This study helps to progress the application of ontological methods within GEOBIA. It demonstrates a novel approach to automatically extracting threshold values for feature attributes used in ontological classification rules. The developed approach distinguishes between transferable and non-transferable ontologies. In addition, it demonstrates the application of these methods to a new domain landslide detection.
An avenue for further research is incorporation of spatial rules and exploration of optimal combinations of segmentation, classification-based-segmentation, and, in turn, the final classification in an ontology driven GEOBIA framework.