Research on the Intelligent Construction of UAV Knowledge Graph Based on Attentive Semantic Representation

Abstract

Accurate target recognition of unmanned aerial vehicles (UAVs) in the intelligent warfare mode relies on a highly standardized UAV knowledge base, and thus it is crucial to construct a knowledge graph suitable for UAV multi-source information fusion. However, due to the lack of domain knowledge and the cumbersome and inefficient construction techniques, the intelligent construction approaches of knowledge graphs for UAVs are relatively backward. To this end, this paper proposes a framework for the construction and application of a standardized knowledge graph from large-scale UAV unstructured data. First, UAV concept classes and relations are defined to form specialized ontology, and UAV knowledge extraction triples are labeled. Then, a two-stage knowledge extraction model based on relational attention-based contextual semantic representation (UASR) is designed based on the characteristics of the UAV knowledge extraction corpus. The contextual semantic representation is then applied to the downstream task as a key feature through the Multilayer Perceptron (MLP) attention method, while the relation attention mechanism-based approach is used to calculate the relational-aware contextual representation in the subject–object entity extraction stage. Extensive experiments were carried out on the final annotated dataset, and the model F1 score reached 70.23%. Based on this, visual presentation is achieved based on the UAV knowledge graph, which lays the foundation for the back-end application of the UAV knowledge graph intelligent construction technology.

1. Introduction

With the penetration of artificial intelligence technology into the total factor of warfare, intelligent warfare is gradually becoming the mainstream combat style [1]. UAVs play the role of both a material foundation and a main support of combat capability in the context of intelligent warfare, which can realize more accurate perception, swifter decision-making, and more efficient action on the battlefield. In the rapidly changing intelligent warfare environment, the generation of large-scale UAV knowledge data leads to changes in knowledge processing, storage, query, and integration technologies, bringing new challenges and opportunities for the application of UAVs in practical mission scenarios [2].

As a new paradigm of knowledge processing, the development of knowledge graphs brings new possibilities for UAV knowledge management [3]. Objectively, knowledge graphs have more powerful data synthesis governance capabilities, and on a large scale, multi-source and different forms of UAV knowledge data can be deeply mined and represent the semantic relation and knowledge systems the graph contains. Subjectively, the knowledge graph is part of the development process of expert systems. However, the knowledge graph construction research on specific domains mainly focuses on medical [4], financial [5], and energy [6] fields. Compared with other domains, there are two main challenges for UAV knowledge graph construction:

• At the level of UAV knowledge and data, several challenges need to be addressed. Firstly, the authority of UAV data is often lacking. These data primarily originate from encyclopedias and news pages, where a direct binding connection between data producers and data credibility is absent. Secondly, the accuracy of UAV knowledge suffers from deficiencies, leading to conflicts and noticeable errors among different UAV data sources. Thirdly, there is a scarcity of systematic data within the UAV domain, and publicly available UAV data are in unstructured text format. Extracting fine-grained knowledge directly from such data becomes challenging, thus hindering comprehensive system research in this field.
• At the level of UAV knowledge graph construction and application, knowledge extraction is the core task in the construction process. The classical knowledge extraction methods can be commendable in the general domain. However, in the face of UAV domain data, it is necessary to develop an algorithmic extraction model adapted to the domain properties, to improve the efficiency and accuracy of extraction, and thus to enhance the generalization ability of the traditional methods. In the face of downstream UAV task applications, there is no direct reference case for the development of domain-oriented system engineering for user requirements and visual interaction. The overall architecture of the system needs to be explored and developed under the guidance of the requirements, in combination with application models from other domains.

Based on the above problems, this paper proposes a knowledge graph construction and application framework for the UAV domain. Firstly, a fine-grained ontology knowledge system model of UAV classes and their relations is defined, and the UAV knowledge extraction dataset is labeled according to the ontology. Then, a two-step UASR knowledge extraction model is designed according to the characteristics of the UAV triple corpus, divided into two subtasks of relation prediction and subject–object entity identification. Then, we design a two-step UASR knowledge extraction model based on the characteristics of the UAV triple corpus. The main contributions of this paper are as follows:

• A fine-grained knowledge ontology is formed by defining the concept and relation attributes of UAVs based on a collection of unstructured UAV data of a significant scale. From this ontology, a UAV knowledge extraction dataset is created by selecting high-quality texts that align with predefined UAV ontology entities and relation annotations.
• A UASR knowledge extraction model is proposed, taking into account the characteristics of UAV knowledge extraction data. The BERT pre-trained language model is utilized to generate character feature encoding. In the decoder stage, the model incorporates the MLP attention mechanism to enhance the representation of relation types in the text for relation prediction. Additionally, a relationship-aware attention approach is employed to assign higher weights to tokens closely associated with relation classification and entity recognition tasks, thus enhancing the contextual semantic representation of subject–object entities.
• The UASR knowledge extraction model undergoes extensive comparison and ablation experiments using a self-built dataset. The experimental results demonstrate the model’s effectiveness in solving knowledge extraction challenges within the UAV corpus. Furthermore, the knowledge graph generated by the UASR model’s extraction enables visual storage applications. These quantitative and qualitative experiments substantiate the efficacy and validity of the UASR framework.

2. Related Works

In this section, we mainly introduce two types of related work: the results of macro-construction of UAV knowledge graphs and micro-knowledge extraction methods.

2.1. Construction of UAV Knowledge Graph

The knowledge graph is a network composed of entities and the relations between them. These entities include entity types, entity type properties, relation types, and relation type properties, all of which require a unified semantic specification, known as the knowledge graph schema [7]. In the context of the Semantic Web, the knowledge graph schema is often referred to as an ontology, primarily aimed at abstracting, semantifying, and conceptualizing the content of knowledge graphs. However, there is no mature ontology in the UAV domain, while more mature ontologies have been constructed in the aviation domain on top of UAVs. For aircraft maintenance faults, Wang et al. [8] proposed an ontology for aircraft faults in order to solve the problem that multi-source aviation maintenance data lead to syntactic and semantic conflicts, thus limiting the integration and sharing of aircraft fault information, which provides a unified and specific description of multi-source data and eliminates the semantic heterogeneity. For aviation intelligence and operational safety, Mi et al. [9] used natural language processing techniques and clustering algorithms to intelligently extract ontologies for navigational announcement information.

Based on the top-level ontology model, the knowledge graph can be formed by combining the underlying data instances. At present, the research on the construction and application of knowledge graphs in the UAV domain is basically in the initial stage, and there are limitations in the specialty and the scale is far from the practical application requirements. Qiu Ling et al. [10] constructed a knowledge graph with more than 900 entities and 1800 relations for UAV faults, but its ontology construction is too simple and lacks certain authority. Nie Tongpan et al. [11] constructed a knowledge graph containing 74 entities and 98 relations for UAV power system fault diagnosis, and although the original fault manual documents are available on the data to ensure professionalism, the graph storage capacity is too small.

Although the above two typical UAV knowledge graphs basically present the form of the knowledge graph in terms of the results, they essentially directly replicate the general domain methods in terms of the methods and cannot meet the needs of the domain for knowledge graphs, while in other specific domains, knowledge graph construction techniques are relatively mature. Li et al. [12] applied the BERT-BiLSTM-CRF model for information extraction in the military domain to achieve extraction of multi-source military intelligence information. In the geographic domain, Molina-Villegas et al. [13] selected the Mexican geological exploration news text as the research object and applied the general word-embedding approach to accomplish information extraction and disambiguation at the same time. In the energy domain, Wang et al. [14] proposed a recognition method applicable to electric power text, which improves the performance of the BiLSTM-CRF decoding recognition model by fusing character-level pre-training models, left-neighbor entropy, and lexical feature encoding. In the aviation domain, Bao et al. [15] used the traditional BiLSTM-CRF framework and incorporated an attention mechanism to extract named entities related to aviation design. Wang Hong et al. [16] used a self-attentive mechanism and BiLSTM to extract the triples of the accident occurrence process and used scenario reproduction to analyze the causality of aviation safety accidents.

2.2. UAV Knowledge Extraction Approach

The UAV knowledge extraction task is essentially an automated process of extracting triples from large-scale unstructured texts. Currently, it is primarily tackled using deep learning methods, which involve training deep neural networks to learn relevant features for knowledge classification. This process can be divided into three main components: an embedding layer, a network layer, and classification.

Since sentences are not directly input into the neural network for computation, textual word element-embedding representations are required, and common approaches include the static word representations Word2vec [17,18] and GloVe [19], and the dynamic pre-training models ELMo [20], BERT [21], and GPT [22] for the input-embedding layers. The main approaches of the network layer include the convolutional neural network (CNN) [23], firstly used for text classification and then further extended to knowledge extraction, mainly implemented using multiple convolutional kernels and multiple windows. Based on this, Zeng et al. [24] proposed a maximum convergent segmental convolution based on location information, which can capture more structured information of entity pairs. Similarly, Recurrent Neural Network (RNN) can also extract relations, such as the SDP-LSTM framework [25], which applies a multi-channel LSTM to the shortest sequence path of entity pairs. In independent sentence representation, graph convolution networks (GCN) [26] can be utilized to encode sentences with relational knowledge, and the multi-headed attention mechanism selects weights corresponding to different relational edges, further proposing a knowledge-adaptive coarse- and fine-grained attention mechanism combined with an information-filtering relation extraction model. The output layer of the model is processed with SoftMax or sigmoid for single and multiclassification cases, respectively.

According to the model structure, there are pipeline and joint extraction models [27]. The biggest drawback of the pipeline model is that it cannot solve the relation overlap problem, which is manifested in the case of one entity corresponding to multiple relations, the case of two relations between the principal and object entities, and the case of multiple nesting within the principal and object entities. These problems are commonly found in UAV knowledge extraction tasks, such as the PURE model proposed by Chen et al. [28]. However, Yu et al. [29] first proposed the ETL model for the overlapping problem of relations, where the main entity is extracted first as a priori information, and then the object entities and relations are classified by sequence annotation. However, this method cannot solve the situation that there are two relations between the main and object entities. The CasRel model, proposed by Wei et al. [30] for this purpose, first identifies all possible principal entities in the sentence as a priori information and extracts both object entities and relations using sequence annotation. However, the method requires judging a large number of redundant relations and can only handle one principal entity at a time, which is less efficient in engineering practice. Wang et al. [31] proposed the TPLinker model to construct the global information matrix, a matrix used for extracting all the principal and object entities, and additionally, for each relation, to construct two matrices for the beginning and end positions of the principal and object entities. Finally, the alignment of entities and relations is achieved by using the corresponding annotation method to decode the triples. The relation extraction model based on potential relations and global correspondence (PRGC) proposed by Zheng et al. [32] can not only solve the overlap phenomenon in the corpus, but can also significantly reduce the number of relation judgments due to the presence of a priori potential relation judgments and significantly improve the training and computational efficiency.

3. Construction of UAV Knowledge Graph Based on UASR

3.1. UAV Ontology Definition

Currently, ontology construction usually adopts a seven-step approach [33], in which the core aspects include listing important terms in the domain, defining classes and their hierarchical relation, defining class attributes and their relation, etc. As for UAV ontology construction, it is more oriented to unstructured text data, and there is currently no ontology that can be directly reused to learn from. Therefore, combined with the specific UAV knowledge graph construction and application scenario requirements, the UAV ontology definition is simplified into the following two steps:

1. Analyze the core knowledge concepts of the UAV domain and sort out the knowledge system of the UAV system. This paper is oriented to UAV ontology construction and was completed under the guidance of domain expert knowledge, which defines UAV-related concepts, attributes, and relations, and prepares “raw materials” for the subsequent steps. This process does not require a completely clear and conflict-free classification of the above elements, but only a list of as many desired elements as possible, for example, for the UAV system components enumerated in the flight control, weaponry, and landing systems, etc. Although there is no UAV ontology that can be directly reused, it can be borrowed from the ontology model of the military domain [34], which has a strong relevance to the UAV studied in this paper and can serve a complementary role in the process of the knowledge system formation. This contains the main basic information of the aircraft and knowledge of derived systems, such as warfare, equipment systems, design manufacturers and facilities, and equipment, which are basically the same concepts as the UAV.

2. Define the UAV classes and their hierarchical and relational attributes. After determining the UAV-related elements, the next step is to filter out the representative elements and further constitute the framework of the UAV classification system with its hierarchical relation, in which the concepts represented by the upper-level categories must fully encompass the concepts represented by the lower-level categories. Relations generally correspond to the categories that need to be interacted with, and attributes refer to the inherent qualities of the categories themselves. However, the UAV relation and attribute constraints can be transformed into each other to some extent. For example, the UAV production manufacturer can be defined as an attribute inherent to the UAV in the description metrics, but often the manufacturer latently has a strong interaction with categories such as the location country. Therefore, this paper provides a unified definition of relation and attribute constraints. According to the above ontology construction method, the first level of the UAV classification system includes two subsystems, UAV equipment and UAV events. Three third-level subsystems, of the composition system, description attributes, and technical indicators, are included under UAV equipment, while UAV events include eleven third-level subsystems, such as design, manufacturing, and operation. The fourth-level subsystems include 70 subsystems, such as appearance, the control system, and duties. The layers of the UAV composition system are shown in Figure 1.

3.2. UASR Knowledge Extraction Model

3.2.1. Problem Formulation

The task of UAV knowledge extraction model is to get the UAV knowledge triples $T(s)=\{(s,r,o)|s,o\in E,r\in R\}$ from the given sentences with $n$ tokens $S=\{x_1,x_2,\dots ,x_n\}$. Among them, $E$ and $R$ denote the set of entities and the set of relations, respectively. In the encoding part, in order to obtain better results for token and relation embedding, the BERT [35] large-scale language pre-training model is used to encode the input sentences and obtain the contextual representation of each word, and the result is represented as: $Y_{enc}(S)=\{h_1,h_2,\dots ,h_n|h_i\in {ℝ}^{d\times 1}\}$. In the decoding part, it can be split into two stages: relation prediction and subject–object entity identification. For a given sentence $S$, the goal of the relation extraction subtask is to predict the potential relations contained in the sentence, and the output is:$Y_r(S)=\{r_1,r_2,\dots ,r_m\in R\}$. Among them, $m$ denotes the number of relations in the subset of potential relations. The entity recognition subtask annotates each token corresponding to a given sentence $S$ and the potential relation $r_i$, predicted by the entity recognition task according to the predefined scheme and BIO rules, and the output is: $Y_e(S,r_i|r_i\in R=\{t_1,t_2,\dots ,t_n\})$. Among them, $t_i$ denotes each label of the annotation. A given sentence $S$ produces a corresponding score for each subject–object entity start token, which corresponds to a higher score for the start tokens and lower scores for the other tokens in a triadic pair. The output results are: $Y_s(S)=M\in {ℝ}^{n\times n}$. Among them, $M$ represents the global correspondence matrix. Combined with the specific UAV knowledge extraction context, Figure 2 shows that considering the input text: “RQ-4 is deployed in the Western Pacific”, the final output triple is: RQ-4, Position (Deployment), Western Pacific. The relation judgment subtask determines the potential relation from the five predefined relations, such as “Position (Deployment)”. The entity recognition subtask predicts “RQ-4” and “Western Pacific” with the labels “B-SUB”, “I-SUB”, “B-OBJ”, and “I-OBJ”. The global correspondence matrix for the subject–object alignment subtask predicts higher correspondence scores for the tokens starting with “Seahawk” and ending with “West Pacific”, and lower correspondence scores for other tokens.

3.2.2. Relation Prediction

For the relation prediction stage, MLP attention [36] is used to generate contextual semantic representations based on tokens and relation representations to further extract key information, and the extraction method is shown in Equation (1), where the weight of each token in the sentence is generated by multi-layer perceptron attention. If token $x_k$ has a tight semantic association in the subject–object entity extraction or relation classification tasks, $x_k$ receives a higher weight.

$$\begin{array}{c}{\mathcal{V}}_k={MLP}_k\left(x_k\right)\mathrm{s}.\mathrm{t}.k\in [1,n]\\ {\alpha }_{\mathbf{k}}=\frac{exp\left({\mathcal{V}}_k\right)}{{\displaystyle \sum _{m=1}^n}exp\left({\mathcal{V}}_m\right)}\\ {\mathcal{F}}_{\mathrm{s}}={\displaystyle \sum _{m=i}^n}{\alpha }_m{x}_m\end{array}$$

(1)

From the MLP attention method, the sentence semantic representation obtained in the relation prediction stage is ${\mathcal{F}}_{\mathrm{s}}$, while the relation classification is simplified into a multi-label binary classification task, $\sigma$ denotes the $sigmoid$ function, ${\mathrm{W}}_r\in {ℝ}^{d\times 1}$ denotes the training parameters, and ${\mathrm{P}}_{rel}$ denotes the probability of the sentence relation classification, as the corresponding relation of the binary classification task will be labeled as 1 (when the relation probability is exceeded); otherwise, it will be 0. Therefore, the subsequent tasks only need to label the sequences with potential relation sequences, and not all relational sequences need to be labeled.

$${\mathrm{P}}_{rel}=\sigma ({\mathrm{w}}_r{\mathcal{F}}_s+{\mathrm{b}}_r)$$

(2)

3.2.3. Subject and Object Extraction

Different from the conventional knowledge extraction model, this paper further incorporates the attention mechanism [37] to obtain the contextual semantic representation. For the general entity–relation extraction method, the subject and object entities are first extracted, and then the relation vectors are spliced and fused with the subject and object entity vectors. Thus, the extraction of entities and relations is inseparable from the semantics of the context, where the relation embedding is represented by a learnable network, and the specific relation attention mechanism is formulated as follows:

$$\begin{array}{c}{a}_{ij}={\alpha }^T\left[W_1{x}_i+b_1;W_2{r}_j+b_2\right],\\ {\lambda }_{ij}=\mathit{softmax}\left(a_{ij}\right),\\ h_s=r_j+{\displaystyle \sum ^n}{\lambda }_{ij}\left(W_3{x}_j+b_3\right),\end{array}$$

(3)

To overcome the overlap phenomenon in the UAV knowledge extraction corpus, this paper utilizes the relation-embedding approach to label object entities. Based on the semantic association of entities and relations, the main entity and relational embeddings are input to a deep neural network model to predict the object entities. In order to enhance the effect of relational embedding in entity classification, the weight of each token in the sentence is calculated based on the relational attention approach under each specific class of relations. In the subject–object entity recognition task, different relational representations for each sentence are generated through relation-aware contextual semantics with the following prediction formula for each token, specifically:

$$\begin{array}{l}{\mathrm{P}}_{i,j}^{sub}=Softmax({\mathrm{w}}_{sub}({\mathrm{h}}_i+{\mathrm{u}}_j)+{\mathrm{b}}_{sub})\\ {\mathrm{P}}_{i,j}^{obj}=softmax({\mathrm{w}}_{obj}({\mathrm{h}}_i+{\mathrm{u}}_j)+{\mathrm{b}}_{obj})\end{array}$$

(4)

Among them, ${\mathrm{u}}_j\in {ℝ}^{d\times 1}$ denotes the $j$-th relation representation in the learning-embedding matrix, $n_r$ denotes the total number of relations in the set of relations, ${\mathrm{h}}_i\in {ℝ}^{d\times 1}$ denotes the encoding of the $i$-th token, ${\mathrm{W}}_{sub},{\mathrm{W}}_{obj}\in {ℝ}^{d\times 3}$ denote the learnable weights, and the three dimensions correspond to the annotation set $\left\{B,I,O\right\}$.

In the process of subject–object entity alignment, the correct subject–object entity pairs are defined using the global correspondence matrix. The prediction of a potential relation can also be simultaneously performed during the learning process of the global correspondence matrix. The process is as follows: first, number the possible subject–object entity pairs, then check the corresponding score of each subject–object entity pair in the global correspondence matrix, and keep the score if it exceeds the threshold ${\lambda }_2$; otherwise, filter and delete it.

For a given global correspondence matrix $M\in {ℝ}^{n\times n}$, $n$ denotes the number of tokens in the sentence, each element of the matrix corresponds to the starting position of the subject–object entity, and the correspondence score is the confidence of matching the subject and object entities. The higher the score, the higher the confidence of matching the two entities. The corresponding score of each element in the matrix is calculated as follows:

$${\mathrm{P}}_{i_{sub},j_{obj}}=\sigma ({\mathrm{w}}_g[{\mathrm{h}}_i^{sub};{\mathrm{h}}_j^{obj}]+{\mathrm{b}}_g)$$

(5)

Among them, ${\mathrm{h}}_i^{sub},{\mathrm{h}}_j^{obj}\in {ℝ}^{d\times 1}$ denote the input encoding of the $i$-th and $j$-th tokens formed by the subject–object entity, ${\mathrm{W}}_g\in {ℝ}^{2d\times 1}$ denotes the learnable weights, and $\sigma$ denotes the $sigmoid$ function.

3.2.4. Training and Inference

The cross-entropy loss function is used in the training inference phase to calculate the final loss to optimize the training model, while sharing the parameters encoded by the BERT model. The loss function can be decomposed into three components:

$$\begin{array}{c}{\mathcal{L}}_{rel}=-\frac{1}{n_r}{\displaystyle \sum _{i=1}^{n_r}(y_i\log {\mathrm{P}}_{rel}+(1-y_i)\log (}1-{\mathrm{P}}_{rel}))\\ {\mathcal{L}}_{seg}=-\frac{1}{2\times n\times n_r^{pot}}{\displaystyle \sum _{t\in \{sub,obj\}}{\displaystyle \sum _{j=1}^{n_r^{pot}}{\displaystyle \sum _{i=1}^n}}}{\mathrm{y}}_{i,j}^t\log {\mathrm{P}}_{i,j}^t\\ {\mathcal{L}}_{global}=-\frac{1}{n^2}{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^n(y_{i,j}\log {\mathrm{P}}_{i_{sub},j_{obj}})+(1-y_{i,j}\log (1-{\mathrm{P}}_{i_{sub},j_{obj}}))}}\end{array}$$

(6)

Among them, $n_r$ denotes the number of all relations and $n_r^{pot}$ denotes the number of predicted potential relations.

The weighted sum of the three-part loss function is:

$${\mathcal{L}}_{total}=\alpha {\mathcal{L}}_{rel}+\beta {\mathcal{L}}_{seg}+\gamma {\mathcal{L}}_{global}$$

(7)

Among them, $\alpha ,\beta ,\gamma$ denote the adjustable hyperparameters, which can be set to 1 to simplify the model calculation.

4. UAV Knowledge Extraction Experiments

4.1. Experimental Setup

4.1.1. Dataset

The experimental dataset was generated based on the UAV ontology defined in Section 3.1 using the manual annotation method, with a total number of 377,670 characters and 8063 sentences. The core elements of relation labeling still revolve around the UAV entity, corresponding to the relation between UAV and UAV, including homologation, modifications before and after production design; operation between the UAV and the organization; UAV and its related locations such as the production, origin and user country etc.; the direct correspondence between the UAV and the engine; the relations between the UAV and event participation, exhibition, and occurrence of accidents, etc. The final annotated corpus entity data and representatives are shown in

Table 1. Statistics and examples of each type of relation.

Relation	Examples	Number
UAV and Engine	RQ-4 Global Hawk—Rolls-Royce AE 3007	88
UAV and UAV	RQ-4 Global Hawk—ACTD Prototype	974
UAV and Events	RQ-4 Global Hawk—Afghanistan War	135
UAV and Country	RQ-4 Global Hawk—United States	363
UAV and Organization	RQ-4 Global Hawk—Northrop Grumman	702

.

4.1.2. Model Setting

The above constructed UAV knowledge extraction dataset was divided into train, validation, and test dataset data models in the ratio of 8:1:1 in the experiment. The accuracy, precision (P), recall (R), and the F1 score were selected as the evaluation index of knowledge extraction task:

$$\begin{array}{c}P=T_P/\left(T_P+F_P\right)\\ R=T_P/\left(T_P+F_n\right)\\ F1=2*P*R/\left(P+R\right)\end{array}$$

(8)

where $T_P$ denotes the number of entities whose true class is positive and that were predicted to be positive as well, $F_P$ denotes the number of entities whose true class is negative but were predicted to be positive, and $F_n$ denotes the number of entities whose true class is positive but that were predicted to be negative.

The RoBERTa-wwm-ext language pre-training model proposed by Cui et al. [38] was chosen as the basis for character-level vector coding, and the experimental base hyperparameters were set as shown in

Table 2. Relation extraction model hyperparameter setting.

Hyperparameter	Value
Batch size	8
Sequence length	256
${\lambda }_1$	0.5
${\lambda }_2$	0.1
Warmup proportion	0.05
Decay rate	0.5
Learning rate	1 × 10−3
Embedding learning rate	1 × 10−4
Dropout	0.3
Loss	CE
Optimizer algorithm	Adam

. The deep learning framework was Pytorch V1.6.0, the basic running environment was Python V3.7.0, and the hardware configuration was 32G RAM, NVIDIA RTX A4500 (20G).

4.2. Experimental Results

4.2.1. Pre-Experiment Result

Before applying the UASR model on the in-house UAV knowledge extraction dataset, a pre-experiment on the public dataset NYT was conducted to justify the feasibility. As depicted in

Table 3. Comparison of the results of different models for the pre-experiment.

Model	P (%)	R (%)	F1 (%)
CasRel	87.71	90.53	89.10
TPLinker	89.35	90.67	90.01
PGCN	93.54	91.62	92.57
UASR (Our Model)	93.58	92.24	92.91

, the UASR model demonstrated superior performance compared to other models on the NYT dataset. Our model outperformed the baselines in terms of precision, recall, and F1 score, with a notable increase of 0.34% in the F1 score over PRGC. While the performance on the public dataset was relatively satisfactory, it should be noted that there is still room for improvement. Nevertheless, this pre-experiment serves as a strong indication that the UASR model is viable for knowledge extraction tasks, providing a solid foundation for subsequent experiments on our in-house UAV knowledge extraction dataset.

4.2.2. Overall Comparison

The UAV knowledge extraction results are presented in

Table 4. Comparison of the results of different models for UAV knowledge extraction.

Model	P (%)	R (%)	F1 (%)
CasRel	67.12	60.16	63.86
TPLinker	62.13	67.58	64.74
PGCN	64.93	68.35	66.59
UASR (Our Model)	67.79	72.86	70.23

. Our proposed UASR model demonstrated the best performance compared to other models, achieving an impressive F1 score of 70.23%. In comparison to two other entity–relation joint extraction models, namely CasRel and TPLinker, the CasRel model exhibited exposure bias and error propagation issues. It overlooked the interaction between entities and relations, resulting in a decrease of 6.37% in the F1 score compared to our model. On the other hand, the TPLinker model performed relatively better, with a decrease of 5.49% in the F1 score. This improvement is primarily attributed to the adoption of a handshake labeling strategy for end-to-end sequence labeling. This strategy connects entity heads with entity tails, head entity heads with tail entity heads, and head entity tails with tail entity tails, using a matrix-based approach for labeling. By training the model to predict the matrix under each relation, the TPLinker model achieved simultaneous entity–relation learning, avoiding the inconsistency between training and prediction orders. Consequently, it demonstrated superior performance on the UAV dataset. Additionally, the PGCN model, which shares closer similarities with our approach, attained an F1 score of 66.59%, showing a decrease of 3.64%. This outcome further emphasizes the advantages of our attention-based relation-aware method in acquiring contextual semantic information.

4.2.3. Ablation Experiments

In order to further demonstrate the effectiveness of the improvement strategies of each part of the UASR model, further ablation experiments were conducted in this paper. The experiments removed the MLP attention module in the relation prediction phase and the relation attention module in the subject–object entity identification phase, respectively. The two main improvement modules of the model shown in

Table 5. Comparison of the results of UAV knowledge extraction ablation experiments.

Model	P (%)	R (%)	F1 (%)
UASR (Our Model)	67.79	72.86	70.23
MLP attention	66.21	67.83	67.01
Relation attention	62.51	73.49	67.55

had significant effects on the overall performance. When the MLP attention module was removed, the F1 score of the model decreased by 3.22%, presumably because the relation prediction stage enhanced the model’s perception of relation classification to a certain extent for the relation-embedding representation. When the relation attention module was removed, the F1 score of the model decreased by 2.68%, presumably because the main object entity recognition stage above the semantic relationship-aware representation further enhanced the information fusion of task associations.

The impact of different components within the loss functions on the performance of our proposed UASR model was analyzed through a comprehensive experiment. The results, presented in

Table 6. Comparison of the results of UASR loss function ablation experiments.

Model	P (%)	R (%)	F1 (%)
UASR (Our Model)	67.79	72.86	70.23
Relation loss	57.32	73.90	64.56
Sequence loss	Null	Null	Null
Global loss	39.26	28.70	33.16

, provide valuable insights into the significance of three components. Upon removing the relation loss function, we observed a decrease in the performance of the UASR model by 5.67%. This validates the crucial role of the relation prediction component, which aims to accurately predict potential relation subsets. Moreover, Table 6 reveals that the removal of the sequence loss function led to the inability of the UASR model to correctly extract UAV knowledge. This can be attributed to the inclination of the model to solely memorize the entity positions rather than comprehending the underlying semantics. Additionally, the limited size of our in-house dataset may have contributed to this limitation. Furthermore, the absence of the global correspondence loss function significantly impacted the performance of the UASR model, resulting in an increased number of predicted triples with numerous mismatched pairs. This emphasizes the importance of the constraint imposed by this component.

To assess the impact of different BERT models on our proposed model, we conducted a comparative analysis of three prominent BERT models: BERT-base, BERT-wwm, and RoBERTa-wwm. These models have demonstrated state-of-the-art performance in related knowledge extraction tasks. The experimental results, presented in

Table 7. Comparison of the results of UASR BERT ablation experiments.

Model	P (%)	R (%)	F1 (%)
RoBERTa-wwm (Our Model)	67.79	72.86	70.23
BERT-base	66.71	69.62	68.13
BERT-wwm	65.22	73.77	69.23

, indicate that RoBERTa-wwm outperformed the other models, exhibiting F1 score improvements of 2.10% and 1.00%, respectively. However, as the focus of this paper is not specifically on pre-training language models, the influence of these models, as indicated by the results of our ablation experiments, can be considered negligible.

4.3. UAV Knowledge Graph Visualization

UAV knowledge extraction was implemented through the UASR knowledge extraction model, and the extracted relation needed to be stored and visualized. In this paper, Neo4j was chosen as the relation storage tool in view of the data structure of UAV and the convenience of interactive visualization. In this paper, the UAV knowledge graphs are represented as a triad of <entity, relation, entity> and <entity, attribute, attribute value>, such as <RQ-4, deployment location, Western Pacific> and <MQ-9, endurance, 42 h>. In this paper, we adopted the LOAD CSV method in Cypher query language to import the above knowledge triples and visualize them using Cypher statements. Figure 3a shows the UAV knowledge graph of 4000 entities and more than 10,000 relations at a macro-level. Figure 3b shows the UAV knowledge graph of a single entity “RQ-4” and its relation attributes at a micro-level.

5. Conclusions

As a basis for intelligent target identification and detection, UAV knowledge graphs can provide fine-grained structured knowledge for downstream situational awareness tasks. In this paper, a knowledge graph construction and application framework for the UAV domain was proposed for the problems at the data, system, technology, and application levels in the process of UAV knowledge graph construction. The UAV ontology was formed by combining the characteristics of the UAV knowledge structure system, and a certain scale of the UAV knowledge extraction triple dataset was labeled according to the ontology model, which solved the difficulties at the data and knowledge levels. Based on the UAV data and schema layers, a two-step UASR knowledge extraction model was proposed, and the contextual semantic representation of the two stages of relation prediction and subject–object entity recognition was optimized using attention representation and a relation-aware attention mechanism, respectively, based on the word meta-embedding of the large-scale language pre-training model, with full consideration of the UAV corpus characteristics. Finally, extensive experiments were conducted on the labeled dataset to demonstrate the effectiveness of the UASR model, and the extracted UAV knowledge was implemented for visual storage and intelligent question-and-answer (Q&A) applications.

However, the proposed framework for UAV knowledge graph construction and ap-plication still has some limitations:

(1) Due to the limited UAV data, there is still room for improvement in the knowledge extraction accuracy of the UASR model. In the future, under the condition that the dataset is large enough, we could divide the sub-datasets according to the categories of overlapping relations and the length of entity texts in the relations. We could then conduct experiments on the sub-datasets, analyze them from multiple perspectives based on the experimental results, and finally, conclude the reasons for the poor performance of the model on the UAV dataset and the directions for improvement.

(2) Although the application of the UAV knowledge graph was simply realized in this paper, there is still a gap in the application technology between intelligent Q&A and UAV target recognition and detection applications. In the future, the method of knowledge graph embedding can be used to input UAV knowledge in the form of pre-trained templates into the downstream situational game perception task to realize the development of UAV knowledge graphs for knowledge understanding and reasoning applications in actual aerospace situation.