Automated Construction Method of Knowledge Graphs for Pirate Events

Abstract

With the development of seaborne trade, international maritime crime is becoming increasingly complex. Detecting maritime threats by fusing the physical movement data from traditional physical sensors is not sufficient. Thus, soft data, including intelligence reports and news articles, need to be incorporated into the situational awareness models of maritime threats. In this regard, this study developed an automated construction method of knowledge graphs for pirate events, which lays a foundation for subsequent maritime threat reasoning and situational awareness. First, a knowledge graph ontology model for pirate events was designed. Secondly, the BERT-BiLSTM-CRF model is proposed for named-entity recognition, and an entity linking algorithm based on distant learning and context attention mechanism is proposed to remove the conceptual ambiguity. Thirdly, based on traditional distant supervision relation extraction, which is based on sentence-level attention mechanism, bag-level and group-level attention mechanism methods are additionally proposed to further enhance the performance of distant supervision relation extraction. The proposed model demonstrated high performance in named-entity recognition, entity linking, and relation extraction tasks, with an overall F1-score of over 0.94 for NER and significant improvements in entity linking and relation extraction compared to traditional methods. The constructed knowledge graphs effectively support maritime threat reasoning and situational awareness, offering a substantial contribution to the field of maritime security. Our discussion highlights the model’s strengths and potential areas for future work, while the conclusion emphasizes the practical implications and the readiness of our approach for real-world applications.

1. Introduction

Maritime shipping is the most important mode of international transportation in the world today and plays a vital role in the development of world trade and economies [1]. However, non-traditional security risks, mainly piracy and maritime terrorism, have become the main issues affecting maritime shipping security. Among them, pirate attacks have the widest impact. Once such attacks occur, they will not only cause property losses but also pose a great threat to the lives of the people on board and may even have an adverse impact on navigation and the environment [2]. Therefore, constructing a situational awareness model for pirate events using available information is particularly important in the early warning of pirate events and maritime crime cracking [3].

Much of the current research focuses on automated processing, fusing, and reasoning from different data sources in maritime surveillance to assist official decision making. Using only the physical movement data from traditional physical sensors (i.e., hard data) makes full-scale situational awareness difficult [4]. Thus, soft data need to be incorporated into maritime threat situational awareness models. Here, soft data refer to structured text data, semi-structured and unstructured news, social media data, maritime incident reports, and so on [5,6,7,8,9,10]. These soft data include structured text data, semi-structured and unstructured news, social media data, maritime incident reports, and other relevant sources. In response, this paper presents methods that can extract such knowledge and construct knowledge graphs for pirate events, which are conducive to subsequent maritime threat reasoning for maritime threat situational awareness in general.

This paper presents an in-depth study on how to construct knowledge graphs for pirate events, and its contributions can be summarized as follows:

(1)

A Pirate Event Model (PEM) is designed as an ontology model of the knowledge graphs for pirate events. Four types of entities are set: pem: Aggressor, pem: Victim, pem: Location, and pem: Date, alongside the entity attributes of these four types of entities. In addition, two major categories of 11 types of relations among entities are designed.

(2)

For entity extraction, the BERT-BiLSTM-CRF model is used for NER from the soft data of pirate events. In the entity linking process, the method of distance learning is introduced for entity linking model training to address the problem of the lack of large amount of labeled text data. In addition, the context attention mechanism is proposed to select the words that are helpful for disambiguation, so as to enhance the performance of entity linking model training.

(3)

For RE, based on the traditional sentence-level attention mechanism, a bag-level attention mechanism is further introduced, which emphasizes the sentences that are related to the bag label at the global level by calculating the correlation between sentences. As such, the information of the bag is expressed, while sentences that are not related to the bag label are suppressed. Furthermore, bags with the same labels are assembled as a bag-group, and comprehensive discriminative features are mined at the bag-group-level as high-quality features of the relations between representation entity pairs.

(4)

For the algorithms involved in instance layer filling, experiments were designed to verify the effectiveness and superiority of the proposed models, including NER, entity linking, and relation extraction models, at the same time. According to the given pirate event text, using the proposed NER, entity linking, and relation extraction models, knowledge triples can be accurately extracted, and a high-quality knowledge graph for pirate events is constructed.

2. Related Works

2.1. Knowledge Graph Ontology Model

Knowledge graph construction includes ontology layer design and instance layer filling. The ontology layer is designed through an event-based ontology model to describe conceptual entities and their relations. Subsequently, through named-entity recognition (NER) and relation extraction, entities, attributes, and relations are extracted from diverse data sources to obtain knowledge triples, thus generating the ontologized knowledge representations and completing the construction of knowledge graphs. Currently, generic knowledge graphs, such as Freebase [11], Wikidata [12], and YAGO4 [13], are widely used, with corresponding vertical industry applications in medical, academic, and financial fields [14,15,16]. However, in the field of navigation, there are few relevant studies, especially studies on knowledge graphs for pirate events. Therefore, constructing a knowledge graph ontology model for pirate events is a primary research task. Currently, the main ontology models used in academia to describe events are the ABC (purposely given an innocuous name) [17], simple event model (SEM) [18], event ontology (EO) [19], and CIDOC-CRM [20] models. Based on the SEM model, a Pirate Event Model (PEM) was designed as a knowledge graph ontology model for pirate events.

2.2. Entity Extraction and Linking

Once the ontology layer is constructed, NER needs to be utilized to extract the required entities from the unstructured text data. Currently, NER based on neural networks has become a mainstream method [21,22], among which the convolutional neural network-conditional random field (CNN-CRF) model [21] and bidirectional long short-term memory network-conditional random field (BiLSTM-CRF) model [22] are the most commonly used. With a large number of entity mentions obtained through NER, there is a need for the further identification of the real-world entity to which an entity mention refers to through entity linking. There are two main steps: candidate entity generation and candidate entity linking. There are three main methods for candidate entity generation: name dictionary-based methods [23], surface form expansion [24,25], and object library-based methods [26]. After obtaining the entity mentions and feature representations of the contextual information of entities, entity linking performs similarity calculation between the entity mentions and representations to select the target entities. There are two main methods: traditional feature methods and representation learning methods. The essence of traditional feature methods is how to manually design effective features. Honnibal et al. [27] used the Bow model to obtain vectors of entity mentions and candidate entities; subsequently, the one with the highest cosine similarity score was regarded as the candidate entity. Background knowledge, priori knowledge, and entity typing information of candidate entities are also important for entity disambiguation; thus, many researchers have taken this information into account to improve the accuracy of disambiguation [28,29]. Traditional feature methods are heuristic for both target entity and entity mention representation, such as the bag-of-words model and term frequency-inverse document frequency (TF-IDF). It is difficult to adjust these heuristic algorithms or to capture the more fine-grained semantic and structural information; thus, traditional feature methods are not dominant. The essence of representation learning methods is how to obtain a distributed representation of the context of entities and entity mentions. The representation of entities is generally complex. An entity may be represented at different granularities, and the representation may also involve the entity type information. Neural networks are often used to automatically learn entities as well as distributed representations of entity mentions [30,31,32]. Usually, there is a lot of contextual information of entity mentions. However, some words in the context are not very relevant to the entity mentions. Therefore, the continuous representation of the trained context contains noise, which affects the accuracy of entity disambiguation. Researchers have proposed combining an attention mechanism with deep neural networks to train the semantic feature vectors of context to improve entity disambiguation models [33]. According to Sun et al. [34], important clues about entity mentions and candidate entities from the surrounding context could be discovered automatically through the attention mechanism, facilitating entity disambiguation. Raiman et al. [35] proposed a strategy in which the symbolic information was explicit-ensembled into the neural network inference process and subsequently applied this strategy to the entity disambiguation task so that entity linking was performed by training a type system. The current work focuses on supervised learning, where the training text contains annotated entity mentions (i.e., entity mentions linked to a knowledge base). However, in many specific verticals, there does not exist a large amount of labeled data, only a knowledge base and collection of unlabeled texts datasets from the corresponding domains. How to conduct the training of entity linking models in this regard was a focus of this research.

2.3. Relation Extraction

After extracting the desired entities from unstructured text data through named-entity recognition and entity linking techniques, the relations between the entities can be obtained through relation extraction (RE). Traditional RE techniques usually adopt the fully supervised learning paradigm. To achieve good performance, these methods require a large pool of well-labeled training corpora [36,37]. However, labeling large corpora involves significant economic and time costs, which hinders the application of these traditional supervision methods in the real world. Mintz et al. [38] first proposed an RE algorithm based on distant supervision. Distant supervision assumes that sentences containing the same entity pairs will express the same relations. Obviously, this assumption would lead to a considerable number of mislabels if distant supervision was applied directly to each sentence. In this respect, several studies [39,40,41] have used the multiple-instance learning paradigm, which packages sentences with the same entity pairs and learnt bag representation. In recent years, as the advantages of the attention mechanism have been gradually recognized, researchers have used it in distant supervision-based RE algorithms [42,43,44]. Such methods apply a sentence-level attention mechanism to score all sentences in the bag, and their contribution to the bag representation can subsequently be evaluated. However, all of the above methods ignore the correlation between sentences, which leads to the loss of supervisory information. In addition, such methods based on a sentence-level attention mechanism are unable to extract global sentence information from the bag.

2.4. Natural Language Processing Applications in the Maritime

Natural language processing technology has been applied in the maritime field. Gan et al. [45] present a knowledge graph construction method for analyzing ship collision accidents, automating the extraction of key insights from 241 CMSA investigation reports using NLP and ontology. Liu et al. [46] introduces a BERT-MCNN deep learning model for extracting key information from Chinese maritime pollution prevention regulations, achieving high precision in knowledge graph formation, which aids PSC officers in efficient decision-making through a Neo4j-based application. Cao et al. [47] employ a Bayesian network model to analyze the impact of Accident Influential Factors (AIFs) on marine accident severity, identifying key factors and potential risks through data-driven analysis and machine learning, ultimately aiding in the prediction and prevention of maritime accidents for stakeholders. Kamal et al. [48] conduct a quantitative assessment of 418 marine accidents in the Istanbul Strait using a Tree Augmented Naive Bayes approach to identify probabilistic relationships between accident occurrence and influencing factors, revealing that smaller vessels are particularly prone to adrift accidents, and providing insights for authorities to enhance maritime safety measures.

3. Construction of Knowledge Graph Ontology Model for Pirate Events

3.1. Entities

The key step in designing a knowledge graph is to define an ontology model [11,12,13,14,15,16]. Inspired by the SEM model, a Pirate Event Model (PEM) was designed as an ontology model for pirate event-oriented knowledge graphs, which can be used to guide the progress of the proposed methods of named-entity recognition, entity linking, and relation extraction. It has four types of entities: (1) pem: Aggressor, i.e., the aggressor in the pirate event; (2) pem: Victim, i.e., victims in the pirate event; (3) pem: Location, i.e., where the pirate event occurred, such as a specific geographical name or latitude and longitude; and (4) pem: Date, i.e., the date of the pirate event. In addition, entity attribute constraints are set for the four types of entities: pem: AggressorTpye, pem: VictimTpye, pem: LocationTpye, and pem: DateTpye.

3.2. Association Relations

The associative relations of the PEM model can be divided into two categories: (1) binary relations between pem: Aggressor, pem: Victim, pem: Location, and pem: Data and (2) behavior relations between pem: Aggressor and pem: Victim. The details of the entity relations are shown in

Table 1. Entity relations of the PEM model.

Relation	Subject(s)	Object(s)
pem: Aggressor-Location	Aggressor	Location
pem: Aggressor-Date	Aggressor	Date
pem: Victim-Location	Victim	Location
pem: Victim-Date	Victim	Date
pem: Attacking	Aggressor	Victim
pem: Boarding	Aggressor	Victim
pem: Firing upon	Aggressor	Victim
pem: Kidnapping	Aggressor	Victim
pem: Hijacking	Aggressor	Victim
pem: Robbing	Aggressor	Victim
pem: Suspiciously approaching	Aggressor	Victim

. In addition, Figure 1 visualizes the proposed ontology model.

4. Instance Layer Filling of Knowledge Graph for Pirate Events

We use pirate events, which are unstructured text reports, as model inputs, for instance, the layer filling of knowledge graph for pirate events.

4.1. Entity Extraction

This section extracts four types of entities: pem: Aggressor, pem: Victim, pem: Location, and pem: Date, from the given pirate event text. The BIO format [21,22] is used to label words in text data, i.e., B: Begin, representing the beginning of the named-entity; I: Inside, representing the inside of the named-entity; and O: Other, not representing any entity.

The BERT-BiLSTM-CRF model is used for NER. Bi-directional Encoder Representations from Transformers (BERT) is a self-supervised deep learning language model that pre-trains the text of multilayer bi-directional transformer coding structures mainly through the masking mechanism. For an input text sequence $S=\left\{w_1,w_2,\cdots ,w_N\right\}$, the BERT model outputs a coding sequence $\mathit{X}=\left\{{\mathit{x}}_1,{\mathit{x}}_2,\cdots ,{\mathit{x}}_N\right\}\in {ℝ}^{N\times d_w}$ in dimension $d_w$. In the following steps, the BiLSTM model maps the coding sequence $\mathit{X}=\left\{{\mathit{x}}_1,{\mathit{x}}_2,\cdots ,{\mathit{x}}_N\right\}$ to a tag score sequence $\mathit{P}=\left\{{\mathit{p}}_1,{\mathit{p}}_2,\cdots ,{\mathit{p}}_N\right\}\in {ℝ}^{N\times K}$, where ${\mathit{p}}_i\in {ℝ}^K$ denotes the score of the $K$ types of BIO-labeled entities corresponding to the word $w_i$. Among them, the BIO-labeled entities are listed in

Table 2. BIO-labeled entities.

Serial Number	Label
1	B-Aggressor
2	I-Aggressor
3	B-Victim
4	I-Victim
5	B-Location
6	I-Location
7	B-Data
8	I-Data
9	O

.

Finally, the tag score sequence $\mathit{P}$ is fed into the Conditional Random Field (CRF) layer, which is parameterized by the label transition matrix $\mathit{A}\in {ℝ}^{K+2\times K+2}$. $A_{i,j}$ represents the score transitioned from label $i$ to label $j$. In addition, the matrix $\mathit{A}$ contains two additional states: the beginning and end of the sequence.

For the input sequence $\mathit{X}=\left\{{\mathit{x}}_1,{\mathit{x}}_2,\cdots ,{\mathit{x}}_N\right\}$ and the corresponding BIO-labeled sequence $\mathit{Y}=\left\{{\mathit{y}}_1,{\mathit{y}}_2,\cdots ,{\mathit{y}}_N\right\},y_i\in \left\{1,2,\cdots ,K\right\}$, the evaluation scores for the CRF layer are given by Equation (1) [49].

$$Score\left(\mathit{X},\mathit{Y}\right)={\displaystyle \sum _{i=1}^N{A}_{y_i,y_{i+1}}}+{\displaystyle \sum _{i=1}^N{P}_{i,y_i}}$$

(1)

where $A_{y_i,y_{i+1}}$ denotes the score for transition from label $y_i$ to label $y_{i+1}$, and $P_{i,y_i}$ denotes the score for the probability that the $i$-th character is labeled as $y_i$.

The model is trained to maximize the log probability of the correct tag sequence [49]:

$$\log \left(p\left(\mathit{Y}|\mathit{X}\right)\right)=Score\left(\mathit{X},\mathit{Y}\right)-\log \left({\displaystyle \sum _{\tilde{\mathit{Y}}\in {\mathit{Y}}_X}{e}^{Score\left(\mathit{X},\tilde{\mathit{Y}}\right)}}\right)$$

(2)

where ${\mathit{Y}}_X$ denotes all possible BIO-labeling sequences.

Finally, Viterbi decoding [21,22] is used to find the highest scoring label sequence from the input sequences as a way to obtain the entity extraction results.

4.2. Entity Linking

Entity linking is the process of mapping entity mentions to real-world entities. Here, ambiguity can be considered a problem for the four types of entity mentions extracted from the event text, due to the different ways of expression. Therefore, entity linking based on context attention mechanism and distance learning is proposed in this paper, such that entity disambiguation can be achieved, and the quality of the knowledge graphs can be improved.

For entity mention $m=\left\{w_h,\cdots ,w_k\right\}$ in the event text $S=\left\{w_1,w_2,\cdots ,w_N\right\}$, $E^{+}$ is defined as a positive candidate entity set, which contains the correct entity $e$ corresponding to the entity mention $m$; $E^{-}$ is the negative candidate entity set, which contains only the incorrect entities. For the construction of the candidate entity set $E^{+}$, matching heuristics [40] are used to select entities from the knowledge base Wikipedia by measuring the overlap between entity mentions and the entity names. Specifically, all entities that contain entity mention $m$ are listed, and these selected entities are subsequently sorted according to the order in the knowledge base. Finally, the top $\left|E^{+}\right|$ ranked entities are selected as the positive candidate entity set $E^{+}$. This method is not particularly accurate in selecting the optimal entity, but the provided candidate entity set usually contains the correct entity $e$ corresponding to the entity mention $m$. For the construction of the negative candidate entity set $E^{-}$, $\left|E^{-}\right|$ candidate objects from the rest of the knowledge base are selected as the negative candidate entity set $E^{-}$.

For the entities $e$ in the candidate entity set $E^{+}$ and $E^{-}$, the word2vec method [50] is used to map them into vectors. Secondly, the BERT model (mentioned in Section 4.1) is then used to map event text $S=\left\{w_1,w_2,\cdots ,w_N\right\}$ into vector $\mathit{X}=\left\{{\mathit{x}}_1,{\mathit{x}}_2,\cdots ,{\mathit{x}}_N\right\}$, which includes mapping vector ${\mathit{X}}_m=\left\{{\mathit{x}}_h,\cdots ,{\mathit{x}}_t\right\}$ corresponding to entity mention $m$. The association scores of each word in the event text $S=\left\{w_1,w_2,\cdots ,w_N\right\}$ are calculated as a measure of its association with the entity $e$, as shown in Equation (3) [33]:

$$u\left(w\right)=\underset{e\in E^{+},E^{-}}{\max }{\mathit{x}}_e^T{\mathit{A}}_{ew}{\mathit{x}}_w$$

(3)

where ${\mathit{A}}_{ew}$ is the trainable parameterized diagonal matrix. The word set corresponding to the top $R$ ranked words is selected based on the ranking of the association scores from highest to lowest [33]:

$$\overline{S}=\left\{w\in S|u\left(w\right)\in \mathrm{top}\mathrm{R}\left(u\left(v\right)\right)\right\}$$

(4)

According to Equation (4), the attention weights are calculated as follows [33]:

$$\beta \left(w\right)=\{\begin{array}{cc}{\displaystyle \frac{\exp \left[u\left(w\right)\right]}{{\displaystyle {\sum }_{v\in \overline{S}}\exp \left[u\left(w\right)\right]}}}& w\in \overline{S}\\ 0& otherwise\end{array}$$

(5)

The representation vector ${\mathit{X}}_m^{att}$ of entity mention $m$ is calculated as follows:

$${\mathit{X}}_m^{att}={\displaystyle \sum _{w=h}^t\beta \left(w\right){\mathit{x}}_w}$$

(6)

Finally, ${\mathit{x}}_e$ and ${\mathit{X}}_m^{att}$ are fed into a feedforward neural network with two fully connected layers:

$$g\left(e,m,S\right)=FFN\left(\left[{\mathit{x}}_e,{\mathit{X}}_m^{att}\right]\right)$$

(7)

The feedforward neural network contains 128 hidden units and uses ReLU nonlinear activation.

During training, it is expected that, through the training model, at least one entity in $E_{train}^{+}$ scores higher than any entity in $E_{train}^{-}$. Max-margin loss is used to achieve the following:

$$\begin{array}{c}L={\displaystyle \sum _{\left(m,S\right)\in D_{train}}l\left(m,S\right)}\\ l\left(m,S\right)={\left[\underset{e\in E^{-}}{\max } g\left(e,m,S\right)+\delta -\underset{e\in E^{+}}{\max } g\left(e,m,S\right)\right]}_{+}\end{array}$$

(8)

where $D_{train}$ is the training set, and $\delta$ is the margin factor.

Based on Equation (8), the Adam optimization algorithm is used to minimize the model parameters. After the model training, for the test set sample $\left(m_i,S_i\right)\in D_{test}$, entity linking is achieved according to Equation (9):

$$e^{*}=\underset{e\in E_{test}^{+}}{\arg \max } g\left(e,m_i,S_i\right)$$

(9)

In summary, the entity linking based on context attention mechanism and distance learning can be summarized as Algorithm 1.

Algorithm 1: Entity Linking Based on Context Attention Mechanism and Distance Learning.

Require: $\mathrm{Event}\mathrm{Text}S=\left\{w_1,w_2,\cdots ,w_N\right\}$$;\mathrm{Entity}\mathrm{Mention}m=\left\{w_h,\cdots ,w_k\right\}$$;\mathrm{Positive}\mathrm{and}\mathrm{Negative}\mathrm{Candidate}\mathrm{entity}\mathrm{sets}{E}^{+}$$,E^{-}$ 1. The entity $e$ in $E^{+}$$,E^{-}$$\mathrm{is}\mathrm{mapped}\mathrm{to}\mathrm{vector}{\mathit{x}}_e$ using word2vec. Event text $S$$\mathrm{is}\mathrm{mapped}\mathrm{to}\mathrm{vector}\mathit{X}=\left\{{\mathit{x}}_1,{\mathit{x}}_2,\cdots ,{\mathit{x}}_N\right\}$, and entity mention $m$$\mathrm{is}\mathrm{mapped}\mathrm{to}\mathrm{vector}{\mathit{X}}_m=\left\{{\mathit{x}}_h,\cdots ,{\mathit{x}}_t\right\}$ using the BERT model; 2. The association scores of each word in the event text $S$ with the entity $e$ are calculated according to Equation (3); 3. The word set $\overline{S}$ corresponding to the top $R$ ranked words is selected based on the ranking of the association scores from highest to lowest; 4. Based on the word set $\overline{S}$ and Equation (5), the attention weights are calculated; 5. $\mathrm{Calculate}\mathrm{the}\mathrm{representation}\mathrm{vector}{\mathit{X}}_m^{att}$ of entity mention $m$ according to Equation (6); 6. $\mathrm{Feed}{\mathit{x}}_e$$,{\mathit{X}}_m^{att}$ into the feedforward neural network and output according to Equation (7); 7. The training loss function is calculated according to Equation (8), and the Adam optimization algorithm is used to minimize the model parameters; 8. $\mathrm{After}\mathrm{model}\mathrm{training},\mathrm{for}\mathrm{the}\mathrm{test}\mathrm{set}\mathrm{sample}\left(m_i,S_i\right)\in D_{test}$, entity linking is achieved according to Equation (9).

4.3. Relation Extraction

For the relations of the PEM model, the corresponding entities can be extracted from the text through NER and entity linking, which in turn can be used to directly build the binary relations between entity pem: Aggressor, pem: Victim and pem: Location, and pem: Date. Therefore, this section focuses on extracting the behavioral relations between entity pem: Aggressor and pem: Victim.

The text sequence of the containing entity pair of pem: Aggressor and pem: Victim $C=\left\{w_1,w_2,\cdots ,w_s\right\}$ is transformed into a pre-trained coding sequence $\mathit{X}=\left\{{\mathit{x}}_1,{\mathit{x}}_2,\cdots ,{\mathit{x}}_s\right\}$ by the BERT model. For word vector ${\mathit{x}}_i\in {ℝ}^{d_w}$, their relative distance to the target entity in the sentence is calculated, i.e., $d_1,d_2$. Its positional embedding vector is subsequently obtained by consulting the positional embedding table [51] ${\mathit{x}}_i^{d_1},{\mathit{x}}_i^{d_2}\in {ℝ}^{d_p}$, and it is spliced with the word vector ${\mathit{x}}_i$ as a word embedding representation ${\mathit{q}}_i\in {ℝ}^d\left(d=d_w+2d_p\right)$. Repeating the above steps, the text sequence $C=\left\{w_1,w_2,\cdots ,w_s\right\}$ is transformed into the embedding representation $\mathit{C}=\left\{{\mathit{q}}_1,{\mathit{q}}_2,\cdots ,{\mathit{q}}_s\right\}\in {ℝ}^{s\times d}$. Subsequently, a piecewise convolutional neural network (PCNN) [52] is used to extract text sequence features. This consists of a convolutional layer and piecewise max pooling layer. The output ${\mathit{c}}_{i,j}$ from the convolutional layer is calculated as follows:

$${\mathit{c}}_{i,j}={\mathit{w}}_i{\mathit{q}}_{j:j+l-1}$$

(10)

where ${\mathit{w}}_i\in {ℝ}^{l\ast d}$ denotes the weight matrix corresponding to the $i$-th convolutional kernel, $l$ denotes the size of the convolutional kernel, and ${\mathit{q}}_{j:j+l-1}$ denotes the splicing of word embedding representation from ${\mathit{q}}_j$ to ${\mathit{q}}_{j+l-1}$.

For the text sequence containing entity pair of pem: Aggressor and pem: Victim, the corresponding ${\mathit{c}}_{i,j}$ is divided into three segments $\left\{{\mathit{c}}_{i1},{\mathit{c}}_{i2},{\mathit{c}}_{i3}\right\}$ by the entity pairs. Max pooling is applied to each segment separately to produce three different output features ${\mathit{p}}_i=\left\{p_{i1},p_{i2},p_{i3}\right\}$. Afterwards, they are concatenated into a feature vector ${\mathit{p}}_{1:n}\in {ℝ}^{3n}$, where $n$ represents the number of convolutional kernels. Finally, the hyperbolic tangent function $\mathit{s}=\tan \mathrm{h}\left({\mathit{p}}_{1:n}\right)$, which is the activation function, is applied to the feature vector ${\mathit{p}}_{1:n}$. The dimensions of the feature vector $\mathit{s}$ are fixed and are no longer related to the length of the text sequence.

A set of text sequences containing the same entity pair is encapsulated into one bag. Suppose the bag $i$ contains $m_i$ text sequences, which can be represented as $\left\{{\mathit{s}}_1^{\left(i\right)},{\mathit{s}}_2^{\left(i\right)},\cdots ,{\mathit{s}}_{m_i}^{\left(i\right)}\right\}$. The representation vector ${\mathit{b}}_i\in {ℝ}^{3n}$ of the bag $i$ is obtained by calculating the weighted sum of $m_i$ text sequences:

$${\mathit{b}}_i={\displaystyle \sum _{j=1}^{m_i}{\alpha }_j^{\left(i\right)}{\mathit{s}}_j^{\left(i\right)}}$$

(11)

where ${\alpha }_j^{\left(i\right)}$ is the attention weight assigned to the $j$-th text sequence in the bag $i$, calculated as follows:

$${\alpha }_j^{\left(i\right)}={\displaystyle \frac{\exp \left({\displaystyle \sum _{j^{\prime }=1,\cdots ,m_i\wedge j^{\prime }\ne j}{\mathit{s}}_j^{\left(i\right)}{\left({\mathit{s}}_{j^{\prime }}^{\left(i\right)}\right)}^T}\right)}{{\displaystyle {\sum }_{k=1}^{m_i}\exp \left({\displaystyle \sum _{j^{″}=1,\cdots ,m_i\wedge j^{″}\ne j}{\mathit{s}}_k^{\left(i\right)}{\left({\mathit{s}}_{j^{″}}^{\left(i\right)}\right)}^T}\right)}}}$$

(12)

Equation (12) shows that if the $j$-th text sequence in the bag $i$ has a high similarity to the other text sequences in the bag, it will be assigned a larger attention weight when calculating the representation vector ${\mathit{b}}_i$ of the bag $i$. Finally, the representation vector $\left\{{\mathit{b}}_1,{\mathit{b}}_2,\cdots ,{\mathit{b}}_M\right\}$ of all bags is calculated.

Bags containing the same relation label are packaged into a bag-group $\mathit{G}=\left\{{\mathit{b}}_1,{\mathit{b}}_2,\cdots ,{\mathit{b}}_m\right\}\in {ℝ}^{m\times 3n}$. ${\mathit{G}}_{avg}$ is first calculated to express the overall discriminative features of this bag-group.

$${\mathit{G}}_{avg}={\displaystyle \frac{1}{3n}}{\displaystyle \sum _{i=1}^{3n}{\mathit{G}}_i^T\mathit{G}}$$

(13)

where ${\mathit{G}}_i$ denotes the $i$-th row of the matrix $\mathit{G}$. Interactions are designed on this basis to calculate the similarity ${\mathrm{sim}}_i$ between ${\mathit{G}}_{avg}$ and ${\mathit{b}}_i$:

$${\mathrm{sim}}_i={\displaystyle \frac{{\mathit{G}}_{avg}\cdot {\mathit{b}}_i^T}{{\Vert {\mathit{G}}_{avg}\Vert }_2{\Vert {\mathit{b}}_i^T\Vert }_2}}$$

(14)

Similarity ${\mathrm{sim}}_i$ can be considered as the similarity of the overall discriminative features that the representation vector ${\mathit{b}}_i$ is able to represent the bag-group $\mathit{G}$. On this basis, we calculate the attention weight ${\beta }_i$ assigned to the representation vector ${\mathit{b}}_i$.

$${\beta }_i={\displaystyle \frac{\exp \left({\mathrm{sim}}_i\right)}{{\displaystyle {\sum }_{j=1}^m\exp \left({\mathrm{sim}}_j\right)}}}$$

(15)

Then, we calculate the representation vector $\mathit{g}\in {ℝ}^{3n}$ of the bag-group $\mathit{G}$.

$$\mathit{g}={\displaystyle \sum _{i=1}^m{\beta }_i{\mathit{b}}_i}$$

(16)

Finally, a fully connected layer is added to the end of the above model to implement model training, using the negative log-likelihood as the training objective function.

$$J\left(\theta \right)=-{\displaystyle \sum _{i=1}^{\left|g\right|}\log p\left({\mathit{r}}_i|{\mathit{g}}_i;\theta \right)}=-{\displaystyle \sum _{i=1}^{\left|g\right|}\log \left(\mathrm{softmax}\left(\tanh \left({\mathit{g}}_i\right)\cdot {\mathit{r}}_i^T+b\right)\right)}$$

(17)

where $\mathit{r}\in {ℝ}^{3n}$ denotes the label of the bag-group, $\left|g\right|$ denotes the number of bag-groups, $\theta$ denotes the model parameters, and $b$ denotes the bias parameters. Based on Equation (17), the Adam optimization algorithm is used to minimize the model parameters.

After the model training, relation prediction is performed using Equation (18):

$$k^{*}=\arg \underset{k}{\max }\left(\mathrm{softmax}\left(\mathit{s}\cdot {\mathit{r}}_k^T+b\right)\right)$$

(18)

The proposed RE method can be summarized as Algorithm 2.

Algorithm 2: Proposed RE Method.

Require: Text sequence containing entity pairs of pem: Aggressor and pem: $\mathrm{Victim}C=\left\{w_1,w_2,\cdots ,w_s\right\}$ 1. $\mathrm{The}\mathrm{text}\mathrm{sequence}C=\left\{w_1,w_2,\cdots ,w_s\right\}$$\mathrm{is}\mathrm{transformed}\mathrm{into}\mathrm{the}\mathrm{embedding}\mathrm{representation}\mathit{C}=\left\{{\mathit{q}}_1,{\mathit{q}}_2,\cdots ,{\mathit{q}}_s\right\}$; 2. PCNN is used to extract text sequence features $\mathit{s}$; 3. $m_i$ text sequences containing the same entity pair are encapsulated in bag $i$ $\left\{{\mathit{s}}_1^{\left(i\right)},{\mathit{s}}_2^{\left(i\right)},\cdots ,{\mathit{s}}_{m_i}^{\left(i\right)}\right\}$; 4. $\mathrm{According}\mathrm{to}\mathrm{Equations}\left(11\right)\mathrm{and}\left(12\right),\mathrm{representation}\mathrm{vector}{\mathit{b}}_i$ of the bag $i$$\mathrm{is}\mathrm{obtained}\mathrm{by}\mathrm{calculating}\mathrm{the}\mathrm{weighted}\mathrm{sum}\mathrm{of}{m}_i$$\mathrm{text}\mathrm{sequences},\mathrm{and}\mathrm{finally},\mathrm{the}\mathrm{representation}\mathrm{vectors}\left\{{\mathit{b}}_1,{\mathit{b}}_2,\cdots ,{\mathit{b}}_M\right\}$ of all bags are calculated; 5. $\mathrm{Bags}\mathrm{containing}\mathrm{the}\mathrm{same}\mathrm{relation}\mathrm{label}\mathrm{are}\mathrm{packaged}\mathrm{into}\mathrm{a}\mathrm{ bag-group }\mathit{G}=\left\{{\mathit{b}}_1,{\mathit{b}}_2,\cdots ,{\mathit{b}}_m\right\}$; 6. $\mathrm{The}\mathrm{overall}\mathrm{discriminative}\mathrm{features}{\mathit{G}}_{avg}$ of the bag-group are calculated according to Equation (13); 7. $\mathrm{Calculate}\mathrm{the}\mathrm{similarity}\mathrm{between}{\mathit{G}}_{avg}$$\mathrm{and}{\mathit{b}}_i$ according to Equation (14); 8. $\mathrm{The}\mathrm{attention}\mathrm{weight}{\beta }_i$$\mathrm{assigned}\mathrm{to}\mathrm{the}\mathrm{representation}\mathrm{vector}{\mathit{b}}_i$ is calculated according to Equation (15); 9. The representation vector g of the bag-group G is calculated according to Equation (16); 10. The training loss function is calculated according to Equation (17), and the Adam optimization algorithm is used to minimize the model parameters. 11. After the model training, relation prediction is performed using Equation (18)

5. Experiment and Analysis

5.1. Data Sources

The source of the soft data in this study was Worldwide Threats to Shipping (WWTTS), which contains a summary of piracy incidents and hostile acts against global commercial shipping since 1994. The data can be obtained in https://msi.nga.mil/Piracy, URL (accessed on 21 December 2023). The data are in English text and annotated automatically based on DeepKE (v2.2.7). Automatic annotation significantly reduces the need for manual labeling, thereby decreasing the time and cost. It also diminishes biases caused by human factors, improving the consistency and objectivity of data annotation. Additionally, training the proposed model with data that has been automatically annotated can lessen the dependency on large model of DeepKE, thereby promoting the development of the proposed method to a lightweight model and better adapt to domain-specific (pirate incident) entity and relation extraction tasks. In this study, 7580 events were extracted from the WWTTS records of the last 8 years (2015–2023) for experimental validation.

5.2. Performance Validation of NER

For the performance validation of BERT-BiLSTM-CRF-based NER, the event text was labeled automatically based on DeepKE using the BIO format according to Table 2, and a NER corpus was built. It was subsequently divided into training and test sets in a ratio of 8:2. The results of the experiments are shown in

Table 3. NER results.

Serial Number	Label	Precision	Recall	F1-Score
1	B-Aggressor	0.9045	0.9225	0.9134
2	I-Aggressor	0.9160	0.9047	0.9103
3	B-Victim	0.9022	0.9118	0.9070
4	I-Victim	0.9239	0.9203	0.9221
5	B-Location	0.9524	0.9524	0.9524
6	I-Location	0.9416	0.9484	0.9450
7	B-Data	0.9875	0.9925	0.9900
8	I-Data	0.9917	0.9913	0.9915
9	O	0.9441	0.9287	0.9363
10	Overall	0.9404	0.9414	0.9409

and Figure 2.

From the results in Table 3 and Figure 2, it is clear that the BERT-BiLSTM-CRF model scores somewhat differently across entities. Compared to the entity pem: Location and pem: Data, the F1-score of entity pem: Aggressor and pem: Victim was lower. However, the F1-score of all types of entities reached more than 0.9, and the average recognition rate reached more than 0.94. Overall, the BERT-BiLSTM-CRF model performed well in NER.

Furthermore, several methods were used for comparison: (i) Hidden Markov Model (HMM); (ii) Machine Learning CRF model; (iii) Deep Learning BiLSTM model; and (iv) Deep Learning BiLSTM-CRF model. The above four named-entity algorithms were tested on the same dataset, and the test results were compared with the BERT-BiLSTM-CRF model. The experimental results of each model are shown in

Table 4. Comparison results of NER of each model.

Model	Precision	Recall	F1-Score
HMM model	0.8116	0.7961	0.8037
CRF model	0.8436	0.8313	0.8374
BiLSTM model	0.8671	0.8784	0.8664
BiLSTM-CRF model	0.9029	0.8979	0.9004
BERT-BiLSTM-CRF model	0.9404	0.9414	0.9409

and Figure 3.

As can be seen from Table 4 and Figure 3, the deep learning models outperformed the traditional machine learning models HMM and CRF in all respects. BiLSTM-CRF was obtained by adding sequence labeling CRF to the deep learning model, and it outperformed the BiLSTM model in all respects, suggesting that the CRF improves the performance of the model after taking into account the global label information of the sequences. After adding the BERT model to the deep learning BiLSTM-CRF model, F1-score was improved by 4.05%, which indicates that the proposed BERT-BiLSTM-CRF model has an obvious advantage in the NER of pirate events compared to other models. Thereby, the quality of the knowledge graph construction for pirate events can be improved.

5.3. Performance Validation of Entity Linking

Firstly, all the events were divided into training and test sets in the ratio of 8:2. For each entity mention in the event text of the training set, the matching heuristics mentioned in Section 4.2 were used to build the training candidate set: $E_{train}^{+}$ and $E_{train}^{-}$, where $\left|E_{train}^{+}\right|=\left|E_{train}^{-}\right|=20$. For each entity mention in the event text of the test set, matching heuristics were also used to build the test candidate set $E_{test}^{+}$, where $\left|E_{test}^{+}\right|=20$. The difference is that $E_{test}^{+}$ includes entity mentions to correctly corresponding entities.

Table 5. Entity linking results for different entity types.

Entity Types	Precision	Recall	F1-Score
pem: Aggressor	0.8965	0.9017	0.8991
pem: Victim	0.8739	0.8693	0.8716
pem: Location	0.9362	0.9540	0.9450
pem: Data	0.9912	0.9925	0.9919
Average	0.9220	0.9244	0.9269

and Figure 4 show the results of the test candidate set. The proposed model showed some variability in entity linking results for different entity types. Specifically, for the entity pem: Data and pem: Location, Precision, Recall, and F1-score of entity linking reached values above 0.93, especially for entity pem: Data, whose Precision, Recall, and F1-score of entity linking were all close to 1. This is due to the fact that these two types of entities are more standardized in their text representations, and there is rarely any ambiguity in their representations. For entity pem: Aggressor and pem: Victim, their Precision, Recall, and F1-score of entity linking were above 0.86, which are lower compared to those of pem: Data and pem: Location. This is due to the fact that these two types of entities have some diversity in textual representations, making the entity linking results subject to some error. However, overall, the average Precision, Recall, and F1-score of the proposed model reached more than 0.92. In particular, the proposed model achieved good test results by conducting entity linking training on the basis of unlabeled texts, which verifies the excellent performance of the proposed model of entity linking in specific vertical domains, consequently helping to improve quality of knowledge graph construction for pirate events.

5.4. Performance Validation of Relation Extraction

After NER and entity linking, the sentences containing entity pair pem: Aggressor and pem: Victim were selected. Next, the relation type was manually discriminated for a particular entity pair, and all sentences containing that entity pair were subsequently packaged into a bag; meanwhile, the above relation type is set as the relation type label of that bag, which serves as the basis for the experimental dataset. On this basis, bags containing the same relation label are packaged into a bag-group.

The performance of the proposed model in RE was evaluated, and the experimental results are shown in Figure 5 and

Table 6. AUC of seven precision–recall curves for the behavior relations.

Relation Types	AUC
pem: Attacking	0.733
pem: Boarding	0.719
pem: Firing upon	0.722
pem: Kidnapping	0.754
pem: Hijacking	0.744
pem: Robbing	0.707
pem: Suspiciously approaching	0.746

. Little variation can be seen in the distribution of precision–recall curves for different behavior relations. In addition, their corresponding AUC values are more similar, and all of them reach values over 0.7, which proves that the proposed model is effective in categorizing all the behavior relations under study.

Subsequently, to verify the superior performance of the proposed model, it was compared with six existing state-of-the-art methods: self-attention mechanism and convolution and self-selective attention (SelfCON + SATT) [44], multicast [53], DSRE variational autoencoder (DSREVAE) [54], false-negative adversarial network (FAN) [55], contrastive instance learning (CIL) [56], and passage relation extraction (PARE) [57]. The results of the experiments are shown in Figure 6 and

Table 7. AUC of seven precision–recall curves of our model and competitors.

Model	AUC
SelfCON + SATT	0.672
Multicast	0.681
DSREVAE	0.707
FAN	0.656
CIL	0.688
PARE	0.625
Ours	0.723

.

As shown in Figure 6, by observing the distribution of precision–recall curves, it can be seen that the vast majority of recalls of the proposed model are better than those of other models. The AUC values can quantitatively characterize the RE performance of different models. Table 7 shows the AUC values corresponding to the precision–recall curves of the proposed model and competitors. The AUC of the proposed model achieves the highest value and is 1.6–9.8% higher than those of other models, which further proves the superiority of the proposed model in RE, consequently helping to improve quality of knowledge graph construction for pirate events.

5.5. Knowledge Graph Generation for Pirate Events

In this section, the corresponding knowledge graphs were generated based on two simple pirate event texts. The texts of the pirate events are as follows:

• Pirate event 1: On 30 July 2019, Somali sailors boarded the general cargo ship OYA underway near position 04:10N-006:59E, 15 nm southwest of Bonny Island. The pirates kidnapped five crewmen and escaped.
• Pirate event 2: On 2 January 2019, Somali sailors attacked a small container ship M/V MANDY, near position 05:28N-002:21E, 55 nm south of Cotonou. Six crewmen were kidnapped.

Based on the event text, the knowledge triples are obtained as shown in

Table 8. Knowledge triples extracted based on pirate events 1 and 2.

Subject	Object	Relation
Somali sailors	cargo ship OYA	pem: boarding
Somali sailors	five crewmen	pem: kidnapping
Somali sailors	30 July 2019	pem: Aggressor-Date
Somali sailors	04: 10N-006: 59E	pem: Aggressor-Location
cargo ship OYA	30 July 2019	pem: Victim-Date
cargo ship OYA	04: 10N-006: 59E	pem: Victim-Location
five crewmen	30 July 2019	pem: Victim-Date
five crewmen	04: 10N-006: 59E	pem: Victim-Location
Somali sailors	container ship M/V MANDY	pem: attacking
Somali sailors	six crewmen	pem: kidnapping
Somali sailors	2 January 2019	pem: Aggressor-Date
Somali sailors	05: 28N-002:21E	pem: Aggressor-Location
container ship M/V MANDY	2 January 0219	pem: Victim-Date
container ship M/V MANDY	05: 28N-002:21E	pem: Victim-Location
six crewmen	2 January 2019	pem: Victim-Date
six crewmen	05: 28N-002:21E	pem: Victim-Location

using the NER, entity linking, and relation extraction model proposed in this paper. The knowledge triples are stored through the neo4j database and visualized in Figure 7.

By comparing the pirate event text with the knowledge triples shown in Table 8 as well as the pirate event knowledge graph shown in Figure 7, it can be seen that the NER, entity linking, and relation extraction model proposed in this paper are able to efficiently extract entities and accurately establish the relations between entities, which further strongly suggests that the proposed model performs well in the construction of knowledge graphs for pirate events, especially in instance layer filling. Therefore, it is able to effectively generate high-quality knowledge graphs for pirate events.

6. Conclusions

In this paper, an automated construction method of knowledge graphs for pirate events was proposed and studied. Firstly, a Pirate Event Model (PEM) was designed as a knowledge graph ontology model for pirate events, on the basis of which the study focused on the knowledge triple extraction method for pirate events. For entity extraction, the BERT-BiLSTM-CRF model was proposed for NER. In the process of entity linking, to address the lack of large amounts of labeled text data, the method of distant learning was introduced for entity linking model training. In addition, using the context attention mechanism, the basis was proposed so that the words that are helpful for disambiguation decision can be selected, enhancing the performance of entity linking model training. For relation extraction, on the basis of the traditional distant supervision relation extraction, which is based on the sentence-level attention mechanism, the bag-level and group-level attention mechanism methods were additionally proposed to further enhance the performance of distant supervision relation extraction. The experimental results demonstrate the effectiveness of the proposed method for the automated construction of knowledge graphs for pirate events.

The knowledge graphs for pirate events achieve the deep knowledge mining of maritime threat information, which can support the knowledge representation of pirate events, and comprehend spatiotemporal activity query and backtracking, so as to achieve the effect of semantic enhancement, which has a certain application value. Furthermore, the knowledge graphs for pirate events are conducive to subsequent maritime threat reasoning for maritime threat situational awareness in general. In future work, the author will conduct maritime threat reasoning research based on the existing knowledge graphs for pirate events, including identifying dangerous areas, predicting threat trends, and detecting potential hostile behaviors, so as to comprehensively achieve maritime threat situational awareness and assist in the detection of illegal maritime activities.