Event Argument Extraction for Rainstorm Disasters Based on Social Media: A Case Study of the 2021 Heavy Rains in Henan

Abstract

Rainstorm disasters have wide-ranging impacts on communities, but traditional information collection methods are often hampered by high labor costs and limited coverage. Social media platforms such as Weibo provide new opportunities for monitoring and analyzing disaster-related information in real-time. In this paper, we present ETEN_BERT_QA, a novel model for extracting event arguments from Weibo rainstorm disaster texts. The model incorporates the event text enhancement network (ETEN) to enhance the extraction process by improving the semantic representation of event information in combination with event trigger words. To support our approach, we constructed RainEE, a dataset dedicated to rainstorm disaster event extraction, and implemented a two-step process, as follows: (1) event detection, which identifies trigger words and classifies them into event types, and (2) event argument extraction, which identifies event arguments and classifies them into argument roles. Our ETEN_BERT_QA model combines ETEN with a BERT-based question-answering mechanism to further improve the understanding of the event text. Experimental evaluations on the RainEE and DuEE datasets show that ETEN_BERT_QA significantly outperforms the baseline model in terms of accuracy and the number of event argument extractions, validating its effectiveness in analyzing rainstorm disaster-related Weibo texts.

1. Introduction

Due to the unique climatic conditions and topography of the country, rainstorm disasters occur frequently and pose a serious threat to human life, infrastructure, and economic stability [1]. The increasing frequency and severity of rainstorm disasters underscore the imperative for effective disaster management. However, traditional approaches encounter considerable obstacles, including the necessity for prompt information, an expedient response, and a unified allocation of resources [2]. It is frequently the case that traditional data sources, such as official reports, are unable to provide real-time, detailed insights that are critical in emergencies [3]. In this context, the extraction of pertinent information from unstructured texts pertaining to rainstorm disasters has become a crucial step in enhancing disaster response and management. The process of identifying and structuring critical event information, otherwise known as event extraction, can provide emergency services with actionable insights that facilitate faster, more targeted interventions.

In recent years, Weibo has become a key source for obtaining data on rainstorm disaster events by virtue of its wide coverage, real-time update capability, and embedded geolocation information [4]. Compared with traditional official report data, Weibo data are instantaneous and can reflect the latest situation of those affected by disasters. In contrast, official reports are usually released late and may lack detailed descriptions of specific disaster areas [5]. The immediacy and extensiveness of Weibo data give it a unique advantage in extracting structured event information from massive unstructured rainstorm disaster texts. Event extraction is a key part of information extraction, which can automatically identify and organize event information in unstructured text and transform it into actionable structured data [6]. This capability is particularly important in rainstorm disaster response because rapid and accurate access to information about the relevant people, content, location, and time of rainstorm disaster events can effectively support decision-making and coordination in emergency management.

Recent research has explored various methods to enhance the effectiveness of event extraction in social media. Panem et al. introduced a method in 2014 to automatically extract structured information (such as attribute–value pairs and fact triplets) from tweets related to natural disasters, which improves the relevance of search results and provides critical real-time information for crisis management [7]. This work highlights the potential of utilizing social media data to quickly and effectively respond to disasters. Similarly, Edouard et al. proposed a graph-based event extraction method in 2017, which detects and clusters tweets describing the same disaster event by constructing a temporal event graph, significantly improving the accuracy of event information extraction [8]. Typically, event extraction can be divided into two subtasks [9]: (1) event detection, where the goal is to identify trigger words in the event text and categorize them into event types. Trigger words are the core words of an event, which can clearly express the occurrence of the event, usually in the form of nouns or verbs. (2) Event argument extraction, the goal of which is to identify event arguments in event texts and classify them into argument roles. An event argument is a participant in an event, while an argument role is the role that an event argument plays in an event [10].

Despite the clear structure provided by these subtasks, current research mainly treats event extraction as a classification problem, focusing on event trigger word extraction and classification as well as event argument role extraction [11,12,13,14]. Despite the progress made, classification-based approaches still lack data and require large amounts of training data to ensure good performance [15,16]. In addition, such methods are usually unable to handle new event types that are never encountered during training [17].

In this study, we introduce a new event extraction learning paradigm that aims to address the above issues simultaneously. Our main motivation is that event extraction can essentially be viewed as a machine reading comprehension (MRC) problem [18] involving text comprehension and matching to discover information about specific events in a text.

This implies a new approach to address event extraction, which has two main advantages. Firstly, by treating event extraction as MRC, we can augment the event extraction task by utilizing the latest advancement in MRC, BERT [19], which may significantly enhance the inference process in the model. Second, we can directly utilize the rich MRC dataset to enhance event extraction, which may alleviate the data scarcity problem (this is known as cross-domain data augmentation) [20]. The second advantage also opens the door to zero-sample event extraction for invisible event types, we can list the questions that define their patterns and use the MRC model to retrieve the answers as event extraction results, instead of obtaining training data for them in advance.

To connect MRC and event extraction, the key challenge is to generate relevant questions that describe the event scenario. Note that we cannot use a supervised problem-generation approach [21,22,23] because of the lack of aligned problem–event pairs. Previous work on connecting MRC and other tasks typically used manually designed templates [24,25,26]. For example, in QA-SRL, the question for the predicate publish is always ‘Who published something?’, regardless of the context. Such a question may not be sufficient to instruct the MRC model to find an answer.

The aforementioned challenges are addressed through the introduction of an unsupervised question generation process, which is designed to produce questions that are both relevant and context-dependent. Specifically, our approach assumes that each question can be decomposed into two parts, reflecting the event type and argument role information, respectively. To illustrate, the question “Q” can be decomposed into two constituent parts: “Personal safety?” and “Number of trapped persons?” In order to generate question expressions, a template-based generation method has been devised which combines event types and argument roles. Subsequently, a BERT-based MRC model is constructed in order to answer each question and synthesize the answers into an event extraction result.

Deep learning methods are data-driven, and well-labeled event extraction datasets are essential for obtaining accurate event extraction results. Currently, representative publicly available event extraction datasets include the DuEE dataset.

In 2020, Li et al. [27] developed the DuEE dataset, a large-scale, high-quality, and diverse event extraction dataset. Available online: https://opendatalab.org.cn/OpenDataLab/DuEE (accessed on 17 October 2024). This publicly available Chinese event extraction dataset provides comprehensive coverage and promotes extensive research and evaluation in the field of event extraction.

The Weibo platform is an important social media platform through which users can publish and share rainstorm disaster-related information in real-time. Collecting and processing rainstorm disaster data can effectively solve the problem of scarcity of data sources for event extraction, thus improving the completeness of the event extraction task. Using Weibo data as a data source, the best deep learning method for event extraction can be identified, which in turn facilitates subsequent research in the field of rainstorm disasters. Therefore, it is crucial to generate event extraction datasets related to rainstorm disasters.

Inspired by the work mentioned above, the main contributions of this work are as follows:

• In view of the importance of rainstorm disaster event information extraction for subsequent natural language processing tasks, this paper generates a rainstorm event extraction (RainEE) dataset based on rainstorm disaster texts on Weibo.
• In order to enhance the perception of trigger words in event text and improve the accuracy and completeness of event argument extraction, a BERT-based question and answer network (ETEN_BERT_QA) is proposed with an event text enhancement network in the coding section.
• To verify the superiority of our proposed network, we conduct experiments on the RainEE dataset, the DuEE dataset, and the 20 July 2021 Henan rainstorm case. The results show that our proposed network outperforms the baseline model in both accuracy and number of event extractions.

2. Materials and Methods

This section introduces the rainstorm disaster event extraction dataset used in this paper.

2.1. Materials

This section introduces the process of conducting a self-made event extraction dataset in the field of rainstorm disasters, as well as the existing event extraction datasets that are currently widely used.

2.1.1. Production of RainEE

Due to the lack of event extraction datasets dedicated to rainstorm disasters, we developed the Rainstorm Event Extraction (RainEE) dataset to fill this research gap. The RainEE dataset is based on a large amount of user-generated content related to rainstorm disasters on Weibo, especially those that contain information about the events and is designed to improve the applicability of rainstorm event extraction methods. Compared with traditional data sources, the immediacy and extensiveness of Weibo data enable us to capture rainstorm disaster field information quickly and accurately. This immediacy not only helps to gain an in-depth understanding of the dynamic development of the disaster event but also provides data support for timely emergency response measures and rational allocation of resources, thus enhancing the responsiveness and decision-making efficiency of disaster management.

In order to construct the RainEE dataset, a bespoke web crawler framework was devised and deployed with the specific objective of extracting content from Weibo pertaining to rainstorm-related disasters. The initial step involved the construction of a keyword list that was highly relevant to rainstorm events. Subsequently, between 15 July 2021 and 3 August 2021, a further 70,000 or more Weibo were collected using the keywords ‘Zhengzhou flood’ and ‘Henan rainstorm’. The selected time period was chosen to encompass the peak of the rainstorm disaster, ensuring that the dataset can comprehensively reflect the public’s real-time response to the disaster and the latest developments. The selection of this period is primarily based on the occurrence of the rainstorm disaster, as shown in Figure 1. Statistical analysis of comment volume reveals a significant surge in posts and user engagement around July 20, coinciding with the peak of the disaster. This sharp increase in activity reflects heightened public attention and ongoing discourse surrounding the event, which offers valuable data for subsequent event extraction and disaster management research.

During the data preprocessing stage, it became evident that retweeting and copying behaviors are pervasive on social media, which presents a significant challenge for text processing. To address this issue, a series of pre-processing techniques were implemented to ensure the high quality and relevance of the data. Initially, redundant text is identified and removed through the utilization of deduplication techniques. Subsequently, texts pertaining to rainstorm disasters were filtered using a keyword-based template filtering method. In this stage, we initially assume that the data collected through keyword-based filtering represents real information for the specific disaster event (i.e., rainstorm disaster). However, as the data application scope expands, especially in multi-disaster scenarios, a series of validation methods, such as source reliability checks, sentiment analysis, and cross-platform comparisons, will be employed to further screen and distinguish true from false information to ensure data accuracy and reliability. Subsequently, the texts were subjected to a syntactic analysis to ascertain the soundness of their syntactic structure. The implementation of these preprocessing techniques resulted in a notable enhancement of the dataset quality, thereby establishing a robust foundation for the subsequent annotation work.

In the process of event annotation, we adhered rigorously to the established guidelines for annotation set forth by the DuEE dataset, to ensure consistency and accuracy in the annotation process. In order to define the various event types, I have made use of the disaster classification framework set out in the China standard SL579-2012 [28]. This provides a scientific basis for the systematic classification of rainstorm-related disaster events. In practice, depending on the amount and type of data, some categories were merged and adjusted to ensure reasonable classification and full utilization of the data. The final dataset’s event schema comprises 7 event types and 28 argument roles, including waterlogging, vegetation damage, traffic congestion, rescue operations, personal safety, landslides, and building collapse/damage.

Table 1. Event schema.

Event Types	Argument Roles
Waterlogging	Time, location, depth of water accumulation
Vegetation damage	Time, location, vegetation name
Transportation	Time, location, transportation name
Rescue operations	Time, location, rescuer, rescued, supplies/equipment
Personal safety	Time, location, victims, number of missing persons, number of dead, number of injured, number of trapped persons, number of disaster victims
Landslides	Time, location, landslide subject
Building collapse/damage	Time, location, subject of collapse/damage

illustrates the argument roles pertinent to each event type, including the victim, time, and location. Furthermore, a visual representation of the event schema and the outcomes of event detection and event role extraction are presented in Figure 2. The classification system facilitated not only the more efficient organization of the dataset but also provided a more targeted basis for subsequent emergency response and rescue plans for different incident types.

To address the possibility of new disaster events emerging in future research, it is important to expand the dataset to include new disaster types as they arise. This will help optimize model training and improve its adaptability. In particular, the continuous evolution of disaster events may require adjustments to the classification system to ensure its relevance and accuracy. By incorporating emerging disaster types, future research can enhance the robustness of the model, providing more comprehensive support for emergency response efforts in a dynamic environment.

During the annotation process, we carefully analyzed the storm disaster text, identified and annotated the event trigger words, classified them into event types, and annotated the event arguments and classified them into argument roles. This information was recorded in a predefined format. The final dataset consists of 1432 labeled data entries, of which, 70% (1021 training data entries) form the training set, and 30% (411 entries) form the validation set. The training and validation sets were selected from data collected between 15 July 2021 and 3 August 2021, covering the period of the heavy rainstorm event in Henan. This specific time frame ensures that the dataset captures a representative range of social media posts related to the disaster event. These labeled events not only provide a comprehensive picture of the public’s response to the disaster but also provide a valuable reference for understanding the evolution of the disaster. To ensure the consistency and accuracy of the data annotations, we regularly review the annotations that have been made. Although we were the sole annotators, the rigorous measures taken ensured that the process of annotating the RainEE dataset met the necessary standards of quality and reliability. The entire process of building the dataset is shown in Figure 3.

2.1.2. Existing Event Extraction Datasets

This study utilized two distinct datasets: the self-constructed RainEE dataset and the publicly available DuEE dataset.

Table 2. Parameters of the datasets used in this experiment.

Dataset	Training Set	Test Set	Validation Set
DuEE	11,958	35,000	1498
RainEE	1021	/	411

provides a detailed comparison of the main parameters of these two datasets.

The DuEE dataset was made available by Baidu in 2020. It is the largest event extraction dataset in China to date, and its influence in this field is considerable. The DuEE dataset is meticulously partitioned into 11,958 training samples, 35,000 test samples, and 1498 validation samples, thereby establishing a robust foundation for the training and evaluation of event extraction models [27]. The DuEE dataset encompasses the most pertinent topics within Baidu search results, with data sourced from Baijiahao News, thereby ensuring the timeliness and relevance of the events. This diversity not only enriches the dataset but also enhances its applicability in different fields and environments, thus representing an important resource for evaluating the generalization ability of event extraction models.

Furthermore, the extensive coverage of diverse event types and argument roles inherent to the DuEE dataset presents a considerable challenge to researchers and practitioners alike. The dataset is notable for its complexity and comprehensiveness, which render it one of the most representative of Chinese event extraction tasks. This feature provides a robust foundation for benchmarking novel methods and techniques, ensuring valid and meaningful performance comparisons.

The RainEE dataset has been constructed with the specific intention of being highly relevant and serves as a pivotal benchmark for the extraction of information pertaining to disasters. In the context of emergency response, the timely and accurate dissemination of event information can have a significant impact on decision-making and the allocation of resources. The utilization of real-time Weibo social media data enables the reflection of the immediate response and actions of affected communities, as observed in the RainEE dataset.

In contrast to the DuEE dataset, which comprises a distinct test set, the RainEE dataset is not divided into separate test sets. This decision was made deliberately to concentrate on the principal event of 20 July 2021, the rainstorm in Henan. The substantial volume of data generated on Weibo social media, coupled with the diverse array of consequences associated with this event, offers a comprehensive repository for examining a multitude of incidents pertaining to heavy precipitation. In lieu of dividing the test set, we elected to conduct a comprehensive analysis of the model’s performance in this real-world scenario.

In the analysis process, we selected 6000 data entries from the crawled data as specific cases for the subsequent experimental part. Through this approach, we can gain a deeper understanding of the actual performance of the model in dealing with the complex challenges posed by the rainstorm disaster in Henan. In addition, it provides valuable insights and references for future research and disaster management strategies. By analyzing this specific case, we can comprehensively assess the adaptability and reliability of the model in responding to actual disaster events, thus providing an important basis for future improvement and application.

In conclusion, the DuEE dataset offers a comprehensive and heterogeneous foundation for event extraction, whereas the RainEE dataset concentrates on the particular domain of rainstorm disasters, providing a specialized resource for event extraction research. The combination of the two not only provides a comprehensive evaluation platform for event extraction methods but also advances the field and enhances the effectiveness of disaster response strategies.

2.2. Methods

This section describes the structure of each part of BERT_QA and the evaluation metrics of the network training results.

2.2.1. Optimizing the BERT_QA Model by Adding Event Text Augmentation Block

Developed by Devlin et al. at Google, BERT (Bidirectional Encoder Representation of Transformers) is a pre-trained language model that sets new standards in a wide range of Natural Language Processing (NLP) tasks, especially machine reading comprehension. BERT uses bidirectional transformer encoders, which, by simultaneously processing text in both pre- and post-contexts, can better capture semantic relationships and contextual nuances [29]. This bi-directional approach significantly improves BERT’s ability to model complex linguistic dependencies, enabling it to excel in tasks such as question-answering (QA) [30].

In QA tasks, BERT generates contextual representations of questions and paragraphs through fine-tuning. In the process, the model learns to predict where the answer will start and end in the context, enabling an accurate understanding of the text [31]. As shown in Figure 4, the architecture of the BERT model consists of a multi-layer transformer encoder that dynamically adapts its understanding of the context to the complexity of the input text [32,33].

This architecture makes BERT perform particularly well when dealing with multi-layered linguistic structures and can cope with the challenges in many NLP tasks, providing strong support for the model’s performance enhancement in the fields of text classification, named entity recognition, and machine translation.

Based on the BERT architecture, this study proposes ETEN_BERT_QA, which aims to enhance the model’s ability to process and understand event-related text through event trigger words. Compared to conventional BERT, which directly processes unaltered input text, our approach achieves more efficient information capture by preprocessing and enhancing event-specific text, especially by highlighting event trigger words. The aforementioned trigger words are identified in the event de-detection task and strategically enhanced in our approach, thus enabling the model to capture event-related information more efficiently, thereby improving the accuracy and effectiveness of the output results.

ETEN is based on the ability to highlight event trigger words in the appropriate context. This approach enables BERT to more effectively utilize these trigger words as key cues, thereby significantly enhancing the model’s ability to understand and interpret disaster scenarios. Compared to traditional approaches, our approach places more emphasis on the fundamental components of an event, thereby enhancing the model’s ability to parse the text of a specific event.

In addition, our model incorporates an intermediate linear layer after the BERT output that facilitates the connection between the BERT representation output and subsequent task processing. This layer combines a ReLU (rectified linear unit) activation function and a dropout operation to optimize feature extraction while reducing the risk of model overfitting. This design ensures that the model is not dependent on specific features in the training data, thus improving the generalization ability of the model.

The generated enhanced event representations can be used as inputs for subsequent tasks, including the question-answering (QA) task. In the QA task, the output of the event-text augmentation network goes through a final linear layer to predict the likelihood of a particular text span (i.e., answer). This layer helps the model to more accurately identify and extract key information from the event text, thus improving the performance of the machine reading comprehension task.

Figure 5 depicts the ETEN_BERT_QA structure, highlighting the interrelationships between the BERT layer, the intermediate linear layer, and the final output layer. This design optimizes the model’s ability to extract and understand event-related information, providing a powerful tool for event extraction tasks and a solid foundation for meeting complex disaster event extraction requirements.

These improvements markedly enhance the model’s capacity to process and interpret event-related text, thereby optimizing the performance of the event extraction task. By enhancing the representation of event trigger words and optimizing the output of the middle layer, the event text augmentation network can provide more accurate and relevant results. This capability is of particular importance when dealing with complex event-related linguistic phenomena, such as ambiguous wording, nested events, and diverse contextual cues. Consequently, the model not only enhances the precision of event recognition but also guarantees that the extracted information is contextualized and actionable, thereby providing crucial support for more effective disaster management and the development of more efficacious emergency response strategies.

2.2.2. Event Text Enhancement Network

In this study, we propose a machine question-answering approach based on a pre-trained model to effectively utilize event schema information for extracting relevant event arguments from a given event text. The task involves obtaining event argument role information $r$ from an event text $x_s$ containing an event type $x_{ts}$ and its corresponding trigger word $x_{tr}$. The objective of the model is to maximize the conditional probability $p\left(\left(\alpha |r,x_s,x_{ts},x_{tr}\right)\right)$, where $\alpha$ represents the most plausible event argument.

The design of question-answering (QA) text plays a crucial role in encoding and decoding processes within the pre-trained model. Therefore, crafting effective QA texts is essential for achieving the methodology’s goals. Machine QA texts are generated by integrating enhanced representations of question templates and event texts to leverage the capabilities of pre-trained models.

To ensure efficient processing and avoid excessive text length that may impair model performance, we use a prompt word template derived from prior knowledge, rather than a traditional natural language template. The question template is formulated as follows:

$$Q=\left\{x_{ts1},\dots ,x_{tsm}{,}^{\prime }{,}^{\prime },r_1,\dots ,r_n{,}^{\prime }{?}^{\prime }\right\}$$

Here, $x_{ts}$ is determined by the event detection subtask, and $r$ is obtained from the event schema corresponding to $x_{ts}$. The variables $m$ and $n$ represent the lengths of $x_{ts}$ and $r$, respectively.

To improve the model’s understanding of event text, the representation of trigger words is enhanced. This involves incorporating trigger words into the event text. The process is as follows:

• Event text construction: For the original event text $x_s$, the text is vectorized by the event content in the form of the following:

  $$x_s=\left\{x_{s1},\dots ,x_{sm}\right\}\in R^{n^{\prime }}$$
  Among them, $n^{\prime }$ represents the length after vectorization.
• Trigger word representation: The trigger word form obtained from the event detection subtask is as follows:

  $$x_{tr}=\left\{x_{tr1},\dots ,x_{tr{m}^{\prime }}\right\}\in R^{m^{\prime }}$$
• Enhanced representation construction: In order to improve the model’s ability to understand event text, we adopted the strategy of adding special separators ‘[SEP]’ before and after the trigger word $x_{tr}$. Specifically, we construct the enhanced text $X$, which is in the following form:

  $$X=\left\{x_{s1},x_{s2},\dots ,\left[SEP\right],x_{tr1},\dots ,x_{tr{m}^{\prime }},\left[SEP\right],\dots ,x_{s{n}^{\prime }}\right\}$$

The question template and the enhanced event text are integrated as the input C of the machine question-answering model. The integration form is as follows:

$$C=\left\{\left[CLS\right],Q,\left[SEP\right],X,\left[SEP\right],\left[PAD\right],\dots ,\left[PAD\right]\right\}$$

Here, $\left[CLS\right]$ marks the beginning, $\left[SEP\right]$ separates components, and $\left[PAD\right]$ ensures uniform input length.

Next, we will use the pre-trained model Chinese-BERT-WWM to encode the integrated question and answer text $C$ to obtain its hidden layer representation $H$.

$$H=BERT\left(C\right)\in R^{\left|C\right|*d}$$

where $\left|\mathrm{C}\right|$ represents the length of the input $C$ and $d$ represents the dimension of the hidden layer representation.

2.2.3. Event Extraction Evaluation Metrics

This paper adopts three accuracy evaluation indicators for instance-level and argument-level evaluation, which are widely used as evaluation criteria for event extraction: precision, recall and F1 score. $TP$ is the number of samples correctly classified as positive, $FP$ is the number of samples misclassified as positive, $TN$ is the number of samples misclassified as negative, and $FN$ is the number of samples correctly classified as negative. The above three precision evaluation metrics are calculated as follows:

This study employs three widely accepted accuracy metrics to evaluate event extraction performance at both the instance and argument levels: precision, recall, and F1 score.

• Precision measures the proportion of correctly identified positive samples out of all samples predicted as positive. It is calculated as follows:

  $$Precision={\displaystyle \frac{TP}{TP+FP}}$$

  where $TP$ (true positive) is the number of samples correctly classified as positive, and $FP$ (false positive) is the number of samples incorrectly classified as positive.
• Recall, also known as sensitivity, indicates the proportion of actual positive samples that are correctly predicted by the model. It is calculated as follows:

  $$Recall={\displaystyle \frac{TP}{TP+FN}}$$

  where $FN$ (false negative) is the number of samples that are positive but incorrectly classified as negative.
• The F1 score is the harmonic mean of precision and recall, providing a single metric to evaluate the model’s performance. It is calculated as follows:

  $$F1score={\displaystyle \frac{2\times Precision\times Recall}{Precision+Recall}}$$

These metrics offer a comprehensive assessment of the model’s accuracy in event extraction, balancing the trade-offs between precision and recall.

3. Experiment and Result

3.1. Experiment

The experiments were conducted in a computing environment configured with PyTorch 2.2.1 and CUDA 12.2, with the use of GPUs optimized to ensure efficient model training and evaluation. To validate the accuracy and effectiveness of the proposed model, a series of comprehensive experiments were conducted using both the customized RainEE dataset and the widely recognized DuEE dataset. This dual approach guarantees robust evaluation in disparate event extraction environments.

Furthermore, an investigation was conducted into a significant rainstorm that took place on 20 July 2021 in the Henan Province. The widespread impact of this event and the substantial volume of Weibo social media data it generated made it an invaluable real-world scenario for testing the model.

The principal objectives of this experiment are twofold: firstly, to evaluate the accuracy of event argument role extraction; and secondly, to quantify the total number of successful extractions performed by the model.

The structured experimental framework is designed to rigorously validate the performance of the proposed model and demonstrate its suitability for real-time disaster management applications.

3.2. Result

3.2.1. Accuracy Results for Event Argument Extraction

The results of the experiments conducted on the DuEE dataset are shown in

Table 3. Accuracy of event argument extraction on the DuEE dataset. The best results are highlighted in bold.

Network	Recall (%)	Precision (%)	F1 (%)
BERT_QA	72.7	73.9	73.3
ETEN_BERT_QA	76.9	75.7	76.3

. The F1 score of our proposed augmented model ETEN_BERT_QA is 76.3%, which is significantly improved by about 3% compared to the F1 score of the baseline model BERT_QA of 73.3%. This indicates that ETEN plays a significant role in improving the accuracy of the model in identifying and extracting event arguments.

This improvement can be attributed to ETEN’s emphasis on event trigger words, which allows the model to better understand complex event narratives and, thus, extract event-related details more accurately. In particular, when faced with complex and context-rich unstructured text, ETEN helps the model to more effectively identify key information that has been obscured.

Experiments conducted on the DuEE dataset not only validate the model’s performance in the event extraction task but also demonstrate its adaptability and robustness in dealing with a wide range of event types. The DuEE dataset covers a wide range of event types and provides a good test benchmark for the model’s generalization ability.

In conclusion, the experimental results on the DuEE dataset validate the effectiveness of our proposed model architecture and provide a solid foundation for subsequent evaluations based on the RainEE dataset.

Next, we evaluated the performance of the model on the RainEE dataset, and the results are shown in

Table 4. Accuracy of event argument extraction on the RainEE dataset. The best results are highlighted in bold.

Network	Recall (%)	Precision (%)	F1 (%)
BERT_QA	82.7	69.0	75.2
ETEN_BERT_QA	79.3	72.4	75.7

. The F1 score of our proposed model ETEN_BERT_QA is 75.7%. Although the recall of 79.3% is slightly lower than that of the benchmark BERT_QA model of 82.7%, the precision of our model is significantly higher than that of the BERT_QA model of 69.0%, reaching 72.4%. The improved accuracy supports our F1 score and highlights its strong performance.

The higher accuracy suggests that the ETEN_BERT_QA model can more reliably extract event arguments from complex, rainstorm disaster-related Weibo texts. This is particularly important in rainstorm disaster management, where the accuracy of the information can have a significant impact on disaster relief efforts.

In summary, these results demonstrate the effectiveness of our model in extracting event arguments from rainstorm disaster-related Weibo texts. These findings not only affirm the model’s ability to handle complex real-world data but also highlight its potential application in improving situational awareness and decision-making in emergency situations.

In summary, the design and improvement of the ETEN_BERT_QA model significantly improves the performance of the event argument extraction task for both general event types and specific disaster-related scenarios, proving its usefulness and effectiveness in real-world data processing. These results lay a solid foundation for future research and applications.

3.2.2. Quantitative Results and Specific Examples of Event Argument Extraction

In analyzing the DuEE dataset, we used a GlobalPointer-based event detection method for the 35,000 test texts in the dataset. Through the analysis, we identified 30,475 texts containing relevant event types and extracted event parameter roles from them.

As shown in

Table 5. Quantitative analysis of event argument extractions on the DuEE dataset. The best results are marked in bold.

Network	Total Number of Items	Number of Extractions
BERT_QA	30,475	25,369
ETEN_BERT_QA		28,261

, the baseline model BERT_QA successfully extracts event arguments from 25,369 texts. In contrast, our enhanced model ETEN_BERT_QA, which incorporates the trigger word enhancement method, successfully extracts event arguments from 28,261 texts with significantly improved performance. This result indicates that the model improvement effectively enhances its ability to recognize relevant event roles.

ETEN was particularly effective in providing additional cues to the model to help identify key event roles. By utilizing the contextual information associated with trigger words, ETEN_BERT_QA improves the accuracy and coverage of event parameter extraction. This is crucial for comprehensive event analysis.

These results highlight the strengths of our proposed event argument extraction model, emphasizing the potential of targeted enhancements to improve the performance of complex natural language processing tasks. Especially in areas such as disaster management, where timely and accurate information is critical.

Table 6. Detailed event argument extraction results on the DuEE dataset.

Network	Text	Event Argument Extraction
BERT_QA	There were multiple demonstrations in Paris on the 21st, local time, with preliminary statistics showing more than 10,000 people participating.	Organizational Behavior_Parade_Location: Paris
ETEN_BERT_QA		Organizational Behavior_Parade_Time: the 21st local time Organizational Behavior_Parade_Location: Paris Organizational behavior_Parade_number of marchers: more than 10,000 people

shows the specific results of the event argument extraction task, demonstrating the significant advantages of the ETEN_BERT_QA model in real scenarios. Compared to the benchmark model BERT_QA, ETEN_BERT_QA performs better in terms of both accuracy and completeness. This is mainly due to its enhanced textual representation capability, which enables it to identify event roles more efficiently.

Specifically, the ETEN_BERT_QA model not only performs well in basic event argument extraction (e.g., location) but also excels in handling complex event details (e.g., time and number of people marchers). This suggests that the model can better understand the nuances in text, especially in variable contexts, which are crucial for event extraction.

Overall, the ETEN_BERT_QA model significantly improves the quality of event argument extraction through a deeper understanding of context. These results not only validate the robust performance of the model in multi-scenario tasks but also demonstrate its potential application in complex event analysis.

To further assess the robustness of the proposed ETEN_BERT_QA, we conducted a comprehensive case study of the 20 July 2021 Henan rainstorm. This rainstorm was one of the most destructive rainstorms in modern Chinese history, causing extensive flooding, casualties, and significant property damage. Consequently, it offers an optimal setting for meticulous evaluation of the efficacy of our event extraction model.

As illustrated in

Table 7. Quantitative analysis of event argument extractions for the Henan rainstorm case. The best results are marked in bold.

Network	Total Number of Items	Number of Extractions
BERT_QA	6000	4713
ETEN_BERT_QA		5461

, the baseline BERT_QA model is capable of effectively extracting event arguments from 4713 transcripts, while the enhanced ETEN_BERT_QA model can extract event arguments from 5461 transcripts, representing a notable improvement of approximately 16%. This result demonstrates the efficacy of our model in accurately identifying and extracting relevant event arguments. The enhanced representation of trigger words plays a pivotal role in enhancing the model’s performance, enabling it to more accurately identify and contextualize event roles.

Furthermore, the case study illustrates the model’s capacity to discern intricate details, such as the precise location of the inundated area, the number of individuals involved in the rescue operation, and the temporal occurrence of the event. This level of detail is of critical importance to emergency management, as it allows emergency responders to make informed decisions based on real-time information.

In conclusion, the results of this case study confirm the effectiveness of our model in intricate real-world environments and emphasize the potential of ETEN to enhance the task of event extraction. The model’s ability to handle complex details and massive amounts of data proves its utility in practical applications in the field of emergency management and disaster response. This will improve the efficiency of disaster relief operations.

Table 8. Detailed event argument extraction results for the Henan rainstorm case.

Network	Text	Event Argument Extraction
BERT_QA	More than one hundred people were trapped at the entrance of the Zhengzhou Confucian Temple Station.	Personal Safety_Location: the entrance of Zhengzhou Confucian Temple Station
ETEN_BERT_QA		Personal Safety_number of trapped persons: More than one hundred people Personal Safety_Location: the entrance of Zhengzhou Confucian Temple Station

provides a detailed comparison of the event argument extraction results for rainstorm disaster texts. The results clearly show that our enhanced model significantly outperforms the baseline model, especially in the extraction of argument roles, such as the number of people trapped, which is an important detail often overlooked by traditional models.

This significant improvement can be attributed to the enhanced text representation capabilities and the effective use of the model to deepen contextual understanding. By accurately capturing the nuances of language in disaster scenarios, our model demonstrates greater accuracy in recognizing complex event details. These details are critical in real-time disaster management.

In disaster-related situations, accurate information can be a watershed for effective response and assistance. Therefore, the ability to extract these subtle and complex event details is critical. The model’s success in identifying these subtle elements not only emphasizes its robustness but also validates its practical application in disaster response and resource allocation. This capability ensures that emergency responders and decision-makers have access to the most relevant information, thereby increasing the efficiency of disaster response efforts. By providing a comprehensive understanding of event nomenclature, our model has become an important tool for responding to and mitigating the effects of natural disasters.

4. Discussion

In this paper, we propose a novel event extraction method, ETEN_BERT_QA, focusing particularly on extracting event argument elements from Weibo texts about rainstorm disasters. Our evaluation using the self-built RainEE dataset and the existing DuEE dataset shows that ETEN_BERT_QA excels in handling noisy and unstructured social media data. The ETEN component significantly improves event extraction in complex contexts by introducing event trigger words and optimizing the textual representations to improve the accuracy of identifying event parameters.

Compared to existing BERT-based event extraction models, ETEN_BERT_QA performs well in the event argument extraction task, especially in the context of noisy and complex Weibo data with enhanced robustness. Although the BERT-based machine reading comprehension model proposed by Du et al. [34] lays the foundation for event extraction, it lacks explicit modeling of event trigger words. This study introduces the trigger word mechanism, which significantly improves the accuracy of event extraction, especially in the context of complex text. Most of the current event extraction studies focus on generic events, and the refinement of natural disasters, especially rainstorm disasters, is significantly underdeveloped. To address this limitation, we construct a specialized RainEE dataset that enables the model to better cope with social media texts related to heavy rainfall. In addition, this study proposes practical solutions to effectively deal with social media data noise, thus opening up new avenues for event extraction tasks in complex environments.

The ETEN_BERT_QA model is founded upon the theoretical framework of event trigger words and augmented text representation. By explicitly modeling event trigger words, our study demonstrates the pivotal role of trigger words in event extraction. This result is in accordance with the theory of event semantic analysis, which posits that the accuracy of event argument extraction can be enhanced by the identification of pivotal information that precipitates the occurrence of events. The model demonstrates the successful application of this theory to event extraction for rainstorm disasters, thereby further enhancing the accurate identification of event arguments.

Notwithstanding the impressive outcomes yielded by the ETEN_BERT_QA model, certain constraints remain. Firstly, the domain specificity of the RainEE dataset constrains the model’s capacity to generalize to other categories of disaster events or broader social media datasets. To enhance the model’s capacity for generalization, the construction of more diverse datasets in the future is imperative. Secondly, the presence of noise and unstructured information in social media texts continues to present a challenge to the accuracy of event detection. The potential for misdetection and omission may impact the extraction of event arguments. Moreover, although existing noise reduction methods are effective in mitigating noise interference, the efficacy of these techniques in large-scale or real-time event extraction scenarios requires further verification.

While the ETEN_BERT_QA method is tailored for Weibo data and the specific context of rainstorm disasters, its core event extraction approach can be adapted to other social media platforms. The event extraction techniques, including the handling of noisy and unstructured text and the use of event trigger words, are not inherently platform-specific and could be applied to other platforms like Twitter, Facebook, or Instagram. However, it is important to note that the RainEE dataset, being based on Chinese Weibo data, is not directly applicable to other platforms or languages. The event extraction methods and preprocessing steps, such as text cleaning, and temporal and spatial information tagging, are transferable and could be applied in the construction of datasets for other platforms. Additionally, the event templates developed in this study, which focus on the extraction of specific event schema like locations, times, and event triggers, are adaptable for broader disaster datasets. Thus, while the RainEE dataset itself may limit the model’s immediate applicability to other platforms, the methodologies developed in this study provide a valuable framework for event extraction tasks on a wider scale. The ETEN_BERT_QA model demonstrates considerable potential for practical applications, particularly in the domains of disaster emergency management and social media monitoring. By effectively extracting data pertaining to rainstorm disasters from social media sources, government agencies, and emergency response organizations can gain insights into disaster dynamics and implement appropriate measures in a more expedient manner.

Further research could be conducted in several different ways. Firstly, the construction of diverse datasets encompassing a greater range of natural hazards is essential to enhance the model’s generalizability. Secondly, future research should investigate more sophisticated automatic event detection techniques, particularly in the context of the intricate nuances of social media texts. The utilization of more sophisticated multi-task learning strategies may prove beneficial in optimizing event detection and event parameter extraction. Furthermore, the incorporation of multimodal data (e.g., images, videos, sensor data, etc.) will facilitate the enhancement of contextual information for event extraction, thereby optimizing the accuracy of the model. Furthermore, real-time event extraction represents a crucial avenue of research, particularly in the context of disaster response and emergency management. Here, the objective is to enhance computational efficiency and reduce latency, which will be pivotal for effective decision-making in high-pressure situations. Furthermore, an investigation into the efficacy of pre-trained models on expansive and heterogeneous datasets, along with an analysis of the potential of migration learning for deployment in disparate domains, will enhance the adaptability and precision of the models. Ultimately, the implementation of reinforcement learning in dynamic environments, particularly in the context of optimizing event extraction decisions, has the potential to facilitate significant advancements in model development.

5. Conclusions

In recent years, social media has played an increasingly critical role in disaster studies, particularly during extreme weather events such as heavy rainstorms. Platforms like Weibo provide vast amounts of real-time data that can inform disaster response and mitigation efforts. However, the unstructured nature of the data presents significant challenges, making it difficult to extract actionable insights effectively. To address these challenges, this paper proposed an innovative event extraction approach utilizing a machine QA model enhanced by event trigger words. This method aims to systematically extract multi-dimensional information related to disasters, including time, location, and specific event details, from social media texts.

Our method demonstrated substantial improvements over traditional BERT-based models in event extraction tasks. Specifically, the ETEN_BERT_QA model was evaluated using both the custom RainEE dataset and the widely recognized DuEE dataset, achieving notable performance gains. On the RainEE dataset, which focuses on rainstorm disaster events, our model achieved an F1 score of 75.7%, effectively handling the complexity and noise typical of social media data. When tested on the DuEE dataset, which covers a broader range of event types, our approach showed a 3% improvement in F1 score over the baseline BERT_QA model, underscoring the effectiveness of incorporating event trigger words into the extraction process. The enhanced text representations allowed our model to more accurately capture relevant event triggers and arguments, thereby improving the completeness and accuracy of the extracted information.

A key strength of our approach lies in its ability to address the nuanced challenges posed by disaster-related social media texts, which often contain incomplete, noisy, and context-dependent information. By incorporating trigger word representations, the ETEN_BERT_QA model significantly enhances the identification of relevant event arguments, resulting in more robust and reliable extraction outcomes. The case study of the 20 July 2021, Henan rainstorm further validated the practical utility of our method, demonstrating its effectiveness in real-world disaster scenarios. During this devastating event, which led to extensive flooding, casualties, and property damage, our model was able to extract critical event details from over 6000 Weibo entries, highlighting its potential to support emergency management efforts with timely and accurate data.