A Novel Method for Named Entity Recognition in Long-Text Safety Accident Reports of Prefabricated Construction

Abstract

Prefabricated construction represents an advanced approach to sustainable development, and safety issues in prefabricated construction projects have drawn widespread attention. Safety accident case reports contain a wealth of safety knowledge, and extracting and learning from such historical reports can significantly enhance safety management capabilities. However, these texts are often semantically complex and lengthy, posing challenges for traditional Information Extraction (IE) methods. This study focuses on the challenge of Named Entity Recognition (NER) in long texts under complex engineering contexts and proposes a novel model that integrates Modern Bidirectional Encoder Representations from Transformers (ModernBERT),Bidirectional Long Short-Term Memory (BiLSTM), andConditional Random Field (CRF). A comparative analysis with current mainstream methods is conducted. The results show that the proposed model achieves an F1 score of 0.6234, outperforming mainstream baseline methods. Notably, it attains F1 scores of 0.95 and 0.92 for the critical entity categories “Consequence” and “Type,” respectively. The model maintains stable performance even under semantic noise interference, demonstrating strong robustness in processing unstructured and highly heterogeneous engineering texts. Compared with existing long-text NER models, the proposed method exhibits superior semantic parsing ability in engineering contexts. This study enhances information extraction methods and provides solid technical support for constructing safety knowledge graphs in prefabricated construction, thereby advancing the level of intelligence in the construction industry.

1. Introduction

Modular construction is a sustainable mode of production and has become a significant development direction in the construction industry [1,2,3]. Driven by technological advancements, the demand for sustainability, and the pursuit of higher efficiency, the global prefabricated construction market is rapidly expanding. In North America, the share of prefabricated buildings in new construction projects increased from 2.14% in 2015 to 6.64% in 2023 [4]. In China, this proportion rose from 2.7% in 2015 to 20.5% in 2020, with plans to further increase it to 40% by 2030 [3]. Despite its numerous advantages, safety issues at construction sites remain prominent in prefabricated construction. New construction processes, such as the unloading, hoisting, and installation of large components in prefabricated projects, have introduced new safety hazards [5]. According to statistics from one hundred prefabricated construction accident cases, falls from height account for 35%, crushing/collision incidents for 27%, and falling objects for 23% [6]. In addition, the frequency of accidents is relatively high. Data from the U.S. Bureau of Labor Statistics in 2018 show that in the construction of prefabricated wood and steel structures, the accident rates are 6.5 and 4.4 per 100 full-time workers, respectively. In contrast, the accident rate in traditional construction is only 2.7 per 100 full-time workers [7]. Ensuring construction safety in prefabricated buildings is a vital part of the United Nations Sustainable Development Goals (SDGs), including Good Health and Well-being (SDG 3), Sustainable Cities and Communities (SDG 11), and Responsible Consumption and Production (SDG 12) [8]. Therefore, emphasizing safety in modular construction is both a critical and urgent imperative.

Safety accident investigation reports provide detailed records of the causes and specific contexts of accidents, offering abundant case information for safety management. Effectively utilizing such textual information is of great significance for enhancing safety management [9]. With the ongoing digital transformation of the construction industry, a large amount of data related to prefabricated construction safety—such as investigation reports and construction logs—has been accumulated. However, these data typically exist in unstructured or semi-structured forms [10] and have not been fully integrated or leveraged, resulting in fragmented risk information that is difficult to effectively support safety risk management. Against this backdrop, Natural Language Processing (NLP) technologies have gradually been introduced into construction safety management with the goal of mining valuable safety information from massive amounts of unstructured or semi-structured accident texts [11,12]. Among these, Named Entity Recognition (NER) is a critical component of NLP that automatically identifies core safety entities such as accident types, causal factors, and construction activities from texts [13]. Meanwhile, NER serves as a bridge in many downstream applications, including question–answering systems, knowledge graph construction, and chatbots. In recent years, deep learning-based NER methods have achieved significant progress, especially pre-trained language models represented by Bidirectional Encoder Representations from Transformers (BERT), which have greatly improved recognition accuracy through their context modeling capabilities. Jeon et al. (2022) [14] proposed the KoBERT architecture, which is capable of recognizing complex entities in noisy texts with an F1 score of 91%, demonstrating its effectiveness in complex semantic environments. However, the standard BERT architecture is limited by an input length restriction of 512 tokens, making it difficult to directly process the long texts commonly found in accident reports [15]. Currently, most NER work is conducted on short texts, which severely limits the effective utilization of safety information. Although segmenting long texts can address this issue, it causes the loss of contextual information and may disrupt entity integrity [16]. To overcome these challenges, this study introduces Modern Bidirectional Encoder Representations from Transformers (ModernBERT), a pre-trained language model optimized for long texts, and integrates it with a Bidirectional Long Short-Term Memory (BiLSTM)–Conditional Random Field (CRF) structure. This model framework can handle context lengths of up to 8192 tokens, significantly extending the application scope of information extraction.

The remainder of this paper is organized as follows. Section 2 reviews the related literature; Section 3 introduces the integrated model framework and its methodology; Section 4 presents the experimental results; Section 5 analyzes the model’s stability and discusses the theoretical and practical implications of this study; and Section 6 concludes the research.

2. Literature Review

The construction industry is a high-risk sector [17]. The number of safety accidents in construction has consistently remained at a high level, resulting in significant casualties, economic losses, and disruptions to productivity [18,19]. Due to the frequent and severe nature of accidents at construction sites, various safety information records and reports are crucial for accident prevention and decision-making. However, most of this information exists in unstructured textual form and requires NLP techniques for effective extraction and utilization [20]. NLP can transform unstructured documents in the construction domain into structured information, facilitating efficient management by safety personnel. Among NLP tasks, text classification, topic clustering, and NERs are the most representative and widely applied branches in construction safety information extraction (IE) [21]. However, the first two lack sufficient depth in semantic understanding and cannot accurately pinpoint specific entities within the text. The core of NER lies in precisely identifying safety entities with specific meanings—such as accident types, causal factors, construction activities, personnel roles, and safety measures—by analyzing semantic information in the text, thus providing foundational semantic support for subsequent knowledge applications [22].

NER technology has evolved through several stages, from rule- and dictionary-based methods to machine learning, deep learning, and finally pre-trained language models [23]. In the early stage, rule- and dictionary-based approaches were primarily used, relying on regular expressions and manually crafted rules to match specific entities [24]. Although effective within specific domains, these methods suffer from limited generalizability. Subsequently, machine learning techniques such as Hidden Markov Models (HMMs), Maximum Entropy Markov Models (MEMMs), Support Vector Machines (SVMs), and CRF were widely applied [25]. However, these methods depend heavily on manual feature engineering and struggle to capture long-distance dependencies. In the initial phase of deep learning, models such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and BiLSTM automatically extracted features, significantly improving recognition performance [26]. In recent years, pre-trained language models have rapidly advanced NER tasks, especially with the introduction of models like BERT, which further enhance entity recognition accuracy. Related studies are summarized in

Table 1. BERT-Based NER studies in construction safety.

Related Work	Methodology	F1 Score	Research Focus
[27]	BERT	93.4%	Identification of Contractual Risk Clauses in Construction Specifications
[28]	CL-CasRel	66.9%	NER in Safety Documents
[29]	BERT-BiLSTM-CRF	88.26%	Identification of Construction Safety Knowledge Entities
[30]	BERT-GPLinker	91.90%	Identification of Risk Factors in Large-Scale Hydropower Construction Projects
[13]	BERT-LSTM	91%	Extraction of Key Information from Construction Accidents
[31]	RoBERTa	85.80%	NER of Risks in the Construction Supply Chain
[32]	BERT-Imseq2seq-CRF	91.47%	NER in Hot-Rolled Strip Rolling Process Texts
[14]	KoBERT	91.0%	Information Extraction from Construction Defect Reports
[15]	RoBERTa	95.6%	Extraction of Compliance Information in Construction Projects
[33]	BERT-BiLSTM-CRF	85.65%	Extraction of Safety Specification Information in Construction
[34]	BERT-base-uncased	87%	Identification of Risk Elements in Unilateral Contract Changes of Construction Projects

.

Currently, BERT and its integrated models still face significant limitations when processing long texts, primarily constrained by a maximum sequence length of 512 tokens, which hinders their ability to effectively handle long documents directly. To address the challenge of long-text processing, researchers have proposed various technical solutions. Chunking is the most common method [16], where long texts are segmented into fixed-length parts and processed stepwise using a sliding window; however, this approach leads to loss of contextual information. Longformer [35] and BigBird [36] are improved Transformer-based models designed to overcome computational bottlenecks in processing long texts. Longformer employs a combination of local sliding window attention and global attention mechanisms to achieve linear computational complexity. Building upon Longformer, BigBird introduces a random sparse attention mechanism. Both models extend text processing length to 4096 tokens and have been successfully applied in NER in medical texts [37] and long-text entity recognition in the construction domain [38]. Although Longformer and BigBird offer innovative solutions for long-text processing, their respective design characteristics bring certain drawbacks, such as sensitivity to hyperparameters, instability, and computational resource constraints, even though these can be mitigated through distributed matrix multiplication [39]. In contrast, ModernBERT demonstrates outstanding performance across various tasks due to its efficient processing capabilities and flexible architecture [40].

Although significant progress has been made in NER, there remain many theoretical and practical deficiencies in safety information extraction within the construction field, especially in the domain of prefabricated construction. First, with the advancement of construction industrialization and the gradual implementation of digital transformation, safety incidents in prefabricated construction have become increasingly complex and diverse. As a result, traditional qualitative analysis methods can no longer meet the practical demands for efficiently handling such situations. Moreover, in the prefabricated construction sector, efforts to mine and apply unstructured or semi-structured textual data remain insufficient. Second, mainstream deep learning models are primarily trained on short texts; therefore, their performance is significantly limited when processing longer texts. This limitation severely restricts the extraction and utilization of safety information.

3. Methodology

This study employs the ModernBERT-BiLSTM-CRF model for training and recognizing entities. The overall model framework is illustrated in Figure 1. The input text sequence undergoes preprocessing and is transformed into word embedding vectors as model input. The input sequence is first encoded by the ModernBERT model. After processing through the ModernBERT layer, each word embedding integrates contextual information, and the resulting semantic representations are passed to the subsequent BiLSTM layer. The BiLSTM captures both forward and backward sequence information, enabling an understanding of each word’s meaning within its context. Finally, a CRF decodes the feature sequence output by the BiLSTM to generate the final label sequence.

3.1. ModernBERT

ModernBERT [40] integrates a range of cutting-edge optimization techniques and introduces systematic improvements over the traditional BERT model in terms of architecture and computational efficiency. The model architecture is illustrated in Figure 2.

It replaces absolute positional embeddings with rotary positional embeddings (RoPE), removes unnecessary bias terms to simplify the architecture, and adds an additional normalization layer after the embedding layer. For normalization, it adopts pre-normalization blocks in combination with standard layer normalization. The activation function is replaced by GeGLU instead of the commonly used GeLU. A key technical innovation of ModernBERT is the introduction of the Alternate Attention Mechanism (AAM), which fundamentally differs from traditional global attention schemes. Technically, this mechanism applies full global attention only once every three layers, while the remaining layers adopt a sliding window strategy, wherein each token attends only to its 128 nearest neighbors (local attention). Considering that the computational complexity of the attention mechanism increases sharply with the number of tokens, this design enables ModernBERT to handle long-sequence inputs more efficiently than existing models. The formula is shown as follows:

$$\begin{array}{c}Attention\left(Q,K,V\right)=softmax\left({\displaystyle \frac{{QK}^T}{\sqrt{d_k}}}\right)V\end{array}$$

(1)

$$\begin{array}{c}AAM\left(Q,K,V\right)=GlobalandlocalAttention\left(Q,K,V\right)\end{array}$$

(2)

where Q denotes the query matrix, K the key matrix, and V the value matrix. ${QK}^T$ represents the dot product of Q and K. $d_k$ is the dimensionality of the key vectors. Softmax denotes the normalization of similarity scores to obtain attention weights.

ModernBERT significantly improves computational efficiency through its innovative unpadding and sequence packing techniques. Traditional approaches rely on padding tokens to standardize sequence lengths, resulting in considerable redundant computation. The unpadding technique in ModernBERT completely removes padding tokens and restructures the valid content into smaller batches. Combined with Flash Attention, this optimization eliminates the overhead of repeated padding and unpadding operations in conventional methods, leading to a performance gain of 10–20%. Furthermore, ModernBERT adopts a sequence packing strategy, using a greedy algorithm to dynamically combine multiple short sequences into compact batches that approach the model’s maximum input length, thereby maximizing GPU parallelism. This approach not only reduces padding waste but also enhances training throughput. Along with Flash Attention’s optimization of RoPE, ModernBERT requires only a single unpadding operation, with no need for subsequent reconstruction, making both pretraining and inference more efficient. These techniques collectively optimize resource utilization and significantly accelerate the model while maintaining its performance.

3.2. BiLSTM

BiLSTM belong to a class of RNN architectures capable of capturing both forward and backward dependencies within a text sequence [41]. In NER tasks, determining the boundaries and categories of entities often relies on contextual information from both preceding and succeeding words. Unidirectional models struggle to effectively capture such symmetrical semantic relationships. In contrast, BiLSTM can more accurately represent the semantic state of words within their context. It achieves this by incorporating two independent LSTM units—one processing the input sequence from left to right, and the other from right to left—thereby comprehensively capturing the full contextual semantics. The model architecture is illustrated in Figure 3.

In this study, BiLSTM is employed to further process the word vectors output by ModernBERT. The computational process of the BiLSTM model is described as follows:

$$\begin{array}{c}{i}_t=\sigma \left(W_i\left[x_t;h_{t-1}\right]+b_i\right)\end{array}$$

(3)

$$\begin{array}{c}{f}_t=\sigma \left(W_f\left[x_t;h_{t-1}\right]+b_f\right)\end{array}$$

(4)

$$\begin{array}{c}{o}_t=\sigma \left(W_o\left[x_t;h_{t-1}\right]+b_o\right)\end{array}$$

(5)

$$\begin{array}{c}{\tilde{c}}_t=\mathit{tanh}\left(W_c\left[x_t;h_{t-1}\right]+b_c\right)\end{array}$$

(6)

$$\begin{array}{c}{c}_t=f_t·c_{t-1}+i_t·{\tilde{c}}_t)\end{array}$$

(7)

$$\begin{array}{c}{h}_t=o_t·\mathit{tanh}\left(c_t\right)\end{array}$$

(8)

where σ denotes the Sigmoid activation function. · represents matrix multiplication. $x_t$ is the input feature vector at time step t. $h_t$ is the current hidden state vector. $W_i,W_f,W_o,W_c$ are the weight matrices for the input gate, forget gate, output gate, and cell candidate, respectively. $b_i,b_f,b_o,b_c$ are the corresponding bias vectors. $tanh$ denotes the hyperbolic tangent function.

The final output is the concatenation of the forward and backward hidden states:

$$\begin{array}{c}\widetilde{h_t}=\left[\overrightarrow{h_t};\overleftarrow{h_t}\right]\end{array}$$

(9)

3.3. CRF

In NER tasks, BiLSTM excels at capturing long-distance textual dependencies but cannot effectively model the dependencies between adjacent labels [42]. CRF are discriminative probabilistic graphical models [43] whose core advantage lies in considering the transition probabilities between labels. By decoding via a globally optimal path, CRF prevents the model from producing invalid label sequences. The model architecture is illustrated in Figure 4.

The output $\widetilde{h_t}$ of the BiLSTM is projected through a hidden layer into an emission score matrix P, whose dimensions match the number of labels, capturing local semantic alignment between “words and labels.” The CRF layer introduces a transition score matrix A, which models dependencies between labels. For example, the entity label “B-Safety Harness” is more likely to be followed by “I-Safety Harness” than by “B-Helmet.” The higher the score, the more semantically reasonable the transition. By combining emission and transition scores, the CRF computes the total score of a label sequence y as follows:

$$\begin{array}{c}S\left({\rm X},y\right)={\displaystyle \sum _{i=0}^T}{A}_{y_i,y_{i+1}}+{\displaystyle \sum _{i=1}^T}{P}_{i,y_i}\end{array}$$

(10)

where X is the input sequence, y is the label sequence, T is the sequence length, A is the transition score matrix, P is the emission score matrix.

To convert the score into a probability and guide model training, the CRF normalizes over all possible label sequences $y^{\prime }$ using the softmax function, yielding the conditional probability:

$$\begin{array}{c}p\left(y|{\rm X}\right)={\displaystyle \frac{e^{S\left({\rm X},y\right)}}{{\sum }_{y^{\prime }\in \mathsf{\gamma }}{e}^{S\left({\rm X},y^{\prime }\right)}}}\end{array}$$

(11)

where γ denotes the set of all possible label sequences.

During training, the model parameters are optimized by maximizing the log-likelihood of the correct label sequence:

$$\begin{array}{c}\mathit{ln}p\left(y|{\rm X}\right)=S\left({\rm X},y\right)-\mathit{ln}{\displaystyle \sum _{y^{\prime }\in \mathsf{\gamma }}}{e}^{S\left({\rm X},y^{\prime }\right)}\end{array}$$

(12)

During inference, the model selects the label sequence with the highest total score among all possible sequences. This problem is solved via Viterbi decoding, which consists of three steps: (i) Initialization: Compute initial scores for each label at the first position, incorporating the start symbol. (ii) Recursion: For each subsequent position and each possible label, compute accumulated scores by considering all previous labels and record the optimal predecessor. (iii) Termination and Backtracking: Determine the final label using the end symbol, then backtrack using the recorded predecessors to recover the optimal sequence $y^{*}$.

$$\begin{array}{c}{y}^{*}=\underset{y^{\prime }\in \mathsf{\gamma }}{\mathit{argmax}}S\left({\rm X},y^{\prime }\right)\end{array}$$

(13)

3.4. Evaluation Metrics for NER

Precision (P), Recall (R), and F1-score are used as evaluation metrics to measure the model’s performance. Their calculation formulas are as follows:

$$\begin{array}{c}Precision={\displaystyle \frac{TP}{TP+FP}}\end{array}$$

(14)

$$\begin{array}{c}Recall={\displaystyle \frac{TP}{TP+FN}}\end{array}$$

(15)

$$\begin{array}{c}F1=2\times {\displaystyle \frac{Precision\times Recall}{Precision+Recall}}\end{array}$$

(16)

where True Positive (TP) refers to the number of correctly identified entities; False Positive (FP) refers to the number of non-entities incorrectly identified as entities; False Negative (FN) refers to the number of true entities that were not identified.

4. Experiments and Results

4.1. Data Collection and Processing

This study collected 1235 construction accident investigation reports related to prefabricated steel structure buildings. Each report was carefully reviewed, resulting in the removal of 597 reports unrelated to prefabricated steel structure buildings and 109 reports with incomplete content (requiring the report to include a complete overview of the accident, accident process, causes, and prevention measures). Ultimately, 529 valid prefabricated steel structure building accident investigation reports were obtained.

Text preprocessing of accident investigation reports is a crucial step in building a high-quality corpus, as illustrated in Figure 5. First, the text format is standardized. Next, the raw texts are cleaned by removing irrelevant information, redundant spaces, and other noise. Then, stop words that do not contribute to semantic analysis are deleted. Subsequently, data quality checks are conducted to ensure the texts are complete, consistent, and accurate. Finally, the corpus is constructed by saving the processed and normalized texts, facilitating subsequent tasks such as information extraction and model training. Statistical analysis of the collected texts shows that the longest document contains 6450 characters, and the average length is 1517 characters—both exceeding the text length typically handled by traditional pre-trained models.

4.2. Entity Type Definition and Annotation

Based on the content of accident investigation reports and literature review, a set of core terms covering the entire accident process was compiled, comprising six categories of entities: accident time, construction activity, accident cause, accident type, accident consequence, and preventive measure, as shown in

Table 2. Categories of entities.

Entity Category	Definition	Description	Source
Accident Time	Time	The specific time point or time period when the accident occurred.	[44,45]
Construction Activity	Activity	The specific construction task being carried out at the time of the accident.	[44,46,47,48]
Accident Cause	Cause	The direct and indirect causes of the accident, such as unsafe worker behaviors, mechanical failures, and managerial deficiencies.	[13,29,44,45,46,47,48,49]
Accident Type	Type	The nature of the accident, including falls from height, struck-by-object incidents, collapses, etc.	[44,45,47]
Accident Consequence	Consequence	The impact of the accident, including casualties, injuries, and economic losses.	[13,29,44,45,46,48]
Preventive Measure	Measure	Recommended preventive actions or improvement strategies to avoid similar accidents.	[46,47,48]

.

After defining the named entity types, the training data annotation process began. This study utilized the open-source tool Yedda [50] and applied the BIO sequence labeling method for entity annotation. The BIO method assigns a label to each character to indicate its position within a named entity: “B” denotes the beginning of an entity, “I” denotes characters inside the entity, and “O” denotes characters outside any entity. This approach clearly marks the boundaries and types of entities within the text, as illustrated in Figure 6. To ensure the reliability of the dataset, this study implemented rigorous quality control measures during the annotation process. All annotations were independently carried out by three PhD students with backgrounds in construction safety management and NLP. In the initial phase, we conducted discussions and performed pilot annotations to standardize the annotation guidelines. Subsequently, a subset of samples was randomly selected for cross-checking to ensure consistency and accuracy. Following a detailed annotation process, a total of 12,772 entities were labeled, with the distribution of annotated entities shown in Figure 7.

4.3. Experimental Setup

In this study, all experiments were conducted on a server equipped with an NVIDIA GeForce RTX 4090 GPU with 24 GB of memory, running the Linux operating system. The code was developed using PyCharm Community Edition 2022, with Python 3.9 as the programming language and PyTorch 2.5.1 as the deep learning framework. During model training, the dataset was split into a training set and a test set at a ratio of 8:2, and five-fold cross-validation was employed to calculate the final results. In the training configuration, the batch size was set to 4, the learning rate to 1 × 10⁻⁴, and the optimizer used was Adaw. The model was trained for 100 epochs, using the cross-entropy loss function.

4.4. NER Results

(1) NER Results on Short and Long Texts and Model Comparison

Comparative experiments were conducted on both short-text and long-text corpora to evaluate the performance of different pre-trained models in NER for safety accident reports. The evaluation metrics used were Precision, Recall, and F1-score. The results are presented in

Table 3. NER performance of different pretrained models on short texts.

Model	Precision	Recall	F1
BERT	0.6712	0.6753	0.6732
BERT-BiLSTM-CRF	0.7012	0.6825	0.6916
ModernBERT-BiLSTM	0.7381	0.6225	0.6656
ModernBERT-BiLSTM-CRF	0.6825	0.6643	0.6731

and

Table 4. NER performance of different pretrained models on long texts.

Model	Precision	Recall	F1
BERT	0.5912	0.5743	0.5827
BERT-BiLSTM-CRF	0.6023	0.5824	0.5922
ModernBERT-BiLSTM	0.7106	0.5756	0.6143
ModernBERT-BiLSTM-CRF	0.6855	0.5828	0.6234

.

Table 3 presents the NER performance of different pre-trained models in short-text scenarios. These short texts were obtained by segmenting the long texts, with lengths kept within 512 characters. The experimental results show that the BERT-BiLSTM-CRF model outperforms other models in terms of Precision, Recall, and F1-score, achieving values of 0.7012, 0.6825, and 0.6916, respectively. This indicates that combining BiLSTM and CRF mechanisms with the BERT model enables more effective capturing of entity boundaries and types in short-text NER tasks, thereby improving recognition accuracy. In comparison, the pure BERT model, the ModernBERT-BiLSTM model, and the ModernBERT-BiLSTM-CRF model exhibit slightly lower performance than the BERT-BiLSTM-CRF model.

Table 4 lists the NER results of various models on long-text tasks. The experiments demonstrate that the ModernBERT-BiLSTM-CRF model achieves the best performance in long-text scenarios. This result fully illustrates that ModernBERT, owing to its improved architectural design, can better capture long-distance dependencies, thereby enhancing entity recognition performance in long texts. In contrast, the BERT-BiLSTM-CRF model, the ModernBERT-BiLSTM model, and the pure BERT model all perform worse than the ModernBERT-BiLSTM-CRF model on long texts. This is primarily due to the contextual window limitations inherent in BERT when processing long sequences.

To validate the effectiveness of the CRF layer, the ablation study results presented in Table 3 and Table 4 indicate that incorporating the CRF layer significantly improves model performance under both short and long text scenarios. This confirms the utility of structured label transition constraints in NER tasks. Overall, within the length of 512 tokens, BERT’s full attention mechanism can fully capture the dependencies among all tokens, especially the close correlations between local and global contexts. That is, in short texts, the full attention mechanism of BERT can model all token relationships without omission. In contrast, ModernBERT for long texts adopts a sparse attention mechanism, which may overlook important information. When the text length exceeds 512 characters, the advantage of this long-range sparse attention mechanism becomes prominent, as it can focus on positional and contextual information in long texts, thereby enhancing the contextual modeling capability. Therefore, for short texts, BERT’s global attention mechanism combined with absolute positional encoding achieves dense interactions within the 512-token limit, making it suitable for local semantic modeling. ModernBERT, by employing strategies such as sparsification and chunking to overcome length limitations while retaining global context, is better suited for tasks involving long-distance dependencies.

To verify the statistical significance of the performance difference between the ModernBERT-BiLSTM-CRF and BERT-BiLSTM-CRF models in long-text NER tasks, a paired-sample t-test was conducted on their F1 scores using the same test set. The results show that ModernBERT-BiLSTM-CRF consistently outperformed the BERT-BiLSTM-CRF model across all five experimental runs, with an average F1 score improvement of 3.6%. The paired t-test revealed a highly significant difference (t = 4.5200, p = 0.0054 < 0.05), and the effect size (Cohen’s d = 2.02) indicates a substantial practical difference. These quantitative findings strongly support the effectiveness of the ModernBERT architecture improvement and demonstrate its statistically significant performance advantage in long-text processing tasks.

(2) NER Results by Entity Category

In long texts, this study investigates the model’s recognition capability and differences across various entity categories within prefabricated construction safety accident reports. Using the experimental model ModernBERT-BiLSTM-CRF, a quantitative comparative evaluation was conducted for six entity categories, with the results presented in

Table 5. NER performance by entity category.

Entity Category	Precision	Recall	F1
Time	0.98	0.65	0.78
Activity	0.49	0.26	0.34
Cause	0.58	0.33	0.43
Type	0.92	0.92	0.92
Consequence	0.93	0.97	0.95
Measure	0.68	0.31	0.41

.

The experimental results indicate significant differences in recognition performance among different entity categories. The model performs best on the “Consequence” and “Type” entities, achieving F1 scores of 0.95 and 0.92, respectively, demonstrating high accuracy and stability in identifying accident consequences such as casualties and economic losses, as well as accident types like falls from height and object strikes. This is attributed to the relatively standardized expression, clear semantic boundaries, short length, and uniformity of these entities in the corpus. The “Time” entity achieves an F1 score of 0.78, with high precision but relatively low recall. Although most predictions are correct, some entities are missed. This phenomenon arises because time expressions vary widely, and the model struggles to handle ambiguous boundaries or semantically unclear temporal segments. In contrast, the recognition performance for the “Cause,” “Measure,” and “Activity” entities is relatively poor, with F1 scores of 0.43, 0.41, and 0.34, respectively, indicating limited capability in recognizing these semantically complex entities. The weakest recognition is for the “Activity” entity, mainly due to its involvement with specific construction tasks that have diverse and inconsistent expressions. For example, terms such as “steel column hoisting,” “component installation,” and “lifting operations” can all refer to similar construction activities. In some cases, even generic expressions like “operation” or “during the operation” are used, making it difficult for the model to establish stable recognition patterns. Additionally, some construction activities do not appear as explicit verb structures or proper nouns but are instead embedded within complex semantic structures. The “Cause” entity is also difficult to identify, as accident causes are diverse. Without the support of a large sample size, the model lacks sufficient contextual understanding when extracting direct or indirect causes of accidents. Additionally, the “Measure” entity involves suggestive and improvement-related expressions with flexible and varied language and longer lengths, making it challenging for the model to accurately capture boundaries and core semantics.

5. Discussion

5.1. Model Performance Analysis

(1) Noise Robustness Experiment

To further verify the robustness of the model in complex real-world application scenarios, this study conducted interference experiments by introducing semantic noise into the long-text corpus, including word order disruption, insertion of irrelevant words, and random character deletions [14]. In this study, the noise perturbation ratio was controlled at the 5% character level during the experimental setup. A comparative analysis was conducted to evaluate the model’s NER performance before and after the introduction of noise. The experimental results show that without noise, the model’s Precision, Recall, and F1 scores were 0.6855, 0.5828, and 0.6234, respectively. After introducing noise, these metrics changed to 0.6691, 0.5857, and 0.6186, respectively. It can be observed that Precision decreased noticeably, indicating that the accuracy of the model’s predictions was somewhat affected in noisy environments, with an increased rate of misidentified entities. Recall showed a slight increase, suggesting that the model’s ability to capture potential entities was marginally enhanced after noise introduction, possibly because the noisy text introduced certain cues related to entity boundaries. The overall F1 score declined, reflecting a reduction in the model’s overall performance under noise interference. Although the recall improved slightly, it was insufficient to offset the negative impact of the decreased precision. These results reveal that the model possesses a certain degree of resistance to semantic noise in long-text NER tasks, as its performance did not exhibit drastic fluctuations.

(2) Comparative Analysis of Existing Long-Text Processing Models

In recent years, numerous improved pre-trained models have been proposed for long-text NER tasks. Among them, models such as Longformer and BigBird have received widespread attention, extending the text processing length to 4096 characters. These models demonstrate high recognition accuracy in domains where entity boundaries are clear and annotation standards are strict, particularly in well-structured corpora like medical texts. To evaluate the practical performance of various models across different domains, this paper summarizes and analyzes current research achievements in long-text NER, as shown in

Table 6. Comparison of existing NER performance on long texts.

No.	Related Work	Model	Data Scale	F1 Score	Text Length
1	[36]	Clinical-Longformer	1304 electronic medical records	Average 0.9055	4096 characters
		Clinical-BigBird		Average 0.8945	4096 characters
2	[37]	Longformer-CASREL	7 engineering consulting standards and specification documents	0.6899	4096 characters
3	[49]	DeBERTa v3-Longformer	1017 medical texts	0.940	4096 characters
4	This method	ModernBERT-BiLSTM-CRF	529 construction accident investigation reports	0.6234	8192 characters

.

In the medical domain, studies by Li et al. (2023) [37] and Schäfer et al. (2023) [51] show that Longformer and BigBird models perform excellently in extracting medical concepts and drug names, achieving F1 scores above 0.9, mainly benefiting from clear domain terminology and standardized data. In contrast, Gao et al. (2024) [38] report relatively weaker performance of the Longformer-CASREL model on engineering consulting texts, primarily due to the diverse expression forms and high language ambiguity of technical texts, which bear some similarity to the characteristics of the corpus used in this study. The corpus utilized in this study not only exceeds the maximum character length manageable by Longformer and BigBird models but also features significant variation in writing styles and implicit semantic expressions and contains many strongly context-dependent and domain-specific terms. Despite these substantial challenges, the proposed model still achieves an F1 score of 0.6234. Although this is somewhat lower than the results in structured domains, it demonstrates good practical applicability in handling complex, unstructured, and semi-structured long texts. Additionally, it should be noted that this comparative analysis aims to present the representative performance of existing mainstream pre-trained models in long-text tasks within their respective domains, illustrating their capability characteristics and application scenarios, rather than serving as a direct performance comparison with the model proposed in this study on the same task.

5.2. Knowledge Contributions

The knowledge contributions of this study in the field of prefabricated building construction safety are evident and primarily manifested in the following two aspects.

1. Previous research has been limited by the constraints of traditional Transformer architectures, focusing mainly on information extraction methods for short texts, with insufficient exploration of long-text processing. To address this gap, this paper proposes a novel long-text NER method specifically designed for the prefabricated construction safety domain. Through an in-depth analysis of the proposed model’s performance across varying text lengths, the effectiveness of the model in long-text NER is validated. Additionally, contrastive experiments introducing semantic noise evaluate the model’s robustness when faced with non-standard text inputs in practical applications, thereby demonstrating the practical potential of the proposed method. This approach significantly improves the efficiency of information extraction from prefabricated construction safety accident texts, opening a new technical pathway for semantic parsing of long texts, and substantially expanding the depth of NLP technology’s semantic understanding and knowledge construction applications within the construction engineering domain.

2. The prefabricated building construction process involves a large volume of unstructured or semi-structured long-text data in various forms, such as accident reports, construction logs, and supervision records. The NER method proposed in this study enables automatic extraction of critical safety information from massive amounts of text, thereby achieving structured knowledge management and improving safety management performance. Moreover, it facilitates the transition of enterprises toward data-driven construction safety management models, enhancing the synergistic development of building industrialization and intelligent construction in the industry.

5.3. Limitations and Future Work

Although the long-text NER method proposed in this study has achieved satisfactory performance on prefabricated building construction safety accident reports, several limitations remain that require further improvement and optimization in future research.

1. The training and testing corpora used in this study are primarily derived from publicly available Chinese reports of prefabricated building construction safety accidents. The overall dataset size is relatively limited, and the distribution of entity types is imbalanced, which may adversely affect the model’s ability to recognize long-tail entity categories and its generalization performance [52].

2. The current dataset is limited to Chinese texts, and the feasibility of transferring the model to other languages, such as English, has not been explored.

3. This study focuses solely on the NER task and does not extend to constructing semantic relationships between entities. To some extent, this undoubtedly restricts the model’s potential applications in higher-level tasks such as knowledge graph construction and safety inference analysis.

Future research will focus on addressing the above issues. To tackle the challenges of limited sample size and cross-lingual adaptation, a progressive knowledge distillation framework will be employed [53]. Additionally, future studies will integrate an entity relation extraction module to advance the construction of knowledge graphs in the field of prefabricated construction safety.

6. Conclusions

This study focuses on construction safety accidents in prefabricated buildings and proposes a new method for NER to address the challenges associated with long text. Systematic experiments and comparative analyses were conducted on actual safety incident investigation reports, leading to the following conclusions.

1. This work integrates the advantages of various models to construct a novel NER architecture based on ModernBERT-BiLSTM-CRF. Compared to traditional BERT and BERT-BiLSTM-CRF models, the ModernBERT-BiLSTM-CRF model outperforms others in Precision, Recall, and F1 metrics, achieving values of 0.6855, 0.5828, and 0.6234, respectively. Notably, it demonstrates excellent recognition performance for the key entity categories “Consequence” and “Type,” with F1 scores reaching 0.95 and 0.92, respectively.

2. In long-text tests involving semantic noise interference, the model’s performance exhibits only minor fluctuations. This indicates strong robustness and adaptability when handling real-world texts characterized by irregular expressions, inconsistent formatting, and semantic ambiguity. Such robustness provides reliable support for safety information extraction and knowledge graph construction in complex contexts.

3. Although models like Longformer and BigBird perform well in medical tasks, they show limited adaptability when confronted with engineering texts featuring complex terminology, semantic ambiguity, and greater text length. The proposed method demonstrates higher semantic parsing capability for longer and more heterogeneous texts typical of engineering contexts, thereby offering greater practical value for engineering applications.